wrotniak.net: Time for omni-aspect sensors?

Trying too avoid confusion (but, possibly, adding to it), we will be talking about multi-aspect or omni-aspect imagers or cameras. Both kinds allow you to change the aspect ratio between one recorded image and another, except that in multi-aspect you cannot switch between landscape and portrait orientation without rotating the camera by 90° around the lens axis.

Actually, avoiding that rotation while shooting in a vertical format is the main subject of this article. Most digital cameras use two traditional form factors from the film era, single-lens reflex (SLR) and rangefinder (RF). In both, most of the rationale behind the body shape is long gone. For example, the designer no longer has to accommodate the film path (including film chambers on both sides of the lens). Except for digital SLRs (a dying breed) there is no need for a viewfinder hump directly above the lens and a mirror chamber with an optical viewing screen.

The traditional form-factor has been tested and refined over more than 100 years of development. It seems to work quite well ergonomically, at least in the landscape orientation. For vertical framing, however, it is not so good, both in terms of camera handling and accessing its controls. The so-called 'vertical grips", whether integral or accessory, don't seem to help much, if at all.

Good news: with the mirror going the way of the dodo, another option becomes feasible: using an omni-aspect imager without rotating the camera.

You can see it in my E-M1X article (update of February, 2019), referred to as "true multi-aspect". While I have yet to see it described elsewhere, the concept is so obvious that it must have been presented somewhere — and not just once.

This one comes in two flavors, described in their own sections that follow the next one.

Multi-aspect imagers

These are quite common and, with one exception I'm aware of, they are implemented in the simplest possible way: by cropping down the full sensor frame (sometimes referred to as native). In Micro Four Thirds this is the full-resolution, 4:3 rectangle, usually cropped down to 3:2 and 9:16, often also 1:1.

This does not really matter, as the same result can be produced in postprocessing; some image managers (like the freeware FastStone, which I can really vouch for) have a quick, lossless crop function for common file types, including JPEGs (which usually suffer some quality loss when re-saved). The only advantage seems to be that the preview is more accurate and meaningful, using the actual aspect ratio as set (you can later change your mind, as long as the camera also saves raw files).

Let me refer to this design as Cropped Multi-Aspect; its geometry is shown at the left below. It i clearly seen that of all frames shown, only the native 4:3 one spreads over the full image circle: frame diagonal equals circle diameter (21.7 mm).

The other picture shows a less common solution by Panasonic, used in (some of) their Lumix DC-GH camera series. I'll be calling it Full Multi-Aspect. It adds just 1.6 mm to the overall width of the sensor. This does not affect the 4:3 frame (limited by the image circle, which cannot be changed), but adds to the area to the cropped wide frames (2:3, 9:16) — see the Area row of the table that follows.

		Cropped Multi-Aspect		Full Multi-Aspect
Aspect Ratio	1:1	4:3	3:2	16:9	Sensor	1:1	4:3	3:2	16:9	Sensor
Frame [mm]	13.0×13.0	17.3×13.0	17.3×11.6	17.3×9.8	17.3×13.0	13.0×13.0	17.3×13.0	18.0×12.0	18.9×10.6	18.9×13.0
Area [mm²]	169 75%	225 100%	200 89%	169 75%	225 100%	169 75%	225 100%	217 96%	201 89%	246 109%
Diagonal [mm]	18.4 85%	21.7 100%	20.8 96%	19.9 92%	21.7 100%	18.4 85%	21.7 100%	26.7 123%
Focal Length	118%	100%	104%	109%	100%	118%	100%	81%

Note the white corner tips of the full sensor area. These are not covered by the image circle, but also (by definition) not used by any of the aspect frames.

I do like this approach, but it suffers from one limitation: not handling vertical (portrait-oriented) formats — or, for that matter, anything less wide than 4:3, even a square (which both Multi schemes handle identically).

Vertical versions of the supported horizontal aspect ratios can be achieved by rotating the whole camera (trivial and cumbersome; not helped by a vertical grip) or just its data back accessory (if available, costly) by 90°. I'm considering these approaches unfit for the 21st century.

Cropped Omni

In this approach, the "native" 4:3 effective image rectangle is cropped to 3:2 and 16:9 aspect ratios, preserving the longer dimension. So far this is what Olympus and (with a few exceptions) Panasonic are doing now.

Vertical versions of these ratios can be achieved by rotating the whole camera (trivial) or just its data back accessory (if available, costly) by 90°. I'm considering these approaches unfit for the 21st century.

Another option is to stretch the sensor to cover the vertical 3:4 rectangle, and crop the image as needed. While that makes the sensor square, the far corners are never used, and all frames fit within the existing μFT image circle as shown below.

Thus, every out-of-range aspect ratio has its own effective focal length multiplier. For example, a 200 mm lens at 16:9 provides an angle like 218 mm at 4:3 (109% of 200), filling 75% of the native frame area (or pixel count).

The simplest implementation of this scheme would require a square sensor, 17.3 mm to a side; a 33% increase in area and pixel count, adding just 4 mm to the camera height. Compare that to "vertical grip" solutions.

The Sensor column of the table refers to this square sensor. The focal length multiplier is in red, as it includes the corner area, partly outside of the image circle. Still, I would see the whole square saved as a raw file, to be cropped only at RGB conversion.

Full Omni

This is an improved version of the previous scheme. Here the frames are made to fit in the image circle, with all corners on its perimeter — but only between a certain, widest aspect ratio and its inverse. 9:16 makes sense here, being the widest of ratios expressly specified in the μFT standard.

Note that the frame diagonal (and therefore the effective focal length) stay constant within the table range, so the image circle provided by the lens is fully utilized..

Again, the Sensor column describes the smallest square covering the widest and tallest supported ratios (16:9 and 9:16). Its area is 58% larger than that of the native 4:3, and so is the pixel count. Obviously, the physical size of the imager is now by 5.9 mm taller and by 1.5 mm wider.

A purist may prefer the True Omni, forgoing the 16:9 limit and using the whole image angle at every aspect ratio. This calls for a 21.7 mm square — a sensor area of 470 mm², more than twice the original μFT. In practice, however , we see a pattern of diminishing returns, and a ceiling or the aspect ratio to 16:9 (or maybe 2:1) seems to make more sense.

Other requirements

Each of the four solutions shown above requires that the camera has a set of related features and/or performs or modifies a set of related functions. While some details (especially numeric values) may differ between brands or models, most remain the same.

Crop to aspect. All four designs need a firmware capability to generate cropped RGB images; also cropped EVF/monitor preview and review.
This is simple for the landscape-only, multi-aspect designs. For omni-aspect, to display also portrait-oriented frames, a painful choice has to be made between two options:
- Increasing the display vertical dimension, making it a square, capable of showing portrait and landscape frames in the same scale. Nice, but possibly costly.
- Keeping the landscape screen proportions and reducing the magnification of portrait-oriented to fit them in. A poor man's solution; still, I'd prefer it over a vertical grip.
I wouldn't be surprised to see the EVF and monitor use different choices here.
Larger, heavier image sensor. Except for Cropped Multi, all sensors gain area and weight as compared to vanilla μFT: by 9%, 33% and 58% (in the order of appearance). This affects functions which involve imager movement: in-body image stabilization, sensor-shift high resolution, and dust removal.
Extended shutter travel, heavier curtains — follow-up of the above. These may bring: more vibration to suppress, lower serial rates, slower flash synch..
More of them pixels or photosites. Assuming a given pixel count for the native, 4:3 output, these sensors have, again, 9%, 33% or 58% extra photosites and have to handle a similar amount of image information. All this uses processor cycles, memory, and battery power.
It also affects the performance of electronic shutter, which has to deal with more scan lines nd/or longer ones. This may affect some performance aspects in this mode: maximum frame rate, synch speed, geometric distortion, and more.
Conclusions
Unless I've missed something really important, a mirror-less digital camera using an omni-aspect imager for portrait-oriented shooting seems to be technically possible and doable. Consisting only of elements similar to ones already used in the industry, it could be also realistically priced.
Sooner or later both the market and csmera manufacturers will recognize this.