Why is full colour passthrough so disappointing?

Last updated on Aug 8, 2023

Virtual Reality (VR) technology has been hotting up in recent years, but there are still some limitations that need to be overcome. One of the biggest issues that VR users face is the poor color pass-through in VR headsets. So let's have a wider look at what could be affecting this.

What is full colour passthrough?

First, let's define what we mean by full color pass-through. In the context of VR, or should I say MR (mixed reality) color pass-through refers to the ability of a VR headset to accurately represent the colors of the real world, much like your camera does, or the camera on your phone.

This is important because VR is designed to create a sense of presence, and accurate color representation is a key component of creating a convincing virtual environment.

Can we take any clues from digital cinematography?

In digital cinema cameras, you have a simlar make up of to that of a VR headset. They both use and rely on digital sensors to interpret image data from single or multiple lenses, either built in or interchangeable.

Canon use thier CMOS sensors.

Arri use varients of thier ALEV2 in thier Alexa's.

Red Digital Cinema have cameras built around specific sensors for particular types of shot needed, for example Monstro VV 8K 16 bit for full frame VFX work, Gemini 5K for low light etc (yes these are a previous generation of Red cameras, but it is clearer for the purpose of this article).

And let's not forget Sony's excellent Venice, used to shoot the original Avatar film.

Before the Red Gemini, which was an excellent dual ISO sensor, meaning you could switch from normal shooting conditions to low light at the flick of a switch, a lot of film makers used a cheap Sony DSLR camera, the A7s.

On paper, this camera looked a bit disappointing compared with, at the time, to the much superior A7R, which had many more pixels (36 megapixels), but for filmmaking, that was it's Achilleas heel. You see the A7S was a 12 megapixel camera (even at the time was low) but this was for a very good reason, a smaller pixel count meant big, juicy, fat pixels that took in more light from the scene, perfect for low light situations, I remember at the time how impressive this was even though it was within an 8bit colour space.

Even with a lower pixel count, the image was really tight, with a nice, natural filmic type grain to it. The irony is today, the resolutions are so massive, that you have to add grain in post production, such is the competance of these new sensors and lenses.

So what can all this tech spiel tell us?

Making VR Headsets or any form of physical product is really, really, damn hard, thats for sure. But VR headsets have to do so much more than a digital cinema camera.

A cinema camera has one lens and one sensor (ok not entirely true, but I am not going to go into sensor stacking or stitching in a VR article) and one job, to digitally interpret an image, and via a codec (RedRaw, Arri Raw, Apple Pro Res, XAVC for canon etc) recorded to internal or external media cards, where all that data can then interpreted in an NLE (Non Linear Editor, think Final Cut Pro, Avid etc) and can then be treated by a colourist and then onto the editors screen.

The above comes with a considerable cost and other implications, lets use the Sony Venice as an example:

Camera body $40k +
Tripod and head $15k
On camera monitor $3k
Directors monitor P3 colour space $5k
Digital follow focus $12k
Media cards for the day $6k
Lenses, $20k each
DIT cart and operator $1300 day
Focus puller op $700 day
The million bits needed "just cos" on set.

Ok you see the point, a fully rigged camera is north of $100k

With VR or MR headsets, the processes are much more complicated:

6 or more cameras in a tiny space
Must not overheat
Must not use use loud fans for cooling
Have great built in audio
Hand tracking
Great colour
Switch quickly to mixed reality
All this, and much more and with a decent battery life all for $800 bucks!

I think you can quickly see why colour pasthrough is not so great at the moment, even though Meta has thrown a lot of coin at the problem.

Conclusion

Maybe, just maybe, in the race for more resolution to apease YouTubers and pixel peepers, they'll take a look right back at the start of the low light revolution in cameras, the Sony A7sii, bigger pixles, more light in and better vision out where its needed, not in a controlled studio/lab enviroment, but in your poorly lit house or apartment, with crappy mixed lighting. No wonder MR is tough atm.

Let me tell you, the nightmare on filmmakers street is arriving on location, at someones house/apartment etc and that heady mix of 3600 kelvin tungsten light, mixed with natural (viariable) daylight through windows and 5600k daylight lightbulbs and trying to make that look fashionable, oh lawrt!

I'm so glad I'm out of that business!

See you next time.