Halide developers break down the technology powering Apple’s new ProRAW image format

Apple’s new ProRAW format is now available to iPhone 12 Pro and 12 Pro Max users running iOS 14.3. The new format promises the power of Apple’s image processing capabilities with the flexibility of a Raw image. But what exactly is a ProRAW image, how useful is the format and what sort of results can we expect from it?

Ben Sandofsky, one of the developers behind the iOS camera app Halide, has answered all of those questions and more in a thorough overview of the ProRAW technology, with illustrations and example photos from Halide designer, Sebastiaan de With.

Sandofsky’s overview is effectively broken down into three parts. That first part explains how a digital camera ‘develops’ an image using the data from the sensor. The second part explains the pros and cons of traditional Raw files. And the third part is where we learn what’s unique about Apple’s ProRAW technology and why it ‘changes the game’ despite ‘its few remaining drawbacks.’

We suggest taking Sandofky’s advice and grabbing a coffee if you plan on reading the entire post, but we’ll summarize a few of the standout details here on DPReview as well in the case you’re in a rush or are out of coffee.

Since most DPReview readers are familiar with how a camera processes data from the sensor, as well as the pros and cons of Raw files, we’re going to skip right to Sandofky’s rundown of the new ProRAW technology from Apple.

Sandofsky gets straight to the point, answering a question we’ve seen a few people ask in the comments of previous articles on Apple’s new ProRAW technology — ‘Technically, there’s no such thing as a ProRAW file.’ ProRAW image files, as he explains, are simply regular DNG files that use little known features in the DNG file format specifications, as well as a few new features Apple worked with Adobe to implement in the new 1.6 specification.

Specifically, Adobe and Apple worked together to add a new set of tags for the open Raw format that enable much of what Apple is doing with its ProRAW-branded DNG files. In fact, the new DNG 1.6 specification went live the day Apple released the iOS 14.3 public beta, showing just how closely the two worked together on the new specification.

As Sandofsky points out, Apple opting to use the DNG format means, despite their attempt at rebranding the technology as its own Raw format, nothing about the resulting files is proprietary. The photos should, in theory, be able to work with any other device or program that chooses to make the most of the new DNG 1.6 standard.

As for what makes ProRAW files different than other Raw file formats, Sandofsky starts off by explaining that ProRAW images store pixel values after the demosaic step. These demosaiced color values still contain all of the original colors and dynamic range of the scene, they just cut out the step of choosing what demosaic algorithm will be used on that data—something usually done in post-processing based on the editing software you’re using.

Sandofsky argues that, aside from saving time and taking another step out of the equation, it’s ‘quite possible that iOS can do a better job demosaicing your images than any third-party RAW editor’ considering Apple’s strength of its vertical integration of hardware and softrware. That is, Apple ‘know[s] exactly the sensor you’re using, and how it behaves with different ISO settings.’ Sandofsky even points out that with Apple’s image recognition technology, iOS could apply a specific demosaic algorithm depending on the scene to achieve an optimal result.

One example he uses is a set of starscape images from photographer Austin Mann, who has also shared a detailed breakdown of the new ProRAW technology on his website alongside a beautiful collection of images.

A ProRAW file before and after editing.

Another benefit of this approach is Apple could, in theory, start to develop its own sensors, eschewing the current bayer sensors they use. By doing so, they could tailor-make the best sensor possible for their smartphones and as long as the resulting image is saved as a ProRAW file, ‘it would work from day one in every pro photography process and app like Lightroom without having to wait for Adobe to write a new demosaic algorithm’ for the new sensor.

Next, Sandofsky explains how Apple is baking in the ‘local tone mapping and other computational photography goodies’ right inside the DNG using the new tags found in the DNG 1.6 standard. By using the new tags, Apple is able to apply its Smart HDR and Deep Fusion technology to the ProRAW images. While Halide is opting to not use all of this technology — Sandofsky explains why in detail within his post — it’s still there to leverage in the image editor of your choice or use with another third-party camera app should the developers choose to. ProRAW images also store semantic maps — the depth data used in Portrait mode — which gives developers yet another piece of information to work with.

A comparison image showing the semantic maps of an image next to the original photo.

Sandofsky goes on to explain the file size flexibility of ProRAW files. A standard ProRAW file, captured with the default 12-bit data, is roughly 25MB. But it’s possible for developers to drop that to 10-bit, which roughly halves the file size while still getting ‘most of the ProRAW benefits.’ If that’s not small enough, it’s also possible for developers to use lossy compression on ProRAW files, which drops the files down to as little as 1MB, but as Sandofsky notes, this results in a ProRAW file that isn’t much more versatile than a standard 8-bit JPEG.

A list of Raw and ProRAW shooting options the Halide team has implemented into its camera app.

Another neat trick is that it’s possible to store a fully-processed JPEG version of an image directly inside the DNG file. Apple doesn’t do this by default in its iOS Camera app, third party developers can choose to do this, which would ensure apps that don’t support DNG files, such as Instagram, can still use the processed JPEG image. Halide says it’s added this option in its app, but do note it will add an extra 4MB or so to the file size, due to the extra baggage.

In conclusion, Sandofsky notes that ProRAW ‘is a leap forward for everyone, but it will be especially impactful for beginning photographers.’ With Apple supporting Raw editing directly in its iOS Photos app, even casual photographers will be able to have the flexibility of editing a ‘Raw’ file format — even if they’re not using more advanced editing apps such as Affinity Photo, Adobe Lightroom, Capture One, etc.

Sandofsky wraps up his post talking about how the Halide team is further pushing the boundaries of ProRAW with its app, but we’ll let you read that over on the full post, linked below. You can download the Halide camera app in the iOS App Store.

Understanding ProRAW

Image credits: All images and illustrations by Halide, used with permission.