MIT Researchers Developing Modulo Camera with Unlimited Dynamic Range

A team of researchers at MIT (Massachusetts Institute of Technology) has developed a new technique that may allow cameras to take photos with the ability to capture high dynamic range scenes without any overexposure of brighter areas.   Here is a simulated image from their research paper using the new algorithms with what they're calling a Modula Camera.

mit1.jpegThey accomplish avoiding overexposure in an image by resetting a photosite after it comes saturated (filled with electrons to its capacity), then capturing light again until it has a readable value of light without over saturation; while keeping track of the number of times the photosite was reset (emptied, with capture of electrons starting over again). That lets them compute the amount of light a photosite with unlimited well capacity would have recorded, by integrating the final value captured with the number of resets performed. That's an over simplified explanation.  

So, to read more detail about it, see this paper from the MIT researchers about the Modulo Camera concept: http://web.media.mit.edu/~hangzhao/papers/moduloUHDR.pdf

Here's a quote from that research paper:

"This paper presents a novel framework to extend the dynamic range of images called Unbounded High Dynamic Range (UHDR) photography with a modulo camera. A  modulo camera could theoretically take unbounded radiance levels by keeping only the least significant bits. We show that with limited bit depth, very high radiance levels can be recovered from a single modulus image with our newly proposed unwrapping algorithm for natural images."

See this project page for much more information about the Modula camera project, with links to articles, videos, sample images, papers presented and other ongoing work: http://web.media.mit.edu/~hangzhao/modulo.html

steves_line.gif
News about it from MIT Media Lab

http://www.media.mit.edu/research/highlights/unbounded-high-dynamic-range-photography-using-modulo-camera

Unbounded High Dynamic Range Photography Using a Modulo Camera

Camera Culture group research may be the beginning of the end for over-saturated images

Trying to take pictures in the dark or through a window is difficult for professional photographers and everyday people alike. A group of researchers at MIT have proposed a camera that can take a perfect picture, no matter what the lighting contrast is. Called a "modulo camera," this camera is designed to never overexpose an image, enabling high dynamic range photography. This achievement was awarded the best paper runner-up at the 2015 International Conference on Computational Photography.

High dynamic range (HDR) imaging is a method that allows both very bright and very dim light sources to be pictured in a single image with no loss in quality. HDR cameras have been created before, but conventional HDR cameras use multiple normal images to create one final HDR image. This means that if the camera is shaking, or if the image is of a moving target, the HDR technique does not work. However, the modulo camera, created in a collaboration between the Media Lab's Camera Culture group, MIT Lincoln Lab, and Singapore University of Technology and Design, only requires one shot to create an HDR image. This not only allows HDR photos to be taken free of blur, but also allows for the possibility of HDR video.

Conventional camera sensors will get "full," or saturated, after receiving an excess amount of light. This is because conventional camera sensors have a limited "well capacity," or a limited amount of light the sensors can take in before they overflow. The modulo camera solves the saturation problem by resetting the sensor capacitors whenever the "well" gets full, and uses an inverse modulo algorithm to calculate how much light the reset sensors took in. This algorithm recovers a much larger dynamic range. For example, if a certain camera sensor can record eight bits of information, then when those eight bits are filled, the capacitor will be reset to zero. The number of resets is recovered by the algorithm, which then calculates the relative brightness of each area of the photo.

There are numerous uses for this technology. No more will photographers or even ordinary people have to fumble with aperture size and exposure length. The algorithm would enable people simply to click the camera button and let the computer deal with exposure problems. The modulo camera can potentially transform the way everyday photography works.

Furthermore, cameras serve as visual input for robots. Clarity in all lighting conditions is crucial to robotic vision. However, good lighting cannot always be guaranteed. When a driverless car drives into a tunnel, an ordinary camera goes blind: the exit ahead is bright, and the surroundings are dark, so the camera cannot see both inside the tunnel and out of it. A real-time HDR camera could guarantee safety in these conditions. The list of real-world applications goes on: laser image speckling, astronomy, and any field that deals with sources of both bright and low light could be transformed by the modulo camera technology.