Computational Photography Around Us
Computational photography as media literacy requirement
Browse through an IKEA catalog and you’ll see a spread of pristine kitchens, living rooms, and bedrooms, each cast with a warm blanket of sunlight. Stools, ottomans, and floor lamps rest stoically in the corner, and silverware sits perfectly knolled in drawers. The mise en scène is spotless.
These images are spotless perhaps because they are not photographs of real places. They are actually 3D renderings of virtual homes, entirely composed of bits and bytes. In 2005 IKEA snuck in its first 3D rendering of their chair “Bertil” into the catalog. No one noticed. By 2014, it was not just one chair: 75% of their catalog’s images were now computer generated. In 2020 we can only guess if any of the images in the catalogue are real at all.
IKEA’s photorealistic 3D renders are just one type of computational photography. Computational photography is a broad term which includes a wide range of techniques. A way to think about computational photography is that it is everything that happens after light has passed through a lens and is processed by a computer. I sometimes call this technique “lensless”, or “post-lens,” photography.
We encounter these types of images every day. You take a photo on your modern smartphone, and it automatically sharpens, deblurs, relights, and detects whose face is in the image. You walk down a street and an incoming car outfitted with depth-sensing sensors, which generates a virtual model of your body, brakes to avoid you. On the way to the park you check your satellite mapping software, which has precise digital models of the city street and surrounding landscape, so you can know what to expect before you get there.
Beyond your day-to-day interactions with computational images, however, computational photography also actively aids in the institutional advancement of health and science. MRIs, CT scans, and X-Rays augment a doctor’s ability to diagnose issues in a patient. Crystallography allows pharmaceutical researchers to visualize the shape of proteins, helping them better develop medicines. Computational photography also helps astronomers visualize the great expanse of the universe beyond our planet.
To this point — the recent image of the black hole at the center galaxy M87 exemplifies computational photography’s power to understand new ideas, which I am personally excited by and an advocate of. The image is computational because it was not captured with one powerful telescope, but with an array of nine different telescopes around the earth, spanning more than half the entire diameter of the world, each precisely oriented and timed with an atomic clock. The Earth was effectively transformed into a planet-size telescope. The 5 total petabytes (one million gigabytes) of image data captured at each station were aligned, denoised, and interpolated with software and output as the famous orange glow we know today. With the help of computational photography, then, the generation of the image helped bolster the theory of General Relativity and provided a more direct calculation of the mass of a black hole.
As with any tech, lensless imaging is a double edged-sword. Yes, the image of the black hole is an indicator of the progress that can arise from computational photography. But complexities arise when it becomes ubiquitous, seamless embedded into the environment, at which point it becomes invisible; it becomes the water. When it is invisible it is simultaneously most useful and the most readily abused. Under the veil of invisibility the technology can express its innate tendencies, reflect its social bias, and invent new problems.
The counter to this complexity is not an amputation of the tech, nor is it an ignorance-is-bliss deal. Contingent on a media literacy of it, computational photography can be properly guided. A guidance from both outside and within a computational photographic institution per se (of which we are all participants) requires a common lingo to even begin to healthily debate its ramifications and decide its limits. To be media literate means to be able to dissect the process by which media is made. If a good cook can taste a dish and pinpoint its ingredients and technique used to make it, a good world-participant should be able to do the same with the media they consume.
