Diffraction blur is the primary cause of resolution loss in long distance imaging. Consider this illustration of imaging a human-sized object 1 km away. (a) A conventional camera using passive illumination with a fixed aperture size of 12.5 mm induces a diffraction spot size of 50 mm on objects 1 km away, destroying relevant image features. (b) Using Fourier ptychography, an array of cameras using coherent illumination creates a synthetic aperture 10 times larger than (a) resulting in a diffraction spot size of 5 mm for scenes 1 km away. Phase retrieval algorithms recover the high resolution image.

(a) Diffraction blur spot size as a function of distance for common and exotic imaging systems. Consumer cameras (shaded in the pink region) are designed for close to medium range imaging but cannot resolve small features over great distances. Professional lenses (green region) offer improved resolution but are bulky and expensive. Identifying faces 1 kilometer away is possible only with the most advanced super telephoto lenses, which are extremely rare and cost upwards of $2 million US dollars. Obtaining even finer resolution (blue region) is beyond the capabilities of modern manufacturing; the only hope for obtaining such resolution is to use computational techniques such as Fourier ptychography. (b) Affordable camera systems (solid marks) are lightweight but cannot resolve fine details over great distances as well as professional lenses (striped marks) which are heavy and expensive. We propose using a camera array with affordable lenses and active lighting to achieve and surpass the capabilities of professional lenses. Note: These plots only account for diffraction blur and do not consider other factors limiting resolution.

For a given high resolution target (a), the corresponding Fourier transform is formed at the aperture plane of the camera (b). The lens aperture acts as a bandpass filter, only allowing a subset of the Fourier information to pass to the sensor. A larger aperture may be synthesized by scanning the aperture over the Fourier domain and recording multiple images. (c) The full sampling mosaic acquired by scanning the aperture. The dynamic range has been compressed by scaling the image intensities on a log scale. (d) Larger detail images shown of four camera positions, including the center. Image intensities have been scaled linearly, note that only high frequency edge information is present in the three extreme aperture locations. Please zoom in to see details.

Constraints on the image domain magnitude and Fourier domain support are enforced in an alternating manner until convergence or a maximum iteration limit is met.

(a) We simulate imaging a 64x64 mm resolution target 50 meters away using sensor with a pixel pitch of 2 μm. The width of a bar in group 20 is 2.5 mm. (b) The target is observed using a lens with a focal length of 800 mm and an aperture of 18 mm. The aperture is scanned over a 21x21 grid (61% overlap) creating a synthetic aperture of $160$ mm. The output of phase retrieval is a high resolution Fourier representation of the target. The recovered image is shown in (d) and the recovered Fourier magnitude (log scale) is shown in (e). The plot in (c) shows the contrast of the groups in the intensity images of the recovered resolution chart (purple solid line) and the observed center image (blue dashed line). Whereas the central image can only resolve elements which have a width of 12 pixels before contrast drops below 20%, using Fourier ptychography we are able to recover features which are only 2 pixels wide. Detail images of groups 2, 5, and 6, for the ground truth, observed, and recovered images are shown in (f).

Holding the synthetic aperture size constant, we vary the amount of overlap between adjacent images. As the amount of overlap increases, reconstruction quality improves. When the amount of overlap is less than 50%, we are unable to faithfully recover the high resolution image, as shown in the RMSE plot in the top left. The contrast plots of the intensity images for four selected overlap amounts (0%, 41%, 50%, and 75\%) shows that insufficient overlap drastically reduces the resolution of the system. Recovered images show the increase in image reconstruction quality as the amount of overlap increases.

For a fixed overlap of 61%, we vary the size of the synthetic aperture by adding cameras to array. As seen in the contrast plot of image intensities, as well as the reconstructed intensities below, the resolution of the recovered images increase as the SAR increases from 1 (the observed center image) to 11.8. We are able to recover group 2 with an SAR of 5.64, which requires 169 images.

We repeat the varying SAR experiment with the Lena image (assuming the scene is 650 meters away), while varying the amount of added Gaussian noise. The signal-to-noise ratio (SNR) of the input images is set to be 10, 20, and 30 dB. As shown in the RMSE plot, we are able to recover the high resolution image even with noisy input images. The observed center images (SAR=1) are blurry and noisy, with a high RMS error. As the SAR increases, the resolution improves, the noise is suppressed, and RMSE decreases. A cropped portion of select images are shown to highlight the performance of our approach. Note: all other simulation experiments are conducted with an input SNR of 30 dB.

We simulate image capture with an inexpensive 1200 mm lens, 75 mm aperture diameter (f/16), and a pixel pitch of 2 μm. (a) Using passive illumination the resulting diffraction blur (530 μm) removes details necessary to identify the fingerprint. (b) Imaging with a coherent camera array (61% overlap) that captures 81 images to create a synthetic aperture of 300 mm reduces diffraction blur to 130 μm, and leads to faithful recovery of minutiae. (c) Using a $150,000 lens with a 215 mm aperture diameter (f/5.6), the diffraction blur reduces to 190 μm, which is roughly comparable to our final reconstruction in (b). (d) Detail views of the three imaging systems. In this simulation diffraction is the only source of blur.

We simulate image capture with an inexpensive 1200 mm lens, 75 mm aperture diameter (f/16), and a pixel pitch of 2 μm. (a) Directly using the inexpensive f/16 lens in passive illumination results in a diffraction blur spot size of 17.8 mm on the face, obliterating detail necessary to recognize the subject. (b) Using Fourier ptychography (61% overlap, 81 images) to achieve a synthetic aperture of 300 mm the diffraction spot size is reduced to 4.4 mm. (c) Using the $150,000 215 mm (f/5.6) lens yields a diffraction spot size of 6.2 mm, 50% larger than the diffraction realized using Fourier ptychography. Detail views of the three systems are shown in (d). In this simulation diffraction is the only source of blur.

From left to right: A helium neon laser passed through a spatial filter acts as the coherent illumination source. A focusing lens forms the Fourier transform of the transmissive object at the aperture plane of the camera's lens. The aperture acts as a bandpass filter of the Fourier transform and the signal undergoes an inverse Fourier transform as it is focused onto the camera's sensor. The camera (Point Grey Blackfly (BFLY-PGE-50A2M-CS) is mounted on a motorized 2D stage to capture overlapping images.

Using the experimental setup, we use a 75 mm focal length lens to acquire images of a fingerprint ~1.5 m away from the camera. A fingerprint was pressed on a glass slide and fingerprint powder was used to make the ridges opaque. (a) Stopping down the aperture to a diameter of 2.34 mm induces a diffraction spot size of 49 μm on the sensor (~20 pixels) and 990 μm on the object. (b) We record a grid of 17x17 images with an overlap of 81%, resulting in a SAR which is 4 times larger than the lens aperture. After running phase retrieval, we recover a high resolution magnitude image and the phase of the objects. (c) For comparison, we open the aperture to a diameter of 41 mm to reduce the diffraction blur to the size of a pixel. (d) Comparing zoomed in regions of the observed, recovered, and comparison images shows that our framework is able to recover details which are completely lost in a diffraction limited system.

Using the same parameters as in the fingerprint capture we image a diffuse water bottle label ~1.5 m away from the camera. The diffuse nature of the water bottle label results in laser speckle. In the observed center image (a), diffraction blur and laser speckle render the text illegible. Using Fourier ptychography (b) we are able to reduce the effect of speckle and remove diffraction revealing the text. In the comparison image (c) the text is clearly legible. Detail views in (d) show the improvement in image resolution when using Fourier ptychography.

We capture a USAF resolution target 1.5 meters away from the camera. For this experiment, we use a slightly different lens and image sensor (focal length = 75 mm, pixel pitch = 3.75 μm). (a) The camera is stopped down to 2.5 mm which induces a 930 μm blur on the resolution chart, limiting resolution to 1.26 line pairs per millimeter (lp/mm). (b) We record a 23x23 grid of images with 72% overlap, resulting in a SAR of 7.16. Following phase retrieval, we are able to resolve features as small as 8.98 lp/mm, a 7.12x improvement in resolution. (c) We show the effect of varying the SAR on resolution. Using a subset of the captured images we vary the SAR from 2.12 up to 7.16. The gains in improvement closely track the SAR.

(a) Multiple coherent sources illuminate a distant scene. (Note: different colors are used to distinguish light sources; however, we assume each source operates in the same wavelength.) (b) Reflected light undergoes Fraunhofer diffraction forming spatially shifted Fourier transforms at the aperture plane of the camera array. (c) A single camera aperture (e.g. center camera) sums the intensities from each diffraction pattern (top), recording a single image (bottom).

We repeat the conditions of the simulation resolution target experiment using a 7x7 camera array. Each camera has an 800 mm lens with an aperture (front lens diameter) of 25 mm. Cameras are placed such that their apertures abut, though don't overlap, in the Fourier plane. Illumination sources provide 66% overlap among neighboring apertures in the Fourier plane. Nmux, in subcaptions denote the number of active illumination sources per image and T denotes the number of images captured with different illumination patterns. Recovered images show that increasing Nmux and T improves the quality of the reconstructed images.