DOI: 10.5061/dryad.6t1g1jx42
title: NeuWS: Neural Wavefront Shaping for guidestar-free imaging through static and dynamic scattering media

description: Diffraction-limited optical imaging through scattering media has the
 potential to transform many applications such as airborne and space-based
 imaging (through the atmosphere), bio-imaging (through skin and human
 tissue), and fiber-based-imaging (through fiber bundles). Existing
 wavefront shaping methods can image through scattering media and other
 obscurants by optically correcting wavefront aberrations using
 high-resolution spatial light modulators—but these methods generally
 require (a) guidestars, (b) controlled illumination, (c) point scanning
 and/or (d) statics scenes and aberrations. We propose Neural Wavefront
 Shaping (NeuWS), a scanning-free wavefront shaping technique which
 integrates maximum likelihood estimation, measurement modulation, and
 neural signal representations to reconstruct diffraction-limited images
 through strong static and dynamic scattering media without guidestars,
 sparse targets, controlled illumination, nor specialized image sensors. We
 experimentally demonstrate guidestar-free, wide-field-of-view,
 high-resolution, diffraction-limited imaging of extended, non-sparse,
 static/dynamic scenes captured through static/dynamic aberrations.
description: <strong>Data:</strong> The
 NeuWS_data directory contains all the measurements and corresponding SLM
 patterns of all the experiments shown in the article. The data are divided
 into four groups based on imaging dynamic/static objects through
 dynamic/static aberrations. The data from the automatic refocusing
 experiments is also provided. In each folder, the
 Zernike_SLM_data subfolder includes the patterns projected on the SLM
 (SLM_sim) and the raw measurements captured by the camera
 (SLM_raw).  For static scenes and aberrations, the
 "uncorrected" image is a measurement with aberrations and
 without SLM modulation. For dynamic scenes and aberrations,
 "Uncorrected GIF" is generated by combining the images from the
 corresponding Zero subfolder (uncorrected measurements). The Zero
 subfolders contain the measurements captured without any SLM modulation;
 i.e., the uncorrected measurements one would observe through the
 aberration.  The "optical_correct" images and
 GIFS show the measurements captured through an aberration when the
 conjugate of the estimated aberration was projected on the SLM. For the
 dynamic experiments, the corresponding raw data is provided in the
 "optical_correct_frames" folder.
 "GT" is the ground truth image captured without any
 aberrations or SLM modulation. We generated each
 modulation pattern by creating a 256 × 256 image made from the weighted
 sum of the first 15 Zernike basis functions, where each weight was sampled
 from an independent Gaussian distribution with a standard deviation of 5
 radians. These patterns were then evenly cropped on the top and bottom to
 144×256 to match our SLM’s aspect ratio.  The optically
 corrected measurements (image data in optical_correct_frames directories)
 are unprocessed 1384 ×
 1036 images captured by our
 camera.  The modulated and uncorrected measurements (image
 data in Zernike_SLM_data and Zero directories) are 256x256 images. These
 were formed by first resizing 1280x1026 camera measurements to
 387 × 290 and then cropping a 256 × 256 area in the center. The images were resized to
 reduce the computational cost of running NeuWS.
 The data collection process is described in detail in our
 paper. <strong>Code</strong>:
 The NeuWS_code directory contains all the code necessary to run
 NeuWS on the provided data. Please see the included README.md file for
 information on setting up the code. It assumes you have access to a recent
 Nvidia GPU.  For a more complete, up-to-date, and
 optimized implementation of NeuWS please refer to the NeuWS
 repository hosted on GitHub.
description: The SLM patterns are stored as Matlab .mat files. They can be
 opened with Python's open-source SciPy module. This is handled by
 "dataset.py" in our code.   Information on
 how to set up your Python environment to run NeuWS can be found in the
 README.md file.