Repository: spacetelescope/jwql

Issue:
Mostly white pixels on preview image and thumbnail
**Describe the bug**
For some NIRISS '_rate.fits' images with high background flux (e.g. https://jwql.stsci.edu/NIRISS/jw01208034001_02101_00001_nis/), the preview image and the thumbnail shows mostly white pixels that make the image look like a failed observation. However, inspecting the image shows the white pixels are not present. 

**To Reproduce**
Steps to reproduce the behavior:
1. Go to https://jwql.stsci.edu/NIRISS/jw01208034001_02101_00001_nis/
2. Click on the radio button for 'rate' file type to see the white pixels.
3. Click on the 'Explore Image' button to see that the pixels aren't white after all but the background is high.

**Expected behavior**
The preview image should look the same as the full image viewed after clicking the 'Explore Image' button

**Additional context**
The background in the affected images are 10 - 25% of the signal flux, pretty high but not unusable. 



Discussion:
I'm seeing this in a few other cases, all with data taken very recently: 
https://jwql.stsci.edu/NIRCam/archive/2514/obs011/
https://jwql.stsci.edu/NIRCam/archive/1783/obs001/
https://jwql.stsci.edu/MIRI/archive/1783/obs002/


And more:
https://jwql.stsci.edu/NIRSpec/jw01491027001_01201_00001_nrs1/

I wonder if this is related to the dependency updates that were made recently. Maybe a color palette has changed, or something like that?
Looking more closely at the data in ds9, I'm thinking this is due to the recent change in the pipeline where bad pixels (those with the DO_NOT_USE flag set in the DQ array) are given a value of NaN in the rate image. Prior to this change, these pixels were given a value of zero. I think that when the preview images are being created, these NaN pixels, which are white in the current color palette, are not disappearing as well as they did previously when matplotlib bins/averages/downsamples the pixels during the creation of the jpg. 

So maybe we need a new step in the creation of the preview images that replaces NaN values with local averages before saving the jpgs? Or return to the previous situation and set them equal to zero?

Resolving patch:
diff --git a/jwql/utils/preview_image.py b/jwql/utils/preview_image.py
--- a/jwql/utils/preview_image.py
+++ b/jwql/utils/preview_image.py
@@ -576,6 +576,11 @@ def make_image(self, max_img_size=8.0, create_thumbnail=False):
             # Find signal limits for the display
             minval, maxval = self.find_limits(frame)
 
+            # Set NaN values to zero, so that those pixels
+            # do not appear as big white splotches in the jpgs
+            # after matplotlib downsamples/averages
+            frame = nan_to_zero(frame)
+
             # Create preview image matplotlib object
             indir, infile = os.path.split(self.file)
             suffix = '_integ{}.{}'.format(i, self.output_format)
@@ -757,3 +762,20 @@ def expand_for_i2d(array, xdim, ydim):
         return new_array_x
     else:
         return array
+
+def nan_to_zero(image):
+    """Set any pixels with a value of NaN to zero
+
+    Parameters
+    ----------
+    image : numpy.ndarray
+        Array from which NaNs will be removed
+
+    Returns
+    -------
+    image : numpy.ndarray
+        Input array with NaNs changed to zero
+    """
+    nan = np.isnan(image)
+    image[nan] = 0
+    return image


Test patch: