Repository: spacetelescope/jwst

Issue:
Spec2 error for NIRISS SOSS full-frame non-TSO exposure
_Issue [JP-3662](https://jira.stsci.edu/browse/JP-3662) was created on JIRA by [Howard Bushouse](https://jira.stsci.edu/secure/ViewProfile.jspa?name=bushouse):_

jw01209-o001_20240614t162206_spec2_00001_asn.json contains a single NIRISS SOSS exposure taken with full-frame readout and in non-TSO mode. The ATOCA algorithm in extract_1d is not setup to handle full-frame images, so extract_1d properly skips with a warning. That however then causes the photom step to fail, because the input datamodel has no "spec" attribute (no extracted spectrum) and the hook in the photom step that's meant to catch the situation where extract_1d was skipped only works if the input is a "CubeModel". If the SOSS exposure was taken in TSO mode, the input would indeed be a CubeModel and the hook in the photom step would work properly and also cause that step to skip. But in this case, because the exposure was not taken in TSO mode, the input to spec2 is a "rate" file (ImageModel), instead of "rateints" (CubeModel). 

We need to modify the logic in either the spec2 pipeline or the photom step to trap this situation and cleanly skip whenever the extract_1d step has been skipped, regardless of input datamodel type.

Note that this problem was thought to be solved by the work done for JP-3325, but those changes only work for the case where the input is a CubeModel.

Discussion:
emolter: regression tests [started here](https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1540/)
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/spacetelescope/jwst/pull/8575?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 58.63%. Comparing base [(`dba65ae`)](https://app.codecov.io/gh/spacetelescope/jwst/commit/dba65ae8e9e7c650eb6aa303566a6fd3d6bf61e6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) to head [(`7b324c9`)](https://app.codecov.io/gh/spacetelescope/jwst/commit/7b324c9ecd58edf020f8a2bb9639e4f17b28caec?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope).
> Report is 2 commits behind head on master.


<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##           master    #8575   +/-   ##
=======================================
  Coverage   58.63%   58.63%           
=======================================
  Files         391      391           
  Lines       39081    39083    +2     
=======================================
+ Hits        22914    22916    +2     
  Misses      16167    16167           
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/spacetelescope/jwst/pull/8575?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope).

emolter: single regtest failure is unrelated
hbushouse: @emolter So I'm curious as to what kind of outputs, if any, now get created by the Spec2 pipeline in this instance. I assume that a "cal" file is created and that it has all the normal calibrations up to the point of the `extract_1d` step, so it's a useful product. But is an "x1d" file created and, if so, what kind of content does it have? Is it actually a copy of the "cal" data that's simply labeled as an "x1d" file? Or is there no "x1d" file at all?

Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 27f019d4b..00baa2337 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -56,6 +56,12 @@ associations
 - Exclude NIRISS SOSS data taken with uncalibrated filter F277W from spec2 and
   tso3 associations. [#8549]
 
+background_subtraction
+----------------------
+
+- Clarified MIRI MRS default/optional background subtraction steps in the
+  documentation pages.  [#8582]
+
 badpix_selfcal
 --------------
 
@@ -246,6 +252,9 @@ photom
   all slits containing point sources are now handled consistently,
   whether they are marked primary or not. [#8467]
 
+- Added a hook to bypass the ``photom`` step when the ``extract_1d`` step
+  was bypassed for non-TSO NIRISS SOSS exposures. [#8575]
+
 pipeline
 --------
 
@@ -269,6 +278,9 @@ pipeline
   FS slits. Final output products (``cal``, ``s2d``, ``x1d``) contain the
   combined products. [#8467]
 
+- Added a hook to skip ``photom`` step when the ``extract_1d`` step was skipped
+  for NIRISS SOSS data [#8575].
+
 pixel_replace
 -------------
 
diff --git a/docs/jwst/background_subtraction/main.rst b/docs/jwst/background_subtraction/main.rst
index 9aa995fca..2dade7dea 100644
--- a/docs/jwst/background_subtraction/main.rst
+++ b/docs/jwst/background_subtraction/main.rst
@@ -98,9 +98,9 @@ when the available data support multiple methods.
 +--------------------------+---------------+-------------------+-----------------------------+
 | **MIRI MRS:**            |               |                   |                             |
 +--------------------------+---------------+-------------------+-----------------------------+
-| Dedicated background     | Default       | Optional          |                             |
+| Dedicated background     | Optional      | Default           |                             |
 +--------------------------+---------------+-------------------+-----------------------------+
-| Nodded point source      | Default       | Optional          |                             |
+| Nodded point source      | Optional      | Optional          |                             |
 +--------------------------+---------------+-------------------+-----------------------------+
 | User supplied            |               | Default           |                             |
 +--------------------------+---------------+-------------------+-----------------------------+
diff --git a/jwst/photom/photom.py b/jwst/photom/photom.py
index c445d9601..7628582e3 100644
--- a/jwst/photom/photom.py
+++ b/jwst/photom/photom.py
@@ -425,6 +425,9 @@ class DataSet():
             raise DataModelTypeError(f"Unexpected input data model type for NIRISS: {self.input.__class__.__name__}")
 
         elif self.exptype in ['NIS_SOSS']:
+            if isinstance(self.input, datamodels.ImageModel):
+                raise DataModelTypeError(f"Unexpected input data model type for NIRISS: {self.input.__class__.__name__}")
+
             for spec in self.input.spec:
                 self.specnum += 1
                 self.order = spec.spectral_order
diff --git a/jwst/pipeline/calwebb_spec2.py b/jwst/pipeline/calwebb_spec2.py
index 9ce8c04d6..9f3004f4a 100644
--- a/jwst/pipeline/calwebb_spec2.py
+++ b/jwst/pipeline/calwebb_spec2.py
@@ -362,11 +362,11 @@ class Spec2Pipeline(Pipeline):
             x1d = self.extract_1d(x1d)
 
             # Possible that no fit was possible - if so, skip photom
-            if x1d is not None:
+            if (x1d is None) or (x1d.meta.cal_step.extract_1d == "SKIPPED"):
+                self.log.warning("Extract_1d did not return a DataModel - skipping photom.")
+            else:
                 self.photom.save_results = self.save_results
                 x1d = self.photom(x1d)
-            else:
-                self.log.warning("Extract_1d did not return a DataModel - skipping photom.")
         elif exp_type == 'NRS_MSASPEC':
             # Special handling for MSA spectra, to handle mixed-in
             # fixed slits separately


Test patch:
diff --git a/jwst/photom/tests/test_photom.py b/jwst/photom/tests/test_photom.py
index 43c7d5e60..2fc0f5b68 100644
--- a/jwst/photom/tests/test_photom.py
+++ b/jwst/photom/tests/test_photom.py
@@ -1357,6 +1357,21 @@ def test_expected_failure_niriss_cubemodel():
         ds.calc_niriss(None)
 
 
+def test_expected_failure_soss_imagemodel():
+    """
+    Test that passing a CubeModel to calc_niriss raises an exception
+    This occurs when extract_1d step is skipped, e.g. for NIRISS SOSS data
+    in FULL subarray.
+    """
+
+    input_model = create_input('NIRISS', 'NIS', 'NIS_SOSS',
+                               filter='CLEAR', pupil='GR700XD')
+    ds = photom.DataSet(input_model)
+    ds.input = datamodels.ImageModel()
+    with pytest.raises(photom.DataModelTypeError):
+        ds.calc_niriss(None)
+
+
 def test_miri_mrs():
     """Test calc_miri, MRS data"""