Repository: spacetelescope/jwst

Issue:
index out of bounds error for NRC_TSGRISM level_3
_Issue [JP-3308](https://jira.stsci.edu/browse/JP-3308) was created on JIRA by [Hien Tran](https://jira.stsci.edu/secure/ViewProfile.jspa?name=htran):_

processing of new nrc_tsgrism level_3 dataset [jw01550-c1001_20230717t083335_tso3_00002](https://pljwdmsweb.stsci.edu/owlgui/logs/directory_view?directory_path=/ifs/archive/ops/jwst/info/owl/logs/owlmgr_jw01550-c1001_20230717t083335_tso3_00002_1689585038.963402&fileset_name=jw01550-c1001_20230717t083335_tso3_00002&dag_name=LEVEL_3) (consisting of observations 1550:28 and 1550:29) with b9.2.1 gave the following error 
```
2023-07-17 09:17:39,554 - stpipe.Tso3Pipeline.extract_1d - INFO - Step extract_1d done
2023-07-17 09:17:39,565 - stpipe.Tso3Pipeline - INFO - Performing white-light photometry ...
2023-07-17 09:17:39,689 - stpipe.Tso3Pipeline.white_light - INFO - Step white_light running with args (<MultiSpecModel from jw01550028001_03103_00001-seg001_nrcalong_calints.fits>,).
2023-07-17 09:17:39,690 - stpipe.Tso3Pipeline.white_light - INFO - Step white_light parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.ecsv', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'whtlt', 'search_output_file': True, 'input_dir': '/ifs/archive/ops/jwst/info/owlmgr/paths/sdp/asn_creation/cal/level3', 'min_wavelength': None, 'max_wavelength': None}
2023-07-17 09:17:39,698 - stpipe.Tso3Pipeline.white_light - INFO - Step white_light done
2023-07-17 09:17:39,699 - stpipe.Tso3Pipeline - INFO - Extracting 1-D spectra ...
2023-07-17 09:17:39,812 - stpipe.Tso3Pipeline.extract_1d - INFO - Step extract_1d running with args (<SlitModel(3, 64, 2048) from jw01550029001_03103_00001-seg001_nrcalong_calints.fits>,).
2023-07-17 09:17:39,814 - stpipe.Tso3Pipeline.extract_1d - INFO - Step extract_1d parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': 'extract_1d', 'search_output_file': True, 'input_dir': '/ifs/archive/ops/jwst/info/owlmgr/paths/sdp/asn_creation/cal/level3', 'smoothing_length': None, 'bkg_fit': None, 'bkg_order': None, 'bkg_sigma_clip': 3.0, 'log_increment': 50, 'subtract_background': None, 'use_source_posn': None, 'center_xy': None, 'apply_apcorr': True, 'soss_atoca': True, 'soss_threshold': 0.01, 'soss_n_os': 2, 'soss_wave_grid_in': None, 'soss_wave_grid_out': None, 'soss_estimate': None, 'soss_rtol': 0.0001, 'soss_max_grid_size': 20000, 'soss_transform': None, 'soss_tikfac': None, 'soss_width': 40.0, 'soss_bad_pix': 'masking', 'soss_modelname': None}
2023-07-17 09:17:40,084 - CRDS - DEBUG - ========================================================================================================================
2023-07-17 09:17:40,085 - CRDS - DEBUG - getreferences() CRDS version: 11.16.22, b11.4.0, 64d96076d89b32a5687a6b77bb910ab93b3a99b3
2023-07-17 09:17:40,085 - CRDS - DEBUG - getreferences() server: https://jwst-serverless-mode.stsci.edu
2023-07-17 09:17:40,085 - CRDS - DEBUG - getreferences() observatory: jwst
2023-07-17 09:17:40,085 - CRDS - DEBUG - getreferences() reftypes: ('extract1d',)
2023-07-17 09:17:40,085 - CRDS - DEBUG - getreferences() context: None
2023-07-17 09:17:40,085 - CRDS - DEBUG - getreferences() ignore_cache: False
2023-07-17 09:17:40,085 - CRDS - DEBUG - CRDS_PATH = /ifs/archive/ops/jwst/ref/tmp_crds/crds/cache
2023-07-17 09:17:40,085 - CRDS - DEBUG - CRDS_SERVER_URL = https://jwst-serverless-mode.stsci.edu
2023-07-17 09:17:40,086 - CRDS - DEBUG - Final effective context is 'jwst_1100.pmap'
2023-07-17 09:17:40,086 - CRDS - DEBUG - Computing best references locally.
2023-07-17 09:17:40,087 - CRDS - DEBUG - Bestrefs header:
{'META.EXPOSURE.NOUTPUTS [NOUTPUTS]': '4.0',
'META.EXPOSURE.READPATT [READPATT]': 'RAPID',
'META.EXPOSURE.TYPE [EXP_TYPE]': 'NRC_TSGRISM',
'META.INSTRUMENT.CHANNEL [CHANNEL]': 'LONG',
'META.INSTRUMENT.CORONAGRAPH [CORONMSK]': 'UNDEFINED',
'META.INSTRUMENT.DETECTOR [DETECTOR]': 'NRCALONG',
'META.INSTRUMENT.FILTER [FILTER]': 'F356W',
'META.INSTRUMENT.LAMP_STATE [LAMP]': 'UNDEFINED',
'META.INSTRUMENT.MODULE [MODULE]': 'A',
'META.INSTRUMENT.NAME [INSTRUME]': 'NIRCAM',
'META.INSTRUMENT.PUPIL [PUPIL]': 'GRISMR',
'META.OBSERVATION.DATE [DATE-OBS]': '2023-07-16',
'META.OBSERVATION.TIME [TIME-OBS]': '20:16:39.327',
'META.SUBARRAY.NAME [SUBARRAY]': 'SUBGRISM128',
'META.VISIT.CROWDED_FIELD [CROWDFLD]': 'F',
'META.VISIT.TSOVISIT [TSOVISIT]': 'T',
'META.VISIT.TYPE [VISITYPE]': 'PRIME_TARGETED_FIXED',
'REFTYPE': 'UNDEFINED'}
2023-07-17 09:17:40,088 - stpipe.Tso3Pipeline.extract_1d - INFO - Using EXTRACT1D reference file N/A
2023-07-17 09:17:40,214 - CRDS - DEBUG - ========================================================================================================================
2023-07-17 09:17:40,215 - CRDS - DEBUG - getreferences() CRDS version: 11.16.22, b11.4.0, 64d96076d89b32a5687a6b77bb910ab93b3a99b3
2023-07-17 09:17:40,215 - CRDS - DEBUG - getreferences() server: https://jwst-serverless-mode.stsci.edu
2023-07-17 09:17:40,215 - CRDS - DEBUG - getreferences() observatory: jwst
2023-07-17 09:17:40,215 - CRDS - DEBUG - getreferences() reftypes: ('apcorr',)
2023-07-17 09:17:40,215 - CRDS - DEBUG - getreferences() context: None
2023-07-17 09:17:40,215 - CRDS - DEBUG - getreferences() ignore_cache: False
2023-07-17 09:17:40,215 - CRDS - DEBUG - CRDS_PATH = /ifs/archive/ops/jwst/ref/tmp_crds/crds/cache
2023-07-17 09:17:40,215 - CRDS - DEBUG - CRDS_SERVER_URL = https://jwst-serverless-mode.stsci.edu
2023-07-17 09:17:40,215 - CRDS - DEBUG - Final effective context is 'jwst_1100.pmap'
2023-07-17 09:17:40,215 - CRDS - DEBUG - Computing best references locally.
2023-07-17 09:17:40,216 - CRDS - DEBUG - Bestrefs header:
{'META.EXPOSURE.NOUTPUTS [NOUTPUTS]': '4.0',
'META.EXPOSURE.READPATT [READPATT]': 'RAPID',
'META.EXPOSURE.TYPE [EXP_TYPE]': 'NRC_TSGRISM',
'META.INSTRUMENT.CHANNEL [CHANNEL]': 'LONG',
'META.INSTRUMENT.CORONAGRAPH [CORONMSK]': 'UNDEFINED',
'META.INSTRUMENT.DETECTOR [DETECTOR]': 'NRCALONG',
'META.INSTRUMENT.FILTER [FILTER]': 'F356W',
'META.INSTRUMENT.LAMP_STATE [LAMP]': 'UNDEFINED',
'META.INSTRUMENT.MODULE [MODULE]': 'A',
'META.INSTRUMENT.NAME [INSTRUME]': 'NIRCAM',
'META.INSTRUMENT.PUPIL [PUPIL]': 'GRISMR',
'META.OBSERVATION.DATE [DATE-OBS]': '2023-07-16',
'META.OBSERVATION.TIME [TIME-OBS]': '20:16:39.327',
'META.SUBARRAY.NAME [SUBARRAY]': 'SUBGRISM128',
'META.VISIT.CROWDED_FIELD [CROWDFLD]': 'F',
'META.VISIT.TSOVISIT [TSOVISIT]': 'T',
'META.VISIT.TYPE [VISITYPE]': 'PRIME_TARGETED_FIXED',
'REFTYPE': 'UNDEFINED'}
2023-07-17 09:17:40,217 - stpipe.Tso3Pipeline.extract_1d - INFO - APCORR reference file name is "N/A"
2023-07-17 09:17:40,217 - stpipe.Tso3Pipeline.extract_1d - INFO - APCORR will NOT be applied
2023-07-17 09:17:40,241 - stpipe.Tso3Pipeline.extract_1d - INFO - Processing spectral order 1
2023-07-17 09:17:40,241 - stpipe.Tso3Pipeline.extract_1d - INFO - Beginning loop over 3 integrations ...
2023-07-17 09:17:40,241 - stpipe.Tso3Pipeline.extract_1d - INFO - Extracting integration 1
2023-07-17 09:17:40,243 - stpipe.Tso3Pipeline.extract_1d - INFO - Using extraction limits: xstart=0, xstop=2047, ystart=0, ystop=63
2023-07-17 09:17:40,638 - stpipe.Tso3Pipeline.extract_1d - INFO - Extracting integration 2
2023-07-17 09:17:41,032 - stpipe.Tso3Pipeline.extract_1d - INFO - Extracting integration 3
2023-07-17 09:17:41,420 - stpipe.Tso3Pipeline.extract_1d - INFO - All 3 integrations done
2023-07-17 09:17:41,450 - stpipe.Tso3Pipeline.extract_1d - INFO - Step extract_1d done
Traceback (most recent call last):
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/bin/strun", line 28, in <module>
step = Step.from_cmdline(sys.argv[1:])
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/lib/python3.9/site-packages/stpipe/step.py", line 179, in from_cmdline
return cmdline.step_from_cmdline(args)
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/lib/python3.9/site-packages/stpipe/cmdline.py", line 343, in step_from_cmdline
step.run(*positional)
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/lib/python3.9/site-packages/stpipe/step.py", line 454, in run
step_result = self.process(*args)
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/lib/python3.9/site-packages/jwst/pipeline/calwebb_tso3.py", line 173, in process
x1d_result.int_times[row[0] - 1] = row
File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/lib/python3.9/site-packages/astropy/io/fits/fitsrec.py", line 574, in __setitem__
self.field(self.names[idx])[key] = value.field(self.names[idx])
IndexError: index 2 is out of bounds for axis 0 with size 2
2023198091741 INFO src=ssb_calibration_wrapper._strun_analyze_and_log_failure fsn=jw01550-c1001_20230717t083335_tso3_00002_asn msg="strun /dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-1.10.1.20230413-py3.9/lib/python3.9/site-packages/jwst/pipeline/calwebb_tso3.cfg FAILED (exit=1) on jw01550-c1001_20230717t083335_tso3_00002_asn.json."
2023198091741 ERROR src=ssb_calibration_wrapper.calibrate._strun._strun_analyze_and_log_failure fsn=jw01550-c1001_20230717t083335_tso3_00002_asn msg="strun exit status=1"
2023198091742-E-ERROR-calibration_wrapper.csh FAILED. Archiving will NOT be attempted.
 1 / 1  ```
it crashed right after +*completing*+ the extract_1d step but just before performing white-light photometry for jw015500{*}29{*}001 . it had successfully completed all steps for jw015500{*}28{*}001 earlier.

could be related to JP-3146?

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/spacetelescope/jwst/pull/7982?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) Report
Attention: `1 lines` in your changes are missing coverage. Please review.

| [Files](https://app.codecov.io/gh/spacetelescope/jwst/pull/7982?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope) | Coverage Δ | |
|---|---|---|
| [jwst/associations/lib/rules\_level3.py](https://app.codecov.io/gh/spacetelescope/jwst/pull/7982?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope#diff-andzdC9hc3NvY2lhdGlvbnMvbGliL3J1bGVzX2xldmVsMy5weQ==) | `95.85% <80.00%> (-0.49%)` | :arrow_down: |


:loudspeaker: Thoughts on this report? [Let us know!](https://about.codecov.io/pull-request-comment-report/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=spacetelescope).

tapastro: Regtests: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/978/
hbushouse: I put the jw01550 pool file in /grp/jwst/ssb/chartreuse/dms_bugs/jp-3308. You could try running this PR branch on that pool (i.e. "asn_generate jw01550_20230830t034805_pool.csv") and see if the c1001 tso3 asn goes away now.
tapastro: > I put the jw01550 pool file in /grp/jwst/ssb/chartreuse/dms_bugs/jp-3308. You could try running this PR branch on that pool (i.e. "asn_generate jw01550_20230830t034805_pool.csv") and see if the c1001 tso3 asn goes away now.

I don't have write permissions in that subdir, but I did this test locally to verify my fix was functional. I can reproduce it in the public dir with a permissions update, and/or we could consider addiing the pool to the regression tests. It's a bit narrow, though, and my reading of the regtest timing on Jenkins may imply that associations are the rate-limiting test? Hard to tell if the individual tests are done in parallel or not.
hbushouse: I updated permissions on the central store directory to include group write, so you should be able to write to it now(?). I ran the existing rules on the pool file, the results of which are in the "old_results" subdir, for comparison.
tapastro: Well, I tried an alternative approach - rather than disallow the groups, I'm instead _only_ allowing observation association IDs. This prevents the "discovered" association IDs from generating associations, which I saw in testing without the `--all-candidates` flag.
tapastro: It looks like that test expects the test pools to return only discovered associations, i.e. a3001 associations. I don't know if it's a test representative of typical use cases - perhaps it's best to double-check with @stscieisenhamer ?
stscieisenhamer: wow, this would be one of the first tests ever written, using a test class. So, the way the generator is run, yes, it will only be creating "a" type associations: no other candidate information/constraints are specified. So, yes, this test could very well be affected if the one association was based on TSO data in that pool. If that association is not to exist any longer, then updating the the truth from "1" to "0" is probably appropriate.
tapastro: Trying an association-specific regtest run: https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/983/


Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 393cc99e5..e2878fb2a 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -6,6 +6,11 @@ assign_wcs
 
 - Increase margin at edges of NIRSpec MOS slits to reduce edge effects in resampling. [#7976]
 
+associations
+------------
+
+- Prevent group candidates from generating level 3 TSO associations. [#7982]
+
 charge_migration
 ----------------
 
diff --git a/jwst/associations/lib/rules_level3.py b/jwst/associations/lib/rules_level3.py
index 9d06b3200..15644e1f5 100644
--- a/jwst/associations/lib/rules_level3.py
+++ b/jwst/associations/lib/rules_level3.py
@@ -906,6 +906,14 @@ class Asn_Lv3TSO(AsnMixin_Science):
             )
         ])
 
+        # Only valid if candidate type is 'observation'.
+        self.validity.update({
+            'is_type_observation': {
+                'validated': False,
+                'check': self._validate_candidates
+            }
+        })
+
         super(Asn_Lv3TSO, self).__init__(*args, **kwargs)
 
     def _init_hook(self, item):
@@ -914,6 +922,26 @@ class Asn_Lv3TSO(AsnMixin_Science):
         self.data['asn_type'] = 'tso3'
         super(Asn_Lv3TSO, self)._init_hook(item)
 
+    def _validate_candidates(self, member):
+        """Allow only observation-type candidates
+
+        Parameters
+        ----------
+        member : Member
+            Member being added. Ignored.
+
+        Returns
+        -------
+        True if candidate type is observation.
+        False otherwise.
+        """
+
+        # If a group candidate, reject.
+        if self.acid.type.lower() != 'observation':
+            return False
+
+        return True
+
 
 @RegistryMarker.rule
 class Asn_Lv3WFSCMB(AsnMixin_Science):


Test patch:
diff --git a/jwst/associations/tests/test_level3_basics.py b/jwst/associations/tests/test_level3_basics.py
index 337f55874..bc92aba92 100644
--- a/jwst/associations/tests/test_level3_basics.py
+++ b/jwst/associations/tests/test_level3_basics.py
@@ -31,7 +31,6 @@ def test_meta():
 @pytest.mark.parametrize(
     'pool_file',
     [
-        'data/pool_005_spec_niriss.csv',
         'data/pool_006_spec_nirspec.csv',
         'data/pool_007_spec_miri.csv',
         'data/pool_010_spec_nirspec_lv2bkg.csv',
diff --git a/jwst/associations/tests/test_level3_spectrographic.py b/jwst/associations/tests/test_level3_spectrographic.py
index 464f52522..8d5aaa8f9 100644
--- a/jwst/associations/tests/test_level3_spectrographic.py
+++ b/jwst/associations/tests/test_level3_spectrographic.py
@@ -18,7 +18,7 @@ class TestLevel3Spec(BasePoolRule):
     pools = [
         PoolParams(
             path=t_path('data/pool_005_spec_niriss.csv'),
-            n_asns=1,
+            n_asns=0,
             n_orphaned=0
         ),
         PoolParams(
@@ -28,7 +28,7 @@ class TestLevel3Spec(BasePoolRule):
         ),
         PoolParams(
             path=t_path('data/pool_007_spec_miri.csv'),
-            n_asns=2,
+            n_asns=1,
             n_orphaned=0
         ),
         PoolParams(