Repository: spacetelescope/drizzlepac

Issue:
Update alignment parameters for pipeline and SVM processing
A fix for the alignment logic is requested for the next drizzlepac build.   These changes are important to avoid skewing the images by allowing for complex fit geometry when there only a few catalog matches.

At the moment we have the following parameters for all instruments/detectors. The code will attempt “rshift”, “rscale”, and “general” fit geometry with the number of required NMATCHES for pipeline products (mosaic_rel_geom) and SVM products (mosaic_fitgeom).   
```
"mosaic_relgeom_list": {"rshift":20, "rscale": 10, "general": 9}
"mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
```

This is not the desired logic which should apply a more complex fit geometry only with a large number of matches.  WFC3 and ACS have been digging into to accuracy our current distortion solutions and based on those results, we would like to change the parameters to the following:
```
"mosaic_relgeom_list": {"rscale": 12}
"mosaic_fitgeom_list": {"rscale": 12, "general": 25}
```

We find errors in the filter-dependent scale in the geometric distortion reference file, so we want to eliminate the 'rshift' option and allow the software to fit for scale terms when aligning images.  We increase the number of required matches from 10 to 12 to account for sigma-clipping.  To fit for skew errors in the distortion (e.g. general fit), we find that a large number of sources are required for a good fit.  

As Michele suggests in https://jira.stsci.edu/browse/HLA-1385, we need to discuss as a group and then ask Matt to run a large regression test so that the impact of this update can be assessed in an isolated manner.

Discussion:
@stscijgbot-hstdp: _This issue is tracked on JIRA as [HLA-1405](https://jira.stsci.edu/browse/HLA-1405)._@stscijgbot-hstdp: _Comment by [Steve Goldman](https://jira.stsci.edu/secure/ViewProfile.jspa?name=sgoldman) on [JIRA](https://jira.stsci.edu/browse/HLA-1405?focusedCommentId=941748#comment-941748):_

will be closed with PR [#&#x2060;1932](https://github.com/spacetelescope/drizzlepac/pull/1932) once SVM regression tests are investigated (HLA-1422). @stscijgbot-hstdp: _Comment by [Jennifer Mack](https://jira.stsci.edu/secure/ViewProfile.jspa?name=mack) on [JIRA](https://jira.stsci.edu/browse/HLA-1405?focusedCommentId=956600#comment-956600):_

From 4/18/25 email:
 
The instrument teams have done a deep dive into the accuracy of the ACS and WFC3 distortion solutions over the past year.  Those findings show that we are introducing linear terms (scale, rotation) when we do not have enough matches.  Additionally, for fields with few stars, it results in observations with mismatched WCS solutions in different filters (eg. UV or narrowband.)
 
The updated logic will result in a slightly larger fraction of images which retain the 'a priori' WCS, but that is preferred to avoid introducing rotation or scale errors when N_matches is low. (From my testing it was just 5%, see below).  We can do a statistical comparison of the WCS solutions after reprocessing and potentially tweak the alignment criteria if desired.
 
------ TESTING results -----
 
Summary of test results for 2000 datasets processed in early Feb:
 
--> 10% fewer datasets with eDR3 solutions (200/2000)

        -> 5% switch from eDR3 to GSC242 (100/2000)
        -> 5% revert to 'a priori' solution  (100/2000).   

These updates change the reference catalog for a fraction of images (eDR3 -> GSC242), typically deep field data.  They also increase the FIT_RMS (e.g. positional errors are larger using resolved sources vs stars) but should give better fitting results by using a larger number of matched sources.

Images with 'a posteriori' solutions are now split 50/50 between eDR3 (stellar fields) and GSC242 (extragalactic fields) which is reasonable considering the rough distribution of HST imaging targets.  
 
Example dataset (galaxy cluster) with a better WCS solution, but a larger fit RMS: 
<https://archive.stsci.edu/cgi-bin/mastpreview?mission=hst&dataid=ICZK01020>
Solution   filename                             RMS_RA   RMS_DEC  NMATCHES   WCSNAME
OLD  hst_14148_01_wfc3_ir_total_iczk01      6.0    13.6       4           FIT_SVM_GAIAeDR3
NEW  hst_14148_01_wfc3_ir_total_iczk01     66.3   71.3      65          FIT_SVM_GSC242@stscijgbot-hstdp: _Comment by [Jennifer Mack](https://jira.stsci.edu/secure/ViewProfile.jspa?name=mack) on [JIRA](https://jira.stsci.edu/browse/HLA-1405?focusedCommentId=956602#comment-956602):_

Adding plots from [Steve Goldman](https://jira.stsci.edu/secure/ViewProfile.jspa?name=sgoldman) 

Solution patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index fd0e8b66..c8a1dfb7 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -24,6 +24,8 @@ number of the code change for that issue.  These PRs can be viewed at:
 
 - Added new header keywords and match requirements for relative fitting. [#1860]
 
+- Updated alignment parameters. [#1932]
+
 - Implemented fixes to address uneven detection thresholds in the HAP catalogs
   due to bugs in the function, make_wht_masks, which intends to create weight
   masks covering the full drizzled output footprint. A somewhat related bug in
diff --git a/drizzlepac/pars/hap_pars/svm_parameters/acs/hrc/acs_hrc_alignment_all.json b/drizzlepac/pars/hap_pars/svm_parameters/acs/hrc/acs_hrc_alignment_all.json
index d4a3ac23..6ff9f327 100644
--- a/drizzlepac/pars/hap_pars/svm_parameters/acs/hrc/acs_hrc_alignment_all.json
+++ b/drizzlepac/pars/hap_pars/svm_parameters/acs/hrc/acs_hrc_alignment_all.json
@@ -16,8 +16,8 @@
       "MAX_FIT_LIMIT": 150,
       "mosaic_catalog_list": ["GAIAeDR3", "GSC242", "2MASS"],
       "mosaic_fit_list": ["match_relative_fit", "match_2dhist_fit", "match_default_fit"],
-      "mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
-      "mosaic_relgeom_list": {"rshift": 20, "rscale": 10, "general": 9}
+      "mosaic_fitgeom_list": {"rscale": 12, "general": 25},
+      "mosaic_relgeom_list": {"rscale": 12}
     },
   "generate_source_catalogs":
     {
diff --git a/drizzlepac/pars/hap_pars/svm_parameters/acs/sbc/acs_sbc_alignment_all.json b/drizzlepac/pars/hap_pars/svm_parameters/acs/sbc/acs_sbc_alignment_all.json
index 9f000f66..f4d91456 100644
--- a/drizzlepac/pars/hap_pars/svm_parameters/acs/sbc/acs_sbc_alignment_all.json
+++ b/drizzlepac/pars/hap_pars/svm_parameters/acs/sbc/acs_sbc_alignment_all.json
@@ -16,8 +16,8 @@
       "MAX_FIT_LIMIT": 150,
       "mosaic_catalog_list": ["GAIAeDR3", "GSC242", "2MASS"],
       "mosaic_fit_list": ["match_relative_fit", "match_2dhist_fit", "match_default_fit"],
-      "mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
-      "mosaic_relgeom_list": {"rshift": 20, "rscale": 10, "general": 9}
+      "mosaic_fitgeom_list": {"rscale": 12, "general": 25},
+      "mosaic_relgeom_list": {"rscale": 12}
     },
   "generate_source_catalogs":
     {
diff --git a/drizzlepac/pars/hap_pars/svm_parameters/acs/wfc/acs_wfc_alignment_all.json b/drizzlepac/pars/hap_pars/svm_parameters/acs/wfc/acs_wfc_alignment_all.json
index ff3ce38c..93ed346c 100644
--- a/drizzlepac/pars/hap_pars/svm_parameters/acs/wfc/acs_wfc_alignment_all.json
+++ b/drizzlepac/pars/hap_pars/svm_parameters/acs/wfc/acs_wfc_alignment_all.json
@@ -16,8 +16,8 @@
         "MAX_FIT_LIMIT": 150,
         "mosaic_catalog_list": ["GAIAeDR3", "GSC242", "2MASS"],
         "mosaic_fit_list": ["match_relative_fit", "match_2dhist_fit", "match_default_fit"],
-        "mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
-        "mosaic_relgeom_list": {"rshift":20, "rscale": 10, "general": 9}
+        "mosaic_fitgeom_list": {"rscale": 12, "general": 25},
+        "mosaic_relgeom_list": {"rscale": 12}
       },
     "generate_source_catalogs":
       {
diff --git a/drizzlepac/pars/hap_pars/svm_parameters/wfc3/ir/wfc3_ir_alignment_all.json b/drizzlepac/pars/hap_pars/svm_parameters/wfc3/ir/wfc3_ir_alignment_all.json
index f5f95708..c421751c 100644
--- a/drizzlepac/pars/hap_pars/svm_parameters/wfc3/ir/wfc3_ir_alignment_all.json
+++ b/drizzlepac/pars/hap_pars/svm_parameters/wfc3/ir/wfc3_ir_alignment_all.json
@@ -16,8 +16,8 @@
         "MAX_FIT_LIMIT": 150,
         "mosaic_catalog_list": ["GAIAeDR3", "GSC242", "2MASS"],
         "mosaic_fit_list": ["match_relative_fit", "match_2dhist_fit", "match_default_fit"],
-        "mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
-        "mosaic_relgeom_list": {"rshift":20, "rscale": 10, "general": 9}
+        "mosaic_fitgeom_list": {"rscale": 12, "general": 25},
+        "mosaic_relgeom_list": {"rscale": 12}
       },
     "generate_source_catalogs":
       {
diff --git a/drizzlepac/pars/hap_pars/svm_parameters/wfc3/uvis/wfc3_uvis_alignment_all.json b/drizzlepac/pars/hap_pars/svm_parameters/wfc3/uvis/wfc3_uvis_alignment_all.json
index 4db56868..a6c8baca 100644
--- a/drizzlepac/pars/hap_pars/svm_parameters/wfc3/uvis/wfc3_uvis_alignment_all.json
+++ b/drizzlepac/pars/hap_pars/svm_parameters/wfc3/uvis/wfc3_uvis_alignment_all.json
@@ -16,8 +16,8 @@
         "MAX_FIT_LIMIT": 150,
         "mosaic_catalog_list": ["GAIAeDR3", "GSC242", "2MASS"],
         "mosaic_fit_list": ["match_relative_fit", "match_2dhist_fit", "match_default_fit"],
-        "mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
-        "mosaic_relgeom_list": {"rshift":20, "rscale": 10, "general": 9}
+        "mosaic_fitgeom_list": {"rscale": 12, "general": 25},
+        "mosaic_relgeom_list": {"rscale": 12}
       },
     "generate_source_catalogs":
       {
diff --git a/drizzlepac/pars/hap_pars/svm_parameters/wfpc2/pc/wfpc2_pc_alignment_all.json b/drizzlepac/pars/hap_pars/svm_parameters/wfpc2/pc/wfpc2_pc_alignment_all.json
index 5a435416..78bdea87 100644
--- a/drizzlepac/pars/hap_pars/svm_parameters/wfpc2/pc/wfpc2_pc_alignment_all.json
+++ b/drizzlepac/pars/hap_pars/svm_parameters/wfpc2/pc/wfpc2_pc_alignment_all.json
@@ -16,8 +16,8 @@
         "MAX_FIT_LIMIT": 150,
         "mosaic_catalog_list": ["GAIAeDR3", "GSC242", "2MASS"],
         "mosaic_fit_list": ["match_relative_fit", "match_2dhist_fit", "match_default_fit"],
-        "mosaic_fitgeom_list": {"rshift": 4, "rscale": 10, "general": 6},
-        "mosaic_relgeom_list": {"rshift":20, "rscale": 10, "general": 9}
+        "mosaic_fitgeom_list": {"rscale": 12, "general": 25},
+        "mosaic_relgeom_list": {"rscale": 12}
       },
     "generate_source_catalogs":
       {


Test patch:
diff --git a/tests/hap/test_svm_ibyt50.py b/tests/hap/test_svm_ibyt50.py
index ffccac8e..0e3c3fe2 100644
--- a/tests/hap/test_svm_ibyt50.py
+++ b/tests/hap/test_svm_ibyt50.py
@@ -29,8 +29,8 @@ from pathlib import Path
 """
 
 POLLER_FILE = "wfc3_byt_50_input.out"
-WCS_UVIS_SUB_NAME = "FIT_SVM_GAIA"
-WCS_IR_SUB_NAME = "FIT_SVM_GAIA"
+WCS_UVIS_SUB_NAME = "FIT_SVM_GSC242"
+WCS_IR_SUB_NAME = "FIT_SVM_GSC242"
 expected_total_point_sources = {
 "hst_13023_50_wfc3_ir_total_ibyt50_point-cat.ecsv": 118,
 "hst_13023_50_wfc3_uvis_total_ibyt50_point-cat.ecsv": 100}