Repository: JeffersonLab/hps-java
Pull request #1088: Update to fix Over and Understimation of Errors
Author: login: rodwyer100
Created: 2025-03-12T16:44:27Z | Merged: 2025-03-28T20:06:47Z

This update is directed at getting amplitude errors and chi square values aligned with their proper values as determined by the chi square probability. The chi square probability, when chi is estimated properly, should be a flat distribution by the inverse CDF transform theorem. Under the assumption we have an unknown multiplicative error in our strip error estimation (that is layer dependent), we can plot the chi square probability with a different multiplicative value of error. We can then make a best fit measurement to determine what this factor is and correct it (below is a drawing of what that scan looks like):
![image](https://github.com/user-attachments/assets/40706c28-bf70-42b0-ad09-1a6608a0a0be)
When it was fit, this factor was determined to also be run dependent:
![image](https://github.com/user-attachments/assets/4833dd6e-a090-430b-9ab0-0c2200f35a4f)
So we did a rather long run dependent calibration, and this is the result. If you are interesting in seeing the run dependent histograms of before and after for the chi square fits, they can be found here:https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/master.html.
You can see that the runs do seem to flatten the chi out. Under the assumption that the error is multiplicatively scaled, this same factor is applied to the amplitudeErrors. This should, in all likely hood, affect almost nothing. I have been working with the reconstruction for a couple of years, and have never seen these used for anything but diagnostics. Still, there are a few tests we may want to employ. The first is that this change seems to motivate in the foremost layers a chi sqr cut at .01 to remove a spike in bad fits. These are not consitent with underestimating errors now; these are genuinely poor fits and removing them should clean things. That needs to be done and a simple evaluation of track number and moller peaks should be checked. Others (particularly those who can do the Moller stuff) may need to help with this. The other test that should be done is to plot the actual strip error (which is another number which should be related to these values but may not be if the way the database errors are logged were wrong) and diagnose the issue that caused this. This shouldn't prevent the proposed changes from being pushed, but its the next plot I will append to this request.


Diff:
diff --git a/tracking/src/main/java/org/hps/recon/tracking/DumbShaperFit.java b/tracking/src/main/java/org/hps/recon/tracking/DumbShaperFit.java
index 4e71d9cd9..c529e281b 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/DumbShaperFit.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/DumbShaperFit.java
@@ -12,6 +12,7 @@ public class DumbShaperFit implements ShaperFitAlgorithm {
 
     private boolean debug = false;
     private String fitTimeMinimizer = "Simplex";
+    private int runNum = 1000000;
 
     public DumbShaperFit() {
     }
@@ -20,6 +21,10 @@ public void setDebug(boolean debug) {
         this.debug = debug;
     }
 
+    @Override
+    public void setRunNum(int RunNum){
+        this.runNum=RunNum;
+    }
     @Override
     public void setFitTimeMinimizer(String fitTimeMinimizer) {
         this.fitTimeMinimizer = fitTimeMinimizer;
diff --git a/tracking/src/main/java/org/hps/recon/tracking/NearestNeighborRMSClusterer.java b/tracking/src/main/java/org/hps/recon/tracking/NearestNeighborRMSClusterer.java
index cfee9dd5c..0978334dd 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/NearestNeighborRMSClusterer.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/NearestNeighborRMSClusterer.java
@@ -30,7 +30,7 @@ public class NearestNeighborRMSClusterer implements ClusteringAlgorithm {
     private double _timeWindow = 48; 
     private double _neighborDeltaT = Double.POSITIVE_INFINITY;
     private double _neighborDeltaTSigma = Double.POSITIVE_INFINITY;
-    private final double _minChiProb = Gamma.regularizedGammaQ(4, 20);
+    private final double _minChiProb = .01;//Gamma.regularizedGammaQ(4, 20);
     private double _doTimeError = 0.0;
     private boolean _doDeadFix = false;
     private boolean _doVSplit = false;
diff --git a/tracking/src/main/java/org/hps/recon/tracking/RawTrackerHitFitterDriver.java b/tracking/src/main/java/org/hps/recon/tracking/RawTrackerHitFitterDriver.java
index 2d03af98f..ee80bfb87 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/RawTrackerHitFitterDriver.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/RawTrackerHitFitterDriver.java
@@ -239,6 +239,7 @@ public void process(EventHeader event) {
             int strip = hit.getIdentifierFieldValue("strip");
             HpsSiSensor sensor = (HpsSiSensor) hit.getDetectorElement();
             //===> ChannelConstants constants = HPSSVTCalibrationConstants.getChannelConstants((SiSensor) hit.getDetectorElement(), strip);
+            fitter.setRunNum(event.getRunNumber());
             for (ShapeFitParameters fit : fitter.fitShape(hit, shape)) {
 
                 if (correctTimeOffset) {
diff --git a/tracking/src/main/java/org/hps/recon/tracking/ShaperAnalyticFitAlgorithm.java b/tracking/src/main/java/org/hps/recon/tracking/ShaperAnalyticFitAlgorithm.java
index 234592a49..b31613a52 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/ShaperAnalyticFitAlgorithm.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/ShaperAnalyticFitAlgorithm.java
@@ -15,12 +15,18 @@
 public class ShaperAnalyticFitAlgorithm implements ShaperFitAlgorithm {
 
     private boolean debug = false;
+    private int runNum = 1000000;
     public String fitTimeMinimizer;
 
     public void setDebug(boolean debug) {
         this.debug = debug;
     }
 
+    @Override
+    public void setRunNum(int RunNum){
+        this.runNum=RunNum;
+    }
+    
     @Override
     public void setFitTimeMinimizer(String fitTimeMinimizer) {
         this.fitTimeMinimizer = fitTimeMinimizer;
diff --git a/tracking/src/main/java/org/hps/recon/tracking/ShaperFitAlgorithm.java b/tracking/src/main/java/org/hps/recon/tracking/ShaperFitAlgorithm.java
index acbc06f65..279be0f24 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/ShaperFitAlgorithm.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/ShaperFitAlgorithm.java
@@ -11,5 +11,7 @@ public interface ShaperFitAlgorithm {
 
     public void setDebug(boolean debug);
 
+    public void setRunNum(int runNum);
+    
     public void setFitTimeMinimizer(String fitTimeMinimizer);
 }
diff --git a/tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java b/tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java
index ca62366fd..7ed400cd4 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java
@@ -55,6 +55,7 @@ public class ShaperLinearFitAlgorithm implements ShaperFitAlgorithm, FCNBase {
     private boolean fitPedestal = false;
     private boolean debug = false;
     private static final Logger minuitLoggger = Logger.getLogger("org.freehep.math.minuit");
+    private int runNum = 1000000;
 
     public ShaperLinearFitAlgorithm(int nPulses) {
         this.nPulses = nPulses;
@@ -89,6 +90,11 @@ public void setFitTimeMinimizer(String fitTimeMinimizer) {
         this.fitTimeMinimizer = fitTimeMinimizer;
     }
 
+    @Override
+    public void setRunNum(int RunNum){
+        this.runNum=RunNum;
+    }
+    
     @Override
     //===> public Collection<ShapeFitParameters> fitShape(RawTrackerHit rth, ChannelConstants constants) {
     public Collection<ShapeFitParameters> fitShape(RawTrackerHit rth, PulseShape shape) {
@@ -142,14 +148,75 @@ public Collection<ShapeFitParameters> fitShape(short[] samples) {
 
         ArrayList<ShapeFitParameters> fits = new ArrayList<ShapeFitParameters>();
 
+        double[][] mults={{0.859375, 0.78125, 0.75, 0.796875, 0.875, 0.921875, 0.90625, 0.90625, 0.859375, 0.953125, 0.984375, 0.953125, 0.984375,1.0},
+        {1.0, 0.859375, 0.765625, 0.890625, 1.078125, 1.125, 0.96875, 0.96875, 0.890625, 0.984375, 1.015625, 0.984375, 0.984375,1.0},
+        {1.046875, 0.90625, 0.828125, 0.890625, 1.0, 1.109375, 1.0, 1.0625, 0.90625, 1.015625, 1.03125, 1.09375, 0.953125,1.0},
+        {0.859375, 0.828125, 0.71875, 0.8125, 1.03125, 1.078125, 1.015625, 1.046875, 0.875, 1.03125, 1.0625, 1.09375, 0.953125,1.0},
+        {0.84375, 0.796875, 0.71875, 0.75, 1.03125, 1.109375, 0.96875, 1.015625, 0.90625, 1.046875, 1.0625, 1.09375, 0.984375,1.0},
+        {0.859375, 0.8125, 0.703125, 0.71875, 1.078125, 1.078125, 1.03125, 1.078125, 0.90625, 1.0625, 1.078125, 1.15625, 1.0,1.0},
+        {0.828125, 0.765625, 0.703125, 0.765625, 1.046875, 1.09375, 0.984375, 1.015625, 0.890625, 1.015625, 1.03125, 1.03125, 1.0,1.0},
+        {0.828125, 0.71875, 0.6875, 0.734375, 1.078125, 1.125, 1.015625, 1.078125, 0.90625, 1.078125, 1.125, 1.140625, 0.984375,1.0},
+        {0.828125, 0.75, 0.71875, 0.765625, 1.0625, 1.125, 1.0, 1.03125, 0.921875, 1.046875, 1.09375, 1.09375, 1.0625,1.0},
+        {0.8125, 0.71875, 0.65625, 0.671875, 1.03125, 1.0625, 0.984375, 1.015625, 0.875, 1.015625, 1.046875, 1.03125, 0.96875,1.0},
+        {0.84375, 0.703125, 0.6875, 0.703125, 1.0, 1.0625, 0.984375, 1.046875, 0.875, 1.03125, 1.046875, 1.078125, 1.0,1.0},
+        {0.765625, 0.703125, 0.625, 0.609375, 1.0, 1.015625, 0.96875, 0.96875, 0.828125, 0.96875, 0.96875, 0.96875, 0.921875,1.0},
+        {0.8125, 0.734375, 0.703125, 0.8125, 1.09375, 1.09375, 1.125, 1.078125, 0.84375, 1.0, 1.0, 1.046875, 0.953125,1.0},
+        {0.8125, 0.75, 0.671875, 0.71875, 1.03125, 1.03125, 0.96875, 1.015625, 0.875, 1.0, 1.015625, 1.078125, 0.953125,1.0},
+        {0.8125, 0.734375, 0.671875, 0.734375, 1.046875, 1.078125, 1.046875, 1.0, 0.875, 1.0, 1.015625, 1.03125, 0.953125,1.0},
+        {0.796875, 0.78125, 0.703125, 0.796875, 1.09375, 1.09375, 1.078125, 1.09375, 0.9375, 1.125, 1.09375, 1.171875, 1.125,1.0},
+        {0.78125, 0.734375, 0.65625, 0.734375, 1.0, 1.078125, 1.015625, 1.015625, 0.859375, 1.0, 1.03125, 1.046875, 0.953125,1.0},
+        {0.71875, 0.65625, 0.640625, 0.640625, 0.90625, 0.9375, 0.96875, 1.0, 0.84375, 1.0, 1.046875, 1.046875, 0.96875,1.0},
+        {0.75, 0.703125, 0.640625, 0.75, 1.09375, 1.140625, 1.015625, 0.984375, 0.828125, 0.96875, 0.921875, 1.03125, 0.921875,1.0},
+        {0.765625, 0.71875, 0.671875, 0.71875, 1.03125, 1.078125, 1.046875, 1.046875, 0.875, 0.984375, 1.03125, 1.046875, 0.984375,1.0},
+        {0.765625, 0.703125, 0.640625, 0.671875, 0.96875, 1.03125, 0.96875, 0.953125, 0.84375, 0.984375, 0.984375, 1.03125, 0.90625,1.0},
+        {0.75, 0.734375, 0.671875, 0.75, 1.03125, 1.046875, 0.984375, 1.046875, 0.84375, 1.03125, 1.03125, 1.09375, 0.984375,1.0},
+        {0.734375, 0.703125, 0.65625, 0.75, 1.046875, 1.046875, 0.984375, 1.03125, 0.875, 1.03125, 1.015625, 1.046875, 0.984375,1.0},
+        {0.734375, 0.71875, 0.703125, 0.765625, 1.125, 1.125, 1.09375, 1.09375, 0.9375, 1.125, 1.109375, 1.1875, 1.109375,1.0},
+
+        {0.71875, 0.71875, 0.6875, 0.71875, 1.0625, 1.03125, 1.0, 1.03125, 0.859375, 1.046875, 1.0, 1.046875, 0.96875,1.0},
+        {0.734375, 0.671875, 0.609375, 0.640625, 1.015625, 1.015625, 1.015625, 1.015625, 0.84375, 1.0, 1.015625, 1.046875, 0.984375,1.0},
+        {0.859375, 0.828125, 0.828125, 0.921875, 1.078125, 1.09375, 1.09375, 1.078125, 0.84375, 1.03125, 1.09375, 1.125, 1.03125,1.0},
+        {0.875, 0.859375, 0.90625, 0.984375, 1.09375, 1.078125, 1.09375, 1.03125, 0.875, 1.046875, 1.0625, 1.140625, 1.0,1.0},
+        {0.890625, 0.859375, 0.828125, 0.859375, 1.0, 1.140625, 1.09375, 1.046875, 0.84375, 1.046875, 1.078125, 1.09375, 0.984375,1.0},
+        {0.78125, 0.890625, 0.953125, 0.953125, 0.984375, 1.03125, 0.984375, 0.953125, 0.859375, 1.03125, 1.046875, 1.03125, 0.96875,1.0},
+        {0.953125, 0.90625, 0.96875, 1.046875, 1.0, 1.078125, 1.09375, 1.078125, 0.859375, 1.0625, 1.046875, 1.09375, 0.96875,1.0},
+        {0.953125, 0.921875, 0.890625, 1.046875, 1.046875, 1.03125, 1.09375, 1.109375, 0.859375, 1.03125, 1.109375, 1.078125, 0.953125,1.0},
+        {0.984375, 0.953125, 1.0, 1.078125, 1.015625, 1.03125, 1.0625, 1.046875, 0.828125, 0.984375, 1.0, 1.078125, 0.984375,1.0},
+        {1.046875, 1.0, 1.078125, 1.21875, 1.078125, 1.078125, 1.09375, 1.109375, 0.875, 1.109375, 1.078125, 1.125, 1.09375,1.0},
+        {1.09375, 1.03125, 1.09375, 1.171875, 1.015625, 1.046875, 1.078125, 1.015625, 0.84375, 1.03125, 1.03125, 1.078125, 0.984375,1.0},
+
+        {1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0}};
+
+        int[] runs = {14170,14185,14191,14200,14204,14210,14222,14232,14262,14272,
+             14291,14314,14332,14340,14381,14391,14408,14421,14443,14460,
+             14470,14482,14506,14520,14540,14550,14573,14596,14612,14654,
+             14710,14720,14731,14750,14772};
+        boolean helper = true;
+        int holder = mults.length-1;
+        //System.out.println("The run number is ");
+        //System.out.println(this.runNum);
+        for (int i = 0; i<runs.length; i++){
+             if(runs[i]<14170){
+                  holder=0;
+                  helper=false;
+             }
+             if((runs[i]<=this.runNum)&&helper){
+                  helper=false;
+                  holder=i;
+             }
+        }
+        if(holder>=mults.length-1){
+                holder=mults.length-1;
+        }
         for (int i = 0; i < nPulses; i++) {
             ShapeFitParameters fit = new ShapeFitParameters();
             fit.setAmp(amplitudes[i]);
-            fit.setAmpErr(amplitudeErrors[i]);
+            fit.setAmpErr(amplitudeErrors[i]/mults[holder][sensor.getLayerNumber()-1]);
+            //Note IF WE CHOOSE TO DO FIT PEDESTAL, THESE WILL NEED TO BE FIXED!!!
             if (fitPedestal) {
-                fit.setChiProb(Gamma.regularizedGammaQ(samples.length - 2 * nPulses - 1, chisq));
+                fit.setChiProb(Gamma.regularizedGammaQ(samples.length - 2 * nPulses - 1, chisq/mults[holder][sensor.getLayerNumber()-1]));
             } else {
-                fit.setChiProb(Gamma.regularizedGammaQ(samples.length - 2 * nPulses, chisq));
+                fit.setChiProb(Gamma.regularizedGammaQ(samples.length - 2 * nPulses, chisq/mults[holder][sensor.getLayerNumber()-1]));
             }
 
             fit.setT0(min.userState().value(i));
diff --git a/tracking/src/main/java/org/hps/recon/tracking/ShaperPileupFitAlgorithm.java b/tracking/src/main/java/org/hps/recon/tracking/ShaperPileupFitAlgorithm.java
index ce8a05eca..da06f8f52 100644
--- a/tracking/src/main/java/org/hps/recon/tracking/ShaperPileupFitAlgorithm.java
+++ b/tracking/src/main/java/org/hps/recon/tracking/ShaperPileupFitAlgorithm.java
@@ -14,6 +14,7 @@ public class ShaperPileupFitAlgorithm implements ShaperFitAlgorithm {
     private int refitAttempts = 0;
     private int refitsAccepted = 0;
     private int doOldDT = 1;
+    private int runNum = 1000000;
     
     public ShaperPileupFitAlgorithm() {
     }
@@ -28,7 +29,10 @@ public void setFitTimeMinimizer(String fitTimeMinimizer) {
         this.onePulseFitter.setFitTimeMinimizer(fitTimeMinimizer);
         this.twoPulseFitter.setFitTimeMinimizer(fitTimeMinimizer);
     }
-
+    @Override
+    public void setRunNum(int RunNum){
+        this.runNum=RunNum;
+    }
     //===> public Collection<ShapeFitParameters> fitShape(RawTrackerHit rth, HPSSVTCalibrationConstants.ChannelConstants constants) {
     public Collection<ShapeFitParameters> fitShape(RawTrackerHit rth, PulseShape shape) {
         Collection<ShapeFitParameters> fittedPulses = onePulseFitter.fitShape(rth, shape);


Reviews:
user: login: mgignac
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-28T19:54:32Z
user: login: rodwyer100
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T19:57:30Z
user: login: mgignac
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-28T19:59:52Z
user: login: mgignac
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-28T20:00:06Z
user: login: bloodyyugo
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-28T20:06:38Z

Review comments:
diff_hunk: @@ -142,14 +148,75 @@ public Collection<ShapeFitParameters> fitShape(short[] samples) {
 
         ArrayList<ShapeFitParameters> fits = new ArrayList<ShapeFitParameters>();
 
+        double[][] mults={{0.859375, 0.78125, 0.75, 0.796875, 0.875, 0.921875, 0.90625, 0.90625, 0.859375, 0.953125, 0.984375, 0.953125, 0.984375,1.0},
+        {1.0, 0.859375, 0.765625, 0.890625, 1.078125, 1.125, 0.96875, 0.96875, 0.890625, 0.984375, 1.015625, 0.984375, 0.984375,1.0},
+        {1.046875, 0.90625, 0.828125, 0.890625, 1.0, 1.109375, 1.0, 1.0625, 0.90625, 1.015625, 1.03125, 1.09375, 0.953125,1.0},
+        {0.859375, 0.828125, 0.71875, 0.8125, 1.03125, 1.078125, 1.015625, 1.046875, 0.875, 1.03125, 1.0625, 1.09375, 0.953125,1.0},
+        {0.84375, 0.796875, 0.71875, 0.75, 1.03125, 1.109375, 0.96875, 1.015625, 0.90625, 1.046875, 1.0625, 1.09375, 0.984375,1.0},
+        {0.859375, 0.8125, 0.703125, 0.71875, 1.078125, 1.078125, 1.03125, 1.078125, 0.90625, 1.0625, 1.078125, 1.15625, 1.0,1.0},
+        {0.828125, 0.765625, 0.703125, 0.765625, 1.046875, 1.09375, 0.984375, 1.015625, 0.890625, 1.015625, 1.03125, 1.03125, 1.0,1.0},
+        {0.828125, 0.71875, 0.6875, 0.734375, 1.078125, 1.125, 1.015625, 1.078125, 0.90625, 1.078125, 1.125, 1.140625, 0.984375,1.0},
+        {0.828125, 0.75, 0.71875, 0.765625, 1.0625, 1.125, 1.0, 1.03125, 0.921875, 1.046875, 1.09375, 1.09375, 1.0625,1.0},
+        {0.8125, 0.71875, 0.65625, 0.671875, 1.03125, 1.0625, 0.984375, 1.015625, 0.875, 1.015625, 1.046875, 1.03125, 0.96875,1.0},
+        {0.84375, 0.703125, 0.6875, 0.703125, 1.0, 1.0625, 0.984375, 1.046875, 0.875, 1.03125, 1.046875, 1.078125, 1.0,1.0},
+        {0.765625, 0.703125, 0.625, 0.609375, 1.0, 1.015625, 0.96875, 0.96875, 0.828125, 0.96875, 0.96875, 0.96875, 0.921875,1.0},
+        {0.8125, 0.734375, 0.703125, 0.8125, 1.09375, 1.09375, 1.125, 1.078125, 0.84375, 1.0, 1.0, 1.046875, 0.953125,1.0},
+        {0.8125, 0.75, 0.671875, 0.71875, 1.03125, 1.03125, 0.96875, 1.015625, 0.875, 1.0, 1.015625, 1.078125, 0.953125,1.0},
+        {0.8125, 0.734375, 0.671875, 0.734375, 1.046875, 1.078125, 1.046875, 1.0, 0.875, 1.0, 1.015625, 1.03125, 0.953125,1.0},
+        {0.796875, 0.78125, 0.703125, 0.796875, 1.09375, 1.09375, 1.078125, 1.09375, 0.9375, 1.125, 1.09375, 1.171875, 1.125,1.0},
+        {0.78125, 0.734375, 0.65625, 0.734375, 1.0, 1.078125, 1.015625, 1.015625, 0.859375, 1.0, 1.03125, 1.046875, 0.953125,1.0},
+        {0.71875, 0.65625, 0.640625, 0.640625, 0.90625, 0.9375, 0.96875, 1.0, 0.84375, 1.0, 1.046875, 1.046875, 0.96875,1.0},
+        {0.75, 0.703125, 0.640625, 0.75, 1.09375, 1.140625, 1.015625, 0.984375, 0.828125, 0.96875, 0.921875, 1.03125, 0.921875,1.0},
+        {0.765625, 0.71875, 0.671875, 0.71875, 1.03125, 1.078125, 1.046875, 1.046875, 0.875, 0.984375, 1.03125, 1.046875, 0.984375,1.0},
+        {0.765625, 0.703125, 0.640625, 0.671875, 0.96875, 1.03125, 0.96875, 0.953125, 0.84375, 0.984375, 0.984375, 1.03125, 0.90625,1.0},
+        {0.75, 0.734375, 0.671875, 0.75, 1.03125, 1.046875, 0.984375, 1.046875, 0.84375, 1.03125, 1.03125, 1.09375, 0.984375,1.0},
+        {0.734375, 0.703125, 0.65625, 0.75, 1.046875, 1.046875, 0.984375, 1.03125, 0.875, 1.03125, 1.015625, 1.046875, 0.984375,1.0},
+        {0.734375, 0.71875, 0.703125, 0.765625, 1.125, 1.125, 1.09375, 1.09375, 0.9375, 1.125, 1.109375, 1.1875, 1.109375,1.0},
+
+        {0.71875, 0.71875, 0.6875, 0.71875, 1.0625, 1.03125, 1.0, 1.03125, 0.859375, 1.046875, 1.0, 1.046875, 0.96875,1.0},
+        {0.734375, 0.671875, 0.609375, 0.640625, 1.015625, 1.015625, 1.015625, 1.015625, 0.84375, 1.0, 1.015625, 1.046875, 0.984375,1.0},
+        {0.859375, 0.828125, 0.828125, 0.921875, 1.078125, 1.09375, 1.09375, 1.078125, 0.84375, 1.03125, 1.09375, 1.125, 1.03125,1.0},
+        {0.875, 0.859375, 0.90625, 0.984375, 1.09375, 1.078125, 1.09375, 1.03125, 0.875, 1.046875, 1.0625, 1.140625, 1.0,1.0},
+        {0.890625, 0.859375, 0.828125, 0.859375, 1.0, 1.140625, 1.09375, 1.046875, 0.84375, 1.046875, 1.078125, 1.09375, 0.984375,1.0},
+        {0.78125, 0.890625, 0.953125, 0.953125, 0.984375, 1.03125, 0.984375, 0.953125, 0.859375, 1.03125, 1.046875, 1.03125, 0.96875,1.0},
+        {0.953125, 0.90625, 0.96875, 1.046875, 1.0, 1.078125, 1.09375, 1.078125, 0.859375, 1.0625, 1.046875, 1.09375, 0.96875,1.0},
+        {0.953125, 0.921875, 0.890625, 1.046875, 1.046875, 1.03125, 1.09375, 1.109375, 0.859375, 1.03125, 1.109375, 1.078125, 0.953125,1.0},
+        {0.984375, 0.953125, 1.0, 1.078125, 1.015625, 1.03125, 1.0625, 1.046875, 0.828125, 0.984375, 1.0, 1.078125, 0.984375,1.0},
+        {1.046875, 1.0, 1.078125, 1.21875, 1.078125, 1.078125, 1.09375, 1.109375, 0.875, 1.109375, 1.078125, 1.125, 1.09375,1.0},
+        {1.09375, 1.03125, 1.09375, 1.171875, 1.015625, 1.046875, 1.078125, 1.015625, 0.84375, 1.03125, 1.03125, 1.078125, 0.984375,1.0},
+
+        {1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0}};
+
+        int[] runs = {14170,14185,14191,14200,14204,14210,14222,14232,14262,14272,
path: tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java
user: login: mgignac
body: What happens if the run is not in this list? i.e. for when we process the whole dataset. Is this interpolated somehow?
created_at: 2025-03-28T19:54:10Z
author_association: COLLABORATOR
diff_hunk: @@ -142,14 +148,75 @@ public Collection<ShapeFitParameters> fitShape(short[] samples) {
 
         ArrayList<ShapeFitParameters> fits = new ArrayList<ShapeFitParameters>();
 
+        double[][] mults={{0.859375, 0.78125, 0.75, 0.796875, 0.875, 0.921875, 0.90625, 0.90625, 0.859375, 0.953125, 0.984375, 0.953125, 0.984375,1.0},
+        {1.0, 0.859375, 0.765625, 0.890625, 1.078125, 1.125, 0.96875, 0.96875, 0.890625, 0.984375, 1.015625, 0.984375, 0.984375,1.0},
+        {1.046875, 0.90625, 0.828125, 0.890625, 1.0, 1.109375, 1.0, 1.0625, 0.90625, 1.015625, 1.03125, 1.09375, 0.953125,1.0},
+        {0.859375, 0.828125, 0.71875, 0.8125, 1.03125, 1.078125, 1.015625, 1.046875, 0.875, 1.03125, 1.0625, 1.09375, 0.953125,1.0},
+        {0.84375, 0.796875, 0.71875, 0.75, 1.03125, 1.109375, 0.96875, 1.015625, 0.90625, 1.046875, 1.0625, 1.09375, 0.984375,1.0},
+        {0.859375, 0.8125, 0.703125, 0.71875, 1.078125, 1.078125, 1.03125, 1.078125, 0.90625, 1.0625, 1.078125, 1.15625, 1.0,1.0},
+        {0.828125, 0.765625, 0.703125, 0.765625, 1.046875, 1.09375, 0.984375, 1.015625, 0.890625, 1.015625, 1.03125, 1.03125, 1.0,1.0},
+        {0.828125, 0.71875, 0.6875, 0.734375, 1.078125, 1.125, 1.015625, 1.078125, 0.90625, 1.078125, 1.125, 1.140625, 0.984375,1.0},
+        {0.828125, 0.75, 0.71875, 0.765625, 1.0625, 1.125, 1.0, 1.03125, 0.921875, 1.046875, 1.09375, 1.09375, 1.0625,1.0},
+        {0.8125, 0.71875, 0.65625, 0.671875, 1.03125, 1.0625, 0.984375, 1.015625, 0.875, 1.015625, 1.046875, 1.03125, 0.96875,1.0},
+        {0.84375, 0.703125, 0.6875, 0.703125, 1.0, 1.0625, 0.984375, 1.046875, 0.875, 1.03125, 1.046875, 1.078125, 1.0,1.0},
+        {0.765625, 0.703125, 0.625, 0.609375, 1.0, 1.015625, 0.96875, 0.96875, 0.828125, 0.96875, 0.96875, 0.96875, 0.921875,1.0},
+        {0.8125, 0.734375, 0.703125, 0.8125, 1.09375, 1.09375, 1.125, 1.078125, 0.84375, 1.0, 1.0, 1.046875, 0.953125,1.0},
+        {0.8125, 0.75, 0.671875, 0.71875, 1.03125, 1.03125, 0.96875, 1.015625, 0.875, 1.0, 1.015625, 1.078125, 0.953125,1.0},
+        {0.8125, 0.734375, 0.671875, 0.734375, 1.046875, 1.078125, 1.046875, 1.0, 0.875, 1.0, 1.015625, 1.03125, 0.953125,1.0},
+        {0.796875, 0.78125, 0.703125, 0.796875, 1.09375, 1.09375, 1.078125, 1.09375, 0.9375, 1.125, 1.09375, 1.171875, 1.125,1.0},
+        {0.78125, 0.734375, 0.65625, 0.734375, 1.0, 1.078125, 1.015625, 1.015625, 0.859375, 1.0, 1.03125, 1.046875, 0.953125,1.0},
+        {0.71875, 0.65625, 0.640625, 0.640625, 0.90625, 0.9375, 0.96875, 1.0, 0.84375, 1.0, 1.046875, 1.046875, 0.96875,1.0},
+        {0.75, 0.703125, 0.640625, 0.75, 1.09375, 1.140625, 1.015625, 0.984375, 0.828125, 0.96875, 0.921875, 1.03125, 0.921875,1.0},
+        {0.765625, 0.71875, 0.671875, 0.71875, 1.03125, 1.078125, 1.046875, 1.046875, 0.875, 0.984375, 1.03125, 1.046875, 0.984375,1.0},
+        {0.765625, 0.703125, 0.640625, 0.671875, 0.96875, 1.03125, 0.96875, 0.953125, 0.84375, 0.984375, 0.984375, 1.03125, 0.90625,1.0},
+        {0.75, 0.734375, 0.671875, 0.75, 1.03125, 1.046875, 0.984375, 1.046875, 0.84375, 1.03125, 1.03125, 1.09375, 0.984375,1.0},
+        {0.734375, 0.703125, 0.65625, 0.75, 1.046875, 1.046875, 0.984375, 1.03125, 0.875, 1.03125, 1.015625, 1.046875, 0.984375,1.0},
+        {0.734375, 0.71875, 0.703125, 0.765625, 1.125, 1.125, 1.09375, 1.09375, 0.9375, 1.125, 1.109375, 1.1875, 1.109375,1.0},
+
+        {0.71875, 0.71875, 0.6875, 0.71875, 1.0625, 1.03125, 1.0, 1.03125, 0.859375, 1.046875, 1.0, 1.046875, 0.96875,1.0},
+        {0.734375, 0.671875, 0.609375, 0.640625, 1.015625, 1.015625, 1.015625, 1.015625, 0.84375, 1.0, 1.015625, 1.046875, 0.984375,1.0},
+        {0.859375, 0.828125, 0.828125, 0.921875, 1.078125, 1.09375, 1.09375, 1.078125, 0.84375, 1.03125, 1.09375, 1.125, 1.03125,1.0},
+        {0.875, 0.859375, 0.90625, 0.984375, 1.09375, 1.078125, 1.09375, 1.03125, 0.875, 1.046875, 1.0625, 1.140625, 1.0,1.0},
+        {0.890625, 0.859375, 0.828125, 0.859375, 1.0, 1.140625, 1.09375, 1.046875, 0.84375, 1.046875, 1.078125, 1.09375, 0.984375,1.0},
+        {0.78125, 0.890625, 0.953125, 0.953125, 0.984375, 1.03125, 0.984375, 0.953125, 0.859375, 1.03125, 1.046875, 1.03125, 0.96875,1.0},
+        {0.953125, 0.90625, 0.96875, 1.046875, 1.0, 1.078125, 1.09375, 1.078125, 0.859375, 1.0625, 1.046875, 1.09375, 0.96875,1.0},
+        {0.953125, 0.921875, 0.890625, 1.046875, 1.046875, 1.03125, 1.09375, 1.109375, 0.859375, 1.03125, 1.109375, 1.078125, 0.953125,1.0},
+        {0.984375, 0.953125, 1.0, 1.078125, 1.015625, 1.03125, 1.0625, 1.046875, 0.828125, 0.984375, 1.0, 1.078125, 0.984375,1.0},
+        {1.046875, 1.0, 1.078125, 1.21875, 1.078125, 1.078125, 1.09375, 1.109375, 0.875, 1.109375, 1.078125, 1.125, 1.09375,1.0},
+        {1.09375, 1.03125, 1.09375, 1.171875, 1.015625, 1.046875, 1.078125, 1.015625, 0.84375, 1.03125, 1.03125, 1.078125, 0.984375,1.0},
+
+        {1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0}};
+
+        int[] runs = {14170,14185,14191,14200,14204,14210,14222,14232,14262,14272,
path: tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java
user: login: rodwyer100
body: It runs the same rules as the database readout scheme. Namely, it goes for the closest previous run's settings and uses that. If something dynamical happens at a finer resolution than these runs provides, it will be a problem; shouldn't be too bad otherwise
created_at: 2025-03-28T19:57:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -142,14 +148,75 @@ public Collection<ShapeFitParameters> fitShape(short[] samples) {
 
         ArrayList<ShapeFitParameters> fits = new ArrayList<ShapeFitParameters>();
 
+        double[][] mults={{0.859375, 0.78125, 0.75, 0.796875, 0.875, 0.921875, 0.90625, 0.90625, 0.859375, 0.953125, 0.984375, 0.953125, 0.984375,1.0},
+        {1.0, 0.859375, 0.765625, 0.890625, 1.078125, 1.125, 0.96875, 0.96875, 0.890625, 0.984375, 1.015625, 0.984375, 0.984375,1.0},
+        {1.046875, 0.90625, 0.828125, 0.890625, 1.0, 1.109375, 1.0, 1.0625, 0.90625, 1.015625, 1.03125, 1.09375, 0.953125,1.0},
+        {0.859375, 0.828125, 0.71875, 0.8125, 1.03125, 1.078125, 1.015625, 1.046875, 0.875, 1.03125, 1.0625, 1.09375, 0.953125,1.0},
+        {0.84375, 0.796875, 0.71875, 0.75, 1.03125, 1.109375, 0.96875, 1.015625, 0.90625, 1.046875, 1.0625, 1.09375, 0.984375,1.0},
+        {0.859375, 0.8125, 0.703125, 0.71875, 1.078125, 1.078125, 1.03125, 1.078125, 0.90625, 1.0625, 1.078125, 1.15625, 1.0,1.0},
+        {0.828125, 0.765625, 0.703125, 0.765625, 1.046875, 1.09375, 0.984375, 1.015625, 0.890625, 1.015625, 1.03125, 1.03125, 1.0,1.0},
+        {0.828125, 0.71875, 0.6875, 0.734375, 1.078125, 1.125, 1.015625, 1.078125, 0.90625, 1.078125, 1.125, 1.140625, 0.984375,1.0},
+        {0.828125, 0.75, 0.71875, 0.765625, 1.0625, 1.125, 1.0, 1.03125, 0.921875, 1.046875, 1.09375, 1.09375, 1.0625,1.0},
+        {0.8125, 0.71875, 0.65625, 0.671875, 1.03125, 1.0625, 0.984375, 1.015625, 0.875, 1.015625, 1.046875, 1.03125, 0.96875,1.0},
+        {0.84375, 0.703125, 0.6875, 0.703125, 1.0, 1.0625, 0.984375, 1.046875, 0.875, 1.03125, 1.046875, 1.078125, 1.0,1.0},
+        {0.765625, 0.703125, 0.625, 0.609375, 1.0, 1.015625, 0.96875, 0.96875, 0.828125, 0.96875, 0.96875, 0.96875, 0.921875,1.0},
+        {0.8125, 0.734375, 0.703125, 0.8125, 1.09375, 1.09375, 1.125, 1.078125, 0.84375, 1.0, 1.0, 1.046875, 0.953125,1.0},
+        {0.8125, 0.75, 0.671875, 0.71875, 1.03125, 1.03125, 0.96875, 1.015625, 0.875, 1.0, 1.015625, 1.078125, 0.953125,1.0},
+        {0.8125, 0.734375, 0.671875, 0.734375, 1.046875, 1.078125, 1.046875, 1.0, 0.875, 1.0, 1.015625, 1.03125, 0.953125,1.0},
+        {0.796875, 0.78125, 0.703125, 0.796875, 1.09375, 1.09375, 1.078125, 1.09375, 0.9375, 1.125, 1.09375, 1.171875, 1.125,1.0},
+        {0.78125, 0.734375, 0.65625, 0.734375, 1.0, 1.078125, 1.015625, 1.015625, 0.859375, 1.0, 1.03125, 1.046875, 0.953125,1.0},
+        {0.71875, 0.65625, 0.640625, 0.640625, 0.90625, 0.9375, 0.96875, 1.0, 0.84375, 1.0, 1.046875, 1.046875, 0.96875,1.0},
+        {0.75, 0.703125, 0.640625, 0.75, 1.09375, 1.140625, 1.015625, 0.984375, 0.828125, 0.96875, 0.921875, 1.03125, 0.921875,1.0},
+        {0.765625, 0.71875, 0.671875, 0.71875, 1.03125, 1.078125, 1.046875, 1.046875, 0.875, 0.984375, 1.03125, 1.046875, 0.984375,1.0},
+        {0.765625, 0.703125, 0.640625, 0.671875, 0.96875, 1.03125, 0.96875, 0.953125, 0.84375, 0.984375, 0.984375, 1.03125, 0.90625,1.0},
+        {0.75, 0.734375, 0.671875, 0.75, 1.03125, 1.046875, 0.984375, 1.046875, 0.84375, 1.03125, 1.03125, 1.09375, 0.984375,1.0},
+        {0.734375, 0.703125, 0.65625, 0.75, 1.046875, 1.046875, 0.984375, 1.03125, 0.875, 1.03125, 1.015625, 1.046875, 0.984375,1.0},
+        {0.734375, 0.71875, 0.703125, 0.765625, 1.125, 1.125, 1.09375, 1.09375, 0.9375, 1.125, 1.109375, 1.1875, 1.109375,1.0},
+
+        {0.71875, 0.71875, 0.6875, 0.71875, 1.0625, 1.03125, 1.0, 1.03125, 0.859375, 1.046875, 1.0, 1.046875, 0.96875,1.0},
+        {0.734375, 0.671875, 0.609375, 0.640625, 1.015625, 1.015625, 1.015625, 1.015625, 0.84375, 1.0, 1.015625, 1.046875, 0.984375,1.0},
+        {0.859375, 0.828125, 0.828125, 0.921875, 1.078125, 1.09375, 1.09375, 1.078125, 0.84375, 1.03125, 1.09375, 1.125, 1.03125,1.0},
+        {0.875, 0.859375, 0.90625, 0.984375, 1.09375, 1.078125, 1.09375, 1.03125, 0.875, 1.046875, 1.0625, 1.140625, 1.0,1.0},
+        {0.890625, 0.859375, 0.828125, 0.859375, 1.0, 1.140625, 1.09375, 1.046875, 0.84375, 1.046875, 1.078125, 1.09375, 0.984375,1.0},
+        {0.78125, 0.890625, 0.953125, 0.953125, 0.984375, 1.03125, 0.984375, 0.953125, 0.859375, 1.03125, 1.046875, 1.03125, 0.96875,1.0},
+        {0.953125, 0.90625, 0.96875, 1.046875, 1.0, 1.078125, 1.09375, 1.078125, 0.859375, 1.0625, 1.046875, 1.09375, 0.96875,1.0},
+        {0.953125, 0.921875, 0.890625, 1.046875, 1.046875, 1.03125, 1.09375, 1.109375, 0.859375, 1.03125, 1.109375, 1.078125, 0.953125,1.0},
+        {0.984375, 0.953125, 1.0, 1.078125, 1.015625, 1.03125, 1.0625, 1.046875, 0.828125, 0.984375, 1.0, 1.078125, 0.984375,1.0},
+        {1.046875, 1.0, 1.078125, 1.21875, 1.078125, 1.078125, 1.09375, 1.109375, 0.875, 1.109375, 1.078125, 1.125, 1.09375,1.0},
+        {1.09375, 1.03125, 1.09375, 1.171875, 1.015625, 1.046875, 1.078125, 1.015625, 0.84375, 1.03125, 1.03125, 1.078125, 0.984375,1.0},
+
+        {1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0}};
+
+        int[] runs = {14170,14185,14191,14200,14204,14210,14222,14232,14262,14272,
path: tracking/src/main/java/org/hps/recon/tracking/ShaperLinearFitAlgorithm.java
user: login: mgignac
body: Thanks @rodwyer100 ! The coverage of run numbers looks pretty good and uniform across the dataset, so this is probably fine. 
created_at: 2025-03-28T19:59:52Z
author_association: COLLABORATOR

PR comments:
user: login: rodwyer100
created_at: 2025-03-13T20:12:34Z
body: I am adding to html files with the results of the previous tests I mentioned. Its side by side hit on track and track pz distributions with text columns so you can see exactly how things change. The change is tiny, but I realize because the chi square determines if we do pileUp or not it will change things slightly. It does seem to remove a very small amount of tracks, but it does add more tracks to the momentum FEE peak and the tracks it removes seem to all be in the really low (perhaps out of physics scale) momentum and hit number variety: https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/momentum.html and https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/hitOnTr.html
author_association: CONTRIBUTOR
user: login: rodwyer100
created_at: 2025-03-13T20:13:50Z
body: I will be doing two simple analyses now. I will be removing the portion where I change the amplitude error (just to check that this indeed does nothing) and a chi sqr cut. You will get htmls for these too. After  that, I think it is relevant to make a choice as to whether to push this. Tim may want to see how the error changes with time, but I am doing the simplest studies first.

author_association: CONTRIBUTOR
user: login: rodwyer100
created_at: 2025-03-13T22:38:28Z
body: Added the runs without the amplitude err change here: https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/noAmp/hitOnTr.html. Compare it to the left sides of the previous htmls. You can see almost nothing changed (even entry wise) maybe singular track changes. I believe this confirms my hypothesis. Now I will implement a chi cut of .01
author_association: CONTRIBUTOR
user: login: rodwyer100
created_at: 2025-03-14T01:03:56Z
body: Here is the chi square cut of .01 applied. I need to figure out a way to plot this and without on top of eachother. You can definitely see the loss in efficiency, but I am not sure you make up for it in momenum: https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/withcut/hitOnTr.html
author_association: CONTRIBUTOR
user: login: rodwyer100
created_at: 2025-03-14T01:05:13Z
body: When it removes alot of tracks (i.e. in the second run) it does seem to do so predominantly in good places to do so, i.e. the junk at ~200 MeV.
author_association: CONTRIBUTOR
user: login: rodwyer100
created_at: 2025-03-14T17:52:23Z
body: Okay last study done; here is how the actual errors drift or dont over runs: https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/Errors/hitOnTr.html. It tells you that the strip error from the database (which is not the dsame as the chi square fit found from actually fitting pulses) doesn't drfit, and this is likely because it was insufficiently recorded or not recorded under beam.
author_association: CONTRIBUTOR
user: login: rodwyer100
created_at: 2025-03-26T16:28:17Z
body: I have done layer studies here: https://s3df.slac.stanford.edu/people/rodwyer1/chiFix/LayerChange/hitOnTr.html. You can find a subtraction of the two plots for 14170 here: https://www.desmos.com/calculator/w0ndtdbzud. There is a clear peak near the first 4, and it dies away with later layers. This was expected.
author_association: CONTRIBUTOR