<!-- Title: 08:45:04  Mon Dec 13 2004 -->
<!-- Crewchief: Dale Stentz (NWU) -->
<!-- Op1: Phil Schlabach (FNAL) -->
<!-- Op2: Andrew Kubik (NWU) -->
<!-- Op3: Ingyin Zaw (HU) -->
<!-- Op4: the many gone -->
<!-- Op5: but not forgotten -->
<!-- Op6: &nbsp; -->
<!-- Notes: -->
<i>Welcome to the CMX elog.</i>


<!-- Date: Mon Dec 13 09:03:12 2004 -->
Recall Ymon plots <a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=7&frame=2&anchor=11:24:47'> 
from previous page</a> in the elog.   
<p> 
There's a section of dead channels in west wedge 18, Ymon 20-23.  Sure enough I have not seen this area populated resently and looking at the TDC check I did way back in 11/17 I see it dead.  I don't remember an entire board being dead so I'm worried that one of the cables (LV or signal) is out.  How I didn't catch this is unknown - all I can say is that a long cosmic run at the end of the shutdown would have made it easier to spot. 
</p> 
<p> 
Here's some more detail:  In the 12/25/03 cosmic run and 6/16/04 (beam?) plots I have I see that wedge 18, YMon 20-22 are alive and 23 is dead.  Also note that YMon 20-23 = TDC {22, 18, 23, 19}.  It's the classic 2 by 2 pattern that I was looking for.  Its almost a given that Ingyin and I were in this area for our t0 study (we don't suspect foul-play). 
</p> 
I'll make a super low level access request - basically if somebody else needs the west toriod open we should go in.  The chances are high that we can spot the problem within the 1st min of looking at it.
<!-- Author: Dale -->
<!-- Comment: Mon Dec 13 14;56;27 comment by...Dale and Phil -->
<a href=
'/cgi/elog/elog.pl?nb=2004&action=view&page=866&frame=2&anchor=13:10:24.057'>
Fixed</a>.  We were able to put the loose LV fuses back in the preamp board.  Once Dale is off shift he'll take a look with the scope to make sure all the channels are good.

<!-- Comment: Mon Dec 13 16;21;51 comment by...Dale -->
SOB.  NW wedge '18' trips on the anode around 2.1 kV.  The rest of CMX comes up ok.  Clearly I did something to the cable/fuse box while I was crawling around.  
<p>Note that the wedge was fine before and was holding at stand-by during and after the access.  Also note that the miniskirt did not trip following the quench (we could blame the beam but this is clearly not the cause).</p>
I disconnected the anode and sure enough the cathode goes all the way up.  I'll switch back and watch it trip on rampup again.  I'm going to have a hard time explain this to ops - sigh...

<!-- Comment: Tue Dec 14 10;09;34 comment by...Dale -->
D0 has requested a long access (tentatively planed for 8:00 tomarrow - although a quench durning today's shotsetup could make it a ~1.5 day access).  Bottom line is that Rob wants to open up the plug on the west side to get access to the COT which gives us lots of time to make sure the NW miniskirt is back in working order.
<br><br>
In the same vein we should be extra careful that we or somebody else doesn't damage anything durning the access.

<!-- Comment: Wed Dec 15 12;38;28 comment by...Dale and Phil -->
We fixed it in less than 15 seconds.  As the toroid was almost all the way back Dale only need to look and feel to find that the 'T' connection was loose all the fathest south anode filter box (this would be the one on the 5 degrees that we worked on before).  <a href=
'/cgi/elog/elog.pl?nb=2004&action=view&page=872&frame=2&anchor=08:27:03.393'>
It's holding HV fine</a>.


<!-- Date: Wed Dec 15 15:51:04 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-378&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-378  width='300' align='top'></a></td><td>channel 4 = 6 (keystone) 
<br> 
channel 5 = 7+8</td></tr></table>
<!-- Author: Dale -->


<!-- Date: Wed Dec 15 15:57:53 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-379&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-379  width='300' align='top'></a></td><td>With a y-axis</td></tr></table>
<!-- Author: Dale -->


<!-- Date: Wed Dec 15 22:27:47 2004 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-380&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-381' ></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-382&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-383' ></a></center>
<!-- Author: Dale -->


<!-- Date: Wed Dec 15 22:56:07 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-384&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-385  width='300' align='top'></a></td><td>This shows what happed.  I'll explain later</td></tr></table>
<!-- Author: Dale -->
<!-- Comment: Thu Dec 16 07;23;02 comment by...Dale -->
Clearly from the iFIX icicle plot we see that the current on the drift voltage [SHPR] for wedge 6 [channel 4 for AR=arch, W=West] makes a drops by about 15 microA (this would likely be 2 chambers +/- 1).  The plot to the right (this would be the current on the anode WIRE for wedge 6) show in increase in current at the same time around 3:00 +/- 10 minutes.
<p>
During this time I was in with Ricardo putting in the BLMs.  We were done there around 2:50 and by this time the plug had just been moved in but several of the electronics crates were still out.  Shortly there after the shift crew started moving in the CMX arches.  About the time when we were finishing up Lew yelled out about something that looked like it was "hanging" - it was in the general dirrection of the overlap area.  I looked out (I'm at y=0 still putting the BLM in) and see that the overlap area does look like somethings out of place.
</p><p>
I asked Lew about it and we quickly got a lift in place and I ran up to see what was the matter.  The overlap area by default looks ugly but several things were wrong.  3 pairs (+1 female end without a male counter part) were unhooked.  The more obvious eyesore was that the Al SHV patch panel that's on the wedge 7 (SW arch) side was not its normal slot.  It was duct tape to the side of a more Northern (farther left) chamber.  More to the point the HV leads which run to 2 preamps and 2 anode wires were right aginst preamp 06-10 and the leads were rather streched out in this configuration.  Also note that this patch panel was grounded by a resistor (100 Ohm?) to a chamber grounding tab.  Now its broken (it actually looked like an anode lead at first.
</p><p>
I attempted to reposition the panel but I didn't have very many degrees of freedom.  In the process something in the upper far south part of wedge 6 sparked (the HV was still on).  I connected the grounding tabs (the least of our worries) and used the guy I couldn't find a mate for and used it in a make shift way to try to directly ground the panel via the broken grounding tab.
</p>
I thought for sure that CMX would have triped but the shift crew reported no trips on CMX.  When I ran up stairs it was holding fine but I noticed the anode current being aroun 11 microA.  I ramped the HV up and down once and it came back happy but drawing to much current (I didn't notice it right away but the current on the drift was a bit smaller).  I ran back down to relook at it but holding up on the end of the access when the wedge was effectively working fine wasn't a strong option.  I decided not to touch anything (it would have taken a lot of time to really put stuff back into place).  In particular I didn't think I could get the patch panel where it need to be without causing more problems.  I told Stefeno and Rob that it looked like it was working but that it was drawing to much current.

<!-- Comment: Thu Dec 16 19;23;09 comment by...Dale -->
I looked at the 2d occupancy plot and I don't see any new dead channels.


<!-- Date: Thu Dec 23 10:38:17 2004 -->
Operations is aware of our request for a special (hopefully one time only) procedure for pulling the SW arch. 
<br><br> 
If we have enough time, we have two basic options: 
<ol> 
<li>Modify things so that they are in a better configuration - 
basicially get things back to the way they were before the end of the 
shutdown.</li> 
<li>Undo the HV for the 4 overlaping chambers (we are currently not 
reading them out for physics but the hope was for somebody in the 
future to recover these guys) - the pre-shutdown setup effectively.</li> 
</ol> 
Either way HV is still going to thoes patch pannels and the chance of something bad happening (in its current state) is better than not.  A tech on a lift should, at most, have to hold up cables for a short duration to make sure nothing gets caught.  Other wise it should close and open with out snaging or being problem-matic.  This is how we thought it was the last 3 times it was moved durning the August 2004 shutdown.
<!-- Author: Dale -->


<!-- Date: Thu Dec 30 14:08:17 2004 -->
<a href= 
'/cgi/elog/elog.pl?nb=2004&action=view&page=917&frame=2&anchor=13:33:43.558'>The honeymoon is over</a>. <p> SW '16' started to trip on the cathode well into a store (lum ~ 30 E30).  Trips on the cathode around 2.4 kV (draws .4 mA at this - certainly goes non-linear around 2.0 kV.  I got the drogie on it - I can get it up to ~2.8 kV for a bit but it draws to much and trips off quickly.  I set it on auto-reset (for trips) and about half way through this e-log entry the current dropped to where I expected for 2.4 kV and I quickly raised it to 3.0 kV.  It's holding drawing .44 mA on the cathode (about right).  I'll inch up the anode - may go to some crazy scheme like before but let me get it to hold for 1/2 an hour before trying to convice ops to stop the run to include it again. 
</p>
<!-- Author: Dale -->
<!-- Comment: Thu Dec 30 14;10;37 comment by...Dale -->
Cathode at 3.0 kV while the anode at 5.2 kV (draws under .1 microA)

<!-- Comment: Thu Dec 30 14;30;13 comment by...Dale -->
Turned the wedge off and on a few times.  It's in its happy state - cannot get it to trip or go bad.  
<br><br>
Raised the cathode to 3.1 kV to see if it would hold.  It does and the current is steady as a rock.  I adjusted the anode up to 5.5 kV just for kicks.  I told the shift crew that run 191726 should be marked bad due to the HV "not being where it should" I also told them that the occupancy might look normal.  Going to wait a few more minutes.  
<br><br>
Options: hook it back up the CAEN and iFIX <b>OR</b> let the droggie stay on and let the Aces check it at the end and start of each run.  I prefer the former - but I hate to waste time if the wedge goes bad again and the run has to be stopped and started...

<!-- Comment: Thu Dec 30 14;33;25 comment by...Dale -->
Checked it and it tripped.  The problem looks like its back.  I'll ramp it up slow and see how it responds.  Clearly I cannot include it back in if it goes bad every 20 min.

<!-- Comment: Thu Dec 30 14;42;15 comment by...Dale -->
Came right back up - back to normal.  Going to wait until 15:00.  We might go for the latter option noted above if it starts to go back and forth.
<br><br>
My Basic impression is that after a while the wedge goes back to being normal.  It is how it acted before (I'll include some links later).  Ingyin and I were able to sneak it back in a few times when other problems came back and we usally made it to the end of the store.

<!-- Comment: Thu Dec 30 14;52;46 comment by...Dale -->
Lots of realvent entries from before the shutdown.  <a href=
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=6&frame=2&anchor=13:54:19'>
This</a> is a good one to look at.


<!-- Comment: Thu Dec 30 15;03;48 comment by...Dale -->
I talked with Bob - he's fine with me including the wedge back in (hooking it back up and running as normal).  I have ramped the HV up and down a few times - it seems happy and ready.  I will likely ramp the thing up while DAQ is in the halt state - if it stays on then we end the run - else we keep going and turn the wedge off for a while.

<!-- Comment: Thu Dec 30 15;26;48 comment by...Dale -->
It's back in and holding fine.  <a href=
'/cgi/elog/elog.pl?nb=2004&action=view&page=917&frame=2&anchor=15:12:32.321'>Link</a.> 



<!-- Date: Fri Dec 31 01:30:07 2004 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-387&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-387  width='300' align='top'></a></td><td>AC_AT_RP1_RAH = Coll. Hall rel. Humidity, NE<br> 
WTR_DEWPT_3 = DEW POINT COLLISION HALL 
<br><br> 
Realtive Humidity went up by 50% (it was realtively warm and wet for a December day).  Not a proof just an observation.</td></tr></table>
<!-- Author: Dale -->
<!-- Comment: Fri Dec 31 01;33;31 comment by...Dale -->
Plots brought to you by <a href= 'http://cdf-fs2.fnal.gov:3000/plug/javachart/icicle.htm'>http://cdf-fs2.fnal.gov:3000/plug/javachart/icicle.htm</a>


<!-- Date: Sat Jan 1 23:02:33 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-388&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-388  width='300' align='top'></a></td><td>Note: its cold but raining now.  In any event, the values came back down to about where they were before.  Also note no HV trips on CMX on the current store.</td></tr></table>
<!-- Author: Dale -->


<!-- Date: Sun Jan 2 01:55:49 2005 -->
I spoke too soon.  <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=4&frame=2&anchor=01:18:26.794'> 
SW '16' started to trip again</a>. 
<br><b> 
I can get it to trip realtively quickly at 2kV on the drift.  When I turn the anode on (around ~2 kV just for "support") I notice that it will trip off with the cathode (it doesn't have to do that).  Not looking good as far as getting back in the run any time soon.  Going to work it for for awhile.
<!-- Author: Dale -->
<!-- Comment: Sun Jan  2 01;57;01 comment by...Dale -->
</b>that should have been a 'br' not a 'b' in side the tag.

<!-- Comment: Sun Jan  2 02;03;47 comment by...Dale -->
It draws .38 mA at 2.4 kV on the drift (2 kV on the anode).  It non-linear but "looks" steady (trip is sudden and quick).  Its trip one at these settings.  I'll know it might be good if the current drops down some.  For now I'll keep teasing it.

<!-- Comment: Sun Jan  2 02;22;21 comment by...Dale -->
I can get it, at certain periods, up to 2.8 kV but it draws .46 mA and hasn't lasted much more than 5 min.  If I auto reset the trip it will usally just trip off right away.  I'm looking at the realtive humidity and dew point - they look like they are heading up.  I'll post a plot later.


<!-- Date: Sun Jan 2 02:39:54 2005 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-389&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-389' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-390&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-390' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-391&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-391' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-392&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-392' width='300'></a></center>
<!-- Author: Dale -->
<!-- Comment: Sun Jan  2 04;04;54 comment by...Dale -->
Simular to the plots posted before - just cooler looking.  The 1st and the 3rd have to do with the <a href=
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=14:08:17'>first occurancy of a problem</a> and the 2nd and 4th have to do with the most resent espoide for HV problems for SW wedge '16' [iFIX 15].
<br><br>I talked with cryo - there's no fast action to reduce the humidity.  <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=4&frame=2&anchor=03:52:04.397'>
Here's</a> a link to the shift e-log with my rather current set of instructions.  



<!-- Date: Mon Jan 3 10:19:21 2005 -->
JDL ask if he could redo / undo the cables we had over by the muon scintlators (in paritucular the NIM area where we have our NIM to ECL converters.  We haven't used in any time so I told him to knock whatever he needed out of the way and that (around the NIM crate) it would be easy to to redo things if we need to anyway.  Basically this is harmless to us and shouldn't represent a problem.
<!-- Author: Dale -->
<!-- Comment: Wed Jan  5 09;23;11 comment by...Dale -->
Update:
<br>Rick need to work in the area too so I went ahead and move the B0RAT* (* = 1...8) cables to go up a differnt cable column.  This should greatly help Rick with all of his cables and certainly stay out of the way of Jonathan's work.
<!-- Date: Tue Jan 4 09:38:51 2005 -->
Sigh...<br><br> 
The on going theory on humidity effecting the miniskirt chambers was tested big time <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=11&frame=2&anchor=08:33:40.540'>as water is everywhere</a> around the miniskirts and CMP bottom.  We eventally turned all of CMU [because the miniskirts are powered by the same CAENs], CMP [bottom + toes], CMX [miniskirt] which were effected clearly effected off when Phil tried to bring things up<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=11&frame=2&anchor=08:23:09.323'>after the power outage</a>.  We have locked the system by turning the "enable" switch on the CAENs for CMU [miniskirts] and CMP (CMX should be fine).
<!-- Author: Phil and Dale -->
<!-- Comment: Tue Jan  4 10;14;18 comment by...Dale -->
Update:
<br>
Basically, the water that's used to cool many of the electronics racks was released.  The west side was effected worse than the east and the SW was perhaps the hardest hit.  We have the toroid open on the east side but the surveyers are working on the west side this morning so the west toroid is closed.  They got fans and air-ducts blowing air everywhere.  Phil suggest that we (Xiaojian and I) wait   ~8 hours before trying to turn to stand-by.
<br>
<br>
Note that there may be some independent problems with the CAEN monitoring but we will pass that bridge when we come to it.  For now we wait and hope things dry out.  Basically as Rob was going to do any way they have spare fans that they will use to blow air through the BMU-toroid and CMX-arch gap (durning normal operations) and where ever they will be best served now.

<!-- Comment: Tue Jan  4 16;12;54 comment by...Dale -->
The arches+keystone came on fine as expected when turned on.  I have tried the SE, NE, and NW miniskirts.  3 of the 6 wedges holds (18+19 on the east and 19 on the west).  Going to wait (an hour) before trying the SW or the others that failed.


<!-- Date: Wed Jan 5 08:17:08 2005 -->
Further update on wedge status: 
<p> 
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=13&frame=2&anchor=01:16:51.933'>During the owl shift</a> Paul was able to get the other NW miniskirt up.  The SW defied all logic and came up when I tried them (<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=12&frame=2&anchor=16:58:11.221'>evening shift</a>). 
</p> 
I've just tried the SE Miniskirts (the only remaining troublemakers).  Taking '16' [iFIX 15] to stand-by caused it to trip.  The anode current was racing up (drawing +100 microA!).  East '17' [iFIX 16-17] was a bit better.  It made it to stand-by and was drawing ~6 microA while holding (still way to much).  It tripped while writing this elog entry so maybe it held for 5 min. 
<br>
<!-- Author: Dale -->
<!-- Comment: Wed Jan  5 14;17;31 comment by...Dale -->
May have gotten iFIX 16/17 back enough to run with.  It draws about 1.5 microA on stand-by.  iFIX 15 is still in trouble.  See <a href='/cgi/elog/elog.pl?nb=2005&action=view&page=14&frame=2&anchor=13:34:59.835'>Phil's entry</a>.


<!-- Comment: Thu Jan  6 12;21;21 comment by...Dale -->
When the store started durning the owl shift all HV had recovered such that we were able to run without any trips to the miniskirts.  Unfortinately, there would be many other problems...


<!-- Date: Thu Jan 6 12:24:37 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=17&frame=2&anchor=11:40:26.109'> 
See status here</a>.  More details to come...
<!-- Author: Dale -->
<!-- Comment: Thu Jan  6 14;08;43 comment by...Dale and Phil -->
<pre>First the LV issue with SE miniskirt [15-17].  The Low Voltage fuses for the gray LV boxes for each set of physical wedges much be blown (or at least one set [+12 or -6] is gone.  It's unlikely its anything else but the possiblities are

1. LV is pluged out at the power source [exteremly unlikely]
2. ASD cards are fried [extremly unlikely - have to be 3 cards and signal cables shouldn't carry an electrical short]
3.  No gas - don't think there is any easy way to turn off the gas flow to 6 sections [fairly unlikely - easy enough to check]

We have summited an access request and will go after the fuses.  Should be quick and dirty (under the toroid that is).

The problem with HV for wedges 4 and 5 a news to us.  There's a high proablity that the problem is with the new addition, wedge 5.  We will start there first.  We will likely have to do it first by 3rds - i.e. hope to recover 2/3 of the bad wedge.

Throughout the morning I have been testing west 4-5 - I'll add what I found in a comment below.</pre>

<!-- Comment: Thu Jan  6 14;57;34 comment by...Dale -->
Here are the highlights:
<p>
The trip occurs on the drift/grid/cathode although 4 out of the 5 trips recorded by iFIX claim that both the anode and cathode triped.  When I got it on the droggie (intially) I see that it would trip on the cathode (just ramping it up).  Would seem to be stable at 2.4 kV but would trip above this around the 1-5 minute or less time scale.  I observed that it liked being brought back up to HV slowly.  The currents have always been reasonable.  For example, 2.6 kV yields a .59 mA draw (recall that its really only about 9 or 10 stacks due to the HV configuration and non-existant chambers in wedge 5).  
</p><p>
After writeing up the control access request I turned it back on and it held for an hour at 2.4 kV.  I later increased it to 2.5 kV just before the 12:30 meeting.  It was holding an hour later.  I even rasied it to 3.0 kV and had it hold there for a few minutes.  Current draw was steady at .66 mA.</p><p>
I'm not sure how real it is but I have observed that the cathode is more likely to trip as I bring up the anode HV.  In any event, there were several times where I had gotten it to a stable voltage that was useful (~2.6 kV) and finished or nearly finished my ramp up on the anode the system would trip.  It's possible that whatever time constant the system has might be effected in a round-about way by the anode.</p>
In any case, it seemed to have a realtively long life (~+15 min) at near nominal voltages but resently it's gone back to how it was this morning.  Just playing with the cathode around 2.5 kV is enought to trip things off.  I'm going to see what value it holds at (currently its holding at 2.5 kV - next to nothing on the anode).  If it proves un-useful I will likely see "measure" the time constant for tripping at 2.8 and 3.0 kV (hopefully it's consistantly under a minute).

<!-- Comment: Thu Jan  6 16;59;52 comment by...Dale -->
I played with the HV several times.  Seems that I could work my way up to nominal voltages but once there was a trip it would make everything flaky so that I couldn't get the cathode past 2.5 kV.  Right I got the cathode at 2.4 kV with the anode at 1.0 kV (just above what it normally feels via the drift field).
<br><br>
The system would trip quickly and semi-reliably once it had had become flaky.  I suggest (to myself) to overvoltage the cathode when testing it out in the hall and to ramp it up quick the first time to get it to trip.  Then turn off, wait, and then ramp up the cathode slowly.  At 2.6 kV it trips on a minute time scale but at 2.8 and 3.0 kV it trips right away once its been "condidtion" to behave badly.  Obviouslly I will look and listen for a spark to start off with.  Otherwise, I will just have to be cautious and march through each 5 degrees until I find the bad section and then limit the problem to a board.  I will likely need 3 sets of SHV cables (2 for the cathode, one anode to share).  
<br><br>
Going to leave it disconnected from the CAEN until there a chance to play with it during quite time.  No solid word on if there will be a access any time soon although the store should end by 6:00 on Friday.

<!-- Comment: Fri Jan  7 09;35;05 comment by...Dale -->
It held at 2.4 kV the whole night.  They are about to put protons in so I don't dare turn the anode up - but for the time being I got the drift over-voltaged at 2.9 kV.


<!-- Date: Fri Jan 7 11:13:51 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=20&frame=2&anchor=09:43:07.114'> 
Going to try 4-5 again</a> in the next store.  We may try to reduce the settings but we'll try things as normal to begin with.
<!-- Author: Dale -->
<!-- Comment: Sun Jan  9 17;57;44 comment by...Dale -->
We <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=20&frame=2&anchor=09:43:07.114'>
tried</a> but it <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=20&frame=2&anchor=11:44:32.870'>
triped</a> on ramp up.  It was turned off.


<!-- Date: Sun Jan 9 18:43:47 2005 -->
After the store was prematurely aborted we had an access. 
<p> 
The executive summary is posted <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=21&frame=2&anchor=18:26:44.722'> 
in the shift elog</a>.  Basically we fixed the LV and recovered most of wedge 5 such that 4-5 would hold HV</p> 
<p> 
The access went virtually exactly as we planed.  We replaced the LV fuses at the SE miniskirt.  In each case the 1 of the 2 fuses from each box was blown.  We had plenty of time to run up and see signals in each section of the SE miniskirt.  The following store shows that the 1-D occupancy looks as it should - we may want to look closer and make sure there aren't any new dead channels. 
</p> 
<p> 
We got the lift in about as quick as we could to go after the wedge 4-5 HV problem.  We looked first at wedge 5 and sure enough it tripped while ramping up around 2.6 kV.  Dale couldn't see the trip but could sometimes hear it.  It was the "clunck" [not "snap"] sound that Ingyin and Dale associated with a problem just inside where the grid voltage enters via the HV jumpers.  The problem seems to be in the 1st 5 degrees (based off of the sound).  We tested this area first.  Sure enough it tripped but it was not reliable.  Also over voltaging it didn't necessarily help or speed things up.  Long story short We had to ramp it up at a modest pace and sometimes it would hold for a few minutes.  It was flakey at best.   
</p> 
<p> 
It appeared that we had it down to the 4 physical chambers of stack 0 with the knowledge that stacks 2-5 held and could be ramped up and hold voltage with ease [note this is the modified binomial search or Stentz's maxim: it is easier to identify a large good section then a small occasionally will trip bad section].</p> 
<p> 
We tested wires 0+2 and 4+6 ('+' means tested together).  0+2 (this would be the 2 inner most chambers of stack 0) seem to trip on ramp up but 4+6 were adding to the confusion.  Namely 4+6 tripped when over voltaged (3.0 kV) due to the current spiking past 80% maximum (this is on the .1 mA scale).  Our guess is that either 0 or 2 (or perhaps the drift jumper in-between) is the bad guy and 4+6 are fine when combined with the rest of the wedge. 
</p><p> 
Unfortunately, time was winding down.  It took Dale some time to get stack 1 connected back to stacks 2-5.  At this point, the best plan was to make sure that stacks 1-5 would work [stack 1 had not been tested by itself].  Luckily Ingyin and Dale had been smart enough to have a relatively long jumper that went from stack 1 wire 7 (or was it 6?) where the drift/cathode HV gets piped into the wedge such that stack 0 could easily be by-passed.  The end result is that stacks 1-5 were hooked up in a very natural way.  Sure enough the wedge without the first 4 chambers in phi held HV and had no trouble when ramped up quickly.  We, of course, also turned the anode all the way on and saw that the current draws were as expected.</p> 
We had the shift crew turn on the HV and sure enough it held.  Our plan is to submit a low level access request (perhaps wait for supervised access) to hopefully recover 3 chambers.  Note that the first store afterwards there were no CMX trips!  
<!-- Author: Phil and Dale -->


<!-- Date: Wed Jan 12 14:10:58 2005 -->
It warms up - rains off and on through the night and morning - miniskirts trip (e.g. <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=35&frame=2&anchor=13:06:37.248'> 
here</a>). 
<br><br> 
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=35&frame=2&anchor=13:30:48.046'> 
Here's a plot of the realtive humidity and dew point</a>.  The "good" thing to note is that the SW has not tripped yet.  The bad news is that the SE miniskirt trips (its off now) and, of course, this was the hardest hit when the collsion hall flooded.  As we already suspect water on the electronics it's easy to believe that it got in around the chamber end plates (the trips were on the grid/drift by the way).  Also note that the trips were recoverable when the humidity was rising and got out of hand near  the maximum in realtive humidity/dew point.  One can take this to mean that the fans are helping out - when the SW would go 'bad' it was durning the rise in humidity (in fact we could recover near the maximum).   
<br><br> 
Going to go back and try it in a little bit...
<!-- Author: Dale [1.0] -->
<!-- Comment: Wed Jan 12 15;59;48 comment by...Dale [1.5] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=35&frame=2&anchor=14:44:18.337'>
Realtive humidity at 42.5%</a> and not moving down.  Called Rob - it might be possible for him to shutoff or lower the incoming air and try to dry out the air inside.
<br><br>
I got the HV on the SE on for a short <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=35&frame=2&anchor=14:53:45.928'>
while</a> but it trips on the order of < 10 min - much too often.  By the way, <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=35&frame=2&anchor=15:42:33.613'>
DAQMon</a> is working and finds that CMX is bad (and also when it's good).
<br><br>
Will leave some instructions for the shift crew - basically it depends on if the humidity goes down - I hope it gets colder as the sun goes down...

<!-- Comment: Wed Jan 12 21;45;24 comment by...Dale [1.0] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=36&frame=2&anchor=21:25:07.013'>
Another plot</a> - things have not improved.  Ace did a test before beam was put in and <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=36&frame=2&anchor=20:58:59.963'>
SE 16-17 did not hold</a>.  Possible that some of the other miniskirts may also trip - I told the shift crew to expect that and to minimize time taken away from data taking if the CMX is already being marked bad.  I hope there is a cold weather front moving in.  Right now it continues to rain...

<!-- Comment: Thu Jan 13 08;57;13 comment by...Dale [1.0] -->
<b>Update:</b>
<p><a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=37&frame=2&anchor=00:09:58.957'>
East (iFIX) 16/17</a> didn't make it (as expected) at the start of the store.  The aces follow my instructions and <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=37&frame=2&anchor=04:46:40.796'>
recover the wedge</a> around 4:30.  Sure enough the humidity had decreased by then.  No trips reported and rel. humidity/dew point are at their normal levels.
</p>
<p>
At the 8:00 meeting Rob and Dervin talked about reconfiguring their system to helpout more on the SE miniskirt.  I pointed out that the SW was bad before the fans and air ducts had been put in place but didn't cause us any trouble durning the resent eposiod.   Thus they were doing their job and helping out.  Dervin thinks that we are blowing the humidity to the SE side.  I don't think it works that way but maybe there is some truth to this. In any case, the SE getting drenched couldn't have help its state when hot and humid.
</p>
The rain turns to snow.  Cold weather front moving in - looking forward to below freezing tempetures.


<!-- Date: Fri Jan 14 14:21:42 2005 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-393&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-393' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-394&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-394' width='300'></a></center>
<!-- Author: Dale -->
<!-- Comment: Fri Jan 14 14;26;42 comment by...Dale [1.0] -->
Rather poor quaility plots but they do the job.  This the run before they started doing all these tests and the beam condidtions got worse (<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=40&frame=2&anchor=07:15:20.326'>
see here</a>).
<p>
The SW CMX arch sees a higher trigger (also something going on over on the SE).  The factor of 5 doesn't make much sences as teh occupancy is cosistant in the occupancy.  I'll check the counters in a moment.
</p>

<!-- Comment: Fri Jan 14 14;29;19 comment by...Dale -->
CSX looks fine.  Note that the above conditions is ture for all the runs that I looked at (most of which are past run #192300).


<!-- Date: Fri Jan 14 14:49:25 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-395&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-395  width='300' align='top'></a></td><td>Compare with above - this is from the previous store.  SW seems to be more in check but still larger then the NW.  Perhaps I'm just paranoid.</td></tr></table>
<!-- Author: Dale -->
<!-- Comment: Fri Jan 14 16;13;27 comment by...Dale [1.0] -->
I put a plot in the <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=41&frame=2&anchor=15:22:35.366'>
elog</a>.  Tom suggested that I look at XFT - didn't see any hot channels in the area expected.  I definately think the rate is high er than it ought to be.  However, as far as I can see CMX/CSX appear fine.


<!-- Comment: Mon Jan 17 12;34;48 comment by...Dale -->
It looks fine in (a later run) #192360 <a href = 'http://www-cdfonline.fnal.gov/internal/mon/cdfdoc/Consumers/LatestConsumerSlides/ts_192360.html#_1_4_3_1_Cmx'>see here</a>.  Also a simular BMU problem appears to be gone.
<!-- Date: Sun Jan 16 21:11:20 2005 -->
In addition to SW  9-10 getting the crap beat out of it by the beam (<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=42&frame=2&anchor=18:31:18.932'>example</a>) SE '17' might be going bad. 
<p> 
We might be catching the back end of the great flood.  SE iFIX 16/17 (<a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=14:10:58'>which had troubles above</a>) has been acting up <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=48&frame=2&anchor=19:41:50.943'> 
again</a>.  I looked at the HVAC varibles.  There was an increase in humidity (basically it looks like a temp. went down a degree and then everything shot up) but it much dryer than before.   
</p> 
If the anodes tripping I don't know what to think - the problem could go away or get worse.  If its the grid/drift voltage then its possible that a chamber has gone bad (or is messed up by a minor variation in temp/humidity, etc.).  Right now the time constant looks to be about 1 per hour but might be getting quicker.  
<!-- Author: Dale [1.0] -->
<!-- Comment: Mon Jan 17 08;40;57 comment by...Dale [.25] -->
<pre>Trip History for SE '17':

Sun Jan 16 01:49:45 2005  Anode - Note time stamp
Sun Jan 16 12:32:55 2005  Both (anode leads)
Sun Jan 16 12:37:46 2005  Both (anode leads)
Sun Jan 16 15:34:10 2005  Both (anode leads)
Sun Jan 16 17:35:45 2005  Both (same time)
Sun Jan 16 19:40:57 2005  Both (Field leads)
Sun Jan 16 19:42:27 2005  Field [just grid, just after above]
Sun Jan 16 19:45:06 2005  Both

Afterwards, smooth sailing...</pre>
<!-- Date: Sun Jan 16 21:43:20 2005 -->
Trigger inhbit problem is talked about <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=42&frame=2&anchor=19:24:24.614'>here</a>. 
<p> 
Trigger inhibit gaps are where CMX is off and being turned on (null = no inhibit).  RC should be paused or halted so that we don't take data.  Two eamples are below.  RC time (measured in seconds from the start of a run) should (duh) increase but not @ event # which is the event # that RC at when the inhibit status changes. 
</p> 
Example of a correct procedure (via run summary) [note: MS trip]<pre> 
date        time    @event  RC time 
2005.01.16 19:44:50 5078846 24911    CMU00 West Trip  
2005.01.16 19:44:58 5078846 24920    CMX HV:iFix CMU00 West Trip  
2005.01.16 19:45:22 5078846 24944    null 
2005.01.16 19:45:31 5078846 24953    CMX HV:iFix  
</pre> 
Example where @event increases during inhibit gap: 
<pre> 
                    @event  RC time 
2005.01.14 19:55:21 614186  9976     CMX Trip  
2005.01.14 19:55:37 614186  9994     CMX HV:iFix CMX Trip  
2005.01.14 19:55:46 614186  10000    CMX HV:iFix  
2005.01.14 19:55:54 614211  10009    null  
2005.01.14 19:56:02 614840  10018    CMX HV:iFix  
</pre>
<!-- Author: Dale [1.0] -->
<!-- Comment: Wed Jan 26 13;17;57 comment by...Dale [.25] -->
From 2005.01.25, run 192878
<p>
Ok this is a bit odd:
<pre>
                @event
-------------------------------------------------------------
 17:17:26 	838708 	4238 	CMU00 West Trip
 17:17:34 	840210 	4247 	CMX HV:iFix CMU00 West Trip
 17:17:58 	840210 	4271 	CMX HV:iFix
 17:18:07 	840210 	4277 	null
 17:18:15 	840210 	4286 	CMX HV:iFix
 17:20:08 	840210 	4400 	null
-------------------------------------------------------------
 17:28:45 	965967 	4916 	CMU00 West Trip
 17:28:53 	965967 	4925 	CMX HV:iFix CMU00 West Trip
 17:29:17 	965967 	4949 	CMX HV:iFix
 17:29:27 	965967 	4958 	null
 17:29:33 	965967 	4964 	CMX HV:iFix
 17:31:42 	965967 	5078 	null
</pre>
The second one is "perfect" event # stays the same.  In the first one 9 seconds go by after the trip and we take ~1k events.  Huh?  How did this happen?  In that run there were 4 trips: 2 good, 2 "bad/odd".  Note 6 trips in total on SE '17' (due to humidity) as noted 
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=75&frame=2&anchor=16:38:37.717'>here</a>.
</p>
<!-- Comment: Wed Jan 26 22;50;42 comment by...Dale [.25] -->
Bill explains that this may just be a lag.  <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=77&frame=2&anchor=15:21:18.118'>See here</a>.
<!-- Date: Tue Jan 18 14:19:27 2005 -->
<p>JJ asked about the muon calibration.  We should make sure that the calibration works as it might be useful in the future to the muon group.  Asked the current Ops Manager (Bob Wagner) to go ahead and try the calibration - we'll note the error and pass it along to more informed types.  Note that the COT calibration is not being done by the shift crew either for (apparently) the same reason (TDC code/version change, etc.).</p>
<!-- Author: Dale [.25] -->
<!-- Comment: Wed Jan 19 11;29;04 comment by...Dale -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=54&frame=2&anchor=20:11:40.804'>
Bill is on the job</a>.  See link for error message.<br>  Summary of Bill's work:<p>"I tried several different versions of fer code:  v3_53, v3_51a, v3_49, but all had the same crash.  Requires further investigation." -Bill</p>


<!-- Comment: Fri Jan 21 11;01;09 comment by...Dale -->
Might already be fixed - <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=60&frame=2&anchor=18:54:05.963'>
it ran yesterday</a>.  Here are the "offical" <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=62&frame=2&anchor=10:46:06.771'>instructions</a>.  
<!-- Comment: Sun Jan 23 12;44;11 comment by...Dale [.25] -->
<p>I talked with Bill, he said that something to the effect that part of the new code was always crashing when loaded into the crate.  The bottom line is that he thought it was working from the DAQ prospective.  At some point somebody in the muon group should look at this and see if the calibrations make any sense.
</p>
<!-- Date: Thu Jan 20 08:50:15 2005 -->
Talk about <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=58&frame=2&anchor=05:15:26.015'>bad timeing</a>, SE miniskirt '17' trips at the start of the store.  They got it back running after waiting a few minutes.  There's a lot of activity in the humidity plot but everything was low when the problem happened.  Note that during the access the air-duct was moved to the SE (this may have helped its hard to say right now).
<!-- Author: Dale [.5] -->
<!-- Comment: Fri Jan 21 10;57;09 comment by...Dale [.5] -->
<p>I tried turning on the HV for SE '17' and it triped the first time up (on the cathode - it had just reached 2.8 kV).  The second trip was on the anode and looked like it was due to MCR putting in beam suddenly (this was done durning non-quite time).  However, the 3rd time was the charm - it held throughout the 8:00 AM meeting.  I turn in on after the quench/abort today and found that it held HV without issue.  The humidity and related quanties were are low (decreasing) levels. </p>




<!-- Comment: Mon Jan 24 21;24;49 comment by...Dale -->
I have seen this behavior a few times.  It acts like (fakes) a charge build up like effect - it holds HV after triping on the first try.  Phil and I looked an it turns out that Steve put the air duct on the inside with the CSX.  This in principle helps to move air towards the middle and the bottom.  It helps and may be the best action to take but it be nice to move air along the floor in front of the miniskirt.  It's hard to say how "bad" the SE miniskirt is we haven't had it on for any long periods of time (no beam).  We'll have to wait and see. 
<!-- Date: Mon Jan 24 15:18:42 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=71&frame=2&anchor=14:37:34.869'> 
Went in on a control access</a>.<p> 
We went after the 4 chambers left for dead back on Jan 7th, <a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=18:43:47'> 
see above</a>.  We were able to recover and hold HV as normal by by-passing only wire 2 [thus wedge-stack-wire = 5-0-2], this is the second chamber from the bottom (interior) on the lowest (physical) part in phi for wedge 5. 
</p> 
<p> 
Durning testing when this chamber and wire 6 (just the cathodes) were independently tested (the anode to preamp board was wire was  unpluged) they would often trip together.  Testing them one-at-a-time wire 2 should up as the bad guy (wire wire 6 held).      Wedge 5 held (with just the 1 chamber by-passed from the orginal group of 4) at 3.0 kV with a current draw of (if Dale memory is good...) ~340 mircoA.  Anode was around .2 microA.  Ace confirmed the expected current draw when we hooked everything together. 
</p> 
We put the NW toroid back in to place (Rob taught Phil and to a lesser extent Dale to move the toroid).  Note that the modified radeka from 5-0-2 was removed.
<!-- Author: Phil and Dale [3.5] -->



<!-- Date: Tue Jan 25 11:00:23 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-396&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-396  width='300' align='top'></a></td><td>2d occupancy - I'll add some comments in the future</td></tr></table>
<!-- Author: Dale -->
<!-- Comment: Wed Jan 26 10;48;20 comment by...Dale -->
As expected wedge 5, YMon = 2 (5-0-2) is dead as it has been by-pass.  The rest of 5 looks ok.  Miniksirts need more stats as always...  At somepoint I will have to update the CO plots now that things are back in working order.
<!-- Date: Wed Jan 26 10:42:14 2005 -->
<center><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-397&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-397' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-398&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-398' width='300'></a><a href='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-399&button=yes' target='xxx' onclick='jswindow()'><img src='/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-399' width='300'></a></center>
<!-- Author: Dale [.25] -->
<!-- Comment: Wed Jan 26 11;11;55 comment by...Dale [.5] -->
The suspect area is where Bill add some funky code [<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=29&frame=2&anchor=15:26:51.050'>link</a>] and continue to (or past) 
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=29&frame=2&anchor=15:52:18.710'>
this point</a>.  This happens all around and just before 192.2.  Bottom line is that things have been constant for a while if you take out the above problem.  I'll post plots that cover well into the previous shutdown (the ones Steve Levy showed) but I'm overly conserned about the question in Xiaojain's 8:00 meeting report:
<pre>DAM [DQM/PhysMon?] people saw a resulting higher ratio of CMUP to CMX muons in Wmunu events. They want to know that in such case, should the run be marked as good or bad?</pre>
I need to see their plots, the problem seems to be definately not on our end.




<!-- Date: Wed Feb 2 14:08:22 2005 -->
sigh... 
<p> 
NE "18" has gone to the darkside.  It slowly started to trip with about a 1 to 3 ratio with the SE 16-17.  Looks like it may have overtaken it now.  I'll add the links and what not later... 
</p> 
It trips on the cathode, fast and sudden.  Certainly related to the humidity changes but at this point it looks like it has gone bad.  Holds at 2.4kV (and currently 2.5 kV) on the drift.  I'll inch it up but it was tripping at 2.6 kV when I started.
<!-- Author: Dale -->
<!-- Comment: Wed Feb  2 14;18;00 comment by...Dale -->
Now it is holding at 2.6 kV (anode 5.2 kV).  When I tried to take it up to 2.8 kV it tripped quickly after I stopped ramping.

<!-- Comment: Wed Feb  2 14;22;09 comment by...Dale -->
I got it up to 2.8 kV.  At some point I will overvoltage it and see if it 1) trips 2) if it does trip does it trip when I ramp it up.  [I may force the first to happen as the second condition determins whether or not we include it any time soon.]

<!-- Comment: Wed Feb  2 15;02;41 comment by...Dale [2] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=98&frame=2&anchor=14:03:05.748'>It's back in</a>.


<!-- Date: Fri Feb 4 13:47:08 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=104&frame=2&anchor=12:14:35.323'>It</a> broke - I <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=104&frame=2&anchor=13:28:40.952'> 
fixed</a> it.
<!-- Author: Dale [1.5] -->
<!-- Comment: Tue Feb  8 12;26;37 comment by...Dale [1.5] -->
Bad periods of running:
<p>
Some problems Saturday as noted <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=106&frame=2&anchor=01:31:08.600'>
here</a>.
</p>
<p>
At the start of the data taking (on Sunday) on Sunday problems abound: <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=109&frame=2&anchor=03:16:12.327'>
see here</a> for instructions and documentation by the ace <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=109&frame=2&anchor=01:50:11.001'>
here</a>.
</p>
Trips well doumented Sunday <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=111&frame=2&anchor=17:18:21.688'>
here</a> and <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=111&frame=2&anchor=17:27:57.463'>
here</a>.
<!-- Date: Mon Feb 7 12:33:53 2005 -->
<pre>turned on cathodes on SE 16/17. One trip at 1-2k. It then turned on.</pre>
<!-- Author: Phil -->
<!-- Comment: Mon Feb  7 12;34;27 comment by...ps -->
<pre>tripped while I was making the entry</pre>

<!-- Comment: Mon Feb  7 12;40;01 comment by...ps -->
<pre>tripped a couple of times after 15-30 sec. @ 3kA. Trying 2.4</pre>

<!-- Comment: Mon Feb  7 12;46;42 comment by...ps -->
<pre>tripped at between 8 and 10 minutes.</pre>


<!-- Date: Mon Feb 7 13:30:03 2005 -->
<pre>there was a hiatus there for a bit. Rob, Dervin and I went in and configured some fans to blow more air into the dead air region on the east side.</pre>
<!-- Author: Phil -->


<!-- Date: Mon Feb 7 13:30:24 2005 -->
<pre>One trip, then got to 2800.</pre>
<!-- Author: Phil -->
<!-- Comment: Mon Feb  7 13;31;05 comment by...ps -->
<pre>again it died while i was making an entry.</pre>

<!-- Comment: Mon Feb  7 13;32;14 comment by...ps -->
<pre>2400.</pre>

<!-- Comment: Mon Feb  7 13;41;05 comment by...ps -->
<pre>lost it at 10 min</pre>


<!-- Date: Mon Feb 7 15:25:09 2005 -->
<pre>it's held 1900 for about an hour (1415-1515h). I'm leaving it on. It has been tripping reliably at 2800 all day. (The last try was about an hour ago.) At some point we should try 2800 to see if the extra air helped. And which way do I want that to go...</pre>
<!-- Author: Phil  -->
<!-- Comment: Tue Feb  8 07;58;00 comment by...Dale -->
It stayed on until 7:55 (the next day).  I slowly took it to nominal HV, it tripped (for several tries) around 2.4-2.6 kV.  I turned it to 2.2 kV - it's been holding for so far.
<!-- Comment: Tue Feb  8 08;30;39 comment by...Dale -->
It held.  Trying it at 2.8 kV - it got up there safely.  Protons being loaded.  We might give it a fair shot at the start of the store(this assumes that it holds voltage until all the anti-p are loaded).  If we do then I have to be there to take the thing out if and when it trips.  
<!-- Comment: Tue Feb  8 08;38;58 comment by...Dale -->
Inched it up to 2.9 kV.  pbars about to be loaded.  I want to turn off the wedge before scraping begins (and perhaps hook it back up to the CAEN).
<!-- Date: Mon Feb 7 15:30:03 2005 -->
Note that air flow to the east side miniskirt was <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&action=view&page=113&frame=2&anchor=13:28:45.683'> 
reconfigured</a> (supplemented) this afternoon. Also note that NE 20 which is the wedge we had to squeeze by to get to the middle of the miniskirt <a href= 
'http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&action=view&page=113&frame=2&anchor=13:51:38.149'> 
tripped twice</a> when voltages were raised after the access. It was on from ~1348-1520h thereafter.
<!-- Author: Phil -->





<!-- Date: Tue Feb 8 10:35:42 2005 -->
Put  
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=116&frame=2&anchor=09:59:11.889'> 
instructions</a> in conserning CMX for this store.  Basically the SE looks fine (for now) although there was 1 single trip (recovered fine) on the NE.  The obvious diffrence is the colder (~dryer?) air outside.
<!-- Author: Dale [2.0] -->



<!-- Date: Wed Feb 9 12:25:45 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-400&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-401  width='300' align='top'></a></td><td>complicated circuits analysis for dale</td></tr></table>
<!-- Author: phil -->


<!-- Date: Wed Feb 9 15:09:20 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=119&frame=2&anchor=09:52:40.389'> 
We are going to have an access</a> for a couple of days.<p> 
The basic plan is to open up and look at the east side CMX.  Look for the "obvious" stuff then segment so that we can re-configure for easy switching (in and out) of 5 degree segments.  </p>
<!-- Author: Dale and Phil -->
<!-- Comment: Wed Feb  9 15;54;46 comment by...Dale [1.0] -->
We have everything ready for segmenting SE wedge '17' into three 5 degree sections (2 SHV to jumper connectors + 2 "medium length" SHV cables).  We should have enough spares to do the same thing to NE '18' but we'll take one thing at a time.  I wish we had our power strip from way back - may take the one in that powers the lamp and (useless) PC in the accembly hall.


<!-- Date: Thu Feb 10 12:25:58 2005 -->
<h2>Access is underway</h2> 
We have segmented SE wedge '17' into three 5 degree sections.  We have a lot of cleaning up/puting in cables etc. but our setup works.  We will likely move on to the NE wedge '18' next. 
<p> 
We by-passed a chamber (details to come) on wedge 17 - stack 0 - wire 4 ("top" channel of board 17-01).  Possible that we might be able to recover this one in the future - like we said more details to follow. 
</p>
<!-- Author: Phil and Dale -->
<!-- Comment: Thu Feb 10 15;43;32 comment by...Phil and Dale -->
More details:
<p>
First we note that several of the preamp boards at various "interfaces" showed crossion due to the Jan 4th flood.  In paritucular where the LV plugs in (cable + connection) and (as seen on preamp boards from before) there is corrosion where the radekas plugs in at.  This agrress completely with our theory that the water tripped off the LV.  At some point a re-investigation will be need to check that all the SE channels that should be on are (basically did any of the preamp boards or radekas buy the farm).
</p><p>
The chamber that we ended up by passing at 17-0-4 was by passed due to the trips on the drift.  In parituclar these were massive trips creating a large spark and sound.  The problem (where the spark comes from) is just to the right and below the anode pin.  This is where there is a nice application of corna dope.  Phil's explaination might be better but it looks like part of the chamber face plate is "pealed up".  In any case, it looks like this area was looked at before.  Certainly as it is now it is no good.  It would trip on a < ~5 minute time scale and was worse (more frequent) as we searched for it.  Two quick thoughts, it is possible that we could recover this chamber by re-applying more corna dope to the suspect area.  The second thought is that the water could have had an effect but it's not clear that something applied ~4 years ago could be removed so easily.  We do note that the problem area is right at the boundry where the corna dope was applied.</p>
<p>
Currently the SE wedge "17" is segmented in 5 degrees sections so that a quick control access is enought to try to get 10 out of the 15 degrees to work by simple cable rearraging.  It has not tripped when we brought the system up.  We plan to leave it on and react to what happens.
</p>
The last thing we did was to work on the NE miniskirt.  We are partially done segmenting this section.  We will finish up our work now (and put in the info on the by-pass chamber for this wedge).


<!-- Comment: Sun Feb 13 00;33;48 comment by...Dale [9.] -->
<pre>By passed chamber is described by Phil below.  Wedge 19 - stack 0 - wire 0.</pre>
<!-- Date: Thu Feb 10 21:55:33 2005 -->
<pre>When I left, the lowest 15 deg. of NE was also segmented and holding voltage. I tried to convince Dale that we hadn't completely checked the anode/grid had the same segmentation. I'm not sure I succeeded. I agree that it's extremely likely.</pre>
<!-- Author: Phil -->
<!-- Comment: Fri Feb 11 09;12;35 comment by...Dale and Phil -->
We tested each 5 degree section for wedges "17" and "18".  All 6 hold fine with the expected current draw.
<!-- Date: Thu Feb 10 22:23:16 2005 -->
a few comments: 
<ul> 
<li>water damage</li> 
<ul> 
<li>On the SE lowest wedge, there is discoloration on every LV cable and connector. It looks like the "salts" left when water evaporates. It is clear to me that water got on the preamps.</li> 
<li>The 2nd thing that occurred to me was that the deposition looked "galvanic". I'm not sure I'm using the right word, but it's too late to bother looking it up. By galvanic, I mean an electrochemical process:  water + voltage in other words leading to something that is not simply water residue.</li> 
<li>There are deposits around the pins of the radeka socket on many preamps that look much the same. This is the other place water and LV could have come in contact.</li> 
<li>That said, George Ott pointed out similar corrosion around the radeka sockets a long time ago.</li> 
<li>There are several points. 
<ol> 
<li>As the LV probably tripped off on SE at the instant it got wet, why the reaction? Water and LV were never there at the same time.</li> 
<li>Are the deposits on the LV connector and the radeka sockets from the same source, especially given the earlier observation?</li> 
<li>Given 1, is is possible it's getting worse?</li> 
<li>Has anything actually failed from this stuff?</li> 
<li>There was one(?) LV connector/cable on NE lowest phi with similar discoloration, but that's all I saw of a similar nature.</li> 
</ol> 
</ul> 
<li>2nd diabled channel is wedge 19, stack 0, layer 0. The grid board is shorting to the ground tab coming off the chamber. This is a massive short. The grid board itself is damaged. Parts of it seem to be blown away. There is a large discolored patch. It sort of looks like it starts on the lower trace on the grid board like  where the 1st diabled channel was failing.</li> 
<li>1st disabled channel is wedge 17, stack 0, layer 4. There is a glop of corona dope on the lower (as viewed) or large R edge of the board more or less on the lowest trace on the grid board. On the right side of the glop, one sees what looks like exposed trace. That's what seems to be sparking. It's possible a piece of the glop flaked off. It's harder to blame this on the water, but not impossible. It's possible more of the insulation of the trace failed. That's not as hard, but doesn't seem as likely. More glopping might fix it.</li> 
<li>Another failure in the making? The grid resister on wedge 17, stack 0, layer 7 was bent down. We bent it back up, but it is obviously damaged. The pins are coming out of the resister package.</li> 
</ul> 
  
<!-- Author: Phil -->


<!-- Date: Thu Feb 10 22:27:34 2005 -->
The bottom line at the end of the day: 
<ul> 
<li>We disabled 1 cathode on each of the two wedges. Were these the ones that were causing all the problems of the last several weeks?</li> 
<li>Given that, how do we want to configure the HV channels down there tomorrow?</li> 
</li> 
</ul>
<!-- Author: Phil -->



<!-- Date: Fri Feb 11 09:46:46 2005 -->
Current state of affairs: 
<p>Given the above 
<a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=21:55:33'>check</a> on each section we went ahead and reconfigured things such that the out outer miniskirts are 20 degrees while the inner ones are 10 degrees.  Here is a table:</p> 
<table 
style="background-color: rgb(0, 0, 0); width: 700px; text-align: left; margin-left: auto; margin-right: auto;" 
border="1" cellspacing="2" cellpadding="2"> 
<tbody> 
<tr> 
<td 
style="vertical-align: top; background-color: rgb(255, 255, 255);">iFIX 
wedge<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">15<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">16-17<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">18-19<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">20<br> 
</td> 
</tr> 
<tr> 
<td 
style="vertical-align: top; background-color: rgb(255, 255, 255);">physical 
wedge:stack<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">15:5 
- 17:0<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">17:1 
- 17:4<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">18:1 
- 18:4<br> 
</td> 
<td 
style="vertical-align: top; width: 18%; text-align: center; background-color: rgb(255, 255, 255);">18:5 
- 20:0<br> 
</td> 
</tr> 
</tbody> 
</table>
<!-- Author: Phil and Dale -->


<!-- Date: Fri Feb 11 15:04:18 2005 -->
The east toroid is closed up with various fans blowing all over.  There have been no trips so far but the true test will be once we go out of supervised access and "normal" HVAC conditions return.   
<br><br> 
See today's work  
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=125&frame=2&anchor=09:10:25.124'> 
summary</a>.
<!-- Author: Phil and Dale -->



<!-- Date: Sun Feb 13 23:17:09 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-402&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-403  width='300' align='top'></a></td><td>It rains all day (and all night) - humidity is going up.  This will definately test things on the (east) miniskirts.  No HV problems in the first post shutdown store and so-far-so-good in this one.</td></tr></table>
<!-- Author: Dale [4] -->
<!-- Comment: Mon Feb 14 00;15;42 comment by...Dale -->
So cruel...SW wedge "16" (iFIX 15) <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=133&frame=2&anchor=00:06:36.664'>trips on the drift</a>.  This guy was fine.  I had the ace take it out.  The HVAC conditions aren't any better (in fact they are worse).

<!-- Comment: Mon Feb 14 00;33;24 comment by...Dale [.5] -->
My<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=133&frame=2&anchor=00:11:57.480'>
instructions</a> and other random thoughts.  Boy I should did jinks myself - <i>what was I thinking...</i>

<!-- Comment: Mon Feb 14 07;53;33 comment by...Dale [.25] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=133&frame=2&anchor=03:09:57.150'>1st attempt</a> didn't work, but <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=133&frame=2&anchor=05:48:43.879'>
2nd attempt</a> did.  The wedge is back up.<br><br>The resent history of this guy is that he tripped on the anode: 
<pre>
Wed Feb 09 00:41:34 2005  CMX  H TRP  Crate # 5  Channel # 37
</pre>

<!-- Comment: Mon Feb 14 07;55;02 comment by...Dale -->
The trips today appear to be always drift/cathode triping off the anode or just the drift it self in the case of some rampups.

<!-- Comment: Mon Feb 14 08;42;41 comment by...Dale [.25] -->
See a much better looking <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=133&frame=2&anchor=08:01:46.278'>
iFIX HVAC "history" plot</a>.  It continues to rain and the weather is rather mild for mid-February.


<!-- Date: Mon Feb 14 19:41:51 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=135&frame=2&anchor=16:33:39.861'> 
It goes bad again</a>. 
<p> 
I got the wedge [west "16" - ifix 15] overvoltaged by 100 V on both the drift and anode (for +5 min).  No trips.  Up to 3.0 kV on on the drift; it draws about .42 mA and is steady like a rock.  Going to give it some more time. 
</p> 
Looked at the HVAC iFIX "history" plot.  The humditiy and dew point are up and level.  They are about at the level where the problems went away (i.e. when the ace was able to re-incorperate the wedge durning her shift).  That's a bit close for me.  I rather these things be solid for the rest of the store or not at all.
<!-- Author: Dale -->
<!-- Comment: Mon Feb 14 20;08;10 comment by...Dale [.75] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=135&frame=2&anchor=20:07:03.388'>
Back in without any problems</a>.


<!-- Comment: Tue Feb 15 11;17;34 comment by...Dale -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=137&frame=2&anchor=10:20:13.921'>Bad again</a>.  I was thinking that the temp. was warming up and the rain continue to drizzle and worried about the HVAC.  Things did not look any better or worse when CMX west wedge 15 tripped (it had remained "fine" throught out the owl shift following my recovery above).<p>
I had time so I took it but the wedge looked like it would still trip on a few minutes time scale.  We ended up turning it off.  The store should end around 1200 (lum < ~10 E30) so no big loss.    Might be possible to move a fan into better position durning a quick access.  With enough time we could connect the wedge ("17") that isn't tripping to the closest 5 degrees that is ("16").  
</p>
Perhaps more and better thoughts later...


<!-- Date: Tue Feb 15 13:43:51 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-404&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-405  width='300' align='top'></a></td><td>Relative humidity / dew point conditions over the last 2 days.  I consider the "treshold" for when the SW should hold HV with little or no trips to be ~35% [or dew point 45 degrees] based off of its recent performace. 
<br><br> 
We've summited an access request.  Short term - get a fan on the lower phi of the SE miniskirt.  Long term - segment into 5 degrees then divide and conqure.</td></tr></table>
<!-- Author: Dale [1.] -->
<!-- Comment: Fri Feb 18 16;26;02 comment by...Dale -->
<pre>conqure --&gt; conquer</pre>
<!-- Date: Wed Feb 16 16:06:39 2005 -->
Another HVAC related problem... 
<p> 
Losses go up due to A4Q3 movement as described <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=140&frame=2&anchor=13:34:28.809'> 
here</a>.  CMX trips follow on the SW 9-10 + 11-12. 
</p> 
Here's a <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=140&frame=2&anchor=15:22:24.912'> 
plot</a>.  The basic explaination is given in the <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=140&frame=2&anchor=15:57:51.095'> 
shift summary</a>.
<!-- Author: Dale -->
<!-- Comment: Wed Feb 16 16;40;08 comment by...Dale [1.5] -->
Some thoughts:
<br><br>
Basically there was work done on the chillers or some related HVAC system that just outside the b0 building.  The bottom line is that this type of work should never even be considered while we are running - the AC maintance work last spring is <u>still</u> causeing us problems.  Better yet, if one of the quadroples got to far out of whack (just west of B0 no less) silicon has a good chance to get much harder than our chambers.


<!-- Comment: Thu Feb 17 13;05;48 comment by...Dale [.25] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=142&frame=2&anchor=05:22:59.681'>
Paged last night</a>.  CMX trips in the same area caused by losses.  
<!-- Date: Thu Feb 17 09:05:10 2005 -->
<pre>Made some quick work of the CSX mean-timers.  I think only 1 of the 8 B0RAT# works.  The 333 scalar says as much - either dumb error on my part or more likely this scalar is bad.  Either way beam is about to be aborted and there's better uses of my time.   
 
I'll post a table and debug if I have time and if there is a strong motivation to use these.  Basically I have it setup to look at counters on the SW CMX arch.</pre>
<!-- Author: Dale -->



<!-- Date: Fri Feb 18 14:53:36 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=146&frame=2&anchor=10:03:19.087'>Trips in SW 9-12</a>.  Same problem as <a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=16:06:39'>above</a> - replacing a chiller part durning data taking.  Marcel put a  plot in with the A4Q3 information [P=pitch?, R=rotation?] as shown <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=146&frame=2&anchor=10:06:34.403'> 
here</a>.  Talked with SungWon - we are in agreement that this crazy pratice needs to stop now.  Not only does CMX take it on the chin but solnoid warms up and quads move causing losses at here and at D0.  I hate to think what would happen if the quad moved another 10 or 15 micro-rad and started to spray SVX and COT.
<!-- Author: Dale [1.25] -->



<!-- Date: Sun Feb 20 16:51:34 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-406&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-407  width='300' align='top'></a></td><td><a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=151&frame=2&anchor=01:00:20.392'> 
Trips on SW wedge 15</a> at the start of the previous store - occured somewhere around the first big peak.  As I was talking to the ace the store quenched (or aborted).  Certainly there looks like a lot more activity then one would expect.  This requires further investigation.</td></tr></table>
<!-- Author: Dale [1.] -->
<!-- Comment: Sun Feb 20 16;53;02 comment by...Dale -->
Note that there was also a <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=151&frame=2&anchor=00:16:14.786'>trip on CMP</a> at the start - might be related.


<!-- Date: Wed Feb 23 12:52:16 2005 -->
We are going to go in on an access at 13:30.<p>  It will be a short (~1 hour) access and we plan to just reposition fans to the SW.  We aggree that we should take one of the standing fans that's blowing in on the NE and move it the SW (so that it blows into the opening and where west "16" [iFix 15] is).  We may also move one of the big grey fans "on wheels" to point differntly.  No other work is plan.  During a longer access (when things become warmer and there's more humidity in the air) we will search for a "weak link" or, if nothing else, resegment so that we can test each 5 degrees as is done on the bottom SE and NE miniskirts.</p>
<!-- Author: Phil and Dale -->
<!-- Comment: Wed Feb 23 13;35;15 comment by...Phil and Dale [.5] -->
Basically things went as planned above.  See 
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=161&frame=2&anchor=13:33:26.418'>
shift elog entry</a>.


<!-- Date: Wed Feb 23 14:38:13 2005 -->
<pre>I got off my lazy butt and asked Dervin to help make some more SHV to jumper connectors.  I gave him 8 jumpers and said that I had plenty more if they need them.  I also found the connector that we use (Kings 1701-1 or something like that) and gave him a sample connector.  The SHV cable I gave to him was damanged and he thought it could easily be repaired.  In any case they could take it apart if needed to help make the others.  In addition I requested to get some small lenght (~2-3 feet SHV cables).</pre>
<!-- Author: Dale [.5] -->
<!-- Comment: Thu Mar  3 08;50;37 comment by...Dale [.25] -->
<pre>Dervin said that everything was ready yesterday and I promptly picked up the new jumper to SHV connectors.  They look nice and strong but still flexible.  Lots of RTV, etc.  I've yet to test them.  The timeing was good as we had an access yesterday - details in next entry.</pre>


<!-- Date: Thu Mar 3 09:10:16 2005 -->
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=183&frame=2&anchor=20:18:37.039'> 
We made an access</a> to segment wedge west "16" before the weather turns nice and our humidity/HV problems return. 
<p> 
Access went cleanly.  We were able to install both connectors (we actually used the 2 old ones we had - we'll use the ones above as spares if need be) by just opening up the middle 5 degree section.  No obvious sign of water damage as was seen on the east side.  No problems just two things to note: (1) preamp board 16-5 [the "bottom" of stack 2] did not have itself attached to a leg to the chambers [one leg had the screw broke off the other the leg came out of the chamber].  (2) Dale droped a brass nut for the miniskirt covers as we were closing up.  It went down (likely) under the miniskirt via the crack between the skirt and the floor. 
</p> 
HV held with the normal current draw both via the Drogie and the CAEN's via the control room.  We also checked each 5 degrees to make sure they were independent and would work by themselves.  The configuration for the SW wedges is the same.  Thus iFIX wedge 15 is still all of what we call wedge "16" (the frist 15 degrees of the SW miniskirt 5/6 of which is physics wedge 16).  The SHV cable we got in there will stretch to the other drift HV cable if need be - thus it would be realtively easy to reconfigure it 20/10 or as need be during a brief access.
<!-- Author: Phil and Dale [2.25] -->


<!-- Date: Wed Mar 9 13:43:36 2005 -->
Here's a HV wedge to CAEN channel to SHV cable map.  (It's a good thing to have.) 
<br> 
<table 
style="background-color: rgb(0, 0, 0); width: 100%; text-align: left; margin-left: auto; margin-right: auto;" 
border="2" cellspacing="2" cellpadding="2"> 
<tbody> 
<tr> 
<td 
style="text-align: center; vertical-align: middle; width: 10%; background-color: rgb(204, 102, 204);">West 
HV <br> 
Wedge<br> 
</td> 
<td 
style="text-align: center; vertical-align: middle; width: 10%; background-color: rgb(204, 102, 204);">CAEN<br> 
</td> 
<td 
style="text-align: center; vertical-align: middle; width: 10%; background-color: rgb(204, 102, 204);">Drift<br> 
channel<br> 
</td> 
<td 
style="text-align: center; vertical-align: middle; width: 10%; background-color: rgb(204, 102, 204);">Anode<br> 
channel<br> 
</td> 
<td 
style="text-align: center; vertical-align: middle; width: 10%; background-color: rgb(204, 102, 204);">SHV<br> 
channel<br> 
</td> 
<td 
style="background-color: rgb(255, 102, 102); text-align: center; vertical-align: middle; width: 10%;">East 
HV<br> 
Wedge<br> 
</td> 
<td 
style="background-color: rgb(255, 102, 102); text-align: center; vertical-align: middle; width: 10%;">CAEN<br> 
</td> 
<td 
style="background-color: rgb(255, 102, 102); text-align: center; vertical-align: middle; width: 10%;">Drift<br> 
channel<br> 
</td> 
<td 
style="background-color: rgb(255, 102, 102); text-align: center; vertical-align: middle; width: 10%;">Anode<br> 
channel<br> 
</td> 
<td 
style="background-color: rgb(255, 102, 102); text-align: center; vertical-align: middle; width: 10%;">SHV<br> 
channel<br> 
</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">0-1<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">3<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">7<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-4</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">0-1<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">25<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">29<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-4</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">2-3<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">8*<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">12*<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-5</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">2-3<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">26<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">30<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-5</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4-5<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">0*<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4*<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-6</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">18<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">22<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-6</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">6<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">9<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">13<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-6</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">7<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">27<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">31<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-6</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">7-8<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">17<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">21<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-5</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">8-9<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">35<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">39<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-5</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">9-10<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">16<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">20<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-4</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">10-11<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">34<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">38<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-4</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">11-12<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">11<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">15<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-3</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">12-13<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">33<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">37<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-3</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">13-14<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">10<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">14<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-2</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">14<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">32<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">36<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-2</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">15<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">5<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">33<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">37<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-7</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">15<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">6<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">33<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">37<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-7</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">16-17<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">5<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">32<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">36<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">SW-1<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">16-17<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">6<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">32<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">36<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">SE-1<br> 
</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">18-19<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">5<br> 
</td> 
<td 
style="text-align: center; vertical-align: middle; width: 10%; background-color: rgb(255, 204, 102);">34<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">38<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-1<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">18-19<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">6<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">34<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">38<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-1<br> 
</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">20<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">5<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">35<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">39<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-7</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">20<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">6<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">35<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">39<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-7</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">21<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">1<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">5<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-2</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">21<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">19<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">23<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-2</td> 
</tr> 
<tr> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">22-23<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">2<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">6<br> 
</td> 
<td 
style="background-color: rgb(255, 204, 102); text-align: center; vertical-align: middle; width: 10%;">NW-3</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">22-23<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">4<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">24<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">28<br> 
</td> 
<td 
style="background-color: rgb(102, 204, 204); text-align: center; vertical-align: middle; width: 10%;">NE-3</td> 
</tr> 
</tbody> 
</table> 
<br> 
*: I had these channels switched (between anode and drift) in my note book - they <i>have to be switch</i> as shown.<br><br> 
The SHV cables are labled by location (quad), type of voltage (drift, anode), and a channel #.  I think I got this map correct for the arches (I know the miniskirt is correct).  Note that the map is for iFIX HV wedges.  Some wedges in the arches are off by a stack or two from where the physics wedge starts.  Silly skemes like what we did <a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=09:46:46'> 
on the east miniskirt</a> can cause additional crazyness.
<!-- Author: Dale [.5] -->
<!-- Comment: Wed Mar  9 13;49;35 comment by...Dale -->
There's also an iFIX channel # that matches to each wedge which is used in the slow controls/icicile plots.  The map is <a href ='http://www-cdfonline.fnal.gov/ops/cdf_muon/maps/cmxtoifix.pdf'>here as a *.pdf</a>.


<!-- Date: Fri Mar 11 23:14:32 2005 -->
A "wet snow" blizzard hits - SW miniksirts ("16") goes <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=210&frame=2&anchor=22:20:07.475'>ape sh*t</a>.  1st trip on NE "19" but that recovered.<p> 
It tripped on first ramp-up - then held for a few minutes at 5.5/2.9 kV - talked to the shift crew some more - triped when came back - continues to trip at drift = 2.9 kV.  It's holding at 2.4 kV (~5 minutes).  I'm inching it up to 2.6 kV. 
</p> 
The realtive humdity / dew point conditions definately changed and were (at a base line) much higher to start off with.  Dew point got to ~45 degrees (now it looks like it's coming down).  More details to come...  
<!-- Author: Dale -->
<!-- Comment: Fri Mar 11 23;17;33 comment by...Dale -->
Forgot to say the current draw was .44 mA when at drift V = 2.9 kV.  Also trips coming from the drift/cathode.<br><br>
Holding 5.2/2.6 kV.  Drift current at a steady .36 mA.

<!-- Comment: Fri Mar 11 23;26;00 comment by...Dale -->
held - upped everything by 100 V - conditions improving in the hall (the base line for dew point ~39 degrees) - I'll post a plot soon.


<!-- Comment: Fri Mar 11 23;33;44 comment by...Dale -->
continues to hold - added anoter 100 V to the drift (2.8 kV)
<!-- Comment: Fri Mar 11 23;49;29 comment by...Dale -->
continues to hold - upped it to 3.0 kV on the drift (5.4 kV on the anode).
<!-- Comment: Sat Mar 12 00;07;53 comment by...Dale -->
I got it going on the CAEN/iFIX - it is holding.  Ended run to start a new one (with CMX good).
<!-- Comment: Sat Mar 12 01;57;59 comment by...Dale [3.5] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=211&frame=2&anchor=01:44:55.739'>So far so good</a>.  CMX is holding.  I left instructions to "speed-up" things if the wedge goes bad again.  The weather when I left: a bit icy but clear skys - most of the snow gone.  The obvious conclusion is that the wedge controls the weather.  Yep - time for me to sleep.
<!-- Date: Fri Mar 11 23:30:40 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-409&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-410  width='300' align='top'></a></td><td>Dew point / rel. humidity</td></tr></table>
<!-- Author: Dale -->






<!-- Date: Sat Mar 12 10:05:51 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-411&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-412  width='300' align='top'></a></td><td>Things look stable and below my trip threshold of 45 degrees / 35%.  No trips reported.</td></tr></table>
<!-- Author: Dale [.25] -->


<!-- Date: Mon Mar 14 16:22:55 2005 -->
One trip on NE 20 reported <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=218&frame=2&anchor=10:55:25.724'> 
here</a> when I was in the control room explaining about the PC inhbits.  It was humidity related but the humidity leveled out just below my threshold values (dew point > 44 degrees) so no trips on the SW miniskirt.  Note that NE 20 is the 20 degree section (wedge "19" and then some).  If we knew at the ~100% level the trip would have come from wedge "18" (iFIX 18-19) as it was before we segmented it then this would uniquely identify the 3rd 5 degree section as the bad guy.  <br><br> 
I think we are ok for now - he has never tripped on reset and is a good marker that problems with SW "16" (iFIX 15) are coming soon.  I haven't done it yet but I plan to write a short access request to segment the SW miniskirt into 20 and 10 degree sections. 
<!-- Author: Dale [.75] -->


<!-- Date: Wed Mar 16 13:33:15 2005 -->
~90 minute control access to start sometime this evening (~9:00 PM).  <p>All I need is an extra SHV 'T', barrel, and cable (just in case; I shouldn't need them) in addition to a flash light, few cable ties, and wire cutters (for the old cable ties).  Everything should be preped before 4:00 PM.</p><p>The plan is to go in with Oleg.  He needs to work on the west side as do I.  Both of our jobs are quick so we should have no trouble doing our business on the SW side and then moving quickly to the NE miniskirt.</p> 
We will configure the NE back to normal (e.g. 15 degree wedge sections).  If NE iFIX 18-19 starts to trip via humidity change we'll know its the 3rd 5 degree section that's the likely trouble maker.  The SW we will hook iFIX 16-17 to the 4 bottom 5 degrees sections and 20 up to the 1st 2 sections.  The SW miniskirt is thus "reversed" from the way the east side is done.
<!-- Author: Dale [.25] -->
<!-- Comment: Wed Mar 16 20;10;04 comment by...Dale [.25] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=225&frame=2&anchor=16:49:54.180'>Speaking of the devil</a> SW "16" trips.  <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=225&frame=2&anchor=19:04:36.891'>Back in</a>, after humdity flatens out.

<!-- Comment: Wed Mar 16 22;36;22 comment by...Dale -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=225&frame=2&anchor=22:26:04.091'>
Access went as planed</a>.  The new current draws:
<pre>
Location/Wedge      Currents (mA) [+/- .002 mA]
iFIX SW 15 [10]            .262
iFIX SW 16-17 [30]         .495
iFIX NE 18-19 [30]         .359
iFIX NE 20 [10]            .415
</pre>
The only thing worth saying was that I had to arrange the new SW 16-17 anode (going from high phi to low phi counting 5 degree sections) as 1-2-4-3 as the 3rd 5 degree section could not fit a SHV 'T' connection.  As the anode is done in paraelle this doesn't make any physical differnce.


<!-- Comment: Wed Mar 16 22;38;08 comment by...Dale [1.5] -->
...

<!-- Comment: Fri Mar 18 02;09;10 comment by...Dale [.5] -->
<table border=0><td><tr><td bgcolor=#e0e030>It is down to the sweet 16 and I don't mean college basketball teams.<br><br>
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=229&frame=2&anchor=01:39:20.104'>
CMX SW 16-17 trips</a> on humidity change.  We can now configure so that only the 3rd (counting low phi to high phi) 5 degree section is connected to 1 HV channel while the rest connects the rest of the SW miniskirt.  We can run "forever" in that mode (we just turn off that section if need be).  Better yet we have a good shot at finding the problem if we inspect that section durning an access with the right (HVAC) conditions.</td></tr></td></table>


<!-- Comment: Fri Mar 18 03;37;08 comment by...Dale [1.] -->
<pre>&lt;a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=229&frame=2&anchor=03:03:42.183'&gt;back in&lt;/a&gt; </pre>
<!-- Comment: Fri Mar 18 03;38;00 comment by...Dale -->
<i>silly elog</i><br>
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=229&frame=2&anchor=03:03:42.183'>back in</a>
<!-- Date: Fri Mar 18 02:16:03 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-413&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-414  width='300' align='top'></a></td><td>Now the question is "Do I go in or have them turn it back on in the halt state?"</td></tr></table>
<!-- Author: Dale -->
<!-- Comment: Fri Mar 18 02;17;09 comment by...Dale -->
humdity conditions during SW iFIX 16-17 trips.




<!-- Date: Fri Mar 18 08:49:19 2005 -->
Back for more. 
<p>There's going to be a 1 hour access today at 9:00.  I feel just confident enough that we really do have the problem down to 5 degrees.  Thus wedge 16 stackes 3 and 4 will be isolated and put on iFIX SW 15 [ms 10].  The rest of the SW miniskirt will be on iFIX SW 16-17 [ms 30].  The only potential problem is jumping the HV over the suspect 5 degree section.  I seem to think that the drift (cathode) will from wedge "17" will reach to stack 2 of wedge 16 but it might be tight.  The anode would be trivial but I'll need to find an "right angle" SHV connector so that an SHV 'T' can be placed on the 5 degree section that crosses over wedge 17 to 16.  I will almost for sure need my extra SHV cable for the anode of iFIX 15 to jump to the bad 5 degree section.</p>
<!-- Author: Dale -->
<!-- Comment: Fri Mar 18 11;15;57 comment by...Dale [2.] -->
HV was reconfigured as planned.  <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=230&frame=2&anchor=10:09:40.943'>
See entry in shift elog</a>.  I'll add more details later.


<!-- Date: Tue Mar 22 10:14:13 2005 -->
The saga continues... 
<p><a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=242&frame=2&anchor=09:34:30.394'>We caught the suspect section</a>.  This is the first time that we are running without the offending HV channel but marking CMX good. <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=242&frame=2&anchor=09:51:10.275'>See my comments in the elog</a>. 
</p><p> 
I talked with the shift crew for maybe 5 minutes and then tried to turn the "wedge" on.  I ramped the drift up to 3.0 kV without any problems.  The anode is at 1.5 kV.  Note: I'm in uncharted waters here - I don't want to be messing with CMX while the data is good.  Thus, I don't think it is wise to turn the anode all the way on nor to ramp it up mid-run as I done before when CMX was being marked bad.</p> 
HV still holding and drawing .15 mA on the drift.
<!-- Author: Dale [.75] -->
<!-- Comment: Tue Mar 22 11;21;27 comment by...Dale [1.0] -->
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=242&frame=2&anchor=10:57:04.644'>Back in</a>.<br><br>
Found another trigmon caliormeter warring as well...

<!-- Comment: Tue Mar 22 11;23;10 comment by...Dale -->
Almost forgot --- When I looked at the HVAC dewpoint plot it showed   low/flat levels.  I'll look at this again a bit later.

<!-- Comment: Tue Mar 22 13;52;26 comment by...Dale -->
Another epsoide of tripping <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=242&frame=2&anchor=13:13:33.360'>
here</a>.<br><br>
The channel trips on the drift "reliably" at 2.3 kV - I think I just got it to trip on the 2.2 kV scale.  Let's hope that this thing is on its last breath.  More details to follow.


<!-- Comment: Tue Mar 22 14;08;09 comment by...Dale -->
<p>I've confirmed that the HVAC dew point plot is flat and below my threshold of ~45 degrees (it's at the "normal" baseline of 40 degrees).  I have a few theories about this which I will save for later.  I've inched it up to 2.4 kV on the drift and 1.2 kV on the anode (this is about 70 V above what the sense wire "feels" due to the grid HV).  It trips quickly when I raise it above 2.5 kV.</p>
It triped at 2.4 kV now set at 2.3 kV (same anode).


<!-- Comment: Tue Mar 22 16;19;27 comment by...Dale [.25] -->
I got it to hold at 2.4 kV for ~2 hours.  Now I ramp it up to 3.0 kV without tripping it.  If only the precognition in the elog worked I would know if it was worth turning on and putting in the run.  I follow Phil's advise and keep it on the drogie for now.
<!-- Date: Tue Mar 22 14:27:16 2005 -->
<b>The plan</b>: 
<p> 
Keep the cathode/drift HV on near whatever the trip threshold is.  Keep the anode on just enough (~1.5 kV) so we aren't building up any charge in the anode electronics.  I've kapton taped the SHV connectors in the back of the CAEN and turned the HV on for iFIX SW wedge 15 (this will fool the shift crew into believing that the detector is all on).  Run this way until there is a reasonable amount of time when the wedge has be on full (on the cathode) without tripping - then think about adding it into the next run/store. 
</p> 
If we are lucky then this guy is going bad.  We'll find an oppertunity to go in on an access to fix him - find the weak link and put all this nastynest behind us.  A lot of wishful thinking here but I've got to be positive about this.
<!-- Author: Dale [1.0] -->


<!-- Date: Tue Mar 22 15:08:21 2005 -->
<i>Just for the record</i>:<br><br> 
<span style="font-weight: normal;">SW iFIX Wedge 15 was off for all of 
run </span><span style="font-weight: normal;"><a 
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195621">195621</a><br> 
<br> 
Then turned off in the middle of 195623 (via the run summary):</span><br> 
<h3><a 
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/Inhibits?RUN=195623">Trigger 
Inhibit</a> Events during Run <a 
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195623">195623</a></h3> 
<table> 
</table> 
<table style="background-color: rgb(0, 0, 0);" border="1" 
cellpadding="2" cellspacing="2"> 
<tbody> 
<tr> 
<th style="background-color: rgb(255, 255, 51);">InhibitTime</th> 
<th style="background-color: rgb(255, 255, 51);">AtEvent</th> 
<th style="background-color: rgb(255, 255, 51);">RunTime<br> 
<i>seconds from activate</i></th> 
<th style="background-color: rgb(255, 255, 51);">Components</th> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 51);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Tue_Mar_22_2005_Day&amp;anchor=11:44:19.623">2005.03.22 
13:05:08</a></td> 
<td style="background-color: rgb(255, 255, 51);" align="right">889688</td> 
<td style="background-color: rgb(255, 255, 51);" align="right">4852</td> 
<td style="background-color: rgb(255, 255, 51);">CMU01 East Trip<br> 
</td> 
</tr> 
<tr align="center"> 
<td colspan="4" rowspan="1" 
style="vertical-align: top; background-color: rgb(255, 255, 51);">- - -<br> 
</td> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 51);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Tue_Mar_22_2005_Day&amp;anchor=11:44:19.623">2005.03.22 
13:11:06</a></td> 
<td style="background-color: rgb(255, 255, 51);" align="right">889688</td> 
<td style="background-color: rgb(255, 255, 51);" align="right">5209</td> 
<td style="background-color: rgb(255, 255, 51);">CMX HV:iFix</td> 
</tr> 
<tr> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 51);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Tue_Mar_22_2005_Day&amp;anchor=11:44:19.623">2005.03.22 
13:11:43</a></td> 
<td style="background-color: rgb(255, 255, 51);" align="right">889688</td> 
<td style="background-color: rgb(255, 255, 51);" align="right">5230</td> 
<td style="background-color: rgb(255, 255, 51);">null [<span 
style="color: rgb(255, 0, 0);">CMX iFIX SW 15 is 
off</span>]<br> 
</td> 
</tr> 
</tbody> 
</table> 
<br> 
Currently the wedge has been off since the time noted in 195623 as will 
any other wedges<br> 
unless noted otherwise in a comment below. The anode will never 
be turned to full voltage<br> 
to keep the purity of the CMX in its "dual" configuration.<br>
<!-- Author: Dale [.25] -->



<!-- Date: Wed Mar 23 07:33:02 2005 -->
Via <a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=10:14:13'> 
above</a>, 
<br><br> 
West Wedge 16, stacks 3-4 held at 3.0 kV for the whole night.  The store will end around 9:00 today and I will likely hook back up the SHV cables to the CAEN (SW iFIX 15 [ms 10]).
<!-- Author: Dale [.25] -->
<!-- Comment: Wed Mar 23 07;54;35 comment by...Dale [.25] -->
I reduced the HV by 200 V so that the anode was below 1.5 kV (it needed just a bit over 1.6 kV with the grid at 3.0 kV).  Additional notes: no CMX related problems (although I did get paged for a CSX gamma problem) and the rel. humdity and dew point have been realtively flat and at normal values.  I also point out that I checked the ymon plot - indeed stacks 3+4 (YMon 24-31) are dead but the effect on the total occupancy in wedge 16 looks good (about equal with 18) and DQMon doesn't complain either.

<!-- Comment: Wed Mar 23 10;33;20 comment by...Dale [.25] -->
I went ahead in re-connected the SHV cables which connect to the 5 degree section of wedge 16.  I ramped it up to full voltage durning quiet time and it held drawing the normal currents, etc.  This was done before shot-setup for store 4057 began.


<!-- Date: Thu Mar 24 21:44:49 2005 -->
Many details to come... Here's what I want to post now: 
<br><br> 
The 5 degree section is in the same state as before - will definately not make it past 2.5 kV and will trip somewhat quickly even at 2.3 kV.  Bad news is that it does seem to get better rather quickly after it's been steady at a lower voltage.  Right now the drift is at 2.4 kV with the anode 1.2 kV to keep it company.  It is out for the store - more details to follow.
<!-- Author: Dale [.75] -->
<!-- Comment: Fri Mar 25 07;49;11 comment by...Dale -->
It held at 2.4 kV - now it is stable - holding at 3.1 kV and 1.3 kV on the anode.  I'll likely lower the voltages a bit (over voltaging doesn't seem "help").

<!-- Comment: Fri Mar 25 07;59;25 comment by...Dale -->
I lowered the droggie to 3.0 and 1.25 kV.  I'm tempted to put it in but we are running well right now and I don't gain that much if I put it in.  Still, if there is some end of store testing I might include it and "sneak" it back in.

<!-- Comment: Fri Mar 25 09;05;09 comment by...Dale [2.25] -->
As promised:<br>
<br>
We made it through store 4057 without any trips.<br>
<br>
During store 4058 SW iFIX channel for wedge 15 started to trip as noted
<a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=16:30:28.856">here</a>.
I happened to be in the control room a full 30 seconds before it
happened as I wanted to look at the IMU HAT trigmon plot as noted <a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=16:34:46.203">here</a>.
Another CMX problem although this time it was the
TDC (<a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=16:54:48.986">b0cmx01
- the east side TDCs</a>). We got Mitch over to come and
eventually the problem was <a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=18:09:07.238">fixed</a>.
In the mean time I tested the 5 degree section - it held HV fine. I put
it back in with some instructions as noted <a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=17:14:23.931">here</a>.
<br>
<br>
It went bad <a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=20:52:21.177">again</a>.
I came in and this is where I made the original comments to this
entry. CMX must have knew I was coming in again as <a
href="/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=249&amp;frame=2&amp;anchor=21:38:58.072">NE
iFIX ms 10 (east 20) tripped</a>. It ended up doing it twice but
in each case it recovered. The humidity did, for this wedge, was
increasing when the trips started. It has been noted <a
href="/cgi/elog/det-system-elog.pl?nb=cmx&amp;action=view&amp;page=8&amp;frame=2&amp;anchor=23:14:32">before</a>
that this
wedge (aka east "19") behaves this way - feeling the effect of the
humidity first but recovering quickly. I think this brings the
total number of trips on this wedge up to 4 since we fix the east
miniskirts in the long access awhile ago. Back to the bad 5
degree section - if we can get lucky enough that the wedge stays in its
bad state (trips reliably above 2.4 kV on the drift) we have an
excellent chance of fixing/finding the problem. I think we have
some options and letting things play out a bit longer is likely a good
idea.<br>
<br>
<span style="text-decoration: underline;">More Details</span>:<br>
<br>
Ok in run 195677 (store 4057) and through run 195703 (store
4058). <br>
<br>
<a
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/Inhibits?RUN=195704">Trigger
Inhibit</a> Events during Run <a
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195704">195704</a>
<table style="background-color: rgb(0, 0, 0);" border="1"
cellpadding="4" cellspacing="2">
<tbody>
<tr>
<th style="background-color: rgb(255, 255, 51);">InhibitTime</th>
<th style="background-color: rgb(255, 255, 51);">AtEvent</th>
<th style="background-color: rgb(255, 255, 51);">RunTime<br>
<i>seconds from activate</i></th>
<th style="background-color: rgb(255, 255, 51);">Components</th>
</tr>
<tr>
<td style="background-color: rgb(255, 255, 51);"><a
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Thu_Mar_24_2005_Eve&amp;anchor=">2005.03.24
16:23:02</a></td>
<td style="background-color: rgb(255, 255, 51);" align="right">686784</td>
<td style="background-color: rgb(255, 255, 51);" align="right">3036</td>
<td style="background-color: rgb(255, 255, 51);">CMU01 East Trip</td>
</tr>
<tr align="center">
<td colspan="4" rowspan="1"
style="vertical-align: top; background-color: rgb(255, 255, 51);">- - -<br>
</td>
</tr>
<tr>
<td style="background-color: rgb(255, 255, 51);"><a
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Thu_Mar_24_2005_Eve&amp;anchor=">2005.03.24
16:24:40</a></td>
<td style="background-color: rgb(255, 255, 51);" align="right">686784</td>
<td style="background-color: rgb(255, 255, 51);" align="right">3132</td>
<td style="background-color: rgb(255, 255, 51);">CMX HV:iFix</td>
</tr>
<tr>
</tr>
<tr>
<td style="background-color: rgb(255, 255, 51);"><a
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Thu_Mar_24_2005_Eve&amp;anchor=">2005.03.24
16:25:17</a></td>
<td style="background-color: rgb(255, 255, 51);" align="right">686784</td>
<td style="background-color: rgb(255, 255, 51);" align="right">3171</td>
<td style="background-color: rgb(255, 255, 51);">null [<span
style="color: rgb(255, 0, 0);">CMX iFIX SW 15 is
off</span>]</td>
</tr>
</tbody>
</table>
<br>
The 5 degree section is turned back on starting before Run <a
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195706">195706</a>.
<br>
<br>
<a
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/Inhibits?RUN=195718">Trigger
Inhibit</a> Events during Run <a
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195718">195718</a><br>
<table style="background-color: rgb(0, 0, 0);" border="1"
cellpadding="4" cellspacing="2">
<tbody>
<tr>
<th style="background-color: rgb(255, 255, 51);">InhibitTime</th>
<th style="background-color: rgb(255, 255, 51);">AtEvent</th>
<th style="background-color: rgb(255, 255, 51);">RunTime<br>
<i>seconds from activate</i></th>
<th style="background-color: rgb(255, 255, 51);">Components</th>
</tr>
<tr>
<td style="background-color: rgb(255, 255, 51);"><a
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Thu_Mar_24_2005_Eve&amp;anchor=20:30:05.089">2005.03.24
20:46:07</a></td>
<td style="background-color: rgb(255, 255, 51);" align="right">181042</td>
<td style="background-color: rgb(255, 255, 51);" align="right">995</td>
<td style="background-color: rgb(255, 255, 51);">CMU01 East Trip</td>
</tr>
<tr align="center">
<td colspan="4" rowspan="1"
style="vertical-align: top; background-color: rgb(255, 255, 51);">- - -<br>
</td>
</tr>
<tr>
<td style="background-color: rgb(255, 255, 51);"><a
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Thu_Mar_24_2005_Eve&amp;anchor=20:30:05.089">2005.03.24
20:48:35</a></td>
<td style="background-color: rgb(255, 255, 51);" align="right">181357</td>
<td style="background-color: rgb(255, 255, 51);" align="right">1163</td>
<td style="background-color: rgb(255, 255, 51);">CMX HV:iFix</td>
</tr>
<tr>
</tr>
<tr>
<td style="background-color: rgb(255, 255, 51);"><a
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Thu_Mar_24_2005_Eve&amp;anchor=20:30:05.089">2005.03.24
20:49:20</a></td>
<td style="background-color: rgb(255, 255, 51);" align="right">181357</td>
<td style="background-color: rgb(255, 255, 51);" align="right">1208</td>
<td style="background-color: rgb(255, 255, 51);">null [<span
style="color: rgb(255, 0, 0);">CMX iFIX SW 15 is
off</span>]</td>
</tr>
</tbody>
</table>
<br>

<!-- Comment: Sun Mar 27 12;26;02 comment by...Dale [.25] -->
Left out the remaining part of store 4058.  However it has been up and hasn't caused any problems in store 4059 and 4060 (on going).  Here's my <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=251&frame=2&anchor=15:52:36.105'>instructions</a>.

<!-- Comment: Sun Mar 27 12;36;26 comment by...Dale [.25] -->
There was a rare trip, reset-trip, reset-recovery in the <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=256&frame=2&anchor=02:49:52.756'>owl shift</a>.  There's no obvious correlation in the rel. humidity.

<!-- Comment: Sun Mar 27 14;05;02 comment by...Dale [.25] -->
Looks like we didn't make it through 4060 - HV channel taken out <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=257&frame=2&anchor=13:53:30.540'>
here</a>.  It looks like there was a sudden drop off in the humidity and that it might have reacted to the quick "correction".  I'll post the plot below.


<!-- Date: Sun Mar 27 14:06:53 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-415&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-416  width='300' align='top'></a></td><td>rel. humidity on the NE and SE side (the problem is of course on the SW but I'll take what I got).</td></tr></table>
<!-- Author: Dale -->


<!-- Date: Sun Mar 27 17:21:06 2005 -->
The HV channel for iFIX SW wedge 15 is holding HV with 3.0 kV on the drift.  It has been quite awhile since this channel was turned off (see links in entry above the rel humidty image) so I'm not surprized.  The store will likely end at ~19:00 so there won't be much harm in keeping it out.  Conversely if there was a down time I wouldn't mind the shift crew turning it back on.
<!-- Author: Dale -->
<!-- Comment: Tue Mar 29 14;06;10 comment by...Dale [.25] -->
We put it back in for the end of the store.  <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=258&frame=2&anchor=19:05:06.921'>Link</a>.


<!-- Comment: Tue Mar 29 14;07;40 comment by...Dale -->
Before 195759 bad - after 195760 good.


<!-- Date: Tue Mar 29 14:21:24 2005 -->
Store 4062 was good until  
<a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=260&frame=2&anchor=10:47:00.178'> 
here</a> when SW 15 starts to complain.  Later in this run (195788) <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=260&frame=2&anchor=12:10:01.916'> 
as NE 20 craps out</a>. 
<br> 
<h3><small><a 
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/Inhibits?RUN=195788">Trigger 
Inhibit</a> Events during Run <a 
href="http://www-cdfonline.fnal.gov/java/cdfdb/servlet/RunSummary?RUN_NUMBER=195788">195788</a></small></h3> 
<table> 
</table> 
<table style="background-color: rgb(0, 0, 0);" border="1" 
cellpadding="4" cellspacing="2"> 
<tbody> 
<tr> 
<th style="background-color: rgb(255, 255, 0);">InhibitTime</th> 
<th style="background-color: rgb(255, 255, 0);">AtEvent</th> 
<th style="background-color: rgb(255, 255, 0);">RunTime<br> 
<i>seconds from activate</i></th> 
<th style="background-color: rgb(255, 255, 0);">Components</th> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 0);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Mon_Mar_28_2005_Day&amp;anchor=10:32:00.936">2005.03.28 
10:42:23</a></td> 
<td style="background-color: rgb(255, 255, 0);" align="right">100716</td> 
<td style="background-color: rgb(255, 255, 0);" align="right">634</td> 
<td style="background-color: rgb(255, 255, 0);">CMU01 East Trip</td> 
</tr> 
<tr align="center"> 
<td colspan="4" rowspan="1" 
style="vertical-align: top; background-color: rgb(255, 255, 0);">- - -</td> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 0);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Mon_Mar_28_2005_Day&amp;anchor=10:32:00.936">2005.03.28 
10:45:48</a></td> 
<td style="background-color: rgb(255, 255, 0);" align="right">100716</td> 
<td style="background-color: rgb(255, 255, 0);" align="right">841</td> 
<td style="background-color: rgb(255, 255, 0);">CMX HV:iFix</td> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 0);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Mon_Mar_28_2005_Day&amp;anchor=10:32:00.936">2005.03.28 
10:46:24</a></td> 
<td style="background-color: rgb(255, 255, 0);" align="right">100716</td> 
<td style="background-color: rgb(255, 255, 0);" align="right">877</td> 
<td style="background-color: rgb(255, 255, 0);">null [<span 
style="color: rgb(255, 0, 0);">CMX iFIX SW 15 is 
off</span>]</td> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 0);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Mon_Mar_28_2005_Day&amp;anchor=10:32:00.936">2005.03.28 
12:01:15</a></td> 
<td style="background-color: rgb(255, 255, 0);" align="right">1344452</td> 
<td style="background-color: rgb(255, 255, 0);" align="right">5368</td> 
<td style="background-color: rgb(255, 255, 0);">CMU00 West Trip</td> 
</tr> 
<tr align="center"> 
<td colspan="4" rowspan="1" 
style="vertical-align: top; background-color: rgb(255, 255, 0);">- - -<br> 
</td> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 0);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Mon_Mar_28_2005_Day&amp;anchor=10:32:00.936">2005.03.28 
12:07:09</a></td> 
<td style="background-color: rgb(255, 255, 0);" align="right">1344957</td> 
<td style="background-color: rgb(255, 255, 0);" align="right">5719</td> 
<td style="background-color: rgb(255, 255, 0);">CMX HV:iFix</td> 
</tr> 
<tr> 
</tr> 
<tr> 
<td style="background-color: rgb(255, 255, 0);"><a 
href="http://www-cdfonline.fnal.gov/cgi/elog/elog.pl?nb=2005&amp;action=view&amp;page=Mon_Mar_28_2005_Day&amp;anchor=10:32:00.936">2005.03.28 
12:07:16</a></td> 
<td style="background-color: rgb(255, 255, 0);" align="right">1344957</td> 
<td style="background-color: rgb(255, 255, 0);" align="right">5728</td> 
<td style="background-color: rgb(255, 255, 0);">null [<span 
style="color: rgb(255, 0, 0);">ended run - NE 20 
off</span>]<br> 
</td> 
</tr> 
<tr> 
</tr> 
</tbody> 
</table>
<!-- Author: Dale [.75] -->
<!-- Comment: Tue Mar 29 14;31;10 comment by...Dale [1.] -->
One or two opertunities to get the NE 20 up (this is east wedge "19") went by - finally I get back from Evanston and put things back together. <a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=261&frame=2&anchor=21:54:03.110'>
See here</a>.  Thus only run 195789 was marked bad for CMX.<p>
I test both wedges before we put them back in.  SW 15 was fine - ramped up 3.0 kV on the first try.  NE iFIX 20 was a bit suspect - We I tried to ramp the HV up it tripped at low voltage on the cathode several times in a row.  Eventally, I proceded to slowly ramp the thing up to see where it was tripping (~400-800 V) but it recovered and was fine all the way up to full voltage.  The way it acted was strange - I got the impression that I was "burning" something off that was causing a short (water?).</p>
The good news is that they came up fine and held for the rest of the store (4062).  Now 4063 is in - so far so good.


<!-- Date: Tue Mar 29 14:43:41 2005 -->
Note on iFIX wedge NE 20 = wedge "19": 
<p> 
Recall that after we "fixed" the NE and SE HV problems by removing a chamber from each (wedges 17 and 18) we re-configured the HV by segmenting the bottom 15 degrees of each east side skirt.  The orgial configuration we left it in was 
<a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=09:46:46'> 
like so</a>. 
</p> 
<p> 
Later (<a href= 
'/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=8&frame=2&anchor=13:33:15'>in an access</a>) I reconfigured the NE back to normal (15+15 not 10+20).  The hope was that if the bottom section (iFIX 18-19) tripped that would get us down to an unique 5 degree section.  The real result was that iFIX 20 trips so the problem is on wedge "19" proper. 
</p> 
Unfortinately, have not segmented the higher-phi part of the NE miniskirt so I can't simply configure it to be 20+10 or 25+5.  There's a slim to none chance (and slim just left town) that we would be able to go into the access planed for tommarow.  What we need is some SHV cables (I've asked Dervin to track down for us) and at least one more SHV 'T' (I gave a whole bunch to George Watt - I'll ask for two back).  I got some from Steve before so that's also an option.
<!-- Author: Dale [1.] -->


<!-- Date: Wed Mar 30 11:17:54 2005 -->
I managed to get my daily "fix" on CMX. 
<br><br> 
Dervin reported some of the fans were off.  <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=266&frame=2&anchor=10:46:26.965'> 
Link</a>.  Also the weather outside+access suggested that humditiy levels might rise (and thus be a good chance to test the threshold when we have an access like conditions, etc.).  Nothing is bitting.  Dew point got to maybe 43-44 degrees.  Just about at the expected theshold.  See my ramblings <a href= 
'/cgi/elog/elog.pl?nb=2005&action=view&page=266&frame=2&anchor=11:11:43.447'> 
here</a>. 
<br><br> 
Note that we were offered to go in but we would only have ~1 hour so that's a no go as moving the toroid takes ~50 round trip.  (And we are not ready...).
<!-- Author: Dale [.75] -->


<!-- Date: Wed Mar 30 12:56:43 2005 -->
<table><tr><td><a href=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-417&button=yes target='xxx' onclick='jswindow()'; align=top><img src=/cgi/elog/det-system-elog.pl?nb=cmx&action=view&page=-417  width='300' align='top'></a></td><td>SW triped when they ramped it up post access / muon calibration.  Dew point got above 50 degrees and rel. humdity peaked near 45%.  I got SW 15 on at 3.0 kV on the drift.  I'll post the details below.</td></tr></table>
<!-- Author: Dale -->
<!-- Comment: Wed Mar 30 13;19;02 comment by...Dale [.45] -->
Note that the BLUE is the dewpoint (degrees F) and the RED is the rel. humdity (%).
<p>
<a href=
'/cgi/elog/elog.pl?nb=2005&action=view&page=266&frame=2&anchor=12:08:43.766'>
Muon cal started at ~12:09</a>.  They would have had the HV at stand-by for the muon chambers by then.  We (Aron and I) figure this is about the same time as the (HVAC) fans in the collision hall were turned back to full/on.  The increase is rather quick - a bit too high and fast for my taste.  SW tripped at 12:23 and then again 2 minutes later.  Everything else came up ok.
</p>
The HV has tripped once after I came back down (it held for a couple of minutes).  Now it looks like it will trip off and on (periods on the order of minutes in which it holds fine) with ~2.8 kV on the drift.  I may try to keep this channel on right up until shot setup.  Also need to see if the levels in the HVAC have come back down yet.

<!-- Comment: Fri Apr  1 13;17;19 comment by...Dale [.25] -->
Store 4065 went in and stayied in for ~1.5 hours before the storm hit and cause the store to be prematurely ended (lighting effected the power grid).  No muon trips.  The humidity levels right now are right where they should be - nice and flat.  Rel. humidty ~ 30% and dewpoint ~ 40 degrees.  The current store 4072 is in and things have been nice and easy going so far.