Need help finding high current drain on my PCB - Page 1
Forum section: electronics / beginners

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 01:26:18 pm
content: Hi, thanks for reading my post. First off I don't know enough to likely pose a decent question but I hope to learn so that I know what to ask an expert. My real goal here is to find an expert that I can engage with to help finally hammer this issue and put it to rest. My developer who built the circuit and firmware has tried to find the issue but it keeps happening. My US contract manufacturer says there are no hardware issues that are obvious.We build a simple PCB running the Murata CMWX1ZZABZ-078 LoRaWAN module which uses the STM32L0 MCU. We followed the reference design. We've now built 2K of these and they mostly work excellent.However an alarmingly high number of devices (1-5%) will exhibit very high current drain and flatten a 14500 lithium primary cell in 1-5 months. We've done extensive power profiling of these on our Keysight N6705C and a good PCB will show ~12µA continuous and a life of 13+years. Devices that are behaving poorly are showing ~450µA drain.I had purposely flashed 500 of these devices and found ~15 or so that dies prematurely. Took a device that lasted 132 days and erased flash with STLink. Reflashed with same firmware and it immediately rejoined the LoRa network as it should. Left it overnight to stabilize the supercap.Put on Keysight and datalogged power profile. Marker 1 is typical, we are sampling 30x in 1 min. Then it should be a log slope down to next sample at marker 2. But 15 min in it goes all wonky. I removed supercap and tested again, seeing 2.77mA resting. Should be 12µA.Hoping I can learn a bit more about debugging this. I can do most things like solder/desolder, power profile, use DMM etc. But anything above that is greek to me. I do have an Agilent scope, Siglent VNA with probes, Keysight N6705C with SMU. And I know what a good device should look like. If you think you can help me, I'm happy to engage.Thx

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 01:38:35 pm
content: This is what a good device looks like for 1 hour. Marker 1 shows our 1 minute sample, then recharging 0.47F supercap until next sample. Looks like average over that hour is 11.7µA.We thought it could be inputs on MCU "not pulled up/down" I forget but I believe we addressed that in firmware.Tried removing power caps 1 at a time but I don't recall that made a difference. We really thought it was firmware but I have no idea.

author: fcb
author_rank: Super Contributor
timestamp: October 19, 2021, 02:10:27 pm
content: Depending on how many channels you have (N6705C has 4 if fully loaded?) can you power the suspects (LAN, MCU, etc..) from seperate channels. Perhaps prove which part is misbehaving?How do the suspect units behave at low temperatures compared to the known good units?Die revisions?Are boot times IDENTICAL?

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 02:26:36 pm
content: Quote from: fcb on October 19, 2021, 02:10:27 pmI have the N6705C loaded but just 1 SMU (N6781A) for battery drain analysis. I guess I'll admit I wouldn't know how to power the various circuits separately. If it involves cutting traces that's probably above my pay grade.When you say die revisions I assume you are talking about the Murata module? They were all from the same 1K reel so they should be identical.I can try boot times and compare. From memory, and I've looked at a lot of my datalogs, it's pretty darn close but I will check and compare this baddie with a good one to see if boot time is same.Thx

author: Siwastaja
author_rank: Super Contributor
timestamp: October 19, 2021, 02:51:00 pm
content: Do binary search.Cut power traces in half, measure the halves separately. You don't need many channels to do that. After you found which side is misbehaving, again cut that half and so on."Half" not necessarily meaning geometrical sides, but half the components, you know.Even easier, just remove some parts completely, this might render the unit nonfunctional of course, but this lets you see if the problem disappears.

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 02:55:36 pm
content: Interesting I was not expecting a difference but there was. I took an identical PCB, removed the supercap as I had done with device I am testing. Erased flash and flashed with same firmware. It joined the LoRa network just fine.Here's the bootup profile from the device behaving poorly. At Marker 2 seeing 469µA, way too high.Takes about 7 secs between packet sends. Not sure if this is ideal way to measure but those plateaus are the TX from LoRa and easy to pick markers from. The small plateau to right of M2 is the RX downlink from LoRa gateway

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 02:59:36 pm
content: Here's the good device that I just flashed. Again no supercap. Shorter bootup. At Marker2, 30µA, and after plateau, it's down to 2µA.

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 03:10:49 pm
content: Quote from: Siwastaja on October 19, 2021, 02:51:00 pmThanks, I think we did that last year when this issue cropped up. Thought we found something related to a cap but could never repeat it. What I did was desolder C13, C14, C15 1 at a time to see if it changed it. It did not. Although I could try again as my memory is fuzzy from back then.Attached schematic, the external temp sensor circuit is not populated on any of these boards as well as diode on bottom right. Capacitor array is what we tried to remove. I believe I even soldered in different caps from another manufacturer to see if that changed it but I should probably repeat this process.My developer had not pulled up/down inputs on MCU where accelerometer was involved. The accelerometer was mounted but not used in firmware. I think there was concern about resonance so he changed that in firmware. Issue remains.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 19, 2021, 03:47:00 pm
content: So it's not a huge number of parts. Just desolder 2-3 ICs and their related passives. Something that is not fundamental to the most basic operation, so that it can still boot up, maybe with a minor firmware change removing some error check and letting it boot and enter low power mode or whatever it needs for you to be able to observe the issue. Accelerometer could be removed, for example.

author: Manul
author_rank: Super Contributor
timestamp: October 19, 2021, 04:13:09 pm
content: I did not read all, but my first impression is that maybe something is floating? I would try like this: make a Vcc/2 divider with two 100k resistors attach a wire to the middle (Vcc/2 voltage) and touch all the pins of every IC. Observe current consumption. It does not take long, and you might find surprises. Perform this on a "good one".

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 04:22:58 pm
content: Quote from: Siwastaja on October 19, 2021, 03:47:00 pmOK I'll ask my developer for some guidance on this. I can removed things with hot air station but getting them back is not so easy for me. I barely can get 0805 caps on lol. I'm a sales guy who happens to need to be involved in this. I wish it weren't the case but that's life.I performed another experiment which we tried last year. Removed all caps C13-15 1 at a time.Removed C15-No change, background current at 2.77mA. After first sample, this dropped to 470µA on its ownRemoved C14-After join dropped to 300µARemoved C13-After join about 2.3µAOK so removing C13 got resting current to where we expect and have observed many times. I then replaced all the caps after much swearing and sniffing fumes. Reused C14 but C13 & C15 were different manufacturer. Back up to 2.36mA.But something happened with that C13 removal that triggered a major drop in current. I don't know what it could have been but the Keysight rarely lies.So then I removed all 3 caps again. Back to 2.36mA

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 04:27:35 pm
content: Quote from: Manul on October 19, 2021, 04:13:09 pmOK I will attempt this as I enjoy learning. Might need to watch a tutorial as my understanding of this concept is very low admittedly.If there was a scale of EE understanding from 0-100, I'm about a 2.

author: Manul
author_rank: Super Contributor
timestamp: October 19, 2021, 05:01:13 pm
content: Quote from: sailah on October 19, 2021, 04:27:35 pmTo say it simple, in digital circuits, every digital line (even unused inputs) should be either at Vcc or Gnd voltage. If voltage is somewhere in the middle - there will be increased current consumption. This is why unused inputs should be connected to Gnd, Vcc (directly or through pulldown/pullup resistors) or configured as outputs (if it's MCU). If left on their own (floating), these input may randomly develop various voltage levels and increase current consumption. Even oscillate in some cases. The test with Vcc/2 divider tries to pull pins to the middle voltage, this should not be succesful if everything is designed/configured properly. But if succesful, you should see an increased consumption and this would mean that it is a misconfigured digital line. I tried to explain as simple as possible. Note that you will see slight increase in consumption because of divider itself, but just ignore it, look for increase which can not be accounted by divider current.

author: fcb
author_rank: Super Contributor
timestamp: October 19, 2021, 06:07:47 pm
content: So there is a difference between a good and bad boot-up.... I'd probably look very closely at the firmware at this stage.If you can drive or toggle a pin when you reach certain places in the boot code, then you should be able to see which routine is not performing correctly (like perhaps not setting the clock tree right, or configuration of a peripheral incorrectly). It maybe that you are relying on a default state of some register somewhere, and that the default state has changed for some reason (die revision, perhaps on SX1276?).Just because the module came from the same batch, doesn't mean that the components that go into the module also came from same batch.

author: fcb
author_rank: Super Contributor
timestamp: October 19, 2021, 06:19:01 pm
content: Long shot. Also, try removing R14.That 10K thermistor (assume NTC) getting grounded by a misconfigured analog pin during sleep might warm up a bit (dropping impedance to say 7.2K...) and perhaps contribute another 456uA of load.If there is enough time between thermistor samples (say a couple of seconds), then you could measure the drop across this in-circuit. Any voltage across R14 when not sampling is not correct.

author: sailah
author_rank: Contributor
timestamp: October 19, 2021, 11:22:53 pm
content: Thanks for everything so far. We didn't get to voltage divider but we did probe around with a scope looking for oscillations.What was weird was that after probing a bunch of stuff (and seeing nothing), my developer said leave it alone during that 1 min of sampling every hour. Sure enough, right after that hour and 30 samples, it drops down to 1.5µA. No matter what I did I couldn't get it back to a fault state.I have a few more to try and will try to replicate the failure. The plan is Friday to do a datalog session and leave the device in the programmer hooked to pogo pins so we can reflash it without touching it. We are going to rebuild firmware from the ground up. First firmware will just be sleep state etc and keep compiling until we can can trigger it.After watching the device fix itself after a 1 hour session, it def feels like a firmware thing. We talked about it at length and he said the unused pins are all pulled up in this latest firmware. There are some pins from the STM32 that we don't have pinned out on the Murata module. He thinks there might be something there so we will investigate what is going on with the unused pins on STM32.Will keep at it, thanks again for reading and replying, feels like I've been alone on this for way too long. If anyone wants to get deeper involved as a consulting thing and feels like this is in their wheelhouse, happy to engage to.

author: Manul
author_rank: Super Contributor
timestamp: October 20, 2021, 12:20:08 am
content: You will likely not see oscillations with a scope in case of floating digital line. The reason is that probe itself will likely be enough load to pull it down and stop it.Again, I'm not saying that floating pins is for sure your problem, but seems worth checking. You will not believe how strange it may act. It might be totaly unpredictable, there are leakage currents inside IC, also PCB level leakage currents (like someones fingerprint, or soldering flux leftovers) it may react to humidity, external EMI and so on. So the line may randomly transition in logic states, hang in the middle causing increased consumption, or randomly wake up MCU if it has interrupt attached. So it would theoretically fit your description, that some boards are affected and some not.Unless your firmware is very complex, it sounds strange that 5% is affected and others not. Typicall, firmware is quite deterministic.I quickly looked your schematic. Noticed, that you do not have pullups on CS lines. Generally, you should have, because while MCU is in reset, they become undefined (this is not likely to be related to your problem). What can be related to your problem is STM32 MISO line, because if no CS line is activated (bus is not used), then it is tri-stated (floating). So check that you have pullup enabled on this one (in firmware).Also, I see you have interrupts attached to accelerometer? Maybe it causes MCU to wake up?

author: Kasper
author_rank: Frequent Contributor
timestamp: October 20, 2021, 04:31:43 am
content: When things happen randomly, it reminds me of when I used to work somewhere that sold wireless doorbells. One of them would ring randomly, not often but sometimes in the middle of the night. After they returned it, I found a pullup resistor on the button net that wasn't soldered properly.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 20, 2021, 01:45:29 pm
content: I have to remind about a case where in a batch of 30 assembled boards, IIRC in two units the accelerometer was not initializing. In all the others, never any issue whatsoever.Must be soldering issue by the (Chinese) CM, right?Wrong. Software issue on my side. Initialization didn't exactly follow the requirements in the datasheet, which made two of the exact same accelerometers from the same batch not initialize, all others just fine.I don't remember what it was exactly but it was not a bus timing violation (in which case it would be obvious thatsomeunits work with marginal timing). It was really weird and totally unexpected.

author: sailah
author_rank: Contributor
timestamp: October 20, 2021, 01:55:14 pm
content: Thx, we are currently working on this in parallel. I have a coder I brought on to help spot any issues and build firmware back from scratch. Doing this right now and monitoring changes in Keysight. Starting at the most basic STM32 firmware and going in stages.Also have original developer of firmware doing the same thing on Friday. I've offered a cash prize to see if someone comes up with the fix first, hopefully a bit of friendly competition.I've mentioned the suggestions here from helpful members, thank you. I will report back on what we find. If it's one of the suggestions made here, I'll be sure to be in touch as you win a prize tooIf you want to be more involved I'm open to proposals...These boards have always been made in US. I use domestic PCBs and PCB-A work is done by my CM in New Hampshire.As always, thank you, this has been helpful to share the struggles as we get our product debugged.

author: Doctorandus_P
author_rank: Super Contributor
timestamp: October 20, 2021, 02:48:15 pm
content: Another cause of weird issues is using an SPI bus without pullup resistor.If the chip select is floating during reset and initialisation of the microcontroller, then external devices may receive faulty data and end up in weird states.

author: Manul
author_rank: Super Contributor
timestamp: October 20, 2021, 03:16:27 pm
content: Just remember, whatever you do (for example, rewriting firmware), try to find proof of what really was a problem. Sometimes it happens, that the issue is fixed, but the doubt remains (that is really annoying). Or even worse - issue randomly apears again after some time, because it was not really fixed.Also, completely rewriting the firmware introduces risk of new bugs. Be careful, you don't want your devices returning back to you again because of some new issue. Remember, 95% of them worked well.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 20, 2021, 03:18:29 pm
content: Yes, I'd say it even more strongly, from experience, that if you seemingly "fix" the issue without completely proving, waterproof, what the cause really was, it's almost 100% guaranteed the bug will come back at the worst possible time.At very least you need to provewherethe bug is, but that is already risky because there is a risk of misidentification of the source when the mechanisms is unknown.

author: fcb
author_rank: Super Contributor
timestamp: October 20, 2021, 03:49:59 pm
content: +1 for what others have said!Absolutely don’t start re-writing code - you need to identify and prove the mechanism that causes the problem.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 20, 2021, 04:37:00 pm
content: Rewrite is OK if the whole thing is a nest of problems like this. If it's otherwise working OK and this is the only major issue - don't even consider rewrite.

author: sailah
author_rank: Contributor
timestamp: October 20, 2021, 04:55:56 pm
content: OK thanks this is helpful. I wasn't thinking we were rewriting the firmware from scratch. I think the plan was to put firmware on the STM32 to just force it to deep sleep and check current. Nothing else. If that works, add the next bit of firmware, get LoRa working, add our application etc. Hopefully along the way the problem arises and we can then figure out what in the code caused this. And then correct original firmware.I really need a "fishing guide" here, someone who can help advise on the plan to debug. I'm the sales person, this is all way above my pay grade. Doing the best i can to get this back on track.It does feel like something simple as we've had devices in the field for years that work perfectly. Our application works awesome. Just when the battery dies it doesn't work so awesomeI know I've mentioned this before but if this is something anyone here is willing to help with, I do have more projects related to this firmware that I will be needing ongoing help with. Happy to have those conversations offline.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 20, 2021, 05:07:28 pm
content: Yes, I agree it's a really good idea to write super minimalistic test firmwares to reproduce HW problems. I do that all the time, and often I have some initial "HW test firmware" which I can always get back into even if not used in "normal" development.

author: alanambrose
author_rank: Frequent Contributor
timestamp: October 20, 2021, 07:58:09 pm
content: >>> But 15 min in it goes all wonky.Well a start would be to see what's going on at the 15 min mark. Can you do / switch on some software logging (hopefully the problem doesn't go away when you do this) and see what happens in that timescale. Is it always 15 mins?Alan

author: fcb
author_rank: Super Contributor
timestamp: October 20, 2021, 08:19:13 pm
content: As you are using lorawan, examine how your receiver is setup.There is a remote possibility that the unit is waiting for some sort of delayed reply packet from the lorawan network before it goes back to sleep.

author: sailah
author_rank: Contributor
timestamp: October 20, 2021, 09:23:50 pm
content: Quote from: alanambrose on October 20, 2021, 07:58:09 pmDon't really know what kind of software I could hook up. No I would say the wonky behavior is hard to predict. Last year we had some devices that went sorta nuts. Was leaving them on Keysight for hours and it would be fine then big spike into the 100s of mA. Caps were getting hot. That's when we thought we had a hardware problem but we could never replicate that.Working with super nice member here probing digital lines at present. Thanks to everyone for their thoughts and help, it is appreciated.

author: sailah
author_rank: Contributor
timestamp: October 20, 2021, 09:27:06 pm
content: Quote from: fcb on October 20, 2021, 08:19:13 pmI think this is a good thought but seems unlikely. We are using stock Murata Semtech ST stack to control LoRaWAN to 1.0.3 specs. I can see the packets come through on our LoRa Network server and see the ACKs on both device using Putty and online.When this draw is present, it's on the entire time as a increase in base current. Over top of that on Keysight I can see the normal activity being performed regarding LoRa activity.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: October 20, 2021, 09:41:02 pm
content: Quote from: sailah on October 19, 2021, 11:22:53 pmDo this before you do ANYTHING else. It is quick and simple to do, and there is a reasonable chance the issue is something floating.Another thing I'd look at is whether there are any hardware registers that are undefined on powerup, and not initialised by your code.Read the device errata - things happening "sometimes" are the sort of issues that make it through to productionPut a resistor in series with the Lora module's supply to see if it is the module or your MCU drawing the current ( but keep in mind the current may be flowing through IO lines).Do you get consistent behaviour between units ? i.e. does one board always draw excess power and another not ?If so, swap the lora modules over and see what changes.Have you checked that it's not a leaky supercap ?Accelerometers typically have register configs to select opemn-drain or push-pull interrupt lines - check this is being set up OKTry powering up a bad board at high and low ambient temps ( hairdryer, freezer spray). This might show anything where there is dependency on some analogue value. e.g. an RC oscillator somewhere that interacts with code execution speed.BTW why are you using Q2 instead of just using the IO pin direct to switch the thermistor ?

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: October 20, 2021, 09:53:19 pm
content: And as others have said, until you can do something that will replicate the issue to make a "good" board behave badly, you can't be sure you've fixed it, or that units in the field won't suddenly develop the issue.May be hard to get through to a big company, but definitely worth talking to Murata about their module - it could be a known issue. I had a similar problem with some 868MHz RF modules a while ago, where they wouldn't always wake up, fortunately it was a small company and they confirmed it was a known bug and reprogrammed the modules with later FW.

author: sailah
author_rank: Contributor
timestamp: October 20, 2021, 10:28:31 pm
content: Quote from: mikeselectricstuff on October 20, 2021, 09:41:02 pmMost of this is above my head. I'm literally working right now with a super generous EEV member doing exactly voltage divider testing as requested. I don't know enough to interpret the results but hopefully he does and is willing to share his conclusions.I will share your other points with my developer when I meet Friday AM.I don't think it's a leaky cap. We routinely remove those for testing with Keysight and leaking current is still there with it removed on bad devices so it's not cap.I can power up boards in high low temps very easily. Will report back. Right now I have a know good board and bad board on bench and we are testing them both. OK member is telling me what to do am I'm following instructions...I don't know the answer to your Q2 questions sorry.We are in direct contact with Murata and they are willing to help. We've had their device FCC certified and generally they have been very helpful. In fact my rep just had me fill out a few pages of questions that they asked.Thank you for your comments, they are much appreciated.Once we get to the finish line I hope to share the results.

author: sailah
author_rank: Contributor
timestamp: October 21, 2021, 01:23:48 am
content: Thanks to an extremely generous forum member who spent a few hours with me via email doing testing, these are the results. They wish to remain anonymous so I'll call them my results but you know what I mean...Hope to take this testing into my Friday meeting and get an aha moment. If not I will bring them to Murata (who has their own line of questions) and see if we can get anywhere. I did ask ST for some help and have a good relationship there as I buy a lot of ST stuff for my boards.1. STM32_SWO on P5 goes nowhere (not important);2. STM32_USART1_RX and STM32_USART1_TX are floating. If UART is not used (and it is clearly not used - not initialized), both of these pins should have pullups enabled;3. STM32_SWCLK and STM32_SWDIO are floating. Generally these should have pullups on the PCB.4. There is a lack of bypass capacitors. Especially for U4 and U5. There should be capacitors very near U4 and U5. Having no capacitors might cause soft or hard failures of IC including data corruption, undefined states, latch up or even complete failure due to exceeding maximum voltage ratings (voltage transients caused by excessive ringing).5. The SPI data bus STM32_SPI_MISO line stays hard pulled (high), while both of slaves (U4 and U5) have CS lines high. Either this is firmware "feature" or some unknown failure. Generally, unused SPI bus should have tri-stated MISO line (with pullup).

author: Manul
author_rank: Super Contributor
timestamp: October 21, 2021, 09:56:56 am
content: It is somewhat strange if SWCLK and SWDIO are floating. I think most stm32 set them internally SWCLK - pulldown, SWDIO - pullup. Maybe it is because of sleep state? Or something misconfigured? Definetly not encouraging to see them float. You should search for firmware misconfiguration or pull them externally and retest. Same for UART, fix that.Bypassing problems is another can of worms. You should probably bodge some caps if they are missing and retest.

author: sailah
author_rank: Contributor
timestamp: October 22, 2021, 04:51:30 pm
content: Spent a number of hours testing this morning with developer.We managed to isolate 5 devices running same firmware, 1 runs perfect and has resting current of 1.6µA. The others have various resting currents of 180µA-4mA. In some cases the current is very stable and high, like 2.3mA. In other cases it will start fairly low (~20µA) and then ramp up to 180µA and continue to show instability.We did find a couple anomalies. Pin #3 MAX9814_MICOUT is the output of the audio amplifier. On a couple devices we were seeing 180-220mV on this pin with picoscope. It did not correlate with high current but it should have been 0.In every case, each device had its supercap removed and all joined the LoRa network without issue.My desk is turning into a war zone hereWill entertain serious cash prize (not sure if this is verboten is so please forgive me) or proposals if anyone wants to remote debug and help me advance closer to solution

author: Kasper
author_rank: Frequent Contributor
timestamp: October 22, 2021, 04:58:28 pm
content: Pretty nice setup for 'just a sales guy'!Can you share pics of the PCB? I'd like to recommend some mods to figure out which part is drawing the extra current but that will be easier if we can see what you've got.

author: sailah
author_rank: Contributor
timestamp: October 22, 2021, 05:13:14 pm
content: Haha, yeah just a sales guy I claim no other skills. Here's one of my baddies. I have the + leg of supercap clipped so it's not part of circuit.Here's a pic of PCB, let me know if you need any other details, I think schematic has already been posted.Thx

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: October 22, 2021, 05:20:19 pm
content: Quote from: sailah on October 22, 2021, 04:51:30 pmThat sounds a lot like a floaty pin. see if it's affected by a mild static field, e.g. wave a charged plastic comb near the PCB and see if it affects the current draw. If it does then it's almost certainly a floating pin

author: Kasper
author_rank: Frequent Contributor
timestamp: October 22, 2021, 05:23:33 pm
content: Quote from: mikeselectricstuff on October 22, 2021, 05:20:19 pmGreat idea.

author: Kasper
author_rank: Frequent Contributor
timestamp: October 22, 2021, 05:36:54 pm
content: Quote from: sailah on October 22, 2021, 05:13:14 pmOf course it's a black PCBGreen is better, it's easier to see the tracks on it and I might recommend cutting power traces but instead, you can remove parts.I would try removing these parts from a bad board and see if it helps. If it does, double check by putting them back on and retest. Note the FW may need update to make it ignore the missing parts.R14U5U2U4If that soldering is beyond you, try a local repair shop, even the little phone repair shops sometimes have people with good skills and equipment.

author: sailah
author_rank: Contributor
timestamp: October 22, 2021, 05:42:26 pm
content: Quote from: mikeselectricstuff on October 22, 2021, 05:20:19 pmThank you I will def try that. I mentioned my VCC/2 testing and he said the pins were not floating. We checked with my Fluke in ammeter mode to ground and looked for an increase in current? But I have a number of people saying they are floating. I will run your test tonight and report back.Quote from: Kasper on October 22, 2021, 05:36:54 pmHaha, yeah but I like black and I picked the color...I can easily remove those. I presume you want this done on faulty board while monitoring current.R14=thermistor resistorU5-Microchip EEPROM to store keysU2-Max mic preamp, needed for firmware but maybe not? Used at end of every hourU4-3 axis accelerometer. Not used in current firmware, so should be shut down.I can unsolder all of this safely I have a rework station and Hakko iron. Not sure I can get the acelerometer back on or even the preamp but I can tryWill report back tonight.Thx

author: Kasper
author_rank: Frequent Contributor
timestamp: October 22, 2021, 05:58:21 pm
content: Quote from: sailah on October 22, 2021, 05:42:26 pmBlack does look cool. Maybe next time go with the after dark style, it's gorgeous and practical, you can see the copper on the outer layers.https://blog.oshpark.com/2019/10/10/introducing-our-after-dark-black-fr-4-service/Yes, use a bad board, remove 1 part at a time and test to see if that 'fixed' it. If testing takes a long time, maybe remove 2 parts at a time or all the parts, whatever works for you.The goal with this is to find where your extra current is going. Once we figure that out then we add pull-ups/downs, improve the FW, etc.For future reference or for an alternative to removing ICs: what is handy for this is to run power for each IC through a resistor. Ideally the PCB is designed with that but you could maybe cut tracks and insert resistors or with big parts like U5, you could lift pin 8 and add a resistor between pin 8 and VCC on the PCB. These resistors can be used to measure current to each IC and they can be removed to cut power to each IC, though you'd want to be careful not to send power to other pins while there is none to Vcc. You can damage ICs doing that.

author: tooki
author_rank: Super Contributor
timestamp: October 22, 2021, 10:03:19 pm
content: Quote from: sailah on October 22, 2021, 04:51:30 pmI wanna see what the roll of stickers say on them! 😂

author: sailah
author_rank: Contributor
timestamp: October 23, 2021, 01:31:19 am
content: Quote from: tooki on October 22, 2021, 10:03:19 pmHaha, from my motorcycle racing days. Free stickers if you figure out the problem !!

author: Psi
author_rank: Super Contributor
timestamp: October 23, 2021, 01:42:33 am
content: Quote from: sailah on October 22, 2021, 04:51:30 pmBe aware that piezo elements can have resistance as low as a few k ohms due to tiny cracks that can form during installation force. Most of the time they are meg ohms/open circuit but not always.I think it's caused by that fine metal power on the ceramic section to enables soldering. It can fall into any micro cracks and start to conduct from top to the metal plate.It's common to put a capacitor in series to prevent this issue causing problems, which you don't have.

author: sailah
author_rank: Contributor
timestamp: October 23, 2021, 02:48:30 am
content: Last test for night, getting frustrated and been working on this all dayTook bad board, and removed:R15-no changeU5-no changeU4-no changeU2-no changeDevice is currently reading 2.38mA. Should be 1.6µA. So other than Murata module, all the ICs are removed. Not sure I'm going to be able to mount them back on successfully.

author: Psi
author_rank: Super Contributor
timestamp: October 23, 2021, 05:52:37 am
content: You're not holding the PCB or probes while testing are you?Skin contact will cause current to flow.

author: sailah
author_rank: Contributor
timestamp: October 23, 2021, 08:34:19 am
content: No I usually have it in a powder coated enclosure sitting on a mat. Sometimes I'll hold the bare board but rare. Probably power profiled 100s of these over the last few years and have many many extensive data logs to know almost immediately when the device is not working well.

author: fcb
author_rank: Super Contributor
timestamp: October 23, 2021, 03:05:29 pm
content: Quote from: sailah on October 23, 2021, 02:48:30 amSo logically, the current drain is either coming from the microcontroller on the murata module or the SX1276 on the murata module? So you pretty much have 4 (2^2) posibilities.One other thing - modules that exhibit high current when asleep - does this fall in a consistent narrow range?

author: Jeroen3
author_rank: Super Contributor
timestamp: October 23, 2021, 03:18:13 pm
content: Could it be ESD damage?

author: ogden
author_rank: Super Contributor
timestamp: October 23, 2021, 03:36:06 pm
content: In the list of suspects I would include ceramic (decoupling) capacitors that could receive mechanical stress during manufacturing or use, like those close to mounting screw. Try to replace caps, measure leakage.

author: Kasper
author_rank: Frequent Contributor
timestamp: October 23, 2021, 04:21:57 pm
content: Quote from: Jeroen3 on October 23, 2021, 03:18:13 pmI am wondering that too.sailah, did you recently change offices? Maybe to one with carpet? Or start wearing sweaters?The people at my work ignored my esd precautions and got away with it until we moved to an office with carpet then 'all of a sudden' boards started dieing.

author: Kasper
author_rank: Frequent Contributor
timestamp: October 23, 2021, 04:31:20 pm
content: Quote from: sailah on October 23, 2021, 02:48:30 amOn the plus side, there are less parts to troubleshoot now. Now keep removing until current goes down. Remove code , caps, transistor.I would start with writing minimal code. Just boot up and put everything to sleep.You might get to a point where all you have on the board is that module and then you can try digging into the module, swapping modules and or improving the FW.

author: sailah
author_rank: Contributor
timestamp: October 23, 2021, 10:44:29 pm
content: Quote from: fcb on October 23, 2021, 03:05:29 pm"we" meaning instructions from my new EEVBlog friend, have been to modify the board with some small caps and ground wires to fix layout issues. With this device we are datalogging current but from all accounts, this bad device has had any layout defects fixed.The answer to your question, if you are asking if the current range is fairly narrow on bad devices, I would say yes. Kinda. However we are seeing some that "climb" from say 160µA to 500µA and then go back down. All with no input. And they should be very low, say 1.6µA for 59 minutes save for watchdog spike.We have a device now that is fairly unstable.But generally bad devices will go to 2.3mA, for example, and stay there. They will join LoRa just fine and have the big pulse during TX as any othe rdevice would. Just background current is very high and consistent.Quote from: Jeroen3 on October 23, 2021, 03:18:13 pmAnything is possible at this point. I assemble these on an ESD mat that is grounded but I can't say I'm super careful with them. I have a slate floor in my office, no rugs or anything. I ground my arms on the mat. I used to wear an ESD wristband but I haven't in awhile.Quote from: ogden on October 23, 2021, 03:36:06 pmCaps seem to be the 1 common link between fixing devices. For whatever reason by soldering/desoldering caps, sometimes a board will fix itself.I had a device behaving poorly last night with high current (2.3mA steady). Had done nothing to it other than let it sit. Tested today, 1.6µA. WTF.Quote from: Kasper on October 23, 2021, 04:21:57 pmNo my office has always been the same, stone floors, no carpet no static shocks ever. We have these made by a good CM in New Hampshire from domestic PCBs. They ship in ESD bags. I assemble into die cast housings and flash them on an aluminum chassis programming jig based on STM32 discovery board with pneumatic pogo pins. Always been that way.One thing of note, since the piezo was brought up, we bond these in place with a special CNA glue. And clamp them in pneumatic fixture with a small air piston holding a rubber foot. It's about the same pressure as a steady thumb would exert but it's fairly flat.Quote from: Kasper on October 23, 2021, 04:31:20 pmWe are working on that now. Hopefully the device we have is a good known state and can debug from there.I'm learning drinking from a fire hose so thank you to everyone, it is sincerely appreciated.

author: jmelson
author_rank: Super Contributor
timestamp: October 23, 2021, 10:59:55 pm
content: Quote from: sailah on October 23, 2021, 10:44:29 pmOHHH! I wonder if it is some contamination / solder flux under parts? Maybe take a misbehaving board and put in an oven at 50 - 70 C for an hour, and see if the problem goes away. The heat will dry out the flux residue. Do you wash the boards after soldering?Maybe you are not getting them completely clean, and solvent is left under parts.Jon

author: sailah
author_rank: Contributor
timestamp: October 24, 2021, 12:50:33 am
content: These are made by my contact manufacturer in the US I assume they do good work but I'm willing to look at anything at this point.

author: Psi
author_rank: Super Contributor
timestamp: October 24, 2021, 06:13:53 am
content: Does this PCB get flexed at all?

author: Siwastaja
author_rank: Super Contributor
timestamp: October 24, 2021, 09:53:58 am
content: Just keep removing parts. You must be quite close, seeing you have removed many suspects already.Guessing is sometimes OK but there are still too many guesses to test out. Just keep narrowing it down, you'll find it that way.

author: fcb
author_rank: Super Contributor
timestamp: October 24, 2021, 10:07:25 am
content: Without sounding like a stuck record, you need to measure the current drawn by the microcontroller and the current drawn by the SX1276 (Lora IC).Find and break a trace for the power to each and insert a 1ohm resistor, measure voltage drop across resistor under ‘fault’ conditions.This will tell you where to look.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 24, 2021, 12:37:58 pm
content: Quote from: fcb on October 24, 2021, 10:07:25 amYes and now this is easier than ever, thanks to many removed parts.

author: sailah
author_rank: Contributor
timestamp: October 24, 2021, 03:28:29 pm
content: Quote from: Psi on October 24, 2021, 06:13:53 amNo, not really. It's mounted securely to a heavy die cast housing made specifically for it. Maybe some of the screws might induce a flex to the board but seems improbable. These devices I have have been sitting in a box not moving along with 100's of other for my customer.Quote from: Siwastaja on October 24, 2021, 09:53:58 amI removed all ICs from that one board. I don't think I could likely put them back on.Here's where a quick test went on that board.Started with just Murata module...3.6mA constant currentRemoved piezo leads...no changeRemoved C15 under power, blew fuse, removed C15 and blown fuse, jumpered fuse pads...1.01mARemoved C14, jumpered fuse pads...550µARemoved C13, jumpered fuse pads...2µAPut on keysight and datalogged it. This is what it looks like. Other than the join not working because no EEPROM for keys, it worked OK. This is what a good device should look like at rest. And I should note that removing this cap array on previous devices has triggered this same result. So for whatever reason, removing C13-C15 is what "fixes" it in many cases. C13 being the bulk 10µF cap makes the most difference.

author: ogden
author_rank: Super Contributor
timestamp: October 24, 2021, 05:03:39 pm
content: Quote from: sailah on October 24, 2021, 03:28:29 pmThis could be event that restarted IC - actual consumer. First thing you shall do - write "blank" firmware that does nothing besides low power sleep function, flash all suspect PCBs and see it fixes problem or not. Sorry if this is already said because TLDR, did not read whole thread.

author: sailah
author_rank: Contributor
timestamp: October 24, 2021, 05:06:36 pm
content: I don't want to prematurely say we found the issue but I think we are fairly sure.After all this messing around with ICs, floating pins etc...I took the board as described above and removed caps, sure enough it went down. But I have seen this before. Put caps back on, still at ideal current.So then we took a device that had never had cap array reflowed or soldered. As in direct from my CM. Cut the traces for the 3 caps, 1 at a time instead of heat.When I cut C13, the current went down to 1.6µA.We then powered up just C13 on the PCB and it was drawing high current.I think we've repeated this a couple times but I want to try it on a fresh device that has never had the cover removed since it died. I think that will validate it.Now the question is, if this is true, how to avoid it? I will certainly talk to my CM but I want it to be constructive. Are there ideal caps to be using in the case?C13 cap that WAS on all these boards ishttps://www.mouser.com/ProductDetail/Taiyo-Yuden/TMK212BBJ106MGHTC13 that I have at my CM for the next round ishttps://www.mouser.com/ProductDetail/Samsung-Electro-Mechanics/CL21B106KAYQNNEAnd I want to say a special thank you to an EEVBlog member, who has chosen to remain anonymous, for literally 100s of emails back and forth and sticking with my stupidity all weekend to track down the issue. As well as everyone else here who offered their thoughts and read through rambles of a person(me) not even qualified to handle a soldering iron.THANK YOU ALL.I will revisit this thread and update if anything important comes along.

author: Kasper
author_rank: Frequent Contributor
timestamp: October 24, 2021, 05:23:37 pm
content: Great to hear the good news! Good job sticking with it!Ceramic caps are fragile and susceptible to problems due to board flex. I am not familiar with those problems but some things to consider re board flex:You have 4 screws and an antenna connector tieing the PCB to the enclosure. That is 5 mounts. Not many people follow this rule but the ideal number of mounts is 3.How flat is the PCB? If amount of copper is not balanced, the PCB can warp. I think that usually happens during fab, not something that creeps up over time.How tight is the antenna connector?What is the tolerance on the hole placement of the antenna connector?Is there more slop between antenna connector and hole than there is tolerance in hole placement.Did you change your antenna tightening procedure?If you draw a line from the antenna connector to the nearest screw, that line goes near the caps in question and it represents an area of possible stress.

author: ogden
author_rank: Super Contributor
timestamp: October 24, 2021, 06:20:40 pm
content: Quote from: sailah on October 22, 2021, 05:13:14 pmI wonder - why you have (expensive) supercap in battery-powered product? [offtopic question, but anyway..]

author: sailah
author_rank: Contributor
timestamp: October 24, 2021, 06:39:49 pm
content: Quote from: Kasper on October 24, 2021, 05:23:37 pmHow flat is the PCB? Not sure, feels pretty flat. Not sure other than putting on surface plate to tell you flatness.How tight is the antenna connector? The SMA is soldered to board. Antenna is put on finger tight but this is done by user. These monitors affected never left my office.What is the tolerance on the hole placement of the antenna connector? Good observation. Tighter than I would like. I'm having the die casting retooled in the US from china right now. I specifically asked for 0.5mm larger hole for this reason.Is there more slop between antenna connector and hole than there is tolerance in hole placement. There is very little clearance. This was my fault as I wanted to be able for this housing to pass IP67 and figured it would be good practice to have close tolerance. Never in my mind imagined this could potentially be an issue.Did you change your antenna tightening procedure? This is done by user to big variable. However all these devices in question (but not all failures in total, we have had devices fail in field) have never had an antenna put on or left my office. They were all installed in housings though.If you draw a line from the antenna connector to the nearest screw, that line goes near the caps in question and it represents an area of possible stress. Yeah starting to see all these things.I don't have any devices that I have programmed outside of the housing so I can't say for sure it's the cause.Wondering if soft caps would be a better idea to get me though the next round until I respin the board. I've had multiple people tell me that my layout has room for improvement as well as a few other upgrades to make the device more reliable. I would like to incorporate all these into next version.Quote from: ogden on October 24, 2021, 06:20:40 pmWe tried a lot of cells with regards to pulse current. In Gen 1 of this product we used a Microchip RN2903 LoRa radio with a CR123 which could handle the pulse.When we switched to Murata module, we opted for a Saft LS14500 due to ATEX (explosive environments) requirements as well as capacity, life. This cell could not handle the pulse of LoRa radio and radio would reset to to voltage sag. Saft told us we need a supercap after we provided datalogs.The 0.47F AVX cap was recommended for a lot of reasons in that it had lowest leakage and was already built for HazLoc. Whether it is overkill or too big/small I can't comment. What I can say is that it works really well in reducing the pulsing of the LoRa TX. In the big picture it's a small overall cost and really helps us get every last bit out of battery with a very low leakage.My goal is not to have the cheapest BOM, especially with some of the clients we have. It's been a very interesting ride in this PCB journey. 5 years ago I didn't even know what a capacitor or IoT was. Like I said, sales guy. Now I know just enough to be dangerous...https://www.mouser.com/ProductDetail/Kyocera-AVX/SCMR14C474PSBA0H

author: ogden
author_rank: Super Contributor
timestamp: October 24, 2021, 06:52:21 pm
content: Quote from: sailah on October 24, 2021, 06:39:49 pmUsually it is about errors in different (X/Y) planes. In your case you shall be checking against antenna socket contact with enclosure.  I see there is no nut on the antenna socket. Meaning PCB is pulled as strong as consumer pleases to tighten antenna.QuoteThat sounds like overkill advice, especially considering battery used. I would suggest to test electrolyte cap in place of supercap, start with 1000uF or so. You can save some money.

author: sailah
author_rank: Contributor
timestamp: October 24, 2021, 07:20:11 pm
content: Running a final test to confirm on an untouched deviceGrabbed a failed monitor, battery was 0.016V.Tested monitor just by powering up. Joined network as expected, current was high but couldn't see precisely because supercap was still on. Removed 1 leg of supercap. Current was 55µA and jumped up to 100µA, very unstable. (Picture 1)Then I cut the trace between C13 & C14, verified isolation. Current drops to 1.5µA as it should be. Very stable, very narrow range. Just the spikes from the watchdog. I've seen this power profile so many times I could draw it from memory. Picture 2Then I replaced C13 with a new cap from Samsung, same 10µF, 25VDC, but X7R. Not sure if this matters or if it was bad soldering. Soldered in a jumper leg across VCC from C13 to C14 to re-establish the trace. Looks exactly as picture 2 does so no need to show it again. Current 1.5µASo then I resoldered AVX supercap, charged it up to avoid blowing fuse.Running a 24 hour datalog to compare current consumption to what we have see in many good devices. Should use approx 5-6J daily. My test will be a little skewed as the cap chargeup takes awhile to stabilize. I'll leave it on logger for 24 hours anyways but plan to run another log tomorrow just to be sure.

author: Manul
author_rank: Super Contributor
timestamp: October 24, 2021, 07:26:14 pm
content: Something was really going wrong, that such high percentage of devices developed mlcc capacitor failure. If that was caused by soldering / mechanical stress, soft / flexible termination capacitors should reduce failure rate by at least 1 or 2 orders of magnitude.Quote from: ogden on October 24, 2021, 06:52:21 pmI think, the supercap was needed because of that diode voltage dropper, made of 3 diodes. At very low (idle) currents, they drop very little, basically "leak" and charge the capacitor very near battery voltage. At higher current, the voltage drop through these diodes immediately becomes much much higher, up to maybe 1V - 1.5V, so that is the reason why small electrolytic was not enough. It is more like design flaw. 3 diodes are used to drop enough voltage at idling 1.6uA, but they are also very high impedance at that point.

author: sailah
author_rank: Contributor
timestamp: October 24, 2021, 08:54:07 pm
content: There are some design considerations that forced decisions based on ATEX Class 1 Div 1 (HazLoc) certifications that we were forced to use for certification reasons. They might not make sense from an electronics perspective but they were needed for protection against spark/thermal rise. I'm only echoing what I had to go through but I don't have any comments on if it could be a better design, of course it could. But here we are.2 hours into datalog and everything looks great. Our device samples 30x last minute of every hour and uses a decent amount of energy, otherwise asleep. Every 8 hours it sends a LoRa Tx. From what I can tell it's all working perfect.I'll update after I speak to CM tomorrow and get their thoughts about capacitor/solder issue.Thx

author: Kasper
author_rank: Frequent Contributor
timestamp: October 24, 2021, 09:28:51 pm
content: For next version, I would look into different ways to attach that antenna. I'm sure lots of people have faced this problem so there are probably lots of good ideas out there for it.For starters, you should be able to get connectors that can be fastened to the enclosure and soldered to the PCB. That is what I assumed you had. I think it'd be ideal to fasten it first then solder it but I'm not familiar with that.What am a little familiar with is fasten connector to enclosure and have coax cable between connector and PCB. That's a bit of a pain though because it means 4 connectors and a cable instead of 1 connector. And you have to connect the cable without twisting it too much or bending it too sharply. And if you damage the cable it might be hard to tell, might just have less range than normal.

author: ogden
author_rank: Super Contributor
timestamp: October 24, 2021, 09:34:47 pm
content: Quote from: Manul on October 24, 2021, 07:26:14 pmOh, right. Low quiescient current LDO shall be used instead in such kind of app.

author: Manul
author_rank: Super Contributor
timestamp: October 24, 2021, 09:40:21 pm
content: Wanted to share a really nice, quite extensive paper focused on mlcc capacitor cracks. Some really interesting data and insights there.https://ntrs.nasa.gov/api/citations/20190001592/downloads/20190001592.pdf

author: Psi
author_rank: Super Contributor
timestamp: October 24, 2021, 10:40:54 pm
content: Yeah, I have also seen this.All is fine until you get a batch of caps that is a bit more susceptible to cracking than all the batches you had used previously. Then the problem appears.It can be a bit misleading too because the short is not always static, it comes and goes based on many factors. Handling, flexing, hot air rework, ambient temp etc.It can also be a part mix up where you spec'ed 25V caps but got 16V version by mistake, and so are overvolting them a bit.

author: sailah
author_rank: Contributor
timestamp: October 25, 2021, 12:13:28 pm
content: In this application the cap in question was a Taiyo Yuden TMK212BBJ106MGHT. Will speak to my CM today about substituting in a flexible termination cap. Might as well do the 10µF, 1µF & 100nF all the same. Good idea or no? So far I have not discovered an issue in any of the other caps in that array, just to 10µF bulk cap.Was looking on Mouser and they have a few in stock in the qty I need. Not looking for the cheapest option, my devices are installed in high ambient heat (up to 70C) and long term reliability in a high temp area is what I want.100µF->https://www.mouser.com/ProductDetail/810-CGA4J1X7S1E106K21µF->https://www.mouser.com/ProductDetail/81-GCJ21BR71E105KA2L100nF->https://www.mouser.com/ProductDetail/581-08053C104K4Z2ARegarding caps, and what little I know about them, and how little they cost in overall terms, is there a "best" cap for me to pick? Is going with a X7R dielectric going to give me higher reliability over a lower temp dielectric? These little $0.06 caps have cost me much frustration. If there is a better cap for $0.25 for example, I want that. Sorry for these most basic questions.

author: Manul
author_rank: Super Contributor
timestamp: October 25, 2021, 07:01:18 pm
content: I have no experience with Taiyo Yuden. Maybe they have quality problems and you were unlucky enough to use them. Known, reputable brands which come to my mind are AVX, Vishay, Kemet, TDK.Normally, dielectric defines things like capacitance stability over temperature, decrease of capacitance due to DC bias, these sort of things. Not directly reliability. Reliability is more a function of manufacturing process quality, capacitor size (density) and, of course, usage.Soft termination capacitors are a valuable improvement for applications, requiring high reliability under harsh conditions (especially flexing, vibration). They are widely used in military, medical, avionics, automotive applications. Not strictly necessary for your application, but would certainly eliminate some risk.Hope you will not develop a capacitor phobia, likely you just got unlucky and it happens.

author: sailah
author_rank: Contributor
timestamp: October 25, 2021, 07:51:18 pm
content: Quote from: Manul on October 25, 2021, 07:01:18 pmThx I ordered a few reels of the soft termination caps from vendors I use regularly like TDK, Murata and AVX. Have call with CM tomorrow and will discuss. I don't know why we were using Taiyo Yuden other than it was on the BOM and was built that way.So far I have performed the C13 cap reflow on about 20 devices and they all work perfectly. I guess the CM can do an autopsy with Xray and see if cap broke or was soldering issue. The solder looks OK in my magnifying glass but what do I know.Now I just need to deal with fallout of all the devices that have failed in the field. Hope my customers aren't too ticked off.Trying to devise how I can do a current test on these to maybe catch the issue. Problem is that the supercap masks the current draw of a couple mA pretty easily during that initial flash. When I have been testing these, I simply unsolder one leg of the cap, test, and then resolder it. Don't want to be doing that during production runs. My cycle time on my programmer is about 20 secs.

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: October 25, 2021, 08:22:40 pm
content: For production test, would it be easier to not connect the battery, charge the supercap and check that voltage was still above a certain value after a fixed tim, with the mcu in a test sleep mode?

author: Kasper
author_rank: Frequent Contributor
timestamp: October 28, 2021, 05:22:56 pm
content: Just came across this IP68 waterproof LoRa sensor that reminded me of your product. I wouldn't expect seeed to be great quality but it could help you get some ideas for waterproof antennas.https://www.seeedstudio.com/Dragino-LSN50-V2-Waterproof-Long-Range-Wireless-LoRa-Sensor-Node-Support-868MHz-Frequency-p-4474.html

author: jmelson
author_rank: Super Contributor
timestamp: October 28, 2021, 06:45:31 pm
content: Interesting paper, thanks!I am just wondering, what the HECK I have stumbled into doing right, so that I have almost NEVER run into this.Now, I do my own reflow here, using a toaster oven with ramp-and-soak thermocouple controller, and reflow at the lowest temp I can get reliable melting at. So, maybe the whole issue is crazy reflow profiles and very rapid heat-cool delta-T ramps. And, maybe I'd never notice a capacitor leaking a few microamps, as I don't usually do battery-powered stuff. But, I am just mystified as to why I have not run across this issue. I have had exactly ONE capacitor that clearly was bad, and split into two pieces when replaced.Jon

author: Kasper
author_rank: Frequent Contributor
timestamp: October 28, 2021, 07:14:06 pm
content: Quote from: jmelson on October 28, 2021, 06:45:31 pmI too am surprised how supposedly fragile ceramic caps are. I used to solder avionics comms equipment. We would hand solder lots of ceramic caps. We'd put them between legs on ICs, solder 0805s ontop of 0603s, etc. We'd do all sorts of mods because it was easier to certify and perform mods than it was to update a PCB and certify it.

author: sailah
author_rank: Contributor
timestamp: October 30, 2021, 11:57:17 am
content: Quote from: mikeselectricstuff on October 25, 2021, 08:22:40 pmPossible but that's not how we flash the firmware, not sure how we could calibrate that. I'm sure that there's a calculation to figure out the voltage drop on a 0.47F cap over time given anything about say 10µA. I suspect that 2.4mA is pretty simple to figure out, but what about 67µA? Our cycle time on programming is about 20 secs. Clearly the Keysight is the right tool for this job but I would need to get someone to program it to turn it on for a set amount of time and then switch off the outputs, measure. Maybe not as big a deal but just one more thing...I need an easy button hereQuote from: Kasper on October 28, 2021, 05:22:56 pmThanks, we have our device certified to IP67, although generally speaking we are strictly in an industrial environment or maybe outside in a pipe rack. I have had instances where customers broke the antenna but usually it was from a ham-fisted installer. In thousands of deployed devices maybe only a couple have broken.Here's the latest in this saga, again, massive thanks to mystery EEVblog member who walked me through numerous debug steps. Thanks and thanks to everyone else for their kindness and interest with a complete stranger.I spoke to my CM, who was also a bit mystified at this issue. They place millions and millions of caps every year and never had an issue. I sent them the faulty device with the traces cut to the bad cap but no other modifications were performed.I had approx 55 devices left from that production run so rather than test them all by unsoldering the supercap, I decided to swap out C13 to a comparable soft termination X7R cap from TDK. I also sent CM a reel of these for our run. Probably not necessary but I also switched C14 and C15 to soft caps as well, belt and suspenders. I had to upgrade my soldering situation and got a soldering smoke collector from Weller, nice 10" microscope to verify solder joins, new flux from Chipquik. I don't do a lot of soldering so it was helpful practice. Most of my joints were pretty good and if they weren't I did them again.The CM is going to make 50 devices with no supercap and ship them to me. I will test all 50, flashed, on Keysight to verify sleep current. If all 50 pass, I'll greenlight the rest of the run and send them back to be finished.Thanks, Peter