Failure Rate Estimation - Page 1
Forum section: electronics / beginners

author: katzohki
author_rank: Frequent Contributor
timestamp: June 17, 2020, 12:53:11 am
content: Not sure where else to put this.We have done some testing that shows that an event in manufacturing may occur during 37.5% (75 out of 200) of power up cycles. This event (a voltage excursion related to the power supply) can cause damage to / failure of the electronics, but does not cause failure every time. In fact, none of those 75 caused damage. We have 3 units out of 240 that have come back for this failure and confirmed the damage. Each device has only 2 power up cycles.Based on this information, is there a way to estimate the total probability that a unit will experience a failure (and therefore the probable number of total bad units)? I figured out something this morning that I thought made sense, but I'm totally friend now and can't remember what my math was.Thanks for any help.

author: jmelson
author_rank: Super Contributor
timestamp: June 17, 2020, 02:50:58 am
content: Quote from: katzohki on June 17, 2020, 12:53:11 amWhat is the magnitude of the voltage excursion? Does it exceed the "do not exceed" limits in the datasheets for all components powered by that supply? If so, how much does it exceed the ratings? Some ICs have a fairly soft maximum, so you can safely exceed the rating for a second or so, and it will almost never fail. Others have a very hard rating, where exceeding the maximum for even milliseconds will almost surely guarantee catastrophic failure.With the data you have provided, nobody could answer your question.Jon

author: bob91343
author_rank: Super Contributor
timestamp: June 17, 2020, 02:54:27 am
content: There are two basic ways to estimate failure rate.The first is to make many tests to see how the performance goes and thus get a number based on how many failures you see vs how many trials.The second is to analyze the situation to determine what causes the failure. Then do a worst case analysis to compute the failure rate.If you don't know why it's failing, you cannot estimate the failure rate because you have no theoretical basis for a calculation. So perhaps you are stuck with the empirical method.How many units are you working with? Why are they failing? Is it a design problem, a testing problem, an environmental problem, a component tolerance problem, or is it random?

author: golden_labels
author_rank: Super Contributor
timestamp: June 17, 2020, 04:49:48 am
content: Note:my background is software development and dealing with statistics mostly in macroeconomy/demographics, not electronics. The advice below is based on what is employed in those branches: I may not be aware of some methods which could be used for electronics in this case. If there are better methods, I’m happy to learn them.IMO with the scarce data available the best guess is using the number of returned units. But it is averyweak estimation — in scientific context the result would begarbage. At 0.95 confidence level, the interval for the population proportion is [-0.0016, 0.0266] or “between -0.16% and 2.66%” if one prefers percentages. The margin of error is larger than the estimated proportion itself, which is bad (as you can see from obtaining negative proportion). And the reason is clear: 3/240·240 < 10 and therefore doesn’t fit even the weak form of condition for calculating the interval for population proportion.And that is based on all units shipped, without taking into account which of them experienced the voltage excursion. If you would have two samples and they would be much larger, right statistical tests could be employed to determineif(1)there is any significant difference and then calculating how many devices that experienced the problem are likely to fail.____(1)Just because two estimations differ doesn’t mean the real value does.

author: Rerouter
author_rank: Super Contributor
timestamp: June 17, 2020, 01:49:07 pm
content: 2 power cycles per unit? results in 3/240 units with failures, that means the chance of any unit failing over 2 power cycles is ~1.25%, so as its an iterative thing, the failure rate for a single power cycle would be closer to 1.12%,My statistics is rather weak I am afraid. So there would be some wide error bars until you have some extra data, but it looks like the mean time between failures would be around 75 power cycles...

author: OwO
author_rank: Super Contributor
timestamp: June 17, 2020, 02:44:53 pm
content: This is called the sunrise problem, which is when you try to estimate the frequency of an event in a population given only some observations of it occurring (and not occurring). In this case the event is a device failing, and the population is the set of all devices that have had the voltage spike.Let f be the frequency of voltage spiked devices failing, e.g. f = 0.5 would mean half of devices having seen the voltage spike will fail. You don'tknowf, f is a random variable. Therefore f has an associated probability density function (pdf). Given a prior estimate of this pdf (you can assume a uniform pdf if you are unsure), and the observations, you can derive a posterior pdf. Taking the expected value from the posterior pdf will tell you the probability of the *next* voltage spiked device failing, but keep in mind this isn't f itself. However you can get some confidence info by looking at the pdf, for example you can say "there is 99% chance that f is less than <value>". I have derived the formula for this pdf before, it only involves first year level calculus.If you are only interested in the expected value there is a simple formula, which is (events observed + 1) / (total observation size + 2), for example in your case if you assume exactly 37.5% of devices have seen the voltage spike, then (3 + 1) / (75 + 240*0.375 + 2) = 2.4%. That means there is a 2.4% chance that the next device that see the voltage spike will fail.Btw I first discovered this problem in high school when I theorized that you can infer your chances with a girl based on your past rejections and acceptances with others.

author: CatalinaWOW
author_rank: Super Contributor
timestamp: June 17, 2020, 03:13:28 pm
content: You can do very little with the information you have. First, you really don't know if the problem is power cycle related. You have a hypothesis, based on your lab cases, but given the lower number of cycles in your field returns this is by no means confirmed. I see three paths you might take. All of which give answers which are weak until a great deal more data is available.1. Based failure rate on units shipped. No dependence on power cycles. The assumption here is that units are failing because of a weakness that is not uncovered in your pre-shipment testing or that there is some sort of rapid aging phenomenon The calculation is simple just returns divided by units shipped. The path forward is to look for differences in the environment at the two sites and do detail failure diagnosis of the returned units. Until you learn more about the real failure mechanism there is little that can improve the estimate other than increasing the population size.2. Base failure rate on number of power cycles. You can use your lab test data to estimate the failure rate and then use that number to estimate the likelihood of a failure in the first two cycles. I haven't done the calculation, but assume that the likelihood will turn out to be very low. This would imply that your lab environment doesn't represent the customer environment and therefore any estimate of field failure rate would be meaningless.3. Assume your hypothesis on failure mechanism is correct, but recognize that your data on power fluctuations is unlikely to be representative. To do an estimate here you will have to measure/find data on power quality at a wide range of locations representative of your customers environment, specifically including data on the fluctuations embodied in your hypothesis. This data can be used to estimate overall failure rate, and more importantly failure rate at specific locations. This approach overlaps the first in that it implies that power at your two failure locations is different than your labs. Your action in this case would be to either strengthen your product to resist those environments, or modify your power specs forcing customers to clean their power.

author: golden_labels
author_rank: Super Contributor
timestamp: June 17, 2020, 03:45:21 pm
content: Quote from: OwO on June 17, 2020, 02:44:53 pmFor clarity: rule of succession. The sunrise problem is a specific application of that rule with the number of failures being 0.However, it only tells how likely it is that one next item will fail. Not failure probability of the population.Another problem with the calculations above are two assumptions: that there is correlation between experiencing the event and failures, and that only those events cause failures. So far neither can be said to be true. Therefore the expression should be: (3 + 1) / (240 + 2) ≈ 1.65%.As a sanity check, use the very same expression to calculate the probability of the next devicenotfailing.

author: katzohki
author_rank: Frequent Contributor
timestamp: June 17, 2020, 03:59:48 pm
content: Thank you all for the help, it has given me a lot to work with, although mostly confirmed my suspicion that I won't be able to go too far with math alone.Based on the responses I realized I left out some critical information or was unclear. The voltage excursion ONLY happens in manufacturing because our device is powered by batteries. So the potential exposure is only twice per device life. Once it ships it won't happen again. It does violate a hard voltage limit on a critical component. I haven't been able to narrow down on the exact voltage vs pulse width numbers that definitely cause a failure. 50% haven't had enough time and 50% not enough boards to kill. Oh also our good scope died, Murphy's law is full in effect here lol.

author: jmelson
author_rank: Super Contributor
timestamp: June 17, 2020, 06:58:11 pm
content: Quote from: katzohki on June 17, 2020, 03:59:48 pmSo, this excursion happens when the battery is installed, or when it is connected to some factory test rig?If when battery is installed, I wonder if a special rig could be set up to ramp up the supply voltage before/while the battery is installed? Then, if that works, there would be no spike.Jon

author: golden_labels
author_rank: Super Contributor
timestamp: June 20, 2020, 12:53:17 pm
content: Let’s say we have produced some items and some portion of them were returned. There is probability, unknown, of an item being faulty. In the original questionit’s assumed that the faults are due to some items experiencing overvoltage, but — while this is not an unlikely factor — there is no sufficient evidence in the data and amount of data is too tiny to even consider thinking about analyzing that difference (hard to derive knowledge from dividing 3 items into 2 groups). Therefore let’s assume, that all items will experience the same adverse conditions with the same probability, and try to determine what we know about all of them in general: the faulty from overvoltage, the faulty from other factors, the working ones that experienced overvoltage, the ones that haven’t, and possibly those which are broken but never returned for whatever reason. In other words, let’s see what we can learn about the probability of items, any items, being faulty.We may model that as theBernoulli process. The model is composed as follows: there is some number of random variables, which may have a value 0 or 1. In our case those will be items, which are either not returned (0) or returned (1). Each item has the same probability of being returned(1)and that they’re independent.Therefore the returned items count has thebinomial distribution, where the probability is the probability of item being returned (unknown). For example see the distribution for 240 items and probability of return 3/240 = 0.0125.Code:n = 0:1:240;scatter(n, binopdf(n, 240, 0.0125));xlabel('n');ylabel('P(n)');xlim([0 15]); # Nothing interesting above thatylim([0 0.3]);grid on;That Octave script produces the following chart (limited to the first 20 values, because probability after that is too small to even be interested in it).The chart tells us what is the probability of obtainingnreturned items from 240,giventhe true probability of a single item being returned is 0.0125. The maximum value is at 3, which is expected for something that, on average, produces 3 returns per 240 items. But we are not much less likely to have only 2 items returned. Less likely, but still possible, to see 1 or 4 items. And so on. In fact there is some, extremely tiny probability of getting all 240 items returned.2Could another return probability make us see 3 items returned? Of course! Let’s try that for 0.01 and 0.02:In fact every probability of an item being returned, except 0 and 1, produces a non-zero probability of getting exactly 3 items.3We can chart that too:Code:p = 0:0.0001:0.06; # Nothing interesting above thatplot(p, binopdf(3, 240, p));xlabel('p');ylabel('P(P(return) = p | returned = 3)');xlim([0 0.06]); # Nothing interesting above thatylim([0 0.25]);grid on;And this is the complete image we get from knowing that we have obtained 3 returned items for 240 shipped items, with some unknown probabilitypof a single item is to be returned.Whatusefulinformation we can get from that? Of course we see that the maximum is at 0.0125, but we already know that from basic maths: it’s 3/240. But now we may determine how likely it is that the true probability lies inside some region around 0.0125. E.g. we can say that if we see 3 items returned from 240 shipped, the return probability is in [0.0125 - 0.0025, 0.0125 + 0.025] with some probability.↑ obtained from extending the previous code with:Code:hold on;pc = 0.01:0.0001:0.0150;area(pc, binopdf(3, 240, pc));plot([0.0125 0.0125], [0 0.25]);legend("P", "0.0125±0.0025", "0.0125");Now, we may be tempted to calculate that probability by integrating the brown area. BEWARE! IT’S A TRAP! This is not aprobability density function. This is just a chart of some probabilities. To obtain a PDF we must scale it, so the integral of the whole function is 1:trapz(0:0.001:1, binopdf(3, 240, 0:0.001:1)). That yields the following PDF:Code:p = 0:0.0001:0.06; # Nothing interesting above thatscale = trapz(0:0.001:1, binopdf(3, 240, 0:0.001:1));plot(p, binopdf(3, 240, p) / scale);xlabel('p');ylabel('PDF');xlim([0 0.06]); # Nothing interesting above thatylim([0 60]);grid on;hold on;pc = 0.01:0.0001:0.0150;area(pc, binopdf(3, 240, pc) / scale);plot([0.0125 0.0125], [0 60]);legend("PDF", "0.0125±0.0025", "0.0125");And now we can also ask about CDF and calculate the value we need:Code:pLow = 0:0.001:0.010;pHigh = 0:0.001:0.015;trapz(pHigh, binopdf(3, 240, pHigh) / scale) - trapz(pLow, binopdf(3, 240, pLow) / scale)The answer is about 0.2658. That is true, but still not very informative.~27% is not really good certainity. So let’s make another plot, which shows usprobability of the true returns probability being in 0.0125±ε:Code:ex = 0.0001:0.0001:0.0250; # Yes, going to the negative sidefunction ret = rp(err)global scale;pLow = 0:0.0001:max(0, 0.0125 - err);pHigh = 0:0.0001:(0.0125 + err);ret = trapz(pHigh, binopdf(3, 240, pHigh) / scale) - trapz(pLow, binopdf(3, 240, pLow) / scale)endfunctionplot(ex, arrayfun(@rp, ex));xlabel("ε");grid minor on;hold on;plot([0 0.025], [0.95 0.95]);That vertical red line marks the 0.95 probability. So, from the data we have, with probability of 0.95, the true probability of a return for a random item is somewhere between 0.0125±0.019. Or [-0.0065, 0.0315]. Why is the information so unhelpful? Because the dataset is so tiny. We have 3 items in 240 and from that we are trying to infer some useful knowledge. That’s not going to happen.So was all tha work useless? No. I hope someone has learned a bit about how to look at those problems. The other thing is that the result obtained,[-0.0065, 0.0315], is saying a thing similar to the one from the earlier interval estimation: [-0.0016, 0.0266]. The range is larger here, but the picture is the same. Despite it was used for data that is not meeting requirements for that tool and it was too small, statistics gave a decent answer. Something to think about the next time you hear “statistics lie”.And now I should pray that, after 5 years of not using my knowledge in practice, I haven’t made a fool of myself by having an error in my reasoning.____(1)This is true, because we are modelling all items together: both those which have experienced the overvoltage and those which haven’t. At this point they’re undistinguishable to us and the final result will cover both cases as one shipped item.(2)Once in the number that has over 450 zeros in its decimal notation.(3)Since the distribution is discrete, we are allowed to use word “exactly”.

author: Siwastaja
author_rank: Super Contributor
timestamp: June 20, 2020, 01:37:03 pm
content: I think that with such low number of devices, and such high failure rate so far, you shouldn't be doing any failure rate statistics, but instead be searching for the root cause for the failure.It's very likely it isn't random, it's very likely it doesn't follow gaussian distribution.It's a fairly "early" problem.Once you are producing inmillions, you start seeing more truly "random" failures, then it pays of to do some statistics, start thinking about component MTBFs, manufacturing process yield, and so on. On that scale, it's about "bad luck", slight process contamination, and such events that can't be fully reduced to zero, but which can be analyzed with statistical methods.But at this point, it's very likely the failures arenotcaused by any "normal" reason. The design is just broken.This means, the failure can disappear on it's own suddenly, then reappear after years. Or it can suddenly causeallunits to fail.Look for the reason.