Mathematician needed! Multiply then Add vs Add then Multiply - Page 1
Forum section: electronics / projects

author: LightBlue
author_rank: Unknown
timestamp: February 26, 2015, 04:55:05 pm
content: I am working on a system that combines the outputs of 16 ADCs. Each 12-bit ADC value is individually scaled (by an 8-bit number) and then all the results summed together. This requires a large intermediate results buffer. I disagree with my colleague's assessment of the size of the intermediate buffer but I lack the mathematical knowledge to prove it (or the key words to search for an answer). I think you would need a 24-bit buffer but I'm being told that 16-bits is OK. I showed an example but still got brushed aside (see below). Can anyone help me?Decimal example:Each ADC output is 57Each ADC is scaled by 20% (20/100)Multiply -> Add -> Divide : ( SUM( ADCn * SCALARn ) ) / MAX_SCALAR12 (ADC bits) + 8 (Scalar bits) + 4 (16 ADCs) = 24 bits to hold the intermediate resultIntermediate1: ADCn * SCALARn = 57 * 20 = 1140Intermediate2: SUM(all Intermediate1 values) = 1140 + 1140 + ... + 1140 = 18240Result = Intermediate2 / 100 = 182.4 (truncated gives 182)Multiply -> Divide -> Add: SUM( ADCn *SCALARn / MAX_SCALAR )12 (scaled ADC bits) + 4 (16 ADCs) = 16 bits to hold the intermediate resultIntermediate1: ADCn * SCALARn / MAX_SCALAR = 57 * 20 / 100 = 11.4 (truncated, result = 11)Result = SUM(all Intermediate1 values) = 11 + 11 + ... + 11 = 176Am I missing something? Some information theory or statistical fact that means I don't need a big intermediate register? I was told that scaling a value doesn't add any new information and so the number of bits is the same. I can see that, but I don't believe that's true if you're adding lots of individually scaled numbers.Hmmph

author: tonyarkles
author_rank: Regular Contributor
timestamp: February 26, 2015, 05:03:39 pm
content: One thing that jumps out at me is that by dividing early, you're going to be losing precision. The bits that you're discarding due to the division will be set to 0 in your sum, instead of retaining their values until the very end."I was told that scaling a value doesn't add any new information and so the number of bits is the same."That's sort of true, but not. It's true that you're not adding any new information, but because you're storing it in the same representation (e.g. 16-bit int), you can end up discarding information if that scalar is less than 1 (or results in an overflow).Edit:For example, if all 16 ADCs are reading 77, and you've got a 0.2 scaling factor:Multiply, add, divide: 77*0.2 = 15.4 --- 15 + 15 .. = 240 --- 240/16 = 15Multiply, divide, add: 77 * 0.2 = 15.4 --- 15/16 = 0 --- 0 + 0... = 0 --- 0/16 = 0

author: suicidaleggroll
author_rank: Super Contributor
timestamp: February 26, 2015, 05:04:25 pm
content: It looks like you have your answer. If you save the division to the end then you get a more accurate result, but it requires more bits to hold the intermediate value. If you perform the division immediately, you introduce roundoff error that then propagates through the addition and compromises the accuracy of the result, but you can do it with fewer intermediate bits.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: February 26, 2015, 05:09:42 pm
content: Quote from: LightBlue on February 26, 2015, 04:55:05 pmThat is incorrect. Scaling a floating point value doesn't change anything, but scaling an integer up can overflow unless you add bits, and scaling an integer down can introduce roundoff error.

author: IanB
author_rank: Super Contributor
timestamp: February 26, 2015, 05:35:15 pm
content: You need a 16 bit buffer.Consider:You have sixteen 12 bit values. Since 16 is 2^4, summing 16 items requires the same amount of space as multiplying one item by 16, i.e. four more bits. So you can guarantee that 16 bits is enough space to sum sixteen 12 bit values without losing any information.So I would say you should sum your values into your 16 bit buffer and then scale the summed quantity as the last step.

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: February 26, 2015, 05:46:32 pm
content: I agree with IanB.You error is to multiply by 20 the results.You should first sum all data 16 * 12 bits numbers : 2^4 x 2^12 = 2^16 maximumthen if you really want to scale by 20/100, the best thing is to divide by 5 the final result.It would be more complicated if you wanted to scale by a ratio of prime numbers 23/101 for example.In this case the best result is obtained by multiplying first by 23 and divide then by 101The problem occurs if by multiplying by 23 you go beyond 2^16 but you can check that.A solution is then to make an integer division with remainder : N *23 /101First write N = q *101 + r ( q = int (N/101) ; r = N-101*q; )then N*23/101 = q*23 + (r*23)/101like that, you never go beyond your 16 bit buffer.

author: hamster_nz
author_rank: Super Contributor
timestamp: February 26, 2015, 05:54:18 pm
content: Quote from: suicidaleggroll on February 26, 2015, 05:09:42 pmI think he is talking integer math, in an environment were you have full control of the number of bits used (e.g. FPGA?).I say 20 bits, as long as the 8-bit scale factor never goes below 16.The worst case is when the scale factor is 0x01, and if you use a 16-bit register (and use the appropriate divisions along the way) you will end up with only the eight most significant bits of the ADC reading making it through to the intermediate result. The next smallest scale factor (0x02) gives you 9 bits into the final result.It is pretty easy to see that as long as your 8-bit scale factors are never less than 16 then every bit of the ADC will make a difference in an intermediate 16-bit register, as 16 of these are being added together then you will need 20 bit final total.With 20 bits you can guarantee that the LSB of every ADC makes a difference in the final total - as long as the 8-bit scale factors are always > 15If this is part of a control loop, the rounding errors might not be all that important - maybe all that maters is when the LSB of the ADC value changes then the final result of the calculation changes appropriately? Maybe the scale factors are only to tune out gain differences between channels, and the full 0x01 to 0xFF range won't be being used?

author: IanB
author_rank: Super Contributor
timestamp: February 26, 2015, 05:58:47 pm
content: Quote from: hamster_nz on February 26, 2015, 05:54:18 pmIt's not multiplying by an 8 bit scale factor it's dividing by 5.So you sum your 12 bit ADCs into your 16 bit buffer and then divide the result by 5. Since it is a division it is going to shift the MSB downwards and roll off some lower order bits. The result will fit into 14 bits since 5 > 2^2.

author: LightBlue
author_rank: Unknown
timestamp: February 26, 2015, 06:02:26 pm
content: Thanks everyone, I didn't expect replies so soon!If it's not clear from my post, each ADC result is individually scaled and all numbers are unsigned integers. This is being performed by multiple FPGAs with a bus carrying the intermediate result. Using more bits has a real impact, i.e more FPGA pins and a more complicated PCB.The type of scaling that might be applied is a half a bell-curve. ADCs at one end might be scaled by 10/256 while ADCs at the other end might be scaled 230/256.The final result that is taken from the last FPGA is currently 16-bits.I think anything less than a 24-bit intermediate register will loose precision in the result. But is there a way to prove(or disprove) it to someone if they don't follow your reasoning? Any good books references or app notes?James

author: dannyf
author_rank: Super Contributor
timestamp: February 26, 2015, 06:18:43 pm
content: Due to rounding errors, add then divide is better than divide then add.

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: February 26, 2015, 06:38:59 pm
content: I am not sure I see exactly what you are doing, but it seems that you could usean additional 16 bit register to store all the remainders, multiplied by the numerator of the scaling factor.I understand that you want to sum 16 values like A_n* b_n/256 where b_n can change from 1 to 255and A_n is a 12 bit numberI am doing a little bit rapidly so forgive me if there is some errors (not checked) butR=0;S=0;for n= 1 to 16 {R += (A_n && FF)*b_n ;S += R>>8;R = R && FF;S += (A_n >> 8 ) * b_n;}Should give a proper result with only 16 bits for R and S.

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: February 26, 2015, 06:41:38 pm
content: It is funny, there is a bug in the Forum softwareIf I writeS += (A_n >> 8X)X*Xb_n;with not spaces in place of the X I getS += (A_n >>*b_n;

author: Mechatrommer
author_rank: Super Contributor
timestamp: February 26, 2015, 06:42:24 pm
content: Quote from: IanB on February 26, 2015, 05:35:15 pmhe is giving the formula:...SUM( ADCn * SCALARn )...n=1 to 16 so there are SCALAR1, SCALAR2, SCALAR3 ... SCALAR16. they could of different values, but OP provided example the same value for all SCALARn. even if you method is applied, later the summed 16bits will need to be multiplied by 8bits value which require 24bits space as a result. if we know precisely what he is facing, we may find tweak and tricks to make it 16bits top if 24bits is really unfeasible.

author: SeanB
author_rank: Super Contributor
timestamp: February 26, 2015, 06:42:56 pm
content: Why do you want a parallel bus outside. FPGA can easily have a serialiser inside to only use a single line out with the data, and either a clocked output or an asynchronous system like RS232 using a start pulse and then the data, along with either a parity or a CRC check, or use some other serial protocol with enough bits in it. Less pins, and you can have 12 of the inputs on the receiving end deserialisers running together with enough FPGA space. That way you only have to use a 24 bit bus internally to the system chips, and if you use a 24 bit bus throughout ( makes the design easier, just either set the top 12 bits low if the scaling is done at the end or use 20 bits out of 24 if the scaling is done with the send side) you do not see it outside, and then your output can be any desired form you like. Depending on the data rate this will either work with plain logic level paths or you will need to use a LVDS scheme with transmission lines, and in that case using serial is a lot easier to route and get working with minimal skew.

author: IanB
author_rank: Super Contributor
timestamp: February 26, 2015, 06:44:06 pm
content: Quote from: JacquesBBB on February 26, 2015, 06:41:38 pmYes, that bites me too. One fix is this:Code:R=0;S=0;for n= 1 to 16 {R += (A_n && FF)*b_n ;S += R>>8;R = R && FF;S += (A_n>>8)*b_n;}

author: IanB
author_rank: Super Contributor
timestamp: February 26, 2015, 06:56:01 pm
content: Quote from: LightBlue on February 26, 2015, 06:02:26 pmI think you are having trouble proving it because your reasoning is flawed (your argument is incorrect). You cannot successfully prove an incorrect premise.If you take a 12 bit value and scale it by a factor between 0 and 1, then it will fit into 12 or fewer bits. If you sum 16 of these values the result will fit into 16 or fewer bits.If you argue that multiplying a 12 bit value by 10/256 you will lose precision in the result, well yes you will. But that is also because you are losing significance in the value. Extra precision is not useful if it is insignificant.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: February 26, 2015, 07:14:48 pm
content: Quote from: JacquesBBB on February 26, 2015, 06:38:59 pmThat's 32 bits (16 for R and 16 for S). If he can't swing 24 bits he certainly can't swing 32.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: February 26, 2015, 07:16:58 pm
content: Quote from: IanB on February 26, 2015, 06:56:01 pmHe already proved it in the first post. Dividing before adding and restricting the working space to 16 bits created a 3.3% error in the result.I agree with you that he only needs 16 bits to hold the final result at the necessary level of precision, but I'm almost positive the working space needs to be larger in order to prevent accumulated roundoff error.Same reason why professional audio is done in 24-bit before dropping to 16-bit for the final result. 16-bit is perfectly fine for the end product, but you need more range during processing in order to prevent accumulated roundoff error from affecting the result.

author: LightBlue
author_rank: Unknown
timestamp: February 26, 2015, 07:19:45 pm
content: Quote from: SeanB on February 26, 2015, 06:42:56 pmI'm constrained to use a particular type of FPGA and cannot use clock multipliers. It would be easy to shove all the ADC outputs into a single 1000-pin BGA device but I can't. The ADC data is coming into the FPGAs at the clock frequency and since I cannot use clock multipliers the data cannot be serialised. I'm stuck with a parallel bus.

author: SeanB
author_rank: Super Contributor
timestamp: February 26, 2015, 07:25:36 pm
content: FPGA does not have a clock multiplier inside it? parallel data bus will be difficult to route over any distance reliably.

author: LightBlue
author_rank: Unknown
timestamp: February 26, 2015, 07:27:41 pm
content: Quote from: IanB on February 26, 2015, 06:56:01 pmWhat if one number is significant and 15 other numbers on their own are not. If I round them down (lose the precision, decimal places, whatever) then the result is the same as the one significant number. But all those little 15 numbers might add up to be a little bit significant and ignoring them early introduces an error.If that's incorrect then please show me a proof (your own or a text book etc). I just want a proof either way.

author: LightBlue
author_rank: Unknown
timestamp: February 26, 2015, 07:31:56 pm
content: Quote from: SeanB on February 26, 2015, 07:25:36 pmThe incoming parallel ADC data is going at the clock speed so unless I use a clock multiplier, I can't serialise it. I can't use any kind of clock multiplier.I don't have to route the parallel data far.

author: IanB
author_rank: Super Contributor
timestamp: February 26, 2015, 07:59:16 pm
content: Quote from: LightBlue on February 26, 2015, 07:27:41 pmI think if you don't want to "lose" any information then you will need a 24 bit accumulator. However, you now have a cost/benefit situation. Does the additional cost of the extra bits make a difference in the practical results of the application?

author: LightBlue
author_rank: Unknown
timestamp: February 26, 2015, 08:15:21 pm
content: Quote from: IanB on February 26, 2015, 07:59:16 pmI can't argue this point because my colleague doesn't believe it...At least it's a little involved. I used to work for a well-known appliance manufacturer and overheard another engineer trying to explain to an industrial designer that the +/- 2kV power supply he'd made gives the 4kV that was asked for. I think he ended up drawing a number line

author: hamster_nz
author_rank: Super Contributor
timestamp: February 26, 2015, 10:08:17 pm
content: Quote from: LightBlue on February 26, 2015, 06:02:26 pmWhy not just write it up both ways software, and then 'diff' the results, find the worst case and then document it (e.g. in a spreadsheet).

author: Galaxyrise
author_rank: Frequent Contributor
timestamp: February 26, 2015, 11:23:23 pm
content: Quote from: LightBlue on February 26, 2015, 08:15:21 pmI would think your proof by example was good enough. On what grounds was it rejected? Here's a direct proof:Let ADCn = (An*256 + Bn), where An and Bn are integersLet SCALARn = Xn, where Xn is an integerIdeally, the sum yields:Code:TRUNC(SUM(An*Xn) + SUM(Bn*Xn)/256)Doing the divide before add changes this to: (assuming the MulDiv is done correctly)Code:SUM(An*Xn) + SUM(TRUNC(Bn*Xn/256))Subtracting the two produces the difference between the two approaches:Code:TRUNC(SUM(Bn*Xn)/256) - SUM(TRUNC(Bn*Xn/256))Let Bn*Xn = Cn*256 + Dn. The difference can now be expressed asCode:TRUNC(SUM(Cn) + SUM(Dn)/256) - (SUM(Cn) + SUM(TRUNC(Dn/256)))since Dn < 256, then that reduces toCode:TRUNC(SUM(Dn)/256)Since the largest Dn can be is 255, the maximum error is SUM(255)/256 = 255*16/256 = 15.

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: February 27, 2015, 12:14:44 am
content: Quote from: suicidaleggroll on February 26, 2015, 07:14:48 pmBut you can just discard R at the end and just transmit S.The final result is on 16 bits, precise up to the latest bit.The only thing that is discarded is the fractionary partR/256 where R< 256

author: Mechatrommer
author_rank: Super Contributor
timestamp: February 27, 2015, 02:13:01 am
content: Quote from: LightBlue on February 26, 2015, 08:15:21 pmthen ask the dickhead to draw his proof. as for you, its as simple as trimming a skinhead proof...Code:12bits ADC x 8bits SCALAR value = log2(2^12 x 2^8) = 20bitssum 16 of that = log2(2^20 x 16) = 24bitsinteger division doesnt require extra bitsyou laid your proof, now its his turn, ymmv..edit: as for requiring 16bits, he has to proof some bits are insignificant and "binary division" (aka bitshift) or arithmetic precision loss is acceptable. that is the "application specific" we cant help.

author: IanB
author_rank: Super Contributor
timestamp: February 27, 2015, 02:29:59 am
content: Quote from: LightBlue on February 26, 2015, 07:27:41 pmWhy round them down? Why not round to the nearest integer? If you do that some will round down, some will round up, and on average over many samples you will keep the original sampling precision.

author: Mechatrommer
author_rank: Super Contributor
timestamp: February 27, 2015, 02:38:57 am
content: whats with the "all round down" or "all round up" case that will lead to error accumulation? that needs some serious analysis well, 24 bits is not so bad, you only need list of 16millions number thats pretty easy with modern computer. i'm not going to do the statistical homework though...

author: Phoenix
author_rank: Frequent Contributor
timestamp: February 27, 2015, 03:17:57 am
content: Why not take the pragmatic approach and program each method into your favourite C compiler, with the appropriate truncations. Then just set up some test case inputs and see which method produces the output you want (vs say the ideal/floating point case).

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: February 27, 2015, 08:31:23 am
content: As I said, not being familiar with FPGA, I do not know if this applies but compared toQuote from: JacquesBBB on February 26, 2015, 06:38:59 pmIf you need a parallel algorithm, you can just use the same in parallel form (A_n 12 bits numbers, b_n 8 bits numbers).S is a 16 bit number.S = SUM((A_n >> 8 ) * b_n) + SUM((A_n && FF)*b_n )>>8;@3roomlab, there is no overflow as the sum comprises only 16 (12 bits) numbers that is <= 16 bits,and in any case the total sum is less or equal to 16 bits.

author: macboy
author_rank: Super Contributor
timestamp: February 27, 2015, 02:51:03 pm
content: Prove it in the simplest way possible to your challenged colleague:You have 12 bit samples. The therefore can be up to 4095. Let's assume worst case scaling of 1/1, so all intermediate results may be up to 4095.You sum 16 of these: 4095 * 16 is 65520. That is the absolute maximum result of all samples summed with worst case scaling.A 16 bit integer goes up to 65535. Ask him if that is big enough or not!EDIT:The other way to scale would be to do the (variable 10~230) multiply portion in the individual FPGA, and the (fixed 256) divide portion in the master FPGA. This would then require a 20 bit bus, and a 24 bit accumulator, which after the divide by 256 (i.e. truncate lower 8 bits) becomes a 16 bit result. This way will significantly reduce rounding errors caused by scaling down individual samples, but since you want to limit the bus size, I assume that you won't be doing it this way.