Conversion of ADC data to Q15/Q31 format - Page 1
Forum section: electronics / beginners

author: syntax333
author_rank: Regular Contributor
timestamp: January 15, 2020, 05:21:06 pm
content: Hello, I am working on a DSP project, but I have never worked on such projects before so I am a newbie in this area.Using the microprocessor PIC32MX795, I need to calculate the FFT of the data I read from the ADC.I can read the data from the ADC, but it says in the library manual that I need to convert this ADC data to the Q15 format in order to use the functions available in the DSP library of the PIC32MX795.Because the FFT function is expecting for the Q15 or Q31 format as input.When I researched the Q15 format on the internet, I found that it would be enough to shift the integers I had read from ADC by 1 bit to the right.For example:ADC_Data = 3086 = 0b110000001110shifting 1 bit to right 0b110000001110 becomes 0b0110000001110Is that the case? Or is there anything else I have to do?As I said I am new to this area so any information would be appreciated. Thanks in advance.

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 15, 2020, 05:59:25 pm
content: Signed Q15 is a fixed point format where you have 15 fractional bits, and can represent values from -1 to 32767/32768≃0.9999695.Logical valuevcorresponds to 16-bit signed integer ⌈32768×v⌋, and integericorresponds to logical valuei/32768.Signed Q31 is a fixed point format where you have 31 fractional bits, and can represent values from -1 to 2147483647/2147483648≃0.99999999953.Logical valuevcorresponds to 32-bit signed integer ⌈2147483648×v⌋, and integericorresponds to logical valuei/2147483648.You do not necessarily need to do anything. It depends on what the ADC readingsrepresent.The PIC32MX795 has a 10-bit ADC, which supports various output modes (page 239 of the datasheet, bits 8 to 10 of the AD1CON1 register).The signed fractional 32-bit format (0b101) is compatible with Q31, and the (low 16 bits of the) signed fractional 16-bit format (0b011) is compatible with Q15.That way thelogical valuethat you operate the FFT on is between -1 and +1 (excluding the upper limit +1), -1 corresponding to the minimum voltage the ADC can measure, and the largest positive value (0.9999695) the maximum voltage the ADC can measure.If you are only interested in some smaller range of values, and want to map them to -1..+1 or some subrange,thenyou do need to do some math conversion. I can show you the exact math you need, although it really is just mapping the source integer values to target integer values with proper rounding, nothing special.

author: syntax333
author_rank: Regular Contributor
timestamp: January 15, 2020, 06:35:49 pm
content: Thank you for your reply.It would be great if you can show me the math I need to convert the integer values to Q15.Also from your explanation, can I assume that the example I have given is incorrect? (With bit shifting like : Value = (Value >> 1) & 0xFFFE))

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 15, 2020, 08:32:57 pm
content: Quote from: syntax333 on January 15, 2020, 06:35:49 pmLike I said, if you use the signed fractional 16-bit format, the 16-bit ADC samples are in a suitable format.If you want to convert values 0 to 1023, inclusive, to correspond to 0.0 to 0.9990234375=1023/1024 in Q15, just shift the value left by65 bits.Quote from: syntax333 on January 15, 2020, 06:35:49 pmYes.Let's start from the beginning, then.The ADC in your microcontroller has 10 bits of precision. It does not just yield values 0 to 210-1, but has 8 different modes in which the results can be obtained:Mode 0 = 0b000: Unsigned 16-bit integer. The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 0000 00dd dddd dddd, i.e. between 0 and 1023, inclusive.Mode 1 = 0b001: Signed 16-bit integer. The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 0000 sssd dddd dddd, i.e. between 0 and 511, or 57344 and 65535, inclusive. As a signed 16-bit integer, the value is between -512 and 511, inclusive.Mode 2 = 0b010: Unsigned 16-bit fractional. The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 dddd dddd dd00 0000, i.e. between 0 and 65472, inclusive, in steps of 64.Mode 3 = 0b011: Signed 16-bit fractional. The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 sddd dddd dd00 0000, i.e. between 0 and 32704, or 32768 and 65472, inclusive, in steps of 64. As a signed 16-bit integer, the value is between -32768 and 32704, inclusive, in steps of 64.Mode 4 = 0b100: Unsigned 32-bit integer. The 32-bit ADC result words (ADC1BUFx) reads in binary 0000 0000 0000 0000 0000 00dd dddd dddd, i.e. between 0 and 1023, inclusive.Mode 5 = 0b101: Signed 32-bit integer. The 32-bit ADC result words (ADC1BUFx) reads in binary ssss ssss ssss ssss ssss sssd dddd dddd, i.e. between 0 and 1023, inclusive.Mode 6 = 0b110: Unsigned 32-bit fractional. The 32-bit ADC result words (ADC1BUFx) reads in binary dddd dddd dd00 0000 0000 0000 0000 0000, i.e. between 0 and 4290772992, inclusive, in steps of 4194304.Mode 7 = 0b111: Signed 32-bit fractional. The 32-bit ADC result words (ADC1BUFx) reads in binary sddd dddd dd00 0000 0000 0000 0000 0000, i.e. between -2147483648 and 2143289344, inclusive, in steps of 4194304.Above, d corresponds to value bits and s is sign bit.Of these modes, 3 is compatible with Q15, and 7 is compatible with Q31. So, if you use these modes, your ADC data is in suitable form for the FFT.Fixed point integers are perfectly normal integers, thatrepresenta different value. For example, ifx= 4251 = 0b0001000010011011, as a Q15 it represents value 4251/32768 ≃ 0.12973.As theFixed-point aritmeticarticle at Wikipedia explains, addition and subtraction of fixed-point integer types is done exactly like for normal integers, but multiplication and division involves an extra scaling (bit shift) operation.The FFT library you have only supports logical values from -1 to just under +1, represented by fixed-point integers.It is up to you to decide how you map your ADC readings for the FFT, but if you use ADC mode 3 or 7, it provides the 16- or 32-bit ADC samples in a form that corresponds to -1 (smallest or most negative voltage the ADC can detect) to +1 (greatest or most positive voltage the ADC can detect) in Q15 or Q31 fixed-point format.You only need to do math on the ADC readings if you need a custom range, so that e.g. an specific ADC reading corresponds to 0, and another to (just under) +1.Mathematically, scaling \$x\$ from \$[x_{min}, x_{max}]\$ to \$[y_{min}, y_{max}]\$ is$$y = (x - x_{min}) \frac{y_{max} - y_{min}}{x_{max} - x_{min}} + y_{min}$$This only applies to integers if we do rounding correctly. Unfortunately, in C/C++, integer division truncates, i.e. rounds towards zero. Most people, including Arduino developers, do not care about such minor details, so theirmap()function behaves differently for integer and floating-point values.If we want to do rounding correctly, we need to use$$y = \frac{(x - x_{min})(y_{max} - y_{min}) + r}{x_{max} - x_{min}} + y_{min}$$where \$r\$ has magnitude just less than half of \$x_{max} - x_{min}\$, but has the same sign as \$y_{max} - y_{min}\$. This yields the same as calculating the value using floating-point numbers using the mathematically correct formula, and then rounding it to an integer. When programming, one needs to notice that the product is calculated first, and therefore the temporary variables need to be large enough (int32_tif usingint16_t, for example).For example, let's say we use ADC mode 0, but for say electrical reasons our readings are always between say 113 and 742, inclusive.For the FFT, we wish to map these to say -0.25 to just under 0.75, which in Q15 correspond to integers -8192 and 24575.Thus, we have \$x_{min} = 113\$, \$x_{max} = 742\$, \$y_{min} = -8192\$, \$y_{max} = 24575\$, and \$r = 314\$. To convert a buffer full of samples we could useCode:/* Maps buffer values 113..742, inclusive, to -8192..24575, with correct rounding. */void prepare_adc(int16_t buffer[], uint32_t count){for (uint32_t i = 0; i < count; i++)if (buffer <= 113)buffer = -8192;elseif (buffer < 742)buffer = ((int32_t)(buffer - 113)*32767 + 314) / 629 - 8192;elsebuffer = 24575;}which produces the same results if you used floating-point calculation,Code:buffer = round( (buffer - 113.0) * 32767.0 / 629.0 - 8192.0);Mathematically, \$r\$ is the additive rounding factor needed to round towards nearest integer instead of towards zero; and the integer division (by 629, above) in C and C++ rounds towards zero.

author: syntax333
author_rank: Regular Contributor
timestamp: January 16, 2020, 03:20:35 pm
content: Wow! Thank you very much for your detailed explanation.Just to make sure that I get it right, if I wanted my 16-bit ADC data [0,4095] to be mapped from 0 to 0.9990234375(in this case 0x7FFF = 32767) my parameters becomes:xmin=0 , xmax=4095, ymin=0, ymax=32767, and r=2047 right? After the mapping I just need to give the mapped array to FFT function?Quote from: Nominal Animal on January 15, 2020, 08:32:57 pmWhile researching I just came a cross a youtube video by Eli Hughes which explains the topic of fixed point math. So correct me if I am wrong, Q15 format contains 1 sign bit right? If I want to convert 10-bit value to represent Q15, don't I need to shift the 10 bit data(0 to 1024) to left by 5 bits?For example:10bit data : bb bbbb bbbbshifting left by 5 bitsbbb bbbb bbb0 0000isn't there a hidden 0 at postion 16 which represents sign? so my Q15 format becomes0bbb bbbb bbb0 0000=sbbb bbbb bbb0 0000and my limits becomes 0 to 32736.oris the sign bit extra to 16 bit data?Do I lose 1 bit to sign information if I used "Mode 3 = 0b011: Signed 16-bit fractional."?

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 16, 2020, 07:31:15 pm
content: Quote from: syntax333 on January 16, 2020, 03:20:35 pmYes; except the range is 0.0 to 0.9999695 (≃ymax/32768).If range 0.0 to 0.99975586 is okay, i.e. 0 to 32760/32768 inclusive, and you don't mind a fractional shift, then you can just shift each sample left by three bits. In that case, ADC 0 would be mapped to 0.0 in Q15, and ADC 4096 to 1.0 in Q15 (except that 4095 is the biggest value).(This corresponds to xmin=0, xmax=4096, ymin=0, ymax=32768, r=0, but is much faster to calculate. The rounding just shifts the entire scale half an integer up.)If you want 0.0 to 0.9990234375 (=32736/32768), then use xmin=0, xmax=0, ymin=0, ymax=32736, r=2047.Quote from: syntax333 on January 16, 2020, 03:20:35 pmYes, that is correct. I had a typo in my answer, fixed now.Quote from: syntax333 on January 16, 2020, 03:20:35 pmIt depends on which kind of ADC sampling mode you use. Your microcontroller supports differential readings, so I believe the signed fractional range is intended for those, because then the ADC can return negative values too.For unipolar measurements, I think you would lose one bit using mode 3, but I could be wrong. (I use AVRs and ARMs myself, and haven't used PICs.)You cannot use mode 2 = 0b010 for unipolar measurements, because that would map upper half of the ADC values to negative Q15 values.If you use Mode 0 = 0b000, unsigned 16-bit, then shifting each sample left by 5 bits is a sensible option. Then, the maximum ADC reading corresponds to 1023/1024 = 32736/32768 = 0.9990234375 for the FFT.If you use Mode 0, but need the ADC values to span 0.0 to 0.9999695 for the FFT, use(x*32767+511)/1023(which is equivalent to the precise formula using xmin=0, xmax=1023, ymin=0, ymax=32767, r=511).That division will be slow, but it just so happens that if we use(x*32799+527)>>10we get the exact same results for x=0..1023, but with the cost of one multiplication, one addition, and one bit shift right. Your microcontroller has a single-cycle 32×16 multiple-add, so this will be very fast.