Datalogging: correcting readout from a sensor that drifts - Page 1
Forum section: electronics / general

author: gildasd
author_rank: Frequent Contributor
timestamp: May 18, 2016, 09:31:44 am
content: Hello, this is for my end of studies project:I'm datalogging the force changes on a large cube due to wind. It does about 65000 read outs a day...One of the sensors is drifting up and down with the voltage changes (Cube 3 column)I'm trying to correct this using Excel, but I'm not sure in what direction to go...I've tried "((Vcc-5.31+1)*Cube3)" and it works at the extremes but over correct the 5.03V values.Not really sure what kind of methodology I should be using to correct thisdatetime temp wind cube2 cube3 vcc"2016/5/12 6:22:1" 18.32 0 332 342 5.31"2016/5/12 6:22:2" 18.32 0 333 340 5.31"2016/5/12 6:22:3" 18.32 0 332 342 5.31"2016/5/12 6:22:4" 18.32 0 314 337 5.31"2016/5/12 6:22:5" 18.32 0 332 341 5.31"2016/5/12 6:22:6" 18.32 0 331 343 5.31"2016/5/12 6:22:7" 18.32 0 335 344 5.31"2016/5/12 6:22:8" 18.32 0 322 340 5.31"2016/5/12 6:22:9" 18.32 1 341 344 5.31"2016/5/12 6:22:10" 18.32 3 337 341 5.31"2016/5/12 1:14:10" 22.19 0 316 281 5.03"2016/5/12 1:14:11" 22.51 0 317 283 5.03"2016/5/12 1:14:12" 22.51 0 313 286 5.01"2016/5/12 1:14:13" 22.19 0 348 289 5.03"2016/5/12 1:14:14" 22.19 0 345 288 5.03"2016/5/12 1:14:15" 22.19 0 305 279 5.03"2016/5/12 1:14:16" 22.19 0 331 282 5.01"2016/5/12 1:14:17" 22.19 4 355 288 5.03"2016/5/12 1:14:18" 22.19 3 319 278 5.03"2016/5/12 1:14:19" 22.19 1 321 282 5.03"2016/5/11 17:46:19" 35.08 0 314 162 4.25"2016/5/11 17:46:20" 35.08 0 331 166 4.25"2016/5/11 17:46:21" 35.08 0 333 170 4.25"2016/5/11 17:46:22" 35.08 0 304 159 4.25"2016/5/11 17:46:23" 35.08 0 323 167 4.25"2016/5/11 17:46:24" 35.4 0 335 170 4.25"2016/5/11 17:46:25" 35.4 1 334 166 4.25"2016/5/11 17:46:26" 35.08 2 311 161 4.27"2016/5/11 17:46:27" 35.08 1 315 164 4.25"2016/5/11 17:46:28" 35.4 1 340 171 4.25

author: rs20
author_rank: Super Contributor
timestamp: May 18, 2016, 09:43:32 am
content: Why only 30 points of data?Even with only 30 points, plotting cube3 vs vcc clearly shows that the relationship is non-linear. Take all your data, plot cube3 vs vcc, and fit a polynomial (or whatever shape makes sense). The equation of the fitted polynomial can then be used as your correction term.See:https://docs.google.com/spreadsheets/d/1Fblyl6y5soteb-Dns6m3vamXS10_gbyAXF2jRYn7AOo/edit#gid=556521029

author: gildasd
author_rank: Frequent Contributor
timestamp: May 18, 2016, 04:17:07 pm
content: Thanks! I'll look into that.And you are right, I'll an extra data point between the min and the average and the max and the average.For now I'm using ax+b variations.The picture is where I'm at now, the min and mas are perfect, but I over-correct the medium values.

author: gildasd
author_rank: Frequent Contributor
timestamp: May 18, 2016, 08:24:22 pm
content: I think I found it.The previous reply put me on the right trackIt's well within the margin of error. Wonderful!Cube3*(sqrt((5/Vcc)^6.5))More a lucky guess than anything... Happy dance.

author: rs20
author_rank: Super Contributor
timestamp: May 18, 2016, 10:13:33 pm
content: Seems like you'd be far better off doing a regression than just guessing formulae, but whatever works for you I guess...

author: gildasd
author_rank: Frequent Contributor
timestamp: May 19, 2016, 09:47:17 am
content: Quote from: rs20 on May 18, 2016, 10:13:33 pmWell, guessing with a bit of maths intuition. I wanted to stay in the y=ax+b complexity.I started with what I know: the difference in Vcc.Taken as an difference this would correct the base and the end of the curve.This was somewhere but not perfect. A simple subtraction did not work.So I decided to try all the fundamentals: addition, subtraction, multiplication, division, square and root.Just by deduction on what the factor could be (in it's simplest form):addition,subtraction,multiplication, division, square and root.The division of the "ideal voltage" of 5V by the detected Vcc gave me a factor that looked great, but how to use it?Your formula was hinting at something like "ax^(bx-c)" that can be read as "(ax^bx)/(ax^c)"This kinda reminded me of some square root results, so I returned to the division above with your idea of a control exponent: Sqrt((5V/Vcc)^x)Frankly that looked too simple to work.I was quite surprised that it did. The mantra "when it doubt, go back to fundamentals" does work!Do you have a source for "real world" regressions?

author: rs20
author_rank: Super Contributor
timestamp: May 19, 2016, 09:59:47 am
content: Quote from: gildasd on May 19, 2016, 09:47:17 amLook at the sheet I linked. Excel, OpenOffice, even Google Sheets will fit a simple quadratic on a series of points and label the curve with the formula.

author: gildasd
author_rank: Frequent Contributor
timestamp: May 19, 2016, 11:09:02 am
content: Quote from: rs20 on May 19, 2016, 09:59:47 amAh, ok, I think it was taught with a totally different name in French, under what menu/help section of Excel should I start with (i am no expert)?

author: rs20
author_rank: Super Contributor
timestamp: May 19, 2016, 11:20:13 am
content: Quote from: gildasd on May 19, 2016, 11:09:02 amPlot the data as an X/Y chart/graph. Then, somewhere in the settings for the graph, you will be able to add a "regression" or "trend line" or something like that.https://support.office.com/en-us/article/Choosing-the-best-trendline-for-your-data-1bb3c9e7-0280-45b5-9ab0-d0c93161daa8