unit resolution per decade choice - Page 1
Forum section: electronics / general

author: julian1
author_rank: Frequent Contributor
timestamp: November 22, 2021, 12:13:14 am
content: Consider an instrument, that outputs some operator controlled parameter (V,I,Ohm,W,Hz etc). Two output resolution choices are available/feasible - either 40k/decade or 50k/decade.For example, a voltage-source/power-supply with a 0-10V output range, could choose either of these control schemes;0-10V/50k count, gives 0.2mV interval. last digit 0.2,0.4,0.6,0.8, 1 mV0-10V/40k count, gives 0.25mV interval. last digits 0.25,0.5,0.75,1 mV.The advantage of the first scheme is a (slightly) higher resolution. While the second approach achieves similar resolution, but also permits a precise half fraction/unit (ie. 0.5mV) to be cleanly expressed.Which is preferred?

author: Tomorokoshi
author_rank: Super Contributor
timestamp: November 22, 2021, 12:39:31 am
content: 1. Is the fundamental variable a 16-bit register?2. Can you explain a little about what "outputs some operator controlled parameter" means? For instance, is this a dial for some setting, where the digital value is then displayed somewhere, or is it a numeric entry field?3. Can the operator input control actually achieve that resolution such that one increment is controllable?4. Is there a key system reason for needing access to the 0.5 mV value over the 0.4 mV value?

author: Benta
author_rank: Super Contributor
timestamp: November 22, 2021, 09:38:39 pm
content: I have feeling that this is a user interface issue. Look at how oscilloscope inputs are graduated: 0.1, 0.2, 0.5; 1, 2, 5.

author: julian1
author_rank: Frequent Contributor
timestamp: November 22, 2021, 10:48:01 pm
content: QuoteGood example. Those values are definitely the cleanest/most intuitive for an operator. And on a scope they are employed for the trace display interval and input ranging.In my case, the ranging is per decade (eg. 0-1V, 0-10V, 0-100V etc), and I am trying to decide how to slice up the resolution of the least significant digit.Having a half-step 0.5 (like a scope) feels like it could be useful. Or maybe, it's a confusing analogy, that doesn't apply to the least-significant digits?QuoteNot in the strict sense - ie. in a tranceiver, a 500Hz half-step increment is required due to conventions around broadcast frequencyBut I keep thinking about scenarios where 4 or 5 step increments might be better for charting and graphs. Or sweep resolution.Or binning observational data by set parameter, to average/aggregate.Or charting on a log10 transformed scale.Actually, I suspect it doesn't matter either way.

author: TimFox
author_rank: Super Contributor
timestamp: November 22, 2021, 11:25:57 pm
content: The advantage of 1, 2, 5, 10 ... for oscilloscopes is how the results fit into a fixed-pitch graticule on the screen.AC voltmeters often use 1, 101/2(labeled "3"), 10 etc. since it fit well into a dB scheme.It depends on your requirement.

author: David Hess
author_rank: Super Contributor
timestamp: November 24, 2021, 05:43:38 pm
content: 20,000 and 40,000 count converters are more common than 50,000 count converters, perhaps for the reason you identify. Offhand I do not know of any 50,000 count converters.All of my multimeters are 2,000, 20,000, 4,000, or 40,000 counts, although I know of some which have 6,000 or 60,000 counts.