next step learning after Arduino ? - Page 1
Forum section: electronics / general

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 12:08:57 am
content: I have a friend that has asked me, what is the next step for learning about embedded systems beyond arduino ??My biggest gripe with Arduino is there there is no in circuit debugger.So any other platform would need that.I have played with ESP32, but I've found how it wants to load the whole thing into the large device, that compile / flash run times are just too long. And Ive found, IME, the debugger experience isnt very good. (maybe I am doing it wrong)SO what's the next step that has friendly, not too complex tools, but has a real debugger and visibility of CPU registers, peripheral registers, memory etc?IE somewhere between Arduino , and AVR with the awful microchip debugger. actually its not that awful from an experienced POV.IE a debugger and IDE without millions of options and setup parameters that means if you get one bespoke thing wrong it doesnt work. like Eclipse....-glen

author: Smokey
author_rank: Super Contributor
timestamp: July 13, 2024, 12:47:47 am
content: Start with a dev board and example programs that do things you are interested in.I would recommend STM32 Nucleo. All the debugging stuff is built into the dev boards so you don't need to buy an expensive programmer/debugger. Wide variety of boards that do various things. If you want to do motor control, get the motor control one. Want to do LCD interfacing, get that one. And they are pretty inexpensive. This is a "Real Embedded" system.Get set up with the manufacturer tools. STMCubeIDE and the HAL, for all the complaining around here, is a pretty good platform. Especially for learning all the stuff you mentioned. You can use it at any level you want: RTOS to bare metal. Plus it is maintained by a real company and not just a bunch of random dudes on the internet.One thing to keep in mind when selecting a platform like this. People tend to "stick to what they know". So pick something with enough variety that you can comfortably live in that ecosystem and expand into new projects. STM works for that as well.

author: nctnico
author_rank: Super Contributor
timestamp: July 13, 2024, 12:52:20 am
content: I'd say any microcontroller with an ARM-Cortex core will do combined with Eclipse or VS Code. For example: ST's CubeIDE (Eclipse based) debugging does work with ST's evaluation boards with integrated programmer / debugging. Although I'm not a fan of STM32 devices due to crappy documentation, finicky peripherals and their bloated and horrible HAL which just gets in the way. So far I've found NXP's LPC series ARM Cortex devices much nicer to work with. Really good documentation plus consistent peripherals across the entire line. In about 20 years, I never needed a debugger to write software for those microcontrollers so I can't comment on debugging.

author: Psi
author_rank: Super Contributor
timestamp: July 13, 2024, 01:00:22 am
content: The next step after arduino is, I think, still using arduino but leaning how to use the MCU hardware registers to do stuff instead of usingthe Arduino helper functions which hide that stuff from you. Read the datasheet, understand the registers and use them to do stuff.Like try the following without using any arduino functions, just registers- Try setup an IO as output and toggle it without using digitalwrite()- Try flipping one bit of a IO port without affecting other bits/pins on that port, and understand why the code works to do that.- Try enable PWM on an pin that has a hardware PWM channel by reading datasheet and setting the hardware registers.- Try setup your own interrupts for something, enable them yourself, set flags bits etc..I think it's good to get an understanding of that before moving to an entirely different MCU/IDE.

author: Smokey
author_rank: Super Contributor
timestamp: July 13, 2024, 01:09:55 am
content: Quote from: Psi on July 13, 2024, 01:00:22 amIn general yes. You can use the Arduino hardware without any of the Arduino software stuff. The problem with this is that a lot of those boards were never designed to easily use a real debugger. Plus depending on which Arduinos you are using, the Atmel (yes I know they are microchip now) might not be a great choice for starting a project from scratch.Adruino world shields you from essentially anything embedded (peripheral config/interfacing, memory, timing, interrupts...) so "knowing Adruinos" doesn't really translate to "knowing AVRs".

author: Psi
author_rank: Super Contributor
timestamp: July 13, 2024, 02:11:30 am
content: Learning to use a debugger is definitely something you need to learn at some point.But I'm not sure it really matters when you learn it.And it only takes an hour to figure out the basics assuming you have it setup and installed correctly and it's working.You can divide programmers into two groups.Those who use the debugger many times a day and those who might use it once every few months when they can't figure something out and it becomes the last resort.It's the absolute worst when you have a bug that doesn't occur when you enable the debugger.

author: MasterT
author_rank: Frequent Contributor
timestamp: July 13, 2024, 02:30:29 am
content: You can use serial monitor to read/write registers on atmega328. No debugger required.I switched to stm32 nucleo boards awhile ago, but continue to enjoy arduino IDE and do low level debugging over serial, registers, set / reset bits, change clock etc.Code:/*TO DO*/String in_String = "";boolean end_input = false;uint16_t adres_reg = 0;uint8_t debug_osm = 0;void setup(){Serial.begin(115200);in_String.reserve(200);Serial.print(F("\n\tSketch: Reg_base_UNO"));Serial.print(F("\n\tSetup done."));}void loop(){int32_t tempr = 0;char * pEnd;// d, a, r, w - (a)ddress variable length./*q, *w, e, *r, t, y, u, i, o, p,*a, s, *d, f, g, h, j, k, l,z, x, c, v, b, n, m,*/if(debug_osm) {}serialEvent();if( end_input) {char cmd = in_String;in_String = '+';if( cmd == 'd' ) {debug_osm = 1 - debug_osm;if(debug_osm) Serial.print(F("\nDebug aktiv."));else Serial.print(F("\nDebug de-aktiv."));}if( cmd == 'a' ) {adres_reg = strtol( in_String.c_str(), &pEnd, 16);Serial.print(F("\n\tReg: "));Serial.print(adres_reg, HEX);Serial.print(F("\tvalue: "));tempr = (*(uint8_t*)adres_reg);Serial.print(tempr, BIN);}if( cmd == 'r' ) {Serial.print(F("\n\tReg: "));Serial.print(adres_reg, HEX);Serial.print(F("\tvalue: "));tempr = (*(uint8_t*)adres_reg);Serial.print(tempr, BIN);}if( cmd == 'w' ) {Serial.print(F("\n\tReg: "));Serial.print(adres_reg, HEX);Serial.print(F("\tvalue: "));tempr = (*(uint8_t*)adres_reg);Serial.print(tempr, BIN);tempr = strtol( in_String.c_str(), &pEnd, 2);(*(uint8_t*)adres_reg) = tempr;Serial.print(F("\tnew value: "));tempr = (*(uint8_t*)adres_reg);Serial.print( tempr, BIN);}in_String = "";end_input= false;}}void serialEvent() {while (Serial.available()) {char inChar = (char)Serial.read();in_String += inChar;if (inChar == '\n') {end_input= true;}}}// Address Book:/*page 536:0x0B (0x2B) PORTD PORTD7 PORTD6 PORTD5 PORTD4 PORTD3 PORTD2 PORTD1 PORTD0 950x0A (0x2A) DDRD DDD7 DDD6 DDD5 DDD4 DDD3 DDD2 DDD1 DDD0 950x09 (0x29) PIND PIND7 PIND6 PIND5 PIND4 PIND3 PIND2 PIND1 PIND0 950x08 (0x28) PORTC – PORTC6 PORTC5 PORTC4 PORTC3 PORTC2 PORTC1 PORTC0 940x07 (0x27) DDRC – DDC6 DDC5 DDC4 DDC3 DDC2 DDC1 DDC0 940x06 (0x26) PINC – PINC6 PINC5 PINC4 PINC3 PINC2 PINC1 PINC0 940x05 (0x25) PORTB PORTB7 PORTB6 PORTB5 PORTB4 PORTB3 PORTB2 PORTB1 PORTB0 940x04 (0x24) DDRB DDB7 DDB6 DDB5 DDB4 DDB3 DDB2 DDB1 DDB0 940x03 (0x23) PINB PINB7 PINB6 PINB5 PINB4 PINB3 PINB2 PINB1 PINB0 94// PORTS********************#define PINB _SFR_IO8(0x03)#define DDRB _SFR_IO8(0x04)#define PORTB _SFR_IO8(0x05)#define PINC _SFR_IO8(0x06)#define DDRC _SFR_IO8(0x07)#define PORTC _SFR_IO8(0x08)#define PIND _SFR_IO8(0x09)#define DDRD _SFR_IO8(0x0A)#define PORTD _SFR_IO8(0x0B)// COMPARATOR***************#define ACSR _SFR_IO8(0x30)#define ACIS0 0#define ACIS1 1#define ACIC 2#define ACIE 3#define ACI 4#define ACO 5#define ACBG 6#define ACD 7// ADC**********************#define ADC _SFR_MEM16(0x78)#define ADCW _SFR_MEM16(0x78)#define ADCL _SFR_MEM8(0x78)#define ADCH _SFR_MEM8(0x79)#define ADCSRA _SFR_MEM8(0x7A)#define ADPS0 0#define ADPS1 1#define ADPS2 2#define ADIE 3#define ADIF 4#define ADATE 5#define ADSC 6#define ADEN 7#define ADCSRB _SFR_MEM8(0x7B)#define ADTS0 0#define ADTS1 1#define ADTS2 2#define ACME 6#define ADMUX _SFR_MEM8(0x7C)#define MUX0 0#define MUX1 1#define MUX2 2#define MUX3 3#define ADLAR 5#define REFS0 6#define REFS1 7#define DIDR0 _SFR_MEM8(0x7E)#define DIDR1 _SFR_MEM8(0x7F)// TIMERS*******************#define TIFR0 _SFR_IO8(0x15)#define TOV0 0#define OCF0A 1#define OCF0B 2#define TIFR1 _SFR_IO8(0x16)#define TOV1 0#define OCF1A 1#define OCF1B 2#define ICF1 5#define TIFR2 _SFR_IO8(0x17)#define TOV2 0#define OCF2A 1#define OCF2B 2#define GTCCR _SFR_IO8(0x23)#define PSRSYNC 0#define PSRASY 1#define TSM 7#define TCCR0A _SFR_IO8(0x24)#define WGM00 0#define WGM01 1#define COM0B0 4#define COM0B1 5#define COM0A0 6#define COM0A1 7#define TCCR0B _SFR_IO8(0x25)#define CS00 0#define CS01 1#define CS02 2#define WGM02 3#define FOC0B 6#define FOC0A 7#define TCNT0 _SFR_IO8(0x26)#define OCR0A _SFR_IO8(0x27)#define OCR0B _SFR_IO8(0x28)#define TIMSK0 _SFR_MEM8(0x6E)#define TOIE0 0#define OCIE0A 1#define OCIE0B 2#define TIMSK1 _SFR_MEM8(0x6F)#define TOIE1 0#define OCIE1A 1#define OCIE1B 2#define ICIE1 5#define TIMSK2 _SFR_MEM8(0x70)#define TOIE2 0#define OCIE2A 1#define OCIE2B 2#define TCCR1A _SFR_MEM8(0x80)#define WGM10 0#define WGM11 1#define COM1B0 4#define COM1B1 5#define COM1A0 6#define COM1A1 7#define TCCR1B _SFR_MEM8(0x81)#define CS10 0#define CS11 1#define CS12 2#define WGM12 3#define WGM13 4#define ICES1 6#define ICNC1 7#define TCCR1C _SFR_MEM8(0x82)#define FOC1B 6#define FOC1A 7#define TCNT1 _SFR_MEM16(0x84)#define TCNT1L _SFR_MEM8(0x84)#define TCNT1H _SFR_MEM8(0x85)#define ICR1 _SFR_MEM16(0x86)#define ICR1L _SFR_MEM8(0x86)#define ICR1H _SFR_MEM8(0x87)#define OCR1A _SFR_MEM16(0x88)#define OCR1AL _SFR_MEM8(0x88)#define OCR1AH _SFR_MEM8(0x89)#define OCR1B _SFR_MEM16(0x8A)#define OCR1BL _SFR_MEM8(0x8A)#define OCR1BH _SFR_MEM8(0x8B)#define TCCR2A _SFR_MEM8(0xB0)#define WGM20 0#define WGM21 1#define COM2B0 4#define COM2B1 5#define COM2A0 6#define COM2A1 7#define TCCR2B _SFR_MEM8(0xB1)#define CS20 0#define CS21 1#define CS22 2#define WGM22 3#define FOC2B 6#define FOC2A 7#define TCNT2 _SFR_MEM8(0xB2)#define OCR2A _SFR_MEM8(0xB3)#define OCR2B _SFR_MEM8(0xB4)#define ASSR _SFR_MEM8(0xB6)#define TCR2BUB 0#define TCR2AUB 1#define OCR2BUB 2#define OCR2AUB 3#define TCN2UB 4#define AS2 5#define EXCLK 6#define TIMER2_COMPA_vect_num 7#define TIMER2_COMPA_vect _VECTOR(7)#define TIMER2_COMPB_vect_num 8#define TIMER2_COMPB_vect _VECTOR(8)#define TIMER2_OVF_vect_num 9#define TIMER2_OVF_vect _VECTOR(9)#define TIMER1_CAPT_vect_num 10#define TIMER1_CAPT_vect _VECTOR(10)#define TIMER1_COMPA_vect_num 11#define TIMER1_COMPA_vect _VECTOR(11)#define TIMER1_COMPB_vect_num 12#define TIMER1_COMPB_vect _VECTOR(12)#define TIMER1_OVF_vect_num 13#define TIMER1_OVF_vect _VECTOR(13)#define TIMER0_COMPA_vect_num 14#define TIMER0_COMPA_vect _VECTOR(14)#define TIMER0_COMPB_vect_num 15#define TIMER0_COMPB_vect _VECTOR(15)#define TIMER0_OVF_vect_num 16#define TIMER0_OVF_vect _VECTOR(16)#define ADC_vect_num 21#define ADC_vect _VECTOR(21)#define ANALOG_COMP_vect_num 23#define ANALOG_COMP_vect _VECTOR(23)*///**//

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 04:41:11 am
content: ....yeah but serial line debugging in arduino is not non-intrusive, there are side effects.Thanks all the for the comments. Keep them rolling.Good ideas of controlling AVR peripherals directly as an intermediate step. There seems to be a big gulf between Arduino and STM cube etc.I cant imagine not using a debugger. That is something from the 1980s.....I see people employed as pros in companies stuck on bugs and they just guess what is going on and seem resistant to using a debugger, what a time waste....QuoteTHose sort of things are usually startup side effects. Starting in the debugger is ooften different. Depends on the platform.

author: brucehoult
author_rank: Super Contributor
timestamp: July 13, 2024, 05:28:48 am
content: Quote from: glenenglish on July 13, 2024, 04:41:11 amYou could step from the ATMega328 to something that supports both the Arduino library (and IDE) and also proper debugging. For example Arm or RISC-V.My sentimental favourite at the moment is the CH32V003. It works on either 3.3V or 5V, is 32 bit, has a similar amount of resources to an ATMega328 (both have 2k RAM, the '003 has 16k of flash vs 32k on the AVR, but also generally more compact code using fewer instructions, especially if using 16 or 32 bit variables. It also runs at 48 MHz instead of 16-20 MHz, and executes 4 bytes of code per two cycles (two 2-byte instructions or one 4-byte instruction) at that speed. Or one instruction per cycle at 24 MHz. It's also not that much more complex than the AVR at the register level and is well documented -- and had a pretty big community around it too. The chips cost $0.10 to $0.16 depending on the pin count (8, 16, 20) if you buy 50 of them (i.e. $5 - $8 total). The official dev board comes with a USB programmer / serial / debug and 5 bare 20 pin chips for $7.23.https://www.aliexpress.us/item/3256804709476544.htmlYou can find 3rd party boards as low as ... well, I see now one for $0.58, five boards for $2.90:https://www.aliexpress.us/item/3256807122251439.htmlOlimex are advertising a board for €1.00 to ship soon, with an Apple I "Wozmon" style monitor program in flash, and a bit-banged VGA output and PS/2 keyboard support, all from an 8 pin chip!https://www.olimex.com/Products/Retro-Computers/RVPC/open-source-hardwareOf course you can re-flash with your own code, modify theirs etc.People are doing all kinds of fun projects with the CH32V003 e.g.And of course they have bigger cousins with more RAM and flash, more pins, native USB and other interfaces etc.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 07:16:31 am
content: Hi BruceSomething that is a bit more interesting than a dev board with a header is a good idea. Like the pics you posted.That's what I will recommend my friend do, his son is proficient with usual Arduino things shields etc, so this is a reasonable next step, and it's a step into knowing any embedded system, IE the skills u get will be good for all micros onwards.cheersYou are right about AVR producing lots of code (compared to a 32 bit micro) with 32 bit values, just like when you shift right by three etc.

author: ralphrmartin
author_rank: Frequent Contributor
timestamp: July 13, 2024, 06:34:58 pm
content: A different take on your friend's question is, rather than learning to use a debugger or about registers etc, get to grips with an RTOS, e.g. FreeRTOS, and how to use it for multitasking.That does not necessarily need to go beyond Arduino, but it is beyond the usual basic Arduino stuff.There's a decent book out there called "FreeRTOS for ESP32-Arduino" which should help.

author: artag
author_rank: Super Contributor
timestamp: July 13, 2024, 07:01:08 pm
content: I think ralphmartin makes a really good point.Don't go tying yourself to an IDE - they're mostly awful in one way or another and tend to force their particular way of working on you. Arduino is very limited but also lightweight. Some people like VSCode, I personally hate it and I think it has severe disadvantages for professional (ie something that has to be maintained by someone else) use. But understanding a bigger concept like an RTOS is growing your ability, not just learning a new tool.Another possibility is digging deeper by writing or rewriting device drivers. The Arduino ones are generally a bit limited and slow - they're trying to make things appear the same, for easy learning. The STM Cube have a similar aim - they want to make it easy for users to migrate across ST products but have little interest in portability beyond that and seem to give no thought to performance. This is not a good model for the budding embedded engineer. They're after low effort to maintain by using common code as far as possible. I would say comparing the arduino drivers for very different hardware such as AVR and ESP32 might be interesting (though I haven't investigated their ESP321 code).

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 07:53:19 pm
content: good take on the concept of learning RTOS, multitasking. lots of new disclipines to learn there. I mean there is a full bottle of stuff to learn with multitasking and multi threading.I hate VS code. I find it maddening. it's pretty and seems to do alot but seems to be alot of secret background stuff going on in scripting, which I get is the advantage of it, but I dont like it.Alright, I think I have enough to go on, The still seems to be a big gap between Arduino and say STM cube.I'll also thing about how I would teach this, as I do a bit of informal teaching, that might lead me to a coherent path.One thing that I like when there is enough RAM available, is reconfiguring the linker script to run the code from RAM. This substantially reduces the usual flash program/restart cycles.Another thing to look at is QEMU, on a high level platform. But eventually, you need the student to come to grips with real hardware and have him/her realise the importance of embedded hardware working like it should, like for example, the student's project , in the extreme case, firing the ejection seat, or jettesoning the service module. That's putting your c0ck on the bl0ck. There are parallels to this in other industries- in airplane maintenance, or building aircraft electronics I get to feel a bit of this with stuff I do.

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2024, 07:59:44 pm
content: Quote from: glenenglish on July 13, 2024, 04:41:11 amActually, not using a debugger is a critical ability. You need to be able to solve problems to really get anywhere, and although a debugger makes some parts of that process easy it is not the solution. Often, something weird happens and you have no idea where the problem might be, so how is the debugger going to help? You can't single-step the entire code, and if you do single-step the chances are the issue may not occur. You're not going to have interrupt interactions timing properly, and anything that relies on timing (perhaps bit-bashing a port) will be stuffed.No, what you need to do is figure out what the problem is before you set about dealing with it. When the weird thing happens you think about it, what could possibly be related. As other things occur you add them to your mental list of clues, much like you might fill in pieces of a jigsaw. Eventually you'll have enough clues to determine that it's probably so-and-so that's causing the problem, and at that point - where you want to test that hypothesis - you 'do' the debugging, which might be to use an LED, console output or, indeed, the debugger.The debugger is might be seen as the Arduino of problem solving.Don't get me wrong - I like a good debugger as much as the next developer, but it's not the end of the world if it's not there. There are many reasons to discard a particular chip as unsuitable for a job, but lack of a debugger is quite a way down the list.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 08:32:58 pm
content: agreed on not using a debugger is a required skill. That is really good point .I would consider that skill to be after proficiency of source level debugging.There's really two situations1) code not functioning because you wrote it wrong- Source level debugger is a huge time saver. That's what I see missing from Arduino.2) cannot use a debugger because any stop/start of the platform would change the timing with hardware or other processes that your program is interacting with. That's where it is essential to learn various tools and methods to solve those ones.These situations are usually tricky and require experience to troubleshoot.The list of (2) is many ! In the simplist terms- interaction with timers, controlling something external that does not stop when you stop the source code execution------------------------------------------------------Remember the old days before jtag etc. We used to toggle pins to see where we were in the program, hooked up to a 2 channel scope or logic probe, or if you were lucky, a 8 or 16 channel logic analyser. Then when we got hardware uarts, we could send a single character to a uart to inform where we were without side effects. Then when there was a bit more memory (maybe 128 bytes..) a push button could interrupt and halt / terminate and send to a uart a small trace dump out of memory to uart or a bit-bashed uart to a teleprinter in 5 bit baudot....

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 08:50:41 pm
content: WHICH brings me to a question...on an embedded platform -Which do you teach first :source level debugging OR non-intrusive hardware based debugging ?

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2024, 10:03:09 pm
content: What do you mean by non-intrusive hardware-based debugging?

author: Postal2
author_rank: Frequent Contributor
timestamp: July 13, 2024, 10:55:57 pm
content: He means the standard method, when a trap (infinite loop) is used at a fixed address, and then an oscilloscope on external memory can be seen which address. This is a well-known common method.

author: PlainName
author_rank: Super Contributor
timestamp: July 13, 2024, 11:07:45 pm
content: That's hardly non-intrusive. In fact, hard to think of anything more so!

author: tggzzz
author_rank: Super Contributor
timestamp: July 13, 2024, 11:15:32 pm
content: Quote from: glenenglish on July 13, 2024, 08:50:41 pmThere are two useful answers:µ/muyes

author: nctnico
author_rank: Super Contributor
timestamp: July 13, 2024, 11:17:50 pm
content: Quote from: glenenglish on July 13, 2024, 08:50:41 pmFirst thing to teach is to write code step-by-step (incremental going through short coding & testing cycles). Make code testable. Having a serial port CLI is extremely handy for this because you can query the state of modules, print debug output and control low-level hardware. This is not only handy during development but also when a device is installed in the field. A field service engineer only needs a serial port and a terminal emulator to diagnose & service the product.However, algorithms which do not need to deal with hardware are not to be debugged on a microcontroller but tested on a PC where debugging is infinitely easier and algorithms can be unit / stress tested in a short amount of time.

author: tggzzz
author_rank: Super Contributor
timestamp: July 13, 2024, 11:19:06 pm
content: Quote from: glenenglish on July 13, 2024, 08:32:58 pmRemember ICE? Used up until at least the 68020 era

author: tggzzz
author_rank: Super Contributor
timestamp: July 13, 2024, 11:22:14 pm
content: Quote from: glenenglish on July 13, 2024, 08:32:58 pmFails on C/C++ code with higher levels of optimisation, where the instruction sequences can't be unambiguously mapped to source code statements.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 13, 2024, 11:27:12 pm
content: Quote from: Postal2 on July 13, 2024, 10:55:57 pmahh no, not quite.I meant where the code toggles or pulses an IO pin as it enters or leaves or goes past a line of code. Non intrusive.When you get to FPGAs this becomes essential...like it was in the 70s and 80s. because even with Xilinx Chipscope, adding logic to the design can change timing.As for true ICE, I could never afford those as a hobbyist / school age kid. They were not cheap, as I remember.thanks for the responses.Gives me some ideas, perspectives that are not my own are useful to me.

author: tggzzz
author_rank: Super Contributor
timestamp: July 13, 2024, 11:30:01 pm
content: Quote from: ralphrmartin on July 13, 2024, 06:34:58 pmAnd then progress to software architectures that don't have/need an RTOS.Then realise that in future highly parallel systems the RTOS can be beneficially implemented in hardware. After all, if you have many cores, you don't need to multiplex multiple processes onto a single core.But with a single or multicore processor, you still need communication between processes. No, not the operating system concept of a process; that's unnecessarily far from the specification/hardware concept of a process

author: Postal2
author_rank: Frequent Contributor
timestamp: July 13, 2024, 11:30:20 pm
content: Quote from: PlainName on July 13, 2024, 11:07:45 pmTry to think about a red lamp into the loop.Quote from: glenenglish on July 13, 2024, 11:27:12 pmOk.

author: tggzzz
author_rank: Super Contributor
timestamp: July 13, 2024, 11:33:54 pm
content: Quote from: glenenglish on July 13, 2024, 11:27:12 pmI used various ICE systems in the 80s. They weren't cheap and their capabilities have be supplanted by JTAG plus competent hardware monitoring.Distressingly little has changed since the 80s, apart from things being smaller/faster/cheaper. We need better.

author: artag
author_rank: Super Contributor
timestamp: July 13, 2024, 11:41:09 pm
content: Quote from: tggzzz on July 13, 2024, 11:15:32 pmI'm inclined to agree with this. It's an incomplete question. There are many other factors to consider when choosing debug method.However, given that you stated embedded, I would say non-intrusive is far more important. Much embedded code is time-sensitive and stepping or breakpointing through it is impossible. It has to keep running at full speed (and don't expect to be able to do too many printf()s either).

author: Psi
author_rank: Super Contributor
timestamp: July 13, 2024, 11:42:16 pm
content: Quote from: glenenglish on July 13, 2024, 04:41:11 amNormally the people who don't use a debugger still do do debugging, just not using the hardware debugger. They watch the code execute in other ways. Usually UART messages that dump out variables as required, or set/clear/toggle pins that they capture and watch on a scope/LA. You can have a pretty advanced debugging system running in real-time with your code. The downside is the loss in execution speed.The reason some people learned to avoid the HW debugger usually stems from being burnt by it too many times.Some debuggers are terrible buggy pieces of shit that crash often and occasionally lie to you, which is something you really don't want in a debugging system. Obviously not all debugger are like that and they have gotten so much better in the last 30 years.But in your formative years learning to code if the debugger enrages you one too many times you tend to swear off it forever.Everyone has their breaking pointQuoteBugs where you're accidently overwriting memory cause this often.As soon as you use the debugger things move around in memory and the data being overwritten is now something else. So the bug you're focusing on disappears and a new bug is created somewhere else.And of cause there's the issue with hardware debugging where you don't have enough flash space to compile your code without optimizations enabled, making the debugger pretty annoying to use. Debugging with UART messages does not have this issue.

author: artag
author_rank: Super Contributor
timestamp: July 13, 2024, 11:46:33 pm
content: Quote from: tggzzz on July 13, 2024, 11:22:14 pmLeading on to those lovely bugs that disappear when you turn optimisation off so your debugger can work properly.

author: Psi
author_rank: Super Contributor
timestamp: July 13, 2024, 11:52:53 pm
content: Even when you compile with optimizations disabled, executing with the debugger running still messes with things and can lead to different results when you look at the code in question with the debugger vs just letting it run.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 13, 2024, 11:59:02 pm
content: Is the Intel processor an mcu? He has a debug-port. Is it necessary to debug the mentioned Arduino in the same way as an Intel processor? What difficulties can we talk about in a two-line program?

author: nctnico
author_rank: Super Contributor
timestamp: July 14, 2024, 12:36:20 am
content: Quote from: Psi on July 13, 2024, 11:42:16 pmMany, but not all have improved. Getting the debugger going on the ESP32 is typically a waste of time. STM32 can be finicky as well (using the official debugger from ST).

author: Postal2
author_rank: Frequent Contributor
timestamp: July 14, 2024, 01:41:50 am
content: I'm guessing that if all the code is under control of one person, then a hardware debugger is redundant. And if a person boasts that he uses excessive tools in his simple work, this does not speak in his favor.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 14, 2024, 01:48:06 am
content: I dont see what the distinction of 'hardware debugger" isthere is roughly three levels in mebedded world :1) You have source level jtag or debug wire etc based debuggers (set breakpoints, hit points , stepping etc, with memory dumps/mods)2) a slight enhancement there these days with register and peripheral viewing and modification included in the above.3) Then, you have lets call it 'pin level debugging' - set / toggle IO pins. dump one character into the UART etcI do not consider anythign with printf() in it to be non intrusive, because a) printf is heavy and b) the processor uses effort to get that buffer out, and if the buffer is full, you stall until the uart gets some chars out. That's intrusive.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 14, 2024, 02:00:19 am
content: Quote from: glenenglish on July 14, 2024, 01:48:06 amThis is necessary in school and in the case of a project developed by many people.If you are making a "blink" program using a debugger.....

author: brucehoult
author_rank: Super Contributor
timestamp: July 14, 2024, 03:09:32 am
content: Quote from: glenenglish on July 13, 2024, 08:32:58 pmYes. Interactive debugging is mostly for beginners who don't understand the basics of how a CPU and programs work.QuoteThis basically NEVER happens to me, for many decades.It's not because I don't make mistakes. It's because I don't write 1000 lines of code in a frenzy and then try to "debug" it. I write 5 or 10 lines of code, and then test it. If it breaks then I already know, ±5 lines of code, where the problem is.If I'm working on a tricky algorithm then I develop it on a real computer, where I can run billions of test cases quickly, or generate tens of GB of debug printf output and then write little Perl scripts to look for problems. Or, in the worst case, use a source debugger, but a printf or assert between every single line of code is better than single-stepping or even setting breakpoints in a debugger. See "Perl", above. Sanity checks written in the code run at the speed of code. Conditional breakpoints in a debugger run a million times slower. OK, maybe a thousand.Single-stepping didn't make sense on a CPU running 400k instruction per second in the 1970s, it sure as heck doesn't make sense on a modern microcontroller running 20 or 30 million instructions per second, let alone on a PC running 10 BILLION instructions per second, per core.The only exception is if you're so fresh you don't yet understand CPU instructions, registers, PC, stack, loops, function calls etc.If I'm working on something that can only be done on the real hardware -- setting up GPIO, or input and output from a UART, or some SPI or i2c communication -- then I write and test ONLY THAT.QuoteIn the mid 2000s I was writing effectively embedded software on pre iOS/Android phones running the BREW operating system. There was no memory protection, no debugging. Not even a serial console. What you could do was draw something on the screen, maybe up in a corner to not interfere too much with using the app, or log printfs to a file in the onboard flash (which was never very large). And even with logging, when the program crashed you lost the last flash sector of printfs unless you closed the file and re-opened for append every time which is very very slow because you're writing a complete flash sector every time.Fun times.But we got the product (AlcheMo Java to native Arm AoT compiler) out and it was extremely solid.

author: Postal2
author_rank: Frequent Contributor
timestamp: July 14, 2024, 03:29:19 am
content: I can only say one thing - not a single problem found recently, including those described by me here, can be detected by debugging tools.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 14, 2024, 03:32:44 am
content: what like a single event upset changing a condition code bit in the ALU ?

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 14, 2024, 03:36:24 am
content: well, I think single steppng, register inspection, etc is enormously useful. Yes sure, develop the algorithm on a high level device first if you want.

author: Psi
author_rank: Super Contributor
timestamp: July 14, 2024, 04:02:10 am
content: Quote from: glenenglish on July 14, 2024, 01:48:06 amIf you are using debugging silicon inside the chip then it's a hardware debugger.If not, then it's a software debugger

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2024, 08:44:19 am
content: QuoteYes, it can be. I think the point is that it's not the end of the world if it doesn't exist - there are other techniques that usefully substitute, albeit they may be more tedious to use, and learning those ultimately leads to a more useful toolbox. Also, on-chip source level debugging isn't the solution for every problem.However, you don't bake a cake without breaking eggs, as they say, and it helps to walk before you run. There is a school of thought that you should be au fait with the hard stuff - times tables, say - before being rewarded with the cool tools - a calculator. But it depends on what the aim of the game is, and you aren't here to learn how to debug but to get down and dirty with the processor.I would second the idea of getting an STM32 dev board, which is ready to go and has the on-chip debugging embedded and just gagging for a USB cable. You don't have to use the Cube IDE or libraries, but doing so is kind of halfway between the Arduino stuff and nothing at all. The hardest decision would likely be what to do with it - some target to work towards is useful.

author: nctnico
author_rank: Super Contributor
timestamp: July 14, 2024, 10:21:54 am
content: Quote from: glenenglish on July 14, 2024, 03:36:24 amIt is not. The reason is that you'll also need to know what each bit and/or instruction means. There is no use in looking at a bundle of data and franticly going through documentation which describes each bit and/or instruction. Errors are easely made. A much better way is to simply print the data as text where the meaning of bits is translated into a human readable form. People don't program in assembly nowadays and scripted languages are also being used increasingly.

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 10:33:55 am
content: Quote from: glenenglish on July 14, 2024, 01:48:06 amNot necessarily: it depends on the hardware and software architecture.I have used printf/scanf over USB to allow a PC to control an instrument. The instrument was operating inhardrealtime, using software to count edges in two 62.5Mb/s serial bitstreams, plus front-panel i/o. Counting the edges took 99% of a processor's cycles; edges were never missed - proven by design and demonstrated by testing.

author: brucehoult
author_rank: Super Contributor
timestamp: July 14, 2024, 10:34:36 am
content: Quote from: nctnico on July 14, 2024, 10:21:54 amI am, at this very moment (or I was 2 minutes ago), programming in assembly language, and am in fact writing an interpreter for a different assembly language (ok, machine code) that I'm making up as I go along, with the intention of being the most compact and hopefully fastest executing (non JIT) bytecode with the smallest interpreter ever made. Some combination of those -- but hopefully all. Well, we shall see how that works out :-)Targeting RV32EC with 2k RAM and with a REPL. Inspired by Transputer, SWEET16, and others.I claim that I am "people", though perhaps a weird one.

author: pardo-bsso
author_rank: Frequent Contributor
timestamp: July 14, 2024, 02:31:12 pm
content: Quote from: glenenglish on July 13, 2024, 07:53:19 pmHow would that work?(I'm assuming the program won't survive a powercycle)

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 03:47:02 pm
content: Quote from: brucehoult on July 14, 2024, 10:34:36 amWeirdness, on its own, is neither good nor badXMOS have a RISC-V processor now. Use a mailinator email address (ugh!) to download the whitepaper fromhttps://www.xmos.com/using-risc-v-to-define-socs-in-software/QuoteThat gives you 16 TransputersI presume, but can't be bothered to check, you can run xC on them - which would give you OccamNo reason you couldn't implement your interpreter in that processor, and invoke the appropriate comms mechanismsSounds fun!

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 14, 2024, 07:55:48 pm
content: Quote from: pardo-bsso on July 14, 2024, 02:31:12 pmThere does not need to be a power cycle. The debugger loads the program into SRAM and executes from there. Flash memory isnt used in that case (some processors cannot execute from RAM, that's true also, and some have limitations on where the interrupt vectors can go so you might need to nail down some things) .Most STM32 can do this . at 1 MHz JTAG clock you can load up 100kbytes of code in about 1 second and its ready by the time you blink, rather than an incremental/segment partial flash erase and reprogram which takes a few seconds.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 14, 2024, 07:59:38 pm
content: Quote from: nctnico on July 14, 2024, 10:21:54 amNot if you use a modern debugger that permits single stepping of source code, or single stepping of the disassembly, wih modification of registers and CC while the CPU is halted...... there's no need to remember the bits and opcodes anymore, unless you want to, like Bruce, which is fine.I think various ops here have different definitions on what 'embedded' is for them. That's what I am seeing with this excellent discussion.

author: nctnico
author_rank: Super Contributor
timestamp: July 14, 2024, 08:16:03 pm
content: Quote from: glenenglish on July 14, 2024, 07:59:38 pmI'm not writing about opcodes but assembler instructions and what bits mean in peripheral registers. Another problem is that you can only step through C / C++ code which isn't optimised OR has been optimised to allow debugging (gcc flags -O0 or -Og). Stepping through optimised code is hit & miss (with mostly miss) as there is no longer a 1 to 1 relation between the machine code and source code.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 14, 2024, 08:20:30 pm
content: Quote from: nctnico on July 14, 2024, 08:16:03 pmHelpful debuggers have all the peripheral registers graphically illustrated, even Microchip AVR studio does that. No need to remember anything, except of course you will have the datasheet for the device open on another monitor as you trying and figure out, for example what the counter-timer bits must be for some bespoke config , some of these timer peripherals are tricky.Yes, the program needs to be able to run with -Og or -O0 otherwise single stepping is a nightmare.Something though which is more difficult with single stepping, is when you have a microprocessor with a deep / reordering pipeline, This is fairly uncommon with small micros.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2024, 08:31:07 pm
content: QuoteI don't think there needs to be. When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitelynotdebugging the compiler of microprocessor.Source debugging on the hardware shows that the branches in the code you expected it to take are actually taken. That variables have the value you expect them to have, that this function is called, etc. For that the source-level on-device debugging is fine, in not actually perfect for the task. The only real question is "what are you looking for"?(If this - the source-level ignore machine code thing - wasn't the case then you couldn't run unit tests.)

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 08:35:08 pm
content: Quote from: nctnico on July 14, 2024, 08:16:03 pmOne of the problems with C/C++ is determining whether you have given the compiler the correct instructions (i.e. source code plus arguments/flags) to generate the code you want. Just looking at disassembled optimised machine code is too difficult to understand within a reasonable timeframe.Stepping through at the machine code level can speed up that understanding. Then all you have to do is work out why the compiler created that spaghetti, and how you have to change your instructions to the compiler. That is left as an exercise for the studentOTOH, I know of a toolchain thatstartswith theoptimisedassembler, and works back to define how many cycles (i.e. min/max) it takes to get fromheretothere. The route isn't always obvious!None of that helps with the interaction between multiple threads/processes. Back in 68020 days, you could capture the bus activity, and (painfully!) work out what was happening; no longer. Yes, I once had to do that to find the fault inside a third-party RTOS; ignoring irrelevant/unexecuted prefetched instructions was tedious in the extreme.

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 08:41:43 pm
content: Quote from: PlainName on July 14, 2024, 08:31:07 pmOne difficulty is to understand why the processor isn't doing what you thought you told it to do. That's a real problem with C/C++ code, where nctnico's statement is correct. A second difficulty is to understand why.A good toolset will minimise problems in the first place, and help to spot and diagnose problems. A bad toolset will start by blaming the developer for not understanding the tools.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2024, 09:02:23 pm
content: Quote from: tggzzz on July 14, 2024, 08:41:43 pmHence the 'yet' in my comment. In order of probability, the code may not be working because 1) you wrote it wrong, 2) you wrote it correctly to do the wrong thing, 3) there's a compiler issue, 4) there's a processor issue. It is exceedingly rare that you get past 2 IME, but it sometimes happens and you find it out by the source level stepping going wrong.

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 09:35:30 pm
content: Quote from: PlainName on July 14, 2024, 09:02:23 pmYou are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.Examples:C++ templates. The committee refused to believe the complexity until Erwin Unruh rubbed their noses in the problem by giving them a simple, correct, valid, conforming program that cannever finish being compiled. Why not? Because the compilation process itself generates the (infinite) sequence of prime numbers. If the committee can't understand what they have created, ordinary engineers employed to "drain the swamp" have no chanceHans Boehm had to forcibly "remind" people that you couldn't - by design - create a threading library in C. As a consequence the committeefinallyrealised C needed a defined memory model. (Finally, since everybody else had known it forseveral decades!)Then there are the compiler/linker flags, which are compiler dependent and whose actions change over time. Hence the requirement that "this code is compiled and linked using tool X version a.b.c" - which became obsolete just after the code was first delivered a decade or more ago.

author: PlainName
author_rank: Super Contributor
timestamp: July 14, 2024, 09:50:41 pm
content: QuoteNo, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.As I noted previously, if single-stepping is useless without intimate knowledge of tiniest part of your toolsetandthe intricacies of the chip then unit tests would be useless. You need to concentrate on the particular abstraction level that you''re debugging at, and at source level I reckon we can hazard a good guess as to what that is.

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 10:07:07 pm
content: Quote from: PlainName on July 14, 2024, 09:50:41 pmThere's some value to that statement in that context.But the point is that the tools have become so complex that they are part of the problem. A change in an apparently unrelated part of the source code can change what you have already "debugged".

author: Postal2
author_rank: Frequent Contributor
timestamp: July 14, 2024, 10:36:53 pm
content: Quote from: PlainName on July 14, 2024, 08:44:19 amI can give you an idea. Make an universal touchscreen chip converter (i2c). This is a big problem when you have a screen with a touch where the touch can't connect anywhere.I have more ideas, but they are little complex.

author: nctnico
author_rank: Super Contributor
timestamp: July 14, 2024, 10:40:22 pm
content: Quote from: PlainName on July 14, 2024, 09:50:41 pmAnd despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

author: tggzzz
author_rank: Super Contributor
timestamp: July 14, 2024, 10:55:50 pm
content: Quote from: nctnico on July 14, 2024, 10:40:22 pmThe defined behaviour has a tendency to change over time, as the compilers become ever more "capable".Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".

author: nctnico
author_rank: Super Contributor
timestamp: July 15, 2024, 12:15:50 pm
content: Quote from: tggzzz on July 14, 2024, 10:55:50 pmI don't think that is 100% true. For example: there is some ambiguity where it comes to how the << and >> work in C. The way compiler makers have implemented these is a way which works as expected. Nobody with a sane mind is going to change this behaviour because it would break a huge amount of code.Most problems with incompatibilities I've encountered are with header files, pre-defined values and things which used to threw a warning becoming an error.

author: RoGeorge
author_rank: Super Contributor
timestamp: July 15, 2024, 12:30:19 pm
content: Try stepping away from the Arduino IDE, and use C and Makefile from the command line, or use some other IDE, like Eclipse.For the hardcore, learn about gdb, and try to make use of "debugWIRE" with an "Arduino UNO" or with an "Arduino nano" board. Many AVR microcontrollers can use debugWire as a hardware debugger, programmer, step-by -step execution, memory and peripheral inspection, breakpoints, etc.Did that setup once for an ATtiny13 microcontroller, and a generic CH340G USB-to-serial adapter as a programmer/hardware debugger:https://www.eevblog.com/forum/microcontrollers/avr-(arduino)-linux-debug/(which reminds me I still didn't post the clean instructions of how to set all that)

author: tggzzz
author_rank: Super Contributor
timestamp: July 15, 2024, 12:52:52 pm
content: Quote from: nctnico on July 15, 2024, 12:15:50 pmBreaking working code/applications is unpleasant wherever it occurs. It is a traditional problem with C and C++. Other languages cause zero or far fewer problems in that way.Previous/historic code presumed the language definition (and language ambiguity) in force at that time. Not unreasonable.Compilers have, over the years, become more aggressive at interpreting the language specification to the compiler's advantage, in the name of gaining minor performance enhancements. That breaks some historic code which used to work, sometimes to the point of omitting to generate the object code! The traditional response is to blame the developer, implicitly for not taking account of what might happen in the future. Personally I think that is disreputable; the tools are the problem. That has been a problem with somewell-known benchmarks; later versions of the compiler have run the benchmarks "infinitely" fast. Some compilers long before VolksWagen and Toyota - detected they are compiling the benchmark, and emitted unjustifiably optimised code .

author: westfw
author_rank: Super Contributor
timestamp: July 15, 2024, 06:53:16 pm
content: QuoteIt is important to learn about the Heisenberg Uncertainty Principle as it applies to software debugging. (well, WE called some problems "Heisenbugs"!)That is, you need to understand in what what ways your debugging process (whatever it is) affects the behavior of your code.Also, different types of debugging imply that different tools are useful.Single-stepping through source code mat be great for debugging your C program flow, especially if you've made some stupid mistake that you're not seeing in the source (I mean, mistakes that you SHOULD be able to see in the source, but...)But of course it's pretty useless for debugging timing-related bugs in your bitbang code.And if you think you're chasing a compiler bug, you'll need other tools.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 15, 2024, 09:03:12 pm
content: Good suggestion RoGeorge.could- stay with simple platform - and learn a bit of the background - make, gdbI think learning the background stuff is important, because once you are in the land of pre-canned GUI environments, like Eclipse, VS code, if something doesnt work in the build sequence, if you dont understand the basics and what is going on in the background, you'll find it time consuming to solve.

author: Johnny B Good
author_rank: Frequent Contributor
timestamp: July 21, 2024, 04:29:05 am
content: I've been drawn to this topic thread on account of a gremlin (most definitelynota bug!) that I've been struggling to resolve over the past few weeks while I was tuning the temperature control algorithm of a rubidium oscillator's base plate to cope with sub 10 degree ambient temperatures.I had finally figured out how to get it to cope with ambient temperatures to within a degree of its theoretical 33 deg maximum (a three degree delta between the base plate's 36.05 deg set point and maximum ambient) and now wanted to verify that it would still cope with 4 deg as it had seemingly done during the winter months test runs in my outside workshop.Unfortunately this is the wrong time of the year to be using the outside workshop as a low temperature test chamber so I had the bright idea to use my fridge as an 'all seasons' low temperature test chamber. The initial test runs exceeded my wildest dreams of milli - Kelvin stability right down to 10 degrees. Unfortunately, my dreams were shattered just as the temperature had dropped to 8 degrees when it suddenly took a deep dive into endless cycles of under/over shooting, only managing to return to a stable set point once the temperature had risen back to 10 degrees.At first I assumed I hadn't scaled the corrective responses at low fan intake temperatures enough to avoid this instability when dealing with a thermal time constant in the region of some 35 to 40 seconds or so. I had been under the impression that the Atmel M328p used on Nano and its slightly larger cousin used on the UNO could do no wrong and therefore it was down to me for not properly considering the increased instability with fan cooling as the sole means of temperature control at these low temperatures.Either that, or I had simply reached the limits of just what was possible and was now "Asking for The Moon", despite my observing overshoot excursions with the fan just barely ticking over, rather than being brought to a total standstill as in my initial versions of my program, suggesting otherwise. Indeed it was my curiosity over just how far I could raise this minimum tick-over drive level to before it would prevent overshoot and start reducing the temperature ever so slightly under such low ambient temperature conditions.To this end, I simply over-rode the fan control with a set of static fanDrive test values, trying to home in on the 'magic number' and it was when I had reached a value of 255 that I spotted a rapid temperature drop rather than the expected small variation in temperature. Sure enough, trying out adjacent values (254 and 256) did give the expected resulting slight change of temperature drift rate and going back to 255 to retest this peculiar behavior did confirm that with pins D9 and D10 configured to output 10 bit PWM, that one specific 'gremlin value' would mimic the effect of generating a value of 1023.Later testing with my bread boarded "Flight Spare" followed by writing a small test program to exercise the UNO's 10 bit PWM performance confirmed this to be a shortcoming in these micro controllers (or possibly some cockup in the Arduino's compiler or a built in library function).With this 'gremlin' only appearing at a rather suspicious 100% for the 8 bit case, I did begin to wonder whether this gremlin repeated itself over the next two multiples of 256 (511 and 767). In this case I resorted to the age old remedy of "Shoot first, ask questions later" approach, using the simple work around of testing for fanDrive for values of 255 and substituting this gremlin value with 254 on every such occurrence.As later testing proved, 255 was the only gremlin value over the full 10 bit range so proved an almost perfect solution until I closed a backdoor I'd forgotten I'd left open in the display update code for the gremlin to have its wicked way. More extreme low temperature testing had hinted at a shadow of its presence elsewhere in my code. It would have been a more obvious presence but for the fact that I'd added a test to override whatever fanDrive value was going to be used with a fanDrive level of the 215 tickover value whenever the base plate temperature undershot the set point by 25mK or more.The whole point of the above description was to highlight the fact that not all bugs can be analysed using debugging tools (or even logic analysers for that matter). More accurately put, debugging tools cant handle gremlins. In such cases, as a few in this thread have already pointed out, careful analysis of the code and the behavior of the hardware under its control may be the only way to resolve such gremlins.If anyone is interested in getting to the bottom of this particular gremlin, I've attached the gremlin test code I'd run on the UNO (should work just the same on a nano). I'd be interested in knowing whether it's just clones that suffer this 10 bit PWM gremlin or whether it also afflicts genuine Arduino boards as well.visit <https://nerdytechy.com/how-to-change-the-pwm-frequency-of-arduino/#Pins_D9_and_D10_Timer_1_10_bit> for the original reference I'd used to set the timer 1 /D9 /D10 PWM to 10 bit mode.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 21, 2024, 04:54:15 am
content: Hi JBGthis should really go in its own thread...... suggest move it there, since this will generate alot of responses ....-----------------------------------------------------------I've never encountered badness in ye-olde AVR (1999-2008 stuff) and given the value, it's likely a cock up in the avr library,.I''ve only ever used peripherals directly, never used arduino for anything more than a doorbell.Might be a good time to manipulate the port directly. The AVR timers are quite simple.For what you're doing, I'd make everything signed so you can never have a rollover F.U.and watch out for non-monotonic sensors !...I assume you are using the PWM libraryhttps://github.com/atmelino/Arduino/blob/master/libraries/PWM/utility/ATimerDefs.cppwatch out there is void pwmWrite(uint8_t pin, uint8_t val) and also there is void pwmWriteHR(uint8_t pin, uint16_t val)in pwmWriteHR(), somehow verify thatif(td.Is16Bit)is being executed correctly., anyway, suggest another thread. cheers

author: PlainName
author_rank: Super Contributor
timestamp: July 21, 2024, 07:29:09 am
content: Quote from: Johnny B Good on July 21, 2024, 04:29:05 amI am not an Arduino guru, but a red flag to me in that code is that you're using a float to store 1033 values that you can then average. A float can represent huge numbers, but it only uses 32 bits to do so, and taking away the sign and scaling info you're left with 23 bits of actual number. So the resolution of a float is very much poorer than a single 32-bit int.I would think about placing a bet that your use of a float like this is introducing the 255 magic value (and, indeed, '255' being a magic value fires off multiple bells).I think, if you want to average that way then you should use adoublelong, which is a 64-bit variable. It will also be quite a bit faster to use.But... assuming that is the answer to your problem, 1033 values is a lot (and oddly precise), and I would think about alternative ways to achieve the end of smoothing that out. Perhaps, through dropping the float as store, an appropriate number of values to average would be fewer.Edit: fixed fingers not catching up with brain error (typed double, meant long). Also it seems the Arduino int is 16-bit whereas I assumed 32-bit. Doesn't make any practical difference though.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 21, 2024, 09:18:22 am
content: I'd be averaging with an integer boxcar filter. no precision issues. maybe use int32 for the PID filter. (decide on a decimal point location perhaps. Also if you do=do float to int, be sure to add 0.5f for all positive float values so that you round to nearest instead of round down

author: brucehoult
author_rank: Super Contributor
timestamp: July 21, 2024, 10:05:43 am
content: If I wanted to smooth values over about 1000 (equal) time periods then I'd do like this...Code:// initint32_t smoothed = first_sample << 10;// then each time periodsmoothed = smoothed - ((smoothed + 512) >> 10) + new_sample;Obviously you need 10 more bits for "smoothed" than for the samples, so if the samples are in 0..255 or 0..1023 you'll need "smoothed" to be 24 bits, which you can do if you're hand-coding it, or just use the 32 bit integers the compiler provides.You could use "smoothed" directly as the control variable for your PID, or maybe shift it right a few places to bring it into range of a 16 bit integer.About a dozen years ago I used an Uno to control my home heating using a 2400W oil column heater. I used a thermistor to read the temperature, did an ADC 100 times a second, and smoothed over 30 seconds. This gave me a nice 0.01 C resolution on the temperature that moved very smoothly, without noise, as the room changed temperature. Obviously the accuracy wasn't to 0.01 C, but it gave a very nice indication of the direction and speed of the room changing temperature.I made a decision whether the heater should be on or off every 30 seconds, using a similar smoothing process to integrate an approximation of how much heat was currently stored in the heater, adding to the stored heat every 30 seconds and assuming the heater decayed to room temperature with a ~15 minute time constant (i.e. assume 63% (?) of the remaining stored energy is transferred to the room every 15 minutes)It worked really really well, with the (self measured) temperature variation in normal use about 0.02 C. A transient such as starting or stopping cooking generally gave about a maximum 0.2 C change in temperature, brought smoothly back to the set value within 20 minutes. I could also make reasonable changes to the set temperature (±0.5 or ±1 C, say) and the temperature would smoothly converge on the new temperature in 20-30 minutes, with just a little overshoot (like 0.2 C)

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 21, 2024, 10:20:05 am
content: agreed with Bruce there if you want to smooth 1000 samples. Boxcar is accurate and super cheap, but needs memory... other option would be carefully precise single pole IIR filter. you might get limit cycles if you look hard enough.just be careful not to try and use any more precision than your control system provides for, specifically relating to non-monotonicity of sensors and actuators. They can cause loops and cycles.Bruce what did it do if you repeatidly opened a door every exactly 30 seconds ?

author: ozcar
author_rank: Frequent Contributor
timestamp: July 21, 2024, 11:23:42 am
content: Quote from: PlainName on July 21, 2024, 07:29:09 amThe documented maximum value for analogWrite is 255.Some random has mislead Johnny into believing that a simple trick will allow 10 bit resolution, with values up to 1023, by messing with the timer registers to invoke 10-bit rather than 8-bit PWM (obviously not possible for 8-bit timers).From what I can see, this happens to work for all values from 0 to 1023,except 255.Source:https://github.com/arduino/ArduinoCore-avr/blob/master/cores/arduino/wiring_analog.cCode:void analogWrite(uint8_t pin, int val){...if (val == 0){digitalWrite(pin, LOW);}else if (val == 255){digitalWrite(pin, HIGH);}else{...... set the appropriate timer OCR register from val -... for a 16bit timer it happens to write to the OCR as a 16-bit field, rather than only to... the low 8-bit half - egOCR1A = val; // set pwm duty... rather thanOCR1AL = val;...}}

author: brucehoult
author_rank: Super Contributor
timestamp: July 21, 2024, 11:26:26 am
content: Quote from: glenenglish on July 21, 2024, 10:20:05 amI didn't have enough doors to do that.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 21, 2024, 02:55:44 pm
content: Quote from: Johnny B Good on July 21, 2024, 04:29:05 amSure they can. Debugging tools are just tools you can use to view things, like a microscope. The trick is not to know how to use a microscope. The trick is to know what to look at. The same with the debugger. You could have stepped through the code to see how the code sets the value of the register. Or, you could have used the debugger to inspect the register value after you have set it. This would immediately reveal the nature of the problem.Moreover, you could have looked at the same things without a debugger. For example, you could have looked at the source code to see how it sets the register. Or you could have inspected the disassembly. Or you could re-write the code and set the value of the register directly (instead of using Arduino function) and then observe if this fixes the problem.

author: Johnny B Good
author_rank: Frequent Contributor
timestamp: July 21, 2024, 07:49:38 pm
content: Quote from: glenenglish on July 21, 2024, 04:54:15 amI plan to make a new thread describing my temperature stabilised and barometrically compensated LPRO 101 based Rubidium Frequency standard one of these days when I've finally completed this project.Now, at last, I really am close to the finish merely requiring that I "Dot the eyes and cross the tees" so to speak before embarking on a MK II version based on an SA.22C I'd purchased just over two years ago now.I want to put all the lessons I'd learned during the previous three years to good use to create an even better RFS.However, I first have to resume my ZED-9T based MK III GPSDO project to provide a more phase stable GNSS based frequency standard than the NEO -M8T based MK II standard I'm having to rely on for syntonising my Rubidium based frequency standards to. The long term (days to decades) frequency stability with a PLL based GPSDO is never in doubt but since the only effective way to syntonise a rubidium based frequency standard to such high precision frequency references is to compare phase drift over hours to days long runs, it helps to minimise the effect of the minute to minute variations in electron density in the ionosphere (space weather) - the whole point of blowing 250 quid on that Sparkfun ZED-9T module.In short, I don't have any idea on when I'll get around to creating such a topic thread.TBH, I was so astonished by discovering such a gremlin in Robert Brown's simple to implement 10 bit PWM code snippets, I felt impelled to find a topic thread where it wouldn't be unreasonable to announce 'My Discovery' and perhaps elicit more considered responses as to what the actual cause was likely to be. After all, this was an ideal example case where debugging and logic analyser tools would not be able to retrieve the situation.Manipulating AVR timers might be simple for those who can peruse the whole 200 odd pages of the "Datasheet" without their eyes glazing over but for me that's a bit beyond my pay grade.When I'm searching for solutions, that meet my less ambitious requirements, ready made solutions like Robert Brown's easy to implement 10 bit PWM code snippets easily satisfied my demand to gain just the extra two bit's worth of fan drive resolution improvement I had craved.Regarding the use of signed integers, that's the default for int anyway. Besides which, it's very rare for me to feel the need to use unsigned integers even for loop counters (I've not had to use counts beyond the 64K mark thus far). I guess this advice is more useful when programming down to the 'bare metal' which is something I've managed to avoid thus far.I've been very mindful of 'divide by zero' errors and other 'close call' situations and used the constrain instruction where such possibilities exist. I've been well aware the monotonicity issues in ready made sensors so have steered clear of those which offer "10mV/deg C" and similar such claims. There's nothing more monotonic by way of a temperature sensor comprised of a bead NTC with its "other" resistor located in the same drilling in a quarter inch thick heat spreader plate paired up with an ADS1115 16 bit delta sigma ADC. As for the BMP280 I'm using to gather plenum temperature and barometric compensation data, that seems to be as good as it gets in this regard.As for your assuming I'm using that PWM library, in view of its 16 bit timer support, I don't think so. Since I don't have an #include reference to any PWM library files, I assume I must just be using the bog standard built in PWM support for the analogWrite commands.By and large, when it comes to such "simple" (anything but!) temperature control software projects like this, I tend to take a pragmatic approach and steer clear of bare metal programming that requires an understanding of register functions in excruciating detail, preferring to make good use of others' published ready made solutions.Anyway, thank you for taking the time to respond to my post. It's not as if this mix of software and hardware is new to me (I did this sort of stuff with a Z80 based Transam Tuscan S100 bus kit computer and a set of bare Philips solenoid operated data cassette drives some 4 decades back). Sadly, I'm just a little too long in the tooth for my worn out brain cells to get a grip on any bare metal programming of the Atmel AVR. KISS (in my case, Keeping It Stupidly Simple) had been the only viable way I could get this project done and dusted.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 21, 2024, 08:37:46 pm
content: Hi JBGPlease make a new forum thread underhttps://www.eevblog.com/forum/projects/that's much more appropriate for project orientated discussions,cheers