How to keep Interupts short or how to write good ISR's - Page 1
Forum section: electronics / general

author: DVX
author_rank: Contributor
timestamp: October 25, 2016, 08:30:38 pm
content: This is a newbie coding question to find out the best way to code ISR's.I am having problems balancing the size of my interupt code and accurately reading external inputs without missing input data. This is for hobby use and my current project is a TFT clock with an IC2 connected DS3231 RTC, a GPS module is used to read time and date and periodically write this data to the DS3231. The MCU is a PIC32 and my code is all in C. The code / hardware currently works but my interupt to read the GPS is long. My question isn't so much for this project but more general one for how to balance the size of code in an Interupt and how to pass data via variables to the main code so no input data is missed. Currently GPS parcing is in the ISR, making it many lines long, it does work reliably but I would like to learn how to do this properly and perform the parsing in the main code. I had a similiar problem when reading a rotary encoder in a previous PIC18 project, for that I read the level input changes via external interupt pins and checked the grey pattern in the IRS, again making the IRS a long one, again it did work reliably. Are there any good publications or online guides for writing ISR's, passing data to the main program and how to make sure the main program does not miss the data being presented by the ISR?

author: suicidaleggroll
author_rank: Super Contributor
timestamp: October 25, 2016, 09:00:57 pm
content: I generally just use a shared circular buffer. Allocate it to 256 bytes and give it an 8-bit read and 8-bit write pointer (you can of course use other sizes, but 256 is convenient because it lets you use 1-byte addressing without having to worry about bounds checking.Your ISR simply writes the incoming byte to the buffer at the write location, and then increments the write pointer. Your main loop compares the write and read pointers, if they're the same then the buffer is empty, otherwise it reads from the read location over and over again until it catches up with the current write location.You just need to make sure your main loop runs often enough that you can't fill up the fifo between passes.eg:Code:volatile uint8_t uart_fifo;volatile uint8_t uart_wri = 0;volatile uint8_t uart_rdi = 0;int main(void) {while (1) {while (uart_rdi != uart_wri) process(uart_fifo[uart_rdi++]);sleep();}}void ISR(void) {uart_fifo[uart_wri++] = byte;}

author: nctnico
author_rank: Super Contributor
timestamp: October 25, 2016, 09:24:37 pm
content: When it comes to interrupts there is always a trade-off between the priorities of other interrupts. Always handing off data to the primary thread (main loop) isn't a good default. Transferring data adds overhead (buffering, keeping track of pointers and semaphores/mutexes to signal the data can be accessed safely) and you will always need X time to process the incoming data whether that processing is done inside an interrupt or not. Also try to see the interrupt controller as a hardware RTOS which runs several tasks (threads) semi-parallel.I think a good rule of thumb is to use the main loop for things which aren't time critical and has the lowest priority and do high priority things inside the interrupt. If for example the communication with the device isn't time critical (for example commands received over the UART) it makes sense to use a thread safe FIFO buffer between the UART interrupt and do the processing of the commands in the main loop. With signal processing tasks it is the other way around. These usually have the highest priority so it makes perfect sense to do all the processing inside the interrupt.By setting the priorities in the interrupt controller you can often have nested interrupts so you have fine grained control over what should take precedence over something else. But remember: there is only Y processing time to do X tasks. Buffering (and handing off data to the main loop) can be used to smear bursts of incoming data over a longer period for processing but this makes the response time uncertain on a system with a lot of tasks on it's plate.

author: dannyf
author_rank: Super Contributor
timestamp: October 25, 2016, 10:04:47 pm
content: "My question isn't so much for this project but more general one for how to balance the size of code in an Interupt and how to pass data via variables to the main code so no input data is missed"Having a long isr isn't always a bad thing: there are cases where this is preferred.With that said, the typical approach is to have a short isr. In cases like yours, I typically use a call back function . this function is called bybthe isr and does two things:1. It sends or received one char at a time.2. It detects the end of a transmission and sets a flag. This flag is read in the main loop and triggers appropriate processing.Let's say that you want to send a string to the gps through a UART. The initialization code will load the address of the string, enable transmission isr, and load the first char into the UART. And then exits.The UART hardware will transmit that char. And the end of it (or even concurrently if there is a buffer), the isr is invoked. The call back function will see if the end of the string has been reached. If not, it advances on to the next char and loads it into the UART. If the end of the string has been reached, it sets the flag.The main loop can detect the flag and do something about it: for example, it can serve as a uartisready flag and starts the next transmission.I have done this on a range of Chios, from lowly avr and pic to cm4 chips. Works like a charm every time.It can be used on other slow peripherals, like i2c.

author: DVX
author_rank: Contributor
timestamp: October 25, 2016, 10:41:07 pm
content: Thanks suicidaleggroll & nctnico, I have made a quick search for 'shared circular buffer' and there are good examples for this technique, just knowing what terms to Google helps. In your example suicidaleggroll I assume there would be need to check for the buffer pointers and when they reach 256 to reset them to provide the effect of the circular buffer?I have a sys tick running as a 2nd interupt (every 50ms) with a higher ISR priority and this provides timming for the main loop. By toggling a port pin high and low at the start of the main loop and high and low at the end, then scoping the pin, the duty time for the main loop has been measured and is currently low, so for most of the time it is waiting for the next sys tick. I can understand you feedback nctnico as this is forming the basis of a simple RTOS or scheduler, both items I have read about but not used before.What would be useful to know is the reason for passing data from the ISR using volitile variables, as I currently use an array?Dannyf, one of the reasons for using a GPS module was to sync the RTC from the GPS making the RTC time second accurate to the GPS second time. If I understand your feedback correctly, I could write the whole GPS string to a buffer and when the end string CHR is detected then process the string in the main loop? My concern for this method is that it would work for most GPS data but the second time update would be inaccurate as the second information in the buffer would be somewhere in the 300-400 chr string and during parsing this could miss a second update. I currently get fast second updates from the GPS string by parsing in the ISR. The GPS time including seconds are then displayed in the main loop on a TFT, the seconds update is OK and doesn't break the secound count. A circular buffer may still be fast enough to resolve the GPS second update as my main loop loading is currently low, I will plan to update my code and try this.Thanks everyone for your help

author: dannyf
author_rank: Super Contributor
timestamp: October 25, 2016, 10:55:00 pm
content: Quotei'm not sure how the parsing time would vary that greatly between doing it in the isr vs. outside.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: October 25, 2016, 11:17:32 pm
content: Quote from: DVX on October 25, 2016, 10:41:07 pmWith anyothersize buffer, yes, but with 256 you can use 1-byte pointers which will naturally roll over at 256.Quote from: DVX on October 25, 2016, 10:41:07 pm"Volatile" tells the compiler that the contents of that variable can change at any time, which prevents it from doing some optimization trickery that could then cause problems if an ISR fires and suddenly changes one of the values.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: October 26, 2016, 01:37:10 am
content: I like to make them re-entrant, at least for suitable use cases.Example: periodic timer interrupt, in a system where the main() task can be interrupted for long stretches without trouble.Suppose you need to perform a variety of services on a regular basis: selecting and sampling ADCs (which themselves might be read by a different interrupt), polling user input, updating a display, managing a serial connection (rx/tx), etc.Most of these tasks can be (or must be) sequenced, so running them from a timer is a natural choice. But not all have to run every time. So you might divide the clock down. Say ~10kHz for ADC samples, ~1kHz for serial updates (1ms is one UART serial frame at ~10kbaud), ~100Hz for user polling and display (HID) update, ~1Hz for slow management functions (reading temperatures? scheduling background tasks?).Simplest way to sequence those, is to put in a static counter variable, which increments on each interrupt. When it hits 10, do the serial update, and increment a different counter. When that counter hits 10, do the HID stuff and increment another counter. And when that counter hits 100, do the slow stuff.Well. What happens when the lucky 10-10-100 rolls around? That management task could be pretty awful; what if it takes more than 100us to complete? It'll get interrupted by another interrupt!But it's already in that same interrupt!...So?If you've taken care that the interrupted section of code does not depend on static variables that are changed by other sections of the interrupt (i.e., their data are isolated), then itdoesn't matterthat you've got two (or maybe even several) interrupts piled on the stack!All that matters is that one task finishes before it's called again (which can be prevented by setting a static "hey, I'm still busy, don't jump here" flag, assuming it's an optional task).Actually, not even that: as long as the stack doesn't underflow (overwriting data space), and the stack of interrupts is eventually able to finish, you're still fine. (Playing thingsthistight would be rather risky. It would be wise to periodically read the stack pointer value, calculate how much stack remains, and take different actions if it's filling up.)Even more specifically, you can have their data overlap (i.e., reading data from public fields that are constantly being updated by the fast interrupt), as long as you preserve atomicity. Standard multitasking rules apply (atomic transfers, buffered transfers, semaphores, mutexes... deadlock..).The biggest danger is twofold:1. If you don't observe atomicity, you will get errors ranging from missing bits (e.g., an interrupt occured while a variable was being updated, and the interrupt reads the half-and-half value, which is now nonsense) to sneaky hair-pulling (it only happens when so-and-so delays happen to line up, tongue at the right angle, etc...), and worse.2. If you don't verify that the interrupts will complete in determined time, or at least put in checks and balances to prevent them stacking too high, then you WILL blow the stack, and the CPU will most likely wander off into the weeds and crash or freeze or reboot. Needless to say, this ain't something you're gonna do in Arduino!Tim

author: ajb
author_rank: Super Contributor
timestamp: October 26, 2016, 12:16:35 pm
content: Lots of good advice about ISRs so far, the other thing to keep in mind is that you may be able to take advantage of hardware features to handle some tasks that would otherwise require time-sensitive software. Most commonly this means using DMA to shuffle data, such as between your serial port and your buffer. Rather than having your serial port receive a bit, then call an ISR that has to jump through some logic so that it can take a byte from the rx data register and shove it into a buffer, you could set up the USART RX event to trigger a DMA transfer straight from the rx data register into the buffer with no software interaction at all. The DMA controller handles indexing the buffer pointer and everything, and can usually be configured to transfer a certain number of bytes or to run continuously using a circular buffer of specified size.Aside from DMA, you might have things like a USART character match facility that would allow you to an execute an interrupt when a particular character is received--usually used for address matching on buses, but if you are receiving messages that always start or end with a particular character, you could use this to start/end DMA transactions or set flags that tell the main loop that the start or end of a message has been received, that sort of thing. Of course these sorts of things vary from chip-to-chip and so may not be worthwhile if you need portability (DMA is pretty common and works more or less the same way everywhere), but it's good to keep an eye out for them where they make your life easier.

author: John Coloccia
author_rank: Super Contributor
timestamp: October 26, 2016, 02:57:56 pm
content: Quote from: DVX on October 25, 2016, 10:41:07 pmI've never done any GPS work so I hope I'm not talking out my butt here, but I think I understand what you're trying to accomplish. It's not so much important to know what time it is RIGHT NOW. You need to know what time it WAS when the GPS data came in. The idea would be to time stamp the data with the real time clock so that you know what time it was when the data came in. Then you can process the data at your leisure and correct for the time difference with another real-time clock read in your main program.To each his own, but I would never depend on a byte rollover to manage a circular buffer. It adds a tiny bit of overhead, but I always maintain head and tail indexes (consumer always increments the head when it reads...ISR always increments the tail....everyone always knows if there's data in the buffer, how much, if an overrun has occurred or is about to occur, etc), and always manage them explicitly. It's just way too easy to botch up the world when you need to resize the buffer for some reason. Anyhow, a good optimizing compiler can probably figure it out and optimize for you regardless.

author: Kilrah
author_rank: Unknown
timestamp: October 26, 2016, 03:06:52 pm
content: Quote from: DVX on October 25, 2016, 10:41:07 pmYou cannot synchronise your clock to anything better than +/1 sec from the UART feed anyway. The output isn't synced with the second ticks, and what you get in the middle of your sentence is whatever the GPS had as a status before it started even generating that string.If you want to be precise you need to sample the 1PPS output of the GPS module, the edge of which will be exactly on the second tick.Which second it is you can know by adding one to the last received NMEA string.

author: Kalvin
author_rank: Super Contributor
timestamp: October 26, 2016, 03:43:07 pm
content: Implementing the parser inside the ISR is typically a bad design decision. Just capture the received characters in the ISR and let the main application poll the receiver buffer for the received characters. Implement the parser as a state machine so that the state machine will be called for each received character. The code will be portable and easily maintainable.If you really want to do it in the ISR, implement is as a state machine, run the state machine for each character and return the ISR as soon as you have handled the characters. Do not stay in the ISR waiting for the complete message string but process the characters as they are received. When the message has been received, signal the main program that the complete message has been received. The main task will then read the message from the buffer and process it.

author: dannyf
author_rank: Super Contributor
timestamp: October 26, 2016, 11:00:23 pm
content: This whole thing depends on implementation. If the message is indeed 300 chars long, at 10kbps, that is about 0.3 second off at max (ie, you can repeated query the GPS until a roll over), and 0.15 second on average.It may be perfectly acceptable for most of the applications out there.A better and more robust approach is to use 1pps.

author: DVX
author_rank: Contributor
timestamp: October 27, 2016, 10:09:05 am
content: Checking on the UART datarate this is typical for a GPS module and is 4800bps, the total string length is over 700 chr's but I only check the first 350. I will change my code to parse in the main loop and read the GPS pulse per second output pin to get more accurate timming for second transitions. The use of DMA could be interesting if processing time becomes more an issue as the project progresses. I also noticed the MCU ISR vector addressing might cause an issue if the interupt routine is too long and writes into the next vector space.Lots of good feedback on implementing ISR's.

author: dannyf
author_rank: Super Contributor
timestamp: October 27, 2016, 10:12:33 am
content: QuoteThat should rarely (=never) be an issue: it is literally an address pointing to the isr. so it is the same regardless of the "length" of the isr.

author: DVX
author_rank: Contributor
timestamp: October 27, 2016, 10:32:52 am
content: I understand for the vector pointing to the start address of the ISR, but if the MCU has many interrupts with each one having a different start vector and they are equally spaced out could long ISR not write into another ISR vector space?

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: October 27, 2016, 10:34:43 am
content: If you are receiving well-defined frames or packets, then instead of a circular buffer, another approach is two seperate buffers, and do a page swap between each reception.This can make handling overrun situations more graceful, as you can make it drop an entire frame rather than getting out of sync.

author: nctnico
author_rank: Super Contributor
timestamp: October 27, 2016, 11:04:43 am
content: Quote from: DVX on October 27, 2016, 10:32:52 amOn some MCUs this is the case because the CPU performs a jump to an addres in the IRQ vector space (Z80 IIRC) based on the type of interrupt but on an ARM Cortex-Mx controller the interrupt vector table is a table with pointers so you can't write any code there.

author: dannyf
author_rank: Super Contributor
timestamp: October 27, 2016, 11:25:20 am
content: "The use of DMA could be interesting "The use of DMA or interrupts for that matter doesn't fundamentally solve your problem.The root cause of your issue is that your only way to sync with the GPS is via a very slow mechanism. You don't know when the roll over takes place precisely. In this case, it could have take place as much as 0.7 seconds ago. The time you obtained from the GPS is old. And worse yet, unpredictable, even if you do continuous polling.That situation will not change with DMA or interrupts.Your only way to alleviate that is through 1pps or speed up the transmission.

author: dannyf
author_rank: Super Contributor
timestamp: October 27, 2016, 12:03:20 pm
content: "Your only way to alleviate that is through 1pps or speed up the transmission."On 2nd thought, I think there is a way, assuming you have a reasonably accurate local clock.At local time 0, you obtain a delayed GPS time n0. At time 1.1, you obtain a delayed GPS time n1... At some point, the delayed GPS time will roll over.That tells you that n0 has an offset from the real GPS time by ni - n0.It is basically a form of dithering. Or randomized sampling.

author: DVX
author_rank: Contributor
timestamp: October 27, 2016, 04:09:21 pm
content: "Your only way to alleviate that is through 1pps or speed up the transmission."This could be a solution as the GPS module does have IC2 and SPI interfaces with the default being UART at 4800bps. Checking the datasheet states 400kbps for IC2, this could then read the whole string in less than 1 sec and providing the parsing of the data string is fast enough the second rollover should be more accurate. This could then move the problem from being fast data collection to one requiring a fast parsing routine, currently both are an issue.Ultimately it's not too important if the second information is delayed compared realtime as long as the delay is constant and this can then be allowed for.

author: Kilrah
author_rank: Unknown
timestamp: October 27, 2016, 04:24:48 pm
content: The delay will not be constant since the whole thing isn't synchronized.BTW... why are you planning to constantly adjust your RTC? If you set it right once it can run freely for quite a while, all you'd want to do is monitor it.Setting it every second is guaranteed to give you discontinuities and jumps.

author: DVX
author_rank: Contributor
timestamp: October 27, 2016, 04:39:22 pm
content: I plan to update the RTC from the GPS every 24 hours same for day rollover so this will then auto correct for for DST. The plan is to make the time update, when it is read from the GPS as accurate as possible. It should be possible to achieve synchronised time even if delayed to realtime by using one pulse per second pin (1pps) on the GPS. The datasheet states..The 1pps pulse length (high state) is 200ms with rising edge is less than 30ns synchronized to GPS epoch.The correspondent UTC time message is generated and put into output FIFO 300ms after the PPS signal. The exact time between PPS and UTC time message delivery depends on message rate, message queue and communication baud rate.

author: John Coloccia
author_rank: Super Contributor
timestamp: October 28, 2016, 07:15:23 am
content: FWIW, DST is implemented at exactly 2:00am (going forward and back), not midnight.Anyhow, it sounds like you're going to need some clever logic if you want this to work reliably. The 300ms sounds reliable, but there's no way to know where the message lives in the FIFO. But if you DID know, then if you consider the technique I suggested of time stamping the message you can do something like this:in ISR:If (newMessage){GetTimestamp();if (haven'tHadAMessageInAWhile)MarkMessageGood();}in main program:If (MessageGood){time = message.gpsTime + (GetTimestamp() - message.timestamp) + 300ms;}And you should really be pretty close. The source of latency is the time from getting into the FIFO to the time you receive the first character, which shouldn't be a terribly long time. The gotcha is you don't know where the message lived in the FIFO, so you'd want to look for a message that happens to come in all on it's own, hence the idea of "haven'tHadAMessageInAWhile".Using the PPS is better, but if you want it accurate then you still need to get a timestamp when you receive the PPS, and then adjust when you process in your main code. (edit: actually just realized that you can keep track of the last time, and then add a second when the next PPS comes in so potentially no need to correct for anything if you can set it up right).

author: T3sl4co1l
author_rank: Super Contributor
timestamp: October 28, 2016, 12:51:15 pm
content: Quote from: John Coloccia on October 28, 2016, 07:15:23 amDST? Time zones? Don't forget, it gets worse... Tom provides just one example:Tim

author: suicidaleggroll
author_rank: Super Contributor
timestamp: October 28, 2016, 01:37:17 pm
content: DST? Why on earth would you have an instrument running on local time? Microsoft made that mistake long ago with Windows and it's been a horrible PITA ever since. Instruments and computers should run on UTC all the time. No time zones, no DST. If the user wants to see local time when they look at something, you convert it to their requested time zone on the fly when showing it to them, but all internal time stamps stay on UTC.

author: John Coloccia
author_rank: Super Contributor
timestamp: October 28, 2016, 02:22:44 pm
content: Quote from: suicidaleggroll on October 28, 2016, 01:37:17 pmI don't know, but he brought it up. I didn't try to read too much into it. He mentioned handling it at the new day, but that's the wrong time anyhow.

author: dannyf
author_rank: Super Contributor
timestamp: October 28, 2016, 02:42:36 pm
content: QuoteYou may want to think about the practical value such an approach may bring for your application.Without a 1pps, the uncertainty with the gps-pulled timing data is anywhere from 0.3 - 0.7 second. A terrible RTC can do 20ppm easily and a half decent one <5ppm for a dollar or so, all included. That means a drift of 0.2 second - 2 seconds over 24 hours.you have to judge whether that's worth the effort.

author: DVX
author_rank: Contributor
timestamp: October 28, 2016, 05:39:55 pm
content: This is turning out an interesting project. I have moved the parsing from the interrupt to the main code and there are now two parsing routines, one each for time and date. As time updating is more critical and the data is found earlier in the UART string I am currently checking the string every 0.25 sec, this produces reliable second counting on the TFT, the date is parsed every second and again is reliable (after filtering). As the project develops these will be checked less frequently, just before 24.00 and 02.00. My clock references are two different radio clocks synced to the UK atomic clock radio reference broadcast at 60Hz and website 'time.is' all three show the same time and roll over the same second together. My GPS code is within 1 second of these references, sometime about 0.5 sec fast or 0.5 slow from cold boot. To get this any better will probably follow the law of diminishing returns. Interestingly GPS time is UTC and should auto adjust as times zones are crossed, this then leaves some code to calculate local DST, I am writing a UK look up table with the dates for October and March changes for the next 30 years, after this time it's unlikely to matter too much.

author: John Coloccia
author_rank: Super Contributor
timestamp: October 28, 2016, 06:14:20 pm
content: Quote from: DVX on October 28, 2016, 05:39:55 pmJust FYI: GPS is not the same as UTC. GPS ALWAYS ticks forward at the same rate, and always increments...always always always. Leap seconds are not taken into account. GPS and UTC time have drifted apart, and are significantly different by a good 10 or 20 seconds. Might be more by now.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: October 28, 2016, 06:48:18 pm
content: Quote from: John Coloccia on October 28, 2016, 06:14:20 pm17 right now, #18 is coming in a couple of months.

author: DVX
author_rank: Contributor
timestamp: October 28, 2016, 06:58:44 pm
content: "Just FYI: GPS is not the same as UTC. GPS ALWAYS ticks forward at the same rate, and always increments...always always always. Leap seconds are not taken into account. GPS and UTC time have drifted apart, and are significantly different by a good 10 or 20 seconds. Might be more by now."Yes I did read about this, there must be more data in the GPS steam which can allow for some correction and the onboard ARM processor in the GPS module is most likley making some timing adjustment for seconds, my parsed results are never more than one second from my other atomic clock sync'ed time sources.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: October 28, 2016, 07:14:59 pm
content: Quote from: DVX on October 28, 2016, 06:58:44 pmYes, the GPS data messages from the satellites include the current leap second count so that receivers can apply the offset to output UTC. Many receivers (and computers) screw up the leap second increment though, so you might get a +/- 1 second error between Dec 31 00 UT and Jan 1 24 UT (a couple of months from now).

author: C
author_rank: Super Contributor
timestamp: October 28, 2016, 08:15:33 pm
content: If your clock is going mobile, You have GPS, so where is handled.Download a copy of tzdata and you get time zones.Gets updated when rules change.

author: IanB
author_rank: Super Contributor
timestamp: October 28, 2016, 08:47:25 pm
content: Quote from: DVX on October 25, 2016, 10:41:07 pmActually, no, you don't need to do this. If you use unsigned integers they wrap around automatically. For instance with unsigned char: 255 + 1 = 0, 0 - 255 = 1.

author: Cerebus
author_rank: Super Contributor
timestamp: October 29, 2016, 02:43:47 am
content: Quote from: IanB on October 28, 2016, 08:47:25 pmI would say that is poor practice. Yes, it works but then one day someone needs to increase the buffer size to 512, or 768 or whatever. If the buffer size is explicitly declared and an explicit wraparound programmed (or a modulo used) then everything just works. If on the other hand there is an implicit, worse still hidden, magic number of 256 then it stops working and we all know how much fun ISRs are to debug, don't we.A better scheme would look something like this (with initialization, dequeueing, 'volatile' and much else omitted):// WARNING - untested example code only.#define BUFFER_SIZE 256Whatever buffer;unsigned bufferStart, bufferEnd;void insertIntoBuffer (Whatever a){unsigned newBufferEnd = (bufferEnd + 1) % BUFFER_SIZE;// You can omit the second modulo below if and only if your dequeue routine always leaves a properly reduced result in bufferStartif (newBufferEnd == (bufferStart % BUFFER_SIZE)){// Buffer full. Take error action.}else{bufferEnd = newBufferEnd;buffer = a;}}

author: C
author_rank: Super Contributor
timestamp: October 29, 2016, 03:02:27 am
content: CerebusTo prevent race conditions think you shouldfor insertcheck spaceInsert itemupdate index saying item therefor removecheck spaceread itemupdate index saying item gone

author: hamster_nz
author_rank: Super Contributor
timestamp: October 29, 2016, 09:38:21 am
content: One thing that always makes me ponder is that a 'n' item circular buffer has n+1 states (between 0 and n items in the buffer). If you test based on a 'read_index' and 'write_index' all the entries in the buffer can't be used as you can't tell the difference between a completely full or completely empty buffer.If this is important to you, consider changing it to be 'read_index' and 'data_in_buffer' rather than using a 'read_index' & 'write_index', however you will need some way to lock access to the variables when they are updated (e.g. disable/enable interrupts when adding or removing data values). You don't have to do if you use write and read indexes, as the reader only updates the read index, and the writer only updates the write index.Oh, and for efficiency using the modulus/remainder is most likely not the best.. maybe consider usingwrite_index++;if(write_index == buffer_size)write_index = 0;or the uglier:write_index = (write_index == buffer_size -1 ? 0 : write_index+1);if you are sure that the buffer size will be a power of two, then you could also use this:write_index = (write_index+1) & (buffer_size-1);The last option won't have any branching/jumps it should be much more efficient on pipelined CPUs.

author: Cerebus
author_rank: Super Contributor
timestamp: October 29, 2016, 12:55:29 pm
content: Quote from: C on October 29, 2016, 03:02:27 amTo quote myself:-Quote from: Cerebus on October 29, 2016, 02:43:47 amI wasn't really worrying about the fine detail of execution order as this is in the context of ISRs and one hopes that by this point folks realize that some sort of mutual exclusion scheme is needed there to protect the whole thing. There can be no race condition as long as the entire enqueue/dequeue process is protected by some sort of mutex/semaphore/etc.If I was going to do the job properly I'd actually code a wait-free/non-blocking implementation but that example (coded on the fly) was just meant to be a snippet to show how cleanly this can be coded with a modulo counter versus the 'orrible practice of hiding an assumed modulo 256 in there by using an 8-bit unsigned control variable.

author: Cerebus
author_rank: Super Contributor
timestamp: October 29, 2016, 01:13:15 pm
content: Quote from: hamster_nz on October 29, 2016, 09:38:21 amI would argue for adopting the coding style that is clearest and least likely to generate bugs, and pushing efficiency down the packing order. If you actually run out of cycles to get the job done you can go back and recode something. Given that even the meanest embedded processor runs at several MIPs and you usually end up throwing most of the processor's power away in wait loops there's not much to be gained in the typical case by making efficiency a primary goal but there is much to be gained from adopting coding styles that encourage correctness and easy maintenance.

author: suicidaleggroll
author_rank: Super Contributor
timestamp: October 29, 2016, 01:27:59 pm
content: Quote from: Cerebus on October 29, 2016, 02:43:47 amQuote from: Cerebus on October 29, 2016, 01:13:15 pmOr you can just leave a note to yourself (or a future developer) in the comments...# This circular buffer is set to 256 bytes so that it can take advantage of the implicit rollover in its 8-bit unsigned pointers.# If you need to change the size, make sure to change the pointer type and uncomment the modulo operation in the read and write functionsThen in your read and write you put the module in place, commented outcirc_buf[wr_pointer++] = val;# if (wr_pointer == circ_buff_size) wr_pointer = 0;Why waste a not insignificant number of clock cycles doing an unnecessary math operation that can be easily avoided? I understand the need/desire for maintainable code, but you can just as easily do that in comments without needlessly bloating up the code.These are embedded systems we're talking about here, efficiency is king. Any option that saves a significant number of clock cycles needs to be seriously considered, especially when any resulting code obfuscation can be explained away in the comments.

author: nctnico
author_rank: Super Contributor
timestamp: October 29, 2016, 02:05:48 pm
content: You can also use buffers with a length which is a power of 2. That way a simple AND operation can do the roll-over of the read/write indices.

author: John Coloccia
author_rank: Super Contributor
timestamp: October 29, 2016, 02:07:10 pm
content: I think what you wrote is practically the definition of bloated code, IMHO. Not only that, but when you pull this kind of stuff in your code for absolutely no practical benefit, you open the door for all sorts of difficult to understand and debug problems. I can't tell you how many times I've had to fix "clever" code that someone wrote because some insignificant details changed somewhere and now the whole thing is broken. Not only that, even if the cleverness is documented, it still requires a brand new round of testing and verification. I see this kind of thing all the time with clever pointer arithmetic and things like that too, but people miss the fine details and cause incredible problems.Part of my background happens to be real time embedded controls. There aren't a whole lot of people that are more familiar with optimizing code and squeezing every last microsecond of performance out of very complex systems. I RARELY ever have to resort to something like this, and the fact is that if you write your code simply and neatly, the compiler usually does a better job at optimizing than you can anyway. Real optimization that matters comes from clever algorithms and intimate understanding of the hardware so that you can take advantage of the memory architecture, caching, pipelines and things like that. Most of the rest is just parlor tricks.But if anyone is really curious, the thing to do is profile the code. Write it a few different ways, get RTC time going in and coming out, and see what the differences are when you do 20k writes. Try different compiler optimization settings too. One thing you'll find, especially when you deal with more advanced processors and algorithms, is that a lot of your code is going to be I/O bound, not CPU bound, and any gains will come from organizing the algorithm and memory in such a way to keep the CPU fed and the instruction pipeline filled. May not apply in this particular case, but I'm not intimately familiar with this processor so I don't really know.

author: Cerebus
author_rank: Super Contributor
timestamp: October 29, 2016, 02:10:48 pm
content: Quote from: suicidaleggroll on October 29, 2016, 01:27:59 pmYou could equally argue: "Why waste a not insignificant number of clock cycles by using a high level language when many unnecessary operations can be easily avoided by writing assembler?".The answer to both is the same. Coding for humans is more important than coding for the machine except in the very few edge cases where the machine is paramount and considerations of maintainability and ease of coding are secondary.I could argue, with I think some justification, that the number of clock cyclesisinsignificant. I could further argue that the bloating is in using several lines of obscure code with too many comments in place of one or two lines of clear, elegant code that doesn't require excessive comments.Itisjust a question of style and where you make your trade-offs. I prefer mine and I prefer working on code that's been written the same way. I've been banging out code for 41 years now. Which means I started back in the days where the mainframe I was coding for literally had less memory and MIPs than the sort of microcontroller you'll find buried inside an electric toothbrush - so I know a thing or two about efficiency for necessity's sake. You're arguing for efficiency for efficiency's sake, I'm saying let it take a back seat to correctness and maintainability.Anyway, why I am I having to justify myself to some young whippersnapper? It should be sufficient for me to stroke my grey beard sagaciously while you bow to my every word.

author: C
author_rank: Super Contributor
timestamp: October 29, 2016, 02:32:47 pm
content: Quote from: C on October 29, 2016, 03:02:27 amTo pick on me,The above is good idea, Would let you do a lot of instructions in the insert item or read item part with out problems.It assumes a bunch of things.Only one to put to put in item and Only one to get item. Not bad until you forget this part when you make copy code and use where this it not true.Also assumes that compiler does not reorder instructions. So what could have been good code gets broken just by using smarter compiler that can reorder instructions. Old code that worked is now bad.So think of big picture, by assuming something and not stating what that assumption is to all that needs to know leads to possible future errors.Do you leave a note that a programmer might read or a note that a compiler will read?Think of modulo hereCompiler decides this is dumb and not needed.Compiler decides this can be a binary mask.Compiler decides this modulo must be done using modulo.All result in working code. Leaving modulo out is counting on Human part of compiler.Ask your self, how great is the Human part of compiler at reading all of a program and catching something like the order of code assumption above?Make compiler work for you or hope you can make human work for you.

author: John Coloccia
author_rank: Super Contributor
timestamp: October 29, 2016, 02:34:45 pm
content: Incidentally, some other things to consider if you use the rollover technique:- you loose the ability to use -1 as a signal or initial condition- You're probably going to get word aligned data anyway and it may actually be SLOWER to do modular arithmetic on 8byte quantities when you're on a 32bit processor, especially if it doesn't have native 8bit support (again, I don't know about this particular processor...just talking general case).

author: Kalvin
author_rank: Super Contributor
timestamp: October 29, 2016, 02:43:03 pm
content: Using a specific variable containing the number of the chars in the buffer is a simple and robust solution. Also, using an write and read index variables comparing to buffer size limit to perform the index wrapping is much more portable way of doing things than using the modulo arithmetics. Yes, modulo arithmetics is nice and cool, but the compiler may produce extremely inefficient code if the buffer size is not power-of-two, the compiler cannot optimize the modulo to bitmask. The runtime overhead is very small compared the mental pain you will have using any other methods.

author: hamster_nz
author_rank: Super Contributor
timestamp: October 30, 2016, 04:59:30 am
content: Quote from: Kalvin on October 29, 2016, 02:43:03 pmCode:#define BUFFER_SIZE (16)static char circular_buffer;static int read_index = 0;static int write_index = 0;static int chars_in_buffer = 0;static int overrun = 0;void enqueue_char(char c) {disable_interrupts();if(chars_in_buffer == BUFFER_SIZE) {/* Overrun error */overrun++;} else {/* Add value to queue */circular_buffer = c;chars_in_buffer++;/* Advance write index */write_index++;if(write_index >= BUFFER_SIZE)write_index = 0;}enable_interrupts();}int dequeue_char(void) {int c = -1; /* Default return value */disable_interrupts();if(chars_in_buffer > 0) {/* Pull value from queue */c = circular_buffer;chars_in_buffer--;/* Advance read index */read_index++;if(read_index >= BUFFER_SIZE)read_index = 0;}enable_interrupts();return c;}As this is only a few lines, I gave it a crack at writing it. I'm yet to actually test it - I just confirmed that a C compiler likes it.I am also assuming it is for a single core micro-controller CPU. Have I made any obvious errors?.I would give enable_interrupts() and disable_interrupts() and internal counter for how many times they have been called, allowing them to be called from within interrupt code without breaking things..

author: obiwanjacobi
author_rank: Super Contributor
timestamp: October 30, 2016, 06:06:57 am
content: Quote from: hamster_nz on October 30, 2016, 04:59:30 amvolatile?Quote from: hamster_nz on October 30, 2016, 04:59:30 amIn this case not needed: disable- and enableInterrupts is nicely paired in each function. Just make sure in these types of scenarios that you have one (and only one) return statement. An early exit would leave the interrupts disabled.

author: hamster_nz
author_rank: Super Contributor
timestamp: October 30, 2016, 08:35:30 am
content: Quote from: hamster_nz on October 30, 2016, 04:59:30 am

author: John Coloccia
author_rank: Super Contributor
timestamp: October 30, 2016, 10:44:15 am
content: Quote from: hamster_nz on October 30, 2016, 08:35:30 amIf you have data that's being accessed by the ISR and anywhere else, it absolutely needs to be volatile.re: interruptsNo need to have a counter. If interrupts are disabled in the main program, the ISR can't ever be called, so no need to reference count it. It's a good idea to disable interrupts going into the ISR and enable coming out if that's not done automatically. All that said, it would be better to write the buffer management so that disabling interrupts in the main program is not required. I'm sure you can find one online that shows how to do it, but if not I'll try to scribble something up. The basic technique is that you have a contract between the ISR and the main program where certain values are only read by the ISR and written by Main(), and vice versa.