Do you use size_t? - Page 1
Forum section: electronics / general

author: pacmann
author_rank: Contributor
timestamp: July 09, 2025, 06:02:28 am
content: Do you use size_t in your code? Should uint be used instead if max loops is known size?

author: hans
author_rank: Super Contributor
timestamp: July 09, 2025, 09:24:44 am
content: Yes.. size_t is a type definition to denote the number of bytes. For example, in memcpy or malloc. Its useful on systems which have a memory space that is bigger than the size of "uint". For example, if uint is 8-bit but your MCU can address more than 256 bytes of RAM. This is why sizeof() etc. all return size_t so the type is always large enough for the types that may exist, but not unnecessarily big (e.g. always using uint32_t) when you don't need it.Source:https://web.archive.org/web/20190604075520/http://www.embedded.com/electronics-blogs/programming-pointers/4026076/Why-size-t-mattersGenerally I always use types from stdint.h. Never ever just int or unsigned int.For loops I may size_t if its a byte loop. Otherwise I tend to use uint_fast8_t, uint_fast16_t, etc. when I can. In the rare chance I'm compiling some code for an 8-bit AVR or 16-bit device, the indices automatically scale back to a size that best fits the architecture but is large enough for the range I need.For application values I often use uint32_t etc.

author: brucehoult
author_rank: Super Contributor
timestamp: July 09, 2025, 09:58:42 am
content: Quote from: pacmann on July 09, 2025, 06:02:28 amOf course! I'm not some kind of savage.Quote"unsigned int" is one of the worst possible choices, as if it is used as an index it may take extra work to zero extend it to the size of a pointer before multiplying it by the element size and adding it to the array base. There are ISAs used as microcontrollers for what that is the case. And if you use "int" then it may take work to sign extend it.Less commonly, it may result in slower code to increment and compare an int or unsigned int vs something the full size of a register.Using long or unsigned long is almost always as good as or better than using int or unsigned int. The only exception is on machines where long is larger than a pointer, which is really only 16 bit MSDOS (long is bigger than a near pointer, but the same size as a far pointer), some DSPs, and some historical machines from when memory was extremely expensive e.g. CDC6600 with 18 bit pointer registers or Cray 1 with 24 bit pointer registers.In some cases long is still smaller than a pointer. The only one I can think of off-hand is 64 bit Windows, where size_t is unsigned long long.But simply put, always use size_t or intptr_t. They will always be the correct size for correctness and efficiency.If you're more worried about being lazy in the amount of typing you have to do than in correctness then "long" will almost always be as good as or better than "int" or "unsigned int" on modern machines.

author: mikerj
author_rank: Super Contributor
timestamp: July 14, 2025, 11:18:59 am
content: Quote from: brucehoult on July 09, 2025, 09:58:42 amI'm absolutely not suggesting anyone not use size_t when required but this seems like hyperbole. It may well be a bad choice for specific targets (particularly 64 bit micros), but at least on the almost ubiquitous 32 bit micros used these days size_t will be typically be typedef'd as an unsigned int or unsigned long.

author: brucehoult
author_rank: Super Contributor
timestamp: July 14, 2025, 11:56:49 am
content: Quote from: mikerj on July 14, 2025, 11:18:59 amIt's absolutely true. 64 bit microcontrollers exist, maybe not so much stand-alone but they are common in SoCs that also have one or more 64 bit applications processors e.g. SiFive FU540 and FU740, StarFive JH7110, ESWIN EIC7700X, Sophgo CV1800B, SG2000/2, Microchip PolarFire SoC FPGAs and PIC64GX.Code:#include <stddef.h>int foo(int *p, size_t i) {return p;}int bar(int *p, int i) {return p;}int baz(int *p, unsigned int i) {return p;}On RV64IC this compiles to:Code:0000000000000000 <foo>:0: 058a slli a1,a1,0x22: 952e add a0,a0,a14: 4108 lw a0,0(a0)6: 8082 ret0000000000000008 <bar>:8: 058a slli a1,a1,0x2a: 952e add a0,a0,a1c: 4108 lw a0,0(a0)e: 8082 ret0000000000000010 <baz>:10: 02059793 slli a5,a1,0x2014: 01e7d593 srli a1,a5,0x1e18: 952e add a0,a0,a11a: 4108 lw a0,0(a0)1c: 8082 retOne extra instruction and six extra bytes for the unsigned int version. Hmm .. actually that bad codegen by gcc. If it just did those shifts in a1 then it would be two 2-byte instructions not two 4-byte instructions. Even trunk gcc on Godbolt does that :-( No version of LLVM does it. So really it should be one extra instruction and two extra bytes.This is the best that can be done on all of the above except the JH7110 and EIC7700X which offer the Zba extension which has instructions that implicitly zero extend 32 bit unsigned to 64 bit unsigned before use.Code:0000000000000000 <foo>:0: 20a5c5b3 sh2add a1,a1,a04: 4188 lw a0,0(a1)6: 8082 ret0000000000000008 <bar>:8: 20a5c5b3 sh2add a1,a1,a0c: 4188 lw a0,0(a1)e: 8082 ret0000000000000010 <baz>:10: 20a5c5bb sh2add.uw a1,a1,a014: 4188 lw a0,0(a1)16: 8082 retSure there are plenty of ISAs for which unsigned int is better than int (ARM .. which is why people started to go to the trouble of typing "unsigned int" instead of "int" in the first place), or which are neutral on the question.But using a type such as size_t which is reliably the size of a pointer on all of 16 bit, 32 bit, and 64 bit CPUs just isn't that hard to do and is the RIGHT thing to do.

author: peter-h
author_rank: Super Contributor
timestamp: July 14, 2025, 02:30:17 pm
content: AIUI, using uint8_t uint16_t uint32_t clearly states the storage unit. But it does not pack the storage so e.g. a uint8_t may still take up 4 addresses on an arm32.There are many cases when loading registers where a specific size must be used.For stuff likefor (int x=0; x<10; x++)I use int because it is supposed to be the basic machine word and thus most efficient. Sofor (uint8_t x=0; x<10; x++)is pointless and certainly no faster than int.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 14, 2025, 02:39:32 pm
content: Quote from: peter-h on July 14, 2025, 02:30:17 pmNo - the whole point of uint8_t is that it's size is exactly 8 bits. It will never take up "4 addresses", if it did, everything would break down completely. This is literally why these accurate-width types were invented in the first place. (Unless, by "taking" you mean compiler adding padding, then sure, it can add 3 wasted locations of padding, but the variable itself would be just 1 byte.)You are thinking about uint_fast8_t, which can be larger than 8 bits.If you need a loop variable for which range 0-255 suffices, uint_fast8_t will work and produce portable code: then on a 8-bit target 8-bit variable gets used, but on 32-bit ARM a 32-bit integer would get used.But size_t is functionally (and coincidentally, also performance-wise) optimal when used as an index to an array, which loop variables often (but not always!) are.

author: peter-h
author_rank: Super Contributor
timestamp: July 14, 2025, 02:42:57 pm
content: QuoteExactly!Obviously 1 byte takes up 1 byte of RAMIsn't size_t same as int, once above 16 bits? (I know int is 16 bits on a Z80 which is why C is so much slower than asm on the old chips).

author: brucehoult
author_rank: Super Contributor
timestamp: July 14, 2025, 02:49:50 pm
content: Quote from: peter-h on July 14, 2025, 02:42:57 pmNo. Almost all 64 bit OSes have int as 32 bits, except I think Windows still has int as 16 bits.Size_t is the same as "unsigned long" on almost all 64 bit OSes, but on Windows it is the same as "unsigned long long".

author: gerbay
author_rank: Regular Contributor
timestamp: July 14, 2025, 03:03:14 pm
content: Different systems and operating systems use different data models. Variable sizes vary depending on the data model used. For some environments, you can see the data types defined by the data model in the table.To avoid this confusion in C/C++, the "stdint.h" header file has been added to define guaranteed data type sizes.The data types required to represent addresses relative to the data bus width also depend on the data bus width.The "size_t" and "ssize_t" data types, optionally defined in the standard, are represented in bits equal to the data bus width. They are 32 bits on 32-bit systems and 64 bits on 64-bit systems.Using "size_t" and "ssize_t" is important for ensuring code portability and generating the most efficient code.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 14, 2025, 03:22:58 pm
content: I've never used it. I size my variables based on the values they're supposed to hold, sometimes other considerations, such as alignment or CPU properties. I don't want them to be dependent on compiler idiosyncrasies. I use uint8_t, uint16_t and similar types which allow me to specify the desired size precisely.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 14, 2025, 09:19:56 pm
content: Quote from: pacmann on July 09, 2025, 06:02:28 amAlways, for in-memory sizes/counts/lengths.I also useuintptr_t/intptr_t(for pointer-size unsigned/signed integer values),uintN_t/uintN_tforN=8/16/32/64 -bit exact-size unsigned and signed integers (signed integers always using two's complement format), as well asuint_fastN_t/int_fastN_ttypes optimal for the target with at leastN=8/16/32/64 bits (but may have more in practice).To limit precision and range, I use casting whenever possible, for both ensuring correct compiler code, and for us humans looking at what the code does, because an explicit cast also indicates software developer intent; just like best comments do.For example, when you use<ctype.h>, and you have a character inchar c, you should useisspace((unsigned char)(c))instead ofisspace(c). The underlined part is the cast to unsigned char type. This commonly occurs when one has a C string, saychar *s, and then does e.g.while (isspace(s)) i++;or similar loop to skip over whitespace characters in the string. This is abugon systems using character sets with whitespace characters for which(char)((unsigned char)(c)) != c(which also requires thatcharis a signed type). The correct form for said loop is to cast the character tounsigned charfirst, because theint cparameter for<ctype.h>functions must have anunsigned charvalue (for historical reasons), as the documentation,man 3 isspace, explains.As a practical example, consider the case where you have some configuration data in a C string (terminated by a NUL byte,\0), and you want to extract the next command token. Let's say you implement (to run under a fully-featured operating system)const char *command(const char **constconfigptr, size_t *constlengthptr, size_t *consthashptr);which returns the pointer to the next command or NULL at the end of configuration data, updating*configptrto point to the first character after the command token (usually whitespace), and if non-NULL, saves the token length at*lengthptrand the DJB2 Xor hash of the token at*hashptr:Code:#include <stdlib.h>#include <locale.h>#include <ctype.h>#include <errno.h>const char *command(const char **const configptr, size_t *const lengthptr, size_t *const hashptr) {if (lengthptr)*lengthptr = 0;if (hashptr)*hashptr = 0;if (!configptr) {errno = EINVAL;return NULL;} elseif (!*configptr) {errno = 0;return NULL;}const char *p = *configptr;if (*p == '\0') {errno = 0;return NULL;}while (1) {// Skip all whitespace, including spaces, tabs, and newlines.while (isspace((unsigned char)(*p)))p++;if (isalpha((unsigned char)(*p))) {// Start of command.break;} elseif (*p == ';' || *p == '#') {// Comment to the end of the line. Skip.while (*p != '\0' && *p != '\n' && *p != '\r')p++;continue;} elseif (*p == '\0') {// End of configuration.*configptr = p;errno = 0;return NULL;} else {// Invalid character in configuration.*configptr = p;if (hashptr)*hashptr = (unsigned char)(*p);errno = EDOM; // Invalid command characterreturn NULL;}}*configptr = p;const char *start = p;size_t hash = 5381;do {hash = (33 * hash) ^ (unsigned char)(*(p++));} while (isalnum((unsigned char)(*p)));if (lengthptr)*lengthptr = (size_t)(p - start);if (hashptr)*hashptr = hash;*configptr = p;return start;}int main(int argc, char *argv[]) {const char *const arg0 = (argc > 0 && argv && argv && argv != '\0') ? argv : "(this)";if (!setlocale(LC_ALL, ""))fprintf(stderr, "Warning: Current locale is not fully supported by the C library.\n");// Other stuff...Thesetlocale()at the beginning of yourmain()tells the C library to adjustisspace(),isalpha(), andisalnum()to the character set used by the current locale if possible (typically only narrow/8-bit locales are supported, and e.g. Unicode requires wide characters via<wchar.h>).Typical use case issize_t len, hash;const char *cfg; // Points to the configuration stringconst char *cmd = command(&cfg, &len, &hash);if (!len) {if (!hash) {//end of config string} else {//unexpected character (hash) in config string}} else {//command starts atcmd,length islen,DJB2 Xor hashis hash// cfgpoints to the first character following the command, usually some whitespace character}If you use*pinstead of(unsigned char)(*p), the code will work fine on ASCII architectures, but fail on e.g. ISO 8859-1 (ISO Latin 1, Western European) whencharis a signed type, failing on non-breaking white space (U+00A0) and accented latin characters like áåïíêë in commands.Having length be represented bysize_tensures that the length of the token is representable, no matter how long, as long as it is in memory (or is mapped to memory).We could discuss which type would be optimal for the hash. Arguably,typedef uintptr_t hash_t;and usinghash_ttype for*hashptrandhashmight be optimal for general use: in the hash table of recognized command tokens and associated pointers (including function pointers), the most often accessed entries tend to be pointer-aligned anyway (and I tend to compare the hash first, and only when the hash matches, compare the parsed command string to the stored command string usingstrncmp(), which needs the length). In practice,size_t(from<stdlib.h>) anduintptr_t(from<stdint.h>) are the same type on all architectures I've used. The difference is in their C standard definition ––size_tbeing the unsigned integer type able to represent the size in bytes of any object in memory, anduintptr_tbeing an unsigned integer type for which any possible value ofvoid *pwill fulfill(void *)((uintptr_t)(p)) == p–– and I just happen to considersize_tmore appropriate as an"unsigned integer of the same storage size as an object pointer"type; some other people do disagree.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 15, 2025, 12:24:20 am
content: IMHO, it depends on the situation.Yes, size_t is a thing, but in a lot cases it's helpful to immediately cast it to the thing you KNOW it should be. And then justify it.This is the safety critical world. You simply have to know your limits, and if it goes wrong you need to know way before you start casting things. The casts are there for clarity and intention...nothing more, and it should flow up.A really good static analysis tool is also helpful. Occasionally finds real bugs.And it takes real thought and planning...design...to get there. But mostly it's done, and it's a PITA, but it's worth it.I WILL say that the idea of choosing my types based on optimizing performance is WAAAYYY down on my list. We program expressively. We worry about optimizations when they come up, which is rarely.

author: radiolistener
author_rank: Super Contributor
timestamp: July 15, 2025, 12:52:29 am
content: Yes, I often use size_t/ssize_t, instead of uint/int when working with sizes, array indices, and similar cases

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2025, 02:34:07 am
content: Quote from: John Coloccia on July 15, 2025, 12:24:20 amIf you KNOW what concrete type (i.e. how many bits) it should be then you won't be using size_t in the first place.The whole point of size_t is portable software where you don't know whether it will be 16, 32, 64 bits on the machine someone will be it running on.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 15, 2025, 03:11:21 am
content: Quote from: brucehoult on July 15, 2025, 02:34:07 amSure. But that's not real unless you're writing libraries.The vast majority of softwares doesn't need to be portable. You can target a platform, for the next 20 years, and then reconsider. Especially if you use stdint, and extra especially if you're not depending on some optimizations to get you there.All of this is about understanding the problem domain, and doing something intelligent looking out for the lifetime of the project. And tracking where the hacks are.IMHO, there's no such thing as truly portable software, with some exceptions for managed languages and some other frameworks.You just do your best to not be an idiot.But some people really need to hit widely different architectures, for whatever reason. So then you do something different. But in my experience, this usually ends up as a mess.We very like stdint these days.Edit:Hey look, I'm not arguing with you. It's just a different point of view that works for us.

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2025, 03:42:04 am
content: Quote from: John Coloccia on July 15, 2025, 03:11:21 amThat may be your experience, but I'm constantly reusing C code across 8 bit AVR, 32 and 64 bit Arm, 32 and 64 bit RISC-V, and 64 bit x86. (I don't remember the last time I cared about 16 or 32 bit x86, but it was before Core 2 came out).Quote"There is no portable software, only software which has been ported".QuoteThat address one set of needs: making the program conform to the problem domain.Stddef.h addresses different needs: making the program conform to whatever hardware it finds itself on. There is no point using a 64 bit number for an array index on a machine with a 64k address space and 8 bit ALU and registers.It might beportableto use a uint64_t to iterate through an array or string on every machine, but it's completely unnecessary and inefficient to do so on an AVR. or even a Cortex-M3.

author: Analog Kid
author_rank: Super Contributor
timestamp: July 15, 2025, 03:49:01 am
content: Quote from: brucehoult on July 14, 2025, 02:49:50 pmNope, 32 bits. SeeMicrosoft Learn.

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2025, 03:59:29 am
content: Quote from: Analog Kid on July 15, 2025, 03:49:01 amOh my bad, apparently they changed that in Windows NT and its successors. But "long" is still 32 bits on 64 bit, which is just as bad.But they still use WORD for 16 bits and DWORD for 32 bits, even on a 64 bit machine? Maybe thats what I was thinking of.I very rarely touch a Windows machine, and never program one lol. Mac, Linux, and embedded.

author: John Coloccia
author_rank: Super Contributor
timestamp: July 15, 2025, 04:03:35 am
content: Well, that's why C is actually a great language. You have great control over your problem domain, and can fine tune it for what you need.And it's also why a lot of people find it difficult. You really have to kind of think about what you're trying to achieve.Which I suppose you should do anyhow, but C is really better thought of as a really good processor abstraction than a high level programming language.I think things will be different in 20 or 30 years. Maybe sooner? Hard to see it right now but there are at least some credible options with ideas. I don't think it's Rust, but it's making some progress, at least.

author: ejeffrey
author_rank: Super Contributor
timestamp: July 15, 2025, 05:10:29 am
content: Quote from: Nominal Animal on July 14, 2025, 09:19:56 pmMy guess is that any platform where uintptr_t exists it is the same as size_t.The 16 bit DOS (and I assume windows) "large" memory model default pointers were far pointers that took up 32 bits, but size_t was still 16 bits. Not only that, offset calculation was done with 16 bits, so you couldn't have an array larger than 64 KB regardless of element size. You had to go up to the huge memory model (which I never had reason to use) to be able to use larger offsets. But I don't doubt any DOS compilers bothered to defined uintptr_t.

author: peter-h
author_rank: Super Contributor
timestamp: July 15, 2025, 05:25:24 am
content: QuoteHow does that work across a range of values? I mean if a variable goes 0 to 1000 it cannot possibly be held in 8 bits. So right away your portability breaks.I would use uint16_t, or just int which will be >= 16 bits.

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2025, 05:38:46 am
content: Quote from: peter-h on July 15, 2025, 05:25:24 amIf you know that a variable can be 0 to 1000 on all machines then you use a type that is at least 10 bits on all machines e.g. uint16_t.QuoteInt might be 64 bits, which would be a huge waste.On the other hand, as I've now said repeatedly, if I want a variable to index into an array then I KNOW that it can be bigger than 256 bytes on an AVR but it can't be bigger than 64k at least on the AVR I've used (ATMega2560 with external SRAM) so a 16 bit variable is big enough, while on my Arm64 Mac or Amd64 laptop or riscv64 SBCs I can easily have an array with more than 4 billion elements, so a 64 bit value is needed.Which is EXACTLY what size_t is for. It is automatically the appropriate size on all machines. 16 bits on an AVR, 32 bits on a Cortex-M3, 64 bits on the M1 Mac Mini."size_t" is in the subject line of this post.

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2025, 05:45:13 am
content: Quote from: ejeffrey on July 15, 2025, 05:10:29 amuintptr_t was only invented in C99. If anyone has made an MS-DOS or Windows 3.1 compiler since that date then uintptr_t needs to be 32 bits while size_t is indeed correctly 16 bits in the normal (efficient) memory model.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 15, 2025, 02:28:28 pm
content: Quote from: brucehoult on July 15, 2025, 05:38:46 amWhy is that "appropriate", whatever this means? The size necessary to store an index to an array depends on the actual size of the array and does not depend on how big other array could've existed on your platform. For example, if you want to look at payloads of IP packets, you know that the array is less than 65536, so you can use uint16_t to hold their length and the indexes. Using bigger variables doesn't usually serve any purpose.For example, if you build a garage, is it "appropriate" to build it big enough to fit the biggest truck ever built in the world? May be it's more "appropriate" to build it to fit the biggest truck you would ever buy?

author: brucehoult
author_rank: Super Contributor
timestamp: July 15, 2025, 02:43:18 pm
content: Quote from: NorthGuy on July 15, 2025, 02:28:28 pmOn a 32 bit or 64 bit machine?Most 32 and 64 bit machines don't even HAVE 16 bit registers available, so even if you declare a variable as uint16_t it's going to use a full 64 bit register to hold it, with zero savings, and quite possibly a cost to make addition etc wrap around at the 65536 boundary with an extra zero-extend instruction or ANDing with 65535.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 15, 2025, 02:46:44 pm
content: Quote from: NorthGuy on July 15, 2025, 02:28:28 pmOf course it does. If you use artificially small data type, smaller than machine register size, as a loop variable, compiler may end up doing unnecessary extra work of extending or zeroing bits.If a package is dispatched to be delivered in a van, you don't call the shipping company "wait, stop, let's reschedule this delivery and find the smallest possible car". Or if you build a garage and all the garage doors on the market are 2 meters wide, you build a garage of that size, instead of spending extra money by custom ordering a 1.5-meter wide special door; assuming you have empty space on your lot anyway. Just like the bits are in the CPU registers anyway.Exact minimum size is usually not the most efficient solution. Which is exactly why we have uint_fast16_t and friends.Now things are different if you need to store a lot of these indices in a memory-efficient manner. Then you would use smallest data type which can store the index, and accept the extra work compiler has to do to work with this data type.But loop variables are VERY often local and live only in CPU registers. For those, smallest possible accurate-width type makes least sense.I'm surprised you have never tried it. I often play around with this kind of stuff when writing some timing-critical piece of code. There you can see by simple experiments how the program size and running time decrease when going for a larger type.Micro-optimizations are not always obvious. As this example proves, it's a good idea to write somewhat portable, high-level code by default, but by all means when you know what you are doing, override that and use specific accurate-width types as necessary.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 15, 2025, 04:35:31 pm
content: Quote from: Siwastaja on July 15, 2025, 02:46:44 pmYes, this is the idea. You use various considerations like these to select the size for your variables, as opposed to just using size_t.Of course, if you're using PIC18 you don't think about the size of CPU registers as there's none, so the conclusions will be different. Moreover, the design itself will be different, so you won't be even implementing the exact same thing.Quote from: Siwastaja on July 15, 2025, 02:46:44 pmOf course, whatever you write should follow the C standard, which makes it "somewhat portable" whether you use specific sizes or general types like size_t.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 15, 2025, 04:42:33 pm
content: My criteria for choosing the type depends on whether the variable is in a structure, or whether it is just a temporary local one like a loop index or function parameter.The one exception issize_t, which I do use everywhere when it is about in-memory or memory-mapped lengths or counts that are not otherwise limited.I often useuint_fastN_t/int_fastN_tfor local variables and function parameters, but almost never in structures. In structures, I use explicit-sizeduintN_t/intN_tinstead.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 15, 2025, 04:50:40 pm
content: size_t in C usually maps to the "native" register size for most common targets, so it's a safe bet in terms of performance and bit width. (Of course, that may not be the case for some "odd" architectures.)size_t is unsigned, the signed variant is ssize_t.What I recommend beyond that is to define your own types *when it helps readability and maintainability*: to convey "semantics" and to make it very easy to change the base type if you ever need to.Let's say you have some array data structure for which you need to have a length (number of items) and index (for accessing it), you can define types like so:typedef size_t myarray_length_t;typedef myarray_length_t myarray_index_t;(In this example, it probably makes sense to use the same base type for the index as for the length, so that's how it's defined, but you can of course adapt it to the particular context. The idea is to define types that again convey some semantics and are easy to maintain.)I'm a proponent of strict typing so that would also explain my approach. C doesn't have strict typing, but you can still do the above to make things easier and clearer as I explained. Just because C *lets* you be sloppy doesn't mean you *have* to. Just my 2 cents.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 15, 2025, 05:34:49 pm
content: Quote from: SiliconWizard on July 15, 2025, 04:50:40 pmVery recommendable, as now you easily can:* Compare size/performance by changing the underlying type in one place of code,* Test for bugs hindering future portability by changing the type in one place of code and running tests

author: NorthGuy
author_rank: Super Contributor
timestamp: July 15, 2025, 06:27:37 pm
content: Quote from: brucehoult on July 15, 2025, 02:43:18 pmOn any machine. Variables take space in memory. Bigger variable takes more space. If you use long arrays or linked lists to store them there may be much more memory.Registers have nothing to do with this. C uses a thing called integer promotion - by the time a variable is loaded into a register, it is already promoted to integer size. The integer promotion also makes unnecessary any actions related to overflows and wrapping around for intermediary results within expressions - the excess gets truncated at the moment where the result is saved into a smaller variable (and only if it is saved to such a variable). That's the rai·son d'ê·tre of integer promotion.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 15, 2025, 06:48:21 pm
content: Quote from: NorthGuy on July 15, 2025, 06:27:37 pmNot always - local variables can get mapped to CPU registers directly. This is especially the case in instances where size_t is used most: as loop iterator, or loop limit. In small functions specifically they just map to registers.Even if they end up in memory, trying to save a few bytes here and there is calledpremature optimization. Especially futile if going for smaller datatype causes compiler to add padding.However, you are right that...QuoteFor arrays (or other multi-element constructs) it makes sense to do a rough initial calculation of how much memory will be spent - hopefully before choosing the microcontroller, if at all possible, and optimize the algorithm and data storage types for size. It won't be a small micro-optimization anymore, due to amount of elements.This is engineering; we need numbers. Trying to save every bit of RAM is futile act if you are not nearly out of RAM. If this leads to performance penalty (code running slower), thenthatmight become a problem instead; or might not - it depends, sometimes RAM is tight, sometimes speed, usually neither. And if neither, don't prematurely micro-optimize; instead write maintainable code intended for human readers and compilers. Portability also doesn'thurteven if you don't initially plan on porting it. Has happened to me many times that I end up running the code on a different target anyway.Communicating ranges with the variable type is crap; it would be limited to only 0 .. 2^N or -2^(N-1) .. 2^(N-1)-1, where N can be only 8, 16, 32, 64. Real-world ranges can be anything. Use some sort of error checking or assert() instead: that clearly documents what the expected range is, and verifies it runtime, and in best case, allows a modern compiler to do optimizations too, or good static analysis tools to check. In a tight performance critical loop where you can't afford the runtime check, comment the range. C unfortunately lacks arbitrary range specifications; stdint types were never meant for that purpose either.

author: cv007
author_rank: Frequent Contributor
timestamp: July 15, 2025, 07:00:41 pm
content: In mcu code I would rather see the size specific types from stdint so the type size is not hidden. For an AVR it is easy to forget about overflow in any math operation if for example you use int/unsigned int, but when you have something like uint16_t/uint32_t in front of you it brings to mind what limits you are dealing with and maybe will be less likely to overflow a relatively small 16bits. For an avr, size_t would also be similar as you do not see its limitation. You also will have to know what a literal number type is by default. Seems hard to get away from knowing what mcu you are working with, but the stdint types do a good job of both allowing the same code on 8-32 bit and also making you decide which one is appropriate to use.QuoteSince size_t is mcu dependent, you will possibly get 3 different hashes for the same input because size_t was used for the hash. Since this is a microcontroller forum I'll skip the 64bit, so you have 2 different results, such as running on an AVR and a Cortex-M. Whether that would a problem or not in this case, I guess it depends on how the hash is used, but you do have different results due to the use of size_t. A better decision would be to determine how many bits of the hash you need and use uint16_t/uint32_t instead of size_t so you get the same results no matter which mcu is used. I'm not sure in this code how you could get something portable (consistent hash results) without the use of these stdint types.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 15, 2025, 08:19:48 pm
content: Quote from: Siwastaja on July 15, 2025, 06:48:21 pmWhen you do your design, you have different storage needs - a circular buffer, a tree, a list, a stack, whatever you need for the design. These will contain certain elements, each element will have its size. You choose the size of the storage elements based on various considerations. Like, for example you can store characters as ASCII (one byte per characters), or you can store them as UCS32 (4 bytes per character), or whatever else you may think of. If you decide to use ASCII, it is not premature optimization. If you decide to use UCS32, this is not waste of memory. Rather you chose one or another based on your design considerations.Similarly, when you need a variable to store array indices, offsets, or sizes, you need to select the size for that variable somehow. The variable cannot exist without the size. I prefer to choose the size by myself. This way I get the best size for my particular design. This also eliminates uncertainty. Not to mention this is better for portability because if you move your code to another CPU, all the sizes and structures will stay the same.

author: hans
author_rank: Super Contributor
timestamp: July 15, 2025, 09:15:00 pm
content: Indeed, there are multiple things to consider.For actual data I will try to use the most apprioriate type that is the smallest. It makes a huge difference if you store an array of ADC samples in uint16 or uint32.I use uint_fast types for scratchpad variables, like loop/array indices, values that live on stack (locals, function arguments), etc.However, I think its sometimes a bit overestimated how much its effect is:https://godbolt.org/z/E3qxG3Ed93 implementations of a simple circular queue push, with uint8, uint_fast8 and size_t.On ARM + Clang, all the simple push functions are "practically" identical. Curiously, its still using byte load/store for uint_fast8_tOn ARM + GCC, only an extra uxtb for uint8 implementation. Not sure why, it is using a load byte instructon.For AVR + GCC, obviously size_t is slower. But since its indirect address also needs a 16-bit load for the buffer operation, the difference is not even that big IMO.All compilers are smart enough to recognize that modulo 32 clears any upper bits anyway, so there is no need for zero extend. For the array pushes, not much of a difference neither, even though the loop variable is a new unknown.I'm guessing because the compiler knows the 'count' argument value cannot exceed 255, and so the lifetime values of the loop counter neither, so there is no need to clear the upper bits.So the biggest difference is seen on AVR with size_t..Not sure if its worth all the trouble to handpick these widths throughout my application for the ofchance I compile some code for AVRs lol.Its ironic because there are so many FLASH devices available with an address space which is larger than 2 bytes. That makes an object 'size' not even fit in size_t on AVR.

author: radiolistener
author_rank: Super Contributor
timestamp: July 15, 2025, 09:16:04 pm
content: Quote from: brucehoult on July 15, 2025, 02:34:07 amYes, the main idea is exactly to avoid hardcoding variable sizes that depend on the platform's architecture. Using size_t ensures that your code automatically adapts to the pointer size of the target platform, whether it's 16, 32, or 64-bit. This promotes portability and correctness, especially when working with memory sizes or array indexing.I often need to compile exactly the same code for very different platforms, and using size_t/ssize_t allows me to delegate this concern to the compiler, ensuring correct and efficient type selection automatically without manual adjustments for each architecture.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 15, 2025, 10:58:26 pm
content: Quote from: cv007 on July 15, 2025, 07:00:41 pmYes, but that is completely okay, because my approach takes that into account. Iknowits range varies from target to target, and I typically cross-compile my projects.Because I work with different architectures where the integer type ranges vary, I've learned to document the effective limits caused by the ranges, and to consider the effects of choosing a particular type. Again, my choice ofsize_tfor hashes stems from me storing those hashes in structures with pointer-sized fields and alignment.I use something like the following to calculate a type-generic DJB2 Xor hash:Code:#include <stdint.h>uintmax_t djb2xor(const char *token, const size_t len, const uintmax_t modulus) {if (!token)return 0;uintmax_t hash = 5381;const unsigned char *p = (const unsigned char *)token;const unsigned char *const q = p + len;while (p < q)hash = ((hash * 33) ^ (*(p++))) % modulus;return hash;}Theuintmax_tis anUINTMAX_WIDTH-bit unsigned integer type that one can parse from string usingstrtoumax()(from<inttypes.h>) and format inprintf()using%ju(decimal) or%jX(hexadecimal with uppercase A-F). It should be the largest integer type supported.The value ofmodulusisSIZE_MAX(from<stdint.h>) on the target system, obtainable using e.g.`echo '#include <stdint.h>' | $(CC) $(CFLAGS) -dM -E - | sed -ne 's/^\r*#[\t ]*define[\t ][\t ]*SIZE_MAX[\t ][\t ]*\(.*[^\t\r ]\)[\t\r ]*$/\1/p'`in aMakefile. Note that the-dM -E -causes the compiler to only run the preprocessing step, so this will work fine when$(CC) $(CFLAGS)is a cross compiler. Since this is run at build time only, there is no need to optimize the modulo operation at all.For other unsigned integer types,modulusis the maximum representable value using the target type, as listed in<inttypes.h>or<limits.h>.I also like to have a compile-time check file not included in the final binary but just compiled/pre-processed to ensure limits and types are of supportable size combinations, for example#include <stdint.h>#if (SIZE_MAX == 65535U) && (UINTPTR_MAX == 65535U)// 8/16-bit arch#elif (SIZE_MAX == 4294967295UL) && (UINTPTR_MAX == 4294967295UL)// 32-bit arch#elif (SIZE_MAX == 18446744073709551615ULL) && (UINTPTR_MAX == 18446744073709551615ULL)// 64-bit arch#else#error Unsupported SIZE_MAX/UINTPTR_MAX#endifand similar checks, with specific tested-okay type limit combinations, perhapstypedef'ing the domain/application-specific architecture-dependent types.When porting the code to a new architecture, I add the corresponding check, comment the addition date, and then run full tests and examine the types, sizes, and ranges on that target architecture, and go through the code to check for incompatibilities.I am old enough to remember the bad times when "everybody" assumed(void *)((int)(p)) ==pfor all possible values ofvoid *p, but was suddenly not true anymore for e.g. AMD64 (x86_64), causing all sorts of breakage and bugs. Hell, because of Cint, Linux low-levelread()andwrite()return short counts for sizes at or greater than 2GiB - one page. Libraries which assume that "short reads/writes do not occur for ordinary local files" thus break when having a larger file than that and plenty of RAM. Worst offenders are various serial libraries, which do not understand anything about POSIX/Linux/BSDtermios, and instead assume things work like they do on Windows, ignoring return values and silently losing data and ignoring errors... which is why Ialwaysprefertermiosover stupid serial libraries.I do tend to do various ELF object file shenanigans at build time. One of these is to use ELF section attribute to collect things like command structure entries from various source files, export them in binary form usingobjcopy, then using a simple host-side C or Python program to read the binary and generate the source file that declares the corresponding array of structures containing the hashes et cetera, suitable for e.g. a hash table.Those shenanigans do limit me to architectures and toolchains that support ELF format object files. I currently use various versions ofsdcc,gcc, andclang, and they all obviously do support ELF format object files and custom section attributes for variables and functions; but so do basically all other compilers that are not limited to a single OS or target, for example AMD Optimizing C/C++ compiler, Intel ICC and ICX, Open64, Portland Group, and Pathscale C compilers. I do not use Keil or IAR compilers, because I don't like being tied to a single vendor. To me, the competition between gcc and clang is extremely valuable (for that reason, and for keeping them from stagnating).The practical way I define "portability" differs from how others define it. To me, it is defined by practical experience and results, not by e.g. assertions about following specific ISO C or C++ standards precisely. The compiler bugs I've encountered have all been about differences in the interpretation of the standards, you see.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 16, 2025, 08:36:06 am
content: Quote from: NorthGuy on July 15, 2025, 08:19:48 pmHow do you define "better"?It really depends what your goal is with portability.If you are writing a software that would run optimally on another target, for the capability of that target, then you would use size_t. For example, if you used uint8_t on AVR that would have limited your data size to 256 elements. If you now compile the same software for 32-bit ARM, or x86-64 PC, then you:- Have the same 256-element limitation- Codemightbe a tiny bit slower and larger than it needs to be,but+ it operates as closely as possible to the original, which can be useful if the reason for porting is to "simulate" how the software works on the original target.Then again, if you used size_t and compile the same software for 32-bit ARM or x86-64 PC, then you:+ Gain an improvement in feature set: you can process larger data (assuming your code otherwise is able to handle that)+ Generated code will be optimal for the target (possiblyh a small performance and size improvement)but- it does not simulate the original behavior exactly. If this is what you wanted, use exact-width types instead.

author: hans
author_rank: Super Contributor
timestamp: July 16, 2025, 12:43:55 pm
content: Or.. you can get fancy pants and apply metaprogramming (type programming) in C++.E.g.:https://godbolt.org/z/1fse8b5xsHere the internal types for counters etc. are determined during compilation dependent on size parameters. It would be consistent across all platforms. Except one caveat: its not portable to some undermaintained builds of GCC, like AVR, which does not contain a large part of C++'s modern STL (even though type_traits is only from C++11).I have used this kind of code before, but for aforementioned (and earlier) reasons I opt'ed out. Not only is it a big problem when you start writing code that gets too smart (it will bite back one day). It also limits the scope of said code. AVRs are still nice chips for various tasks, and being able to reuse these generic embedded tailored containers is the purpose of writing and thoroughly testing this kind of code in the first place.Moreover, like I stated in my previous post, the amount of overhead is not that big when compilers get this smart. I can have fun writing this kind of stuff, but its more like solving puzzles is fun as mental exercise, but its not really solving a real world problem.In the above example, the difference between uint8 and uint32 implementation is a single zero-extend instruction on the return path of the PushAll/PopAll functions. These only cost 1 cycle over potentially hundreds of cycles when transferring a large amount of bytes. Practically nothing was optimized here, only the uint32 implementation is a few bytes bigger in storage.Realistically you could polish the zero extend away if you make the function arguments use size_t instead of Tpos ( arguably better for a type which is exposed to the caller).

author: Siwastaja
author_rank: Super Contributor
timestamp: July 16, 2025, 01:37:39 pm
content: Quote from: hans on July 16, 2025, 12:43:55 pmYes. It's funny to see how the extremes play out: computer sciency people and also most of the people "in the industry" want to drive extremely generic and abstracted solutions, and unfortunately this drives to over-engineering, bloat, uncertainty of how the thing actually performs, bugs, hard to predict delivery schedules, exploding development and maintenance costs...... and then the other extreme do not even want to use size_t in C because they feel they have not enough control and certainty over it - it being "too generic".I believe there must be a golden middle road with right amount of genericity applied where it is most beneficial, and when unsure, some of it applied "just in case" (to make future work easier, or avoid bugs that arise from manual micromanagement), but not too much (having to do extra work to gain some feature you likely won't ever need).C does not offer much genericity or advanced metaprogramming, but it offerssome. It rarely is too much even when used to "the fullest" - just avoid ugly hacks when they are not necessary. C++ is very different in that regard - using it to "the fullest" results in a collapsing cathedral of epic proportions.So for C programmers, I would say, "get a bit fancier", but to C++ programmers, "try to think simpler, no need to be that fancy".

author: brucehoult
author_rank: Super Contributor
timestamp: July 16, 2025, 01:57:13 pm
content: Quote from: Siwastaja on July 16, 2025, 01:37:39 pmHave you seenhttps://mcyoung.xyz/2025/07/14/best/?I look forward to the Swisstable PR getting merged. That will make it a very usable and higher performance but considerably simpler replacement for the standard C+ library.I also have used a C++ implementation of the J2ME library and found it quite convenient.Something I like about the J2ME specification more than full Java is that J2ME doesn't expose (and therefore mandate) a particular implementation of e.g. hash table internals. And Best has a similar relationship to ordered and unordered maps in the standard C++ library, with the standard library ordered map mandating a Red Black binary tree implementation and unordered map exposing (mandating) a linked list representation of hash table buckets.Both of these are inherently inefficient on modern CPUs. Once you've fetched 64 bytes into L1 cache you should use ALL of them and vastly reduce the number of random pointer chasing. Trees should be multi-way e.g. some kind of btree, and hash tables should store items with the same hash code (modulo table size) in contiguous elements of an array (at least in the common case).Best does both of those things.

author: DiTBho
author_rank: Super Contributor
timestamp: July 16, 2025, 05:04:17 pm
content: Quote from: NorthGuy on July 15, 2025, 08:19:48 pmI use the same reasoning. 100% the same thinkening.In my case, it's a mental way to keep everything under control, no doubt about, especially since there's no size_t in tool like Stood when you're designing: it forces you to explicitly choose a size.So I isolate the C modules that "could" use "size_t" so that it's clear that they'll need to be revised if ported to other architectures.

author: DiTBho
author_rank: Super Contributor
timestamp: July 16, 2025, 05:13:01 pm
content: Quote from: Nominal Animal on July 15, 2025, 10:58:26 pmI do it toobut I preferred to use sizeof() because there's a big problem with hedears: the defines change with the toolchain versions.For example, on my Xscale Japanese PDA, I can't use modern rootfs because they take up too much RAM(1), so I use on old one ~ 2008, all compiled by gcc v4.1.2, which has library defines that are sometimes "slightly" different from those of gcc v13.Basically, I would have to check the version first, then include a file that checks the size, which becomes very annoying to manage.(1) running kernel 5.4 I only have 24Mbyte free ram, over 64Mbyte physical ram.

author: DiTBho
author_rank: Super Contributor
timestamp: July 16, 2025, 05:19:12 pm
content: Quote from: Nominal Animal on July 15, 2025, 10:58:26 pmNot only AMD64, it was also a problem for PowerMacs when we moved from PPC750 and PPC74xx to PPC970 and POWER4.From 32-bit to 64-bit. Same problem. That kind of bug was a bloody disaster!

author: NorthGuy
author_rank: Super Contributor
timestamp: July 17, 2025, 02:12:15 pm
content: Quote from: Siwastaja on July 16, 2025, 08:36:06 amProbability that the code works the same on different platforms. For example, I sometimes run my code on PC to test something. Then the goal is to write code that is guaranteed to yield the same results on PC as on MCU.Quote from: Siwastaja on July 16, 2025, 08:36:06 amI would guess, on AVR, size_t is 16-bit, so using size_t will be quite inefficient. On 32-bit ARM it won't make any difference, let alone on x86.If you relay on overflows (for example you have a 256-byte circular buffer that rolls over automatically), then using size_t instead of uint8_t would change the behaviour.size_t is more of an abstract concept. For example strlen() returns size_t. Because it must returnsomething. size_t fits the bill. Using size_t everywhere is just sort of political correctness.

author: brucehoult
author_rank: Super Contributor
timestamp: July 17, 2025, 02:27:05 pm
content: Quote from: NorthGuy on July 17, 2025, 02:12:15 pmIf you don't KNOW in advance that a string or buffer or array will ALWAYS be smaller than 256 bytes then using something smaller than size_t will not be more efficient, it will be a bug.328's have 2k RAM. 2560s have 8K and can be transparently expanded to 64K with external RAM. It's absolutely entirely possible that an array of buffer on an AVR could be bigger than 256 bytes.Unless you design your program in such as way that it can't be.But if you're making a general purpose function such as memcpy() then you have to assume that the size can be larger than 256 bytes.QuoteIf you KNOW the size is 256 bytes, will always be 256 bytes, even if ported to a 64 bit machine, and you rely on wrap-around ... then for fuck sake .. yes OF COURSE you'd use uint8_t. Because you know what it is. No one here is going to try to argue against that.size_t is for things that you don't know what it is and you WANT it to adapt.QuoteZero terminated strings can be longer than 255 bytes, even on an AVR (especially if they're in ROM). Using something smaller than 16 bits for strlen() would be simply wrong.QuoteNo one is advocating blindly using size_t everywhere.

author: radiolistener
author_rank: Super Contributor
timestamp: July 17, 2025, 03:56:38 pm
content: Quote from: brucehoult on July 17, 2025, 02:27:05 pmWhy waste bytes? Use uint8_t for strlen!If string is longer than 255, just return 0xFF.So caller can add 0x100 to pointer and calls strlen() again.Repeat as needed. Simple, elegant, efficient!(joke)

author: NorthGuy
author_rank: Super Contributor
timestamp: July 17, 2025, 04:45:24 pm
content: Quote from: brucehoult on July 17, 2025, 02:27:05 pmThat's the whole point. You design and write code for things as they are, not as they could've been. You know the size of your buffers and you choose the type accordingly.Quote from: brucehoult on July 17, 2025, 02:27:05 pmIMHO, you can write much better memcpy() when you know what you're designing for - then you can write really efficient memcpy() in assembler (read Agner Fog for some examples). The generic one will be much slower than the compiler-supplied memcpy() and thus doesn't have any value.Even if I was designing a generic memcpy() I would not use size_t. Instead I would create my own type so that the user could choose what they need for their particular design. Or, if the distribution is binary, I would use a fixed size as, for example, Microsoft did with their lstrlen() Windows API function which returns 32-bit result no matter where it runs.Quote from: brucehoult on July 17, 2025, 02:27:05 pmSo what? The question is whether they can be longer than 255 bytesin your particular design. For example, if you have a 64-byte buffer, you somehow obtain characters to fill in the buffer, you put the zero character at the end of a received string. Now you know it's no more than 255. You can either use stock strlen() and assign the result to uint8_t which will dismiss the higher bytes, or you can write your own strlen(). If you use AVR, then your uint8_t constricted code will be somewhat more efficient at the expense of writing your own strlen(). If you use it in 100+ projects the effort of writing your own function is immaterial. But if you use ARM there will be no benefits.

author: brucehoult
author_rank: Super Contributor
timestamp: July 17, 2025, 05:06:54 pm
content: Quote from: NorthGuy on July 17, 2025, 04:45:24 pmYou are completely ignoring that people write libraries that are used in more than one situation. And other people use them.Incidentally you are also wrong on the efficiency.Take this code:Code:long get(long *p, int8_t i){return p;}I've used long as the array element type just to force a multiply by 4 for the index. That means it also needs to load four bytes for the return value, but that will be the same across all versions.Compiled with AVR gcc 15.10 with -O1. I've added comments.Code:get:// sign extend indexmov __tmp_reg__,r22lsl r0sbc r23,r23// multiply index by 4lsl r22rol r23lsl r22rol r23// add scaled index to base, move to Z registeradd r24,r22adc r25,r23mov r30,r24mov r31,r25// load the resultld r22,Zldd r23,Z+1ldd r24,Z+2ldd r25,Z+3retBut of course int8_t is stupid. Array indexes can't be negative.Code:get:// zero extend indexldi r23,0// the rest is identical// multiply index by 4lsl r22rol r23lsl r22rol r23// add scaled index to base, move to Z registeradd r24,r22adc r25,r23mov r30,r24mov r31,r25// load the resultld r22,Zldd r23,Z+1ldd r24,Z+2ldd r25,Z+3retThat's better.How about using the awful size_t?Code:get:// nothing to do here!// the rest is identical// multiply index by 4lsl r22rol r23lsl r22rol r23// add scaled index to base, move to Z registeradd r24,r22adc r25,r23mov r30,r24mov r31,r25// load the resultld r22,Zldd r23,Z+1ldd r24,Z+2ldd r25,Z+3retIf you're using C, then size_t is the shortest code.If you're writing in asm then you could make a shorter sequence for uint8_t, but ONLY if you know there are no more than 64 elements in the array. In theory a compiler could do that too, but the compiler we actually have doesn't.

author: Nominal Animal
author_rank: Super Contributor
timestamp: July 17, 2025, 06:43:53 pm
content: Quote from: brucehoult on July 17, 2025, 05:06:54 pmThat is exactly what I mean by "practical portability", too: what matters is what we end up with. I myself here focus on avoiding the worst pathological cases instead of chasing the "best" cases, often testing both gcc and clang on a couple of different architectures, but brucehoult's example shows exactly why and how.Another example is writing anend_of_string()function using either array indexing,Code:const char *end_of_string(const char *s) {if (!s)return NULL;size_t i = 0;while (s != '\0')i++;return s + i;}or using pointers,Code:const char *end_of_string(const char *s) {if (s)while (*s != '\0')s++;return s;}or using a library function,Code:const char *end_of_string(const char *s) {return (s) ? s + strlen(s) : NULL;}This is particularly sensitive to optimization flags (-Osvs.-O2), compiler, and compiler version.None of the compilers I use generate horrible code for any of the three variants in any case, so I'm happy with all three. In general, thebestoptimization is one that avoids the function call altogether, of course.Examining the assembly code different compilers generate for different targets using different options illustrates thelimitationsof compiler optimization engines; Compiler Explorer at godbolt.org is a perfect free tool for this. If you happen to have some critical code, it is a good idea to add it as a separate "porting/" folder within your project, with an up-to-date README file explaining why a specific variant is chosen compared to alternatives, and revisit the topic whenever you update compiler (major) version, change optimization flags for the project, or switch target architectures.Using DJB2 Xor hash function for human-readable strings is a good example of avoiding work and repeated function calls. A common approach would be to have one function to obtain the string length, and another to calculate its hash, but a better one does them at the same time. DJB2 Xor itself is an extremely simple hash function, but has a pretty good distribution across alphanumeric ASCII strings, and is very suitable for hash tables of string tokens. One could even include it in an UART character-by-character input state machine, calculating the hash and length for each command token (and converting numeric parameters to suitable integer types).

author: NorthGuy
author_rank: Super Contributor
timestamp: July 17, 2025, 08:00:39 pm
content: Quote from: brucehoult on July 17, 2025, 05:06:54 pmHow's that? For example a TCP/IP stack library work with entities of known size. IP packets have the same size regardless of where you process them. I have already written about this earlier in the thread.Quote from: brucehoult on July 17, 2025, 05:06:54 pmWhy would you want to force multiplication? Perhaps because it requires loading 0 into a register, while addition doesn't. Very cleverQuote from: brucehoult on July 17, 2025, 05:06:54 pmNot really. The zero needs to be loaded into a register anyway. In case of your size_t function, it is loaded outside of the function when loading the parameters. In case of uint8_t it is loaded within the function. That's all the difference.You only analyzed loading, but there are much more things done with the indices. Look at more realistic example. Say, you want to load data from a 64-byte buffer which is simultaneously filled in inside an interrupt handler:Code:typedef size_t index_t;uint8_t buf;volatile index_t head, tail;void check_buffer(void) {while (head != tail) process(buf[tail++&0x3f]);}Here's what it did with size_t for the check_buffer() function at -O2 -fno-inline:Code:rjmp .L7.L5:lds r30,taillds r31,tail+1mov r18,r30mov r19,r31subi r18,-1sbci r19,-1sts tail+1,r19sts tail,r18andi r30,lo8(63)ldi r31,0subi r30,lo8(-(buf))sbci r31,hi8(-(buf))ld r24,Zrcall process.L7:lds r18,headlds r19,head+1lds r24,taillds r25,tail+1cp r18,r24cpc r19,r25brne .L5retAnd here's uint8_t version:Code:rjmp .L7.L5:lds r30,tailldi r24,lo8(1)add r24,r30sts tail,r24andi r30,lo8(63)ldi r31,0subi r30,lo8(-(buf))sbci r31,hi8(-(buf))ld r24,Zrcall process.L7:lds r25,headlds r24,tailcpse r25,r24rjmp .L5ret16 instructions compared to 22 (if I counted correctly).Moreover, with size_t you're facing another problem. tail and head loadings and tail++ are no longer atomic, so an interrupt may happen in the middle of a loading or incrementing. This has a potential to screw up things because the interrupt handler may modify head in the middle of its loading, or read tail in the middle of the increment. So, you now need to make sure this doesn't cause any harm.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 17, 2025, 09:14:00 pm
content: Quote from: NorthGuy on July 17, 2025, 08:00:39 pmReal code would need a sequence point before increment, but this doesn't change much:Code:typedef size_t index_t;uint8_t buf;volatile index_t head, tail;void check_buffer(void) {while (head != tail) {process(buf[tail&0x3f]);tail++;}}

author: bson
author_rank: Unknown
timestamp: July 18, 2025, 07:51:58 pm
content: Yes, I use size_t for memory range sizes and typically convert to it early and use it for address arithmetic.Obviously it's machine dependent, so can't be used where a size is independently defined, such as, for example, the 16-bit datagram size field of an IP header. That's clearly a uint16_t. But it gets turned into a size_t when used, like 'const size_t iphlen = frame.iph.len;' I don't mix 16-bit and 32-bit, or 64-bit, address calculations and memory block sizes - that path only leads to the stuff nightmares are made of with implicit conversions and whatnot. Stick to the machine sizes. Religiously.When passing around virtual addresses I similarly use either void* or uintptr_t, for similar reasons. Although I do admit to sometimes being lazy and converting to and from 'const uint8_t*' for address calculations, mostly out of habit, but any machine that doesn't work on is also dead to me.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 18, 2025, 09:38:51 pm
content: When implementing shared queues, I strongly recommend using atomic types and operations (when available) rather than just volatile.stdatomic is available on all ARM Cortex-M except the M0/+, and on all RISC-V supporting the A extension (which is very commonly implemented even on small MCUs). And of course on anything x86-64.On the M0 or other targets that do not directly support stdatomic, that can be emulated as was discussed in a previous thread.

author: westfw
author_rank: Super Contributor
timestamp: July 20, 2025, 04:42:56 am
content: I do get annoyed by the warnings I get when I have a function that returns "index, or -1 if not found" when I try to use the (signed) result (having already checked for < 0) with functions that want size_t(if I'm remembering the annoying case correctly...)

author: gf
author_rank: Super Contributor
timestamp: July 20, 2025, 03:23:29 pm
content: Quote from: SiliconWizard on July 18, 2025, 09:38:51 pmBasically, you should use atomic types since they are the only way how the language standard guarantees atomicity. However, there is still a caveat: not all atomic operations are guaranteed to be lock-free for every type. If the required native instructions are unavailable on the platform or an atomic object is not properly aligned (for example, if it is a member of a packed struct), the compiler can use locks to achieve atomicity. Therefore, a program that requireslock-free atomic operationsfor particular types would still not work on a specific platform if these operations are unavailable on that platform. At least the atomic library allows you to test the availability in a machine-independent way, so that you can add corresponding assertions to the program.

author: bson
author_rank: Unknown
timestamp: July 20, 2025, 08:31:30 pm
content: Quote from: westfw on July 20, 2025, 04:42:56 amC++:Code:typedef unsigned int size_t;enum : size_t { NOT_FOUND = (size_t)-1 };extern size_t memsize;bool initialized() { return memsize != NOT_FOUND; }

author: NorthGuy
author_rank: Super Contributor
timestamp: July 20, 2025, 08:53:53 pm
content: Quote from: gf on July 20, 2025, 03:23:29 pmYou need to show compiler your intentions and all that.You know, I just went to ChatGPT, asked it to write an efficient and thread-safe 128-byte circular buffer. And it produced wonderful code, which is politically correct to the highest possible point, covering every little detail and providing the safest code you can imagine. It uses atomic_size_t!!! ChatGPT also told me that this is as efficient as it gets with C11. Just look at this:https://godbolt.org/z/v8YbebbfMThen I wrote a straightforward implementation, which is still thread-safe and portable (well, if you wanted to run it on a multi-core machine you would need to add memory barriers) but much simpler. I didn't test it, so it may have bugs, but I don't see any:https://godbolt.org/z/9cjbd4xf8It does produce 5 times less code on AVR and will run much faster, but even if it didn't I would take mine over ChatGPT's any time. In engineering, simplicity is key.

author: peter-h
author_rank: Super Contributor
timestamp: July 20, 2025, 09:07:27 pm
content: QuoteDo you mean the compiler might use a mutex in some hidden way? How would that work?We have a thread on this, on which arm32 types are inherently atomic in storage. A single float is but a double is not. A int32_t is but a int64_t isn't.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 20, 2025, 10:02:40 pm
content: Quote from: peter-h on July 20, 2025, 09:07:27 pmOn ARM32, there are LDREX and STREX instructions. They are used in pairs and must access the same value. First you do LDREX to read a value, then you do STREX. If someone else accessed this memory location in between them, STREX fails and you need to start over.For example, you can use these for an atomic increment. You read the value with LDREX, increment it and try to write it back wih STREX, but if some other thread accessed the memory in between, STREX will fail, so you loop back to LDREX and keep trying.Similarly, you can organize exclusive access to something. You allocate a control word in memory. You write there either 0 (meaning the shared object is currently free) or 1 (meaning it is used). When you want access, you can read the value of the control word with LDREX and if you get zero then your try to write back 1 with STREX. Once this succeeds, you've got your object and can do anything you want with it. But if STREX fails, some other thread got the object first, and you cannot touch it. If you did get the object, once you're done, you write 0 to the control word thereby releasing the object and making it available to others.These instructions should be used by RTOS, so just look at their code. It'll explain things better than words.

author: gf
author_rank: Super Contributor
timestamp: July 20, 2025, 10:34:55 pm
content: Quote from: peter-h on July 20, 2025, 09:07:27 pmGCC does not use a mutex directly, butif it cannot generate lock-free inline code, it emits a call to an external support function in libatomic. And this function may use a lock. For example, on Linux, libatomic has external references to pthread_mutex_lock() and pthread_mutex_unlock().Code:$ nm /usr/lib/gcc/x86_64-linux-gnu/6/libatomic.a | grep -i threadU pthread_mutex_lockU pthread_mutex_unlockI don't know how libatomic realizes locks on other platforms, e.g. RTOS.From the GCC documentation:https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.htmlQuotehttps://gcc.gnu.org/onlinedocs/libstdc++/manual/ext_concurrency_impl.htmlQuoteEDIT: Look at this example:https://godbolt.org/z/WM13nr7aqBoth functions increment the value atomically with respect toother threads, but the 64-bit variant eventually uses a hidden lock to achieve atomicity. Also note that only the lock-free 32-bit variant is signal-safe (interrupt-safe) when it is called from a threadandfrom a signal handler (or interrupt handler).

author: cv007
author_rank: Frequent Contributor
timestamp: July 20, 2025, 11:45:18 pm
content: QuoteThat excludes many microcontrollers typically discussed on this forum (such as Cortex-M0, AVR), but if available the use of atomic libraries seems to require some effort to make sure they are used correctly (I never have it available, so no experience using these libraries).A simple example-https://godbolt.org/z/T165hGPed//avrhttps://godbolt.org/z/3T8bazsaK//cortex-mThe problem here is it has 2 vars that need to change as a pair- count_ cannot be out of sync with inIdx_/outIdx_. Treating each atomically does no good as you then left an opportunity for both to be out of sync. It may be, depending on the mcu in use, that some of these vars can be read atomically (16bit var on 32bit mcu) and there are ways to check that at compile time but helps very little because you still have write(s) to protect.Other variations of code could also be created, such as pointers instead of count/index, but you still end up with several reads (the pointers) and at the very least a single write to protect.It seems to me the simple solution where you cannot get it wrong, is to just disable interrupts for the 'transaction' (append/remove in this case). The examples above do that, and it will work the same whether on an AVR, Cortex-M0 or a Cortex-M33. No real need to micromanage the differences each mcu may have and break up the problem into separate parts since one solution can take care of all of them, with a major advantage in that there is no doubt about whether it is correct in all cases. Interrupts, rtos, or whatever else is going on in the mcu these 'transactions' cannot be split.The cost is maybe a dozen or so cpu cycles where interrupts were disabled, but you most likely have the same or worse already in place where interrupts are effectively disabled because you are already in another interrupt of the same priority or higher. I can't speak to the use of atomic libraries but I find this interrupt protection solution satisfactory for my needs, whether an avr or a cortex-m33.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 21, 2025, 12:46:33 am
content: Quote from: cv007 on July 20, 2025, 11:45:18 pmThere's much easier solution that doesn't disable interrupts - instead of maintaining count, calculate it on-the-go.It makes for shorter code as well. Here's my example from an earlier post modified to deal with buffer of pointers (or objects if you will) on ARM32:https://godbolt.org/z/vedaT7j6GUsually it's not hard to find a solution that doesn't require disabling interrupts, let alone on ARM where you can use LDREX and STREX.

author: cv007
author_rank: Frequent Contributor
timestamp: July 21, 2025, 03:14:35 am
content: I'll focus on a single thing I picked out of the code. I could be wrong but don't think so (and part of the problem for me is it is not obvious this code does not contain a problem).Look at the line marked ---interrupt---. This is after the buffer has been written, but before the head value was stored. I think its safe to assume this buffer could be storing events produced in an interrupt. Someone is adding an event to the buffer, it gets added, interrupt occurs at the ---interrupt--- line in the code below, this new code that is now running (in an isr) also adds an event to the buffer, but since head was not updated it overwrites the event previously just added, successfully updates head, returns, previous code updates head. We have just lost an event as it was overwritten.Code:circular_buffer_write:ldr r3, .L4ldr r2, ldr r1, [r3, #4]subs r2, r2, r1cmp r2, #128beq .L3ldr r2, and r2, r2, #127add r2, r3, r2, lsl #2str r0, [r2, #8]ldr r2, movs r0, #1adds r2, r2, #1----interrupt----str r2, bx lr.L3:movs r0, #0bx lr.L4:.word .LANCHOR0As I said, maybe I am missing something but do not think so. I also did not look at any other parts of the code so do not know if there are other problems if any, but suspect there is.Another difference is the C++ version is a single piece of code that will work on avr/cortex-m for as many buffers as you want, where this C code is a single buffer of a single type geared toward a 32bit mcu. A C version can certainly do the same, but it will be different code than the example given.I think we can agree (maybe) that you can look at the C++ version and say there can be no problems due to interruption as its obvious it cannot be interrupted. The C code shown requires some thought, even when picking out a single part of the code which I have done.edit- the reason I prefer to keep a count is so I do not have to deal with calculations on where the head/tail are in relation to each other. Its a simple increment/decrement and eliminates the other calculations, and that extra var is meaningless in ram usage even if a dozen buffers are in use. I also have long ago dropped the notion that all buffers have to be a power of 2. That may have been necessary when mcu's were very limited and writing was done in asm, but see no need to require all buffers sizes be rounded up the next power of 2 (and the next power of 2 up starts to be big jumps soon enough). I would note that the C++ append function is 4 more lines of asm code (did not compare cpu cycles) than the C circular_buffer_write function, which adds interrupt protection and the ability to use any size buffer.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 21, 2025, 04:33:25 am
content: Quote from: cv007 on July 21, 2025, 03:14:35 amOf course. This particular code is for a single consumer and a single producer. All writes are done in the main, and all reads are done in the interrupt. Or, all reads are done in the main, and all writes are done in an interrupt.If you need to write from different IRQ levels, you will need to modify the code (there are different ways to do that depending on the situation), or perhaps put all the writers on the same IRQ level. For example, in Windows there's so called DPC level where most of the things run. It is below the hardware interrupts which need urgency, but above threads.To me disabling interrupts is not a good thing - the usual purpose of the interrupt is to do something urgent that cannot wait.

author: brucehoult
author_rank: Super Contributor
timestamp: July 21, 2025, 04:39:50 am
content: Quote from: cv007 on July 21, 2025, 03:14:35 amYeah, there is no significant difference between...Code:idx++;idx &= (SZ-1);... and ...Code:idx++;if (idx == SZ) idx = 0;The former requires SZ to be a power of two, the latter works with any value.

author: peter-h
author_rank: Super Contributor
timestamp: July 21, 2025, 06:41:42 am
content: I see thisCode:__STATIC_FORCEINLINE uint32_t __STREXW(uint32_t value, volatile uint32_t *addr){uint32_t result;__ASM volatile ("strex %0, %2, %1" : "=&r" (result), "=Q" (*addr) : "r" (value) );return(result);}in cmsis_gcc.h but nowhere else (not in FreeRTOS for example). It doesn't get used. It's off topic but FR disables interrupts to implement mutexes. I stepped through the code.I too much prefer keeping a count for a FIFO / circular queue.

author: cv007
author_rank: Frequent Contributor
timestamp: July 21, 2025, 06:49:03 am
content: QuoteI didn't specify my examples were to be used for any purpose, but it had interrupt protection and assumed your example was for the same universal purpose. Yes, problems disappear when there is only one writer and one reader. It also requires you do not forget this buffer code has those restrictions. I assumed you were trying to say you had an equivalent code that needed no interrupt protection, so had me confused and I was trying to be delicate to someone smarter than me.The use of the head/tail is clever but also requires the use of power of 2 sizes, and as soon as you remove that restriction you will eventually end up adding a count var to keep the code readable. I certainly am not opposed to something simpler than what I showed in the examples, but I don't think there is anything left to simplify and am not about to require power of 2 sizes simply to save a couple instructions.QuoteThen you cannot have more than a single interrupt at the same priority, as that also blocks for most likely a lot longer. The interrupts off I don't think is that big of a deal as the interrupt will still get serviced and the possible dozen cpu cycles it was delayed (most likely a rare occurrence), should not break the ability for it to do its job.I want buffer code that can be used for any purpose, add/remove anytime/anywhere- events, uart buffers, or anything else. I think the price is minimal to get that, and you end up with a single piece of code that is simple and you can easily see that any possible corruption is already prevented (no need to examine asm, no need to figure out size types, where the writes are taking place, who is accessing, no need for an atomic library which you may not have, and so on).

author: Siwastaja
author_rank: Super Contributor
timestamp: July 21, 2025, 07:10:11 am
content: Quote from: NorthGuy on July 21, 2025, 04:33:25 amI prefer to have numbers in my engineering, and don't agree with the idea of interrupt disabling being "generally" bad just because it increases the latency of servicing of "urgent instructions" from, say, 100 nanoseconds to 150 nanoseconds, and jitter from, say, 10 nanoseconds to 50 nanoseconds. Of course, you want to keep the section which disables the interrupts really short, and if it's more than just reading, writing or RMWing a few words, then something is being done wrong.In some very specific case that 50 extra worst case nanoseconds, or added jitter of 40 nanoseconds, might be a problem. In such cases I have always managed with priority levels. Instead of disabling all interrupts, you can do stuff like, for example on ARM Cortex-M, of disabling only the priority levels above 2 using BASEPRI, or disabling just the individual interrupt that needs to be protected against.

author: peter-h
author_rank: Super Contributor
timestamp: July 21, 2025, 07:18:05 am
content: Disabling ints is fine. CPUs these days are so fast. I timed a ~50 line ISR at several microseconds, which is nothing relative to servicing UARTs, DMAs, ADCs, DACs etc.If this is a problem then the system design is probably defective.FreeRTOS disables them for maybe 1us.I've still never used size_t and can't understand the need

author: Siwastaja
author_rank: Super Contributor
timestamp: July 21, 2025, 09:14:56 am
content: Quote from: peter-h on July 21, 2025, 07:18:05 amNah, not defective, nothing wrong in trusting the latency of ISR entry itself - then just never disable that interrupt priority level, and don't assign the same priority to other interrupt sources, simple as that. I have done that a few times, some protection stuff in switch mode converters.Now granted if you have many of those that all need to operate in parallel, then indeed you need something else. Maybe an expensive FPGA, or how about XMOS XCORE.QuoteIt's for writing reusable/portable things that need to iterate over small or large arrays. That way you can support e.g. count of 5 billion, without having to use uint64_t which would be waste on smaller CPUs that can't support the count of 5 billion anyway.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 21, 2025, 01:03:25 pm
content: Quote from: cv007 on July 21, 2025, 06:49:03 amActually, there's only one interrupt you should worry about - the one at the highest priority. Other interrupts may get delayed by the time necessary to process the highest priority interrupt. Therefore all interrupt except one will already have considerable latency and jitter.For example, if you have a highest interrupt executing 2 us and then you have a timer interrupt of lower priority, your timer interrupt already will have 2 us jitter because if the timer interrupt happens simultaneously with the high priority interrupt it will be delayed by 2 us. Extra delay of 200 ns won't make much difference.Therefore, disabling all interrupts except the highest priority will do no harm. If your highest priority interrupt does not require low latency/jitter, then disabling it for a short time will not do any harm neither.

author: cv007
author_rank: Frequent Contributor
timestamp: July 21, 2025, 01:44:48 pm
content: QuoteThere is always the M0/plus, not an uncommon version, which of course does not have that.I can live the with dozen clock delay. Existing priorities are still in effect, and the highest priority still gets serviced when done if was pending. I just delayed it a little on the rare occasion it happens to fire inside these functions.

author: westfw
author_rank: Super Contributor
timestamp: July 22, 2025, 02:00:22 am
content: QuoteDoes disabling interrupts have an effect on pipelines?

author: John Coloccia
author_rank: Super Contributor
timestamp: July 22, 2025, 05:19:24 am
content: Quote from: westfw on July 22, 2025, 02:00:22 amI think the broader question is "does disabling interrupts have an effect on your application." Which, in some cases, might be yes. For example, a BLDC controller we did. In other cases, you're using interrupts to do things like DMA, communication, etc, and a delay is irrelevant.Not picking on you at all, btw. I get what you're pointing out and why, and it's totally valid. It's a good question, actually. I don't actually know the answer, or even if there's a generic answer. I'd guess that it does what you might expect...everything before the disable is theoretically up for grabs, maybe even subject to flushing and just start over when you return (though it's already in the pipeline, so not sure what good flushing would do). Everything after is sacred.Good question.

author: Siwastaja
author_rank: Super Contributor
timestamp: July 22, 2025, 06:05:57 am
content: Quote from: John Coloccia on July 22, 2025, 05:19:24 amIMO (having done a few BLDC controllers) it's an example of not-so-timing-critical thing. Like, your control ISR is usually triggering once per PWM cycle, so around 20kHz. There you read ADC inputs, do a few trivially small calculations, and output to PWM control registers, which are buffered and will apply for the "next" cycle. So even if the 50µs period interrupt gets sometimes delayed by 10µs, it still isn't a problem. And 10µs in huge.I remember doing a software DC/DC with switching frequency closer to 200kHz where I did do a specific shutdown sequence on inductor saturation with software. That is one of those rare cases where I was able to show that the 12-cycle interrupt entry latency (plus some cycles to actually do the IO operations, maybe ~10) was sufficient, but without a huge margin. Disabling interrupts for another, say, 20-30 cycles, would not have been acceptable. But these cases are rare. Someone else would have used external glue logic, I don't know. I had never any problem with this. I just never disable this interrupt level.

author: uer166
author_rank: Super Contributor
timestamp: July 22, 2025, 07:46:24 am
content: Quote from: Siwastaja on July 22, 2025, 06:05:57 amWith more modern MCUs with uber fast >5MSPS ADCs you could do main control loops (like peak current or hysteretic) purely in ADC conversion complete ISR. In that case you'd not onlynotdisable ISRs, but I suppose you have to micro-optimize ISR entry and register stacking.It would be a fun way to sidestep having custom timer and PWM setups via brute force, something like PIC32MK comes pretty close for modest frequency converters.. TI also solves this issue also using smaller independent cores ("Control Law Accelerators")..

author: NorthGuy
author_rank: Super Contributor
timestamp: July 22, 2025, 02:03:28 pm
content: Quote from: uer166 on July 22, 2025, 07:46:24 amPIC32MK may not be a good candidate for this. It uses MIPS core which does not have native interrupt levels, rather they emulate levels in software. Even when a lowest priority interrupt is starting, all interrupts get disabled until the software saves the context and re-enables interrupts.Look at dsPIC33AK or PIC32AK instead. They have 40 Msps ADC and a powerful CISC core, so that you can do a lot with a few assembler instruction you can squeeze into a fast interrupt. Also they have several DACs, analog and digital comparators, logic cells, Op Amps etc., so that you can offload lots of stuff to periphery.

author: cv007
author_rank: Frequent Contributor
timestamp: July 31, 2025, 06:07:15 am
content: Just a thought experiment with circular buffers-I was looking at some popular mcu projects like mbed, zephyr, nuttx, and some others (I don't use any, not sure if mbed is much of a thing anymore). For mbed, they interrupt protect the whole push/pop functions, and zephyr ring buffer documentation indicates concurrency is left up to the user, nuttx also leaves concurrency up to the user. With an os it may be easier to deal with using the available locking options, although that has its own downsides. Only nuttx is using size_t in the buffer code, while zephyr is using defines for sizes and size types, and mbed is using a template parameter for a size type. My example linked below uses what mbed does, where the user can optionally specify the count type (default uint32_t) so for example if the code is used for an avr and the count value is <256 a uint8_t can be specified as the count type (mbed asserts the count type can contain the template parameter specified buffer size, my example simply lets the compiler error). This also does not requite something like the type_traits header, which the avr will not have available.There is one caveat with the simple code example I showed before (whole function interrupt protected, 6 lines of code each for append/remove)- since templates are in use, it allows common simple types like any other but it will also allow types like structs. All works the same using the same code (advantage C++), but if the size of the buffer type gets beyond a certain size the compiler may start calling memcpy and becomes a good indication the size of the buffer type is no longer small and the complier is asking a library function to do the job. And, whether that starts to become a problem would depend on the application, but it will then be taking more time (with interrupts disabled) than the simple types.Coming up with a method to allow higher priority interrupts to fire inside the append/remove, and also allowing that interrupt (or interrupts) to also append/remove is an easy thing to get wrong. I think this code would work, and is about as simple as I could think of (keep in mind this is an M0 and no OS in use, wanting to allow higher run levels to append/remove)-https://godbolt.org/z/bc49x8oTn(edit- removed wasBusy bool from functions, iCount can do the job, made no difference to times but code looks simpler)I don't know if this is an original idea or not, and I'm sure circular buffers have been done to death by now, but I don't recall seeing anything like this in the mcu world before. Could be its not that useful and shouldn't be seen.There is not much of a difference in maximum interrupt disable time between the previously shown simple Q class when the buffer types are 'single' types (char/uintN_t/etc.), but if the buffer type is more complex such as the struct S example which has an event enum and a u64 time, and which may end up using memcpy, then interrupt disable time increases to more of an unknown depending on how big the struct is and/or how long memcpy takes (and which version is linked in), where the newer Q class remains about the same interrupt disable time regardless of type. So, I guess the slightly more complex Q class is harmless if not really needed, but could eventually be helpful as it will allow for interruption during the copy or memcpy, allowing a higher irq level to run sooner and the interrupt can also use append/remove on the same buffer if needed. This may be more beneficial for something like an avr, which is probably on the slower side already, and is also slower dealing with anything larger than 8bit.I do not know for sure whether this example is without a flaw, but thinking it through thoroughly it seems to be ok. I do have this code running on an stm32, with a loop appending and removing, along with several timer interrupts at different priorities also doing the same (1000Hz, only one is highest priority). They all write fixed/unique values (buffer type is a struct of 2 u64's, 2 u16's), and the removes are checked to see if both u64 values match and are one of the unique values. For instance, the main loop appends {0x1111111111111111, 0x1111111111111111,0,0}, other timer interrupts use 2's , 3's, etc. The highest priority timer also records its counter when it enters the isr as a means to gauge how soon the isr code started to run after overflow, and updates the count min/max values seen (put in the buffer struct 2 u16's). This test has been running for a while (days) without any corruption, so at least it seems to be ok in use also. Whether it is that useful is a different question.The max response times I get from the highest priority timer (via its counter) using the new Q class is about half of the max response time of the original Q class (which protected the whole function). I think that makes sense, as irq's are given an opportunity to run during the copy phase which probably is roughtly the halfway point (initial setup/protected, copy/unprotected, completion/protected).

author: NorthGuy
author_rank: Super Contributor
timestamp: August 01, 2025, 02:53:12 pm
content: Quote from: cv007 on July 31, 2025, 06:07:15 amInteresting idea. Looking at this code:Code:MAKE_CALLED T* //normally MAKE_INLINEnext (AR& ar){CPU::InterruptLock lock; //all below done atomicallyif( ar.n == 0 ) return nullptr; //full (append) / empty (remove)T* tp = &t_[ar.idx]; //pointer to current t_ for append or removeif( ++ar.idx >= N ) ar.idx = 0; //inc for next append / remove--ar.n; //--used (remove) or --free (append)ar.iCount++; //inc iCount (will now be >=1)return tp;}MAKE_CALLED auto //normally MAKE_INLINEcommit (AR& from, AR& to){CPU::InterruptLock lock; //all below done atomicallyif( from.iCount > 1 ) return true; //we interrupted another append or remove, so nothing more to doto.n += from.iCount; //inc to.n total count of append/removes (1 or more)from.iCount = 0; //clear iCountreturn true;}Some tweaks are needed. For example, if two appenders call "next", iCounter will become 2. Once this happens, the "commit" will always return "true" and iCount will never decrement. You probably need two counters - one for busy appenders (the ones who have called "next", but have not called "commit" yet) and the other for finished appenders (who have called "commit" already, but have not been commited because other appenders were active at the time).

author: cv007
author_rank: Frequent Contributor
timestamp: August 01, 2025, 03:28:58 pm
content: QuoteI did have a local var called 'wasBusy', which of course would be exclusive to each 'append'er/'remove'er. At some point I decided that iCount could do the same job, but that is a 'global' var. Funny I can run my tests hammering append/removes without problems, but my test could also be flawed.My previous version, where the 'wasBusy' local var is used-https://godbolt.org/z/v1na1cMWM//line 166 was badhttps://godbolt.org/z/Yx5T7zTTohttps://godbolt.org/z/obqzhq8ve//not a big deal, but the wasBusy check in commit can also be done before the interrupt lockI also need to keep track of how many appends/removes are successful in case it gets 'stuck'. Its possible my removes were doing nothing, and although I would 'dump' the buffer every 10 seconds just to get a view of its values it may have been showing the same values over and over and did not notice.As I said, easy to get wrong, both the code and the testing. And could still be wrong.edit- and is still wrong- I now keep track of total removes, and that value is not advancing. If I change to using the simple Q class, I get >100,000 successful removes every 10 seconds, so I think my test code is valid and I just have a problem with the new class.edit- new class now working correctly, I had left in a slight code change I was trying earlier but did not ctrl+z my way back all the way. I normally try new ideas with a copy then #if 0 the original, but I guess I thought this change was simple enough to try quickly. Now getting >100'000 removes in 10 seconds just as the simple Q class was.Thanks for pointing out the flaw. I sort of knew switching from from a local var (wasBusy) to using iCount would now be using a 'global' var but somehow thought because it was interrupt protected it would be ok. Bad thinking. I try to reduce the code until there is no more to reduce, and replacing wasBusy with the use of iCount went one step too far.

author: NorthGuy
author_rank: Super Contributor
timestamp: August 01, 2025, 05:22:27 pm
content: I would do something like that (with both of the counters initially reset to 0):Code:MAKE_CALLED T* //normally MAKE_INLINEnext (AR& ar){CPU::InterruptLock lock; //all below done atomicallyif( ar.n == 0 ) return nullptr; //full (append) / empty (remove)T* tp = &t_[ar.idx]; //pointer to current t_ for append or removeif( ++ar.idx >= N ) ar.idx = 0; //inc for next append / remove--ar.n; //--used (remove) or --free (append)ar.iAllocated++; //inc iCount (will now be >=1)return tp;}MAKE_CALLED auto //normally MAKE_INLINEcommit (AR& from, AR& to){CPU::InterruptLock lock; //all below done atomicallyif (from.iAllocated == ++from.iCommited) {to.n += from.iAllocated;from.iAllocated = 0;from.iCommited = 0;}return true;}

author: cv007
author_rank: Frequent Contributor
timestamp: August 01, 2025, 06:25:23 pm
content: I now have that running, and works ok also.The change is in the class Q1-https://godbolt.org/z/Mdvsj3zMM(I changed to an early return in 'commit', as I just like keeping indented code to a minimum- compiler doesn't care either way)it appears the 'next' function spends a few less instructions with interrupts off with this change (I didn't count cycles, but is something like 16 vs 19 instructions).