Want a Rant? (RTOS and "libraries" vs. the dark side) - Page 1
Forum section: electronics / general

author: wek
author_rank: Frequent Contributor
timestamp: May 20, 2023, 08:48:13 am
content: Over at the STM32 forum, Dave Nadler asked:STM32F4 library issues fixed?. He started with mentioning past problems withtoo many problems with ST-provided drivers/stacks, especially Ethernet/lwip and USB, and very poor FreeRTOS support. Lots of Ethernet driver problems, and the USB CDC stack had a nasty habit of losing packets.In response, I wrote this:I see many RTOS-related issues and rants (includingDave's own printf() rant).My stance is, that RTOS-s - and, for the very same reasons, "hardware-layer libraries" - are to be simply avoided in microcontrollers.The reason is, that they simply exchange one source of complexity (perceived or real) with a different - and less obvious - one. In doing so, they consume resources, introduce limitiations (inevitably crippling down innate capabilities of hardware to the least common denominator of all covered cases, often defended by "good enough"), and, being relatively complex, they simply contain bugs. Their usage also requires to align one's mindset to their creators', which is a learning process, and, as such, requires significant effort and simply lots of time (yes, years). Nobody talks about this publicly - on one hand it's a dirty secret for the knowledgeable, and those who are in opposition are publicly intimidated by being told to be unable to recognize the benefits and straight stupid to learn new things. (I try to fight back by publicly acknowlidging, that yes, I am too stupid to learn these new things 🙂 ) There's also significant monetary incentive to those who propose these tools.Often, the appeal of these tools is enhanced by providing plenty ready-made templates for many use cases (and, in case of Cube/HAL, a shiny clicky interface), and impressive examples. The limitations involved in using those templates in one's work are not obvious at all.What makes users of these methods confused is the lightness with which certain - arguably, the most common - cases can be solved. These tools are designed so that the initial threshold is low, efforts initially appear to be reasonable, results are instant and impressive. The real learning curve is delayed to the point of no return.There's of course also the argument, that this is what made the "big" computers what they are today. Sure, except that microcontrollers are a different world. "Big" computers are in comparison to mcus in principle incredibly simplistic, requirements put on them are very extensive but relatively trivial (megabytes (okay, tera these days), megahertzs, megapixels solved themselves thanks to Gordon Moore, RIP - whereas microseconds, single bits and single pin/analog are the really hard stuff) - and by unification the economy of scale starts to show. Remember, how long it took a multi-billion company to produce a half-decent multitasking operating system.What's your budget?2 eurocents.JWPS. My strong impression is, that Adam Dunkels's early work, not just lwip, was in line with these lines, i.e. the point of his works was to help avoiding RTOSes in microcontrollers. Eventually, he too gave in, as there's no money in proposing blood, toil, tears and sweat.

author: nctnico
author_rank: Super Contributor
timestamp: May 20, 2023, 08:59:16 am
content: Quote from: wek on May 20, 2023, 08:48:13 amNot true. Just don't use anything from ST! If you pay peanuts, you get monkeys. Consider ST provided software to be example level software at best.You can use ethernet stacks and RTOS on microcontrollers just fine but you have to make sure to use production ready code. And ofcourse you need to know what you are doing. FreeRTOS in itself is perfectly mature. There are other manufacturers that offer production ready code to use on their microcontrollers. Sure you pay a bit more for the microcontroller but NRE costs are much lower.

author: westfw
author_rank: Super Contributor
timestamp: May 20, 2023, 09:27:07 am
content: QuoteWhere do you draw the line as to how big a microcontroller has to be to support an RTOS?After all, many modern microcontrollers have upwards of a megabyte of flash and 256k of RAM; rather significantly more than the larger minicomputers of yesteryear that were capable of running multitasking multiuser operating system, even with much slower CPUs. Some of them have better memory protection features as well.TCP/IP and multitasking in general were all developed on systems that were smaller and slower than an ARMCM4 with QSPI flash.

author: hans
author_rank: Super Contributor
timestamp: May 20, 2023, 10:36:04 am
content: I think the biggest difference is that on a MCU, all code you use is now your liability. For that old PC with similar capabilities that ran Windows or Linux, if something failed, then it's library XYZ or software package ABC its fault. Usually bugs are quite obscure and there are different upgrade paths/regimes for LTS, stable, unstable, etc. You wouldn't run Arch on an enterprise server, maybe it's fine for your NAS at home.On embedded, all the garbage of a vendor is on you. Things move slowly, so a "fix" for library XYZ may only appear months later, while you need it to work by next week for a MVP demo. And then backporting your fixes with their fixes is another mess to consider. Its unfortunate that vendor code can be downright bad that it's buggy. I sometimes wonder how the firmware team has ever validated if the silicon is OK.I think a positive tangent on this is:https://hackaday.com/2023/05/18/an-entire-risc-v-operating-system-in-2000-lines/It's admirable to have everything "visible" in one eyes view. The problem however I foresee is that these works, including some of the works that is referred to from Adam Dunkel's; is they are software packages with a philosophy in its purest form. Although its useful and educational, it also has some severe limitations. Protothreads for example can't do "blocking" statements outside of the original function, as its a syntactic sugar library. Protocol stacks are always a fuss, where I often apply the 20/80 rule: 20% of the time is spent on getting things to work, and 80% is on error handling. RTOS environments often identify themselves as being an one-stop shop, as virtually many libraries (in particular protocols) to be written rely on it. Does anyone use the POSIX or CMSIS-RTOS standard in embedded? I don't think so.To be honest I don't see these things changing quickly. Sometimes a new OS/Framework comes around that promises to do things "well", but it then draws into custom build systems, stretching code organization to its limits by stubbornly sticking to C, abstraction bloat, taking shortcuts by using malloc, etc.

author: tggzzz
author_rank: Super Contributor
timestamp: May 20, 2023, 11:27:37 am
content: Quote from: wek on May 20, 2023, 08:48:13 amThat's the key issue: running into a brick wall at some unpredicted/unpredictable point.There's no simple solution nor single solution.Pragmatism and good taste is required.Good taste is the result of forethought and/or experience.TANSTAAFL

author: peter-h
author_rank: Super Contributor
timestamp: May 20, 2023, 03:14:47 pm
content: I get your drift,wek, and largely agree with the actual situation we have with various open source code. I've ranted on about this herehttps://www.eevblog.com/forum/programming/a-rant-about-lack-of-support-on-open-source-software-or-anything-else-actually/msg4348588/#msg4348588But actuallyFreeRTOSworks very well, more or less out of the box, and in 2-3 years of my usage I have found zero issues. It performs a fairly basic function: switching tasks according to some rules. I use some of its other features (the dynamically allocated timers, which are handy) but don't use stuff like message queues, ISRs kicking off tasks, etc. There is minimal glue to the CPU (systick, and not much else). The website is quite readable.LWIPis a bloody nightmare. The function is inherently complex but lots of people try to use it because they "need internet". There is no support (well, it is open source soof coursethere is no support; it is > 15 years old so the devs moved on many years ago). The website is crap and there is a ton of API clickbait. The glue to the CPU (the ETH subsystem) is one of the most complex things in embedded, and possibly apart from rare narrow cases there is no solid code published. Not to say one can't get it to work but very hard to get it to work at decent data rates i.e. everything interrupt-driven. As an RTOS task, much easier. My project works fine but at probably 10% of the PHY data rate (but easily good enough). There are people on the ST forum who taunt everybody by pointing out problems, but never posting complete tested code. "Piranha" is the worst example but there are others. Dave Nadler is a good guy but again while his website lists various issues and suggested approaches, it is IMHO incomplete. None of this "slag this off" stuff is of much use unless people publish complete working code.MbedTLS(which tends to go with LWIP, due to the market demanding "security" even where meaningless) is horrible and someone else did the implementation in my project. It does work but if something breaks, finding it is almost impossible. You have to get it running and then freeze the code. The crypto suite variability issues will always be present, however.FatFSworks great and is fairly simple to integrate. The glue to the storage medium is quite simple.The situation withUSBis not good. I have MSC and CDC working OK. MSC was not hard to get going IIRC. CDC took much longer. Almost nobody understands USB, and it is highly complex.The problem is that the above stuff is really complex. Nobody will be writing it from scratch. So people have to use libs, and normally you dig around for open source initially.Etc.As I've often posted, most people develop their own fixes but never post them, either because they are just leechers or because they did in company time etc. So everybody else struggles...Getting back to RTOS, how else would one do it? I've been coding since ~1980. One did a "main loop" (i.e. a state machine) and then a) ISRs for comms etc and b) a timer ISR (say 100Hz on a 4MHz Z80) in which you did all "background" stuff. I developed dozens of products, some really complex, using that. But eventually, when working on a product which did protocol conversion on many channels concurrently, I wrote my own simple RTOS, which made everything much easier.The memory footprint of FreeRTOS is small. The main usage is in the task stacks, which run inside a RAM block which you give it. It runs a sort of private heap inside that, and you don't need to know anything about it. The gotcha is that if say you have a function (thread-safe one) which runs a 1k buffer on its stack, and you call it from 5 RTOS tasks, each of those tasks needs room for that 1k buffer. Obvious, I know. Well, you could use a static 1k buffer and mutex that function (might have fun and games with what academics call "priority inversion" then i.e. one task locking out others). In my project I gave the 64k CCM to the RTOS task stacks. So I have 128k RAM, plus the 64k for the RTOS.

author: Nominal Animal
author_rank: Super Contributor
timestamp: May 20, 2023, 05:28:40 pm
content: Selecting ones own tools wisely is a complex, complicated, and underappreciated art that is often replaced with rules of thumb, anecdotes, and unfounded beliefs.

author: Siwastaja
author_rank: Super Contributor
timestamp: May 20, 2023, 05:59:17 pm
content: Managers and even some developers tend to underestimate the effort that goes into reusing existing work, and overestimate the effort that goes into creating new work, biasing their ability to make correct decisions, resulting in projects that are seriously overbudget, overtime, and underperforming. Sometimes seriously underperforming system is accepted as a compromise when time simply runs out.This is not to say never to reuse existing work. Just how hard it is to evaluate when to do it and when not. And pointing out in which direction the error usually goes.Managers are not the only ones to make this mistake. Even developers who have always worked under such management learn this mindset. And if you never design and write your own application specific frameworks and tooling, you don't ever get better at that. If you always maximize amount of libraries and pre-existing frameworks used, the skills you learn are debugging them, working around their issues, copypasting usual solutions from Stack Overflow. This becomes a problem when youneedto design something non-trivial of your own anyway. It is painful when you lack experience.

author: Marco
author_rank: Super Contributor
timestamp: May 20, 2023, 06:15:17 pm
content: I just looked at a Bluetooth low energy stack ... 183 kLOC ...Just fight the library, sometimes there is no alternative.

author: peter-h
author_rank: Super Contributor
timestamp: May 20, 2023, 06:17:24 pm
content: I think business owners prefer to re-use code, while employees prefer to write new code. The reasons are obviousIt's a miracle that Bluetooth works...

author: Siwastaja
author_rank: Super Contributor
timestamp: May 20, 2023, 06:19:43 pm
content: Quote from: Marco on May 20, 2023, 06:15:17 pmOh I hate this argument.Because, what do you think it proves? It can prove two completely opposite things:* It's 183kLOC because it's bloated and overly complex piece of shit and not worth using, likely buggy and nightmare to debug, OR;* It's 183kLOC because what it does is so difficult and complicated to do, yet you need every LoC, so you obviously should use it instead of replicating those 183kLOC.We can't know without actually looking closely at the case. This is the part which requires expertise, because if the truth happens to beoneof the mentioned extremes, and you choose the wrong one (either way!), the project will be doomed!

author: tggzzz
author_rank: Super Contributor
timestamp: May 20, 2023, 06:28:30 pm
content: Quote from: Nominal Animal on May 20, 2023, 05:28:40 pmYup. Cargo-cult engineering is all too common.A good plan is toconsciouslydecide which knowledge and choices will be valid for decades, and which will "time out" at the end of the current project.

author: coppice
author_rank: Super Contributor
timestamp: May 20, 2023, 07:24:08 pm
content: Quote from: peter-h on May 20, 2023, 06:17:24 pmNot really. A lot of business owners and managers want to develop code they "own" and protect like Gollum in Lord of the Rings protecting his precious. Mostly they have no idea just how long it will take to have a polished reliable solution. Equally, many employees do fully understand that, and want to use something existing and proven, even if it mean paying royalties.

author: peter-h
author_rank: Super Contributor
timestamp: May 20, 2023, 07:37:41 pm
content: A business owner who- doesn't understand how much work is involved, and- won't listen to people who apparently knowis a dickhead.

author: coppice
author_rank: Super Contributor
timestamp: May 20, 2023, 08:30:57 pm
content: Quote from: peter-h on May 20, 2023, 07:37:41 pmSo, who do you think knows? You already said the employees want to develop from scratch, so you don't consider them a reliable source of input.

author: peter-h
author_rank: Super Contributor
timestamp: May 20, 2023, 08:35:21 pm
content: I think the "boss" (or the project manager appointed by him) needs to know enough to make an informed judgement.Otherwise, you are totally trusting the employee[ s ] to recommend an approach which will put less bread on the table at home than another approach.This issue goes all the way to the topmost decision: which CPU to use.

author: coppice
author_rank: Super Contributor
timestamp: May 20, 2023, 08:39:10 pm
content: Quote from: peter-h on May 20, 2023, 08:35:21 pmThat's unrealistic. Nobody has broad enough knowledge to be an expert in every aspect of a complex project. Capable software people on this very forum have been quoting widely variable expectations for the effort needed to implement a viable TCP/IP stack in just this past month, because they haven't tackled that exact project before.

author: peter-h
author_rank: Super Contributor
timestamp: May 20, 2023, 09:04:01 pm
content: Or because they are extremely clever.At any time since I knew what a TCP/IP stack is (basically since about 1990) I would not have attempted a project developing one from scratch.

author: ddavidebor
author_rank: Super Contributor
timestamp: May 20, 2023, 11:29:18 pm
content: I use Zephyr RTOS and it's absolutely wonderful

author: westfw
author_rank: Super Contributor
timestamp: May 21, 2023, 07:27:59 am
content: Quoteor 3: It's bloated because it does a bunch of things that I don't actually need. Even (especially?) if it does them well, it makes the whole thing so difficult to understand that it's not worth fighting to find a more minimal subset.

author: wek
author_rank: Frequent Contributor
timestamp: May 21, 2023, 07:32:50 am
content: Thanks all for comments.Here, I was talking about RTOS and "hardware-layer libraries" (together with clicky configurators - think Arduino, or Cube+CubeMX) considered harmful in microcontrollers, and why.Libraries like protocol stacks, algorithmic/math libraries, they are for a different discussion. The difference is, that they are not replacement for a more appropriate methodology, and they also hit hard up front.> Where do you draw the line as to how big a microcontroller has to be to support an RTOS?I draw the line at the word "controller". How "big" is that, is up to you to decide.As all "rules", this of course has exceptions, too. For example, Arduinos are very useful for applications where relatively few and simple things are simply glued together. These would require low effort anyway, but Arduinos help to avoid the up-front cost of learning the details for those who just want that. Using particular RTOS may be required for administrative/regulatory reasons, or particular protocol stacks may be unavailable without using them.JW

author: peter-h
author_rank: Super Contributor
timestamp: May 21, 2023, 09:03:44 am
content: QuoteMy project started life from Cube MX (although that bit wasn't done by me) and I reckon that works if you bought some ST demo board and want to get that up and running quickly. It doesn't work once you deviate a bit from the demo board setup, and a lot of people get stuck at that point. A quick look at the ST forum makes this clear. MX isn't going to deliver "understanding"Also the code it generates is often buggy.QuoteYes; I think LWIP can work without an RTOS but it does so poorly.

author: SiliconWizard
author_rank: Super Contributor
timestamp: May 21, 2023, 08:40:48 pm
content: Quote from: peter-h on May 21, 2023, 09:03:44 amYeah, well even FatFS isn't that great if you don't use an RTOS (at least a preemptive scheduler) as all its calls are blocking, which can be an annoyance.As to USB, I do recommend TinyUSB these days. It's easy to use, works very well and allows porting to a different (as long as supported) MCU rather easily.I'm all for minimizing the number of third-party libraries one uses in general, but this one is one of the rare that I'll recommend.

author: peter-h
author_rank: Super Contributor
timestamp: May 21, 2023, 09:20:28 pm
content: The FatFS stuff gets a bit more involved. From my project notes:The _FS_REENTRANT parameter in FatFS is set to 0 to disable re-entrancy. This was done because if this is enabled, all kinds of extra code gets brought in which basically needs the RTOS to be running. It does run fine with _FS_REENTRANT=1 (lots of testing was done under RTOS) but this prevents the file functions being used early on in main() for checking the config file or checking for firmware upgrade files.The above would result in the file functions being “not thread-safe” which would mean that they would all need to be all in one RTOS thread. This would greatly complicate any function where file ops are driven from external events, such as during web based config file editing. So each of the published file ops is protected with a mutex.Having a genuinely non-blocking filesystem gets really messy. The storage device won't be re-entrant, so you have to lock it somewhere lower downThere was a long thread here on USB MSC writing to a FLASH with a 15ms sector program time, and what to send back to the Host during that time. The answer, implemented by flash sticks too, seems to be "nothing"; you just make it hang. It doesn't time out. People suggested all kinds of fancy solutions e.g. "writing a driver for the FLASH chip" but that just pushes the problem another layer down.I know someone who implemented TinyUSB and that has some subtle bugs too, apparently. There seems to be no free lunch anywhere and especially not on USB which is really complex.

author: nctnico
author_rank: Super Contributor
timestamp: May 21, 2023, 11:02:56 pm
content: Quote from: SiliconWizard on May 21, 2023, 08:40:48 pmNot really a problem. Just run time critical processes from the hardware scheduler AKA interrupt controller (triggered by a timer or other peripheral) that is available in all microcontrollers.

author: Siwastaja
author_rank: Super Contributor
timestamp: May 22, 2023, 07:10:32 am
content: An OS is tempting when one has fixated into the mindset of writing linear, blocking code. And if you need to use existing code (e.g., libraries), they are often written that way. Some can be easily converted into event-driven mindset (events and callbacks), in which case they work fine in bare metal interrupt based system.When you can freely design everything, nothing holds you back from designing it all ground up event-driven and thus interrupt based.

author: peter-h
author_rank: Super Contributor
timestamp: May 22, 2023, 08:21:29 am
content: That adds complexity though.You can write "blocking" ("I own this machine") code under RTOS and whenever the task is waiting for something you just runosDelay(1);or taskYield();(there is a subtle difference w.r.t. priorities)and you are yielding to the RTOS and allowing everything else to run. It works superbly.

author: Siwastaja
author_rank: Super Contributor
timestamp: May 22, 2023, 08:24:42 am
content: Quote from: peter-h on May 22, 2023, 08:21:29 amI would not say so. Event-driven programming may look difficult to someone who has never done it, but I would not call it "complexity". The idea of having triggers and reacting to them is intuitive and lends itself very well to logging and analysis.Having to write event-driven state machines basically prevents you from copypasting large pieces of linear code from Stack Overflow and instead forces you to do a lean design. Which is opposite of complexity.

author: peter-h
author_rank: Super Contributor
timestamp: May 22, 2023, 09:41:22 am
content: It is harder to debug though, especially if one is waiting on some hardware status bit which is supposed to generate an interrupt but doesn't. On something like an ARM32, the whole interrupt controller portion is hugely complicated and I am sure very few people fully understand it. I know you dobut for sure I do not, so I use interrupts only for stuff which really needs them.I think that in most cases doing what I suggest above (polling with osDelay() yield if busy) ends up taking up pretty much the same % of CPU time as being IRQ-driven. Obviously not in silly-extreme cases like 200 RTOS tasks polling status bits but in typical applications it is similar. "Co-operative multitasking" is actually really good - IFyouhave written all the code. It never worked properly under Windows, but we all know why...QuoteOr Github.But IME much of that stuff doesn't work anyway. I'd say 50% of Github is dud code which either never worked, or worked by luck, on some 16MHz AVR. And the other 50% comes with various open-ended difficulties.

author: Siwastaja
author_rank: Super Contributor
timestamp: May 22, 2023, 10:34:04 am
content: ARM Cortex-M has some of the simplest interrupt system (for the user) ever invented in any microcontroller - configure priority as you wish, enable the interrupt, attach regular C functions to them; job done. As usual, you have created some pretty weird strawmen inside your head which you fight against, and call anyone who does not believe in your demons a "genius" or something like that.Difficult to debug? Set a variable in ISR for logging or something like that.

author: nctnico
author_rank: Super Contributor
timestamp: May 22, 2023, 11:00:52 am
content: Quote from: Siwastaja on May 22, 2023, 08:24:42 amThe problem with event driven is that you decouple the cause and effect which makes code less easy to understand AND make it easier to create deadlocks / unwanted 'paths'. It definitely doesn't make code easier to understand & debug, especially when a project gets bigger. For complex projects linear programming using a pre-emptive multi-tasking OS makes a higher quality solution in terms of predictability and maintainability.

author: BadeBhaiya
author_rank: Regular Contributor
timestamp: May 22, 2023, 11:22:10 am
content: Quote from: Siwastaja on May 20, 2023, 05:59:17 pmThis is something that I have noticed time and time again. I think there is a massive gap between the perception of what developers do and what we actually do, and how hard things we do are seen as.I do see the argument from the perspective of the managers, its obvious. "Hey boss shall I write my own blinky library or use this existing one?". Is that even a question, of course just use the existing one, save company time and money! What managers don't know (and mostly don't care to know) are the technical specifics. Will the external code cause issues? Is it easy to integrate or will it require some refactoring? I do not expect the managers to be up to date with all the technical details (that's what we devs are for!), they just need to be receptive to changes.If I encounter managers with non-technical backgrounds, I spend a lot of time giving them an overview of the technical aspects. Its not for altruistic purposes, but because they can better understand my motivations and not veto obvious design decisions just because of reasons like they take too long.There are times when using an existing library is absolutely worth it and trivial, case in point, FatFS. I've had to include it into projects many times, its as simple as writing 3 simple functions, and registering the callbacks to the fatfs init function. Done.There are other times when "integrating a library" means rewriting half the library.Sadly, its very difficult to which is it going to be before starting the work.

author: tggzzz
author_rank: Super Contributor
timestamp: May 22, 2023, 12:34:10 pm
content: Quote from: nctnico on May 22, 2023, 11:00:52 amThat's arguable.It is easier to specify and reason about complex system in terms of events and states; that's why large-scale commercial systems (e.g. telecoms) do it that way. Deadlocks that areinherent in the problemare then visible (with effort!) in the FSM, and no implementation technology can avoid them.It is a good implementation strategy to attempt to have a direct correspondance between an implementation and the specification. In my experience (with event-driven FSMs in this case) that has been possible without introducing the possibility of extra deadlocks.Of course, if you have inadequate specifications and/or unnecessarily poor implementation techniques, then all bets are off. That's true forallengineering!QuoteI don't see why you have introduced linear programminghttps://en.wikipedia.org/wiki/Linear_programming.Apart from that, I strongly disagree. Logging the sequence of events+states is a simple and powerful technique. It makes it easy to see not only where the problem has become visible, but also how it got to that point. That enables rapidly demonstrating that your system is blameless and the other person/company system isn't behaving properlyBeing able toclearlyshow the spec in conjunction with "you did X1, I did Y1, you did X2, I did Y2 etc is a great way of ensuring lawyers don't get involvedIt is also excellent for performance monitoring and analysis, and can made low cost in terms of time and space.While they can be used, pre-emptive RTOSes aren't necessary. My preference is for one level of interrupts and one level of worker task(s) reacting to the interrupts. Anything more can be a pain to debug. (Ask the Mars Pathfinder's programmers)

author: nctnico
author_rank: Super Contributor
timestamp: May 22, 2023, 12:46:35 pm
content: In my book linear programming is having everything in 1 flow. Do A, wait for B, do H, wait for K. Or wait for message to arrive, process message and go back to waiting. The latter has 1 entry point where messages enter instead of several call-back functions that react to various events and have no obvious relationship. You could ofcourse make this easier to follow by using C++ and packing all call-back and related function into an object so the shared (private) data & states are more obvious.

author: tggzzz
author_rank: Super Contributor
timestamp: May 22, 2023, 01:00:14 pm
content: Quote from: nctnico on May 22, 2023, 12:46:35 pmAh, an non-standard usage of the termI too find naked call backs to be confusing, and therefore class them as a "suboptimal" implementation technique.Event->struct/object passed to an FSM implemented in an inheritance-based OOP is a very natural, succinct and understandable implementation technique. It is easy to:inspect the mapping between specification and implementationdetermine which function/line caused the bugensure that changing that function/line won't introduce other bugs. Or at least other "infelicities" in the specificationC++ is, of course, a suboptimal implementation technology, and we know what happens to things mortals build on sand

author: Siwastaja
author_rank: Super Contributor
timestamp: May 22, 2023, 01:16:34 pm
content: Quote from: tggzzz on May 22, 2023, 12:34:10 pmI agree - because during design, the interactions are normally visualized (hopefully) as events leading to state transitions, it's a weird idea that then re-structuring that into linear code that waits for mutex releases in parallel with dozen other pieces of linear code makes it somehow more maintainable or understandable. I don't agree with that idea at all. It is just a description of how software is mostly writtentoday, with built-in assumption that because it's popular, it must be superior.My bad introducing term "linear programming" to denote a completely different thing to that Wikipedia link. It didn't occur to me it's some pretty obscure term already in existence. But I think we all have some sort of idea what me and nctnico mean by it: large piece of procedural code (with brances and abstractions as functions, of course) where interactions are communicated by wait-for-it type statements.

author: tggzzz
author_rank: Super Contributor
timestamp: May 22, 2023, 01:51:57 pm
content: Quote from: Siwastaja on May 22, 2023, 01:16:34 pmIn some cases a specification can be written in terms of "wait for (X and do A) or (Y and do B) then do...". That can work for relatively simple FSMs, but becomes impractical when there are many events and/or orders of events. It is also more difficult to ensure that the specification is complete.In such cases an implementation can sensibly be based around a cooperative RTOS's yield() function, but the logging and performance analysis is either more convoluted/complex or less comprehensive. i.e. easy mapping between code and spec.

author: nctnico
author_rank: Super Contributor
timestamp: May 22, 2023, 01:57:44 pm
content: You can also argue whether a specification of a piece of software in the form of a state diagram is the most optimal form. Hatley & Pirbhai basically say that a complete specification has at least two models: one which defines the states and one that defines the data flow. These models should match. IOW: you could also implement the data flow instead of the state diagram.

author: peter-h
author_rank: Super Contributor
timestamp: May 22, 2023, 09:43:02 pm
content: QuoteBS.One bit I agree with is the clever arm32 interrupt handling (magic numbers on the stack, etc) allowing a normal C function to work as an ISR.

author: tggzzz
author_rank: Super Contributor
timestamp: May 22, 2023, 10:09:55 pm
content: Quote from: nctnico on May 22, 2023, 01:57:44 pmHaving been on a course taught by Pirbhai in the late 80s, and quizzed him extensively, I don't think that's quite right.The methodology described used any and every technique helpful in the requirements specification, including but not limited to FSMs. Such specifications were almost but not quite executable. Perhaps I might to go back and speed read their bookOne thing I liked was that they had twoindependenthierarchies: one for specification and one for implementation. It was entirely possible for one specification requirement to be implemented in multiple places and technologies in the implementation hierarchy. Thus a specification requirement that the wheels should not be retracted while an aircraft was stationary on the ground could reasonably be implemented using hardware and/or software and/or mechanics.The only requirement was that requirements in the specification hierarchy should have corresponding entries in the implementation hierarchy. When creating computer support for their methodology used in the 777 programme, the only thing they chose to implement was a database for the crossreferences between the entities in the two hierarchies. Very pragmatic!

author: nctnico
author_rank: Super Contributor
timestamp: May 22, 2023, 10:35:54 pm
content: Quote from: tggzzz on May 22, 2023, 10:09:55 pmI think I have their book somewhere as well but it is utterly unreadable due to the obsessive use of many acronyms. It is also quite possible I threw it away; it is not on my bookshelf and can't remember where I would have stored it elsewhere.

author: tggzzz
author_rank: Super Contributor
timestamp: May 23, 2023, 12:04:17 am
content: Quote from: nctnico on May 22, 2023, 10:35:54 pmHm.The course was very understandable. The book came with the course, but I don't remember reading it since the course notes (long gone) were sufficient. Hence I have no opinion on your comment.

author: westfw
author_rank: Super Contributor
timestamp: May 23, 2023, 02:02:38 am
content: QuoteBut ... most of the gratuitous use of RTOS I've seen has been BECAUSE some "core feature" is a protocol stack.So ESP32 has FreeRTOS so that it can have a pretty complete IP/TCP stack, Nordic and Arduino typically throw in mBedOS because ... network connectivity (IP or BT.) And etc.There have been some attempts to add multitasking just so users can "combine two sketches and run them at the same time", but those are less common and less successful (presumably for the reasons you mention.)(meanwhile, The Arduino MBed core for rp2040 takes 4us to run micros(), because of all the resource locking that really doesn't need to be there. Sigh.)

author: wek
author_rank: Frequent Contributor
timestamp: May 23, 2023, 06:15:25 am
content: Quote from: westfw on May 23, 2023, 02:02:38 amYes, Siwastaja wrote that too, far above - some (arguably most) protocol stack implementations are not written with mcu usage in mind. But for a given implementation that's a known thing, so you should consider it as part of the "hits up front".One, not unseen, way, how to tackle this, is to quarantine such stacks to a separate mcu/core.Quote from: westfw on May 23, 2023, 02:02:38 amSo, "how many cores does it take to blink a LED" is no more a joke.JW

author: SiliconWizard
author_rank: Super Contributor
timestamp: May 23, 2023, 06:40:01 am
content: Quote from: nctnico on May 21, 2023, 11:02:56 pmWell, that's just essentially an ad-hoc preemptive scheduler. That was my point. Without that, it's (often) unusable in any use case where an unknown, and possibly longish delay is unacceptable.It's a simpler system than a full-fledged RTOS, but it's still a primitive preemptive scheduler.For similar reasons, most moderately complex network stacks are usually built on top of RTOSs.Makes things much easier for the developers. Not necessarily less complex or safer, but definitely easier.A counter-example is (yeah I mentioned it already) TinyUSB. Can be used with an RTOS of course, but otherwise you just need to call some "idle" function frequently enough, and it only blocks for short amounts of time.And yes, it's (usually) more difficult to develop, as you need to keep internal state and design it so that the blocking time is short and bounded.

author: 5U4GB
author_rank: Super Contributor
timestamp: May 23, 2023, 09:20:30 am
content: Quote from: tggzzz on May 20, 2023, 06:28:30 pmDon't forget the ever-popular "The SoC/ASIC we're using has hardware X therefore our code needs to use the hardware X because there's a hardware X there". Never mind the fact that no-one else has ever managed to get the hardware X doing anything useful, we'll just get an intern/contractor to do it and it won't cost much.Every IP-block is sacred,Every IP-block is great,If an IP-block is wasted,The boss gets quite irate.

author: 5U4GB
author_rank: Super Contributor
timestamp: May 23, 2023, 09:25:12 am
content: Quote from: peter-h on May 20, 2023, 03:14:47 pmBut billion-dollar corporations (seriously!) will use it, because it's free, like a puppy. And then some poor guy eight levels down the food chain spends a year struggling to get useful performance out of it on their particular hardware, ends up creating an unmaintainable private fork for their particular device, and it only cost them around ten times the licensing fee for a supported commercial stack in the wages they paid him over the time it took to get working, combined with the time he wasn't available to develop the actual product he was supposed to be working on.

author: peter-h
author_rank: Super Contributor
timestamp: May 23, 2023, 09:52:56 am
content: QuoteIndeed; arm32 code is quite efficient. It may be "RISC" but with goodies like conditional instructions it is pretty good. I have a loader which reads data out of an SPI FLASH, does AES, etc, and it is under 3k.QuoteI am sure that happens, but the counter argument (assuming it isn't literallyonebloke) is that it pays to develop in-house expertise for a key product line, whereas if you are licensing you may not get the sources, etc. And the continuity of support will be only as good as, ahem, what the company can deliver, which may be bugger-all once all relevant people move on.I've paid "heavy money" (~50k) for commercial network stack stuff in the 1980s and abandoned the project after the vendor decided they were spending too much of their time on it, and started dragging it out to get us off their back. In the end I developed the whole thing myself in a few months. Of course someone could argue it wasn't well enough defined to start with, but that's another art, which is less critical if you do it in-house because you just bang heads together in the office, whereas if you change the spec to an outside vendor they will want $xxxxx extra (and actually this - ripping the customer off because you know his Spec v1.0 will need amending - isa key profit generatorin custom software development and licensing).

author: 5U4GB
author_rank: Super Contributor
timestamp: May 23, 2023, 11:46:02 am
content: Quote from: peter-h on May 23, 2023, 09:52:56 amIn the case I'm thinking of it is literally one guy, an external contractor, and the thing they're asking him to work on has nothing to do with their products. Imagine something like a network stack on a refrigerator (not the actual product but a placeholder) from a refrigerator manufacturer, any time above zero spent on mucking around with the network stack internals is wasted effort because it'll never be touched again once it's up and running, their business is refrigerators, not networking software.

author: tggzzz
author_rank: Super Contributor
timestamp: May 23, 2023, 12:25:23 pm
content: Quote from: wek on May 23, 2023, 06:15:25 amWell, with XMOS xCORE it would be one core, and using the SERDES outputs you can turn the LED on/off as you choose with a 4ns clockFor other time-critical stuff like ethernet USB processing etc, you would have to make do with the remaining 7/15/31 cores, depending on the device used. Quite a hardship.Oh yes. The RTOS is implemented in hardware

author: peter-h
author_rank: Super Contributor
timestamp: May 23, 2023, 04:29:58 pm
content: QuoteI don't think you can do that indefinitely into the future.You can make the same refrigerator "for ever" but external connectivity stuff will need a re-visit. The original dev will be long gone...I spoke to a customer today who bought a load of my product (a user-programmable box) 15-20 years ago. The dev (who wrote the code for him) is so long gone they don't even know who it was.I don't think any of this has a good solution, but if they did it in-house they would have a better chance, even if the approach was sub-optimal. But maybe not; most companies lose everything after 5-10 years

author: westfw
author_rank: Super Contributor
timestamp: May 24, 2023, 12:35:25 am
content: QuoteThis is not dissimilar to how a lot of consumer are created. Company "USAcomp" says to some contractor "Cont2ndW" in some country where development labor is cheap and says "we want a box that does X; it's just this generic SoC reference design running linux, with this special ZChip, but we want the user interface to look like Y." Product quality then depends on the choice of Cont2ndW, how close the relationship is, and how proaactive USAComp is about getting bugs fixed.