Continuous delivery applied to mcu firmware at work, and how mcu tooling is b.s. - Page 1
Forum section: electronics / general

author: magnus0re
author_rank: Contributor
timestamp: August 20, 2022, 11:09:10 am
content: I work on an IoT-like product, with multiple microcontrollers. I'm a new hire, and I have responsibility for the entire code base. The first edition of the code was made by consultants some time ago.Previously I've worked in academia and an IT company, so I've learned two things: From Academia is how to work independently and make an ungodly mess (of code), and from the IT company I learnt the value of good code, and that one should not deliver crap code.As usual for code that you inherit from others; there are no tests, comments are sparse, and the code has never been cleaned up. Luckily some parts of the code has undergone code review, and those parts are nice.So I have a quite large set of code to work with, wherein I have to pick my battles. One battle was to rewrite the firmware over the air receiving code, It is probably still ugly, but now it is readable.The one thing that I've done, that I think has worked out extremely well is to never have the firmware more than 10 minutes away from being deployable. This has made me, and the workflow, extremely responsive to changing requirements. Only three times in the past 4 months has the firmware been in a state where I could not just comment out a small piece of code, and it would compile, and I could put the newest working version of it into our machines. What's been extremely important here has been the firmware version numbering scheme. Every different firmware has to have a different number, or you cannot tell what has gone wrong. I use three numbers <major>.<minor>.<bugfix/test>, where I often put an extra character on the bugfix/test field just to remember what I was testing, like 0.1.0,0.1.a1, 0.1.a2, 0.1.a3, and finally 0.2.0 when that new feature (feature 'a') I worked on has been put into the master git branch.So that has worked really well. It's almost like continuous delivery, but without any rigorous testing, because I haven't had the time available to set up the testing. I have a checklist that I go through on a few units.Eventually I want a build of the firmware that runs on a computer (cross compiled to linux, or maybe qemu), so that I can run tests and some analysis of performance only on the computer.I've collaborated with the back-end software engineers, so now we can automatically build the programming files in the cloud, which is a first step to a controlled ci/cd process, like making sure that every tag compiles, maybe start to enforce warnings, etc..I've had a few bugs that hit the machines, but with firmware over the air updates, I've been able to fix and deploy in minutes. One bug that was horrible was that someone commented out the initialization of the watchdog!Soon we get a new hire, fresh out of school. I want the newbie to feel comfortable with making mistakes, and at the same time I expect that learning happens quite fast.This means that I have to communicate that this is a good way of working, and nudge the new hire into working somewhat like this. I think it will be challenging, but totally possible. Any advice?Does anyone else have experience with this kind of way of making firmware, where they are able to deliver something that works at a moments notice (but of course maybe not functionally complete)?I find it freeing compared to waterfall-like processes. It's also been nice for my colleagues, because usually, they don't have to wait long because the firmware code is a minute away from working.Anyways, I'm pushing CI/CD. You should do it. At the same time, I want to complain about all our mcu manufacturers. The specialized tools they make for every single chip is a horrible way of doing things.Every device family has its own certain tooling, so you have to learn a lot of different tools to do effectively the same things. Its a bad way of life. Nordic has a thing for v.s. code with Zephyr RTOS, Microchip has mplab, which has DIFFERENT tools inside the software for different chips (because it was only pic before, and now they added arm mcus), so it is nested different tools.The linux/open source community figured this out almost thirty years ago.* One way of creating makefiles* One debugger* One standard way of configuring* One standard way of accepting flags to your commands(I know this is a simplified version of reality)We need one workflow that you can script as you like, but it is still the same things below, and you are able to modify stuff. I really miss that from normal software. In the software world, you can even switch languages and much remains the same. While in the MCU world, you are forced to use a specific editor, with specific menus, or weird XML config files. It could be so much better guys! Also, manipulating a compiler via a GUI is horrible, you never know what is hiding behind that button. And don't get me started on the effort required to make a dockerfile for mplab, jeez, I feel so sorry for the guy that had to do that. Besides that, the special tooling per microcontroller REALLY limits your opportunities, because some people will only hire you if you've worked with 'their' tooling, which is the tooling for microcontroller 'abc'. So if things were standardized, the people who work with mcu development would also get a job market with more opportunities. (I personally have heard of a company which its engineers refused to hire anyone who did not use the pic32 toolchain)Do CI/CD, and standardize your tooling. Please?rant over.

author: hans
author_rank: Super Contributor
timestamp: August 20, 2022, 12:17:21 pm
content: I've first hand experience with what happens in Academia. We call it PhD-ware here! It's a bunch of mess only 1 person understands, and if he/she doesn't pursue a further career, then those papers be written damn well in order to transfer the knowledge/progress over..Anyway, I digress. I'm in Academia but still like to CI/CD techniques. For my current research project I don't have a full blown CI setup right now, but I can tool it around soon(TM) if I may have a student working on the project (or I let him/her do a standalone assignment). I do use the basic stuff like unit tests to make sure that algorithm changes that may seem too good to be true are in fact true and do not break other setups (like algorithms, correct RF modulations/demodulations, etc.)Nonetheless, for home projects I do have a CI server setup. No CD yet. I run my local git server (Gogs) and have it setup to push HTTP callbacks to a Jenkins server, which is configured to build half a dozen several projects and targets. I've written a CMake file (because I hate Makefiles) that is reconfigurable for different targets; but it exclusively uses GCC compilers. MPLAB XC16/XC32 is GCC, MSP430/AVR has a decent GCC version, x86 and ARM obviously has, and RISC-V as well. So that's already a very wide range of chips that can be covered with a standard (C)Make file.The Jenkins server runs in a docker container with a custom Dockerfile script. It pulls the latest Jenkins container, but then installs the extra compilers necessary for ARM, RISC-V, etc. I don't rebuild the container often, but it's nice to know that the container is a throw-away and does not need to be manually reconfigured.The automated targets I currently run are based on ARM (Cortex-m0 through to m7), PIC32MX, x86 unit tests, and RISC-V emulation target which is coupled with a DIY RISC-V emulator that does performance estimations. It's not a cycle accurate emulator of any kind of core implementation, but it's supposed to emulate a basic rv32im core with a basic 1 IPC (so barrel shifter and 1 cycle multilier), and additionally 1 cycle branch penalty to 'punish' code that's branching a lot (pretty ordinary MCU). I think I measured it at around 2.5 - 3 Coremarks/MHz. It should give an indication when I make changes to underlying firmware structures (like circular buffers/FIFOs/queues, LinkedLists, etc.) whether the performance improves or plummits.The CI server currently only keeps track of succesful/failing unit tests, code coverage, performance estimations and succesful firmware builds. I keep all built images in Jenkins so I can download and revert state. This has been useful sometimes when I only have built 1 board of a project that's in day to day use, but I've been tinkering with some firmware changes and want to test them (but also take it back when the day is done). Now for hobby projects I don't have a very well established release system, but I imagine that you could make an automation where a git tag will also create a tagged firmware image in Jenkins.. (or perhaps push Jenkins data back to the Git frontend server)What would be really awesome is taking a few steps further. One is to also program firmware onto real hardware and run some automated tests. The problem is you need a dedicated hardware bench for that, and having reprogrammable test equipment is kind of expensive. But it can make sense to do it, for example if you're building battery powered equipment and therefore it's critical not to introduce any power-bug. Then you certainly want to have something like this:https://medium.com/tado-product-development-blog/building-an-iot-product-continuous-battery-lifetime-testing-3245b6cdfa40It's a CI/CD setup of a battery powered product (an IoT thermostat), and the test setup measures energy consumption of the firmware build with a JouleScope, standard powersupply and a small server.I do feel like this is one of the big plus points of using open source tools. IAR/Keil also have integrations for CI IIRC, but I'm not sure how cheap it is concerning licensing and debug probes that are needed to work with that software. Also to me it sounds more like a PITA to setup, as you got to manage it on a basic Windows server machine.. Now granted, the setup time with OSS is also not zero, but once you get familiar with it I reckon it can save quite a lot of time and/or be of a rescue if someone pushes code that breaks half the code base.

author: magnus0re
author_rank: Contributor
timestamp: August 20, 2022, 01:38:48 pm
content: Yes! I am guilty of making some PhD-ware myself. For a project I worked on a few years ago, we had a test setup that was connected to real hardware, resulting in SUPER SLOW test cycles, as the test cases grew the testing became something that could only run overnight. This made everybody super slow in delivering any changes, and pull requests suddenly took 1 day per reviewed comment. I ended up being getting a distaste for hardware as a test bench.But the idea of using a dedicated bench for power profiling over time is super nice. I'll probably do this later.I tend to think of CMake as a superset of Makefiles. I use CMake for my own stuff. Your approach seems really nice to me. How much work do you think it was to get the Cmake file for XC32 and gcc-arm working? Is that something that you have shared/could share? In that case it would save me a few days of work. Additionally I'm no CMake expert.What other measures would you like the CI server to keep track of? For me it could be stuff like lines of code, how often certain functions are edited, maybe RTOS parameters like how much margin the scheduler has, and also link it to a bug tracker.In my mind, I would rather build dedicated testers for each product because I think that off-the shelf programmable testers are basically guaranteed to lack something that you want, for example if one has 12 GPIOs, you will invariable end up needing 13 GPIOs, or another ADC channel, or something. And your next gen product will need measurements that requires that you swap out all of your equipment for something slightly better. Thus dedicated specialist hardware is expensive and must be (possibly) upgraded, introducing even more costs. But the Joulescope seems quite nice, and seems to have a wide range of uses. Maybe I should clone it for my hobby stuff.. It would be fun to work on something like that.

author: tellurium
author_rank: Frequent Contributor
timestamp: August 20, 2022, 05:01:17 pm
content: We used to use Github Actions as a CI engine.Multiple firmwares were built by multiple architectures on each commit.Each toolchain was Docker-ized.No CMake or similar bloatware, just plain GNU Make. Tip: if Makefiles become a mess, keep refactoring until they're simple.That way, each commit was verified across multiple architectures for build regressions.Built firmwares were pushed to a dedicated box, which was a Linux machine with a bunch of USB hubs attached, to whose devboards were connected. Firmwares were flashed to corresponding devboards, UART output collected and upload for further analysis if required, and success/error status reported to GA.That way, each commit was verified on each hardware board.

author: IanB
author_rank: Super Contributor
timestamp: August 20, 2022, 05:22:54 pm
content: Quote from: magnus0re on August 20, 2022, 11:09:10 amI think you have described what the software industry calls Agile.There is much written about it, and it has become the thing that everyone likes to do these days.It has a couple of problems. One is that it is very easy to create "not-Agile" processes and still call the result Agile, thus undermining the benefits that should be obtained. Another problem is that it seems to have been invented in a bubble, without drawing from and learning from the knowledge and practices of those engaged in all the other engineering disciplines.The second drawback is related to the poor state of software engineering as a field, compared to mechanical, civil, electrical, etc. Software engineering really doesn't have the formal stature that is needed to engineer solid, high quality, reliable systems.

author: nctnico
author_rank: Super Contributor
timestamp: August 20, 2022, 05:29:58 pm
content: Just some random thoughts that pop up in my mind:I think the interesting part of this discussion is how to automate software testing to a level where you can test (small) changes quickly. I do wonder how to change the test vectors for the changes that where made for every commit. There must be some considerable amount of effort going into setting up the test environment.So far I have managed to use agile methods (together with a team) to implement a set of the highest priority features and then create a release followed by a cycle of regression / bug fixing. Not quite continuous delivery as in a daily release that fixes issues. But then again, the projects I typically work on don't allow to install potentially buggy software in the field.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 20, 2022, 06:48:50 pm
content: Automating testing and building is a good thing.Now the whole agile/CD thing has a whole set of other issues that could take days to describe.

author: ajb
author_rank: Super Contributor
timestamp: August 21, 2022, 04:09:17 am
content: Quote from: evb149 on August 21, 2022, 01:03:24 amWhat requirements?Sure, exhaustively defining a set of requirements and then exhaustively designing the software implementation before youwritea single line of code, let alonedeployit is a great way to lay the foundation for robust software. Some projects absolutely demand it. Many projects can’t justify it. Most projects can’t afford it whether or not they need it. A certain class of projects can’t realistically discover the project’s requirementsexcept by starting to build it.That’s not to say we shouldn’t do any planning ahead, or shouldn’t take all appropriate steps to produce good and reliable software, but the level of rigor that should be applied to defining a project’s requirements is, itself, a definable project requirement. It’s usually has to be balanced against time-to-market and development costs, which are also important project requirements. This doesn’t just apply to the project as a whole, but also to individual requirements, feature requests, and sometimes even bugfixes.Pragmatic Programming describes the idea of “Tracer Bullet Development“, which is the method of identifying requirements by taking a shot, seeing how close to the target you hit, and correcting your aim from there. It’s not really an efficient way to write software, and missing is part of the process, but it can be a very efficient way of discovering requirements. It’s sometimes the only realistic way, as sometimes a stakeholder can’t fully describe what they want until they see something they can say yes or no to, and in some ofthosecases shipping half-finished code—with appropriate disclosures!—so you can ask “is this what you had in mind?” is a perfectly reasonable way to avoid wasted development effort. Once the requirements are sufficiently defined you can write and ship the fully finished version, which may or may not involve throwing away the half-finished version and starting fresh.QuoteNone of this refutes what the OP is saying. IME, rather, it’s hard to achieve the level of responsiveness and release reliability they’re describingwithoutdoing a lot of this. The whole point of encapsulation, after all, is to minimize the impact that changes to one part of a project have on other parts of the project. The whole point of modularization is to allow functional chunks of the project to be more easily rearranged and exchanged. Both make it a lot easier to respond to changing requirements, so if the OP has been as successful as they’re describing I would tend to assume that they’ve already been doing a lot of these things.

author: westfw
author_rank: Super Contributor
timestamp: August 21, 2022, 05:57:43 am
content: It's not clear whether OP is using some sort of version control software...They should be! None of this "comment out the recent changes" stuff; there ought to be the unchanged version sitting there, without code being committed to it until after it has been reviewed, tested, and proven. And another, older, version for all the customers not adventuresome enough to be "beta testers.""Develop and Deploy" changes dramatically when you have more than one person working on a project, and again when you end up with more than one person working on the same file.

author: magnus0re
author_rank: Contributor
timestamp: August 21, 2022, 08:13:36 am
content: In no way do I believe what you are saying about requirements.First of, requirements are not a tablet written by an omnipotent god. Requirements always have some error, or the potential for error, same with standards.Secondly, some requirements can be firm and measurable, but others always have some subjective measure to them. Where I work I don't think it would be appropriate to ask for the kind of requirements that you describe.Third, any requirements you do have might change in the face of reality.So in other words, many requirements will become stale or irrelevant with time, no matter how hard you plan ahead.The agile bridge is a bit funny. But what you are describing is not anyone's process. I don't like to use the word 'agile' because it brings out reactions like that. Also, never trust a guy that is selling you 'Agile'.If you've ever had to add a delay somewhere for something to work, like a peripheral, would you not add that delay even if it was not in the specification?With infinite time and money I might investigate, but my schedule is already so tight that it would not be responsible to do this.For me, the SOLID principles point towards a way of coding where you can deal with reality. Reality is mushy and has soft edges, software needs to be changed to respond to this, and if that takes it outside of a specification, it is the spec that is wrong, not reality.

author: magnus0re
author_rank: Contributor
timestamp: August 21, 2022, 08:28:22 am
content: Quote from: westfw on August 21, 2022, 05:57:43 amI don't allow anyone who does not know git near real code. That has become a people specification, so I take git for given nowadays. The not commenting out code rule taken to its extreme is a bit silly, reminds me of the quote about the hobgoblin. But in principle any master branch commit should not contain dead/commented code. I usually test and then commit, not commit and then test. This means less broken commits. I have a thing with broken commits, I don't want to see them. But this also means more comments and tweaking.If you use git well, "Develop and Deploy" can be almost as easy with multiple people and a good team lead (I've experienced this once). It can also be a rat race. I've seen several points on this spectrum. I want to be on the easy side.

author: hli
author_rank: Frequent Contributor
timestamp: August 21, 2022, 10:23:11 pm
content: Quote from: IanB on August 20, 2022, 05:22:54 pmI recommend readinghttps://www.hillelwayne.com/post/are-we-really-engineers/. We should stop pitying ourself (or, more likely, using this as an excuse for doing shitty work). Yes, we surely are not where software engineering should be, but when compared with other fields its not as bad as we always claim it to be (esp. when considering that software projects are the most complex project mankind does, by several orders of magnitude)..

author: snarkysparky
author_rank: Frequent Contributor
timestamp: August 21, 2022, 11:01:10 pm
content: Software can't really be engineering because the product can never ( except in trivial cases ) be proven correct.Software is always an informed art.

author: hli
author_rank: Frequent Contributor
timestamp: August 22, 2022, 08:23:23 pm
content: Quote from: snarkysparky on August 21, 2022, 11:01:10 pmThats wrong in two ways.First, you can prove that software is correct. AFAIK ESA actually proves that their Ariane software conforms to the specification. (That the specification is the correct one to solve the problem is another matter). The issue here is that this makes the process quite more expensive, so usually this step is skipped and replaces by other mechanisms (test coverage, TDD, whatever else we have). It also means that you need to limit yourself to a certain subset of programming language capabilities (e.g. MISRA C) and avoid the dangerous / non-provable ones (or use proper languages right away).Second, it assumes that other engineering disciplines can actually prove that their solutions are correct. If that would be the case, we should never ever have collapsing dams, or buildings, or whatever else you can read in the news. I guess searching youtube for 'engineering failure' will yield quite a lot of cases of 'but we thought', and the the reality did not conform to expectations. (I have a relative working on simulating waterflow in rivers and dams and so on - by building scale models and the testing the water flows. Thats not 'proving to be correct' - thats testing and hoping the model actually is somewhat similar to real life.)Or more pointedly: for the space shuttle, it was never the software that failed, but the real engineering. (Maybe they would have been able to prove something to be correct, but also choose not to do?)

author: IanB
author_rank: Super Contributor
timestamp: August 22, 2022, 09:24:21 pm
content: Quote from: hli on August 22, 2022, 08:23:23 pmOr worse still, to be told something is incorrect, but choose to ignore the advice.

author: hans
author_rank: Super Contributor
timestamp: August 23, 2022, 01:47:34 pm
content: Luckily alot of people don't go to college for a degree in software engineering, but most MSc and PhDs are awarded for Computer Science. That's can be just as math intensive (if not more abstract) as an average EE course. It's a completely different story that many students just want to program some Javascript site together with some turnkey AI...But honestly, the courses I took in my embedded systems college time from theoretical CS had casual induction proof questions toshowhow amodelof a real-time system is correct or not. The courses I took, from EE or CS, in general showed that the more details a model or verification grammar can contain, the less assertive it becomes in actually checking programs. You can quite easily show if programs are always real-time if you strip out all control flow instructions from CPUs. But for practical reasons and implementation necessities, we need those control flow instructions, so we can't really work with that limitation.So both EE and CS works with margins instead. If you can't model WCET on cached-CPUs, then you measure them and multiply by them with a factor you're comfortable with. You can't always buy a big enough diode for the peak in-rush current of a circuit. So either design an in-rush limiter or pick a diode with an appropriately sized SOA to meet your requirements. The probability of failure can't be zero, but with careful design it can be pretty low.Unfortunately a lot of the statements about Agile were removed, because I thought it raised an interesting question. Surely the components we design with (both firmware and electronics) are reasonably deterministic, but humans are far from it. How are you going to be certain a programmer put back all his commented 'disabled for debugging' lines before committing and releasing a FW update? How do you want to test some piece of code if you can't write down how to test it, in code? It's not like a human debugger is very efficient.. or never forgets things.. or won't skip half of the todo check-list when daily updates of FW are released and 99% of the features stay the same (so they only test the 1% difference, and may miss anything that regresses in the 99%), etc.This is why I think design paradigms like Agile, TDD etc. became a trend, because it's a way to manage expectations for the software world. Sure there are shortcomings with it (like Agile burnout), but so is with the classic waterfall model. In terms of managing human workflow/resources, I don't think there is a solution written in stone as it's a design and development philosophy. Just like EE's may have different philosophies about CAD tools, library management, etc. One engineer may draw a separate library component for each resistor size, value, tolerance, etc. while somebody else may just place down a "Resistor" and change some parameters per component and then fill in BOM's manually. Which one is better if both get the job done, in time (for now)?Anyhow I digress. I had been asked to share some snippets of my CMake file. Honestly, it's a bit of a mess as I want to refactor it sometime. But in short, here are the most important bits to switch between several crosscompilers and CPU flags:Code:# Determine CPU_TYPE above here depending on project, etc.# CPU typesif (CPU_TYPE MATCHES "native")add_definitions(-DSIMULATOR)endif()if (CPU_TYPE MATCHES "m0")# Cortex-m0 CPUs (STM32L011)SET(CPU_FLAGS "-mcpu=cortex-m0")endif()if (CPU_TYPE MATCHES "m4")# Cortex-m4 CPUs (STM32F427)SET(CPU_FLAGS "-mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv4-sp-d16 -mthumb -mthumb-interwork")add_definitions(-DARM_MATH_CM4 -DARM_MATH_MATRIX_CHECK -DARM_MATH_ROUNDING)endif()if (CPU_TYPE MATCHES "m7")SET(CPU_COMPILER "arm")# Cortex-m7 CPUs (STM32F722)SET(CPU_FLAGS "-mcpu=cortex-m7 -mfloat-abi=hard -mfpu=fpv5-sp-d16 -mthumb -mthumb-interwork")add_definitions(-DARM_MATH_CM7 -DARM_MATH_MATRIX_CHECK -DARM_MATH_ROUNDING)endif()if (CPU_TYPE MATCHES "m33")SET(CPU_COMPILER "arm")# Cortex-m33 CPUs (STM32U575 project)# https://github.com/RT-Thread/rt-thread/blob/master/bsp/stm32/stm32u575-st-nucleo/rtconfig.pySET(CPU_FLAGS "-mcpu=cortex-m33 -mthumb -mfpu=fpv5-sp-d16 -mfloat-abi=hard -mthumb -mthumb-interwork")add_definitions(-DARM_MATH_CM33 -DARM_MATH_MATRIX_CHECK -DARM_MATH_ROUNDING)endif()if (CPU_TYPE MATCHES "riscv")SET(CPU_FLAGS "-march=rv32im -mabi=ilp32")endif()# Architcture compilerif (CPU_ARCHFAM MATCHES "ARM")if (DEFINED CPU_ID)add_definitions(-D${CPU_ID})endif()# specify cross compilers and toolsSET(CMAKE_C_COMPILER_WORKS 1)SET(CMAKE_C_COMPILER arm-none-eabi-g++)SET(CMAKE_CXX_COMPILER_WORKS 1)SET(CMAKE_CXX_COMPILER arm-none-eabi-g++)set(CMAKE_ASM_COMPILER arm-none-eabi-gcc)set(CMAKE_AR arm-none-eabi-ar)set(CMAKE_OBJCOPY arm-none-eabi-objcopy)set(CMAKE_OBJDUMP arm-none-eabi-objdump)set(SIZE arm-none-eabi-size)include_directories(${CPUPATH}/arm/inc)set(CMAKE_C_FLAGS "-x c -ffunction-sections -fdata-sections")set(CMAKE_CXX_FLAGS "-fno-rtti -fno-exceptions -ffunction-sections -fdata-sections")set(COMMON_FLAGS "${COMMON_FLAGS} -specs=nano.specs -ggdb")SET(CMAKE_EXE_LINKER_FLAGS "-Wl,-gc-sections")set(CMAKE_CXX_STANDARD 20)elseif (CPU_ARCHFAM MATCHES "RISCV")SET(COMMON_FLAGS "${COMMON_FLAGS} -flto")if (DEFINED CPU_ID)add_definitions(-DCPU_${CPU_ID})endif()SET(RISCV_PATH "/opt/riscv/bin/riscv32-unknown-elf-")# specify cross compilers and toolsSET(CMAKE_C_COMPILER_WORKS 1)SET(CMAKE_C_COMPILER ${RISCV_PATH}g++)SET(CMAKE_CXX_COMPILER_WORKS 1)SET(CMAKE_CXX_COMPILER ${RISCV_PATH}g++)set(CMAKE_ASM_COMPILER ${RISCV_PATH}gcc)set(CMAKE_AR ${RISCV_PATH}ar)set(CMAKE_OBJCOPY ${RISCV_PATH}objcopy)set(CMAKE_OBJDUMP ${RISCV_PATH}objdump)set(SIZE ${RISCV_PATH}size)include_directories(${CPUPATH}/riscv/inc)set(CMAKE_C_FLAGS "-x c -ffunction-sections -fdata-sections")set(CMAKE_CXX_FLAGS "-fno-rtti -fno-exceptions -ffunction-sections -fdata-sections")set(COMMON_FLAGS "${COMMON_FLAGS} -specs=nano.specs -ggdb")SET(CMAKE_EXE_LINKER_FLAGS "-Wl,-gc-sections,--print-memory-usage")set(CMAKE_CXX_STANDARD 20)elseif (CPU_ARCH MATCHES "PIC32")if (EXISTS "/opt/microchip/xc32/v3.01/bin/")SET(XC32_PATH "/opt/microchip/xc32/v3.01/bin/")elseif (EXISTS "/opt/microchip/xc32/v2.50/bin/")SET(XC32_PATH "/opt/microchip/xc32/v2.50/bin/")else()message(FATAL_ERROR "Could not find XC32")endif()SET(XC32_EXEC "${XC32_PATH}xc32")SET(XC32_BIN2HEX "${XC32_EXEC}-bin2hex")message(NOTICE "DFP=${XC32_PATH}")# specify cross compilers and toolsSET(CMAKE_C_COMPILER_WORKS 1)SET(CMAKE_C_COMPILER ${XC32_EXEC}-gcc)SET(CMAKE_CXX_COMPILER_WORKS 1)SET(CMAKE_CXX_COMPILER ${XC32_EXEC}-g++)SET(CMAKE_ASM_COMPILER ${XC32_EXEC}-gcc)SET(CMAKE_AR ${XC32_EXEC}-ar)SET(CMAKE_OBJCOPY ${XC32_EXEC}-objcopy)SET(CMAKE_OBJDUMP ${XC32_EXEC}-objdump)SET(SIZE ${XC32_EXEC}-size)SET(COMMON_FLAGS "-ggdb")SET(CMAKE_C_FLAGS "-g -std=c99 -mafrlcsj -ffunction-sections -fdata-sections")SET(CMAKE_CXX_FLAGS "-g -std=c++11 -mafrlcsj -mno-jals -fno-rtti -fno-exceptions -fno-check-new -fenforce-eh-specs -ffunction-sections -fdata-sections")SET(CMAKE_EXE_LINKER_FLAGS "-D__${CPU_ID}__ -mafrlcsj -Wl,-gc-sections,--defsym=__MPLAB_BUILD=1,--defsym=_min_heap_size=2048,--no-code-in-dinit,--no-dinit-in-serial-mem -DXPRJ_default=default ")SET(CMAKE_CXX_STANDARD 11)add_definitions(-D__${CPU_ID}__)elseif (CPU_ARCHFAM MATCHES "X86")SET(COMMON_FLAGS "${COMMON_FLAGS} -flto -ggdb")set(CMAKE_CXX_STANDARD 20)set(CMAKE_C_FLAGS "-Wno-address -Wall -Werror -fPIC -fprofile-arcs -ftest-coverage")set(CMAKE_CXX_FLAGS "-Wno-address -Wall -Werror -fPIC -fprofile-arcs -ftest-coverage")else()message(FATAL_ERROR, "Invalid compiler chosen")endif()SET(CMAKE_ASM_FLAGS "${COMMON_FLAGS} ${CMAKE_ASM_FLAGS} ${CPU_FLAGS}")SET(CMAKE_C_FLAGS "${COMMON_FLAGS} ${CMAKE_C_FLAGS} ${CPU_FLAGS}")SET(CMAKE_CXX_FLAGS "${COMMON_FLAGS} ${CMAKE_CXX_FLAGS} ${CPU_FLAGS}")After these flags have been set up, the CMakeFile searches for project sources (in several folders, of which some depend on the chosen architecture/chip family), and add several targets to be built. The linker script and map file are added per target with:Code:file(GLOB TARGETS "projects/*.*" "projects/${BOARD}/*.*" "projects/Common/*.*")foreach(TARGET_FILE ${TARGETS})# [...]target_link_options(${TARGET_OUT}.elf PRIVATE "SHELL:-T ${LINKER_FILE}" ${LINKER_FILE_SUPPLEMENTS})target_link_options(${TARGET_OUT}.elf PRIVATE "-Wl,-Map=${PROJECT_BINARY_DIR}/${TARGET_OUT}.map")# [...]endforeach(TARGET_FILE)Supplementary outputs to .elf are also built:Code:add_custom_command(TARGET ${TARGET_OUT}.elf POST_BUILDCOMMAND ${CMAKE_NM} -l -S --size-sort -C $<TARGET_FILE:${TARGET_OUT}.elf> > ${SZ_FILE}COMMAND ${CMAKE_NM} -l -S -n -C $<TARGET_FILE:${TARGET_OUT}.elf> > ${ADDR_FILE}COMMAND ${CMAKE_OBJDUMP} -d $<TARGET_FILE:${TARGET_OUT}.elf> > ${LST_FILE}COMMAND ${CMAKE_OBJDUMP} -S -d $<TARGET_FILE:${TARGET_OUT}.elf> > ${LSTSRC_FILE}COMMAND ${SIZE} ${TARGET_OUT}.elfCOMMENT "Building supplementary files")if (NOT DEFINED GENERATE_NO_BIN)set(BIN_FILE ${PROJECT_BINARY_DIR}/${TARGET_OUT}.bin)add_custom_command(TARGET ${TARGET_OUT}.elf POST_BUILDCOMMAND ${CMAKE_OBJCOPY} -Obinary $<TARGET_FILE:${TARGET_OUT}.elf> ${BIN_FILE}COMMENT "Building .bin file")endif()if (CPU_ARCH MATCHES "PIC32")# use PROPRIETARY hex conversion tool.....add_custom_command(TARGET ${TARGET_OUT}.elf POST_BUILDCOMMAND ${XC32_BIN2HEX} ${TARGET_OUT}.elf)add_custom_command(TARGET ${TARGET_OUT}.elf POST_BUILDCOMMAND python3 ${CPUPATH}/PIC32MX/memoryusage.py "${TARGET_OUT}.xml")else()add_custom_command(TARGET ${TARGET_OUT}.elf POST_BUILDCOMMAND ${CMAKE_OBJCOPY} -Oihex $<TARGET_FILE:${TARGET_OUT}.elf> ${HEX_FILE})endif()In which the bin2hex tool of XC32 is used for PIC32s (combined with generate_no_bin), as the fragmented flash architecture would otherwise generate huge .bin files..

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 23, 2022, 06:30:47 pm
content: Quote from: hans on August 23, 2022, 01:47:34 pmAnd, this is spot on. Those methods are not engineering methods whatsoever - they aremanagementmethods, and most of them are actually the opposite of what engineering usually teaches you. Like, developing without clear requirements upfront. Or designing by testing. Yeah, while TDD may look like a good idea on the surface, testing should not DRIVE design. This is a completely flawed approach if you care to dig a little deeper.