Reproduceable build with an IDE - Page 1
Forum section: electronics / general

author: trampas
author_rank: Regular Contributor
timestamp: December 19, 2024, 07:21:33 pm
content: I have been looking for a way to create reproduceable builds, including IDE and JLink debugging.The problem is that I do a lot of development with various microcontrollers. The test and verification time is usually long and tedious. As such we lock in the compiler version, build tools, etc. Then we verify that every developer can recreate the same binary exact hex files from the source code. As you can imagine after doing 10 projects with 10 different compilers and tools the problem can become tedious to keep everything organized. I have found xpack,https://xpack.github.io/, which does a good job for specifying the compiler, tools, etc for each project. However the next problem is the debugger and IDE.For setting up debugger and IDE, it has become a pain, we have to write multipage instructions on how to install IDEs, JLink software, and configure the IDEs to use the correct tools, etc. This has proven to be difficult for many, and causes a support nightmare.I would like to move to something like VScode but the nightmare is just as bad, as you have to tell people which extension to install, as well as other tools. Even telling people to install node.js then install xpack has been a pain. I have debated on putting binaries for all the tools in git with source code and then including a batch file to install everything, but this seems wrong.I figure I can not be the only one with this problem, so how do others solve this?ThanksTrampas

author: Dazed_N_Confused
author_rank: Contributor
timestamp: December 19, 2024, 07:31:48 pm
content: Docker containers an option

author: cfbsoftware
author_rank: Regular Contributor
timestamp: December 19, 2024, 10:13:56 pm
content: If you are using Windows I strongly recommend FinalBuilder:https://www.finalbuilder.com/finalbuilder

author: amyk
author_rank: Super Contributor
timestamp: December 20, 2024, 12:15:33 am
content: Quote from: trampas on December 19, 2024, 07:21:33 pmThe simplest solution is often the best. Self-contained and easy to understand. "If it looks stupid but it works, it's not stupid."

author: betocool
author_rank: Regular Contributor
timestamp: December 20, 2024, 02:10:41 pm
content: Second docker containers.Setting them up may be a pain and there is a lot of software lingo involved, and there's differences between Linux and Windows, and I think you need to use actual hardware to program the chips.Once configured however, they should give consistent results. I think there are tutorials for STM32s and NRF, but I've been too lazy to look them up myself!Cheers,Alberto

author: peter-h
author_rank: Super Contributor
timestamp: December 20, 2024, 02:57:55 pm
content: Build it inside a VM.I use VMWARE. All freeThere is a performance loss but it is OK on a fast PC with loads of RAM. I have my whole Cube IDE project set up in a win7-64 VM.

author: jan28
author_rank: Contributor
timestamp: December 21, 2024, 01:01:41 pm
content: I use platformio (platformio.org) for reproducible builds of firmware and tools to upload/debug. The only external dependency is python (AFIAK). All other stuff gets pulled in/build according to the specs in your platforms.ini file. It also manages your build environments (without messing up your other projects!).It has a learning curve to understand that you have to ‘let it mange build environments and libraries for you’ and how to pin these dependencies (semantic versioning, semver.org) and setup different builds within a single project.The VS Code extension is not required. You can also use the CLI. The VS Code integration doesn’t influence your build setup, it’s just a UI frontend.

author: Siwastaja
author_rank: Super Contributor
timestamp: December 22, 2024, 11:08:15 am
content: I understand you want to standardize the compiler versions to guard against compiler bugs, (un)intentional implementation defined or undefined behavior in your code, or (un)intentional timing-critical code, but why standardize IDEs and debugging environments? They should not affect the output. Why not let developers use whatever they want, maybe giving recommendations if you have young players in your team?

author: Warhawk
author_rank: Frequent Contributor
timestamp: December 23, 2024, 07:31:27 pm
content: Ten years back, when I worked for one of the leaders in industrial automation, ours FW guys always built the RC and release packages in virtual machines. Sometimes, it was needed to fix some bugs in 10 years old products and it was quite interesting to see people working in w2k or similar.

author: harerod
author_rank: Frequent Contributor
timestamp: December 23, 2024, 08:11:49 pm
content: And only fifteen, twenty years ago we would embalm whole PC's plus make backups of the drives. This would happen on two occasions:- developer left the company- important enough project with ongoing supportEven then a PC's cost was peanuts in comparison to lost time.These days a virtual machine is a good bet. At the same time it is important to have rigid control over updates to the development system. Again - a full drive image before updating is a cheap way to ensure full recovery, should some update foul up the works. This is rarely necessary, thank goodness.Afterthought: at that time, PC's' performance greatly improved in a relatively short time. Embalming PC's was a good excuse to get a new main machine.

author: Siwastaja
author_rank: Super Contributor
timestamp: December 24, 2024, 08:19:56 am
content: Archived hardware or virtual machines should besafety nets. Too often, the whole development 100%relieson them, and that's just insane and risky.At very least you should be frequently (say, twice a year?) test what happens if you get a fresh machine, with different compiler versions, pull fresh copy from revision control and just compile it, preferably by a person who has not worked with the project closely. Does it work? This:* Reveals missing files from version control,* Reveals poor documentation (how to setup & compile),* Possibly reveals buggy code (like depending on IDB/UB in C, or unintended timing criticality (race conditions etc))You can keep shipping the product compiled on the locked-down HW or virtual machine, but at least you get input what to maintain, what to fix. Optimally you maintain the project so that fresh copies Just Work too, even if, for traceability reasons you still keep using the locked machine for official releases.This way your risk goes away, as anyone sufficiently competent can take over maintaining it.

author: peter-h
author_rank: Super Contributor
timestamp: December 24, 2024, 01:04:21 pm
content: One would build the project inside the VM and check the binary matches byte for byte.I have some products which are programmable (in C) by unspecified present and future others and one can- have 10 pages in the RM telling them how to install and set up Cube IDE, the project sources, etc- give them a VM with the whole thing ready to goIn reality one has to do the 1st one, because the 2nd one contains a copy of win7-64 which is "technically not right" and when you run the VM for the first time, you have to answer "moved it" instead of "copied it" otherwise you get forced to buy a new win7-64 product keyBut the 2nd option will rapidly get that user out of trouble... Also the 1st option is no good after a few months because that version of Cube and GCC tools will no longer be available online (so I have to achieve the .msi install package and host it somewhere, where nobody can find it unless I give them the full URL).I suppose a VM running linux instead of windoze would avoid the above issue...VMs work great, on a fast machine. I have a 15 year old win2000 (visbas) program, high value, running on a win10 PC in a VM. The only issue I found is that a win10 VMWARE VM runs very poorly on a win7-64 host. Luckily win7-64 runs great.The full VM, win7-64 with Simplix updates to latest version, Cube IDE, the whole project (about 200MB), zipped up for corruption protection, is about 10GB.Keeping an old PC is OK but one tends to find the hard disk doesn't start rotating after say 10 years

author: Silenos
author_rank: Regular Contributor
timestamp: December 24, 2024, 03:23:30 pm
content: I had once some company installation system with a hipster name I don't remember which was installing all the system level dependencies, like IDE, tools, msvc components. Pretty sure it was ment for corporate admins so they fling the worker's computer with a installation profile. If I pull the name of it, I'll edit the post.Apart from that I'm recently fiddling with cmake and I'm pretty sure it can be made to perform all necessery checks for system dependencies, pull them, build, install and what not with minimal to moderate external scripting.

author: Siwastaja
author_rank: Super Contributor
timestamp: December 24, 2024, 04:20:19 pm
content: Quote from: Silenos on December 24, 2024, 03:23:30 pmWhile cmake and similar are useful to some people, in some circumstances, they are by definition the polar opposite of reproduceable build: they are "buildsomethingeverywhere, with whatever you happen to have available on the computer".