Has anyone used ASAN & MSAN, and relevance to embedded arm32 - Page 1
Forum section: electronics / general

author: peter-h
author_rank: Super Contributor
timestamp: April 27, 2024, 01:57:22 pm
content: This came up on the MbedTLS mailing list, where somebody had some weird bug.A suggestion was running the code with Asan(https://github.com/google/sanitizers/wiki/AddressSanitizer) and Msan(https://github.com/google/sanitizers/wiki/MemorySanitizer), bothnatively available on Ubuntu.But doesn't this require building the code for a "PC"?In the case of MbedTLS one can indeed build a win32 version (and I do have that, done by the guy who did my MbedTLS implementation) although this was quite a lot of work, especially to use the same memory allocation strategy as the embedded target uses, which is quite important since you are processing external data (certificates of various sizes and with different hashes etc).In my case I do not use the heap other than a one-off malloc for an optional feature which is never freed so fragmentation is impossible.

author: ataradov
author_rank: Super Contributor
timestamp: April 27, 2024, 04:06:07 pm
content: I don't think those would be very useful on embedded code. Asan adds instrumentation in the code and then replaces low level malloc() and free() calls with custom tracing versions.This typically increases code size by 2-3 times, decreases execution speed by the same 2-3 times. And it also needs a lot more RAM to store all those allocation records. On embedded with custom allocators, you will essentially get one or no real malloc() calls, so the tool will be useless.In theory, it might be possible to work with the instrumented code and match it with the custom allocators, but I'm not aware of anyone doing it.

author: peter-h
author_rank: Super Contributor
timestamp: April 27, 2024, 06:18:40 pm
content: Thanks... so I thought.FWIW the general heap I use is the one in libc.a i.e. newlib. The block starts as usual after end of BSS and goes all the way to the general stack. The code seems OK; I had to mutex it to make it thread-safe, and in the RM for the box I tell coders to not use the heapMbedTLS runs its own private heap within a static block you give it. Well that's one mode; I give it 60k and that is obtained with the newlib heap, and is de-allocated when the TLS session ends but because 60k is such a huge block - most of my heap - fragmentation is not possible. And I think it is that heap which the MbedTLS guys were talking about. MbedTLS is incredibly complicated and almost nobody understands it, so chucking away the whole block is probably a good idea.FreeRTOS also runs its own private heap for allocating memory to tasks as they are defined during startup, but I never de-allocate those either.

author: NorthGuy
author_rank: Super Contributor
timestamp: April 28, 2024, 01:08:49 am
content: If you want to test memory allocation/use on Linux, use Valgrind.

author: ataradov
author_rank: Super Contributor
timestamp: April 28, 2024, 01:30:14 am
content: Asan is much faster and easier to use than Valgrind. But it won't show much if the stack uses one big allocation and then uses allocated chunk for the custom allocator. It will catch gross out of buffer access, of course.

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 28, 2024, 01:56:56 am
content: Note that FreeRTOS has stack checking features, if you use it:https://www.freertos.org/Stacks-and-stack-overflow-checking.html

author: peter-h
author_rank: Super Contributor
timestamp: April 28, 2024, 11:33:42 am
content: Yes; those help with a) using the FreeRTOS internal heap (which I don't use) and b) overrun of the task stack due to e.g. some function which has big local variables.The problem with using those checks is that they only tell you if the thing has blown up. One needs a separate tool to check if there is enough margin. I got a graphical tool developed on freelancer.com which shows the 64k RTOS stack area and it shows the unused stack space etc. Anyway that's a bit off topic

author: __george__
author_rank: Contributor
timestamp: April 28, 2024, 08:56:14 pm
content: There is also puncover:https://github.com/HBehrens/puncoverwhich can help you to analyze the stack usage statically (also integrated with Zephyr/west). Apart from that I never tried ASAN/MSAN with embedded, from what I have heard around it is too much overhead for most use cases.

author: tellurium
author_rank: Frequent Contributor
timestamp: April 28, 2024, 11:28:36 pm
content: GCC can do static stack usage analysis. Just add "-fstack-usage" to the build flags, then GCC spits BINARY.su file with data.

author: djacobow
author_rank: Super Contributor
timestamp: April 29, 2024, 12:18:50 am
content: At my company we have taken quite a bit of effort to be build out a hal that allows us to native compile our firmware projects on the host and run them against fake peripherals also on the host: things like UART and SPI show up as network sockets, timers are simulated with SIGTIMER, etc.Anyway, we use clang for this and do use -fsanitize=address and we've caught a fair number of bugs this way.You can achieve a lot of what these tools do by making your own instrumented memory guards. Create functions for allocating memory that allocate a larger chunk and return a pointer that skips the first 1/2 of the extra space, then have a registration function that colors the space at the beginning and and, and a checker that you run periodically on all the allocations to check for clobberage. This is very straightforward to do if you use malloc (just call this instead) but even if you use 100% automatic variables, you can achieve the same thing with some clever macros.Here is a VERY quick-n-dirty example of what I am suggesting:https://github.com/djacobow/memguard. Basically, if you have a variable you suspect is getting clobbered you can create it using these macros and then call a checker occasionally. Not as good as a real sanitizer, but helpful in a pinch on an embedded system.