Alternatives to Newlib-Nano? - Page 1
Forum section: electronics / general

author: Sal Ammoniac
author_rank: Super Contributor
timestamp: September 29, 2016, 08:28:26 pm
content: I'm doing embedded C programming using Eclipse/GCC for the ARM Cortex-M4 and have been using newlib-nano for a C runtime library up until now. While debugging some code last night I happened to notice that several nano functions internally call malloc(). I don't think dynamic memory allocation in an embedded environment is a good idea in general, especially when it happens behind the scenes. On rare occasions I'll use malloc() myself, but I make that choice consciously.Is there another C runtime library compatible with ARM GCC thatdoesn'tuse hidden dynamic memory allocations?

author: langwadt
author_rank: Super Contributor
timestamp: September 29, 2016, 09:55:40 pm
content: Quote from: Sal Ammoniac on September 29, 2016, 08:28:26 pmgoogle seems to indicate that newlib-nano uses malloc in printf and you can get around it by not using it or implementing your own

author: Sal Ammoniac
author_rank: Super Contributor
timestamp: September 29, 2016, 10:11:04 pm
content: It's more than just printf. I haven't discovered the full extent of nano's use of malloc yet, but intend to.I'm just surprised that a library that bills itself as a "C library (libc) targeting embedded microcontrollers" uses dynamic allocation at all.

author: nctnico
author_rank: Super Contributor
timestamp: September 29, 2016, 10:26:53 pm
content: I have been using the extremely small C library which comes with MSP430 GCC for ARM controllers for over a decade:https://sourceforge.net/p/mspgcc/msp430-libc/ci/master/tree/It is pretty simple to hack the makefile and some source files so it compiles for ARM.

author: rstofer
author_rank: Super Contributor
timestamp: September 29, 2016, 10:53:10 pm
content: I think string functions also use the heap.So, I don't use those parts of the library. Instead, I build up the functions from code given in "The C Programming Language". I wind up with enough string and conversion routines to meet my needs.The portion of the standard C library that I wind up needing doesn't seem to use the heap.

author: Sal Ammoniac
author_rank: Super Contributor
timestamp: September 29, 2016, 11:18:37 pm
content: The only thing that I really need that I haven't already written myself is sprintf. I use this to format log entries. I'll have to search for an embedded-friendly version or write one myself.

author: nctnico
author_rank: Super Contributor
timestamp: September 29, 2016, 11:29:41 pm
content: May I suggest using snprintf? This has a maximum buffer length parameter so it helps against buffer overflows. Either way most C libraries route anything printf-ish into a standard routine (IIRC) vsprintf.

author: Sal Ammoniac
author_rank: Super Contributor
timestamp: September 29, 2016, 11:58:36 pm
content: Quote from: nctnico on September 29, 2016, 11:29:41 pmGood point. Actually I do use snprintf rather than sprintf.I've traced through the code for some library versions of sprintf and some of them are almost unbelievably complex and execute literally thousands of instructions to format a simple string.

author: langwadt
author_rank: Super Contributor
timestamp: September 30, 2016, 12:22:25 am
content: Quote from: Sal Ammoniac on September 29, 2016, 11:18:37 pmgoogling for tiny printf shoudl give you some hints, likehttp://www.sparetimelabs.com/tinyprintf/tinyprintf.php

author: nctnico
author_rank: Super Contributor
timestamp: September 30, 2016, 01:14:20 am
content: Quote from: Sal Ammoniac on September 29, 2016, 11:58:36 pmTrue. The regular C libraries have become bloated but the interface still dates back to when C got invented and it is very lean and mean. IMHO one of the most common failures is that people throw all the string functions overboard because printf is too large. A much better solution is to use a small C library combined with tiny printf (as langwadt proposed) and keep the good stuff.

author: westfw
author_rank: Super Contributor
timestamp: September 30, 2016, 09:46:29 am
content: avr-libc has looked promising, has a good license, and seems like it wouldn't be too difficult to port to something else.I was really put off by what newlib does to implement stdio, compared to avr-libc... (OTOH, a CM4 is more likely to actually have enough of a filesystem to make the newlib scheme have some advantages...)

author: mac.6
author_rank: Regular Contributor
timestamp: September 30, 2016, 12:37:10 pm
content: Most uses of malloc() are for string functions, and it's not true dynamic allocation. Allocation is only valid during function call to avoid using too much stack.For me it's fine as no string function shoould be in the critical path anyway.And it's better to reimplement malloc with more efficient algorithm than reimplementing string function which are the number 1 entry point for exploits.And remember that we are in 201d, string are not longer char*...

author: legacy
author_rank: Super Contributor
timestamp: September 30, 2016, 01:01:01 pm
content: Quote from: mac.6 on September 30, 2016, 12:37:10 pm

author: mac.6
author_rank: Regular Contributor
timestamp: September 30, 2016, 01:10:36 pm
content: to be more specific: string are no more ascii nowadays, so implementing anything more than a debug_printf requires a lot of work and testing...

author: nctnico
author_rank: Super Contributor
timestamp: September 30, 2016, 01:15:14 pm
content: Quote from: mac.6 on September 30, 2016, 01:10:36 pmUTF-8 is perfectly backwards compatible with '8 bit' strings but any string operation must be limited to the number of characters which fit in the receiving buffer.

author: Sal Ammoniac
author_rank: Super Contributor
timestamp: September 30, 2016, 02:34:49 pm
content: Quote from: mac.6 on September 30, 2016, 12:37:10 pmThey are for me, but then again I'm a provincial American doing this as a hobby and don't need to support other weird languages. ASCII's good enough for me--fuhgettabout that Unicode malarkey!

author: nctnico
author_rank: Super Contributor
timestamp: September 30, 2016, 02:49:46 pm
content: Consider yourself lucky.. in one of my projects I had to support several languages with weird signs so I had to write a UTF-8 to LCD display character set converter. The UTF-8 texts also had to go through the editing and compile stage without getting mangled to avoid cumbersome linking in of the texts as binary object. Fortunately Eclipse and GCC managed to do that.

author: Brutte
author_rank: Frequent Contributor
timestamp: October 05, 2016, 07:05:29 am
content: Quote from: Sal Ammoniac on September 29, 2016, 08:28:26 pmYes, it does, for printf-ing for example. But it calls sbrk only once (at first printf) and allocates ~1kB buffer for parsing printf string. When printf does its job, it just passes the buffer pointer further (to UART, semihosting, etc). This 1kB is not free()-d but hogged till exit(). Speeds up processing. Imagine a newlib-nano without malloc(), that does the printf in byte by byte fashion (like avr-libc)

author: legacy
author_rank: Super Contributor
timestamp: October 05, 2016, 09:43:12 am
content: well, the Oracle of Delphi (who promotes pascal & ADA)had said that things like *printf*MUSTbe avoidedlike a deadly sin which can causebad thingslike whenit rains cats and dogsand even outside the Matrixthey all know (including the Architect and his henchmen) thatit's actually raining a ton ofcode-bugsdue to lazy humanstherefore, every attempt to re-implementthose bad things must be banned and punishedand all of those dissidents and rebelsreduced in bondage and obliged to the hammer & stonesto build pyramids with spiral stairs and elliptical templesin the name of goddess ADAhad said the wise -don't make the Oracle so angry-but we still staid in those things because we still had H.O.P.E.(was itHold On, Pain Ends!, ain't it? ... even under the whip?)(kidding, it's my humor)

author: Kremmen
author_rank: Super Contributor
timestamp: October 05, 2016, 11:02:49 am
content: I didn't read through every single post so maybe this was mentioned already, but i'll say it anyway:Have you checked the allocators available with FreeRTOS? It comes with several heap managers that behave differently. heap4.c seems to be the most popular but there are others. The descriptions should be available at FreeRTOS site and the code is validated by use in countless projects.

author: westfw
author_rank: Super Contributor
timestamp: October 05, 2016, 09:11:41 pm
content: QuoteAn interesting thought. On an AVR, the likely next-level destination is a UART or other byte-oriented peripheral, so it probably doesn't make much difference. On more complex systems (say, those involving an OS), the next level is likely to be a filesystem on some block-mode media, and there WOULD be a significant advantage to an intermediate buffer... (in fact, avr-libc pretty much leaves out the "filesystem" layer, having their stdio functions work directly on top of "devices.")

author: Brutte
author_rank: Frequent Contributor
timestamp: October 06, 2016, 07:21:03 am
content: Quote from: westfw on October 05, 2016, 09:11:41 pmYes, but once you hitCode:printf("Chicken counter: %d\n",chickens);the function calls deeper and deeper internals and once some int2str function produces a char, that single byte is sent to uart, then once it is TXed the UART returns, int2str produces second char, etc,etc...PITA.The effect would be that you would never have a complete printf string in memory, only single bytes. That "avr printfing" style would also render semihosting impossibly slow.The alternative for newlib-nano could have been to allocate the buffer statically (without sbrk) at bss:Code:char stdio_buffer;which has the advantage that it can be tied to designated SRAM address (easier debugging and overrun control).

author: Kalvin
author_rank: Super Contributor
timestamp: October 06, 2016, 07:43:42 am
content: Quote from: langwadt on September 30, 2016, 12:22:25 am+1 for this. I have used this as a starting point / idea when creating my own printf() and sprintf() functions. My major modification was to pass a pointer to a sprintf struct which contains a pointer to a string buffer (char*) and the maximum length given while calling the sprint(). As each character is produced, the function will check whether there is still room in the string buffer. However, if the pointer is NULL, it will pass the characters to the serial port instead. Small overhead, but has proven handy. It can be tailored using various compilation time ifdefs to exclude the features I do not need.

author: nctnico
author_rank: Super Contributor
timestamp: October 06, 2016, 07:52:48 am
content: Regarding (re)directing output: I usually do that in putchar (which is called by any string output function) and if speed is an issue in 'puts' as well.

author: Kalvin
author_rank: Super Contributor
timestamp: October 06, 2016, 08:00:10 am
content: Quote from: nctnico on October 06, 2016, 07:52:48 amThat was how I implemented as well, but it requires passing the pointer throughout the functions. However, one can use a global variable for the buffer struct which reduces the overhead a little but the sprintf will not be re-entrant any more. The printf is never a re-entrant by its nature unless one uses a semaphore.

author: nctnico
author_rank: Super Contributor
timestamp: October 06, 2016, 08:14:45 am
content: I use a global variable so everything stays the same. Another option for redirecting would be to implement fprintf and so on but that could end up being a lot of work.

author: Kalvin
author_rank: Super Contributor
timestamp: October 06, 2016, 09:08:31 am
content: I had a PIC24F project which was required to run on a battery for two-three years, so the energy consumption needed to be reduced as much as possible. I implemented the printf so, that the printf configured the serial port, then printed out the printf arguments, and finally closed the serial port. The closing function took care of waiting until the last character was sent, and then disabled the UART peripheral block.

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 09:30:06 am
content: I don't understand the why? on the why?you all need to implement that printf thingscan't you just implement dedicated functions,one for each data_type? in ADA style?out_uint32(string_t msg_head, uint32_t data, string_t msg_tail);out_sint32(string_t msg_head, sint32_t data, string_t msg_tail);out_fp32(string_t msg_head, fp32_t data, string_t msg_tail);out_fx32(string_t msg_head, fx32_t data, string_t msg_tail);msg_head can be empty "", or "\0", therefore skippedmsg_tail can contain the CR "\n"from my point of view, it'ssimplersaferless error pronedoesn't require any stack manipulationconsumes less resourcesfp32 is for floating pointfx32 is for fixedpointthey both need some additional work

author: Kalvin
author_rank: Super Contributor
timestamp: October 06, 2016, 11:54:28 am
content: Quote from: legacy on October 06, 2016, 09:30:06 amFor the convenience mostly. Also, each function call will create some overhead which may be a problem with a MCU with a constrained code memory. I have used the Ada-style print-functions you pointed out in the past, too.

author: nctnico
author_rank: Super Contributor
timestamp: October 06, 2016, 01:11:46 pm
content: Quote from: legacy on October 06, 2016, 09:30:06 amSimple: printf and (more generally speaking) the C libraries where invented while the amount of memory available on computers was tiny so they went for the most efficient interface. I have seen people trying to re-invent the wheel many times but in the end the core of printf gets linked in anyway because it is so cleverly constructed.

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 01:36:19 pm
content: Quote from: nctnico on October 06, 2016, 01:11:46 pmit's not a matter of re-inventing the wheel,it's a matter or mind-set: those C-ish thingsare not error-prone, and it's time to movedon't expect to find any printf-like in any avionic productQuote from: nctnico on October 06, 2016, 01:11:46 pmit doesn't seem "cleverly" constructed at allin first place it's composed by unsafe codeCode:void printf(char *fmt,... <---------------------- this is considered unsafe codeCode:va_list va;va_start(va, fmt);...va_end(va);}all of these are unsafe codeand crap like the followingCode:*(*((char * *) p))++ = c;I have just checked with our QA-tooland it violates 43 rulesanyway, even considering implementation detailsthe ADA-like approach does not produce more codetherefore, you actually have choices, so it's only up to youpersonally I have already replaced every printf since a while

author: Kalvin
author_rank: Super Contributor
timestamp: October 06, 2016, 01:39:25 pm
content: C as a language is broken anyway, why bother. Yes, you can pretend that C will be a safer with some constructions and limited subset, but nevertheless the C as a language is unsafe.

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 02:13:36 pm
content: Quote from: Kalvin on October 06, 2016, 01:39:25 pmI agreehere, if someone for some reason needs to use the stackpractically if he or she needs to useva_list,va_start,va_endand all things defined bystdarg.h, then he or she must classifythe code as "unsafe" and this has the consequence of keepingmore attention from the testing dudes first, and then from the QAand their acceptance more difficult to be achieved(if he or she accidentally uses those things without classifyingthe code as "unsafe" ... the code will be rejected immediately)

author: westfw
author_rank: Super Contributor
timestamp: October 06, 2016, 09:37:16 pm
content: QuoteBecause it's a considerable aid to program readability to have a sort of "picture" of what your output will actually look like. Fortran has it, COBOL has it, printf() provides it with C.QuoteAnd by the time you implement all the different formatting options that people want, how many functions do you have? Or how many options on each "dedicated" function? With how much overhead for each option? Field widths, justification, precision, sign location, leading character choice...QuoteWhy are these considered so unsafe, when most languages with built-in IO implement plenty of "statements" with variable numbers of arguments, even those languages considered to be much safer than C (Pascal, for example)? Note that printf() has had strong type checking for its arguments in many C compilers for a decade or so now (much to the chagrin of those who have implemented "custom" versions of printf()...) A lot of these "QA-tools" seem to be thinking "the way C handled variable numbers of arguments in the distant past was clearly dangerous. Therefore any modern modification must be dangerous. Therefore printf() is dangerous no matter how many checks have been added to it..."

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 10:43:44 pm
content: Quote from: westfw on October 06, 2016, 09:37:16 pmPascal does not have it, you need to pass through assemblyADA does not allow it without passing through the unsafe declarationQuote from: westfw on October 06, 2016, 09:37:16 pmif you pass the following to some (old) C compiler you will see a crashCode:printf("%s", 0xdeadbeaf);some old compilers don't actually check it at all, since everything is acceptedgcc and llvm do some checks, but it's not guaranteed by the "C language"it's up to the implementation, and it might pass with jus warning shoot

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 10:59:28 pm
content: Quote from: westfw on October 06, 2016, 09:37:16 pmin first place because having variable numbers of arguments in variable sizemakes the testing activity more complex, even for debuggers and validators

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 11:09:15 pm
content: Code:#include <stdio.h>int main(){printf("%s", 0xdeadbeaf);return 0;}Code:co-nix app # echo "nowarnings" > project.opt.myprjco-nix app # mygcc-build-project-makefile-v5co-nix app # make- generating interface public for app ... done- generating interface private for app ... done- linking to myCode:co-nix app # echo "warnings" > project.opt.myprjco-nix app # mygcc-build-project-makefile-v5co-nix app # make- generating interface public for app ... done- generating interface private for app ... done- compiling app ... app.c: In function ‘main’:app.c:5:1: warning: format ‘%s’ expects argument of type ‘char *’, but argument 2 has type ‘unsigned int’ [-Wformat]done- linking to myCode:co-nix app # obj/co-nix-mips32r2/mySegmentation fault

author: nctnico
author_rank: Super Contributor
timestamp: October 06, 2016, 11:13:36 pm
content: Quote from: legacy on October 06, 2016, 10:59:28 pmTry and create a versatile formatting function in any language which can do the same as printf and see how testable/verifiable it is.

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 11:46:23 pm
content: a good programmer should always remember thatthose printf are extremely unsafe constructions.and those things is that if a formatting specifiersomehow gets inside the string, it will lead tounpredictable consequencesQuote from: nctnico on October 06, 2016, 11:13:36 pmADA has safe mechanisms to do that, e.g. generic data instantiateC++ also has something similar, e.g. the use of "<<" operator

author: legacy
author_rank: Super Contributor
timestamp: October 06, 2016, 11:49:15 pm
content: oh, even the Go language has called it "unsafe"go-static-check/unsafe-printf.go

author: nctnico
author_rank: Super Contributor
timestamp: October 07, 2016, 12:17:07 am
content: Quote from: legacy on October 06, 2016, 11:46:23 pmI'm not seeing much use of the << in C++ probably due to the lack of formatting. And try to use a non-constant string as a format string... I gave up on that over 15 years ago. GCC won't allow it and goes to great lengths to prevent it.

author: westfw
author_rank: Super Contributor
timestamp: October 07, 2016, 09:49:13 am
content: Quotehuh? I'm talking about IO support built into the language, not user procedures. Pascal's standard "read" and "write" procedures support an arbitrary number of arbitrary base-typed arguments.(I tend to find it particularly annoying when a language defines procedures that permit a syntax that can't be duplicated by user-written procedures. Though perhaps it is a fine idea from a "safety" perspective. :-( I think C is one of the first languages where this is rigorously avoided, though I don't know that I've ever seen that expressed as a design goal of the language. (It used to be common for timesharing system user commands as well; you could do things from the user interface (which was part of the operating system, rather than a separate program) that you could not accomplish from within a user-mode program...)

author: langwadt
author_rank: Super Contributor
timestamp: November 07, 2016, 11:14:13 pm
content: Quote from: Kalvin on October 06, 2016, 01:39:25 pmsoldering irons are unsafe, there is only limited number of ways you can use one without breaking things or burning you fingers

author: legacy
author_rank: Super Contributor
timestamp: November 08, 2016, 06:12:38 pm
content: Quote from: langwadt on November 07, 2016, 11:14:13 pmis it humor?

author: Harjit
author_rank: Regular Contributor
timestamp: October 19, 2018, 05:50:38 pm
content: Did you end up with an alternative that didn't use dynamic memory? Thanks.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 19, 2018, 06:44:26 pm
content: Quote from: legacy on November 08, 2016, 06:12:38 pmIrony.

author: richardman
author_rank: Frequent Contributor
timestamp: October 20, 2018, 02:50:38 am
content: It would be interesting to know WHICH C string functions use malloc, because ... they shouldn't and I have difficulty believing that nano-newlib string functions use dynamic memory.

author: technix
author_rank: Super Contributor
timestamp: October 20, 2018, 04:42:10 am
content: It kind of make sense to implement printf using malloc since the resulting string has a undetermined length. If you need to avoid that you can stack or static allocate the memory and snprintf into it, then output it. (All the printf-class functions calls vsnprintf internally, which requires the caller to supply a buffer and its length.)Code:ssize_t printf(const char *format, ...){va_list args, args2;va_start(args, format);va_copy(args2, args);ssize_t length = vsnprintf(NULL, 0, format, args2);va_end(args2);if (length > 0){char *buffer = malloc(length);if (!buffer){va_end(args);return -1;}length = vsnprintf(buffer, length, format, args);length = fwrite(stdout, buffer, length);free(buffer);}va_end(args);return length;}

author: rstofer
author_rank: Super Contributor
timestamp: October 20, 2018, 03:26:16 pm
content: When I first start an embedded project, I get the UART working - getc() putc()Then I grab up "The C Programming Language" (Kernighan & Ritchie) and copy the string and conversion functions.No heap, simple code, perfect!I usually build a bunch of layered functions to display (in hex) a nibble, byte, short and int with the '0x' prepended. I might also use the decimal conversion code - itoa().To the extent possible, I don't want to use the library code. Obviously, I can't avoid the C runtime code but I certainly don't need the newlib string functions.

author: technix
author_rank: Super Contributor
timestamp: October 20, 2018, 06:03:38 pm
content: Quote from: richardman on October 20, 2018, 02:50:38 amObviously printf and fprintf (vprintf and vfprintf) use malloc but sprintf and snprintf (vsprintf and vsnprintf) don't. strdup does as per specified by its contract as a chained call of malloc and memmove. Some implementation of sscanf (vsscanf) uses malloc for internal buffers, and if so through that fscanf and scanf (vfscanf and vscanf) will also use malloc.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 20, 2018, 06:21:32 pm
content: printf(), at least in its original implementations, is basically a character-by-character function calling putc() for each character. It doesn't need any dynamic allocation per se except a few local variables AFAIK.I haven't looked at a "modern" implementation of it in ages, so some libs may use dynamic allocation to speed things up, but I have no clue. It's not really necessary. (Reminds me of a discussion we had on dynamic allocation though. Depends on what we call that. Using local variables and function parameters is still some kind of dynamic allocation. So I guess we should add "on the heap".)As for using it in embedded contexts - I think I've never used printf() directly in any embedded software in ages. Admittedly I do use sprintf() or better, snprintf() when available and mostly for non-timing critical tasks such as UI needs. I don't see the point of using printf() and having to define a custom putc(), as using snprintf() and the likes and then sending the resulting buffer to whatever peripheral will be much more efficient since you can take advantage of the specifics of the peripheral in terms of buffering (FIFO, DMA...), something you can't with putc() AFAIK. Some might argue that printf() allows a non-limited buffer size by nature, and it's true, but I actually don't like that, especially in embedded settings. Far too slippery for my taste. I prefer bounding every transaction that I issue.That said, in terms of performance or code size, yes sprintf() does basically the same thing as printf() except it doesn't need to call a callback function like putc(). I wouldn't use it when I need performance. printf() and its siblings is very inefficient (although admittedly very flexible and useful) compared to implementing direct formatting functions as needed. Even on a fast PC. Just test a sprintf() call with a moderately complex format in a loop and time it. Compare to the same thing implemented with direct formatting. Of course if neither space nor timing is critical, have at it. But then avoid sprintf() if you can and use snprintf() and the likes instead. Avoids the possibility of nasty buffer overflows.Other basic functions from the standard lib are much more efficient and take up little space, such as memcpy(), strcpy(), strcat(), memset(), memcmp()... With optimizing compilers, they are even often compiled as direct and very efficient inlined code. So there's no reason to refrain from using those.

author: richardman
author_rank: Frequent Contributor
timestamp: October 21, 2018, 02:29:11 am
content: Quote from: technix on October 20, 2018, 06:03:38 pmYes of course strdupby definitionuses malloc. The OP though said STRING functions, and we can all agree that printf, sprintf, etc. are stdio.h functionsIn other words, I am not convinced of the OP's claims. I think other than printf and friends, and strdup, nanolibs or any C library implementation, for that matter, should or WOULD use malloc.*as a compiler writer, I have written my share of C library functions*

author: technix
author_rank: Super Contributor
timestamp: October 21, 2018, 04:52:58 am
content: Quote from: richardman on October 21, 2018, 02:29:11 amFrom a maintainer's aspect, using malloc in fprintf (and in turn printf) makes sense, as it means that there would be only one instance of printf code in vsnprintf and all other whatever-printf functions directly or indirectly calls it. A malloc-less version would have to include two copies of almost the same code in vsnprintf and vfprintf respectively, or use some function pointer magic and slash vsnprintf efficiency almost in half.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 21, 2018, 05:47:59 pm
content: Quote from: technix on October 21, 2018, 04:52:58 amI beg to differ. They just have to call a 'putc' function instead of writing to the given buffer's current pointer and incrementing it. Just one test.

author: technix
author_rank: Super Contributor
timestamp: October 21, 2018, 06:19:39 pm
content: Quote from: SiliconWizard on October 21, 2018, 05:47:59 pmTechnically this can be done: (Please be patient while reading the code.)Code:ssize_t _real_vnprintf(ssize_t limit, void (*action)(void *context, char ch), void *context, const char *format, va_list args){while (...){char ch;// Actual printf codeaction(context, ch);}}struct _vsnprintf_context{char *buffer;off_t *location;}void _vsnprintf_action(void *context, char ch){struct _vsnprintf_context *ctx = context;ctx->buffer[*ctx->location] = ch;*ctx->location++;}ssize_t vsnprintf(char *buffer, size_t limit, const char *format, va_list args){struct _vsnprintf_context ctx = {buffer, 0};ssize_t rv = _real_vnprintf(limit, _vsnprintf_action, &ctx, format, args);ctx.buffer[ctx.location] = 0;return rv;}ssize_t vfprintf(FILE *file, const char *format, va_list args){return _real_vnprintf(-1, fputc, file, format, args);}The overhead of calling _vsnprintf_action for each character will add up - not ideal for slow CPU. Or if _real_vnprintf is a macro the resulting code will have two copies of printf code, one for vsnprintf and one for vfprintf - not ideal when Flash memory is constrained.The implementation of vfprintf through malloc and vsnprintf allows those two scenarios to be averted: there is only one copy of printf code in vsnprintf, and that is the fastest possible version. newlib seemed to me lacked dynamic stack allocation which is usually a better option for this though.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 22, 2018, 12:32:37 am
content: I can definitely think of a couple ways of implementing just one core function for all the printf() variants and use it to implement them without duplicating code. Definitely doable without code duplication.

author: technix
author_rank: Super Contributor
timestamp: October 22, 2018, 02:55:13 am
content: Quote from: SiliconWizard on October 22, 2018, 12:32:37 amI had put up an example above, but that code uses function pointers and reduces efficiency of vsnprintf significantly due to all the function call overheads. The best solution I know of would be using a dynamic stack allocation (similar to malloc but the memory is allocated on stack and freed when exiting the current stack frame.) which also builds vfprintf using a fast vsnprintf like that malloc example above but does not rely on malloc or sbrk.

author: richardman
author_rank: Frequent Contributor
timestamp: October 22, 2018, 09:52:58 am
content: Speaking as a compiler vendor who has multi-thousands of users since 1994 for MCU from HC11, HC08, CPU12, ARM V7, Propeller, MSP430, Cortex... we have always used a single core printf-format function and use function pointers to either write using putc, or stuff the output to a buffer, depending on whether it is printf or sprintf (and the variants).Indeed, for the smaller MCUs, it's a worse sin to have multiple similar code functions than printf being slow as customers can easily understand that printf can be and will be slow, due to its features.

author: technix
author_rank: Super Contributor
timestamp: October 22, 2018, 01:03:48 pm
content: Quote from: richardman on October 22, 2018, 09:52:58 amI had the misfortune of having to work with some folks that really loves to use printf, and they need a fast implementation of printf with a limited memory space.

author: Karel
author_rank: Super Contributor
timestamp: October 22, 2018, 02:47:49 pm
content: Quote from: langwadt on November 07, 2016, 11:14:13 pmC language is not the problem. C is a compromise between overhead/performance andabstraction level. C does require an experienced programmer who knows what he is doing.Or is a surgeon with a scalpel also a problem?

author: legacy
author_rank: Super Contributor
timestamp: October 22, 2018, 03:16:16 pm
content: Quote from: Karel on October 22, 2018, 02:47:49 pmIf the C language wasn't the problem, we wouldn't have the need to schedule up to the 45% of the activities to make it MISRA compliant and then to make it DO178B-level{A-C} compliant. I can assure you nobody is happy to spend his paid time for these activities since they are very boring and frustrating.Can't those developers write their code already compliant? No, with C, usually they can't, but in avionics, we don't need to schedule so many % of activities with everything is written in Ada.Unfortunately, we can't write the firmware of our AFDX, ARINC, and stuff in Ada, which is only used for everything is on the top of the BSP, iBSP, and OS.I don't want to be put in the same basket of those who are arrogant, but personally, I think the C language carries the design and philosophy of the 80s, whose attempts to fix, C89 and C99, have helped, but not resolved the problem.

author: Karel
author_rank: Super Contributor
timestamp: October 22, 2018, 03:44:29 pm
content: Quote from: legacy on October 22, 2018, 03:16:16 pmI understand your worries when writing software for safety-critical purposes.For all other purposes where you don't want slow and bloated and energy consuming software,C is a good choice for motivated programmers who understand the internals of computers and electronics.Anyway, whether you like it or not, C is here to stay and it's not going anywhere in the foreseeable future.In fact, it's one of the most used programming languages. (only Java is more used)

author: legacy
author_rank: Super Contributor
timestamp: October 22, 2018, 04:11:22 pm
content: Quote from: Karel on October 22, 2018, 03:44:29 pmI am not sure about this. There are other languages in development, and for example, The Rust Programming Language is very interesting and promising

author: JPortici
author_rank: Super Contributor
timestamp: October 22, 2018, 05:03:30 pm
content: Quote from: legacy on October 22, 2018, 04:11:22 pmBut until the MCU manufacturer or somebody else come up with a certified rust (or whatever language) compiler for said MCU, I (as in developer) am going to what's available, which is usually a C compiler, which is why C is going to stay for a long time..

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 22, 2018, 05:36:35 pm
content: Quote from: richardman on October 22, 2018, 09:52:58 amYes, that's pretty much what I was meaning. I don't think there is another sane way of implementing it on small targets, and probably not even on large ones.Quote from: richardman on October 22, 2018, 09:52:58 amStrictly speaking of printf() and its siblings, I am not even convinced whatsoever than using malloc() would speed anything up in the general case compared to using function pointers and so on. It may, on some specific platforms and with a specific set of test cases, but most likely not at all in the general case. The standard malloc() has undeterminate execution time and will be probably much slower on average than calling callback functions for each character.That may prove marginally faster on VERY large formatted sequences, but most likely worse in general.

author: richardman
author_rank: Frequent Contributor
timestamp: October 22, 2018, 06:37:39 pm
content: Programs are written by people who know how to program. The military and the avionic industrymandatedthe use of Ada in the 90s and on. People thought the era of "better software engineering" for such use would be upon us...... and yet, most of those software are still and will be written using C and C++ after the mandatory Ada requirement was dropped. From what I understand, even though there has been decent, even "free" Ada compilers such as GNU Ada, it was and is difficult to attract enough talent.So back to a case of supply and demand.

author: legacy
author_rank: Super Contributor
timestamp: October 22, 2018, 07:56:18 pm
content: Quote from: richardman on October 22, 2018, 06:37:39 pmjudging by all the effort we have to invest in our activities, let me say it may be a wrong assumption, and the C language doesn't put any barrier on this.e.g. we considered an embedded FAT filesystem(1) given us by a third company, and it ended we had to spend three times the effort we would have invested in writing it from scratch(I had told it to my boss, but he didn't want to listen ...)why? apart the big mistake made on the wrong assumption that we would have to use GCC(2) (which is extremely bad since it means the code is NOT portable), the authors were pretty ignorant about software design, they have completely ignored all the error propagation in their interfaces, and worse still, their poor modularity made the testing activity like a nightmare, which then usually costs more time to sort it out.now, a lot of these concepts would have been mitigated with Ada, resulting in us to had spent less effort on it.Quote from: richardman on October 22, 2018, 06:37:39 pmunfortunately, we can't use Ada for an OS, or for a BSP, we neither use the C++ for this, but it's irrelevant: the point is everything needs to be written in C requires a lot of effort from the engineering step the production step, and the effort we have to invest with C is several orders of magnitude higher than the effort we have to invest with Ada, which is ridiculous since what is written in Ada is usually of higher complexity.From my experiences:C: low complexity design requires usually high effort (can't Imagine the effort for a high complexity design ... probably it's of the level of several chronic headaches)Ada: high complexity design usually requires low effort, and it's less error proneI am one of the those have written the AFDX switch's firmware purely in Ada. It has already passed all the qualifications with a quarter of the effort requested by a C implementation, but ... for some strange reasons is not in production, and it's frustrating because for a reason, or for another, it really seems people really don't want to move on(1) it's for the navigation maps of tactical missions. It needs to burst-read large files from a USB pen drive. There is a dedicated board for this, with a dedicated interface to the tactical viewer on the pilot's helmet.(2) we are usually with Diab and GreenHills.

author: richardman
author_rank: Frequent Contributor
timestamp: October 22, 2018, 09:02:42 pm
content: Legacy, you just proved my point

author: legacy
author_rank: Super Contributor
timestamp: October 22, 2018, 10:05:08 pm
content: Quote from: richardman on October 22, 2018, 09:02:42 pmmaybe, but what really matters is that soon or lateI will convince the project managers to accept my work in Ada, and it will be the end of the trouble we are having with the AFDX and ARINC stuff written in C!That D-day, in the near future, will be remembered in every local calendar until the end of time, with at least one week of celebration and a boxed set of bottles of champagne offered to all the team, including all the secretaries, the guys, and girls working at the reception

author: westfw
author_rank: Super Contributor
timestamp: October 23, 2018, 07:36:36 pm
content: QuoteWe never DID get a full description of which "string functions" were supposed to be using malloc(), did we?OTOH, whether a given set of functions CAN be written without malloc() is only slightly comforting to the user faced with a library that DOES use malloc(). Yeah, lots of us are in favor of writing low level C code without resorting to "bloated vendor libraries", but we don't usually extend that to writing our own versions of the standard libc functions...QuoteWell, that's going to be pretty highly dependent on what is "underneath" putc(), isn't it? I mean, putc() is defined to send characters to a file, and newlib (for example) has a whole very general stdio implementation underneath putc(), with quite a lot of overhead for each call to putc(), not counting any underlying "file" implementation (like an actual disk) that might be really "unhappy" being asked to do individual byte writes (probably there's another whole layer of malloc() for the file buffers.)I don't have any trouble believing thatmanycases are more efficient with an intermediate buffer somewhere in vsnprintf()Even if you do "short-circuit" sprintf() via function pointers (or hard-coded check), vsnprintf() is still going to REFERENCE malloc(), in order to support asprintf() variants. (I'm a little surprised that newlib doesn't have an "omit variations of functions that require malloc()" option, given MISRA and other aversions to malloc().(Ah, and doesn't "have libc functions that reference malloc(), but don't actually implement a working malloc because it's not allowed" sound like exactly the sort of thing that got Ariane's "AdaTran" code in trouble?)Alas, newlib looks like the sort of code that's been so generalized and option-filled and modularized that it would be difficult for anyone to figure out by looking at the source code, unless they spend a lot of time looking at it :-(https://github.com/32bitmicro/newlib-nano-1.0/blob/master/newlib/libc/stdio/putc.c#L92