Dhrystone 2.1 on mcus - Page 1
Forum section: electronics / general

author: dannyf
author_rank: Super Contributor
timestamp: March 01, 2014, 06:40:22 pm
content: I ran dhrystone 2.1 (a useless benchmark) on a few mcus that I have.I would run 10,000 times the benchmark, and then flip a pin. By measuring the duration between pin flips, we measured the duration of the benchmark. The shorter the duration, the faster the execution.No optimization of any kind, for any chip - the exact same code ran on all chips.[edit: represent the data in terms of Dhrystone / Mhz per second, from high to low]QuoteI was surprised:1) pic24f was really fast. and stm32f1/3 sucked on a per Mhz basis.2) avr sucked wind.3) stm8s did OK. Evident of 6502's staying power.Didn't run on 8051 but would expect it to hold its own reasonably well.edit: 1) added stm32f100 numbers.2) added stm32f100 + iar-arm, vs. gcc-arm.3) added results from IAR-ARM and represented the data to make it easier for the eyes.4) added results from pic18f5) updated PIC24F results (fat fingers) and added -O3 optimization6) added mdk-arm numbers for STM32F3. Pretty much identical unoptimized.7) added jaxbird's results for STM32F4 and PIC24H.added PIC24F results under XC16 free/pro. Still very high.9) added jaxbird's and hans' results for pic24h and pic24f/stm32f4, respectively.10) added LM4F120 under IAR-ARM and MDK-ARM.11) added LPC1343 under gcc-arm and MDK-ARM.12) added STM32F4 under gcc-arm and iar-arm, and mdk-arm too.13) added LPC1114 under iar-arm. The first CM0 chip in the comparison.14) added C51 (P87C51MC2) under Keil C51. Optimized for size and speed.15) added PIC32MX320 under C32, 2.x. (simulated)16) added PIC32MX440F512H results, for 80Mhz and 20Mhz17) added msp430. Fairly respective scores.18) added LPC1227 / IAR scores.19) added MSP430F2370 - similar to the MSP430 scores obtained earlier.20) added simulated results for PIC32MZ, and PIC24F (XC16 and C30)21) added the results for STM32F030F, from the ghetto thread.

author: Bored@Work
author_rank: Super Contributor
timestamp: March 01, 2014, 07:03:23 pm
content: Quote from: dannyf on March 01, 2014, 06:40:22 pmNot surprising. GCC without any optimization produces absolutely crap code.

author: tszaboo
author_rank: Super Contributor
timestamp: March 01, 2014, 07:50:49 pm
content: Quote from: Bored@Work on March 01, 2014, 07:03:23 pmYeah, kinda like ";" will be "NOP" after complying just to be able to put a breakpoint on it. Not to mention, cortex has a very well defined 1.25 DMIPS/MHz for M3 and 0.93 DMIPS/MHz for M0+ so running it again is kinda pointless. Unless you compare different complier settings, etc...

author: hans
author_rank: Super Contributor
timestamp: March 01, 2014, 08:24:40 pm
content: I was doing CoreMarks last week (as it so happens), to see what compiler settings did for PIC24.I chose CoreMarks because it focuses so much on Embedded Controllers and thus includes tests like pointer handling capability, branching etc. over pure integer speed.I just did some more tests and with XC16 enabled for optimizations, I got:-O 0 = 9.911 iterations / second (2700b RAM, 8015b FLASH)-O 1 = 23.56 iterations/second (2630b RAM, 6486b FLASH)-O 2 = 29.64 iterations/second (2630b RAM, 6634b FLASH)-O 2 -unroll = 31.13 iterations/second (2630b RAM, 8083b FLASH)-O 3 = 28.95 iterations/second (2630b RAM, 9454b FLASH)-O 3 -unroll = 30.09 iterations/second (2630b RAM, 12348b FLASH)-O s = 21.11 iterations/second (2630b RAM, 6484b FLASH)Interesting compiler/benchmark, where grade 2 is faster than optimize grade 3 and s(size).No optimizations being so slow doesn't surprise me, as it's for development and the debugger needs to be able to find the instructions in program memory.Also funny to see how well -O1 actually compares for this benchmark. However, as always with optimizations depending how stuff is written: YMMV.The coremarks benchmark is really easy to port (initialize uart for printf, insert timebase with 32-bit timer, done). Except.. for PIC18, because the program has a matrix multiply test, and uses 32-bit integer array indexing. The XC8 didn't like that. It said:../coremark_v1.0/core_matrix.c:244: error: can't generate code for this expressionAnyway, I see STM8 has 16-bit divide instructions, where the AVR only has multiply (8-bit). PIC24 has 32-bit by 16-bit divide, and 17-bit by 17-bit multiply. So I guess it wins out on a lot. Not sure why it would be faster than Cortex m3 though , because it has 32-bit multiply/divide. That wouldn't be logical, unless the bus is being stalled all the time or something (highly doubt that though).

author: GiskardReventlov
author_rank: Frequent Contributor
timestamp: March 01, 2014, 09:11:36 pm
content: Help me understand the numbers.pic24fj64ga102 @ 4Mhz (8Mhz crystal), duration = 2.12 seconds -> 8.5 seconds @ 1MhzIs this 8.5 Dhrystones/second?What compiler flags did you use and what versions of the compilers did you use?As a casual, naive observer I conclude that you didn't reach any conclusion. But maybe I missed how the chips you tested are similar or dissimilar. The only commonality I saw was that you have them.Correction:avr90usb1286should beat90usb1286

author: dannyf
author_rank: Super Contributor
timestamp: March 01, 2014, 09:22:47 pm
content: QuoteYou may try some earlier compilers from hi-tech.QuoteAgreed. I think we will know for sure once I have a look at the list file.I also didn't keep track of code size but that's not that interesting to me.

author: westfw
author_rank: Super Contributor
timestamp: March 02, 2014, 07:40:13 am
content: QuoteWhen you say that, do you mean that you didn't do any chip-specific optimizations, or that you completely turned off compiler optimizations (-O0) as well? The latter is practically worthless; some compilers do a lot more in the "optimization" step than others. Pick a generic compiler optimization parameter ("-O3"?) and use the closest equiv for each compiler.

author: dannyf
author_rank: Super Contributor
timestamp: March 02, 2014, 12:02:03 pm
content: Finished running the code on 18f. The XC8 free compiler did quite well vs. the good old picc18.

author: nctnico
author_rank: Super Contributor
timestamp: March 02, 2014, 12:35:48 pm
content: Quote from: dannyf on March 01, 2014, 06:40:22 pmUnless you used the same C library the same code didn't run on all chips. Drystone results are tainted by the efficiency of the C library.edit: After a short peek I see several calls to strcpy and strcmp. These should be replaced with functions inside the Dhrystone test so the results are not tainted by differences in the C library.

author: HackedFridgeMagnet
author_rank: Super Contributor
timestamp: March 02, 2014, 02:14:49 pm
content: QuoteYour right, but at least he has gone to the trouble to produce some results, and given a rough indication of how he did them.Which is a start, and most of us are stuck with the efficiencies of those C libraries anyway.

author: jaromir
author_rank: Unknown
timestamp: March 02, 2014, 09:11:20 pm
content: dannyf: maybe I overlooked something, but could you, please, provide your sources, say for PIC24? I'd like to verify it and maybe do tests with different MCUs.

author: dannyf
author_rank: Super Contributor
timestamp: March 02, 2014, 09:28:41 pm
content: It is the standard dhrystone package. It comes in one .h file and two .c files dhry_1 dhry_2.cgoogle it. They compile right away, with minimum changes.

author: GiskardReventlov
author_rank: Frequent Contributor
timestamp: March 02, 2014, 09:40:24 pm
content: Quote from: HackedFridgeMagnet on March 02, 2014, 02:14:49 pmToo rough to reproduce the numbers but it's all just for fun anyway.avr-gcc -varm-gcc -vetc.What voltages?The Dhrystone says that it's not meant to use any optimizations, unless you just want to test optimizations.But that's not the goal here. So they're just some numbersQuoteInefficiencies are bigger problem.

author: westfw
author_rank: Super Contributor
timestamp: March 03, 2014, 04:15:58 am
content: QuoteOf course they are. They're "tainted" by how good the C compiler is too. That's THE POINT. It's a benchmark of the chip+compiler+library SYSTEM. (and that's what SHOULD be meant by "applied no optimizations.") Benchmarks that only measure the chip are much MORE tainted.QuoteYeah, but we can't come up with numbers that compare directly to the ones in your table, without seeing exactly the wrapper code or your external calculations work. (It currently says "Drystones/sec/MHz"; but that doesn't look right, nor does it match up very well with the data (compared to published estimates.))

author: nctnico
author_rank: Super Contributor
timestamp: March 03, 2014, 09:57:30 am
content: Quote from: westfw on March 03, 2014, 04:15:58 amNot if you are comparing compilers. IMHO there is not much use to include a slow and fast C library in a benchmark test since you can always replace a slow C library function with a faster one.

author: jaxbird
author_rank: Frequent Contributor
timestamp: March 03, 2014, 10:48:59 am
content: Interesting comparison of mcus and compilersRan a couple for comparison and adding more data, same standard Dhrystone 2.1 code I assume.Results:STM32F4 (168 MHz) with KeilCompiler OptionsDhrystones/MHz/Second-o0806-o1932-o2942-o3955-o3 -otime3621 (likely not valid)PIC24HJ64GP502 (80 MHz) with XC16 (free/pro)Compiler OptionsDhrystones/MHz/Second-o0 (free)489-o1 (free)865-o2 (pro)1158-o3 (pro)1170Edit: Added more results for PIC24H

author: dannyf
author_rank: Super Contributor
timestamp: March 03, 2014, 11:30:51 am
content: -Yeah, but we can't come up with numbers that compare directly to the ones in your table, without seeing exactly the wrapper code or your external calculations work. (-as I said in the beginning, there is nothing in the my code others than calling dhrystone and flip a pin. You the measure the duration of the pun flip and calculate dhrystone per MHz per second.for example: say it takes 5 seconds to run 10,000 loops of dhrystone on a 10 MHz mcu. That's 10000 / 10 / 5 or 200 dhrystone per MHz per second.it may be clearer if you take a look at the code.

author: dannyf
author_rank: Super Contributor
timestamp: March 03, 2014, 12:06:06 pm
content: --o3 -otime 3621 -I ran dhrystone on a lpc2106 @ 30mhz (12mhz x 5 / 2) and I got 2000 plus. No time to debug it but it wouldn't surprise me that some thing is being optimized away.running a keil vs iar vs gcc comparison would be interesting.

author: dannyf
author_rank: Super Contributor
timestamp: March 03, 2014, 12:54:11 pm
content: On the last one: all the benchmarking I have seen, including keils own, suggests an upper end of 1000 Dhrystone per MHz for armv7 chips. The cortex chips are crippled or smaller so lower numbers for them make sense.

author: jaxbird
author_rank: Frequent Contributor
timestamp: March 03, 2014, 01:54:30 pm
content: Quote from: dannyf on March 03, 2014, 12:06:06 pmAgree, the difference is so large, it's likely not a valid result. I have not added any asserts on the outputs of the Dhrystone runs. Will try do a validation at some point.Yeah, it would be interesting to get some comparable results between the most used compilers and their optimizations. Of course it wouldn't necessarily mean the best result is always the best compiler, but would still be interesting though.

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 01:27:45 am
content: Ran PIC24F again (just to be sure). The earlier numbers actually were slightly lower due to key-in errors.Also ran -O3 on PIC24F. Over 2400 Dhrystones / Mhz. That is unbelievable,

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 01:46:23 am
content: Also added STM32F3's numbers under mdk-arm (4.x). A virtual tie with IAR when unoptimized.

author: nuhamind2
author_rank: Regular Contributor
timestamp: March 04, 2014, 01:56:33 am
content: Could flash memory speed affect performance ? On higher clock wait state need to be inserted so the performance/Mhz is worse on high freq. I know that STM32 has flash accellerator or something, but how efficient is that I don't know and I think flash accelerator won't help for indirect jump. Don't know though whether Dhrystone generate indirect jump.just my 2c

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 02:20:11 am
content: I combined jaxbird's results for F4 and 24H with mine in the first post. When I get some time, I will try XC16 on the 24F as well.On wait state, I think so as well, particularly at high speed so performance is unlikely to increase with speed linearly. But it would be difficult for me to quantify.

author: nuhamind2
author_rank: Regular Contributor
timestamp: March 04, 2014, 03:05:10 am
content: Quote from: dannyf on March 04, 2014, 02:20:11 amHow about running the benchmark on slower frequency and disabling wait state.

author: westfw
author_rank: Super Contributor
timestamp: March 04, 2014, 03:47:31 am
content: So you essentially replaced the printf code in the original benchmarks with a pin toggle for measuring the timing? And these numbers should result in the nominal DMIPS/MHz if divided by the usual magic constant (1757 for a VAX780)? That puts the PIC24 at about 1.4, which seems to be in the usual range for modern microcontrollers. (1.25 to 1.89 is what ARM quotes for CM3.)The interesting question is why most of the other chips' numbers are so much lower than expected.

author: nuhamind2
author_rank: Regular Contributor
timestamp: March 04, 2014, 11:50:57 am
content: I doubt that's the case for AVR,AVR flash is always as fast as the core when fetching instruction. The only slow down happen when doing literal load from flash which take 3 cycles compared from RAM which take 2 cycles. But you only load constant from flash if you state to store your data implicitely in flash using some kind of modifier.Otherwise your data will be copied to RAM and loaded from there.

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 12:10:34 pm
content: QuoteWill look into that later.

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 01:47:52 pm
content: Played with the flash wait state (aka latency) on gcc-arm. Without optimization, the default latency setting produced a Dhrystone / Mhz score of 766 on STM32F3. Pushing it to wait state of 0 pushes the score up about 100, and pushing it to wait state 2 pushes the score down about 100.

author: Kjelt
author_rank: Super Contributor
timestamp: March 04, 2014, 01:56:32 pm
content: Interesting results, thanks

author: nuhamind2
author_rank: Regular Contributor
timestamp: March 04, 2014, 02:32:15 pm
content: Just adding some info , the stm32f3 page on mouser state 62 DMPIS/72Mhz (2 wait state) or 0.861 DMIPS/Mhz ( pretty close to the benchmark posted here )and 94DMIPS/72Mhz when running from CCM-RAM (0 wait state) or 1.305 DMIPS/Mhz

author: jaxbird
author_rank: Frequent Contributor
timestamp: March 04, 2014, 02:40:09 pm
content: I'm surprised by the difference in results between the pic24f using C30 and pic24h using XC16. It's close to twice the score for the pic24f/C30 combination.I had a go at a few different frequencies the pic24h to see if that makes any difference. My original test was run at ~80MHz.4 MHz - 119520 Mhz - 119780 MHz - 1190(All with pic24h, XC16, -O3 optimization)Very similar, I'd put the differences down to not 100% accurate time keeping, plus using the internal 7.37MHz oscillator with feedback divisor and pre/post scaler results in frequencies slightly below or above the target.

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 05:34:18 pm
content: Yeah. I am with you on that. The PIC24 numbers are simply too good to be true. My first reaction then was that the compiler may have been coded to recognize dhrystone code.When I get some time, I am going to insert some pin flipping code in the dhrystone itself to see if all pieces of the code are actually executed.

author: hans
author_rank: Super Contributor
timestamp: March 04, 2014, 06:54:37 pm
content: It could also be pipeline differences between MCU's that can cause Dhrystone to perform very well or very bad. PIC18 does 1 instruction every 4 ticks, PIC24 does 1 every 2, and ARM 1 every 1, however they probably run a pipeline (others could do too). But if one instruction modifies the data of the next instruction, then that next instruction can be stalled until the preceding instruction was completed.I don't think this changes with clock speed, because the pipeline only runs at the instruction clock.I've now become interested to see why PIC24 is so fast, and "forked" this version of the Dhrystone:https://github.com/rkrajnc/amber/tree/5f1fc912d06346cc3266a0ed0148f0b4272f1e43/sw/dhry( used for testing Amber ARM FPGA softcore)Interesting part is that the test lists no of cycles for other processors like Intel i3: it says that takes 389 cycles per Dhrystone. This means 2570 Dhrystone/MIPS.For PIC24FJ64GA004 it took 1001 cycles(XC16 -O0), so that means I got 999 Dhrystones/MIPS with XC16 -O0.With -O3 -unroll I get 483 cycles per Dhrystone, which means 2070 Dhrystones/MIPS .With -O3 I get 452 cycles per Dhrystone -> 2212 Dhrystones/MIPS .With -O3 and small code model, small data model, constants in RAM I get 421 cycles per Dhrystone -> 2375 Dhrystones/MIPS .Very much in line with what dannyf tested. But, as the PIC24 does 0.5 instruction per Hz, I argue that the actual performance is half. So I actually think you get ~1187 Dhrystone/MHz.Time for STM32F407. I set up timer TIM2 with prescaler 0 (1:1), en clock div 1. The input frequency of the timer is the APB1 RCC clock, 37.5MHz, where the CPU runs at 150MHz. So each timer value we get, is actually 1:4 resolution.With that setup, the figures for IAR are (FLASH / 150MHz):2044 cycles/Dhrystone @ no optimisations -> 489 Dhrystone/MHz1904 cycles/Dhrystone @ low -> 525 Dhrystone/MHz1228 cycles/Dhrystone @ medium -> 814 Dhrystone/MHz1182 cycles/Dhrystone @ high (size) -> 846 Dhrystone/MHz1086 cycles/Dhrystone @ high (balanced) -> -> 920 Dhrystone/MHz860 cycles/Dhrystone @ high (speed) -> 1162 Dhrystone/MHzI had a plan to run the code from RAM, but if I place add __ramfunc to every function, it actually gets slower. With no optimisations I get 2026 cycles/Dhrystone, but with high speed optimisation it still takes 1396 cycles/DhrystoneSo I tried lowering the clock speed, to 75MHz, and even 37.5MHz, but it makes no difference. So it certainly is not the FLASH wait state.By the way, how did you trick the PIC18 into running Dhrystone? The version I got wants to allocate 5K bytes in 1 array, which is larger than the whole memory of the chip.

author: dannyf
author_rank: Super Contributor
timestamp: March 04, 2014, 07:13:13 pm
content: I came across this document, produced by TI:http://www.ti.com/lit/an/slaa205c/slaa205c.pdfThe chart in the back does suggest that PIC24 has lower cycle counts for math operations and comparable cycle counts with ARM7TDMI (LPC2106 class chips) in thumb mode (which is similar to STM32F chips).So maybe what we observed is indeed within realm of possibilities.

author: smashIt
author_rank: Contributor
timestamp: March 04, 2014, 08:14:28 pm
content: Quote from: hans on March 04, 2014, 06:54:37 pmi think you went the wrong directionif the pic has 2000 dhrystone/MHz and 2 cicles per instruction, it should do 4000 dhrystone / million instrucions / second

author: hans
author_rank: Super Contributor
timestamp: March 04, 2014, 09:32:06 pm
content: Maybe I should have explained that with cycles I meant instructions.As 2 Hz are 1 instruction. So 2MHz yields 1MIPS.I understand how you could assume a cycle is 1 Hz.

author: westfw
author_rank: Super Contributor
timestamp: March 05, 2014, 08:11:34 am
content: QuoteWhere did you get that idea? The pic24 manuals say "up to 40 MIPS" (and 40MHz clock) and:Quote(Of course, the 8-bit PICs also say something like that, and elsewhere define a "cycle" as "four oscillator clocks", but I believe that the PIC24 is really 1 instruction per clock...)

author: hans
author_rank: Super Contributor
timestamp: March 05, 2014, 09:12:13 am
content: Look up the datasheet of a PIC24FJ64GB004. Even the summary page says;"Up to 16 MIPS Operation @ 32 MHz"PIC24 and dsPIC practically all use the same core, but with different speeds, DSP instructions, and the E/H series have some instructions removed & added.For example: a PIC24EP128GP202 can run at 60/70 MIPS (depending on temperature range), and the PLL has a maximum output of 120 or 140MHz.A DSPIC33FJ128GP804 can clock up to 80MHz, which yields 40MIPS.That's why I focused on the no. of instructions PIC24 uses to complete a Dhrystone, and calculate from there.

author: JTR
author_rank: Regular Contributor
timestamp: March 05, 2014, 10:28:59 am
content: Quote from: westfw on March 05, 2014, 08:11:34 amThe PIC24 is a minimum of two clocks per instruction, gospel...

author: nuhamind2
author_rank: Regular Contributor
timestamp: March 05, 2014, 10:31:11 am
content: Perhaps the better wording is every instruction take the multiple of 2 clock cycles

author: dannyf
author_rank: Super Contributor
timestamp: March 05, 2014, 11:01:52 am
content: QuoteThe datasheet? It is fairly easy to pick it up there, actually.QuoteYeah. It has been well known for quite some time by now,QuoteYeah.

author: westfw
author_rank: Super Contributor
timestamp: March 05, 2014, 11:28:06 am
content: Huh. I guess the PIC24H parts can double the (up to) 40MHz external clock, yielding an 80MHz internal clock and 40MIPs instruction rate... I hadn't realized that they internal clock rate had gotten so high!

author: legacy
author_rank: Super Contributor
timestamp: March 05, 2014, 12:23:18 pm
content: can i see the C code of the test ? i'd like to test my board/toolchain

author: dannyf
author_rank: Super Contributor
timestamp: March 05, 2014, 01:58:04 pm
content: I did some testing. I put some pin-flipping patterns into the various Procx() called in dhrystone benchmark. Those patterns differ from each other so by observing them on the pins I get to confirm that those routines are called.I did in fact observe those patterns so they are indeed called by the benchmark, both in debug and release modes. That would suggest that the dhrystone numbers for PIC24F are real -> fairly remarkable I think.Two possible shortfalls here:1) maybe the routines are only called with those patterns inserted: instead of inserting those patterns, I did an OR 0x00 on the output port and the time I got is similar to the original score.2) maybe the routines are called by the results are faulty: I did not investigate that.Too bad that PIC24 is really under-marketed by Microchip and under-appreciated by the mass.

author: diyaudio
author_rank: Frequent Contributor
timestamp: March 05, 2014, 02:04:42 pm
content: Quote from: dannyf on March 05, 2014, 01:58:04 pmI just got on-board with the PIC24 a month ago (after I received my samples) as I ran out of I/O using a 18F series, thus far its very interesting, true that its under marketed, few open source projects or material on the net.

author: jaxbird
author_rank: Frequent Contributor
timestamp: March 06, 2014, 02:24:58 pm
content: No doubt, that Microchip designed a killer 16 bit series with the pic24/dsPic33 series. It's been my favorite in this class for the last couple of years. For me it started with a $25 microstick dev board including a couple of dip package mcus to try it out.Just want to make sure we agree that we are calculating results per MHz and not per MIPS. The results posted by Hans and my results are very similar 1100+ from Hans with pic24f and XC16. Mine at 1190 using pic24h and XC16.Dannyf: I know you used C30, did you have a chance to run your tests with XC16 for comparison?Just for fun I had a go at overclocking. The pic24h I tested will run at 120MHz+, not bad at all. But of course pointless as it's not guaranteed to run stable within the temperature specs at that clock speed. But you could probably run it at 100MHz without any problems as long as it's not exposed to extreme temperatures.

author: dannyf
author_rank: Super Contributor
timestamp: March 06, 2014, 06:52:51 pm
content: I did. The numbers are posted earlier and updated in the first post. The XC16 produced comparable but marginally slower performance vs. C30.

author: JTR
author_rank: Regular Contributor
timestamp: March 06, 2014, 10:18:21 pm
content: Anyway, for all that has been said and argued, the simple fact is that there is no difference between the PIC24F and the PIC24H in terms of instructions per MHz. Both of them scale linearly and in lock step to each other. Ergo, there cannot be a difference in Dhrystones per MHz. The fact that the PIC24F is listed here as having pretty much double the performance of the PIC24H (with same compiler and settings) itself has to be a red flag that something is wrong with the calculations.

author: dannyf
author_rank: Super Contributor
timestamp: March 07, 2014, 12:48:32 am
content: Quote"double the performance" = 2x.I wonder what might have caused that,

author: dannyf
author_rank: Super Contributor
timestamp: March 07, 2014, 12:49:37 am
content: Added numbers for LM4F120 (CM4) and LPC1343 (CM3).

author: BravoV
author_rank: Super Contributor
timestamp: March 07, 2014, 04:37:45 am
content: Quote from: dannyf on March 01, 2014, 06:40:22 pmQuote from: dannyf on March 07, 2014, 12:49:37 amCool numbers, never did a through comparison on my own, but this at least confirms my limited observation on TI CM4, it "feels" so fast even on my bloated noob code, btw I migrated from AVR as my sole mcu in the past, thanks.The worst part is, I become too spoiled, lazy and totally screws my effort to learn code optimization.

author: dannyf
author_rank: Super Contributor
timestamp: March 07, 2014, 12:24:20 pm
content: QuoteThat's what I suspected. I used a 8Mhz crystal in my test and used 4Mhz for the calculation, as the cpu is actually running at 4Mhz. I understand if you used 8Mhz - both approaches have rationale. Thus I kept the numbers the way they are, knowing that people may think one is more valid than another.

author: dannyf
author_rank: Super Contributor
timestamp: March 08, 2014, 01:59:11 pm
content: STM32F4 added for gcc-arm (running at 16Mhz): inline with the F3's numbers.

author: dannyf
author_rank: Super Contributor
timestamp: March 08, 2014, 02:09:23 pm
content: STM32F4 under iar-arm added. I think this is the first time in this test where gcc-arm is faster than iar-arm, with optimization turned on. The validity of the test with optimization turned on, however, is dubious without further investigation.

author: dannyf
author_rank: Super Contributor
timestamp: March 08, 2014, 03:22:32 pm
content: Also added are the STM32F4 numbers under mdk.IAR and Keil seem to be running neck to neck. GCC appears to be quick a bit slower than either IAR or Keil.

author: nctnico
author_rank: Super Contributor
timestamp: March 08, 2014, 08:49:55 pm
content: Replace the C library functions with internal ones and test again to make sure it's not the C library making the difference between IAR and GCC.

author: dannyf
author_rank: Super Contributor
timestamp: March 09, 2014, 12:46:30 am
content: Added LPC1114 (=CM0 running at 12Mhz). The unoptimized Dhrystone/Mhz number is actually quite comparable to STM8S' (=good old 6502).The old clunker isn't that bad, after all.Or to put it another way, the OEMs are reasonable honest when they say that the CM0/1 chips are meant to compete with the 8-bitters.

author: jaxbird
author_rank: Frequent Contributor
timestamp: March 09, 2014, 11:51:38 am
content: Quote from: dannyf on March 07, 2014, 12:24:20 pmIn my tests, using the internal oscillator, the configuration was like this:Fosc = ((7.37MHz * M) / N1) / N2Where M, N1 and N2 are PLLFBD, PLLPOST and PLLPRE.And the actual values used:M = 65N1 = 2N2 = 3Giving Fosc = 79.841 MHz. (+/- 2%)The datasheet/reference does list most of the instructions as executing in a single cycle, but I do find that a bit questionable as it's clearly not oscillator clock cycles they are referring to.

author: nctnico
author_rank: Super Contributor
timestamp: March 09, 2014, 12:49:36 pm
content: Quote from: dannyf on March 09, 2014, 12:46:30 amWhat you wrote above makes all my alarm bells ring. I'm very much doubting comparing unoptimised results has any real value. Unoptimised code is mostly used for debugging purposes where each line of code is represented by some assembly language. The aim is not even to make production grade code as no-one in their right mind would use that in a product. A real test would be to optimise for size and for speed.

author: dannyf
author_rank: Super Contributor
timestamp: March 09, 2014, 01:10:46 pm
content: .-The datasheet/reference does list most of the instructions as executing in a single cycle, but I do find that a bit questionable as it's clearly not oscillator clock cycles they are referring to.-you are using a 24h part, right? Take a liook at the clock tree. Fcy is at least 1/2 of fosc, assuming doze is not set.on 24f parts, you have to go through the datasheet to find that out.

author: Kjelt
author_rank: Super Contributor
timestamp: March 09, 2014, 10:48:17 pm
content: Quote from: dannyf on March 09, 2014, 12:46:30 amYou keep on saying that, I didn't know that, is there some info on that?I know the STM8 also has only the X and Y register (unfortunately, if they had added some extra registers that would have been nice).But is that the only similarity?

author: jaxbird
author_rank: Frequent Contributor
timestamp: March 23, 2014, 01:24:32 pm
content: Quote from: dannyf on March 09, 2014, 01:10:46 pmYeah, it's defined as Fcy = Fosc / 2. So 2 clocks required minimum per instruction.Anyway, not important, my motivation was primarily to find the main reason for the large differences in measured performance. I believe we agree this is where our calculations differ, so I'm satisfied

author: GiskardReventlov
author_rank: Frequent Contributor
timestamp: March 25, 2014, 04:24:27 pm
content: Quote from: dannyf on March 01, 2014, 06:40:22 pmHold its own what? I would say that this all falls under the category of "Premature optimization", but maybe not.Can you provide a few cases where this kind of performance is the keystone in a design?I'm curious to know where this metric would be the top design decision.I've been learning more about uC and I see that some uC have 8051 cores in them, don't know if any you tested do.Do you know if any do?

author: dannyf
author_rank: Super Contributor
timestamp: March 25, 2014, 10:58:57 pm
content: Added (simulated) scores for C51 (a nxp P87C51MC2, in order to hold the data). Scores are obtained in simulation under Keil C51, on 24Mhz crystal, and calculated off a 2Mhz core frequency (the chip I think is a 12-cycle C51).

author: dannyf
author_rank: Super Contributor
timestamp: March 26, 2014, 12:44:22 pm
content: STM8s are advertised by ST as a 0.25DMIPS/Mhz chip. That translates into about 430 dhrystones/Mhz, consistent with our measurements here.Unfortunately, for the CMx chips, we are getting about 50 - 75% of the numbers published by ARM / vendors.

author: dannyf
author_rank: Super Contributor
timestamp: March 26, 2014, 02:42:38 pm
content: The DMIPS/Mhz numbers for 8051 varies a lot, from the lows of <0.1DMIPS/Mhz to the highest of 0.5DMIPS/Mhz. 0.25DMIPS/Mhz (about 400+ dhrystones per Mhz) being quite often quoted.Fairly remarkable in that a chip from the 1980s is as fast as a chip introduced in the last 10 years (STM8).Not sure what 6500 has in terms of dhrystones scores.

author: westfw
author_rank: Super Contributor
timestamp: March 26, 2014, 05:13:09 pm
content: QuoteYou're still measuring DMIPS/MHz, right? That's not "speed", that's just "architectural efficiency at running C code" or something like that. The RISC claim is not so much that their architectures are fundamentally faster, just that they permit building SIMPLER chips, which in turn allows the clock rate to be pushed up and give you an overall faster chip.

author: hans
author_rank: Super Contributor
timestamp: March 26, 2014, 05:37:57 pm
content: That's true. The Pentium 3 at 1GHz was faster than a Pentium 4 at 1GHz, but the Pentium 4 could clock far higher with it's new pipeline design. The end of their range it maxed out just under 4GHz or so, and we haven't seen much higher ever since (for example, my i5 3570K steps up to 3.9GHz 1-core load). The only thing that keeps pushing for more performance has been multi-threading and more efficient CPU's, with larger/better caches, more instructions to play with (if programs are enabled for them), etc.An interesting dimension to add is power consumption per MHz. From that you could then calculate a performance/energy, as you have both Dhrystones/MHz, and mA/MHz, which divided on each other would give Dhrystone/mA ratio, or simply put "computing efficiency". That would be interesting for low power electronics like battery powered stuff which main driver is the MCU doing stuff on an regular basis.I don't know if it's acceptable to take these figures from the datasheet.. it can depend a lot of what peripherals are turned on (ARM) or supply voltage.I think I have a board lying around with a PIC32 on it. I will see if I can run the test on that too, see how well MIPS4k compares. They claim 1.65DMIPS/MHz on that.

author: westfw
author_rank: Super Contributor
timestamp: March 26, 2014, 05:42:16 pm
content: (In this case, we're saying STM8 is as fast as CM0 (in DMIPS/MHz), but STM8 tops out at 16MHz, while CM0 in the same price range run 48-72MHz...) (I count that as about 4x the DMIPS/Dollar...)

author: dannyf
author_rank: Super Contributor
timestamp: March 26, 2014, 09:08:25 pm
content: Added the simulated results for PIC32MX320F128H, under an old C32 compiler.The unoptimized figure translates to 0.75 DMIPS/Mhz, and 2.0 DMIPS/Mhz optimized - not that believable, however.

author: dannyf
author_rank: Super Contributor
timestamp: March 26, 2014, 09:09:24 pm
content: QuoteWould love to see where the real thing comes out to be.

author: Kjelt
author_rank: Super Contributor
timestamp: March 27, 2014, 08:04:14 am
content: Quote from: westfw on March 26, 2014, 05:42:16 pmsmall correction: STM8 tops at 24MHz but then needs an external oscillator.The CM0 will give a great increase in speed BUT with cost of codesize, the codesize of a CM0 is 30-35% larger then STM8 codesize and that also adds up to the final cost.

author: dannyf
author_rank: Super Contributor
timestamp: March 27, 2014, 03:43:49 pm
content: The dhrystone benchmark for 6502 (actually 65C02), that I can find, suggests a 0.022 DMIPS/Mhz (not sure if it is scaled by 2 or not). That would translate into a dhrystone score of 30 / Mhz. Slower than a PIC,

author: GiskardReventlov
author_rank: Frequent Contributor
timestamp: March 27, 2014, 05:37:28 pm
content: Was curious when I discovered that 8051 (and 80C51?) cores are in a lot of uC. A quick digikey search shows 600-700 (plus have to subtract tape&reel, etc.). So let's say 500 (but still less if you subtract pkg types), but still a lot. Or is that the only way to get an 8051? i.e. they only come as a core?What does the C in 80C51 designate? (or 65C02)

author: miguelvp
author_rank: Super Contributor
timestamp: March 27, 2014, 06:07:18 pm
content: Quote from: GiskardReventlov on March 27, 2014, 05:37:28 pmCMOS

author: hans
author_rank: Super Contributor
timestamp: March 29, 2014, 05:12:33 pm
content: Okay, I've got a PIC32 target board (PIC32MX440F512H @ 80MHz) and ran the Dhyrstone benchmark.SetupCycles @ 80MHzDhrystone @ 80MHzCode Size @ 80MHzCycles @ 20MHzDhrystone @ 20MHzCode Size @ 20MHzNo optimizations131276217436996100417436GCC optimize level 1631158415260455219715256GCC optimize level 2481207915156345289815152GCC optimize level 3445224715216339294915212GCC optimize level 3 + unroll loops437228815308326306715300GCC optimize level s(ize)680147015400483207015396See text / speed423236415308326306715300See text / size140771010552123281110548After the standard GCC 0/1/2/s , I tried the following:"Remove unused sections" -> Code size dropped to 14880 bytes, execution time increased to 445 (?). Slower, so I turned it off."Optimization level stdlib" set to level 3 -> 436 cycles / Dhrystone! (-1 cycle) Code is 16176 bytes (+868byte)."Isolate each function in it's own section" -> 423 cycles / Dhrystone (-14 cycles /2364Dhrystone/MIPS). Code size was 15308 bytes (+0 bytes)."Use legacy stdlib" -> Back to 439 cycles, code 33400 bytes. Nope, not attractive feature at all"Generate 16-bit code" -> 803 cycles / dhrystone, code dropped to 15144 bytes. Quite a performance hit.As personal interest, the smallest code I could get was 10552 bytes, with GCC 16-bit code, GCC optimize s, no unroll loop, remove unused functions, allow section overlap, optimize s for stdlib, 16-bit in linker (for stdlib?). Execution took 1407 cycles, though.Now, I first did the test only @ 80MHz, wrote the above and thought "something doesnt seem right". As the 1.65DMIPS/MHz claim should give 2900Dhrystone/MIPS. I quickly thought about the FLASH accelerator, and I suspect it's not as agood as on most modern ARM chips or the new PIC32MZ. So I reran the test at 20MHz (which seems to run at full speed, e.g. lowering further doesn't yield faster results) and got these. They seem about right, or better than claimed.Also glad to see the new PIC32MZ has a FLASH accelerator that should do the business @ full speed

author: dannyf
author_rank: Super Contributor
timestamp: March 29, 2014, 06:33:53 pm
content: Thanks. Your numbers are fairly consistent with mine (simulated). The poorer efficiency at higher frequency makes sense to me - maybe due to flash wait state.The optimized numbers, not just for pic32 but other mcus as well, are not that believable as you have to investigate each routine to make sure that they are not being optimized away - one way to do that is actually to write something to the port in those routines so they don't get cut by the compiler - and you can watch the port to make sure that they are actually run. Too much work.

author: hans
author_rank: Super Contributor
timestamp: March 29, 2014, 06:56:24 pm
content: Yes I suspect it's the FLASH accelerator, as the PIC32MZ datasheet goes into the prefetch module, which is only 16 bytes deep. So I am pretty certain any call will result in a performance drop.Unless the compiler inlines all the test functions, of course. In that case maybe a "dummy" function is possible so it tricks the compiler into thinking those functions are used more often, and "can't" inline them anymore. However, as the compiler will try to do the same on my "real" programs, I take it as a featureAlso, too much effort, and would need to do the same for every other chip as they also use GCC.Sorry I forgot to add the compiler; it's XC32 v1.21

author: dannyf
author_rank: Super Contributor
timestamp: March 29, 2014, 07:14:53 pm
content: Updated the list for your compiler.I compared XC32 vs. C32 in simulation and the numbers I got are almost identical so I think they are (essentially?) the same, just different branding.

author: dannyf
author_rank: Super Contributor
timestamp: March 30, 2014, 08:53:50 pm
content: Added MSP430 scores. Fairly respectable but considerably lower than the PIC24 did.

author: Kjelt
author_rank: Super Contributor
timestamp: March 31, 2014, 12:01:35 pm
content: Danny what was the difference in compiler settings between these two measurements?Quote

author: dannyf
author_rank: Super Contributor
timestamp: March 31, 2014, 10:57:31 pm
content: Kjelt, the 1053 score came from me, IAR-ARM, no optimization.The 494 score came from one of the participants and I can look back and see exactly what it is.

author: dannyf
author_rank: Super Contributor
timestamp: March 31, 2014, 11:01:06 pm
content: hans got a score of 489 for STM32F4 using IAR, no optimization. I may have mis-transcribed it.Corrected now.

author: Kjelt
author_rank: Super Contributor
timestamp: April 01, 2014, 08:11:07 am
content: But a difference of a factor two with the same compiler and compiler settings does not compute?

author: dannyf
author_rank: Super Contributor
timestamp: April 01, 2014, 10:52:06 am
content: Sure.But I am not in the business of validating someone's results. They are what they are, as reported here.

author: dannyf
author_rank: Super Contributor
timestamp: May 26, 2014, 09:43:58 pm
content: Updated simulated benchmark for PIC32MZ (not a single chip available) and PIC24F (XC16 and C30).First, PIC32MZ vs. PIC32MX:QuoteThe results look to be roughly comparable: non-optimized at around 1200 and optimized around 3400.The compiler for PIC32MZ is XC32.

author: dannyf
author_rank: Super Contributor
timestamp: May 26, 2014, 09:46:48 pm
content: 2ndly, PIC24F actual vs. simulation:QuoteVery consistent results between the measured benchmarks vs. simulated benchmarks.Not unexpected.

author: dannyf
author_rank: Super Contributor
timestamp: May 26, 2014, 09:52:27 pm
content: Now, XC16 vs. C30:QuoteSlight edge to C30, about 1% higher scores.QuoteC30 still leads, with a 20% margin.The old dog does have some tricks,

author: dannyf
author_rank: Super Contributor
timestamp: August 14, 2014, 11:31:32 am
content: I was quite impressed with the dhrystone performance of PIC24 - they are surprisingly fast in the benchmarks we have done earlier.It turns out that TI has also done a series benchmarking of 16-bit / 8-bit mcus, as published in SLAA205.Here is a screen shot of one of the tables in the appnote. Look at the cycle counts for pic24/dspic.Wow!For a job well done, Microchip. Wish they had done more to push that chip.

author: Kjelt
author_rank: Super Contributor
timestamp: August 14, 2014, 12:03:41 pm
content: Here they use another benchmark: coremark, might be interesting for comparison also.http://www.eembc.org/coremark/

author: bwat
author_rank: Frequent Contributor
timestamp: August 14, 2014, 12:29:25 pm
content: Quote from: Kjelt on August 14, 2014, 12:03:41 pmCoremark was mentioned on the very first page.

author: amyk
author_rank: Super Contributor
timestamp: August 14, 2014, 12:30:39 pm
content: There's a list of DMIPS/MHz here that you may find helpful for comparison:http://en.wikipedia.org/wiki/Instructions_per_second#Timeline_of_instructions_per_second

author: dannyf
author_rank: Super Contributor
timestamp: August 14, 2014, 05:32:51 pm
content: I have a workstation with dual quad-core Xeon (3.0Ghz+), equivalent to 250,000 dmips.Pretty impressive vs. any Cortex M3 machines,.

author: dannyf
author_rank: Super Contributor
timestamp: September 11, 2014, 11:26:59 am
content: Added STM32F030F - from the ghetto thread.Not terribly impressive, more on the lower-end of the CMx chips and not that much faster than typical 8-bit chips. Its advantage I think is in its ability to run really fast - I have clocked the little guy at over 64Mhz (flash wait = 1) or 52Mhz (flash wait = 0). So its raw MIPS numbers are still impressive, in spite of its mediocre MIPS/Mhz numbers.Among the 8-bit / 16-bit chips, PIC24F remains the champ in terms of the MIPS/Mhz race. It just has limited frequency range.

author: dannyf
author_rank: Super Contributor
timestamp: September 24, 2014, 12:30:31 am
content: Dug up an old STM8 training presentation.Notice the 0.29dmips figure? That translates to about 500 dhrystones / Mhz.Very close to our test figures:Code:STM8S: 482, IAR-STM8, optimizedSTM8S: 434, IAR-STM8,PIC18F26K20: 380, XC8 proPIC18F26K20: 323, XC8 freePIC18F26K20: 322, PICC18 proAVR90USB1286: 237, gcc-avrPIC18F26K20: 168, PICC18 liteThe presentation is also right that that kind of speed is 1.5 - 3x of some PICs and AVR.

author: BravoV
author_rank: Super Contributor
timestamp: September 24, 2014, 02:46:14 am
content: Any plan for TI's TM4C123x or TM4C129x series ?

author: coppice
author_rank: Super Contributor
timestamp: September 24, 2014, 06:37:56 am
content: Quote from: GiskardReventlov on March 27, 2014, 05:37:28 pmThe 8051 is a core anyone can use without paying royalties, and multiple reasonably good toolchains are available to support it. That means its a no brainer for many people who need to drop a simple low performance core into a chip to drop in an 8051 core. The original 8051 takes 12 clock cycles for one machine cycle. There are versions today which run at one clock cycle per machine cycle, so it can be reasonably fast. You will find an 8051 at the heart of a lot of devices you didn't even realise were processor based, as the user just sees them as a black box.Every single engineer graduating in China has studied the 8051 in detail. This is a huge incentive for people to put it in their chips, as a huge number of the engineers developing with 8 bit MCUs today are in China. If you try to sell them a chip with an 8051 core you can talk about the things which make the chip interesting. If you have any other core the conversation ends up mostly about tools, and how much hassle it will be to get up to speed with the core. Even very popular cores, like the PICs, have this disadvantage in China. Many people will recognise a parallel with ARM cores in bigger MCUs on a global level. If a 32 bit MCU doesn't have an ARM core it will be a big struggle to sell it to a lot of people.

author: Kjelt
author_rank: Super Contributor
timestamp: September 24, 2014, 06:44:28 am
content: 8051 has been put in its grave for a long long time, its now a zombie created by some companies lacking innovation effort.Unfortunately zombies refuse to stay in their grave and act dead.

author: coppice
author_rank: Super Contributor
timestamp: September 24, 2014, 08:16:26 am
content: Quote from: Kjelt on September 24, 2014, 06:44:28 amThat is the exact opposite of reality. People being really innovative seldom give a damn about the core. If that's where your innovation lies its pretty weak. It would be great if a better free to use core with good tools were available, but the 8051 is good enough for a lot of devices.

author: Kjelt
author_rank: Super Contributor
timestamp: September 24, 2014, 03:35:23 pm
content: Quote from: coppice on September 24, 2014, 08:16:26 amWhich reality is that then? If you look around almost all innovations start with the availability of new technology, being or faster having more processing power to establish and realizing new technologies or having less powerusage to be run on different (mobile) platforms.

author: mikerj
author_rank: Super Contributor
timestamp: September 24, 2014, 05:01:20 pm
content: Quote from: Kjelt on September 24, 2014, 03:35:23 pmNot so. Simply using a faster microcontroller does not make a design innovative, it's what you do with that microcontroller that is important. Innovative new designs using old (and extremely cheap) 8 bit micros like the 8051 are regularly developed.

author: coppice
author_rank: Super Contributor
timestamp: September 24, 2014, 05:52:31 pm
content: Quote from: Kjelt on September 24, 2014, 03:35:23 pmAlmost every part of an MCU except the CPU is an interesting place for innovation. That's why the ARM is becoming so dominant. It just doesn't matter, so people use the ARM as a default. Its something people are familiar and happy with, and it has good tool support. That's all that matters about the core for the vast majority of MCU applications. You might be surprised how few MCUs are actually running at their rated speed. People typically use a fraction of the potential core speed, to save a little current. The main reason for the growth of 32 bit MCUs has less to do with speed, than the ability to work smoothly with big memories.Most high volume users buy MCUs for their interesting peripherals. Mixed signal peripherals are a particularly strong area for competition. Special memory features , such as fancy weak cell checking, or EDMI error corrrection, are often critical features for flash. These are the areas where you compete in the MCU market.There are clear high volume niche exceptions to the above - DSP and control loops often push the core pretty hard. However, these are not the bulk of the market.

author: Kjelt
author_rank: Super Contributor
timestamp: September 24, 2014, 06:20:40 pm
content: I know that but if you look at the last two technology waves naming the cellphone and the iPad you can hardly say that those applications could be done 10 years before they were done, because they could not at least with decent battery life time and computing power to do something that the user "must" have. The new tech waves arise when the application and the technology are both just mature enough to make it work. And yes you can do innovative things with 30 year old microcontrollers but you can also do them then (and even better) with the current generation of uC's. At least that is my opinion. And if you talk about a 8051 you probably talk about the modern 8051 based uC's with modern peripherals and state of the art silicon but just the old school core and not about the 8051 with the parallel data and adress bus and external memory to be able to do anything.

author: dannyf
author_rank: Super Contributor
timestamp: September 25, 2014, 11:09:47 am
content: The sources of advancement in computer power have been more limited to raw speed at which the cpu runs vs. architecture advances.Take the chart here for example:http://www.netlib.org/performance/html/dhrystone.data.col0.htmlOn a DMIPS/Mhz basis, the early chips (towards the bottom), like 6502/8086/186/286 are more on par of 0.1DMIPS/Mhz. The newer chips, like Pentium, etc. are more 0.5 - 1 or 1.5 DMIPS/Mhz, over the course of 20-30 years. Or 5 - 15x.In the mean time, the speed at which they run had gone from 4Mhz to 200Mhz, or 50 times, and even more if you benchmark against today's Xeon / Core chips.So there is not a whole lot to be gained going from one architecture to another - I think our own numbers here show as well. The increase in performance comes more from running the chips faster and faster.Plus, for many applications, faster isn't as needed as other measurements, like current consumption.

author: Kjelt
author_rank: Super Contributor
timestamp: September 25, 2014, 11:25:13 am
content: Quote from: dannyf on September 25, 2014, 11:09:47 amAnd as we know from the big father processors as example the Core ixfrom Intel we see these also go together, each next silicon process reduces the voltage of the core thus decreasing the power usage thus enabling faster frequencies (or more cores) for the same temperature/power footprint.