Cortex-M7 pipeline idiosyncrasies… - Page 1
Forum section: electronics / general

author: ZaneKaminski
author_rank: Contributor
timestamp: November 04, 2022, 09:03:01 pm
content: I was cycle-counting for an audio DSP project intended to run on a 180 MHz ARM Cortex-M4 microcontroller. The code makes extensive use of the ARM’s FPU. After extensively benchmarking the compiler output and then rewriting the crucial inner loop in assembly, I concluded that the 180 MHz STM32F4 just doesn’t have enough oomph to get the job done at the desired sample rate. I figured we can upgrade to one of the faster 216 MHz STM32F7s with the superscalar Cortex-M7. Target ship date is a year out and the product has a lot of cost margin so I figured there’s not a big issue specifying a chip that’s been in short supply as of late.Anyway, I’ve never actually used the Cortex-M7 before so I was not aware of some of the big drawbacks. Firstly (and I expected this), FPU ops can’t be dual-issued. No big deal since loads and FPU ops can be paired. But neither can FPU ops and ALU ops dual-issue. Only ALU+ALU or FPU/ALU+load/store can be issued together. That’s not too bad since scalar DSP stuff often has equal number of load and FPU operations. The real kicker is the combination of the M7’s longer pipeline and lack of FPU bypassing. On the Cortex-M4, there’s a 2-clock latency for FPU operations so you suffer a stall if you use the result of an FPU op in the next instruction. On the M7, the FPU latency is 3 clocks, and of course to get more speed, you want to dual-issue. So you have to schedule two pairs of two instructions in between generating and consuming an FPU result. That’s a lot of latency! The arm-none-eabi-gcc compiler (with mcpu=cortex-m7 of course) is not exceptionally good at reordering instructions to get more speed. Best performance at least in a convolution kernel like I’m working on is still obtained by writing assembly. Of course the FPU latency means you have to sort of skew all the computations forward and then backfill other ops in between. Makes the code hard to read! I am always hopeful that the compiler will do a good job but in branchless unrolled loop kind of code where I know how to make it as fast as possible, I find I can usually do a fair bit better than the compiler. So with hand-optimized assembly, I was able to get a 1.85x speedup per clock on the convolution inner loop.Anyone have any thoughts about this kind of thing? I could imagine that bypassing the FPU would be very costly in terms of transistor budget, but I am surprised to see that porting to the M7 from the M4 requires a recompile or else you may lose a lot of performance per clock. Makes more sense for an embedded microarchitecture than in a PC CPU but still, my perception of the M7 as a “modern” core has been somewhat dashed by the huge performance regression on the same branchless code. We might have to go for one of the faster 480 MHz STM32H7s anyway for parts sourcing reasons, and it's not a strongly power-constrained application, so like, whatever, it's solved. Still the M7 has left me with a slightly bad taste in my mouth. Am I being too harsh since a recompile is kind of a given in embedded applications and you can just get a faster clock speed Cortex-M7 MCU?

author: uer166
author_rank: Super Contributor
timestamp: November 04, 2022, 09:20:12 pm
content: I've never hand-optimzied assembly to this degree before, but it sounds like you're trying to use a general-purpose core "with DSP features" as a true FP DSP, so that might be part of the disconnect. What I've seen done on stuff like TMS320 is use their "control law accelerators" to do this kind of work (https://training.ti.com/control-law-accelerator-cla-hands-workshop), instead of the cores. A recompile between M4 and M7 doesn't sound too obscene, they're different in some important ways for your application.Out of curiosity, were you executing out of CCM RAM? How did you do the cycle counting?

author: dietert1
author_rank: Super Contributor
timestamp: November 04, 2022, 10:47:19 pm
content: Instead of hand-optimizing assembly code one could try and write some smart level/scale supervision so the convolution can be done with integers. As far as i remember the Cortex DSP supports 32 x 32 with 64 accumulation as one instruction. I think that covers the dynamics of audio.Regards, Dieter

author: eutectique
author_rank: Frequent Contributor
timestamp: November 05, 2022, 12:40:12 am
content: Some decades ago I worked at Intel, writingintegrated performance primitivesfor image and signal processing on StrongARM (ARMv4) and XScale (ARMv5) CPUs. At the time, the latter existed in a form of cycle-accurate simulator only. All work was done in hand-written assembly. Data formats included integer 8-, 16-, 32-bit, and fractional s15 and s31 fixed-point.Some years later, at different place, I've done several sound-processing projects with ARM926 processors. Again, heavy use of DSP instructions, fixed-point fractional arithmetic, and hand-optimised assembly.I would dump FPU completely and do the job with integer DSP instructions. MAC (multiply-accumulate), saturating arithmetic, cache and PLD instruction, branch prediction, TCM, assembly -- all that you would need. Forget the C compiler.

author: ZaneKaminski
author_rank: Contributor
timestamp: November 05, 2022, 09:01:43 am
content: Quote from: uer166 on November 04, 2022, 09:20:12 pmAh yeah, you hit the nail on the head. It's not really a DSP.As for the cycle counting, I had two methods. First (in the course of board bringup) I just hooked a scope up to a GPIO pin and ran a sequence of copies of the inner loop from internal SRAM bookended by toggling the GPIO. Then I divided the time between GPIO toggles by the clock period and number of iterations. The result is lent a bit more credibility if it rounds to something divisible by the number of copies of the code concatenated together between toggling the pin. That's easy but the scope is annoying when doing a lot of software stuff so I switched to counting with one of the STM32's timers. Basically the same methodology but you get the total time from the counter, which may run slower than the CPU clock, so you have to apply an additional scaling factor.Re: CCM/TCM RAM, I don't think the STM32F446 has it, in the sense that all internal SRAM is accessed over an AHB crossbar bus. I tried to mitigate possible instruction fetch latency in my benchmarking by putting a bunch of NOPs before the GPIO/timer stuff kicking off the big run of benchmark code.Quote from: dietert1 on November 04, 2022, 10:47:19 pmQuote from: eutectique on November 05, 2022, 12:40:12 amYeah, fixed-point would have offered more options for performance optimization if we pursued it in the first place. Indeed the current software is basically an emulation of an old ASIC with fixed-point processing. Unfortunately we (read: someone else) developed the "meat" of the solution on the PC (using double-precision) and I'm just trying to take that code and make it work on the MCU. Obviously the difference between double/float doesn't matter by the time you send the sample to the 16-bit DAC so the conversion to 32-bit float was a no-brainer. But I don't wanna change from floating to fixed point at this time... I think I have the critical inner loop worked out just about as well as possible using floats, with only a few instructions in the big convolution kernel incurring stalls or not paired for dual-issue. Basically the inner loop is a bunch of repeated y += w * (a + b). Conveniently, there are exactly as many FPU ops as loads, so 32-bit fixed-point wouldn't be faster and we could only gain speed by going to 16-bit fixed point and SIMD, I think, which maybe wouldn't be precise enough. Not all 16 bits are needed since the original chip was less precise than that but I dunno if 16-bit fixed-point will suffice.

author: Siwastaja
author_rank: Super Contributor
timestamp: November 05, 2022, 09:53:25 am
content: I did this once,1) Disable dynamic branch prediction and, I found out, undocumented DISFOLD bit needs to be set, too, for predictable branch timing (write 0x04006004 to ACTRL)2) Just unroll the heck out of your loops to get rid of branching as much as you can3) Use core coupled memory (ITCM) for the routineWith above, I was able to get completely predictable timing. At that point, it did not matter whether I did it by counting cycles (complicated because you need to understand how exactly dual issue works) or just test & measure.Of course, a "true" DSP would make your life easier, but M7 CPUs are cheap and ubiquitous and come with good set of peripherals so if the DSP is just a small part of your application, then having a simple one-MCU-solution instead of MCU + DSP is definitely an advantage, but you pay the price with more complicated development and uncertainty whether you can actually pull it off or not.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 05, 2022, 08:03:50 pm
content: And, of course, as we repeatedly say, you shouldn't count on precise execution timings on those processors anyway. There's a way of achieving what you want achieved in a much better way in 99.9% of the cases using proper coding and proper use of peripherals.

author: eutectique
author_rank: Frequent Contributor
timestamp: November 05, 2022, 09:54:01 pm
content: Quote from: ZaneKaminskiThere is a third way.DWT unithasDWT_CYCCNTregister, which shows the CPU cycle count. Usage pattern isCode:static volatile int32_t cycles = 0;cycles -= DWT->CYCCNT;... // critical code herecycles += DWT->CYCCNT;The names are from STM32 SDK, but you get the idea. A further enhancement would be to makecyclesan array and collect independent counts on several runs.Quote from: ZaneKaminskiYou are on the right track then. Pass test vectors through PC version and compare your MCU results against them.Quote from: ZaneKaminskiYes, that's the way of doing DSP on general-purpose CPU (which Cortex-M is) with a general-purpose compiler -- compile C code to assembly, and then optimise it to death by hand. Add comments with pipeline and data latencies.And good luck!

author: Marco
author_rank: Super Contributor
timestamp: November 05, 2022, 10:51:00 pm
content: Quote from: Siwastaja on November 05, 2022, 09:53:25 amCan't the M7 do zero overhead looping if you do the compare early enough in the body?

author: brucehoult
author_rank: Super Contributor
timestamp: November 06, 2022, 02:29:56 am
content: Quote from: ZaneKaminski on November 04, 2022, 09:03:01 pmSorry .. a 1.85x speedup compared to what? The M4F? So you've got near perfect dual-issue (85% of the time) despite the longer FPU latency? Sounds pretty awesome!Dual-issue speedups are often in the 1.4x to 1.6x range, which is usually reckoned to be an excellent efficiency gain as they only use about 15% more hardware.And that's in addition to the 216/180 = 1.2x faster clock? So you've actually got 2.22x performance increase?M7s are available in 600 MHz and the one on my Teensy 4.0 seems perfectly happy (if a little hot) at 960 MHz, as in perfect performance scaling to that speed.

author: ZaneKaminski
author_rank: Contributor
timestamp: November 06, 2022, 05:49:18 am
content: Quote from: Siwastaja on November 05, 2022, 09:53:25 amOoh thanks for the info. There weren't any branches in the DSP kernel anyway anyway so I didn't worry about that. It's unrolled 192 times over.Quote from: eutectique on November 05, 2022, 09:54:01 pmYeah, I figured there would be something like that but I just used the general-purpose interval timer which I had already figured out how to use rather than go through the docs. I'll use this in the future though, thanks.Quote from: brucehoult on November 06, 2022, 02:29:56 amWell, that's with the hand-optimized code. I was sort of lamenting the fact that the arm GCC compiler did quite a bad job compared to my handwritten assembly. I can't remember the exact figure, but yeah, it was in the 1.4x range. This was particularly disappointing because from a compiler-algorithmic standpoint, what I did in the handwritten assembly to get the 1.85x speedup wasn't too complicated.I started with a naive, unrolled implementation in assembly. The implementation used 29 floating point registers (out of 32). Having a few spare registers is key for the transformation I did. Knowing that the M7's FPU latency is 3 clocks and knowing that I had equal numbers of loads and FPU ops, I cut-and-pasted all the loads from the naive implementation into a separate table which would form the backbone of the fully-optimized implementation. Loads can't dual-issue with other loads so my goal was to pair each load with an FPU or ALU operation. With the loads isolated from the FPU/ALU ops, I reordered the loads to minimize "latency skew" between two loads whose results would then be used by the same instruction. My original assembly code reused some registers, so I had to rename all the registers to temporary virtual names. Then I started adding in FPU ops to the big table of loads, pairing them with loads and renaming the registers in the FPU ops for consistency. The first few loads had to be unpaired since I assumed a 3-clock latency between loading and using the result. I didn't measure load latency so I chose 3 clocks to match the (known) FPU latency. I think the load latency from internal SRAM is actually two clocks. Were I working from external RAM, maybe I'd assume more load latency. Because of the load and FPU latency, I had to space out dependent operations within one iteration of the unrolled loop and interleave them with independent operations from subsequent loop iterations. The FPU and load latency determines the interleave factor and the number of operations per loop iteration determines the span of time over which a single loop iteration's result is calculated. Once I had everything written out, I re-renamed the registers from virtual names back to physical registers. Since the computation of a single iteration's result was interleaved with the computation of the results of other iterations, a few more registers were required than I originally used in the naive assembly implementation.Isn't this basic optimizing compiler stuff? You'd think it would do what I described automatically, since it's algorithmically pretty simple. I tried -funroll-all-loops and it unrolled the loop but it didn't interleave the iterations properly to hide the FPU latency. So yeah, I knew I could get almost 2x if I wrote the code myself, I'm just disappointed in the toolchain. And the 3-clock FPU latency is sort of inconvenient, though not a big deal since you just have to do a wider interleave factor between independent loop iterations to compensate. I would also add that I stuck this big unrolled loop in the middle of some C code as inline assembly. I chose the registers to use and moreover, I specifically put a few instructions at the beginning and end which are not paired for dual-issue. These few unpaired instructions at the beginning and end are responsible for the speedup being only 1.85x instead of 2x. If only the compiler could do the optimization I described, maybe it could find some stuff in the preceding or subsequent C code that could be dual-issued with those unpaired instructions in my inline assembly. Or maybe it could do the register allocation better. My optimized implementation doesn't use register s1 but that's caller-save in the usual calling convention. Maybe using s1 and avoiding the use of one of the callee-save s16-s31 registers would eliminate a push/pop in the surrounding function. Too many things to consider for such a small gain and as you said, there's a 2.22x performance increase so the problem is solved. Just disappointing that the compiler doesn't seem to know much about the latencies in the core and that it doesn't try a slightly tricky optimization like I described.

author: brucehoult
author_rank: Super Contributor
timestamp: November 06, 2022, 06:21:12 am
content: As a compiler engineer I'm not even slightly surprised that you were able, with quite some effort (how long?), to make code 30% faster than the compiler.One of the most important properties is a compiler is to compile code fast. They do only those optimisations that they can do quickly. Most people would be upset if the compiler spent even 10ms on your kernel, let alone 100ms. In fact probably it spends under 1ms. Code gets compiled again and again and again.What you want is some other tool that will happily spend 100ms or 1s or 10s or 10 minutes on your kernel and spit out amazingly optimal assembly language for that one exact pipeline. And you only ever run it once, unless the algorithm changes.

author: DC1MC
author_rank: Super Contributor
timestamp: November 06, 2022, 09:08:18 am
content: Quote from: brucehoult on November 06, 2022, 06:21:12 amSo far I kind of understand what's happening, but this let me scratching my head, where is it coming from, could you please elaborate ? Personally I don't mind and never knew or heard of anyone in the industry that will not trade even a day for a compiled code 30% faster than stock compile options ? Or was it some undetected sarcasm ?Cheers,DC1MC

author: wek
author_rank: Frequent Contributor
timestamp: November 06, 2022, 09:16:47 am
content: The DWT unit in Cortex-Mx has a few "performance" counters, CYCCNT is one of them. In CM7, there's also a FOLDCNT among others. Might be fun to have a look a these, while doing this sort of hand-asm-optimization. I never needed to cycle-count so have no personal experience with these.JW

author: brucehoult
author_rank: Super Contributor
timestamp: November 07, 2022, 02:34:03 am
content: Quote from: DC1MC on November 06, 2022, 09:08:18 amYou wouldn't like it if it took a day instead of 10 seconds to compile your project every time you made a change. You, having read the datasheet, know far more about the intricacies of the M7's pipeline than a compiler such as gcc or llvm knows. They know it's dual-issue and they have some heuristics to not put dependent instructions directly adjacent when possible. But that's about it. A skilled human with a day to spend will beat a compiler on optimising 50 lines of code every day of the week. And that's fine. Do what you're best at. The compiler is best at producing good competent, but not perfect, machine code from thousands of lines of code a second.

author: DC1MC
author_rank: Super Contributor
timestamp: November 07, 2022, 04:20:24 am
content: @bruce Is there no middle road I've always wondered, after half of century of research and implementation of compilers is this what we get, a mediocre and somehow fast translation of language vs well, nothing, really, because I have yet to see the mythical animal that spends a day on 50 lines of code and produce perfect optimal code ?All these -Omfgand -m cpu=cortex-m7 for nothing ? AI is producing museum quality art and context winning poetry, and we can't describe a CPU architecture good enough to produce "hand un-optimizable" code, without a human in the loop counting painfully instruction cycles and trying to remember which little register bit buried in the 5000 pages description (over 20 documents) of the CPU may give some performance increase ? This is really sad.And if this very slow but wise animal may sometimes show up, we don't have to apply it all the time, for every code module, is not like we didn't solve at least this issue with the "beautiful" #pragma invention, what is one more to have. Personally I love to tell Jenkins "Use -O "Absolute" and optimize the hell out of the code base over the week-end". Then the dream goes away and in three hours, back to gcc.

author: brucehoult
author_rank: Super Contributor
timestamp: November 07, 2022, 05:11:34 am
content: https://www.gnu.org/software/superopt/https://web.stanford.edu/class/cs343/resources/superoptimizer.pdfThis paper describes, for example, the quite unexpected discovery that on the 68000 the C function ...Code:int signum (int x) {if(x > 0) return I;else if(x < 0} return -I;else return 0;)... can be optimised to the branch-free code ...Code:(x in d0)add.l d0,d0 ;add d0 to itselfsubx.l d1,d1 ;subtract (d1 + Carry) from d1negx.l d0 ;put (0 - d0 - Carry) into d0addx.l dl,dl ;add (d1 + Carry) to d1(signum(x) in d1)Such trickery with the carry flag!Interestingly, you can write this function in C as ...Code:int signum(int x){return x>>31 | !!x;}... which would compile quite badly on 68000, but on a modern ISA such as RISC-V is very simple and direct (argument and result both in a0):Code:srai a1,a0,31snez a0,a0or a0,a0,a1Anyway, the point is that such tools do exist, both for straight-line code and for optimising loops.

author: ejeffrey
author_rank: Super Contributor
timestamp: November 07, 2022, 05:22:13 am
content: Quote from: DC1MC on November 06, 2022, 09:08:18 amFor this particular application, a 100 ms compile time would obviously be irrelevant, but go compile a big project like the Linux kernel, xorg, chromium, those kinds of projects. They take 10s of minutes on the low end, and many hours on the high end.You can do a debug build with minimal optimizations for a faster build, but ultimately even optimized builds need to be fast to be usable, both for developers and for automated regression testing.QuoteThere is a middle ground, that is why compilers have optimization levels. As for the "mega optimizer" that spends a long time on short sections of code, I wonder how good that would be in practice. Yes, you could write an optimizer that would solve this particular problem easily. But if you had 100 different heuristics and a good scoring system, how often would it actually help? How much would people *actually* use it?QuoteIt does seem like those options should be providing some input to optimize instruction scheduling in exactly this way. If i had to guess, it might be that such a heavily unrolled loop that is using so many of the registers is simply too big for the optimizer. That's a lot of branch free instructions. It's not big for a global optimization algorithm like simulated annealing, but it might be larger than the heuristics in a compiler are designed for since they rarely have so many instructions to optimize without branches. The OP also took advantage of some high level structure such as the 1:1 ratio between loads and FP ops to simplify the problem.Finally, it's probably relevant that this is the sort of problem that out-of-order processors are designed to solve. deeply pipelined, dual issue CPUs are... not exactly rare, but less common than smaller simpler cores or larger out-of-order cores. While several important CPU cores fit that description, I wonder how much compiler writers really optimize for that case. After all, if generating efficient code for that type of system was easy, we would all be using Itanium.

author: brucehoult
author_rank: Super Contributor
timestamp: November 07, 2022, 08:51:54 am
content: Quote from: ejeffrey on November 07, 2022, 05:22:13 amYes ... an exact point that I made in my head but didn't get into a comment.The A53 is probably the most important of these in the current market, and Pentium and PPC601 and PPC603(e) for a brief period in the mid 90s -- 2 1/2 years from Pentium to the OoO Pentium Pro, for example.Even the A55 has just enough pseudo OoO in the pipeline (as do SiFive U74 and Western Digital SweRV in the RISC-V world) that code scheduling into dependency-free pairs becomes a lot less critical than on A53.But the vast vast majority of cores are either single-issue in-order or else fully OoO.

author: ZaneKaminski
author_rank: Contributor
timestamp: November 07, 2022, 09:36:34 am
content: There is a funny disconnect. Over here on EEVBlog we're all in agreement that the compiler is only so-so at writing branchless "non-tricky" code like this DSP unrolled loop stuff even though we can articulate the algorithm to write the perfect code in certain cases. Whenever I bring up writing assembly language to my software developer friends, they are are all like, "assembly? who uses that anymore" "just use the compiler since it generates very efficient code" "the [Intel|AMD|etc] compiler knows much more than you about the target microarchitecture so it will schedule the instructions best". I guess GUI software, CRUD apps, etc. are branch-heavy code with lots of pointers, variables, etc. The compiler is (in my experience) actually pretty good at allocating the registers and organizing the branching in that kind of code. And of course it changes so much, it would be inefficient not to use the compiler.Quote from: brucehoult on November 07, 2022, 02:34:03 amAnother not-so-nice aspect of the Cortex-M7--none of the instruction latencies are documented. There is a section on the instruction timings in the Cortex-M4 Technical Reference Manual. In the M7's TRM, the analogous section is missing! I guess that's part of why I was offended by the 3-clock FPU latency. I wish they told me! Way before my time, but I think Intel was a lot more specific in the Pentium's datasheet. I've got nothing official on what instructions can be issued together, latencies, etc. beyond the basic pipeline block diagram.Quote from: ejeffrey on November 07, 2022, 05:22:13 amJust too bad there aren't better flags or options I can provide the compiler to tell it what to do here. I recall watching a talk by Chris Lattner about LLVM. One of the points that stuck with me was that after the code was translated to LLVM IR, you could sort of generically apply a number of optimizations to it. (Was my generated code bad because I'm using GCC? haha) Itanium failed because you can't actually solve the instruction scheduling problem statically, especially because over subsequent CPU generations, memory latency per instruction throughput increases. And what about speculating past multiple branches that you just can't predict with great accuracy? Here the latency is all very static so I wish there was a way I could write my loop in straightforward C and then specify how the unrolling and interleaving should be done. Explicit specification of the transformations would avoid the compiler needing to use heuristics or know about the target microarchitecture.

author: wek
author_rank: Frequent Contributor
timestamp: November 07, 2022, 11:28:00 am
content: Quote from: ZaneKaminski on November 07, 2022, 09:36:34 amQuote from: Donald Knuth, some 50 years ago

author: abyrvalg
author_rank: Frequent Contributor
timestamp: November 08, 2022, 03:22:40 pm
content: Quote from: brucehoult on November 07, 2022, 05:11:34 amModern x86/amd64 compilers (MSVC I guess) produce such tricks automatically. Recent Windows executables are full of hardly readable constructs likeADD A, B, C; SBB D, E, F; SETNZ ALreplacing things likebool x = ((y>42) && (y<63)).ARM GCC's output feels more architecture-/platform-agnostic, even compared with commercial ARM compilers (i.e. no MOVT usage, no data section compression - the latter is really annoying, a nonconst 1k struct with a single initialized nonzero byte would occupy full 1k in flash to be copied to RAM).

author: jnk0le
author_rank: Regular Contributor
timestamp: November 13, 2022, 12:05:24 pm
content: Hi,here are my notes from cm7 analysis:https://github.com/jnk0le/random/tree/master/pipeline%20cycle%20test#cortex-m7Not the most up to date, and didn't touch FP so far.My advice is to try llvm, they had better pipeline model than gcc, the last time I checked. (it's coremark went from 5.01 up to 5.29 on ARM website for a reason)

author: jnk0le
author_rank: Regular Contributor
timestamp: June 28, 2024, 12:17:00 am
content: Just touched the M7 "vloating" point, and the best approach seems to be what was discovered on arm forum long ago:- mass load- mass FMA (dual issue with integer code if possible)- mass storeFMA has to be interleaved by at least 3 independent accumulations.FMA cannot be dual issued with vldr/vstr even though it is said to have 4 read and 2 write ports.Load to use latency is 1 cycle except when consumed as FMA accumulators, where it's 3 cyclesAlso, if you want addition after executing FMA (within 3 next cycles) then FMA against 1.0f instead of simply adding.EDIT: another approach could be to interleave FMAs with 2 or 4 loads/stores to get rid of accumulator interleavingQuote from: brucehoult on November 06, 2022, 02:29:56 amM4 FMA is 3 cycle or 2 cycle for scheduled MUL+ADD. So in theory, dual issuer (with 1 FMA pipe) should be 4-6x faster in FMA dominated code.

author: bson
author_rank: Unknown
timestamp: June 28, 2024, 09:26:52 pm
content: Quote from: ZaneKaminski on November 07, 2022, 09:36:34 amCompilers are great for larger bodies of code, which frankly is pretty much anything these days. Just changing a constant from 1 to 0 can make it completely overhaul register usages, stack ordering, inlining, removing unreachable code, change unrolls, etc. Doing this on scale in assembly means significant rewrites, which means it isn't done so the code ends up showing its history. Large assembly code bodies also rely heavily on conventions, which then means due to the lack of inlining (lots of small functions usually, with inlining being in the form of macros) there is tons of "glue" code to rearrange register contents between function calls and even macro invocations.And while compilers generate pretty good code mostly, there's no doubt sometimes assembly is a win. But it's also nice to have a more naive reference implementation in C/C++ to compare test results against, to verify that the highly optimized assembly implementation works correctly and doesn't regress if touched. This also makes porting easier, in that initial bringup work can focus on functional correctness and punt on optimization. (It's hard to optimize something that isn't running to begin with, and even counterproductive often as performance optimization should ideally be data-driven. I.e., the numbers determine what's a good optimization and what's counterproductive.) In most cases also, naive code is just fine while being more maintainable and understandable. I'm a big fan of writing code that's simple and straightforward, and which does what it needs to and nothing else. Compilers usually like code like this as well and generate better code than for do-everything frameworks and engines with a gazillion configurable constants, buttons, and levers to pull.But yeah, optimized assembly code, limited to the functions that actually benefit from optimization is definitely a thing. Not sure how anyone could think otherwise.

author: brucehoult
author_rank: Super Contributor
timestamp: June 28, 2024, 11:05:06 pm
content: Agree with all.

author: nimish
author_rank: Regular Contributor
timestamp: June 29, 2024, 03:47:08 am
content: Reducing data dependencies is critical to getting any perf out of superscalar CPUs. It's not like they can magically break those requirements, even OOO are stuck when confronting raw hazardsRewrite your code to better break up dependent calculations, e.g. use Estrin's scheme and not Horner's. You can get near-theoretical speedup doing that + forcing the use of fused instructions when the compiler can't or won't infer them. Worth checking via sollya/megalibm and the like too.If you have an armv8.1-m processor there's some low overhead looping instructions that help too.Anywayhttps://www.quinapalus.com/cm7cycles.htmlwas very useful to plan out execution.

author: jnk0le
author_rank: Regular Contributor
timestamp: June 29, 2024, 01:58:00 pm
content: Quote from: jnk0le on June 28, 2024, 12:17:00 amNot the case, can't interleave FMAs with dual loads/stores (2x vldr.f or vldr.d)anything but vstr.f which can't dual issue with itself nor FMA's. mass load, mass compute mass store is the only way.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 01, 2024, 09:34:21 pm
content: For CM7, I've found using the CMSIS DSP libs gets me pretty close(or better) to hand optmized assembler, and by looking at the instruction ordering in the CMSIS DSP libs, you can get some ideas about the ordering tricks. On some general purpose DSP (speech codec), I found approx 1.4x improvement for CM7 over CM4, clock for clock, *and using ITCM DTCM on the M7) and almost 2x using CMSIS DSP libs versus original CM4 optimized C code.

author: tggzzz
author_rank: Super Contributor
timestamp: July 01, 2024, 09:50:14 pm
content: The solution to making i/o have predictable timing is to dotwothings:have hardware dictate the i/o timing, not the time taken for the code to execute. That's easy, standard practice, and competent peripherals support it.ensurethe code executes within the timings specified by the hardware. That is easy - provided the processor is guaranteed to be grossly underutilised. The intractable problems arise with hardware features that minimise theaverageexecution time. (In a hard realtime system the average time/performance is completely and utterly irrelevant!)There is one processor designed for embedded hard realtime systems. Its software tool chaindefinesthe minimum/maximum code execution time by examining the compiled object codewithout execution. There's none of this "measure and hope we blundered across the worst case" nonsense.So, go to DigiKey, and buy an XMOS MCU (up to 4000MIPS/chip and expandable), and program it in xC using the xTimeComposer IDE.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 02, 2024, 11:06:08 am
content: tggzzzI like how you wrote "That is easy - provided the processor is guaranteed to be grossly underutilised."AMENso true, there's a project I worked on --- the customer was trying to use a Nvidia Jetson to do a load of video processing.The chip/board data sheet claims it can do "mountains of x", "tons of y " , "sh1tloads of z". It reads very impressive -But not all simultaneously....The busses all just saturate, the CPU starts to stall and wait , irqs and buffers dont get emptied, and then arbitration and throughput goes to hell .That design is now on an FPGA...Someone posted here somewhere talking about heavy DMA chewing up the bus , that might kill the CM4/CM7/dspic perf....

author: Siwastaja
author_rank: Super Contributor
timestamp: July 02, 2024, 11:40:18 am
content: ... and buying a 32-core XCORE^TM and only utilizing 17 cores for the application somehow doesn't count as "underutilization"To put the underutilization thing in perspective, CPU time seldom is a limiting factor for a microcontroller selection. Performance has increased year after year and 32-bit chips running at over 100MHz cost some $1 in volume today. Almost always something else is limiting, such as peripherals, maybe flash or RAM; rarely CPU time.The idea that you somehow should "fully utilize" a part is ridiculous. What you pay for vs. what you get is what matters. If a $1 CPU does the job and has excess performance, then what?And think about using 1/16W rated resistors as pull-ups, where they only dissipate 10% of their ratings. What underutilization!

author: NorthGuy
author_rank: Super Contributor
timestamp: July 02, 2024, 01:38:18 pm
content: Quote from: Siwastaja on July 02, 2024, 11:40:18 amQuote from: Siwastaja on July 02, 2024, 11:40:18 amI think his point was that high CPU frequency forces the part to use various caches and long pipelines. They give you decent average performance. However when it comes to the worst case, the caches and long pipelines don't help. For example, the worst interrupt latency of a 400 MHz part won't be 100 times less than the interrupt latency of 4 MHz part. Hence, to get decent worst-case interrupt latency, you may have to choose a part with much higher frequency (and much higher average performance) than you would otherwise need.I agree with this to some extend. Instead of being obsessed with bits and MHz, you should look at the characteristics of the CPU and periphery and evaluate how they may be used for your task. For example, the OP went for higher MHz, did a very good job studying the internals of the CPU and then wrote lots of assembler with reshuffled instructions to avoid stalling the pipeline (which is hard to do). But wouldn't it be easier to just get a DSP for his convolutions, or even use a small FPGA?

author: tggzzz
author_rank: Super Contributor
timestamp: July 02, 2024, 02:21:03 pm
content: Quote from: Siwastaja on July 02, 2024, 11:40:18 amDon't make irrelevant and easily disprovable contentions...The OP's question is about an application where heis specificallycycle counting. Your contentions conflict with that.CPU time may not be a limitation in many applications, but as performance increases, so do the requirements. That is inevitable and has been demonstrated over the decades. That is particularly apparent as hardware functions are migrated into software.Almost every embedded realtime application I've been involved in has pushed the envelope in one way or another; sometimes performance, sometimes simplifying hardware - andalwaysdeciding on a partition between hardware and software. Yes, that boundary has changed over time, but not as much as "raw headline specs" imply.However, if the application's requirements and economics allow a $1 CPU, then by all means use it. Would I consider using an MCU instead of a 555 timer? Yes, in the right circumstances. (But to remind you, the OP's question involves pushing the performance).You can get 8..32 core xCORE devices.Your resistor analogy, as with most analogies, generates more heat than illumination. If you want to follow that analogy, then consider a resistor that had only a 1/16Wpeakrating without a "DC rating".

author: dietert1
author_rank: Super Contributor
timestamp: July 02, 2024, 03:48:03 pm
content: One criterion for using the M7 core is the double precision FPU. It can certainly make a difference in some applications.And i'm currently studying a STM32H7 ethernet application. Although the ethernet peripheral is available with M4 as well, i'd guess it got some new tricks on the H7.But getting into building computers really doesn't make a lot of sense nowadays. Very hard to design something competitive in terms of performance and cost in comparison to a RPi. Better learn how to use it properly.Regards, Dieter

author: nimish
author_rank: Regular Contributor
timestamp: July 02, 2024, 07:53:09 pm
content: Quote from: tggzzz on July 01, 2024, 09:50:14 pmIf you use more than 5 cores then it's:This means that for more than five logical cores, the performance of each core is often higher thanthe predicted minimum but cannot be guaranteedCortex-M7 isn't all that complicated even with a cache, TCM, and pipeline. And there's a reason core 2 is a cortex m4.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 02, 2024, 08:32:36 pm
content: and also, the op needs to consider how much left over for future feature expansion.How many of us have started with plenty of performance and been crushed at the end with extra stuff in the micro ?---If you are struggling and cycle counting at the architecture stage you set yourself up for failure when more inevitably needs to be added to the design.My POV is something like CM7 is one of the best general purpose machines, and you dont need to worry about the pipeline and ordering etc and all that at the start off the project- .IE at the beginning, if you need to optimize, then you probably have not started with enough horsepower. However, as you get squeezed and need to get blood from the stone, you will find that there is plenty of performance available if you start using DTCM, ITCM, and CMSIS DSP libraries, and then finally, hand coded assembler, if you really must.I'm not a ARM fanboy, but I feel CM7 is a fantastic all-jobs machine,Do be aware of cache effects ! referring to global variables all over the place will KILL the throughput (if cache is under pressure) - put your globals in DTCM .Do use the CMSIS DSP libraries- they are excellent.If you are keen, try the pre-canned tools aware RISC-V soft core on EFINIX FPGA with custom instructions (option) - then, you can sub-contract the hard DSP stuff out of the CPU inline with the code. RISCV is good and the Efinix tools are fairly simple, a much easier learning curve than Xilinx.

author: tggzzz
author_rank: Super Contributor
timestamp: July 02, 2024, 09:26:25 pm
content: Yes and no.Yes, on bog standard me-too processors+toolchains.No, on processors+toolchains which have realised that is undesirable and that Things Can Be Better.When considering the architecture, inevitably it will be desirable to understand whether key functions can be implemented in software, or what hardware support will be required. That's basic hardware/software architectural partitioning.On the xCORE devices it is perfectly possible toaccuratelydetermine the performance of key algorithmsbeforehardware is availableaddmajorfunctionality in the knowledge itcannotaffect the performance of the key algorithms. Example of major functionality: bidirectional comms (including printf()!) over USB or ethernet, and front panel i/oThat's very powerful, and removes a major cause of uncertainty.In effect the XMOS hardware+toolchain brings FPGA-like predictability to embedded software systems. It shifts the hardware-software boundary. It also removes the need for an RTOS: scheduling and intertask comms functions are implemented in hardware

author: Siwastaja
author_rank: Super Contributor
timestamp: July 03, 2024, 12:18:03 pm
content: Quote from: NorthGuy on July 02, 2024, 01:38:18 pmExcept that it (nearly, roughly) does. For example, limiting to ARM Cortex-M universe, if you take the lowest cost M0 part with its 16-cycle interrupt latency and run it at 4MHz, and compare to M7-part with 12-cycle interrupt latency running code from ITCM at 400MHz, you getbetterthan 100x improvement. If you consider worst case difference in presence of multiple interrupt sources, tail-chaining of the M7 compared to M0 not having it increases the advantage even further.Now granted, if doing an IO from the beginning of the ISR is critical, as it often is, then that 400MHz part will lose a few cycles from synchronizing the write to the GPIO peripheral's address space's clock, and that pretty much loses those few clock cycles we gained from lower interrupt latency compared to M0, but oh well, it's still pretty close to 100x difference, give or take.On the other hand, if you compare that 12-cycle latency at 400MHz to an AVR's 4-cycle latency at 4MHz, the difference isn't 100x, but still 25x. Except that the AVR does not do hardware stacking so only in rare cases you can actually execute something meaningful on the first instruction of the ISR. So in reality where you have to stack something first it's close to 100x again, or, if you just write ISRs in C and not hand-crafted assembly, stacking is done first so the M7@400MHz is probably again at least 100x as fast.So it all depends, but in any case, rules of thumbs originating fromdesktop computingare utterly useless, and that isexactlywhere these market shills try to divert the discussion by mentioning irrelevant stuff such as caches, TLBs and whatnot, which are utterly irrelevant to the topic, because every Cortex-M7 runs timing critical code from ITCM if you so wish, bypassing caches completely and running with predictable memory performance equivalent to what you would get with guaranteed zero cache misses; and it doesn't even have the concept of TLBs.Also, their pipelining is simple enough that we are discussing about a jitter offew cycles. Say a loop often takes 12 cycles but sometimes it takes 11. And thesefew cyclesare exactly what this thread is all about. Not caches, not large differences, not timing guarantees, not gross underutilization; but cycle accuracy and getting rid of even small jitter in those few cases where itdoesmatter.It is unsurprising this particular market shill then tries to again divert the discussion to a totally different off-topic subject, using the same proven-irrelevant claims that they never stop using, despite the fact the claims have been again and again proved irrelevant. It is nearly comical to see they have guts to complainothersdiverting off-topic when simply replying to their off-topic. Which probably was a mistake, as the good old saying, don't feed a troll, applies pretty well to this master troll who has been doing this for years nonstop.Caches inmicrocontrollersare some of the most widely misunderstood features. Just like JPEG decoder peripheral, they are extras that only cost a little bit of die area (maybe a bit of static power consumption too) if not used, but otherwise do not hinder performance. Becausetiming-critical control using interrupts is a colossally basic use case for anymicrocontroller, those caches have absolutely nothing to do with it; they reach decent performance running directly off flash, and excellent performance running off core-coupled extra instruction RAM, which is a basic feature in nearly every MCU which has cache, too.And when you discuss further with this particular troll, you will see what happens. Despite the forum section saying microcontrollers, and discussion being about microcontrollers, suddenly application processors like Cortex-A pop up as example cases. No wonder the misconception that "high-end microcontrollers" somehow are not proper microcontrollers timing-wise is so widespread. Some people actually have a day job of confusing the field for others.But in actual microcontrollers, just like some applications need SPI and other need I2C, some applications also need ITCM and others cache. Say, an UI application drawing animations or playing MP3 files out of SD card could enable caches and greatly benefit from them. And DC/DC or motor controller would again enable ITCM and not bother enabling caches at all because they are irrelevant.In such case, a cache which is disabled is simply irrelevant, all it does is increase the chip cost by a few cents. Except that having fewer different parts (and instead, more capable parts with many options) increases the economy of scales, so in the end selling parts that have cache, even if most users would run them disabled, is still the cheapest option.But as a result, there is underutilization. Which is a big thing in MCUs; nearly every MCU is 90% underutilized, if not more, if you count CPU cycle utilization, RAM, FLASH let alone peripheral utilization. But an XCORE with 9 cores out of 16 being used and executing NOPs 99% of the time emulating HW peripherals is clearly not underutilization. (And I like the concept and the idea, just not their marketing presence here.)

author: NorthGuy
author_rank: Super Contributor
timestamp: July 03, 2024, 01:51:48 pm
content: Quote from: Siwastaja on July 03, 2024, 12:18:03 pmWe've been through this already:Quote from: mino-fm on April 10, 2022, 11:32:24 amIn these tests, 216 MHz CPU had 110 ns response latency, while 550 MHz CPU had 70 ns response latency. These tests have been done in ideal conditions. Running other tasks on the same CPU caused roughly 40 ns jitter (see earlier posts), further increasing the worst response latency of the 550 MHz CPU to 110 ns.For comparison, PIC18 would have roughly 190 ns response latency. Or compare this to 10 ns (or better) response latency you can achieve with 550 MHz clock in FPGA."Grau, teurer Freund, ist alle Theorie, Und grün des Lebens goldner Baum"

author: dietert1
author_rank: Super Contributor
timestamp: July 03, 2024, 03:30:49 pm
content: Cycle counting for an audio project means there is something fundamentally wrong. It doesn't make sense to force the CPU into precision timing. Clearly interrupt timing won't be deterministic, neither DMA, nor GPIO.And it isn't necessary. Peripherals that need precision timing support that. The SAI peripheral device will take care of digital audio. It works with DMA buffers and FIFOs to assure precise timing. Of course there will be some latency.The OP didn't explain what it's all about. It sounds more like a rant. For me cycle counting means reading the CPU cycle counter. It's fairly easy to determine whether a function doesn't finish in time or is close to that. Maybe it takes more time or starts late.Regards, Dieter

author: tggzzz
author_rank: Super Contributor
timestamp: July 03, 2024, 04:22:52 pm
content: Quote from: dietert1 on July 03, 2024, 03:30:49 pmAgreed. The issue is almost always toguaranteethemaximumtime is less than therequiredtime.QuoteIt might be easy enough to measure and state that youhaven't observeda timing violation. That's insufficient inhardrealtime systems.Even a 486 (which hadtinycaches compared with many modern embedded MCUs) and tested under ideal conditions showed a peak ISR latency ~8* the mean latency.Modern embedded systems often have software components from more than one person and/or more than one company. If they are just bolted together and execute in a single core, it rapidly becomes very difficult topredictworst case performance.And then you have the question of how networking components might behave when something has gone wrong with the network components outside your system. Been there, narrowly avoided the scars.So you have to measure andhopeyou've hit the worst case performance. Unsatisfactory.

author: tggzzz
author_rank: Super Contributor
timestamp: July 03, 2024, 04:51:45 pm
content: Quote from: Siwastaja on July 03, 2024, 12:18:03 pmFirst of all I will assume there is a language issue with the meaning of "shill". I gain absolutely nothing from noting the beneficial characteristics of the XMOS ecosystem. Therefore I am not a shill. I am, however, obviously a fan.Secondly, as embedded processors and systems increase (mean) performance by using the same techniques as desktop systems, so embedded processors will suffer the same problems. How could it be otherwise?!(Those with a long memory and wide experience will realise that the same is true for high performance computing and workstation/desktop computing)QuoteAnother strawman argument. Caches increase mean performance, but not worst-case performance. (Actually, they can slightly reduce worst case performance, but that is a lesser concern)Hard realtime systems only "care" about worst case performance. Guaranteeing worst-case performance in a system with caches is problematic, to say the least.QuoteWell, there's a mishmash rant of intertwined statements. Not worth trying to disentangle them and refute them.QuoteDisabling the caches will indeed remove the unpredictability; the processor then reverts to pessimistic performance. Better not to have the cache in the first place!QuoteSuch cores are devoting 100% of their computation to a single taskpredictably. That is a necessary goal.

author: dietert1
author_rank: Super Contributor
timestamp: July 03, 2024, 05:30:25 pm
content: Quote from: tggzzz on July 03, 2024, 04:22:52 pmYet that's how everybody handles this. The problems of complexity you mention are a good example. After integration demonstrate the error rate to be less than 10 ** -10 or 10 ** -15 and together with code revision that's enough of a proof. No reason for assembly language, on the contrary.Let's mention the histogram method. An important tool in statistics for the analysis of actual statistical distributions and it should be applied to the timing analysis of complex MCU apps. One should learn to handle this instead of thinking endlessly about hardware details that can't be changed anyway.Regards, Dieter

author: tggzzz
author_rank: Super Contributor
timestamp: July 03, 2024, 05:46:16 pm
content: Quote from: dietert1 on July 03, 2024, 05:30:25 pmHow exactly do youdemonstratefailure rates of <10-15? If there is a possible failure every microsecond (fast!), that is 109s = 30years!It is a unpleasant to have to resort to measurements and statistics when it is possible to have reliable prediction. Why presume you must settle for second best?Let's fight for improved techniques

author: dietert1
author_rank: Super Contributor
timestamp: July 03, 2024, 06:41:53 pm
content: 10 ** -15 could be for example be a failure rate for audio bit stream output. The STM32H7 SAI maximum frame length is 256 bits. Assume digital audio at 96 KHz, so bit rate of 24 MHz. After one year that gives 7.56E14, not so far off. It can be faster if you have multiple test setups. I'd guess the difficulty will be to make a tester that is as reliable.I am always stunned by how easy things have become. Still remember how much time i once spent on making a Z8000 computer. Wiring buses with RAM, ROM and peripherals, creating nonstandard clock generators, TTL glue etc. etc. I am equally stunned by the fantastic reliability of modern MCUs. They can run for years without ever crashing. Of course if one includes error recovery for I2C and the like.Regards, Dieter

author: tggzzz
author_rank: Super Contributor
timestamp: July 03, 2024, 08:35:35 pm
content: Quote from: dietert1 on July 03, 2024, 06:41:53 pmI was presuming program-level event failures, not bit-level events.Have things changed since I designed by first computer, a 6800 with 128bytes of RAM (i.e. not enough for an SMS message)? Yes and no.Yes: smaller faster cheaper.No: 8-bit processors and C are still being used.After 40+ years I think it is reasonable to expect afundamentalimprovement. After all, multicore processors are the current standard and are definitely the future.So far the best available new concepts is xC+xCORE, and possibly Rust.But even they are old. XCORE=Transputer (early 80s), and xC=Occam++/CSP, is effectively Occam early 80s, and Tony Hoare mid 70s respectively. CSP is continually reinvented, e.g. in Go.

author: NorthGuy
author_rank: Super Contributor
timestamp: July 03, 2024, 08:50:50 pm
content: Quote from: dietert1 on July 03, 2024, 03:30:49 pmIf you maintain a buffer which gets transferred somewhere with I2S, you don't need anything deterministic. Your timing can vary. You just need to maintain some amount of data in the buffer. This can be say 100 elements on average. But it can go down to 50, or it can increase to 150. In either case you don't care as long as there's something in the buffer. You're not concerned with jitter and what you need is average performance.If you need to do more extensive calculations for the signal processing, it may so happen that the calculations are so slow that your buffer dries out very quickly. This is similar to the situation that OP has faced. He took faster CPU and he re-wrote the code in assembler. Thereby he increased average performance. Now the CPU calculates his convolutions twice as fast and the buffer no longer dries out. This also has nothing to do with being deterministic.

author: nimish
author_rank: Regular Contributor
timestamp: July 03, 2024, 09:02:58 pm
content: Quote from: tggzzz on July 03, 2024, 04:51:45 pmthis is why Cortex M7 almost always have DTCM and ITCM. Get the best of both scenarios: predictable latency when you want it, caching when it doesn't matter for speed.XMOS barrel processors are a neat little artifact from an alien planet. Interesting, but mostly a curiosity. I'd rather use a SoC-FPGA and a standard language over some tiny company's proprietary custom stuff anyway.

author: uer166
author_rank: Super Contributor
timestamp: July 03, 2024, 09:04:01 pm
content: Quote from: tggzzz on July 03, 2024, 08:35:35 pmI sincerely hope to you're getting paid some serious bucks to shill for them so much, seriously wasted potential otherwise.

author: tggzzz
author_rank: Super Contributor
timestamp: July 03, 2024, 09:31:42 pm
content: Quote from: uer166 on July 03, 2024, 09:04:01 pmI write about what interests me, and why it interests me.Why do you write about things?

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 03, 2024, 10:06:16 pm
content: seems a bit discussion about not much except that all good embedded people know that if something has to happen on time, everytime, then it has to be hardware driven and you pick a processor that can keep up with the interaction with the hardware (producing or consuming samples or whatever).... CM7 provides best of both worlds with a mix of complex ugly C (flash/caches) , and lean and mean C/ASM Interrupt code (ITCM). I'm in the basket of mixing a small FPGA (in order to make hardware deadlines) with a softcore micro - if the CM7 cant do it.

author: tggzzz
author_rank: Super Contributor
timestamp: July 03, 2024, 10:16:56 pm
content: Quote from: glenenglish on July 03, 2024, 10:06:16 pmWith most MCUs, that is the case.But the xCORE+xC+toolchain has many FPGA-like properties, especially parallelism and timing guarantees. It shifts the boundaries of what can be achieved in software without needing an FPGA.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 03, 2024, 10:32:51 pm
content: While using interrupts requiring "extremely" low latency was common in the days were MCUs were slow and only had very basic peripherals, nowadays it rarely makes sense to rely on interrupt latency (as long as it's not ridiculously excessive) to ensure strict timings down to the µs or less. Modern MCUs almost all have chainable peripherals ("hardware triggers"), fancy PWM and whatnot, that would not require relying on interrupts for such tasks. Some even have programmable IO blocks, such as the RP2040's PIO and NXP's FlexIO, which can also do quite a bit. Use them, don't try to be clever with ISRs.If that still doesn't cut it, you can of course consider using an FPGA - sometimes a small FPGA, linked to a MCU, will dramatically reduce your headaches while adding maybe just a couple bucks to the BOM. Worth it.And then, yes, you otherwise have the xCORE stuff, which is great too. The only downside I personally see to them, which is not intrinsic, but just a market fact, is that going for xCORE will tie you up to a single vendor.

author: brucehoult
author_rank: Super Contributor
timestamp: July 04, 2024, 12:17:41 am
content: Quote from: nimish on July 03, 2024, 09:02:58 pmThe wacky thing about XMOS is the instruction set and programming language, not that it's a barrel processor, which is a perfectly fine way to design a multi-core CPU if you care about total throughput not peak single-thread speed.At last week's RISC-V Summit in Munich, Ben Abdelhamid, a postdoc at Heidelberg University, showed his "BRISKI" RISC-V RV32I barrel processor which implements up to 16 harts/core. It uses fewer than 800 LUTs and runs at 650+ MHz in a VU9P FPGA, with CPI = 1 (always .. no cache or branch predictor needed), achieving a pretty amazing 0.8 MIPS/LUT. It fits 1024 cores (16384 hardware threads) in that FPGA.

author: nimish
author_rank: Regular Contributor
timestamp: July 04, 2024, 02:31:31 am
content: Quote from: brucehoult on July 04, 2024, 12:17:41 amBarrel processors are fine but they are uncommon enough and require a re-think of how computation works, so they are wacky for general purpose use.Not a great idea to shackle yourself to a single vendor either, if the chip shortages weren't enough convincing.

author: brucehoult
author_rank: Super Contributor
timestamp: July 04, 2024, 02:48:59 am
content: Quote from: nimish on July 04, 2024, 02:31:31 amExactly the same re-think as any other computer with multiple cores (or anyway more than 2 or maybe 4), which has been common for quite some time.The Mac Pro my employer provided me in 2009 had 8 cores. Another employer got me an i9 with 18 cores in late 2017. The Linux PC I built in early 2019 has a 32 core (64 thread) ThreadRipper. Macbook Pro laptops now come with up to 16 cores. My Lenovo laptop has an i9 with 24 cores (32 threads).Pretty much everyone who wants to take full advantage of the computer they have now has to ask themselves "How can I divide work up into lots of threads?"Interestingly, Apple was pushing their answer to using lots of cores in a single application -- Grand Central Dispatch -- a long time ago and it was built in to OS X Snow Leopard (2009) and iOS 4 (2010).

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 04, 2024, 05:51:01 am
content: Well, I decided to take a look at this famed XMOS productI have an estemed colleague, who about 5 years ago, a hard SHARC DSP guy, experimented with XMOS and gave it away, went back to SHARC.https://www.xmos.com/download/xCORE-Architecture(1.2).pdfI think that that flyer is a whole load of snake oil marketing crap. and quite misleading the way its competition are portayed, and they're like comparing it against a snail.and its written inferring you dont have to synchronize anything, just write code in lots of units and it will all just work.L ooking,. naa it seems xmos is alive and well, and quite capablehttps://www.xmos.com/xcore-aiIt reads well. But I am sure the devil is in the detail. Or is it another modern day take on a GPU architecture ?Seems the multicore - is not really multicore- more like cores with hyperthreading x 8 per core.Someone must have bought their chips ?!

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2024, 08:29:54 am
content: Quote from: brucehoult on July 04, 2024, 12:17:41 amBarrel processors have a lot of advantages for "embarassingly parallel" problems. Basically you can avoid thesignificantsilicon real-estate used by caches, prediction and super-scalar operation, and use it to provide many simple cores.Sun used that to great advantage in their UlteaSPARC T-series processors for server-side operations. When I used them for a soft real-time telecoms server system, other engineers were stunned by the (easily achieved) performance. Shame Big Red borged them.It is easy to produce fast parallel hardware; many companies have done it. Where they have fallen down is normally the software - especially when they expect people to be able to write correct parallel C/C++!The key point about XMOS is not the processor (and who cares about the instruction set) but the way the cores, switch fabric, peripherals, language and toolchain have been integrated into a remarkably pleasant ecosystem.CSP starts with the presumption that processing is parallel, and has a decent theoretical basis. Unsurprising when it was invented by Tony Hoare (QuickSort, semaphores, NULL, Turing Award), and its concepts have been continually implemented (TMS320, Go, etc)xC is C with the bits that make parallelism "difficult" removed (e.g. aliasing), and added RTOS and CSP parallelism constructsthe Eclipse-based IDE takes full advantage of everything being integrated, compiles the xC (and C/C++ ugh) code, then examines the optimised binaries to determine the min/max times ofallpaths from here to thereWe urgently need hardware+software ecosystems that are based on parallelism from thegroundsilicon up to application. 1970s tech with a few warts bolted on 40 years too late simply won't cut it in the future. So far xC+xCORE+IDE are the sole demonstration of a better practical system. Rust has promise, but is too far from the hardware.We need more highly parallel ecosystems. they have even had chips where one of the cores was an ARM processor. The different processor was fully integrated into the hardware+xC+IDE ecosystem. start with the concept of a memory model, and never forget that C defined parallelism as being part of the library - and explicitly avoided providing the properties that would enable a library to implement parallel functions in C

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2024, 08:44:41 am
content: Quote from: nimish on July 04, 2024, 02:31:31 amAgreed and agreed.Barrel processors cannot overcome "Amdahl's law", and are not relevant to desktop applications. But theyarewell-suited to "embarassingly parallel" problems. Such systems are commercially important, at a low embedded level (XMOS) and at the server application level (Sun UltraSPARC T series, Niagara etc).The single source vendor is indeed a significant issue. It occurs at all sorts of levels from resistors and transistors up to complete computers. Examples:resistors: Caddock resistors in some of HP's scope probes, resistors in 8 digit DVMstransistors: some high frequency PNP BJTs, jFets, terahertz componentssystems: just look for products fromanysmall company with highly competent employees (e.g. Phil Hobbs, John Larkin)computers: IBM360 - 'nuff said!So the single-source supply issue has to be taken into account, but is astandard problem.For an individual engineer's career, it is also an issue. You simply cannot and should not jump on the next shiny geegaw that is similar to existing geegaws, since such experience has a half-life of 3-5 years. OTOH, it is very beneficial to spot the general trends that will become important in the future, and try to anticipate them; that makes youveryemployable.

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2024, 08:52:19 am
content: Quote from: brucehoult on July 04, 2024, 02:48:59 amThe alternative of trying to make a faster single-core processor has run out of steam, e.g. infamously with HP/Intel'sItanicItanium processors, and look at all the times x86 processors (from Intel and AMD) have to be neutered because someone else has found a way to abuse the hardware performance enhancementsPushing barrel processors as being useful for general purpose desktop use would indeed be ridiculous.There are many commercially important applications that are highly multi-threaded. Such "embarassingly parallel" applications work very well on barrel processors, e.g. telecoms/web servers running on Sun's T-series. Shame Sun was Borged.

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2024, 09:09:18 am
content: Quote from: glenenglish on July 04, 2024, 05:51:01 amThere are many reasons why that might be the appropriate decision. Technical prowess is often subordinate to commercial and training requirements.QuoteWhat do you expect from a flyer? If you expect to grok everything from an advertising flyer, then you will always be disappointed.QuoteStrange statement.That's much like saying "just write code in lots of functions/procedures and it will all just work". Such misstatements could be used to incorrectly imply procedural languages and von-Neumann architectures are a waste of time!However, the synchronisation primitives and conceptsdo"just work".In addition, the hardware is very simple to explain and use, the documentation isblessedlyterse (no 1000 page datasheets!), and I haven't run into any errata.QuoteYes, barrel processors and hyperthreading are intimately related. Intel invented the term "hyperthreading" for their two-way x86 barrel processor.I've no interest in ML/LLM applications and architectures, since their response to stimuli cannot be predicted from the design. Hence I cannot comment on xcore-ai.I believe there are many chips in use, e.g. in high-end audio and "Alexa loudspeakers/microphones" type things. I have no interest in either.XMOS has been delivering chips from (IIRC) ~2007, so they must have been around for almost 20 years. They must be doing something right!

author: dietert1
author_rank: Super Contributor
timestamp: July 04, 2024, 10:45:02 am
content: In our lab there is a XMOS evaluation kit plus some prototype boards made here.When i looked into it, the training examples proposed using one thread as CPU and then one thread for each peripheral. So peripherals were implemented in firmware. They provided libraries for common peripherals including source (open for improvement).Forcing one of the threads into precise timing is less damaging than doing it to the Cortex M7 CPU, but things work similar as soon as the threads need to interact.A design method somehow intermediate between MCU and FPGA. The project would be a little more complex than a MCU project, but without the need to implement a CPU on a FPGA. Of course, nowadays there are MCUs that include FPGA fabric and FPGAs that include hardware CPUs.Regards, Dieter

author: tggzzz
author_rank: Super Contributor
timestamp: July 04, 2024, 11:08:36 am
content: Quote from: dietert1 on July 04, 2024, 10:45:02 amAgreed, but you can include the ISR plus some heavyweight processing in such a "low-level front-end" thread/core. Such processing might include "DSP-like" operations.The details depend on the application, of course.QuoteAgreed, however...Where threads need to interact is ideally an consequence of the application and/or the solution's architecture. That is inherent to the problem being solved, and hence is inescapable. We have to live with thatThe "damage" caused by forcing everything onto a single core is an consequence of the implementation and toolset.I dislike problemscaused by the tools, preferring to concentrate on the problemsinherent in the application. We can - and as engineers we should - strive to remove "problematic tools"QuoteAgreed. That is a good summary. The boundary has been shifted.The toolsets for FPGAs are much more heavyweight and ponderous than those for software development. The software is usually implementing concepts closer to the application being solved. Hence shifting the boundary is beneficial.We seem to be in violent agreement