Cortex M55 - Page 1
Forum section: electronics / general

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: February 23, 2020, 07:00:01 am
content: Hi,Have you seen the news?https://developer.arm.com/ip-products/processors/cortex-m/cortex-m55Any Ideas who would make the first chips? and when they became available? I prefer to have something like M75 too, a Cortex M7 with a lot of improvements over it and added DSP and AI capabilities, and some math accelerators for Cordic and Filter algorithms.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 23, 2020, 04:00:09 pm
content: Quote from: ali_asadzadeh on February 23, 2020, 07:00:01 amYes.Quote from: ali_asadzadeh on February 23, 2020, 07:00:01 amNope. This is all very recent.

author: lucazader
author_rank: Regular Contributor
timestamp: February 23, 2020, 06:52:55 pm
content: Interesting that they have implemented half-precision float calcs in hardware (fp16). As well as fix point acceleration in their dsp.Also interesting that the FPU and the Vector unit share their registers to save silicon area.Although I dont personally use AI inferencing in my line of work so this announcement overall doesn't impact me, I could definitely make use of the the fp16 support.

author: ataradov
author_rank: Super Contributor
timestamp: February 23, 2020, 06:55:07 pm
content: Quote from: lucazader on February 23, 2020, 06:52:55 pmFor what? This stuff puzzles me.I can see if you use FP16 to save resources on massive SIMD instructions, but otherwise why limit yourself to 16 bits?

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 23, 2020, 07:15:41 pm
content: Quote from: ataradov on February 23, 2020, 06:55:07 pmYeah, I personally don't see much use for this either. What's the typical use case for FP16?Accelerated fixed point, OTOH, yep!

author: lucazader
author_rank: Regular Contributor
timestamp: February 23, 2020, 07:23:59 pm
content: My use case is in some of my bldc motor control applications where i dont need the full precision of fp32, and the speed increase of moving to an fp16 calc would be beneficial to allow higher max commutation rate of the motor.That is one example that i have, I'm sure i could find others.This is based on the assumption that the fpu doing an fp16 calc will return a result in fewer cycles than fp32. If I am wrong, and there is no speed-up then of course there is no benefit.

author: Yansi
author_rank: Super Contributor
timestamp: February 23, 2020, 07:43:39 pm
content: Saturating arithmetics and SIMD and FLOAT FPU is present since Cortex M4.Since some newer Cortex M7 revision, there is even a hard double FPU.So what improvements are we talking about here?

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 23, 2020, 07:57:10 pm
content: Quote from: lucazader on February 23, 2020, 07:23:59 pmOK, I see. I think this is a typical use case where using fixed point would make sense, but of course FP is usually easier (less to worry about with overflows and underflows, although you'd quickly reach those with FP16 too).Now I'm not sure how critical your calculations could be for motor control. Given the inertia of a typical motor, how could you ever be limited by computation speed unless you do really HEAVY computation on a really slow MCU?Quote from: lucazader on February 23, 2020, 07:23:59 pmI would suspect this would be the case for divides, maybe multiplies (not sure, depends on the FPU) and probably not additions and subtracts.

author: mach665
author_rank: Newbie
timestamp: February 24, 2020, 04:59:33 pm
content: The vector register size in all previous Cortex-M is only 32bit. The vector register size in Helium extension in ARMv8.1-m is 128 bit. Plus some other instructions to speedup some math calculations. Think about the increase of the throughput.As a comparison, intel's Advanced Vector Extensions (AVX-512) has a vector size of 512bits and includes preparations to extend 1024 bits.Copied from Wikipedia for ARMV8"The Scalable Vector Extension (SVE) is a new extension for ARMv8 allowing "implementation choices for vector lengths that scale from 128 to 2048 bits"; a complementary extension that does not replace NEON. A 512-bit variant has already been implemented."

author: brucehoult
author_rank: Super Contributor
timestamp: February 24, 2020, 11:53:17 pm
content: Quote from: mach665 on February 24, 2020, 04:59:33 pmSo far, almost all shipping consumer Intel CPUs have either been limited to AVX-256 or else they slow the clock rate for AVX-512, which negates much of any theoretical performance increase.QuoteSVE and MVE are different things.SVE has 32 vector registers and 16 predicate registers (which contain one bit for each data element in a vector register). The vector registers are, as stated above, anywhere from 128 bits to 2048 bits in size, varying from CPU to CPU. SVE was announced 3.5 years ago in August 2016. ARM has, as far as I know, not yet announced let alone shipped any CPU core implementing SVE. Fujitsu has prototype chips of their own CPU core design implementing SVE for an uncoming supercomputer.SVE is quite similar to the RISC-V "V" extension which is nearing final specification status (I'm on the Working Group and implemented the first Spike simulator support for it, and first assembly-language kernels). RVV has 32 vector registers of length anywhere from 32 bits to 2^32 bits (i.e. much more scalable in both directions than SVE). If your code doesn't require a full 32 register variables you can gang the registers together into 16, 8, or 4 registers of 2, 4, or 8 times the hardware length for greater efficiency. Predicates are stored in normal vector registers and always take exactly one register even when the registers are ganged for data elements. SVE and RVV both support using only the initial part of the vector register for short vectors or for the last elements of a long application vector. RVV has a "current vector length" register while SVE does this purely by setting the predicate bits for the tail of the vector to false.MVE (aka Helium) is a completely different and simplified but again similar vector instruction set announced by ARM out of the blue a year ago in February 2019. It has a fixed set of 8 registers of 128 bits each -- the same as the smallest RVV in ganged x4 mode.The advantage of SVE and RVV is that you can write one assembly-language (or compiled) program that will run at maximum efficiency over a wide range of different CPU vector sizes. You don't have to rewrite your program to go from MMX to SSE to SSE2 to AVX to AVX-512. And new CPUs don't need entirely new instructions every time alongside all the older instuctions.

author: mach665
author_rank: Newbie
timestamp: February 25, 2020, 01:00:04 am
content: I understood that MVE is not SVE. It is not even NEON, but after all we are talking about micro-controllers. With 128bit vector registers, we can do 4 times as many calculations as Cortex-M7 in an ideal world. It's a lot faster for DSP or ML jobs. And I would assume the right way to use MVE is using CMSIS libraries, or we may have to embed assembly.

author: Yansi
author_rank: Super Contributor
timestamp: February 25, 2020, 08:29:32 am
content: Assembler? That is what the intrinsics are for. Perfectly fine to be used within the C environment. Even the CMSIS DSP library is written completely in C, using the intrinsics.(I would not call that an assembly language).

author: tszaboo
author_rank: Super Contributor
timestamp: February 25, 2020, 08:56:45 am
content: Quote from: SiliconWizard on February 23, 2020, 07:15:41 pmFP16 is used for AI, especially for image related stuff.

author: brucehoult
author_rank: Super Contributor
timestamp: February 25, 2020, 09:12:20 am
content: Quote from: Yansi on February 25, 2020, 08:29:32 amNot sure what you are replying to, but I seem to be the only one who has mentioned assembler.You can't use intrinsics in C until you add them to the compiler. I helped with that work in Oct/Nov/Dec. In the case of vector instructions you also need to first add to the compiler the new register file, a register allocator, C types representing the vector registers -- which are tricky for both SVE and RVV because previously gcc and llvm had no concept of a register that you don't know the size of at compile time, which complicates a lot of things from stack frame allocation and spilling to argument passing.You can start trying new instructions in assembly language as soon as you have added them to the assembler (which the C compiler also needs), and the emulator. That was Jan/Feb/Mar last year.

author: mach665
author_rank: Newbie
timestamp: February 25, 2020, 04:56:27 pm
content: Hi brucehoult,What is the current compiler status? For example, does GCC optimize C source code into those SIMD instruction with wide vector registers? If it can, how efficient is it. What is the recommended way to do it now? Using library like NEON lib, or relying on GCC is good enough?

author: brucehoult
author_rank: Super Contributor
timestamp: February 25, 2020, 06:19:52 pm
content: Quote from: mach665 on February 25, 2020, 04:56:27 pmHell no.GCC and LLVM can auto-vectorize only the most basic and trivial things, on *any* ISA. Support for SVE and RVV is not there at all yet, at least in released versions. It'll be years before there is good support. I think SVE may have intrinsics support released. RVV has intrinsics support in company internal testing and that's the best you can hope for on initial hardware releases. Basically assembly-language programming but the compiler handles the register allocation for you.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 26, 2020, 05:15:49 pm
content: Quote from: brucehoult on February 25, 2020, 06:19:52 pmAny? A bit much here. any that supports vector operations probably. Surely obvious, but "any" sounded weird. And I don't think they would (at least for GCC) unless you enable the corresponding optimizations (either automatically through some -O level or specifically.)I had done some experiments with GCC for x86-64 targets.In order to guide GCC to make the most of it, it's often useful to use specific qualifiers/builtins, otherwise your chances of getting something remotely efficient are much lower, since the compiler can't make the required assumptions.To illustrate, you can compare the compiled assembly of the two versions (it would be interesting to try this for various targets at least supporting some SIMD instructions, you may need to adjust the alignment hint in __builtin_assume_aligned()):Code:void MAC1(double *Y, double *X1, double *X2, unsigned int n){for (unsigned int i = 0; i < n; i++)Y += X1 * X2;}void MAC2(double * restrict Y, double * restrict X1, double * restrict X2, unsigned int n){Y = __builtin_assume_aligned(Y, 32);X1 = __builtin_assume_aligned(X1, 32);X2 = __builtin_assume_aligned(X2, 32);for (unsigned int i = 0; i < n; i++)Y += X1 * X2;}The second one will have less overhead.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 26, 2020, 05:25:59 pm
content: Quote from: NANDBlog on February 25, 2020, 08:56:45 amAre there really benefits compared to fixed point for this application? Or is it just because people don't want/don't know how to use fixed point anymore?

author: tszaboo
author_rank: Super Contributor
timestamp: February 27, 2020, 09:05:37 am
content: Quote from: SiliconWizard on February 26, 2020, 05:25:59 pmMost AI calculations involve multiply-add, where the operand is between 0 and 1. You cannot represent that number with INT16. Also, you most likely would need to take care about overflows and that would mean that instead of 1 operation, you have to do more than 1. There is something called neural network quantification, which moves the model from floating point to fixed, but apparently, it is very computing intensive.There is also bfloat16, which apparently takes much less gates than fp16, and porting models to it is simple. Look at the Edge TPU. I really think high performance microcontollers will have AI coprocessor built in with FP arithmetic.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2020, 01:39:22 pm
content: Quote from: NANDBlog on February 27, 2020, 09:05:37 amExactly like "most" DSP calculations. It's been done on DSPs with fixed point for ages and still is. What's the problem?Quote from: NANDBlog on February 27, 2020, 09:05:37 amOh really? I'm talking about fixed point. You can put the virtual point wherever you like. If you absolutely need to represent "1" exactly (which I'm not even sure matters in most cases, as all calculations are approximate anyway, especially with FP!), you can just use a 1.15 fixed point number. Done, you can get your exact '1'. You'll actually get better accuracy than FP16 (except for the very low values) which has a mantissa of only 10 bits and an exponent of... 5 bits! Now that's very limited.Quote from: NANDBlog on February 27, 2020, 09:05:37 amThis is again very typical in many (most) DSP applications using fixed point. Nothing new here. Oh and FP16 will overflow/underflow pretty quickly too. You don't have a lot of leeway.I guess the real point is as I hinted earlier, relative ease of use, and most of all, that FPUs are now a commodity, so they are used everywhere without really giving that a second thought.I'm just saying that fixed point can have definite benefits especially in terms of accuracy, and especially in applications dealing with this kind of numbers in the [0, 1] interval. Low-precision FP, like FP16, will quickly lose a LOT of accuracy after just a few operations.I get the idea of using FP instead of fixed point in a processor that already contains one or more FPU(s) and nothing specific for fixed point, but here we were talking about a new core that apparently implements accelerated fixed point as well, so to me using FP16 on this instead of fixed point makes a lot less sense from a technical and computational POV. IMO. Again fixed point is still heavily used on conventional DPSs for a large range of applications, and if used exclusively, would cost significantly less silicon area (even with nice accelerated MAC operations). Now sure if an FPU is required for other uses, then reusing it for this and not bother with fixed point makes sense, but when you implement both, I'm not so sure. And OTOH, if you're going to implement a dedicated "AI" coprocessor, I would personally at least consider fixed point seriously.Thing is, as I said above, FPUs are now so ubiquitous that just getting FPU IPs to implement accelerators is likely a lot simpler - I would guess that there are a lot fewer (and good) accelerated fixed-point units available as IPs.

author: mach665
author_rank: Newbie
timestamp: February 27, 2020, 05:00:11 pm
content: Quote from: SiliconWizard on February 27, 2020, 01:39:22 pmFrom what I read, an important usage of FP16 is for applications that needs high dynamic range, but not very high SNR. Speech recognization as an example, the audio samples can have huge dynamic range, but recognizing the sound may only need a moderate SNR. Either you speak right next to the microphone or 10 feet away, algorithms implemented with FP16 works well in both cases.Someone may argue that a good AGC plus VGA for the microphone can get the work done too. But obviously that's harder to get right and much less appealing in this digital world.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2020, 05:04:56 pm
content: Quote from: mach665 on February 27, 2020, 05:00:11 pmI see, but FP16 has only a 5-bit exponent... the dynamic range will still not be very "high".As to dynamic range vs. SNR, that would be an interesting discussion.Quote from: mach665 on February 27, 2020, 05:00:11 pmYes that too. Well if you're relying on FP operations, you're still in the digital domain. So if it can be done with FP, it can be done with a purely digital AGC with fixed point (and get you better SNR).But yeah it again all comes down to keeping things very simple, and likely relying on ready-made FPUs as I hinted above.

author: langwadt
author_rank: Super Contributor
timestamp: February 27, 2020, 06:33:40 pm
content: Quote from: SiliconWizard on February 27, 2020, 01:39:22 pmwith signed Q1.15 you can't quite get 1, you can get exactly -1, so -1*-1 overflows

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2020, 06:51:58 pm
content: Quote from: langwadt on February 27, 2020, 06:33:40 pmI was implying unsigned fixed point here (which covers many of the algorithms in which only values in the [0, 1] range are used and combined through positive weighted sums).The Q notation (that I didn't use on purpose) is always a bit confusing. Strictly speaking, from the usual definition, Q1.15 would be signed indeed and wouldn't fit within 16 bits actually, as it means one integer bit, 15 fractional bits, and an implied sign bit. So the correct notation would be Q1.14... just a small detail.

author: langwadt
author_rank: Super Contributor
timestamp: February 27, 2020, 07:03:54 pm
content: Quote from: SiliconWizard on February 27, 2020, 06:51:58 pmI've never seen Q1.15 used for anything but 16 bit with the range -1,0.999..

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2020, 07:25:43 pm
content: We're digressing a bit, but anyway. The Q notation is always a bit ambiguous and there are actually two conventions, one including the sign bit and the other, not. If you know the word length is 16-bit, Q1.15 usually means that the leading 1 bit is just the sign bit (as you assumed), but that's confusing as it's meant to be the integer part, so definitions vary a bit even from one vendor/textbook to the other. As long as you know what is talked about exactly, it's fine...So anyway, if you want to represent exactly -1 and +1 (assuming that you really need that), you'd use Q1.14 that would mean 1 integer bit, 14 fractional bits, and an implied sign bit.I've often used unsigned fixed point. A whole range of calculations don't need negative numbers. Some definitions would write what I meant as: UQ1.15 ('U' as in unsigned).

author: Yansi
author_rank: Super Contributor
timestamp: February 27, 2020, 07:46:27 pm
content: Quote from: SiliconWizard on February 27, 2020, 07:25:43 pmHowever that implies you have a 15bit register. That is a pretty nonstandard use of the notation and I would strongly suggest against using it.For a 16bit register:Q1.15 gives you -1.00000 to +0.99997Q2.14 gives you -2.00000 to + 1.99994Q3.13 gives you -4.00000 to +3.99988etc...Q15.1 gives you -16384 to +16383.5The range is from-2m-1to2m-1- 2-nfor signed fractional fixedpointQm.n(if I did not make a mistake in the formula). m+n shall always equal to the number of bits of the register in question.

author: langwadt
author_rank: Super Contributor
timestamp: February 27, 2020, 07:50:47 pm
content: Quote from: SiliconWizard on February 27, 2020, 07:25:43 pmif you add a signbit you lose all the advantages of two-complement

author: Yansi
author_rank: Super Contributor
timestamp: February 27, 2020, 07:51:05 pm
content: Regarding unsigned fractional, UQm.n, it is like so:UQ1.15 means from 0.00000 up to 1.999something.I think wikipedia is quite clear about this:https://en.wikipedia.org/wiki/Q_(number_format)

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2020, 08:02:15 pm
content: Quote from: Yansi on February 27, 2020, 07:51:05 pmYes. Exactly what I meant. Allows to represent '1' exactly. If that ever matters in a given situation.What NANDBlog was meaning earlier with "You cannot represent that number with INT16" is that he was likely thinking of either Q1.15 (for which I personally find the Q1.14 notation more consistent), or UQ0.16.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 27, 2020, 08:09:15 pm
content: Quote from: Yansi on February 27, 2020, 07:46:27 pmNope. Read again the very article you linked to.Code:here are two conflicting notations for fixed point. Both notations are written as Qm.n, where:Q designates that the number is in the Q format notation – the Texas Instruments representation for signed fixed-point numbers (the "Q" being reminiscent of the standard symbol for the set of rational numbers).m. (optional, assumed to be zero or one) is the number of bits set aside to designate the two's complement integer portion of the number, exclusive or inclusive of the sign bit (therefore if m is not specified it is taken as zero or one).n is the number of bits used to designate the fractional portion of the number, i.e. the number of bits to the right of the binary point. (If n = 0, the Q numbers are integers – the degenerate case).One convention includes the sign bit in the value of m, and the other convention does not. The choice of convention can be determined by summing m+n. If the value is equal to the register size, then the sign bit is included in the value of m. If it is one less than the register size, the sign bit is not included in the value of m.And I just do prefer the second convention that doesn't include the sign bit in "m", as I find it more consistent. I know the other notation is also common.

author: Yansi
author_rank: Super Contributor
timestamp: February 27, 2020, 08:53:53 pm
content: I have actually never seen anyone use a notation where (m+n) != number of register bits. But I have not done that much DSP stuff yet.

author: langwadt
author_rank: Super Contributor
timestamp: February 27, 2020, 09:31:22 pm
content: Quote from: SiliconWizard on February 27, 2020, 08:09:15 pmif it is signed magnitude that might make sense, but for the majority using twos-complement calling the msb a sign bitis iffy because it doesn't just set the sign, i.e. you can negate a number just by inverting the "sign bit", the msb is a bit just all the others it just has negative weight

author: Howardlong
author_rank: Super Contributor
timestamp: February 27, 2020, 09:55:40 pm
content: Quote from: Yansi on February 25, 2020, 08:29:32 amUnless it’s been updated recently, CMSIS DSP is unfinished. The filter/decimator for example is very unoptimised.Sometimes you need to amalgamate algorithms knowing the architectural register limitations to minimise store/load inter-algorithm memory buffer operations which are very expensive. Optimisers generally won’t know to do this because they have little clue about your data or use case.I did an AM radio on an M4F not so long ago, with an antenna connected to the ADC input. To do the quadrature oscillator, mixer/downconversion and polyphase filter using CMSIS DSP functions was expensive because you need to store and read buffers between blocks. If you can achieve all three functions in-register without resorting to storage between stages, you save a lot of time.I can’t remember why, but I couldn’t get the compiler to do what I wanted, including pipelining, unrolling loops, intrinsics and so forth, so there’s a hundred lines of assembler or so at the pinch point.Otherwise I agree, assembler these days is to be avoided.

author: Howardlong
author_rank: Super Contributor
timestamp: February 27, 2020, 09:57:54 pm
content: The only reason I can think for the 16 bit float is that you can reduce expensive store/load operations if 32 bit is unnecessary.

author: ehughes
author_rank: Frequent Contributor
timestamp: February 28, 2020, 02:25:59 am
content: meh. Kids these days and there fancy 32-bit floating point.We had 8-bit floatshttp://synthmuseum.com/ensoniq/ensmirage01.html

author: brucehoult
author_rank: Super Contributor
timestamp: February 28, 2020, 03:55:39 am
content: Quote from: Howardlong on February 27, 2020, 09:57:54 pmAnd total storage. And you fit twice as many of them into a given size vector register.

author: DBecker
author_rank: Frequent Contributor
timestamp: February 28, 2020, 03:05:37 pm
content: Quote from: SiliconWizard on February 27, 2020, 05:04:56 pmYou are mis-understanding the targeted application area -- neural networks.The trained networks are very fuzzy, in the sense of being imprecise. No element or weight has been studied and calculated to have the correct value, or even a reasonable value. Instead they just run the training until the wrong results are reduced in influence enough that a not-obviously-wrong output is produced. Each element contributes a little bit of the influence for the result, and each intermediate response is weighted.Like in the real world, some voices get a weight of 0.0000000001 and others get a weight of 0.99. With enough inputs, it doesn't matter if a particular weight is 0.51 or 0.52. But you don't want the 0.000000001 rounded up to 0.01 or forgotten completely.The point here is that, while superficially similar to SNR, it's quite different in detail. In particular the concept of AGC doesn't apply because you simultaneously want to add in very tiny contributions while being heavily influenced by the strong weightings.

author: mach665
author_rank: Newbie
timestamp: February 28, 2020, 05:29:37 pm
content: Quote from: DBecker on February 28, 2020, 03:05:37 pmWell, I understand your explanation. But a lot of ML algorithms use 8bit fix point. What's the difference here? What kind of ML algo needs floating point, what kind of ML algo only requires 8bit fixed point?

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 28, 2020, 06:18:52 pm
content: Quote from: DBecker on February 28, 2020, 03:05:37 pmAs mach665 saw as well, I think your reply was more to him than to me, as he was the one mentioning applications using AGCs to deal with an extended dynamic range, and my point to this was just to say that if you're dealing with it purely in the digital domain, FP16 would not provide any special benefit IMO compared to fixed point.I for one know quite well what the basic algorithms for NN are, and as I mentioned a few times already, the core of them is basically series of weighted sums.I can see how FP16 would allow to represent smaller values in the [0, 1] interval compared to fixed point of similar width. What I'm still not convinced about is the real benefit over fixed point. I'd like to see comparative examples which clearly show that FP16 would yield better results overall. Thing for instance is, when at some point you have numbers that are so small that they would be represented as 0 with fixed point, and some value (but with low precision) with FP16... but would that contribute to the overall weighted sum enough to matter? (Given that even big NNs are usually limited in the number of "neurons", and as I understood, we tend to favor more layers these days rather than more neurons at each layer level...)To sum it up, I would like to see a real comparison between FP16 and fixed point that would clearly show the benefits in real cases.Maybe in the end, FP16 takes less resources overall than fixed point, given that even though fixed point itself is less expensive to implement, more care is needed for the calculations using that, so possibly in the end fixed point would be more expensive. I'm just not quite sure or convinced at this point, and would be interested in reading papers about that specifically if there are any.

author: jnz
author_rank: Frequent Contributor
timestamp: March 01, 2020, 03:25:18 am
content: I don’t get it.Who is really doing AI / ML at the far edge on cortex processors? I’m sure someone is, but really how many people could this really apply to?

author: brucehoult
author_rank: Super Contributor
timestamp: March 01, 2020, 11:11:54 am
content: Quote from: jnz on March 01, 2020, 03:25:18 amYou may be right. This is an area which is still in very rapid flux and many people are going to RISC-V because of the ease (and legal ability) to experiment with custom functional units and instructions and data types to find out what is actually best for price / performance / efficiency for any particular application.See for example the Kendryte K210 chip, or this very recent research paperhttps://arxiv.org/pdf/2002.12151.pdf

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 01, 2020, 11:45:50 am
content: QuoteIs there a complete Datasheet and user-manual for that thing, what about tool-chains and tutorials for it, it seems very interesting but docs and info are lacking, or at least I could not find them

author: SiliconWizard
author_rank: Super Contributor
timestamp: March 01, 2020, 03:03:19 pm
content: Quote from: ali_asadzadeh on March 01, 2020, 11:45:50 amThere isn't any decent documentation, and that's the key issue with Kendryte.But the toolchain itself is the usual binutils/GCC, and they provide an SDK and a few examples. So far you need to use those to get started, and as some of us have done a few months ago with a Sipeed dev board (see corresponding thread), you can do quite a bit with the SDK, it's not hard to grasp. But yes, documentation is definitely lacking.

author: SiliconWizard
author_rank: Super Contributor
timestamp: March 01, 2020, 03:15:39 pm
content: Quote from: jnz on March 01, 2020, 03:25:18 amPotential applications are many. What exactly would make you think "Cortex" processors are not appropriate for this?(BTW, "Cortex" doesn't mean much - it defines a whole range of CPU cores, from ones for very small MCUs to the most powerful ones. So you probably meant to say "Cortex-M" processors, which are the cores which target MCUs. A Cortex A53 is very far away from a Cortex M0 for instance. Just a note, agreed the M55 is in the Cortex M line and thus targets MCUs. But still, there's also a large gap between M0 cores and M7 cores for instance.)Whether we think shoving AI everywhere is dubious in itself or not is another debate entirely, but the benefits of being able to implement some AI on "small" processors (low-cost, low-power) directly seem pretty obvious to me. What's the alternative? Big and hungry processors? Remote AI on servers? (Inefficient, high latency, big privacy concerns, unavailable if no network access...)TensorFlow is currently usable on a whole range of MCUs, it's not there just to be cute. Any hardware able to accelerate this (and do as much for lower power) will win some markets.

author: tszaboo
author_rank: Super Contributor
timestamp: March 02, 2020, 11:57:16 am
content: Quote from: SiliconWizard on February 28, 2020, 06:18:52 pmThere are many research papers, that show even INT8 provides acceptable results for neural nets, if it is translated to that. Usually the training works in FP32, that takes large memory (GBs) and a long time to do. The end result is a few megabytes large, describing the neural net. Take this as an example:https://github.com/tesseract-ocr/tessdataThis is google's text recognition neural net. One of the goal of reduced precision is to shrink this descriptor, that the main functions of an OS, this descriptor, and the actual net running fits into the cache of a CPU. If it is in the cache, it runs a lot more efficient and fast, compared to memory. English is 22 MB, which is already kinda doable, as there are CPUs with 32MB+ L2 cache. Similarly, you might want to run the thing on GPU, where cache is similarly small, and distributed.So if you reduce the precision from FP32 to FP16 or the new bfloat16, the model takes half the size, and the internal date is half the size. But you have to translate the network to run on this, and verify that it still works. bfloat is great, because it is just a truncation, so this is an easy step, without much risk.Note that for a Cortex M type MCU, the entire external memory size is about the size of the cache, so you are IO limited anyway.I've read some papers about the INT8, which apparently very fast and energy efficient, with only a small drop of accuracy (meaning for example word recognition).The real problem is that most big companies are doing their own thing, even with new silicon produced around a concept, so results are varying.In any case, I have an application, where I have an expensive link (GSM or 4/5G) to the mothership, and there is relatively large amount of data being generated that needs analysing (audio stream). I would need to process the data locally, as much as possible, and send as little as possible, if an event happened. The more type DSP or hardware acceleration I have the better.

author: wizard69
author_rank: Super Contributor
timestamp: March 02, 2020, 12:57:41 pm
content: It seems really strange to have ML in a processor targeting embedded, battery powered devices.

author: brucehoult
author_rank: Super Contributor
timestamp: March 02, 2020, 01:43:02 pm
content: Quote from: wizard69 on March 02, 2020, 12:57:41 pmThere is a massive difference between training the NN and applying the results of the training.

author: SiliconWizard
author_rank: Super Contributor
timestamp: March 02, 2020, 03:26:17 pm
content: Quote from: brucehoult on March 02, 2020, 01:43:02 pmYup of course. Training algorithms (and the amount of data required to train NNs) are impractical to run on small targets - and that's not the point at all. Even on big machines using NNs, the training phase is usually done beforehand and comparatively takes a lot of time.There are many applications (and this is the case when using TensorFlow) in which NNs are just pre-trained, and only the trained NNs (a bunch of coefficients in practice) are stored and executed on the target.Beyond this difference, I think there's a lot of misconception about what AI is used for (and how) in real applications (or what it can be used for).Some people seem to be influenced by the big solutions and the big words associated with them. One example would typically be Google translate, which requires enormous amounts of data and which constantly adapts its training while being used. There are many applications for AI in which pre-trained NNs are perfectly adequate though, and relatively small ones can yield useful results.Typical examples: voice/face/fingerprint/iris recognition, basic classification and localization of objects (such as identifying faces in a picture)...

author: ejeffrey
author_rank: Super Contributor
timestamp: March 02, 2020, 05:58:22 pm
content: Quote from: mach665 on February 28, 2020, 05:29:37 pmA lot of this is very recent research (relatively speaking). So for a long time it was conventional to use relatively high precision computation such as 32 bit floats. But with the demand to push ML into lower power devices, or increase throughput with SIMD operations, people have packed them into smaller word sizes and discovered to some peoples surprise that they still often work fine. I think the first generation Google TPU used some some custom 8 bit word, some kind of logarithmic encoding. Other people have used 16 bit floats, 16 bit integers, 8 bit integers, and probably a bunch of other things. That doesn't mean that every application uses (or can use) 8 bit.16 bit floats are also very useful in graphics applications because they are easier to use than fixed point and have enough resolution for human vision and common display technology. Probably a lot of other applications have this same character: half-precision float has about 0.1% resolution which is good enough for almost any sort of physical world interaction. Any application that either benefits from the high dynamic range of a floating point number (including for intermediate calculations) or just can't be bothered to deal with the annoyances of fixed point can probably use 16 bit floats. So implementing it is kind of a no-brainer since it is both ML buzzword compliant and has real applications