A couple questions about Milk-V Duo boards - Page 1
Forum section: electronics / general

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 24, 2024, 07:52:50 am
content: Interested in evaluating the CPU they use. I haven't ordered boards yet.Does anyone know:- How the C906 core compares to say a Cortex-M7 (namely the main C906 core @1GHz vs a iMXRT10xx @600MHz, for instance)?- The approximate current draw under typical load, and any "sleep" mode it may have?- Whether it's possible (probably) but, most of all, documented, to program the main core baremetal (no Linux)? If so, is there any documentation out there? (I've seen the typical setup was Linux on the main core and some RTOS on the second core, I'd like to use both cores baremetal).Thanks!

author: brucehoult
author_rank: Super Contributor
timestamp: April 24, 2024, 08:16:56 am
content: On my primes benchmark:Code:// 27.196 sec Teensy 4.0 Cortex M7 @ 960 MHz 228 bytes 26.1 billion clocks// 27.480 sec HiFive Unleashed RISCV U54 @ 1.45 GHz 228 bytes 39.8 billion clocks// 30.420 sec Pi3 Cortex A53 @ 1.2 GHz T32 204 bytes 36.5 billion clocks// 36.652 sec Allwinner D1 C906 RV64 @ 1.008 GHz 224 bytes 36.9 billion clocks// 39.840 sec HiFive Unl RISCV U54 @ 1.0 GHz 228 bytes 39.8 billion clocks// 43.516 sec Teensy 4.0 Cortex M7 @ 600 MHz 228 bytes 26.1 billion clocks// 47.910 sec Pi2 Cortex A7 @ 900 MHz T32 204 bytes 42.1 billion clocks// 48.206 sec Zynq-7010 Cortex A9 @ 650MHz 248 bytes 31.3 billion clocksThe D1 is not the same chip as the Duo but it's the same core at the same speed, and this benchmark runs entirely in cache.> The approximate current draw under typical load, and any "sleep" mode it may have?See:In the comments:Without SD card: 15mABooting: 80-114mA, average about 90.Idle after boot: 62mA> Whether it's possible (probably) but, most of all, documented, to program the main core baremetal (no Linux)?Sure. The UBoot is, after all, open source, so you can either use that to launch your own code (replacing OpenSBI) or else simply hack on the UBoot SPL. The biggest task will be initing the DDR2 RAM, but the code is there.

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 24, 2024, 10:27:21 pm
content: Thanks! So, in terms of pure performance, at least from your benchmark, the C906 looks way behind a Cortex-M7 at the same clock freq, but not too far between a M7 @600MHz and C906@1GHz.In terms of power draw, the "missing" info is whether it has any kind of sleep mode, which you can go to and resume from, either when running Linux, or baremetal.62mA on "idle" is not bad for a CPU @1GHz, but the ability to put it in some standby mode drawing much less, and from which it could get out of within maybe less than 1s, would be nice. I haven't found the info so far and may have to order boards and figure it out myself.

author: brucehoult
author_rank: Super Contributor
timestamp: April 25, 2024, 01:35:42 am
content: Quote from: SiliconWizard on April 24, 2024, 10:27:21 pmSure. The M7 is a pretty good dual-issue uarch -- better than A7, A9, A53, maybe close to A55. And running from SRAM not DRAM helps too. The C906 is purely single-issue, in order.Note that the Duo has massively more RAM with 64 MB (and 256 MB and 512 MB models just becoming available for only a few dollars more)

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 25, 2024, 01:41:44 am
content: Yep. The amount of RAM can be pretty interesting. Using a Cortex-M7, you rarely have access to anything else than PSRAM if you want more than the internal RAM, which is slow and limited to 8MB/chip (at least for the QSPI ones), or maybe SDRAM for some MCUs?The Duo's with 256MB and over are slightly different beasts as I've seen, they include an ARM core on top of the 2 RISC-V cores. I haven't figured out yet how all 3 can work together.

author: brucehoult
author_rank: Super Contributor
timestamp: April 25, 2024, 04:18:18 am
content: I think you have to choose one or the other at boot time, but I don't know.Also see:https://github.com/orangecms/sg_boot"loader for CVItek/Sophgo SoCs (CV1800B, SG200x)"He said today initial stuff is working and asked for collaborators.https://t.me/riscv/1/20904https://t.me/riscv/1/20913

author: brucehoult
author_rank: Super Contributor
timestamp: April 26, 2024, 02:40:05 am
content: More progress today:"Daniel aka CyReVolt 🐢: Transferring code to the CVITek 1800B worked with my tool. There was no output on the UART though. I'll see if the SRAM address is different, which isn't described in the manual. UART0 seems to have the same base address per the manual.""plat/cv180x/include/platform_def.h296: #define VC_RAM_BASE 0x3BC00000 // Shadow_vc_mem""Daniel aka CyReVolt 🐢: aha aha - 0x3BC00000 - neither "VC RAM" nor that address is mentioned in the manual""Daniel aka CyReVolt 🐢: THAT WORKS! VROOM VROOM! 🥳""I have now successfully run my own code on- Duo (CV1800B)- Duo 256M (SG2002)- Duo S (SG2000)The CV1800B has its SRAM at a different base address.UART0 is mapped to the same address for all three."His "own code" is written in Rust. Blergh for some people, ++ for others :-)A note on chip names and companies ... Sophgo bought Cvitech a year or two ago, so CV1800B and SG200* share lineage. Also, a well informed Chinese person on Telegram said a couple of days ago that Sophgo and Radxa are parts of the same company, or fall under the same umbrella company (Bitmain?). I just found this repo, which lends support to this:https://github.com/radxa/bm-bootloader-arm64

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 02, 2024, 05:29:29 am
content: I ordered a couple boards and dug into the SDK and datasheet to figure out the details and possibly how to use it "baremetal".The SDK repo:https://github.com/milkv-duo/duo-buildroot-sdkDatasheets & schematics:https://github.com/milkv-duo/duo-filesThis post which gave a starting point:https://forum.sophgo.com/t/use-opensbi-to-boot-your-own-operating-system/340I was able to build the FSBL with mainline GCC 14.1, but that took quite a few changes in various files, so I suggest sticking to the toolchain they provide (which unfortunately is based on GCC 8 ), unless you are brave enough. It takes a while.Note that recent GCC now includes the T-head extensions in mainline, I guess with GCC 8 this was patched:https://gcc.gnu.org/onlinedocs/gcc/RISC-V-Options.htmlThe T-head extensions are described here:https://github.com/T-head-Semi/thead-extension-spec/tree/masterSo with that info and the datasheet, I should be able to write baremetal stuff. I should receive the boards in a week or so, I'll see how it goes.The second core should also be freely usable, as far as I got it, they share the same resources, and apart from the fact it doesn't have the vector extension and runs at 700MHz max "only", it's pretty much the same core. In the SDK distribution with Linux on the first core and FreeRTOS on the second, they reserve the upper 768KB of DRAM for the second core, but I think there shouldn't be any problem sharing RAM differently when going baremetal. All this looks pretty cool.There's even a third core, 8051-based, but which can run at up to 300 MHz. Not sure I'll bother with that for now, but it's also described in the datasheet.Also, it should be able to boot from eMMC or even from a SPI flash chip. Not sure yet how you'd program the flash chips though, at least initially. Probably before assembly? Haven't gotten into all details about the ROM bootloader. It's not documented in details, just a few sentences.

author: Nominal Animal
author_rank: Super Contributor
timestamp: June 03, 2024, 12:56:25 pm
content: Do keep us informed about your experiences; I for one am very interested, and might go the same way.The main use case for myself is, once again, as an intelligent controller for small displays to use with headless SBCs ("IoT"'y servers) and routers and switches (non-technical human interface for status reporting).You see, if the interface is slow or sluggish, the nontechnical folks just don't trust it. It it is slick with small animations (my favourite one is having Tux, my mascot and shown as my avatar, roll its eyes in different directions when booting or checking out something), they seem to trust it much better.Iam intelligent enough to not try and convince them otherwise, since the technology is here to do the jobtheir way. I just need to learn enough to make the software needed for the job.In the Milk-V Duo case, I might even run a simple network server (tracking which machines are connected and how) on the Linux side, and the display driver on the RTOS side. Typical display sizes I use are 320×240 and 480×320, mostly IPS panels from BuyDisplay/EastRising; I don't want the expensive HDMI stuff here. The simple Linux network server would make it easy to add support for e.g. some ESP32's with temperature and humidity sensors to measure otherwise hard-to-reach nooks and crannies in a big rural house, for example. It's just that I'm finding it difficult to find/get SD cards I can trust, so would prefer eMMC or something more durable.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 03, 2024, 10:40:47 pm
content: I will. I'll have to confirm with actual tests, of course, but so far I think there should be no problem using this chip as a MCU on steroids.The only thing that lacks documentation - that I could find - is the TPU, if you're at all interested in using that at a low level as some kind of "general-purpose" accelerator.One other annoyance (albeit minor) is that this chip has 2 C906 cores, which are good overall but were designed when the vector extension was not ratified yet. So it implements rvv 0.7.1, while rvv has been ratified and is 1.0 now, and recent binutils only support rvv 1.0. That bites as there are a number of actual chips on the market that have implemented rvv 0.7.1 and they are now "let down" by the toolchains - you'd have to stick to a pretty old version, and find it. This has been discussed already a couple years ago (found discussions about it with Bruce), so this is not a new problem. From what I know, so far, very few actual, available chips implement rvv 1.0. There is the newer K230 which apparently does, but I haven't had all that great experience with Kendryte in terms of support (with the K210), so I would personally shy away from Kendryte at this point.Of course, the chip is perfectly usable without the vector extension. Alternatively, one can either find a version of binutils that supports rvv 0.7.1 and just use 'as' for assembly code which uses rvv and still use a recent and mainline GCC/binutils for the rest. Or implement your own assembler.Regarding displays, yes, note that the CV1800B doesn't support any video output, only input. But of course you can drive TFT panels which have a SPI controller. Note that the chip doesn't have any kind of "parallel"/FSMC interface, so you either have SPI, UART or SDIO. (It has 2 SDIO controllers, one is used by the SD card, but I'm thinking of using the second for communicating with a FPGA. You can otherwise use it to connect another flash chip, SD card or even modules with a SDIO interface such as some WiFi modules.)Yes in your case, going "baremetal" may not be the most efficient approach: using the provided Linux on the main core and FreeRTOS on the second will work, and it's readily supported.For using eMMC, from what I gathered, the board has a footprint (not populated) for a Flash chip that can be used instead of a SD card - note that the pins are routed in parallel, so it's either one or the other, they share the same SDIO controller. And you'd need a eMMC chip with a SDIO interface, from what I got. They give a list of supported chips in the following file: duo-files/duo/hardware/CV1800B-Flash-Support-List.pdfAs I said, I don't know yet how you can flash it this way, as one can't solder a Flash chip and boot from a SD card at the same time. Either you'd have to flash the chip beforehand with an external programmer, or maybe there's a USB mode in the bootloader that I don't know how to use yet. Something to figure out.

author: brucehoult
author_rank: Super Contributor
timestamp: June 03, 2024, 11:18:48 pm
content: Quote from: SiliconWizard on June 03, 2024, 10:40:47 pmNot if by "recent" you mean gcc 14.1, binutils 2.42 which support both RVV 1.0 and RVV 0.7.1 (under the name XTHeadVector).You can choose which version you want using command line -march.Even better, if you write RVV code using the C intrinsic function interface then the same source code can be compiled to either 0.7.1 or 1.0.The only missing feature is auto-vectorisation doesn't support 0.7.1.QuoteYes, literally billions of chips shipped according to THead, and on Linux-capable boards priced as low as $5 (such as the Duo, but also Pine64 ox64 and Sipeed M1s (BL808 chip) both of which also have 64 MB RAM.0.7.1 is, obviously, not quite as good as 1.0, otherwise we wasted 2 1/2 years of committee time between those releases, but it's still a darned good vector ISA, and better than SIMD ISAs. Probably as good as Arm's SVE.Meanwhile, what's the cheapest thing you can get SVE (spec published in 2016) or SVE2 (2019) in? No SBC that I know of -- not the RK3588, for example. High end phones only.QuoteNot a problem, I've had the relevant GCC/binutils snapshot athttps://github.com/brucehoult/riscv-gnu-toolchaineven since the first C906 boards came out in 2021.But now you can simply use current GCC.QuoteK230 was the first, with the CanMV-K230 board released back in October. It's got two cores with the larger (1.6 GHz, dual issue C908) one supporting RVV 1.0. There are a couple of annoyances with it, the big one being that it's only got 0.5 GB RAM, the small one that the vendor OS runs Linux on the small (non RVV) core reserving the big core for microcontroller/realtime use. 3rd parties had figured out how to run Linux on the big core, with RVV enabled.https://code.videolan.org/Courmisch/k230-boot/See also this excellent video which demos running RVV 1.0 code on the CanMV-K230:youtube.com/watch?v=Ozj_xU0rSyYThe BananaPi BPI-F3 started getting delivered to customers in the last month. It has 8 dual-issue cores running at 1.6 GHz (and reportedly better than Arm A55 or SiFive U74) each with RVV 1.0. Also the price is good at $63 for a board with 2 GB RAM or $75 for a board with 4 GB RAM.https://www.aliexpress.us/item/1005006921744822.htmlThe problem is I don't regard 4 GB as enough for an 8 core system. The SoC supports 16 GB RAM, which would be great, but boards with this are not (yet) available.There are a couple of faster boards coming up using SiFive P550 cores. They will be comparable to RK3588 boards except they DON'T have RVV.Late in the year the SG2380 boards with 16 SiFive P670 cores with RVV 1.0 should appear. Those are OoO with roughly Arm A78 performance, including around 2.5 GHz clock for the P cores. That should make a really usable PC, getting up into early i7 territory I think.QuoteYup. Been doing that for years.But now just use the current GCC version, 14.1.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 03, 2024, 11:42:35 pm
content: Quote from: brucehoult on June 03, 2024, 11:18:48 pmAh thanks! Didn't realize that was what the XTHeadVector implemented. Great. I'm using GCC 14.1 now, so that's all good. Solved.

author: brucehoult
author_rank: Super Contributor
timestamp: June 03, 2024, 11:55:08 pm
content: Quote from: SiliconWizard on June 03, 2024, 11:42:35 pmThere are a couple of minor differences:- XTHeadVector doesn't include the "ediv" feature from 0.7.1 (which was optional and C906/C910 don't implement)- XTHeadVector adds the "vlenb" CSR (vector register length in bytes), which was not in the 0.7.1 spec.I think that's it!

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 14, 2024, 08:19:51 pm
content: I should have received the boards almost a week ago, but the package got stuck at a transit facility for an unknown reason. Annoying.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 24, 2024, 08:51:16 am
content: I finally received the boards and got some good news, more on that later. The baremetal approach does work just fine.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 25, 2024, 06:13:30 am
content: So, some results so far:As shown in the post I linked to, but not really explained, the approach here for going baremetal is to build OpenSBI and have it start your own code, in place of u-boot. This is the easiest and fastest way to go baremetal. The DDR controller and all clocks get configured for you. You don't need u-boot, and going via u-boot would probably make it not just longer to boot, but harder, unless you know u-boot very well and are meaning to write some kernel that resembles Linux.The boot time from power on to your own code, as I measured using the integrated 'time' CSR, and giving a result consistent with what I timed manually, is, in my case, 1816 ms. So, that's just the CPU booting and OpenSBI running to completion. Not bad, but I was expecting shorter. It's probably possible to get shorter boot time by configuring OpenSBI appropriately, for instance by disabling the console output, which is configured for UART output @115200 Bd, so that's kinda slow.There's a tool in the SDK (as also shown in the linked post) that generates a 'fip.bin' file which amalgamates OpenSBI and your own code (or u-boot), and additionally the code for the second core. For the SD card, you just to format it with a primary FAT32 partition, and copy the fip.bin file in the root of it. I have created the partition as a 128 MB one with the boot flag, and labeled it 'boot', as is done my milkV too, but I'm not sure this is at all required. Possibly all you need is a FAT32 partition as the first primary partition, and that's it. But anyway, 128 MB is more than enough to fit this file, knowing that you only have 64MB of RAM in total anyway. You can always create other partition to store more data, as they do for Linux.I measured the current draw from USB (so @5V) with my test program running on the main core. I get an average of about 70 mA. Note that the second core is (for now) unused, and I don't know whether it's parked in a low-power state, or if it runs an infinite loop, or whatever. I don't know the intricacies of OpenSBI on this chip quite yet. Also note that the board uses only buck converters (and not LDOs) for generating all voltages, so this is more or less really the power that the chip draws. Plus the LED that I'm flashing.My first test was freezing the CPU once OpenSBI started my code, and I finally found out why (no debugger with this...): the generated code used some vector (from rvv 1.0) instructions for optilmizing memory copy operations, because I had forgotten to remove the 'v' in the march options, using GCC 14.1. So it was just triggering an exception when hitting the first vector instruction. (This is done by GCC's optimizer, at no point I was explicitely using vector instructions. So, something to know.)I use a riscv64-unknown-elf GCC.These are the compiler options I use:-mabi=lp64d-march=rv64imafdc_xtheadcmo_xtheadsync_xtheadvector_zicsr_zifencei-mcmodel=medanyYou'll need the medany memory model, because with this chip (like with many other 64-bit RISC-V CPUs), the DDR RAM lies beyond the 2GB limit.So you'll need a GCC compiler that was built with support with medany. Not all riscv64-unknown-elf toolchains support it, so be careful, in particular if you build your own toolchain.Other than that, the CV1800B datasheet is decent. There's a lot that is missing (it's only 692 pages for such a complex chip), but there's enough to use all peripherals. The only thing that I've found missing so far, unless I missed it, is the 'pinmux' registers that are not documented. But you'll find the "doc" in form of source code (good luck) in the buildroot SDK.Some additional random notes:The SPI peripherals only support master mode. I2C supports both master and slave, though.There are 8 32-bit timers, but only downcounting.The initial boot process is not documented. There are barely a few lines to hint at how we can boot from SPI Flash instead of SD, but that's pretty much it.I have tested GPIOs and UART in baremetal code so far, both working as expected. I tried reading the temperature sensors, but I only get 0 so far for the readings. I haven't found code about them in the SDK, so not sure (or almost sure not) they are using them with Linux. Maybe the clock for them is not enabled by default. I'll have to figure it out.Also, for those curious, this is the console output you get initially from OpenSBI:Code:C.SCS/0/0.WD.URPL.SDI/25000000/6000000.BS/SD.PS.SD/0x0/0x1000/0x1000/0.PE.BS.SD/0x1000/0xc400/0xc400/0.BE.J.FSBL Jb28g9:g34fb84e8-dirty:2024-06-01T08:27:38+02:00st_on_reason=d0000st_off_reason=0P2S/0x1000/0x3bc12400.SD/0xd400/0x1000/0x1000/0.P2E.DPS/0xe400/0x2000.SD/0xe400/0x2000/0x2000/0.DPE.DDR init.ddr_param=0x78075562.pkg_type=3D3_1_4DDR2-512M-QFN68Data rate=1333.DDR BIST PASSPLLS.PLLE.C2S/0x0/0x0/0x0.No C906L image.MS/0x10400/0x80000000/0x22000.SD/0x10400/0x22000/0x22000/0.ME.L2/0x32400.SD/0x32400/0x200/0x200/0.L2/0x414d3342/0xcafe04ed/0x80200000/0x1a00/0x1a00COMP/1.SD/0x32400/0x1a00/0x1a00/0.DCP/0x80200020/0x1000000/0x81500020/0x1a00/1.DCP/0x23e0/0.Loader_2nd loaded.Use internal 32kJump to monitor at 0x80000000.OPENSBI: next_addr=0x80200020 arg1=0x80080000OpenSBI v0.9____ _____ ____ _____/ __ \ / ____| _ \_ _|| | | |_ __ ___ _ __ | (___ | |_) || || | | | '_ \ / _ \ '_ \ \___ \| _ < | || |__| | |_) | __/ | | |____) | |_) || |_\____/| .__/ \___|_| |_|_____/|____/_____|| ||_|Platform Name : Milk-V DuoPlatform Features : mfdelegPlatform HART Count : 1Platform IPI Device : clintPlatform Timer Device : clintPlatform Console Device : uart8250Platform HSM Device : ---Platform SysReset Device : ---Firmware Base : 0x80000000Firmware Size : 160 KBRuntime SBI Version : 0.3Domain0 Name : rootDomain0 Boot HART : 0Domain0 HARTs : 0*Domain0 Region00 : 0x0000000074000000-0x000000007400ffff (I)Domain0 Region01 : 0x0000000080000000-0x000000008003ffff ()Domain0 Region02 : 0x0000000000000000-0xffffffffffffffff (R,W,X)Domain0 Next Address : 0x0000000080200020Domain0 Next Arg1 : 0x0000000080080000Domain0 Next Mode : S-modeDomain0 SysReset : yesBoot HART ID : 0Boot HART Domain : rootBoot HART ISA : rv64imafdcvsuxBoot HART Features : scounteren,mcounteren,timeBoot HART PMP Count : 16Boot HART PMP Granularity : 4096Boot HART PMP Address Bits: 38Boot HART MHPM Count : 8Boot HART MHPM Count : 8Boot HART MIDELEG : 0x0000000000000222Boot HART MEDELEG : 0x000000000000b109

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 26, 2024, 03:14:05 am
content: Some additional facts.I managed to make the temperature sensor work. I was initializing it correctly, except for one thing: I was passing "3" for the channel selection (tempsen_ctrl register, see datasheet of the CV1800B), as was documented, to enable both sensors.Looking for some clues, I found this patch:https://lore.kernel.org/linux-kernel/20240531-oyster-duct-9eaf5f3e6f35@spud/T/(which hasn't made it to the milkv sdk yet, that I know of). The driver contains code for converting the read values in °C, which the datasheet does not document, so that was handy too.They do pass "1" for the channel selection. They seem to use only 1 channel anyway. Since everything else was exactly the same, I tried passing 1 instead of 3, and bam, it worked. So, I'm assuming there's either a fault in the datasheet, or this is a silicon bug. What leads me to think of a silicon bug is that the interrupt flags (in tempsen_intr_raw register) show that both channels 0 and 1 have completed a measurement, yet the second channel only ever returns 0. When passing 3 as the channel selection, both channels return 0 and the interrupts flags are all 0. So, yeah. If anyone reads this and has worked on the Linux driver, and has a clue about that... (I guess they had access to documentation/info that is not in the public documents?)Note that the SG2000/2002 are very similar to the CV1800B (that I'm assuming Sophgo bought, and then released new chips under their own name), and the docs are there:https://github.com/sophgo/sophgo-doc/releases/These docs are nicer to navigate, and while very similar in content, they at least document the pinmux functions, while the CV1800B DS did not. So, that'll help. But regarding the temperature sensors, they document that in the exact same way as in the CV1800B. So, I dunno. To see if both channels actually work in the SG2000/2002, I'd have to get a board with one of these (Duo 256M, Duo S).Regarding power consumption, I got about 0.35W on average, but the given figure of 0.5W with video input active and the TPU seems to be about right. Conversely, all clocks seem to be enabled by default, so disabling clocks for peripherals that aren't used should lower power consumption significantly. Something I'll try.Back to temperature, the internal temp sensor gives me around 46.5°C (for 25°C ambiant) after 1 hour or so. So that shouldn't require any heatsink whatsoever. Maybe if you intend to use it in very high temperature environments, you could stick a small passive heatsink and call it a day. Note that the internal temp sensor peripheral implements thresholding and that can be tied to interrupts, or even just reset the CPU in case the temp is outside the thresholds you defined.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 27, 2024, 03:55:02 am
content: I've run Bruce's countPrimes() on the main core. Here's the result:Code:37.918 sec CV1800B C906 @ 1.0 GHz ??? bytes 32.2 billion clocksI also computed the average CPI for this function and got: 1.147. (The code is very branchy.)Not bad at all. It sits between the SiFive U54 and SiFive U74.The discrepancy between the execution time and number of clock cycles indicates that either the CPU clock is not set for 1 GHz exactly, or that the 'time' CSR that I use for timing execution is not accurate. (Although I doubt the latter as it's clocked from the 25 MHz crystal oscillator.)

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2024, 05:06:27 am
content: That's fairly much in line with the original AWOL Nezha D1 dev board:36.652 sec Allwinner D1 C906 RV64 @ 1.008 GHz 224 bytes 36.9 billion clocksThat was run in March 2021 via ssh to an early board in China, so a fairly old gcc version (though the benchmark isn't very susceptible to compilers or optimisation).

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 27, 2024, 05:40:24 am
content: Yeah, trying to figure out clock settings. The results I got seem to indicate that the core runs @ 850 MHz.My code, as I explained, runs after OpenSBI, and I'm not sure how they set up clocks at this stage, or if they leave them as default.And the datasheet, regarding clocks, is uh, not very explicit.My best bet might be to have a look at the Linux code and how they handle CPU clock frequency.Edit: OK, I think I found it. They set clocks in the FSBL (first-stage bootloader). Can be seen in: duo-buildroot-sdk/fsbl/plat/cv180x/platform.c , in sys_pll_nd() , line 317.They set the first core at 850 MHz, and the second at 594 MHz. I don't know what the rationale for these frequencies is. I'm curious.In fact, the first core is clocked from the MPLL, which is set at 850 MHz, while the chip's default for MPLL is 1000 MHz. Again, why, it's anybody's guess.But the source code of this function has all I need to change frequencies.So, while it's the number of clock cycles that matters for your benchmark, please change the "1 GHz" value to "850 MHz" to make it correct. I'll re-test when I manage to switch to 1 GHz. I may also try "overclocking" the core to see what happens.

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2024, 08:09:18 am
content: Quote from: SiliconWizard on June 27, 2024, 05:40:24 amAhh, so the lower number of clocks may be correct if, for example, your compiler is using the THead extensions. Though 15% lower is a lot.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 27, 2024, 11:18:47 pm
content: Just to make sure it was clear: I use the cycle CSR to count cycles and the time CSR (which on this chip is clocked from the main 25 MHz oscillator) to count time. So, the number of cycles (+/- obviously a few cycles) is exact and does not rely on timing. This is also why I got a time that wasn't consistent with the number of cycles (that should have been @ 1 GHz), which is how I figured that the CPU was not running at 1 GHz, but 850 MHz. Later confirmed by the above code.So, the CPU clock frequency shouldn't change the number of cycles whatsoever. Just the time. And at 1 GHz, it will be about 32.2 s.I looked at the disassembled code, and no vector extension is used, nor anything specific to C906, as far as I saw. So, either the C906 is more efficient on this chip than on the D1, or it's just a matter of compiler optimizations. Which is likely. Note that I got this result at -O3. I can check with -O1 for a comparison.Otherwise, the differences will mainly be about cache differences, I guess. Also, the C906 may not be configured exactly the same way as in the D1 for some other parameters. Might be worth rechecking on the D1 with GCC 14.1, but before that, I'll test at -01 and may get the same result as you saw for the D1.Result with -O1: 39.1 billion clocksResult with -O2: 32.2 billion clocksSo, definitely a significant difference here between -O1 and -O2 / -O3, with GCC 14.1. There is almost no difference between -O2 and -O3, although very slightly fewer clocks at -O2 (but less that 1% difference).

author: brucehoult
author_rank: Super Contributor
timestamp: June 27, 2024, 11:56:56 pm
content: Wow that 39.1 with -O1 is weirdly more than the 36.9 on the D1 in 2021.There won't be any vector instructions, but things from Zba such as `sh2add` (`(rs1 << 2) + rs2`) make a noticeable difference to array addressing on SiFive's U74 -- see the difference between the HiFive Unleashed and the VisionFive 2.The `xtheadba` extension (supported in gcc 13 and later) has the similar `th.addsl rd, rs1, rs2, imm2` instruction, which does `rs1 + (rs2 << imm2)`https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=8351535f20b52cf332791f60d2bf22a025833516

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 28, 2024, 12:14:53 am
content: Nope, it seems to only use instructions from the base RV64IM. Unless I missed something.You can check for yourself, with the same options I used:https://godbolt.org/z/Ef4TY844xand compare with -O1.

author: brucehoult
author_rank: Super Contributor
timestamp: June 28, 2024, 01:16:02 am
content: Quote from: SiliconWizard on June 28, 2024, 12:14:53 amYou do have to add the relevant extensions :-)https://godbolt.org/z/1KExjPfaaFor some unknown reason gcc uses the Zba instructions later on in the .L9 block too, but not for XTHeadba.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 28, 2024, 02:12:24 am
content: I'll add this in the list of extensions.Note that it may explain the difference between the D1 and CV1800B at -O1 (if this extension was enabled when testing on the D1), but GCC doesn't seem to use this instruction at -O2 and -O3.I'll do some testing with this enabled, to see what I get.So, with xtheadba and xtheadbb enabled, -O1, I get: 39.06 billion cycles (rounding to 2 decimals just because my previous result was slightly above 39.1, so there is a difference, but extremely small.)Would be worth retesting on the D1 with GCC 14.1 just to see. If the compiled code is the same, then one possible reason is a difference with caches.At -O3, GCC uses th.ext/th.extu from xtheadbb, and it does make a very small difference too. Now down to 32.1 billion clocks:Code:32.144 sec CV1800B C906 @ 1.0 GHz ??? bytes 32.1 billion clocksI managed to change the CPU clock to 1 GHz. The temperature increases by about 1°C. Haven't checked the power draw, I do expect a corresponding increase (about 80 mA @ 5 V).

author: brucehoult
author_rank: Super Contributor
timestamp: June 28, 2024, 04:15:18 am
content: Of course this is just fiddling around the edges. It is possible to construct a loop wheresh2addand friends (similar toleaon other ISAs) allow a loop to run 30% faster  but that's very much an outlier and most code will see just 1% or 2% speedup.https://godbolt.org/z/jMofYTd8aStill, the Zba and Zbb instructions are cheap to implement (exceptclzandcpop, which also provide the biggest speed gains in code that needs them).I have a suspicion that a modern 64 bit adder has lower worst-case latency than a 64 bit barrel shifter, and taking the adder input from after the first two shift stages (i.e. shifting all normal adds by 0, twice) might not add any latency to the ALU stage. assuming thatmulis fast. Replace it withaddorsllor something if you want.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 28, 2024, 05:34:15 am
content: I tried 1.2 GHz for the main core and it appears to run absolutely fine. Temperature is ~ 1.4°C higher than @ 1 GHz. Shall I try 1.5 GHz?Well, for the record: the CPU freezes @ 1.5 GHz. I tested a couple intermediate values, 1.25 GHz seems to be the highest it can go in a stable way.@ 1.25 GHz, the primes benchmark takes 25.715 s.

author: brucehoult
author_rank: Super Contributor
timestamp: June 28, 2024, 07:29:38 am
content: Oh nice. That puts it faster than the dual-issue Teensy 4.0 (Cortex M7) overclocked to 960 MHz -- which is NOT running at all cool at that speed. (and has 1 MB RAM, and costs $23.80 these days)

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 28, 2024, 07:54:07 am
content: It stabilizes @ 47°C @ 1.25 GHz.We may be able to make it work at higher frequencies by tweaking the core voltage, which is adjustable via PWM.But anyway - that's nice as it is. This chip is less than $4 in low quantities and probably less than half that in higher quantities, imagine that.

author: brucehoult
author_rank: Super Contributor
timestamp: July 01, 2024, 01:25:13 pm
content: Original 64 MB Duo is now $2.99!https://arace.tech/products/milk-v-duoI hope that's not an EoL sale.And in other news Milk-V Jupiter -- SpacemiT K1 (4 GB or 8 GB) or M1 (16 GB) 8 core dual issue 1.6 GHz with RVA22 + 256 bit Vector in MiniITX form factor -- is now ready to preorder with shipment promised within 30 days. They demo it with an RX 550 GPU in the PCIe slot, so you can get a proper accelerated desktop without the ImgTech driver rubbish.$59.90 for the 4 GB, to $115 for the 16 GBhttps://arace.tech/products/milk-v-jupiter-spacemit-m1-k1-octa-core-rva22-rvv1-0-risc-v-soc-2tops-miniitxOh, and the M1 chip is 2.0 GHz instead of 1.6 GHz and comes in a metal case that will give better cooling (they say the junction temperature is 20 C lower than the K1 under the same load WITHOUT any additional heatsink) and perhaps overclocking potential.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 01, 2024, 10:20:37 pm
content: Saw that for the Duo, I'll order a bunch more.Note that the Duo 256M is based on a SG2002 which is an improved CV1800B (while very similar), with more cache, a Cortex A53 core (that I personally wouldn't use, and am wondering whether it draws, even from just leakage, some power even if not used), more RAM in DDR3 instead of DDR2 (much faster), and video output. But for baremetal uses I have in mind, the CV1800B is fine and 64MB is already a lot of RAM compared to what you have with any MCU.The NXP iMXRT10xx, which support PSRAM in QSPI, only support up to 16MB, and I had benchmarked it (on a Teensy 4.1) - it's approx. 30MB/s sustained, read and write almost the same figure. So that's pretty slow. The DDR2 in the CV1800B is much faster, and all inside a single package.And the Milk-V Jupiter looks interesting.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 01, 2024, 10:43:05 pm
content: A couple more random points regarding my baremetal journey with the Duo. As I said before, you get "kickstarted" by OpenSBI. What that means is:- Your code will start in Supervisor mode.- You have access to SBI calls (although I haven't tested that yet).I found out that OpenSBI protects the area for the CLINT registers, so that you can't access them in your own code. That looks like making sense in the general case, as those are mainly MTIMECMP registers (and those for software interrupts), normally only accessible in machine code. But the C906 core adds STIMECMP registers (and same for software ints) that you can use in S-mode. Due to the PMP protection made by OpenSBI (which protects the whole CLINT area), you can't access these STIMECMP registers in S-mode. That kind of bites. It's more efficient, for instance, for implementing some RTOS, than having to resort to peripheral timers and going through the PLIC.There is an SBI call to configure the MTIMECMP registers, but that call mechanism kind of defeats the benefits you get using the CLINT directly. Note that exceptions and interrupts are delegated to the S-mode.I have tested configuring a peripheral timer and the PLIC, and an interrupt handler, and it does work as expected. Keep in mind you are in S-mode, so you'll have to use sret instead of mret to return from exceptions/interrupts. The GCC attribute for writing such handler in C is:Code:__attribute__((interrupt("supervisor"))).Now of course, one next step could be to bypass OpenSBI entirely, and start at a lower level. That's probably going to yield a shorter boot time, and access to M-mode. But, one would have to configure the DDR controller, caches, etc.

author: brucehoult
author_rank: Super Contributor
timestamp: July 02, 2024, 01:31:41 am
content: Quote from: SiliconWizard on July 01, 2024, 10:43:05 pmIncorrect.Configuring clocks, DDR etc is the job of U-Boot SPL. By the time you arrive in SBI (or if you simply replace SBI with your own M mode code) you have a fully functioning machine.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 02, 2024, 02:58:40 am
content: Quote from: brucehoult on July 02, 2024, 01:31:41 amAh, I haven't looked at the details of the boot process yet, I was assuming this was part of OpenSBI.I'll have a deeper look when I work on this step. So that would mean we can take over just before OpenSBI is loaded, and still have some basic hardware configuration. I'll see how they did it exactly, and how to generate the corresponding binary.There is a bl1, bl2 and the part I generate for now is what they call bl33.

author: brucehoult
author_rank: Super Contributor
timestamp: July 03, 2024, 11:54:49 am
content: Well, so I finally took the Duo I bought when they came out  out of its box and tried it. Took less than 30 minutes and I had to do the whole find software, flash it, enable RDNIS etc routine.Quick results with my primes and memcpy (with and without RVV) benchmarks:https://www.reddit.com/r/RISCV/comments/1du9qnf/comment/lbfcuvi/It's running at (as previously reported in this thread) 850 MHz, and the buildroot Linux has only been allocated 32 MB RAM, and I haven't expanded the disk partition yet.If you want me to try anything, the board is running :-) Also happy to flash another SD card with any custom image you might come up with. there are also in unopened boxes two Duo S (one with Wifi/BT, one without), a Mars, a Star64, a Pi Pico, a CH32V003 kit, a couple of ox64, a LicheeRV Dock, a few Longan Nano, an Orange Pi 5 Plus, a Pi 5, a CanMV-K230 ... yes I have a problem. The good news is that when I find time & inclination to play with one of them I can satisfy the itch instantly, as I just did with the Duo, not wait weeks. I do have a number of boards in near-daily use, including 4 and 8 GB VisionFive 2, and 8 and 16 GB Lichee Pi 4A, and original HiFive1.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 04, 2024, 02:12:36 am
content: "Interesting" that the CPU clock is still 850 MHz when running Linux. I would have expected it to be bumped to 1 GHz by Linux. I have stuck to working baremetal on it, not even tried Linux yet, so I can't say, but isn't there a way to configure the CPU clock from Linux on this? Still, that's good to know that 850 MHz is the default. That means that users who wouldn't care checking would be misled into thinking that it runs @ 1 GHz, as they put this 1 GHz figure everywhere. Why it's not the default, nobody knows (yet). Were they afraid that it would only be marginal already @ 1 GHz? That may not be completely wrong, I don't know. My tests show that it's stable up to 1.25 GHz (but keep in mind I haven't tested it extensively at this frequency), but completely freezes beyond that (I tested 1.35 GHz and it failed miserably.)Anyway, I've done some more performance testing (still in baremetal). I've run a FFT test: double-precision FP, 512 points, FFT written in vanilla C without any trick. The FFT takes 107 µs @ 1 GHz (107162 cycles), with an average CPI of 1.55.The FFT has 0.5*N*log2(N) iterations, so that's approx. 46.5 cycles/iteration. There is: 8 memory accesses per iteration, 8 multiplications, and 8 additions. Plus of course index management and loops.That can probably be optimized of course.I think Bruce said rvv 0.7.1 on this chip didn't support 64-bit numbers, so that would mean we can't use vector instructions to double the speed of double-precision FP? Haven't looked into this yet. But my goal was mainly to benchmark double-precision FP. I'll test with single-precision too.

author: brucehoult
author_rank: Super Contributor
timestamp: July 04, 2024, 02:31:31 am
content: Quote from: SiliconWizard on July 04, 2024, 02:12:36 amIt does. That was the next thing I checked. It may be that very early versions of the C906 (some Allwinner D1? All of them?) didn't, but the Duo certainly does...Code:bruce@i9:~$ cat vadd64.c#include <stdio.h>#include <stdint.h>void vadd64(uint64_t *t, uint64_t *n, uint64_t loops);uint64_t t = {0,0};uint64_t n = {1,1000};int main(){vadd64(t, n, 1000*1000*1000);printf("%lu %lu\n", t, t);return 0;}bruce@i9:~$ cat vadd64.s.globl vadd64vadd64:beqz a2,2fli t0,2th.vsetvli zero,t0,e64,m1th.vle.v v0,(a0)th.vle.v v1,(a1)1: th.vadd.vv v0,v0,v1addi a2,a2,-1bnez a2,1bth.vse.v v0,(a0)2: retbruce@i9:~$ riscv64-unknown-elf-gcc -march=rv64gc_xtheadvector -O vadd64.c vadd64.s -o vadd64bruce@i9:~$ scp vadd64 duo:vadd64 100% 99KB 3.7MB/s 00:00bruce@i9:~$ ssh duo time ./vadd641000000000 1000000000000real 0m 4.81suser 0m 4.72ssys 0m 0.00sbruce@i9:~$Changingtandnand the corresponding prototype arguments from uint64_t to double (and the printf to%fobv) and theth.vadd.vvtoth.vfadd.vvworks too, and takes 1.2 seconds longer. That's 5 cycles per loop vs 4 for the 64 bit integer.Code:1000000000.000000 1000000000000.000000real 0m 6.00suser 0m 5.91ssys 0m 0.00s

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 04, 2024, 02:37:37 am
content: Great. Gives me an incentive to implement the FFT in assembly using rvv.IIRC (that was a few years ago already), about 100 µs was what it took the FFT accelerator in the K210 to compute a 256-pt FFT, in 16-bit resolution only. And a 512-pt FFT is 2.25 times the number of iterations. So that means we get a 2.25 times faster FFT, with 64-bit FP, compared to 16-bit on the K210 with its accelerator.Just tested the same FFT, but in single-precision FP. Takes 87 µs, and an average CPI of 1.27.Here's what the core of the inner loop looks like (just the computation part, outside of that, there's mainly index calculation and branching):Code:fldfa2,0(a3)fldfa5,8(a4)fldfa3,8(a3)fldft0,0(a4)fmul.dft1,fa2,fa5fmul.dfa1,fa3,fa5fldfa4,0(a5)fldfa5,8(a5)addia5,a5,16adda3,a3,a1addia4,a4,16fmadd.dfa3,fa3,ft0,ft1fmsub.dfa2,fa2,ft0,fa1fadd.dft0,fa5,fa3fadd.dfa1,fa4,fa2fsub.dfa5,fa5,fa3fsub.dfa4,fa4,fa2fmul.dfa2,ft0,fa0fmul.dfa3,fa1,fa0fmul.dfa5,fa5,fa0fmul.dfa4,fa4,fa0fsdfa2,-8(a5)fsdfa3,-16(a5)fsdfa5,-8(a4)fsdfa4,-16(a4)

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 06, 2024, 08:49:36 am
content: Working with the XtheadVector extension. And, damn. I had a good use for the 'th.vrgather.vv' instruction, namely to swap parts of a vector register.GCC "supports" it when the xtheadvector extension is enabled. But the CV1800B triggers an illegal instruction exception on it.To me, it's unclear why it's not implemented and why GCC let it pass - possibly developers were not aware either that the instruction was not available.Here is the short doc:https://github.com/XUANTIE-RV/thead-extension-spec/blob/master/xtheadvector.adocThey mention: "No whole register move instructions." - Do you think this instruction would be part of what they call whole register move instructions? Puzzled.The encoding is as follows:325002d7 th.vrgather.vv v5,v5,v0

author: brucehoult
author_rank: Super Contributor
timestamp: July 06, 2024, 09:13:49 am
content: No. "Whole register move" refers to RVV 1.0 instructions such asvs8r.v v8, (a1)which storesv8tov5in memory, starting at the address ina1, ignoringvtype,vlenetc. This is for compilers spilling and reloading registers, or the operating system saving and restoring the vector state.RVV 0.7.1 can do this, but only by first saving the CSRs, then setting vl and lmul and using regular register store. It's a couple of extra instructions which is ok for the OS< but inconvenient for e.g. function prolog/epilog.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 06, 2024, 09:29:40 am
content: Do you have any idea for the vrgather instruction? Or a way to achieve the same as efficiently (I just want to swap the two 64-bit parts of a v register for now.)

author: brucehoult
author_rank: Super Contributor
timestamp: July 06, 2024, 09:49:46 am
content: It works for me on the Duo, in the sense that it doesn't crash. However it is slightly buggy ... damnit Edit: not it's not buggy, the mistake was mine :-( See later message.Code:bruce@i9:~$ cat gather.c#include <stdio.h>// max len VLENB*8 i.e. 128 chars on C906, C910void gatherb(char *dst, char *src, char *indexes, long len);int main() {char r;char indexes[] = {1, 17, 20, 2, 4, 26};gatherb(r, "abcdefghijklmnopqrstuvwxyz", indexes, 26);printf("|%s| %d %d %d %d %d\n", r, r, r, r, r, r);return 0;}bruce@i9:~$ cat gather.s.globl gatherbgatherb:th.vsetvli zero,a3,e8,m8th.vle.v v8,(a1)th.vle.v v16,(a2)th.vrgather.vv v0,v8,v16th.vse.v v0,(a0)retbruce@i9:~$ riscv64-unknown-elf-gcc -march=rv64gc_xtheadvector -O gather.c gather.s -o gatherbruce@i9:~$ scp gather duo:gather 100% 99KB 2.8MB/s 00:00bruce@i9:~$ ssh duo ./gather|bruce| 98 114 117 99 101What is buggy? It should have worked passing 5 forlen, not 26. But that gives:Code:|b| 98 0 0 99 101They missed in the spec: "If the element indices are out of range ( vs1 ≥ VLMAX ) then zero is returned for the element value." They returned 0 ifvs1 ≥ vl:-( :-(And YOU missed in the spec: "For any vrgather instruction, the destination vector register group cannot overlap with the source vector register groups"

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 06, 2024, 09:53:01 am
content: Ahhh... Thanks a ton!

author: brucehoult
author_rank: Super Contributor
timestamp: July 06, 2024, 10:39:47 am
content: Oh, I'm dumb! There is no bug in the CPU. Or at least not this one :-)When I passed 5 as the len I was only loading the first 5 items from the lookup table. I needed separate arguments for the length of the src/dst and the length of the lookup table, and an extra vsetvli to load the table.Code:.globl gatherbgatherb:th.vsetvli zero,a3,e8,m8th.vle.v v8,(a1)th.vsetvli zero,a4,e8,m8th.vle.v v16,(a2)th.vrgather.vv v0,v8,v16th.vse.v v0,(a0)retCode:#include <stdio.h>// max len VLENB*8 i.e. 128 chars on C906, C910void gatherb(char *dst, char *table, char *src, long tablelen, long srcdstlen);int main() {char r;char indexes[] = {1, 17, 20, 2, 4};gatherb(r, "abcdefghijklmnopqrstuvwxyz", indexes, 26, 5);printf("|%s| %d %d %d %d %d\n", r, r, r, r, r, r);return 0;}Works perfectly. Tested on both Duo and Lichee Pi 4A (C910).

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 07, 2024, 07:13:31 am
content: So, my results so far are not that great. My attempt at implementing the FFT with vector instructions yields a 1.8x slower execution, compared to the compiled C (which only uses regular FP instructions).I still have a small difference in output, so there is a bug in my implementation (that I'll figure out), but this shouldn't change the exec time much - the number of iterations is correct.First thing is that one crucial part of the innermost loop is a complex multiplication, which doesn't lend itself all that well to vectorization, but of course if anyone's got a good improvement on just a complex multiplication with vector instructions compared to FP instructions, I'm all ears. Could be a challenge.Keep in mind I only tried with double-precision FP and thus, "only" have 2 numbers per vector. I certainly don't get a two-fold improvement - almost the opposite.I haven't benchmarked vector instructions in this case, in a simpler way, just to have results easier to interpret. If you're up for it...One small point about the exec time I gave earlier - RAM caching was a non-negligible part of it. That's bound to happen when your run algorithms on a significant amount of data in a very short time. (This chip "only" has 64KB of data cache.)Running the FFT twice and timing the second occurence gives me ~ 79 µs instead of 107 µs.Regarding vectorization, once I get benchmarks on the vector instructions themselves and if they are worth it, since optimizing just a single iteration of the innermost loop looks like a dead-end, one obvious approach will be to unroll this innermost loop and vectorize 2 (or 4 if using single-precision FP) iterations. My attempt was a bit naive, was hoping to get a better result than this, considering that we're working with complex calculations so on pairs of FP numbers. But, again, the multiplication... bites.

author: brucehoult
author_rank: Super Contributor
timestamp: July 07, 2024, 08:41:11 am
content: Oh, are you just implementing the textbook algorithm?That sucks for sure. It's absolutely essential to inline the bottommost several layers of butterflies into each other. Are you doing bit reversal? Are you taking advantage of 0, 1, -1 coefficients? 1/sqrt(2)? Higher?Are you familiar with FFTW? It's more than 25 years old now, but still very valuable for finding an optimal configuration for scalar computations on any particular hardware (FPU speed, OoO, cache organisation), and I think would also give a good indication of what amount of inlining to to in a vector implementation.Nick Knight and Aliaksei Chapyzhenka (commonly known on the net as "drom", active in things such as wavedrom and SDR) did a lot of work on RVV FFTs at SiFive in 2019-2020 -- I mean many months of effort. I don't know if their work was ever published or is now SiFive "secret sauce".You're of course never going to get much speedup from 2 element vectors. There isn't much potential gain and organisational overhead can kill it. Bigger LMUL (2 or 4) can hide bookkeeping overhead in many algorithms, but if you inline/unroll then there is less of that anyway.And the C906 is just a low end simple in-order CPU ... wadda ya want? Using C910 / A72 level hardware will give a lot of gain (OoO), or the SpacemiT K1 in the BPI-F3 with its 256 bit VLEN, or the SiFive X280 cores with 512 bit VLEN in the space-rated PIC64 or (in the much nearer term) in the SG2380 which should be in multiple boards around New Year.I think Arm doesn't have *anything* with more than 128 bit vector/SIMD length? The "Scalable" in SVE doesn't seem to mean anything yet -- either upwards OR downwards (they made MVE/Helium instead, a design point RVV can also cover). But the Pi Zero W (or the chip in it, if you can get it) might offer the most bang for the buck rn.All those things are a lot more than $3 of course.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 07, 2024, 08:51:53 am
content: Yep. I'll still be doing some more benchmarking, with simpler stuff. FFT is not the simplest thing to optimize. But 79 µs for a double-prec 512-pt FFT isn't too bad, and 36 µs for a 256-pt one.I'm running into something odd but unrelated, that may sound fully obvious to you. I have set up an exception handler as I said earlier. Just a plain C function with the interrupt("supervisor") attribute.It does work plenty fine... until, from what I gathered now, my code grows and the exception handler gets located at a higher address (at least, that's all I got for now). At which point, the exception handler doesn't get called anymore. But no crash or anything like that. Is there a max address for the exception handler (set with stvec) on this chip?Edit: Ouch. My silly bad. Problem is that the address ended up at an address that was aligned to 2 bytes, but not 4 (as required for exception handlers in direct mode). I added an align(4) attribute to this function. It fixes it. Obviously.So, bonus question: is a 2-byte alignment only guaranteed with RV(32 or 64) when using GCC, in all cases, or is it because I enabled the C extension?

author: brucehoult
author_rank: Super Contributor
timestamp: July 07, 2024, 09:15:25 am
content: Spec: "The mtvec register must always be implemented, but can contain a read-only value. If mtvec iswritable, the set of values the register may hold can vary by implementation."It's got to be 4-byte aligned, of course.> Is there a max address for the exception handler (set with stvec) on this chip?No idea. Read back the value you set and see if you get back the same thing. ALWAYS a good idea with WARL CSRs.

author: glenenglish
author_rank: Frequent Contributor
timestamp: July 07, 2024, 09:20:17 am
content: Fantastic discussion. Love it.I referred to this treatment at page 40 onwards when getting to know RVV.https://eupilot.eu/wp-content/uploads/2022/11/RISC-V-VectorExtension-1-1.pdfThe bit I didnt know what about what compilers try to do with register allocation. (IE if you let the compiler figure it out instead of being implicit)

author: brucehoult
author_rank: Super Contributor
timestamp: July 07, 2024, 09:23:53 am
content: Oh you edited while I was writing my reply.Sure, if you have C extension then only 2 byte alignment is guaranteed by the compiler/linker. Without C you're guaranteed 4 byte alignment.Many people add some set of -falign-functions -falign-loops -falign-labels with 4 or 8 or something to their CFLAGs to make branches cause less of a stall, especially with superscalar CPUs, but even with single-issue you might get a penalty for jumping to a misaligned 4 byte instruction -- or even a 2 byte instruction that is not on a 4 byte boundary. Depends on the bus width between L1 icache and the fetch/decode unit. It might only be instructions (4 byte only of course) that straddle an 8, 16, 64, 4096 byte boundary that need two fetches to get the whole instruction.

author: brucehoult
author_rank: Super Contributor
timestamp: July 07, 2024, 09:46:32 am
content: Some interesting data from the dav1d porting effort ... 128 bit RVV on the Kendryte K230 (first RVV 1.0 chip on the market, back in November) is very similar to NEON on the A53 in an Odroid C2 ... and the C908 core is significantly better on scalar versions of the code.Best to start a bit before that, but that's the dataThe K230 uses the THead C908 core which is dual-issue but otherwise very much appears to be just an updated C906, especially in the vector unit -- just updated from RVV 0.7 to 1.0 without any serious performance changes, it appears.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 08, 2024, 02:33:15 am
content: A more conclusive result, with something simple: a weighted sum, which lends itself very easily to vectorization.I compared a pure C version:Code:double WeightedSum(double *X, double *Y, uint64_t nPts){double Sum = 0.0;for (uint64_t i = 0; i < nPts; i++)Sum += X * Y;return Sum;}Which gets compiled as:Code:fmv.d.xfa5,zeromva5,s4mva4,s0.L708:fldfa3,0(a5)fldfa4,0(a4)addia5,a5,8addia4,a4,8fmadd.dfa5,fa3,fa4,fa5bnes0,a5,.L708With the following hand-written assembly, still using double-precision FP:Code:VectD_Zero:.double 0.0.double 0.0.global Vector_WeightedSum.type Vector_WeightedSum, @functionVector_WeightedSum:beq a2, zero, Vector_WeightedSum_Endaddi sp, sp, -16li t0, 2th.vsetvli zero, t0, e64, m1lla t0, VectD_Zeroth.vle.v v1, (t0)mv t1, a0th.addsl t3, a0, a2, 3mv t2, a1Vector_WeightedSum_Loop:th.vle.v v2, (t1)th.vle.v v3, (t2)addi t1, t1, 16addi t2, t2, 16th.vfmacc.vv v1, v2, v3bgtu t3, t1, Vector_WeightedSum_Loopth.vse.v v1, 0(sp)fld ft0, 0(sp)fld ft1, 8(sp)fadd.d fa0, ft0, ft1addi sp, sp, 16Vector_WeightedSum_End:ret(nPts must be even. It's not checked here, and I didn't bother adding code to adjust for odd values, but this is easy to add.)This time, the results are more conclusive. I tested with nPts = 1024.C version: Cycles = 6169, CPI = 1.003Assembly (vectored): Cycles = 3679, CPI = 1.188 (so about 1.7 times faster).So, even with 2 64-bit elements per vector, this is still worth it.I've since seen that one could load all zeros to a vector with the following sequence, instead of loading from memory as I did in the above piece:Code:fmv.d.x ft0, zerovfmv.v.f v1, ft0Probably much faster if the loaded "zero vector" is not in cache, otherwise possibly slower (as there are 2 instructions). Not sure which one should be prefered.

author: brucehoult
author_rank: Super Contributor
timestamp: July 08, 2024, 04:05:07 am
content: Floating point zero is all 0s, so you can just splat an integer 0, orxorthe total register with itself, or whatever.Instead of saving the total vector to memory and then loading back the individual elements to FP registers you can usevfredsum.vs. This will also make the code independent of whether you're using single or double precision, different LMUL etc. In that case you'd also want to use vsetvli properly for the loop control, to deal with any number of elements (not just even / not even).The one trick is you'd need to setvltoVLMAXbefore thevfmaccto make sure you don't inadvertently zero the tail elements of the totals. Technically you only need to do that on the last iteration (ifvl<VLMAX) but it's no great harm to do it every time. Again, the tail elements after the short final vector you loaded from the input arrays will be zeros, and so not affect the totals in those positions (but there will be wasted multiplies of 0*0).

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 08, 2024, 04:30:43 am
content: Quote from: brucehoult on July 08, 2024, 04:05:07 amI had seen this one actually, but problem being to extract the value to a FP register after that anyway. How would you do it without passing via memory?Edit: got it, th.vfmv.f.s does that - I had missed this one. So new version is this:Code:Vector_WeightedSum:beq a2, zero, Vector_WeightedSum_Endli t0, 2th.vsetvli zero, t0, e64, m1fmv.d.x ft0, zeroth.vfmv.v.f v1, ft0mv t1, a0th.addsl t3, a0, a2, 3mv t2, a1Vector_WeightedSum_Loop:th.vle.v v2, (t1)th.vle.v v3, (t2)addi t1, t1, 16addi t2, t2, 16th.vfmacc.vv v1, v2, v3bgtu t3, t1, Vector_WeightedSum_Loopth.vfmv.v.f v0, ft0th.vfredsum.vs v2, v1, v0th.vfmv.f.s fa0, v2Vector_WeightedSum_End:retNow, surprise. Which I don't get. The number of cycles jumps to 3875 compared to the previous version, with a CPI of 1.253. The number of cycles for the C version remains identical, so it's reproductible.Is it a difference with the cache maybe? Would be odd, since I execute a dry run before timing the second run, to get cache misses out of the way. What's up...And... what do you know: circles back to what we discussed earlier. Alignment of instructions. I added .option norvc before my assembly function. And, bam, I'm back to 3679 cycles. Fun fact is that it's the exact same number of cycles, but I bet that's because everything was already in data cache, otherwise the difference would have been much greater. And, it's cleaner this way, without passing via memory.So, yes, I can confirm that instruction alignment on this CPU makes a significant difference.

author: brucehoult
author_rank: Super Contributor
timestamp: July 08, 2024, 04:58:06 am
content: vfmv.f.s fd, vs2

author: brucehoult
author_rank: Super Contributor
timestamp: July 08, 2024, 05:03:08 am
content: Gah .. your edit was 39 seconds before my reply...

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 08, 2024, 05:07:41 am
content: I'll have to learn more about the strided addressing mode.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 20, 2024, 07:34:16 am
content: Having some questions about the CLINT and PLIC. I can't really figure out how these are mapped in memory for multiple cores, at least regarding the CV1800B, for which the documentation is basically absent about this. There is documentation for the C906 (open), but it's also not extremely helpful regarding this point,, as they document the CLINT and PLIC the C906 implements only as if there was only one core. Thanks.So, for the CLINT: I suppose that on SoCs that have multiple RISC-V cores, each core has its own CLINT? How is it memory-mapped? Is each CLINT (registers) mapped to a different address, or all mapped at the same address? (If the latter, that shoud mean each core has a special case for accessing it that doesn't go through the AXI bus?) Possibly obvious question, but it's confusing to me.About the PLIC: I suppose that there is only one PLIC for the whole SoC. The official spec defines PLIC registers for each hart (0, 1, etc). But on the CV1800B, the fun fact is that both cores have the same hart id (0). They apparently like jokes. So I can't figure out how to configure the PLIC for the second core.The SDK source code didn't really help. There's a TON of source code all over the place, with references to the PLIC and CLINT, but all coded in different styles with a bunch of macros that make it hard to figure out what exactly they use in actual code that runs.I was able to configure interrupts fine on the main core, but for the second core, this is puzzling. Any ideas?

author: brucehoult
author_rank: Super Contributor
timestamp: July 20, 2024, 10:04:32 am
content: Quote from: SiliconWizard on July 20, 2024, 07:34:16 amIf the HARTs are intended to make up a multi-CPU computer under one OS then, yeah, the registers have different addresses, and each HART can access the registers of the other HARTs. It would be pretty difficult to do a software interrupt between HARTs otherwise!For the way things are intended to be done it's usually useful to read SiFive documentation. For example the manual for the U74-MC core complex that is at the heart of SiFive's FU740 SoC (HiFive Unmatched) and StarFive's JH7100 and JH7110 SoCs (VisionFive, VisionFive 2, Pine64 Star64, Milk-V Mars, DC-Roma laptop etc). The U54-MC in the HiFive Unleashed, PolarFire SoC FPGA, PIC64GX are basically the same.CLINT memory map on P190.https://sifive.cdn.prismic.io/sifive/f24c0f97-cd86-4a88-9f2d-af23e8e32a10_u74mc_core_complex_manual_21G1.pdfNOw if two cores are NOT intended to be used under one OS ... which both having HART id 0 would indicate! ... then I don't know. Maybe they're at the same address and only accessible to their own core.QuoteYeah, tricky. PLIC is normally one for the computer. But, again, if they're treating this as two computers that just happen to share RAM (etc) then I don't know! There must be a way to route pins to the right core.Maybe look at the Arduino library source code? That runs on the 2nd core, and is probably simpler than the RTOS. I assume they support attaching a handler function to a pin interrupt, same as on other Arduino.

author: SiliconWizard
author_rank: Super Contributor
timestamp: July 20, 2024, 10:29:48 pm
content: Quote from: brucehoult on July 20, 2024, 10:04:32 amYeah. Obviously, best scenario would just be that they had documented it. But they haven't. At least not in the public documentation.So, there is reverse-engineering. Yes I've looked at the source code for the provided FreeRTOS which runs on the second core, but as I said, it's quite confusing. There are definitions all over the place, it's hard to follow. I haven't looked at the Arduino code, if it's less confusing to decipher, I may have a look.Anyway, from what I've gathered from the source code in the SDK and my tests, I can confirm that:- They have assigned the same hart ID for both C906 core. Which, yes, I understand that it's "ok" if both cores are not supposed to be used in a SMP context, since they can't run the same code anyway, but I still find it rather odd, and an unfortunate decision. But it's just a detail here - it was more confusing about how the CLINT and PLIC were shared.- The CLINT registers are mapped exactly at the same addresses for both cores. But, clearly they are separate. So the SoC must be internally mapping these addresses to different actual registers for either core. That's very confusing. But that's how it works here. For the PLIC, I haven't tested it yet on the second core, but from what I've gathered in the SDK again, I suspect this is the same thing. Actually, rather than a single PLIC, I suspect that each core has its own PLIC, which would explain why the PLIC interrupt numbers for each core are different.Now, as the RISC-V specs (and even the C906 specs) do not mandate anything specific for this kind of cases, I get it that every SoC designer is free to do whatever they want. I just find it a pretty odd decision, using the same memory-mapped addresses for separate registers. I haven't looked at the source code for the OpenC906 - possibly it would explain this decision in an obvious way.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: May 05, 2025, 12:23:26 pm
content: The SG2002 seems interesting, and this is the first time I'm considering it, I want to use it as a MCU replacement by doing only baremetal, SiliconWizard how did your tests run on it ok? what about debugging the code and flashing the SPI flash? did you find something, where should I start to read.

author: SiliconWizard
author_rank: Super Contributor
timestamp: May 05, 2025, 01:19:38 pm
content: Quote from: ali_asadzadeh on May 05, 2025, 12:23:26 pmHaven't tested the SG2002 yet, only CV1800B, although they seem very similar. So I can't tell for the SG2002 specifically.Haven't rolled my own board yet with it, but on the milkV duo boards, there doesn't seem to be any access to a RISC-V debug port. So the debugging is limited to sending and receiving stuff via UART. I don't find this a huge limitation that's how I debug most of my MCU designs. I rarely use a debugger. But this lack of debugging access is something to consider. I'm going to have a look at the chip(s) itself to see if there's a debugging port on some pins. But if that were the case, we'd have to figure out which debug probe can be used if any and with what tool. I don't know if OpenOCD supports the C906, I'll have a look at that.I recommend starting here:https://github.com/sophgo/sophgo-docThe ref manual for the SG2002 can be downloaded in PDF here:https://github.com/sophgo/sophgo-doc/releases

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: May 05, 2025, 01:35:33 pm
content: QuoteThanks, If you can test on CV1800B for debugging, I think it would show us the path to the newer SG2002 too.

author: SiliconWizard
author_rank: Super Contributor
timestamp: May 05, 2025, 05:23:10 pm
content: From what I can read, they do support JTAG. There are only two JTAG pins though (JTAG_CPU_TMS, JTAG_CPU_TCK) multiplexed with other pin functions, but JTAG appears to be the default config at power-on.Apparently, the debug interface is also available via I2C.There is a "debug firewall" though so JTAG access can be disabled. Doesn't look too trivial.There may be more info on the debug protocol itself in the (open)C906 manual.The only tools that appear to officially support the C906 I could find so far are from Lauterbach:https://www.lauterbach.com/supported-platforms/chips/c906So, at this point that looks like a rabbit hole, possibly you could contact Lauterbach to see if they directly support the SG2002 and have more info on this.Digging a bit more: the closest I could find to getting us to JTAG debugging seems to be there:https://wiki.sipeed.com/hardware/en/maix/m1s/other/start.html#Use-JtagIt's for a BL808, so a completely different chip, but also with a C906 core. So I guess it should be a good starting point and may work with the milkV chips too.They use a "CKLink" as a JTAG probe:https://wiki.sipeed.com/hardware/en/logic_analyzer/combo8/use_cklink_function.htmlthe JTAG server software is there:https://github.com/UDTeD/T-HeadDebugServerunfortunately, it's not open-source, they only release binaries, but it's free. And it's from T-Head, so I'm assuming it should work with all T-Head cores.Using that on a CV1800B/SG2002 is probably unsupported though, at least from Sipeed - and from T-Head, I doubt you'd get much direct help either.But, that's something to test. I don't much like the fact there are only binaries for the debug server though, and they only release the Windows version (there's apparently a Linux version available on demand... also binary only)I haven't seen any support from the OpenOCD side, although there may be some plan to support the T-Head cores, don't know. That would be much better.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: May 06, 2025, 12:20:00 pm
content: I hoped you could shed light on us, since Debugging complex projects is a MUST, I thought maybe something in OpenOCD and FT2232 would help us.