Is there any standard reference designs for a high speed inter-FPGA bridge? - Page 1
Forum section: electronics / general

author: technix
author_rank: Super Contributor
timestamp: June 05, 2019, 04:29:49 am
content: For example, I have a board with a Zynq-7020 and a Cyclone IV connected using some traces. Now I want to implement some kind of high speed bridge that exposes an AMBA AHB interface from the Zynq to the Cyclone IV. Is there any standard implementation for that, which can encapsulate AMBA AHB transactions into some form of external signaling format and shoot it over the traces, and decapsulate it on the other end?I want to avoid SerDes at all costs (since my chips don't have any to begin with) but differential serial transmission is preferred (so length matching between pairs can be less critical.)

author: Berni
author_rank: Super Contributor
timestamp: June 05, 2019, 05:53:39 am
content: That is a bit of a odd design to join two vendors of FPGAs like that.But standardization in FPGAs is pretty bad to say the least. Each vendor pushes there own kind of memory or streaming bus that does the same thing, but ever so slightly differently so its not directly compatible.In principle you can just run the internal bus onto the pins and run it right to the other chip. Perhaps trough some sort of memory bus bridge to transform it into variant of the bus with a 8 or 16bit wide bus with perhaps address latching modes to cut down the number of pins required, some extra latency can also be set for the bus to make sure the timings still work out when they make it to the other chip trough the PCB. Another way is so have the master chip act as a synchronous SRAM controller and the slave chip to act like its SRAM memory.To use diff pairs you can just run the whole bus into a softcore serdes block, those will generally give you about 300 to 1000 Mbit/s on most FPGAs. On the other side you simply get your big parallel bus out of the serdes block again. Tho some trace length matching might still be required because you will not tend to have individual clock recovery for each pair to be able to tune out the delays. But due to the lower speeds it will tolerate a lot more trace length mismatch.

author: technix
author_rank: Super Contributor
timestamp: June 05, 2019, 06:20:14 am
content: Quote from: Berni on June 05, 2019, 05:53:39 amSo for using diff pairs, use one soft core LVDS serdes on each side, somehow have AHB directly connected to it on one side, and have AMBA AHB come out on the other side? Is it going to be transparent regardless of latency? As of clock synchronizaton, is it a good idea to run a LVDS clock pair along with the two unidirectional LVDS data pairs? And is 1.8V LVDS a good option for that?As of poking the parallel bus out, is there any transparent solutions that can pack a 32-bit bus into something that is 8-bit wide and restore the bus on the other end?

author: Berni
author_rank: Super Contributor
timestamp: June 05, 2019, 07:56:12 am
content: Well of course you have to make sure your timings and latencies work out for the bus you are running across. Buses are usually configured for certain timing settings, so its just a matter of setting up the bus correctly.LVDS obviously adds extra latency to the whole thing because the process of serialization takes time, but the time it takes is always constant so the bus can be configured for the correct latency. Clock + unidirectional LVDS pairs is the way to go, tho you may want more than one pair going each way if you want to have lots of speed and low latency.As for narrowing the bus down to 16bit or 8bit, this is done by the same memory bridge block that you will likely need to sort out the bus timings (Internal buses likely run on timings that are too fast). Various memory bus bridges and adapters are often used inside FPGAs for connecting modules with the wrong bus type onto the given main memory bus. So if you connect a module with a 8bit interface to a 32bit bus there will have to be a bus converter between to make it work. Vendors often provide such bus converters, or provide a tool that builds your bus system automatically, inserting these converters wherever needed like magic (Like Altera SOPC builder and whatever Xilinxes equivalent tool is called).Even so these bus width/timing adapters are not hard to make yourself as long as you don't try to support all the advanced functionality of a complex memory bus configuration (Like transaction interleaving, reordering, buffering, special DMA related stuff etc) that in most cases you will not use or need. So in the end all you need to handle is "Please write X to location Y" and "Please tell me what is at location Y" along with perhaps a busy signal to pause the bus if something clogs up. Its your choice how to get that information across depending on your speed and performance needs. Heck you could do it all over a simple SPI bus if high bandwidth is not required.

author: tszaboo
author_rank: Super Contributor
timestamp: June 05, 2019, 08:37:10 am
content: You know, that big companies, like Intel and AMD is struggling connecting their chips together transparent, with memory coherency. AMBA, AHB and others are on chip buses. Meaning that they are designed to stay on one chip.

author: technix
author_rank: Super Contributor
timestamp: June 05, 2019, 10:39:46 am
content: Quote from: NANDBlog on June 05, 2019, 08:37:10 amI have some FPGA overflow but it just isn't worth it to step the main FPGA chip up from CLG400 package to FBG484 package, which requires me adding PCB layers to hold that increased package size. So while staying on the original layer count I had the idea of offloading some modules into a separate, smaller and cheaper 144-pin FPGA and bridge the two.

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 11:23:46 am
content: There is a free AXI Chip2Chip IP provided by Xilinx for that purpose. It can use either parallel or MGT serial physical connection. Check Xilinx document PG067.

author: tszaboo
author_rank: Super Contributor
timestamp: June 05, 2019, 11:30:02 am
content: Usually unless you need a lot of high speed interfaces, you typically just put a bunch of 74AC164 and 165s on your board.

author: technix
author_rank: Super Contributor
timestamp: June 05, 2019, 12:07:19 pm
content: Quote from: NANDBlog on June 05, 2019, 11:30:02 amThese are not just simple GPIO blocks.Quote from: asmi on June 05, 2019, 11:23:46 amCan it use soft-core serdes though, and can it be used on the Intel chip?

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 12:33:53 pm
content: Quote from: technix on June 05, 2019, 12:07:19 pmI don't know what is "soft-core serdes", and the latter is likely a "no".

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 12:40:41 pm
content: Quote from: technix on June 05, 2019, 10:39:46 amThis is a bad idea. Increasing layers count is very likely to end up significantly cheaper than adding another chip (with all required components it will require - like power DC-DCs, QSPI flash, JTAG, etc.).There is also CLG484/485 package if you want to stick to 0.8 mm pitch and not go to much larger 1.0 mm one, but the latter will obviously much easier to route.

author: NorthGuy
author_rank: Super Contributor
timestamp: June 05, 2019, 01:19:18 pm
content: How fast do you need?

author: technix
author_rank: Super Contributor
timestamp: June 05, 2019, 02:40:54 pm
content: Quote from: asmi on June 05, 2019, 12:40:41 pmThe 144-pin one shares all its power rails with the CLG400 one, so no additional DC-DC needed. No QSPI used either as the interconnect traces does include provision for the ARM cores in the Zynq to load bitstream into that satellite FPGA. And JTAG is daisy chained. I am talking TQG144 too which is a package doable even in a dual-layer board.

author: OwO
author_rank: Super Contributor
timestamp: June 05, 2019, 03:18:59 pm
content: More pins != harder to route, because you don't need to route out all the IO pins. It's usually very much the opposite, since the used IO density is lower meaning you have more space between vias to route signals through. Also going to 1.0mm pitch helps a lot. How many layers are you currently using?

author: Berni
author_rank: Super Contributor
timestamp: June 05, 2019, 05:40:40 pm
content: Well who knows maybe the design makes sense to use a 2nd smaller FPGA. You do sometimes see a FPGA and a small CPLD on the same board, so why not a big FPGA+SOC and a smaller FPGA on the same board.Oh and a so called "softcore serdes" is a serdes that is implemented inside of the FPGA fabric rather than being a real physical serdes block. As a result its much slower in terms of max bitrate, but can often be used on any IO pin and can be more flexible in terms of data formats. Often this serdes implementation uses the help of some extra hardware inside the IO pin such as DDR flip flops in order to achieve higher speeds than are generally possible inside the FPGA fabric. Despite being a lot slower they are often still fast enough to run a lot of LVDS interfaces you might run into.Since you can shove the memory write/read requests down any kind of "pipe" by writing your own simple bus adapter on each side, means that the choice of the bus is mainly driven by how much bandwith you need. On the top end you could have a LVDS going 8 pairs one way and 8 pairs back, costing you 18 pins and getting about 1GB/s in both directions. Cutting it down to a single pair gets your IO cost down to 6 pins to get you about 100MB/s. Going for a 8bit data bus will cost about 20 normal general purpose pins and get you about 300 MB/s but with nice latency due to no serdes, or cutting it down to a simple SPI bus with 4 pins could get you up to 300Mbit/s or about 35MB/s. Heck even UART running at a ridiculous 100Mbit+ baud rate could be used, tho that's not a great idea for FPGAs.

author: technix
author_rank: Super Contributor
timestamp: June 05, 2019, 05:42:23 pm
content: Quote from: OwO on June 05, 2019, 03:18:59 pmI am on 6 and seriously considering going to 4 and move the Zynq to a COTS SoM. And since I am talking Zynq the memory bus is a routing bitch forcing the layer count up as for that side almost all pins must be routed out.Quote from: Berni on June 05, 2019, 05:40:40 pmMy processor maxes out at 667MHz anyway so the latecy of a serdes can be just a few CPU cycles, so I would take the serdes route. Also I am offloading the slower peripherals here that has a maximum bus frequency lower than 50MHz, so having something like 800MB/s should pose no more latency than internal implementation.

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 06:04:49 pm
content: Quote from: Berni on June 05, 2019, 05:40:40 pmThe question is "what's the point?" 7 series FPGAs and Zynqs have a pair of ISERDES/OSERDES for each and every IO pin so you will never run out of them unless you're doing something really cheesy like using SERDES cascading with single-ended IO.

author: OwO
author_rank: Super Contributor
timestamp: June 05, 2019, 06:12:38 pm
content: Quote from: technix on June 05, 2019, 05:42:23 pmI have a reference design for Zynq-7010/7020 400 pin BGA on 4 layers:https://github.com/gabriel-tenma-white/sdr5#zynq_som_2The DDR3 memory runs at full speed with no errors, 2 units prototyped so far.

author: OwO
author_rank: Super Contributor
timestamp: June 05, 2019, 06:19:28 pm
content: Cost of a Zynq 7010: $15 (new). Cost of a commercial SoM with DDR3: over $100. Similar story for 7020. I wouldn't use a SoM especially when you have already spent the effort designing the DDR3 layout.

author: Berni
author_rank: Super Contributor
timestamp: June 05, 2019, 06:31:51 pm
content: Well since internal memory buses are often 32bits wide that would means 50MHz bus would only transfer 200MB/s. So you would probably need about 3 to 4 diff pairs in each direction to make it work when counting in overhead for control signals and that you might not want to run the pairs at max speed to get some extra timing slack and reliability.In theory the 4 pairs will need 8 clock cycles to transfer 32bits and likely need a 9th for control signals. So 450Mbit per pair would do nicely, this is something most FPGAs can easily do with soft serdes. So this transfers the 32bits in the time of a single 50MHz bus clock cycle. You will also need to get your address across tho. So its 1 cycle to move the address, 1 cycle to move the data, 1 cycle to execute the operation on the slave bus, 1 cycle to receive the data response, all together 4 cycles to execute a read in the slave FPGA. This is not too bad, but you could also have support for burst transfers to get more speed for large chunks. There you would need the 4 cycles to get the first read done, but each next read is only 1 cycle, thus getting you the 200MB/s theoretical max speed until the burst transfer finishes.You might need to add a extra clock cycle here or there if some extra time is needed to make a decision in there. Bus transfers always have some sort of latency associated with them so you will never get to 0 cycle latency. This was one of the motivations for Intel in x86 to get rid of the north bridge and bring the memory controller onto the CPU die.EDIT: By the way OwO where can you buy a Zynq 7010 for $15?

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 06:42:37 pm
content: Quote from: OwO on June 05, 2019, 06:19:28 pmNot everyone is willing to put up with dodgy Chinese sellers selling you random parts while trying to fool you into believing it's something else. Some like to only buy through official channels. This is especially so for commercial products when you need to be sure the seller you bought chips from will still be there in 5 years from now. I tried it personally and the seller sent me a SG1 device instead of SG2 I ordered and somehow though that it was OK to do soI got my money back (and still got a chip, thou I haven't soldered it yet to see if it works at all), but if it would be a commercial project, I'd be totally screwed as for some time I'd have no money nor chips.Oh, and please stop calling this crap "new". It's not. It might be from old stocks, or it may be desoldered reballed junk that may or may not work, or (my case) it might be a different chip altogether from the one you ordered.

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 06:43:47 pm
content: Quote from: Berni on June 05, 2019, 06:31:51 pmGo to aliexpress and go for it - if you're willing to play Russian roulette with what you will actually get (if at all).

author: NorthGuy
author_rank: Super Contributor
timestamp: June 05, 2019, 07:12:56 pm
content: Quote from: asmi on June 05, 2019, 06:43:47 pmHow about the chips you bought few months ago. Did they work Ok?

author: asmi
author_rank: Super Contributor
timestamp: June 05, 2019, 07:16:56 pm
content: Quote from: NorthGuy on June 05, 2019, 07:12:56 pmDon't know yet. The board I was going to put it on was really designed for SG2 device (DDR3L and all that), and so are most of my projects, so such board won't be very useful to me. Which is why I don't feel like spending hours assembling the board to check it out.

author: brucehoult
author_rank: Super Contributor
timestamp: June 06, 2019, 08:17:20 am
content: At SiFive, we used TileLink, originally developed at Berkeley University, for all on-chip communications. SOme of our customers prefer AXI so we have fully-featured bridges between them:https://www.sifive.com/documentation/tilelink/tilelink-spec/https://static.dev.sifive.com/AXI2TL.pdfhttps://static.dev.sifive.com/TL2AXI.pdfFor inter-chip or inter-board, we have "ChipLink". For example this is used to send coherent memory traffic between the HiFive Unleashed board and either a Xilinx VC707 FPGA board, or MicroSemi "HiFive Unleashed Expansion Board" over an FMC connector using 2x35 pins @200 MHz. The boards bridge this to PCIe (and other things) and we run video cards, SSDs etc there.There's no official spec of ChipLink yet. The source for the interface is herehttps://github.com/sifive/sifive-blocks/tree/master/src/main/scala/devices/chiplinkand a presentation herehttps://content.riscv.org/wp-content/uploads/2017/12/Wed-1154-TileLink-Wesley-Terpstra.pdfThis will all be proposed as a standard once it settles down a bit.Western Digital has also been working on a TileLink-over-Ethernet implementationhttps://github.com/westerndigitalcorporation/omnixtendWe also very much like NVMe as a physical layer, so I think you'll see some work around that emerge later.A quote from Wes (ChipLink designer):QuoteAnd one from Krste Asanovic (SiFive founder):QuoteBoth these fromhttps://forums.sifive.com/t/chiplink-isnt-an-open-spec-apparently-or-can-i-see-it-somewhere/1479

author: tszaboo
author_rank: Super Contributor
timestamp: June 06, 2019, 08:24:08 am
content: Quote from: asmi on June 05, 2019, 06:42:37 pmYou do realize that FPGAs are not sold at the same price "as seen on digikey". It is a lot lower.

author: asmi
author_rank: Super Contributor
timestamp: June 06, 2019, 09:53:13 am
content: Quote from: NANDBlog on June 06, 2019, 08:24:08 amNowhere in my post you'll see mentions of a price. That is intentional.The problem with buying FPGAs on aliexpress is not price. It's the fact that it's not an official channel, so there are no guarantees whatsoever that you will actually get what you've ordered, and that it will be brand new chips, and they will work. It's true that you can get your money back if things go south, but for commercial projects this is not good enough because time wasted costs money too.

author: nctnico
author_rank: Super Contributor
timestamp: June 06, 2019, 12:49:58 pm
content: Quote from: technix on June 05, 2019, 04:29:49 amSome peope at CERN have cooked up something for the Wishbone bus called Etherbone (http://cds.cern.ch/record/1563867).

author: asmi
author_rank: Super Contributor
timestamp: June 06, 2019, 01:17:26 pm
content: Quote from: brucehoult on June 06, 2019, 08:17:20 amWhy haven't you just integrated PCIe root complex instead? This would allow building a RISC-V PC without having to use $5k FPGA devboards, not to mention all schematic complexities this brings. If it would have PCIe, I would've bought these chips in a heartbeat to design a motherboard for such PC. This feels such a missed opportunity. Nobody in the right mind would make a baseboard with super-expensive FPGA only to get PCIe capability. And decision to use home-brewed bus is even more annoying.

author: technix
author_rank: Super Contributor
timestamp: June 06, 2019, 03:31:28 pm
content: Quote from: asmi on June 06, 2019, 01:17:26 pmActually how about just adopting AMD AM4 socket in the next ASIC version? That socket can require just DDR4 and PCIe (to both the graphics card slot and the southbridge) to work, so RISC-V in AM4 means people can just buy the chip and drop it in a standard COTS AMD Ryzen motherboard, and it would more or less work.

author: asmi
author_rank: Super Contributor
timestamp: June 06, 2019, 03:45:26 pm
content: Quote from: technix on June 06, 2019, 03:31:28 pmNo it would absolutely not work. For example, BIOS/UEFI code is x86 binary which RV would obviously not be able to execute. Same goes for video and network BIOSes - they also contain x86 code. Using a socket would only increase the price of a chip (and a baseboard) for zero gain.

author: brucehoult
author_rank: Super Contributor
timestamp: June 06, 2019, 08:00:53 pm
content: Quote from: asmi on June 06, 2019, 01:17:26 pm"just"Because it's a test chip, SiFive's first 28nm chip, the world's first Linux-capable 64 bit quad core 1.5 GHz RISC-V, and by far more complicated than SiFive's only previous chip, a single core 32 bit low speed FE310 microcontroller.*Obviously* future volume production chips are highly likely to have PCIe built in -- that's hardly giving any secrets away -- but it's needless complication for a first test chip when PCIe can be provided externally with off-the-shelf boards without severely limiting performance (unlike, say, DDR).If those boards cost $2000 or $3500 (vc707), that's an absolute bargain compared to the cost and time and risks of developing in-house PCIe and associated SerDes and PHY technology for a test chip that's only ever going to have a few hundred made. (or licensing it, obviously, but people already complain enough about the proprietary DDR4 IP that was used)QuoteNot exactly home-brewed. TileLink has been under development and used in numerous Berkeley projects for a number of years.Your "home brewed" is other people's "free and open non-proprietary standard" that is just as good as AMBA4 that is controlled by the company with the most to lose if RISC-V or other open ISAs gain significant market share.

author: NorthGuy
author_rank: Super Contributor
timestamp: June 07, 2019, 04:00:47 am
content: Quote from: brucehoult on June 06, 2019, 08:00:53 pmI guess by the time it gathers enough financing to climb to PC-level, it won't be free and open source any more

author: technix
author_rank: Super Contributor
timestamp: June 07, 2019, 04:25:02 am
content: Quote from: asmi on June 06, 2019, 03:45:26 pmIf Loongson boards is something to go by, the CPU itself contains some internal ROM code serving the purpose of a BIOS. The x86 UEFI is just skipped and unused.

author: Berni
author_rank: Super Contributor
timestamp: June 07, 2019, 05:23:24 am
content: Don't know how we ended up on RISC-V but reusing a AMD motherboard doesn't make much sense.These days motherboards only really provide IO ports and power, that's it. All of the major parts just connect directly to the CPU socket like DDR memory, PCIe slots, integrated video etc.. What the motherboard actually does is turn some of those PCIe lanes into other ports (SATA,USB,Audio,Network...) using the south bridge and some extra chips around it, since some of these ports are critical for booting it also provides a bios that can bring these ports up and load the OS from them. It also converts 12V into whatever core voltages it needs. That's about it for a typical consumer motherboard.If RISC-V had a built in PCIe controller (as a modern motherboard would require it) then making a dev board with PCIe would just be a matter of running a few traces to a PCIe card edge connector.Devboards for brand new technology will always be expensive. You need the large mass volume to bring production costs down and cover the huge engineering costs.

author: brucehoult
author_rank: Super Contributor
timestamp: June 07, 2019, 06:18:26 am
content: Quote from: NorthGuy on June 07, 2019, 04:00:47 amThat's entirely up to whoever develops said PC-level chip. If it's you, you're free to open-source the whole thing if you want.Right now we're at about early 2000s PC level, or low end Android phone now (plenty of them for sale with quad core A53 only).My guess is the proprietary RISC-V chips will stay three to five years ahead of the totally open ones, but not much more than that.

author: asmi
author_rank: Super Contributor
timestamp: June 07, 2019, 12:23:25 pm
content: Quote from: Berni on June 07, 2019, 05:23:24 amBIOS/UEFI is also responsible for detecting memory modules and configuring memory controller(s), performing PCIe enumeration, bus assignment and BAR mapping (among other things). Also south bridge (at least on Intel platform) is connected using DMI bus, which is substantially different from PCIe.

author: asmi
author_rank: Super Contributor
timestamp: June 07, 2019, 12:35:07 pm
content: Quote from: brucehoult on June 07, 2019, 06:18:26 amI'm actually working on PCIe root complex for my own hobby RISC-V PC that is going to be based on FPGA (since I have no money for ASICs), which I plan to open source once there will be enough of it to actually work. This is why I got so excited when I heard about your chip, and was so disappointed to see the lack of PCIe onboard - this is a really major showstopper. At least in my case I don't have to design my own PCIe PHY, data link and transaction layers as there is pre-exising hard IP for that purpose integrated into FPGAs I'm planning to use, so I only have to design higher-level logic of handling TLPs. The idea for the module is to be as similar to x86 PCIe as possible as it's by far the most straightforward implementation I've ever seen (all PCIe IPs I've read about in ARM SoCs are just a big mess). I want using it to be as easy as it is on x86, where once configured and mapped, PCIe endpoints become essentially the same as using any other memory mapped peripheral with zero PCIe-related programming required.Quote from: brucehoult on June 07, 2019, 06:18:26 amThe latter is probably more correct, as the cornerstone of PC (and the reason they were so successful) has always been extensibility (via ISA, PCI, PCIe buses).Quote from: brucehoult on June 07, 2019, 06:18:26 amI hope they will be open from the beginning, but I'm afraid you'll probably be right. All ideals of OS tend to be tossed aside when big money are getting involved

author: Berni
author_rank: Super Contributor
timestamp: June 07, 2019, 01:07:56 pm
content: Quote from: asmi on June 07, 2019, 12:23:25 pmYes PCIe enumeration is part of "getting it to boot into a OS" because PCI and PCIe cards can act as boot devices since they can hold PCIe SSDs, SATA controller cards, network cards with network boot support etc.. Same reason why USB is enumerated in BIOS since it might be holding a USB drive with bootable code. Legacy software also might not support a USB mouse/keyboard so it enumerates those and provides its keyboard abstraction driver for it. The boot code and all of this also needs to fit somewhere so it also brings the RAM so it has room to put it into.Once the OS really gets onto its feet, it offten throws a lot of this out, such as tossing out the BIOS keyboard driver and grabbing the USB controller to talk to it directly. Similar is done to a lot of the other hardware (This is why older Windows OSes are so difficult to boot from NVME SSDs, they start booting fine, but then upon tossing out the default bios disk driver find themselves without a working system drive).On other platforms such as ARM this whole job is typically done by Uboot or similar. That Uboot firmware is only compatible with one board/cpu so its not portable at all.And yes Intel does use a different bus to the southbridge that does the same thing really. But AMD pretty much uses a repurposed PCIe bus for this.

author: technix
author_rank: Super Contributor
timestamp: June 07, 2019, 02:30:26 pm
content: Quote from: Berni on June 07, 2019, 01:07:56 pmMost boot-related PCIe and USB drivers are standardized: PCIe itself, XHCI, EHCI, OHCI and UHCI for USB, AHCI and NVMe for local storage and USB MSC for USB-attached storage. It is normal for a modern OS to perform a full system re-enumeration during boot to establish driver usage too.The whole point of reusing an AMD motherboard is allowing the ASIC to carry just the processing cores, DRAM controller, PCIe root complex and an alternative BIOS, while with little external effort achieve a usable system.

author: asmi
author_rank: Super Contributor
timestamp: June 07, 2019, 05:57:56 pm
content: Quote from: technix on June 07, 2019, 02:30:26 pm1. It will increase the price of the chip. BGA packages are cheap, specialized sockets/connectors are not - especially if you only want to do a (relatively) small batch.2. AM4 socket is probably a proprietary IP of AMD. Who says AMD is going to let them use it?3. The voltage rails are going to be incompatible. The ASIC is powered by 0.9 V rail, while AMD CPU use 1.37-1.4 V. If you power RV ASIC with 1.4 V, chances are it's going to go bust before you can say "oops".4. DRAM controller designed to work only with a small selection of DRAM chips, and the one that can work with pretty much anything out there on a market - are two very different things in terms of complexity.5. There is much more going on on a motherboard than meets the eye. It's far from being just a bunch of connectors.6. Most importantly, PCIe connectors are cheap, and so are PCBs and regular BGA assembly. So there is little sense in trying to save anything.But fundamentally what's the point? If someone would want to build an RV PC, he would have to buy all components anyway. And I seriously doubt that motherboard is going to be the most expensive part.

author: technix
author_rank: Super Contributor
timestamp: June 07, 2019, 07:27:13 pm
content: Quote from: asmi on June 07, 2019, 05:57:56 pm1. Since the RV chip is significantly smaller physically than a Ryzen, instead of a specialized PGA package, it can be a RV BGA chip + a QSPI Flash chip for the alternative BIOS on a piece of FR4 interposer. This is actually a trick some Shenzhen sellers use to convert Intel laptop chips for their desktop motherboard, by adding a FR4 interposer.2. I don't think the pinout of a chip socket is copyrightable or patentable. And all signals we are interested here are open standards.3. Depending on the chip's power consumption and the size of the interposer, it should be possible to integrate a Vcore regulator on the interposer. Alternatively since this level of design change more likely than not requires a new chip revision anyway, the new chip revision can be made with a maximum Vcore of 1.45V to tolerate both native BGA design and AM4-based design.4. Self-calibration of DRAM controller is a problem you need to solve before RV chips can be taken seriously, as the use of DIMMs are practically required as soon as you leave the embedded/mobile market. Virtually all desktop computers and all servers uses DIMMs.5. What might go wrong then? All the interfaces involved here are either open standard, or at least have open source drivers in Linux kernel you can learn from.6. A small FR4 interposer is cheaper than even a mini-ITX board, and a mass market COTS AM4 board is more likely than not to be cheaper than a small volume native BGA board too if matched feature for feature.

author: brucehoult
author_rank: Super Contributor
timestamp: June 07, 2019, 07:41:24 pm
content: Quote from: asmi on June 07, 2019, 12:35:07 pmWhen the FU540 taped out about 1.7 years ago (and first working boards were ready 1.3 years ago) SiFive had fewer than 30 employees. Now there are over 400. You might imagine there are a lot of things in the pipeline.QuoteYes, FPGAs give you a lot. That's why we use them for the PCIe too, for the first test boards.They make really low performance CPUs though :-(MicroSemi (a Microchip subsidiary) will soon be selling new PolarFire FPGAs (the "PolarFire SOC") with SiFive's quad core (plus 1) 64 bit FU540 as a hard macro inside the FPGA, the same way Xilinx Zynq has ARM cores.https://www.microsemi.com/product-directory/soc-fpgas/5498-polarfire-soc-fpga

author: asmi
author_rank: Super Contributor
timestamp: June 07, 2019, 08:09:34 pm
content: Quote from: brucehoult on June 07, 2019, 07:41:24 pmWell I hope one day I will see a CPU with PCIe, DRAM that is going to be available in a reasonable package (not a 0.3 mm pitch BGA with bazillion of balls, something like 0.7 mm and up BGA with <500 balls), and it won't have a million of useless (for PC) peripherals (which is my personal grief with existing ARM SoCs - all PCIe-enabled ones have a crap load of peripherals that I don't want).Quote from: brucehoult on June 07, 2019, 07:41:24 pmLow per-core performance can be somewhat compensated by having a large amount of cores. Which is what's my current goal - I'm working on a 64 bit core that will run at 250 MHz on a cheap Artix-7 FPGAs, which can be pushed up to ~300 MHz (maybe even 350 MHz) on Kintex if I can get my hands on them for reasonable money, and then just bunch up however many of them that will fit inside FPGA. All FPGA 64 bit RV cores I've tried so far can barely go above 100 MHz on A7, which is why I'm on a mission to change that. MUL/DIV will obviously be multicycle, but the rest should give close to 1 IPC (fully-pipelined).Quote from: brucehoult on June 07, 2019, 07:41:24 pmAre their design tools just as free as Xilinx's ones are? Never encountered their FPGAs yet.

author: brucehoult
author_rank: Super Contributor
timestamp: June 08, 2019, 06:08:55 am
content: Quote from: asmi on June 07, 2019, 08:09:34 pmThat would be very cool! Good luck!As well as 1 IPC, are you expecting to be able to run dependent integer instructions back to back?PicoRV32 runs at a pretty high clock rate, but doesn't even try to be 1 IPC.SiFive evaluation FPGA bitstreams are 1 IPC but are deliberately generated from RTL designed for SoC not for FPGA, to help verify that the eventual SoC will work, even though this results in lower clock frequency and higher LUT usage than FPGA-optimised RTL would.

author: asmi
author_rank: Super Contributor
timestamp: June 08, 2019, 10:49:42 am
content: Quote from: brucehoult on June 08, 2019, 06:08:55 amThank you! Right now the summer is somewhat getting in the way (my wife doesn't like me staring into the monitor while the weather is so nice outside), but I already have some blocks ready, and the project is slowly progressing.Quote from: brucehoult on June 08, 2019, 06:08:55 amYes, absolutely. DSP blocks that I use for ALU have a dedicated feedback path from output back to input, so there shouldn't be any issues with that. I actually already have designed ALU and tested it in hardware at 250 MHz and it worked OK, but of course when you integrate modules together there might be some routing issues. These DSP tiles can be run as high as 500 MHz when fully pipelined, but I'm trying to avoid making pipeline too long as pipeline flushes become progressively more expensive.Quote from: brucehoult on June 08, 2019, 06:08:55 amWell as I understand it's design goal was to be small. I don't have such constraint - there are lots of LUTs and registers in A7 FPGAs, so I can afford to "invest" them into the proper pipeline.