Fast data transfer between FPGA (Xilinx Spartan 6) and 30 microcontrollers - Page 1
Forum section: electronics / general

author: Momchilo
author_rank: Regular Contributor
timestamp: February 04, 2019, 12:03:44 am
content: Hi,I have to send and receive data in both directions between an FPGA (Xilinx Spartan-6) and 30 microcontrollers, which aren't chosen yet. Here is the basic sequence as seen in the attached picture:1. Every µC makes one measurement simultaneously with 16bit (30 different voltages)2. Each µC send the measured values (30 different voltages) to the FPGA3. FPGA writes the 30 values to SRAM4. DSP and FPGA process data and calculate 30 PWM values5. A specific PWM value (16bit) must be sent to each µC (30x)6. Each µC converts the PWM value into the corresponding PWM signal simultaneouslyMy problem is point 2 and 5. The FPGA receives 30x16bit=480bit from the µCs and then send 30x16bit=480bit to the µCs. What are the options to do this?Of course there are SPI, I2C and similar buses, but I think they're way too slow. Because there are only ~10µs to send the measured voltages and send the PWM values. All in all the data rate is like 100Mbit/s (50Mbit/s in each direction) and the wire length is about 0.5m.The selected DSP and FPGA are given, but the 30 microcontrollers could be anything and aren't chosen. It would be even possible to use FPGAs instead of microcontrollers.Do you know any bus systems or other options to solve the problem? It would be really nice, if the implementation wouldn't be too complicatedI'm grateful for every help.Best regardsMomchilo

author: KaneTW
author_rank: Frequent Contributor
timestamp: February 04, 2019, 12:32:59 am
content: At these latency requirements, I'd say a direct FPGA to FPGA connection is required. I'm not too familiar with FPGA interfaces, but maybe JESD204B?

author: hamster_nz
author_rank: Super Contributor
timestamp: February 04, 2019, 01:28:43 am
content: Is it possible to add a bit more on the timing information?What is your sample rate? It sounds like it is 100kS/s, but am not sure.How simultaneously is simultaneous? Will you need to sync the µC's clocks? How can you reliably trigger the ADC process at the same time?How can a µC's convert a PWM value to a PWM signal simultaneously? That would require them all to be in lockstep. On the surface it seems easier to generate the PWM at the FPGA, and just send that signal down the wire rather than a PWM value.I would suggest a remote SPI ADCs, using LVDS driver/receivers over 0.1" ribbon cables would be my first stab at a design. You should have no issue with a few Mb/s per pair. A simple state machine in the FPGA should be all you need to acquire the data.What are you thoughts on grounding all these ADCs? I don't know much about it, but I think thirty 0.5m cables and signal-ended signalling will not play well with the 16-bit ADCs.

author: nctnico
author_rank: Super Contributor
timestamp: February 04, 2019, 01:42:03 am
content: I'd skip the microcontrollers at the end points and use LVDS to connect from the FPGA to the gate driver and ADC directly. Distributed systems are no fun to get working reliably and synchronously. When it comes to the gate driver I'd add some extra logic to detect over-current conditions and shut it down. Modern FPGAs can deal with LVDS directly and have lots of pins anyway but be sure to have some TVS diodes and series resistors in the signal paths to protect the FPGA from off-board signals.

author: Marco
author_rank: Super Contributor
timestamp: February 04, 2019, 01:59:44 am
content: Assuming the pulse who's width is being modulated has to be generated within that 10us timewindow that's in the range of 150 ps resolution ... not impossible, but not a lot of microcontrollers which can do it.

author: brucehoult
author_rank: Super Contributor
timestamp: February 04, 2019, 02:09:43 am
content: With 100 Mbps needed and a range of 0.5m I think you're getting away from what a simple bi-level signal protocol can do, and start to need a proper analogue signal and serdes.I was wondering about USB3 or gig Ethernet. The data rate vs distance is fine, but you've not going to turn them around in 10 us. 300 - 1000, yes.NVMe might be the best bet.

author: hamster_nz
author_rank: Super Contributor
timestamp: February 04, 2019, 02:29:03 am
content: Quote from: brucehoult on February 04, 2019, 02:09:43 amI read the OP's post as 100Mb/s for aggregated 30 channels, so 3.2Mb/s per camera, which is 16 bits in each direction, one 16-bit sample and PWM value every 10us.But could be (and most likely am) wrong....

author: brucehoult
author_rank: Super Contributor
timestamp: February 04, 2019, 02:50:35 am
content: Quote from: hamster_nz on February 04, 2019, 02:29:03 amAh, yes that's a bit easier. Should be no problem with SPI.Very curious what kind of control system needs 100 kHz response.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: February 04, 2019, 04:20:13 am
content: Quote from: Momchilo on February 04, 2019, 12:03:44 amYou seem to have not looked at separate links ? - if you send 16b each way in 10us, that's a fairly modest 1.6Mb/s duplex link.That's easily within UARTS and SPI ports on most any of the better small MCU's.

author: donotdespisethesnake
author_rank: Super Contributor
timestamp: February 04, 2019, 11:48:12 am
content: Previous discussion on this topic :https://www.eevblog.com/forum/projects/can-bus-timing-164140/It seems you have the classic ambitious design with a black box marked "magic happens here" which makes it all work. The ideal design is scalable so you could just drop in new nodes on a serial bus, but with this sort of timing you will probably need to drop that requirement. Instead you will need to build a system tailored for a specific number of nodes.I also have a problem with the "simultaneous" requirement, without numbers that is a bit meaningless. Perhaps you mean "within 500ns" or something.One question that doesn't seem to have been addressed, are just trying to save wiring by using a bus? Clearly, 30 connections from FPGA to each cell is a lot more wiring but equally a lot more bandwidth which makes the design a hell of a lot easier. I suspect there is no off the the shelf bus that will meet your requirement, you will have to roll your own.I see a lot of MCUs doing little but driving comms, that is a red flag for me... the MCUs are there for the comms, but the comms requires MCUs. Eliminate the comms, remove the MCUs. You just need some very dumb peripherals, do the work in the FPGA. You could probably put a softcore in the FPGA as well, eliminate the DSP.

author: OwO
author_rank: Super Contributor
timestamp: February 04, 2019, 12:45:44 pm
content: What is the physical layout of the whole system like? are the "cells" spread out over a distance of 0.5m? or are they all clustered together but have to be 0.5m away from the main processor?

author: legacy
author_rank: Super Contributor
timestamp: February 04, 2019, 01:01:10 pm
content: just for comparing, we usually use these three solutions in avionics- ARINC429_highspeed @ 100Kbit/s (low speed is 10Kbit/s)- CANbus_highspeed @ 1Mbit/s (low speed is 125Kbit/s) (1)- EtherCAT_highspeed @ 100Mbit/sec (low speed is 10Mbit/sec) (1)- AFDX_hightspeed @ 100Mbit/s (low speed is 10Mbit/s)usually to connect up to 60(2) devices.(1) experimental, not always approved by QAEtherCAT is usually also used in the industry for big machines.(2) in practice, no more than about 20 units can be sharing a bus to avoid data collisions so we havemultiple cables.

author: tszaboo
author_rank: Super Contributor
timestamp: February 04, 2019, 01:06:16 pm
content: You can use multiple buses for the communication. If you have an UART going from the FPGA to each microcontroller, the resulting speed is just 1-2 mbit/sec. Or couple them into smaller groups. RS485 on such small lengths could reach 10-20Mbit/s. Otherwise, you can use simple devices like shift registers, like SN74AHC594 and SN74AHC595 to write the data parralel and read out serially (1980s serdes). They are fast, if you need immunity, convert the signals to LVDS or something.

author: Scrts
author_rank: Frequent Contributor
timestamp: February 04, 2019, 02:44:25 pm
content: I'd say ethernet. The FPGA should have 1Gbps Ethernet, then distributed over 16 ports Ethernet switch and on the microcontroller side you'd use a simple microcontroller with 100Mbps Ethernet, like STM32.

author: NorthGuy
author_rank: Super Contributor
timestamp: February 04, 2019, 04:11:36 pm
content: If only for PWM and ADC, you don't seem to need any MCUs.However, if you do, the speed you need is not that great - roughly 3 MHz SPI is not a problem for an MCU. If you don't have much noise, it might work Ok over 0.5 m, especially if you fine-tune the timing with FPGA.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 04, 2019, 04:29:34 pm
content: I'd say LVDS. Full-fledged ethernet looks totally overkill for this IMO at first sight, and the timings not that easy to ensure with it, but we may not be aware of all your requirements. Is the "30" number fixed forever, or is it going to change over time? Are separate LVDS lanes OK, or would it have to be a single bus which you can attach as few or as many "peripherals" to as you want? If you implement a separate communication channel for each of your "peripherals", MCUs would do the trick without much issue and you won't need 100Mb/s. Now if you want a single bus on which to multiplex all of the communication from the main FPGA to the peripherals, using MCUs will become tricky.So, you seem to have thought about a solution with a single bus, but have you thought about separate communication channels as hinted above? Would it be acceptable? The added benefit would be that you could communicate will all the peripherals exactly at the same time (as opposed to multiplexed), so that the communication itself could serve as a synchronization means.

author: rstofer
author_rank: Super Contributor
timestamp: February 04, 2019, 06:18:51 pm
content: I would do the entire project inside an FPGA - it's easy to meet the timing when everything is done in parallel. I would tie the ADCs directy to the FPGA with one SPI channel for each ADC. Trivial! Design one SPI channel, instantiate it 30 times.If I still wanted distributed uCs I would still instantiate 30 SPI channels operating concurrently. I would add one signal named, oddly, "GO" and use it to synchronize the uCs such that they take samples concurrently. Maybe use it to generate an interrupt to begin processing. It would be as cleanly synchronized as if everything were in a single clock domain of the FPGA but it could be pretty close.I don't know anything about SERDES or LVDS, I would just have the FPGA output the PWM directly and perhaps add an off-chip buffer. Design one PWM driver and instantiate it 30 times.The power of the FPGA is in synchronized parallel processes and multiple instances of components. Want 50 channels? No problem, just get a chip with more pins and logic. Want 100 channels? No problem...

author: PCB.Wiz
author_rank: Super Contributor
timestamp: February 04, 2019, 06:55:13 pm
content: Quote from: Momchilo on February 04, 2019, 12:03:44 amIf you choose separate bus star design, as given above almost any MCU can manage SPI or UART to a few MBd.For physical layer, if you need more than a direct wire, you could look at M-LVDShttps://wiki.analog.com/resources/technical-guides/mlvds-design-guideIf you need a 16b ADC MCU, that will largely self-select as not many MCUs come with fast 16b ADC. Maybe Kinetis ?SiLabs do have a 14b ADC in their EFM8LB1 series.

author: Marco
author_rank: Super Contributor
timestamp: February 04, 2019, 06:59:49 pm
content: A 16 bit digital to time converter for a max 10us pulse is not trivial in a FPGA, certainly possible with tapped delay lines, but not trivial.

author: hamster_nz
author_rank: Super Contributor
timestamp: February 04, 2019, 07:10:53 pm
content: Quote from: Marco on February 04, 2019, 06:59:49 pmOn the sketch now attached to the original post the PWM frequency is 12kHz... That's about 0.2uS resolution for 16-bit PWM.

author: NorthGuy
author_rank: Super Contributor
timestamp: February 04, 2019, 07:16:21 pm
content: Quote from: hamster_nz on February 04, 2019, 07:10:53 pm1.27 ns, I would say.

author: rstofer
author_rank: Super Contributor
timestamp: February 04, 2019, 07:23:11 pm
content: One thing to consider with ADCs is that they sometimes need 'n' number of clocks to do an 'n' bit conversion and then they need another 'n' clocks to shift out the results. So, bandwidth = 2 * n. Check the datasheet carefully.When working out latency, maybe the individual times don't simply add. Maybe they are pipelined. Maybe you can be doing one conversion while processing the previous conversion. FPGAs do things in parallel and can be overlapped in time.

author: Marco
author_rank: Super Contributor
timestamp: February 04, 2019, 08:22:52 pm
content: Quote from: NorthGuy on February 04, 2019, 07:16:21 pmOkay, that's more reasonable. In that case just use a whole bunch of spi ADCs and 800 MHz LVDS to transmit the PWM, with a fast LVDS to single ended converter on the other side.

author: hamster_nz
author_rank: Super Contributor
timestamp: February 04, 2019, 08:27:39 pm
content: Quote from: NorthGuy on February 04, 2019, 07:16:21 pmDid I divide the wrong way in my head again... 12kHz / 65k = !0.2us?Oh, 12kHz*64k = ~800MHz.Sigh. Should learn to use a pen and paper again.We might all be talking into a void, as the OP has been quite quiet...

author: Momchilo
author_rank: Regular Contributor
timestamp: February 04, 2019, 09:03:26 pm
content: Thank you so much for all the helpful answers so farSorry for the late answer. I went to sleep after the initial post and had to work today.It's part of my master thesis, so I'm not very familiar with all the required stuff and I have to do more research, solve all the upcoming problems and choose the "right" topology. Maybe there are some questionable decisions on my part. The control board with the DSC/FPGA combo is already built and works just fine. Anything else is more or less my decision. But in the end the converter must work.The PWM frequency will be 12kHz. There are 83µs time to do the calculations and data transfers. But the DSC needs about 50µs for the calculations based on the 30 measured voltages and send the results to the FPGA, which further processes them and calculates the PWM values. Then the PWM values are send to the MCUs. That's the idea so far.Quote from: hamster_nz on February 04, 2019, 01:28:43 amThe sample rate is 12kS/s. The PWM frequency is 12kHz.Quote from: hamster_nz on February 04, 2019, 01:28:43 amBecause of the 12kHz, the dead time will be few microseconds. I would call it simultaneously with 1µs delay between the first and last signal. That shouldn't be any problem.Quote from: hamster_nz on February 04, 2019, 01:28:43 amYes, you're right. But the scalability of the cells is very important. It should be quiet easy to increase and decrease the number of the cells. It shouldn't be necessary to flash the FPGA and with more cells the number of I/O pins of the FPGA could be an issue.It's an option, but the last.Quote from: hamster_nz on February 04, 2019, 01:28:43 amEvery ADC is on one PCB/plugin card with a floating power supply. All those cards are plugged in to a "motherboard". The motherboard is connected via 0.5m cables to the main DSP/FPGA board.I hope there won't be grounding issues. 16 bit is chosen, but I need only 14bit. Small inaccuracies are expected and are already included.Quote from: PCB.Wiz on February 04, 2019, 04:20:13 amThat would be for one cell. But there are 30 and it should work with more cells. The faster the better.Quote from: donotdespisethesnake on February 04, 2019, 11:48:12 amI understand what you mean and you named the trade-off. The problem is the scalability. Of course I could build this with fixed 30 cells connected directly to the FPGA and use a PWM frequency of 4kHz or reduce the sample rate (with same multiple sequenced PWM outputs) for a nice 3 phase 50Hz output. But at the moment I figure out what is possible with respect to the scalability without increasing the hardware costs astronomical. There is one point where I have to say "Ok, more than XY cells are impossible with my design and/or the sample rate must be reduced." But I want to have the number of possible cells as big as possible. I will build it with 30 cells. And as I wrote prior in this post, for my requirements simultaneous within 1µs should work well enough. At least I hope soQuote from: OwO on February 04, 2019, 12:45:44 pmEvery cell is one plugin card. They're clustered together, but are 0.5m away from the control board with the DSP and FPGA.Quote from: NorthGuy on February 04, 2019, 04:11:36 pmThe MCUs will do a little bit more. They monitor the temperature and for some events they control a switch to bypass the cell. Not quiet a lot, but every cell would need at least 5 I/O pins of the FPGA.But yes, maybe it's the best solution. I don't know yet.Quote from: SiliconWizard on February 04, 2019, 04:29:34 pmThe 30 cells aren't a fixed number. More and less should be possible. I didn't think about a separate communication channel between the FPGA and every cell. At the moment I don't know how many channels can be implemented in the used FPGA (Spartan 6). With I2C only 2 pins of the FGPA would be used. Maybe it's a nice solution. Have to look into this.

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: February 04, 2019, 10:00:45 pm
content: To send the PWM values to the µCs through a single comms channel, you could do something similar to DMX512: send all the PWM values in order, one after the other, in a single frame, and every µC reads the data that corresponds to its slot/µC ordinal/id number.In the other direction, to send the ADC values from the µCs to the FPGA with a single comms channel you'd need something like that too, but it's not easy to do because every µC would have to insert its data at its slot and there's no built-in ad-hoc comms peripheral to help with that. Ethernet can't do it, CANBUS can't do it, spi, i2c, uart/serial etc, aren't designed to do that. It can be bodged in software with a serial channel and interrupts at low(er) speeds, but there's no (sane) way to do it at 50 Mbit/s.I think you can do with only one channel to send data from the FPGA to the µCs, but you're going to need more than one comms channel to send the data that goes in the other direction (from the µCs to the FPGA), or help from additional, dedicated hardware.

author: NorthGuy
author_rank: Super Contributor
timestamp: February 04, 2019, 10:14:48 pm
content: Quote from: Momchilo on February 04, 2019, 09:03:26 pmJust imagine you wanted to sell something on eBay and, of course, you would want to get as much money for it as possible, so instead of giving a price you would list it for "as much money as possible". Designing for as many cells as possible is equally silly.You first need to decide on the maximum number of cells you're willing to support and then you design everything else around this number. Since your control block is already designed and made, what is the maximum number of cells it can handle? Is this acceptable for you?

author: Momchilo
author_rank: Regular Contributor
timestamp: February 04, 2019, 10:44:27 pm
content: Quote from: NorthGuy on February 04, 2019, 10:14:48 pmThe control block could handle basically, infinitely many. That's the whole point of this multilevel converter concept. The number of cells doesn't matter for the DSP. The FPGA could handle many thousand cells. There is only one limitation in the number of cells and this is the data transfer, which increases with every added cell. Either the I/O pins of the FPGA are the limitation (without using MCUs) or the data rate from the bus.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: February 04, 2019, 11:00:35 pm
content: Quote from: Momchilo on February 04, 2019, 09:03:26 pmYes, the idea is you add more simple but identical links, rather than try to connect all those slaves onto asinglebus. KISS is usually better.FPGAs have plenty of pins. Each MCU only needs 3 or 4.Search Digikey for LVDS, Multipoint and you will find a list of suitable cable driver candidates, that can do half or full duplex.

author: nick_d
author_rank: Regular Contributor
timestamp: February 04, 2019, 11:05:31 pm
content: Based on the previous thread and this thread it would seem that the design is not really set in stone at this stage. It would be a good idea to go back a few steps and explain what you are really trying to achieve. Wikipedia has an article on HVDC converters which went some way to enlightening me, but it talks about thyristors, whereas I understand that you're making a multilevel modular converter which is apparently similar but uses transistors?Are the levels isolated? In that case, how are you planning to get your bus isolated? And would it not make sense to use a separate wire(s) for each level in such case?What we really need to know is the "givens" of the design. Sometimes these are set by customer, sometimes by hardware or software or physical considerations, sometimes by politics... but the proposed designs (FPGA, 30 uCs, ADCs etc) are from what I understand, not givens. It's certainly good to try to solve the problem yourself, but this shouldn't unduly constrain the advice asked for/received.cheers, Nick

author: NorthGuy
author_rank: Super Contributor
timestamp: February 04, 2019, 11:17:01 pm
content: Quote from: Momchilo on February 04, 2019, 10:44:27 pmThe biggest Spartan-6 has 180 DSP blocks. If you run at 200 MHz, this is about 3 million operations per your 12 kHz cycle (if you manage to utilize every DSP at every cycle). If you have 3000 cells, it's only 1000 operations per cell - top max - is this enough?If you only need 1000 operations per cell per cycle, it's roughly 12 MOp/s per cell, which can easily be done by rather small CPU. May be you can off-load this (entirely or partially) to MCUs and decrease traffic?

author: Momchilo
author_rank: Regular Contributor
timestamp: February 05, 2019, 12:38:47 am
content: Thanks a lot for all your help. I'm happy to share my thoughts. I'm not an expert, I'm learning. Please point out every possible mistake/oversight I made and makeYou learn from mistakes, but there will be several, few less would be niceQuote from: nick_d on February 04, 2019, 11:05:31 pmYou're right, everything except the control board (DSC and FPGA) can be changed.Quote from: nick_d on February 04, 2019, 11:05:31 pmExactly. Here you can see the output of one phase. With every additional level (each level is built by 6 cells; 1 positive, 1 negative for every phase) the voltage steps become smaller at the output. With enough levels, you have a superb sinusoidal output without harmonics and additional filters (big expensive heavy inductors) aren't needed. Few years ago the breakdown voltage wasn't high enough of transistors and/or R_on was too high, instead they used thyristors. But with modern IGBTs it's possible to use transistors.Quote from: nick_d on February 04, 2019, 11:05:31 pmEach level (more precisely every cell) has an own floating power supply for the ICs. The measured voltage is galvanic isolated (optical) and the gate driver is also galvanically isolated. Only the digital side of each level/cell has a common ground with the control board.Quote from: nick_d on February 04, 2019, 11:05:31 pmThe DSC and the main FPGA are given. It's highly desired to use a MCU or FPGA on every cell with a shared bus/buses, but if not possible without extremely high effort, other options are possible.For the ADC with the required isolation an ACPL-C87B (Optically Isolated Voltage Sensor) and an ADC (ADS8320) with differential inputs could work.But there aren't finished boards or even layouts. So everything can be changed with the exception of the control board.Quote from: NorthGuy on February 04, 2019, 11:17:01 pmLess traffic than receiving one measured voltage per cell and send one pwm signal to each cell isn't possible. There is no MCU with at least 30 PWM outputs. Both values are critical. The cell voltage is critical for the control and for emergency stops. All PWM signals are based on the cell voltages, phase currents and leg currents (3x positive, 3x negative).

author: NiHaoMike
author_rank: Super Contributor
timestamp: February 05, 2019, 12:46:03 am
content: Have the FPGA directly generate the PWM signals and interpret PDM, to allow the use of simple isolators. TI makes isolated ADCs specifically for that purpose.

author: Momchilo
author_rank: Regular Contributor
timestamp: February 05, 2019, 12:55:55 am
content: Quote from: NiHaoMike on February 05, 2019, 12:46:03 amYes, this would be possible or isolated Delta-sigma modulators like AD7401 with the demodulation in the FPGA. But if possible I would use some kind of busses and separate MCUs/FPGAs.I will look into all your suggestions. But I need some time to do thisBest regardsMomchilo

author: free_electron
author_rank: Super Contributor
timestamp: February 05, 2019, 01:06:09 am
content: DMA bulk transfer linked to an external clock.the FPGA supplies the external clock.A frame is fixed length.you use a wired-or mechanism (READY signal)once all microcontrollers are ready the FPGA begins transport. The first clock cycle after ready generates an interrupt to the microcontrollers that they can start acquiring new data. Once they are done acquiring they need to wait for the DMA transfer to complete. once complete they load the memory with new acquired data and set the ready flag.so in flowchartcpu's acquire 30 readings .cpu copies 30 readings into DMA channel.cpu sets READY HIGH and waits for CLK to go high.when all CPU's are ready the ready pin will go high. ( the slowest one is the lst one to release the 'ready pin'. ready pin is open collector )the FPGA waits for ready to go high.if ready goes high the FPGA starts clocking.the first rising edge of the CLK the CPU clears the ready pin ( makes it low ) and starts a new sequence of acquiring measurements into a buffer.After the last transport clocktick the DMA controller will notify the cpu 'transport complete'. the cpu now copies its buffer into the DMA block and sets its readay pin high.the cycle begins anew....This gives you 2 control signals ( CLK and READY ) and one data line from CPU to FPGA.you can use the DMA controller in conjunction with a USART ( not a UART , you need the SYNCHRONOUS type , most CPU's have such a mode where the send in synch with an external clock ) Most modern cpu;s can do this under DMA control. here is a block of bytes : send using usart under control of clock.Minimal wiring , minimal code, everything synchrounous and self timing.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: February 05, 2019, 01:36:19 am
content: Quote from: Momchilo on February 05, 2019, 12:38:47 amWhat charges these cells (batteries?) ? What appx voltage are they ?There were Audio Amplifiers that used multiple power rails, and switched them in and out to lower the power losses, but I think they rather faded with Class-D technology.You mention 16b PWM, but do not show exactly what that controls ?Does each cell have an IGBT Pair and an inductor, and the PWM sets the % of full cell out ?How do the ON and OFF energies flow ?If they switch at ~12KHz, that indicated ~800MHz of edge placement, which limits MCU choice more, so you may be better sending modulation info from the FPGA, and skip the local PWM entirely.I'd expect you want to have tight control over every edge, not have some serial link added into the mix.A device like EFM8LB1 could read 14b of Analog, and transmit serially at 18MBit over a Digital Isolator, for ~1.1 us link times.The TI isolator ISO7721 can manage info flow both ways, and specs typical skews of 0.5ns, for precise edge control.

author: dmendesf
author_rank: Frequent Contributor
timestamp: February 05, 2019, 02:13:59 am
content: Seems you're doing a multilevel inverter. I helped make one before. Used a uC for each cell + 485 bus to drive them from a stm32. Used 3 buses, one for each "leg" (phase).

author: David Hess
author_rank: Super Contributor
timestamp: February 05, 2019, 03:03:37 am
content: Quote from: PCB.Wiz on February 04, 2019, 11:00:35 pmI agree with the above except for the need for dedicated drivers and receivers. The FPGA has plenty of I/O to divide the MCUs into individual channels or small groups. The much lower data rates, 16 bits in 10 microseconds is 625 nanoseconds per bit, combined with the transmission line delay of roughly 3 nanoseconds considerably relaxes the driver and receiver requirements if some care is used.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: February 05, 2019, 04:35:01 am
content: Quote from: David Hess on February 05, 2019, 03:03:37 amYes, the OPs' original post did not mention any voltage offsets, but later posts suggest this certainly will need voltage isolation.In that case, the TI isolator ISO7721 I mentioned is likely good enough, and LVDS would not be required.LVDS-M was suggested from a view of EMC and noise rejection, as much as 'can it work', plu sit is lower power then RS485 alternatives.

author: NorthGuy
author_rank: Super Contributor
timestamp: February 05, 2019, 04:48:02 am
content: I think you overcomplicated things a bit.I don't think it's a good idea to isolate ADCs. I would have few ADCs with ground on the bottom rail, which would be constantly measuring voltage at all the necessary points through resistive dividers. Each ADC can handle several channels. Even one ADC with a high sample rate would do. There are MCUs which have enough channels and speed for adequate sampling.Then I would have wires going through optoisolators to switches.So, a cell would contain optoisolators, switches, voltage divider for ADC, temperature sensor and interface.IMHO, an MCU could do all this quite well. Certainly won't be a problem for FPGA.

author: Marco
author_rank: Super Contributor
timestamp: February 05, 2019, 08:55:24 am
content: A resistive divider might be too slow depending on the voltages involved.

author: nick_d
author_rank: Regular Contributor
timestamp: February 05, 2019, 09:06:43 am
content: About multi-drop buses:10 Mbit/s Ethernet, PCI and PATA are examples of multi-drop buses. They originally ran at 10 MHz (Ethernet) or 33 MHz (PCI, and intermediate versions of PATA). Enhanced versions have gone up to 66 MHz (PCI) and even 133 MHz (PATA with the special 80 pin ribbon cable). In the PCI case, it's all within the same PCB, which makes things a lot easier. PATA uses a ribbon cable. So in theory, if they can do that, you can do it too. However I suspect that some black magic is involved, I personally wouldn't vouch for a multi-drop bus running over any kind of cable at more than 10 MHz. But, remember this is MULTI-DROP. If you go to point-to-point links, the rules change.100 Mbit/s Ethernet, PCIE and SATA are examples of point-to-point buses. By using a point-to-point link, you have much better control over the inductance of the link and issues like termination and reflection. This is good for 100 MHz without significant problems, even several Gbit/s if special techniques are used. However, note that you should really use dedicated transceivers for this kind of thing, if you just took a 74AC gate and tried to send 100 MHz data over it with a long cable, it would be problematic for a number of reasons. Noise pickup, ground issues, etc. So you should definitely be going differential, and preferably it should be isolated, as in Ethernet.Now with that background, considering your problem, it would be 100% easier and better to have a separate connection or connections from your FPGA to each converter level. You've specified 100 MHz, and this just won't work well over a shared cable. Even if you used special transceivers and cables to make it work, there is also the problem of the connector, which is always going to add a lot of stray impedances, and a loose connector could bring the whole system down. If there are wiring considerations that make the single cable approach unavoidable, the best I could suggest would be a daisy-chain, that is, a series of point-to-point links (repeaters) from each level to the next.Another way to do the multi-drop bus (if it absolutely must be a multi-drop bus) would be to put everything on the same PCB: the DSP, the FPGA, and the 30 levels. It would be a big PCB but quite workable. If you have a multi-drop bus WITHIN the same PCB, then with good layout techniques you could make it work up to 100 MHz. However, there'd be no point because the PCB would allow a star topology just as easily as the multi-drop bus.So clearly the right thing to do is a star topology: reserve a couple of FPGA pins for communicating with each converter level, and have a big bundle of wires going to the level PCBs. Given that modern FPGAs have hundreds of pins, it would not place significant limitations on your design. And, the data rate would be so much reduced (kbit/s not Mbit/s), that no special techniques would be required, you could just connect the FPGA pins directly to the cable with say a 1k resistor and potentially a couble of diodes (one to each rail) for protection.About isolation:Isolated ADCs are going to be very expensive. Isolated PWM output slightly less so, since in general the high side driver needs to be isolated anyway, but even so, it would be much easier to put the isolation in between the micro and the FPGA, rather than between the micro and the PWM + ADC. A good way to isolate the communication between FPGA and micro would be to do it like the old MIDI standard: an opto-isolator is placed on the receiving PCB, and the sending PCB gets a pair of wires connected to the opto, which it can use to turn on the LED of the opto. (The opto can also be placed on the sending PCB and the open-collector output sent along the wire to the receiving PCB).Another approach might be to use fibre-optics instead of wires. So basically the LED is on the sending PCB and the receiver (light detector) is on the receiving PCB. Look up S/PDIF with TOSLINK connectors for an example of this in practice.Also, the micro could possibly be deleted if you use the right ADC. Certain kinds of ADC only require a clock input, and will regularly initiate a conversion and clock out the results. For example, many audio ADCs work like this, and can send the results in S/PDIF over a TOSLINK connector. I don't suggest to use an audio ADC as such, because they often can't handle DC and you need to measure the DC, but I am sure that an appropriate ADC could be found. If the conversions need to be synchronized, you'd need an extra opto (or fibre-optic cable) to distribute the conversion clock. As to the PWM, how about having one opto (or fibre-optic cable) for the low side and another for the high side, it has the advantage of simplicity and that the FPGA can generate the needed dead time, but may be unsafe as both can be turned on simultaneously. Or possibly just have an opto letting the FPGA drive the low side, and generate the high side with appropriate dead time locally based on what you see at the low side.cheers, Nick

author: Rerouter
author_rank: Super Contributor
timestamp: February 05, 2019, 10:03:40 am
content: I'll start out simply, you want to do as much as possible in the least amount of effort possibleThe simplest way would indeed be to give each micro its own SPI line, dedicated comms, and the data rate can be very high, while keeping to really cheap micro's, It would only take 16 SPI clocks to cycle in and out the values you needed, as long as its a 16 or 32 bit device, you write your ADC value in the SPI register, then as it writes in the new PWM value it clocks out the current ADC readingNext step up involves the same design, but with cost reduction in mind. have a SPI ADC and a 8 pin Micro with an SPI port and a timer that can handle your desired frequency and resolution.So you have 5 wires running through an isolator of your choice. SCLK, MOSI, MISO, CS, PWM_CLK, 4 in, 1 out, You have the ADC and the Micro share the Chip select line, but with opposite activation states, Select the ADC, clock out the reading, Select the Micro, clock in new PWM value.The second option would be a pair like soMicro:https://au.element14.com/microchip/attiny402-ssnr/mcu-8bit-avr-20mhz-soic-8/dp/2915533main pain then is just finding the ADC, 12 Bit is easy, but 16 bit at low cost is a right pain, Is there a reason for the 16 bit requirement? its seriously hard to layout circuits to make proper use of the last bit.

author: vealmike
author_rank: Regular Contributor
timestamp: February 05, 2019, 11:24:20 am
content: Sounds like a job for Daul / Quad SPI, one link to each endpoint.For the processors, take a look at the PSoC. The integrated CPLD will allow you to have minimal processor intervention in your control loop and to implement your own FPGA comms if necessary.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: February 05, 2019, 06:30:18 pm
content: Quote from: Marco on February 05, 2019, 08:55:24 amYes, plus you lose precision, and a local MCU is likely needed for failure handling and overcurrent reaction. MCUs are very cheap these days.Of course, a host side MCU could be addedas well, as another system-monitor task, but it's unlikely to be fast enough for control and protection.

author: NorthGuy
author_rank: Super Contributor
timestamp: February 05, 2019, 06:47:32 pm
content: Quote from: PCB.Wiz on February 05, 2019, 06:30:18 pmYou need a voltage divider anyway. As to the resolution, you're unlikey to get accuracy better than 0.1%, which, in case of a 16-bit ADC, leaves you 6 bits of extra resolution.But there's a huge advantage. When they're all referred to common ground you don't need isolation whether you use one MCU or a separate MCU for every cell.

author: Marco
author_rank: Super Contributor
timestamp: February 05, 2019, 07:33:45 pm
content: A resistive divider is probably an option at 1kV, probably not at 10 kV.

author: Scrts
author_rank: Frequent Contributor
timestamp: February 05, 2019, 09:25:02 pm
content: Quote from: vealmike on February 05, 2019, 11:24:20 amQSPI over 0.5m cable will end up with garbage. Been there. We had to use LVDS drivers/receivers for such application and it still wasn't 100% robust.