Venturing into the brave new world of LVDS... - Page 1
Forum section: electronics / projects

author: SpacedCowboy
author_rank: Frequent Contributor
timestamp: April 20, 2022, 06:56:56 pm
content: So I have an FPGA on a board I'm designing, and I want to be able to plug "daughter cards" (also with an FPGA on) into a PCIe connector - I'm just using PCIe as a convenient connector, it's not going to be PCIe signaling.The FPGA provides SERDES on LVDS pins so I can run a slow internal clock (say 100MHz) with a gearing ratio of 8:1 to get an 800mpbs LVDS signal. This is *way* faster than anything else I've ever sent over even a single board, let alone something with a plug-in...The FPGA also provides internal 100R termination for LVDS at the receiver, it's just a matter of enabling it, which presumably means I don't have to do anything to the traces themselves to terminate them.Beyond that, my ideas for how to get this to work are:Impedance match the trace-pairs to a differential impedance of 100R - probably using microstrip referenced to a second-layer ground plane wherever possibleLength-match the _P/_N traces from FPGA pad to PCIe pad to within 5 mils of each other - this seems to be the accepted wisdom from reading around a bit. Plan to do this on both sides of the slot (motherboard and daughterboard), and hope that's good enough - I'm not really sure what else I can doLength-match the separate trace-pairs to within a cm or so - which seems huge, but at 800 MHz, the period is 1.25ns, microstrip propagation is about 60ps/cm, so if there's 1cm of skew, the signal could be ±60ps or ~5%Make the diff-pairs use symmetric layout around a common center, so if there are resistors along the way, the paths are mirror-images of each other.Make sure the GND,Power planes are continuousAngle corners at 45 degrees rather than 90 for EMI. Rounded would be better, but I don't think my skills extend that farDon't run diff-pairs away from the GND plane, or close to the edge of the PCB. If the _P,_N are separated by x, ake sure there's 2x separation between diff-pair sets of traces, and 3x between diff-pairs and single-ended.If I'm missing anything here, I'm quite keen to hear about itI do have questions I'm not really finding answers to...Does it matter if I reference a ground plane or a +vcc plane as long as the differential traces are one plane away ? So if the layer stack-up is {Signal, GND, +3.3v, Signal, +1.8v, Signal} can I route diff-pairs on the bottom/center layer as well, or is my reference plane always to GND (in which case, I'd need multiple GND planes, presumably) ?Assuming I have to use vias - some of the pins on the FPGA will require that - is it better to run the LVDS as stripling throughout on the interior layers to the PCIe pins (which are through-hole, so available on every layer), or to pop back up to the top layer as soon as possible and run on the top layer as microstrip ? The assumption here is that I match the vias across _P,_N traces, and preserve the separations of the traces as far as possibleI know the standard advice is not to run power traces such that they cross a diff-pair, but if there's a GND/power plane in-between the layers where the diff-pair is placed, and the power signal is run, does that plane "shield" the diff-pair signal, and I could cross over the diff-pair with (say) Vsupply = +5v - as in {layer with diff-pair}{GND}{+5v signal routed perpendicular to diff-pair} ?I also have a different, tangentially related, question to do with higher-speed routing... I want to put a microcontroller on the daughterboard, and I want to be able to program the flash of that microcontroller from the motherboard - but the flash is accessed via QSPI @100 MHz by the microcontroller, and I'm concerned about having long traces going all the way back to the motherboard microcontroller when it's being accessed as QSPI rather than being programmed over SPI. I was wondering whether an analogue mux close to the flash on the motherboard might alleviate that - there'd still be small stubs on the QSPI signals, but it seems like it might be a lot better than a 10-15cm trace...Thanks very much in advance for any help - hopefully I can reduce the number of board-spins I have to do to find out the answers myself

author: T3sl4co1l
author_rank: Super Contributor
timestamp: April 20, 2022, 08:13:28 pm
content: Use lots of grounds in the connector. Alternate signal and ground pins, at least. Or GND, diff_P, diff_N, GND.Check that the connector specifies differential impedance; for PCIe, it probably does, and so, follow that layout (probably as above, but if different, do that). These don't have ground plane I think, so aren't real great, but that's also something differential can deal with okay. The CMRR should be good enough not to have emissions problems, though it can still fail on ESD. Keep both boards inside a shielded enclosure, to sink ESD away from them. (For development, doesn't really matter, I mean for, if this becomes a proper project.)Planes: crossing over gaps, or between layers, is fine. Some CM noise can be picked up, due to the higher local plane impedance (i.e. around edges). Ideally you'd put a bypass cap (or several, flanking) around each such crossing. In practice, it doesn't matter, maybe unless you're doing extremely precise timing / high spectral purity or something.You can imagine the wavefront from a transmission line, crossing over such a gap, or along a via between layers: the energy of that wave, crosses the gap, and curls underneath the edge, and spreads out (all around the via, or fanning out the one side of the gap). As the wave spreads out, the impedance decreases, going inversely with distance. So, the impedance has an inductive characteristic: higher at higher frequencies, i.e. shorter waves see more cycles (or a greater fraction of a cycle) closer to the via. Simultaneously, the image current (of this fanning-out wavefront) appears on the plane below the gap (say, GND behind a VCC1-VCC2 plane gap), carrying that current over to the opposite side, either through a via hole to the opposite face, or following the trace between inner layers (stripline between planes, 6+ layer build), or following back up the opposite side plane in a mirrored fashion, thus crossing the gap.The planes are much, much wider than the signal traces, so even up close (say within 10ps), the impedances are low 10s of ohms, dropping to single ohms in not much more time. So you really need quite a sharp wavefront to even sense all of this. It's a kind of thing that's evident on a SD-24 TDR or the like, but outside of such screaming-fast comms, really isn't an issue. And again, digital levels still get there fine, we're talking upsets of low 10s % in the waveform; you'll get as much loss on the leading edge due to FR-4 losses. Digital data that's synchronized at the target (separately clocked with adaptive phase, or with clock recovery from the same bitstream), will be fine. And indeed, this is, roughly speaking, what PCIe does, so you're in good company. (Your serdes might not -- YMMV?)So, as long as planes are wide enough to have good coupling together, this will happen, the wavefront spreads out inbetween, dropping to imperceptible signal levels (~mV) and causing only minor upset to signal quality.And also, mind that the above story goes for a single trace over such features -- a differential pair, their wavefronts superimpose and largely cancel out at a distance, so the only effect on the (differential) signal is a tiny blip, on the order of the pair spacing. For which you'd need extremely sharp signals indeed, to run afoul of.And yes, planes are good shields. Keep signals away from edges, gaps, via or other holes, generally, any negative space in one or both planes -- but if you have to approach such features, it's not the end of the world, and doing so symmetrically is the important part. Traces on other layers (opposite the planes) are essentially irrelevant.Note that likely noise on power supplies, is much lower frequency than signal lines are. You could run a diff pair, top layer, even over an SMPS switching node trace, mid layer 1, without much effect; SMPS noise is "only" low 100s MHz, at least for typical devices. The amount coupled in, by one trace-pair crossing, is negligible. (Some newer fancier parts do run faster, and obviously you don't want to invite such circumstances to begin with -- but for sake of argument, you'll likely still have fine reclocked data coming out. Which, again, if it's not reclocked but sensitive to timing, that's a bigger concern, etc.) Likewise, separation to other signals isn't a big deal, but it adds up over the distance -- momentary close approaches are fine, tightly-grouped buses less so.What else... 90 degree corners are essentially irrelevant, that's really only a concern for 10s GHz, where the wavelength / wavefront struggles to stay within the transmission line in the first place (TL width/height a sizable fraction (<0.25x) of wavelength) -- but basically everything does 45 degree traces out of the box, it's handy and ready to go -- and most of all, it looks better.And can always go through and clean up corners with arcs when you're ready to fab, if you really want to polish it up -- may not be that difficult depending on what you're using.As for the QSPI thing, yeah, a mux/switch might be a good idea. Not sure where you'd place the source termination resistors in that case; maybe a set right by the mux, another set by the source (on the motherboard)? Signal quality won't be great, but if you aren't trying to blast it at 100M from that end as well, that should be okay. Bidirectional switches/muxes are available with quite good signal performance at 3.3/5V supplies, and that's probably better than having to manage signal direction with bus transceivers (the usual parallel-bus solution, but QSPI is not a usual parallel bus). May be a bit expensive/annoying to get all the traces muxed in this way; if you can run single lane (SPI) and not mind a slower transaction from the motherboard, you can likely use fewer / a smaller/cheaper mux. If the daughterboard MCU can be disabled (e.g. held in reset, GPIOs tristate?), SPST switches will do, otherwise, you may want/need SPDT switches.Tim

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 20, 2022, 08:21:34 pm
content: Note that you can run into clock skew issues especially for higher clock frequencies rather easily - that is, if you implement a communication scheme transporting the clock signal on a separate pair. Probably note an issue if you route signals properly and limit lengths to a few inches. But for longer cables, even at 800 Mbps, you'll get problems.To avoid this, at higher clock rates, we avoid transporting a separate clock signal and combine data and clock on the same pair, using some line code. Of course, for this, you need a DCR (data/clock recovery) at the receiving end, which complicates things.That's something I'm currently working on, so that piques an interest of mine.

author: SpacedCowboy
author_rank: Frequent Contributor
timestamp: April 20, 2022, 09:53:48 pm
content: I know I clicked on 'Say Thanks' to both of you, but just wanted to say it again - especially Tim - that's a *lot* of incredibly useful informationI was planning on putting lots of GND pins on the interface - the placeholder at the moment is a PCIe x8 slot, which is ... somewhat overkill for 4+1 trace-pairs . There will be other single-ended lines too, but nothing of any significant bandwidth, maybe 6 more signal lines tops (SPI + a couple of others). I was thinking of downsizing to a PCIe x1 slot (36 pins) which still gives me a *lot* of pins for power and ground.The total trace length is reasonably small - the motherboard is basically an FPGA, assorted other chips and a separate high-speed bus (parallel, this time) with all that positioned "south". The slots will be "east" and "west" of the FPGA, probably 3" at max, and then maybe extending another inch on the daughterboard. I actually think it'll be less than that - these are absolute "worst case" figures.I haven't actually tried this out yet, it's still in the "thinking" stage, but the plan was to send a clock signal as a separate LVDS-pair to the data - although to be fair the *extent* of said thinking is "HDMI and MIPI do it that way and they know more about this stuff than I do"... So I'm open to persuasion that embedding the clock in the data stream is a better option - it does seem that it would make the logic at both ends more complex though, and I don't think the IP on the FPGA for LVDS does it for me. The FPGA claims are:• Dedicated LVDS TX and RX channels (the number of channels is package dependent), and one dedicated LVDS RX clock• Up to 800 Mbps for LVDS data transmit or receive• Supports serialization and deserialization factors: 8:1, 7:1, 6:1, 5:1, 4:1, 3:1, and 2:1• Ability to disable serialization and deserialization• Source synchronous clock output edge-aligned with data for LVDS transmitter andreceiver• 100 Ω on-die termination resistor for the LVDS receiverSo the plan was to run 4 of the LVDS channels + a clock channel and see how that went. Ideally I would (eventually) want two of these plugin daughter cards, but it *looks* like I can designate any RX as a clock input and feed it into the PLL that drives the rest of the RX LVDS blocks with appropriate phase manipulation. I need to confirm that, otherwise embedded clocks may be in my future, given there's only one dedicated RX clock pairThe QSPI thing, I'm fine with just running it as 5-10 MHz SPI from the motherboard, and the target chip will indeed be held in reset while I'm reprogramming its flash, so I'll have a look at the muxes available.I don't know if this will ever get to "official project" status, at the moment it's just a hobby-thing, but I may have aspirations , and I was thinking a metal box would be a good idea in that case - or at least metallized spray on the inside though I don't know how well that stuff works.... A bridge to cross some time in the futureOn that note, though, I've never quite been sure how terasIC get away with things likethisThanks again to both of you - *really* appreciated.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: April 20, 2022, 11:21:03 pm
content: Oh, also left out but meant to add: power and ground are identical for AC purposes. That's what all the shared plane area and bypass caps are doing. So it's fine to have e.g. VCC sig_P sig_N GND and it counts the same. Maybe better isolate any supplies that need to be super clean/quiet, but then again, you're probably going to need to filter them locally anyway, in which case whatever is picked up over the connector is just about irrelevant.Yeah, for a basic starting-out project, if you don't have hardware/IP blocks handy to do clock recovery, probably easier to send a separate clock pair... The other thing you can do then, is make a phase shift / delay block at one or the other end, and adjust it (from NVRAM say, or perhaps even automatically, given a measure of bit error rate?) to account for clock skew. Which can be applied to each lane if you like, to resync any number of pairs.If you've ever adjusted the phasing/timing on an LCD monitor connected with VGA or analog DVI, you've seen exactly this in action. The exact pixel values are fairly stable during a symbol time, but exactly out of phase with the pixel clock, it suddenly becomes noisy and blurry -- as timing errors are magnified across transitions, smearing the level between adjacent pixels. And, tweaking the RX pixel clock, this can occur as a wave across the screen (as phase rotates around during a scan), or altogether (when frequency is matched but phase is out). You of course want to set it 180 degrees away from this transition level, for best results. Or, maybe like 90 degrees ahead so there's more settling time but still plenty of timing margin; it depends. (Probably matters more for analog video, whereas digital levels, it doesn't really matter how much it's settled out during a bit time, as long as it's well past the threshold.)Some things do this timing automatically, like, RAM starting with DDR3 or 4 I think?, has sub-buses, where every 8 bits or so are synced to within some modest amount, then every byte in the interface is lined up over a somewhat larger margin, and, I forget if there's compensation for the clock and bus signals specifically, separate from the rest of the bus, but something in any case. So they're quite tolerant of variations in trace length, within certain limitations: bytes need to be routed carefully enough, but words/dwords have some slack between their bytes, easing the amount of length equalization needed on board. In addition to preshoot (to compensate for FR-4 losses), hysteresis, and edge blanking (no transition is received within some 100s ps of an edge, so as to ignore ringing due to stub lengths). And semi-differential, with individual signals being single-ended but intended to route over a mid-supply VREF plane, and the receivers are differential with respect to that; so the noise margin is much larger than plain old LVCMOS (which aside from the preshoot trick, the transmitters otherwise mostly are).Or maybe I'm confusing that with some features of PCIe or other high speed standards, I don't remember -- it's been some years since I looked things up, haven't had to work with them in a while. Anyway, there's some very clever stuff done with mass-market high speed standards such as these, to deal with the vagaries of cheap FR-4 PCBs and their routing. Things which, perhaps, you'll miss on a more from-scratch build -- but also appreciate quite well as a result? Heh!Quote from: SpacedCowboy on April 20, 2022, 09:53:48 pmEasy -- dev kits aren't commercial products, in the same sense that is. They don't need FCC or other approval, they're sold for operation by qualified personnel (well, hopefully.. ideally..), and not sold with any kind of guarantee, I mean it's a dev kit after all, it's up to you to make it do anything.Rather galling when it comes to say SMPS eval boards, when they belch out nanosecond spikes on all wires because an idiot did the layout -- especially so when the manufacturer makes some claim about EMI performance of the parts used, or you ask their FAE and they don't know anything better -- but again, such it is, presumably the eval board works for the thing it's meant to do (whatever it was measured for in the appnote), and anything beyond that is merely incidental.Tim

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 21, 2022, 02:36:05 am
content: Quote from: SpacedCowboy on April 20, 2022, 09:53:48 pmAs I said, you don't need that if you route your signals carefully, for just short board-to-board communication at sub-GHz rate. Just thought I'd mention it, as I'm currently working on it, and clock skew can become a real problem for high-speed communication over cables, especially at rates higher than the GHz mark.Yes it would make things more complex. Not awfully complex on the TX side though - but problematic on the RX side, as that would require a clock/data recovery block that you won't get in most FPGAs (maybe you can get that on the very high-end ones...) So, your options for RX would be either to use an external CDR chip (TI and AD have them, among others), or use oversampling - but doing proper CDR at even 800 Mbps using oversampling is probably not going to be feasible either unless you use maybe a very high-end one. So forget about that. =)Just for the record, HDMI has a separate clock signal, but DisplayPort doesn't. Typical busses using embedded clock in data are many these days: USB, Ethernet, DisplayPort, Thunderbolt, PCIe (at least starting with a certain revision I think)...But anyway, you can surely forget about that in your case. You may need to consider other factors though, such as measures you can take to minimize EMI. For instance, HDMI may have a separate clock, but it still uses a certain encoding of data (TMDS)... You have also some sort of scrambling for PCIe, etc. Fun stuff.

author: SpacedCowboy
author_rank: Frequent Contributor
timestamp: April 21, 2022, 03:05:25 am
content: Quote from: SiliconWizard on April 21, 2022, 02:36:05 amYeah, this is a very *low* end FPGA - an Efinix Trion T20Q144C3 - 20k LE's for $12. It was originally going to be the QFP T8 for $6, but someone bought the last few thousand of them in the last couple of days...I've done some display-port stuff before, but only on the microcontroller firmware side, and I think the FPGA peeps used Xilinx' DP IP anyway. I know it was a bugger to get the link up - Xilinx hardware is pretty top notch, the software division aren't as great, IMHO.Anyway, There's an app-note for how to do source-synchronous designs, so hopefully following that won't lead me too far astrayThanks for all the help, as well