PXIe instrument in a regular PC - Page 1
Forum section: electronics / projects

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 29, 2023, 08:41:35 pm
content: I started a thread in the RF section because the device I'm interested in is an RF test instrument.https://www.eevblog.com/forum/rf-microwave/ni-vector-signal-transceiver/The actual realization more belongs here I believe so I'm posting this for folks who care.Here's where I'm at right now.-Electronics is pretty much done.-I'm trying to figure out if I can get a sheet metal cage going (probably sendcutsend.com)Depending on my mood I'm certain it's going to work easy-peasy, but on lonely cold nights, while contemplating about the meaning of life and the universe and stuff I'm not sure if-the PCIe extension cable won't be too long-there is some sort of software check for the driver to check the chassis parameters (I know there is some on newer models)-cooling will be adequate or efficient enough (even with modifications to the PC drive cage)The PXIe standard is free, but it heavily references the CompactPCI Express standard which is not and I didn't feel like paying $750 for it, so I'm trying to piece this together from random internet resources.The cage sketch around the device on the first picture has the dimensions of a double height 5.25" drive bay. I was thinking getting a refurbished Dell Optiplex for ~$150 to put it in.

author: coromonadalix
author_rank: Super Contributor
timestamp: May 30, 2023, 06:58:54 am
content: something like this ?https://www.sundancedsp.com/2562-2/smt580-pcie-to-pxie-adapter-card/385 pound i think ? and yes darn expensive for that boardIs it only pcb or cable lanespossible / direct interfacing without dedicated ic's ?? really intrigued by your projecthttps://www.scientific.net/AMR.718-720.1262

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 30, 2023, 12:53:25 pm
content: Yes, totally like that. I haven't seen this one, so I'm glad as it serves a proof for the validity of the electrical concept.I just want to make mine somewhat better integrated mechanically.I was struggling with fitting the whole thing into double 5.25" bay and can't find a good solution.I was looking at PC cases or old refurb PCs that have 3x 5.25" bays, but they are much rarer, and accessibility is a major point here.Another idea that I might implement is to ditch the direct PCIe cable and use a Mini SAS HD adapter and cable. I'd have to redo layout completely on the board, but it'd fit a lot better and the cables are cheap. Even with the adapter it's below $50, while the 3M PCIe cable I was looking at before cost almost $100.https://a.co/d/7kzoLJdhttps://a.co/d/4mQAcyVhttps://mou.sr/3IM0VR1My only concern is signal integrity. I don't have a good feel for how much margin I have here. The PXIe-5644, that is the target device for me is Gen 1 PCIe and I found some contradicting info on how does it behave when cabled.

author: Berni
author_rank: Super Contributor
timestamp: May 30, 2023, 01:05:56 pm
content: You can push PCIe pretty far in terms of signal integrity.I ran PCIe trough some ribbon cable and a bunch of bodge wiring at one point (Agilent PC based scope with a proprietary pinout), worked perfectly fine.I also seen some youtuber chain together a bunch of PCIe extenders to see how far it would work. The huge chain ended up something like 2 meters long before it stopped working.

author: coromonadalix
author_rank: Super Contributor
timestamp: May 30, 2023, 03:59:18 pm
content: I'm no pci / pcie specialist, but i think you have differential pairs on some signalsit normally would not be a problem

author: samofab
author_rank: Regular Contributor
timestamp: May 30, 2023, 06:11:41 pm
content: How about this? Create an adapter for FFC cable to motherboard then mate this cable to connector on the PXIe end (as opposed to mating it to adapter). this should be much more compact on the PXIe side, perhaps enought for original plan of fitting into 2x 5.25" drive bay?

author: coromonadalix
author_rank: Super Contributor
timestamp: May 31, 2023, 12:30:45 am
content: problem would be power lines ??

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 31, 2023, 12:32:16 am
content: Quote from: samofab on May 30, 2023, 06:11:41 pmI'm going for something similar I just want to choose something that's at least semi-available in the consumer space.I like the MiniSAS HD, except it has a really irritating pinout when used for PCIe. The lanes are organized in a square pattern and if I break that out it becomes 0-1-3-2 it annoys the f out of me.But going down this rabbit hole at least saved me a board spin, because I realized that in CompactPCI Express and thus PXI the peripheral lane directions are swapped on the backplane. (the system slot's TX pins go to peripheral slot's RX pins) I was doing TX to TX as that's how it's referenced in regular PCIe. So I totally had that messed up.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 31, 2023, 12:34:01 am
content: Quote from: coromonadalix on May 31, 2023, 12:30:45 amI have to power it separately anyway. The full SATA power connector has all 3 voltages the PXIe module needs (3.3V/5V/12V) and I have terminals for 3 sets to make sure I can pull enough current through.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 31, 2023, 12:43:01 am
content: Who wants to bet if I'm going to mess this one up too?

author: jhenderson0107
author_rank: Regular Contributor
timestamp: May 31, 2023, 03:04:26 pm
content: Herehttps://www.pxisa.org/pxi5_pxiexpresshw_r11/#dearflip-df_rand225093977/11/is a link to the PXI Express standards document. Be sure to incorporate all of the control signals so that all operational corner-cases are covered. For example when rebooting the host or powering the target chassis after the host has already booted, the target cards must re-enumerate. This process relies on sideband signals.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 31, 2023, 05:04:34 pm
content: Luckily, at least if I understand correctly, hot swap is not a required feature, and that significantly simplifies all the control signals.The PXIe spec is not much help though because for anything covered there it references the CompactPCI Express spec, which is unfortunately not free.I'm not going to have any of the PXI specific signals either, e.g. triggers, but I'm only intending to connect one module anyway.

author: jhenderson0107
author_rank: Regular Contributor
timestamp: May 31, 2023, 06:09:05 pm
content: If you omit hot-swap, then the slave chassis must be powered prior to the host so that the targets are available during PCI bus enumeration. You'll still need the ref clk, perst* and a few other critical signals, similar to the excerpt below (which is XMC, not PXIe).

author: Marsupilami
author_rank: Frequent Contributor
timestamp: May 31, 2023, 09:10:56 pm
content: There is no host chassis, it's all one computer. There is no PCIe switch or bridge in the way that has to init before the host does.I have the reflclk, reset, present ,wake and so on.I even added a spot for an XO to generate the 100MHz PXIe clock as I've seen a module before that refused to start up without that. I doubt it's needed though.This is where I'm at right now:Added a bunch of SATA plugs (P10,P11,P12) to use the 15 pin power connector from the ATX PSU. I couldn't find any that's 15 pin.I'll keep the wire terminals too for flexibility. (P5,P6,P7)Amazon meanwhile bough me this. I ordered it ahead to be able to check the pinout. It's definitely an easier form factor than the twinax ribbon cables and the card and the cable was less than $25 combined.Routing the diff pairs isn't as bad from the SAS connector as I though it would be. There is one awkward loop, but nothing too bad. Still have to do length matching within pairs.

author: Hydron
author_rank: Super Contributor
timestamp: May 31, 2023, 10:33:35 pm
content: RE the worries about cable length - if you use a good cable then for any reasonable length the losses in it will be WAY lower than what you get from the short traces running over the motherboard/card PCBs. Spend your time worrying about connectors and adapter PCBs, not about a half meter of (appropriate) PCIe cable. The SAS stuff should be a good choice I think.

author: coromonadalix
author_rank: Super Contributor
timestamp: June 01, 2023, 01:02:04 am
content: Does it need signal traces the same length / balancing from one connector to the other ?

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 01, 2023, 04:47:17 am
content: Quote from: Hydron on May 31, 2023, 10:33:35 pmThanks, that makes me feel good about it.I'm trying to impedance match the traces well and they will be only about 1.5in long max.My only concern is about the stubs presented by the connector pins. On the ADF connector (CPCIe) the pins are longer than the board width, so the worst case is for signals that connect to the pin on the primary side of the board. I found apresentationdetailing the use of "boomerang vias" but it was referring to Gen 4 16GT/s PCIe and for me the target device is only Gen 1 2.5G, so I'm hoping I can get away with it.I could simulate it, but I'm not sure it's worth the effort and I don't really have an alternative anyway.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 01, 2023, 04:51:15 am
content: Quote from: coromonadalix on June 01, 2023, 01:02:04 amThe PCIe spec says that the lane-to-lane output skew should be less than 1300ps, which is about 8 inches of trace, so even with the unknown parts of the signal chain I'm certain it's a no issue. The two conductors of each differential pair has to be matched pretty close, I try to get them within a few mils. It's not done yet on the screenshot I posted, but will add little wiggles to even them out where needed.

author: Berni
author_rank: Super Contributor
timestamp: June 01, 2023, 05:16:56 am
content: Yep PCIe actually synchronizes itself on a pair by pair basis, so the length matching between pairs is not really required (as long as you don't go out of the synchronization range)You can even swap the P and N wire in a pair on PCIe randomly and it will still work due to the encoding used. This lets you route the PCIe traces around without having to use extra vias to swap pairs around.Making sure you are getting full PCIe speed can also be tricky since even with broken pairs PCIe will still work, it just falls back to a more narrow bus.

author: Hydron
author_rank: Super Contributor
timestamp: June 01, 2023, 07:23:27 am
content: Yeah try very hard to match within a pair, but you're free to totally ignore matching between pairs or the P/N polarity as noted. I try and not leave the long stub from the pin by being careful about what side I route on, but I've also seen it work at gen4 speeds even with a connector stub. You'll almost certainly get away with it at gen1.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 01, 2023, 08:50:38 pm
content: Quote from: Berni on June 01, 2023, 05:16:56 amOn that note do you guys know of any software that can get diagnostic info like that?

author: switchabl
author_rank: Frequent Contributor
timestamp: June 01, 2023, 08:54:50 pm
content: On Linux, I thinklspci -vvgives a lot of diagnostic info.

author: Hydron
author_rank: Super Contributor
timestamp: June 01, 2023, 09:51:47 pm
content: Quote from: switchabl on June 01, 2023, 08:54:50 pmYep this is what you need (run as root), a warning though it's VERY spammy. Look for the LnkCap and LnkSta lines for the device in question, these show the card capabilities and then the actual speed/width. Much easier to get this sort of diagnostics info in linux than windows btw.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 02, 2023, 08:24:16 pm
content: More progress yaaay.I'm getting close.I decided to use proper subrack hardware to support the board. I will have to cut the rails, but I can live with that.Cost estimate so far:-Sheet metal top, bottom, front bezel and rear cover from SendCutSend ~$130(including bending, countersinking and threading the top and bottom and anodizing the front)(it would be much cheaper with quantity, and structurally neither the front nor the rear panel is needed, but it looks nice and won't short)-Subrack hw - 1pc 19" horizontal rail with threaded insert, 10pc card guides ~$20-PCB ~120x60mm 6 layer, JLBCB more expensive stackup to meet the trace impedance 10pcs. ~$140-Press fit connectors - MiniSAS, CompactPCI 4pcs - $75-Other PCB parts + screws ~$15-------------------------------------------$380 for the prototype (with PCBs to spare)I was planning to put fan(s) in the cage. A 70x70x10mm would fit fine. I decided to ditch that though, as the PC case this would go into needs modification anyway to allow airflow through the drive bay and if I have to do that then fitting a much larger fan (120mm) is not a lot more hassle, promising better cooling.Thanks for the PCIe diagnostics tip. I'm not too much of a linux person, but I can get a live image running or something I guess to check if I can't find a good Windows alternative.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 05, 2023, 06:41:46 am
content: Orders placed.Here's how much it is so far in case anyone interested:

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 13, 2023, 06:33:18 am
content: Update!Mechanical integration tests.The 3D printed pieces will be replaced when the final parts come in.I had a surprisingly hard time trying to find the card guide rails.PCB are in transit, expected on Thu. That's gonna be the moment of truth. Fingers crossed... exciting.

author: Gribo
author_rank: Frequent Contributor
timestamp: June 13, 2023, 01:10:16 pm
content: In a future rev, You might want to break out the PXI trigger lines and 10MHz Reference to an internal connector.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 13, 2023, 03:01:56 pm
content: Quote from: Gribo on June 13, 2023, 01:10:16 pmIn case of the NI VST all of that is accessible on the front panel, but maybe there's other practical use of the floppy-drive-backplaneso I can add an extra connector or two to it.

author: zrq
author_rank: Frequent Contributor
timestamp: June 13, 2023, 04:22:42 pm
content: Card guide rails may be salvaged from broken PXI chassis I guess.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 13, 2023, 05:55:15 pm
content: Quote from: zrq on June 13, 2023, 04:22:42 pmYeah, I was looking at that but I couldn't find any on eBay that were cheap enough. Also I can't be sure they'd fit.I though they are standard but apparently only the card facing interface is.I designed my cage with Schroff horizontal rails and the guides from Vector or Wakefield don't fit.I think I found a bag of them on Amazon now though. ETA Friday.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 16, 2023, 01:16:05 am
content: Pickle Riiiiiiick!!!I'm so happy that it shows up.

author: coromonadalix
author_rank: Super Contributor
timestamp: June 16, 2023, 07:46:21 pm
content: noob question do i see the pcb connector simply pushed inside the holes, pressure contacts ??now it's the quest for driver(s) lollKudos

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 16, 2023, 11:46:26 pm
content: And we're up and running.Still need a bit of mechanical work for cooling and aesthetics.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 16, 2023, 11:50:17 pm
content: Quote from: coromonadalix on June 16, 2023, 07:46:21 pmI wanted to respond immediately, that naaah the driver is easy, but then I run into an issue with the RF generator side. The soft front panel application freezes mid startup. I had to install an older version. Interestingly no such issue with the analyzer part.You're right about the connector, they are press-fit. It's major pain to install them without appropriate tooling. I went through 3 iteration of 3d printed backers and it was still finicky pressing them it with a vice. They are not coming out of there before the end of times.

author: zrq
author_rank: Frequent Contributor
timestamp: June 17, 2023, 10:40:27 am
content: Congratulations. I hope the cooling will not be an issue as I read on NI knowledge base articles recommending setting the fan of PXIe chassis to high to avoid FPGA overheating.I'm so tempted to get a 5645 or 5644 from ebay and make a similar adapter board.

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 17, 2023, 02:03:39 pm
content: Quote from: zrq on June 17, 2023, 10:40:27 amThanks.Yes, I'm a bit worried about the cooling. I will have 2x 50x10mm fans on the intake below and 2x pulling on the other side. I will set it up and monitor the device temperature. If this is not enough then some creative solutions will be needed.I haven't decided yet, but I might make the board design open source and/or maybe try to sell it in various build stages.Either way if you decide to build your own and have questions about the specifics of mine feel free to ask!

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 18, 2023, 05:35:00 am
content: I am not proud of the Dremel carnage I did on the PC case, but at least I got the fan mounting brackets and those are nice.

author: oilburner
author_rank: Contributor
timestamp: June 20, 2023, 02:00:25 am
content: I like those rivet/nut things, nice!g

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 20, 2023, 03:21:41 am
content: Quote from: oilburner on June 20, 2023, 02:00:25 amMy second favorite thing after heat-set insertshttps://a.co/d/89Cz6SIhttps://a.co/d/8vGicUl

author: Marsupilami
author_rank: Frequent Contributor
timestamp: June 27, 2023, 03:30:17 am
content: TADA!I'm pretty happy with this, but I'm already waiting for parts for Rev B.The modifications to the PC case are not nice nor easy so I decided to ditch the double 5.25" bay compatibility and get a PC case that will house the instrument cage vertically with one big fan below, venting through the top of the case. It will be more silent and mechanically robust.Until then check this out:

author: Marsupilami
author_rank: Frequent Contributor
timestamp: October 01, 2023, 06:10:58 pm
content: For anyone that might care this is as done as probably it ever will be.PCB files and info onhackaday.

author: jhov
author_rank: Newbie
timestamp: October 11, 2023, 04:44:42 pm
content: This is an awesome project. Your end result is very clean. How flexible is the software for signal generation? Is it useful out of the box or is it just a basic demo and the user is expected to develop their own applications?

author: coromonadalix
author_rank: Super Contributor
timestamp: October 12, 2023, 05:24:37 pm
content: just saw on ebay that you have dual / quad sff-8643 connectors pcie adaptors,maybe it need a new backplane pcb to fit theses, if you expand it in the future ...

author: zrq
author_rank: Frequent Contributor
timestamp: November 23, 2023, 08:43:04 pm
content: Very cool, just pulled the trigger for mine 5644R, should arrive in my apartment for <$400. Thanks very much for posting the files, it will be a nice Christmas project for me.However I noticed a problem in 20 minutes after paying, the semi-rigid link between CAL IN and CAL OUT was removed by some dumb guy.Despite the seller claim it to be in “Excellent working conditions”. I'm going to replace it with the best SMA cable I can find, and hope the VST not that sensitive to this.I also want to help hacking together the Gnuradio blocks, although not sure I'm competent enough in the hard core C/C++ part, but I can try.

author: zrq
author_rank: Frequent Contributor
timestamp: November 24, 2023, 12:10:56 am
content: Order placed at JLCPCB (101 EUR for 10pcs). So if anyone in the Europe (I'm an expat) interested in a board, you may let me know and I can mail you one.

author: zrq
author_rank: Frequent Contributor
timestamp: November 26, 2023, 08:06:29 pm
content: May you bother to discuss a bit more on the software side?If I understand right, if the description ini files in C:\ProgramData\PXISA\Chassis are created correctly (and the pxiesys.ini and possibly also the ChassisDescriptionFilePath registry), the adapter board can get recognized as a chassis and the VST board get enumerated (if it's already enumerated at UEFI booting and seen by Windows).Or is it compulsory to write a stub DLL like NIPXIeSMu.dll to help enumeration and implement the SMBus operations?

author: Minki
author_rank: Contributor
timestamp: November 27, 2023, 11:30:26 am
content: Quote from: Marsupilami on June 27, 2023, 03:30:17 amHi Marsupilamiwould you mind to share the case 3D print file...?I want to print one...and does the main frame still need make by AL?Thank