USB Sniffer using RP2040 - Page 1
Forum section: electronics / projects

author: ataradov
author_rank: Super Contributor
timestamp: February 16, 2022, 05:02:57 am
content: To better understand PIO of the RP2040 I decided to make a useful project using it. The result is a project of a USB sniffer I just published -https://github.com/ataradov/usb-sniffer-liteThe sniffer supports Full Speed and Low Speed modes. It is very easy to use. It requires no external software and the whole interface happens over a virtual COM-port.It may not beat $600 tools, but going over my previous projects where I needed a USB Sniffer, it would cover 90% of them easily. And the ease of setup would make me not hesitate to actually use it, instead of trying random things in hopes that firmware finally works.All you need to test it is a Raspberry Py Pico or virtually any board based on RP2040 and a USB cable you can spare. Add a splice to the USB cable, program the provided UF2 file and you are done.I'm attaching a couple sample log files showing what you should expect to see.

author: ataradov
author_rank: Super Contributor
timestamp: February 16, 2022, 05:05:26 am
content: And on the PIO part - I like it a lot. I see some potential for improvement, there are some obvious things that come up in many scenarios I thought about, and even in their own demo code. But generally, it is very capable and very well done.This project also uses multi-core stuff. And again, reasonably well done compared to some other devices. And makes life easier for sure, even on a Cortex-M0+ device.PIO program is far more complicated than it needs to be. It tries to re-synchronize the sampling point using edge transitions (similar to how a real USB controller would do it), but based on my observations, it is probably not necessary, at least for the standard packets up to 64 bytes. It might be needed for 1023 byte isochronous packets, I have not tried them. I kept the program as is to have a more complex example and because it fit into the memory. And obviously it will help if there is a bigger deviation between the clocks.

author: magic
author_rank: Super Contributor
timestamp: February 16, 2022, 08:35:18 am
content: No need to splice cables, though. Get yourself some male/female USB connectors, a female pin header and a bit of perfboard and you have a clean and much more mechanically sound solution. It will be OK at full speed.

author: ataradov
author_rank: Super Contributor
timestamp: February 16, 2022, 08:40:00 am
content: Non-molded connectors are all bulky and suck. This picture is just a demo for people that want a quick solution.The version I'll be using will be with a custom board (on a way) that fits into this boxhttps://www.ebay.com/itm/393626280408(the type that all cheap logic analyzers use). The cables would be just cut off from female A and male A molded cables, and strain relief and soldering will be hidden inside the box.I'll add pictures of that when I have all the parts.

author: magic
author_rank: Super Contributor
timestamp: February 16, 2022, 08:44:02 am
content: I meant PCB mount connectors. Solder them down on a piece of perfboard, connect USB cables on both sides, plug into the Pi, done.That's a quick and dirty solution if you don't want to bother with a case.

author: ataradov
author_rank: Super Contributor
timestamp: February 16, 2022, 08:49:08 am
content: Yes, I guess that works too. But standard connectors don't fit too well on a perfboard. You can drill out the holes, but then mechanically it would still be questionable. Cable approach is better, just strain relief the cable. I also like captive cables, because I hate the rats nest of cables that ensues every time I have to use USB analyzers.I just made this cable today to take that picture.For the development I used a hacky setup with a chunk of a PCB from a different USB sniffer project (HS capable). 0.1" connectors go where USB B connector was designed to go.

author: SiliconWizard
author_rank: Super Contributor
timestamp: February 16, 2022, 05:28:00 pm
content: Quote from: ataradov on February 16, 2022, 05:05:26 amI agree. I find it pretty flexible and easy to use at the same time. NXP has FlexIO, which is kind of similar, but much more complex, and much harder to grasp IMO.

author: thm_w
author_rank: Super Contributor
timestamp: February 16, 2022, 11:40:59 pm
content: Quote from: magic on February 16, 2022, 08:35:18 amYou can use these for USB sniffing as well:https://www.aliexpress.com/item/33025145386.html

author: ataradov
author_rank: Super Contributor
timestamp: February 17, 2022, 12:15:27 am
content: Quote from: thm_w on February 16, 2022, 11:40:59 pmWow. I don't know why I'm surprised this exists, but it is such a non-obvious thing.Yes, as far as easy ways to connect things to RP2040, this is the easiest except for the need to wait for it to arrive.

author: tim_
author_rank: Regular Contributor
timestamp: February 19, 2022, 07:26:05 am
content: Quote from: ataradov on February 16, 2022, 05:02:57 amAwesome use of the PIO. I also saw that you freed the RP2040 from the shackles of the bloated SDK. That's very great, I really didn't like all the cmake stuff.

author: ataradov
author_rank: Super Contributor
timestamp: February 20, 2022, 09:26:24 pm
content: Here is the final version of the hardware I'll be using, I'm pretty happy with it. It is not a dedicated board, so wiring is a bit messy. This is based on the board specifically designed for this case though -https://github.com/ataradov/breakout-boards/tree/master/rp2040.Also, USB cables are always made out of the shittiest wires. They are not fun to solder at all.

author: Renate
author_rank: Super Contributor
timestamp: February 21, 2022, 04:14:59 pm
content: Did I miss something? Where is the case from? It doesn't look printed. Or is it?

author: ataradov
author_rank: Super Contributor
timestamp: February 21, 2022, 05:35:05 pm
content: This is a standard project box. It is the same design as used by the cheap logic analyzers. But it does not have any holes in it. Here is the exact listinghttps://www.ebay.com/itm/393626280408But they tend to appear and disappear, generally they go by something like "1.97 x 1.1 x 0.59inch" and/or "50 x 28 x 15mm".I bought a bunch of them as this is the most sane small project box I could find. And the board was designed to fit into that box, so that I could use it for projects like this.

author: Renate
author_rank: Super Contributor
timestamp: February 21, 2022, 07:22:30 pm
content: Quote from: ataradov on February 21, 2022, 05:35:05 pmThe lowest common denominator is the Radio Shack boxes.I used to use a bunch of 2" x 1" x1" but they seem to have discontinued all but the largest sizes?Velleman makes some decent boxes that you can get at Micro Center.

author: dcd
author_rank: Newbie
timestamp: January 20, 2023, 07:01:15 pm
content: Thanks for sharing your projectI noticed your code has crc16 and crc5 for USBI actually found this thread when doing some research on the rp2040 SNIFF_CTRL, SNIFF_DATA, and CTRL.SNIFF_EN related with DMAthe rp2040 has support for these, and others• CRC-32, MSB-first and LSB-first• CRC-16-CCITT, MSB-first and LSB-firstdid you consider using the rp2040 hardware CRC calculations? ( I don't know if there is any hardware assist for crc5 or if you need the speed :-)

author: ataradov
author_rank: Super Contributor
timestamp: January 20, 2023, 07:19:07 pm
content: USB uses a different polynomial compared to CCITT, so hardware acceleration is not going to work.And for CRC-5 there is no acceleration, but the packets that use this CRC are very short, so the software calculation would be faster anyway.

author: ataradov
author_rank: Super Contributor
timestamp: April 12, 2023, 04:59:16 am
content: This project recently got a lot of attention and I see people actually using it, so I decided to spend a bit more time and clean some things up.The most annoying thing about the initial firmware was probability of starting in the middle of a packet and completely screwing up the capture state. In the new update, I've added second PIO to monitor the bus for end-of-packet or bus reset markers, which serve as a synchronization point. It should be way more robust now.I've also made custom hardware for it. This board integrates FE8.1 USB hub IC, so there is no need for two connections to the host. The setup is very easy now, just plug the sniffer into the PC and your device into the sniffer. No need for messy wiring.

author: voltsandjolts
author_rank: Unknown
timestamp: May 06, 2023, 11:01:09 am
content: This looks really neat. As always, your firmware source is so clear and succinct, a style I would like to emulate in my own code.I'd never heard of the FE8.1 FS USB hub IC, useful little chip to have in the toolbox. Aliexpress seems to be the main stockist. I've put a parts order together and bits should be here shortly. My parts list is attached with some suggested suppliers and PNs.I'm not setup for reflow so will be soldering the old fashioned way. The RP2040 has one ground connection only, on the exposed underside pad, doh! I'll need to drill a hole under it to cludge the ground soldering. Maybe I'll do a little PCB re-hash to make a hand solderable board design with a hole under the RP2040.FYI The PCB doesn't bring out the trigger input, not sure if that was intentional.

author: ataradov
author_rank: Super Contributor
timestamp: May 06, 2023, 03:45:24 pm
content: SC662K is also available from LCSC. It is discontinues in its original form, but it lives on in thishttps://www.lcsc.com/search?q=XC6206P332MRI have no idea what is the story there, but it is basically the same voltage regulator with the same "662K" top markings. It has 300 mA output current vs 250 mA of the original.I don't have reflow soldering setup myself. MCU was mounted using hot air, the rest by hand.

author: voltsandjolts
author_rank: Unknown
timestamp: May 06, 2023, 07:55:46 pm
content: Quote from: ataradov on May 06, 2023, 03:45:24 pmAhh, OK, updated the parts list accordingly.

author: ataradov
author_rank: Super Contributor
timestamp: May 06, 2023, 08:24:46 pm
content: Also, USB-A connector footprint is for this style of connectorhttps://www.lcsc.com/product-detail/_Shenzhen-Kinghelm-Elec-_C2979055.htmlIt is the same dimensions and everything, but the mounting lugs are straight and the PCB has slots. I assume your connector would fit, but might need a bit of messing around when inserting, or just straighten the lugs.I like that connector because it does not have flared out opening, it is just a straight shield. So, your connector would also not fit into the 3D printed case, you would have to modify that if you want to use use one.

author: voltsandjolts
author_rank: Unknown
timestamp: May 06, 2023, 09:01:52 pm
content: Agreed, and updated parts list.

author: voltsandjolts
author_rank: Unknown
timestamp: May 22, 2023, 11:35:04 am
content: Switching option 't' to "Time display format changed to Relative to the bus reset" is ideal for for watching the enumeration process. Start capture, plug in device and you got it all. Nice. Alex's sniffer lite combined with wireshark makes a very useful toolset for FS USB.Just to build confidence in it, I did a little comparison to the Beagle. Note the enumeration process is not exactly the same, so there are minor differences, but it's all there.

author: pmr
author_rank: Contributor
timestamp: June 01, 2023, 02:51:40 pm
content: Wow, very cool project. Will definitely build this -- just ordered a USB splitter board for that. One question: if the sniffer is in FS mode and a PRE header arrives, will it understand the LS packet that follows?

author: ataradov
author_rank: Super Contributor
timestamp: June 01, 2023, 03:21:02 pm
content: Quote from: pmr on June 01, 2023, 02:51:40 pmNo, not at all. it is likely to be confused by that.There is not enough processing power to decide on the fly and switch the speed.

author: pmr
author_rank: Contributor
timestamp: June 01, 2023, 08:08:18 pm
content: QuoteAh, of course. It is already surprising that the uC is fast enough to bit bang a FS connection. Even without this feature the sniffer is exciting and imminently useful!Still (clutching at straws here), according to the spec there is a 4 bit time pause after the PRE and the line polarity does not need to be switched, just the clock speed needs to change. Maybe the 4 bit times is just enough. On the other hand, you have probably already thought about this a lot more than I; I guess the hard part is in the case that the PID is not PRE and another byte immediately follows the PID -- checking and deciding on the PID might already take too much processing time in this case.

author: ataradov
author_rank: Super Contributor
timestamp: June 01, 2023, 08:36:24 pm
content: The capture is performed autonomously by the PIO, it does not know anything about the captured data, it can't tell the type of the packet or if the packet is even valid. It just receives the stream of bits, it does not even remove bit-stuffing. It just captures bits as long as memory allows. All the processing for display is done later on the CPU, after capture is done.I'm preparing a release of a much more capable sniffer with HS/FS/LS support using FPGA and a real USB PHY, but even that would not do PRE handling, at least initially. But it won't be confused by the mix of packets.What is the actual use case for this?

author: pmr
author_rank: Contributor
timestamp: June 01, 2023, 08:56:41 pm
content: I am working on an FPGA based USB1.1 host (PHY / SIE in gateware and protocol using a tiny riscv softcore) on a ULX3S board. The host will handle 1 level of hub which has a keyboard and mouse attached. I was thinking about sniffing the host-hub traffic. I can debug the host-hub traffic first with a FS device in the hub and later the LS hub-device traffic as needed. The sniffer as-is will allow for that. Sniffing the host-hub traffic with an LS device attached would be even nicer -- but don't get me wrong: I am already super happy and impressed with the way it is now.I'm trying to understand the code in capture.c (unfortunately I am new to RP2040). As far as I understand now, the PIO supports small programs per pin for fast responsiveness. the PIO0 program seems to do NRZI decoding and the PIO1 program seems to detect EOP's. The buffer seems to get filled with raw capture data in lines 581-609. Is that understanding correct?Not sure I currently understand the format of the data capture buffer, either in raw format or processed format. Is that documented somewhere?

author: ataradov
author_rank: Super Contributor
timestamp: June 01, 2023, 10:00:22 pm
content: Quote from: pmr on June 01, 2023, 08:56:41 pmYeah, that would be the use case. I assumed not a lot of people would be doing host/hub implementations, especially capable of multiple speeds.Quote from: pmr on June 01, 2023, 08:56:41 pmThey are not really per pin, a subset of pins available to each PIO, but the ways they can use them are pretty limited.Quote from: pmr on June 01, 2023, 08:56:41 pmPIO0 does most of the work. It does not actually remove NRZI, but it handles re-synchronization to bit edges. This is likely unnecessary in practice, but I already had the code, so I left it there.Quote from: pmr on June 01, 2023, 08:56:41 pmPIO1 was a later addition. This is an attempt to synchronize capture to the stream. Otherwise if you start the capture in the middle of the packet, the whole thing will get messed up. PIO0 can only track its state if it started from the idle line state.PIO1 only runs once at the begging looking for any EOP. The line is assumed to be in the idle state after that, so it signals PIO00 to start running.Quote from: pmr on June 01, 2023, 08:56:41 pmYes, this is the main loop that pulls the data from the PIO.Quote from: pmr on June 01, 2023, 08:56:41 pmIt is just raw bits. separated by the size word. PIO only captures 31 bits at a time, so that bit 32 remains 0 for normal data. And the size (in bits) is calculated by subtracting 1 from 0xffffffff for each bit. At the end of the packet that value is also pushed in the stream. But this value will always have bit 32 set to 1.Now you can scan the buffer and find the first word with the MSB set, and you know the exact number of bits in the preceding data stream.The parsing happens in the display.c.

author: pmr
author_rank: Contributor
timestamp: June 01, 2023, 10:14:32 pm
content: QuoteThat is probably a fair assessment. There does not seem to be a lot of host-hub stuff out there in the hobby scene.Maybe at some point I'll experiment with detecting PRE in the buffer fill loop and switching the clock there. The main issue that I see now is that it runs on the other side of the FIFO, making timing difficult.

author: ataradov
author_rank: Super Contributor
timestamp: June 01, 2023, 10:24:59 pm
content: Quote from: pmr on June 01, 2023, 10:14:32 pmSwitching speed here is a complete reprogramming of the PIO program, since FS and LS are handled by the different implementations, since PIO instruction set is very limited.And yes, the timing would be hard to manage because you would have to interpret each packet on the fly to figure out which of them are PRE. This may be easy enough, since PRE would fit entirely in a single word, so you can find it using a simple compare without decoding anything. And I think it would also be unique because there will be a tail of a lot of 0s, which is impossible in a normal data stream (except may be at the very end, but that is solvable by looking at the length word).So, detecting the PRE may not be as hard. But switching PIO programs in time may be a bit tricky, but may be doable.For the bigger sniffer, I'll keep it in mind.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 01, 2023, 11:31:26 pm
content: I'm looking forward to the next version of the RP2040, if there is ever one. With a more advanced PIO, maybe a bit more RAM, and why not Cortex-M4-based.Wondering what the RPi has in store.

author: pmr
author_rank: Contributor
timestamp: June 02, 2023, 06:36:16 am
content: QuoteThe LS signalling after a PRE uses FS polarity for J and K, so I guess the same PIO code should work?Thinking about something like this in the buffer fill loop, in rough pseudo code:Code:if (v==EOP) {EOP_flag = true;<set clock divider to normal value><insert EOP marker in buffer>} elseif (v==PRE pattern && EOP_flag) {<set clock divider to LS speed><insert PRE marker in buffer>EOP_flag = false} else {<handle regular bits>EOP_flag = false}The remaining problem is that the loop only sees the PRE pattern after 31 bit times. The PRE pattern lasts only 16 bits, followed by a pause of 4 bits, 20 bits in total. That means the PIO code has already seen 11 FS bit times of the LS sync pattern by the time the loop detects it and switches the clock. This would mean that the switch happens in the first J of the low-speed KJKJKJKK sync pattern -- assuming that the host indeed produces a pause of 1...9 FS bit times. With the spec pause being 4 bit times, that should allow for a bit of jitter between the PIO loop and the buffer fill loop.

author: ataradov
author_rank: Super Contributor
timestamp: June 02, 2023, 03:05:38 pm
content: Quote from: pmr on June 02, 2023, 06:36:16 amIn that case yes. The bit-rate difference is handled by the clock prescaler. It should be fast to change, but obviously would need some experimentation.Quote from: pmr on June 02, 2023, 06:36:16 amThis is not the case. The PIO pushes out incomplete word as soon as EOP is detected. So all your main loop logic has to do is scan all the words, set a flag if a word has MSB set. This would indicate end of a previous packet. if you get a word that is PRE (compare full 32 bits, only 16 bits would be meaningful, the rest are guaranteed to be 0), then you have a PRE packet. You can double check the next word to see if it is a length of 16 to be absolutely sure. But I don't think this situation is possible normally.Or you can just scan for PRE payload and wait for the next word to be a length of 16, no need to look for the end of the previous packet.

author: pmr
author_rank: Contributor
timestamp: June 02, 2023, 03:46:08 pm
content: Thanks for the feedback. The special thing about the PRE header is that it is not terminated by an EOP, only the LS package that follows has the EOP. This is described in section 8.6.5 of the spec. Hence the PRE packet will not be pushed to the FIFO individually. This is for another day, though.-------The RPI Pico arrived in the mail this afternoon and I'm happily using it to debug my stuffI'm seeing intermittent errors on my SOF packets, so it has already found a first bug:Code:... : Folded 115 frames1458583 : ERROR : SYNC = 0x80, PID = 0xa5, DATA: 0a 7c... : Folded 24 frames1483626 : ERROR : SYNC = 0x80, PID = 0xa5, DATA: 23 7c... : Folded 52 frames1536719 : ERROR : SYNC = 0x80, PID = 0xa5, DATA: 58 7cNot sure I understand the error report. What is error "NBIT"? Does that mean the packet is not the required 24 bits? If so, what is error "SIZE"? I'm not asking for help on debugging this issue, just for a clarification of what the error message indicates.

author: ataradov
author_rank: Super Contributor
timestamp: June 02, 2023, 03:51:35 pm
content: If it is not terminated, then I'm not sure it is possible to receive it correctly. As the whole LS frame would be sampled in FS mode. This is likely to confuse the sate machine. And I'm not sure trying to intercept the 31-bit word and switch things mid-frame would work either. It might, but I would not expect it to be reliable.NBIT means that the number of bits received after bit stuffing removal was not a multiple of 8.SIZE means the number of bytes is not enough (or too many) for a given token type.But given that it folds a lot of frames, the error seems to be intermittent, which is strange. It is possible that bit-stuffing is messed up, and sometimes does not work correctly for some values.

author: pmr
author_rank: Contributor
timestamp: June 02, 2023, 09:28:36 pm
content: QuoteIndeed: it made a mistake when a packet ended with 5 ones and a 0, mistakingly inserting an extra 0.Many thanks for for having made this tool, it is really nice.

author: tszaboo
author_rank: Super Contributor
timestamp: June 02, 2023, 11:38:03 pm
content: Quote from: SiliconWizard on June 01, 2023, 11:31:26 pmYeah, I wonder if there is any info on that. They definitely planned more than one micro, hence the naming.I also hupe the signal integrity guy realizes, that you don't need to have every second pin as ground or power for the thing to work.

author: PeterZ
author_rank: Regular Contributor
timestamp: June 19, 2023, 12:55:29 pm
content: Thanks for such nice and useful project.I have altered the pcb design a bit to fit into one of those cheap "usb dongle" transparent boxes from aliexpress.Built, programmed, plugged in, reports on ttyACM0 as "USB Sniffer Lite. Built on Apr 11 2023 20:41:25" - all works fine!

author: pmr
author_rank: Contributor
timestamp: June 20, 2023, 09:30:18 pm
content: Well, I got my simple FPGA usb1.1 host working. About 1,000 lines of verilog and 1,000 lines of C code. Supports hubs, keyboards and mice. About 4,000 LUTs including the RV32I softcore that runs the control code. Repo here:https://gitlab.com/pnru/usb_hostMany thanks for the RP2040 USB Sniffer that greatly helped tracking down some issues!

author: royk
author_rank: Newbie
timestamp: August 26, 2024, 11:20:08 pm
content: How will this work with the RP2350 in regards to performance bottlenecks with the RP2040?

author: ataradov
author_rank: Super Contributor
timestamp: August 26, 2024, 11:34:15 pm
content: Probably about the same. The only thing that would improve is the capture buffer size due to increased memory size. The rest of the code is not really hitting any performance limitations.But I personally have zero interest in RP2350, so I'm not going to do the port.

author: moffy
author_rank: Super Contributor
timestamp: August 26, 2024, 11:49:41 pm
content: Thanks ataradov, I now have a use for the RP2040 I purchased. I was going to try and use it to interface some fast ADCs to USB via the PIO but only when I received it did I find that it was limited to USB full speed.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 27, 2024, 12:14:59 am
content: Quote from: ataradov on August 26, 2024, 11:34:15 pmWhy is that?

author: ataradov
author_rank: Super Contributor
timestamp: August 27, 2024, 12:58:09 am
content: Quote from: SiliconWizard on August 27, 2024, 12:14:59 amRP2040 was a first version of a new product, so I gave it a pass for may drawbacks. RP2350 does not address any of those things and makes things worse in new interesting ways.The price point is not a huge differentiator for me personally. I don't ship anything in volumes where it matters, and if I did, I doubt any RP device would be a good fit.I'd really rather see a solid and coherent design than that hodgepodge of IP they managed to bum from ARM and Synopsys.For me personally the value in the MCU is in the ease of use - I just want to drop the device and provide it power and at most decouple the core supply. I don't want to add a flash IC and a bunch of passives. I don't want to use a very specific inductor nor a separate core regulator.Not even managing to embed the USB serial termination resistors is just lame.

author: Colt45
author_rank: Contributor
timestamp: March 15, 2025, 08:29:27 am
content: Cool project, I'm trying to debug an STM32 project and quickly built one of these. Seems it is setting nak when the buffer is full and never recovering for some reason.very handy tool!

author: voltsandjolts
author_rank: Unknown
timestamp: March 15, 2025, 08:45:41 am
content: If you find you need a more capable sniffer, Alex also made this:https://www.eevblog.com/forum/projects/low-cost-lsfshs-usb-sniffer-with-wireshark-interface/

author: Colt45
author_rank: Contributor
timestamp: March 15, 2025, 11:29:55 pm
content: I saw that as well. I now ordered something based off it from ali - wireshark interface will be nice.