Minimal USB CDC for PIC32...Update: Say goodbye to COM ports - Page 1
Forum section: electronics / general

author: voltsandjolts
author_rank: Unknown
timestamp: November 03, 2020, 10:12:27 am
content: In the past I have always used one of the dedicated USB-Serial converter ICs (FTDI/Prolific/CH340 etc) for serial comms.I have a fairly simple project using PIC32MX795 which would be suited to using the USB peripheral instead of a converter IC.So, I fired up Harmony for the first time (normally I would just use the datasheet for PIC32 programming) and tried out the USB option.Harmony automatically pulls in FreeRTOS as a dependencyBefore I trawl through the Harmony code to sift out the crud, can anyone point me to a minimal PIC32 USB CDC implementation?For example, someone has done a nice job of this for PIC18, one source file and a few headers, et voila. Nice.https://pic-microcontroller.com/a-minimal-usb-cdc-acm-aka-virtual-serial-port/

author: JPortici
author_rank: Super Contributor
timestamp: November 03, 2020, 10:24:11 am
content: Yes, check Mister Hemi's workhttps://www.microchip.com/forums/m1083508.aspx

author: voltsandjolts
author_rank: Unknown
timestamp: November 03, 2020, 12:02:56 pm
content: Thank you, that looks very promising, one source file and one header!I will give that a try.[Edit: I looked at it but it's just so wrong its silly. Ignore Mister Hemi's code.]I see you responded a few times in that thread - did you get it running?I wish manufacturers would provide a simple CDC implementation for their USB peripherals, surely that would cover a lot of use cases where folks are looking for an easy UART equivalent over USB. It really shouldn't be this time consuming.

author: JPortici
author_rank: Super Contributor
timestamp: November 03, 2020, 02:28:13 pm
content: I actually did not, because i always wanted to use USB on different targets (MX and MK) which use different controllers and i didn't know how to port it.For the MX i went on using the good old MLA, as i'm doing in other products using dsPIC/32MM.. performance is more than acceptable.For MK, project was put on hold indefinetly (hobby stuff)

author: voltsandjolts
author_rank: Unknown
timestamp: November 03, 2020, 02:32:35 pm
content: Ahh, I didn't realise the USB controllers were different, I had just assumed they'd be the same across PIC32I'll probably just switch micro from MX to MZ, easier than porting USB stuff from MZ to MX.

author: voltsandjolts
author_rank: Unknown
timestamp: November 03, 2020, 05:13:58 pm
content: Can someone give me some guidance on which are the newer/improved PIC32 MZ micros?They had a bad reputation for being buggy in the early days.I am looking at a MZ EF series, 512K/128K in 64 or 100 QFP...or such like.

author: JPortici
author_rank: Super Contributor
timestamp: November 03, 2020, 06:04:05 pm
content: PIC32MZ EC: The old bugged onesPIC32MZ EF: Those which replaced the EC, with also a FPUPIC32MZ DA: PIC32MZ EF, but with also a graphics and SDRAM controller. selected versions have also up to 32MB of SDRAM in the same packagePIC32MK: Promising on paper, but implementation sucked. Not nearly as bugged as the MZ EC but has some dealbreaker bugs (VBAT not working, Prefetch not working and can lock up processor, DMA error can lock up processor, single SRS for interrupts) supposedly solved in the new versions (which will replace the old ones, which will add ECC flash, CAN FD and more improved stuff)PIC32MX: Several families, with all their quirks. However i quidte like the 1xx/3xx for cheap CAN+USBPIC32MM: Has some dealbreaker bugs (namely ADC with missing codes) but overall quite pleasant for reeeeally cheap usb

author: voltsandjolts
author_rank: Unknown
timestamp: November 03, 2020, 07:40:23 pm
content: Great summary, thank youThat should be on front page of Microchip website

author: Renate
author_rank: Super Contributor
timestamp: November 03, 2020, 08:25:33 pm
content: If you like, you just do the FTDI implementation of a serial interface.It's simpler that the spec USB CDC.

author: JPortici
author_rank: Super Contributor
timestamp: November 03, 2020, 08:36:38 pm
content: or better yet, don't use CDC at all.Low bandwidth, i just use one or more interrupt endpoints (custom HID device) so no driver needed.. and 64kb/s transfer speed is usually plenty, even compared to UARTHigh bandwidth, or packets bigger than 64 bytes, i prefer to go the vendor route (one or more bulk endpoints and be done with it)Sure, going CDC means that you don't have to deal with libusb/winusb but then you have to deal with all the limitations of COM ports under windows. Don't know if there are simillar nuisances with linux/mac, don't care

author: Renate
author_rank: Super Contributor
timestamp: November 03, 2020, 09:57:48 pm
content: Quote from: JPortici on November 03, 2020, 08:36:38 pmOh, yeah, definitely!I mostly do HID, the only "serial" I use is debug/console.Everybody is always using serial. It's unstructured. It stinks.Collect characters until newline or timeout or did you miss the start of the sentence?

author: dmendesf
author_rank: Frequent Contributor
timestamp: November 04, 2020, 02:42:33 am
content: Quote from: Renate on November 03, 2020, 08:25:33 pmYou mean using FTDI drivers with a firmware emulating a FTDI device inside a microcontroller? Do you know how they transfer data?

author: voltsandjolts
author_rank: Unknown
timestamp: November 04, 2020, 08:36:05 am
content: Quote from: JPortici on November 03, 2020, 08:36:38 pmYeh, COM ports are shit aren't they.USB CDC was created for legacy equipment which used COM ports.At that time a more modern standardised replacement for COM ports should also have been implemented, with a simple API maybe like this:ListDevices( 256-bit unique device type ID )OpenDevice()CloseDevice()SendPacket( bytes )GetPacketByTimeout( timeout )Callback for received packet ( *fptr )Callback for device disconnected ( *fptr )So, cross-platform drivers available on every system, no baud rate crap, no searching through COM ports for your device, simple packetised data.Our technology leaders failed us - looking at you USB-IF.So here we are in the year 2020, basterdising the USB HID class and many of us still using frickin COM ports.It actually makes me mad that, for the want of something so simple, literally thousands of engineer hours (and user hours messing with driver installs) have been wasted on simple serial comms over USBFTDI with the DLL interface works good for me but something similar should have been made available as a standard USB class with standard driver many years ago.

author: JPortici
author_rank: Super Contributor
timestamp: November 04, 2020, 10:36:26 am
content: Quote from: voltsandjolts on November 04, 2020, 08:36:05 amI would not be surprised if keeping visual basic alive was also a reason.After all, i am still mantaining a 20+ yo application that went through VB4,5 and 6 which i can't really migrate to a new framework because the budget to do that is zero. USB CDC and classic bluetooth means that i can still use a COM port and not have to come up with monstrous ideas like having another process running written in a more up to date framework such as QT and communicate between the two processes via winsocket or something like that so i can finally use real USB and BLE (though BLE support in windows is still atrocious and i prefer to have an android tablet and a specific app for such occasions)Then we have spare laptops that run XP or worse 98 because there is some piece of software written in the 90s that was never updated with 32bit binaries, that will not work inside a VM because reasons (sometimes windows XP mode in win7 worked though)

author: Renate
author_rank: Super Contributor
timestamp: November 04, 2020, 12:50:25 pm
content: Quote from: dmendesf on November 04, 2020, 02:42:33 amI must know. Or did know.I wrote a replacement once when I didn't have an actual FTDI around.The Windows driver (written by FTDI) is more particular as FTDI tries to prevent your using clones.The Linux driver is easy-peasy.All the info you need can be found in the header file for the Linux driver:https://github.com/torvalds/linux/blob/master/drivers/usb/serial/ftdi_sio.hIt's two bulk endpoints and the control endpoint is used for vendor specific control/status.The OUT transactions are just raw, the IN transactions have two bytes at the head, 0x01, 0x01.The good thing is that you don't need to really implement the parts you don't need, baud rate, 5-8 bit data size, parity, etc.You also only need to implement a single model.

author: dmendesf
author_rank: Frequent Contributor
timestamp: November 05, 2020, 03:00:55 am
content: Thanks for the info. Will try someday...

author: voltsandjolts
author_rank: Unknown
timestamp: November 05, 2020, 10:03:25 am
content: Quote from: JPortici on November 03, 2020, 08:36:38 pmQuote from: Renate on November 03, 2020, 09:57:48 pmMy preferred solution for packetisation over serial links is the simple and elegant COBS encoding.https://www.eevblog.com/forum/microcontrollers/implementing-uart-data-packets-with-consistent-overhead-byte-stuffing-(cobs)/So if we can use COBS to send arbitrary length data packets over USB HID (broken up into 64 byte chunks), that leaves the speed limitation of 64,000 bytes/sec (in each direction simultaneously, I believe..?)In comparison to good-old-fashioned COM ports, that would need a 8N1 baud rate of around 640kbaud for same data rate.Hmmm, maybe that would suffice for my next project. I need to look into this HID stuff a bit more closely.

author: voltsandjolts
author_rank: Unknown
timestamp: November 09, 2020, 10:54:31 am
content: Ahha, found a neat library:M-Stack: Free USB Stack for PIC 16F, 18F, 24F, and 32MX Microcontrollershttps://github.com/signal11/m-stackLGPL or Apache license.Includes examples for CDC, HID, MSC.Just a few minutes after downloading I had the CDC example running on a PIC32MX795F512, using just under 12K FLASH and 1K RAM (-O2 free XC32 version)All self contained.No hunting for old Microchip libraries or compilers, it just compiled first try on MPLABX 5.45 and XC32 v2.5 (latest versions as of November 2020)That's refreshing!

author: JPortici
author_rank: Super Contributor
timestamp: November 09, 2020, 01:19:03 pm
content: It does? I never was able to use that one

author: voltsandjolts
author_rank: Unknown
timestamp: December 03, 2020, 08:15:01 pm
content: Okay, I've been looking into this a bit more and I've come to the conclusion that I don't need CDC and COM ports anymore, woo-hoo!Firstly, I looked at using USB HID which is a nice solution because there is no driver install.usbhid.sys/usbhid.dll are used as kernel/userspace drivers and hidapi.dll provides an easy to use cross-platform API.The maximum data rate of 64,000 bytes/sec full-duplex is ample for many applications...but what I've recently discovered (many of you probably knew years ago) is that you can use any USB transport type without a driver install and get FS/HS/SS data rates. Just as easy as HID.With Windows 8.1 onwards, your USB device can respond with certain string descriptors which cause it to be assigned winusb.sys/winusb.dll as the kernel/userspace drivers, with no user interaction.Plug and play. Your host software can then use winusb.dll directly, or via libusb for easier API and cross-platform ability.This system is often referred to as Windows Compatability ID (WCID). There are some links below  to get you started if you want more background info.There is an open source embedded USB stack called 'M-Stack' .It's for all PICs which have the Microchip Full Speed USB peripheral, and that includes PIC16/18/24/32, not PIC32MZ (which has HS USB).IMHO it's well written and much easier to understand than any of Microchip's USB code, which has so much abstraction and pointer shenanigans it makes me dizzy.Useful to look over even if you are going to use Microchips code in your device.Anyway, M-Stack includes this WCID feature and I'm attaching a ready configured project here.FYI M-Stack needed one tiny fix, the last byte of this struct was zero but it actually must be 0x01:Code:struct microsoft_os_descriptor os_descriptor ={0x12, /* bLength */0x3, /* bDescriptorType */{'M','S','F','T','1','0','0'}, /* qwSignature */MICROSOFT_OS_DESC_VENDOR_CODE, /* bMS_VendorCode */0x01, /* bPad MUST BE 0x01 */};This project has been tested on PIC32MX but should work for other PIC families.Change MPLABX project settings and hardware.c for your device as appropriate.Build the project (no massive external dependencies like Microchip USB stuff, all self contained) and program your device.Plug it into a USB socket and it appears in Win10 Device Manager under 'Universal Serial Bus Devices' as 'USB Stack Test Device'It is setup with VID=0x1209 PID=0x000E which is assigned for test devices by pidcodes .USB Tree View shows all the details for the device, with the four endpoints established (pipes), in addition to endpoint 0.Code:---------------- Connection Information ---------------Connection Index : 0x06 (Port 6)Connection Status : 0x01 (DeviceConnected)Current Config Value : 0x01 (Configuration 1)Device Address : 0x0E (14)Is Hub : 0x00 (no)Device Bus Speed : 0x01 (Full-Speed)Number Of Open Pipes : 0x04 (4 pipes to data endpoints)Pipe : EndpointID=1 Direction=IN ScheduleOffset=0 Type=BulkPipe : EndpointID=1 Direction=OUT ScheduleOffset=0 Type=BulkPipe : EndpointID=2 Direction=IN ScheduleOffset=0 Type=BulkPipe : EndpointID=2 Direction=OUT ScheduleOffset=0 Type=BulkThe firmware just has a loopback on endpoint 1 and same for endpoint 2 (see main.c).I have attached a very simple Python script which puts some data through the loopback - read it for install instructions.Goodbye COM ports!HIDAPI library was originally developed by Alan Ott.It is now part of the libusb project.https://github.com/libusb/hidapiKeywords are WinUSB, WCID, Microsoft OS Descriptors/Extended Descriptors. Some links:https://stackoverflow.com/questions/38906880/how-to-work-with-winusbhttps://github.com/pbatard/libwdi/wiki/WCID-Deviceshttps://www.silabs.com/community/mcu/32-bit/forum.topic.html/using_microsoft_osd-lMCDhttps://docs.microsoft.com/en-us/windows-hardware/drivers/usbcon/automatic-installation-of-winusbGuess what, also developed by Alan Ott!https://github.com/signal11/m-stackhttps://pid.codes/1209/USB Tree Viewhttps://www.uwe-sieber.de/english.html

author: asmi
author_rank: Super Contributor
timestamp: December 03, 2020, 08:56:07 pm
content: OMG. So much trouble to get rid of COM ports, what did they do to you?As for me, I will take zero lines of code + 1$ FT230X chip over this nonsense with Python and other crap please & thank you. My sanity is worth more than these pennies. And I can use it with any MCU or FPGA which has two free GPIO pins.

author: voltsandjolts
author_rank: Unknown
timestamp: December 03, 2020, 08:57:31 pm
content: It's only a hassle once and now I am over the learning curve it's easy-street all the way

author: voltsandjolts
author_rank: Unknown
timestamp: December 03, 2020, 09:03:12 pm
content: To implement PnP WCID using Microchip libraries, see this post by NICK7 using Harmony libraries V2.02:https://www.microchip.com/forums/FindPost/977964

author: JPortici
author_rank: Super Contributor
timestamp: December 03, 2020, 09:25:09 pm
content: you actually managed to make the automatic windows driver thingy? I haveat this so many times without success..Ah, i prefer to use libusb over windows' dlls, i havea nice wrapper around QT (derived and improved from microchip's mla examples on github) but that's another story

author: voltsandjolts
author_rank: Unknown
timestamp: April 30, 2021, 11:43:18 am
content: I've done a couple of projects with PIC32MM. Quite a nice cheap entry level 32-bit device family, must always check errata though.Here's a MPLABX project for the PIC32MM0256GPM064 on the USB Curiosity Board (DM320107).It implements WCID for PnP installation, with WinUSB as the driver. Uses M-Stack (mentioned above) for USB.The included python script does a basic comms check, run it and you should see:Code:Manufacturer: Signal 11 Software LLC.Product: USB Stack Test DeviceSerial: 12345678Endpoint 1 loopback: 244 kbytes/secEndpoint 2 loopback: 246 kbytes/secEdit: 27DEC22 Improved comments in the python test script, no functional changes.

author: JPortici
author_rank: Super Contributor
timestamp: April 30, 2021, 09:15:02 pm
content: Quote from: voltsandjolts on April 30, 2021, 11:43:18 amwould have become my goto if not for the ADC errata. Crazy cheap and good performance with USB. why would anyone use a M0+ is besides me. (unless for the ADC errata)

author: orphean
author_rank: Unknown
timestamp: May 02, 2021, 11:05:03 pm
content: Quote from: JPortici on April 30, 2021, 09:15:02 pmCode:ADC 12-Bit Conversion The ADC may miss one or more of the following codes in 12-bit mode: 1023, 2046, 2047, 3070 and 3071.ADC Format Options 32-bit signed format option is the same as the 16-bit signed format option.These seem to be the remaining ADC errata as of revision A4. Would you mind sharing what makes these a deal breaker for you? I've been evaluating the pic32mm with a usb dev board and just curious what your thoughts are, thanks!

author: voltsandjolts
author_rank: Unknown
timestamp: May 03, 2021, 03:35:39 pm
content: Different applications have different requirements.For many applications a few missing ADC codes would not be an issue, but yeh, it's not ideal.Peruse the errata and datasheet then you can decide if it meets your application requirements.With Microchip parts its often worth reading the errata first

author: JPortici
author_rank: Super Contributor
timestamp: February 27, 2023, 03:16:03 pm
content: Quote from: orphean on May 02, 2021, 11:05:03 pmSimply put, i require the 12bit resolution. In the meantime, my goto has become the PIC24FJ GU series (specifically the parts in the 40x sub family)Quote from: voltsandjolts on May 03, 2021, 03:35:39 pmwith all parts, read the errata firstno errata available, you're in for a bad time..almost two years later, i finally had to work on a new usb gadget for which HID wouldn't cut it, thanks to your pointers earlier i managed to make windows enumerate the device without a driver.It's actually easier than i thought.Taking the "USB Device - Vendor Throughput Test" in the MLA as an example, one just have to remove the GUID part from the extended property windows descriptor.Code:const MS_EXT_PROPERTY_FEATURE_DESC ExtPropertyFeatureDescriptor ={//----------Header Section--------------sizeof(ExtPropertyFeatureDescriptor), //dwLength0x0100, //bcdVersion = 1.00EXTENDED_PROPERTIES, //wIndex0x0000, //wCount};Assign a new PID, the device will enumerate.In case one doesn't want to use MCC and keep using the MLA code (it's actually exactly the same but i don't want to use that slow useless plugin) with the PIC24FJ GU series, this is the code to initialize the FRC for active clock tuning - no crystal requriedCode://Set up PLL - PLL parameters are already set in hardwareCLKDIVbits.RCDIV = 1; //FRC divided dy 1:2CLKDIVbits.CPDIV = 0; //PLL Output Divide by 1:1CLKDIVbits.PLLEN = 1; //PLL always enabled__builtin_write_OSCCONH(0x01);__builtin_write_OSCCONL(0x01);while(!OSCCONbits.LOCK);//Enable Active Clock TuningOSCTUNbits.STSRC = 1; //Tuning Soruce is USBOSCTUNbits.STEN = 1; //Enable Active Clock Tuningand in order to use the MLA one has to add these lines in the usb_hal_pic24f.h file, in the IFDEF block that selects the processor type (i only added one of the chips, the rest is left as an exercise to the reader)Code:#elif defined(__PIC24FJ256GU406__)#define DEVICE_SPECIFIC_IEC_REGISTER_COUNT 7 //Number of IECx registers implemented in the microcontroller (varies from device to device, make sure this is set correctly for the intended CPU)#define USB_HAL_VBUSTristate() {TRISFbits.TRISF7 = 1;}That's it