urgent: need to write a program that plots data from uart very fast (scope-like) - Page 1
Forum section: electronics / projects

author: JPortici
author_rank: Super Contributor
timestamp: December 01, 2016, 10:44:25 am
content: Hi.We are having a problem with a customer and one of our devices that must be solved as soon as in about a week.Problem #1: he is on the other side of the world so we can't go there.Problem #2 he barely knows how to operate a multimeter if he's following instructions but that's not enough. we need to see the waveforms.So i took one of our boards which has a UART interface for the customer to use and reprogrammed it so it acquires data from 2 analog channels and send it over the UART, we will send it and give him a custom cable to connect to the system so he just have to plug it in, plug the uart cable he already have and wait for a sufficient amount of data.Because we must use UART and because of the MCU on board the baud rate is limited to 115.2 kb/s, an effective frame rate of 2.5k frames per second... so 2.5 kS/s(All the limitation come from problem #3: Issue came up yesterday evening and we have to ship stuff to him in a couple of hours. but the hardware is complete.)Problem is in the computer software: due to haste and my lack of PC programming skills, i am using VB6. I came up with a simple program that samples and process the UART every 1ms and plots the data but it's not very fast/precise.5k Lines, 2.5k per trace, two traces (not sure if switching to polylines would be better or just the same)Don't care for triggering, the routine is very simple:EVERY 1ms (shortest time for timer event):- Check MSCOMM input buffer lenght- If Greater than frame_size:- Find Start Of Frame- If Start of Frame Found:- Adjust Y coordinate of Segment at Horizontal_Idx- Increment Horizontal Idx and Wrap Around- If enough elements remainin in the buffer, update another segment- End of routineIt's semi working. i'm getting partial data every redraw

author: Codemonkey
author_rank: Regular Contributor
timestamp: December 01, 2016, 10:56:53 am
content: Can't you just log the data to a .csv file and create the plots in Excel or something later ? Should be a bit faster.

author: JPortici
author_rank: Super Contributor
timestamp: December 01, 2016, 11:07:59 am
content: it will do that too but i think that a realtime screen could be of some help.. something like this will be re-used for sure in other applications/products (with a more relaxed development phase).playing around some more, it's not outputting garbage anymore. still some dropouts but it's manageable. have to find if dropouts are because of issues with the code.. or vb6 can't keep up with this task of generating new video but csv will be okay...forgot to mention: visual aid is necessary so we can tell him "no the connector is wired wrong. move this pin there" etc

author: nctnico
author_rank: Super Contributor
timestamp: December 01, 2016, 01:26:25 pm
content: I'd send him a Picoscope or a similar USB scope and do remote control using Teamviewer. Otherwise your time is probably better spend driving to the customer even if he is on the other side of the world. One of the things I've learned that in these kind of situations you don't want to develop test tools (too little too late anyway) but use something off-the-shelve for measurements and concentrate on fixing the product.After the problem is fixed you evaluate how things could have been done better. In some cases you also learn why the competitor's product is so expensive compared to yours (hint: the competitor isn't losing money on their product).

author: Twoflower
author_rank: Frequent Contributor
timestamp: December 01, 2016, 02:10:07 pm
content: I haven't used it but how about Octave (http://wiki.octave.org). It is a free, open source version of Matlab. Thus creating diagrams should be easy. It seems to have at least some interface support (http://wiki.octave.org/Instrument_control_package).At least Matlab can create Scope, read updating. Depending of the required feature set with only a few lines of code plus the code to decipher the input data.If you're really desperate:Alternatively (depending on the levels, sample-rate, common GND issues, ...) you might use a sound-card as Scope. Many recording SW shows the waveform. If you need floating GND you can use a Smartphone with something like "phyphox".

author: max_torque
author_rank: Super Contributor
timestamp: December 01, 2016, 03:49:14 pm
content: There is NO POINT in drawing a real time graph at 1KHz, unless your particular customer has VERY fast eyes indeed!Get the raw data at 1Khz, store it to a .csv for any post processing, and either:1) Update the graph with every 10th, 20th or even 50th valueOR2) Every 100s, add the last 100 samples to the plot (or the average the last 100 samples down to say 10 and add those)

author: Twoflower
author_rank: Frequent Contributor
timestamp: December 01, 2016, 04:01:48 pm
content: The OP says two signals at 2.5kS/s. Assuming the graph is 1250 pixel wide that's an update rate of two graphs per second. Not very fast. And depending on the signal that needs to be observed that's easily observable.

author: vk6zgo
author_rank: Super Contributor
timestamp: December 01, 2016, 04:14:02 pm
content: Quote from: JPortici on December 01, 2016, 10:44:25 amYes you can,they have these wonderful new devices called aeroplanes!Seriously,though,if it is that urgent,& the customer is worth keeping,send someone with the right equipment ,or find a third party in his country who can do the testing.As someone "on the other side of the world" I have been on the receiving side of the reluctance of people in EU to move out of their comfort zone.Buying parts for Picture Monitor repair is one example:Sony:"OK.it will be on the next airfreight".Barco: "We'll put it in the next container going to Australia ".(No they didn'treallysay that,but it's what it amounted to}

author: vodka
author_rank: Frequent Contributor
timestamp: December 01, 2016, 04:53:49 pm
content: Quote from: vk6zgo on December 01, 2016, 04:14:02 pmThe problem isn't the plane . The problem is the logistic cost:At first you have to find a worker that he is disposed for traveling to the Conchinchina.Second , Negotiatiating the salary and the extra-hour of the worker.Third , Reserving the plane bill and the acomodation , checking the custom laws of the country for items and the workers.All these issues cost money and time for the corporation and more if they haven't provided.

author: JPortici
author_rank: Super Contributor
timestamp: December 01, 2016, 05:50:15 pm
content: aha! we were speaking with the customer before and considered again going there but the timing couldn't be worse at the moment. It's not really about not wanting to leave the comfort zone. also no corporation, small business. losing one of us for a week can be a problem.on monday/tuesday, when the package arrive we will try this route: "DAQ" + "Scope software" and see if we can at least pin point the problem(additional info: it's a throttle pedal -or, slightly less possible, an ECU- in a new chinese car doing some funky business and shutting down the power whenever we attach our device to the car. most likely the throttle has a different connection scheme and we need to see the signals or at least how do they look like. if it's that and it's probably that the customer is able to solve the problem, if not someone will have to go over to him as we can't get our hands on the car from here. the urgency was that we were sending other products to him today)again, once we have a brief look at the realtime waveforms i can just do uart->csv directly and analyze the data offline. in the future the "daq" solution could make things simple as we have a standard cable so the customer plugs the logger in, reads all the channels (4/6/8) and sends over the data, he can be a mechanic or an office guy.. oscilloscope.. probing... all stuff that sound too difficult in their heads

author: max_torque
author_rank: Super Contributor
timestamp: December 01, 2016, 07:17:40 pm
content: Quote from: JPortici on December 01, 2016, 05:50:15 pmOf course, you know that breaking in to a safety critical system like a throttle pedal can have er, significant legal implications:http://nypost.com/2014/03/19/toyota-to-pay-1-2b-settlement-in-vehicle-acceleration-lawsuit/but, if your ok with all that, and have done your FMEM properly, then what you are looking at is the host ecu noticing that the throttle pedal output is no longer within acceptable error tollerances (usually a dual, inverse 5v analogue signal pair). So this is easily determined with a simple multimeter, no data logging or what ever required.Simply measure voltage on pins without your device fitted, then with, at zero and WOT, and all should become quite clear!

author: nctnico
author_rank: Super Contributor
timestamp: December 01, 2016, 07:39:38 pm
content: Quote from: JPortici on December 01, 2016, 05:50:15 pmIn my experience there are only two options which work for these kind of projects: go there yourself or get the hardware (a stripped down version) in your own office. It is not going to work otherwise and the customer (at the management level) will become increasingly dissatisfied with the lack of progress.

author: janoc
author_rank: Super Contributor
timestamp: December 01, 2016, 11:36:42 pm
content: Quote from: JPortici on December 01, 2016, 05:50:15 pmSeriously, if he is in China, why not have him buy the cheapest Rigol/Siglent/Atten/whatever low end scope they could get locally, hook up Skype with a webcam and tell him where to stick the probes and which knob to turn? The scope can even record, so you could get the data on a USB stick and then transferred over Internet for analysis.That will be likely much more effective than trying to rig up something using an UART and Visual Basic (whatever). And $200-$300 for the cheap scope is still way less than having to fly over in person. It is also a lot faster than getting a Picoscope and DHL-ing/Fedexing it over.

author: mjkuwp
author_rank: Unknown
timestamp: December 02, 2016, 01:47:28 am
content: I've used the language Processing at Processing.org to do something a little bit like this. It is a high level language that you can use on windows/mac/linux and it's main job is to draw graphics. there is a serial library that can do a callback when data arrives. It is easy but...still with only a week it can be tough to learn something new.

author: Siwastaja
author_rank: Super Contributor
timestamp: December 02, 2016, 06:12:56 am
content: Quote from: nctnico on December 01, 2016, 07:39:38 pm+1. It's only going to work by luck, only if this is a super simple problem that is easily found and fixed. Sounds like it's very improbable, and it's going to need a professional debug session.I can't understand the OP's logic of not being able to lose one of their teamfor a week. Even if we allow for two full days for travel, if the actual debugging takesseveral full daysfrom the professional designer to debug and fix, what are the chances that it's possible to do it at all by this "remote scope" strategy, even if you had a year?

author: kaz911
author_rank: Super Contributor
timestamp: December 02, 2016, 08:00:47 am
content: remember it can take a week or two to get a Visa for China.. ...

author: Molenaar
author_rank: Regular Contributor
timestamp: December 02, 2016, 09:52:15 am
content: This sounds like something that would not be too hard to implement in Python. But I second the customer buying a cheap scope and using skype to tell him/her what to connect where and which knobs to turn.

author: ebclr
author_rank: Super Contributor
timestamp: December 05, 2016, 04:11:59 am
content: Where is that Conchinchina ?

author: ebclr
author_rank: Super Contributor
timestamp: December 05, 2016, 04:19:26 am
content: https://github.com/sebnil/RealtimePlotter

author: vodka
author_rank: Frequent Contributor
timestamp: December 06, 2016, 07:32:40 pm
content: Quote from: ebclr on December 05, 2016, 04:11:59 amhttps://en.wikipedia.org/wiki/French_Cochinchina"Send you to Cochinchina " This is a typical spanish expression for referring to place that is very far , few known and very hard to arrived.

author: ebclr
author_rank: Super Contributor
timestamp: December 07, 2016, 01:01:24 am
content: And where is that far place ?

author: technix
author_rank: Super Contributor
timestamp: December 07, 2016, 01:31:08 am
content: You can set up a server receiving data, and send him a client software that would upload the data every second or so. Then analyse the data on your end, using your tools, to your hearts' content. You can even allow for some control commands to be sent over the server so you can change configs remotely too. (Apple uses this method to diagnose almost all of their devices remotely when you call them for tech support, with diagnose software built into every copy of iOS and macOS)Or you can use the protocol used by Arduino's serial plotter. Then send your customer a copy of Arduino software so he can use that feature.

author: Back2Volts
author_rank: Unknown
timestamp: December 07, 2016, 06:47:06 am
content: Quote from: ebclr on December 07, 2016, 01:01:24 amIt is just an expression referring to such a far away place that having to go to it would be an annoyance. Historically speaking seems to refer to a part of current South Vietnam, which when I was in school was Indochina. You know... that was when they used to print Google Maps in big colorful books

author: ebclr
author_rank: Super Contributor
timestamp: December 07, 2016, 08:38:09 am
content: Where means where, I don't care about Spanish expression, I would like to know where is the country that problem supossed to be solved, simple as that

author: Back2Volts
author_rank: Unknown
timestamp: December 07, 2016, 10:29:37 pm
content: Quote from: ebclr on December 07, 2016, 08:38:09 amSorry, it sounded to me like you were asking where Conchinchina was, (to which I answered), and not in what country the problem OP client is. It seems the client is in China.

author: RoGeorge
author_rank: Super Contributor
timestamp: December 07, 2016, 10:45:49 pm
content: Real Time Plotting Tool ( Oscilloscope )http://forum.arduino.cc/index.php?topic=58911.0