Taking data on LCD(320x240) to PC - Page 1
Forum section: electronics / projects

author: tlhsglm
author_rank: Contributor
timestamp: November 02, 2021, 06:56:00 am
content: Hi fellows,We have a machine that display screen is Hitachi SX14Q001, datasheet is below. In that display machine shows a couple of sensor values.I don't want to take data from sensors .I have 2 main ideas to do this (I prefer 1st if i can);1-) Putting a camera front of LCD and using OCR to take data.I can't put a camera front of machine because display has to be seen clearly. So i have to add one more display and putting camera front of new display but this is expensive.Another thing that comes to my mind is to mirror the image of the existing screen somewhere in the machine(but very tiny space i have), without buying a new screen. Then take this image to the camera and apply OCR.It may sound like science fiction, but I think I've heard of it -or so it seems to me- , isn't there a screen protector-like glass that I can stick on top of the screen and take the image of the screen, even if it is faint? If not feel free to laugh2-) Reading LCD from it's 16 pins.I searched how to do this but every source is about 16x2 LCD displays. Are they same? In datasheet there is timings of the pins is this can help me out?How do I distinguish between text and color when I read the pins? Do I need to create the same screen on the computer by doing visual programming?http://www.farnell.com/datasheets/29979.pdfBest Regards, thanks for reading.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: November 02, 2021, 10:34:57 am
content: Sure, it can be done.Interface is fairly simple, 8 bits with sync. It's streaming live raster from a controller, so on the one hand you don't need to remember anything, it's always right there for the viewing; on the other, it's drinking from a firehose, and preferably you'll have some sort of frame grabber, or enough bandwidth and good timing, to select and capture the pixels in question.Probably there are LCD capture cards that can do this, maybe with some descrambling since the colors are in bit-sequential order but most sources are high-color (16 to 24 bit/pixel). They could conceivably be used with this, with some image processing to restore the original format (8 colors, 3 bit/pixel).Doing it yourself, will take at least an MCU with good timing; preferably an FPGA, or any kind of interface chip that's purpose made for this (up to and including a whole-ass capture thing, if it exists). You also need an electrical interface, probably some connectors to mate the original display (flex) to, while tapping off the signals to capture. So there will be at least some hardware involved, and then some amount of software development to make it run. I would guess anywhere from a few days' labor to adapt an off-the-shelf device, and a couple days more to get it to connect and get the formats right; up to several months building it from scratch. So, worth spending a few hours first, checking if solutions exist, to see if it's on the easy or hard side of that range.Are you sure there are no better solutions? The display machine (or anything else in the system) doesn't have an accessible MCU? Could its input protocol be reverse-engineered faster than this? Does it have outputs (e.g. serial or other comms) that might already provide this information?Tim

author: voltsandjolts
author_rank: Unknown
timestamp: November 02, 2021, 10:50:37 am
content: Quote from: tlhsglm on November 02, 2021, 06:56:00 amWhy not?Depending on the sensor type, it might be simpler than all the display decoding.But if the sensor has a complex calibration that you don't have access to, then screen-scraping may be simpler.

author: Renate
author_rank: Super Contributor
timestamp: November 02, 2021, 12:39:35 pm
content: Are you sure that the screen format stays the same? That it doesn't go to alarm or confirm or login or anything?If the sensors are I²C or SPI or serial you can "spy" on them and just let the stock CPU query them.OTOH, if this is like DRO (digital readout) on a milling machine or something, you won't have access to the axis zero.

author: tlhsglm
author_rank: Contributor
timestamp: November 02, 2021, 02:10:30 pm
content: Thank you all for your kind replys.@T3sl4co1lOh lord! I don't have months.1-) Can BusPirate-like cards help me out?2-) If the SX14Q001 has RAM, wouldn't it be easier to read it?@T3sl4co1l @Renate @voltsandjoltsSensors are expQuote from: voltsandjolts on November 02, 2021, 10:50:37 amYes and some sensors i cant reach and allowed. So it seems screen-scraping is easiest way for now. Which brings us two questions;1-) Mirror the image of the existing screen somewhere in the machine(but very tiny space i have). Then take this image to the camera and apply OCR. If there isn't a screen protector-like glass that I can stick on top of the screen and take the image of the screen.2-) Adding a new screen and connecting it to the pins of the old one with a parallel connection.

author: coromonadalix
author_rank: Super Contributor
timestamp: November 02, 2021, 05:18:33 pm
content: Dont want to be rude BUTYou dont give any specs of your machine and or the sensors in it, you have National instruments usb daq and many thingsYou overthink things and add complexity for nothing, or somekind of lazyness to do so ?Give us more details of the sensors you need to readNobody here will design things with poor explanations like this ?? it's pure guessing ...As other previous threads, some sensors can be really easy to spy / sniff without creating problems, they could be 4-20 ma models or say 0-10 volts etc ... you see what i mean it's guessing this or that ... you could have an plc controller who could/can be read ?

author: Renate
author_rank: Super Contributor
timestamp: November 02, 2021, 05:24:05 pm
content: Quote from: coromonadalix on November 02, 2021, 05:18:33 pm+1How many guesses do we get?Are you dead sure that this device doesn't have any expension interfaces for a optional feeder, debug port, logger, who knows.An optical solution is silly and will bite you in the butt when somebody knocks it a bit and it's 5 degrees off.

author: tlhsglm
author_rank: Contributor
timestamp: November 02, 2021, 08:22:39 pm
content: It is a very big textile machine tomorrow i will add pictures of it but there is a lot of cables and circuits inside it. Purpose of the my work is to report when machine shows problem and stops. I didn't detailed sensors because machine have maybe one hundred sensors. You get angry a little, this because you cared my questions so thanks a lot for your care.Since it is a man sized textile machine and it have a lot of big circuits, i dont know where to capture photo for right track. Is it nice to track backwards from lcd display?

author: Renate
author_rank: Super Contributor
timestamp: November 02, 2021, 10:43:53 pm
content: Quote from: Renate on November 02, 2021, 05:24:05 pmHmm. Is this a circular knitting machine with a thread out sensor on both the entrance and exit of each of the positive feeders (along with a broken needle detector)?

author: tlhsglm
author_rank: Contributor
timestamp: November 03, 2021, 07:41:32 am
content: Quote from: Renate on November 02, 2021, 10:43:53 pmYes but not just them, there are pneumatic sensors, vacuum sensors, etc a lot of sensors. I also want to reach speed of the machine, all i wants are at lcd display.

author: Renate
author_rank: Super Contributor
timestamp: November 03, 2021, 11:42:55 am
content: You're not going to get a good solution until you define your problem a lot better.You have a manufactured device that runs by a microprocessor. If you want to improve some aspect of operation or reliability you're going to have to come up with something smarter and more reliable than the manufacturer and you're going to have to come up with a precise definition of the problem (something that we can understand).Adding some half-baked modification to something that is fundamentally working is only an invitation to disaster. Reading a display is half-baked. There is delay and reliability of the information getting to the display, there is delay and reliability reading the display.You've never explicitly said whether you are talking about the equipment not meeting a guaranteed spec or whether you require something above and beyond what the equipment was originally supposed to do.

author: Buriedcode
author_rank: Super Contributor
timestamp: November 03, 2021, 11:26:20 pm
content: I'll side-step the X-Y problem, and whether your plan is sound, and comment on the LCD.I have one of those displays on my desk, old style CSTN, without a controller. This means data is constantly streamed to the display, the character displays you mention have a controller on board, with RAM that constantly refreshes the display - you send a character to display, once, and the controller puts on the display. Without a controller, the lines going to your display will essentially constantly clock in pixel data every frame.That makes it tough to decode, but no impossible. Others have mentioned an FPGA which is pretty much the only viable option (although a relatively fast MCU with lots of RAM might work). What complicates matters is the fact its acolourSTN display. In monochrome displays, there is one data bit per pixel, with 4-bit interface, so thats 4 pixels every clock. Thats quite easy to read into a block of memory and I've done it with an ESP8266.But colour displays have 3 sub pixels, so each 8-bit "word" it sends on D7-D0 is 2 and 2/3 pixels (R1,G1,B1,R2,G2,B2,R3,G3), so you'll probably have to read in an entire frame as raw data, go through that frame of data to convert it to an easier format, then use this to pluck out values. It's a hell of a task. Certainly possible, but a LOT of work.At the other end of the argument is reading the sensors directly.Perhaps theres a middle ground here: If you have access to the machine, then it must have a controller for the display on board. That will either be an ASIC, like an Epson LCD controller, or built in to an MCU (lots of Phillips ARM MCU's have on board STN/CSTN controllers.Ifit is a separate controller, then you can tap the lines going from the host MCU/MPU to this chip - just like tapping the lines on a 16x2 character display.Whilst it'll have pixel data for graphics and the like, often these controllers have built-int character generators, so the host just sends ASCII to the controller, which then displays that specific character.They also often have an 8-bit 8080 or 6800 interface, that can be read pretty easily just by using external interrupts. This way you don't have to faff about trying to match bitmaps to characters and just read the data as ASCII characters. I'm not saying thats what your machine has, just that, if it is using a standalone controller, it'll make life a lot easier.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: November 04, 2021, 02:14:37 am
content: Yeah, it sounds like there probably isn't a cheap solution, so the question comes down to: which less-cheap solution do you want to pursue?First call is the manufacturer. What support do they offer, if any? Is this a very old (unsupported) machine? Is a newer machine available, how much would it cost (and would they buy back the old machine, or can it be sold, so you're only out the difference), and how much value might a newer model provide (not just concerning the present issue but raw throughput, functionality, etc.). It's a business decision, it may not be great (it may cost something to take any action, the best choice simply being the least-bad one) but hopefully some of these options lead to increased productivity and thus some payback time.Second is aftermarket, if applicable. Now, I'm guessing these things are pretty specialized and captive, so no one's marketing products to fit them.Third is engineering a new solution. No one else has one so you have to figure it out yourself.You can do it actually-yourself, but expect to work very patiently on it. I gather your electronics knowledge is limited (please correct me if I've read this wrong!), so you will need to study the basics first, and then whatever the various parts of the machine are doing, and then how to solve this problem. By "very patient", I mean, most kids graduating from university probably wouldn't know where to start; of course, a grad has a big leg up on the basics, the theory, and where to look, what questions to ask. So it's not that that'd be a bad route; a recent grad will certainly be cheaper than a veteran, but expect it to take longer, too.As for experienced engineers, that's what I mean: worst case, having to reverse-engineer everything, and build an interface and decoder from scratch, could take months. It could also take just a few days, if there are off-the-shelf solutions that can be (nearly) dropped in, or there are interfaces built in to the system that you just didn't know about.And as far as spending months doing it, it may well be profitable to reverse-engineer whatever digital logic or controller(s) the system has; maybe they have interfaces built in, or that can be activated with a little cleverness (e.g. a debug or service mode). This isn't unusual; a famous present-day case is McDonald's ice cream machines, which break down notoriously often, having a service mode that owners aren't supposed to know about, and someone made a dongle to attach to it, greatly easing maintenance... which the manufacturer didn't approve of.Or, if such an interface truly isn't provided, perhaps it can be created anyway. A likely case, is there's a microcontroller onboard, which can be reprogrammed with relative ease (but, it depends how old/obscure it is, and if it's locked or one-time-programmable), and some spare pins can be found on the board to attach a convenient interface (like a serial port).RE-ing software could take quite a while indeed (weeks to several months), so it's probably not a route to stake the job on, but it would be worth a few days, or a week or two even, to see how feasible it is.In summary, an engineered solution comes down to:- Someone knowledgeable has to see the machine, or otherwise be familiar with it.- Someone will need a few days, at fair market rate for such services in the area (maybe a few USD/hr in India/China, up to 100+ for EU, US), to determine feasibility of the methods.- And then a few more days, to months, to implement the method.And it doesn't have to be the full stack, if you find an engineered solution is just too slow/expensive, it might simply be best to deal with it, or try the hacky solution. If it's not actually hard at all, there you go, no need for hackery.Generally speaking, engineering isn't cheap, and if this is a big expensive piece of industrial machinery, it isn't cheap either. So, it's just a range of alternatives. Such is business.And, by the way -- sure -- it might well prove to be the case that the dumbest solution is the best. If it's dumb, but it works -- as the saying goes. It's certainly feasible to set up a webcam on a bracket, and do image analysis on that. This incurs a HUGE number of assumptions, which mean you might not get useful data -- any obstruction in the optical path, your reading is gone; the image analysis itself is likely fragile and prone to error -- but as complicated as it is, it's really just a few Python libraries from being done. The challenge is which ones and how; and, me personally, I don't have that knowledge, I would prefer the electronic solution -- but YMMV, those more skilled in Python or other languages may have more insights to offer.Tim

author: dmendesf
author_rank: Frequent Contributor
timestamp: November 04, 2021, 02:46:23 am
content: I can totally understand why you want to get the data from screen. All you need is. Stm32 with camera interface (or even better the parallel slave port". Both can accept a stream of data and generate a XY graph in memory where you can read the interesting pixels.

author: voltsandjolts
author_rank: Unknown
timestamp: November 04, 2021, 08:49:10 pm
content: Quote from: tlhsglm on November 02, 2021, 08:22:39 pmIf the machine stops, it's current consumption from the mains power would likely drop significantly

author: Renate
author_rank: Super Contributor
timestamp: November 07, 2021, 04:52:03 pm
content: There certainly has to be an alarm circuit out of this device. it's not cheap and a production floor would have lots of them.There could be SNMP or serial or MODBUS or something.Do you have full schematics? Are they any empty connectors on this?

author: tlhsglm
author_rank: Contributor
timestamp: November 08, 2021, 07:06:49 am
content: Quote from: Buriedcode on November 03, 2021, 11:26:20 pmYES! There is Epson S1D13503F00A2 LCD Controller. Datasheet is below. Now i read the datasheet but im not so sure which pins i have to look closer for? Kindly can you please tell me where i should take a closer look?Lcd while workingLcd driver circuit from aboveLcd driver circuit from leftLcd driver circuit from middleLcd driver circuit from righthttps://www.alldatasheet.com/view_datasheet.jsp?Searchword=S1D13503F00A2Quote from: dmendesf on November 04, 2021, 02:46:23 amI've used STM32F407 before. Where can I connect STM32 and perform the operation you specified? By conneting parallel to the pins going to the LCD screen?@T3sl4co1lThank you very much for your inspiring and instructive engineering perspective@RenateCan you please update from using info from above?

author: T3sl4co1l
author_rank: Super Contributor
timestamp: November 08, 2021, 10:15:31 am
content: Oh neat, parallel interface. I wonder what MCU/CPU it uses. Anything on the back side of the board? Follow the ribbon cables? (Some of which I think go up to, solenoids or something, I just wonder about whatever goes back to the main / CPU board.)Tim

author: tlhsglm
author_rank: Contributor
timestamp: November 08, 2021, 11:33:12 am
content: Quote from: T3sl4co1l on November 08, 2021, 10:15:31 amGood to hear "neat" word from you mate. There is nothing on backside of board, image is below for traces. Main MCU is Intel i960.Main cpu (i guess)Wide look in circuitsBack side of Lcd driver(Epson)'s circuitRibbon cables source (Two ribbon is coming from this circuit

author: Renate
author_rank: Super Contributor
timestamp: November 08, 2021, 04:14:29 pm
content: That's quite a sophisticated piece of equipment. Do I read that PCB correctly, there is a barcode reader too?It must have some interfaces for external systems. Have you asked the manufacturer?Are there any accessories that it could have that you don't have (like a logging line printer or something).

author: Cerebus
author_rank: Super Contributor
timestamp: November 08, 2021, 04:26:34 pm
content: Quote from: Renate on November 07, 2021, 04:52:03 pm^This.This kind of machine is a prime candidate to have some fault/alarm output, probably on a relay or three, probably designed to drive a green/amber/red 'traffic light' pole. Ten minutes with the installer's manual would probably find the details.

author: tlhsglm
author_rank: Contributor
timestamp: November 09, 2021, 06:55:43 am
content: Quote from: Renate on November 08, 2021, 04:14:29 pmWhere is barcod reader i cant see it?The manufacturer is not supporting this model.Which circuits do we need schematics of? The circuit that have LCD Driver. Two ribbon cables source circuit. Do we need another?

author: Renate
author_rank: Super Contributor
timestamp: November 09, 2021, 12:08:06 pm
content: The PCB says "Bar Co..." in one of the photos. Not that this does us any good exactly but if it's taking data like that in there's probably some way it goes out too.Installation manual would be good.There could be any number of accessories, a remote status panel (LCD 2 x 20 would be much easier to read than graphic LCD) or some supervisory circuit.Maybe photos of the external connector panel. How many connections are there? Or is there really no bulkhead termination and all the cables just sneak into the cabinet?

author: tlhsglm
author_rank: Contributor
timestamp: November 10, 2021, 02:05:52 pm
content: Quote from: Buriedcode on November 03, 2021, 11:26:20 pmI've found the lcd driver. Kindly could you give me a next trace ?

author: Buriedcode
author_rank: Super Contributor
timestamp: November 11, 2021, 07:23:53 pm
content: Quote from: tlhsglm on November 08, 2021, 07:06:49 amNice photos. The good news is that it appears that the board is using 74LVC541's as level translators or just buffers - the SX14Qxxx series run at 3.3V, and that Epson controller only has one power supply (sometimes they allow different votlages for host IO and display IO) so it runs at 3.3V. Iassumethat the host conotrller that supplies signals in those ribbon cables are 5V, so the 541's are there to make 5V - > 3.3V.Thats good news because it means you can tap those lines, either by soldering wires to the ribbon cable shrouded headers (on the underside of the board) or inserting a board between the ribbon cable and the header. This sounds like a lot but its easier than soldering kynar wire to the Epson chip or the 541's. But this leads to thebad news:Whilst most graphics controllers can use a simple 8-bit interface - where all data is transfered in byte blocks with 4/5 control lines, meaning only 12/13 connections are required, the more complicated controllers for larger displays - or colour displays that take a lot more data - use direct address. It appears the one here uses the full address/data interface, where the host controls up to 20 address lines, plus 16 data lines PLUS the control lines. In your photos, specifically "Lcd-driver-middle" - you can see that all those pins along the bottom of the Epson chip are connected to 541's, that themselves are connected to the ribbon cable headers.So, my nice idea of "just tap the data lines!" suddenly gets a hell of a lot more difficult because you're having to look at 20 + 16 + 5/6 = 41/42 signals just to read in data. Plus, going over the epson datasheet, it looks like it doesn't generate its own fonts - the host literally draws characters as bitmaps. It's still, in a way, easier than looking directly at the LCD signals, as those are RGB bits for each pixel, where-as the data from the host is probably one byte-per-pixel. Bit its still an arse as you'll have to recognise characters based on what pixels are on/off.So, whilst it'll be a nice challenge, I'd go with the other suggestions or either tapping the sensors directly - maybe analogue sensors are conditioned by hardware and read by ADC's, in which case you could jsut buffer and read those too. Or perhaps the other end - optical character recognition with a camera.Pity, I got all excited as I've been working with those controllers for a while and they're old, and pretty complicated. As I said, it's still technically possible, but a lot of work in decoding it all, and wouild need either a PCB to tap the signals or a lot of wires soldered to the back of that board.edit: after mulling it over, it might be slightly easier if certain things are constant:1) if the same font (and font size!) is used all the time: make it easier to recognised characters from the pixel data. If you're only looking forvaluesthen thats even easier as you only have to recognised 0-9 and perhaps +/-, decmial point etc.. From those photos it looks like they're using standard 5x7 with one pixel spacing. The popup menu looks like larger font - hopefully thats not the area of interest.2) if the values of interest are in the same place on screen all the time. This means you can ignore most of the datastream, and only "record" a specific part, that you can find by counting the pixel clock.With the above two conditions it gets easier, but I'm still thinking OCR is the better way. It'll cost more in hardware, but should be much quicker in terms of writing software - prewritten libraries. Trying to match pixel data in an array to fonts can be done even by 8-bit micros, but its a lot of faffing about, bit shifting, comparisons, etc.. it's just a lot of hassle.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: November 11, 2021, 09:28:41 pm
content: Would be interesting to know what the host is. Something with an exposed parallel bus, in any case. If it's an MCU with exposed bus (e.g. STM32 FSMC, or various others across the market), or bit-banged GPIOs, it might be a right PITA. Or if it's like a PC104 single-board computer, it might be as trivial as stacking on a comm port expansion board (or it even has unused ports available as-is) and shimming in some monitoring software (the nature of which is still a big question, mind).Tim