Are there any DIY Automated Optical Inspection (AOI) solutions available? - Page 1
Forum section: electronics / general

author: protoneer
author_rank: Regular Contributor
timestamp: March 30, 2016, 12:29:38 am
content: I have been building lots of boards lately and I was wondering if there are any DIY / Opensource Optical Inspections libraries or software available?Most of the time I did manual pick and placing but I am moving to machine placing and it opens me up for to making more mistakes with volume.I found this blog that seemed very helpful...http://bobbaddeley.com/2015/12/creating-an-automated-optical-inspector-for-50/But this is only a small part of the picture and I don't want to re-invent the wheel but is willing to contribute towards a project like this.

author: mrpackethead
author_rank: Super Contributor
timestamp: March 30, 2016, 01:45:59 am
content: I was discussing this with Thommo, last night, and with the open-pnp guys.. Open-CV already provides a way to do pattern recongiition.It would'nt take too much to run the downfacing camera of your pnp machine across the board. and do the checking?

author: rx8pilot
author_rank: Super Contributor
timestamp: March 30, 2016, 01:51:20 am
content: Joining this discussion - hoping for the same thing.

author: ChunkyPastaSauce
author_rank: Unknown
timestamp: March 30, 2016, 02:49:36 am
content: If you can provided a good training data for your application, then you may want to look into machine learning methods. I've had very good results with LIBSVMhttps://www.csie.ntu.edu.tw/~cjlin/libsvm/in other applications, and it's fast enough for many realtime applications after training; I haven't tried it for OCR but I'd expect the SVM method to work well for many OCR problems.

author: mrpackethead
author_rank: Super Contributor
timestamp: March 30, 2016, 02:54:36 am
content: nice nick. buy the way.

author: ChunkyPastaSauce
author_rank: Unknown
timestamp: March 30, 2016, 03:06:09 am
content: Quote from: mrpackethead on March 30, 2016, 02:54:36 amThanksI just took a look at the open-cv libraries. It includes a bunch of machine learning algorithms, including a SVM implementation [actually it's LIBSVM lol]........awesome!

author: mrpackethead
author_rank: Super Contributor
timestamp: March 30, 2016, 03:35:24 am
content: http://bobbaddeley.com/2015/12/creating-an-automated-optical-inspector-for-50/"n the world of electronics assembly, parts are really small, quantities are really large, and equipment is really expensive. The process of putting all the components onto circuit boards thousands of times per hour with really high accuracy is challenging, but tools are getting easier and cheaper every day.At my job (Quietyme), we’re doing our own circuit board assembly. We had to purchase some equipment, like a pick and place, which for $5000 automates the process of taking the small components off of reels and putting the components on the circuit board. We also have a stencil screenprinter and a reflow oven (toaster oven). Total investment so far has been under $6000.The most recent tool to add to our arsenal is an Automated Optical Inspector. The purpose of the AOI is to examine the circuit board to find defects in the assembly process. Sometimes a component doesn’t solder on correctly, or there is a bridge where there shouldn’t be. These need to be found and fixed before they are programmed and used. We had been doing it manually with just visual inspection, but that was tedious and not very accurate, especially after the 100th board."

author: Smallsmt
author_rank: Frequent Contributor
timestamp: March 30, 2016, 07:10:07 pm
content: Some months before I thought about to add this function to our PNP machine.But there are some more complicated things to solve.You need to add more than one camera from minimum 2 directions and light from different directions if you want to inspect the solder points too.It's easy to detect if there is a part or not but solder point inspection is more complicated.

author: thommo
author_rank: Frequent Contributor
timestamp: March 30, 2016, 07:41:41 pm
content: Congrats NickNice progress.As SmallSMT points out - no solder inspection, but every project starts somewhere and what you've achieved to date is all currently required to be done using the 'eye strain' method on every board anyway - so a big early step.It seems most Cartesian machines don't populate max PCB area anyway, so mounting an additional camera on the head should solve the problem. It doesn't really matter how many views or board sections are required in order to cover the whole area. I don't think image stitching would help either because interpolation could add some artefacts to the mix.We've dealt with a machine vision company called PointGrey previously and their product is great. With a good high resolution image, things like solder bridges should be detectable with high reliability also, given the footprint data already exists.Well done!!!

author: protoneer
author_rank: Regular Contributor
timestamp: March 31, 2016, 01:54:22 am
content: Quote from: mrpackethead on March 30, 2016, 01:45:59 amThat's a brilliant idea!!! I have a TVM802 arriving shortly and it sounds very hack-able as the cameras are just normal USB type and it looks like the Machine control comms have been hacked aswell.

author: protoneer
author_rank: Regular Contributor
timestamp: March 31, 2016, 02:11:55 am
content: I am also interested in checking solder-paste before starting the run,so some kind of inspection would be better than nothing.Importing component location data from an EDA(Kicad) might also be an option to filter out only the bits you are interested in.

author: l0wside
author_rank: Regular Contributor
timestamp: March 31, 2016, 12:50:28 pm
content: Solder paste print should be pretty simple. The solder paste is grey, the pad is either silver or gold, and the solder mask is green.Checking QFNs will be a pain.Max

author: mikeselectricstuff
author_rank: Super Contributor
timestamp: March 31, 2016, 01:38:24 pm
content: I though of one way to do it super-cheap. Set up a decent camera with view of the whole PCB, with an alignment jig ( or a decent area), take a photo of a reference PCB, then place the DUT in teh jig, take a pic and show the two pics alternately - the eye is avery good at spotting changes, so although not automatic, I think it could be very quick, quicker than 'real' AOI. You could add enhancement like emphasising changes, and a pan/zoom that tracks the two images.Considering the high resolution of modern digital cameras I think it could be a viable approach.

author: kayvee
author_rank: Regular Contributor
timestamp: March 31, 2016, 01:53:49 pm
content: A low cost solution would be a huge help in my setup too. I have been looking around for something but commercial options are crazy expensive.I've trained SWMBO on the task but it's not the most foolproof solutionKeen to see where this goes...

author: l0wside
author_rank: Regular Contributor
timestamp: March 31, 2016, 03:26:09 pm
content: Not having the slightest clue about the inner works of image processing, I just played around with an Android demo:http://www.barrythomas.co.uk/machinevision.htmlThe "Colour Range" function is quite simple. After setting up Python/OpenCV, finding a PCB in an image should be possible during an evening of playing around with a webcam and a computer. Finding all grey areas and checking if they match with the stencil definition should not pose a major problem either. The biggest effort might be comparing the Gerber data to all these rectangles that OpenCV will return.Complete solder joints are a different story. Professional systems usually do not look from straight above, but rather at 45°. Otherwise, you could not detect a solder joint where a TQFP pin dangles above the pad.Max

author: johnwa
author_rank: Frequent Contributor
timestamp: April 01, 2016, 07:55:50 am
content: I had a bit of a play around with this before using ImageMagick to find the difference between the sample and reference images. This was primarily targetting missing components, not bad soldering, I had good results transforming to HSV colour space, and just examining the saturation channel, though a better approach would be to use some function of all three channels. The saturation channel is colourised to red, and 'eroded', to reduce the noisiness, then blended back into a darkened version of the original.I didn't get round to building a proper light box and alignment jig, but results were promising just using a desk lamp and a the camera in a smart phone that was resting on a stack of books.I used the following shell script, if anyone wants to experiment:#!/bin/shREFERENCE="20160118_111042.jpg"SAMPLE="20160118_111108.jpg"# Using BMP for these doesn't seem to give the same result, colour depth/gamma issues?convert $REFERENCE -colorspace HSV -separate /tmp/reference%d.pgmconvert $SAMPLE -colorspace HSV -separate /tmp/sample%d.pgmcd /tmpcomposite reference1.pgm sample1.pgm -compose difference out.pgmconvert out.pgm -morphology Erode Square out2.pgmconvert out2.pgm -channel G -evaluate set 0 -channel B -evaluate set 0 out3.bmpconvert out3.bmp -brightness-contrast 40x50 out3.bmpconvert "$REFERENCE" -modulate 100,30 -brightness-contrast -30 reference-dark.bmpcd -composite /tmp/reference-dark.bmp /tmp/out3.bmp -compose plus out4.pngPGM was used for the intermediate files for speed, with the eventual intent that the process could be applied to a live video stream - put the board under the camera, and as soon at is registered correctly, the result would be displayed on the monitor. I think processing time was only a second or so for an image of a few megapixels; no doubt it could be accelerated further if desired.

author: Brumby
author_rank: Unknown
timestamp: April 03, 2016, 02:17:53 am
content: Not sure if this is covered in the software being used - but ....Whenever you take a photo, particularly at close range (less than a metre or so) you will get some degree of barrel distortion. The longer the distance and/or the longer the focal length of the lens, the less it will be - but practical situations for something the size of a PCB (or panel) means it is a real consideration.When you take a photo of a reference board and then, using the same optical setup, compare that to the board being examined, these distortions will be the same, allowing a fast and accurate examination - whether using a human (such as the 'blink comparator' method mentioned above) or machine vision.If you then want to take your CAD quality data and map it to the image, the distortion will need to be corrected or at least accounted for, since the drawing files will have digital precision on perfectly rectangular coordinates - and the photo won't.

author: JardCrocker
author_rank: Newbie
timestamp: November 02, 2017, 05:13:26 pm
content: Hi..i am a new user here. As per my knowledge Open-CV already provides a way to do pattern recognition.It will take too much to run the downfacing camera of your pnp machine across the board.turnkey pcb

author: mrpackethead
author_rank: Super Contributor
timestamp: November 02, 2017, 06:16:51 pm
content: The commerical machines i've seen seem to run a X/Y gantry over boards with their cameras. They also seem to use differnet color light for differnet process's.If i only had time to spend on this, it would be awesome. Compared to building a PNP machien though, mechanically this shoudl be easy!

author: protoneer
author_rank: Regular Contributor
timestamp: November 02, 2017, 06:52:48 pm
content: Quote from: mrpackethead on November 02, 2017, 06:16:51 pmI am slowly starting to look at a project doing this. The hardware for movement is simple. Consistent lighting and camera is a huge factor.Software will be the magic sauce but can only be looked at as soon as the other stuff can be done consistently.

author: protoneer
author_rank: Regular Contributor
timestamp: November 02, 2017, 06:56:25 pm
content: One good thing about running a small company is that resources are limited and it forces you to innovate.Funny that Quality and Inventory management is still my major issues...

author: mk_
author_rank: Regular Contributor
timestamp: November 02, 2017, 07:36:28 pm
content: Quote from: protoneer on November 02, 2017, 06:52:48 pmhttps://www.mevislab.de/This is sw written for - at the first sight - completly different field but - comparing pictures in a medical disipline isn`t very different from good/bad placement of a pcb...there is also a free version aviable with some minor restrictions, which shouldn`t harm a solution for pnp-verification, seehttps://www.mevislab.de/mevislab/versions-and-licensing/hthmichael

author: cdev
author_rank: Super Contributor
timestamp: November 02, 2017, 08:54:54 pm
content: The concept of eigenvalues and eigenvectors - related terms (for example, eigenimage, eigenface) may bring you to a fair amount of stuff on differences from average values.I also remember having seen writeups of projects using various machine vision toolkits (there are quite a few tools out there) Opencv is just the best known of many- etc. to do electronics quality control.For example, you might want to look at the world of GIS tools.Your application is actually a GIS application, its just thearea you are looking at is smallerthan most of them usually are.Quote from: johnwa on April 01, 2016, 07:55:50 amPGM is actually used as theintermediate file formatbecausethats its purpose, for use in applications exactly like the one you describe.

author: mrpackethead
author_rank: Super Contributor
timestamp: November 02, 2017, 09:22:55 pm
content: have you got some pictures you can post? of the input/reference output of those?

author: cdev
author_rank: Super Contributor
timestamp: November 02, 2017, 09:23:14 pm
content: Look at GIS software and software for interpreting space imaging. Lots of it is free. For example, the ESA has a bunch of tools, fairly advanced ones which can give you quantitative analysis from bitmaps - they are designed for automated analysis of things like vegetation, terrain cover, changes in geographical things like roads and waterways.. that maps very closely to circuits, from the point of view of the software.Another area is segmentation software. There look at medical imaging software. If any of you have a Mac, check out Osirix, which is for medical imaging, you can perhaps get some ideas on what those kinds of terms mean. For example, you might have certain areas on your boards where the same errors always come up. Likely you could hack together some automated way of examining them using this wealth of free software.One good free resource is theNASA Technical Reports Server.Try a search on machine vision in the context of electronics quality and process control.There is tons of useful scientific information on things like that on NTRS.There is a good chance you could find something specific to your problem domain that you did not need to adapt.

author: sasul
author_rank: Newbie
timestamp: March 28, 2019, 10:32:26 am
content: I was also looking for open source AOI software, and could not find any.So I wrote my own software for this purpose.It's based on dlib object detection using fhog.Seehttps://bitbucket.org/guenterneust/examinator/wiki/Home

author: SWR
author_rank: Regular Contributor
timestamp: April 26, 2019, 12:35:31 pm
content: Quote from: Smallsmt on March 30, 2016, 07:10:07 pmI think it's a very good idea.Instead of multiple cameras couldn't the first step be to use the camera that is already there and change the position to get different angles?It's using the aperture angle of the camera to create different viewing angles.Example with 2 90° angles: position below left of pad and use upper right of picture, position upper left of pad and use lower right of picture.Example with 3 90° angles: position below pad and use upper picture, position left of pad and use right of picture, position above pad and use lower picture.It should be possible to define inspection points for each footprint type in the library and offset them with the P&P file X, Y and rotation.That would be a cool feature to have.