Lego Automatic Sorting LegoLAS 2 0 - Further Development of a Mechatronic Lego Sorting Machine
Channel: Jörn Schlingensiepen
Date: 2021-08-23

Lego Automatic Sorting (LegoLAS 2.0) - Further Development of a Mechatronic Lego Sorting Machine - Student Projects in the Laboratory for Computer Science in Engineering and Computational Mathematics - i2cm (Arduino, Raspberry Pi, 3D printing, Image Processing, Machine Learning, TensorFlow)

Lecturer: Prof. Dr.-Ing. Jörn Schlingensiepen

Deutsche Version dieses Videos:
https://www.youtube.com/watch?v=nvlKVkN6aeg

Project page on gitHub:
https://github.com/LegoAS/LegoAS

CAD data on OnShape: 
https://cad.onshape.com/documents/987d7bcb5ba09db685ee5959/w/9b6ee89cc72c5f3be05c2815/e/2b4e90a536956ffc8c740721

Video about first version of LegoLAS:
https://www.youtube.com/watch?v=dJLl3fBxQ1Q

As part of a study project 10 students supervised by Professor Jörn Schlingensiepen developed and built the first Lego sorting machine in the Laboratory for Computer Science in Engineering and Computational Mathematics in the summer semester 2011. 

Time did not stand still, but the subject of Lego bricks retained its fascination, also sorting Lego bricks is a simple and all-time available example of a complex process. Engineers love building machines for those tasks.

Since the project tasks for students projects shall always represent a complete product development, it was obvious to build a machine in the summer semester 2019 with a similar topic. This time the solution should use the cheap embedded systems available today and the new and easy to integrate machine learning tools available on the market. So 10 students developed and built the LegoLAS during the summer semester of 2019. LegoLAS is a recursive acronym for ‘LegoLAS - Lego Automatic Sorting’. Maybe some of you already know this one (https://www.youtube.com/watch?v=dJLl3fBxQ1Q). In the same academic year, the team was awarded the prize for the best project work.

We learned a lot while developing LegoLAS, but unfortunately it was permanently attached to the lab equipment, so it had to remain a mock-up. Based on this preliminary work, Vinzenz Malke developed and built a new machine, the LegoLAS 2.0, in the winter semester 2019/2020. Markus Steuerer undertook the task of developing and integrating the recognition of the different bricks.

Both wrote their bachelor's theses on this under supervision of Professor Schlingensiepen. Those theses are now the basis for the documentation Wiki on gitHub. Indeed, the complete documentation is available there and on OnShape (see links above).

Along the way over 72,000 photos and 300,000 rendered images of Lego bricks were taken, which are available on the project page. Maybe you'd like to try your hand at machine learning and accept the challenge of developing a better solution than ours.

First step in product design is defining a functional structure. As with the first version of LegoLAS the functions are:
- Pick up stones
- Separate stones
- Recognize stone
- Classify stone
- Place stone
- Collect stones
 
Depending on the solution there are different machine parts the stone has to pass, therefor another function

- Transport stone or stones

has to be implemented.

The stones are delivered in an overhead storage container, because gravity serves as the energy supply for the transport steps.
First the stones are pre-separated in a rotating tube.
After pre-separation, the stones fall into a chute, which completely separates the stones by swinging back and forth.

The individual stones fall into a recognition box, a simple Plexiglas tube. Here they are recorded by three cameras. Due to the different recording angles, it is possible to recognize not only the color but also the extension of the stones in all spatial directions.

The recognition is done by a neural network, which was created and trained by TensorFlow.

The assignment to the collection bins is done by a rotating tube whose end is bent. After recognition, the kinked part is rotated in the direction of the matching collection container and the recognition box is rotated so that the recognized stone falls into the kinked tube at the top and then comes out at the bottom in the direction of the matching collection container.

Using this simple approach 10 different outputs were realized, which are sufficient for the 10 basic colors of the Lego world. In this video you see the demonstration at the Landesgartenschau in Ingolstadt, where the machine sorts only by color.

For a real life application we have implemented a stack mode, where different bricks are first sorted together into joint storage bins. These joint storage bins contain a limited number of different types of bricks, thus re-sorting them results in fully sorted bins.

We hope you have enjoyed our little demonstration. The links to the documentation of the LegoLAS 2.0 can be found above. Get inspired to build your own machine or build an improved version, we would love to hear from you.

In the meantime: Take care and stay safe!

Music “Waiting For The Train“ by musicfox: https://www.musicfox.com