Artificial Intelligence in Mechanical Engineering | by Sunny K. Tuladhar | Towards Data Science

Artificial Intelligence and Machine Learning seems to be the current buzzword as everyone seems to be getting into this subject. Artificial Intelligence seems to have a role in all fields of science. According to Britannica , “Artificial intelligence (AI), is broadly defined as the ability of a digital computer or computer-controlled robot to perform tasks commonly associated with intelligent beings.” By intelligent beings it basically means humans ... but maybe not all humans...so anyway,It is usually classified into three subsets as shown below.
Artificial Intelligence is a broader term which in cooperates Machine Learning. Machine learning uses statistical methods to allow machines to improve with experience.
Deep Learning, again, is the subset of Machine Learning which uses multi layer neural networks that mimic the human brain and can learn incredibly difficult tasks with enough data.
We are going to talk about Deep learning methods and its possible role in the field of Mechanical Engineering. Some common examples could be Anomaly Detection(Machine Learning) and Image based Part Classification(Deep Learning). The focus will be on Image based part classifiers and why we need them.
Firstly, what is an image classifier? The ever famous AI which recognizes cat-dog pictures should come to mind. Here’s a link to the code of such a program. The data-set used contains images of cats and dogs, the algorithm learns from it and then is able to guess with 97% accuracy whether a randomly shown image is a cat or a dog.
We will attempt a similar code but using Nuts, Bolts, Washers and Locating Pins as our Cats and Dogs..... because mechanical engineering.
So how does it work? An algorithm is able to classify images(efficiently) by using a Machine Learning algorithm called Convolutional Neural Networks(CNN) a method used in Deep Learning. We will be using a simple version of this model called Sequential to let our model distinguish the images into four classes Nuts, Bolts, Washers and Locating Pins. The model will learn by “observing” a set of training images. After learning we will see how accurately it can predict what an image (which it has not seen) is.
Go directly to the code in github
We downloaded 238 parts each of the 4 classes (Total 238 x 4 = 952) from various part libraries available on the internet. Then we took 8 different isometric images of each part. This was done to augment the data available, as only 238 images for each part would not be enough to train a good neural network. A single class now has 1904 images(8 isometric images of 238 parts) a total of 7616 images. Each image is of 224 x 224 pixels.
We then have our labels with numbers 0,1,2,3 each number corresponds to a particular image and means it belongs to certain class
#Integers and their corresponding classes{0: 'locatingpin', 1: 'washer', 2: 'bolt', 3: 'nut'}
After training on the above images we will then see how well our model predicts a random image it has not seen.
The process took place in 7 steps. We will get to the details later. The brief summary is
Data Collection : The data for each class was collected from various standard part libraries on the internet.Data Preparation : 8 Isometric view screenshots were taken from each image and reduced to 224 x 224 pixels.Model Selection : A Sequential CNN model was selected as it was simple and good for image classificationTrain the Model: The model was trained on our data of 7616 images with 80/20 train-test splitEvaluate the Model: The results of the model were evaluated. How well it predicted the classes?Hyperparameter Tuning: This process is done to tune the hyperparameters to get better results . We have already tuned our model in this caseMake Predictions: Check how well it predicts the real world data
Data Collection : The data for each class was collected from various standard part libraries on the internet.
Data Preparation : 8 Isometric view screenshots were taken from each image and reduced to 224 x 224 pixels.
Model Selection : A Sequential CNN model was selected as it was simple and good for image classification
Train the Model: The model was trained on our data of 7616 images with 80/20 train-test split
Evaluate the Model: The results of the model were evaluated. How well it predicted the classes?
Hyperparameter Tuning: This process is done to tune the hyperparameters to get better results . We have already tuned our model in this case
Make Predictions: Check how well it predicts the real world data
We downloaded the part data of various nuts and bolts from the different part libraries on the internet. These websites have numerous 3D models for standard parts from various makers in different file formats. Since we will be using FreeCAD API to extract the images we downloaded the files in neutral format (STEP).
As already mentioned earlier, 238 parts from each of the 4 class was downloaded, that was a total of 952 parts.
Then we ran a program using FreeCAD API that automatically took 8 isometric screenshots of 224 x 224 pixels of each part. FreeCAD is a free and open-source general-purpose parametric 3D computer-aided design modeler which is written in Python.
As already mentioned above, each data creates 8 images of 224 x 224 pixels. So we now have a total of 1904 image from each of the 4 classes, thus a total of 7616 images. Each image is treated as a separate data even though 8 images come from the same part.
The images were kept in separated folders according to their class. i.e. we have four folders Nut,Bolt, Washer and Locating Pin.
Next, each of these images were converted into an array with their pixel values in grayscale. The value of the pixels range from 0 (black), 255 (white). So its actually 255 shades of gray.
Now each of our image becomes a 224 x 224 array. So our entire dataset is a 3D array of 7616 x 224 x 224 dimensions.7616 (No. of images) x 224 x 224 (pixel value of each image)
Similarly we create a the label dataset by giving the value of the following integers for the shown classes to corresponding indexes in the dataset. If our 5th(index) data in the dataset(X) is a locating pin , the 5th data in label set (Y) will have value 0.
#integers and the corresponding classes as already mentioned above{0: 'locatingpin', 1: 'washer', 2: 'bolt', 3: 'nut'}
Since this is an image recognition problem we will be using a Convolutional Neural Network (CNN). CNN is a type of Neural Network that handles image data especially well. A Neural Network is a type of Machine learning algorithm that learns in a similar manner to a human brain.
The following code is how our CNN looks like. Don’t worry about it if you don’t understand. The idea is the 224 x 224 features from each of our data will go through these network and spit out an answer. The model will adjusts its weights accordingly and after many iterations will be able to predict a random image’s class.
#Model descriptionModel: "sequential_1"_________________________________________________________________Layer (type)                 Output Shape              Param #   =================================================================conv2d_1 (Conv2D)            (None, 222, 222, 128)     1280      _________________________________________________________________activation_1 (Activation)    (None, 222, 222, 128)     0         _________________________________________________________________max_pooling2d_1 (MaxPooling2 (None, 111, 111, 128)     0         _________________________________________________________________conv2d_2 (Conv2D)            (None, 109, 109, 128)     147584    _________________________________________________________________activation_2 (Activation)    (None, 109, 109, 128)     0         _________________________________________________________________max_pooling2d_2 (MaxPooling2 (None, 54, 54, 128)       0         _________________________________________________________________flatten_1 (Flatten)          (None, 373248)            0         _________________________________________________________________dense_1 (Dense)              (None, 64)                23887936  _________________________________________________________________dense_2 (Dense)              (None, 4)                 260       _________________________________________________________________activation_3 (Activation)    (None, 4)                 0         =================================================================Total params: 24,037,060Trainable params: 24,037,060Non-trainable params: 0
Here is a YouTube video of Mark Rober (a NASA mechanical engineer) explaining how neural networks work with very little coding involved.
Now finally the time has come to train the model using our dataset of 7616 images. So our [X] is a 3D array of 7616 x 224 x224 and [y] label set is a 7616 x 1 array. For all training purposes a data must be split into at least two parts: Training and Validation (Test) set (test and validation are used interchangeably when only 2 sets are involved).
The training set is the data the model sees and trains on. It is the data from which it adjusts its weights and learn. The accuracy of our model on this set is the training accuracy. It is generally higher than the validation accuracy.
The validation data usually comes from the same distribution as the training set and is the data the model has not seen. After the model has trained from the training set, it will try to predict the data of the validation set. How accurately it predicts this, is our validation accuracy. This is more important than the training accuracy. It shows how well the model generalizes.
In real life application it is common to split it even into three parts. Train, Validation and Test.
For our case we will only split it into a training and test set. It will be a 80–20 split. 80 % of the images will be used for training and 20% will be used for testing. That is train on 6092 samples, test on 1524 samples from the total 7616.
For our model we trained for 15 epochs with a batch-size of 64.
The number of epochs is a hyperparameter that defines the number times that the learning algorithm will work through the entire training dataset.
One epoch means that each sample in the training dataset has had an opportunity to update the internal model parameters. An epoch is comprised of one or more batches.
You can think of a for-loop over the number of epochs where each loop proceeds over the training dataset. Within this for-loop is another nested for-loop that iterates over each batch of samples, where one batch has the specified “batch size” number of samples. [2]
That is our model will go through our entire 7616 samples 15 times (epoch) in total and adjust its weights each time so the prediction is more accurate each time. In each epoch, it will go through the 7616 samples, 64 samples (batch size) at a time.
The model keeps updating its weight so as to minimize the cost(loss), thus giving us the best accuracy. Cost is a measure of inaccuracy of the model in predicting the class of the image. Cost functions are used to estimate how badly models are performing. Put simply, a cost function is a measure of how wrong the model is in terms of its ability to estimate the relationship between X and y. [1]
If the algorithm predicts incorrectly the cost increases, if it predicts correct the cost decreases.
After training for 15 epochs we can see the following graph of loss and accuracy. (Cost and loss can be used interchangeably for our case)
The loss decreased as the model trained more times. It becomes better at classifying the images with each epoch. The model is not able to improve the performance much on the validation set.
The accuracy increased as the model trains for each epoch. It becomes better at classifying the images. The accuracy is for the validation set is lower than the training set as it has not trained on it directly. The final value is 97.64% which is not bad.
The next step would to be change the hyperparameters, the learning rate,number of epochs, data size etc. to improve our model. In machine learning, a hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters (typically node weights) are derived via training.[3]
For our purpose we have already modified these parameters before this article was written, in a way to obtain an optimum performance for display on this article. We increased the dataset size and number of epochs to improve the accuracy.
The final step after making the adjustments on the model is to make predictions using actual data that will be used on this model. If the model does not perform well on this further hyperparameter tuning can commence.
Machine Learning is a rather iterative and empirical process and thus the tuning of hyperparameters is often compared to an art rather than science as although we have an idea of what changes will happen by changing certain hyperparameters, we cannot be certain of it.
This ability to classify mechanical parts could enable us to recommend parts from a standard library based only on an image or a CAD model provided by the customer. Currently to search for a required part from a standard library you have to go through a catalogue and be able to tell which part you want based on the available options and your knowledge of the catalogue. There are serial codes to remember as a change in a single digit or alphabet might mean a different type of part.
If an image can be used to get the required part from the standard library, all we will need to do is to make a rough CAD model of it and send it through our algorithm. The algorithm will decide which parts are best and help narrow down our search significantly.
If the classification method gets detailed and fine-tuned enough it should be able to classify with much detail what type of part you want. The narrowed search saves a lot of time. This is especially useful in a library where there are thousands of similar parts.
Deep Learning (Artificial Intelligence) is a field of study that has immense possibilities as it enables us to extract a lot of knowledge from raw data. At its core it is merely data analysis. In this age of the internet, data is everywhere and if we are able to extract it efficiently, a lot can be accomplished.
This field has a lot of possible applications in the domain of Mechanical Engineering as well. Since almost all studies in Deep Learning need a domain expert it would be advisable for all engineers with interest in Data Analytics, even though they haven’t majored in Computer Sciences, to learn about data science, machine learning and examine its possibilities. The knowledge of the domain plus the skills of data analysis will really help us excel in our own fields.
I am thankful to Pro-Mech Minds for letting me do this and specially to its data science team which includes Gopal Kisi, Bishesh Shakya and Series Chikanbanjar for their immense help with this project. Pro-Mech Minds & Engineering Services is one of the companies in Nepal working with both mechanical and IT solutions in engineering. This idea popped as an attempt to combine design engineering with Data Science. Last but not the least I would like to offer my special thanks to Saugat K.C. for acting as a mentor for our Data Science team.
Link to the code in github
References
[1]Conor McDonald, Machine learning fundamentals (I): Cost functions and gradient descent(2017), Towards data science
[2]Jason Brownlee, Difference Between a Batch and an Epoch in a Neural Network(2018), machinelearningmastery.com
[3] Hyperparameter_(machine_learning), Wikipedia
[4] Andrew Ng, Convolutional Neural Networks of the Deep Learning Specialization by deeplearning.ai. (n.d.). Retrieved from Coursera