# Application of Intelligent Recognition Technology in Recognition of Mechanical Material Structure

Authors: Xiaoyan Zhang; Chensi Wang; Tao Wu; Yanfeng Wang
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

In order to solve the related problems of mechanical material structure identification, the author proposes a method that integrates intelligent identification technology into mechanical material identification. Intelligent image recognition technology is based on image features, focusing on the main features of each image, and eliminates redundant input information, finds the key information needed, and completes the sorting of image information in stages to form a complete intuitive image. The experimental results show that the proposed method performs well in most cases, reaching 90.8% for coarse classification, 76.6% for medium classification, and 73.4% for fine classification. It is proved that artificial intelligence recognition technology is of great significance for the application of mechanical material structure recognition.


---

## Body

## 1. Introduction

Artificial intelligence is a branch of computer science and technology, and it is currently the most representative technology in this field; it tries to understand the essence of intelligence and make similar intelligent machines by simulating human thinking and consciousness [1, 2]. Virtual reality technology, simulation technology, speech recognition technology, etc. are all research contents of artificial intelligence. It can also be seen from this that artificial intelligence is also a comprehensive and practical technology.First of all, the budding stage of artificial intelligence is precisely the stage of computer technology, because the computer technology at this time has just emerged, and it has only been applied in a limited field and has not yet been popularized. At this stage, the production mode of society mainly relies on manual work, and artificial intelligence technology has begun to appear, but it has not been able to play a greater role in production and life [3, 4]. Secondly, electronic information technology has developed rapidly, and the shadow of the Internet can be seen in people’s study, life, and work. People have some understanding of artificial intelligence, but the degree of cognition is not deep, and the technology has not been widely and profoundly applied in various fields. But with the improvement of people’s ideology, people have gradually realized the importance of artificial intelligence technology, which is developing rapidly. Finally, in the stage of popularization of artificial intelligence technology, although artificial intelligence technology is applied in production and life, the application of technology is not deep and proficient enough, and it has not been applied on a large scale [5].Mechanical design, manufacturing, and automation are a highly integrated discipline with mechanized engineering as its core and also integrate automation, electronic engineering, and computer disciplines; this technology has a wide range of applications in the field of production. Especially with the development of science and technology and economy, this technology has been widely used in large industrial enterprises. With the development and update of network information technology, mechanical design, manufacturing, and automation will also penetrate into people’s life field, which will facilitate people’s life and promote the development of society.The pace of science and technology is advancing rapidly, which has caused great changes in social production and people’s lives, the production efficiency is getting higher and higher, and the pace of life is gradually accelerating. The application of artificial intelligence technology in the field of mechanical design, manufacturing, and automation, such as the use of information processing, fault diagnosis, and the use of neural networks for data storage and calculation, can effectively improve its work efficiency and quality and promote its development towards more intelligent development. From another point of view, mechanical design and manufacturing and automation also provide a platform and opportunity for artificial intelligence technology, and the two promote and develop each other.


## 2. Literature Review

With the rapid economic development and the continuous improvement of scientific and technological level, the research on the development and application of computer technology has become the focus of scholars. In particular, the research on artificial intelligence recognition has become more and more in-depth, and its application has become more and more extensive. The application of artificial intelligence recognition technology not only improves the efficiency of production but also provides convenience for people’s production and life.Intelligent identification technology is based on computer systems, scanning equipment, and camera equipment to intelligently identify the data information of the target [6]. The current artificial intelligence recognition technology is gradually developed from the speech recognition technology. Now, a number of intelligent recognition methods have been formed, such as face recognition, image detection, image retrieval, target tracking, and style transfer. The emergence of these intelligent identification technologies has improved people’s quality of life, reduced people’s workload, and improved the efficiency of production and life, which is of great significance for promoting the development of modern technology.Image recognition technology is an important field and branch of computer artificial intelligence technology. Its core is to perform corresponding object recognition on images in order to distinguish between targets and objects in different modes [7]. In terms of development, image recognition technology has gone through three stages. They are text recognition, image processing and recognition, and object recognition. Through the corresponding processing and analysis of the image information, the research objectives we need are obtained. Today, image recognition is not only recognized by the naked eye but also recognized by computer which is an important recognition method. In the recognition principle, the computer image recognition technology is the same as the human customer’s naked eye recognition. Image recognition by humans is based on the characteristics of the image itself, in order to identify the image. When we see a picture, our brain will quickly respond to the picture we know and classify and recognize and store memory. Artificial intelligence image recognition technology is based on image features, pays attention to the main features of each picture, eliminates redundant input information, finds the key information needed, and completes the sorting of image information in stages to form a complete intuitive image. In the process of artificial intelligence image recognition, pattern recognition is the key. Pattern recognition is to analyze and process the confidence of different forms of things, so as to realize the description, identification, and classification of a thing or phenomenon. As shown in Figure 1, it is a complete image recognition process.Figure 1
Complete image recognition process.Every object is made of one or more materials, and one can basically get a good idea of what it is made of just by glancing at it. People can easily judge whether the table they see is made of wood, whether the computer is made of metal, and whether the carpet is made of soft fibers; this ability to identify and distinguish materials and their properties is called material perception. Human perception of materials is obtained through a multitude of sensory organs such as hearing, sight, and touch. The material perception system based on computer vision mainly uses computer technology to process and analyze the image of the target object, obtain an understanding of the types and properties of materials, and provide the basis for analysis and judgment for subsequent actions or operations. The research on material visual perception technology is of great significance; for example, when operating a target object, the robot system needs to adaptively grasp the target object according to its physical properties such as weight, surface roughness, and softness. The robot judges whether the road is smooth during the navigation process and judges whether the food and fruit are fresh during the production process. Through the material visual perception technology, the robot can perceive the surrounding environment and interact like a human and adopt different action strategies according to different materials, such as intelligently avoiding the sharp edges of blades or broken glass, but not so sensitive to the corners of clothes. Be more careful when handling fragile ceramic cups than plastic ones. Initially, some researchers used manually selected features (such as color and SIFT) to provide standard classifiers. Based on this material identification system, Jia et al. downloaded 1000 images (including 10 materials, each the material contains 100 images) to build the Flickr material database (FMD). This database became the classic database for human and computer identification materials [8]. Liu et al. used crowdsourcing technology to build a database of materials in natural environments, which consisted of 23 materials containing 3 million hand-marked area pictures. There is no doubt that with the establishment and improvement of the material recognition database, the development of material visual perception will be promoted [9]. Yao et al. believed that the inherent characteristics of illumination will affect the surface gloss estimation ability [10]. Sáez et al. have experimentally demonstrated that surface gloss perception is spatially distributed inconsistently, and it suffers from specular highlights. For semitransparent materials like jade and porcelain, information such as specular highlights, render shading, and background has a greater impact on glossiness estimation [11]. Parvini et al. emphasized the influence of geometric shape on intensity gradients, and the human visual system encodes changes in the geometric surface normal of an object into a rate of intensity change [12]. Gupta et al. simulated glassy pebbles in front of a textured background; maximum likelihood difference scaling (MLDS) is used to measure how the refractive index changes under changing conditions of the physical refractive index. They discovered a nonlinear compression function; this function has a strong correlation with the degree of background texture distortion caused by refraction [13].In the field of computer vision research, the research on visual perception of mechanical material structures is far behind object recognition, both in terms of recognition accuracy and the number of research literature. Compared with object recognition, face recognition, and scene recognition, the research on visual perception of mechanical material structure is in the initial stage, and researchers are applying conventional methods to visual perception of mechanical materials. According to the characteristics of mechanical materials, the author proposes a novel and effective method to extract the robust features of materials, which is still a very challenging topic; therefore, the author will propose the application of intelligent recognition technology in mechanical material structure recognition. Figure2 shows the image recognition technology.Figure 2
Image recognition technology.


## 3. Methods

### 3.1. Material Visual Perception System

In the computer vision system structure, the general processing flow is from image acquisition to image processing to image recognition. The process of material visual perception system is basically the same as the above; the difference is that image processing is divided into low-level image processing, intermediate image processing (estimating material properties), and high-level image processing (high-level feature space).


### 3.2. Research Status of Visual Perception of Materials

According to different research directions, the research methods of material visual perception can be divided into two categories: material identification and classification and material properties and parameter estimation. Material identification and classification is top-down, that is, using object identification methods to identify materials and infer their properties and parameters. For example, when people identify the target object as a glass, its transparency, hardness, roughness, and other information can be inferred. On the contrary, the estimation of material properties and parameters is bottom-up; that is, the material properties and parameters are obtained by intermediate image processing and then the material is identified, such as transparency, hardness, roughness, and other property information to identify whether the target is glass. These two approaches are not mutually exclusive, but mutually reinforcing.


#### 3.2.1. Mechanical Material Identification and Classification

One of the main functions of visual perception of materials is to identify and classify them. However, there are still problems in the identification of materials by the existing methods of target recognition and scene recognition. There is no one-to-one mapping between object types and material types, especially man-made products, although currently in object recognition, the recognition success rate of computer vision surpasses that of humans, but in terms of material recognition, the performance of computer vision is far inferior to that of humans. Some researchers believe that the reason why material recognition lags behind object recognition may be the lack of training databases. Table1 shows the databases for material identification created by researchers in recent years, among which CUReT and KTH-TIPS are mainly used for material property and parameter estimation.Table 1
Material identification database.
DatabaseCreatorCreation timeNumber of pictures/pieceType of materialCUReTDana19991250561KTH-TIPSHayman200481010FMDSharan2009100010MINCBell2015300000023GMDWieschollek20161000010


#### 3.2.2. Material Properties and Parameter Estimation

Invited some volunteers, experiment on a subset of FMD, neither of these volunteers was explicitly told that the pictures were from different categories, there was also no requirement to categorize the materials, but they were 90 percent correct on nine attributes that help material identification. This shows that pictures can be used for accurate material identification, and material properties are closely related to material types. Using the method of classification and identification to classify materials (wood, leather, glass, plastic, etc.) cannot reflect the rich subjective feelings of human beings to materials. Computer pattern recognition technology assigns corresponding boundaries and labels to images; this makes them ignore most of the material’s sensory characteristics, yet it is these sensory characteristics that make different materials attractive, precious, and distinctive. Human visual perception of materials is the perception of certain inherent properties and parameters of objects (reflectivity, stiffness, and translucency), which are called material properties and parameter estimation. The estimation of material properties and parameters can be roughly divided into two categories: estimation of optical properties of materials, such as surface reflectance, gloss, and transparency, which are currently studied, and mechanical properties of materials, such as viscosity and elasticity.


#### 3.2.3. Optical Property Estimation of Materials

When light hits the surface of an object, the light may be absorbed, reflected, or transmitted, and different objects exhibit different optical properties. For opaque objects, the light propagation characteristics can be described by the bidirectional reflection distribution function (BRDF) [14]. The distribution function (1) describes the distribution of the reflected light at various angles after the incident light is emitted. BRDF is defined as
(1)fl,v=dL0vdEl⇒fθi,φi,θr,φr=dL0θr,φrdEθi,φi,where l is the incident light direction, v is the viewing direction, dL0v is the differential radiance of the reflected light reflected from the surface to the v direction, and dEl is the differential irradiance on the surface from the incident light direction l. The original model of the bidirectional reflection distribution function has many variables, and an approximate analytical model containing some variables is often used to replace the original model in the actual processing process.At present, research on the optical properties of materials mainly focuses on the perception of surface reflectance. Surface reflectance perception is the process of estimating unknown parameters in the bidirectional reflectance distribution function from photographs [15]. However, it is still unclear which parameters are inferred by humans and how many parameters are used, most research works only consider those simple picture information, and some recent studies have begun to consider picture information that can better express the real world. Pictures of real objects and pictures synthesized with image software are used to identify real materials. Studies have shown that image information such as the shape of a grayscale image of light is closely related to the reflectance of diffuse and specular reflections [16].Only opaque objects have been discussed above; however, people often encounter transparent objects such as glass, water, jam, and crystal in real life [17, 18]. These transparent objects cannot be described by BRDF, because for transparent objects, light incident from point A will be emitted from other points. This property of transparent objects can be described by a bidirectional surface scattering distribution function model. Since these objects are all optically denser than air, this will result in (a) specular emission, which makes most transparent objects appear glossy [19], and (b) refraction, a feature that makes the internal visible view of a transparent object appear fragmented or distorted. When the transmitted light is transmitted into the interior of the object, part of it is dispersed and part is absorbed [20]. Scattering fills the interior of the material with light, making the background pattern invisible and causing the material to exhibit a unique milky, slightly luminous, translucent appearance, such as marble and jade [21].Most studies on transparent materials idealize the object as a transparent filter sheet, so that refraction and scattering effects can be ignored [22]. When viewing the surface through the filter, the resulting image patch is the result of a fusion of the background and filter colors [23].


## 3.1. Material Visual Perception System

In the computer vision system structure, the general processing flow is from image acquisition to image processing to image recognition. The process of material visual perception system is basically the same as the above; the difference is that image processing is divided into low-level image processing, intermediate image processing (estimating material properties), and high-level image processing (high-level feature space).


## 3.2. Research Status of Visual Perception of Materials

According to different research directions, the research methods of material visual perception can be divided into two categories: material identification and classification and material properties and parameter estimation. Material identification and classification is top-down, that is, using object identification methods to identify materials and infer their properties and parameters. For example, when people identify the target object as a glass, its transparency, hardness, roughness, and other information can be inferred. On the contrary, the estimation of material properties and parameters is bottom-up; that is, the material properties and parameters are obtained by intermediate image processing and then the material is identified, such as transparency, hardness, roughness, and other property information to identify whether the target is glass. These two approaches are not mutually exclusive, but mutually reinforcing.


### 3.2.1. Mechanical Material Identification and Classification

One of the main functions of visual perception of materials is to identify and classify them. However, there are still problems in the identification of materials by the existing methods of target recognition and scene recognition. There is no one-to-one mapping between object types and material types, especially man-made products, although currently in object recognition, the recognition success rate of computer vision surpasses that of humans, but in terms of material recognition, the performance of computer vision is far inferior to that of humans. Some researchers believe that the reason why material recognition lags behind object recognition may be the lack of training databases. Table1 shows the databases for material identification created by researchers in recent years, among which CUReT and KTH-TIPS are mainly used for material property and parameter estimation.Table 1
Material identification database.
DatabaseCreatorCreation timeNumber of pictures/pieceType of materialCUReTDana19991250561KTH-TIPSHayman200481010FMDSharan2009100010MINCBell2015300000023GMDWieschollek20161000010


### 3.2.2. Material Properties and Parameter Estimation

Invited some volunteers, experiment on a subset of FMD, neither of these volunteers was explicitly told that the pictures were from different categories, there was also no requirement to categorize the materials, but they were 90 percent correct on nine attributes that help material identification. This shows that pictures can be used for accurate material identification, and material properties are closely related to material types. Using the method of classification and identification to classify materials (wood, leather, glass, plastic, etc.) cannot reflect the rich subjective feelings of human beings to materials. Computer pattern recognition technology assigns corresponding boundaries and labels to images; this makes them ignore most of the material’s sensory characteristics, yet it is these sensory characteristics that make different materials attractive, precious, and distinctive. Human visual perception of materials is the perception of certain inherent properties and parameters of objects (reflectivity, stiffness, and translucency), which are called material properties and parameter estimation. The estimation of material properties and parameters can be roughly divided into two categories: estimation of optical properties of materials, such as surface reflectance, gloss, and transparency, which are currently studied, and mechanical properties of materials, such as viscosity and elasticity.


### 3.2.3. Optical Property Estimation of Materials

When light hits the surface of an object, the light may be absorbed, reflected, or transmitted, and different objects exhibit different optical properties. For opaque objects, the light propagation characteristics can be described by the bidirectional reflection distribution function (BRDF) [14]. The distribution function (1) describes the distribution of the reflected light at various angles after the incident light is emitted. BRDF is defined as
(1)fl,v=dL0vdEl⇒fθi,φi,θr,φr=dL0θr,φrdEθi,φi,where l is the incident light direction, v is the viewing direction, dL0v is the differential radiance of the reflected light reflected from the surface to the v direction, and dEl is the differential irradiance on the surface from the incident light direction l. The original model of the bidirectional reflection distribution function has many variables, and an approximate analytical model containing some variables is often used to replace the original model in the actual processing process.At present, research on the optical properties of materials mainly focuses on the perception of surface reflectance. Surface reflectance perception is the process of estimating unknown parameters in the bidirectional reflectance distribution function from photographs [15]. However, it is still unclear which parameters are inferred by humans and how many parameters are used, most research works only consider those simple picture information, and some recent studies have begun to consider picture information that can better express the real world. Pictures of real objects and pictures synthesized with image software are used to identify real materials. Studies have shown that image information such as the shape of a grayscale image of light is closely related to the reflectance of diffuse and specular reflections [16].Only opaque objects have been discussed above; however, people often encounter transparent objects such as glass, water, jam, and crystal in real life [17, 18]. These transparent objects cannot be described by BRDF, because for transparent objects, light incident from point A will be emitted from other points. This property of transparent objects can be described by a bidirectional surface scattering distribution function model. Since these objects are all optically denser than air, this will result in (a) specular emission, which makes most transparent objects appear glossy [19], and (b) refraction, a feature that makes the internal visible view of a transparent object appear fragmented or distorted. When the transmitted light is transmitted into the interior of the object, part of it is dispersed and part is absorbed [20]. Scattering fills the interior of the material with light, making the background pattern invisible and causing the material to exhibit a unique milky, slightly luminous, translucent appearance, such as marble and jade [21].Most studies on transparent materials idealize the object as a transparent filter sheet, so that refraction and scattering effects can be ignored [22]. When viewing the surface through the filter, the resulting image patch is the result of a fusion of the background and filter colors [23].


## 3.2.1. Mechanical Material Identification and Classification

One of the main functions of visual perception of materials is to identify and classify them. However, there are still problems in the identification of materials by the existing methods of target recognition and scene recognition. There is no one-to-one mapping between object types and material types, especially man-made products, although currently in object recognition, the recognition success rate of computer vision surpasses that of humans, but in terms of material recognition, the performance of computer vision is far inferior to that of humans. Some researchers believe that the reason why material recognition lags behind object recognition may be the lack of training databases. Table1 shows the databases for material identification created by researchers in recent years, among which CUReT and KTH-TIPS are mainly used for material property and parameter estimation.Table 1
Material identification database.
DatabaseCreatorCreation timeNumber of pictures/pieceType of materialCUReTDana19991250561KTH-TIPSHayman200481010FMDSharan2009100010MINCBell2015300000023GMDWieschollek20161000010


## 3.2.2. Material Properties and Parameter Estimation

Invited some volunteers, experiment on a subset of FMD, neither of these volunteers was explicitly told that the pictures were from different categories, there was also no requirement to categorize the materials, but they were 90 percent correct on nine attributes that help material identification. This shows that pictures can be used for accurate material identification, and material properties are closely related to material types. Using the method of classification and identification to classify materials (wood, leather, glass, plastic, etc.) cannot reflect the rich subjective feelings of human beings to materials. Computer pattern recognition technology assigns corresponding boundaries and labels to images; this makes them ignore most of the material’s sensory characteristics, yet it is these sensory characteristics that make different materials attractive, precious, and distinctive. Human visual perception of materials is the perception of certain inherent properties and parameters of objects (reflectivity, stiffness, and translucency), which are called material properties and parameter estimation. The estimation of material properties and parameters can be roughly divided into two categories: estimation of optical properties of materials, such as surface reflectance, gloss, and transparency, which are currently studied, and mechanical properties of materials, such as viscosity and elasticity.


## 3.2.3. Optical Property Estimation of Materials

When light hits the surface of an object, the light may be absorbed, reflected, or transmitted, and different objects exhibit different optical properties. For opaque objects, the light propagation characteristics can be described by the bidirectional reflection distribution function (BRDF) [14]. The distribution function (1) describes the distribution of the reflected light at various angles after the incident light is emitted. BRDF is defined as
(1)fl,v=dL0vdEl⇒fθi,φi,θr,φr=dL0θr,φrdEθi,φi,where l is the incident light direction, v is the viewing direction, dL0v is the differential radiance of the reflected light reflected from the surface to the v direction, and dEl is the differential irradiance on the surface from the incident light direction l. The original model of the bidirectional reflection distribution function has many variables, and an approximate analytical model containing some variables is often used to replace the original model in the actual processing process.At present, research on the optical properties of materials mainly focuses on the perception of surface reflectance. Surface reflectance perception is the process of estimating unknown parameters in the bidirectional reflectance distribution function from photographs [15]. However, it is still unclear which parameters are inferred by humans and how many parameters are used, most research works only consider those simple picture information, and some recent studies have begun to consider picture information that can better express the real world. Pictures of real objects and pictures synthesized with image software are used to identify real materials. Studies have shown that image information such as the shape of a grayscale image of light is closely related to the reflectance of diffuse and specular reflections [16].Only opaque objects have been discussed above; however, people often encounter transparent objects such as glass, water, jam, and crystal in real life [17, 18]. These transparent objects cannot be described by BRDF, because for transparent objects, light incident from point A will be emitted from other points. This property of transparent objects can be described by a bidirectional surface scattering distribution function model. Since these objects are all optically denser than air, this will result in (a) specular emission, which makes most transparent objects appear glossy [19], and (b) refraction, a feature that makes the internal visible view of a transparent object appear fragmented or distorted. When the transmitted light is transmitted into the interior of the object, part of it is dispersed and part is absorbed [20]. Scattering fills the interior of the material with light, making the background pattern invisible and causing the material to exhibit a unique milky, slightly luminous, translucent appearance, such as marble and jade [21].Most studies on transparent materials idealize the object as a transparent filter sheet, so that refraction and scattering effects can be ignored [22]. When viewing the surface through the filter, the resulting image patch is the result of a fusion of the background and filter colors [23].


## 4. Results and Discussion

In the experiment, there are a total of 736 training samples and 184 test samples, the classification experiments are carried out at three levels of coarse, medium, and fine; the results of a single random experiment are shown in Figure3. The proposed method performs well in most cases, reaching 90.8% for coarse classification, 76.6% for medium classification, and 73.4% for fine classification. GKSC has better performance than KNN, and in medium classification experiments, the performance of GKSC is similar to KNN [24]. However, in most cases, the performance of both KNN and GKSC degrades as the classification accuracy increases. An obvious reason is that the total number of samples is fixed, and as the degree of refinement of the classification increases, the number of categories also increases. In addition, the confusion matrix of the coarse recognition result is shown in Figure 4; it can be observed that the main part of identification errors occurred during the identification of C5 and C3, where 75% of the test samples for C5 were incorrectly identified as C1, C2, and C6. In medium classification, the confusion matrix is shown in Figure 5; it can be clearly observed that due to the improvement of the recognition precision, the recognition error rate has increased compared with the rough classification, especially in the recognition process of C8S1, C8S4, C7S2, C3S2, C3S4, C1S3, and other categories; the error rate is higher [25].Figure 3
Experimental results.Figure 4
Confusion matrix of coarse classification and recognition results.Figure 5
Confusion matrix of classification and recognition results.


## 5. Conclusion

The author proposes an intelligent recognition technology, and the technology is applied to the identification of mechanical material structure. In the process of artificial intelligence image recognition, pattern recognition is the key, and pattern recognition is to analyze and process the confidence of different forms of things, so as to realize the description, identification, and classification of a thing or phenomenon. With the continuous improvement of economic level and science and technology, the application of artificial intelligence identification technology is becoming more and more extensive, so the research on artificial intelligence identification technology is of great significance to improve the quality of people’s production and life.



---

*Source: 8909122-2022-08-05.xml*