# Automatic Annotation of Functional Semantics for 3D Product Model Based on Latent Functional Semantics

Authors: Zhoupeng Han; Hua Zhang; Weirong He; Li Ba; Qilong Yuan
Journal: Scientific Programming
 (2023)

---

## Abstract

To support effectively function-driven 3D model retrieval in the phase of mechanical product conceptual design and improve the efficiency of functional semantics annotation for 3D models, an approach for functional semantics automatic annotation for mechanical 3D product model based on latent functional semantics is presented. First, the design knowledge and function knowledge of mechanical product model are analyzed, and the ontology-based functional semantics for assembly product is constructed. Then, some concept about functional region is defined, and the 3D product model is decomposed into functional regions with different levels of granularity. The similarity of the functional region is evaluated considering multisource attribute information and geometric shape. Subsequently, the similarity based on latent functional semantics annotation model for functional regions is established, which is employed for annotating automatic latent functional semantics in the 3D product model structure. Finally, mechanical 3D models in the model library are used to verify the effectiveness and feasibility of the proposed approach.


---

## Body

## 1. Introduction

More and more 3D models have been accumulated with 3D CAD software applied widely in enterprises, which are important knowledge resources and can be employed for reference and reuse during developing new products. When 3D CAD model is reused, it is very difficult for meeting the practical needs of engineering designers by retrieving the CAD model only considering the topological and geometric information. Especially, in the process of product conceptual design, engineers pay more attention to searching for 3D CAD model structures with specific functions by making use of product function requirements’ information. The existing 3D model structures embodying plenty of design/assembly intent can achieve specific function requirements. However, there is no explicit and clear correlation mapping relationship between structure and function. The functional semantics tagging correspondingly to the model structure is becoming a hot issue in the 3D model reuse field.In the process of design and development for mechanical products utilizing computer-aided design systems, designers mostly design the structure/geometry of 3D product model in terms of the product’s functional requirements, parameters, and performance. The functional requirements’ information and the 3D model structure are independent. In this case, there is a gap between functions and structures for the 3D product CAD model [1]. It means that there is no explicit and clear structure-function correlation for an isolated 3D CAD model, lacking of functional information that the structure can accomplish. It will have an impact on the level of 3D model reuse.Currently, the research on the 3D CAD model retrieval is mainly grouped into two classification: geometry-based [2, 3] retrieval and topology-based [4, 5] retrieval. But these query information is uncertain and vague; at the stage of product conceptual design, it is difficult to support the reuse of 3D CAD model knowledge. With the emergence of semantic modeling tools, some researchers are concerned for visualization and semantic expression for resolving the semantics gap between the high-level design semantics and the bottom-level geometric shape of the 3D model. To simplify the process of semantic-based 3D model retrieval, Attene et al. [6] presented a semantic description of the local shape of the 3D model, where the 3D model is segmented into regions and the segmented region is labeled with marking semantics by using ontology-based concept instance semantics. Cheng et al. [7] proposed a complex surface shape design method for the CAD model based on semantic features, where the different geometric constraints and geometric surface types are defined as corresponding semantic features, and the semantic features are used for modeling complex surfaces of the CAD model. To capture the design intent of the 3D model, Abdul-Ghafour et al. [8] integrated semantic web into the CAD system and built a common design feature ontology for feature modeling of the CAD model, where the semantics is used for modeling and reasoning, and the association relationship between the product data can be effectively managed and discovered.The existing semantics annotation methods for 3D models mainly considered the geometric shape and topological structure information. On the basis of semantic labeled sample 3D models, machine learning algorithm is employed for automatic semantics annotation [9]. However, there are fewer researchers on functional semantics annotation of 3D models. In fact, functional semantics as a special class of semantics is not only an abstraction of geometric shapes, but also a high-level semantic description for potential information such as design or assembly intent, usage, and efficacy of the 3D model. The functional semantics annotation of the 3D model is much more difficult and complex than the shape or structure semantics annotation; it still faces many challenges for functional semantics-based retrieval of the 3D model. To meet the requirements of the CAD model retrieval at the stage of mechanical product conceptual design, Wang et al. [10] constructed function semantic ontology of 3D CAD models, where the overall functional semantics of 3D models are annotated, and the functional semantics-based retrieval of 3D models is realized. It is concerned about functional semantics annotation of the 3D part model, lacking functional analysis and annotation of the 3D assembly model at different granularity structure. Han et al. [11] proposed an approach for structure-function semantic correlation analysis and annotation of the 3D assembly model, where the 3D assembly model is divided into different functional regions for structure-function semantic correlation analysis, and the polychromatic sets is used to describe correlations of structure-function semantics. This method is concerned with the structure-function analysis of the 3D parts/assembly model, and functional semantics is labeled by the human-computer interaction mode so that the annotation efficiency is not high. Hao et al. [12] took key parts in the assembly model as the object and evaluated the similarity of the participated shape structure key parts. The functional semantics is annotated based on the shape and structure similarity, without consideration attribute information of the 3D model structure. The accuracy of the functional semantic annotation is affected to some extent. The function is not only related to the geometric shape of the 3D model, but also related to the geometric parameters, type of kinematic pair, and so on. Therefore, a reasonable and feasible solution for structure-function semantic correlation analysis and automatic annotation of the 3D product model can be provided by comprehensively considering geometric structure and attribute information of the 3D model.Functions can reflect the specific requirement, objective, and purpose that the product needs to achieve, which exists throughout the life of the product design. In the stage of the product conceptual design, it is usually on the premise that the product function requirements have been determined to find correspondingly the structural design scheme, that is, the product function-structure mapping. Function-structure mapping is the key process of the top-down product design, which is an approach of transforming the product function model to parametric structure model [13]. The function is the abstraction of the product structure and the purpose of structure realization, and the structure is the shape and composition structure that is shown to achieve certain functions. The mapping relation of function-structure is a many-to-many relationship, where a certain function may be completed by one structure or multiple structures, and a structure may also implement one or more functions. The current more typical design process model about function-structure mapping relation include function-structure (Function-Structure, FS) [14] and function-behavior-structure [15]. To realize the relationship between the function and structure, a behavioral layer is introduced to describe the action and principle which are performed to complete the function, and explain “how the structure realizes the function.” Based on that, the models including function-behavior-state model [16] is proposed. The issue about function-structure mapping can be effectively solved by using the abovementioned approaches in the product conceptual design.To improve the efficiency and accuracy of the structure-function semantic annotation for the 3D CAD product model, and support 3D CAD model retrieval by means of functional semantics, an approach for structure-function semantic automatic annotation of the 3D product CAD model is presented. The structure of the 3D product model is decomposed into functional regions (i.e. assembly region and flow-activity region) with different levels of granularity. Multisource attributes information and structural similarity are considered for evaluating the similarity between any two functional regions. Based on that, an approach for automatic annotation of latent functional semantics for 3D model structure based on semi-supervised learning is built for improving the accuracy and efficiency of functional semantics annotation of the 3D model structure.This rest of the paper is organized as follows: ontology-based functional semantics is constructed for providing standard and formal terms for functional semantic annotation in the 3D product model is discussed in Section2. In Section 3, similarity evaluation of the 3D product model structure is given by taking advantage of multisource information. A latent functional semantic annotation model of functional regions is presented in Section 4. Cases are designed and tested in Section 5. In Section 6, the paper is concluded and further works are provided.


## 2. Function Ontology of the 3D Product Model

### 2.1. Product Function Classification

Design knowledge is mainly related to the product conceptual and detailed design phase, which contains design document, domain expert experience, and so on. The design knowledge can help capture design and assemble intent of the 3D CAD model, which can provide basis and support for design thinking analysis for the correlation between the structure and function of the 3D model. It can be obtained from design documents, PDM, BoM, machining, assembly process documents, and so on.The function is viewed as an abstract description of the task that a system or product can accomplish, and the parameters or state changing when the system or subsystem is input/output. It is an understanding for the system from the perspective of technical realization, as explanatory description and behavior abstraction of the structure, which is the main goal and core of the product design. Product function is generally determined in the stage of the product conceptual design and has an essential role in the life of the product design.The concept of function basis, classification of function, and flow is proposed [17]. On this basis, the function is described as “verb + object,” where the function is expressed through the verb and noun form. In this paper, the basic classifications of the function are further summarized and refined for covering the function’s description of the mechanical product as far as possible. Based on that, function ontology is constructed for providing functional semantic specification and expression of the 3D product model structure in the mechanical field. The functional classification and common terms of the mechanical product are shown in Table 1.Table 1
Functional classification and common terms of the mechanical product.
Functional classificationFunctional basisVerb vocabularyBranchSeparateIsolate, disconnect, cut, and detachRemoveCut, lathe, mill, drill, grind, polish, bore, and removeDivideDisassembly, isolate, release, classify, group, and divideRefineExtract, filter, purify, permeate, wash, clean, and refineDistributeDisperse, spread, dissipate, disperse, and distributeChannelImportEntry, accept, permit, capture, and importExportDischarge, eject, jet, release, destroy, eliminate, and exportTransferConvey, handle, carry, move, lift, conduct, communicate, and transmitGuideGuide, drive, switch, move, transfer, migrate, rotate, turn, flip, limit, release, and orientedConnectCoupleConnect, assemble, install, tie, and coupleMixMerge, merge, package, mix, add, and mixControlActuateBegin, start, launch, initiate, and actuateRegulateControl, balance, limit, block, interrupt, delay, close, forbid, and allowChangeAdapt, correct, reverse, adjust, modify, increase, enlarge, magnify, enhance, enlarge, expand, strengthen, reduce, weaken, shrink, reduce, compress, transform, construct, form, and changeStopInhibit, protect, seal, insulate, isolate, shield, end, close, terminate, stop, interrupt, and preventConvertConvertConversion, transform, concentrate, melt, liquefy, solidify, evaporate, fusion, integrate, and processSupplyStoreStore, contain, include, encapsulate, enclose, accumulate, gather, collect, reserve, occupy, and retainSupplySupply, furnish, fill, supplement, and exposeSignalSenseSense, identify, distinguish, and confirmDisplayDisplay, show, indicate, register, record, expose, choose, and selectMeasureMeasure, calculate, process, estimate, check, proofread, examine, and compareSupportStabilizeStabilize, firm, and propSecureSecure, place, arrange, press, clamp, and tightenPositionPosition, orient, limit, and guide


### 2.2. Function Ontology

Ontology, as explicit and formal specification of conceptualization, can offer a formal description and defining common terms, classification, and relationship for conceptualization, which can be employed to represent, storage, share, and integrate knowledge [18]. To reduce and avoid the randomness and ambiguity of functional description and ensure standardization of the definition of functional vocabulary, a function ontology should be constructed to standardize the functional vocabulary of the product, so that functional terms of the 3D product model structure have a unified and standard expression form. According to the classification of the function and flow of mechanical products, the function ontology was constructed by Protégé through the analysis of function and flow concepts and their logical relationships. Parts of function ontology are shown in Figure 1, where the solid blue line represents parent-child relationship of concepts and the dashed red line represents object attribute relationship between concepts [11].The function ontology provides standard terms of functional semantic annotation of the 3D product model.Figure 1
Parts of function ontology for the CAD product model.


## 2.1. Product Function Classification

Design knowledge is mainly related to the product conceptual and detailed design phase, which contains design document, domain expert experience, and so on. The design knowledge can help capture design and assemble intent of the 3D CAD model, which can provide basis and support for design thinking analysis for the correlation between the structure and function of the 3D model. It can be obtained from design documents, PDM, BoM, machining, assembly process documents, and so on.The function is viewed as an abstract description of the task that a system or product can accomplish, and the parameters or state changing when the system or subsystem is input/output. It is an understanding for the system from the perspective of technical realization, as explanatory description and behavior abstraction of the structure, which is the main goal and core of the product design. Product function is generally determined in the stage of the product conceptual design and has an essential role in the life of the product design.The concept of function basis, classification of function, and flow is proposed [17]. On this basis, the function is described as “verb + object,” where the function is expressed through the verb and noun form. In this paper, the basic classifications of the function are further summarized and refined for covering the function’s description of the mechanical product as far as possible. Based on that, function ontology is constructed for providing functional semantic specification and expression of the 3D product model structure in the mechanical field. The functional classification and common terms of the mechanical product are shown in Table 1.Table 1
Functional classification and common terms of the mechanical product.
Functional classificationFunctional basisVerb vocabularyBranchSeparateIsolate, disconnect, cut, and detachRemoveCut, lathe, mill, drill, grind, polish, bore, and removeDivideDisassembly, isolate, release, classify, group, and divideRefineExtract, filter, purify, permeate, wash, clean, and refineDistributeDisperse, spread, dissipate, disperse, and distributeChannelImportEntry, accept, permit, capture, and importExportDischarge, eject, jet, release, destroy, eliminate, and exportTransferConvey, handle, carry, move, lift, conduct, communicate, and transmitGuideGuide, drive, switch, move, transfer, migrate, rotate, turn, flip, limit, release, and orientedConnectCoupleConnect, assemble, install, tie, and coupleMixMerge, merge, package, mix, add, and mixControlActuateBegin, start, launch, initiate, and actuateRegulateControl, balance, limit, block, interrupt, delay, close, forbid, and allowChangeAdapt, correct, reverse, adjust, modify, increase, enlarge, magnify, enhance, enlarge, expand, strengthen, reduce, weaken, shrink, reduce, compress, transform, construct, form, and changeStopInhibit, protect, seal, insulate, isolate, shield, end, close, terminate, stop, interrupt, and preventConvertConvertConversion, transform, concentrate, melt, liquefy, solidify, evaporate, fusion, integrate, and processSupplyStoreStore, contain, include, encapsulate, enclose, accumulate, gather, collect, reserve, occupy, and retainSupplySupply, furnish, fill, supplement, and exposeSignalSenseSense, identify, distinguish, and confirmDisplayDisplay, show, indicate, register, record, expose, choose, and selectMeasureMeasure, calculate, process, estimate, check, proofread, examine, and compareSupportStabilizeStabilize, firm, and propSecureSecure, place, arrange, press, clamp, and tightenPositionPosition, orient, limit, and guide


## 2.2. Function Ontology

Ontology, as explicit and formal specification of conceptualization, can offer a formal description and defining common terms, classification, and relationship for conceptualization, which can be employed to represent, storage, share, and integrate knowledge [18]. To reduce and avoid the randomness and ambiguity of functional description and ensure standardization of the definition of functional vocabulary, a function ontology should be constructed to standardize the functional vocabulary of the product, so that functional terms of the 3D product model structure have a unified and standard expression form. According to the classification of the function and flow of mechanical products, the function ontology was constructed by Protégé through the analysis of function and flow concepts and their logical relationships. Parts of function ontology are shown in Figure 1, where the solid blue line represents parent-child relationship of concepts and the dashed red line represents object attribute relationship between concepts [11].The function ontology provides standard terms of functional semantic annotation of the 3D product model.Figure 1
Parts of function ontology for the CAD product model.


## 3. Similarity Evaluation of the 3D Product Model Structure Considering Multisource Information

### 3.1. Functional Region

To analyze convenient assembly constraints of the product model, assembled parts are grouped into two categories: functional part and connector [19]. The connected relationships between assembled parts are classified into soft connection and hard connection in terms of corresponding connection relationship [20]. Connectors contain a type of part used for sealing or fastening, such as gaskets, screws, bolts, nuts, pins, keys, and other standard parts. The functional part is viewed as some function characteristics apart from connectors.Functional Region: For a 3D product model, the function mainly reflects in the structure at different granularity level. The structure unit with certain function characteristics is considered as a functional region. Functional region may be a subassembly structure composed of multiple parts or shape structure of the part. To annotate easily functional semantics’ indifferent granularity structure for the 3D assembly model, functional regions are grouped into assembly region and flow-activity region. The functional structure of the 3D product model can be considered comprehensively by the assembly region and flow-activity region from the aspects of assembly structure and part shape. The correlation relationships between the function and structure is analyzed comprehensively from the coarse and fine-granularity structure of the 3D product model.


### 3.2. Functional Region Similarity Considering Multisource Information

The similarity for functional regions of the 3D product model is evaluated by considering multiple attributes and geometric shapes.


#### 3.2.1. Multisource Information Similarity of the Assembled Part

Multisource information is mainly grouped into qualitative attribute, quantitative attribute, text attribute [21], and geometric shape. The similarity calculation rule and method of each classification information of the assembled part are given as follows [22]:(1)
Quantitative attribute: The attributes are quantitative including size, surface area, volume, and so on. It can be expressed by a numeric value. The similarity of the quantitative attribute can be expressed as follows:(1)sim_attributeqki,pkj=1−qki−pkjmaxqki,pkj,whereqki denotes the kth quantitative attribute of part i for the product model Q and pki denotes the kth quantitative attribute of part j for the product model P.(2)
Qualitative attribute: The attributes are not quantified, such as face type of the part, design/assembled feature type, and so on. The similarity of the qualitative attribute is expressed as follows:(2)sim_attributeqki,pkj=1,qki=pkj,0,else,whereqki denotes the kth qualitative attribute of part i for the product model Q and pki denotes the kth qualitative attribute of part j for the product model P.(3)
Text attribute: The attributes are expressed mostly by string/text set for assembled constraints [23]. The set theory is employed to the similarity of the text attribute, which is expressed as follows:(3)sim_attributeqki,pkj=qki∩pkjqki∪pkj,whereqki denotes the kth text attribute of part i for the product model Q and pki denotes the kth text attribute of part j for the product model P.(4)
Geometric shape: The geometric shape of assembled parts is taken into account during the similarity of the functional region. The similarity calculation about the geometric shape is mainly referred to the literature [24]. Here, it can be expressed as sim_shapeqki,pkj.(5)
Multisource information similarity of parts: Based on similarity evaluation of assembled parts considering multisource information, the weight coefficientα is introduced for describing the importance degree of each factor in terms of requirements of engineering designer. The similarity of parts considering multisource attributes sim_partqi,pj can be expressed as follows:(4)sim_partqi,pj=ω1•∑t=1Tsim_attributeqki,pkj•αk+ω2•sim_shapeqki,pkj,∑t=1Tαt=1,wheresim_attributeqi,pj denotes the similarity of part i for the assembly region Q and part j for the assembly region P considering multisource attributes, T denotes the total number of attributes, αt denotes the weight of the tth attribute, w1 denotes the weight of the similarity of attribute information, and w2 denotes the weight of the similarity of the geometric shape.


#### 3.2.2. Similarity Measure of the Functional Region

The similarity of the functional region mainly contains the similarity of the assembly region and the flow- activity region.(1)
Similarity measure of the assembly region: Similarity of assembly region should not only consider the shape structure and attribute information of assembled parts, but also take into account the information of assembly relation, such as connection form, contact form, degree of freedom, assembly feature pair and so on. By using the above similarity evaluation through multisource attribute, the similarity for assembly relationship could be expressed as follows:(5)sim_relationshipqi,pj=∑h=1Hsim_attributeqki,pkj•βh,∑h=1Hβh=1,wheresim_relationshipqi,pj denotes the similarity for assembly relationships between assembled parts i and j for the product models Q and P, H denotes the number of attributes for assembly relationships, and βh denotes the weight of the hth attribute.Considering the attributes about the assembly relationship and assembled part comprehensively, the similarity of the assembly region can be expressed as follows:(6)sim_assemqi,pj=sim_partqi,pj•w1+sim_relationshipqi,pj•w2,wherew1 denotes the weight of similarity of the assembled part and w2 denotes the corresponding assembly relationship similarity.The assembly region may consist of different assembled parts. It is necessary that the assembled parts should be considered as matching problem between regions while calculating the similarity of assembly region. In fact, it has found the best assembled parts pair among two part-sets from corresponding functional regions when the total sum value of the similarity between functional region pairs is maximal. The matching problem can be solved by using maximum matching in the weighted bipartite graph. The Kuhn–Munkres algorithm [25] adopted the similarity simass(Q, P) between the assembly regions Q and P can be expressed as follows:(7)simassQ,P=∑lqi,pj∈L0sim_assemqi,pjmaxm,n,whereM denotes the maximal matching, L0 denotes the set of corresponding edges, lqii,pjj∈L0 denotes the best matching between assembled parts qi and pj, and m, n denote respectively the number of assembled parts in assembly regions Q and P.(2)
Similarity measure of the flow-activity region: Flow-activity area mainly refers to the shape structure of the part that does not participate in the assembly region. In fact, it specifically refers to the similarity evaluation of the part model. Similarity of the flow-activity regionq′ and the flow-activity region p′, simpart(q′, p′) can be expressed as follows:(8)simpartq',p'=sim_partq',p',during the similarity evaluation, the weight values can be determined by domain expert and experience generally. Moreover, to avoid subjective interference, the weight values can be determined by using AHP and TOPSIS [26].


## 3.1. Functional Region

To analyze convenient assembly constraints of the product model, assembled parts are grouped into two categories: functional part and connector [19]. The connected relationships between assembled parts are classified into soft connection and hard connection in terms of corresponding connection relationship [20]. Connectors contain a type of part used for sealing or fastening, such as gaskets, screws, bolts, nuts, pins, keys, and other standard parts. The functional part is viewed as some function characteristics apart from connectors.Functional Region: For a 3D product model, the function mainly reflects in the structure at different granularity level. The structure unit with certain function characteristics is considered as a functional region. Functional region may be a subassembly structure composed of multiple parts or shape structure of the part. To annotate easily functional semantics’ indifferent granularity structure for the 3D assembly model, functional regions are grouped into assembly region and flow-activity region. The functional structure of the 3D product model can be considered comprehensively by the assembly region and flow-activity region from the aspects of assembly structure and part shape. The correlation relationships between the function and structure is analyzed comprehensively from the coarse and fine-granularity structure of the 3D product model.


## 3.2. Functional Region Similarity Considering Multisource Information

The similarity for functional regions of the 3D product model is evaluated by considering multiple attributes and geometric shapes.


### 3.2.1. Multisource Information Similarity of the Assembled Part

Multisource information is mainly grouped into qualitative attribute, quantitative attribute, text attribute [21], and geometric shape. The similarity calculation rule and method of each classification information of the assembled part are given as follows [22]:(1)
Quantitative attribute: The attributes are quantitative including size, surface area, volume, and so on. It can be expressed by a numeric value. The similarity of the quantitative attribute can be expressed as follows:(1)sim_attributeqki,pkj=1−qki−pkjmaxqki,pkj,whereqki denotes the kth quantitative attribute of part i for the product model Q and pki denotes the kth quantitative attribute of part j for the product model P.(2)
Qualitative attribute: The attributes are not quantified, such as face type of the part, design/assembled feature type, and so on. The similarity of the qualitative attribute is expressed as follows:(2)sim_attributeqki,pkj=1,qki=pkj,0,else,whereqki denotes the kth qualitative attribute of part i for the product model Q and pki denotes the kth qualitative attribute of part j for the product model P.(3)
Text attribute: The attributes are expressed mostly by string/text set for assembled constraints [23]. The set theory is employed to the similarity of the text attribute, which is expressed as follows:(3)sim_attributeqki,pkj=qki∩pkjqki∪pkj,whereqki denotes the kth text attribute of part i for the product model Q and pki denotes the kth text attribute of part j for the product model P.(4)
Geometric shape: The geometric shape of assembled parts is taken into account during the similarity of the functional region. The similarity calculation about the geometric shape is mainly referred to the literature [24]. Here, it can be expressed as sim_shapeqki,pkj.(5)
Multisource information similarity of parts: Based on similarity evaluation of assembled parts considering multisource information, the weight coefficientα is introduced for describing the importance degree of each factor in terms of requirements of engineering designer. The similarity of parts considering multisource attributes sim_partqi,pj can be expressed as follows:(4)sim_partqi,pj=ω1•∑t=1Tsim_attributeqki,pkj•αk+ω2•sim_shapeqki,pkj,∑t=1Tαt=1,wheresim_attributeqi,pj denotes the similarity of part i for the assembly region Q and part j for the assembly region P considering multisource attributes, T denotes the total number of attributes, αt denotes the weight of the tth attribute, w1 denotes the weight of the similarity of attribute information, and w2 denotes the weight of the similarity of the geometric shape.


### 3.2.2. Similarity Measure of the Functional Region

The similarity of the functional region mainly contains the similarity of the assembly region and the flow- activity region.(1)
Similarity measure of the assembly region: Similarity of assembly region should not only consider the shape structure and attribute information of assembled parts, but also take into account the information of assembly relation, such as connection form, contact form, degree of freedom, assembly feature pair and so on. By using the above similarity evaluation through multisource attribute, the similarity for assembly relationship could be expressed as follows:(5)sim_relationshipqi,pj=∑h=1Hsim_attributeqki,pkj•βh,∑h=1Hβh=1,wheresim_relationshipqi,pj denotes the similarity for assembly relationships between assembled parts i and j for the product models Q and P, H denotes the number of attributes for assembly relationships, and βh denotes the weight of the hth attribute.Considering the attributes about the assembly relationship and assembled part comprehensively, the similarity of the assembly region can be expressed as follows:(6)sim_assemqi,pj=sim_partqi,pj•w1+sim_relationshipqi,pj•w2,wherew1 denotes the weight of similarity of the assembled part and w2 denotes the corresponding assembly relationship similarity.The assembly region may consist of different assembled parts. It is necessary that the assembled parts should be considered as matching problem between regions while calculating the similarity of assembly region. In fact, it has found the best assembled parts pair among two part-sets from corresponding functional regions when the total sum value of the similarity between functional region pairs is maximal. The matching problem can be solved by using maximum matching in the weighted bipartite graph. The Kuhn–Munkres algorithm [25] adopted the similarity simass(Q, P) between the assembly regions Q and P can be expressed as follows:(7)simassQ,P=∑lqi,pj∈L0sim_assemqi,pjmaxm,n,whereM denotes the maximal matching, L0 denotes the set of corresponding edges, lqii,pjj∈L0 denotes the best matching between assembled parts qi and pj, and m, n denote respectively the number of assembled parts in assembly regions Q and P.(2)
Similarity measure of the flow-activity region: Flow-activity area mainly refers to the shape structure of the part that does not participate in the assembly region. In fact, it specifically refers to the similarity evaluation of the part model. Similarity of the flow-activity regionq′ and the flow-activity region p′, simpart(q′, p′) can be expressed as follows:(8)simpartq',p'=sim_partq',p',during the similarity evaluation, the weight values can be determined by domain expert and experience generally. Moreover, to avoid subjective interference, the weight values can be determined by using AHP and TOPSIS [26].


## 3.2.1. Multisource Information Similarity of the Assembled Part

Multisource information is mainly grouped into qualitative attribute, quantitative attribute, text attribute [21], and geometric shape. The similarity calculation rule and method of each classification information of the assembled part are given as follows [22]:(1)
Quantitative attribute: The attributes are quantitative including size, surface area, volume, and so on. It can be expressed by a numeric value. The similarity of the quantitative attribute can be expressed as follows:(1)sim_attributeqki,pkj=1−qki−pkjmaxqki,pkj,whereqki denotes the kth quantitative attribute of part i for the product model Q and pki denotes the kth quantitative attribute of part j for the product model P.(2)
Qualitative attribute: The attributes are not quantified, such as face type of the part, design/assembled feature type, and so on. The similarity of the qualitative attribute is expressed as follows:(2)sim_attributeqki,pkj=1,qki=pkj,0,else,whereqki denotes the kth qualitative attribute of part i for the product model Q and pki denotes the kth qualitative attribute of part j for the product model P.(3)
Text attribute: The attributes are expressed mostly by string/text set for assembled constraints [23]. The set theory is employed to the similarity of the text attribute, which is expressed as follows:(3)sim_attributeqki,pkj=qki∩pkjqki∪pkj,whereqki denotes the kth text attribute of part i for the product model Q and pki denotes the kth text attribute of part j for the product model P.(4)
Geometric shape: The geometric shape of assembled parts is taken into account during the similarity of the functional region. The similarity calculation about the geometric shape is mainly referred to the literature [24]. Here, it can be expressed as sim_shapeqki,pkj.(5)
Multisource information similarity of parts: Based on similarity evaluation of assembled parts considering multisource information, the weight coefficientα is introduced for describing the importance degree of each factor in terms of requirements of engineering designer. The similarity of parts considering multisource attributes sim_partqi,pj can be expressed as follows:(4)sim_partqi,pj=ω1•∑t=1Tsim_attributeqki,pkj•αk+ω2•sim_shapeqki,pkj,∑t=1Tαt=1,wheresim_attributeqi,pj denotes the similarity of part i for the assembly region Q and part j for the assembly region P considering multisource attributes, T denotes the total number of attributes, αt denotes the weight of the tth attribute, w1 denotes the weight of the similarity of attribute information, and w2 denotes the weight of the similarity of the geometric shape.


## 3.2.2. Similarity Measure of the Functional Region

The similarity of the functional region mainly contains the similarity of the assembly region and the flow- activity region.(1)
Similarity measure of the assembly region: Similarity of assembly region should not only consider the shape structure and attribute information of assembled parts, but also take into account the information of assembly relation, such as connection form, contact form, degree of freedom, assembly feature pair and so on. By using the above similarity evaluation through multisource attribute, the similarity for assembly relationship could be expressed as follows:(5)sim_relationshipqi,pj=∑h=1Hsim_attributeqki,pkj•βh,∑h=1Hβh=1,wheresim_relationshipqi,pj denotes the similarity for assembly relationships between assembled parts i and j for the product models Q and P, H denotes the number of attributes for assembly relationships, and βh denotes the weight of the hth attribute.Considering the attributes about the assembly relationship and assembled part comprehensively, the similarity of the assembly region can be expressed as follows:(6)sim_assemqi,pj=sim_partqi,pj•w1+sim_relationshipqi,pj•w2,wherew1 denotes the weight of similarity of the assembled part and w2 denotes the corresponding assembly relationship similarity.The assembly region may consist of different assembled parts. It is necessary that the assembled parts should be considered as matching problem between regions while calculating the similarity of assembly region. In fact, it has found the best assembled parts pair among two part-sets from corresponding functional regions when the total sum value of the similarity between functional region pairs is maximal. The matching problem can be solved by using maximum matching in the weighted bipartite graph. The Kuhn–Munkres algorithm [25] adopted the similarity simass(Q, P) between the assembly regions Q and P can be expressed as follows:(7)simassQ,P=∑lqi,pj∈L0sim_assemqi,pjmaxm,n,whereM denotes the maximal matching, L0 denotes the set of corresponding edges, lqii,pjj∈L0 denotes the best matching between assembled parts qi and pj, and m, n denote respectively the number of assembled parts in assembly regions Q and P.(2)
Similarity measure of the flow-activity region: Flow-activity area mainly refers to the shape structure of the part that does not participate in the assembly region. In fact, it specifically refers to the similarity evaluation of the part model. Similarity of the flow-activity regionq′ and the flow-activity region p′, simpart(q′, p′) can be expressed as follows:(8)simpartq',p'=sim_partq',p',during the similarity evaluation, the weight values can be determined by domain expert and experience generally. Moreover, to avoid subjective interference, the weight values can be determined by using AHP and TOPSIS [26].


## 4. Latent Function Semantic Annotation of the Functional Region

Functional region as a structure with certain functions in the 3D product model, functional semantic annotation of the functional region is viewed as essentially functional semantic annotation of the model structure. Through analysis of the function region, the function that the corresponding structure of the function region can achieve will be obtained. Based on that, the corresponding latent functional semantics can be annotated. The process of latent functional semantic annotation of the functional region consists of two steps: ① functional semantic annotation of sample CAD models, that is, the structure-function is analyzed for 3D CAD product sample models, and corresponding functional semantics is annotated interactively. ② Latent functional semantic annotation based on the similarity. A latent functional semantic prediction model is built for automatically annotating latent functional semantics in the 3D assembly model structure. The process of latent functional semantic annotation of the functional region is shown in Figure2.Figure 2
The process of latent functional semantic annotation of the functional region.


### 4.1. Functional Semantic Annotation of the Sample Model

For mechanical product, functions are mainly reflected in assembly regions and flow-activity regions. Through analyzing the structure-function of the product model from the two aspects of the assembly region and the flow-activity region.(1)
Functional semantic interactive annotation of the assembly region: The assembly region can be expressed by assembled parts, assembly feature, and corresponding attribute information involved in the assembled structure. By analyzing the part attributes, connection types, matching types, degrees of freedom, and other information in the assembly region, together with the mechanical design theory and design experience, designers can capture the assembly intent, function, and degree of freedom, which is contained in the features of the assembly region. According to the function ontology, functional semantics of the assembled structure are labeled, thereby absorbing and understanding the assembly structure knowledge.(2)
Functional semantics interactive annotation of Flow-activity region: Flow-activity region is a description of the geometric shape with some functions in the functional parts, except for participating fit or assembled structure. The structure represented by a flow-activity region is considered as a carrier in the process of function flow from input to output. Therefore, when analyzing the flow-activity region, it is needful to consider the input and output of flow during the working process of the assembled structure. The key integral/local structures and corresponding functions in the assembled function part represented by the flow-activity region is acquired. The flow-activity region can be described by the design feature, attribute, and other information of the part, for example, the shape feature, geometric shape surface, and other attribute descriptions of the part model.


### 4.2. Latent Function Semantic Annotation of the Function Region

There are many-to-many mapping relationships between structures and functions, that is, the same structure will perform different functions in different products, and different structures will also achieve the same function. In most cases, structures with the same attributes may achieve the same functions for different assembly models. It is a probability event that the same structure is provided with the same function. As the similarity of the structure increases, the probability having the same function also increases. Based on that, a prediction model for latent functional semantic annotation of the 3D CAD product model is given using sample models annotated functional semantics. Drawing on the historical information of the structure-function semantic annotation, the probability that the product model structure can achieve certain functions is inferred. Finally, combining with the participation of artificial knowledge, the functional semantics that can be realized is determined. It enhances the efficiency of functional semantics annotating to some extent.Assuming that the annotated functional semantic model library isM, the unannotated structure (functional region) is Stru. The structure in the model library M that has the maximum similarity with the structure Stru is Strumax, and the latent functional semantic annotation model based on the similarity can be expressed as follows:(9)pfi|Stru=SimStru,Strumax∗countfi|StrumaxsumStrumax,where p(fi|Stru) denotes the probability that the structure Stru can achieve the function fi. Stru represents the unannotated structure (functional region). Strumax represents the structure (functional region) in the model library M that has the greatest similarity with the structure to the unannotated structure Stru. Sim(Stru, Strumax) represents the similarity between Strumax and Stru, equations (7) and (8) are used for similarity measurement of functional regions. Sum(Strumax) denotes the number of the structure Strumax in the model library M. Count(fi|Strumax) denotes the number of the structure Strumax that has the function fi in the model library M.


## 4.1. Functional Semantic Annotation of the Sample Model

For mechanical product, functions are mainly reflected in assembly regions and flow-activity regions. Through analyzing the structure-function of the product model from the two aspects of the assembly region and the flow-activity region.(1)
Functional semantic interactive annotation of the assembly region: The assembly region can be expressed by assembled parts, assembly feature, and corresponding attribute information involved in the assembled structure. By analyzing the part attributes, connection types, matching types, degrees of freedom, and other information in the assembly region, together with the mechanical design theory and design experience, designers can capture the assembly intent, function, and degree of freedom, which is contained in the features of the assembly region. According to the function ontology, functional semantics of the assembled structure are labeled, thereby absorbing and understanding the assembly structure knowledge.(2)
Functional semantics interactive annotation of Flow-activity region: Flow-activity region is a description of the geometric shape with some functions in the functional parts, except for participating fit or assembled structure. The structure represented by a flow-activity region is considered as a carrier in the process of function flow from input to output. Therefore, when analyzing the flow-activity region, it is needful to consider the input and output of flow during the working process of the assembled structure. The key integral/local structures and corresponding functions in the assembled function part represented by the flow-activity region is acquired. The flow-activity region can be described by the design feature, attribute, and other information of the part, for example, the shape feature, geometric shape surface, and other attribute descriptions of the part model.


## 4.2. Latent Function Semantic Annotation of the Function Region

There are many-to-many mapping relationships between structures and functions, that is, the same structure will perform different functions in different products, and different structures will also achieve the same function. In most cases, structures with the same attributes may achieve the same functions for different assembly models. It is a probability event that the same structure is provided with the same function. As the similarity of the structure increases, the probability having the same function also increases. Based on that, a prediction model for latent functional semantic annotation of the 3D CAD product model is given using sample models annotated functional semantics. Drawing on the historical information of the structure-function semantic annotation, the probability that the product model structure can achieve certain functions is inferred. Finally, combining with the participation of artificial knowledge, the functional semantics that can be realized is determined. It enhances the efficiency of functional semantics annotating to some extent.Assuming that the annotated functional semantic model library isM, the unannotated structure (functional region) is Stru. The structure in the model library M that has the maximum similarity with the structure Stru is Strumax, and the latent functional semantic annotation model based on the similarity can be expressed as follows:(9)pfi|Stru=SimStru,Strumax∗countfi|StrumaxsumStrumax,where p(fi|Stru) denotes the probability that the structure Stru can achieve the function fi. Stru represents the unannotated structure (functional region). Strumax represents the structure (functional region) in the model library M that has the greatest similarity with the structure to the unannotated structure Stru. Sim(Stru, Strumax) represents the similarity between Strumax and Stru, equations (7) and (8) are used for similarity measurement of functional regions. Sum(Strumax) denotes the number of the structure Strumax in the model library M. Count(fi|Strumax) denotes the number of the structure Strumax that has the function fi in the model library M.


## 5. Case Study

### 5.1. Function Semantic Annotation of the Sample Model

The 3D CAD model of gear oil pump is used as an example for analyzing and annotating functions in the 3D product model. The correlation relationship among the structure, function, and attribute of the 3D model is expressed by using the polychromatic set, and corresponding functional semantics is annotated interactively. The 3D CAD product model of gear oil pump and parts information is shown in Figure3.Figure 3
3D CAD model of gear oil pump.The 3D model structure of gear oil pump is divided into several independent and stable assembly regions AR1∼AR8. The parts and assembly feature pairs attribute information contained in the assembly region are analyzed for determining the functions that can be realized in each assembly region. The function analysis of the flow-activity region is mainly about the function analysis of the local shape structure that can transmit the flow in the 3D part model. The flow-activity region of the functional part that can transmit flow is analyzed from the perspective of the material flow, signal flow, and energy flow. Through the structure function analysis and functional semantic annotation of the flow-activity region, the key structure in the parts can be discovered and the design intent can be captured. It improves the efficiency of complex parts and key structures reuse. The functional region and functional semantic annotation in the 3D CAD model of gear oil pump is shown in Table2.Table 2
Functional regions and their functional semantics.
CategoryFRAssembled partFunctional semanticsAssembly regionsAR1{p1, p4, p5, p6, p7}F1: sealing liquidF2: position pump coverF3: stabilize pump coverAR2{p1, p2, p3, p7}F4: position-driven gear shaftF5: support-driven gear shaftF6: transform motionF7: supply liquidAR3{p3, p7, p8, p9}F8: prevent liquidAR4{p1, p5, p6, p7, p15}F9: fasten-pump coverAR5{p3, p11, p14}F10: position driving gearF11: stabilize driving gearAR6{p3, p11, p12, p13}F11: stabilize driving gearF12: fasten driving gearAR7{p3, p7, p9, p10}F9: fasten pump coverF13: stabilize bushFlow-activity regionsFAR1p3: gear structureF7: supply liquidFAR2p2: gear structureF7: supply liquidFAR3p6: boss structure 2F14: export liquidFAR4p6: boss structure 1F15: import liquidFAR5p11: gear structureF16: transfer torqueF17: export liquidF18: transfer powerFAR6p6: cavity structureF17: export liquid


### 5.2. Latent Functional Semantic Annotation Based on Similarity

Some 3D CAD product models with annotated functional semantics can be obtained through the structure-function correlation analysis and functional semantics labeling for 3D CAD product sample models. Based on that, the latent functional semantic prediction model based on the similarity can be used to automatically annotate the functional semantics in 3D CAD product models in the model library. Through equation (9), the latent functional semantics can be predicted and annotated probabilistically.Several typical 3D models are taken as example, which is given with latent functional semantics corresponding probability, as shown in Table3. Among them, the maximum similarity structure is the annotated model structure in the model library, and the probability of latent functional semantics is calculated by equation (9). Since the calculation of probability is related to the annotated sample model library, if sample models are fewer, the accuracy of latent functional semantic annotation will be low. In this case, the designer’s design history knowledge and experience should be considered for determining the final functional semantics to be annotated.Table 3
Latent functional semantic annotation of the 3D model.
No.Unannotated modelMaximum similarity structureLatent functional semantics and probability1F1: transfer-torque (0.856)F2: transfer-power (0.856)F3: change-speed (0.762)2F1: transfer-torque (0.758)F2: convert-motion (0.758)F3: transform-motion (0.758)F4: change-speed (0.652)3F1: position-part (0.755)F2: fasten-part (0.755)4F1: transfer-torque (0.763)F2: convert-motion (0.763)F3: transform-motion (0.763)F4: change-speed (0.672)5F1: position-part (0.765)


## 5.1. Function Semantic Annotation of the Sample Model

The 3D CAD model of gear oil pump is used as an example for analyzing and annotating functions in the 3D product model. The correlation relationship among the structure, function, and attribute of the 3D model is expressed by using the polychromatic set, and corresponding functional semantics is annotated interactively. The 3D CAD product model of gear oil pump and parts information is shown in Figure3.Figure 3
3D CAD model of gear oil pump.The 3D model structure of gear oil pump is divided into several independent and stable assembly regions AR1∼AR8. The parts and assembly feature pairs attribute information contained in the assembly region are analyzed for determining the functions that can be realized in each assembly region. The function analysis of the flow-activity region is mainly about the function analysis of the local shape structure that can transmit the flow in the 3D part model. The flow-activity region of the functional part that can transmit flow is analyzed from the perspective of the material flow, signal flow, and energy flow. Through the structure function analysis and functional semantic annotation of the flow-activity region, the key structure in the parts can be discovered and the design intent can be captured. It improves the efficiency of complex parts and key structures reuse. The functional region and functional semantic annotation in the 3D CAD model of gear oil pump is shown in Table2.Table 2
Functional regions and their functional semantics.
CategoryFRAssembled partFunctional semanticsAssembly regionsAR1{p1, p4, p5, p6, p7}F1: sealing liquidF2: position pump coverF3: stabilize pump coverAR2{p1, p2, p3, p7}F4: position-driven gear shaftF5: support-driven gear shaftF6: transform motionF7: supply liquidAR3{p3, p7, p8, p9}F8: prevent liquidAR4{p1, p5, p6, p7, p15}F9: fasten-pump coverAR5{p3, p11, p14}F10: position driving gearF11: stabilize driving gearAR6{p3, p11, p12, p13}F11: stabilize driving gearF12: fasten driving gearAR7{p3, p7, p9, p10}F9: fasten pump coverF13: stabilize bushFlow-activity regionsFAR1p3: gear structureF7: supply liquidFAR2p2: gear structureF7: supply liquidFAR3p6: boss structure 2F14: export liquidFAR4p6: boss structure 1F15: import liquidFAR5p11: gear structureF16: transfer torqueF17: export liquidF18: transfer powerFAR6p6: cavity structureF17: export liquid


## 5.2. Latent Functional Semantic Annotation Based on Similarity

Some 3D CAD product models with annotated functional semantics can be obtained through the structure-function correlation analysis and functional semantics labeling for 3D CAD product sample models. Based on that, the latent functional semantic prediction model based on the similarity can be used to automatically annotate the functional semantics in 3D CAD product models in the model library. Through equation (9), the latent functional semantics can be predicted and annotated probabilistically.Several typical 3D models are taken as example, which is given with latent functional semantics corresponding probability, as shown in Table3. Among them, the maximum similarity structure is the annotated model structure in the model library, and the probability of latent functional semantics is calculated by equation (9). Since the calculation of probability is related to the annotated sample model library, if sample models are fewer, the accuracy of latent functional semantic annotation will be low. In this case, the designer’s design history knowledge and experience should be considered for determining the final functional semantics to be annotated.Table 3
Latent functional semantic annotation of the 3D model.
No.Unannotated modelMaximum similarity structureLatent functional semantics and probability1F1: transfer-torque (0.856)F2: transfer-power (0.856)F3: change-speed (0.762)2F1: transfer-torque (0.758)F2: convert-motion (0.758)F3: transform-motion (0.758)F4: change-speed (0.652)3F1: position-part (0.755)F2: fasten-part (0.755)4F1: transfer-torque (0.763)F2: convert-motion (0.763)F3: transform-motion (0.763)F4: change-speed (0.672)5F1: position-part (0.765)


## 6. Conclusion

In this paper, an approach for functional semantic annotation of the mechanical 3D product model is proposed. 3D product model-oriented function ontology is constructed to provide standard, unified feature and function vocabulary for functional semantic annotation. The 3D CAD product models break up into functional regions at different granularity. Functional region is used as semantic structure unit, together with product design experience, working principle, and so on; the functional semantics of sample models is annotated interactively. The similarity of functional regions is evaluated comprehensively considering multisource information. Similarity-based latent functional semantic annotation model is given for automatic annotation of functional semantics in CAD models.The proposed approach improves efficiency and accuracy of functional semantic annotation to a certain extent, which can assist designers accelerating the speed of the functional semantic annotation. It provides support for searching 3D CAD model using product function requirement, promotes the understanding, absorbs and reuses the complex 3D product model knowledge structure, and inspires the product design innovation. Further research will focus on functional semantics-driven retrieval and product structure optimization of the 3D CAD model.



---

*Source: 9885859-2023-02-04.xml*