## 1 Introduction
[7] integrated data from different information systems into a KG to solve the problem of data islands in the plant planning process, providing data quickly according to the needs of planners. Therefore, KG technology can realize the storage, management, and visualization of causal knowledge of product surface defects. At present, the storage of information about product surface defect problems is mainly based on images and texts. Intuitively, the images store the feature information of defects, and text stores the description of defect features, causality, and other aspects. From the above analysis, it can be seen that the classification of defect images can rely on computer vision technology, while the storage and visualization of defect causal knowledge can rely on KG technology. Considering the point-to-point relationship mapping of \"surface defect-defect causal knowledge\", it can be found that the surface defect detection of industrial products based on the deep convolutional neural network has obvious shortcomings, namely:  
1. The DCNN only obtains the data output of the category and location information of product surface defects but cannot express the implicit causal knowledge behind the image defects, which ignores the rich semantic information of the surface defect image itself and the huge potential connection among information. 2. The knowledge graph can visualize knowledge, making finding the relationship between knowledge easier. Therefore, to enrich the semantic information of surface defect images through knowledge graph technology, it is necessary to fuse the content of image classification technology and knowledge graph technology. Still, they have a vast semantic gap, and the joint task is challenging to solve. Therefore, some researchers have begun to integrate computer vision technology and KG technology for multimodal information processing. [PERSON] et al. [8] proposed an image captioning system, CNET-NIC, to produce better image captions by detecting objects in images. The background knowledge of the image is stored in the form of a knowledge graph. Firstly, the objects in the image are detected by a convolutional neural network, and then the detected objects are used to identify related terms and concepts. Then, these termsor concepts are connected to the corresponding nodes in the knowledge graph. Finally, the subtitle or image description produced by the machine is improved by the relationship between nodes. [PERSON] et al. [9] proposed an image classification technology based on KG technology, which improves the performance of image classification by constructing an image knowledge graph (IKG). However, there are still few studies on multimodal information processing in the cause analysis of product surface defects. In addition, the text datasets of defect cause in the engineering field are limited, which cannot represent the potential knowledge of poor design and manufacturing related to defect images. Based on the technology of multimodal information processing and inspired by the success of the semantic refinement method [10], we propose an image semantic refinement recognition method based on causal knowledge for product surface defects. It aims to visualize the output of image defects and the complete causal knowledge behind them through KG technology, which transforms them into a visually understandable structured form, enriching the image semantic information of product surface defects. The rest of this article is organized as follows. Section 2 introduces the related work about computer vision technology and knowledge graph technology. In Section 3, we propose the framework of a semantic refinement recognition structure for product surface defect images based on causal knowledge. The Section 4 carries on the relevant experiment through the application case. Section 5 presents a framework for developing a visual system. Section 6 discusses the conclusions and future work.