## 2 Paper Research and Evaluation
### Bibliometric Analysis  
From the previous query, a total of 244 documents were returned (of which one was excluded due to it not staying within the topic), ranging in publication date from 2016 to 2023. From the initial period up to 2021, an annual growth rate of 133.22% was found, making this research topic very important to the field, as illustrated by Figure 2. Furthermore, from the entire range of publication dates, it was found that there is an international co-authorship of 25.1%. In addition to measuring the co-authorship, this indicator includes hidden relationships between the co-authors of a scientific paper, such as motivation and the research rank of their institutions [47]. This level of international co-authorship involves a high degree of international collaboration and a positive contribution to the citation impact [48]. As a result, there is a noteworthy flow of researchers between institutions in different parts of the world, enabling deep investigation of DRL applications in manufacturing. This is illustrated in Figure 3, where the most active researchers in the field are linked to the most relevant topics they have investigated and the research institutions they belong to. It can be noted that most of the researchers have belonged to several research institutions, showing the close collaboration between them, and they have participated in several scientific publications on different topics within this field. These collaborations allow progress in the development of new DRL technologies in the industry. The institutions shown in Figure 2 are leading in deep reinforcement learning research. If this is translated geographically, it can be observed that most of them belong, in this order, to China, the United States and the United Kingdom. Figure 4 depicts the collaborations between institutions by country, where the strongest research relationships in this field emerge. The strong and stable relationship between American and Chinese institutions is noteworthy, with a flow of researchers in both directions. In fact, for China, the United States is the only country with which close collaboration is observed. However, the United States does have collaborations with institutions in European countries, such as Imperial College London and Oxford University from the United Kingdom and the Robotic Systems Lab belonging to ETH Zurich (Switzerland). Lastly, although not many active researchers from Canadian institutions have been detected, the analysis shows a large number of collaborations between those countries. Figure 3: Three-field plot: author keywords on the left, authors in the middle and affiliations on the right. Figure 2: Annual scientific production of the query. Finally, an analysis of the journals was carried out following Bradford's Law, a law of diminishing returns in which journals are divided into three zones depending on the frequency of citation [49]. Figure 5 shows in descending order the essential sources of information when looking for DRL knowledge. The most cited document was published by [PERSON] et al. (2016) [50], which presents a DRL algorithm applied to robot manipulation tasks. It was cited globally 701 times, making it the most prolific document in the search. Regarding the appearance of the main approaches to DRL, of the 243 documents: 131 of them mention a model-free approach (53.91%), 113 model-based (46.50%), 30 off-policy (12.35%) and 14 on-policy (5.76%) approaches. Moreover, 35 documents mention both model-based and model-free approaches (14.40%); one mentions both model-based and on-policy approaches (0.41%); one mentions model-free and on-policy approaches; four mention model-free and off-policy approaches (1.65%); and four mention on-policy and off-policy approaches. Overall, it seems that the DRL applications in the industry are more focused on the need for a model of the environment, highlighting the vast presence of model-free and model-based approaches.