# Virtual reality beyond design reviews in shipbuilding: the need for industry-tailored immersive data interaction

Nicolas Fourrier, Mustapha Benaouicha, Guillaume Moreau, Jean-Marie Normand

## ► To cite this version:

Nicolas Fourrier, Mustapha Benaouicha, Guillaume Moreau, Jean-Marie Normand. Virtual reality beyond design reviews in shipbuilding: the need for industry-tailored immersive data interaction. ICCAS 2022 - International Conference on Computer Applications in Shipbuilding, RINA, Sep 2022, Yokohama, Japan. hal-04340647

HAL Id: hal-04340647

https://hal.science/hal-04340647v1

Submitted on 13 Dec 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# VIRTUAL REALITY BEYOND DESIGN REVIEWS IN SHIPBUILDING: THE NEED FOR INDUSTRY-TAILORED IMMERSIVE DATA INTERACTION

**N Fourrier**, SEGULA Technologies · Naval and Energy engineering Research and Innovation Unit, France & Nantes Université, ENSA Nantes, École Centrale Nantes, CNRS, AAU-CRENAU, UMR 1563, F-44000 Nantes, France.

**M Benaouicha**, SEGULA Technologies · Naval and Energy engineering Research and Innovation Unit, France

**G Moreau**, IMT Atlantique, LAB-STICC UMR 6285, France

**JM Normand**, Nantes Université, ENSA Nantes, École Centrale Nantes, CNRS, AAU-CRENAU, UMR 1563, F-44000 Nantes, France.

## SUMMARY

In the context of shipbuilding, the potential of virtual reality (VR) as a visualization tool has been demonstrated [1, 2]. However, many conditions must be met for VR to be widely adopted in the shipbuilding industry. It primarily must be seamlessly integrated into current workflows. The shipbuilding industry will likely first need to tame its data with efficient models accessible throughout the ship's lifecycle. However, some productivity issues with VR have not yet been identified clearly. While working to develop a VR design tool for ship outfitting, it became clear to us that there is a need for industry-driven development of efficient data interaction and manipulation tools in VR. As an example of such new manipulation tools, we are currently developing a new way to input text in VR that is efficient and easy-to-learn through handwriting. The evaluation of this system highlights the need for more controlled and systematic testing of VR tools in the shipbuilding industry to justify investments and adoption.

## 1. INTRODUCTION

The popularity of virtual reality (VR) in industry is at an all-time high. Even in shipbuilding, industrials and academics investigate the usefulness of VR, pushed by growing demands from many shipyards around the world. At ICCAS, the topic of mixed reality (MR) has been tackled regularly for at least a dozen years [3-5]. From marketing to visualization, the range of VR potential applications is as vast as the range of operations related to shipbuilding.

In the consumer market, immersion and social interaction are the main reasons for VR to be used. In an industrial setting, immersion can provide better visualization of 3D models and VR's social aspects may be used to design efficient collaborative tools. As such, there is no unique way to use VR and its potential is demonstrated in many fields: sales, design reviews, training and more. However, integrating new and disruptive tools into existing workflows is technologically complex, time-consuming and expensive. The first step, which has already received major contributions, is to define properly all the conditions for a successful integration of VR into current industrial processes.

Design and conception are two core industrial processes and they both require manipulating data and not only visualizing it. Paradoxically, the industry only uses VR for visualization purposes. The usage of VR as a data manipulation tool remains mostly unexplored today. Indeed, this use case requires massive changes in the way industries handle their data. This kind of transformation is already happening though, hidden behind terms like digitalization and Industry 4.0. Using VR as a design tool can then become a new incentive to accelerate the transition towards standardized and easily consumable data. This would make it possible to develop more efficient design tools that can leverage all the advantages VR has to offer.

## 2. CONDITIONS FOR LARGE-SCALE INDUSTRY ADOPTION

There are many barriers to large-scale adoption of virtual and augmented reality (AR) systems in an industrial setting, and especially in shipbuilding. Defining them properly is necessary as a first step towards better implementations of VR tools.

### 2.1 COMMUNICATION BETWEEN VR AND CAD APPS

Integrating new tools into established processes requires bidirectional communication and data transfer with existing tools. This is especially difficult when the tools to link have different data structures. This becomes increasingly apparent with innovative tools like VR. Indeed, they are often developed with game engines that represent 3D models in a discrete manner using triangles, called tessellation. This is highly efficient for the real-time rendering that is necessary for video games and VR. However, this model is incompatible with Computer-Aided Design (CAD) data that mostly uses continuous representations like NURBS. The conversion from CAD to VR, from a geometrical standpoint, is possible thanks to many algorithms that aim to discretize and optimize 3D models for real-time rendering [6, 7]. However, when it comes to metadata such as part hierarchy or textures, the transformation often requires some amount of work that cannot be streamlined and depends on both the CAD and VR file formats [8]. There is a push for CAD data structures to represent
tessellated geometry, for example with the new versions of STEP AP242 (ISO 10303-242) files but most design software
cannot yet interpret this new norm. The CAx-Interoperability Forum regroups the companies actively working on this
topic with the aim of accelerating the development of STEP AP 242 translators [9]. Those companies evolve in different
industries such as automotive, defence, aerospace and energy. CAD software developers such as Autodesk, Dassault
Systèmes, Datakit and others, accompany them in this endeavour. This shows there is a growing demand from every
industry for better interoperability of CAD file formats especially when it comes to representing tessellated geometry.

Unity, one of the most commonly used game engine to develop VR applications, acquired Pixyz in 2021, which is a plugin that aims to let users import CAD files directly into the game engine. The import is automatically optimized for 3D rendering and supports files coming from most CAD software. There also exist file conversion software such as CAD Exchanger, that lets users transform CAD file formats into others, including ones compatible with game engines. Also, there are open libraries that aim to ease the development process of conversion software such as Assimp [10]. This type of transformation, no matter the software used, often comes with a partial loss of geometrical data and metadata that prevents two-way data transfer.

Indeed, VR to CAD is even more complex as this conversion cannot be done strictly with only the deteriorated data – because it has been discretized and tessellated – that emanates from the VR system. The process of converting back geometric data from a tessellated format into one that is readable by CAD software mainly relies on interpolation [11, 12] or primitive shape extraction techniques [13]. In some cases, it might be possible to reconstruct fully – including metadata – the CAD data but this would rely on accessing 3D model databases or creating a third-party software that has access to both domains. Some software use this kind of technology to create conversion-less import of CAD models in VR scenes and even allow some amount of bidirectional real-time modifications :

* Exxar CAD integrates with standard CAD applications such as Aveva Marine, CADMatic, Catia... to stream the 3D data directly in a VR review application. Users working on the same model both in a CAD software and in VR can annotate the models. Those annotations propagate in real-time.

* Techviz interfaces with CAD applications like Catia and NX for real-time visualization of 3D models. Modifications made in the CAD software can be visualized while a VR session is ongoing. Some interactions with the models are possible in VR such as measuring or moving parts, but it is not possible to modify the CAD model in VR.

* Mindesk also integrates with many CAD applications like Solidworks, Revit or Rhino. It allows streaming CAD data directly into VR for visualization purposes. Leveraging Rhino's data format, Mindesk allows VR modifications to be reflected in real-time on the CAD model through Unreal Engine, which is another game engine often used for developing VR applications.

These observations highlight the need for industries, including shipbuilding, to tame their data. Data must become easily consumable which means having a unique model used throughout the lifecycle of the project. Having access to up-to-date models is necessary to widen the range of possible VR applications [2, 14], as they are currently limited to non-modifiable visualization when it comes to CAD. In the context of Industry 4.0, many powerful models are emerging:

* Building Information Modelling (BIM): a technology that aims to simulate construction projects in virtual environments. A unique model is constructed for "design, procurement, fabrication, construction [...], operations and maintenance" [15]. The model is accessible by all teams for the whole lifecycle of the project. This model serves as the basis for digital transformation in the architecture and construction industry, which are highly similar to the shipbuilding industry. A recent open-source Unity plugin aims to allow bidirectional communication between BIM authoring platforms and the game engine [16].

* Product Lifecycle Management (PLM): a technology that aims to manage the complete lifecycle of a project from ideation to disposal. This model contains all the necessary information from the specifications, technical solutions, to fabrication and maintenance details. Data is streamlined as to create a "coherent data flow, avoiding redundancies and gaps" [17]. In BIM, the digital mock-up is at the center of the model while PLM also formalizes the processes of modification and validation of the model.

* Digital Twins: the virtual representation of real world assets with the goal of representing the real time state of said assets. This virtual representation should span the whole lifecycle of the asset. The virtual model can be used for monitoring, diagnostics, design iterations, simulation... to improve productivity. The link between digital twins and VR is a popular research topic [18].

* Industrial Internet of Things (IIoT): application of Internet of Things (turning every object into a connected device) to industry. It enables ubiquitous monitoring and communication between machines for better automation [19]. This kind of technology requires data standardization and is often paired with augmented reality to monitor interactively the state of a machine [20].
* Knowledge-Based Engineering (KBE): aims to increase productivity and reduce costs by automating design tasks through the formalization of expert knowledge [21]. The technology often uses an inference engine to determine if a set of facts validates a set of rules [22]. The rules should be representative of the expert knowledge.

The objective of such technologies and processes is to create a unique data model that is easily consumable during the whole lifecycle of the project: from early designs all the way to the supply chain. It can enable multidisciplinary teams to interact simultaneously with the same data. They can also reduce iteration time and costs if used as decision-helping tools. This makes these systems ideal candidates for application in the ship building industry.

## 2.2 EASE OF USE AND DEVELOPMENT

The communication between VR and CAD tools is important but is not the only condition preventing wider adoption of VR. Martin [4] stated that VR tools must be easy-to-use even for non-CAD experts. Indeed, the expertise needed to run and be proficient with VR tools nowadays far outweighs the benefits. First, as stated before, the amount of work required to prepare data for VR is non-negligible and non-trivial. While there are solutions to convert the data for visualization as demonstrated in Section 2.1, they often require human intervention or expertise. As of today, teams that want to use VR regularly and efficiently for design reviews purposes would need a dedicated VR expert.

Secondly, the development of VR prototypes is difficult and often lacks collaboration with field experts. This means systems end up missing features or are clunky to use. It might not be a priority to create a perfect user experience because we are still experimenting and trying to prove the potential of VR, but it does not help with convincing experts of VR capabilities. VR development is not easy also because there are many systems that are not all compatible with each other. Indeed, different levels of VR can be experienced depending on the output (head-mounted displays (HMDs), CAVEs, 3D screens...) and input (controllers, navigation, body and gesture tracking) devices [23]. This means some amount of work is required to standardize the experience across platforms.

Finally, when it comes to actual usage, it may be difficult for the less tech-savvy to get the hang of VR. New users must first realize what it means to be immersed in a virtual environment, and they must then learn how to interact with said environment. New users may also fall into the trap of expecting too much from VR. Misconceptions about the capabilities of VR can lead to disappointment, which may harm one's ability to adapt to the technology. We can however expect VR acquaintance to grow as time passes. Nick Clegg, President for Global Affairs at Meta Platforms, recently published a manifesto about the vision of Meta – ex-Facebook – regarding the future of VR [24]. This vision revolves around standardisation and a new social way to experience the internet called metaverses.

With more people acquainted with VR, it will be easier for industry to adopt the technology. Moreover, Meta's short-term objectives are all about releasing new HMDs designed with productivity and industrial use in mind. It is currently pushing for more collaboration with lawmakers, academics and industrials. This is a major opportunity to define properly what is necessary for VR in industry. For the first time, workshops [25, 26] were dedicated to these topics during the IEEE Virtual Reality 2022 conference (the premier international conference on VR) and the push for VR in industry is one of the main factors that will influence the transition towards metaverse-like VR implementations. Such standardization of the VR ecosystem would help greatly the development of industrial applications.

VR applications are often developed using game engines and SDKs that are suited for video games. As such, visualization, interaction and manipulation techniques in virtual environments are vastly different from CAD software. VR apps are not traditionally created with productivity and industrial use in mind, and this shows even when industrials are involved in the making. As such, making VR easy-to-use is probably not sufficient, and new innovative ways to interact with data in VR, that differ radically from the ones used today, must be studied with industry as a priority.

## 2.3 THE NEED FOR INDUSTRY-TAILORED DATA INTERACTION TECHNIQUES

The main hindrance to adopting VR in industry remains data compatibility. As stated before, major VR and CAD actors globally are actively working on this issue. We can realistically believe it will be resolved in a not-so-far future. We must then anticipate the next problem in adopting VR, which is how to make it productive enough to rival current processes. Indeed, current data interaction and manipulation techniques in VR are not compatible with productivity and industrial use. For example, in a traditional CAD software, a keyboard and mouse setup lets the user select and modify data. This setup is efficient in both interacting with and manipulating data and does not really require any training, as everyone has some experience with the setup. In VR though, while there exist many performant and intuitive ways to select data [27], manipulating and changing it is challenging. Indeed, it is difficult to embark a physical keyboard in a VR setup, even if doing so results in unmatched data entry speeds – measured in words per minutes (WPM) – varying from 40 to 80 WPM [28]. This kind of system however requires specific setups that are difficult, from a logistical and technological standpoint,
to integrate with standard HMDs. In general, “plug and play” VR apps have only tried to emulate and virtualize classic – keyboard and mouse setups – interaction methods. This makes sense in a first approximation because the familiarity helps when it comes to understanding VR. However, ultimately, since being immersed in data is wildly different from interacting with it through a flat-screen, the virtualization of classic techniques (such as virtual keyboards) proves not to be efficient enough. As such, it is understandable that we cannot expect VR to succeed in an industrial setting – where productivity is paramount – if even basic tasks such as typing text are daunting.

There are a lot of research and commercial works exploring new ways to handle locomotion, interaction and collaboration in virtual reality. However, entertainment industries are the ones driving this exploration. The needs in those fields is vastly different from industrial needs. Industry must be the driver behind the creation of efficient VR data interaction and manipulation techniques. Three factors are primordial when designing VR applications for industrial use: efficiency, ergonomics, and ease-of learning and mastering. Combining these factors and bidirectional communication between VR and CAD will allow, in the end, to use VR as a design tool beyond its traditional usage of visualization and design reviews.

### 3. VR AS A DESIGN TOOL

There are three main reasons VR would be great for design purposes. First, VR is immersive, which makes it an ideal tool for visualizing difficult situations at 1:1 scale. Indeed, in standard CAD tools, it may happen that some design tasks, and especially in the highly constrained environment of shipbuilding, are rendered difficult because of the less than optimal visualization strategies [29]. Secondly, VR is becoming more and more of a social platform that lets its user interact and share experiences with each other. Some amount of standardization in the domain of identity and avatars in VR will happen thanks to the rise of metaverses, which will make it easier to integrate collaborative components into industrial VR tools. Collaborative tools are great for better communication between designers and between teams, which will in turn improve design efficiency and reduce iteration costs. Finally, users in VR are able to interact with the environment in a more intuitive and close to the real world manner. As such, VR tools can help the decision making process. For example, one could verify if doors can be opened by trying them, could walk through maintenance spaces to check if they are accessible, or even simulate the process of evacuation after pushing in an incendiary alarm button.

<figure><img src="image_2.png" /><figcaption>Figure 1 – a) The green zone represents an accessibility constraint. It is associated with an electric box object that has an open door highlighted in yellow. The green zone must remain free to let the door open. b) Whenever another object enters the area, the constraint is detected as being violated and the volume turns red to indicate the problem to the user.</figcaption></figure>

We are leveraging the advantages of VR through the development of an immersive, interactive, and collaborative VR design tool for ship outfitting. Outfitting is a highly multi-disciplinary process that involves many trades such as piping, electrical, plumbing, HVAC (heating, ventilation, and air-conditioning) and even furnishing. This process is often sequential and iterative. Its goal is to ensure that all the different constraints and norms from each trade are respected in contrived spaces. Combining the visualization power of VR with a decision-helping component will make the outfitting process more efficient. The goal of the tool is to let outfitters immerse themselves at 1:1 scale in the technical locals for better visualization, and be able to place new objects. Interacting with those objects will let them verify certain properties such as accessibility. Additionally, we are developing a decision-helping tool which goal is to let outfitters know when constraints are not respected.

By working hand-in-hand with outfitting experts, we aim to formalize knowledge [30] and exploit it in real-time in the VR tool. An example is given in Figure 1: placing an electric box in the environment comes with an accessibility constraint. Indeed, it must remain possible to open the door at all times, even after subsequent changes in the environment. The tool under development must be able to verify in real-time that all constraints are respected, and lets the outfitter know if there
are any problems. As such, by formalizing knowledge, the amount of work necessary to verify everything is in order should be reduced.

In the other hand, while developing this VR design tool, it became abundantly clear that text entry in VR, which is a common task in design, was not efficient enough for industrial use.

## 4. A NOVEL TEXT ENTRY TECHNIQUE

When it comes to conception and design, one of the main fundamental tasks is data entry. Indeed, creating and manipulating textual and numerical data is necessary in all design processes. In industry, these processes rely heavily on the keyboard as a mean to input data. Keyboards are overall highly efficient: they reach high words per minute (WPM), are not physically and mentally tiring, and most importantly they require little training since they are so common. In the context of using VR tools for design, creating and interacting with this type of data is not trivial. Three types of virtual keyboards are often used for tools that want to only rely on a standard HMD setup [31]:

* Head/eye pointing: the user uses the movement of its eyes or head to move a pointer on a virtual keyboard. Pressing a button on a controller will select the pointed-at key [32].

* Controller pointing: the user uses a laser-like apparatus associated with a controller to point directly at a virtual keyboard. Pressing a button on a controller will select the pointed-at key [33].

* Drum-like: the user uses virtual drumsticks to hit on a virtual keyboard that reacts to the collisions between the drumsticks and keys. No other input than the collision is required to type a character [34].

Compared to a classic keyboard that has a 75 WPM average typing speed for commonly skilled users [35], these virtual keyboards all have drawbacks that make them not practical for industrial purposes. Head/eye pointing keyboards have low typing speeds (16 WPM) and are physically demanding. Controller pointing keyboards suffer from the same problems in a lesser extent. Finally, drum-like keyboards report the best performance (21WPM) but it is still far from the performance of traditional keyboards and it remains physically tiring [36]. A lot of work has been done trying to create innovative data input techniques for VR using T9-like keyboards [37] or gestures-based methods [38]. While they can be better when it comes to physical and mental load, they are slower than virtual keyboards for beginners and their mastery requires a lot of training. It is then important to create a VR text input system that is suitable for industry: efficient, ergonomic and easy-to-learn.

### 4.1 HANDWRITING IN VR

In our work to create an interactive and collaborative VR tool for ship outfitting, we faced the problem described previously that it is hard to do design without performant data input, which does not really exist in the context of virtual reality. The scenarios where we need to input text or numbers are, non-exhaustively, the following:

* Filter databases to find 3D models to import in the virtual environment.

* Find items in large lists.

* Create or modify textual values like object names and annotations.

* Create or modify numerical values like object positions, scales...

All these scenarios are concise but reoccurring typing tasks. As such, we need an input technique that is efficient and reactive mainly for single – or few – word scenarios. In a real-life setting, when asked to provide short textual information without the use of a keyboard, one would grab a piece of paper and a pen. We believe mimicking this type of behaviour in VR could be more efficient than virtualizing keyboards. As such, we developed a system that enables handwriting in VR on a board with a pen. An off-the-shelf handwriting recognition software (HWR), that we integrated into the Unity game engine, transforms the handwritten text in real-time in an exploitable data format as shown in Figure 2. Such a system would indeed answer all the criteria for an industry-suitable input system in VR: handwriting speeds surpass the performance of virtual keyboards for short writing sessions as they average ~38 WPM [39, 40] and requires no training since virtually everyone knows handwriting. Moreover, writing in VR has been gamified many times (The Lab, Half-Life Alyx, DryErase...) and can become an entertaining task, which would reduce drastically the mental load associated with typing in VR.

Nevertheless, writing on a virtual surface that is not present physically comes with two challenges:

* It is necessary to create appropriate feedback for the user to know when the pen is indeed in contact with the virtual whiteboard.

* It can be tiring as it is not possible to rest against the writing surface.
To make the experience more intuitive, we added visual and haptic feedbacks. Visual feedbacks consist of enabling collisions between the virtual pen and the virtual whiteboard, shadows that help estimate the distances to the board, and an adaptive line thickness that varies following the amount of virtual pressure applied on the board. Haptic feedback consists of a varying controller vibration based on the pressure applied to the board. We are also experimenting with visual haptics by representing the contact of the pen with the board by adding a rippling texture on the board, substituting a haptic cue by a visual one. These feedbacks allow for writing that is more natural and lifelike, even if the cues do not perfectly match reality [41]. Adding guides, such as horizontal lines to follow when writing, will maximize the effectiveness of the handwriting recognition software because its performance depends greatly on each character's size and verticality.

<figure><img src="image_3.png" /><figcaption>Figure 2 – Handwritten text on a virtual blackboard in a Unity VR scene is transformed into exploitable string data thanks to the HWR software. The number provided with the data represents the recognition confidence of the algorithm on a 0-100 scale.</figcaption></figure>

We are also studying the influence of posture on the physical load of the virtual handwriting task. Indeed, for long writing sessions in a classroom whiteboard setup, not being able to rest on the board makes the task tiring. As such, it is possible to imagine new writing positions where the arm can be at rest either by providing a physical support or by finding an ergonomic position for the board. However, this problem only arises for long writing sessions, which are not the use case we envision for this system. Preliminary testing has demonstrated the potential of the system when it comes to writing speed, ease-of-use and general appreciation compared to virtual keyboards in a MacKenzie type test [42]. We are planning a two-part user study that will first determine implementation details of the system (feedback types, posture...) and then prove the effectiveness of the system in a more industrial task that is related to design in shipbuilding.

## 4.2 DESIGNING A USER STUDY TO ASSESS INDUSTRIAL USABILITY

One common flaw found in many of the VR tools developed for shipbuilding [3, 5, 29] is that there is no quantification of their usability. It is important to prove the potential of VR in both quantitative and qualitative manners. In the rare cases where user or case studies were conducted [1, 43], either there were not enough participants to be relevant, the study was not structured in a way a quantitative measure could be performed or statistically analysed, or the population of the study was made of students who could not give an appropriate industrial evaluation of the system. If we do not make more effort to provide metrics for the usefulness of VR in industry, it will remain nearly impossible to justify investing massively into this kind of system, especially if no comparisons with current systems are available. The “lack of perceived benefits” described by Morais et al. [14] is in part due to this lack of metrics.

Quantitative measures will always depend on the task; it is not possible to provide a general framework to assess quantitatively the performance of an industrial VR system. However, measuring qualitatively the usability of a system is a well-known exercise [44]. The SUS (System Usability Scale) questionnaire is the most often used: it evaluates 10 items related to usability in a task-abstracted fashion. This evaluation is often conducted in pair with the NASA-TLX questionnaire that aims to quantify the mental and physical loads of a task [45]. Combining these questionnaires should provide an objective metric of a VR tool's potential as well as a way to compare them to the currently used tools. It is important for the questionnaires that the tested population in a user study is representative of the target population. In academics settings, it is often the case user studies are conducted on students. It is debatable whether such results can justify industrial use. In industrial settings however, while technicians and engineers are often involved in the development
process of VR tools, proper case studies with quantifiable metrics are rarely performed. As such, more collaboration would benefit greatly the implementation of VR in industry. Indeed, such collaboration can lead to better-controlled environments for testing systems with relevant populations. We will follow these principles when carrying out the studies for the handwriting input system presented in Section 4.1 and the ship outfitting design tool presented in Section 3.

## 5. CONCLUSIONS

The difficult communication between current tools and processes and VR is the main limiting factor to large-scale VR adoption. Indeed, in industry, processes require smooth and streamlined transitions between the different cycles of a project. Additionally, this makes VR cumbersome to use because of the amount of work necessary to prepare data even for basic visualization. We showed these are popular research topics that industrials and academics are actively trying to resolve.

We argue there are additional limiting factors that have not been yet properly identified when it comes to expanding the scope of VR in industry to tasks like conception and design. Indeed, there is a dire need for more performant, more ergonomic, and easier-to-use data interaction techniques in VR that are suited for industrial use. Moreover, it is also important to start evaluating industrial VR applications in a more rigorous manner in controlled studies that can provide metrics of VR usefulness and usability. Morais & al. stated VR has to “offer a solution that is ten times better than the current way of doing things” [14]. It is important to be able to measure this statement, and compare VR to current systems, to justify massive investments towards industrial adoption. We believe that close collaboration between industry and academia would better advance this topic.

## 6. REFERENCES

1. J.J. GARCIA AGIS; P. BRETT; A. EBRAHIMI, ‘The potential of virtual reality (VR) tools and its application in conceptual ship design’, *Conference on Computer Applications and Information Technology in the Maritime Industries*, 2020.

2. J. MARTIN; A. CONNELL, ‘Accessible immersive visualisation for shipbuilding’, *ICCAS*, 2015.

3. H. LÖDDING; A.FRIEDEWALD; M.HEINIG; S. SCHLEUSENER, ‘Virtual Reality Supported Assembly Planning in the Shipbuilding Industry’, *Journal of Ship Production and Design*, 2011.

4. J. MARTIN, ‘Virtual reality: tool or toy for shipbuilding?’, *ICCAS*, 2019.

5. C.-J. CASSAR; R. SIMPSON; N.BRADBEER; G.THOMAS, ‘Integrating virtual reality software into the early stages of ship design’, *ICCAS*, 2019.

6. J. GUO; F. DING; X. JIA; D.-M. YAN, ‘Automatic and high-quality surface mesh generation for CAD models’, *Computer-Aided Design*, 2019.

7. A. MAGLO; G.LAVOUË; F. DUPONT; C. HUDELOT, ‘3D Mesh Compression: Survey, Comparisons, and Emerging Trends’, *ACM Computing Surveys*, 2015.

8. K. MCHENRY; P. BAJCSY, ‘An Overview of 3D Data Content, File Formats and Viewers’, *Technical Report for Image Spatial Data Analysis Group*, 2008.

9. ‘CAx Interoperability Forum’, https://www.cax-if.org/cax/cax_introduction.php.

10. ‘Open Asset Import Library (assimp)’, https://github.com/assimp/assimp.

11. P. PAL, ‘A reconstruction method using geometric subdivision and NURBS interpolation’, *International Journal of Advanced Manufacturing Technology*, 2008.

12. P.PAL; R. BALLAV, ‘Object shape reconstruction through NURBS surface interpolation’, *International Journal of Production Research*, 2007.

13. R. BÉNIÈRE; G.SUBSOL; G. GESQUIÈRE; F. LE BRETON; W. PUECH, ‘A comprehensive process of reverse engineering from 3D meshes to CAD models’, *Computer-Aided Design*, 2013.

14. D.MORAIS; M. WALDIE; D. LARKINS, ‘The Evolution of Virtual Reality in Shipbuilding’, *COMPIT*, 2017.

15. S. AZHAR, ‘Building Information Modeling (BIM): Trends, Benefits, Risks, and Challenges for the AEC Industry’, *Leadership and Management in Engineering*, 2011.

16. K. W. MAY; J. WALSH; R. T. SMITH; N. GU; B. H. THOMAS, ‘UNITYREV – Bridging the gap between BIM authoring platforms and game engines by creating a real-time bi-directional exchange of BIM data’, *CAADRIA*, 2022.

17. S. TERZI; A. BOURAS; D. DUTTA; M. GARETTI; D. KIRITSIS, ‘Product lifecycle management – From its history to its new role’, *International Journal of Product Lifecycle Management*, 2010.

18. J. PIRKER; E. LORIA; S. SAFIKHANI; A. KUNZ; S. ROSMANN, ‘Immersive Virtual Reality for Virtual and Digital Twins: A Literature Review to Identify State Of The Art and Perspectives’, *IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)*, 2022.

19. E. SISSINI; A. SAIFULLAH; S. HAN; U. JENNEHAG; M. GIDLUND, ‘Industrial Internet of Things: Challenges, Opportunities, and Directions’, *IEEE Transactions on Industrial Informatics*, 2018.
20. M. HUSINSKY; A. SCHLAGER; A. JALAEEFAR; S. KLIMPFINGER; M. SCHUMACH, 'Situated Visualization of IIoT Data on the Hololens 2', *IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)*, 2022.

21. W. J. C. VERHAGEN; P. BERMELL-GARCIA; R. E. C. VAN DIJK; R. CURRAN, 'A critical review of Knowledge-Based Engineering: An identification of research challenges', *Advanced Engineering Informatics*, 2012.

22. M. T. SHAHZAD; J. HOFFMANS, 'KBE/AI for Ships Construction – A Feasibility Study', *COMPIT*, 2021.

23. C. ANTHES; R. J. GARCIA-HERNANDEZ; M. WIEDEMANN; D. KRANZLMÜLLER, 'State of the art of virtual reality technology', *IEEE Aerospace Conference*, 2016.

24. N. CLEGG, 'Making the metaverse: What it is, how it will be built, and why it matters', Medium, 2022.

25. 'Metaverse as a promise of a bright future? – social interactions in a world of isolation', *IEEE VR Workshops*, https://workshop.htt.events/, 2022.

26. 'Metabuild – Building the Foundations for the Metaverse', *IEEE VR Workshops*, https://metabuild-workshop.herokuapp.com/, 2022.

27. F. ARGELAGUET SANZ; C. ANDUJAR, 'A Survey of 3D Object Selection Techniques for Virtual Environments', *Computer and Graphics*, 2013.

28. D.-M. PHAM; W. STUERZLINGER, 'HawKEY: Efficient and Versatile Text Entry for Virtual Reality', 25th *ACM Symposium on Virtual Reality Software and Technology*, 2019.

29. V. ALONSO; R. PÉREZ; L. SANCHEZ; R. TRONSTAD, 'Advantages of using a virtual reality tool in shipbuilding', *COMPIT*, 2012.

30. Z. ZHU; M. VAN TOOREN; G. LA ROCCA, 'A KBE Application for Automatic Aircraft Wire Harness Routing', 53rd *AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference*, 2012.

31. T. J. DUBE; A. S. ARIF, 'Text Entry in Virtual Reality: A Comprehensive Review of the Literature', *Human Computer Interaction. Recognition and Interaction Technologies*, 2019.

32. C. YU; Y. GU; Z. YANG; X. YI; H. LUO; Y. SHI, 'Tap, Dwell or Gesture? Exploring Head-Based Text Entry Techniques for HMDs', *Proceedings of the CHI Conference on Human Factors in Computing Systems*, 2017.

33. M. SPEICHER; A. M. FREIT; P. ZIEGLER; A. KRÜGER, 'Selection-based Text Entry in Virtual Reality', *Proceedings of the CHI Conference on Human Factors in Computing Systems*, 2018.

34. C. BOLETSIS; S. KONGSVIK, 'Text Input in Virtual Reality: A Preliminary Evaluation of the Drum-Like VR Keyboard', *Technologies*, 2019.

35. A. S. ARIF; W. STUERZLINGER, 'Analysis of text entry performance metrics', *IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH)*, 2009.

36. C. BOLETSIS; S. KONGSVIK, 'Controller-based Text-input Techniques for Virtual Reality: An Empirical Comparison', *International Journal of Virtual Reality*, 2019.

37. D. YU; K. FAN; H. ZHANG; D. MONTEIRO; W. XU; H.-N. LIANG, 'PizzaText: Text Entry for Virtual Reality Systems Using Dual Thumbsticks', *IEEE Transactions on Visualization and Computer Graphics*, 2018.

38. S. CHEN; J. WANG; S. GUERRA; N. MITTAL; S. PRAKKAMAKUL, 'Exploring Word-gesture Text Entry Techniques in Virtual Reality', *CHI Conference on Human Factors in Computing Systems*, 2019.

39. N. VAN DREMPT; A. MCCLUSKEY; N. A. LANNIN, 'A review of factors that influence adult handwriting performance', *Australian Occupational Therapy Journal*, 2011.

40. J. SUMMERS; F. CATARO, 'Assessment of handwriting speed and factors influencing written output of university students in examination', *Australian Occupational Therapy Journal*, 2003.

41. N. COOPER; F. MILELLA; C. PINTO; I. CANT; M. WHITE; G. MEYER, 'The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment', *PLOS ONE*, 2018.

42. I. S. MACKENZIE; R. W. SOUKOREFF, 'Phrase Sets for Evaluating Text Entry Techniques', *CHI: NEW HORIZONS*, 2003.

43. K. NORDBY; S. BØRRESEN; E. GERNEZ, 'Efficient Use of Virtual and Mixed Reality in Conceptual Design of Maritime Work Places', *COMPIT*, 2016.

44. C. KUN; N. ASHAARI, 'Utilitarian or Experiential? An Analysis of Usability Questionnaires', *International Journal of Computer Theory and Engineering*, 2015.

45. S. G. HART; L. E. STAVELAND, 'Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research', *Advances in Psychology*, 1988.

## 7. AUTHORS BIOGRAPHY

Nicolas FOURRIER holds the current position of PhD student at Segula Technologies in collaboration with AAU (Ambiances, Architectures, Urbanités) laboratory. He is working on a thesis titled "Interactive and collaborative ship outfitting in virtual reality".
**Mustapha BENAOUICHA** obtained his PhD in Numerical Modelling from La Rochelle University, France in 2007. He is currently Head of the Naval and Energy Engineering Research and Innovation Unit at SEGULA Technologies. He is responsible for the project concerning the use of VR technology in decision support for ship outfitting.

**Jean-Marie NORMAND** obtained his PhD in Computer Science from Nantes University, France in 2008. He is currently Associate Professor at École Centrale de Nantes, France. His research interests are VR, especially Virtual Embodiment, as well as AR, especially perception and interaction.

**Guillaume MOREAU** obtained his PhD in Computer Science in 1998 from Rennes 1 University. He is currently full Professor in Computer Science at IMT Atlantique, France. His research interests are Virtual and Augmented reality with a focus on how these environments are perceived.