Author: Joni Rajamäki, University of Turku
Introduction
I completed my Master’s thesis as part of the Virtual Sea Trial and as a member of the Department of Computing at the University of Turku. The topic of my thesis is the development and evaluation of a proof-of-concept artificial intelligence-enhanced extended reality tool for ship commissioning.
“A shipbuilding project consists of vast amounts of contractors, subcontractors and suppliers, and the commissioning of ships is reliant on engineers navigating documentation and manuals from each of these parties. Extended reality technologies present a new paradigm of human computer interaction, and can be utilized to present data in revolutionary ways, and artificial intelligence can be utilized to compound large sets of information, such as technical documentation related to ship commissioning and distinguish the relevant information. These two technologies can be combined to make artificial intelligence enhanced extended reality solutions.”
Purpose
The research in my thesis serves multiple purposes. The proof-of-concept works as an introduction to extended reality technologies to the partners of the Virtual Sea Trial. The evaluation of the proof-of-concept and the findings of my research demonstrate the potential of contemporary extended reality devices as industrial tools. Finally, the developed proof-of-concept also acts as a starting point for future development conducted in tandem with VST partners.
This research demonstrates, that modern XR devices have reached a level of capability, where they can be considered as serious tools in industry. The video passthrough and tracking qualities of the Quest 3 head mounted display device used for the proof-of-concept was deemed fit by the testers, the artificial intelligence generated instructions included were considered precise and understandable, and the testers could identify potential use-cases for such a tool within the commissioning process. I am personally very happy with these findings.
Figure 1. An oscilloscope tutorial within the extended reality environment.
The making of
My research took place over the summer and autumn of 2024. The initial phase consisted of developing the extended reality solution and writing the background chapters regarding XR and AI technologies. After completion, the solution was presented to VST partners during a seminar held at the University of Turku in August of 2024. Next, the results gained from the demo session were compounded and analyzed. With the findings collected, the rest of my thesis could be written. Once complete, the thesis made its rounds amongst the steering group of the VST project, during which key members of the project were given the opportunity to verify the contents before the thesis would be published. The final evaluation of the thesis was conducted in early 2025 and has since been published in the UTU thesis collection.
Figure 2. Demo session held at University of Turku during VST seminar.
Next steps
Based on the research conducted as part of this thesis, the publication: “Passthrough Extended Reality for Maritime Commissioning” was written and has been accepted to the AHFE 2025 International Conference held in Florida in July of 2025.
Link to thesis: https://urn.fi/URN:NBN:fi-fe2025030415734