Cybersickness in extended reality

The FLSC department was exploring the potential of immersive technology in fighter pilot training. I secured an internship with them, eager to work with the Varjo XR-3 headset and gain experience with the Unreal Engine. Cybersickness, a common issue in VR, posed a challenge for pilot training. My proposal aimed to assess the Varjo XR-3's effectiveness in reducing cybersickness and develop a flight simulator using Unreal Engine. The FLSC approved the idea, and I initiated a pre-study report.

My pre-study detailed background on cybersickness, related work, and how I would go about developing the flight simulator. Previous research had indicated that a mixed reality (MR), with a lower amount of virtual environment, elicit less severe cybersickness than a fully virtual reality (VR). I would, therefore, develop a VR version and a MR version and compare these.

The targets to achieve boiled down to:

• Develop a VR and MR flight simulator in Unreal Engine, being as realistic as possible
• Determine if there is a difference in cybersickness between the two through experiments on users

While I was relatively new to Unreal Engine, I had dedicated the Christmas Holidays to learning its basics prior to getting invited to the FLSC. Given the project's scope, creating a realistic flight simulator from scratch was impractical. Instead, I discovered a helpful guide on building a flight simulator (here). This resource allowed me to swiftly incorporate realistic terrain generation via Cesium and realistic flight dynamics physics using JSBSim. Although I explored ArcGIS as an alternative for terrain generation, I encountered issues and ultimately opted for Cesium.

A quickly developed proof-of-concept.

A lot more work had to be done to make the simulator work with the Varjo XR-3 headset. The Varjo developer documentation helped a lot (although their support for Unity was more extensive). They had a plugin for Unreal where a lot of their features were implemented as examples. The FLSC also had a cool cockpit platform with realistic JAS 39 Gripen joysticks which I interfaced with Unreal, using the plugin WM Input Manager, since Unreal Engine could not read XINPUT data natively. There were a lot of issues that was duct taped as well, which hurts as a perfectionist, but it was just a demo simulator I needed to get done quickly.

To determine differences in cybersickness, I designed a experiment where participants would carry out certain tasks. I measured their cybersickness using a well-established self-report instrument, called the Simulator Sickness Questionnaire. Data from this instrument (and one other) formed the base data for statistical analyses I performed in SPSS.

The flight simulator was completed in approximately 8 weeks, allowing me to proceed with conducting experiments involving participants. You can view demonstrations of both the VR and MR simulators in the videos below.

I recruited 29 volunteers who participated in two experimental sessions, each conducted on separate days. One session took place in virtual reality (VR), while the other occurred in mixed reality (MR). A particularly noteworthy finding was that cybersickness appeared to be less severe in the MR sessions. This suggests that FLSC (and other simulation centers) should consider prioritizing the development of MR flight simulators as a means to reduce cybersickness. The final report received positive feedback from both FLSC researchers and individuals at my university who had the opportunity to review it.

(You can download the pdf [here].)

Working on this project was both immensely rewarding and enjoyable. Drawing from my prior experiences in the master's program, developing different applications, and writing several reports, I personally believe this thesis stands out among my works, for its thorough research, written report, and the quickly developed flight simulator it encompasses.

I exceeded the intended duration of 20 weeks by approximately 5 weeks, even though I was consistently working more than 40 hours per week. Reflecting on the project, one aspect I would reconsider is the experiment procedure. Originally, I structured it in a way that required me to handle multiple simultaneous tasks and a significant amount of manual work. This approach made conducting the sessions incredibly demanding and prolonged the process beyond the intended timeline. Doing it again, I would aim to streamline and automate processes to enhance efficiency. Even if it initially would require a bit more of my time.