How big an issue is the nausea problem for Virtual Reality products?

Very big. Or should I say was very big because here’s a hot tip; 2016 is the year the last VR nausea problem was solved.

Let’s first be absolutely clear about the source of the issue so we can understand how it was mitigated, to what degree, and also judge how much remaining discomfort is “acceptable” for VR to be adopted as a legitimate consumer platform.

Vestibular Disconnect:

99% of the discomfort and nausea associated with VR stems from this one phenomena. As we move about in waking life, the eye constantly perceives our changing surroundings whilst the brain subconsciously checks this data matches up with the motion of our limbs and the jostle of fluid in the inner ear (the vestibular balance apparatus). Even when we travel in a vehicle, the scenery rushing past our view is matched by the g-forces stimulating this inner ear ‘accelerometer’. When the signals received from the eye conflict with motions experienced by the body (a common experience with children focused on reading or playing handheld games in a moving car), the brain registers a disconnect and assumes something is wrong. In nature, we encounter this disconnect as an early symptom of chemical intoxication (i.e. eating bad berries) and the body’s natural defence mechanism is to try to vomit and purge itself of the toxic contents. Thus nausea.

Type 1 Nausea and VR:

We run afoul of the above phenomena in VR in two seperate circumstances. The first is passive scene viewing. When you put on a VR headset and look around, the computer sends images of the virtual scene to the head mounted display (HMD), constantly updating it to correspond to your changing perspective. If the computer delays in sending the coresponding image (lag) or incorrectly judges the movements of the user and sends the wrong image (loss of tracking), or the experience shifts the perspective itself (poor content design), the brain begins to perceive the disconnect and VR sickness kicks in. Thankfully 99% of the population all share roughly the same tolerance levels for these triggers. Not only do we know what they are, modern computational power, display technology and scene rendering technologies have all coalesced at a price, performance and form-factor that meet the necessary standards to fool our biology. This alone kicked off the modern VR Renaissance we are all seeing.

Type 2 Nausea (this is the interesting part, directly impacting VR’s successful mass adoption):

Static or passive enjoyment of virtual worlds has been largely solved. The problem with the VR market now is that people want to actively explore and enjoy virtual scenes. That means locomotion. Traveling through and around virtual landscapes, walking up to and touching things. Thanks to ‘roomscale’ VR platforms such as the HTC Vive, we understand that as long as a users movements are faithfully tracked at 1:1 accuracy (one step forward in real life is mirrored by one step forward in the VR scence), no nausea is experienced. This however also presents a clear problem: If every step in VR must be matched by a step in real life, how are people able to explore large virtual environments when restricted by relatively small living spaces? Traditional gaming platforms acheive this by pressing buttons or moving joysticks for movement but when attempted in VR, the eye perceives all the motion while the body experiences none of the limb motion or vestibular stimulation and severe nausea is often the result, even among the experienced.

Most early VR experiences are relying on point and click ‘teleportation’ as a method to explore larger scenes. Clever environment design also helps where the minimum amount of space is laid out in such a way that users feel less limited than they actually are (redirected walking). In the meantime, many people are trying other solutions to bridge the gap between virtual world exploration and the confines of real world space e.g:

• Purpose built treadmills
• Foot based peripherals
• Special headphones that mimic inner ear stimulation
• Dedicated play centres catering to large scale VR experiences

While many of these ideas have their own merits, a true mass market solution can only depend the on the actual hardware and peripherals sold as standard with simple VR kits like the HTC Vive or Oculus Rift. It must depend on standard equipment to maintain a non-fractured userbase so that developers can roll out content for a mass audience.

The name Dewhirst may one day be famous for this solution: It strikes a good balance between borrowing from real world body mechanics and the forgivenesses of virtual locomotion. So far testing shows massive improvements in solving VR nausea. Again, it seems another threshold that must be met to fool our biology has been met.

Aside from comfortable movement, there will always be a place for motion-sickness and stomach turning in VR when it’s intentional. Vertigo while standing on a virtual rooftop or queasiness riding a virtual rollercoaster can be a thrilling and valuable part of an experience. It is only where nausea is imparted as an unintentional side effect that VR suffers. Thankfully much of the hardwork has been done by engineers, display experts, graphical processor producers and coders. All that remains now is to align the current hardware and software in a configuration that hits all the big targets so the fun can begin.

It is a significant problem, but not a show-stopper for VR in general. It limits possible use of VR with current technology, but there are ways to work around it for many contexts and more advanced solutions to address the fundamental problems are in development.

I have been working with VR in different ways since around 2000. I have developed VR applications as a programmer during this period, I have a PhD focusing on the connection between presence in VR and human brain function and I currently work with VR and Mixed Reality research and teaching in relation to Human-Computer Interaction and Interaction Design. I have several HMDs at home.

Primary sources of cyber sickness are:

1. Latency in the head tracking.
2. Conflicting motion in VR and the physical world.
3. Conflicting visual input from optics.

1 Latency

There is a large degree of individual variation in how sensitive people are to these issues, but with a high-end PC and a recent commercial HMD #1 should be negligible for most people. For example, the latency of the Oculus Rift DK2 tracking is 2 ms and by using asynchronous time warp motion-to-photon latency has been measured to around 5 ms. (Thanks to Doc Ok (Oliver Kreylos) for the research on this).

Oculus Rift DK2’s tracking update rate

A Trip Down the Graphics Pipeline

With trivial graphics and some advanced tricks motion-to-photon latency can approach 1 ms.

Zero-latency Rendering

Compare this to the traditional gold standard of 20 ms latency. Of course, it’s still easy to run into issues with latency if you are not careful, but the problem is very well know, there are solutions and a lot of money is being spent on implementing them.

2 Motion

#2 should be where you have the most variation in sensitivity currently, where some people can stomach quite a lot while others are very sensitive. It is certainly a fundamental problem for applications such as flight simulators or FPS games, but if you check the currently available VR games on Steam you will see that the most popular ones sidestep #2 completely by relying on a combination of real-world walking/movement and teleportation. To be clear, the VR applications that are primarily being pushed in reality right now are qualitatively and importantly different from traditional VR applications such as flight simulators and similar. Developers are well aware of the problem and they are working around it.

Prominent examples of current games:

Skipping virtual motion completely is the only way to avoid nausea for as many as possible but there are many tricks that can be used to reduce the chance that users feel nausea and to allow a large portion of users to move around in a virtual space. Some examples are:

Threadmills - reducing the conflict between proprioception and virtual motion.

Cockpits etc (mentioned in other answers here). Elite Dangerous is a great example of this that I know that many are able to spend many hours in without problems:

Manipulating Field of View to reduce nausea when accelerating:

Ubisoft's VR Movement System Looks Incredible

A note on the dangers of cyber sickness and car driving: I think a lot of people would forgo driving for a while if that was necessary for VR experiences. I know that some parts of US are very adapted to driving around in cars, but that’s not the world. Certainly many cities have large populations that do not own a car at all, and I think it is very likely that self-driving cars will be commonplace within 10 years. But that’s a side-note. :)

3 Optics

While recent HMDs employ custom combinations of lenses and low-latency displays made specifically for VR there are still no solutions to the fundamental vergence/accommodation conflict that results from having to focus on an actual screen. At the moment, however, I would put this into the comfort category as a limitation on how long most people would feel comfortable using VR. I think this is certainly a significant limitation with the current technology, but one that will be addressed in the near future. For the vergence/accommodation conflict we need something like what Magic Leap are claiming to have in the works.

Magic Leap: just why is this super secretive tech company valued at $4.5 billion?

Finally

That’s scratching the surface. Suffice it to say, people are on this. Personally, I am a strong believer in the potential of Mixed Reality and in the power of grounding people in the physical world and enabling tangible interaction with familiar objects. But this answer is long enough already so I will not go further into that now.

The Oculus Documentation has a pretty solid description of Simulator Sickness etc. It certainly demonstrates that these companies are well aware of the issue.

Simulator Sickness

I’m fully convinced it’s a huge problem.

When I demoed the Oculus Rift at GDC I already had one on order. They told me if you start to feel nauseous take it off right away because it won’t get any better. Wait until the next day and then try it again. It should become less and less of a problem as time goes by. They said that people usually start getting nauseous around 15 minutes, but with repeated use, it would get longer, and longer until it went away.

My experience was the opposite. It took about 15 minutes to get nauseous the first time and it was terrible. I had to lay down for an hour for it to go away. The next time was about 12 minutes, then 8, then 5.

I got to the point where I was literally feeling sick before I even put it on. I’d open the clasps on the box, and start feeling it.

Initially I thought it might be the DK1, and the DK2 would be better. Nope, I had to work with one on a project… and it got to the point where I refused to wear it.

I’ve tried other VR products since and it’s the same thing.

I guess I’m just one of those people who can’t do VR. It’s terrible because I was super excited about it, I was an early adopter.

I refuse to put a VR headset on now.

If it happens to me it’s going to happen to other people. I can tell you more than an hour of laying down feeling like you’re going vomit isn’t something your average consumer is going to be happy about.

It’s a big problem. Not only VR has a difficult quest to adopt into the people’s routines, it will turn away first time viewers of VR, if they experience and connect nausea or sickness with VR. It’s hard to make yourself go back in there and play/watch a VR content because your body associates the game with feeling sick.

I ate an old BBQ-ed eggplant once, and I would stay away from this vegetable whenever possible. Serisouly, how can people like it. It might be stupid and immature, but majority of curious mind will stay away from an initial experience.

There are quite a lot of explanations what causes sickness. I’ll try to mention the most common one and theories how to prevent them.

a) Information Overload & Novelty Shock - we have been consuming virtual media through rectangular flat screens for dozens of years and we have gotten used to it.

I’ve written a couple of tips on how to tackle storytelling in VR with a special stress on easing your viewers into the experience. Actaully, the “emergent cardinal rule of VR development seems to be, under no circumstances take the control of the camera away from the viewer – an alteration of viewpoint for someone whose head remains static is an instant one-way ticket to nausea.”

b) Balance - This is arguably the most often cause of sickness. Because VR tricks your brain to accept VR as a temporary reality, every annomality that doesn’t occur in “real reality” confuses the viewer in VR. This sensory conflict states that dissonance between what your eyes see on screen and the kind of motion your body feels lead to disorientation and feelings of nausea.

You’re probably aware of vestibular system (the tubes of liquid in our ears that help us to gauge our position in the world) that’s sits in our ear and is responsible for balance. So imagine that you’re in a VR rollercoaster and you’re just about to hit a downward spiral. Your eyes register the accelerated decline but the vestibular system remains unchanged.

I’m one of those people who gets sick if I read in the car. It’s the same principle. You’re focusing on Atlas Shrugged with visual system telling your brain you’re still but in fact the car driver is making S-turns on the mountain road.

Some funny solutions were adding a virtual nose in the VR content or adding a frame making brain think you’re in a moving vehicle even though you’re actually sitting still.

c) Picture Quality

Unless you have some sort of condition, the world you’re receiving with your eyes isn’t grainy. For example when Oculus Rift DK1 was introduced, most of the people trying it experiences sickness. The problem was the LCD, which was replaced by better one - OLED.

“The refresh rate of on-screen images is often not high enough when VR sickness occurs. Because the refresh rate is slower than what the brain processes, it causes a discord between the processing rate and the refresh rate, which causes the user to perceive glitches on the screen. When these two components do match up, it can cause the user to experience the same feelings as simulator and motion sickness which is mentioned below.” - Wikipedia

d) Relative Sickness Resilience - Sickness is felt different by different people. If you’re someone who gets sick on airplanes or ships, chances are VR will get you sick as well.

Quite a lot of people get sick in cinema’s as well. Few years ago, there was a movie that caused so much sickness in the audience that it was recognized by big news. Cloverfield, a 2008 US Horror movie used jerky hand-held camera footage that makes up most of film (remember Blair Witch Project). The problem was so bad some cinema chains have posted signs warning audiences they could "experience side effects associated with motion sickness, similar to riding a rollercoaster."

I’m afraid there’s little you can do here. A redditor who used sickness medications said:

“They can help a little but not for everyone and not for severe cases. The best way is to never fight the feeling. The second you feel odd, just get out and chill. VR isn't going anywhere so take it slow.”

My suggestion is to stop the video and get out of VR mode, until you relax and get better. If a certain content makes you sick, it’s probably best to skip it altogether or try it after a few weeks.

The closest solution for curing VR sickness is coming from Mayo Clinic. The hospital has developed a couple of electrodes that are placed behind each ear, which “trick the brain into syncing what you're seeing in VR with physical stimulation within a tenth of a second.”

But here’s the major issue. The more you have to add “technology” which requires more preparation, more money and sacrificing comfortability to experience each VR content, the more significant drop off users VR is going to suffer.

I believe the sickness solution must come from content itself. The quality has to be good enough, the stitching mistakes has to go away and the storytelling and cinematography has to be adapted to the medium and tested before hand.

On other hand, while VR is battling these annoyances, AR and MR are gaining ground and making its presence known.

Well, it’s a pretty big problem.

My university teaches a lot of pilots who go on to fly for airlines. They actually get their FAA Commercial license through their coursework. Now obviously, this doesn’t require the use of a CRJ full-flight simulator, but it definitely helps build up resume bullets. We also have a number of simulators on campus and each one is designed specifically to minimize simulator sickness.

In fact, we have an entire simulator designed to show pilots all the ways they can get disoriented (which frequently comes with nausea) in an aircraft/simulator. My favorite one, I can’t remember what it’s called, involved the aircraft being ordered into the pattern opposite its direction of travel. This wouldn’t ever happen, it’s a controller screw up. You end up in a standard right hand turn for about a minute. Then the test conductor tells you to reach down and pick something up off of the aircraft’s floor.

WHOO! You normally have to pay for a swerve like that!

All of the motion simulators have SIM STOP buttons, and the location and operation of the SIM STOP button is a critical part of the pre-flight briefing. It’s a half a million dollar simulator. Cleaning out chunks is expensive and time consuming. Even the fixed-base simulators have barf bags within reach.

Simulators are a form of virtual reality, by the way, and I point out this whole thing above to illustrate that it’s partially an issue of balance.

The problem with virtual reality is that it enables a conflict of sensory input. The brain is visually dominant, vision is by a wide margin our sharpest sense, and so it reacts most strongly to the visual sensory input. So, when you put on a VR system, as far as your brain is concerned you are very much in that world. Your inner ear may disagree, though, and when it does…

The reason motion/simulator sickness occurs is because your brain is trying to resolve this errant proprioceptory and visual data. You can be flying through the air or even space or leaping off a cliff visually but if your inner ear doesn’t receive that same signal, trouble follows on swift wings. Your body thinks it’s hallucinating. Which is a symptom of poisoning. So it sends the signal to your stomach to void that sucker, which it happily obliges.

The inner ear has three fluid filled tubes that help it determine orientation based on the fluid’s contact with sensory organs inside the tubes called otoliths. The way the fluid contacts these otoliths determines your perceived orientation in space. If that fluid is a floating ball inside the ear, as it would be in orbit or other microgravity environments, it may contact otoliths randomly providing wild and unpredictable proprioceptory inputs that your brain can’t make sense of against the visual input. The resulting nausea that can be extreme. So extreme, in fact, that no extravehicular activities are in the flight plan for the first three days of any spaceflight because that’s about how long it takes the body to learn to disregard the errant inputs.

Some people are more susceptible to simulation sickness than others, but most people have a threshold and simulation systems have to be carefully designed to not exceed that threshold.

Many aircraft or closed simulations achieve this by tricking the body into feeling the appropriate inputs. For example, full flight simulators frequently induce a slow, subtle pitch up below the sensory threshold so the occupants can’t feel it (this is called a sensory washout filter), but what they do feel is an increased force from what they perceive as the forward direction. That’s how you simulate acceleration on a takeoff run without actually accelerating the simulator. Similarly, if you increase your roll beyond a certain level, the simulator has trouble keeping up with that and will right itself below the washout filter and then pitch over some more. It’s very fascinating stuff.

So without the false vestibular inputs, nausea in VR will continue to be an issue that will negatively affect the user experience.

You might think this is a recent identified problem. It’s not. You can play with some of this at the Exploratium and maybe the Technorama in CH. They have simple academic tests for disorientation.

I’m aware of an minor incident in a classified facility documented from the 1960s in Washington DC (the building was just knocked down; this information was declassified in the 1990s, and I don’t have nor want a security clearance):

Edwin Land of the Polaroid Corp. (he’s responsible for one of the most used optical systems) suggested polarizing lenses (turn your glasses 90 degrees (at screen or outside)). But when it came time in those early crude days of 3-D vision systems in certain windowless labs, he discovered for himself that his ideas needed a little more work. Back to the drawing board.

Even going back to the 1920s when motion pictures (2-D) were first released people thought the train movies had the train headed right at them (keep how this was done to yourself (special effects and clever editing)). And some of this is cultural. To this day, a small number of people can’t watch motion pictures (the Phi-phenomena) in theaters. This is irksome to one girlfriend, and she waits for it on the small screen.

The first head mounted VR system (with Pop Optics out of MA) that I was a guinea pig for is now in the National Air and Space Museum (most people associate Scott Fisher with this system, but it was Mike McGreevy who deserves the credit; what a hack job story! it was a great demo). BTW: you don’t need high resolution. His first system was 100x100 B&W pixels (driven by 2 unused E&S Picture Systems). Many managers did not “get it.” I don’t know if they felt sick or lacked the managerial vision needed to see the demo (common; Edwin Land found this out on the first 3-D movie preview for Sam Goldwyn (a lesson every person working in 3-D must know)). And I also saw the SciAm photos from Ivan Sutherland’s 1968 article on computer graphics which Fred Brooks does work on haptic interfaces. Let me know if you want the photo of it at the NASM here: (photos take up more bandwidth than is necessary), not included at this time.

I’m a trained photogrammetrist and can sit viewing 3-D for days. This was something the well intended but clueless guys who wrote the Sci-Viz report in the late 1980s missed (the language guys don’t even have the term photogrammetry in their vocabulary or spelling dictionaries (try it) and wikipedia has it poorly covered). So we are aware of this problem in the field (some preliminary students weed themselves out and become photointerpreters).