Interaction Methods for Virtual Reality Applications

Monday, August 03 2009

The potential of Virtual Reality (VR) technology has not been fully realized because the user interfaces are not designed to effectively support the user. One of the most critical interface aspects is the possibility to interact with the system within the virtual environment. The initial idea of optimizing the user interaction was to transfer the interaction that occurs in the real world as accurately as possible into the VR environment. In this case the user, theoretically, would have no problems interchanging between the two worlds.

But, VR technology is not yet mature enough to do this. Some elements of the real-world interaction, such as haptic feedback, cannot be transferred into the VR environment, and others only are transferred with insufficient precision. Examples of common problem areas are haptic feedback, operation of controls requiring fine motor skills (e.g. rotary controls), matching the movement of a real person and their virtual representation, and visualization quality.

Differences Between VR and Reality

It is normally not possible to generate an exact copy of the real world. Therefore, fundamental differences exist between the real and the virtual worlds. To illustrate those differences, imagine the simple example of holding a ball in two hands (Figure 1). In reality, a ball is kept hold of with two hands, the position of the hands is determined by the surface of the ball, and control of the ball is provided by the haptic and visual senses of the human. In a virtual environment, where no haptic feedback is provided, the control of the ball must exclusively be accomplished by the visual system.

There are several aspects that make it much more difficult to control the ball in the virtual environment than in reality:

• The hands seen by the user are not his own hands, but more or less realistic reproductions.
• The ball is also only a reproduction.
• The positional quality of the reproduction of both the ball and the hands is in most cases not perfect.
• The user is not really holding a ball (in fact, he or she is not holding anything); the user has to position his real hands according to the visual movement of the reproduction of his hands. This causes a completely different stimulation of his muscles than with a real ball (no weight and no counterforces from the ball).

A task that is simple in reality turns out to be very challenging in a VR environment, and the only way to make the user able to deal with these issues is through training. Where noticeable differences exist between reality and the virtual world that cannot be overcome, it is not worthwhile to put a lot of effort into the design of interaction methods that try to mirror reality as exactly as possible. In these cases, other methods of interaction should be taken into consideration. Examples from aircraft design are given below.