From: "STARS::PSOTKA" Subject: SCI: Immersion and Vestibular System There have been frequent comments on this list about the control of depth of immersion by many visual and imaginative factors: detail and texture of the object models; stereo; all the depth cues; shading; color; etc, etc. Now there has even been a report about the effectiveness of the number of immersion levels, the number of virtual HMDs donned in a VW. Surely this ought to fall in the same category as excitement, interactivity, involvement, and willingness to suspend disbelief. Another constant star in thsi firmament of discussion is lag and accuracy of 3-D tracking. The better the accurcay, and the smaller the lag; so it goes the better the immersion. Yet, how does this work? What are the processes behind this kind of HMD immersion? What does moving your head (yaw, pitch, and roll) contirbute? How does this compare with linear motion (zoom, dolly up or across)? There are many factors to untangle her; one being the cognitive/perceptual factors versus sensory. For instance, when you move your head, you KNOW you are generally doing it, if it is planful rather than passive. But many sensory systems provide feedback: proprioceptive senses from your skin (the collar will rub against it), muscles (stretching and twisting - also kinaesthetic from special sensors in the ligaments and muscles); and then there are a special set of sensors called the vestibular system. They are really acomplex set of sensors that detect both angualr and linear motion. Rotational motion is mainly sensed by three sets of canals on each side of the head in the inner ear. Each set of conals is oriented in all 3 directions of x, y, and z axes. They detect acceleration by letting fluid flow through the circular canal against a trap door. It is easiest to visualize this with a drawing of these semicircular canals, so look them up if you want to understand it better. Linear acceleration is sensed by otoliths (small ear stones) in some sensors right near by. >From some experiments underway here, it is beginning to look like these semicircular canals are really important for immersion to take place. We do "cognitive tracking" experiments, asking people to play camera by moving a Hanging Omin-Orientational Display (HOOD) to synchronize with changes in the visual scene on the display. Only if they are able to synchronize quite precisiely with the visual changes do they feel deeply immersed. This seems to begin to rule out visual, cognitive, proprioceptive, and kinaesthetic factors in the control of depth of immersion; and place the burden of visual immersion on the lowly vestibular system. It seems strange that such a powerful and centrally cognitive state as immersion is so profoundly influenced by a lowly mechanical system, but that seems inescapable. The relaitve influence of linear and rotational motion seems to favor rotational motion as more influential on immersion. Cognitive tracking appears to be a very frutiful paradigm for investigating HMD visual immersion.