understanding immersivity: image generation and transformation processes in 3d immersive environments

Most research on three-dimensional (3D) visual-spatial processing has been conducted using traditional non-immersive 2D displays. Here we investigated how individuals generate and transform mental images within 3D immersive virtual environments, in which the viewers perceive themselves as being surrounded by a 3D world. In Experiment 1, we compared participants’ performance on the Shepard & Metzler (1971) mental rotation task across the following three types of visual presentation environments; traditional 2D non-immersive (2DNI), 3D non-immersive (3DNI - anaglyphic glasses), and 3D immersive (3DI – head mounted display with position and head orientation tracking). In Experiment 2, we examined how the use of different backgrounds affected mental rotation processes within the 3D immersive environment. In Experiment 3, we compared electroencephalogram (EEG) data recorded while participants were mentally rotating visual-spatial images presented in 3DI vs. 2DNI environments. Overall, the findings of the three experiments suggest that visual-spatial processing is different in immersive and non-immersive environments, and that immersive environments may require different image encoding and transformation strategies than the two other non-immersive environments. Specifically, in a non-immersive environment, participants may utilize a scene-based frame of reference and allocentric encoding whereas immersive environments may encourage the use of a viewer-centered frame of reference and egocentric encoding. These findings also suggest that mental rotation performed in laboratory conditions using a traditional 2D computer screen may not reflect spatial processing as it would occur in the real world.