Most research on realistic visual-spatial processing has been conducted using traditional non-immersive 2D displays. The stimuli were presented on a traditional computer screen or another flat surface, which defines a fixed frame of reference. The stimuli were viewed from “outside” the scene in which they are presented. This might lead to allocentric encoding. We investigate how individuals generate and transform mental images within 3D immersive (3DI) virtual environments, which are similar to the rea-world, where the viewers perceive themselves as being surrounded by a 3D world. In such immersive environments, an observer constitutes a part of the scene and is “immersed” in it, which might lead to egocentric encoding.

Our study is the first attempt to understand immersivity from a cognitive neuroscience perspective. In our experiments, we compared participants’ performance on the Shepard and Metzler (1971) mental rotation (MR) task across the following three types of visual presentation environments; traditional 2D non-immersive (2DNI), 3D non-immersive (3DNI – anaglyphic glasses), and 3DI (head mounted display with position and head orientation tracking). We also compared electroencephalogram (EEG) data recorded while participants were mentally rotating visual-spatial images presented in 3DI vs. 2DNI environments.

Overall, the findings of the three experiments suggest that visual-spatial processing is different in immersive and non-immersive environments, and that immersive environments may require different image encoding and transformation strategies than the two other non-immersive environments. Specifically, in a non-immersive environment, participants may utilize a scene-based frame of reference and allocentric encoding whereas immersive environments may encourage the use of a viewer-centered frame of reference and egocentric encoding.

These findings also suggest that MR performed in laboratory conditions using a traditional 2D computer screen may not reflect spatial processing as it would occur in the real world. Finally, the results of our ERP study demonstrate that 2DNI and 3DI environments do evoke differential parietal ERP response. One interpretation is that this early modulation of ERP activity marks egocentric vs. allocentric scene-based orienting in preparation for subsequent mental rotation from 400ms onwards.

Relevant research articles: