Embodied Virtual Reality for Training and Performance

This proposal sought to perform a feasibility study for a fully immersive (proprioceptive/kinesthetic, physiological, visual, auditory) virtual environment that leads to the development of a flexible platform for embodied athletic training and psychomotor response measurement. The context for the training simulator was the decision-making process of an American football quarterback. The goal of this proposed feasibility study was to develop a viable prototype of a system that meets the basic requirements outlined below, thereby proving that further development of a robust system for a variety of studies is a viable outcome. The following technical requirements have been addressed in creating the prototype:

1. Player motion capture, character modeling, rigging and animation
2. Trainee/participant and ball tracking, avatar and ball modeling
3. Voice recognition for scenario selection, start/stop
4. Player physiological measurements and models
5. Immersive sound based on Lane Stadium recordings
6. HMD mounted in the helmet for the football task
7. Plays with CG offensive and defensive players modeled, rigged, and animated based on known playbook and opponent tendencies

The technology in our system includes a VR headset , an immersive audio display, a high-end motion capture system, and custom software to allow for play selection, immersive simulation, and post-action debriefing. The system our team is building is high-fidelity, highly flexible, and allows full physical performance of the quarterback’s task.

The existing simulator is being prototyped in the ICAT Cube with its state-of-the-art systems for motion tracking (to track the quarterback and ball) and spatial audio (to simulate crowd noise, possibly affecting player performance). The intended use-case for commercialization would have the trainer/simulator installed in an indoor athletics practice facility or gymnasium.