>Education, Engineering, Computer Science, Technology

Embodied Virtual Reality for Training and Performance

Embodied Virtual Reality for Training and Performance

Update: A recent major milestone has been met to implement a realistic play scenarios, with consultation from a VT quarterback coach, that can simulate the many two play options a quarterback might face.

This proposal sought to perform a feasibility study for a fully immersive (proprioceptive/kinesthetic, physiological, visual, auditory) virtual environment that leads to the development of a flexible platform for embodied athletic
training and psychomotor response measurement. The context for the training simulator was the decision-making process of an American football quarterback. The goal of this proposed feasibility study was to develop a viable prototype of a system that meets the basic requirements outlined below, thereby proving that further development of a robust system for a variety of studies is a viable outcome. The following technical requirements have been addressed in creating the prototype:

  1. Player motion capture, character modeling, rigging and animation
  2. Trainee/participant and ball tracking, avatar and ball modeling
  3. Voice recognition for scenario selection, start/stop
  4. Player physiological measurements and models
  5. Immersive sound based on Lane Stadium recordings
  6. HMD mounted in the helmet for the football task
  7. Plays with CG offensive and defensive players modeled, rigged, and animated based on known playbook and opponent tendencies

The technology in our system includes a VR headset , an immersive audio display, a high-end motion capture system, and custom software to allow for play selection, immersive simulation, and post-action debriefing. The system our team is building is high-fidelity, highly flexible, and allows full physical performance of the quarterback’s task.

The existing simulator is being prototyped in the ICAT Cube with its state-of-the-art systems for motion tracking (to track the quarterback and ball) and spatial audio (to simulate crowd noise, possibly affecting player performance). The intended use-case for commercialization would have the trainer/simulator installed in an indoor athletics practice facility or gymnasium.

Accomplishments to date:

  • Interviewed training personnel at VT Football for design exploration
  • Basic trainee and ball tracking has been tested successfully in the Cube
  • Primary 3D assets are complete, with 90% of animations complete
  • Scenario for demo designed
  • Biometric data collection programming complete and integrated into Unity code
  • Unity game code functional with first person view and ball tracking working, biometric data integrated for “coach” view, basic start and stop functionality working

Near term next steps include:

  • Live testing of full speed throws in the Cube
  • UV mapping of fully animated player characters complete by end of Fall semester

Looking forward, we plan to:

  1. Join the NSF iCorps Regional Node program (alternatively, we may join the ICAP Short Course offered through the SBDC)
  2. Complete submission of a provisional patent application through VTIP
  3. Perform customer discovery (and potentially additional subject matter expert support) from VT Athletics.

Submissions for publication:
None to date, as we are seeking to patent protect aspects of the system.

Brief discussion in demos with industry partners via the Center for Human-Computer Interaction

ICAT Innovation Day, April 30, 2018

Student Involvement:
More than 12 graduate students have participated in the project, one via a 1-semester full assistantship, two as volunteers either with their professor or as a result of prior course involvement, and the remainder through course projects in virtual environments and wearable computing courses.

Educational components (K-12):

Supplemental resources used to complete the project:

Media coverage:
NA, as we are seeking commercialization.

Anticipated external funding which may result from this project:
We intend to seek NSF SBIR funding or private investment in the future, based on how our iCorps participation proceeds.

Todd Ogle, Library
Doug Bowman, CHCI/Computer Science
Robin Queen, Biomedical Engineering and Mechanics
Nathan Lau, Industrial and Systems Engineering
Stefan Duma, Institute for Critical Technology and Applied Science.

Collaborative Colleges:
Biomedical Engineering and Mechanics
Industrial and Systems Engineering
Institute for Critical Technology and Applied Science

Computer Science