Like very few other infrastructure projects, "Mirror Worlds" has the potential to connect people around the world (researchers, educators, students, and the general public) to computational research on human behavior, interaction, and understanding.

Mirror Worlds is an interdisciplinary project focused on creating a computing infrastructure for research across physical and digital environments. Each environment within Mirror Worlds is a two part system composed of a physical room/building and a coexisting virtual model of that physical space. Users of an environment are not just visitors to a physical space or virtual model, but are also participants whose behavior influences the environments’ forms and functions. The virtual and physical aspects of the environment are connected with one another through a coordinate-based data networking system to enable user interaction with the rest of the system including environment objects and other users.

A project funded by the National Science Foundation, Mirror Worlds researches human interaction by studying behavior and emotion in both the physical and virtual environment. Sensors placed around the Moss Arts Center track people in the physical space and then generate a digital version of those people at those same locations in a virtual building. People will also be able to access the virtual model of the building online and see both online visitors and the tracked representations of real people currently in the physical building. The project hopes to create a shared space, with portals between the physical and virtual, where people can interact with each other and explore the Moss Arts Center.

The Mirror Worlds infrastructure provides the foundation for experimentation and research on a wide variety of topics. Specific areas of interest include parallels between behavior in real and virtual spaces, the effects of fidelity in virtual environments on performance in physical environments, crowd model simulation and evaluation, the relationship between behavior and affect in a public venue, and computational models of energy-efficient buildings.

Interactions

People interact with Mirror Worlds and engage with environments by both providing and receiving data. As someone physically moves through a building, their location data is extracted using cameras placed throughout the building and computer vision software. This data is reduced to a string of information (rather than raw image) that can then be communicated with a central server. A person’s presence in a physical space can also be detected aurally using microphones and audio triangulation algorithms. In virtual environments, their location information is mapped by the view they are currently experiencing.

Displays in both physical and virtual spaces allow for people to receive information about the environment. Televisions and tablet displays placed throughout the building create “mirrors” from a physical environment and reflect the virtual counterpart back to the viewer. People present in a physical space can also be identified virtually as avatars that have a unique shape or color, and people in a virtual space can be represented in physical spaces through floor projections.

Broader Impact

The project brings together computer scientists, architects, and humanities researchers to address a variety of cross-disciplinary research questions about digital and physical environments and the activities that happen in those spaces. Through these explorations, the project will increase opportunities for a diverse group of people to discover, learn about, and participate in a variety of fields through these collaborative research questions. The project will have lasting benefits to society through its exploration of distance learning, human experience, and social interaction.

This material is based upon work supported by the National Science Foundation under Grant No. 1305231. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.