Interactive Realities Lab

Audio Research

Wednesday, 28 June 2006

Audio is an essential component of any effective simulation, whether for training, education, or entertainment.  The sense of hearing extends 360 degrees around the user and can alert him or her to events that happen outside the visual field of view.  Additionally, a properly rendered audio environment can contribute to the user’s sense of presence with cues that suggest the size and nature of the virtual environment.  For example, an empty warehouse sounds very different from a furnished living room, however most interactive simulations ignore this fact. Evolution of Interactive Audio In the past few years, PC audio hardware, driven by the computer gaming industry, has evolved significantly.  New audio products can provide 5.1- and 7.1-channel surround sound and digital signal processing capability for more accurate acoustic environment modeling and rendering.  Virtual reality simulations can benefit greatly from this advanced technology.  The University of Central Florida Institute for Simulation and Training (IST) is beginning a research effort to bring these advanced audio capabilities to its virtual environment applications and platforms, and to help advance the state of the art in interactive audio.  This research will be conducted by the Audio Simulation Laboratory (ASL), a co-lab of IST’s Interactive Realities Laboratory (IRL) and Media Convergence Laboratory (MCL). Existing Capabilities IST/IRL has created a virtual reality library known as VESS (the Virtual Environment Software Sandbox), which serves as a basis for many of virtual environment applications.  VESS includes a Sound Library, based on the cross-platform OpenAL library.  With VESS, an application developer can create sound sources and directly attach them to objects in the virtual environment.  As the virtual object moves around, the attached sound will automatically move with it.  Similarly, the user’s listening point can be attached to the scene (on the user’s avatar or at any other point).  Audio data can come from digital audio (.wav) files or from any audio stream, such as a microphone or a network source.   Sounds can be looped (for ambient effects) or can be triggered by events in the simulation. ASL Research The initial focus of the ASL’s interactive audio research is to tap into the audio capabilities available to the gaming industry and bring it into IST’s existing virtual environment applications and software.  The goal of this work is to create a baseline of software to support more advanced work and to identify how existing audio Application Programmer Interfaces (API’s) can work with virtual environment software.  A large part of this initial work is aimed at creating an Acoustic Geometry Interface (AGI) for controlling the environmental audio parameters.  This interface will create the necessary acoustic settings to properly render the aural environment based solely on a geometric description of the virtual world.  This will significantly reduce the amount of work necessary for sound design (the process of modeling the acoustic environment). Initially, this research will make use of the advanced hardware capabilities available on the Microsoft Windows platform (using Creative’s EAX API).  However, since many of IST’s virtual environment applications use Linux, the ASL is also working to bring some of these advanced audio capabilities to the Linux platform as well. Future work will include the geometric modeling of early acoustic reflections and diffraction.   This will provide a much more accurate model of sound traveling through the virtual environment, allowing multiple propagation paths to be rendered and heard by the listener.  The ultimate goal of this will be to create a “complete” sound system, providing geometrically-driven acoustic reflections with more conventional late reverberation.  Along with the basic audio source and listener capabilities provided by VESS, this will create a complete audio system, suitable for any virtual reality application.