CHI ’21: The 2021 ACM CHI Conference on Human Factors in Computing Systems
Session: Late-Breaking Work Day 3 – D
Recent years have seen an explosion in eXtended Reality (XR) experiences. Until now, researchers and developers were limited to relatively expensive, closed-source, consumer level hardware when creating high fidelity XR experiences, as these were the only devices that can create a high fidelity XR experience out of the box with minimal setup times. In this paper we present Project Esky, a complete open-source modular software framework that enables the rapid creation of high fidelity XR experiences on any combination of display and tracker hardware. Our framework includes several components that handle 6DOF head tracking with head pose prediction to minimize latency, hand interactions with virtual content, and network components to facilitate between-user co-located experiences. Our framework also includes an asynchronous rendering pipeline that simplifies the representation of optical distortions, as well as a simplified planar-based temporal re-projection technique to minimize rendering latency. It is our hope that the techniques described in this paper, and the open source software implementations that support it, will be a stepping stone towards bringing high-fidelity XR content to researchers and hobbyist users alike.
Pre-recorded Presentations for the ACM CHI Virtual Conference on Human Factors in Computing Systems, May 8-13, 2021