Novel system allows untethered high-quality multi-player VR

WEST LAFAYETTE, Ind. – Virtual reality headsets and application programs for VR are not gaining traction with users because of a chicken-and-egg dilemma, lack of VR content and slow market penetration of custom-made VR units.

Now, Purdue University researchers have created a new approach to VR that allows multiple players to interact with the same VR game on smartphones and provides new opportunities for enterprise, education, health care and entertainment applications.

The Purdue VR system, called Coterie, uses a novel way to manage the challenging task of rendering high-resolution virtual scenes to satisfy the stringent quality-of-experience (QoE) of VR. Those include high frame rate and low motion-to-photon latency, which is the delay between the movement of the user's head or game controller and the change of the VR device's display reflecting the user's movement. The new approach enables 4K-resolution VR on commodity mobile devices and allows up to 10 players to interact in the same VR application at a time.

“We have worked to create VR technology that someone can use on a typical smartphone with a Wi-Fi connection,” said Y. Charlie Hu, the Michael and Katherine Birck Professor of Electrical and Computer Engineering, who led the Purdue team. “Our solution not only allows multiple players to participate in a VR game at the same time, but also provides a better and more cost-effective option for single-player use.”

The technology is detailed in a paper published in ASPLOS 2020, an international conference for interdisciplinary systems research, intersecting computer architecture, hardware and emerging technologies, programming languages and compilers, operating systems and networking.

One reason for the heavy computational workload of high-resolution VR apps is the constant need to render updates to both the foreground interactions with the players and the background environment in the virtual world.

“The heavy load simply cannot be handled by even high-end smartphones alone,” Hu said.

VR apps using Coterie split up this heavy rendering task between the smartphone and an edge server over WiFi in a way that drastically reduces the load on the smartphone while allowing the sub frames rendered on both to be merged into the final frame within 16ms, satisfying the VR QoE.

Hu said this approach not only reduces the network requirement so multiple players can share the same WiFi, but also reduces the power draw and computation demand on each mobile device and provides a better user experience.

“Our technology opens the door for enterprise applications such as employee training, collaboration and operations, health care applications such as surgical training, as well as education and military applications,” Hu said. “You could have multiple doctors and health care professionals interacting in a VR operating room.”

The National Science Foundation and Intel Corp. provided support for the research. The other members of the research team include Jiayi Meng and Sibendu Paul, both doctoral students in Hu’s lab.

The researchers are working with the Purdue Research Foundation Office of Technology Commercialization to patent the technology. The office recently moved into the Convergence Center for Innovation and Collaboration in Discovery Park District, adjacent to the Purdue campus.

The researchers are looking for partners to continue developing their technology. For more information on licensing and other opportunities, contact Matt Halladay of OTC at mrhalladay@prf.org and mention track code 2020-HU-68970.

About Purdue Research Foundation Office of Technology Commercialization

The Purdue Research Foundation Office of Technology Commercialization operates one of the most comprehensive technology transfer programs among leading research universities in the U.S. Services provided by this office support the economic development initiatives of Purdue University and benefit the university's academic activities through commercializing, licensing and protecting Purdue intellectual property. The office is managed by the Purdue Research Foundation, which received the 2019 Innovation and Economic Prosperity Universities Award for Place from the Association of Public and Land-grant Universities. The Purdue Research Foundation is a private, nonprofit foundation created to advance the mission of Purdue University. Contact otcip@prf.org for more information.    

About Purdue University

Purdue University is a top public research institution developing practical solutions to today’s toughest challenges. Ranked the #6 Most Innovative University in the United States by U.S. News & World Report, Purdue delivers world-changing research and out-of-this-world discovery. Committed to hands-on and online, real-world learning, Purdue offers a transformative education to all. Committed to affordability and accessibility, Purdue has frozen tuition and most fees at 2012-13 levels, enabling more students than ever to graduate debt-free. See how Purdue never stops in the persistent pursuit of the next giant leap at purdue.edu.

Writer: Chris Adam, 765-588-3341, cladam@prf.org 
Source: Y. Charlie Hu, ychu@purdue.edu

Abstract

Coterie: Exploiting Frame Similarity to Enable High-Quality Multiplayer VR on Commodity Mobile Devices

Jiayi Meng, Sibendu Paul and Y. Charlie Hu

In this paper, we study how to support high-quality immersive multiplayer VR on commodity mobile devices. First, we perform a scaling experiment that shows simply replicating the prior-art 2-layer distributed VR rendering architecture to multiple players cannot support more than one player due to the linear increase in network bandwidth requirement. Second, we propose to exploit the similarity of background environment (BE) frames to reduce the bandwidth needed for prefetching BE frames from the server, by caching and reusing similar frames. We find that there is often little similarly between the BE frames of even adjacent locations in the virtual world due to a “near-object” effect. We propose a novel technique that splits the rendering of BE frames between the mobile device and the server that drastically enhances the similarity of the BE frames and reduces the network load from frame caching. Evaluation of our implementation on top of Unity and Google Daydream shows our new VR framework, Coterie, reduces per-player network requirement by 10.6X-25.7X and easily supports 4 players for high-resolution VR apps on Pixel 2 over 802.11ac, with 60 FPS and under 16ms responsiveness.