SA '22 XR: SIGGRAPH Asia 2022 XR

With the development of the digital information, the digitalization and creative representation of the intangible culture heritage (ICH) dances have become the core of dance safeguarding. "Dance of Drums", an interactive installation combining virtual and reality for Chinese traditional bronze-drum dance representation, provides both external projected interface and internal virtual reality (VR) experiences. Specifically, "Dance of Drums" rebuilds the motions and folklore elements of bronze-drum dance through internal VR scenarios, and shows the history and ancient form of dance through the projected interface in real space, which reconstructs the static display of dances to promote the preservation and dissemination of bronze-drum dance.

We demonstrate a dual-motor adaptation over 2 virtual bodies with opposite visuomotor rotation (+ 15° and − 15°). A related study explored the effect of view perspective (1st person view perspective, 1pp, and 3rd person view perspective, 3pp) on a dual motor adaptation behavior. They showed that (1) in 1pp, the adaptation is explicit, while in 3pp, the adaptation is implicit, and (2) the implicit adaptation led to a better dual adaptation. Here we let the participants experiment with explicit and implicit dual motor adaptation by doing a reaching task in 1pp and then in 3pp. We present an ad-hoc motor adaptation score and discuss which adaptation behavior led to a better dual adaptation.

PseudoJumpOn is a novel locomotion technique that allows step-up jumping movements using actual jumping motions on a flat floor with a common virtual reality (VR) setup. In this demo, we introduce an installation utilizing PseudoJumpOn where the player repeatedly makes step-up jumps to climb up to higher platforms. The aim is to provide players with the experience of richer vertical locomotion in VR without causing them to feel unnaturally manipulated, even though they know there are no physical steps in reality. While jumping, the system applies two types of viewpoint manipulation (i.e., gain manipulation and peak shifting) based on the user’s physical jumping height and the height of the jumping target. This calculation previously assumed that the height of the jumping target was known, but we improved this by predicting it using the user’s facing direction when taking off.

We present an installation demonstrating the applicability of RedirectedDoors, a redirection technique that occasionally manipulates the user’s orientation during door-opening motions. In this demo, the player explores an indoor virtual environment containing doors while wearing a head-mounted display (HMD), and their orientation in reality is manipulated as a function of the door’s opening angle. In addition, when the player opens the door by pushing or pulling the doorknob in virtual reality, the corresponding passive haptic feedback is provided by the self-actuated doorknob-type prop. When reaching the goal, they can see the manipulation results by comparing their virtual position with a real landmark position. Consequently, this demo both makes the player’s experience more realistic and presents the virtual environment in a comparatively small physical space.

GENESIS embarks on an emotionally intense virtual reality journey to experience the dramatic milestones in the evolution of Earth and mankind. As a 360°, stereo 3D film, GENESIS has been pre rendered (no real time) and is usable as a three Degrees of Freedom (3DOF) experience. The project can be watched on all VR glasses that can play 3DOF, like 360° movies. Glasses such as Pico G2 or Oculus Go were developed specifically for 3DOF content. Of course, all glasses which are able to present 6DOF experiences can also play 3DOF content. The users can watch the content while standing or sitting.

Guernica, éres tu is a first-person anti-war storytelling experience that reconstructs Pablo Picasso's famed painting Guernica into a virtual environment. Guernica, éres tu, through the use of 8i's volumetric capture technology in a virtual reality environment, the VR application tries to retain the essence of human into the artwork and fits the creative style of abstract and surreal art by modifying the 3D captures. As a result, it persuasively delivers a message that the belligerents, the casualties, and the observers are all actual human beings, not merely statistics nor images on the news.

Learning Chinese characters and characters of Chinese origin is a challenging and time-consuming task for native speakers and second language learners alike. Traditional methods to acquire Chinese characters include repeatedly writing the characters on paper or air writing. In scientific research, computer- and mobile-assisted approaches have been implemented that allow users to be interactively guided through the learning process. The recent advancements and availability of virtual reality (VR) headsets with six degrees of freedom tracking and controllers provide an interesting new platform for education applications. The proposed VR application utilizes novel interaction and visualization methods tailored for VR with guided repetition as well as sound and haptic feedback. The augmentation of air writing further enables motor-based learning and allows for the application to be used as an educational exergame.

Boxing is a combat sport that involves fierce movement and close body contact between two boxers. In addition to the boxer, the coach is also essential for training. The coach could be injured or too tired to continue consecutive training sessions for many boxers. Thus, we present MovableBag, a substitutional robot that provides room-scale mobility and the ability to carry sport props with an easier tracking setup to simulate the boxing opponent or coach in virtual reality with encountered-type haptic feedback. In addition, we integrate our system with a wireless VR headset and develop a VR boxing application to show the capacity of the device and inform future studies.

"Out of the Cave" is an immersive VR comic. Its story revolves around a girl struggling with the regrets of her past relationship and stepping on a journey to heal the scars. This work blends webcomics into a spatial medium to create a brand-new narrative experience. The audience can dive into the story world and complete the journey of pursuit and self-healing with the heroine.

Recent demonstrations of 3D telepresence provide a glimpse into the future where 2D video communication is replaced with photo-realistic virtual avatars rendered on 3D displays by using dedicated hardware. Our platform lets visitors experience real-time end-to-end 3D teleconferencing using commodity hardware. This demo integrates current state-of-the-art face reconstruction and rendering algorithms into an end-to-end system to illustrate the utility and capabilities of commodity telepresence on a range of 3D displays.

Transcendental Avatar is a virtual reality (VR) system focused on stress relief to support relaxation techniques, using biofeedback, a well-known therapy technique to improve physical and mental health. Biofeedback as a method uses visual and audio feedback of one’s physiology to reflect, recognize, and help one gain awareness of many physiological functions, improving their cognitive and emotional state. Yet, there has been little work on how this can be appropriately leveraged in virtual reality (VR). In Transcendental Avatar, we proposed a system that shows the personification of the avatar, reflecting in real-time user’s biofeedback to enhance their cognitive and emotional state. In this study, we examine whether stress, and self-reported anxiety symptoms can be relieved or reduced by the proposed immersive experience.

This educational content concerns fundamental nursing skills by using virtual reality. Blood sugar testing and the administration of insulin subcutaneous injection are two of the most frequently used among the 20 core fundamental nursing skills. The content pertaining to these tasks can be used for self-management education for diabetic patients. This contents designed a core fundamental nursing skill on immersive VR contents by dividing the learner's experience into three stages: guide, mission, and feedback with these skills. And it is designed by tracking the movement of the hand through finger joint recognition without using a controller for immerse in training. This content will help improve nursing students’ clinical practice competency.

XRAYHEAD creates the striking illusion of seeing a skeleton inside one’s head when the experimenter touches the interior skeleton. This illusion is tested in a dark room using a setup in which the participant views a specially designed smart skeleton through a half mirror, and the experimenter touches the participant and the skeleton’s head with both hands. In this specific arrangement, the distance between the skeleton head and the palm of the experimenter’s hand determines how the light intensity of the LED embedded in the skeleton head increases. Alternatively, the whole part of the participant’s head is kept illuminated. Additionally, to enhance illusory tactile sensation to one’s skeleton, an auditory stimulus is supplied through a speaker or a bone-conduction headphone and/or modulated in sync with the degree of translucence. This creates the illusion of a head with a clearly viewed skeleton inside its translucent surface, allowing the physical touch to the skeleton’s surface. This technology was tested in our laboratory exhibition, and 77 of 101 participants reported a strong tactile sensation to the illusory skeleton inside their heads.