Papers - Minamizawa, Kouta

asmVR: Enhancing ASMR Tingles with Multimodal Triggers Based on Virtual Reality

D Peng, T Person, K Skierś, R Cui, M Armstrong, K Minamizawa, YS Pai

SIGGRAPH Asia 2023 XR, 1-2  2023.11

　View Summary

Anxiety and stress have gradually become commonplace mental health problems afflicting many people worldwide. We propose asmVR, a novel approach to enhance the autonomous sensory meridian response (ASMR) experience by combining multimodal triggers including visual, auditory, tactile, and emotional stimuli. asmVR helps users enhance ASMR tingling sensations through online and offline modes, provides realistic VR environments and remote avatar ASMRtist for users, showcasing its potential for stress relief, emotion regulation, and customization. Additionally, it reveals new possibilities for the future application of VR in the field of psychotherapy.

• Access to Document (DOI)

FoodMorph: Changing Food Appearance Towards Less Unhealthy Food Intake

R Cui, W Chen, D Peng, K Minamizawa, YS Pai

SIGGRAPH Asia 2023 Posters, 1-2  2023.11

　View Summary

Human dietary experiences are influenced by multiple senses. To promote healthy eating and reduce the consumption of unhealthy food, we have developed FoodMorph, a virtual reality system that immerses users in visually simulated food textures that are inedible. By presenting users with these textures, FoodMorph aims to diminish interest in and intake of unhealthy food while also assessing the dining enjoyment associated with common non-food textures. We found that concrete textures on food tend to lead to the lowest enjoyment score, which potentially effects their dietary habits.

• Access to Document (DOI)

Conversation Echo: Communication in virtual environments that reflects conversation contents

S Hachisu, S Wakisaka, K Minamizawa

SIGGRAPH Asia 2023 Posters, 1-2  2023.11

　View Summary

In this study, we propose Conversation Echo, a system that reflects the topics of conversation in a VR environment in real time. This method uses AI to convert speech data into text, extract conversation topics, and generate panoramic images to generate a VR environment, which dynamically changes the environment in real-time. This method aims to realize an experience that generates conversation topics and inspiration.

• Access to Document (DOI)

asmVR: VR-Based ASMR Experience with Multimodal Triggers for Mental Well-Being

D Peng, T Person, R Cui, M Armstrong, K Minamizawa, YS Pai

SIGGRAPH Asia 2023 Posters, 1-2  2023.11

　View Summary

Individuals are besieged by anxiety and stress throughout the world. Solutions to improve mental well-being can vary, and Autonomous Sensory Meridian Response (ASMR) is one potential method that has proven to be able to reduce stress. To that end, we introduce asmVR, a novel approach to enhancing ASMR experiences using multi-modal triggers. Combining online and offline modes, asmVR enhances ASMR tingling, offering immersive VR environments and remote ASMRist embodiments. Initial user testing shows tingles enhance and stress relief potential, along with new possibilities for VR in psychological therapy.

• Access to Document (DOI)

Multichannel Haptic Communication Platform with Wearable Sensing and Display

H Taguchi, Y Kamiyama, K Kan, Y Ju, A Horie, Y Tanaka, H Ishikawa, ...

SIGGRAPH Asia 2023 Emerging Technologies, 1-2  2023.11

　View Summary

Our goal has been to create an environment in which anyone can take advantage of tactile technology. Content that incorporates tactile sensations in addition to visual and auditory sensations will activate people's interaction with it. On the other hand, both sensing and display are complicated to create high-quality content, and ease of use is lost. In this paper, we propose a platform that can easily develop multi-channel vibrotactile contents by utilizing existing workflows. By preparing a 4-channel fingertip-mounted vibrotactile device as a display, a high-resolution experience with less load during the experience is realized. We demonstrate sensor-based pre-recorded or real-time multi-channel tactile contents and show the concept of near-future tactile technology including use cases.

• Access to Document (DOI)

Waylet: Self-contained haptic device for park-scale interactions

T Maeda, J Yamamoto, T Yoshimura, H Sakai, K Minamizawa

SIGGRAPH Asia 2023 Emerging Technologies, 1-2  2023.11

　View Summary

Although various haptic devices have been proposed so far, most of them are limited to use in laboratory or indoor environments, because such haptic devices are wired to external equipment for power supply and control. This can be a critical issue, particularly when using them in freely moving situations. In this study, we proposed a self-contained haptic device called Waylet for park-scale interactions. Waylets can provide translational and rotational pseudo-forces via asymmetric vibrations. To demonstrate the feasibility of our concept, we demonstrate an intuitive haptic navigation system in a park-scale mixed-reality environment with haptic rendering.

• Access to Document (DOI)

Transcale: Embodiment Transition Toward Multi-Verse Exploration

H Onoda, S Wakisaka, T Saito, K Minamizawa

SIGGRAPH Asia 2023 XR, 1-2  2023.11

　View Summary

Transcale is a VR experience where the users can change the body scale smoothly while walking. By continuously modifying their perceived body scale, this interaction method provides the users to explore the virtual world from various points of view from a dwarf to a giant.

• Access to Document (DOI)

Synced Drift: A Novel Sport Using a System that Harmonises Human Movement to Transcend Distance and Ability

R Ando, G Barbareschi, M Kawaguchi, K Minamizawa

SIGGRAPH Asia 2023 Emerging Technologies, 1-2  2023.11

　View Summary

"Synced Drift"provides a new inclusive sport experience that allows two individuals to play a drifting race together while sharing a single body. Synced drift leverages a system consisting of a large-diameter omni-wheel mechanism enabling omnidirectional movement, two sensing seats that detects the user's center of gravity shift, a mobile vehicle, and a viewpoint sharing system. The movement of the mobile vehicle is achieved by proportional control of the body movements acquired by the two sensing seats, one placed on the vehicle and one remote, from two or more individuals. The speed and direction are adjusted based on the harmonization of the inputs from the two users. This system allows people who were previously unable to participate in sports due to physical limitations, for example people with spinal cord injuries, or those with who had a stroke, to participate equally, regardless of individual impairments. Using the above system, the rules of the sport were designed and implemented. In this demonstration, the system explores the possibility of people coming together, racing and competing as one, transcending physical characteristics and physical locations.

• Access to Document (DOI)

Somatic Music: Enhancing Musical Experiences through the Performer's Embodiment

A Uyama, Y Kamiyama, S Wakisaka, A Horie, T Saito, K Minamizawa

SIGGRAPH Asia 2023 Posters, 1-2  2023.11

　View Summary

Music serves as a tangible embodiment of the performer’s expression, bearing a distinct musicality that transcends auditory perception. This study delves into the distinctive musicality inherent to musicians through physical data analysis. It introduces a novel approach to the auditory experience by incorporating tactile stimulation via vibration and pressure to present an innovative channel for conveying musicality to the audience. The primary aim is to enhance the realm of music appreciation by amplifying the scope of performers’ expressive capacities, thereby revolutionizing the conventional paradigm of music experiences.

• Access to Document (DOI)

Phantom Walls: Spatial perception and navigation without vision

T Ikeda, S Wakisaka, K Minamizawa

SIGGRAPH Asia 2023 Emerging Technologies, 1-2  2023.11

　View Summary

"Phantom Walls"is a novel technique that establishes continuous spatial perception without vision. By creating an auditory environment where obstacles emit sounds, users can perceive and navigate around visually imperceptible "Phantom"obstacles by listening to the generated "soundscapes". This method allows individuals to perceive and avoid these obstacles while walking without vision.

• Access to Document (DOI)

OwnDiffusion: A Design Pipeline Using Design Generative AI to preserve Sense Of Ownership

Wu Y., Minamizawa K., Pai Y.S.

Proceedings - SIGGRAPH Asia 2023 Posters, SA 2023  2023.11

　View Summary

Generative Artificial Intelligence (AI) has been a fast-growing technology, well known for generating high-quality design drawings and images in seconds with a simple text input. However, users often feel uncertain about whether generative art should be considered created by AI or by themselves. Losing the sense of ownership of the outcome might impact the learning process and confidence of novice designers and design learners who seek to benefit from using Generative Design Tools. In this context, we propose OwnDiffusion, a design pipeline that utilizes Generative AI to assist in the physical prototype ideation process for novice product designers and industrial design learners while preserving their sense of ownership. The pipeline incorporates a prompt weight assessing tool, allowing designers to fine-tune the AI’s input based on their sense of ownership. We envision this method as a solution for AI-assisted design, enabling designers to maintain confidence in their creativity and ownership of a design.

• Access to Document (DOI)

Heightened empathy: A multi-user interactive experience in a bioresponsive virtual reality

M Armstrong, K Skiers, D Peng, TS Gunasekaran, A Kundu, T Person, ...

ACM SIGGRAPH 2023 immersive pavilion, 1-2  2023.07

　View Summary

We present Heightened Empathy, a multi-user interactive experience in bioresponsive virtual reality (VR) designed to visualize emotional states and stimulate different types of empathy between two players in various interactive modes. The experience immerses users in a VR representation of each other's emotional state, while also reflecting this to the audience using a table-top social robot and projection as they interact with each other. In competitive mode, the goal is to promote cognitive empathy where each user needs to understand the new emotional representation in VR to win. In communication mode, we have users act on one another verbally to promote emotional empathy. The experience aims to enhance empathy perception by placing the players in virtual environments that require them to understand, share, and act on each other's perspectives and emotions.

• Access to Document (DOI)

"I am both here and there" Parallel Control of Multiple Robotic Avatars by Disabled Workers in a Café

Barbareschi G., Kawaguchi M., Kato H., Nagahiro M., Takeuchi K., Shiiba Y., Kasahara S., Kunze K., Minamizawa K.

Conference on Human Factors in Computing Systems - Proceedings  2023.04

　View Summary

Robotic avatars can help disabled people extend their reach in interacting with the world. Technological advances make it possible for individuals to embody multiple avatars simultaneously. However, existing studies have been limited to laboratory conditions and did not involve disabled participants. In this paper, we present a real-world implementation of a parallel control system allowing disabled workers in a café to embody multiple robotic avatars at the same time to carry out different tasks. Our data corpus comprises semi-structured interviews with workers, customer surveys, and videos of café operations. Results indicate that the system increases workers' agency, enabling them to better manage customer journeys. Parallel embodiment and transitions between avatars create multiple interaction loops where the links between disabled workers and customers remain consistent, but the intermediary avatar changes. Based on our observations, we theorize that disabled individuals possess specific competencies that increase their ability to manage multiple avatar bodies.

• Access to Document (DOI)

Dementia eyes: Co-Design and Evaluation of a dementia education augmented reality experience for medical workers

X Shen, YS Pai, D Kiuchi, K Bao, T Aoki, H Meguro, K Oishi, Z Wang, ...

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems …  2023.04

　View Summary

Dementia describes a syndrome of cognitive degeneration, and Behavioural and Psychological Symptoms of Dementia (BPSD) is the non-cognitive symptom. BPSD can be improved by care services. To aid better care service, we explore the potential of using Augmented Reality (AR) to support dementia education for medical workers in three steps: (1) We explore medical workers' perspective on dementia care lived experience and XR, (2) we co-design an educational experience containing an AR-based application and a 5-min activity with medical workers, (3) we evaluate the effectiveness of the system through a mixed method study. Our result shows that the AR experience successfully touches participants, and motivates them to reflect on the provision of care service. On this basis, we discuss the elements and challenges of designing XR-enabled dementia education for users unfamiliar with novel technology, and the potential of using XR in clinical education.

• Access to Document (DOI)

Towards Embodying Emotions in Play with Neurodivergent Children using Haptic Technologies

Y Ju, M Shibasaki, CC Kim, K Kunze, K Minamizawa

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing …  2023.04

　View Summary

As neurodivergent cases in children are steadily increasing, we are also seeing increased awareness regarding personalized neurological development and efforts towards social inclusion. Children struggling with these conditions should engage in education curricula custom-tailored and personalized for their individual needs. In this initial work, we describe the first steps of the development of an interactive system that created a multi-sensory environment embedded into a participatory performance to help neurodivergent children learn about emotions. We first report on the co-design process and research insights from our work integrating haptic technology and neurodivergent children's emotion learning activity. Then we report on a study in which 28 children were involved in testing the implementation of our system and share our insights that we gained about designing haptic interventions for neurodivergent children and the collaborative process with their instructors and other related parties.

• Access to Document (DOI)

Phantom Undulations: Remote Physiological Sensing in Abstract Installation Works

D Hynds, D Zheng, Y Zhang, H Ma, K Ragozin, G Chernyshov, JA Ward, ...

Proceedings of the Augmented Humans International Conference 2023, 367-370    367 - 370 2023.03

　View Summary

Phantom Undulations is a mixed-media work in which an artist's physiological data is being used remotely to manipulate the sounds and visuals of an abstract artifact in a gallery setting. This work relies heavily on the concept of showing the artist's presence or liveness in an abstract and remote manner through changes in the harmony, rhythm, and timbre of a loosely structured soundscape as well as the physical appearance of the artifact. We propose a method of utilizing real time physiological sensing data through a custom built sensing wristband and accompanying software. This system reads the physiological data of the artist and sends it to the Internet, where it can be received by the artifact anywhere on Earth. In addition to the artist's physiological data, we also offer a way for the audience to incorporate their own data into the work via several sensing wristbands which will accompany the artifact. Through this collaborative process, we wish to invite the audience to join the artist in manipulating the sonic and visual characteristics of this artifact and create a contrapuntally fluid and responsive musical experience.

• Access to Document (DOI)

Linking audience physiology to choreography

J Han, G Chernyshov, M Sugawa, D Zheng, D Hynds, T Furukawa, ...

ACM Transactions on Computer-Human Interaction 30 (1), 1-32 (ACM Transactions on Computer-Human Interaction)  30 ( 1 )  2023.03

ISSN  10730516

　View Summary

The use of wearable sensor technology opens up exciting avenues for both art and HCI research, providing new ways to explore the invisible link between audience and performer. To be effective, such work requires close collaboration between performers and researchers. In this article, we report on the co-design process and research insights from our work integrating physiological sensing and live performance. We explore the connection between the audience's physiological data and their experience during the performance, analyzing a multi-modal dataset collected from 98 audience members. We identify notable moments based on HRV and EDA, and show how the audience's physiological responses can be linked to the choreography. The longitudinal changes in HRV features suggest a strong connection to the choreographer's intended narrative arc, while EDA features appear to correspond with short-term audience responses to dramatic moments. We discuss the physiological phenomena and implications for designing feedback systems and interdisciplinary collaborations.

• Access to Document (DOI)

Accessible Parallel Embodiment of Multiple Robotic Avatars in a Cafe

G Barbareschi, M Kawaguchi, H Kato, K Takeuchi, M Nagahiro, Y Shiiba, ...

XRDS: Crossroads, The ACM Magazine for Students 29 (2), 36-41  2023

The Empathic Metaverse: An Assistive Bioresponsive Platform For Emotional Experience Sharing

Y Suen Pai, M Armstrong, K Skiers, A Kundu, D Peng, Y Wang, ...

arXiv e-prints, arXiv: 2311.16610  2023

The Empathic Metaverse: An Assistive Bioresponsive Platform For Emotional Experience Sharing

YS Pai, M Armstrong, K Skiers, A Kundu, D Peng, Y Wang, ...

arXiv preprint arXiv:2311.16610  2023