Exploring the Relationship Between Emotion, Movement, and Sound Through Interface
Why did we choose to study how sound affects movement?
Affection explores the subtle ways our environment influences both emotion and movement. We wanted to explore how sound—an often overlooked yet powerful external factor—can shape the way people physically and mentally respond to their surroundings.
Our experiment was guided by the hypothesis that changes in sound levels would influence both the physical movements and mental states of participants, leading to observable differences in their walking styles before and after exposure to unexpected sound. Through a series of motion-capture experiments, we observed how changes in sound levels influence walking patterns, posture, and mood, revealing the invisible connection between auditory stimuli, emotion, and the body.
Designing the Sound and Movement Experiment
How did we conduct our sound and movement experiment?
For our testing, we asked each participant to walk from point A to point B while blindfolded. To observe shifts in behavior before and after exposure to sound, we conducted three trials for each test: the first without any sound, the second with a loud object dropped during the walk, and the third again without sound.
Trial #1 : Before Sound
Trial #2 : Exposure to Sound
Trial #3 : After Sound
Observing Participants' Movement Responses
How did participants respond to the sound?
Before Sound
After Sound
Participants are focused on their task, but cautious
Participants are scared something will happen
Blind fold allows them to become more focused with their senses
Participants are more cautious and careful with their steps when walking
Confidence level is lower because of blind fold
Confidence is lost due to fear from the sound
The participants’ responses to sound in our experiment revealed a wide range of emotional and physical reactions, highlighting the powerful influence of sound on human behavior.
•
Some participants were slightly hesitant, showing slight fear while blindfolded, whereas others were confident
•
When the sound was introduced, emotional responses varied: some participants remained relatively neutral, while others were visibly startled, even screaming
•
Afterward, participants often appeared to expect something else to happen, with increased awareness and sensitivity visibly affecting their confidence in walking the same path
Overall, these observations show that sound triggers not only immediate reactions but also impacts participants' movement patterns and confidence levels over time.
Capturing Motion Through Body Tracking
How did we utilize the motion capture?
We imported the motion capture data into a default avatar in Rokoko Studio to visually compare each participant’s before-and-after states. After observing how the data translated onto the avatar, we began designing the interface and screen flows to demonstrate the comparison.