Muso

Turn daily objects into instruments in MR

MUSO is an XR music synthesizer that encourages children to turn ordinary everyday objects into fascinating musical instruments in XR. That feeling captures the essence of childhood wonder and the limitless potential of the imagination.

MUSO offers users such a musical journey while introducing them to the basics of music theory. MUSO believes that music learning should be easy and fun, and hopes to bridge the gap in children's music education through a low-cost, entertaining approach.

Academic Project

2023

Timeline

20 weeks

Key practices

Tech-center Research

C Sharp

Unity

VR/AR Development

Tech-center Research, C Sharp, Unity, VR/AR Development

Other contributors

Monica Zhang, Hattie Li

01 The Core Idea

MUSO believes that music learning shouldn't be boring and inaccessible, by blending education with imagination and creativity, bridging the gap with affordable, engaging method. We aim to tackle <10% access to music education in the developing world.

📒

This video explains our pain points, mission, and key features with in 90 seconds.

02 The Problem

According to the statistics of 2022, the popularization rate of music education in developing countries is less than 10%. In China, for example, there is a general lack of music education for children due to factors such as the cost of music learning, the level of teachers, and the lack of widespread popularization and use of professional, effective, and fun children's music pedagogies for music initiation.

What do Chinese young people think of when it comes to musical initiation? Learning to play the piano or the violin, which in turn conjures up images of children crying by the side of their instruments.

Sadly, this is what young Chinese people have in mind when they think of music education...

So, what is MUSO

MUSO is an XR music synthesizer that empowers kids to explore music by transforming surrounding everyday objects into magical instruments, and teaching knowledge while kids play in XR.

03 Key Features

Unlocking Creativity

MUSO's encourage kids to observe and experiment day life. By recognize the shape, size of a object, MUSO can turn it into playful instrument.

Easy to start

Using MR recognition technology, a round bowl can be transformed into a percussion in just three steps.

Learn through play

MUSO uses fun animated characters to teach music theory while children play with "instruments".

Go wild with exploration!

Create your brand, do your experiments!

04 Development Hightlights

The whole project was development in Unity, using Oculus Quest 2 and Oculus Quest Pro as the testing device. Meanwhile, MUSO continues to iterate after user interviews and user testing.

Even though this isn't my first project in C#, I'd like to share the funny things I've learned about struggling to implement the code logic in order to achieve functionality. :)_

Behind the scenes at Immersion Lab: ideation, development, testing, filming, editing...

Enable Gesture Control!
The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.
Visual Effects
We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.
Physical Computing
We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.
User Testing & Iterations
During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.
Enable Gesture Control!
The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.
Visual Effects
We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.
Physical Computing
We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.
User Testing & Iterations
During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.
Enable Gesture Control!
The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.
Visual Effects
We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.
Physical Computing
We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.
User Testing & Iterations
During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.
Enable Gesture Control!
The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.
Visual Effects
We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.
Physical Computing
We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.
User Testing & Iterations
During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.

User Testing & Iterations

During the development and exploration of our demo, we conducted some user testing. The results helped us understand how users subconsciously want to interact with different shapes, how users' posture and position affect their gestures, and whether MUSO is Easy to use.

Physical Computing

We believe we need to restore the logic of real instrument interaction, so we set up various triggers such as acceleration trigger, height trigger, and force trigger to restore the real physics. We also envisioned designing VR gloves to better utilize sensors and haptic feedback.

Visual Effects

We believe that visual effects add more variety and interest to the interaction between children and music. Therefore, we have incorporated unique visual effects for each musical interaction. Taking circular visual effects as an example: The program checks, during each play, whether the current hand movement speed, `currentSpeed`, is greater than or equal to 0. If the hand is moving in the positive direction, the `MoveCount` counter is incremented by 1. This allows the program to generate pink and blue circles sequentially based on the speed of the play.

Enable Gesture Control!

The first thing about music playing is to get rid of the controllers, because in reality we are not playing instruments with a controller. We embedded Oculus' gesture API to implement gesture control functionality. Moreover, we tuned as many joints as possible for different triggers of the playing process to make the interaction more natural and smooth. Natural interaction has always been our main concern, we want to design a game for children with low learning costs.

No heading elements found. Showing placeholder content.