A worldwide pandemic hit, forcing the majority of us to isolate. As people found new ways to fill their time indoors, there was a growing interest in picking up new skills, especially learning musical instruments like the piano. This presented a chance to meet this need with a fresh, digital approach.

Problem Statement

The pandemic has changed how we connect and learn, pushing many of us to look for hobbies we can dive into from home. Traditional ways of learning the piano aren't as accessible now, making it tough for beginners to get started.


Bridge this gap with a digital solution that makes learning the piano more accessible, enjoyable, and effective than ever before.

‍Competitive Analysis

I put in a good amount of work into understanding where we fit in the piano learning world. By putting together a Competitive Analysis Matrix, I compared  what's already out there with our ideas in context. This wasn't just about seeing who was doing what. It was about finding those spots where we can jump in with something fresh by identifying the strengths and weaknesses of current options, and to spot gaps in the market's providings. Check out my findings and further analysis on the available opportunities below.

Target User Survey

After diving into the competitive analysis, I needed to get a real feel for what potential learners were looking for. So, I sent out a survey asking people about their piano learning journey - why they want to learn, what's been stopping them, and what they dream of in a piano learning app. The feedback was insightful, directly influencing our future design and MVP features. The joke responses gave me a laugh as well 😄. Here’s what they shared with us.

User Interview Key Takeaways

The survey uncovered key insights: folks are interested in learning the piano out of curiosity, looking to impress others, or moved by music to start learning piano. Yet, they're hitting common snags - from finding a starting point to keeping motivated, especially when lessons are pricey or embarrassment creeps in. They imagine an ideal learning blend of video lessons, intuitive play, and clear, visual cues. But here's the thing - they might not know what tech can best address their needs. They may not realize what's possible with current technology. That's where we step in, sifting through their feedback to see if cutting-edge solutions such as Augmented Reality could be the tool they didn't know they needed.

Project Scope

Initially, several ideas were considered, including live video lessons, online courses, and a mobile app with video tutorials. However, you'll discover as you keep reading that augmented reality technology, otherwise known as AR, becomes the path I pursed due to its potential use cases which could transform piano learning, making the process more interactive and engaging in juxtaposition to comparable methods.

Minimum Viable Product (MVP)

Given the wide range of feedback on user goals, issues, and preferred learning methods, I concluded that Augmented Reality is the technology best suited to address these diverse needs. AR's versatility allows us to create a solution that can adapt to various learning preferences, whether it's visual guidance for hand positioning, interactive lessons for those seeking structure, or engaging, game-like experiences to boost motivation and reduce embarrassment. This technology aligns well with the feedback, offering a unique, comprehensive path forward in piano learning.

The Minimum Viable Product (MVP) focuses on leveraging AR to enhance piano learning, ensuring compatibility across most smartphones. A curated selection of 2-5 songs, transforming them into an intuitive and visual AR experience. Users can interact with an in-app keyboard, providing a comprehensive, hands-on experience. This MVP aims to validate the core idea, testing the waters for AR's potential to enhance music learning by combining accessibility with an interactive toolset.

Competitive Benchmarking

Understanding the competitive landscape through a benchmark analysis allows us to closely examine the features, strengths, and weaknesses of  music-learning tools. This type of comparison sheds light on what currently exists in the market, offering insights into user preferences and technological advancements. By analyzing competitors like Yousician, AR Pianist, and Rock Band 3, we gain valuable knowledge on successful strategies and potential areas for innovation, helping us identify opportunities to fulfill unmet needs or improve upon existing solutions in the music education space.


AR Pianist

Rock Band 3

In the competitor benchmark analysis, I scrutinized features of three music-related pieces of software: Yousician, AR Pianist, and Rock Band 3. Yousician stands out with its extensive song database and weekly challenges, though it lacks in intuitive visual cues for the user and MIDI file support. AR Pianist impresses with its 3D models allowing users to watch another pianist play and its visual cues when playing yourself, but suffers from a small song selection and an un-intuitive UI. Rock Band 3 excels in intuitive visual guidance and a diverse song database via MIDI file support, the only downside being the technical knowledge barrier for adding new songs. Each platform offers unique strengths, highlighting opportunities to innovate and address gaps in the learning experience.

Diving into the potential of using Augmented Reality after reviewing various music-learning apps, we see how AR could really change the game. It could overlay visual play instructions right on your actual piano, merging real tactile experience with guided learning. This direct, augmented feedback can make learning intuitive, addressing the common request for structured, visually clear guidance. It would even be possible to improve what AR Pianist did with their 3D model pianist visual, creating a virtual 3D instructor. While AR opens up exciting possibilities, the key will be to innovate within this space, using AR's unique capabilities to enhance the learning experience in ways traditional apps haven't explored or expanding on tried and true methods.

Future Features

As we look to the horizon of our AR piano learning app, envisioning the full suite of features that could enhance the user experience, we imagine a blend of innovation, accessibility, and personalization. Here’s a glimpse into potential future capabilities designed to elevate the learning journey:


As progress with development of the AR piano learning app unfolds, understanding and managing constraints is key to navigating the journey ahead. Here’s a concise overview of the critical factors we need to consider:

User Experience Objectives

In designing an AR piano learning app, my aim is to craft an experience that embodies five key characteristics:

These adjectives will guide my design philosophy, ensuring a product is created that not only meets, but exceeds user expectations for a rewarding educational journey.

Heading 5

How Might We...?

As we navigate through the creation of an AR piano learning app, several pivotal questions arise. How do we translate traditional piano learning into an augmented reality environment? What innovative methods can we employ to tackle physical and technical barriers? And importantly, how do we keep users motivated and engaged over time? Below, we explore potential solutions to these challenges, aiming to blend technology with accessibility in ways that enrich  the learning experience.


This section showcases my initial design concepts and mockups, informed by the user-research and market/competitor analysis documented in previous sections. These visuals are crafted to give you a glimpse into the user experience I'm aiming to create. Each mockup/sketch represents a piece of the puzzle in building an app that's not just functional, but immersive and engaging for learners. Through these mockups and sketches, my aim is to present a clear vision of what I believe the future of piano learning can look like.

Main Menu Design: The gateway into the app, prioritizing user-friendly navigation and simplicity.

AR Menu Integration: Demonstrating how users might interact with app features directly within their physical environment.

Piano Detection Feature: Conceptualizing the technology needed to accurately identify and map the user's piano for AR overlays.

Voice Control and Hand Tracking: Considering the application of voice commands for a hands-off approach during app usage. Examining the potential for hand tracking to interact with the UI via gestures.

Notes Overlay Concept 1: Showing how visualized key instructions could guide users on a real piano, enhancing engagement through visual cues.

Notes Overlay Concept 2: Visualizing how notes could dynamically guide users on a real piano, enhancing learning through visual cues.

Looking Back and Forward

Jumping into this AR piano app has been quite the ride. I've dug into what makes learners tick, eyed the competition, and started sketching out what will hopefully be a worthwhile competitor in the piano learning sphere. There's a lot ahead—polishing up the sketches, wrestling with the tech development, and chatting with real users to get it just right.  Here's to the next steps, where we begin development and fine-tune our vision for an engaging and fun way to learn how to play the piano.

All Recent Work