Playing Together, Apart
How can musicians jam virtually? This is a question many people are struggling to answer. Bands playing music spanning from classical to acapella to rock to pop all rely on in-person rehearsal. Typically bands need to rehearse physically together because their music requires musicians to be playing at the same tempo and pitch and the volume needs to be balanced across instruments. With COVID-19 restrictions in place, many bands and jam groups are increasingly looking for a virtual playing experience. However, this same problem expands to post-COVID times. We set out to create technology that would allow jam groups to play across the country and across the world. We started with designing for the key features that survey participants identified virtual solutions either were lacking or features that were necessary for the respondents to try a virtual platform. In our survey, lag was brought up as the biggest issue facing virtual play. Additionally, complicated set-up was a major barrier for virtual rehearsing. We set out to address both of these issues.
Presenting a Solution
One way to minimize lag is to use a wired network connection. This is impractical for many users who move their computers around so we introduce the Smart Mic. The box contains a microphone and a processor, which allows the microphone to be plugged into ethernet and then act as the processor for an application on the users computer. Because the microphone can then be easily unplugged from the user’s device and neatly stored by the, our device allows the technical lag-busting properties of a wired connection without the inconvenience of tethering your computer to a router. Next, our solution conquers the difficulty of setting up two ways. First, our system’s use of a specific microphone allows the software to make initial assumptions about input so the user doesn’t need to manually adjust settings. The device also automatically pops up an installation prompt when plugged into a new computer, making the first time user experience straightforward. Secondly, our entire software is designed with minimalism in mind, with few basic states and popups layering over them with helpful information so all users remain informed and in control.
Preliminary Testing
We started with a paper prototype for both the hardware and software components. For our paper prototype for the Smart Mic, we prioritized having a simple and portable design. Our main focus for paper prototyping was the accompanying software application. To test our design, we gave participants this scenario: “You and Anika are musicians who like to jam together. You play guitar and Anika is on vocals. Anika has moved away but you still want to play together. You have invested in this new technology to do so virtually.” We then asked our participants to do the following tasks:
- Install the software
- Get Anika into a shared session with you
- Attempt to add a new friend Natalie (who then refused connection)
- Accept Natalie’s request to connect
- Leave the shared session
We observed each of the test users’ ability to complete these tasks and made note of any difficulties or confusions they were having throughout. At the end, we asked them about their frustrations and general experience.
Because one of our main tasks we were designing for was easy setup, we automatically flagged any issues users had figuring out how to start a session. Throughout our user sessions, we realized we needed to further streamline our components. On the hardware component, we saw a pattern of users being confused on where to place their microphone and whether or not the microphone was on. To address this, we decided to include a clear picture and description on the microphone about how it should be placed. We also decided to remove the on/off switch and instead solely have the audio be controlled in the application. With the software prototype, we saw that a lot of users were confused about how interactions with friends worked: how to add a friend, how to accept a friend’s invite, knowing whether a friend was available or not. To make these tasks involving friends easier, we decided to rethink how friends existed in the platform and streamline sessions. We adopted a similar session to Zoom and have there be a unique code associated with a session, which would allow users to join a session with a session code or invite other users using a session code. These results indicated that some of our initial features were too complex to follow and we needed to simplify our features.
High-Fidelity Prototyping
Incorporating the results of our design research, we put together a high-fidelity digital mockup of the Smart Mic software. You can check here. In this section, we’ll walk through two high level flows. First we demonstrate the first-time user experience to emphasize the ease of setup and second we walk through the tasks of joining a session.
User plugs in their Smart Mic.
Once plugged in they receive the following popup (show here on the left). Clicking start will lead them through the installation process and through account creation. From now on, when they plug in their device, they will be prompted to the login screen.
The user has then completed setup and the app is launched. At this point, more advanced users can fine-tune audio settings to achieve their perfect sound.
Now the user is ready to join a session. They have the opportunity to start their own or join another. In contrast to our original prototype, it is the guest that initiates the joining interaction. For this walkthrough we will imagine that the user’s friend Dylan has just texted them his session id (visit the full prototype for the flow of starting a new session). Clicking on the session button, the user can input the code and join (with Dylan’s permission)
The user now enters the jam session, where they can see their audio and that of their collaborators. If there are multiple other collaborators, the input of one will be highlighted in green with the average input of the others in background grey.
From here they can access a menu of participants and change who is highlighted. This is the biggest change from our original mockup, which only supported two collaborators. This was also motivated by a desire to make the interface more personable. When they are finished, they can leave the session by clicking the bottom right button.
Wondering what happens in a scenario we didn’t cover? Head over to the full prototype and find out for yourself!
Going Forward
Musicians practice in-person because they rely on the specific audio cues that being in-person gives. The timing across instruments has to be perfectly aligned and each instrument must be balanced against the whole ensemble. However, a virtual platform would allow musicians to be much more flexible. Location would no longer be a barrier to jamming together. Our smart mic set out to fix the two most salient issues we discovered in research of current virtual platforms: audio lag and complex setup. Through our hard-wired mic connection, the lag which many musicians that we surveyed had mentioned would be removed. Because our mic has an accompanied streamlined platform, we also reduce the barriers to setup that respondents of our survey identified. The smart mic would allow a trio composed of members as far apart as New York, Moscow, and Tokyo to play live together with a 5 minute or less setup time.
Meet the Team
Wadji Boubakri (Designer)
Anika Padwekar (Designer, Communications)
Natalie Parry (Designer, Project Management)
