Outcome

Intention

The Moody Frame is all about connecting to friends and family through the experience of music. Combining subtle light output of the Ambient Orb and the smart, but not intelligent pictures displayed in a smart frame, The Moody Frame turns pictures of friends and family into an interactive IoT device that subtly shares your "music mood" using light to outupt what kinds of music you are listening to. Additionally, if you are listening to a song you really like and want to share with friends, you can press a button on The Moody Frame which will subtly broadcast that to your friends with in a cascade of colored light, and automatically add it to a shared playlist you have set up.

We started off this project knowing that we wanted to create a device that revolved around music. We’re all music lovers and enjoy listening to and sharing music with each other. Music is a powerful tool for expressing emotions and can connect people across languages and cultures.

We also knew that we wanted to create a device that not only connects people but also allows people to communicate with each other.

We started with an idea that we had come up with during one of our in-class co-creation sessions which was a digital frame that cycles through photos of friends and family who we have not communicated with for a certain amount of time. What we liked about this idea was the photo frame aspect which is a throwback to our childhood when special people and memories in our lives were commemorated by being framed. Since we wanted to create a device that connected people, we decided to use the photo frame form where a person can put a Polaroid of the person they want to be connected to in the device.

We knew we wanted to create a device that connects people through music, but felt that creating an audio IoT device would feel intrusive and demanding of our attention, which goes against the principles of glanceability. Therefore, we decided to take music as the input and lights as the output.

Historical Case

We were inspired by two devices for this project - The Ambient Orb and the Good Night Lamp. We really enjoyed the glanceable nature of the Ambient Orb and how it was able to communicate information in a simple and non-intrusive manner. The Ambient Orb was our inspiration for how we wanted to use light in our device - simple and elegant.

Additionally, we liked how the Good Night Lamps used light as a visual cue to communicate across far distances and allowed for social connectedness through the network of lamps. We wanted to replicate the concept of connecting photo frames together and using light as a communication tool.

Approach

Initially, we started with just a single device, which would retrieve two pieces of information: 

1) Currently playing song from Spotify

2) Currently playing song's genre

Based on the genre of the currently playing song, the frame would display a specific color (we had created a genre-color mapping in the code).

We realized that there were shortcomings to just using one device, so we decided to create another device that would communicate with the first device. This second device would show what genre of music the first device's owner was listening to. In this way, it could 'communicate' the mood of one person to the other. Obviously, there are limitations to using genre as a way to communicate mood, as different genres can have very different feelings associated with them. We decided to use genre, though, because it can generally communicate overarching mood (think classical music = relaxed, pop = high energy, rock = high energy, cool jazz = relaxed, etc.)

An alternative would have been to use other pieces of meta data associated with a song that is captured by Spotify, such as danceability (which would indicate how fast-paced and rhythmic a song is).

Next, we wanted to leverage the goodnight lamp to communicate songs that make you think of the other person. To do this, we added a button to the frame that, when pushed, would pull the currently playing track and add it to a shared playlist so that the other person with the other frame could receive notification that a song had been added to their shared playlist.

Process

Below, we included a process for how the frame works.

With respect to a reflection on the process itself and the many iterations, refinements, and challenges:

Initially, we wanted to pull the 50 most recently played songs and display the ratio of those genres on the neopixel (i.e. if there are 50% pop songs, 50% rock songs, 50% of the neopixels would show as one color and 50% another). However, we realized that there were limitations to the API and how many webhooks could be used, so we had to be very deliberate with the webhooks we used for our device.

Knowing that the particle can only use 4 webhooks in a program, we decided to leverage the 'currently playing' webhook to retrieve the track's artist (and their primary genre).

We struggled initially with the Sptofiy APIs, too. For example, with the add song to playlist webhook, we were unsure initially how to pass the value of one API (get currently playing track) to the other. After some research, we were able to figure out how to execute this task.

Additionally, as we were finalizing the completion of our frames, we ran into an unexpected issue. For some reason, our program was no longer running. We hadn't changed the code, so we began to look at the wiring to see if perhaps there was loose wire. We also reviewed the code to ensure nothing had changed. After nearly an hour of pouring through the code and checking the breadboard and the wires and all the connections, we realized that the only explanation was that the particle had been fried. When we checked the code on another device (and the same wire setup), we realized that this was indeed the case. 

Resulting Prototype

Deliver a functional mock up of the device(s) prototyped using Particle. Document the outcome itself (code, circuit diagrams, photos, design files, 3d models, video demonstrations, etc. as required) and provide a short narrative. This should include: a workflow diagram and a bill of materials (sensors, input devices, actuators, and other components). This documentation should be sufficiently rich to allow anyone to repeat / recreate it.

FEATURE 1:  DISPLAY LIGHT BASED ON GENRE OF CURRENTLY PLAYING SONG

FEATURE 2: ADD CURRENTLY PLAYING TRACK TO A SHARED PLAYLIST

Initial Prototype:

Building on a simple picture frame, an initial 3d model was constructed in Rhino to determine what the look and feel of the device could be. Initially, we took the dimensions of a Polaroid and a reasonably sized frame and attempted to allocate the remaining space for the electronics and Neopixels. The construction is simple: in the model, pink represents translucent acrylic to allow the light to shine through, grey is black acrylic, and orange is 3mm plywood. Unfortunately, after laser cutting and building this first version we realized we needed more space for the device and image, and the lights were too close to the outer frame and did not provide the right diffuse glow effect.

Revised (Final) Prototype:

After testing the first prototype we went back to the drawing board to make some adjustments. First, we decided to enlarge the frame to accommodate the entire Polaroid and not just the image portion. Next, after testing the Neopixels and translucent acrylic, we determined that the LED's needed to sit 10mm from the edge of the acrylic to give the correct light effect. Using this as a guide, we revised the frame, and added some internal tabs for the plywood component to allow for a tight press fit that would keep the device closed, but still operable. Last, we developed two types for the devices, one wall mounted, and one desk mounted. The difference between the two devices rests in how they meet the horizontal plane. In the desk mounted version, the device is raised off of the horizontal surface to allow an even cast of light on all sides.

Bill of Materials:

It is recommended to build a pair of devices, this list allows you to build a minimum of 2 devices:

Circuit Board

Video

Conceptual Design: 

If we had more time we would consider the following enhancements:

Increase the interactive potential with Spotify: Make a more seamless connection with Spotify that allows you to log in once and not have to refresh the oauth token every hour. With this approach we could allow users to set up their own color patterns related to genres that they could then share with their friends.

Consider adding more dynamic light display. Instead using a static color to display mood/genre, the device could be improved by  incorporating other elements such as danceability or BPM, which could be used to flash the lights at a certain rate.

Add additional feedback loops to allow friends to communicate more effectively. This could be ways to add additional interactivity through other points of interaction with the device, or other light patterns that may showcase or signal other moods. One specific option is to create a "together" mode that allows connected individuals to team up for a visual listening session through the device where lights move to the beat of a song that a friend is currently playing.

Iterate on the form: Is a picture frame the right form factor? We would experiment with alternatives to see what is more effective. It would be great to include a smart element to the picture display to allow users to swipe through different friends and see what they are listening to visually.

Reflection and Critique:

As predicted in our premortum, our biggest challenge was indeed configuring the Spotify API. The circuitry was relatively simple; however, we did have some difficulty in soldering the neopixels and ensuring that they would fit within the specifications of the frame.

While we believe that the value this product exists, there are certainly shortfalls to our approach - and several ways in which it could be improved. Using genre to communicate mood may not be the most effective approach, as genres do not necessarily reflect mood. Some genres can better be generalized in reflecting a mood, such as EDM, but other genres, such as rock, pop, and blues, are more nuanced and therefore less indicative of mood. Instead, we would have wanted to explore using other elements associated with a Spotify song, such as BPM or danceability, to communicate one's mood. Higher BPM would likely indicate a more elevated mood whereas lower BPM would indicate a more relaxed mood. The same is true for danceability. Nonetheless, we think the frame in its current state, and in the context of the time we had to work on it, poses a powerful way to communicate with your friends. It sits passively in the background and informs you of what your friends are listening to, their 'mood,' and also allows you to communicate to your friend(s) that you're thinking of them via the addition of a song to a shared playlist.

Another avenue we wish we had more time to explore is privacy. One question that comes to our minds if what if you don't  want your friends, or others, to see what you're feeling or listening to? Is there some way to receive incoming messages but prevent outgoing messages? This is a feature we would want to explore in future iterations; though, as for the immediate version, one could argue that privacy can simply be addressed by turning the frame off.

One thing we really enjoy about this communication approach was that it could fail gracefully. If the IoT mechanisms were to fail, the frame would still be fully functional - that is, it would still function for what it was ultimately intended for - to hold and display a picture.

The value of looking back at overlooked, forgotten, or otherwise older works has been infinitely valuable in our approach. We were able to apply different features that worked well and learn about what didn't work well (and making changes accordingly). We were able to synthesize these features into an ordinarily functional device into one that was enchanted.

Acknowledgements & References: 

Provide a list of academic papers, articles, videos, etc that you make reference to in your documentation.

Rose, David. Enchanted Objects: Innovation, Design, and the Future of Technology, Scribner, New York, NY, 2015, pp. 173–178

Spotify APIs:  https://developer.spotify.com/documentation/web-api/reference/#/

Ambient Orb: http://www.ambientdevices.com/about/consumer-devices

DioT Lab Website: https://diotlabs.daraghbyrne.me/

3D files

3D model & Laser cut files available on request, they will not upload to the ideate website.