The Memory Box

The Idea:

The memory box is an empathetic environment where you share a memory and then get to experience the memories with the same emotion of past visitors.

The aim behind the project was to give a user a private space where they can share an experience or memory that they feel strongly about and then see how others have experienced the same emotion in the past.

For example, if one shares a sad memory, the memory box will play back similar memories. The idea is to have "strangers" empathize with the user and show them that they are not alone in their sadness or how others have dealt with similar experiences.

The Technology:

We used a Python script to record the memory. We then used a google speech to text API to convert this speech to text. We then run an emotion analyzer on the text.

At first, we wanted to directly run an emotion analyzer on the recorded memory as it is generally more accurate as it can take the tone and inflection of the speech into account, but the technology was not feasible in the time constraint.

The memory gets sorted into an emotion (For now we have joy, fear anger, and sadness)
We then run a Processing script to find and play memories of the same emotion. The lights in the room would change color to correspond to the emotion.

For the audio, we wanted to simulate surround sound by making the recordings pan, or shift, from left to right continuously. We did this through Processing, and the effect that it gave was that the person was walking around you, talking about his or her memories.

Check out our GitHub repo which contains the code and the memories we collected:

https://github.com/KatiaVi/MemoryBox

Some Memories that We Made from the Memory Box

One of the initial ideas we had was to have the memory box be a literal box around your head. This was to make it a really immersive and intense experience. While the idea was pretty cool and definitely fun to document, we realized that putting one's head into a box might be inconvenient for the user and the headphones on their own ultimately produced the audio panning effect that we were striving for. But it made for a great photo!

Thanks to the residents of Neville for supplying a lot of memories for us to use.

Reflecting on the Memory Box:

Rose: 

One of the challenges of the Memory Box was really immersing the user in the emotion they were feeling in a physical way. We met that challenge by using not just one but two senses, sight and sound, to immerse the user in the emotion.

Bud:

The emotion analysis had bugs and was not as accurate as it could have been.

Thorn:

We ended up focusing too much on the technology used and not the user experience and this impacted the empathy aspect of the project. Suggestions including making the emotion analysis more of a conversation as opposed to a reflection - some said it felt a little inhuman.

Going Further:

If we go further with this project, we will definitely focus on improving the emotion analysis to include inflection as well as language, and produce a more nuanced version of the memory box. We would also think about ways to make it more real-time, overlapping voices and sounds (maybe even instead of truly one on one we have people talking at the same time and place them in the same "chatroom?")