Outcome

Prototype

Exhibit Design

For the exhibition, we decided to build a story that showed the progression of the development, release, and recall of the Aura Carbon. Since we were telling a time-based story, we decided to utilize multiple touchpoints as a way to focus on different elements of the story. We also decided to incorporate a timeline which would be an easy way for the viewer to see the story in entirety. 

Designing/testing the exhibit space

Aura Carbon Prototypes

We prototyped 3 versions of the Aura Carbon for our exhibition to help support the story we were telling.

(1)  A model to represent a first prototype of the Aura Carbon that was built by the Aura engineers. 

The purpose of this model was to support the development phase of the Aura Carbon. We decided to make the prototype look distinctly different than the product released version of the Aura Carbon. This contrast makes the product released version look for real and finished. 

(2) A model to represent the first publically released version of the Aura Carbon. 

The purpose of this model was to support the released portion of the story. This prototype was to be used for the interactive/conversational element of our exhibit design. Because this was supposed to represent a commercially released product, we 3d printed and finished it by sanding it smooth.

Making the Aura Carbon Model

(3) A model to represent a hacked version of the Aura Carbon that was used in the financial fraud scandal that was part of our timeline story

The purpose of this model was to support the hacking scandal portion of our story. We made the prototype look like someone tried to crack into it by making a finished version and breaking into ourselves. We also incorporate things like wires coming out of it to imply it was hacked. 

Physical Computing Prototype

For the final product, we added a few lighting effects to communicate different states of the device, including a listening state, a speaking state, and a rest state. We used a Particle Photon and NeoPixel 24 to prototype the LED lighting effects. The listening state is a spinning ring; the speaking state is a randomly changed color from blue to purple that matches with Aura's branding; the rest state is a slow breathing effect that also transitions from blue to purple color. 

Conversational Interaction

For the conversational interface, we decided to pre-record a few key scenarios rather than prototype a functional voice recognition system. Because it is hard to anticipant how the audiences interact with our device, and we expect the exhibition would be very noisy for real voice recognition. But allowing audiences to trigger different scenario individually, we can demonstrate all the key contexts of how this device would be used.

Then we finalized four key scenarios, including onboarding of voice mimicking, access to a user's personal data, reserve a restaurant and answer the user's mom call. We write down the scripts for each dialogue and record audios for each. 

We also want to build a visual interface that helps audiences see the conversation, and understand the role of each dialogue.  As such, we built a processing prototype, that can display captions along with the recorded audios. The processing code reads audio files paired with text files, and the text files contain all the scripts, roles, as well as the timing of every dialogue. Therefore, when an audio file is playing, a related caption will display simultaneously.

In order to allow audiences to interact with the dialogue easily, we used a physical dial to trigger it. Daragh borrowed us a PowerMate dial, and we coded it to trigger different contexts of dialogue. 

Ultimately,  we use projector mapping to project images onto the table from the top down. So the image aligns with the physical objects as if the projected image is an extension of the objects. The processing code is also real-time communicate with Particle Photon, so it can trigger different lighting mode while playing the dialogue.

Timeline and Supporting Articles

Product Video

We developed a product video that was incorporated into the timeline to support the product release Aura Carbon prototype. This video mimicked the product videos that current tech companies release to promote their products. The purpose of the video was to help show the tension and differences between the intention of the product and the misuse of it.

Precedents

To build our future scenario, we took cues from current applications of technology, specifically Google Duplex. We decided to use the Google Duplex was an existing technology to build off of because it provided a basis to help ground our idea for the use of the technology we were working with. We thought about how this technology would continue to expand and evolve with the advancement of natural language processing and artificial intelligence. Currently, Google Duplex can sound more human-like and can be used to do things like book an appointment. We believed that the human-like sound would eventually become advanced enough to mimic the voice of the user. 

Process

Early on we had a clear idea of the topic and question we wanted to explore for this project. We focused on refining that idea and working in a scope that would make it easy to execute and build a scenario that would feel believable. We decided to have a narrowed scope because we felt that it would be more approachable and believable especially because we wanted to design a product that was a finished released product as opposed to a research experiment. Originally we wanted to have the design of the actual product be more organic looking, like a pebble shape. However, once we decided on using the round led ring, we changed our design of the shape so that it would work better with the movement of the light. We also originally had our backboard timeline with dates, however, after our first crit, we decided to remove the dates and use "versions" instead, which references the product cycles that current tech companies use. 

Reflection

Reflect on making this project. What did you learn? What would you do differently? Did you get where you wanted to? If not, why not? What do you need to get there, etc?

1. One thing works really well is displaying the dialogue caption on the table. It helps audiences understand who is speaking, and clearly see what is saying in a noisy environment. Also, triggering the prerecorded audio avoids users from using voice to interact with a voice system, but still convey the context clearly. We've seen many audiences enjoyed the interactivity and spent a lot of time on our booth.

2. What did not work as expected is that we have multiple pieces of content, so it is confusing what to look at first and what to look at the next. Our initial expectation is to let our audiences read the introduction and watch the product video to get a sense of what the product is about, then listen to the dialogue and fake media news to understand how this is being misused. But we found that most audiences directly interacted with the dial, so it is difficult for them to understand the rationale and concept at the very first time. 

3. Another big challenge is the multiple sound pieces are playing at the same time. We have a product video and several dialogues. They are all very loud and noisy,  so we have to turn down the volume of product video to make the dialogue audio clear. Without an audio narrative, the product video does not make much sense. Therefore, we believe using a headphone and adding captions for our product video would largely solve the problem.

Attribution and References

Reference any sources or materials used in the documentation or composition.