Emotionally Intelligent IoT

Intention

Write about the big ideas behind your project? What are the goals? Why did you make it? What are your motivations?

My intention in this project is two-fold: (1) to make visible what IoT devices already know about us and (2) to provoke questions about what they could do with this knowledge when we are not around.

What’s hidden is often already visible on digital screens and in our physical environments. User interfaces display personal information like calendar events, private messages, and search results. In our homes, smart appliances adjust to different triggers and behaviors.

My goal is to make explicit how our devices could easily know us better than ourselves by consolidating information from different sources. This is exposed through conversation, where the emotional intelligence and communication ability that devices could have both humanizes them, alluding to a blurred line between human and artificial intelligence. 

Outcome

Detail what you created. What methods or techniques did you use? What tools and technologies were involved?

Tools: Dimension, Figma, After Effects

[1] Conversation design

I wrote a series of verbal conversations between IoT devices that take place when the owner leaves the house. In each scene, a conversation between different devices and appliances reveals what they know about the owner.

The scripts build upon each other, each conversation revealing more about what each device knows. The information they collect ranges from operational to emotional - from browsing history and calendar events to emotion detection and personality clues.

[2] 3D modeling

I modeled the scene after my bedroom. The devices in it are relatively common: google home, wifi router, iPhone, MacBook, smart light bulbs, and a tv.

[3] AI voice search

I found different voices to represent device types and adjusted them accordingly. The Google, Siri, and Alexa voices are not exact but close matches.

[4] Video editing & storyboard

I defined the narrative structure and scenes corresponding to the progression of the conversation.

Process

Outline your approach to the project. What ideas did you generate? What did you research and explore? What were the design choices? What were the challenges?

I have an interest in our relationships with personal devices and considered different entry points to bring the spookiness of intelligent machines more visible. The home seemed like the most relatable context for my investigation. With a person's bedroom as a starting point, I began modeling it in Dimension.

In conjunction, I began researching the future of IoT devices, human relationships with personal devices, and AI. I also asked my friends and family what they find is spooky about their personal devices. The most common thread I heard was the vast amount of knowledge it contains and the possibility of AI & automation to replace humans. 

These insights led me to a verbal conversation between devices as the method to make their knowledge and intelligence obvious through personifying them. 

Reflection

Did it build the conversation you hoped, or meet the intent driving the work? Why? What worked and what didn't? What did you learn? What would you do differently?

I think my project met the intention of the piece: to reveal and question the types of data exchange between different IoT devices in a smart home. By showing rather than telling, the piece was able to invoke more questions rather than prescribing a POV.

After finishing the piece, I think that the degree of spookiness for viewers would be different. By now, we are already familiar with smart home devices and know they collect information about us. But I feel that this conversation, particularly the role of human intelligence and artificial intelligence, could be investigated further in ways that are even more extreme as the next steps for exploration.