Outcome

Revised think piece: Portal to your Brain

According to computer scientist and CTO of Xerox Parc, Mark Weiser, “the most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it” [1]. Similarly, the concept of calm technology [2] first introduced by Weiser and Seely Brown intends to make the technologies that typically compete for our attention provide true calmness and comfort. Instead of constantly being at the center of our attention, a calm technology can move from the periphery of our attention, to the center, and back so that people can most fully command technology without being commanded by it. This idea of technology as being both a force for boosting productivity and allowing people to find comfort and relaxation is an intersection we found particularly interesting for further provocation.

Today, people interact with devices primarily through touch and increasingly through voice user interfaces or gestures for extended reality. But what if we were able to control technology in the most seamless and accessible way possible–with our minds? The brain-computer interface (BCI) can provide a direct communication link between brain signals as the input to control a computer or external devices as the control. Of many emerging technology capabilities that feel like magic, the idea that technology can become integrated with human consciousness is perhaps the most advanced of all. The capability to control technology with our thoughts alone has the potential to offer immense flexibility and accessibility to people in fields like rehabilitation, affective computing, gaming, and neuroscience [3]. Less commonly explored, however, are the BCI’s opportunities to improve life, including work life and life and home.

Accelerated by the COVID-19 pandemic, Conflict between work and life demands have forced millions of people around the world to work from home. This experiment struggled to gain traction prior to the pandemic [4], yet remote work and hybrid collaboration models have proven to be more desirable for employees, offering them increased flexibility and autonomy. In particular, the home office has become a unique backdrop inducing both comfort and stress, and where awareness of self and sense of place easily become ambiguous. Given this context, we were led to ask, “how might we influence a person's mindful awareness of self through environmental cues that adjust accordingly to their level of mental focus?”

Interestingly, the use of BCI technology can truly capture human intention by identifying our subconscious reactions more quickly than we can process them. Rather than pressing a button or using a voice invocation, user inputs are our thoughts. As a result, BCI can bypass accessibility, inclusivity, and mobility concerns that other user interfaces are still working to accommodate for. As they are projected to move towards the mass market, we envision new possibilities where BCI can allow people to interact with technology in the most natural way possible. Already, there is a growing number of professionals leveraging BCI tools to improve their performance at work and to make visible employee’s mental states visible in order to lower stress and monitor attention levels [5].

In response, Portal to your Brain is a project that onboards people to BCI capabilities, where users can begin to familiarize themselves with how to control interfaces using their changes in mental state. Rather than the technology being front and center, the control of the interface is the user’s mind. Through the audio-guided experience, users are prompted to focus on their mindful awareness of themselves. By developing greater sensitivity toward our own mental states, we also become more in control of BCI. This technology significantly decreases human contact between devices and can enable technology to truly fade into the background while truly knowing what users want and need. Rather than personalizing user experiences based on identifying patterns in data and generating predictions, it could one day have direct access and accurate communication to our minds. 

Still, ethical and privacy challenges remain. While direct access to brain activity can provide users with another modality of control, brain information is the most intimate and private of all information. Additionally, the capabilities and mechanisms of BCI are hard to understand for an average user [6]. The use cases are currently limited to medical uses and are beginning to expand into other industry verticals. In response, our project aims to provide a starting point and case study for suggesting useful capabilities for becoming more aware of one’s own mental state in order to better control the technology. It also provides an opportunity to reflect on the appropriateness of this technology by envisioning a future when it becomes advanced enough to capture more information about our mental and physical states than we can express ourselves.

[1] Weiser, M. (1991). The Computer for the 21 st Century. Scientific American, 265(3), 94–105. http://www.jstor.org/stable/24938718

[2] Weiser, M., & Brown, J. S. (1997). The coming age of calm technology. Beyond Calculation, 75–85. https://doi.org/10.1007/978-1-4612-0685-9_6

[3] Saha, S., Mamun, K. A., Ahmed, K., Mostafa, R., Naik, G. R., Darvishi, S., Khandoker, A. H., & Baumert, M. (1AD, January 1). Progress in brain computer interface: Challenges and opportunities. Frontiers. https://www.frontiersin.org/articles/10.3389/fnsys.2021.578875/full

[4] Lund, S., Madgavkar, A., Manyika, J., & Smit, S. (2021, March 3). What's next for remote work: An analysis of 2,000 tasks, 800 jobs, and nine countries. McKinsey & Company. https://www.mckinsey.com/featured-insights/future-of-work/whats-next-for-remote-work-an-analysis-of-2000-tasks-800-jobs-and-nine-countries

[5] What brain-computer interfaces could mean for the future of work. Harvard Business Review. (2020, October 6). https://hbr.org/2020/10/what-brain-computer-interfaces-could-mean-for-the-future-of-work

[6] Drew, L. (2019, July 24). The ethics of brain–computer interfaces. Nature News. https://www.nature.com/articles/d41586-019-02214-2

Catalog description

Title: Portal to your Brain

Credits: Gabriel Alvarez, MHCI 22’; Elaine Lu, MDes 23’; and Rohan Sonecha, BSECE 24’.

Project description:
What if the technology that helps you work and relieve stress knows you better than yourself?

Accelerated by the COVID-19 pandemic, domestic life has been altered by the pervasive presence of video telephony applications at home. And while the majority of workers deem the flexibility added onto their lives as something positive, some questions related to privacy remain unresolved.

Portal to your Brain is an exhibition that recreates the home-office space of the future. It proposes an alternate reality where innovations such as holographic displays and brain-computer interfaces become highly integrated into our environments, allowing for smooth transitions between productivity and relaxation.

In the context of the Haunted House, this project raises concerns about the risks associated with technology’s integration with our homes and our minds, the lack of clear boundaries and the users’ need for control.

Letting headsets establish a direct communication path between our brain's electrical activity and any external device, like a holographic projection, might mean granting external agents access to our inner self. Think about your biggest fear, what would you not like others to know about yourself —“Oh, careful not to think about it. Or else we’ll capture it.”

Link to video content only: https://vimeo.com/708013718

Process Logs

1. Margaret Morrison

For the first log, the Team went to the space of the exhibition Margaret Morrison, Room 127. We rested a sheet of reflective acrylic on the wall and projected an image onto it, in between we put a sheet of translucent acrylic at 45 degrees and tried catching the reflection. Thanks to this exercise, we realized how uncomfortable the projector’s glare might be for a user and how important it was for the project to have control of the ambient light.

2. Pepper’s Ghost

For the second log, the Team continued exploring the pepper’s ghost technique, the illusion of capturing a three dimensional image via reflection. Similar to the previous log we started by using a projector, although this time we turned all the lights off. Afterwards, we decided to transition and employ a laptop screen as a way to increase the image fidelity. 

3. Working With Fabric

For the third log, the team pinned a series of fabric sheets and projected onto them from the back to generate redundancy and depth. At this stage, we felt that the effect was good enough to convey three dimensionality, the perception that the image was no longer flat.

4. Live Video Pixelation

In parallel to our hologram experiments, we investigated ways in which we could distort live capture video as a means to provide feedback for our brain-computer interface. The idea being was that a user wearing the headset, would be able to control the projection according to their level of focus (alpha wave).

In order to achieve this, we began exploring Processing’s video library and immediately became interested in pixelation: “Imagine being able to control the fidelity (pixel size) of the hologram with your mind…”

5. Live Video Tessellation

After gaining familiarity with the code, we thought of trying another distortion effect for the live capture video: tessellation. We played with Processing’s mesh library for a while but decided to go back to the previous version because of time constraints.

As a next step, it would be interesting to adapt this code and try to achieve a similar outcome.

6. Muse Meets Processing

For the final run, we designed a dialogue that would guide participants from a relaxed and focused state to a stressed one. For this reason, we thought of integrating the audio protocol with the video feedback to make the experience consistent on the day of the event, which meant going from live capture to pre-recorded footage.

Most importantly, at this stage we began feeding Processing’s pixel size function with the readings from the Muse headset (see diagram).

Technical Documentation

Includes: 1) Journey Map, 2) Exhibit Protocol, 3) Bill of Materials, 4) Source Code

1) Museum Visitor Journey Map

2) Exhibit Protocol  (Final version of script copied below)

Introduction of the exhibit [1 min]

Hello, welcome to your home office of the future. For the next 5 minutes, you will take a study break with us. On your desk, you will find a headband. Please put it on, and follow the voice guidance. The headband you are wearing will be used to track your brain activity. Specifically, we'll be measuring how mentally focused you are. The screen in front of you will provide visual and audio feedback that is reflective of your level of focus. Are you ready?

Activity 1 [2 min]

In this activity, we’ll use the same technologies that are constantly pushing you to be more productive and work longer hours, to give your peace of mind. At least for a moment.

Now, we will begin. Relax your mind. For the next minute, think of spring. Flowers are blooming. Use all your senses to imagine this setting in great detail. Don’t just think fleetingly about this place–really imagine it. What do you see around you? What sounds can you hear? Are you eating or drinking something enjoyable? What can you feel? What scents are present?

Activity 2 [2 min]

For the next activity, think of something that has made you feel anxious lately. This can be a time when you thought that “something bad will happen” or when you thought “you will make a mistake.” It might be finals, family, friends, teamwork, relationships, self doubt, the state of the world around you, or the pandemic. It can be anything that had a great impact on you, and caused you to feel anxiety.

What made you feel this way? Ask yourself: Is my thought based on facts or feelings? If your fear comes true, will it still matter in a week? A month? A year?

Reflection [1 min]

Sit tight, we are now reaching the final activity. Congratulations on making it this far. Now, think to yourself: What if the headband that you are wearing right now can capture all of your thoughts, including everything that crossed your mind while you were wearing the headband.

How does that make you feel? What if the technology that helps you stay calm, to take a break, and to relieve stress… knows you better than yourself. What are your biggest fears? What do you not want us to know about yourself? Oh, careful not to think about it. Or else we’ll capture it. Does it make you want to take off the headband?

End

Thank you for participating. We appreciate your information. 

Please return the headband, and enjoy the rest of the exhibit.

3) Bill of Materials

Roadmap

If you had to build this project over summer 2022 and make it a real, working, interactive prototype / improved exhibit for public audiences, what would be involved? 

If we had more time, we would like to incorporate feedback from our exhibit to improve the experience. This includes exploring different video content based on what resonates with people more, considering other music tracks to play in the background and whether the audio could also correspond to the user's mental focus, and exploring novel work/life applications where BCI could be incorporated to push this idea to more applications. For further provocation, we'd also like to explore more of the ethical components of BCI within the home office context.

Tactically, we hope to also make the experience even more multi-sensory. The video and audio instructions are currently exported as part of the same video. As next steps, we plan to export them as separate files. This way, the guided instructions could be played at a consistent pace and the video being projected could not only become more/less pixelated, but also speed up or slow down more dramatically based on the user's mental focus. Decoupling both aspects would have allowed for an even more compelling experience where the user can feel like they are controlling the projection in more dramatic ways.

Critical Reflection

Documentation of the visitor experience with your installation to articulate what was successful and what could be refined?

As innovation moves forward experiences become increasingly personalized, but the only way that technology can truly keep up with human affects is by having access to the mind. In an all-connected world powered by artificial intelligence, brain-computer interfaces (BCI) might be able to appease those needs. In such a world, what would the role of designers be?

When dealing with these kinds of problems, critical makers build functional narratives to find problems rather than solutions. Therefore, Portal to your Brain utilizes focus and presence to deliver an immersive experience that questions the integration of technology with our homes and our minds.

In a world where your headset might know you better than yourself, what is left of human agency and privacy?