Stay There

Made by Angie (Chuyi) Wang

Stay there. Don’t move. Don’t take notice of the devices following you. Tracking your every move and thought. If you stray, we will warn you, it will be uncomfortable. Scary even. This project is meant to convey that although its become a more mainstream concept to be aware of giving away data to known platforms, there are so many more devices, or eyes, constantly watching and manipulating us.

Created: May 8th, 2023

0
Stay There: Eye
Img 9610.thumb Angie Wang and Anya Singhal (2023)
0

Credits

This is a project built by Angie Wang and Anya Singhal.

Thank to -

Daragh Byrne, our instructor, for introducing the theme of spooky technologies and the 'haunted house'  and led us to think about deeper meanings. We also appreciate him inviting a lot of professors and professionals to our critique and the exhibition, so we gain a lot of powerful feedback.

Zhenfang Chen, our TA, for providing technical support and useful feedback. 

Lab Assistant in TechSpark. for providing technical help.

We also appreciate all professors and professionals who discussed our project with us during the critique and exhibition. 

0

Description


Intentions

In the information age, it is common knowledge that smart devices often collect and even steal personal information. However, the use of various AI models in everyday software and devices is not always apparent to users, leading to unawareness of manipulation by AI. For instance, some software only presents content it predicts will interest the user. Prolonged human-machine interaction can make individuals narrow-minded and increasingly dependent on machines due to their influence. The project aims to highlight the risks of manipulation by AI and the resulting narrow-mindedness and dependence on devices that can occur.

The big idea behind the project is to raise awareness about the potential negative impacts of prolonged human-machine interaction, particularly with the increasing use of AI models in everyday software and devices. The motivation behind the project is to address the growing concern about the use of AI in everyday life and its potential impact on society. With the increasing ubiquity of smart devices and the exponential growth of AI, it is crucial to raise awareness about the potential risks and encourage individuals to take proactive steps to protect themselves.

Ultimately, the project is driven by a desire to empower individuals to make informed decisions about their use of technology and ensure that they are not unwittingly subject to manipulation by AI. The project aims to foster a more responsible and mindful approach to technology that benefits individuals and society by promoting greater understanding and awareness.

0

At the Exhibition...

This project comprises three distinct parts that work together to create an interactive and thought-provoking experience for the audience.

0

I. Stay There: Phone

"Oh, hello there!

Make sure to grab your phone! Can’t leave without it..."

 This first part involves using a phone provided to the visitor upon entering the space. The phone screen displays a message stating that "the connection between the mobile device and the smart home server has failed," raising questions about the reliability and security of smart home technology.

0

II. Stay There: Eye

"I am watching you. Stay there.

If you leave, I will warn you: dangerous things will happen. Things that will make you scared. Uncomfortable.

You know I exist. I have your focus. I watch everything. Just focus on me and don’t stray."

In the second part of the project, visitors will encounter a large mechanical eyeball that tracks their movements as they walk around the room. When visitors stand in the location designated by the machine, the eyeball turns blue, indicating a normal operation. However, if visitors attempt to leave the room or move towards the exit or entrance, the eye turns red, warning them not to proceed. 

0

III. Stay There: Screen

"Oh whoops! Ignore us, just turn back around and go about your life. 

  We know everything, but you don’t need to know that."

Finally, the third part of the project consists of an electronic screen displaying an array of eyeballs that follow visitors' movements as they move around the room. This creates a sense of being watched and monitored by smart devices that people are normally unaware of, raising questions about privacy and surveillance in modern society.

0

Process


Throughout the development of this project, we underwent multiple iterations and refined our approach to improving the final outcome. How the project worked was that we created a grid with four UWB(Ultrawide Band) chips placed in the corner of the room and send signals to the handheld chip. The handheld chip will send the location data to the Arduino Nano according to its position in the grid. The motor and the eye in the P5 controlled by the Arduino Nano will move according to the received location data. Our process can be broken down into three key parts.

I. Stay There: Phone

The first part involved the production of the 'cell phone' device. We first laser-cut and 3D-printed the phone shell and installed the UWB receiver chip for positioning in the phone. However, during the exhibition, we encountered issues with the UWB signal reception which resulted in us having to repeatedly disassemble the phone and reset the chip to reconnect the signal. Despite this setback, we resolved the issue and ensured the device functioned as intended.

Additionally, we designed our screen cover to have a message saying "Welcome Home! The connection between the mobile device and the smart home server has failed, " indicating that the phone shouldn't be communicating with anything in the room. However, the phone is actually communicating with our 'smart home'. By this, we want to let audiences feel a sense of being unsecured but being told that they are safe. 

0

II. Stay There: Eye


We went through a thorough process to create the mechanical eye for our project.

Our first step was to 3D print the shell of the eye, but we quickly realized it wasn't visible enough in the room. To address this issue, we decided to scale the model up and print it in two parts, which took a total of 3 days(34 hours + two times failures due to troubleshooting) to get the desired result. 

Next, we experimented with different materials and spheres for the eye until we discovered the heat sinker from the iDeate lab, which we decided to use as the pupil. However, the heat sinker's weight posed a challenge as the servo motor we used to control the rotation was relatively light and small. We sought assistance from the lab assistant at Techspark, and after some punching, sanding, and gluing, we finally managed to assemble the eye.

Finally, is to assemble the eye in the exhibition. We faced some difficulties attaching the eyes to the motor on the exhibition table due to their weight. Therefore, we decided to glue them to a black acrylic plate to keep them upright. During the process, we came up with the idea of using hot glue to create the effect that the eyes had legs and were trying to climb out of the table. We implemented this idea to give the eyes a more "animistic" appearance.

0

III. Stay There: Screen

Our project's third and final part involved creating a screen of 20 eyeballs using P5.js. We received feedback during the critique phase suggesting we make the eyeballs more realistic, but we ultimately decided against it. We intended to represent intelligent devices that go unnoticed daily, always watching us but not necessarily as obvious as a mechanical eye. Therefore, we wanted to distinguish them from the realistic mechanical eye we had created earlier.

One of the challenges we faced in this part was making the pupils of the eyes follow the audience's movements and turn accordingly. This involved connecting P5 and Arduino Nano through serial communication and ensuring that the Arduino Nano received data from the UWB sender to control the motor to rotate the mechanical eyeball and P5 eyeballs. However, our connection was unstable throughout most of the exhibition, requiring us to restart the chip constantly.

0

Build

Bill of Materials

  • Arduino Nano
  • UWB Device(Sender and Receriver)
  • P5.js + Display Monitor
  • Servo Motor
  • Heat Sinker
  • Neopixel Ring
  • 3D Printing + Laser Cutting
0

Diagram

As mentioned, the project worked by creating a grid with four UWB(Ultrawide Band) chips placed in the corner of the room and sending signals to the handheld chip. The handheld chip will send the location data to the Arduino Nano according to its position in the grid. The motor and the eye in the P5 controlled by the Arduino Nano will move according to the received location data. Here is a diagram of how the UWB locator, Arduino Nano board, the 'phone'(hand-held UWB receiver), and the P5 screen communicate with each other when the user is holding the 'phone' and walks around the room.

0

The UWB transmitter operates by emitting numerous pulses throughout a broad range of frequencies. A receiver that corresponds to it picks up the signal and decodes the data by detecting a familiar sequence of pulses transmitted by the transmitter. These pulses are transmitted at a frequency of approximately one every two nanoseconds. Below is a graph of how UWB works from how2electronics.com .

0

Codes

0
// P5.js Code

let eyes = [];

function setup() {
  createCanvas(windowWidth, windowHeight);
  background(0);
  for (let i = 0; i < 50; i++) {
    let size = random(20, 100);
    let x = random(size, width - size);
    let y = random(size, height - size);
    let eye = new Eye(x, y, size);
    for (let j = 0; j < eyes.length; j++) {
      let d = dist(x, y, eyes[j].x, eyes[j].y);
      if (d < size + eyes[j].size) {
        x = random(size, width - size);
        y = random(size, height - size);
        eye.x = x;
        eye.y = y;
        j = -1;
      }
    }
    eyes.push(eye);
  }
  serial = new p5.SerialPort();
  serial.open("/dev/ttyUSB0");
  serial.on("data", serialEvent);
}

function serialEvent() {
  let data = serial.readStringUntil("\n");
  if (data) {
    let location = split(data, ",");
    if (location.length == 2) {
      let x = map(location[0], 0, 1023, 0, width);
      let y = map(location[1], 0, 1023, 0, height);
      for (let i = 0; i < eyes.length; i++) {
        eyes[i].update(x, y);
      }
    }
  }
}

function draw() {
  for (let i = 0; i < eyes.length; i++) {
    eyes[i].display();
  }
}

class Eye {
  constructor(x, y, size) {
    this.x = x;
    this.y = y;
    this.size = size;
    this.pupilSize = this.size / 3;
    this.pupilX = this.x;
    this.pupilY = this.y;
    this.angle = 0;
  }

  update(x, y) {
    let d = dist(x, y, this.x, this.y);
    let maxDist = this.size / 2 - this.pupilSize / 2;
    let mappedDist = map(d, 0, maxDist, 0, 1);
    let angle = atan2(y - this.y, x - this.x);
    angle = constrain(angle, -PI / 4, PI / 4);
    this.angle = lerp(this.angle, angle, 0.1);
    this.pupilX = this.x + cos(this.angle) * mappedDist * maxDist;
    this.pupilY = this.y + sin(this.angle) * mappedDist * maxDist;
  }

  display() {
    noStroke();
    fill(255);
    ellipse(this.x, this.y, this.size);
    fill(0);
    ellipse(this.pupilX, this.pupilY, this.pupilSize);
  }
}
Click to Expand
0
//Nano BLE - UWB Sender Code

#include <ArduinoBLE.h>

 // BLE POS Service
BLEService POSService("11a953ef-0809-4d89-9a56-25ab22841ecd");

// BLE Orientation Characteristic
BLEStringCharacteristic POSSensorData("e624a94b-4a14-41f3-ae53-ec99c6025855", BLERead | BLENotify, 512);

unsigned long previousTime=0;           // end time of the last period in microseconds
unsigned long currentTime=0;
unsigned long deltaTime=0;
const unsigned long period=1000;       // time period for pulse counting in microseconds

String c; //some temporary string variable

char s[60]; //snprintf buffer
String pos;

void setup() 
{
  delay(10000);
  Serial.begin(115200);
  Serial1.begin(115200); //dwm1001 require 115200 baud
  Serial1.write(0x0D); //send the first "Enter"
  Serial1.write(0x0D); //send the second "Enter" to get into Shell Mode
  c = Serial1.readString(); //read the response from dwm1001
  Serial1.write(0x6C); //send "l"
  Serial1.write(0x65); //send "e"
  //Serial1.write(0x63); //send "c" lec shows distance to all anchors & position
  Serial1.write(0x70); //send "p" lep shows only position
  Serial1.write(0x0D); //send "Enter"

  // initialize the built-in LED pin to indicate when a central is connected
  pinMode(LED_BUILTIN, OUTPUT); 

  // begin BLE initialization
  if (!BLE.begin()) {
    Serial.println("starting BLE failed!");
    while (1);
  }

  /* Set a local name for the BLE device
     This name will appear in advertising packets
     and can be used by remote devices to identify this BLE device
     The name can be changed but maybe be truncated based on space left in advertisement packet
  */
  BLE.setLocalName("POSSensor");
  BLE.setAdvertisedService(POSService); // add the service UUID
  POSService.addCharacteristic(POSSensorData); // add the POS characteristic
  BLE.addService(POSService); // Add the POS service
  POSSensorData.writeValue("hello"); // set initial value for this characteristic

  /* Start advertising BLE.  It will start continuously transmitting BLE
     advertising packets and will be visible to remote BLE central devices
     until it receives a new connection */

  // start advertising
  BLE.advertise();

  Serial.println("Bluetooth device active, waiting for connections...");  
  String address = BLE.address();
  Serial.print("Local address is: ");
  Serial.println(address);  
  Serial.flush();  
} 

void loop() {
  // wait for a BLE central
  BLEDevice central = BLE.central();

  // if a central is connected to the peripheral:
  if (central) {
    Serial.print("Connected to central: ");
    // print the central's BT address:
    Serial.println(central.address());
    // turn on the LED to indicate the connection:
    digitalWrite(LED_BUILTIN, HIGH);  

    // check the POS every period
    // while the central is connected:
    while (central.connected()) {
      currentTime = micros();
      deltaTime = currentTime-previousTime;   

      /* Check if the time in between samples equals sampling period 
         before taking a new sample */
      if (deltaTime >= period) {
        /* Display the current time in microseconds */
        Serial.print(currentTime);
        Serial.print(",");
        previousTime = currentTime;         

        // Stores each POS data type in a string 'buff' // 
        String buff;
        pos = Serial1.readStringUntil('\n');

        // Sends buff using the BLE connection
        buff += String(pos);
        Serial.println(buff);
        POSSensorData.writeValue(buff);
      }
    }
  }
        
  // when the central disconnects, turn off the LED:
  digitalWrite(LED_BUILTIN, LOW);
  Serial.print("Disconnected from central: ");
  Serial.println(central.address());  
}
Click to Expand
0
//Nano BLE - UWB Receiver Code

#include <ArduinoBLE.h>
#include <Servo.h>
#include <Adafruit_NeoPixel.h>

Servo myservo;  // create servo object to control a servo

//int pos0 = 0;    // variable to store the servo position
int pos = 0;
int servo = 3;


#define LED_PIN D3
#define LED_COUNT 16

Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  Serial.begin(115200);

  myservo.attach(A2);  // attaches the servo on pin 9 to the servo object  

  strip.begin();
  strip.show();
  strip.setBrightness(100);

  // begin initialization
  if (!BLE.begin()) {
    Serial.println("starting BLE failed!");

    while (1);
  }

  Serial.println("BLE Central - Receiver");
  Serial.println("Make sure to turn on the device.");

  // start scanning for peripheral
  BLE.scan();
  pinMode(LEDR, OUTPUT);
  pinMode(LEDG, OUTPUT);
  pinMode(LEDB, OUTPUT);
  pinMode(LED_BUILTIN, OUTPUT);

  digitalWrite(LED_BUILTIN, LOW);         
  digitalWrite(LEDR, HIGH);     
  digitalWrite(LEDG, HIGH);     
  digitalWrite(LEDB, HIGH);       
}

void loop() {
  // check if a peripheral has been discovered
  BLEDevice peripheral = BLE.available();

  if (peripheral) {
    // discovered a peripheral, print out address, local name, and advertised service
    Serial.print("Found ");
    Serial.print(peripheral.address());
    Serial.print(" '");
    Serial.print(peripheral.localName());
    Serial.print("' ");
    Serial.print(peripheral.advertisedServiceUuid());
    Serial.println();

    setRingColor(false);
    delay(200);
    setRingColor(true);
    delay(200);
    // Check if the peripheral is a POSSensor, the local name will be:
    // "POSSensor"
    if (peripheral.localName() == "POSSensor2") {  
      // stop scanning
      BLE.stopScan();

      monitorBLEperipheral(peripheral);

      // peripheral disconnected, start scanning again
      BLE.scan();
    }
  }
}

void monitorBLEperipheral(BLEDevice peripheral) {
  // connect to the peripheral
  Serial.println("Connecting ...");
  if (peripheral.connect()) {
    Serial.println("Connected");
  } else {
    Serial.println("Failed to connect!");
    return;
  }

  // discover peripheral attributes
  Serial.println("Discovering service 0xffe0 ...");
  if (peripheral.discoverService("11a953ef-0809-4d89-9a56-25ab22841ttt")) {
    Serial.println("Service discovered");
  } else {
    Serial.println("Attribute discovery failed.");
    peripheral.disconnect();

    while (1);
    return;
  }

  // retrieve the POSSensorData characteristic
  BLECharacteristic POSSensorData2 = peripheral.characteristic("e624a94b-4a14-41f3-ae53-ec99c6025999");
  
  // subscribe to the simple key characteristic
  Serial.println("Subscribing to POSSensorData characteristic ...");
  if (!POSSensorData2) {
    Serial.println("no POSSensorData characteristic found!");
    peripheral.disconnect();
    return;
  } 
    else if (!POSSensorData2.canSubscribe()) {
    Serial.println("POSSensorData characteristic is not subscribable!");
    peripheral.disconnect();
    return;
  } 
    else if (!POSSensorData2.subscribe()) {
    Serial.println("subscription failed!");
    peripheral.disconnect();
    return;
  } 
    else {
    Serial.println("Subscribed to POSSensorData characteristic");
  }

  while (peripheral.connected()) {
    // while the peripheral is connected

    // check if the value of the simple key characteristic has been updated
    if (POSSensorData2.valueUpdated()) {

// THIS SECTION CONVERTS THE RECEIVED CHARACTERISTIC FROM UNSIGNED CHAR TO STRING // 
      String str;
      int length = POSSensorData2.valueLength();
      const uint8_t* val = POSSensorData2.value();
      str.reserve(length);

      for (int i = 0; i<length; i++){
        str += (char)val[i];
      }

      Serial.println(str);

      if (str.indexOf("POS") != -1) {
        int posIndex = str.indexOf("POS");
        int firstComma = str.indexOf(',', posIndex + 1);
        int secondComma = str.indexOf(',', firstComma + 1);
        int thirdComma = str.indexOf(',', secondComma + 1);

        float x = str.substring(firstComma + 1, secondComma).toFloat();
        float y = str.substring(secondComma + 1, thirdComma).toFloat();
        float z = str.substring(thirdComma + 1, str.indexOf(',', thirdComma + 1)).toFloat();        
        
        pos = 180 - x/2.85 * 180;
        myservo.write(pos);
        delay(100);

void setRingColor(bool off) {
  for (int i = 0; i < strip.numPixels(); i++) {
    if (pos/3 < pos< pos *2/3 ) {
           strip.setPixelColor(i, strip.Color(off ? 0 : 0, 0, 255));
        }
        else {
            strip.setPixelColor(i, strip.Color(off ? 0 : 0, 0, 255));
        }
      strip.show();  
  }
        

        // if (0 <= x && x <= 2.85 && 0 <= y && y <= 3.15) {
        //   digitalWrite(LED_BUILTIN, HIGH);
        //   digitalWrite(LEDR, HIGH);
        //   digitalWrite(LEDG, HIGH);
        //   digitalWrite(LEDB, HIGH);
        // }

        // if (0 <= x && x <= 2.85 && 3.15 <= y && y <= 6.3) {
        //   digitalWrite(LED_BUILTIN, LOW);
        //   digitalWrite(LEDR, LOW);
        //   digitalWrite(LEDG, HIGH);
        //   digitalWrite(LEDB, HIGH);
        // }

        // if (2.85 <= x && x <= 5.7 && 0 <= y && y <= 3.15) {
        //   digitalWrite(LED_BUILTIN, LOW);
        //   digitalWrite(LEDR, HIGH);
        //   digitalWrite(LEDG, LOW);
        //   digitalWrite(LEDB, HIGH);
        // }

        // if (2.85 <= x && x <= 5.7 && 3.15 <= y && y <= 6.3) {
        //   digitalWrite(LED_BUILTIN, LOW);
        //   digitalWrite(LEDR, HIGH);
        //   digitalWrite(LEDG, HIGH);
        //   digitalWrite(LEDB, LOW);
        // }
        
        Serial.print("x: ");
        Serial.print(x);
        Serial.print(", y: ");
        Serial.print(y);
        Serial.print(", z: ");
        Serial.println(z);
      } else {
        Serial.println("POS not found in the data string");
      }      
    }
  }
  

  Serial.println("BLE disconnected!");
}
Click to Expand
x
Share this Project

Courses

48-528 Responsive Mobile Environments - Spooky Technology

· 12 members

As part of this project-based course, we’ll get hands-on with emerging technologies, concepts and applications in the internet of things through a critical socio-technical lens. Over it’s 14-weeks,...more


About

Stay there. Don’t move. Don’t take notice of the devices following you. Tracking your every move and thought.

If you stray, we will warn you, it will be uncomfortable. Scary even.

This project is meant to convey that although its become a more mainstream concept to be aware of giving away data to known platforms, there are so many more devices, or eyes, constantly watching and manipulating us.