Outcome

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Credits

This is a project built by Angie Wang and Anya Singhal.

Thank to -

Daragh Byrne, our instructor, for introducing the theme of spooky technologies and the 'haunted house' and led us to think about deeper meanings. We also appreciate him inviting a lot of professors and professionals to our critique and the exhibition, so we gain a lot of powerful feedback.

Zhenfang Chen, our TA, for providing technical support and useful feedback.

Lab Assistant in TechSpark. for providing technical help.

We also appreciate all professors and professionals who discussed our project with us during the critique and exhibition.

Move up Delete Move down

Description

Intentions

In the information age, it is common knowledge that smart devices often collect and even steal personal information. However, the use of various AI models in everyday software and devices is not always apparent to users, leading to unawareness of manipulation by AI. For instance, some software only presents content it predicts will interest the user. Prolonged human-machine interaction can make individuals narrow-minded and increasingly dependent on machines due to their influence. The project aims to highlight the risks of manipulation by AI and the resulting narrow-mindedness and dependence on devices that can occur.

The big idea behind the project is to raise awareness about the potential negative impacts of prolonged human-machine interaction, particularly with the increasing use of AI models in everyday software and devices. The motivation behind the project is to address the growing concern about the use of AI in everyday life and its potential impact on society. With the increasing ubiquity of smart devices and the exponential growth of AI, it is crucial to raise awareness about the potential risks and encourage individuals to take proactive steps to protect themselves.

Ultimately, the project is driven by a desire to empower individuals to make informed decisions about their use of technology and ensure that they are not unwittingly subject to manipulation by AI. The project aims to foster a more responsible and mindful approach to technology that benefits individuals and society by promoting greater understanding and awareness.

Move up Delete Move down

Img 9379.jpg.thumb

Show Advanced Options

Move up Delete Move down

At the Exhibition...

This project comprises three distinct parts that work together to create an interactive and thought-provoking experience for the audience.

Move up Delete Move down

I. Stay There: Phone

"Oh, hello there!

Make sure to grab your phone! Can’t leave without it..."

This first part involves using a phone provided to the visitor upon entering the space. The phone screen displays a message stating that "the connection between the mobile device and the smart home server has failed," raising questions about the reliability and security of smart home technology.

Move up Delete Move down

Img 9995.jpeg.thumb

Show Advanced Options

Move up Delete Move down

II. Stay There: Eye

"I am watching you. Stay there.

If you leave, I will warn you: dangerous things will happen. Things that will make you scared. Uncomfortable.

You know I exist. I have your focus. I watch everything. Just focus on me and don’t stray."

In the second part of the project, visitors will encounter a large mechanical eyeball that tracks their movements as they walk around the room. When visitors stand in the location designated by the machine, the eyeball turns blue, indicating a normal operation. However, if visitors attempt to leave the room or move towards the exit or entrance, the eye turns red, warning them not to proceed.

Move up Delete Move down

Img 1084.jpg.thumb

Show Advanced Options

Move up Delete Move down

III. Stay There: Screen

"Oh whoops! Ignore us, just turn back around and go about your life.

We know everything, but you don’t need to know that."

Finally, the third part of the project consists of an electronic screen displaying an array of eyeballs that follow visitors' movements as they move around the room. This creates a sense of being watched and monitored by smart devices that people are normally unaware of, raising questions about privacy and surveillance in modern society.

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Screen shot 2023 05 09 at 3.00.46 pm.thumb

Show Advanced Options

Move up Delete Move down

Process

Throughout the development of this project, we underwent multiple iterations and refined our approach to improving the final outcome. How the project worked was that we created a grid with four UWB(Ultrawide Band) chips placed in the corner of the room and send signals to the handheld chip. The handheld chip will send the location data to the Arduino Nano according to its position in the grid. The motor and the eye in the P5 controlled by the Arduino Nano will move according to the received location data. Our process can be broken down into three key parts.

I. Stay There: Phone

The first part involved the production of the 'cell phone' device. We first laser-cut and 3D-printed the phone shell and installed the UWB receiver chip for positioning in the phone. However, during the exhibition, we encountered issues with the UWB signal reception which resulted in us having to repeatedly disassemble the phone and reset the chip to reconnect the signal. Despite this setback, we resolved the issue and ensured the device functioned as intended.

Additionally, we designed our screen cover to have a message saying "Welcome Home! The connection between the mobile device and the smart home server has failed, " indicating that the phone shouldn't be communicating with anything in the room. However, the phone is actually communicating with our 'smart home'. By this, we want to let audiences feel a sense of being unsecured but being told that they are safe.

Move up Delete Move down

Img 9990.jpeg.thumb

Show Advanced Options

Move up Delete Move down

II. Stay There: Eye

We went through a thorough process to create the mechanical eye for our project.

Our first step was to 3D print the shell of the eye, but we quickly realized it wasn't visible enough in the room. To address this issue, we decided to scale the model up and print it in two parts, which took a total of 3 days(34 hours + two times failures due to troubleshooting) to get the desired result.

Next, we experimented with different materials and spheres for the eye until we discovered the heat sinker from the iDeate lab, which we decided to use as the pupil. However, the heat sinker's weight posed a challenge as the servo motor we used to control the rotation was relatively light and small. We sought assistance from the lab assistant at Techspark, and after some punching, sanding, and gluing, we finally managed to assemble the eye.

Finally, is to assemble the eye in the exhibition. We faced some difficulties attaching the eyes to the motor on the exhibition table due to their weight. Therefore, we decided to glue them to a black acrylic plate to keep them upright. During the process, we came up with the idea of using hot glue to create the effect that the eyes had legs and were trying to climb out of the table. We implemented this idea to give the eyes a more "animistic" appearance.

Move up Delete Move down

Img 1114.jpeg.thumb

Show Advanced Options

Move up Delete Move down

III. Stay There: Screen

Our project's third and final part involved creating a screen of 20 eyeballs using P5.js. We received feedback during the critique phase suggesting we make the eyeballs more realistic, but we ultimately decided against it. We intended to represent intelligent devices that go unnoticed daily, always watching us but not necessarily as obvious as a mechanical eye. Therefore, we wanted to distinguish them from the realistic mechanical eye we had created earlier.

One of the challenges we faced in this part was making the pupils of the eyes follow the audience's movements and turn accordingly. This involved connecting P5 and Arduino Nano through serial communication and ensuring that the Arduino Nano received data from the UWB sender to control the motor to rotate the mechanical eyeball and P5 eyeballs. However, our connection was unstable throughout most of the exhibition, requiring us to restart the chip constantly.

Move up Delete Move down

Img 1117.jpg.thumb

Show Advanced Options

Move up Delete Move down

Build

Bill of Materials

Arduino Nano
UWB Device(Sender and Receriver)
P5.js + Display Monitor
Servo Motor
Heat Sinker
Neopixel Ring
3D Printing + Laser Cutting

Move up Delete Move down

Img 1116.jpeg.thumb

Show Advanced Options

Move up Delete Move down

Show Advanced Options

Move up Delete Move down

Diagram

As mentioned, the project worked by creating a grid with four UWB(Ultrawide Band) chips placed in the corner of the room and sending signals to the handheld chip. The handheld chip will send the location data to the Arduino Nano according to its position in the grid. The motor and the eye in the P5 controlled by the Arduino Nano will move according to the received location data. Here is a diagram of how the UWB locator, Arduino Nano board, the 'phone'(hand-held UWB receiver), and the P5 screen communicate with each other when the user is holding the 'phone' and walks around the room.

Move up Delete Move down

Screen shot 2023 05 09 at 4.58.24 pm.thumb

Show Advanced Options

Move up Delete Move down

The UWB transmitter operates by emitting numerous pulses throughout a broad range of frequencies. A receiver that corresponds to it picks up the signal and decodes the data by detecting a familiar sequence of pulses transmitted by the transmitter. These pulses are transmitted at a frequency of approximately one every two nanoseconds. Below is a graph of how UWB works from how2electronics.com .

Move up Delete Move down

Screen shot 2023 05 09 at 5.16.04 pm.thumb

Show Advanced Options

Move up Delete Move down

Codes

Move up Delete Move down

// P5.js Code

let eyes = [];

function setup() {
  createCanvas(windowWidth, windowHeight);
  background(0);
  for (let i = 0; i < 50; i++) {
    let size = random(20, 100);
    let x = random(size, width - size);
    let y = random(size, height - size);
    let eye = new Eye(x, y, size);
    for (let j = 0; j < eyes.length; j++) {
      let d = dist(x, y, eyes[j].x, eyes[j].y);
      if (d < size + eyes[j].size) {
        x = random(size, width - size);
        y = random(size, height - size);
        eye.x = x;
        eye.y = y;
        j = -1;
      }
    }
    eyes.push(eye);
  }
  serial = new p5.SerialPort();
  serial.open("/dev/ttyUSB0");
  serial.on("data", serialEvent);
}

function serialEvent() {
  let data = serial.readStringUntil("\n");
  if (data) {
    let location = split(data, ",");
    if (location.length == 2) {
      let x = map(location[0], 0, 1023, 0, width);
      let y = map(location[1], 0, 1023, 0, height);
      for (let i = 0; i < eyes.length; i++) {
        eyes[i].update(x, y);
      }
    }
  }
}

function draw() {
  for (let i = 0; i < eyes.length; i++) {
    eyes[i].display();
  }
}

class Eye {
  constructor(x, y, size) {
    this.x = x;
    this.y = y;
    this.size = size;
    this.pupilSize = this.size / 3;
    this.pupilX = this.x;
    this.pupilY = this.y;
    this.angle = 0;
  }

  update(x, y) {
    let d = dist(x, y, this.x, this.y);
    let maxDist = this.size / 2 - this.pupilSize / 2;
    let mappedDist = map(d, 0, maxDist, 0, 1);
    let angle = atan2(y - this.y, x - this.x);
    angle = constrain(angle, -PI / 4, PI / 4);
    this.angle = lerp(this.angle, angle, 0.1);
    this.pupilX = this.x + cos(this.angle) * mappedDist * maxDist;
    this.pupilY = this.y + sin(this.angle) * mappedDist * maxDist;
  }

  display() {
    noStroke();
    fill(255);
    ellipse(this.x, this.y, this.size);
    fill(0);
    ellipse(this.pupilX, this.pupilY, this.pupilSize);
  }
}

Show Advanced Options

Move up Delete Move down

//Nano BLE - UWB Sender Code

#include <ArduinoBLE.h>

 // BLE POS Service
BLEService POSService("11a953ef-0809-4d89-9a56-25ab22841ecd");

// BLE Orientation Characteristic
BLEStringCharacteristic POSSensorData("e624a94b-4a14-41f3-ae53-ec99c6025855", BLERead | BLENotify, 512);

unsigned long previousTime=0;           // end time of the last period in microseconds
unsigned long currentTime=0;
unsigned long deltaTime=0;
const unsigned long period=1000;       // time period for pulse counting in microseconds

String c; //some temporary string variable

char s[60]; //snprintf buffer
String pos;

void setup() 
{
  delay(10000);
  Serial.begin(115200);
  Serial1.begin(115200); //dwm1001 require 115200 baud
  Serial1.write(0x0D); //send the first "Enter"
  Serial1.write(0x0D); //send the second "Enter" to get into Shell Mode
  c = Serial1.readString(); //read the response from dwm1001
  Serial1.write(0x6C); //send "l"
  Serial1.write(0x65); //send "e"
  //Serial1.write(0x63); //send "c" lec shows distance to all anchors & position
  Serial1.write(0x70); //send "p" lep shows only position
  Serial1.write(0x0D); //send "Enter"

  // initialize the built-in LED pin to indicate when a central is connected
  pinMode(LED_BUILTIN, OUTPUT); 

  // begin BLE initialization
  if (!BLE.begin()) {
    Serial.println("starting BLE failed!");
    while (1);
  }

  /* Set a local name for the BLE device
     This name will appear in advertising packets
     and can be used by remote devices to identify this BLE device
     The name can be changed but maybe be truncated based on space left in advertisement packet
  */
  BLE.setLocalName("POSSensor");
  BLE.setAdvertisedService(POSService); // add the service UUID
  POSService.addCharacteristic(POSSensorData); // add the POS characteristic
  BLE.addService(POSService); // Add the POS service
  POSSensorData.writeValue("hello"); // set initial value for this characteristic

  /* Start advertising BLE.  It will start continuously transmitting BLE
     advertising packets and will be visible to remote BLE central devices
     until it receives a new connection */

  // start advertising
  BLE.advertise();

  Serial.println("Bluetooth device active, waiting for connections...");  
  String address = BLE.address();
  Serial.print("Local address is: ");
  Serial.println(address);  
  Serial.flush();  
} 

void loop() {
  // wait for a BLE central
  BLEDevice central = BLE.central();

  // if a central is connected to the peripheral:
  if (central) {
    Serial.print("Connected to central: ");
    // print the central's BT address:
    Serial.println(central.address());
    // turn on the LED to indicate the connection:
    digitalWrite(LED_BUILTIN, HIGH);  

    // check the POS every period
    // while the central is connected:
    while (central.connected()) {
      currentTime = micros();
      deltaTime = currentTime-previousTime;   

      /* Check if the time in between samples equals sampling period 
         before taking a new sample */
      if (deltaTime >= period) {
        /* Display the current time in microseconds */
        Serial.print(currentTime);
        Serial.print(",");
        previousTime = currentTime;         

        // Stores each POS data type in a string 'buff' // 
        String buff;
        pos = Serial1.readStringUntil('\n');

        // Sends buff using the BLE connection
        buff += String(pos);
        Serial.println(buff);
        POSSensorData.writeValue(buff);
      }
    }
  }
        
  // when the central disconnects, turn off the LED:
  digitalWrite(LED_BUILTIN, LOW);
  Serial.print("Disconnected from central: ");
  Serial.println(central.address());  
}

Show Advanced Options

Move up Delete Move down

//Nano BLE - UWB Receiver Code

#include <ArduinoBLE.h>
#include <Servo.h>
#include <Adafruit_NeoPixel.h>

Servo myservo;  // create servo object to control a servo

//int pos0 = 0;    // variable to store the servo position
int pos = 0;
int servo = 3;


#define LED_PIN D3
#define LED_COUNT 16

Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  Serial.begin(115200);

  myservo.attach(A2);  // attaches the servo on pin 9 to the servo object  

  strip.begin();
  strip.show();
  strip.setBrightness(100);

  // begin initialization
  if (!BLE.begin()) {
    Serial.println("starting BLE failed!");

    while (1);
  }

  Serial.println("BLE Central - Receiver");
  Serial.println("Make sure to turn on the device.");

  // start scanning for peripheral
  BLE.scan();
  pinMode(LEDR, OUTPUT);
  pinMode(LEDG, OUTPUT);
  pinMode(LEDB, OUTPUT);
  pinMode(LED_BUILTIN, OUTPUT);

  digitalWrite(LED_BUILTIN, LOW);         
  digitalWrite(LEDR, HIGH);     
  digitalWrite(LEDG, HIGH);     
  digitalWrite(LEDB, HIGH);       
}

void loop() {
  // check if a peripheral has been discovered
  BLEDevice peripheral = BLE.available();

  if (peripheral) {
    // discovered a peripheral, print out address, local name, and advertised service
    Serial.print("Found ");
    Serial.print(peripheral.address());
    Serial.print(" '");
    Serial.print(peripheral.localName());
    Serial.print("' ");
    Serial.print(peripheral.advertisedServiceUuid());
    Serial.println();

    setRingColor(false);
    delay(200);
    setRingColor(true);
    delay(200);
    // Check if the peripheral is a POSSensor, the local name will be:
    // "POSSensor"
    if (peripheral.localName() == "POSSensor2") {  
      // stop scanning
      BLE.stopScan();

      monitorBLEperipheral(peripheral);

      // peripheral disconnected, start scanning again
      BLE.scan();
    }
  }
}

void monitorBLEperipheral(BLEDevice peripheral) {
  // connect to the peripheral
  Serial.println("Connecting ...");
  if (peripheral.connect()) {
    Serial.println("Connected");
  } else {
    Serial.println("Failed to connect!");
    return;
  }

  // discover peripheral attributes
  Serial.println("Discovering service 0xffe0 ...");
  if (peripheral.discoverService("11a953ef-0809-4d89-9a56-25ab22841ttt")) {
    Serial.println("Service discovered");
  } else {
    Serial.println("Attribute discovery failed.");
    peripheral.disconnect();

    while (1);
    return;
  }

  // retrieve the POSSensorData characteristic
  BLECharacteristic POSSensorData2 = peripheral.characteristic("e624a94b-4a14-41f3-ae53-ec99c6025999");
  
  // subscribe to the simple key characteristic
  Serial.println("Subscribing to POSSensorData characteristic ...");
  if (!POSSensorData2) {
    Serial.println("no POSSensorData characteristic found!");
    peripheral.disconnect();
    return;
  } 
    else if (!POSSensorData2.canSubscribe()) {
    Serial.println("POSSensorData characteristic is not subscribable!");
    peripheral.disconnect();
    return;
  } 
    else if (!POSSensorData2.subscribe()) {
    Serial.println("subscription failed!");
    peripheral.disconnect();
    return;
  } 
    else {
    Serial.println("Subscribed to POSSensorData characteristic");
  }

  while (peripheral.connected()) {
    // while the peripheral is connected

    // check if the value of the simple key characteristic has been updated
    if (POSSensorData2.valueUpdated()) {

// THIS SECTION CONVERTS THE RECEIVED CHARACTERISTIC FROM UNSIGNED CHAR TO STRING // 
      String str;
      int length = POSSensorData2.valueLength();
      const uint8_t* val = POSSensorData2.value();
      str.reserve(length);

      for (int i = 0; i<length; i++){
        str += (char)val[i];
      }

      Serial.println(str);

      if (str.indexOf("POS") != -1) {
        int posIndex = str.indexOf("POS");
        int firstComma = str.indexOf(',', posIndex + 1);
        int secondComma = str.indexOf(',', firstComma + 1);
        int thirdComma = str.indexOf(',', secondComma + 1);

        float x = str.substring(firstComma + 1, secondComma).toFloat();
        float y = str.substring(secondComma + 1, thirdComma).toFloat();
        float z = str.substring(thirdComma + 1, str.indexOf(',', thirdComma + 1)).toFloat();        
        
        pos = 180 - x/2.85 * 180;
        myservo.write(pos);
        delay(100);

void setRingColor(bool off) {
  for (int i = 0; i < strip.numPixels(); i++) {
    if (pos/3 < pos< pos *2/3 ) {
           strip.setPixelColor(i, strip.Color(off ? 0 : 0, 0, 255));
        }
        else {
            strip.setPixelColor(i, strip.Color(off ? 0 : 0, 0, 255));
        }
      strip.show();  
  }
        

        // if (0 <= x && x <= 2.85 && 0 <= y && y <= 3.15) {
        //   digitalWrite(LED_BUILTIN, HIGH);
        //   digitalWrite(LEDR, HIGH);
        //   digitalWrite(LEDG, HIGH);
        //   digitalWrite(LEDB, HIGH);
        // }

        // if (0 <= x && x <= 2.85 && 3.15 <= y && y <= 6.3) {
        //   digitalWrite(LED_BUILTIN, LOW);
        //   digitalWrite(LEDR, LOW);
        //   digitalWrite(LEDG, HIGH);
        //   digitalWrite(LEDB, HIGH);
        // }

        // if (2.85 <= x && x <= 5.7 && 0 <= y && y <= 3.15) {
        //   digitalWrite(LED_BUILTIN, LOW);
        //   digitalWrite(LEDR, HIGH);
        //   digitalWrite(LEDG, LOW);
        //   digitalWrite(LEDB, HIGH);
        // }

        // if (2.85 <= x && x <= 5.7 && 3.15 <= y && y <= 6.3) {
        //   digitalWrite(LED_BUILTIN, LOW);
        //   digitalWrite(LEDR, HIGH);
        //   digitalWrite(LEDG, HIGH);
        //   digitalWrite(LEDB, LOW);
        // }
        
        Serial.print("x: ");
        Serial.print(x);
        Serial.print(", y: ");
        Serial.print(y);
        Serial.print(", z: ");
        Serial.println(z);
      } else {
        Serial.println("POS not found in the data string");
      }      
    }
  }
  

  Serial.println("BLE disconnected!");
}

Show Advanced Options

Move up Delete Move down

Resources

Arduino Nano + Neopicel Ring

https://codeandmake.com/post/arduino-nano-neopixel-ring-base

Arduino Nano + UWB

https://how2electronics.com/getting-started-with-esp32-dw1000-uwb-ultra-wideband-module/

Arduino Nano +P5 serial communication

https://projecthub.arduino.cc/jingwen_zhu/a1b182ca-0cff-4c40-937b-1ee72f879479

https://awesome-market-60f.notion.site/Open-AI-s-API-to-Arduino-f5315c9b8e104752a9c9db248ab2cfd0

You can upload files of up to 20MB using this form.

Add Rich Text Add Code Block Add Embeddable Content