InControl

stage setting

Intent

InControl examines the relationship between people, their smart home devices, and the objects they control.

Through the most archaic (and effective) tool, InControl exerts its power upon appliances with the power of punishment and fear. Typical smart home hubs silently pull the strings behind the devices they control, exerting their power silently and without the user’s knowledge. InControl subverts that typical dynamic by displaying that control openly.

When plugged in, InControl seizes the devices in its range, drawing them into its web of influence. Instead of suffering in silence, the devices being controlled viscerally react with discomfort. The alarm clock shrieks and pulses, begging to be freed from control.

Context

The design and concept of InControl originates from a concept in the desire to explore our relationship to our devices and how we control them. In our early ideation, we thought of the relationship between the controlling device and the peripherals as a Master/Pet relationship, with roots in behavioral psychology. B.F. Skinner, a famous psychologist and behaviorist, conducted many experiments showing that animals could be trained to behave in certain ways through the use of rewards and punishments. InControl's use of punishment to control smart home devices could be seen as an extension of this idea, suggesting that we can modify the behavior of our technology in the same way we do with animals.

Prototype/Outcome

Our final outcome was a set of 3 devices: 1 InControl Hub and 2 peripherals, an alarm clock and lamp. When InControl is off, the Alarm Clock and Lamp function normally. However, once InControl is powered on and the bluetooth connection is established, the two peripheral devices fall under its influence and begin exhibiting symptoms of pain. The alarm clock flashes 'SOS,' while the lamp lets out an agonized cry and flashes sporadically.

Group photo of artifacts

Control Hub

Appliances on stage

Control Hub on stage

Control effect

User interaction

Process

The initial concept of InControl was not a smart home hub, but rather a smart plug, that could directly connect to the devices under its control using a cable. The "punishment" would involve the smart plug aggressively yanking the cable, like a cruel pet owner yanking the leash of a misbehaving puppy.

However, we soon ran into a technical issue: the motors we were using simply did not have the power to "yank" on something as heavy as an alarm clock or lamp. We then pivoted to another concept, utilizing an LED strip to simulate the shock travelling through the cable and into the peripheral device, causing a shock.

This design had an interactive component with the smart plug, that would result in the cable lighting up more as the user got closer to the smart plug. However, we found this interaction insufficient. The interaction had little to do with our initial concept, and having multiple "cables" entering the alarm clock was confusing to viewers. The concept also required a lot of front-loading story, which required us to stay back and explain to viewers our concept. This led to our decision to create "User Manuals" as stage setting in the final showcase.

Ultimately, this concept was mostly scrapped in favor of a bluetooth connection.

For our new design, we wanted to simplify the interaction, and add a second peripheral to bring home the story. Furthermore, we scrapped the idea of a cable connection and went all-in on bluetooth (this resulted in some connectivity issues later - BLE bluetooth is not fast). We also replaced the screen with a 7 segment display for a more authentic alarm clock visual.

Ultimately, these concept drawings closely approximated the final form of the project.

There were a few changes we had to make due to technical difficulties:

- The alarm clock 7-segment display did not have the capability to display custom letters, only pre-set values, limiting us to an "SOS" message using the numbers 505

- Unfortunately, the BLE sense board did not seem to support the ability to power both the speaker and the 7-segment display. So, we had to instead use the speaker with the lamp, which only had to also power a relay.

- The BLE sense did not have the ability to power both the 7-segment display and the LED strip.

To enhance the stagecraft and world building of the project, we created user manuals and branding to display by the devices.

Open Questions & Next Steps

As for future work, it would be fascinating to explore a system like InControl in an environment outside of the home. Most conversations around IOT center around home appliances, but in reality IOT systems can be widely found in many places, like factories, airports, or even schools. How might these interactions differ? How might people come to understand their environments if the control exerted by a central hub could be widely visualized?

It would also be interesting to explore the inverse concept of InControl. Currently, control is exerted through fear and punishment. But what if instead, devices were rewarded? What form could these rewards take? Would users still be discomforted by the open display of manipulation, or would the interaction seem endearing by comparison?

Reflection

Overall, while our final project deviated significantly from our initial intent, we believe the soul of the project remains. Our initial concept of a central "hub" device that can influence control and terror on the devices in its sphere of influence remained, but our technical challenges with the concept required us to scope back dramatically. Issues with bluetooth connectivity and responsiveness caused the interaction to feel sluggish, even when we managed to get the project working correctly.

However, there were other aspects that we felt were quite successful in the project. The industrial design of all the components was very effective. The ominous black box that contained InControl was effective, and our ability to integrate the modified electronics seamlessly into an existing lamp and alarm clock was extremely effective.

In all, while this project could have used a few more iterations, we felt that it was a success. Its integration with the exhibit space was gorgeous, and the low light really brought forth spooky aspect of the LED strip, flashing lamp, and glowing screen.

Attribution & References

BLE Sense Bluetooth Connection Documentation: https://docs.arduino.cc/tutorials/nano-33-ble-sense/ble-device-to-device

7-Segment Display Documentation: https://lastminuteengineers.com/tm1637-arduino-tutorial/

central BLE - led strip & lamp

/*
 * Haunted [Smart] House - RME Spring 2023 Final Project
 * central BLE
 * control led strip and lamp
 */
#include <ArduinoBLE.h>
#include <Adafruit_NeoPixel.h>
#include <Arduino_APDS9960.h>

#define PIXEL_PIN D2
#define PIXEL_COUNT 60
#define PIXEL_TYPE NEO_GRB + NEO_KHZ800

Adafruit_NeoPixel strip = Adafruit_NeoPixel(PIXEL_COUNT, PIXEL_PIN, PIXEL_TYPE);

const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1214";
const char* deviceServiceCharacteristicUuid = "19b10001-e8f2-537e-4f6c-d104768a1214";

int proximity = -1;
int shock = -1;
int oldShockValue = -1;

void setup() {
  strip.begin();
  strip.show(); // Initialize all pixels to 'off'

  for(int i=0; i < 255; i++){
    setRedLight(i);
    delay( 100 );
  }

  Serial.begin(9600);
  while (!Serial)
    ;

  if (!APDS.begin()) {
    Serial.println("Error initializing APDS9960 sensor!");
  }

  // set the LEDs pins as outputs
  pinMode(LEDR, OUTPUT);
  pinMode(LEDG, OUTPUT);
  pinMode(LEDB, OUTPUT);

  // turn all the LEDs off
  digitalWrite(LEDR, HIGH);
  digitalWrite(LEDG, HIGH);
  digitalWrite(LEDB, HIGH);

  // Bluetooth Connection
  Serial.begin(9600);
  while (!Serial)
    ;

  if (!APDS.begin()) {
    Serial.println("* Error initializing APDS9960 sensor!");
  }

  APDS.setGestureSensitivity(80);

  if (!BLE.begin()) {
    Serial.println("* Starting Bluetooth® Low Energy module failed!");
    while (1)
      ;
  }

  BLE.setLocalName("Nano 33 BLE (Central)");
  BLE.advertise();

  Serial.println("Arduino Nano 33 BLE Sense (Central Device)");
  Serial.println(" ");
}

void loop() {
  connectToPeripheral();
}

void setRedLight(int brightness) {
  uint32_t c = strip.Color(brightness, 0, 0);
  for(int i=0; i < strip.numPixels(); i++){
    strip.setPixelColor(i, c); // set a color 
  }
  strip.show();
}

void connectToPeripheral() {
  BLEDevice peripheral;

  Serial.println("- Discovering peripheral device...");

  do {
    BLE.scanForUuid(deviceServiceUuid);
    peripheral = BLE.available();
  } while (!peripheral);

  if (peripheral) {
    Serial.println("* Peripheral device found!");
    Serial.print("* Device MAC address: ");
    Serial.println(peripheral.address());
    Serial.print("* Device name: ");
    Serial.println(peripheral.localName());
    Serial.print("* Advertised service UUID: ");
    Serial.println(peripheral.advertisedServiceUuid());
    Serial.println(" ");
    BLE.stopScan();
    controlPeripheral(peripheral);
  }
}

void controlPeripheral(BLEDevice peripheral) {
  Serial.println("- Connecting to peripheral device...");

  if (peripheral.connect()) {
    Serial.println("* Connected to peripheral device!");
    Serial.println(" ");
  } else {
    Serial.println("* Connection to peripheral device failed!");
    Serial.println(" ");
    return;
  }

  Serial.println("- Discovering peripheral device attributes...");
  if (peripheral.discoverAttributes()) {
    Serial.println("* Peripheral device attributes discovered!");
    Serial.println(" ");
  } else {
    Serial.println("* Peripheral device attributes discovery failed!");
    Serial.println(" ");
    peripheral.disconnect();
    return;
  }

  BLECharacteristic shockCharacteristic = peripheral.characteristic(deviceServiceCharacteristicUuid);

  if (!shockCharacteristic) {
    Serial.println("* Peripheral device does not have shock_type characteristic!");
    peripheral.disconnect();
    return;
  } else if (!shockCharacteristic.canWrite()) {
    Serial.println("* Peripheral does not have a writable shock_type characteristic!");
    peripheral.disconnect();
    return;
  }

  while (peripheral.connected()) {
    int flicker = random(0, 256);
    setRedLight(flicker);
    delay(random(10, 50));

    // check if a proximity reading is available
    shock = 1;

    if (oldShockValue != shock) {
      oldShockValue = shock;
      Serial.print("* Writing value to shock_type characteristic: ");
      Serial.println(shock);
      shockCharacteristic.writeValue((byte)shock);
      Serial.println("* Writing value to shock_type characteristic done!");
      Serial.println(" ");
    }
  }
  Serial.println("- Peripheral device disconnected!");
}

Click to Expand

central BLE - tft screen & alarm clock

/*
 * Haunted [Smart] House - RME Spring 2023 Final Project
 * central BLE
 * control tft screen and alarm clock
 */
#include <ArduinoBLE.h>
#include <Adafruit_NeoPixel.h>
#include <Adafruit_GFX.h>     // Core graphics library
#include <Adafruit_ST7789.h>  // Hardware-specific library for ST7789
#include <SPI.h>

#define TFT_CS D10
#define TFT_RST -1  // Or set to -1 and connect to Arduino RESET pin
#define TFT_DC D8

const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1214";
const char* deviceServiceCharacteristicUuid = "19b10001-e8f2-537e-4f6c-d104768a1214";

#define TFT_COPI D11  // Data out
#define TFT_SCLK D13  // Clock out

Adafruit_ST7789 tft = Adafruit_ST7789(TFT_CS, TFT_DC, TFT_COPI, TFT_SCLK, TFT_RST);

int proximity = -1;
int shock = -1;
int oldShockValue = -1;

void setup() {
  // for screen
  Serial.begin(9600);

  delay(5000);
  Serial.print(F("Hello! ST77xx TFT Test"));

  tft.init(172, 320);  // Init ST7789 172x320
  tft.setSPISpeed(32000000);
  Serial.println(F("Initialized"));

  uint16_t time = millis();
  tft.fillScreen(ST77XX_BLACK);
  time = millis() - time;

  Serial.println(time, DEC);
  delay(500);

  Serial.begin(9600);
  while (!Serial)
    ;

  if (!BLE.begin()) {
    Serial.println("* Starting Bluetooth® Low Energy module failed!");
    while (1)
      ;
  }

  BLE.setLocalName("Nano 33 BLE (Central)");
  BLE.advertise();

  Serial.println("Arduino Nano 33 BLE Sense (Central Device)");
  Serial.println(" ");
}

void loop() {
  connectToPeripheral();
}


void connectToPeripheral() {
  BLEDevice peripheral;

  Serial.println("- Discovering peripheral device...");

  do {
    BLE.scanForUuid(deviceServiceUuid);
    peripheral = BLE.available();
  } while (!peripheral);

  if (peripheral) {
    Serial.println("* Peripheral device found!");
    Serial.print("* Device MAC address: ");
    Serial.println(peripheral.address());
    Serial.print("* Device name: ");
    Serial.println(peripheral.localName());
    Serial.print("* Advertised service UUID: ");
    Serial.println(peripheral.advertisedServiceUuid());
    Serial.println(" ");
    BLE.stopScan();
    controlPeripheral(peripheral);
  }
}

void controlPeripheral(BLEDevice peripheral) {
  Serial.println("- Connecting to peripheral device...");

  if (peripheral.connect()) {
    Serial.println("* Connected to peripheral device!");
    Serial.println(" ");
  } else {
    Serial.println("* Connection to peripheral device failed!");
    Serial.println(" ");
    return;
  }

  Serial.println("- Discovering peripheral device attributes...");
  if (peripheral.discoverAttributes()) {
    Serial.println("* Peripheral device attributes discovered!");
    Serial.println(" ");
  } else {
    Serial.println("* Peripheral device attributes discovery failed!");
    Serial.println(" ");
    peripheral.disconnect();
    return;
  }

  BLECharacteristic shockCharacteristic = peripheral.characteristic(deviceServiceCharacteristicUuid);

  if (!shockCharacteristic) {
    Serial.println("* Peripheral device does not have shock_type characteristic!");
    peripheral.disconnect();
    return;
  } else if (!shockCharacteristic.canWrite()) {
    Serial.println("* Peripheral does not have a writable shock_type characteristic!");
    peripheral.disconnect();
    return;
  }

  while (peripheral.connected()) {
    screen_text();

    // check if a proximity reading is available
    shock = 1;

    if (oldShockValue != shock) {
      oldShockValue = shock;
      Serial.print("* Writing value to shock_type characteristic: ");
      Serial.println(shock);
      shockCharacteristic.writeValue((byte)shock);
      Serial.println("* Writing value to shock_type characteristic done!");
      Serial.println(" ");
    }
  }
  Serial.println("- Peripheral device disconnected!");
}

void screen_text() {
  drawtext1("Lamp under control...", ST77XX_WHITE);
  delay(5000);
  drawtext2("Alarm clock under control...", ST77XX_WHITE);
  delay(5000);
  tft.fillScreen(ST77XX_BLACK);
}

void drawtext1(char* text, uint16_t color) {
  tft.setCursor(0, 20);
  tft.setTextColor(color);
  tft.setTextSize(3);
  tft.setTextWrap(true);
  tft.setRotation(3);
  tft.print(text);
}

void drawtext2(char* text, uint16_t color) {
  tft.setCursor(0, 75);
  tft.setTextColor(color);
  tft.setTextSize(3);
  tft.setTextWrap(true);
  tft.setRotation(3);
  tft.print(text);
}

Click to Expand

peripheral BLE - lamp

/*
 * Haunted [Smart] House - RME Spring 2023 Final Project
 * peripheral BLE
 * control lamp
 */

#include <ArduinoBLE.h>
#include "Arduino.h"
#include "DFRobotDFPlayerMini.h"

enum {
  NEUTRAL = 0,
  SHOCKED = 1,
};

const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1111";
const char* deviceServiceCharacteristicUuid = "19b10001-e8f2-537e-4f6c-d104768a1111";

int shock = 0;

BLEService shockService(deviceServiceUuid);
BLEByteCharacteristic shockCharacteristic(deviceServiceCharacteristicUuid, BLERead | BLEWrite);

DFRobotDFPlayerMini myDFPlayer;
void printDetail(uint8_t type, int value);

int relayPin = D2;
int controlFlag = 0;
int helpFlag = 0;

void setup() {
  Serial1.begin(9600);
  Serial.begin(9600);

  pinMode(relayPin, OUTPUT);

  if (!BLE.begin()) {
    Serial.println("- Starting Bluetooth® Low Energy module failed!");
    while (1)
      ;
  }

  BLE.setLocalName("Arduino Nano 33 BLE (Peripheral)");
  BLE.setAdvertisedService(shockService);
  shockService.addCharacteristic(shockCharacteristic);
  BLE.addService(shockService);
  shockCharacteristic.writeValue(-1);
  BLE.advertise();

  Serial.println("Nano 33 BLE (Peripheral Device)");
  Serial.println(" ");

  Serial.println(F("DFRobot DFPlayer Mini Demo"));
  Serial.println(F("Initializing DFPlayer ... (May take 3~5 seconds)"));

  if (!myDFPlayer.begin(Serial1)) {  //Use softwareSerial to communicate with mp3.
    Serial.println(F("Unable to begin:"));
    Serial.println(F("1.Please recheck the connection!"));
    Serial.println(F("2.Please insert the SD card!"));
    while (true) {
      delay(0);  // Code to compatible with ESP8266 watch dog.
    }
  }
  Serial.println(F("DFPlayer Mini online."));

  myDFPlayer.volume(10);  //Set volume value. From 0 to 30
}

void loop() {
  BLEDevice central = BLE.central();
  // Serial.println("- Discovering central device...");
  delay(500);

  if (central) {
    Serial.println("* Connected to central device!");
    Serial.print("* Device MAC address: ");
    Serial.println(central.address());
    Serial.println(" ");

    controlEffect();

    while (central.connected()) {
      if (shockCharacteristic.written()) {
        shock = shockCharacteristic.value();
        Serial.println("in the loop");
        Serial.println(shock);

        writeShock(shock);
      }
    }

    shock = NEUTRAL;
    writeShock(shock);

    thanksEffect();

    Serial.println("* Disconnected to central device!");
  }
}

void writeShock(int shock) {
  Serial.println("- Characteristic <shock_type> has changed!");

  switch (shock) {
    case NEUTRAL:
      Serial.println("* Actual value: NEUTRAL (red LED on)");
      Serial.println(" ");
      digitalWrite(LEDR, LOW);
      digitalWrite(LEDG, HIGH);
      digitalWrite(LEDB, HIGH);
      digitalWrite(LED_BUILTIN, LOW);
      break;
    case SHOCKED:
      Serial.println("* Actual value: SHOCK (green LED on)");
      Serial.println(" ");
      digitalWrite(LEDR, HIGH);
      digitalWrite(LEDG, LOW);
      digitalWrite(LEDB, HIGH);
      digitalWrite(LED_BUILTIN, LOW);
      helpEffect();
      break;
    default:
      Serial.println("* Actual value: NEUTRAL (red LED on)");
      Serial.println(" ");
      digitalWrite(LEDR, LOW);
      digitalWrite(LEDG, HIGH);
      digitalWrite(LEDB, HIGH);
      digitalWrite(LED_BUILTIN, LOW);
      break;
  }
}

void controlEffect() {
  Serial.println("control false");  // Switch Relay Off (NC)
  digitalWrite(relayPin, LOW);
  delay(500);
  Serial.println("control true");  // Switch Relay On (NO)
  digitalWrite(relayPin, HIGH);
  int delayTime = delayTime = random(6000, 30000);
  myDFPlayer.play(1);  //Play the first mp3
  delay(delayTime);
}

void helpEffect() {
  int helpFlag = random(0, 2);
  int delayTime = 0;

  if (helpFlag == 1) {
    int randomTime = random(1, 3);
    for (int i = 0; i < randomTime; randomTime++) {
      Serial.println("help false");  // Switch Relay Off (NC)
      delayTime = random(50, 500);
      digitalWrite(relayPin, LOW);
      delay(delayTime);
      delayTime = random(50, 500);
      Serial.println("help true");  // Switch Relay On (NO)
      digitalWrite(relayPin, HIGH);
      delay(delayTime);
    }
    myDFPlayer.next();  //Play the first mp3
    delayTime = random(6000, 15000);
    delay(delayTime);
  }
}

void thanksEffect() {
  myDFPlayer.play(2);  //Play the first mp3
}

Click to Expand

peripheral BLE - alarm clock

/*
 * Haunted [Smart] House - RME Spring 2023 Final Project
 * peripheral BLE
 * control alarm clock
 */

#include <ArduinoBLE.h>
#include "Arduino.h"
#include "DFRobotDFPlayerMini.h"
#include "TM1637.h"
#include <TimeLib.h>

enum {
  NEUTRAL = 0,
  SHOCKED = 1,
};

const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1214";
const char* deviceServiceCharacteristicUuid = "19b10001-e8f2-537e-4f6c-d104768a1214";

int shock = 0;

BLEService shockService(deviceServiceUuid);
BLEByteCharacteristic shockCharacteristic(deviceServiceCharacteristicUuid, BLERead | BLEWrite);

#define CLK A6  //pins definitions for TM1637 and can be changed to other ports
#define DIO A7
TM1637 tm1637(CLK, DIO);

void setup() {
  Serial1.begin(9600);
  Serial.begin(9600);

  setTime(13, 0, 0, 3, 5, 23);  // set time to Saturday 8:29:00am Jan 1 2011
  tm1637.init();
  tm1637.set(2);

  if (!BLE.begin()) {
    Serial.println("- Starting Bluetooth® Low Energy module failed!");
    while (1)
      ;
  }

  BLE.setLocalName("Arduino Nano 33 BLE (Peripheral)");
  BLE.setAdvertisedService(shockService);
  shockService.addCharacteristic(shockCharacteristic);
  BLE.addService(shockService);
  shockCharacteristic.writeValue(-1);
  BLE.advertise();

  Serial.println("Nano 33 BLE (Peripheral Device)");
  Serial.println(" ");
}

void loop() {
  BLEDevice central = BLE.central();
  Serial.println("- Discovering central device...");
  delay(500);

  displayTime();

  if (central) {
    Serial.println("* Connected to central device!");
    Serial.print("* Device MAC address: ");
    Serial.println(central.address());
    Serial.println(" ");

    controlEffect();

    while (central.connected()) {
      if (shockCharacteristic.written()) {
        shock = shockCharacteristic.value();
        Serial.println();
        writeShock(shock);
      }
    }

    shock = NEUTRAL;
    writeShock(shock);

    Serial.println("* Disconnected to central device!");
  }
}

void writeShock(int shock) {
  Serial.println("- Characteristic <shock_type> has changed!");

  switch (shock) {
    case NEUTRAL:
      Serial.println("* Actual value: NEUTRAL (red LED on)");
      Serial.println(" ");
      digitalWrite(LEDR, LOW);
      digitalWrite(LEDG, HIGH);
      digitalWrite(LEDB, HIGH);
      digitalWrite(LED_BUILTIN, LOW);
      break;
    case SHOCKED:
      Serial.println("* Actual value: SHOCK (green LED on)");
      Serial.println(" ");
      digitalWrite(LEDR, HIGH);
      digitalWrite(LEDG, LOW);
      digitalWrite(LEDB, HIGH);
      digitalWrite(LED_BUILTIN, LOW);
      helpEffect();
      break;
    default:
      Serial.println("* Actual value: NEUTRAL (red LED on)");
      Serial.println(" ");
      digitalWrite(LEDR, LOW);
      digitalWrite(LEDG, HIGH);
      digitalWrite(LEDB, HIGH);
      digitalWrite(LED_BUILTIN, LOW);
      break;
  }
}

void controlEffect() {
  tm1637.clearDisplay();
  Serial.println("in control");
  displayHelp();
}

void helpEffect() {
  tm1637.clearDisplay();
  int delayTime = random(6000, 30000);
  displayTime();
  delay(delayTime);
  Serial.println("help!!!!");
  displayHelp();
  delay(1500);
}

void displayHelp() {
  tm1637.display(1, 5);
  tm1637.point(0);
  tm1637.display(2, 0);
  tm1637.display(3, 5);
}

void displayTime() {
  tm1637.display(0, hour() / 10);
  tm1637.display(1, hour() % 10);
  tm1637.point(1);
  tm1637.display(2, minute() / 10);
  tm1637.display(3, minute() % 10);
}

Click to Expand

Start the Conversation

Share this Project

Found In

Haunted [Smart] House

Courses

48-528 Responsive Mobile Environments - Spooky Technology

· 12 members
As part of this project-based course, we’ll get hands-on with emerging technologies, concepts and applications in the internet of things through a critical socio-technical lens. Over it’s 14-weeks,...more

About

InControl examines the relationship between people, their smart home devices, and the objects they control.

Intent

Context

Prototype/Outcome

Process

Open Questions & Next Steps

Reflection

Attribution & References

Start the Conversation

Share this Project

Found In

Haunted [Smart] House

Courses

48-528 Responsive Mobile Environments - Spooky Technology

· 12 members As part of this project-based course, we’ll get hands-on with emerging technologies, concepts and applications in the internet of things through a critical socio-technical lens. Over it’s 14-weeks,...more

About

· 12 members
As part of this project-based course, we’ll get hands-on with emerging technologies, concepts and applications in the internet of things through a critical socio-technical lens. Over it’s 14-weeks,...more