Incontrol

Intention

InControl intends to examine the relationship between smart home devices, the objects they control and how people situate and intervene in the relationship. What will happen in this unequal relationship when the central device requests from all the peripheral devices? How will people interact with them to intervene in this relationship? And how could we express this narrative with speculative design approaches and tangible interaction?

Context

The design and concept of InControl originates from a concept in the desire to explore our relationship to our devices and how we control them. In our early ideation, we thought of the relationship between the controlling device and the peripherals as a Master/Pet relationship, with roots in behavioral psychology. B.F. Skinner, a famous psychologist and behaviorist, conducted many experiments showing that animals could be trained to behave in certain ways through the use of rewards and punishments. InControl's use of punishment to control smart home devices could be seen as an extension of this idea, suggesting that we can modify the behavior of our technology in the same way we do with animals. Besides, the project also attempts to unveil the potential issues behind the omnipotent control devices:

Prototype/Outcome

We have two artifacts to express our intention in the prototype: a master socket and a pet alarm clock.

The master controls its pet with Bluetooth communication. Once building the connection, the enslaved alarm will switch to control mode and try to send help messages through the TFT screen from time to time. When people notice the message and try to free the alarm clock by unplugging the master socket, the proximity sensor will detect the human approach. The cable will shock and burn the alarm (expressed by the led strip attached to the cable) and make it glitch (random text and flicker on the TFT screen).

Process

The initial concept of the "punishment" would involve the smart plug aggressively yanking the cable, like a cruel pet owner yanking the leash of a misbehaving puppy.

However, we soon ran into a technical issue: the motors we were using did not have the power to "yank" on something as heavy as an alarm clock or lamp. We then pivoted to another concept, utilizing an LED strip to simulate the shock travelling through the cable and into the peripheral device, causing a shock. The shock effect will reflect on the TFT screen with random text and flickers.

Reflection

Due to the time limit, this is just a lo-fi prototype, the socket hasn't been encapsulated, and the alarm is just a simple box with a screen and a button. During the desk crit, we found the following issues:

People need help understanding the narrative we were trying to convey by only interacting with the artefact because of its complex storylines.
The lo-fi quality confused people by mismatching the leader's role among the artifacts, and they often ignored the socket.
People can't perceive the feedback from the alarm clock due to the small screen. And the LED strip's effect made no sense to them: they couldn't associate the electricity effect with it.
The BLE code failed, which made the button not react to the press action. Thus, the glitch on the screen won't show up sometimes.
The alarm clock is unrealistic; people only regard it as a strange box and think of how to use it instead of treating it as a regular clock in their bedroom, severely harming our context-making.

Open Questions and Next Steps

We were trying to build a rich world to investigate the human and smart home device relationship. But summarizing all the issues we found in the desk crit, we failed to convey it to our guests. Therefore, we must improve our narrative by letting the artifacts tell the story. We decided to enhance our project from the following perspective:

Make a user manual for storytelling - situate people in the context and function as an instruction about interacting with the artifacts.
Improve the control effect - use sound/haptic etc., to make the effect more intuitive and perceptive to users.
Hack the device to make it more realistic and immerse users into our context and staging setting in the final exhibition.
Do we need a socket? Bluetooth technology creates a more spook control method than explicit connection. What's the reasonable product to integrate invisible control and technology? We need a substitute.

Attribution and References

[1] Working with a TFT Screen (notion.site)

[2] BLE Sense Bluetooth Connection Documentation: https://docs.arduino.cc/tutorials/nano-33-ble-sense/ble-device-to-device

#include <Adafruit_GFX.h>     // Core graphics library
#include <Adafruit_ST7789.h>  // Hardware-specific library for ST7789
#include <SPI.h>
#include <TimeLib.h>
#include <TimeAlarms.h>
#include <ArduinoBLE.h>

#define TFT_CS D10
#define TFT_RST -1  // Or set to -1 and connect to Arduino RESET pin
#define TFT_DC D8
#define TFT_COPI D11  // Data out
#define TFT_SCLK D13  // Clock out

enum {
  NEUTRAL = 0,
  SHOCKED = 1,
};

const char *deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1214";
const char *deviceServiceCharacteristicUuid = "19b10001-e8f2-537e-4f6c-d104768a1214";

int shock = 0;
BLEService shockService(deviceServiceUuid);
BLEByteCharacteristic shockCharacteristic(deviceServiceCharacteristicUuid, BLERead | BLEWrite);

Adafruit_ST7789 tft = Adafruit_ST7789(TFT_CS, TFT_DC, TFT_COPI, TFT_SCLK, TFT_RST);

const int BUTTON_PIN = D4;  // the number of the pushbutton pin

int corruptCounter = 0;

int lastState = HIGH;  // the previous state from the input pin
int currentState;      // the current reading from the input pin

void setup() {
  // initialize serial communication at 9600 bits per second:
  Serial.begin(9600);

  tft.init(172, 320);  // Init ST7789 172x320
  tft.setSPISpeed(32000000);
  drawText("init", ST77XX_WHITE);
  tft.fillScreen(ST77XX_BLACK);

  pinMode(BUTTON_PIN, INPUT_PULLUP);
  setTime(10, 30, 0, 4, 20, 23);  // set time to 10:30:00am Apr 20 2023

  if (!BLE.begin()) {
    Serial.println("- Starting Bluetooth® Low Energy module failed!");
    while (1)
      ;
  }

  BLE.setLocalName("Arduino Nano 33 BLE (Peripheral)");
  BLE.setAdvertisedService(shockService);
  shockService.addCharacteristic(shockCharacteristic);
  BLE.addService(shockService);
  shockCharacteristic.writeValue(-1);
  BLE.advertise();
}

void loop() {
  BLEDevice central = BLE.central();
  if (central) {
    while (central.connected()) {
      currentState = digitalRead(BUTTON_PIN);
      digitalClockDisplay();
      Serial.println("- Discovering central device...");

      if (lastState == LOW && currentState == HIGH && corruptCounter < 3) {
        tft.fillScreen(ST77XX_BLACK);
        drawText("14:00:00", ST77XX_WHITE);
        delay(3000);
        tft.fillScreen(ST77XX_BLACK);
        Serial.println("HIGH");
        drawHelp("FREE ME!", ST77XX_RED);
        delay(3000);

        while (1) {
          if (shockCharacteristic.written()) {
            Serial.println("one");
            Serial.println(shock);
            delay(5000);
            shock = shockCharacteristic.value();
            Serial.println("two");
            Serial.println(shock);
            writeShock(shock);
            break;
          }
        }
        tft.fillScreen(ST77XX_BLACK);

      } else {
        digitalWrite(LED_BUILTIN, HIGH);
      }
      // save the last state
      lastState = currentState;
    }
  }
}

void writeShock(int shock) {
  Serial.println("- Characteristic <shock_type> has changed!");

  switch (shock) {
    case SHOCKED:
      glitch();
      Serial.println("SHOCKED");
      break;
    case NEUTRAL:
      Serial.println("NOT SHOCKED");
      break;
    default:
      Serial.println("NEUTRAL");
      break;
  }
}

void glitch() {
  corruptCounter++;
  drawTextMiddle("WR%$%^&*()&^%$vdfdksnjfwvadvbjbva", ST77XX_RED);
  tft.invertDisplay(true);
  delay(100);
  tft.invertDisplay(false);
  delay(50);
  drawTextSmall("WVJasAvJsdncJDN^%&&*&&%$%^&*(AdKDAFJANFKsndlcd", ST77XX_RED);
  tft.invertDisplay(true);
  delay(100);
  tft.invertDisplay(false);
  delay(50);
  drawTextLarge("NJWJFDNSCndaaevbwrlnf^&*()&^%$ECJKE", ST77XX_RED);
  tft.invertDisplay(true);
  delay(100);
  tft.invertDisplay(false);
  delay(50);
  tft.invertDisplay(true);
  tft.fillScreen(ST77XX_BLACK);
}

void digitalClockDisplay() {
  char buffer[10];
  sprintf(buffer, "%d:%d:%d", hour(), minute(), second());
  if (corruptCounter >= 3) {
    drawText(buffer, ST77XX_WHITE);
    drawText(buffer, ST77XX_BLACK);
    glitch();
  } else {
    drawText(buffer, ST77XX_WHITE);
    drawText(buffer, ST77XX_BLACK);
  }
}

void drawText(char *text, uint16_t color) {
  tft.setCursor(0, 65);
  tft.setRotation(3);
  tft.setTextColor(color);
  tft.setTextSize(6);
  tft.setTextWrap(true);
  tft.print(text);
}

void drawHelp(char *text, uint16_t color) {
  tft.setCursor(0, 65);
  tft.setRotation(3);
  tft.setTextColor(color);
  tft.setTextSize(6);
  tft.setTextWrap(true);
  tft.print(text);
}

void drawTextSmall(char *text, uint16_t color) {
  tft.setRotation(3);
  tft.setCursor(0, 55);
  tft.setTextColor(color);
  tft.setTextSize(3);
  tft.setTextWrap(true);
  tft.print(text);
}

void drawTextMiddle(char *text, uint16_t color) {
  tft.setRotation(3);
  tft.setCursor(70, 65);
  tft.setTextColor(color);
  tft.setTextSize(5);
  tft.setTextWrap(false);
  tft.print(text);
}

void drawTextLarge(char *text, uint16_t color) {
  tft.setRotation(3);
  tft.setCursor(-50, 75);
  tft.setTextColor(color);
  tft.setTextSize(7);
  tft.setTextWrap(true);
  tft.print(text);
}

Click to Expand

Start the Conversation

Share this Project

Found In

RME 2023 Haunted [Smart] Homes

About

Ever feel like controlling all of your devices individually is tedious? Introducing: InControl, the only smart hub you’ll need.