Solomon's Box

Catalog Description

Title:

Solomon’s Box

2023

Jeffrey Li, Sarah Kwok, David Oladosu

Credits:

We’d like to thank our instructor, Daragh Byrne, for presenting us with the spooky frame for referencing technology. We are very appreciative of all the love and support he’s shown us, and all of the effort he’s placed into this research project. We are very grateful for being given the opportunity and resources we need to explore these sorts of themes. We’d also like to thank our teaching assistant, Zhenfang Chen, for all of his help, and the entire IDeATe team for everything.

Description:

Solomon’s Box is an intricate puzzle box that is a true feat of craftsmanship and design. The box, inspired by the Celtic symbol known as Solomon’s Knot, is adorned with interlocking shapes and various symbols that represent the interconnectivity of life. But the box is more than just a decorative piece, it responds to movement, gestures, hand signs, and other actions within its environment. This box is an examination of people’s understanding of systems; it can only be unlocked through a series of intricate gestures and movements carried out by multiple people in different locations.

Technology has allowed us to transcend the need for direct contact and gain a level of influence over other people without being in front of them. Solomon’s box is symbolic representation of the power of technology to create interactions between people without their being in each other’s presence.

Creative Documentation

Process

The theme we wanted to explore was the feeling of interconnectedness. Something that is spooky about technology today is its power to enable human interactions that occur through invisible means. For example, remote control allows people to control machines across space by means of invisible signals, Bluetooth allows devices to “talk” with each other without any sort of physical connection, and the internet enables people to share information and experiences across the world while remaining in singular locations. We took inspiration from all these sorts of interactions because they enable things to affect other things through indirect, mysterious, seemingly other-worldly means rather than through direct contact.

For inspiration regarding the physical medium through which we would decide to explore the previously mentioned themes, we looked at mechanical puzzles, escape rooms, kinetic art, and other mechanical systems. We believed that the idea of interconnectedness would better be experienced if people could interact with something and see and feel how it would respond to them. Specifically, we took inspiration from a $30,000 puzzle box designed by Labsterium and various mechanical puzzles designed by local shops.

Our exploration resulted in an interactive installation that immerses people into an interconnected space. Solomon’s Box was an innovative creation that combined electronics, mechanical systems, and Bluetooth technology to create an engaging experience for users.

The box itself is made mainly from laser cut parts. It has several wood components including its gears, shelves, walls, designs, and connectors, as well as acrylic components. Most of those parts are fastened with either M3 hardware or industrial glue. The box contains 3 stepper motors along with 3 gear trains. The stepper motors have a driving gear connected to their shafts via gear couplers. Those driving gears turn their adjacent gears which then turn the patterns that they are connected to.

For the electronics, we used 4 Arduino Nano 33 BLE Sense devices that communicated with each other via Bluetooth. One Arduino controlled the rabbit and caused its LEDs to light up every time its proximity sensor was triggered. Another Arduino controlled the twin clocks which would spin if their proximity sensors were triggered. The third Arduino was placed inside the jewelry box and would send signals whenever someone placed their hand inside of the box. The last Arduino was the central one; it took in the signals from the 3 peripheral Arduinos and used their data to control the state of the puzzle box.

Although we ran into technical difficulties during the exhibition, our overall design was captivating. Ideally, it would be refined to utilize Bluetooth across different spaces so that people would interact with objects in front of them, but then hear sounds from across the room. In general, our project has the potential to be further explored to create more interactive, engaging aspects that unearth the theme of interconnectedness further.

A big challenge that we ran into toward the start was developing the concept we wanted to explore. We knew that we wanted to investigate a topic involving human interaction and the unintended consequences that can result from them, but it was difficult to think of the sorts of interactions we wanted people to have. We decided to create a puzzle because it would create a sense of mystery that makes people ask, “what happens when I solve this?” This would serve as a good framework to work around since we could use it to frustrate people’s expectations. The idea would be that when one person interacts with a thing, it might get them closer to a solution, but when another person interacts with another thing, it would hinder the process that the former person had just made. People would eventually realize that the actions of other people were also affecting the thing that was directly in front of them.

The next phase of the process focused on designing the puzzle box. For that, we took inspiration from mechanical puzzles and escape rooms. We decided that we wanted the scale of the puzzle box to be large so that it would create a more engaging experience.

Once the design was completed, we completed the next stages of our project in parallel: one person on electronics and two on fabrication.

One problem we ran into was that our device couldn’t operate due to factors that weren’t considered in the first design. Our gears weren’t spinning because of the friction between each wooden piece. After we realized this issue, we redesigned the part to incorporate ball bearings that would allow for much smoother motion. The use of bearings enabled our gears to spin properly but required us to alter several other parts of the box to properly incorporate them. In the end though, they greatly increased the functionality of our artifact.

Another huge issue we ran into was using BLE (Bluetooth® Low Energy). We originally intended to use the HC-SR05 Bluetooth modules since they are simple, reliable, and very user friendly, but the Physical Computing Lab didn’t carry them in stock and we weren’t able to acquire them in a reasonable amount of time. Because of this, we attempted to use the built-in Bluetooth modules that the Arduinos use, but since they are relatively new and complex, we struggled to find any decent documentation on how to incorporate them. We experienced a lot of inconsistencies and bugs when trying to use the built-in BLE modules, so we switched to serial communication.

Overall, we are happy with our final product. We learned a good deal about design for manufacturing, integrating electronics with mechanical systems, and programming with communication in mind. In the future, we’d love to further explore ways we can create interactive environments utilizing these tools.

Build (Technical Documentation)

Below is a collection of the technical resources involved in the creation of our artifacts. These include schematics, system diagrams, a bill of materials, a tech rider, and source code, as well as details that one might need if they were to recreate the exhibit themselves.

Tech Rider

The following is a list of additional components that are necessary for installing the exhibit.

Cardboard boxes

1 large table for puzzle box
x4 pedestals/small tables/things to place other small objects on
x2 tall purple lamps
x2 purple floor lamps
x2 12v power supply (goes into 5mm barrel jack)
x4 5v power supply (goes into 5mm barrel jack)

Bill_of_Materials_-_Sheet1.pdf

67.818 KB · Download / View

Schematic

System Diagrams

Solomon's Box Rotation

Jeffrey Li - https://youtube.com/shorts/o9cQGoD1Yd4?feature=share

Solomon's Box Demo

Jeffrey Li - https://youtu.be/VDokhw1MGNE

Code for Central Arduino (Solomon's Box)

//____________________________________________________________________________________________________________________________
//DESCRIPTION-----------------------------------------------------------------------------------------------------------------
/*This code is for the central arduino that controls Solomon's Box in the Solomon's box installation. It communicates with the
other devices via bluetooth and controls 3 stepper motors.*/


//____________________________________________________________________________________________________________________________
//LIBRARIES AND DEFINITIONS---------------------------------------------------------------------------------------------------
#include <ArduinoBLE.h>
#include <Arduino_APDS9960.h>
#include <AccelStepper.h>


//____________________________________________________________________________________________________________________________
//GLOBAL VARIABLES------------------------------------------------------------------------------------------------------------
//MOTOR---------------------------------------------------------------
int motorPos1 = 0; //track motor positions
int motorPos2 = 0;
int motorPos3 = 0;
unsigned char pos1 [4];
unsigned char pos2 [4];
unsigned char pos3 [4];

const int stepPin1 = 2; //assigns pins to motors
const int dirPin1 = 3;
const int stepPin2 = 4;
const int dirPin2 = 5;
const int stepPin3 = 6;
const int dirPin3 = 7;

AccelStepper motor1(1, stepPin1, dirPin1); //creates motor objects
AccelStepper motor2(1, stepPin2, dirPin2);
AccelStepper motor3(1, stepPin3, dirPin3);


//BLE-----------------------------------------------------------------
//Services and Characteristics of other devices
const char* jewelryBoxServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a2008";
const char* motor1CharacteristicUuid = "19b10000-e8f2-537e-4f6c-d104768a1990";
const char* motor2CharacteristicUuid = "19b10000-e8f2-537e-4f6c-d104768a1991";
const char* motor3CharacteristicUuid = "19b10000-e8f2-537e-4f6c-d104768a1992";
BLEService jewelryBoxService;
BLECharacteristic motor1Characteristic;
BLECharacteristic motor2Characteristic;
BLECharacteristic motor3Characteristic;


//OTHER-----------------------------------------------------------------
bool connectedToJewelryBox = false;


//____________________________________________________________________________________________________________________________
//INITIALIZATION--------------------------------------------------------------------------------------------------------------
void setup() {
  Serial.begin(115200);
  while (!Serial);
  delay(3000);

  //APDS--------------------------------------------------------------
  if (!APDS.begin()) {
    Serial.println("* Error initializing APDS9960 sensor!");
  } 

  APDS.setGestureSensitivity(80); 


  //BLE---------------------------------------------------------------
  if (!BLE.begin()) { //initialize bluetooth low energy
    Serial.println("* Starting Bluetooth® Low Energy module failed!");
    while (1);
  }

  //Add characteristics to service, add service to device
  BLE.setLocalName("Solomon's Box (Central)");

  //Advertise
  BLE.setDeviceName("Solomon's Box (Central)");
  BLE.advertise();

  Serial.println("Solomon's Box (Central Device)");
  Serial.println(" ");

  //Motor------------------------------------------------------------
  motor1.setMaxSpeed(1000); // measured in steps per second
  motor1.setAcceleration(500); // measured in steps per second squared
  motor2.setMaxSpeed(1000);
  motor2.setAcceleration(500);
  motor3.setMaxSpeed(1000);
  motor3.setAcceleration(500);


}

//____________________________________________________________________________________________________________________________
//LOOP------------------------------------------------------------------------------------------------------------------------
void loop() {
  alterMotorPos(); 
  motor1.moveTo(motorPos1); // and tell the motor to go there
  motor2.moveTo(motorPos2);
  motor3.moveTo(motorPos3);
  motor1.run(); //commands motor to run
  motor2.run();
  motor3.run();
}


//____________________________________________________________________________________________________________________________
//HELPER FUNCTIONS------------------------------------------------------------------------------------------------------------

void alterMotorPos() {
  if (!Serial.available()) {
    return;
  }
  char num = Serial.read();
  //Serial.println(num);
  if (num == '1') {
    //Serial.println("1 detected");
    motorPos1 += 300;
    motorPos2 += 500;
    motorPos3 += 600;
  }

  if (num == '2') {
    //Serial.println("2 detected");
    motorPos1 += 700;
    motorPos2 += 800;
    motorPos3 += 650;
  }

  if (num == '3') {
    //Serial.println("3 detected");
    motorPos2 += 80;
  }

  //Debugging
  // Serial.print("motorPos1: ");
  // Serial.print(motorPos1);
  // Serial.print("                ");
  // Serial.print("motorPos2: ");
  // Serial.print(motorPos2);
  // Serial.print("                ");
  // Serial.print("motorPos3: ");
  // Serial.println(motorPos3);
}

Click to Expand

Code for Arduino Controlling Rabbit (Peripheral)

//____________________________________________________________________________________________________________________________
//DESCRIPTION-----------------------------------------------------------------------------------------------------------------
//This code is for the rabbit in the Solomon's Box exhibit. If someone triggers the proximity sensor in front of
//it, it lights up and sends a signal over to the master Arduino (Solomon's box).

//____________________________________________________________________________________________________________________________
//LIBRARIES AND DEFINITIONS---------------------------------------------------------------------------------------------------
#include <ArduinoBLE.h>
#include <Arduino_APDS9960.h>
#include <Adafruit_NeoPixel.h>

//____________________________________________________________________________________________________________________________
//GLOBAL VARIABLES------------------------------------------------------------------------------------------------------------

//BLE
const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1214";
const char* deviceCharacteristicUuid = "19b10000-e8f2-537e-4f6c-d104768a2006";

int gesture = -1;
int motorPos1 = 0;
BLEService bearService(deviceServiceUuid);
BLEIntCharacteristic bearCharacteristic(deviceCharacteristicUuid, BLERead | BLEWrite);

//LED
#define LED_PIN    4
#define LED_COUNT 12
Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRB + NEO_KHZ800);
bool stripWhite = true;

//Other Stuff
int distanceLimit = 36;
bool inView; //tracks whether or not somebody is in front of the box
const int echoPin1 = 3;
const int trigPin1 = 2;
bool timeConnectedStarted = false;
int timeConnected = 0;
bool goingUp = true;

//____________________________________________________________________________________________________________________________
//INITIALIZATION--------------------------------------------------------------------------------------------------------------
void setup() {
  Serial.begin(115200);
  delay(10000);
  
  //APDS------------------------------------------------------------
  while (!Serial);
  if (!APDS.begin()) {
    Serial.println("Error initializing APDS9960 sensor!");
  }
  // for setGestureSensitivity(..) a value between 1 and 100 is required.
  // Higher values makes the gesture recognition more sensible but less accurate
  // (a wrong gesture may be detected). Lower values makes the gesture recognition
  // more accurate but less sensible (some gestures may be missed).
  // Default is 80
  //APDS.setGestureSensitivity(80);

  //BLE--------------------------------------------------------------
  BLE.begin();
  if (!BLE.begin()) {                                //if bluetooth initiation fails, stop running code
    Serial.println("Bluetooth initiation failed");
    while(1);
  }

  BLE.setLocalName("Bear");
  BLE.setDeviceName("Bear");
  BLE.setAdvertisedService(bearService);
  bearService.addCharacteristic(bearCharacteristic);
  bearCharacteristic.writeValue(0);
  BLE.addService(bearService);
  BLE.advertise();

  //LED----------------------------------------------------------------
  strip.begin();           // INITIALIZE NeoPixel strip object (REQUIRED)
  strip.show();            // Turn OFF all pixels ASAP
  strip.setBrightness(50);
  for(int i=0; i<12; i++) {
    strip.setPixelColor(i, 0, 0, 0);
  }
  strip.show();

  //Ultrasonic sensor initialization------------------------------------
  pinMode(trigPin1, OUTPUT);
  pinMode(echoPin1, INPUT);

  Serial.println("Advertising Bear");
}




//____________________________________________________________________________________________________________________________
//LOOP------------------------------------------------------------------------------------------------------------------------
void loop() {
  BLEDevice central = BLE.central();
  
  if (central) {
    //While connected to central, detect gesture, turn servo, alter characteristic value
    if (central.connect()) {
      if (!timeConnectedStarted) {
        timeConnectedStarted = true;
        timeConnected = millis();
      }
      if (millis() - timeConnected >= 15000) act();
    }
  }
  else {
    timeConnected = millis();
    timeConnectedStarted = false;
    central.disconnect();
    Serial.println("Looking for central device to connect to");
    delay(500);
  }
}





//____________________________________________________________________________________________________________________________
//HELPER FUNCTIONS------------------------------------------------------------------------------------------------------------

//Used for detecting gesture and altering servo motor position
void act() {
  //Ultrasonic sensors
  int dist1 = getDist(trigPin1, echoPin1);
  //Serial.print(dist1);
  //Serial.println("cm");

  if (dist1 <= distanceLimit) {
    for(int i=0; i <= strip.numPixels(); i++) {
      strip.setPixelColor(i, 0, 0, 255);
    }
    strip.show();
    if (!inView) {
      updatePos();
    }
    inView = true;
  }
  else {
    for(int i=0; i <= strip.numPixels(); i++) {
      strip.setPixelColor(i, 0, 0, 0);
    }
    strip.show();
    inView = false;
  }
}

void updatePos() {
  int motorPos1 = bearCharacteristic.value();
  bearCharacteristic.writeValue(motorPos1 + 77);
  Serial.println(bearCharacteristic.value());
}

//Returns distance detected by ultrasonic sensor
int getDist(int trigPin, int echoPin) {
  long duration, cm;
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  duration = pulseIn(echoPin, HIGH);
  cm = microsecondsToCentimeters(duration);
  return cm;
}

//For Ultrasonic Sensor From https://www.tutorialspoint.com/arduino/arduino_ultrasonic_sensor.htm
long microsecondsToInches(long microseconds) {
   return microseconds / 74 / 2;
}

long microsecondsToCentimeters(long microseconds) {
   return microseconds / 29 / 2;
}

Click to Expand

Code for Arduino Controlling Clocks (Peripheral)

//____________________________________________________________________________________________________________________________
//DESCRIPTION-----------------------------------------------------------------------------------------------------------------
//This code is for the clocks in the Solomon's Box exhibit. If someone triggers the proximity sensor in front of
//either clock, it spins the hand of the opposite clock and sends a signal over to the master Arduino (Solomon's box).

//____________________________________________________________________________________________________________________________
//LIBRARIES AND DEFINITIONS---------------------------------------------------------------------------------------------------
#include <ArduinoBLE.h>
#include <Arduino_APDS9960.h>
#include <Adafruit_NeoPixel.h>
#include <SPI.h>


//____________________________________________________________________________________________________________________________
//GLOBAL VARIABLES------------------------------------------------------------------------------------------------------------

//BLE
const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a1213";
const char* deviceCharacteristicUuid = "19b10000-e8f2-537e-4f6c-d104768a1212";

int motorPos1 = 0;
BLEService clocksService(deviceServiceUuid);
BLEIntCharacteristic clocksCharacteristic(deviceCharacteristicUuid, BLERead | BLEWrite);

//Ultrasonic Sensors
const int echoPin2 = 7;
const int trigPin2 = 6;
const int echoPin1 = 5;
const int trigPin1 = 4;

//Motors
const int motorPin1 = 3;
const int motorPin2 = 2;
const int motorSpeed = 100;

//Other Stuff
bool inView1 = false; //Keep track of whether someone is in front of the tower or not
bool inView2 = false;
unsigned long notInViewTimerStart1 = millis();
unsigned long notInViewTimerStart2 = millis();
bool timerStarted1 = false;
bool timerStarted2 = false;
bool twinClockTowers; //tracks whether or not a message should be sent to Solomon's box
int timeLimit = 200;
int distanceLimit = 30;
bool timeConnectedStarted = false;
int timeConnected = 0;

//____________________________________________________________________________________________________________________________
//INITIALIZATION--------------------------------------------------------------------------------------------------------------
void setup() {
  Serial.begin(115200);
  delay(10000);
  
  //APDS------------------------------------------------------------
  while (!Serial);
  if (!APDS.begin()) {
    Serial.println("Error initializing APDS9960 sensor!");
  }
  // for setGestureSensitivity(..) a value between 1 and 100 is required.
  // Higher values makes the gesture recognition more sensible but less accurate
  // (a wrong gesture may be detected). Lower values makes the gesture recognition
  // more accurate but less sensible (some gestures may be missed).
  // Default is 80
  //APDS.setGestureSensitivity(80);

  //BLE--------------------------------------------------------------
  BLE.begin();
  if (!BLE.begin()) {                                //if bluetooth initiation fails, stop running code
    Serial.println("Bluetooth initiation failed");
    while(1);
  }

  BLE.setLocalName("clocks");
  BLE.setDeviceName("clocks");
  BLE.setAdvertisedService(clocksService);
  clocksService.addCharacteristic(clocksCharacteristic);
  clocksCharacteristic.writeValue(0);
  BLE.addService(clocksService);
  BLE.advertise();

  //Ultrasonic sensor initialization------------------------------------
  pinMode(trigPin1, OUTPUT);
  pinMode(echoPin1, INPUT);
  pinMode(trigPin2, OUTPUT);
  pinMode(echoPin2, INPUT);

  //Motor initialization-------------------------------------------------
  pinMode(motorPin1, OUTPUT);
  pinMode(motorPin2, OUTPUT);

  Serial.println("Advertising Clocks");
}




//____________________________________________________________________________________________________________________________
//LOOP------------------------------------------------------------------------------------------------------------------------
void loop() {
  BLEDevice central = BLE.central();
  
  if (central) {
    //While connected to central, detect gesture, turn servo, alter characteristic value
    if (central.connect()) {
      if (!timeConnectedStarted) {
        timeConnectedStarted = true;
        timeConnected = millis();
      }
      while (millis() - timeConnected >= 10000);
    }
  }
  else {
    timeConnected = millis();
    timeConnectedStarted = false;
    central.disconnect();
    Serial.println("Looking for central device to connect to");
    delay(500);
  }
  act();
}






//____________________________________________________________________________________________________________________________
//HELPER FUNCTIONS------------------------------------------------------------------------------------------------------------

//Used for detecting gesture and altering servo motor position
void act() {
  //Ultrasonic sensors
  int dist1 = getDist(trigPin1, echoPin1);
  int dist2 = getDist(trigPin2, echoPin2);
  //Tower 1
  //If out of view of tower 1, start a timer and stop spinning after certain duration. Else, someone is in view so spin.
  if (dist1 < distanceLimit) {
    analogWrite(motorPin2, motorSpeed);
    inView1 = true;
  }
  else {
    analogWrite(motorPin2, 0);
    inView1 = false;
  }

  //Tower 2
  if (dist2 < distanceLimit) {
    analogWrite(motorPin1, motorSpeed);
    inView2 = true;
  }
  else {
    analogWrite(motorPin1, 0);
    inView2 = false;
  }
  
  if (inView1 == true && inView2 == true) updatePos();

}

void updatePos() {
  int motorPos1 = clocksCharacteristic.value();
  clocksCharacteristic.writeValue(motorPos1 + 230);
  Serial.println(clocksCharacteristic.value());
}

//Returns distance detected by ultrasonic sensor
int getDist(int trigPin, int echoPin) {
  long duration, cm;
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  duration = pulseIn(echoPin, HIGH);
  cm = microsecondsToCentimeters(duration);
  return cm;
}

//For Ultrasonic Sensor From https://www.tutorialspoint.com/arduino/arduino_ultrasonic_sensor.htm
long microsecondsToInches(long microseconds) {
   return microseconds / 74 / 2;
}

long microsecondsToCentimeters(long microseconds) {
   return microseconds / 29 / 2;
}

Click to Expand

Code for Arduino Controlling Jewelry Box (Peripheral)

//____________________________________________________________________________________________________________________________
//DESCRIPTION-----------------------------------------------------------------------------------------------------------------
//This code is for the jewelry box in the Solomon's Box exhibit. If someone triggers the gesture sensor in front of
//the box, it spins its design and sends a signal over to the master Arduino (Solomon's box).

//____________________________________________________________________________________________________________________________
//LIBRARIES AND DEFINITIONS---------------------------------------------------------------------------------------------------
#include <ArduinoBLE.h>
#include <Arduino_APDS9960.h>
#include <Servo.h>
#include <SPI.h>



//____________________________________________________________________________________________________________________________
//GLOBAL VARIABLES------------------------------------------------------------------------------------------------------------
//Servo
Servo motor;

//BLE
const char* deviceServiceUuid = "19b10000-e8f2-537e-4f6c-d104768a2008";
const char* deviceCharacteristicUuid = "19b10000-e8f2-537e-4f6c-d104768a1990";

int gesture = -1;
int motorPos1 = 0;
BLEService jewelryBoxService(deviceServiceUuid);
BLEIntCharacteristic jewelryBoxCharacteristic(deviceCharacteristicUuid, BLERead | BLEWrite);

//PINS----------------------
//LED
const int ledPin = 11;

//Servo Motor
const int servoPin = 8;

//Ultrasonic Sensors
const int echoPin3 = 10;
const int trigPin3 = 9;
const int echoPin2 = 7;
const int trigPin2 = 6;
const int echoPin1 = 5;
const int trigPin1 = 4;

//DC Motors
const int motorPin1 = 3;
const int motorPin2 = 2;
const int motorSpeed = 100;



//Other Stuff
bool inView1 = false; //Keep track of whether someone is in front of the tower or not
bool inView2 = false;
unsigned long notInViewTimerStart1 = millis();
unsigned long notInViewTimerStart2 = millis();
bool timerStarted1 = false;
bool timerStarted2 = false;
bool twinClockTowers; //tracks whether or not a message should be sent to Solomon's box
int timeLimit = 200;
int distanceLimit = 30;
bool timeConnectedStarted = false;
int timeConnected = millis();



//____________________________________________________________________________________________________________________________
//INITIALIZATION--------------------------------------------------------------------------------------------------------------
void setup() {
  Serial.begin(115200);
  delay(10000);
  
  //APDS------------------------------------------------------------
  while (!Serial);
  if (!APDS.begin()) {
    Serial.println("Error initializing APDS9960 sensor!");
  }
  // for setGestureSensitivity(..) a value between 1 and 100 is required.
  // Higher values makes the gesture recognition more sensible but less accurate
  // (a wrong gesture may be detected). Lower values makes the gesture recognition
  // more accurate but less sensible (some gestures may be missed).
  // Default is 80
  //APDS.setGestureSensitivity(80);

  //Servo------------------------------------------------------------
  motor.attach(servoPin);

  //Ultrasonic sensor initialization
  pinMode(trigPin1, OUTPUT);
  pinMode(echoPin1, INPUT);
  pinMode(trigPin2, OUTPUT);
  pinMode(echoPin2, INPUT);
  pinMode(trigPin3, OUTPUT);
  pinMode(echoPin3, INPUT);

  //Motor initialization
  pinMode(motorPin1, OUTPUT);
  pinMode(motorPin2, OUTPUT);

  //BLE--------------------------------------------------------------
  BLE.begin();
  if (!BLE.begin()) {                                //if bluetooth initiation fails, stop running code
    Serial.println("Bluetooth initiation failed");
    while(1);
  }

  BLE.setLocalName("Jewelry Box");
  BLE.setDeviceName("Jewelry Box");
  BLE.setAdvertisedService(jewelryBoxService);
  jewelryBoxService.addCharacteristic(jewelryBoxCharacteristic);
  jewelryBoxCharacteristic.writeValue(0);
  BLE.addService(jewelryBoxService);
  BLE.advertise();

  Serial.println("Advertising Jewelry Box");
}




//____________________________________________________________________________________________________________________________
//LOOP------------------------------------------------------------------------------------------------------------------------
void loop() {
  BLEDevice central = BLE.central();
  
  if (central) {
    //While connected to central, detect gesture, turn servo, alter characteristic value
    if (central.connect()) {
      if (!timeConnectedStarted) {
        timeConnectedStarted = true;
        timeConnected = millis();
      }
      if (millis() - timeConnected >= 15000) act();
    }
  }
  else {
    timeConnected = millis();
    timeConnectedStarted = false;
    central.disconnect();
    Serial.println("Looking for central device to connect to");
    delay(500);
  }
}






//____________________________________________________________________________________________________________________________
//HELPER FUNCTIONS------------------------------------------------------------------------------------------------------------

//Used for detecting gesture and altering servo motor position
void act() {
  box();
  clocks();
}

void box() {
  if (APDS.gestureAvailable()) {
    // a gesture was detected, read and print to serial monitor
    int gesture = APDS.readGesture();
    switch (gesture) {
      case GESTURE_UP:
        Serial.println("Detected UP gesture");
        motor.write(90);
        motorPos1 = jewelryBoxCharacteristic.value();
        Serial.println(motorPos1);
        jewelryBoxCharacteristic.writeValue(motorPos1 + 100);
        Serial.print("Jewelry box characteristic val is now: ");
        Serial.println(jewelryBoxCharacteristic.value());
        break;
      case GESTURE_DOWN:
        Serial.println("Detected DOWN gesture");
        motor.write(270);
        motorPos1 = jewelryBoxCharacteristic.value();
        jewelryBoxCharacteristic.writeValue(motorPos1 + 100);
        Serial.println("Just rewrote motor 1 value");
        break;
      case GESTURE_LEFT:
        Serial.println("Detected LEFT gesture");
        motor.write(180);
        motorPos1 = jewelryBoxCharacteristic.value();
        jewelryBoxCharacteristic.writeValue(motorPos1 + 100);
        Serial.println("Just rewrote motor 1 value");
        break;
      case GESTURE_RIGHT:
        Serial.println("Detected RIGHT gesture");
        motor.write(0);
        motorPos1 = jewelryBoxCharacteristic.value();
        jewelryBoxCharacteristic.writeValue(motorPos1 + 100);
        Serial.println("Just rewrote motor 1 value");
        break;
      default:
        break;
    }
  }
}

void clocks() {
  //Ultrasonic sensors
  int dist1 = getDist(trigPin1, echoPin1);
  int dist2 = getDist(trigPin2, echoPin2);
  //Serial.print("Left: ");
  //Serial.print(dist1);
  //Serial.print("cm            ");
  //Serial.print("Right: ");
  //Serial.print(dist2);
  //Serial.println("cm            ");

  //Tower 1
  //If out of view of tower 1, start a timer and stop spinning after certain duration. Else, someone is in view so spin.
  if (dist1 > distanceLimit) {
    if (!timerStarted1) {             //if a timer hasn't already started, start timer
      notInViewTimerStart1 = millis();
      timerStarted1 = true;
    }
    if (millis() - notInViewTimerStart1 >= timeLimit) {
      inView1 = false;
      digitalWrite(motorPin2, 0);
      //for (int i=0; i < 12; i++) {
      //  strip2.setPixelColor(i, 255, 255, 255);
      //}
      //strip2.show();
    }
  }
  else {
    digitalWrite(motorPin2, motorSpeed);
    timerStarted1 = false;
    inView1 = true;
    //for (int i=0; i < 12; i++) {
    //  strip2.setPixelColor(i, 0, 0, 255);
    //}
    //strip2.show();
  }

  //Tower 2
  if (dist2 > distanceLimit) {
    if (!timerStarted2) {             //if a timer hasn't already started, start timer
      notInViewTimerStart2 = millis();
      timerStarted2 = true;
    }
    if (millis() - notInViewTimerStart2 >= timeLimit) {
      inView2 = false;
      digitalWrite(motorPin1, 0);
      //for (int i=0; i < 12; i++) {
      //  strip1.setPixelColor(i, 255, 255, 255);
      //}
      //strip1.show();
    }
  }
  else {
    digitalWrite(motorPin1, motorSpeed);
    timerStarted2 = false;
    inView2 = true;
  //  for (int i=0; i < 12; i++) {
  //    strip1.setPixelColor(i, 0, 0, 255);
  //  }
  //  strip1.show();
  }

  //If both towers are on, send signal to Solomon's Box
  if (inView1 == true && inView2 == true) twinClockTowers = true;
  else twinClockTowers = false;
  
}

//Returns distance detected by ultrasonic sensor
int getDist(int trigPin, int echoPin) {
  long duration, cm;
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
  duration = pulseIn(echoPin, HIGH);
  cm = microsecondsToCentimeters(duration);
  return cm;
}

//For Ultrasonic Sensor From https://www.tutorialspoint.com/arduino/arduino_ultrasonic_sensor.htm
long microsecondsToInches(long microseconds) {
   return microseconds / 74 / 2;
}

long microsecondsToCentimeters(long microseconds) {
   return microseconds / 29 / 2;
}

Click to Expand

Roadmap

To take this project forward over a 12 week period, there are several areas that we would focus our time on. From a mechanical and physical point of view, we would create a prototype that is generally more robust. From a hardware perspective, we would switch the pieces in order to be more reliable pieces, so that we would be able to have a more reliable bluetooth connection between all of the elements. As mentioned earlier, we planned on using HC-SR05 Bluetooth modules for simple, user-friendly communication, but we had no means of acquiring them in a reasonable amount of time. Because of this, we attempted to use the built-in BLE capabilities of the Arduino Nano 33 BLE Sense, but these devices are relatively new and unreliable for consistent Bluetooth communication between multiple devices. They also lack documentation because of how new they are, so it was very frustrating to work with them. For this reason, we decided to switch to serial communication which forced us to place all of our electronics in close proximity. And from the software end, we felt that we just needed a bit more time as we had to make some last minute adjustments.

In regards to the interactions itself, we had discussions about the puzzle box opening or performing an action once it is solved. Taking this project forward, we would implement an extra mechanism, whether it is a part of the box or another isolated artifact, that opens or moves once the puzzle is solved. Another interaction that we had discussed for a while but ultimately fell out of favor was using hand signs, or even body positions to control the rings on the box. We had begun some shallow explorations of these interactions, but ultimately ran out of time to fully incorporate them. Incorporating these interactions would make the experience much more immersive, and provide a contrasting interaction from the smaller artifacts scattered around the room.

In regards to resolving the software issues that we had, having extra time would be sufficient for us to solve those issues, as we did not feel that they were massive impediments. In regards to the puzzle box, having a bit more time to consider all of the pieces and mechanics would be ideal. For example, we did not consider how we might attach an aluminum bearing, and we needed to retrofit it to work. By slightly redesigning the box, we would be able to build a much more robust prototype.

Aesthetically, we received a number of comments regarding the affordances that our objects provided. If we were to take this forward, we would spend more time experimenting with which kinds of symbols and forms would be most suggestive of the types of interactions that we were hoping to create.

With more time, we would implement user testing at several stages of the project, to better understand where our shortcomings and successes are.

Critical Reflection

Our final installation was a bit more staged than we would have liked. We feel that the overall aesthetics and visual presentation was successful, even without the full functionality of the project. On the other hand, something that could have been refined was the staging of the overall project. We could have spent more time thinking about how we could have covered and hidden the wires of our project to work more towards the experience we were envisioning. Due to the technical difficulties that resulted in us hard wiring the electronics as opposed to using bluetooth, our various props were concentrated on one table. Because our initial proposition was to demonstrate the unseen, intangible interconnectedness of technology, being more deliberate in spacing and staging would have further reinforced that "spooky" connectivity even with the wiring. Furthermore, when presenting some comments we received had to do the user experience and specific narrative of our box - why is the box in the attic? Why did we choose the attic specifically? Why would someone be interacting with this box in the first place? We had focused our project energies on the development of the electronics and how interfacing them with each other, as well as how that communicated our overall concept, that we did not address more clearly how the user actually interfaced with the project.