Outcome

Developed by Luke Hottinger and Chris Williams

For 48-390 Physical Computing Studio (Spring 2015). Instructor: Eric Brockmeyer. 

Full documentation is also available here and here.

Brief Answers:

Course + Assignment (What were you asked to make? What course was this made in?)

We were asked to make an assistive technology with mobile physical computing techniques for Physical Computing Studio class.

Concept (what is the big idea behind this work? What was the goal of the project?)

The big idea behind this work was to explore alternate perspectives in inclusive ways. The visually impaired explore their senses more for better interaction with the world. The sighted can take vision for granted. The project is an invitation to change focus.

Outcome (What did you make and why?)

We developed a wooden, electronic, audio memory game for the visually impaired.

Wooden, because we wanted to use non-traditional materials with electronics. This space is under-explored.

Electronic, because the project require the use of digital elements.

Audio, because there is a richness and subtlety to sound that can create interesting responses.

Memory, because much interaction with the world is based in this area.

Game, because the visually impaired are people. People like entertainment and amusement. Amusement can affect quality of life positively.

Process (How was it built and what was it built with)

In the final version of Coral, the exterior is made of laser-engraved poplar wood. The electronics include three main components: a microcontroller, an accelerometer, and a wireless communication module. Specifically, it had an Arduino Uno, an Invensense MPU-6050 Accelerometer, and an NRF-8001 Bluetooth Low Energy ( BLE) module.

Collaboration (How did you work with each other, who was involved, who was the team, what was the learning experience)

The team was Luke Hottinger and Chris Williams.

The design process grew organically. First, we realized that we were seeing our concepts on different scales. Early ideas ranged from specific and utilitarian to abstract and collaborative. We did not eliminate ideas at first. We allowed for a mutual exchange of thoughts.

Once we had more specific concepts, we learned to simplify, simplify, simplify. Time was a constraint. With limited time, we pared ideas down to their essence. We asked questions such as, "What did we really want to express or communicate?"

When we reached a dead end, we re-considered our constraints and simplified. We aimed for efficacy and quality.

Reflection: What was the best part of making this? Why was this project and your experience in the class successful?

It might be cliche, but it is hard to isolate just one best part. Among other aspects, the project involved humor, respect, and realization. For example, being blind does not have to be a miserable experience. The visually impaired are people and have fun, too!

A seemingly absurd idea can have a grain of feasibility. Teasing out the grains led to more unexpected progress.

Luke and I became more cognizant of our assumptions as sighted people. For example, how do you give directions to someone that cannot see? Visual landmarks are not effective.

We were encouraged to perceive differently. We learned how to create within a space that is radically different for us. In addition, we learned from visually impaired people and better understood their concerns.

Detailed Answers:

Implementation:

Product Design:

The initial motivation for Coral was exploration of alternative perspectives, as well as collaboration.

The primary and secondary interaction modes of Coral are auditory and tactile, respectively. The project emerged from a desire to work with the visually impaired. Within that culture, sound and touch are often used in daily interaction with the world.

The form of Coral was simplified to a cube. A cube has discrete surfaces and allowed for easier mapping of tones to surfaces. Historically, the wooden block is one of the oldest and simplest toys for young children.

Coral was designed with intended use by people age seven and older. As a result, the design was lightweight.

The enclosure is 64 cu. in. (or 4 in. X 4 in. X 4 in.). The electronics were optimized to fit in a space of 27 cubic inches (or 3 in. X 3 in. X 3 in.).

Sensors:

Coral used a number of inputs and sensors to help the user navigate through the menu as well as play the game. Our project used three main components: an Arduino Uno, an Invensense MPU-6050 Accelerometer, and an NRF-8001 Bluetooth Low Energy (BLE) module. The accelerometer transferred data to the Arduino using the I2C protocol while the BLE module communicated with the Arduino using the SPI protocol. Other components used include: a piezo speaker for playing tones and a 1000 mAh LiPo battery.

Accelerometer:

The MPU-6050 is a 6-axis gyroscope/accelerometer sensor that was the basis of our gesture sensing. The sensor records both the current direction of gravity as well as the rotational speed and position of our box. These gestures were recognized through a state-based sensing mechanism. In this, each side of our cube was given a state number that represented if it was the active side (facing upward). If the cube detects that a state change was made, it begins to record the string of states after it. In 1 player mode, this string of state changes is then compared to the provided string of states randomly chosen by the program to see whether the user entered the correct pattern.

BLE Module:

The NRF-8001 is a UART based BLE module that allows for serialized data to be transmitted between the chip and another BLE enabled device. In our project, this was intended to transmit gameplay data between an iOS device and our cube for a 2 player mode. However, it served as a means of debugging and game servicing. We were able to send commands from an iOS app to the cube to test various functions in the gameplay program.

Arduino:

The Arduino Uno microcontroller acted as the central processing unit for our project, running the gameplay program as well as managing all of the connections and gestures entered. It handled a various number or protocols, such as I2C and SPI to connect to the various sensors and components. The Arduino also served as the project’s power regulator, taking in 12V and stepping it down to a component friendly 3.3V and 5V.

LiPo Battery:

The battery used was a 1000mAh, 3 Cell, 12V HobbyKing battery. This battery had plenty of power to keep our project running for upwards of 15 hours of continuous use.

Piezo Speaker:

The piezo we used had a frequency response range of 30 Hz to 15,000 Hz.

Wiring Diagram:

Material:

The enclosure is made of laser-etched and laser-cut poplar wood. Wood was chosen for its organic properties, such as its general texture and relative malleability.

Poplar was chosen due to its softness and straight, uniform grain. The uniform grain allowed for greater resolution of texture into the surface.

Interface:

Four sides of the enclosure are used during gameplay. Two sides are used for utility: the top and bottom surfaces. The top surface has a shell design that serves as a tactile “home” cue and houses the power switch. The bottom surface has a concentric pattern that led to a proposed, centrally-located USB power jack.

To readily identify the enclosure’s walls, each exterior surface was given a unique texture. The textures were modeled after the organism coral. As a species, coral has diverse textures. The use of coral provided a unifying visual and tactile motif. Similarly, the shell atop the “Home” square is modeled after a gastropod called a “limpet”.

Gameplay encouraged grasping the cube with two hands. The use of both hands insures secure rotation of Coral.

Sound Design and User Experience:

The Neutral state is defined as the Home square facing upward and the tapered side of the shell pointing at the player.

Tones:

Coral has families of codified tones. They are categorized as follows:

A. Utility:

Welcome: A set of three tones indicating that the device was first powered on.

Menu Confirmation: In Utility Mode, a tone indicating that a player’s gesture was received.

Orientation: When the Home square is upside-down, a tone encouraging the device to be returned to the Neutral state.

B. Gameplay:

In-Game: Four tones total. One tone is mapped to one side of the device.

Acknowledgement:

a. Positive Tone: During Gameplay mode, the tone indicating that a tonal sequence was matched.

b. Negative Tone: During Gameplay mode, the tone indicating that a tonal sequence was unmatched.

All tones were distinct. The Utility Tones were differentiable from the Gameplay Tones. Players can understand whether they are in the menu or in the game based on sound.

Similar to the game Simon, Coral has four In-Game Tones, which are loosely based on the F Major chord. These notes are:

F4: 349 Hz

A4: 440 Hz

C5: 523 Hz

F5: 698 Hz

The F notes differed by an octave.

Wherein the tones of Simon are reminiscent of trumpet fanfare, the In-Game Tones of Coral were inspired by an instrument called the “handpan“. All Coral tones were in consonance, with the exception of the Negative Tone.

Example of User Experience (UX):

Player turns on device

Welcome Tone plays

Player selects Game Mode

Game Modes:

1 Player:

Tilt device towards self and return to Neutral State

Play confirmation tone

2 Players:

Tilt device away from self and return to Neutral State

Play confirmation tone

Game Mode: One Player

Play In-Game Tone:

If matched, then play Positive Tone. Then, device plays a different In-Game Tone plus one more In-Game Tone.

If unmatched, then play Negative Tone. Then, game over. Return to Game Mode Selection.

Play two In-Game Tones:

If matched, then play Positive Tone. Then, device plays two different In-Game Tones plus one more In-Game Tone.

If unmatched, then play Negative Tone. Then, game over. Return to Game Mode Selection.

Name:

The name Coral was chosen for numerous reasons. There is an obvious reference to the visual motif. The more subtle aspect is that the name can double as a person’s name. This feature alludes to the history of games, like Simon and Henry.

Discussion:

Humans are visually-oriented. This preference is seen in such forms as movies, books, magazines, and live theater. There are numerous facets of life that demonstrate a bias toward the sighted, such as advertising. Focusing on under-explored aspects can yielded new and surprising results.

There are numerous technologies for the blind. However, they tend to be utilitarian, such as the walking stick. Quality of life includes ways of enjoying life, such as entertainment.

The use of wood in Coral refers to the tradition of wooden toys that spans centuries and diverse cultures, such as Ancient Egyptian. There are designers of contemporary wooden toys, such as Playsam.

Some lessons learned:

Challenge your assumptions:

There are many assumptions that sighted people make in their interactions with the world, such as using landmarks for providing directions visually. When visual cues are inaccessible, a new mode of communication needs to be established. In developing Coral, consideration was taken in making components accessible tactilely, such as the power switch.

Many sighted people were drawn to the textures of Coral. Their visual study of the forms often led to touching and exploring the cube.

Prototypes:

The first prototype was made of cardboard, which was useful in understanding the basic interaction with the cube. The second prototype was made of Open Beam and acrylic, which helped in understanding weight and robustness. The third prototype emerged from tests of wood and textures.

Differences in Visual Texture and Physical Texture:

Laser etching in wood can produce textures that are visually distinctive. However, the textures feel smooth and indistinguishable to the touch. Methods were developed to produce physically distinctive textures for Coral. As a by-product, the visual textures were more striking.

The blind painter John Bramblitt discovered the ability to paint by identifying differences in the textures of color paints. He used “haptic visualization” as a means for seeing based on touch. Similar ideas were useful in developing the textures. Players can “see” their location on the cube by touch.

Happy Accidents:

For our team, new discoveries were made with the laser engraver. We discovered that a laser engraver was capable of such varied, dimensional textures.

Due to the laser engraving method, the wood also developed the appearance of a glazed finish. The laser engraving also created a warm wood scent that lasted for days after engraving.

Conclusion:

Coral introduces a new form of entertainment into an underserved community as well as a new perspective on gameplay to an existing larger group. Coral also occupies a novel niche of wooden electronics. It is more dynamic and interactive than traditional wooden toys. Similarly, it provides a unique tactile interface that is missing in modern audio games. The greater context is the exploration of traditional materials in a non-traditional context.

View code here.

Table of Unique Harmonic Tones.