← Projects

Gossiper

Made by ihans and Theo Mandin-Lee

Make and Spread gossip

Created: April 4th, 2022

0

Intention

Privacy is not secrecy but rather a dynamic process of having control over what one wants to share with selected others and what not (Altman 1975). When seen from this lens, it has three aspects: 

Collecting Information, Deducting / Inferencing, and Sharing. 

Out of these three, the collection aspect is the one most often talked about. To subtly shed light on the less visible other two components of this phenomenon, we created a 'playful' and 'fickle' Gossiping device situated in a co-working space. This gossipper hears everything about the people in its environment and combines it with other things it learns from elsewhere or through observation. It then decides to share this information back with everyone rather publicly, sometimes being helpful, sometimes being hurtful, and sometimes being annoying or even creepy. No one knows of its identity, no one knows when it might share something, but everyone knows it's someone or something around them!

We foresee this Gossiper to be a counterfactual, which over a period of time may even become attached to the people or place it is in.

0

Context

Through this project, our intent was to position something in between a smart home assistant and a presence in the space that remembers the events, makes its deductions, and intervenes in the life of people when and how it desires. In that sense, it has autonomy and a will of its own but it is very much situated in the place it has experienced these events. For this reason, we drew inspiration from three different projects: 

  1. Project Alias: Even though this wasn't directly in line with our idea, it is an interesting project for how it brings in an element of autonomy. It attempts to balance the power relationship between the users and consumer smart home product companies by giving users the ability to temporarily disconnect.
  2. Conversnitch: This project is very close to our except that it talks about surveillance from the perspective of only listening. The resources from this project served as a useful guide for us.

0

Process

Part 1: Mind Mapping

As a team, we first began looking for connections between themes, concepts, and ideas. From early on, the idea was grounded outside the realm of the home, in more public spaces that are shared by people both professionally and socially. Since there are multiple actors in these spaces, the idea evolved to include listening, inferencing and sharing, as is the case in such spaces. At this early stage, we didn't have clarity on the output channels, but we were intrigued by the idea of printing so that the stories created by this gossiper could be left for anyone to chance upon. 

0

Part 2: Working with GPT-3

Since we wanted this device to make inferences and the device had to be 'fickle', we had to find software that would be able to generate responses without us needing to write exact conditions for all possible scenarios. This quickly ruled out Alexa since it is based heavily on conditional logic. Open AI's GPT-3 was our ideal choice and became the main mode of interpretation and responding. 

We spent a good amount of time experimenting with GPT 3 for our prototype. The first iteration started with writing a script for two people conversing in a studio setting about a group project. This script had names and corresponding dialogues, which our actual device would never be able to produce in terms of the fidelity as well as the names of the speakers because at this point in the process it's acting as a mere recorder. However, it gave as a good start to first get a hang of tweaking the GPT 3 parameters and how they created different outcomes. We went through about 15 scripts and multiple variations of adjusting three parameters before we started receiving responses that seems like a good balance of what we fed and what GPT 3 gave as an output.

Parameter 1: Prompt: The obvious one is to ask GPT to write a summary for the script, but the outcome wasn’t playful by any means. We tried two three other variations before literally asking it to create a gossip based on the above text, which worked much better. We borrowed the personality of Gossip girl and wrote our ghost story in a similar format, and realized that specific text and format could be included in the command like start with “Hey….” And end with two funny questions and XOXO.

Parameter 2: Presence Penalty: The description said this means that the likelihood to talk about new topics would increase with this and we found this to be a really interesting feature because it added randomness to the outcome.

Parameter 3: Maximum length: This was tricky in finding the right balance because we didn’t want the gossip to be too long or descriptive yet not too short to feel like a tweet. We finally settled on around 260 tokens given that we were commanding GPT 3 to start and end the response with a specific sentence.  

0

After multiple trials, we created a repository of scripts, outputs, parameters, and API that could be put in Python.

0

Part 3: Connecting all APIs in Python

The technical implementation was challenging, to say the least. Compatibility between systems, API SDK libraries, and computing devices proved to be the most difficult, as a number of different libraries and packages were necessary for each piece of hardware and API. The output for the gossip was changed from printing to a simpler Slack channel because of difficulty with the emailing API. The program takes audio input with the Azure Speech-to-Text SDK, handles it and extracts just the text, sends the text in a request to the OpenAI SDK, recieves and parses the response and posts it to Slack.

0

Part 4: IF we had more time

What remains unresolved (in the concept, implementation, or conversation around this outcome)? What are the things we should pay attention to for future explorations? What questions remain to be addressed?

For implementation with the Raspberry Pi & Python, there are a few things left to be resolved. Currently, the program is designed for recording for 15 secs or short utterances and the interpretation can be mixed. In the theme of inferencing by GPT3, this could be technically refined along with better inferencing and better triggering for the audio recording. By refining the implementation, the concept is strengthened by the unsettling or remarkably accurate things that GPT3 can say about the conversations that are ‘overheard.’ Slack was used as an output for the program for ease of technical implementation, though the original concept for output printing to one of many workspace printers. Implementing this would close the loop of the physical-digital-physical flow of information. 

0

Product

Describe your experience/working prototype: What did you create, how, etc.? What tools and technologies were involved? Include appropriate content and illustration (e.g. a concept video, a video of the device in operation, diagrams, code, etc.) How does it relate or build on existing work (provide acknowledgements or cite this work).

You should report this in sufficient detail that anyone knowledgeable with electronics etc would be able to reconstruct your implementation. Be sure to include a system diagram, annotated images, code, and a bill of materials. 

0

Physical Form

Our device has no physical form to emphasize the fact that it could literally be anyone or anything who is gossiping!

0
Miss Whispers: a secret gossiper at Margaret Morrison at CMU
Isha Hans - https://youtu.be/xUdzER6MsLA
0

There's been a lot of gossip flying around at Margaret Morrison, sometimes it's helpful sometimes hurtful, sometimes witty sometimes provoking conflict, sometimes funny and sometimes just non-sense. No one knows who knows, no one knows who's gossiping, but everyone is gossiped about, like my friend Hannah here. She was just walking to school when everyone just got a text about her new exciting job offer that she wasn't ready to share about yet...

0 
import os
import sys
import azure.cognitiveservices.speech as speechsdk
import requests
import json
import openai

#SETUP

#GPT 3 API key
openai.api_key = "APIKEY"
#initialize prompt
prompt = "Say this is a test."


#Azure API function speech-to-text
def recognize_from_microphone(prompt):
    speech_config = speechsdk.SpeechConfig(subscription="APIKEY", region="eastus")
    speech_config.speech_recognition_language="en-US"

    #To recognize speech from an audio file, use `filename` instead of `use_default_microphone`:
    #audio_config = speechsdk.audio.AudioConfig(filename="YourAudioFile.wav")
    audio_config = speechsdk.audio.AudioConfig(use_default_microphone=True)
    speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)

    print("Recording...")
    speech_recognition_result = speech_recognizer.recognize_once_async().get()

    if speech_recognition_result.reason == speechsdk.ResultReason.RecognizedSpeech:
        prompt = speech_recognition_result.text
        print("Recognized: {}".format(speech_recognition_result.text))
        return prompt
    elif speech_recognition_result.reason == speechsdk.ResultReason.NoMatch:
        print("No speech could be recognized: {}".format(speech_recognition_result.no_match_details))
    elif speech_recognition_result.reason == speechsdk.ResultReason.Canceled:
        cancellation_details = speech_recognition_result.cancellation_details
        print("Speech Recognition canceled: {}".format(cancellation_details.reason))
        if cancellation_details.reason == speechsdk.CancellationReason.Error:
            print("Error details: {}".format(cancellation_details.error_details))


#Store s2t response
speech = recognize_from_microphone(prompt)

#Request GPT3 with text from transcription
try:
  response = openai.Completion.create(
  engine="text-davinci-002",
  prompt=speech + "\n\nCreate gossip about this conversation starting with 'Hey Gossip Bot here. Overhead at Maggie-mo, one of my many sources sends us this:\n' and ending with 2 funny questions", 
  temperature=1,
  max_tokens=224,
  top_p=1,
  frequency_penalty=0.31,
  presence_penalty=0.15
  )
  answer = response
except requests.RequestException:
  print("ERROR: Not able to contact OpenAI API")

#parsing JSON from GPT3
gossip = answer['choices'][0]['text']

#setup variables 
payload = {'text': gossip,}
webhook_url = "https://hooks.slack.com/services/"

#slack post function
def post_gossip(payload, webhook_url):
   return requests.post(webhook_url, json.dumps(payload))

#call slack post function
post_gossip(payload, webhook_url)
Click to Expand
0

Reflection

  • This was definitely an ambitious project in terms of its technical implementation because neither of us had prerequisite, or even basic, knowledge of working with SDK packages. We managed a working prototype and were able to successfully demo the gossip bot in real-time but we recognize that this still needs a lot of refinement. We demo-ed the prototype from the laptop and the first refinement would be to integrate Raspberry Pi so that we can place this as a bot in any location for further experimentation.

0

Acknowledgements, Attributions and Credits

Acknowledge and attribute any sources or materials used in the production of this work (e.g. code snippets, tutorials, guides). Give credit where credit is due. 

0

References 

List any sources, references and material that others might need to consult to get a better understanding of this work (e.g. articles, readings, perspectives, etc.)


Precedents:

https://brianhouse.net/works/conversnitch/

x

Start the Conversation

Share this Project

Found In
Courses

48-528 Responsive Mobile Environments

· 5 members

As part of this project-based course, we’ll get hands-on with emerging technologies, concepts and applications in the internet of things through a critical socio-technical lens. Over it’s 15-weeks,...more

About

Make and Spread gossip