← Projects

MeBot: Your San Francisco Expert

Made by Angela Wang

Powered by DataSF Open API

Created: September 23rd, 2018

0
Talk to the MeBot about a movie or TV show you like, and it can tell you if the film was filmed in San Francisco! Or ask the MeBot about the movies that I like :). 

The MeBot is chat bot with which you can text to learn more about me and get movie fun facts about San Francisco – all by using San Francisco's open source API.

Previously practiced as a planner, this chat bot project is a fun marriage of my interest in civic engagement and interaction design. While this MeBot take on a more casual topic of city culture. It lays the ground work for more professional uses such as looking up ongoing development projects in cities or learning about the public space maintenance.


Youtube link to video (in case it doesn't show up below: https://youtu.be/wou1skeBnhQ)

0
0

Choosing Databases

When I started this project I knew I wanted to use government data – it's what I use a lot previously as a planner for technical purposes. I looked into San Francisco's data base and found a lot of them with open API! I eventually narrowed down to two very different types of data that I really liked, weighed the pros and cons, and chose the filming location data that may attract larger audience. 

0

Designing Conversation Structure

In choosing the database I actually sketched the conversation structure for both – it definitely helped me see how one is more challenging than the other. The final structure as shown below was very similar to the one I had when I started coding, but the coding process definitely also help clarify the routes.

0

Composing Speech with Available Data

The Film Location In San Francisco database (https://data.sfgov.org/Culture-and-Recreation/Film-Locations-in-San-Francisco/yitu-d5am) has a lot of information – from title, release, locations, fun facts, distributors, directors, writers, and actors (actor_1, actor_2, and actor_3). I started with wanting to cram in everything but quickly realized that it would make the speech bubble too long. 

I decided to only use 4 data points for my chatbot and quickly realized that I have to make quite a few pieces of speech to accommodate different data quantity and make the bot sound natural.

0

Constructing Code in Local Environment

DataSF's API is powered by SODA (https://github.com/socrata/soda-ruby), which is a gem that let you access the data simply with a weblink to the JSON file. The output of the data is stored as Hashie (https://github.com/intridea/hashie), which is a growing collection of tools that extend Hashes.

I spent the longest time trying to figure out how to get data from hashie. It was especially tricky because hashie doesn't store empty value, so the location of each item can be different depending on how complete each data point is. 

For instance, this is the output for the movie "A Smile Like Yours":

0

You'll see that it skipped the field actor_3 and fun_facts because there was no value for those fields. 

Eventually I figured out a way to simply add each unique value to the another array. (note: I'd add locations.uniq to delete duplicated values, but I know this dataset doesn't have any.)

0 
locations = []
fun_facts = []
 

output.each do |row|
      if row["locations"]
            locations.push(row["locations"])
      end
      if row["fun_facts"].to_s != ""
            fun_facts.push(row["fun_facts"])                    
      end 
end
Click to Expand
0


I coded everything locally and tested it in the local HTML as well as terminals (puts  "#"*90 FTW!)

0 
get "/apitest" do
    
    film = params[:film]
    output = client.get("https://data.sfgov.org/resource/wwmu-gmzc.json", {"$where" => "title like '#{film}%'"})
    
    
    # ---------- if the response does not match with any film title, ask to try again ----------
    if output == []
        message = "I don't recall and film or TV series that match what you said. Maybe try again? (Note: case and space sensitive)"
   
        
    # ---------- if the response match with any film, move forward ----------
    else
        titles = []
        release_year = output.first["release_year"]
        locations = []
        fun_facts = []
        director = output.first["director"]
        writer = output.first["writer"]
        actors_1 = output.first["actor_1"].to_s + output.first["actor_2"].to_s + output.first["actor_3"].to_s


        output.each do |row|
            if row["title"]
                titles.push(row["title"])
            end  
        end

        titles = titles.uniq

        # ---------- if the response matches with more then 2 film titles ----------
        if titles.length >=2

            message = "There are actually quite a few. It'd really help if you can be more specific! For example: 'Star Trek IV' or 'Change Seasion 1'."

        # ---------- if the response matches with 2 film titles, ask ----------    
        elsif titles.length == 2
            message = "Do you mean '" + titles[0].to_s + "' or '" + titles[1].to_s + "'?"

        # ---------- if the response matches with 1 film title, produce answer ----------    
        elsif titles.length == 1

            output.each do |row|
                if row["locations"]
                    locations.push(row["locations"])
                end
                if row["fun_facts"].to_s != ""
                    fun_facts.push(row["fun_facts"])                    
                end    
            end

            # ---------- edit message about location ----------

            locations = locations.uniq
            
            if locations.length >=3
                location_list = locations.sample(2)
                location_1 = location_list[0].to_s
                location_2 = location_list[1].to_s
                message_location = output.length.to_s + " locations, including " + location_1 + ", " + location_2 + ", and more places in SF."
            elsif locations.length == 2
                location_list = locations.sample(2)
                location_1 = location_list[0].to_s
                location_2 = location_list[1].to_s
                message_location = output.length.to_s + " locations, including " + location_1 + " and " + location_2 + " in SF."
            elsif locations.length == 1
                location_1 = locations[0].to_s
                message_location = "one location – " + location_1 + " in SF."
            elsif locations.nil?
                message_location = "one location in SF." 
            end
            
            # ---------- edit message about fun fact ----------
            
            #fun_facts = fun_facts.uniq
            
            if fun_facts == []
                message_fun_facts = ""
            else
                message_fun_facts = " Also fun fact: " + fun_facts.sample.to_s
            end
            
            
            # ---------- print answer about the movie ----------
            message = "Bingo! " + titles.to_s + " was released in " + release_year + " and featured " + message_location + message_fun_facts
            
            
            puts "#"*90
            shuffle = IO.readlines("angela_film_pick.txt").sample.to_s.chomp
            puts determine_message shuffle
            puts determine_congrats shuffle
            puts "#"*90
            
            
        end

    end
    
    
end
Click to Expand
0

While I was able to work around most response variations, there remain quite a few issues that I didn't have time to get to. They are listed below:

0

Adjusting Conversation in Context

Once I was confident with the code I pushed it to Heroku and started to test it on my phone. It allowed me to catch a few bugs and awkward dialogue flow. 

0

Cleaning Up the Code

My code would be working if I just copy-and-paste the local code. However, I wanted to consolidate them into methods so my code can be cleaner. Here are my final two method.

0 
def determine_message x
    
    client = SODA::Client.new({:domain => 'explore.data.gov', :app_token => '1ItWOJ1oGYyvN9Ajo5UvHCSaa'})
    output = client.get("https://data.sfgov.org/resource/wwmu-gmzc.json", {"$where" => "title like '#{x}%'"})
    
    
    # ---------- if the response does not match with any film title, ask to try again ----------
    if output == []
        "I don't recall and film or TV series that match what you said. Maybe try again? (Note: case and space sensitive)"
   
        
    # ---------- if the response match with any film, move forward ----------
    else
        titles = []
        release_year = output.first["release_year"]
        locations = []
        fun_facts = []
        director = output.first["director"]
        writer = output.first["writer"]
        actors_1 = output.first["actor_1"].to_s + output.first["actor_2"].to_s + output.first["actor_3"].to_s


        output.each do |row|
            if row["title"]
                titles.push(row["title"])
            end  
        end

        titles = titles.uniq

        # ---------- if the response matches with more then 2 film titles ----------
        if titles.length >=2

            "There are actually quite a few. It'd really help if you can be more specific! For example: 'Star Trek IV' or 'Change Seasion 1'."

        # ---------- if the response matches with 2 film titles, ask ----------    
        elsif titles.length == 2
            "Do you mean '" + titles[0].to_s + "' or '" + titles[1].to_s + "'?"

        # ---------- if the response matches with 1 film title, produce answer ----------    
        elsif titles.length == 1

            output.each do |row|
                if row["locations"]
                    locations.push(row["locations"])
                end
                if row["fun_facts"].to_s != ""
                    fun_facts.push(row["fun_facts"])                    
                end    
            end

            # ---------- edit message about location ----------

            locations = locations.uniq
            
            if locations.length >=3
                location_list = locations.sample(2)
                location_1 = location_list[0].to_s
                location_2 = location_list[1].to_s
                message_location = output.length.to_s + " locations, including " + location_1 + ", " + location_2 + ", and more places in SF."
            elsif locations.length == 2
                location_list = locations.sample(2)
                location_1 = location_list[0].to_s
                location_2 = location_list[1].to_s
                message_location = output.length.to_s + " locations, including " + location_1 + " and " + location_2 + " in SF."
            elsif locations.length == 1
                location_1 = locations[0].to_s
                message_location = "one location – " + location_1 + " in SF."
            elsif locations.nil?
                message_location = "one location in SF." 
            end
            
            # ---------- edit message about fun fact ----------
            
            #fun_facts = fun_facts.uniq
            
            if fun_facts == []
                message_fun_facts = ""
            else
                message_fun_facts = " Also fun fact: " + fun_facts.sample.to_s
            end
            
            
            # ---------- print answer about the movie ----------
            titles[0].to_s + " was released in " + release_year + " and featured " + message_location + message_fun_facts
            
            
        end

    end
    
    
end
Click to Expand
0 
def determine_congrats x
    
    client = SODA::Client.new({:domain => 'explore.data.gov', :app_token => '1ItWOJ1oGYyvN9Ajo5UvHCSaa'})
    output = client.get("https://data.sfgov.org/resource/wwmu-gmzc.json", {"$where" => "title like '#{x}%'"})
    
    if output == []
        ""
    else
        titles = []

        output.each do |row|
            if row["title"]
                titles.push(row["title"])
            end  
        end
        titles = titles.uniq
        if titles.length == 1
            "Bingo! "
        else
            ""
        end

    end
    
    
end
Click to Expand
0

Final Reflection

While we have been building the foundation for this chatbot in weekly exercises, it was REALLY CHALLENGING for me to finally getting it to work the way I wanted (with major room for improvement). Here are some of my final thoughts:

Things I was proud of: 

  • Learning how to use government open data
  • Designing conversation pieces that together sounds natural 
  • Solving some of the edge scenarios

Things I could do better

  • Incorporating natural language processes capability in my Chat Bot
  • Allowing for more back and forth in the conversation design


That's it :) it was a fun project and I learned a lot :)

x

Start the Conversation

Share this Project

Skills
Tools
About

Powered by DataSF Open API