An Adaptive Audio Storytelling Agent

This project updates the length of a story according to the users wish. The end goal is to develop a storytelling agent that can adjust the telling of a story to passive user input, creating a better storytelling experience in the same way that a human storyteller would, by reading his/her audience.

First, various versions of the story are created - each containing different amounts of detail. They are organized as a folder of files with a particular naming convention.

A PHP script generates a JSON metadata file from this folder containing the mp3 files and their respective duration.

// 20151215164826
// http://smokeyhasa.com/projects/adaptivestory/stories/Happy%20Family/0story.json

{
  "name": "Happy Family",
  "story": [
    [
      {
        "mp3": "01a.mp3",
        "duration": 39
      },
      {
        "mp3": "01b.mp3",
        "duration": 28
      },
      {
        "mp3": "01c.mp3",
        "duration": 21
      }
    ],
    [
      {
        "mp3": "02a.mp3",
        "duration": 45
      },
      {
        "mp3": "02b.mp3",
        "duration": 27
      },
      {
        "mp3": "02c.mp3",
        "duration": 21
      }
    ],
    [
      {
        "mp3": "03a.mp3",
        "duration": 55
      },
      {
        "mp3": "03b.mp3",
        "duration": 46
      },
      {
        "mp3": "03c.mp3",
        "duration": 35
      }
    ],
    [
      {
        "mp3": "04a.mp3",
        "duration": 28
      },
      {
        "mp3": "04b.mp3",
        "duration": 9
      },
      {
        "mp3": "04c.mp3",
        "duration": 10
      }
    ],
    [
      {
        "mp3": "05a.mp3",
        "duration": 63
      },
      {
        "mp3": "05b.mp3",
        "duration": 53
      },
      {
        "mp3": "05c.mp3",
        "duration": 32
      }
    ],
    [
      {
        "mp3": "06a.mp3",
        "duration": 22
      },
      {
        "mp3": "06b.mp3",
        "duration": 10
      },
      {
        "mp3": "06c.mp3",
        "duration": 10
      }
    ],
    [
      {
        "mp3": "07a.mp3",
        "duration": 56
      },
      {
        "mp3": "07b.mp3",
        "duration": 44
      },
      {
        "mp3": "07c.mp3",
        "duration": 42
      }
    ],
    [
      {
        "mp3": "08a.mp3",
        "duration": 34
      },
      {
        "mp3": "08b.mp3",
        "duration": 28
      },
      {
        "mp3": "08c.mp3",
        "duration": 26
      }
    ],
    [
      {
        "mp3": "09a.mp3",
        "duration": 30
      },
      {
        "mp3": "09b.mp3",
        "duration": 19
      },
      {
        "mp3": "09c.mp3",
        "duration": 13
      }
    ],
    [
      {
        "mp3": "10a.mp3",
        "duration": 29
      },
      {
        "mp3": "10b.mp3",
        "duration": 22
      },
      {
        "mp3": "10c.mp3",
        "duration": 21
      }
    ],
    [
      {
        "mp3": "11a.mp3",
        "duration": 35
      },
      {
        "mp3": "11b.mp3",
        "duration": 35
      },
      {
        "mp3": "11c.mp3",
        "duration": 31
      }
    ]
  ]
}

Click to Expand

The playback engine - the user interface - is a single webpage powered by an html5 <audio> tag and JQuery. It looks at the metadata and searches through a variety of possible paths, finding paths that are closest to the users preference. The algorithm remains efficient by only searching a small subset of the domain of all possible paths, yet it re-checks the current path at every intersection in the story.