What's Next?: Emergent Storytelling from Video Collections

Related comments: no other posts on this paper, comment on Kerry's blog.


Summary: 
Three researchers from MIT's Media Laboratory (Edward Yu-Te Shen, Henry Lieberman and Glorianna Davenport) developed Storied Navigation, a video editing system that lets users quickly search and browse a video library to compose a story.  The system works by annotating video clips with sequence information and story attributes; users can then search with words or sentences to find clips with relevant characters, themes, emotions and story elements.  In this way, users simply type in an emotion or story sequence they wish to convey and the program returns clips matching their criteria -- the user doesn't even have to be familiar with the video library to be able to tell a story.  Storied Navigation uses natural language  processing (NLP) to parse user input for story elements, characters, etc.  It features an "edit-by-typing" function that sequences video clips based on a story the user types in English, and an "edit-by-recommendation" feature that can either find similar alternatives to selected clips or provide clips that will continue the current story sequence ("what's next").


The researchers conducted two separate studies, the first of which focused on whether the system helped users to develop their story threads.  The subjects reported the system's ease of use, helpful features, powerful search, the improved efficiency with which they could sift through large amounts of video to find what they needed, ability to compose stories on-the-fly, and the way the interface helped them understand the logic behind a story.  The second study focused on browsing an unknown video library.  All seven subjects reported that the system helps them find what they want, mentioning that the edit-by-typing function was the most useful to them -- some of the subjects even wanted to create stories from the clips that were returned from their queries.


Discussion:
BRILLIANT.  As a video editor, I can tell you that there is nothing more painful than trying to sift through hours of documentary footage to find an appropriate clip -- this tool is literally the coolest thing a documentary filmmaker could ever want (aside from one of these, maybe).  There are times during editing when I even know what all of the footage is (in the event that it isn't dozens of hours of material), and just don't know how to move forward with it.  The Storied Navigation system answers my unspoken question ("what's next?") and even provides multiple ways of filtering the results it provides by matching emotions or placement in the story structure.  The ability to compose stories on-the-fly or quickly change a train-of-thought to another direction greatly streamlines the film editing process.  I could save so much time and tedious browsing by using this system; the "ways to think" that it provides are simply brilliant and ought to spur a storytelling sense in any filmmaker.