Detecting and Correcting User Activity Switches: Algorithms and Interfaces

Related comments:

Summary:
In this paper from Oregon State University’s School of EECS, authors Jianqing Shen, et al. present an updated version of an interface they had previously designed to detect and catalog switches in user activity, called TaskTracer.  TaskTracer applies machine learning methods to associate sets of resources with a particular user activity and make those resources more readily available to the user.  In this context, the term resources include documents, folders, email messages, contacts, and web pages.  TaskTracer configures the desktop in several ways to make these resources easy to get to: the task explorer presents a unified view of all resources associated with the current activity; the folder predictor modifies the Windows Open/Save dialogs by defaulting to folders associated with the current activity and adding shortcuts to them; the system has a time reporting feature that allows the user to show how much time was spent on each activity in a given period; finally, TaskTracer automatically tags incoming and outgoing emails associated with the activity.  The first version of TaskTracer had several problems including incorrect associations, unnecessary interruption of users with dialog boxes, and very slow learning algorithms.  TaskTracer2 fixes these issues by implementing an improved association engine, a desktop state estimator, a more intelligent switch detector, a notification controller that minimizes user interruption cost, and a clearer two-panel UI.

The researchers conducted a study on two people, a “power user” who recorded 4 months of data, and a second user who used the system for 6 days.  Overall, the participants found TaskTracer2 to rarely make an incorrect prediction, though it didn’t always make the exact correct one.

Discussion:
I think the user study for this application speaks the most clearly in regards to how I feel about TaskTracer: the “power user” found it very useful and accurate and was even described by the researchers as “fairly careful about declaring switches.”  I think associating resources with particular tasks is a great idea (like a more powerful, dynamic version of Windows’ “Recent Documents” feature), but having to explicitly declare when I’m switching activities would annoy me very quickly, especially if the system started pestering me about it with dialog boxes.  I think TaskTracer is best suited to the sort of conscientious power users that the researchers studied; the average user probably wouldn’t reap enough benefit for all the explicit task-switch declaration to be worth it.

Annotating Gigapixel Images

Related comments:


Summary:
Qing Luan, et al. present their developments in the annotation of very large (billions of pixels or gigapixel) images in this paper.  Their aim was to augment the pan-and-zoom interface used by applications like Google Earth and Virtual Earth with visual and audio annotations, driven by an intelligent rendering system that takes into account the viewer's perceptual distance from the objects being annotated.  They mentioned related work in areas like zoomable UIs, map labeling, human psychophysics, and augmented reality systems.  The system developed by the researchers runs on HD View and consists of text labels, audio loops, or narrative audio.

The system gauges viewer perspective, depth, and field of view relative to each annotation; it then assigns strengths to the various annotations based on these elements.  Thus, farther-off text labels will be smaller or farther-off audio annotations will be played at lower volume.


Discussion:
I haven't played with Google Earth very much, but according to the authors this system is a lot like it, so I think I have a pretty good idea of how the interface works.  I think it's a great idea to render annotations dynamically based on viewer position -- if all available annotations were rendered simultaneously on a map of the United States it would be completely unreadable.  Hiding or showing labels as the user pans and zooms encourages the user to explore the image to see what annotations can be uncovered.  This would be great if applied to an educational setting where children could browse a map of, say, Philadelphia and read excerpts from historical documents or hear audio clips about important locations.

Inky: A Sloppy Command Line for the Web with Rich Visual Feedback

Related comments:


Summary:
In this paper, Robert C. Miller, et al. (researchers at MIT CSAIL and the University of Southampton) present Inky (short for Internet Keywords), which is basically a command line interface that provides shortcuts to common web browser tasks.  It functions as a hybrid between a command line and GUI interface, giving dynamic visual feedback to the user as they type and utilizing sloppy syntax which frees the user from learning any new command syntax.  Keywords can be provided in any order, replaced with synonyms, or entered in a variety of ways; the Inky interpreter attempts to match the user's input against available commands.  Inky is a Firefox extension built using HTML, CSS, JavaScript, Java, and XML.  The various construction, command, and content-highlighting specifics are discussed in the rest of the paper, covering details about the keyword interpreter, functions and usage.

The authors conducted a small user study of seven participants and found that 95 out of 131 user commands were correctly interpreted, and that Inky was fairly easily learnable based on user response.  Most of the commands that were not correctly interpreted were simply attempts by users to invoke commands that did not exist (by entering website names, for example).  The researchers used their findings about Inky to begin development on a second prototype called Pinky (Personal Information Keywords) which focuses on lightweight data capture for personal information management.


Discussion:
My first thought about this application was that it seemed an awful lot like Quicksilver for Mac -- an application launcher/wizard's tool that interprets user input to execute commands and launch applications.  This paper even mentions Quicksilver, in fact, when talking about how Inky is invoked.  It also shares many similarities with Mozilla's own Ubiquity.  Like, a lot of similarities.  As in, these tools are nearly identical in function and aim.  Having used Quicksilver and Ubiquity pretty extensively, I think a command-line-style interface for quick command execution is a great idea that can really streamline a user's experience (if they are willing to take on a small learning curve).

Passages Through Time: Chronicling Users' Information Interaction History by Recording When and What They Read

Related comments:


Summary:
In this paper, Karl Gyllstrom of the University of North Carolina Computer Science Department outlines his research in building interaction history for users to improve information retrieval and characterize document activity.  His system Passages collects data from text-based desktop activities like web-page content, emails, and other files, as well as precise timing information about these items' visibility.  These data combine to provide a very detailed record of what content was viewed and when.  This record can then be used to answer user questions about their content such as, "when did I read this paper," or "which documents have I spent the most time composing?"


The system is comprised of two subsystems: the tracing subsystem for recording event streams in the GUI and filesystem, and the retrieval subsystem for handling artifact requests ("when was the last time I read this document?") and temporal retrieval ("what files did i work on the most during this time period?").  The author goes on to detail the algorithms behind these subsystems and compare it to existing approaches, which do not currently consider how long users view material and give as much weight to glanced-over pages as ones that have been thoroughly pored over.  The author's user study examined 15 participants' use of Passages over a total of 14.27 hours of activity; the results indicated that the system was well-suited to adding nuance and precision to document history requests.


Discussion:
For the super-organized, this probably a dream come true.  Being able to know which documents you were reading during a certain period of time or finding out how much time you spent reading a paper could potentially be very useful.  I found a couple of the author's assumptions about the algorithm to be somewhat ill-founded; for starters, it is based on whether content is visible as the active window.  I have a 27" monitor and frequently have multiple documents open simultaneously, with one or neither set as the active window.  I may be reading either of them but not having that time or content logged into the system because of the way Passages is set up.  This obviously isn't going to be a problem for everyone, but I feel like the productivity-hawks who would use something like this would probably have large or multiple displays, thus rendering the system less effective.