HCI with Documents

Summary

The authors of this paper wanted to make a new interface for working with documents on a computer that would allow the user to be immersed. They accomplished this by allowing the users to interact naturally with gestures and postures and created a program that also allows users to teach the gestures to be recognized.

According to the paper, users that encounter environments that resemble the real world can use natural capabilities to remember spatial layout and to navigate in 3D environments, which allows them to multitask. The program had multiple visualization methods for the documents. In PlaneMode, users can type multiple search queries into several panels. Documents that match more search queries are moved closer to the user. Important documents pulse to catch the user's attention and the colors the documents are represented by indicate the category.

ClusterMode: The most relevant documents are moved to the front and center of the plane. The documents are clustered with like colors from the search queries. In one variation, the clusters are rings where the documents rotation. Two possible ways to connect the rings are to connect rings with one colors to rings that also contain that color (ie blue only, to blue and red) or to connect clusters that have the same colors except one additional color by a line of that color.

Relations between documents can by having semi transparent green boxes around related documents.

To interact with documents, users will use a P5 data glove to perform gestures which have an associated action to them. Since the data could be noisy due to the P5 cheapness, there needs to be a filter to make sure the data is accurate. Gestures need to be held from 300 to 800 milliseconds for it to be recognized. There is also the gesture manager which keeps track of known postures and the ability to manipulate the database.

Discussion

I think this is a unique way to interact with computer documents. However, the gestures do seem intuitive, but I guess even if they're not, the users can change the gestures to something they feel more comfortable with. Nevertheless, I think the idea is a good way to improve the way we organize files on the computer.

-------------------------------------------------------------------------------------

Andreas Dengel, Stefan Agne, Bertin Klein, Achim Ebert, Matthias Deller. Human-Centered Interaction with Documents. HCM'06.