Ethnography / dancing ideas

Ethnography

I think that for an ethnography, they can study how people who have differing ideas from the majority in a forum (a liberal posting in a far right wing forum) are treated in their posts. For example, in a thread, see how quickly minority opinions are answered with hateful remarks. This is very similar to a project that Sarah and I did in the CHI class a few years ago.

Salsa Dancing

Other than having a similar idea to TIKL, where students learn the correct moves by having vibrotactile sensors vibrate when they move off beat or not closely match the teacher's movement, I don't really see how teaching students how to dance can be improved by a haptic device.

Xwand: UI for Intelligent Spaces

Comments

Franck

Manoj

Summary

The authors created the XWand wireless sensor package. It "enables styles of natural interaction with intelligent environments." If the user wants to control a device, they just point and perform simple gestures. THe system would rely on intelligence of the environment to determine the user's intention.

The device itself has the following

• 2-axis MEMS accelerometer
• 3-axis magnetoresistive permalloy magnetometer
• 1-axis piezoelectric gyroscope
• FM transceiver
• flash-programmable microcontroller
• Infra-red LED
• green and red visible LED
• pushbutton
• 4 AAA batteries

In the user study, pointing accuracy increased when audio feedback was added. From the study, users are comfortable with good tracking, but if there isn't good tracking, audio feedback would help.

Analysis

I don't know much about the hardware aspect of the device. However, it does seem intuitive. We like to point and control objects.

Webcam Mouse Using Face and Eye Tracking in Various Illumination Environments

Comments

Franck

Manoj

Summary

In this study, the authors present an illumination recognition technique that combines K-Nearest Neighbor classifier and adaptive skin model to provide real-time tracking. Their accuracy reached over 92% in different environments. The system tracks face and eyes features at 15 fps under standard notebook platforms. The system permits user to define and add their favorite environments to the nearest neighbor classifier, but it comes with 5 initial environments. They used the Webcam Mouse system on a laptop PC with a Logitech Webcam. The system takes up 45% of window resources.

Analysis

I think this requires more usability tests to see if their methods is robust enough for different environments. However, I did not think the KNN classifier was useful in this area.

Real-Time Hand Tracking as a User Input Device

Comments

Franck

Manoj

Summary

The authors sought to create an easy-to-use and inexpensive system that facilitates 3D articulated user-input using the hands. THe system optically tracks an ordinary cloth glove that has a specific pattern on it. This simplifies the pose estimation problem. First, they construct a database of synthetically generated hand poses. These images are normalized and downsampled to a small size. THey do the same for a queried image from the camera. They use it to look up the nearest neighbor pose in the database, by defining a distance metric between two tiny images. They chose a Hausdorff-like distance to do this.

Analysis

I agree this is a cost effective method of tracking the hand. I am uncertain about the way they choose to measure the distance metrics they use. Well, I don't know anything about them so I do not know the efficiencies of those methods.

That one there! Pointing to establish device identity

Comments

Franck

Manoj

Summary

The authors sought to identify devices using a pointing gesture involving custom tags and stylus called the gesturePen. They chose a two-way line-of-sight communications link to interact with devices in a dense computing environment. The pen had a irDA compliant infrared transceiver and developed tags that can be fixed to active and passive devices. The communication flow between the pen, a tag, and the associated device is:

• the user points the pen towards a tag and presses a button on the pen
• the tag receives the ping message, blinks the light, and sends its identity information to the pen.
• it receives the ID, validates it, and sends it to the attached device.
• Information is transferred over the network to the other device.

They tested their device with some user studies. First was a cognitive load task, where the users would play a jigsaw puzzle on an handheld computer using the gesturePen as a normal stylus. Then the participant would be interrupted to choose a tag by reading the IP address lab and choosing it from the list, or pointing towards the tag with the pen and clicking a button. The next study was a mobile environment task, where the participant was required to select the target device. Their method choices were the same as for the first test. For the most part, the gesturePen, was well suited to the dynamic ubiquitous computing environments. The users learned the system quickly and became comfortable using it fast.

Analysis

It seems like an innovative way of selecting a device. I'm surprised it is not that common now. The only problem I see is the limited range and maybe confusion if there are two devices close by. Anyways, it seems like you would need to be close for this device to be accurate.

Human/Robot Interfaces

Comments

Franck

Summary

The paper pretty much is about designing a system that can recognize gestures interactively and learn new gestures with only a few training examples.

Their approach is automated generation and iterative training of a set of Hidden Markov Models. These models are assumed to be discrete. For gesture recognition, the process involves reducing the data from the glove to a sequence of discrete observable symbols.

The procedure for interactive learning:

1. the user makes a series of gestures.
2. The system segments the data stream into separate gestures and tries to classify each one. If it is certain, then it would perform the associated gesture. If not, it would ask the user to confirm.
3. The system adds the encoded gesture to the example list.

They use the Baum-Welch algorithm to recognize the gestures and automatically update the HMM

They decided to focus on the gesture recognition and the way to generate new HMMs. They decided to solve the new HMM problem by beginning with one or a small number of examples. Then run Baum Welch until it converges, iteratively add more examples, and update the model with Baum Welch after each example.

To process the signal, they needed to represent the gesture as a sequence of discrete symbols. They decided to view the hand as a single dimensional sequence of symbols.They chose vector quantization as the algorithm to preprocess the data.

The vector quantizer encodes a vector by returning the index of a vector in the codebook that is closest to the vector. The preprocessor is coded as a data filter, so a symbol is sent to the recognition system as soon as enough data has been read to generate it.

After the data is preprocessed, it is evaluated by all HMMs for the recognition process, and is used to update the parameters of the proper HMM. They used 5-state Bakis HMMs, where a transition from a state can only go to that state or the 2 next states. They defined a confidence measure, where if the returned value is negative, the model is correct. if it is between -1 and 1, the classification maybe wrong. if it's less than -2.. then it is certain.

They tested on 14 letters of the alphabet in sign language. They didn't worry about measurements in 6D for the hand, so they chose the gestures that will not be confused with another one. The results show that their algorithm is reliable.

Discussion

I think their method is much better than the one I used in my project. However, I think they kind of took the easy way out in their testing by only the letters in the alphabet that would be easiest. I would have liked to see how it would work with all letters.

Real-Time RObust Body Part Tracking for Augmented Reality Interface

This paper seeks to provide an interface to track body parts without limiting the user's freedom. The system can recognize whether the user wears long sleeves or short sleeves. They use a calibrated camera to obtain images of hands, head, and feet. They transfer 2D detected body parts in an approximate 3D posture. Their algorithm is the following (pretty much copied here):

• obtain a foreground image by deleting the background and shadow of the user from the original image
• using the face texture, detect a face from the image.
• extract contour features
• tracks the users head using a particle filter
• detect and tracks the two hands by segmenting the skin blob image
• using the contour of the lower body, detect and track the feet and estimate the 3D body pose
• extract meaningful gestures with the position of the right hand
• visualizes all augmented objects and the user

They performed an experiment, evaluating 2D tracking performance with short and long sleeves, separately. They used an a BeNature system that recognizes simple gestures. They calculated the error when the user wears long sleeves is 5.48 pixels and 9.16 with short sleeves.

Analysis

I think this is quite unique method of tracking the human body, well, the hands, feet, and head. I like the robustness of the system in detecting whether the user is wearing long or short sleeves. However, I would like to see more user studies to see if this can be used for other purposes.

Liquids, Smoke, and Soap Bubbles - Reflections on Materials for Ephemeral User Interfaces

This paper looked at using bubbles as a form of human-computer interaction. They call this ephemeral user interfaces and their properties can be use for "novel playful and emotionally engaging interactions." The system included a round transparent tabletop surface with a diameter of about 20 inch and a thin layer of dark liquid on top. After blowing soap bubbles on the surface, the position and movement of the bubbles can be tracked by a camera. The bubbles leave a visible ring on the surface of the glass plate. A user can blow on the bubble or push it with the hand. They determine that this type of systems can be used in the home and entertainment. They imagine being able to use this in the form of "buttons on demand,"and ambient displays.

Analysis

I don't really see how this interface can be useful. It seems a bit too short term. Movement is a big hassle, and accuracy (blowing on the bubbles to move them) seems to be lacking. Also, the other method requires a bit of luck and finesse. I don't think this will be a good method of interaction

Recent developments and Applications of Haptic Devices

Summary

Haptic feedback is becoming more prevalent in society as technology. He starts out by describing some vocabulary. Force feedback links the user to the computer by applying forces on the user. Tactile feedback is sensed by human receptors lying near the surface of the skins. Haptic feedback now is widely used to include tactile and force feedback. Degrees of freedom refers to the number of rotation and translations the device utilized. Actuators allows the device to exert the force on the user. Then he mentions the different types of devices that exist.

• desktop devices
• tactile devices
• 2 and 3 degree of freedom desktop devices: includes many game accessories.
• 5-7 degree of freedom desktop devices: pen devices
• haptic feedback gloves
• arm exoskeleton haptic devices
• workbenches
• human-scale haptic devices
• motion platforms
• locomotion interfaces

Discussion

These are a lot of haptic feedback devices. I never knew i was using a haptic device until i read this list, mainly the 2 and 3 DOF devices.

Summary

The paper does 2 studies to analyse the effect of design choices on the kinds of interactions performed and the effect on learning opportunities. One of the studies was with an interactive tabletop. There was a LED illuminated glass surface. Many plastic objects were used as input devices. There is a marker on the underside of the object. It is tracked when the tagged object is placed on the table. Different digital effects are generated on the screen when different objects are recognized. The next study is with Wiimotes. The goal of this study is to explore the use of tangible
"extertion interfaces" to understand the concepts of motion and acceleration through body-based interaction. As the Wiimote moves around, a different effect is generated. Technically, they don't have to point the mote towards the screen.. However, to see the visual feedback, they need to look at it.

The results show how the different interactions with the system influenced the kinds of learning opportunities promoted. The location of the representations was found to have a direct impact on where the children's focus is located and how they are aware of other actions. Observing and directing fellow peers also contribute to learning.

Analysis

I like the idea of tangible interfaces. I think we have read a paper dealing with that earlier in the semester. However, I like to see more applications of tangible interfaces in learning.

Eyedraw: Enabling Children with Severe Motor Impairments to Draw with their Eyes

Summary

The authors wanted to help children with disabilities to draw. They knew that although they are physically handicapped, they can still move their eyes around. Through the use of eye tracking technology, they devised two version of EyeDraw and test both non-disabled and disabled people in both studies. The first version had a minimal set of features: tools for drawing lines and circles, an undo button, a grid of dots to help the user dwell at a chosen location, and a facility to save and retrieve drawings. Version 2 was implemented after feedback from the first test. They also added new features to it. Also, users can stop the group of dots, so they can look around without accidentally issuing commands. Although both groups enjoyed using both versions, the disabled group had more success using the second one.

Analysis

I feel this is an interesting way to help handicapped people draw. However, my experiences with the eye tracker indicates it would not be very useful. Of course, I do not know how precise the eye tracker in the labs are. Also, it seems that after a while, the users will get tired or get a headache.

Coming to Grips with the Objects We Grasp

Summary

The authors wanted to create a wrist worn sensor that can read nearby RFID tags and the wearer's gestures in order to identify the interaction. They used a Porcupine sensor, an accelerometer-based module that allows power-efficient capturing of inertial data and a real-time clock and calendar chip. To read the RFID tags, the M!-mini from SkyeTek was chosen. They performed the box test to evaluate wrist-worn RFID antennas, and found out that a reading rate of 1 Hz balances capturing tags and saving power. They tested the sensing in an hour long gardening session. They also tested to see how long the battery would last. After charging it overnight, the longest continuous log lasted 18 hours, and the battery was never depleted. They estimate being able to run this for at least 2 days continuous.

Commentary

Although this seems like an interesting way to detect objects and interactions, it still seems awkward to wear the wrist sensors. Also, this would work if everything had a RFID sensor, which I'm not too sure of.

---------------------------------------------------

E. Berlin, J. Liu, K. van Laerhoven, and B. Schiele. Coming to Grips with the Objects We Grasp: Detecting Interactions with Efficient Wrist-Worn Sensors. 2010

Natural Gesture/Speech HCI

Summary

The authors in this paper examined hand gestures made by a weather person narrating in front of a weather map. Since the gestures are embedded in the narration, they have plenty of data from an uncontrolled environment to study the interaction between speech and gesture.

They implemented a continuous HMM based gesture recognition framework. First they classified gestures as either pointing, area, or contour gestures. Then they chose a parameter space. Since it requires capturing the hand motions, they performed color segmentation on the stream of video input. Then, they determine distance between the face with the hands and the angle from the vertical. They used two multivariate Gaussians to model the output probability distribution at every state of each HMM. In the testing, continuous gesture recognition has lower recognition rate than isolated recognition.

They also did a co-occurrence analysis of different gestures with spoken keywords. With this, they were able to improve the continuous gesture recognition result based on the analysis of gestures with keywords.

-------------------------------------------------------

Discussion

I think this is a good study into how speech and gestures are related. It sounds promising, but the limited case studies they did could be expanded upon. I think that this is still a proof of concept kind of thing, so there was no real user study.

The Wiimote with Multiple Sensor Bars

Summary

The authors wanted to develop a tracked virtual reality controller that can be used in low-budget non-technical scenarios. They used 5 wireless sensor bars with 2 IR sources each, a Wiimote controller, and a Nunchuk. They placed the bars vertically along a 2 display VR environment. Some of the constraints include having 2 IR sources visible at all times, 2 sensor bars are visible when moving from one bar to another, and no more than 4 sources are visible at one time. For each iteration:

1. The software groups the tracked IR points to determine which pair is from the same bar. Then it creates a new dynamic model.
2. It determines if it confirms or contradict the existing model.
3. The software aligns the dynamic model with the static model using the previous alignment as a guide.
4. The software derives the scaling function using the IR points and their corresponding screen coordinates
5. It computes the cursor location.

They performed 3 tests. The first test was aimed a determining whether the visual disruption of the MSB array in front of the display would be distracting. The study consists of 12 participants and were given a navigation task of locating characters in response to audio cues and a manipulation task of stacking boxes on top of each other using the games physics engine. No one said the bars were distracting.

The second phase was aimed at establishing the usability of the complete MSB interface with a demanding FPS game.The authors wanted to see how often the game reset. 8 participants were in this study where they would complete two early levels from Half-Life 2. Some participants had trouble with vertical aiming and resetting.

The third phase involved moving the seat further back inside the coverage area. Some of the participants were from the second study and had high levels of reset errors. They were asked to replay the first level and a custom level where most of the enemies were at the horizon. The reset error rate dropped for all participants.

Discussion

I like this way of FPS interaction. The way they used the Wiimote was very intuitive, since it was like you're holding the gun. However, 2 handed guns may not feel right, and the other hand must be free for movement.

The PepperMill

Summary

The authors of this paper sought to create a device that can be powered through the physical effort required to operate it. Their system consists of a small DC motor as a rotary input sensor that can create a temporary 3.3V power supply. The first stage of the circuit determines the direction of the input. The second stage rectifies the output of the motor via a diode bridge. The 3rd stage uses a pair of resistors as a voltage divider, reducing the variable output voltage to a level that can be directly sampled by an analog-to-digital converter in a micro controller. The final stage uses a 3.3 V low drop-out regulator to stabilize the variable voltage to a level that is readily usable by a micro controller.

They created a prototype device and called it the peppermill. When the user turns the knob, the micro controller powers up and samples the inputs from the supply circuit and the states of the 3 buttons. It transmit this as a single wireless packet. They tested with a simple video browsing and playback application. It is used similar to a remote control. They found the users like using the Peppermill. Some users would turn the knob too slowly, but they instinctively knew to turn faster until it works.

Discussion

I think this is a good idea. There are times where physically charging up the battery would be more helpful than an actual battery. However, I doubt there would be much use for this, since it seems only to work for appliances that don't require large amounts of power. Also, I would have liked to see a better user study than the one provided.

------------------------------------------------------------------------------

Nicolas Villar and Steve Hodges. The Peppermill: A Human-Powered User Interface Device. TEI 2010.

User-Defined Gestures for Surface COmputing

Summary

The authors wanted to see how showing the non-technical users the effect of a gesture before asking the user to perform the gesture. They took 1080 gestures from 20 participants and paired them with 27 commands performed with 1 and 2 hands. They found out that the users do not really care about the number of fingers they employ, 1 hand is preferred to two, desktop idioms strongly influence users' mental models, and there is a need for on screen widgets, since some commands are hard to come up with a common gesture. They use a Microsoft Surface prototype with a C# application to present recorded animations and speech.

Discussion

I think this is an interesting method of interaction. Although I don't know much about the Microsoft Surface, this looks promising. Also, I wonder how this study would look if they used children or people from other cultures.

------------------------------------------------

Jacob O. Wobbrock, Meredith Ringel Morris, ANdrew D. Wilson. User-Defined Gestures for Surface Computing. CHI 2009.

Whack Gestures

Summary

The authors introduce Whack Gestures in order for people to interact with devices with minimal attention and without taking the device out. They introduced a small vocabulary of gestures intended to interact with a small mobile device. To counter the possibility of false positives, the user must use a pair of whacks to frame the gesture to be recognized. There were 3 different gestures: whack-whack, whack-whack-whack, and whack-wiggle-whack.

They used a Mobile Sensor Platform, which is small enough to be attached to the waist. They tested their system with 11 users. They wore it for 2 hours each and then performed the 3 gestures 3 times each. The results from the test was a 97% true positive rate.

Discussion

I think this is an interesting way to solve the problem of interacting with a device without taking it out. However, whacking the device may not be silent enough. I wonder if the device is sensitive to tapping. It seems that if the point of this was to interact silently, tapping would be more silent than whacking.

--------------------------

Scott E. Hudson, Chris Harrison, Beverly Harrison, Anthony LaMarca. Whack Gestures: Inexact and Inattentive Interaction with Mobile Devices. TEI 2010.

Gameplay Issues in the Design of 3D Gestures for Video Games

Summary

The authors sought to identify points to be considered in the design of 3D gestures in space as a means of interacting with video games. They tested on 4 game scenarios

• tilt: They used the game Neverball, where the goal is to send the ball to the exit by tilting a wireless controller.
• Alarm: using data from the accelerometer, the alarm demonstrator will emit a loud ringing sound should an acceleration threshold be exceeded.
• Heli: In a 2D helicopter game, the user must move the helicopter up and down to avoid boulders by shaking the controller.
• Battle of the Wizards: The user uses the wireless controller to gesture runes in the air for offensive and defensive spells.

The test involved 2 people: 1 male with lots of gaming experience and one female with limited experience. The results showed that users like games with simple gestures so they can play immediately instead of learning the gestures.

Discussion

I think this is a decent look into some user interaction issues. I would have preferred a much more in depth user study. But I agree that simple gestures is much better overall for games in the marketplace, because not everyone will be willing to learn complex gestures, and by including only those gestures, a large segment of the market would be alienated.

-----------------------------------------------------------------

John Payne, et all. Gameplay Issues in the Design of 3D Gestures for Video Games. CHI 2006.

3D Gesture Recognition for Game Play Input

The authors sought to use gestures to provide intuitive and natural input mechanics for games. They created a game called Wiizard, where the goal is to cause as much damage to an enemy while taking a little damage.. The users cast spells by performing gestures. The gestures are placed in a queue, which is released when the user cast the spell. They created a user interface with a bar revealing the state of all gestures available to the user, the playing field and the queue for each player. The software uses a Wii controller, the gesture recognition system and a graphical game implementation. Each gesture is a collection of observation, and they use the accelerometer data from the Wii controller. They created a separate HMM for each gesture to be recognized. THe probability of a gesture is the distribution of the observations and the hidden states.

The user study consisted of 7 users performing the images from the game over 40 times each. 90% recognition with 10 states, over 93% with 15.

Discussion

I think this is a unique method for game interaction, which is similar to how our project is going. However, 40 times per gesture seems a bit excessive. Also, they said that the user interface had both queues on it. That may be confusing. I wonder how this would work with fewer training data.

----------------------------------------------------------

Louis Kratz, Matthew Smith, Frank J. Lee. Wizards:3D Gesture Recognition for Game Play Input. FuturePlay 2007.

Hidden Markov Models

Summary

The authors focused on identifying the necessary elements to use an HMM system regardless of the sensor device being used. It would work as long as it has information about the 3 axis of motion. In a hidden Markov model, a sequence is modeled as an output generated by a stochastic process progressing through discrete time steps, where a symbol from the alphabet is outputted at each step. Only the sequence of emitted symbols is observed. HMM requires a 1:1 ratio of state to alphabet. They carved their space into subcubes, where they got alphabet sizes of 27, 64, and 125. 27 was the chosen size since recognition time decreased and were able to achieve recognition of 800 gestures in a second. 250 samples in a training set is good for detection results.

In testing, the user would press a button, perform the gesture, and let go. less than 27 was considered short and greater than 27 was considered long. They determined that left and right hand data sets were different enough to throw off the results.

Discussion

I still don't know much about HMM. It would take a while to fully understand it. That's the reason why I didn't use them in my robotics project.

------------------------------------------------------

Anthony Whitehead, Kaitlyn Fox. Device Agnostic 3D Gesture Recognition using Hidden Markov Models. GDC Canada 2009.

Gesture based control in Multi-Robot Systems

Summary

The authors design a way to use hand gestures to control a multi-robot system. Hidden Markov Models are used to recognize gestures from the CyberGlove. There were 6 gestures, opening, opened, closing, pointing, waving left and waving right. They added states for each of these, plus a wait state. They also use a gesture spotter which selects the gesture that corresponds to the last state with the highest score, or the wait state.

They ran some tests on the HMM. Using codewords with gestures and non-gestures, the HMM with the wait state recognized gestures with 96% accuracy and 1.6 per 1000 false positives.

To control the robots, there are 2 modes of interaction. Local robot control allows the user to control the robot from the robot's POV. If the user points forward, it moves forward, regardless of orientation. Global robot control allows the user to point at where he wants the robot to go.

Discussion

This work is similar to what I am doing for my robotics project. I am focusing on single robot control and the global robot control described here. I only use 1 nearest neighbor to recognize gestures. Anyways, this paper is very interesting. I would like to have expanded my work into using HMM and a 6D tracking system so I can get more accurate readings for my project. However, HMMs were too difficult to learn and the Flock did not work.

------------------------------------------------------------

Soshi Iba, J. Michael Vande Weghe, Christiaan J. J. Paredis, and Pradeep K. Khosla. An Architecture for Gesture-Based Control of Mobile Robots

HCI with Documents

Summary

The authors of this paper wanted to make a new interface for working with documents on a computer that would allow the user to be immersed. They accomplished this by allowing the users to interact naturally with gestures and postures and created a program that also allows users to teach the gestures to be recognized.

According to the paper, users that encounter environments that resemble the real world can use natural capabilities to remember spatial layout and to navigate in 3D environments, which allows them to multitask. The program had multiple visualization methods for the documents. In PlaneMode, users can type multiple search queries into several panels. Documents that match more search queries are moved closer to the user. Important documents pulse to catch the user's attention and the colors the documents are represented by indicate the category.

ClusterMode: The most relevant documents are moved to the front and center of the plane. The documents are clustered with like colors from the search queries. In one variation, the clusters are rings where the documents rotation. Two possible ways to connect the rings are to connect rings with one colors to rings that also contain that color (ie blue only, to blue and red) or to connect clusters that have the same colors except one additional color by a line of that color.

Relations between documents can by having semi transparent green boxes around related documents.

To interact with documents, users will use a P5 data glove to perform gestures which have an associated action to them. Since the data could be noisy due to the P5 cheapness, there needs to be a filter to make sure the data is accurate. Gestures need to be held from 300 to 800 milliseconds for it to be recognized. There is also the gesture manager which keeps track of known postures and the ability to manipulate the database.

Discussion

I think this is a unique way to interact with computer documents. However, the gestures do seem intuitive, but I guess even if they're not, the users can change the gestures to something they feel more comfortable with. Nevertheless, I think the idea is a good way to improve the way we organize files on the computer.

-------------------------------------------------------------------------------------

Andreas Dengel, Stefan Agne, Bertin Klein, Achim Ebert, Matthias Deller. Human-Centered Interaction with Documents. HCM'06.

Haptic Drum Kit

Comments

Franck Norman

Drew Logsdon

Summary

The authors sought to create a tool to help users develop multi-limb coordination. They also mentioned developing skills in recognizing, identifying, memorizing, retaining, analyzing, reproducing, and composing polyphonic rhythms. Their work builds on the theories of Dalcroze, a music educator, the entrainment theory of human rhythm perception and production, and research in embodied cognition, especially sensory motor contingency theory.

They created a Haptic Drum Kit, which contains a set of four vibrotactiles and elastic velcro bands, an Arduino electronic circuit board with a pouch belt, a midi drm kit, a computer running Max/Msp and software for audio and midi recording a playback, the Haptic Drum Kit program, and a stereo audio system for playback. They used a 12000 rpm (typo?) rotary-motor type vibrotactile device for the haptic signal.

Their user study consisted of 5 users, 4 novices and 1 experienced user. There was 20 reference rhythms from 4 broad categories

• metric rhythms, 8 and 16 beat
• figural rhythms, involving syncopation, based on the Cuban clave
• simple regular beats rendered figural by the way events are distributed across limbs, thus subtly varying tone color
• polyrhythms

The results from the experiment showed that users typically had a positive experience wearing the device. They all preferred haptic with the audio feedback. They do complain that the haptic feedback was soft and quiet, feedback was blurry for fast rhythms, and all the pulses were the same regardless of meaning.

Commentary

I think this would be a fun device to try. I am pretty sure that I would have trouble with this because my reflexes tend to be slow, but after a while, I can learn to coordinate myself better, which is the point of the device.

---------------------------------------------------

Simon Holland, Anders J. Bouwer, Mathew Dalgleish, and Topi M. Hurtig. Feeling the Beat Where it Counts: Fostering Multi-Limb Rhythm Skills with the Haptic Drum Kit. TEI 2010.

Office Activity Recognition using Hand Posture Cues

Comments

Manoj

Drew

Summary

This paper focused primarily on determining if hand postures can be used to help determine the objects a user interacts with. Another goal was to determine how different users have different hand gestures for the same interactions. They used the CyberGlove device with sampling at 10 readings a second. 8 users participated in the experiment performing 12 interactions at 5 times each. They averaged the values of each of the 22 sensors for each interaction to input into the classifier. They decided on the 1 nearest neighbor algorithm for the classifier. In the user-independent system, they performed the leave-on-out cross-validation across all users. The average accuracy was 62.5%, ranging from 41.7% to 81.7%. In the user-dependent system, they trained the classifier using only one test user. They first chose one random example of each interaction to train the classifier and then ran the classifier on the remaining four examples in testing. Next, they did 2 examples to train. Average accuracy was 78.9% for one training example to 94.2% for 4 training examples. They determined that the user-dependent system was better for recognizing user interactions in a natural, unconstrained manner.

--------------------------------------------------------

Commentary

I think this paper proves their goals of determining whether or not hand postures can determine an interaction and seeing the variability in hand postures for the same interaction across different users. However, since the experiment used the CyberGlove, I don't see how this could be useful in practice, since I don't think all office workers would agree to being required to wear a glove. I think for most practical purposes (like security mentioned in the related works), a vision based system is more helpful.

--------------------------------------------------

Brandon Paulson, Tracy Hammond. Office Activity Recognition using Hand Posture Cues. The British Computer Society 2007.

$3 Gesture Recognizer

Summary

Since the assigned paper was more of a brief version of their longer paper, I'll summarize the longer paper. Their work was based on the $1 algorithm by Wobbrock. They extended the work to include 3D acceleration data. There are no exact positioning since acceleration data is clouded by noise and drift error. Their algorithm does not require library support and needs only minimal parameter adjustment and training, and provides a good recognition rate.

First, they determine the change in acceleration by obtaining an acceleration delta. The summations of the deltas would give the gesture trace in 3D or projected into 2D. To match the gesture class, they compare the trace at point i of the input to all the traces of all training gestures in the library and generate a score table comparing the two. For resampling, they settled for 150 points. They also rotate along the indicative angle and scaled to fit in a normalized cube of 100^3 units to compensate for scaling differences.

The scoring heuristic reduces the number of false positives. They determine a threshold score. If the highest score is higher than 1.1 times this threshold, they return the gesture ID. If 2 out of the top 3 are of the same gesture class and scores higher than .95 times the threshold, return the ID of the gestures.

They evaluated the algorithm on twelve participants, making 10 unique gesture classes. Each class was entered 15 times on the wiimote. The recognition algorithm took the first 5 as training sets, and then compared the remaining ten. The recognition rate was between 58% and 98% with an average of 80%. The scoring heuristic worked since only 8% of all detected gestures were false positives.

Some of the limitations of this algorithm was that only explicitly started and ended were recognized. Also, the size of the library is a limiting factor since the computational overhead would start growing as the library gets larger.

----------------------------------

Commentary

I think this is a good followup on the $1 algorithm. It could use some improvement since 80% seems a bit low. But since this is 3D recognition, there may be additional problems involved. Overall, I think this is useful.

----------------------------------------

Sven Kratz, Michael Rohs. A $3 Gesture Recognizer - Simple Gesture Recognition for Devices Equipped with 3D Acceleration Sensors. IUI 2010

$1 Recognizer

Summary

This paper seeks to create a gesture recognizer that would allow novice programmers to incorporate gestures into their UI. The $1 algorithm is easy, cheap, and usable anywhere. It involves only basic geometry and trigonometry and requires about 100 lines of code. It supports configurable rotation, scale, and position invariance, does not require feature selection or training examples, is resilient to variations in input sampling, and supports high recognition rates.

The algorithm has 4 steps.

1. Resample the Point Path. It resamples the gestures by splitting the path into N equidistant points.
2. Rotate Once Based on the Indicative Angle. The indicative angle is the angle formed between the centroid of the gesture and the gesture's first point. The gesture is rotated so that this angle is 0 degrees.
3. Scale and translate. The gesture is scaled to a reference square and then translated to a reference point (the origin of the frame)
4. Find the optimal Angle for the Best Score. A candidate is compared to each stored template to find the average distance between corresponding points.

The recognizer cannot distinguish gestures that depends on orientations, aspect ratios, or locations. Horizontal and vertical lines are abused by non-uniform scaling. Also, the recognizer does not distinguish based on time.

The user study consisted of 10 subjects using a Pocket PC with a stylus. They were given a series of gestures to do at slow, medium, and fast speeds. They compared the recognizer with Rubine and Dynamic Time Warping. $1 had a 99.02% accuracy. The number of templates affected the recognition error rate. $1 improved as more templates were added, with an error rate from 2.73% at 1 template to 0.45% at 9 templates. Slow and fast gestures had higher errors than medium. $1 took 1.6 minutes to run 14400 tests for 160 gestures.

---------------------------------------

Commentary

I like this algorithm. It is easy and fast. However, there are some drawbacks (ie the limitations listed). I think that there has to be some drawbacks for a "simple" algorithm. In order to simplify things, you have to leave some things out, otherwise it would be too complicated. I liked the paper since it went into the quantitative and qualitative aspects of the experiment.

--------------------------------------------------------

Jacob O. Wobbrock, Andrew D. WIlson, Yang Li. Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. UIST 2007.

American Sign Language Recognition in Game Development for Deaf Children

Summary

The authors of this paper created CopyCat, an American Sign Language game, which uses gesture recognition to help young deaf children learn sign language. One of the main problems in this area is continuous, user independent sign language recognition in classroom settings. They implemented a Wizard of Oz version of CopyCat and collected data from deaf children who used the system. They attempted to overcome the problems in continuous signing, such as clothing and skin tone differences and changes in illumination in the classroom. The dataset consisted of 541 phrase samples and 1959 individual sign samples.

Their solution used color histogram adaptation for hand segmentation and tracking. The children would wear colored gloves with wireless accelerometers. Both data is used to train hidden Markov models for recognition. The game is interactive. It has tutorial videos to demonstrate the correct signs, live video, and an animated character mimicking what the child is doing. They evaluated the result using the leave-one-out technique, by iterating through each child and removing him from the data set, training the data on the four other children, and testing it on the remaining child's data. They achieved between 73 to 92% accuracy.

Commentary

I think this is a great application for the research area. It is helpful to teach deaf children how to do sign language, especially if their parents aren't fluent in it as well. I think future work will improve on the interface and allow the programmers to get away from the Wizard. Also, the dataset was filtered out to include only the clear signs. In everyday use, that would not be possible. Also, the software needs to deal with individuality in signs. Not everyone would do the signs the exact same way. There are always variations. Overall, though, I enjoyed this paper. A very useful application for HCI.

---------------------------------------------

Helene Brashear, Valerie Henderson, Kwang-Hyn Park, Harley Hamilton, Seungyon Lee, Thad Starner. American Sign Language Recognition in Game Development for Deaf Children. ASSET 2006.

An Empirical Evaluation of Touch and Tangible Interfaces for Tabletop Displays

Comments

Franck Norman

Murat

Summary

The authors wanted to conduct a performance study on tangible interfaces and determine if there is an actual improvement in tangible interfaces compared to other ones. They compared speed and error rates for a touch interface and a tangible interface. They built a top projection tabletop system that can support both a touch and tangible interface. There is a camera mounted on top of the table to detect tagged objects placed on the table using ARTag. The table can also track multiple fingers. The experiment used a shelf and a wall. The touch interface had a toolbar that contains items to drag and drop into the work area. The tangible interface had real tangible objects. There were several interaction methods provided by the interface: addition, lasso selection, translation, etc.. The user study took 40 students and each had to implement a series of 40 layouts using both interfaces. The experiment showed that the users were faster with the tangible interface. They also concluded that manipulating the tangible shelves was much easier compared to the tangible walls. The user preference section indicated that the tangible interface was easy to use, but had more "fun" with the touch interface. Also, users stated that they were more stressed and irritated by the touch interface.

-----------------------------------

Discussion

I think that this is a pretty good intro study into tangible interfaces. It is intuitive and is faster as the result shows. Speaking of results, this paper was also good since it went into depth about the results from the experiment. There were quantitative results (the error and completion time) and qualitative results (fatigue, easy to use, fun).

---------------------------------------

Aurelien Lucchi, Patrick Jermann, Guillaume Zufferey, Pierre Dillenbourg. An Empirical Evaluation of Touch and Tangible Interfaces for Tabletop Displays. TEI 2010.

Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound

Comments

Franck Norman

Drew

Summary

The authors sought to create a new method of interacting with 3D objects with tactile feedback. Although there has been previous methods implemented, one method, the Cyber Glove, is not optimal because it provides tactile feedback at all times from the glove touching the skin. They proposed using ultrasound as the way to provide tactile feedback. Their method is based on acoustic radiation pressure. When the airborne ultrasound is applied on the surface of the skin, almost 99% of the incident acoustic energy is reflected on the skin. This removes the need to place an ultrasound reflective medium on the skin. Their prototype device consists of an annular array of airborne ultrasound trandsucers, a 12 channels amplifier circuit, and a PC. It was designed to produce a single focal point along the center axis perpendicular to the radiation surface.

They measured the total force using an electronic balance. The measured force was 0.8 gf (gram-force) and 2.9 gf at 250 mm and 0 mm respectively. To measure the spatial resolution, a microphone probe was attached to an XYZ stage. They measured that the diameter of the focal point was about 20 mm in diameter, and the maximum intensity decreases as the distance of the focal point from the array increases. When they measured the temporal properties of the radiation pressure, they noticed that the radation pressure decreases at the onset of radiation pressure each half period.

They did a user study and said that they felt vibrations when the radiation pressure was modulated. If it was constant, they can only feel on and off pressure.

----------------------------------

Discussion

I feel like that this experiment is not going to be very useful in future applications. Although the idea is innovative, using ultrasound doesn't seem to be very useful in current interactions. There would also need to be a way to detect the position of the hand without the use of sensors on the hand itself. Also, within the paper, I would've liked to have seen more of a user study. It gives no statistics of what the study was. I assume that the participants just put their hand over the array and felt something.

-----------------------------

Takayuki Iwamoto, Mari Tatezono, Hiroyuki Shinoda. Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound. EuroHaptics 2008. pp. 504-513

FreeDrawer - A Free-Form Sketching System on the Responsive Workbench

Comments

Franck Norman

Manoj

Summary

The authors of this paper sought to use 3D tools for curve drawing and deformation techniques for curves and surfaces. In their setup, the user draws in a virtual environment, using a tracked stylus as an input device. At a workshop about the needs of designers in virtual environments, the modeler should follow certain guidelines

1. be useful as a combined tool for the conceptual phase up to a certain degree of elaboration
2. hide the mathematical complexity of object representations
3. direct and real time interaction
4. full scale modeling, large working volume
5. be intuitive, easy to learn

Their modeler requires the designer to have some drawing skills, which is not too different from the traditional method. They support direct drawing of space curves and 2D curves, projected onto a virtual plane. The features included in the modeler also includes creating curve networks, changing a curve in a network, filling in the surface, smoothing, sharpening, and dragging curves, sculpting the surface, and creating surface patches. Their interface uses a hand held 3D widget, consisting of a set of virtual pointers starting at the stylus. The pointers will have a specific function like copy, move, delete, smooth, and sharpen. The user will touch the object with the tip of the corresponding pointer and pressing the stylus button afterward. Their user study consisted of one user who had experience with the program before. The user drew a seat.

Discussion

I think this application is okay for the specific area they were going for: the designers. It is understandable that one of the constraints of using the product well was having drawing skills. If this product is catered to the product designers, then those people should already have this skill. I think that the user interface could be better. From what I understand, the user moves the stylus and must touch the proper pointer on the curve. That seems quite difficult especially if there are a lot of curves in the area. Also, they only tested the product on one user. I would like to see some stats on multiple users to see how different designers would do the same task.

______________________________

Source: Gerold Wesche and Hans-Peter Seidel. FreeDrawer - A Free-Form Sketching System on the Responsive Workbench. VRST 2001.

Computer Vision Based Gesture Recognition for an Augmented Reality Interface

Comments

Franck Norman

Drew

Summary

In this paper, they try to create a vision based gesture interface for an Augmented Reality system. It can recognize a 3D pointing gesture, a click gesture and 5 static gestures.

To define the gestures, they just use a closed fist, and various numbers of fingers open. They asked the users to perform the gestures in the same plane, which reduced the recognition problem to a 2D issue. The recognition method relies on a pre-segmented image of the hand. They use a color pixel-based approach to account for varying size and forms from image to image. After segmenting the hand pixes from the image, the next step is to detect the number of fingers that are outstretched. It does this by measuring the smallest and largest radii from the center of the palm where there is a non finger pixel. After this, they would recognize the point if there is only one finger detected and click was done by the thumb.

The user study was done by several users. They just said that the users adapted quickly.

------------------------------------

This paper wasn't that good because it didn't go into more detail about the results. We don't know what they meant by several, and they didn't give any quantitative analysis of the results or mention what kind of tests the users went through. Overall, it was a good idea. The results were lacking.

Eyepoint: Practical Pointing and Selection Using Gaze and Keyboard

Comments

Drew

Manoj

Summary

The authors of this paper seek to create an alternative to mouse and keyboard based interaction. It uses eye gaze tracking technology instead of the mouse.

They did an inquiry into how able bodied users use the mouse. They discovered several things in common when using the mouse

1. use the mouse to click on links on a webpage
2. launching applications from the desktop or start menu
3. navigating through folders
4. minimizing, maximizing, and closing applications
5. moving windows
6. positioning the cursor when editing text
7. opening context sensitive menus
8. hovering over buttons/regions to activate tooltips

They determined that any good gaze based pointing techniques must have single click, double click, right click, mouse-over, and click-and-drag capabilities.

EyePoint uses a two-step progressive refinement process stitched together in a "look-press-look-release" action. It requires a one time calibration. The user will look at the desired point on the screen and press a hotkey for the desired action. EyePoint will then zoom in and the user will look at the target again and release the hotkey.

They tested EyePoint on 20 subjects that were experienced computer users. 6 of the users needed vision correction either through glasses or contact lens. They evaluated 3 variants of EyePoint: with focus points, with gaze marker, and without focus points. The first study was to click on a hyperlink highlighted in orange on a webpage. Next, the test subjects had to click on a red balloon, which moves everytime it was clicked. Lastly, they would have to click a target and then type a word.

In the first study, users took 300 milliseconds longer using EyePoint than the mouse (1915 to 1576). EyePoint had an error rate of 13% vs the mouse's 3%. In the second study, EyePoint was about 100 ms slower than the mouse. In the last study, EyePoint was faster than the mouse. Overall, it was determined that EyePoint might have better performance, it was more prone to errors. Also, users were split on whether the mouse or EyePoint were easier to use or faster.

-----------------

I think this is a good technique to use for people who have trouble using the mouse due to some physical injury (ie broken hand). EyePoint also seems to use a much more accurate eye tracking software than the other eye tracking papers we've read. Also, I like the tie in with fitz's law and the book Emotional Design which I have read before.

However, one area of improvement would be to allow the program to be more usable when dealing with glasses, especially the narrow frames which had caused problems

Motion Editing with Data Glove

Commented on:

Drew Logsdon

Franck Norman

---------------------------------------

Summary

In this paper, the authors develop a new method to edit captured motion data by using a data glove. They use the human fingers to simulate the motion of the legs of a human figure. In the case of walking, the animator would map the motion of the fingers to walking. After this, he could generate running by moving the fingers faster.

They used the P5 glove which can detect the position and orientation of the wrist. They divide the procedure to edit motion data into the capturing stage and the reproduction stage. In the capturing stage, they generate the mapping function that define the relationship between the motion of the fingers. The algorithm gathers parameters, such as the cycle of the motion, the minimum and maximum output value, duration of the motion and range. After this is completed, there is a reproduction stage, where the animator performs a new motion with the hand. It must be a similar but different motion.

Since the body can move in more directions than the fingers, they must determine the proper animated motion from the finger motions. They fixed the matching in advanced. For example, when the person walks, they move right leg forward and left arm back. They matched this motion by seting the middle finger to the left shoulder and index finger to the right shoulder.

They tested this by having an animator go through the process. First, the normal walking motion was mapped. Then, they did a hopping motion and walk in a zigzag path. It resulted in an unnatural motion.They found out many limitations of the mapping function, especially the requirement that the new motion must be similar to the old one.

---------------------------------------------------------------

I think it is a good preliminary study into the possibility of motion editing. However, I think the project would have benefited from a better glove. Also, the mapping function relies too much on the finger movements. There are a lot of factors involved in our walking motion, so they need to account for this in the simulation. A lot of these factors cannot be measured from finger motions alone.

lab days

Here are my thoughts for the lab days

cybertouch gloves

I liked the gloves because they were much easier to use than the head mounted goggles and the eye tracker. It was rather comfortable although the vibrating would get tiresome after a while. I followed up on Franck's code to try to have a button detect whether or not the hand was open or closed. When we clicked on the button, the code updates the data received from the sensors for the index finger. We determined that was good enough for a preliminary attempt. We ran the code a few times to determine the dividing value to separate open and close after adding the 2 sensor values for the index finger together. After determining this value, we added a command to start vibrating when the glove is closed, and stop when the glove is open. If we had more time, we would not require having to press the button every time to update the sensor values.

head mounted display

This was the first device I actually used. Josh had us do a user study while using the device. After turn it on, our first task was to do a basic and advanced walking test. I nearly bumped into the whiteboard on the basic test. Next, we had to stack the books on a table in a specific order while sitting and then while standing up. Then, we had to read and write on the board. Finally, we measured how wide and close our periphery vision was limited by measuring how close our hand was before it comes into view.

I thought that using this device was ok, but prolonged usage would give me a headache. It took me a while to get accustomed to the goggles because it seemed like everything was zoomed in . I couldn't see the periphery very well, and I don't want to do any difficult movements while wearing them.

eye tracker

When I used the eye tracker, I just had to figure out how to put it on, and calibrate it. I think the tracker had trouble tracking my eyes because of my glasses. Also, I don't think I calibrated it correctly because the cursor did not "follow" my eyes. I do not know much about the eyes, but the cursor did twitch a lot and quickly moved to one edge of the screen. I also couldn't "double click" by blinking very well either. I got a headache trying to use this device, so I probably should limit using this device.

TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning

Commented on the following blogs
Franck Norman
Drew Logsdon


The authors research ways to use real time tactile feedback, through a wearable robotic system. This way would be used along with verbal and visual feedback from the teacher. In a nutshell, the teacher and student will both be wearing a robotic feedback suit. A motion capture system will track movement for both. The software will see the deviation between the teacher and student and give the student feedback through the actuators placed on the skin. The joints that move in error will receive some feedback through the system proportional to the amount of error. The software has some equation it uses to determine that amount of error.

The experiment was run on 40 individuals divided into 2 groups: 1 that receives visual feedback, the other receives visual and tactile feedback. Both groups wear the suit to make sure the movements would be about the same. After 10 minutes of calibration, the user is shown a series of still images and told to mimic them. After that, 3-10 second motion videos are shown and they were told to mimic it. To study learning over time, the videos were repeated 6 times. Afterwards, they were given a questionnaire to fill out.

The questionnaire revealed that most users felt reasonably comfortable in the suit, but the tactile group needed to concentrate more. Over time, the tactile group stated that their ability to use the feedback improved. Some issues included discomfort in the sitting and elbow positioning. Most agreed that this was a good way to teach motor skills. The authors went on to a more math based analysis of their results.

----------------------------------

I think this is one of the best papers we've read to date, since they did a good analysis of the experiment, results, and future work. They admit certain pitfalls with their project, like that the setup would most likely be too expensive for every day use. However, they used this to prove their assertion that vibrotactile learning is much better than just visual or audio learning alone. Also, they did a good job of describing the aspects of their experiment and they split the groups into a test and control group. Overall, I think this is a good start to the line of research. I think it would be useful to reteach motor skills to people that just came off of injuries or lost the ability to do certain motor skills (i'm sure some medical conditions can cause this).

3DM: A Three Dimensional Modeler Using a Head_mounted Display

Blogs I commented on
Franck Norman
Drew Logsdon

The author's goal was to design a 3D modelling program that uses the same techniques from other programs, but presents it in a more intuitive manner for beginning users to use. The program uses a head mounted display, which places the user "in the modelling space" Placing an object in 3D requires 6 parameters. However, using a 2D mouse and a keyboard makes the spatial relationships unclear.

3DM uses a VPL eyephone to display the image and trackers to track the head and hand. The input device was a 6D 2 button mouse from UNC-CH. Image rendering was done by Pixel-Planes 4 and 5 high performance graphics engines. The user interface has a cursor and a toolbox. Some icons are tools, where the cursor will change based on what tool was selected and commands perform a single task. Toggles can change the global setting for 3DM. There is continuous feedback for the user through predictive highlighting.

3DM has multiple methods of creating surfaces, triangle creation tools and the extrusion tool, which draws a poly line or takes an existing one and stretch it out. There is another tool which allows for the creation of standard shapes, like boxes, spheres, and cylinders. Also, there are methods to edit the surface by grasping and moving the object, scaling, cut and paste, and there is an undo/redo button as well. The grouping feature allows the user to change one copy and have that change go to every other copy.

The results show that organic shapes are easily created in 3DM. Users feel in control because they can grasp something to change it. However, there are some weaknesses like keeping two shapes parallel.

------------------

I think this paper was good for something written in 1992. I liked the way they allowed the user to get some feedback of their actions. However, their results was lacking the quantitative aspect. They didn't give any stats from the user study, like who was tested, and what they were told to do. They gave general analysis of their results. Nevertheless, I liked the research and I wonder how this would be improved with current technology. and i'm sure that this paper was referenced by another paper we read earlier.

Wearable EOG Goggles: Eye-Based Interaction in Everyday Environments

Commented on the following blogs
Franck Norman
Drew Logsdon

This paper presented an embedded eye tracker for context-awareness and eye-based human computer interaction. The author designed a goggle with dry electrodes integrated into the frame and a small microcontroller for signal processing. Eye gazing is a good way to express intention and attention covertly, which makes it a good input mode. However, dwelling is still used for confirmation. The authors used EOG as a substitute for other methods. It is easily implemented in a lightweight system.

The goggles were designed to be wearable and lightweight, require low power, provide adaptive real-time signal processing capabilities to allow for context-aware interaction, and to compensate for EOG signal artefacts. The system would detect and ignore blinking, detect the movement of the eyes, and be able to map the eye movement into basic directions. There would be a string representation of the eye movement, where certain combinations would be recognized as a gesture.

The trial consisted of a computer game with 8 levels where they had to perform a specific gesture. High scores are given for those who complete it in short times.They found the EOG signals can be efficiently processed to recognize eye gestures. However, 30% of the subjects had trouble focusing.

Commentary

I think that relative eye tracking is probably much better than exact eye tracking for future methods to use mouse free interaction. The user can glance in the general direction of where they want the mouse to go.

The results were relatively lacking. There were no statistics. 30% was given for the number of people who had trouble concentrating, but how many people were tested? What was the average time for each level? I would like to see where this research goes and see if there is a much more thorough testing of the system.

HoloSketch: A Virtual Reality Sketching / Animation Tool

Commented on the following blogs
Drew Logsdon
Franck Norman

Summary

HoloSketch is a tool, designed for nonprogrammers to create and manipulate images in 3D. It uses a 20 inch stereo CRT with 112.9 Hz refresh rate. A new viewing matrix is calculated separately for each eye. There is a 3D mouse with a digitizer rod, which is used to control many of the different functions in HoloSketch. The menu design in HoloSketch is engaged by pressing and holding the right wand button. The menu would then fade in. The user would select an item by poking the button and when the user releases the right wand button, the item would be selected. There are many features in HoloSketch that I would not discuss here.

According to the results, it was very easy for first time users to create complex 3D images. Most users however keep their head stationary so they don't look around the object they're creating. They also got a real artist to try to use it for a month. The artist started cold and did not get any documents to help her, but within a short time, complex objects were created with ease.

HoloSketch was designed to be a general purpose 3D sketching and simple animation system.

Commentary

This is a very interesting paper, considering it was made in 1995. I did not even think that 3D virtual imaging was around back then. I still wonder about some things in the paper. The author stated that CPU instructions that could be executed per graphics primitive rendered is steadily going down. I don't quite understand what this means. Does it mean that the CPU is being used to render graphics so it can't do something else? Also, the author did not give numbers on the results, like how many novices were in the trial, and the average time before completing a task. Also, did they give the user an instruction on what to create and they didn't define "simple creatures"

Noise Tolerant Selection by Gaze-Controlled Pan and Zoom in 3D

This paper dealt with using eye tracking technology to try to type something without use of a keyboard. Other methods in the past used dwell time, but that was deemed to be wasteful of time.

They used StarGazer to do the panning and zooming. There is a circular keyboard for the typing. The users can zoom in on specific areas of the keyboard and type using only their eyes.

They ran some test to see if it was intuitive and easy for novice users, and to see if size of display and noise would be a factor. Noise slowed down words per minute and smaller sizes slowed it down as well. Of interest, the accuracy was not increased with noise. This showed that StarGazer was tolerant of noise

Commentary

This would be interesting to see how people can type without using a keyboard. At first, I thought this could be like the Macbook Wheel from the Onion a while ago. But this seems workable. Of course, more work needs to be done for people that have bad eyesight (but are not blind). Also, it seems prolonged usage could hurt the eyes.

Distant Freehand Pointing and Clicking on Very Large, High Resolution Displays

This paper deals with trying to using hand motions to interact with a large screen from a long or short distance (up to touching the screen). They determined 5 characteristics of a device to be used with the display
1. accuracy
2. acquisition speed
3. pointing and selection speed
4. comfortable use
5. smooth transition between interaction distances

Then, they discussed previous work with handheld indirect pointers, laser pointers, eye, hand, and body tracking device. They discussed the problems faced by each of these methods, so they implemented their own clicking and pointing techniques.

Clicking
1. AirTap
2. ThumbTrigger

Pointing
1. RayCasting
2. Relative Pointing with Clutching
3. hybrid RayToRelative pointing

They did some tests and determined that RayCasting was poor in accuracy and comfort for the subject. The actual experiment involved selecting various targets in sequence.

Commentary

This research would do well in finding ways to replace the mouse. Since computer screens are getting bigger, a mouse would not be feasible especially if the screen takes up the whole wall. Obviously there is much more work to be done, like trying to find a balance between comfort for the user and accuracy and reliability of the device and software. Also, unless they're using a mac, they would need a second click (right click). I'm sure that other tracking systems mentioned in the previous work can also help out in this area.

Email: shran2009 at gmail dot com
Academic: 1st semester Master of Computer Science

I am from Houston, Tx.

I am taking this course because this is one of the areas I wanted to study while I'm in grad school

In 10 years, I have no idea where I am going to be. I hope the economy will be better so I can get a job and put my 6 - 7 total years of computer science education to work.

I think the next big improvement in computer science is more integrated touch screen technology (ie, no more mouse and keyboard)

If I could meet anyone in history, I would talk to my grandfather before he went off to war against the japanese in the chinese army

My favorite movie(s) would include the star trek movies and hunt for red october.

Some interesting fact: I am a star trek fan.. although that is probably an understatement.