From: "Carlton J. Sparrell" <carltonj@media.mit.edu>
Subject: Re: SOC: Speech input or gesturing
Organization: MIT Media Laboratory

I would have to argue that there is no *one* best way to interact in a
VE.  The best interface would be one that would allow the user to
choose from many input modes, selecting the best for the current task.
These input tools should at least include speech recognition, gesture
input, eye tracking, and sketching.

As far as hand input is concerned, it need not be limited to direct
manipulation.  Gesture can be combined with speech for what you refer
to as communication.  This had been shown as early as 1980 for simple
deictic (pointing) gestures with "Put That There".  More recently,
David Koons and I completed a prototype system called "Iconic" which
allows a user to specify object placement and movement using coverbal
iconic gesture (gesture in which the hands depict an object being
simultaneously referred to in speech).  This system has no predefined
symbolic gestures, the hand data is interpreted in the context of the
speech with features of the hand shape being mapped to features of the
object shape.

If there is any interest in finding out more about this work, I could
somehow make a postscript version of my Master's thesis available.

Carlton J. Sparrell
Spatial Imaging Group
(Formerly of the Advanced Human Interface Group)
MIT Media Laboratory