Wave to your computer! Gestural UIs (SXSW day 2)

Synve waves to her computer

Scroll through your video libary by waving your hand in the air. Press fast forward by spinning your hand faster and faster og copy documents by rubbing a screen. These types of interactions have for a long time been confined to the research labs or the science fiction movies. Gabriel White from Punchcut talked about the challenges we face when we try to develop this new techology into great interaction for the ordinary users. I had a great time during his talk yesterday and my head has been buzzing with ideas ever since.

Gestural UI

Gestural UI is the use of body or hand movement to control what happens on a device or a piece of software. Though this is really not something new, the rise of new products for the mass marked, like the iPhone,  has opened up a new area of great possibilites for interaction designers. What I enjoyed about this talk was that he looked at the premises for making gestural UI work.

White points out two main problems that are critical in developing successful gestural UIs for the masses.

Affordance

An affordance is a quality of an object, or an environment, that allows an individual to perform an action. (Wikipedia).

The most important issue according to White is how people and devices communicate intent.  White pointed to two different ways to do this- declarative and implicit. With declarative intent the interface explicitly tells us what it wants us to do. An example is when an ATM asks you to press a button. Implicit refers to when we behave in a particular way and the devices react. Most touch- screens are still in a declarative mode of directions. Some devices like the Wii who depend on gestures still use explicit ways to express intent.

Language – what does shake mean?

Creating a common language for gestural UIs is according to White another critical issue to solve. Lets look at what shaking can mean on your iPhone.

• Urban spoon app (iPhone app to find a restaurant)- Shake your iPhone and get a new type of restaurant suggestion.
• Facebook app – Shake to reload
• Answer incoming call
• Friendbook app – Shake to exchange contact info. +++ Many more

It is obvious that this has great potential for confusion and  the success of gestural UI in our everyday appliances is dependent on a common language. White stated that we need to explore the possibilites of interaction and narrow it down to a clear vocabulary.

Embodied mind and gestural UIs

During Whites talk I got loads of ideas and thoughts. One of them was about the embodied mind. In cognitive psychology the principle of the embodied mind is that how we think is shaped by aspect of the body.  It also points out the problems of having a brain developed for a natural and spatial environment and using it in the very abstract digital world we live in. There is little mapping between pushing a little red knob or ctrl+alt+del to remove something.  What striked me during the session was that gestural UIs can be a great possibility for improving this mapping. In other words give us an interaction with technology that more resembles how the human brain works. Lack of mapping increases complexity and the risk of errors.  Thus, this can potentially be used to minimize errors and accidents, reduce training time etc.

Cowgirls 4 Ever Score (6 cows is best score)

Technical content:

Presentation:

Other presentations we attended

• Social Media Nonprofit ROI Poetry Slam. Panelist were Beth Kanter – Beth's Blog, Danielle Brigida – National Wildlife Federation, Wendy Harman – American Red Cross – National Headquarters and David Neff – American Cancer Society. Evaluation: Really cool and well executed panel on rhyme! Even the judges were making poems. They even gave me some good practial tips on measuring ROI. The room vas unfortunately to big and they should have had a smaller room and served beer. The great show deserved better audience.
• Emerging Trends of Mobile Technology. Rob Gonda – Sapient and Juan-Carlos Morales – Sapient Interactive. Evaluation: The panel made some interesting points but they seemed a bit unorganized and their slides were not too good.
  
• The Seven Rules for Great Web Application Design.
  Robert Hoekman Jr – Miskeeto LLC. Evaluation: Really just basic usability rules. Not especially interesting, unless you were a total novice to the field. The presenter seemed a little bored.
• Making Web Widgets Accessible: Tools and Techniques. Panellists: Michael Cooper (W3C), Shawn Henry (W3C WAI), Catherine Laws (IBM), Thomas Logan (HiSoftware), Richard Schwerdtfeger (IBM). Excellent people, I’m sure, but not very enthusiastic, so they lost me (KH).
• Microformats: A Quiet Revolution. Panellists: Tantek Çelik (tantek.com), Leah Culver (Six Apart Ltd), Karsten Januszewski (Microsoft), Glenn Jones (Madgex), Jeremy Keith (Clearleft). Good panel, interesting topic, great examples. Check out the wiki.
• Collaborative Filters: The Evolution of Recommendation Engines. Pannellists: Anton Kast (Digg), Scott Brave (Baynote Inc), David Maher Roberts (The Filter), Jon Sanders (Netflix), Erik Frey (Last.fm). Educational, great line-up.

If you have any questions about any of the presentations we attended, or about the South By South West Interactive Festival in general, drop us a line. Or just follow us on twitter – http://www.twitter.com/khamnes & http://www.twitter.com/synver.