The fake Apple gestural movies, a critique by request

I’m subscribed to just about every RSS that contains the words NUI or Multi-Touch in the title. One of the better one’s is http://theclevermonkey.blogspot.com. He made a post the other day about how impressed he was with a set of videos on youtube of a mockup from an anonymous person based on what they thought Apple’s new tablet will consist of interaction-wise. It was titled “Move Over Jonathan Ive, Hear comes Jon Doe”. That grabbed my attention.

Jonathan Ives is one of the best designers working today. I think he has great talent and he got some wonderful opportunities that really allowed him to flourish. He is great, but the only reason he is great is because he has made several errors. Big errors, as we all do in our careers. Bill Buxton was talking to me about this very thing a few days ago. Great artists make errors, learn from them, and move on. Jonathan has made his share of errors, as well as his share of wins. So, no question about Jonathan’s skills, but who is this John Doe?

I made an offhand comment on theclevermonkey about how unimpressed with the gesture language that was demonstrated in the videos. He replied that I should write a constructive and helpful post to point out a few things. So here we are.

The first video in the series:

So, I will breeze over a few of the main concepts that I feel the person missed in their mockup. I only watched 2 of the videos, and will only comment on one, but here is what I gathered from it.

• In watching the gestures I see confusion between single tap and double tap. At around 4 seconds he does a single tap to change its state after he used single tap to get it into the state? Then does another double tap to zoom out? I’m not quite sure I understand the metaphor around each of the taps. If single tap is select, then how can it also be activate and unzoom/state change.  It seems here that the user is going through a pre-determined path of usability rather than real motions.
• At around 10 seconds after the OnScreenKeyboard use, he does a single tap all the way across the screen on a very small hit target. This seems problematic and does not obey Fitt’s Law.

Fitts’s law (often cited as Fitts’ law) is a model of human movement in human-computer interaction and ergonomics which predicts that the time required to rapidly move to a target area is a function of the distance to and the size of the target. Fitts’s law is used to model the act of pointing, either by physically touching an object with a hand or finger, or virtually, by pointing to an object on a computer display using a pointing device). It was proposed by Paul Fitts in 1954.

• The closing of the keyboard should be thought about in a broader term. Why have the button so small? Why have the button on the other side of the screen? Why not put buttons on both sides to accommodate ambidextrous bi-manual input. Why not change its color slightly to ease speed of use? A very thin row of black buttons is not conducive to a natural interface.
• at 17 seconds he uses double tap twice. Once to do an “execute” on the video clip, and then again on the File Explorer window to perform a maximize/full screen. Unless in this operation if you “execute” the File Explorer that also goes into a Slideshow mode? I’m hazy on this.
• at 20 seconds he does a double tap on an image in the slideshow and that propels him into a scatterview experience? Maybe that is a Mac thing that I am unaware of, but if he just performed an “execute” on the icon, why does it go into scatterview?
• at 22 seconds he uses two-fingers to draw a marquee around items which also produces a context menu. If you had a context menu pop up everytime you selected a group it would not be a good experience. Menus need to be deliberately activated, not activated as a secondary action. There would be emails sent. :)
• at 27 seconds he uses two-fingers to then move the carousel around? Why has movement now shifted chords? Two-fingers were just used to draw a marquee and do a group select, and now they are used for movement? This is a confused metaphor and not consistent.

Thats pretty much everything in the first video besides the simple movements with single touch. To really be critical I would also say that using single touch for movement on a background is a little easy don’t you think? Certain things need to be forgivable, the background being one of them.

My main critique over the set is that multi-touch means just that, Multi-Touch. That means more than just two fingers side by side, or even three fingers next to each other. It means thinking about systems in a whole new way. What about offhand manipulation? What about modifiers? Think about occlusion and ease of use. If you are playing a game do you really want to have 3 fingers on the screen to run? then keep them on the screen the whole time? As the author noted in the last movie he had doubts.

To be more helpful and instructive I’ll throw out a few ideas. Why not use the offhand as movement and the primary hand as action? So, with your left hand you place down a finger and hold it outside of the quick menu area, this establishes the center of movement, now with any other touch in a direction near your initial touch you are running. This leaves your other hand free to perform actions that might be needed. Now with another tap with your left hand your running is now ‘frozen’ forward so you can remove your hand. Your character is now running in a direction that you chose without much primary attention. Want to change directions? Use two-fingers and slide. Want to stop running? Two-fingers and slide down. Side-step? You get the idea. Using the initial two taps to set motion then two fingers at the same time to give more detailed instructions eases the attention you would need to give it. Think how fast you could do that?

That ends the lesson for tonight. Overall, great video. I was very impressed with the videos themselves actually. High quality for a mock-up. Now hopefully you understand my thoughts when I see a lot of passion in this effort, but not quite enough expertise. The thing is …. expertise can be learned. Passion cannot. I hope he continues down this path, we could always use more passionate designers out here.

• Related Posts
• Bookmark
• Tags

One Response to “The fake Apple gestural movies, a critique by request”

1. Stephen says:

   September 24, 2009 at 8:18 am

   Very even-handed critique. I enjoyed reading this.

   Reply