Gesture motions for window-and-mouse based interfaces

Back in early 2007, I designed a method to introduce mouse-based “gestures” into a windowed interface environment. Fast forward to 2010, when the “year of the tablet” might mean the beginning of the end for mouse-based computing…

Still, there are older devices that are still supported, and other projects that may be able to make use of these gestures, and I wanted to put these out for people to consider with their windowed-based applications.

At the time I designed these, I was working with a group to create a series of “personality-based” electronic trading cards — sort of a pop superstar version of baseball cards. These cards were designed to live in an interface (similar to a computer desktop) and would expand or contract to be a mini-card versus a full card (each card would automatically shrink when they lost focus, if a new card opened or the desktop was clicked).

I wanted a way to let a user manage a group of cards, and be able to arrange them in piles. I also wanted there to be a more “3d” feel to this management, versus a typical tile-based layout, *but* I wanted it to be subtle (so most users would experience typical tile/icon layouts (like a computer desktop) but some users would discover subtle motions that would enhance their experience and eventually lend itself into new ways of interacting with these cards). I also hoped that some of these mouse-based gestures would be copied and used in other interfaces as well.

zones of window/card for mouse-gesture interfaceClicking on a card (holding down the mouse-button), moving the mouse, and then releasing the button would result in the card moving — very standard and expected behavior. The enhancements would come if the user *kept* moving the mouse in the same direction after releasing the mouse-button. In my system, that would confer extra velocity to the card, and have one of a couple of effects on the card: it would be spun, tossed, or slid, based on where the initial mouse-click was on the card.

The diagram above shows an example of the zones where these effects would take place. If the user clicked in a corner, and put some “english” on it, the card would move, but would also spin (based on the corner where the user clicked). If they clicked and dragged on an edge (and then kept the mouse moving when they released the button) the card would act like it was tossed (as if it gained some height off the desktop). Clicking and moving from the center of the card would cause it to slide (and potentially move other cards that it hit).

In this way, a simple click and move and release (with no further immediate mouse movement) would act as if the user picked up and placed a card. All other interactions with a card could have immediate effects on the cards surrounding it (either covering or pushing them depending on how they hit).

To recap:

  • If a card is clicked, moved, and then the mouse-button released, the card will be moved (like a standard window in Windows or Mac OS)
  • If the card is clicked, moved, released, and the mouse continues to move in the same direction, it is spun, tossed, or slid, depending on where the card was grabbed.
  • The “toss” area is the border around the outside 20-30% of the card.
  • Toss: effectively lifts a card and tosses it onto a pile. There is a little bit of lift (off the table or desktop) and it lands on top of the other cards (only slightly moving them if anything based on it’s momentum).
  • Slide: slides the card across the desktop, which could have the effect of displacing other cards (pushing them) as well as sliding the card underneath a pile of cards. Some rotational movement can affect the other (hit/displaced) cards as well depending on where they are hit.

(This is much easier to visualize if you use coasters on a table and practice the motions.)

I also came up with some extra effects for the corners: spinning, versus rotating, versus flipping a card. To that end, the *direction* the mouse was moving would give added physical effects to the card. For example:

  • Spin: Clicking the Spin handle and flicking (moving) the card will give it slight rotation momentum as well as direction momentum.
  • Rotate: Clicking the Spin handle and moving in a clockwise/counter-clockwise direction will cause the card to rotate.
  • Flip: Clicking the Spin handle and moving internally (toward the center of the card itself) will lift the card and cause it to flip over.
spin, rotate, and flip gestures for a card

These motions were designed to be slight, subtle, intuitive, and non-obtrusive.

Users familiar with basic windows operating systems (Windows or Mac OS) would operate the way they expect to. However, users could discover these motions the more they use the application and could adopt them based on their personal usage needs. To aid them in this, the mouse cursor would need to change state (depending on the location of the cursor relative to the card) in order to give them a visual cue that something else could happen.

mouse cursors for gesture-based interface


About jeffrey

Jeffrey Berthiaume is a multimedia developer and internet architect who has designed and built award-winning websites, kiosks, and content management systems. He bridges the gap between creative and technology with an ability to balance the needs of designers and marketing with the capabilities of existing technology.
This entry was posted in iOS, Tutorials. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>