Floating Image Layers Using DeviceMotion

At Pushplay, we’re always experimenting with different interesting, interactive effects. As difficult (or painful) as it is to be “first” at something, we always are trying to push things and experiment.

One of the things that inspired us to get into mobile development (coming from the web world) was the fact that a mobile device gave us the shaking interaction. As much fun as it would have been to get web users to lift and shake their monitors, it just isn’t practical (although it would be very amusing to watch)…

One of our clients was an agency that was pitching a fast food chain (to take on all of their digital projects). They wanted to differentiate themselves by showing a working iPhone app (as opposed to comps in a Powerpoint), and they designed one specifically to demo in their pitch.

They designed an app that had a menu of basic options – a food menu, a location screen, a “favorites” screen, and a contact form.

We encouraged them to take it one step further. We asked them to design a series of coupons. To access an offer, the user would be prompted to shake the phone. Upon shaking it, the app would taunt them with messages like “you can shake me harder than that!” After about three messages, they would be shown a special offer…or given the chance to shake it for a different offer.

The client meeting went incredibly well — after a day of watching agency after agency pitch with Powerpoint slides, our agency came in and handed iPhones to all of the clients. The clients started to look at the app as if it was a mobile web site — a nice way of presenting, but nothing special. Once they tapped on the coupon though, they realized they had to do something more than tapping to interact. After shaking it the first time, and seeing it taunt them (the messaging was based on their brand) a number of them chuckled and started shaking the phone. Suddenly they were all smiles and they all stood and shook the phone trying to see their offer as well as comparing them with other people. It completely changed the attitude in the room from stuffy boredom to exuberant enthusiasm.

Suffice it to say, our agency client won the work.

That was in 2009. Fast forward to today, and we’re still looking for things that would engage a user in something more than voyeuristically watching something entertaining. It’s not enough to display buttons for tapping, or even have panels with swiping or pinching — we are looking for other ways to make someone feel completely immersed in a new world.

One of the effects we’ve been playing with is a pseudo-hologram effect. The idea is to take advantage of the DeviceMotion APIs to access the position of the device, and then adjust all of the on-screen elements based on how the user tilts it. It should look like all of the flat, 2d images are almost floating above the device in 3d space.

When we first tried this with the accelerometer, it just wasn’t right for the effect we wanted. It was both too sensitive (the data fluctuated wildly, even with normalization), and not sensitive enough (if the device was moved laterally, it wouldn’t register any change). Fortunately, with the advent of the iPhone 4 and the gyroscope, we had something much more accurate. The DeviceMotion APIs also helped aggregate all of these different device capabilities into one blended method, removing the need for complex 3d physics operations to try to calculate the proper changes in angle or device orientation.

There is still a lot that needs to be done, but the current state of this effect works like this:

As images are added to a ViewController, they are translated along a Z axis (at different heights)

img.layer.transform = CATransform3DMakeTranslation(0.0f, 0.0f, myZ);

(in the example project, myZ is a range from 0 to 80 — the larger the value, the further “out” the image looks like it is from the device — almost like it is floating a couple of inches above the device’s screen)

Then, we do the following to change the display angle of the ViewController’s view based on the current device’s motion:

myAppDelegate *appDelegate = (myAppDelegate *)[[UIApplication sharedApplication] delegate];
	
	if (appDelegate.motionManager.isDeviceMotionAvailable) {
		appDelegate.motionManager.deviceMotionUpdateInterval = 1.0 / 60.0;
        
		[appDelegate.motionManager startDeviceMotionUpdatesToQueue:[NSOperationQueue currentQueue]
													   withHandler:
		 ^(CMDeviceMotion *motion, NSError *error) {
			 
			 CATransform3D transform;
			 
			 double newPitch = motion.attitude.pitch;
			 double newRoll = motion.attitude.roll;
			 
			 transform = CATransform3DMakeRotation(newPitch, 1, 0, 0);
			 transform = CATransform3DRotate(transform, newRoll, 0, 1, 0);
			 
			 self.view.layer.sublayerTransform = transform;
		 }];
		
    } else {
        NSLog(@"No motion manager on device.");
    }

And don’t forget to import:

#import <CoreMotion/CoreMotion.h>
#import <QuartzCore/QuartzCore.h>

The DeviceMotion API does use the new “dark magic” of handler pointers. If you haven’t used them yet, get ready…it looks like a lot of apps are going to be using these in the future…

The link to the project is here: floating image demo. Please let us know if you come up with cool changes, or neat implementations of this!

jeffrey

About jeffrey

Jeffrey Berthiaume is a multimedia developer and internet architect who has designed and built award-winning websites, kiosks, and content management systems. He bridges the gap between creative and technology with an ability to balance the needs of designers and marketing with the capabilities of existing technology.
This entry was posted in Development, iOS, Tutorials. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>