Want to plop a 3D T. rex into a video of your hand? The Motion Stills app for Android phones now lets you do that with a rough-and-ready technique.
If you have a Google Pixel or Pixel 2 phone, you’ve been able to try Google’s AR stickers to plop 3D objects like Star Wars characters into real-world views. Now the company has a new technique that brings the augmented-reality skill to vastly more Android devices.
The AR sticker ability now is built into the company’s Motion Stills app, which can convert ordinary videos and live photos into a variety of short videos with various effects. Among them: stabilization of shaky video, combination of multiple clips into a longer movie, addition of “motion text” that follows an element of the video, or conversion into cinemagraphs that isolate one aspect of the video while keeping the rest still.
Google’s earlier, higher-end approach to AR stickers uses new foundational but hard-to-get Android technology called ARCore. The approach in Motion Stills, called instant motion tracking, is a bit more rough and ready. You can pick an object, like a chicken or gingerbread man, then place it on a horizontal, flat surface at which you’re pointing a phone camera. The object sticks in the view, tracking its perch and changing size appropriately — for example if you put a T. rex in the palm of your hand and then move your hand toward the camera.
The Motion Stills app can put AR stickers into any Android device with a gyroscope, which is nothing special these days.
Of course it lets you share your creation with your friends, a necessity for any photo or video app these days. It’s not a substitute for apps like Snapchat or Facebook’s Instagram — both of which already offer AR effects, by the way — but as a new source of video entertainment for your friends.
The approach doesn’t need the higher-end hardware ARCore requires. On the other hand, it’s not able to match a 3D object to the real-world scale. That’s why you can use it to seemingly hold a two-inch chicken in your hand.
Google researchers detailed the Motion Stills AR stickers approach in a blog post Tuesday. The Motion Stills app is officially published by Google Research, signaling its somewhat experimental status.
The technology works by treating two types of camera motion — rotation and side-to-side translation — as different chores. The side-to-side motion is enough for the app to figure out how much to scale the 3D object, then the gyroscope data is used to orient it correctly on your phone’s viewfinder.
“Because we treat rotation and translation separately, our instant motion tracking approach is calibration-free and works on any Android device with a gyroscope,” Google programmers Jianing Wei and Tyler Mullen said in the blog post.
