Courtesy of The Guardian, an interesting look at augmented reality. As the article notes:
Don’t act too surprised if, some time in the next year, you meet someone who explains that their business card isn’t just a card; it’s an augmented reality business card. You can see a collection and, at visualcard.me, you can even design your own, by adding a special marker to your card, which, once put in front of a webcam linked to the internet, will show not only your contact details but also a video or sound clip. Or pretty much anything you want.
It’s not just business cards. London Fashion Week has tried them out too: little symbols that look like barcodes printed onto shirts, which, when viewed through a webcam, come to life. Benetton is using augmented reality for a campaign that kicked off last month, in which it is trying to find models from among the general population.
Augmented reality – AR, as it has quickly become known – has only recently become a phrase that trips easily off technologists’ lips; yet we’ve been seeing versions of it for quite some time. The idea is straightforward enough: take a real-life scene, or (better) a video of a scene, and add some sort of explanatory data to it so that you can better understand what’s going on, or who the people in the scene are, or how to get to where you want to go.
Sports coverage on TV has been doing it for years: slow-motion could be described as a form of augmented reality, since it gives you the chance to examine what happened in a situation more carefully. More recently cricket, tennis, rugby, football and golf have all started to overlay analytic information on top of standard-speed replays – would that ball have hit the stumps, the progress of a rally, the movement of the backs or wingers, the relative flights of shots – to tell you more about what’s going on. Probably the most common use is in American football where the “first down” line – the distance the team has to cover to continue its offence – is superimposed on the picture for viewers.
But those required huge systems. AR took its first lumbering steps into the public arena eight years ago: all that you needed to do was strap on 10kg of computing power – laptop, camera, vision processor – and you could get an idea of what was feasible. The American Popular Science magazine wrote about the idea in 2002 – but the idea of being permanently connected to the internet hadn’t quite jelled at that point.
“AR has been around for ages,” says Andy Cameron, executive director of Fabrica, an interactive design studio which works with Benetton, “maybe going back as far as the 1970s and art installations that overlaid real spaces with something virtual.” He mentions in particular the work of pioneering computer artist Myron Krueger.
What’s changed in the past year is that AR has come within reach of all sorts of developers – and the technology powerful enough to make use of it is owned by millions of people, often in the palms of their hands.
The arrival of powerful smartphones and computers with built-in video capabilities means that you don’t have to wait for the AR effects as you do with TV. They can simply be overlaid onto real life. Step forward Apple’s iPhone, and phones using Google’s Android operating system, both of which are capable of overlaying information on top of a picture or video.
Within the small world of AR, one of the best-known apps is that built by Layar, which – given a location, and using the iPhone 3GS’s inbuilt compass to work out the direction you’re pointing the phone – can give you a “radar map” of details such as Wikipedia information, Flickr photos, Google searches and YouTube videos superimposed onto a picture you’ve taken of the scene. For Americans, it will also pull in details from the government’s economic Recovery Act – so that if you’re on Wall Street and want to see how many billions went into which building, it will show you.
Or, more usefully, Yelp offers an augmented reality application that will show you ratings and reviews for a restaurant before you walk in – the sort of thing that could make restaurants quiver with delight, or shudder in horror.
Or maybe it wouldn’t need to know where it is; only who it’s looking at. A prototype application demonstrated at the Mobile World Congress in Barcelona in February took things a little further again. Point the phone at a person and if it can find their details, it will pull them off the web and attach details – their Twitter username, Facebook page and other facts – and stick them, rather weirdly, into the air around their head (viewed through your phone, of course). “It’s taking social networking to the next level,” says Dan Gärdenfors, head of user experience research at The Astonishing Tribe, a Swedish mobile software company.
And there are fabulously useful applications: at Columbia University, computer science professor Steve Feiner and PhD candidate Steve Henderson have created their Augmented Reality for Maintenance and Repair (Armar) project. It combines sensors, head-up displays, and instructions to tackle the military’s maintenance needs: start working on a piece of kit, and the details about it pop up in front of you. Imagine if you could put on a pair of special goggles when you needed to investigate your car’s engine, or a computer’s innards, and the detail would pop up. That’s the sort of idea that Armar is trying to implement, though for the military at first..
Yet it’s fashion which seems to have leapt quickest into this technology. The T-shirt with AR in London Fashion Week was developed by Cassette Playa, a label that has been worn by Lily Allen, Rihanna and Kanye West. Carri Munden, who designed it with the Fashion Digital Studio at the London College of Fashion, described it as “mixing reality and fantasy”. Adidas, too, has launched trainers with AR symbols in the tongues: hold them to a webcam and you are taken to interactive games on the Adidas site.
The process by which the strange symbols get translated into images is simple enough: the website takes the feed from your webcam (you have to explicitly allow it to do so, so there are no security worries) and analyses it for the particular set of symbols that the program is looking for. (Some easy calculations mean the symbols can be detected whichever way up you hold the item.) Videos and pictures are then sent back to you.
Andy Cameron says that the arrival of an open-source, hence free, AR tool kit has let companies build their own AR applications, using Flash – the pervasive animation and video technology used for many online ads and YouTube’s videos – “which immediately meant you had huge penetration, because Flash is everywhere”. (Something like 98% of all computers are reckoned to have Adobe’s Flash Player installed.)
“If you build your AR application with Flash, then you can get it out to everybody in the world with a computer with a webcam,” says Cameron.
Benetton is using AR in its latest campaign, called “It’s My Time” which aims to get members of the public to put themselves forward as potential models, and uses AR to show more details about existing models. But its first most visible use of AR was last year in issue 76 of Benetton’s Colors magazine, a quarterly fashion product. Dozens of pages have AR symbols: hold the page up to a webcam, and you see film and more photos of the person on the page. “The Colors editor and the creative director of Fabrica got very excited about it,” says Cameron.
Cameron can see huge potential which could even revive the fortunes of print advertising. “Think of a commercial page, an advert, in a fashion magazine. It’s pretty expensive. With this – and this is the way that the more hard-nosed people in Benetton saw the advantage – it means that you can get more products on the page.” Print an AR code, get people to come to the site, and you can show them so much more, while measuring the return from your effort.
The technical cost is a tiny part of the overall effort. “The printing and photography cost [of the advert] is the same. And the development cost is pretty small.”
And of course where advertisers go, the publications that house them are sure to go as well. Esquire magazine in the US and Wallpaper* in Europe have done “augmented reality” editions, with Robert Downey Jr coming to life on the cover of the former, and AR text providing videos and animation in the latter. But there are more possibilities for journalism using AR: for example if you “geotag” newspaper articles (so that you say that an item relates to a particular place) then someone visiting a site could learn about events relevant to the area via their smartphone.
Book publishers too are leaping in: Carlton Publishing will release an AR book in May, featuring dinosaurs that pop out of the pages when viewed, yes, through a webcam. Future releases include war, sport and arts titles which will also have extra AR elements.
Yet in media it’s the advertisers who are most excited. The possibilities of geotagged, targeted adverts – which in effect hang in the air until someone comes along to find them with a smartphone – or of AR adverts which open up a whole new world of opportunities (and perhaps discounts or loyalty bonuses) when you follow them through – are yet another glimpse of the holy grail ofads that know exactly who and where you are.
Is there a risk that we’ll all become AR’d out – that it will become boring as advert after advert invites us to hold it up to a webcam? “What’s hot today is ancient history tomorrow,” says Cameron. “There have been a lot of bad uses of this technology with a rush to use it. We have had the chance to reflect on what it means and how to use it. The key is that it should be an enhancement of the stuff on the printed page.”
Even so we’re still in the early stages, he argues. “It’s very primitive – having to use a webcam, holding a magazine up to it. Obviously we’re really interested in the opportunities with handheld devices. It’s very frustrating that the iPhone doesn’t allow access to the live video stream.” (Nor does it run Flash, another problem for would-be AR designers.) “People in design are very annoyed with Steve Jobs,” he observes. “We don’t really understand why Apple won’t allow that.”
Given that access, he says, “you could hold your iPhone up to a billboard and get something amazing right there”. What about the alternative, such as Google’s Android-based Nexus phone? “It looks like you could do it on that,” he says. But of course the iPhone is a target market. “Maybe Apple wants to keep that for itself,” Cameron says. “Maybe they’re lodging patents. Or maybe the processor on the iPhone isn’t fast enough.”
Yet there are some who think that AR has already had its brief time in the sun. At the Like Minds conference in Exeter at the beginning of March, Joanne Jacobs, a social media consultant, described an AR application that demanded you buy a T-shirt and then go and sit in front of your webcam – so you could play Rock, Paper, Scissors. By yourself.
“It’s hopeless,” Jacobs said.
Cameron admits to some uncertainty about AR’s measurable impact. “I don’t know if it sells more things, but it seems clearly a good thing if we can get people who may be customers to participate in the adverts.” But, he adds: “If people start to play with the adverts in a way that exposes them to more products, that’s got to help bring a commercial return.”