I got to attend ARE2010 recently, which was an amazing lineup of speakers and a great show of the current state of Augmented Reality. The big takeaway for me, though, had nothing to do with AR and everything to do with AI.

Augmented Reality is, at its essence, just a new interface. In some cases, it’s a better interface, in some cases it’s not, but that’s basically what AR brings to the table: a new means of displaying information, albiet now in a way more integrated with the user’s physical surroundings. The promise is in that integration: visions of sci-fi heads-up displays, the sort of stuff the anime artists have been playing with for years, the ability to leverage our ridiculously well-developed online abilities in the real world.

The problem, though, is that we’ve got almost zero spare cognitive bandwidth to play with. The unfortunate fact is parsing what an alert from a mobile system is takes effort, and effort takes focus, and focus is a finite quantity limited to One: What am I focusing on Right Now? This is easily made apparent just by observing people trying to walk while using smart phones, and it’s also why the headset requirement in California is basically window-dressing: the problem isn’t where our hands are, it’s where our heads are.

So AR poses an interesting conundrum: On the one hand, with some work on machine vision and context filters (and goggles - god knows we don’t need “smartphone elbow” to become the carpal tunnel of the 21st century), it could be a radically new and useful way to get highly readable information about the world around you. On the other hand, current systems display entirely, totally, utterly too much information, even assuming someone’s actually focused on the information being displayed. It’s rare I need to know where Every coffee shop in the city is - normally I’m just looking for the best one. A smart, useful AR system needs to be able to filter the infinite feed of information down to the absolute most useful things you could need to know right now and display only those things.

The question of usefulness of the information itself is only half the battle, though. There’s also an issue of context - What am I doing Right Now? What sort of mood am I in? Am I looking for a coffee shop to meet a friend for a chat or a coffee shop to work for the day? Did I already have lunch, or should the place have good food, too? Am I a creature of habit, or do I crave novelty? What’s the weather like? Information Utility varies by person, by day, by time, and by circumstance.

This applies to notifications, too. If I’m engaged in a (physical) conversation, I probably don’t need to get that email about a coupon for a bookstore. The email about the fire in the server room, though, might warrant an interruption. Likewise, if I’m wandering around looking for something to do, the fact that a friend of mine just checked in to the bar a block away is relevant, whereas if I’m in the office, it’s probably not. What’s even trickier is that mood plays a big role, too. Maybe I’m just not feeling like going out today, or maybe I’m on a super-productive streak, in which case anything shy of the second coming of jesus should probably wait until I run out of steam.

What this boils down to is a system that knows you, knows where you are, knows what you’re doing, and knows how to filter all of that down to just what you need. It remembers what you’ve done before, recognizes by the amount of time you’ve spent somewhere and your overall mood there what you thought of the place, can recognize by your behavior what your likely mood is, can tell by your surroundings what you’re up to, and from all that can tailor its recommendations to what’s best for you right now. Basically, it’s a personal assistant in software form - an AI.

We’ve reached a point in the evolution of computers, the internet, and mobility where we no longer have to try to find information or to connect with people. We’ve wired everyone and everything to everywhere (or at least to Google) such that it’s not a question anymore about whether the information exists, it’s whether it floats to the surface of whatever interface we’ve slapped on top of the fire hose of data aimed at our eyeballs and our brains. We don’t have an Access problem anymore, we have an Excess problem, and there’s just no way that dumb filters (or even just slightly smart filters) are going to be able to carry us forward. We need learning filters. We need systems to sit between us and the fire hose, to act as the gatekeeper to the ludicrously limited cognitive time we’ve got. We need something to take the terabytes terabytes of information we’re constantly exposed to and strip it down to the most relevant 40 bits and deliver it to us so that we Know it, we don’t just see it or read it.

I’ve become increasingly aware of the value of cognitive downtime lately. Our brains need time to parse the materials they’re exposed to, and without taking time to pull out of the feed and parse the information we’ve obtained, we lack context and the ability to assimilate the knowledge we’ve gained. Minimizing non-essential or extraneous pulls on our attention allows us to focus more fully on the world around us and even allows us to relax and reflect. With mobility as it exists today, we’re fast-tracking ourselves towards a sort of mass social ADD, and AR without context is only going to worsen the matter.

(Credit to S.Applin, M.Ito, & others working on this problem. Apologies to any other unattributed shoulders on which I’m standing.)