Love to be wrong: Four guesses about the Apple headset

The last few weeks my brain’s been absolutely buzzing about the Apple AR/VR headset. I’m squarely in the target market — hell, I am the target market — for this thing:

• I’m into AR and VR — you would not believe how many headsets I have, from the Magic Leap to the Valve Index to older, jankier stuff.

• I’m a sucker for a well-designed, opinionated, oddball product. Teenage Engineering’s OP-1 is a triumph, in my eyes, price be damned. Here’s hoping for a weird one.

• I’ve always been obsessed with new modes of human-computer interaction. They’re fascinating! What could we do with eye tracking, or voice, or context-aware devices, or, or...

Like I said: my brain’s been buzzing. And I thought it’d be fun to write down some predictions, if only to see just how wrong I can be.

Four predictions guesses about the Apple headset

1. It’ll be pitched as a new kind of computer

There’s been lots of hand-wringing about how Apple will position the headset. Is it a developer-focused device? Is it for entertainment? Is Apple going to get into games?

My guess? They’ll pitch the headset as a new kind of computer, meaning: a device that does all the things a MacBook does, and then does some new things, too.

The announcement will closely parallel the iPad’s announcement. Remember that one? Steve Jobs, on stage, talking about a device that could fit between a phone and a laptop — or, as he put it:

Everybody uses a laptop and a smartphone. And the question has arisen, lately: is there a room for a third category of device, in the middle? Something that’s between a laptop and a smartphone. …

The bar’s pretty high. In order to really create a new category of devices, those devices are gonna have to be far better at doing some key tasks. They’re going to have to be far better at doing some really important things — better than the laptop, better than the smartphone.

There’s no one, single, game-changing, killer use case for a VR headset. But there’s also nothing like that for a computer — what’s the single, killer use case of a MacBook Pro?

Apple will pitch their headset as a new computer, which can do everything your MacBook or iPad can do, and can also do a few things “far better.”

2. You won’t need a controller, and that’s the big deal

Apple’s at its best when it’s working on new kinds of input. The mouse and the Mac, for example, or — obviously — multitouch and the iPhone.

The thing about multitouch, though, is that it feels very direct. Compared to multitouch, using a mouse seems like a step backward. And using a clunky AR/VR controller feels like several steps back. (I’ve got a whole rant about this, if you’re interested.)

My guess: Apple’s headset will have unbelievably good input, using just your fingers. Precise enough for pro apps — like Final Cut Pro, or Logic — and intuitive enough to need (almost) no instruction.

How? They’ll use a combo of eye tracking and hand tracking to make a good guess about what exactly the user is trying to interact with. This’ll be mostly invisible to the user — I don’t expect them to call attention to the eye tracking. But it’ll feel like it just works, like touching an icon on the iPhone.

This, really, is the killer feature of the Apple headset.

2b. ...and that’s why it’ll run iPad apps

Mark Gurman has reported that the headset will run iPad apps. That makes sense to me.

Every iPad app is designed to be used with your fingers. Fingers aren’t very precise. And as good as Apple’s hand-and-eye tracking system will be, it won’t be as precise as a mouse.

You’ll be able to use a mouse and keyboard with the headset — probably the mouse and keyboard of any Mac you happen to be sitting next to (the iPad can do this already). But your hands are the default, and the iPad just happens to have a whole ecosystem of apps designed for them.

It’s super handy that Final Cut Pro and Logic just launched on iPad, huh?

3. It’ll be Apple’s most social device

FaceTime is about to get a huge upgrade. I imagine they’ll keep the name — or maybe tweak it ever-so-so slightly (“SpaceTime!”). But it’ll feel entirely new, and more importantly: it’ll be an OS-level feature, that can run alongside any other software.

If you’re an editor, you’ll be able to sit in your home office, pull up Final Cut Pro, open a project... and then have the director appear beside you, as if they were in the same room — even though they’re in their own home, far, far away (read: west side of LA).

You’ll be able to do this anytime, while using (almost) any app, in either AR or VR mode. My hope is Apple will attempt photorealistic 3D avatars — but mostly because I find Memoji creepy.

This isn’t a metaverse play, though. It’s really just Apple trying to find a way to flip the script on VR headsets — which are isolating by nature — and turn their headset into their least isolating, most collaborative computer.

4. Miscellany

None of these predictions are terribly interesting, but I do have a lot of them, so here’s all the rest:

• You’ll be able to use any app in either AR or VR mode. The hardware dial will let you switch between the two. This probably means you’ll have a “home” environment, which is what you’ll see when using apps that don’t need anything else. But even games will be able to mark objects as “important, always show” so you can play them in either AR or VR.

  • Also: the animation for switching between the two will be something delightful. You just know it will; Apple’s good at that. I can’t wait to see it! (My guess: like an old TV set turning off.)

• Speaking of that hardware dial, my guess is it’ll be much bigger than the Apple Watch’s dial, and maybe even embedded into the frame of the device. It’ll be oriented like the focus dial on a camera lens — so it’s easy for your finger to spin it up or down — but probably positioned near your right temple.

• I think there will be some external display. If you’ve been following the rumors, you know there’s debate over this — the idea of a screen that’s just to show other people where your eyes are looking sounds bonkers. But it’s bonkers in a way I like, and I bet Apple can make it look good. (It’ll probably be much more stylized than not.)

• The price will start around $1,500. The number that keeps leaking out in the press is $3,000, but that reminds me of when we were convinced the iPad was going to be $1,000. Apple knows how to set expectations. ($1,500 is still very expensive, by the way! It just feels cheap compared to $3K.)

• It’s going to get bad press, sell out at launch, and then pick up steam over the course of years. Tech press always underestimates how important getting the input right is to a new kind of computer. Apple will get that part right, but the price will be steep and there will be questions about what exactly the headset is for.

• You’ll be able to watch MLS games from the touchline. Apple acquired a company that was doing sports-in-VR, and then bought the rights to MLS for a decade. As a passionate fan of the very-worst team in the league (go LA!), I’m pretty much thrilled by this.

Okay! That’s that. I’ve hammered out a thousand-plus words and can feel the caffeine wearing off. WWDC starts June 5th — and we’ll get to see just how wrong I can be.

The conviction we’re missing

I can’t get over touchscreens.

I can’t get over the move they made; the pivot — not in a Silicon Valley sense, but in a basketball-player-pivoting-toward-a-shot-sense. Touchscreens really made a move, a full step beyond the mouse and cursor, a move so foundational that I can’t get over it.

It’s about degrees of separation, right?

Mouses and cursors are a delightful thing. You, the lumbering mammal, push this hunk of plastic across a desk. Above that desk, on another, more ethereal plane, a cursor moves in perfect sync. There’s magic there, in the synchronicity. It feels good to use. It feels very close to direct.

But it’s not direct.

Touchscreens took that whole thing a quantum leap forward. They removed a degree of separation: now, you’re not moving the thing that moves the thing. You’re just moving the thing. With your own fingers. The same way you move other, real-er, stuff.

This pivot is profound, and it wasn’t easy. It took more than just the technology — on its own, multitouch was a gimmick. It took things like the way a scrollable view “bounces back” when you reach the end, or the way pinching on a thing expands it, as if it were a real, malleable material that could be stretched at will.

It took all of that. No one part is more important than the others. If you lose any of it — the tech, the bounce-back, the pinch, the Frames Per Second — you lose the whole thing.

Touchscreens. Multitouch. The iPhone.

When I say those words you’re thinking of technology or devices. But I’m thinking of a sea change in degrees of separation. Software is no longer a thing I use other tools to interact with. It’s the thing I interact with, the same way I interact with a sink full of dirty dishes so we can have dinner tonight.

If you want AR to happen, we need a similar sea change.

You can see the pieces lining up. The tech is nearly there, and shrinking every year. The UI ideas are doing that healthy thing where they pull from the past, but also let the future run wild. It’s all very untamed. That’s good. We have the ideas, the tech, and even the direction.

But we could use some conviction.

Somebody needs to be like the person who said “on the iPhone, every scrollable view will bounce back when you reach the end. Every single view. That way it feels like the thing you flicked had momentum, heft, and weight. That way it feels real.”

Somebody needs to be that person — but about AR.

Somebody needs to be the person who says: “When you let go of a piece of software (any piece of software) it falls. Every single time. That way it feels like the thing you let go of had weight, physicality, and an obedience to gravity. That way it feels real.”

Or maybe it’s: “When you nudge a piece of software with your finger, it slides across the surface it’s on — with friction and resistance. Every single time. That way it feels like the piece of software is present, and predictable; less like the future and more like a vase.”

Or even: “When we’re designing objects to add to a person’s living room, they can’t all look like capital-d Design Objects. They need to be varied, but down-to-earth; less Design Milk and more end-caps-at-Target. That way they can be part of a home.”

I’m not sure gravity is the thing, for what it’s worth. I’m not sure friction or aesthetics are, either. And there’s no reason to think it’s just one thing anyway: multitouch has pinch-to-zoom, bounce-back-on-scroll, swipe-to-unlock, and more.

In the same way quantum physicists can compare their model of the universe to experimental results and go, “yeah, something is missing here,” we should be able to look at the current AR interaction model and wonder: what else? What’s missing?

Because I am sure of this:

We’re never moving back a degree. We’re never moving back from directly manipulating software. Take all your ideas that aren’t that, and throw them out. Direct manipulation is table stakes.

The question for us is: what comes after that?

Should interfaces look good?

When I’m on my iPad, I’m restless. I flit between different websites and apps, reading a snippet here, and a tweet there, before moving on again. It’s meaningless to talk about my “focus” in this state. My focus couldn’t be less. Using an iPad (or a computer, or my phone) puts me in a restless, unfocused state, and it can be hard to escape — I’m too all over the place to even notice.

I’ve long viewed this as a personal failing. And look: it probably is. But I’ve worked in UX for a few years now, and if there’s one thing I’ve learned, it’s that when you’re designing for humans, you should be designing for all humans.

Including the restless ones. Including the ones who can’t stay focused.

Should the iPad be harder to use? Should it be slower? Should they remove split-screen multitasking? Can we go backward? Would that be enough?

The icons on the iPad’s home screen are colorful, inviting things. Tapping one whisks me into it — it grows and morphs to reveal the app within. There’s a strong sense of place, of spatial awareness; I know where I’m at, and I know how to get back to where I was, at all times.

These all sound like good things.

The iPad’s OS is chock-full of the kind of animations I adore. With a touch, it springs forth. With a swipe, it glides along. Opening things opens them; deleting things tosses them, literally, into a small trash can icon. The bin bobbles in place, to let me know it worked. Trash, trashed.

These all sound like good things.

It’s Saturday morning and I’m up early. I grind coffee, bring water to 172 degrees, and let it all steep. The day is filled with possibility. And thank God — it has to be; otherwise what was that whole week of work for?

I take my mug to the living room. I pick up my iPad; it scans every last centimeter of my face before unlocking the screen. This takes no time. I touch, swipe, and scroll. Content leaps forth.

I’m just going to read a bit before I start my day. Next thing I know, it’s the afternoon.

At work I’m a stickler for aesthetics. I don’t know why — my job is writing — but I’m a stickler anyway. If we don’t push ourselves, we’ll never be able to make something beautiful. We’ll be tempted to ship too quick. We’ll create UIs that weren’t loved over, and we won’t feel good standing by our own work.

Whoever designed the iPad’s OS could stand by it. It looks great. It feels loved over. But that same love, that same care, can eat up my entire Saturday morning. It eats up possibilities. It drains me of time, and I know that’s my fault, but I can’t help but think maybe it’s also the fault of a bunch of designers, doing their best to design something beautiful — but never questioning the value of that.

We’ve gotta design for people who aren’t ideal — because that’s most of us. We’ve gotta design for people who are just human. For people who are restless. For people who can’t stay focused.

Should interfaces look good? Should they be immediate, and beautiful? Should they let you glide, frictionlessly, between content and some more content?

Is that a good thing?