We Need a Revolution in Software Interaction Design

We need a revolution in software interaction design. Apple and Google will not provide it. They are too big. They are not the solution. They are the problem.

This revolution will probably come from some unsuspecting source, like the Maker Movement, or an independent group of people or company that is manufacturing physical goods. Here’s why: as computers increasingly inhabit physical objects, as the “internet of things” grows, as more and more computing makes its way into cars, clothing, and houses, there will come new modalities of interacting with software. And it will be dictated by properties of the physical things themselves. Not by the whims and follies of interface designers, whose entire universe consists of a rectangle of pixels and the touch of a user.

Let me explain what I mean when I say that software interaction design needs a major paradigm shift.

Affordance

I hate having to use the word “affordance”. It’s not a very attractive or colorful word. But it’s the best I’ve got. If you’ve read my other blogs posts or my book, Virtual Body Language, you have heard me use it before. The word was given higher currency in the user interface design world thanks to Donald Norman, whose book, The Design of EveryDay Things, I highly recommend. (He was forced to change the name from the “Psychology of Everyday things”. I like his original title better).

Affordance, originally used in J.J. Gibson’s theory of ecological psychology, refers to the possible ways an animal or human can interact with an object (which can be another animal). We often use it in reference to the ways that one interprets visual, tactile and sonic features of a thing, be it an egg-beater, a frightened dog, or a new version of iMovie.

A “natural affordance” is a property that elicits an understanding or response that does not have to be learned  – it’s instinctual. In reference to industrial design: a knob affords twisting, and perhaps pushing, while a cord affords pulling. The term has more recently been used in relation to UI design to indicate the easy discoverability of possible actions.

UPDATE…

Bruce Tognazzini pointed out to me that Donald Norman has more recently been using the term “Perceived Signifier”. And this article explains some of the new semantic parsing going on regarding the word “affordance”. Personally, I would be happy if all of this got subsumed into the language of semiotics.

I believe we have WAY TOO MANY artificial affordances in our software interfaces. I will repeat the call of many wise and learned designers: we need to build tools with natural affordances in mind. Easier said than done, I realize. Consider a common modern interface, such as the typical drop-down dialog of the Apple that allows one to download a file:

As a general rule, I like to download files to my desktop instead of specifying the location using this dialog. Once a file is there, I then move it to the appropriate place. Even though it takes me a bit longer, I like the feeling of putting it there myself. My muscles and my brain prefer this.

Question: have you ever downloaded a file to a specific, somewhat obscure folder, and then later download another file, thinking it was to your desktop, and then not being able to find it? Well, you probably didn’t think to check the dialog box settings. You just hit SAVE, like you usually do, right? It’s automatic. You probably forgot that you had previously set the dialog box to that obscure, hard-to-remember folder, right? Files can get lost easily, right? Here’s the reason:

Software files are abstract concepts. They have no physical location, no mass, no weight. All the properties that we associate with files are virtual. The computer interface is just a bundle of physical metaphors (primarily desktop metaphors) that provide us with affordances so we can think about them as if they were actual things with properties.

The dialog I showed you doesn’t visually express “in” in a natural way. The sensation of the action is not like putting a flower in a vase or drawing a dot in a circle.

Herein lies a fundamental problem of software interface and interaction design. Everything is entirely arbitrary. Natural laws do not apply.

Does the natural world present the same kind of problem as we have when we lose files? Sometimes, but not so often. That’s because the natural world is full of affordances. Our memories are decorated with sensations, associations, and connections, related to our actions. If I physically put a rubber band in an obscure bowl on the top shelf, I have reason to remember this action. Muscle-memory plays a major part in this. With downloading files on a computer, you may not know where you put the file. In fact, sometimes you can’t know!

In fact, it’s not fair to use the word “put”, since “putting” is a deliberate, conscious act. An accidental fumble on the keyboard can cause a keyboard shortcut command that deletes a file or opens up a new window. When this happens, the illusion breaks down completely: this is not a real desktop.

Am I getting too esoteric? Okay, I’ll get more down-to-earth and gritty…

My Deteriorating Relationship with Apple

Bruce Tognazzini says: “While Apple is doing a bang-up job of catering to buyers, they have a serious disconnect at the point at which the buyer becomes a user.” 

My recent experiences with Apple software interfaces have left me worse than disenchanted. I am angry. Apple has changed the interface and interaction of one too many of its products, causing my productivity in some applications (like iMovie) to come to a screeching halt.

At the end of the day, I’d rather keep my old computer with my trusty collection of tools than to have a shiny new, sexy, super-thin Macbook that replaces my trusty old tools. There are years of muscle memory that I have built up in learning and using these tools. My career depends on this muscle memory. When Apple changes these interactions with no clear reason, I become very angry. And so should all its customers.

And then…there is iTunes.

Let’s not talk about iTunes.

Apple’s latest operating system disabled many of the interactions that I (and many others) have been using for decades.

One annoyed user said: “I wish Apple wouldn’t change the fundamental functions of the Finder like this. First, they reversed the scroll button directions then they changed the default double-click to open a folder in a new window function. I’m just glad that they don’t make cars otherwise we would have the brake and accelerator pedal functions reversed (with a pull on the wiper lever to revert to the original)!”

And from the same thread… “If a company cares about users, it doesn’t make a change just for the sake of change that wipes out thirty years of muscle memory.”

Am I saying that a company like Apple should never change its interfaces? Of course not. But when they do, they should do it carefully, for valid reasons, and gracefully. And they should ALWAYS give their customers the choice of if, when and how to adapt to these changes. Easier said than done, I know. Apple is like most other companies. They feel they have to constantly make NEW products. And that’s because our capitalist system emphasizes growth over sustainability.

Will the Maker Revolution Cure Us of Arbitrary, Ephemeral Design?

This is why I believe we need to return to natural affordances that are as intuitive as putting a spoon in a bowl or carving the bark off a stick. The more natural the affordance, the less arbitrary the design. Designers will have to be less cocky, more reverent to human nature and physical nature.

When real physical things start dictating how we interact with software, the playing field will be different. And software interaction designers will have to fully understand natural affordances, and design for them. That’s a revolution I can get behind.

.