Thoughts on the Evolution of Communication

My dog and I engage in a lot of signaling. But it is not always deliberate, and it is not always conscious, and it is not always a two-way process.

In the morning, Otto licks my bald head. He can probably smell what I have been dreaming. I hold him and we have a nice cuddle. Just one of our many routines. He looks at me and I look at him. He is always checking me out. In the process of getting to know each other over several years we have come to read each other’s signals – our body language, interactions, responses, vocalizations…and smells.

image from http://projectdolittle.com/

Semiosis emerges in the process. If there is a coupling of signals – a mutually-reinforcing signaling loop – two-way communication emerges. It is not always conscious – for either of us. Sometimes, a mutually-reinforcing signaling process which I was previously unaware of becomes apparent to me. When this happens, I become an active agent in that semiosis.

Otto is so intensely attentive to me – my routines (and deviations from them). He probably tunes-in to many more of my signals than I do to his. But then again, I am a human: I generate a lot of signal. Does he see this as “communication?” It is not clear: his brain is a dog brain, and mine is a human brain. We don’t share the same word for this experience (he only knows a few English words, and “communication” isn’t one of them).

I can be sure of one thing: we share a lot of signaling. And, as members of two highly-social species, we both like that.

I would conclude from this that communication among organisms in general (the biosemiosis that has emerged on Earth over the last few billion years) came about pretty much the same way that Otto and I established our own little world of emergent semiosis. As life evolved, trillions of coupled signaling channels reinforced each other over time and became more elaborate. Eventually, this signaling became conscious and intentional.

And so here we are: human communication has reached a level of sophistication such that I can type these words – and you can read them. And we can share the experience – across time and space.

Advertisements

Cute Yet Creepy. Animal Yet Human.

Screen Shot 2016-08-14 at 8.54.16 PM

I have been thinking about the uncanny valley for decades. Here are some things I’ve written on the subject:

Screen Shot 2016-08-14 at 8.17.38 PM

The Uncanny Valley of Expression

Uncanny Charlie

How Does Artificial Life Avoid the Uncanny Valley?

Augmenting the Uncanny Valley

Over time, animated filmmakers have become more savvy about the uncanny problem. They are getting generally better at avoiding the creeps. According to this article, Disney learned its lesson, the hard way….

Screen Shot 2016-08-14 at 8.30.42 PM

“And that’s why realism-fetishizing technology like motion capture is much more susceptible to creeping us out than more “primitive” or stylized animation: it’s only when you’re purporting to offer that level of detail in the first place that you can totally, utterly screw it up.”

Despite the fact that animators are more savvy about the Valley, I still can’t help but notice a nagging, low-grade fever of optical realism that has crept into the lineup of popular animated characters (even as the accidental monsters get shuffled off to quarantine). Consumers of animated films may be unaware of it…because it has become normalized. The realism has increased, bit by bit, so that now we have quivering hair follicles, sparkling teeth, and eyeballs reflecting the light of the environment.

Imagine if our favorite classic characters were rendered like this.

real-animals-mickey

But the discomfort we call the uncanny valley doesn’t only occur when the thin veneer of visual realism unexpectedly reveals a mindless robot where “nobody is home”. The phenomenon could be seen in a larger context: it is caused by the clash of any two aspects of an artificial character that operate at incompatible levels of realism. For instance…

Can Animals Become Too Human?

I recently saw Zootopia. I really enjoyed it. Great film. But I must say, I did catch a glimpse of the Valley. There’s no denying it.

I also recently saw Guardians of the Galaxy, with Rocket Raccoon, who exhibits two very different kinds of realism: (1) Raccoon! (2) A tough guy with attitude – and a very human intelligence.

Screen Shot 2016-08-14 at 9.55.27 PMCan contradictory behavioral realism create a different sort of valley? Technology for character animation has enabled a much higher level of expressivity than has ever been possible, with fine detail in subtle eye and mouth movements. One might conclude that since behavioral realism has caught up with visual realism, the uncanny valley should now be a thing of the past. But then again, that depends on whether the behavior and the visuals apply to the same species!

Nothing abnormal about a cartoony raccoon throwin’ shapes and talkin’ tough. But when this animal is rendered in a hyper-realistic manner, AND evoking high-res human expression, things start to feel odd.

Silvery-marmoset-6Pandas, ants, lobsters, bison, eels … in order for all of these various animals to assume the range of human emotion needed to deliver a clever line, they have to be equipped with a face with all the expected degrees of freedom. The result is what I call “rubber mask syndrome”.

273-24099

One example is the characters in Antz, whose faces stretch in very un ant-like ways in order to express very human-like things. More and more animals (unlikely animals even) are being added to the cast of movie stars. They are snarky, sly, witty, sexy, clever…and oh so human. It has all gotten a little weird if you ask me.

The Information EVOLUTION

I remember several decades ago learning that we were at the beginning of an information revolution. The idea, as I understood it, was that many things are moving towards a digital economy; even wars will become information-based.

The information revolution takes over where the industrial revolution left off.

I am seeing an even bigger picture emerging – it is consistent with the evolution of the universe and Earth’s biosphere.

Screen Shot 2016-06-16 at 10.20.47 AM

At the moment, I can hear a bird of prey (I think it’s a falcon) that comes around this neighborhood every year about this time and makes its call from the tree tops. When I think about the amount of effort that birds make to produce mating calls, and other kinds of communication, I am reminded of how much importance information plays in the biological world. The variety and vigor of bird song is amazing. From an evolutionary point of view, one has to assume that there is great selective pressure to create such energy in organized sound.

money+gorilla+teeth+omg+weird+primatesThis is just a speck of dust in comparison to the evolution of communication in our own species, for whom information is a major driver in our activities. Our faces have evolved to give and receive a very high bandwidth of information between each other (Compare the faces of primates to those of less complex animals and notice the degree to which the face is optimized for giving and receiving information).

Our brains have grown to massive proportions (relatively-speaking) to account for the role that information plays in the way our species survives on the planet.

Now: onto the future of information…

Beaming New Parts to the Space Station

Screen Shot 2016-06-16 at 10.29.58 AM

Guess which is more expensive:

  1. Sending a rocket to the space station with a new part to repair an old one.
  2. Beaming up the instructions to build the part on an on-board 3D printer.

You guessed it.

And this is where some people see society going in general. 3D printing will revolutionize society in a big way. Less moving atoms, More moving bits.

To what degree will the manipulation of bits become more important than the manipulation of atoms?

Not Just a Revolution: Evolution

My sense is that the information revolution is not merely one in a series of human eras: it is the overall trend of life on Earth. We humans are the agents of the latest push in this overall trend.

Some futurists predict that nanotechnology will make it possible to infuse information processing into materials, giving rise to programmable matter. Ray Kurzweil predicts that the deep nano-mingling of matter and information will be the basis for a super-intelligence that can spread throughout the universe.

Okay, whatever.

For now, let’s ride this information wave and try to use the weightlessness of bits to make life better for all people (and all life-forms) on Earth – not just a powerful few.

The Miracle of My Hippocampus – and other Situated Mental Organs

I’m not very good at organizing.

pilesThe pile of papers, files, receipts, and other stuff and shit accumulating on my desk at home has grown to huge proportions. So today I decided to put it all into several boxes and bring it to the co-working space – where I could spend the afternoon going through it and pulling the items apart. I’m in the middle of doing that now. Here’s a picture of my progress. I’m feeling fairly productive, actually.

10457290-Six-different-piles-of-various-types-of-nuts-used-in-the-making-of-mixed-nuts--Stock-PhotoSome items go into the trash bin; some go to recycling; most of them get separated into piles where they will be stashed away into a file cabinet after I get home. At the moment, I have a substantial number of mini-piles. These accumulate as I sift through the boxes and decide where to put the items.

Here’s the amazing thing: when I pull an item out of the box, say, a bill from Verizon, I am supposed to put that bill onto the Verizon pile, along with the other Verizon bills that I have pulled out. When this happens, my eye and mind automatically gravitate towards the area on the table where I have been putting the Verizon bills. I’m not entirely conscious of this gravitation to that area.

Gravity Fields in my Brain

What causes this gravitation? What is happening in my brain that causes me to look over to that area of the table? It seems that my brain is building a spatial map of categories for the various things I’m pulling out of the box. I am not aware of it, and this is amazing to me – I just instinctively look over to the area on the table with the pile of Verizon bills, and…et voilà – there it is.

Other things happen too. As this map takes shape in my mind (and on the table), priorities line up in my subconscious. New connections get made and old connects get revived. Rummaging through this box has a therapeutic effect.

The fact that my eye and mind know where to look on the table is really not such a miracle, actually. It’s just my brain doing its job. The brain has many maps – spatial, temporal, etc. – that help connect and organize domains of information. One part of the brain – the hippocampus – is associated with spatial memory.

hippocampal-neurons_0-1

User Interface Design, The Brain, Space, and Time

I could easily collect numerous examples of software user interfaces that do a poor job of tapping the innate power of our spatial brains. These problematic user interfaces invoke the classic bouts of confusion, frustration, undiscoverability, and steep learning curves that we bitch about when comparing software interfaces.

This is why I am a strong proponent of Body Language (see my article about body language in web site design) as a paradigm for user interaction design. Similar to the body language that we produce naturally when we are communicating face-to-face, user interfaces should be designed with the understanding that information is communicated in space and in time (situated in the world). There is great benefit for designers to have some understanding of this aspect of natural language.

Okay, back to my pile of papers: I am fascinated with my unconscious ability to locate these piles as I sift through my stuff. It reminds me of why I like to use the fingers of my hand to “store” a handful of information pieces. I can recall these items later once they have been stored in my fingers (the thumb is usually saved for the most important item).

Body Maps, Brain, and Memory

inbodymaps

Screen Shot 2016-02-07 at 9.03.46 PMLast night I was walking with my friend Eddie (a fellow graduate of the MIT Media Lab, where the late Marvin Minsky taught). Eddie told me that he once heard Marvin telling people how he liked to remember the topics of an upcoming lecture: he would place the various topics onto his body parts.

…similar to the way the ancient Greeks learned to remember stuff.

During the lecture, Marvin would shift his focus to his left shoulder, his hand, his right index finger, etc., in order to recall various topics or concepts. Marvin was tapping the innate spatial organs in his brain to remember the key topics in his lecture.

My Extended BodyMap

18lta79g5tsytjpgMy body. My home town. My bed. My shoes. My wife. My community. The piles in my home office. These things in my life all occupy a place in the world. And these places are mapped in my brain to events that have happened in the past – or that happen on a regular basis. My brain is the product of countless generations of Darwinian iteration over billions of years.

All of this happened in space and time – in ecologies, animal communities, among collaborative workspaces.

Even the things that have no implicit place and time (as the many virtualized aspects of our lives on the internet)… even these things occupy a place and time in my mind.

Intelligence has a body. Information is situated.

Hail to Thee Oh Hippocampus. And all the venerated bodymaps. For you keep our flitting minds tethered to the world.

You offer guidance to bewildered designers – who seek the way – the way that has been forged over billions of years of intertwingled DNA formation…resulting in our spatially and temporally-situated brains.

treblebird

bodymapping.com.au

We must not let the no-place, no-time, any-place, any-time quality of the internet deplete us of our natural spacetime mapping abilities. In the future, this might be seen as one of the greatest challenges of our current digital age.

Hippocampus_and_seahorse_cropped

Failure and Recovery – an Important Concept in Design..and Life

I have observed that good design takes into consideration two important aspects of use:

  1. Failure Rate
  2. Recovery Rate

Well-designed products or software interfaces have low failure rates and low failure amounts. This is related to the concept of fault tolerance. A well-designed product or interface should not fail easily, and failure should not be complete.

“If its operating quality decreases at all, the decrease is proportional to the severity of the failure, as compared to a naively designed system in which even a small failure can cause total breakdown.”

A well-designed product or interface should also be easy to recover from failure.

81oCqPfe5wL._SX522_I recently bought a set of headphones. These were good headphones in most respects…until they broke at the complicated juncture where the ear pieces rotate. Once these headphones broke, there was really nothing I could do to fix them. But I decided to try – using a special putty that dries and holds things into place.

 

photoIt took a long time to figure out how to do this. When I finally repaired the broken part, I realized that the wires had been severed inside. There was no sound coming through. I had no choice but to put them into the garbage bin where they will contribute to the growing trash heap of humanity. Bad design is not just bad for consumers: it’s bad for the planet.

While most people (including myself) would claim that Audio Technica headphones are generally well-designed, we are usually not taking into account what happens when they break.

13687716887463pSometimes the breakdown is cognitive in nature. There’s a Keurig coffee machine at work. It uses visual symbols to tell the user what to do.

As I have pointed out in another article, visual languages are only useful to the extent that the user knows the language. And designers who use visual language need to understand that natural language includes how something behaves, and shows its internal states, not just what kinds of icons is displays on its surface.

The Keurig coffee machine is a nice specimen in many respects. But I had discovered that if I apply the necessary actions in the wrong order, it fails. Namely: if I add the little coffee supply and press down the top part before the water has finished heating up, it doesn’t allow me to brew the coffee.

So…after the water finished heating up, I saw the buttons light up. “Cool” – I said.

But nothing happened when I pressed a button to dispense the coffee. “WTF” – I said. Then I decided to open up the lid and close it again. That did the trick. The lights started blinking. But I was not satisfied with the solution. The discoverability of this bit of behavioral body language ranks low on my list.

Hint: “Blinking Lights” Means “You Can Press a Button”

I have to say, though: I have experienced worse examples of undiscoverability with appliances – especially appliances that are so simple, sleek, and elegant that they have no body language to speak of. This infuriates me to no end. It is not unlike the people I meet on occasion who exhibit almost no body language. It makes me squirm. I want to run away.

Now, thanks to YouTube and the interwebs in general, there are plenty of people who can help us get around these problems…such as this guy who has a solution to a related blinking light problem:

Screen Shot 2016-01-31 at 12.04.09 PM

I realize there are not many people who are bringing up this seemingly small problem. But I bring it up because it is just one of many examples of poor affordance in industrial design that are so small as to be imperceptible to the average user. However, the aggregate of these small imperceptible stumbles that occur throughout our lives constitutes a lowering of the quality of life. And they dull our sense of what good design should be about.

Tiny Rapid Failures and Tiny Rapid Recoveries

148159580_GeneralBicycleNow consider what happens when you ride a bicycle. When riding a bike, you may occasionally lose balance. But that balance can easily be recovered my shifting your weight, turning the wheel, or several other actions – many of which are unconscious to you.

Think of riding a bike as a high-rate of tiny failures with a high-rate of tiny recoveries.

Taken to the extreme: a bird who is standing on one leg has neuromuscular controls that are correcting the balance of the bird’s center of gravity at such a high rate and in such tiny amounts, that we don’t even notice it (and neither does the bird).

flamingo-on-one-leg

Natural Affordance: Perceived Signifiers

User interfaces (in computer software as well as in appliances) should use natural affordances whenever possible so that users can make a good guess as to whether something is about to fail, whether it is failing, how much it is failing, and how to recover.

The best design allows for rapid, and often unconscious correction while using the product. Good design is invisible!

Donald Norman brought some of these ideas to the fore when he wrote the Design of Everyday Things. We still have a lot to learn from his teachings.

Design is a way of life. If you design your life with resilience in mind – with the ability to recognize failures before they happen, and with the ability to recover from those failures, you will live a better life. The same goes for designing the technology that increasingly fills our lives.

The Body Language of a Happy Lizard

lizardhappy-dog-running-by-500px-600x350I love watching my dog greet us when we come home after being out of the house for several hours. His body language displays a mix of running in circles, panting, bobbing his head up and down, wagging his tail vigorously, wagging his body vigorously, yapping, yipping, barking, doing the down-dog, shaking off, and finally, jumping into our laps. All of this activity is followed by a lot of of licking.

There was a time not long ago when people routinely asked, “do animals have intelligence?” and “do animals have emotions?” People who are still asking whether animals have intelligence and emotions seriously need to go to a doctor to get their mirror neurons polished. We realize now that these are useless, pointless questions.

Deconstructing Intelligence

self-cars-300x190The change of heart about animal intelligence is not just because of results from animal research: it’s also due to a softening of the definition of intelligence. People now discuss artificial intelligence at the dinner table. We often hear ourselves saying things like “your computer wants you to change the filename”, or “self-driving cars in the future will have to be very intelligent”.

The concept of intelligence is working its way into so many non-human realms, both technological and animal. We talk about the “intelligence of nature”, the “wisdom of crowds”, and other attributions of intelligence that reside in places other than individual human skulls.

imgres-1

Can a Lizard Actually Be “Happy”? 

I want to say a few things about emotions.

The problem with asking questions like “can a lizard be happy?” is in the dependency of words, like “happy”, “sad”, and jealous”. It is futile to try to fit a complex dynamic of brain chemistry, neural firing, and semiosis between interacting animals into a box with a label on it. Researchers doing work on animal and human emotion should avoid using words for emotions. Just the idea of trying to capture something as visceral, somatic, and, um…wordless as an emotion in a single word is counterproductive. Can you even claim that you are feeling one emotion at a time? No: emotions ebb and flow, they overlap, they are fluid – ephemeral. Like memory itself, as soon as you start to study your own emotions, they change.

And besides; words for emotions differ among languages. While English may be the official language of science, it does not mean that its words for emotions are more accurate.

Alas…since I’m using words to write this article (!) I have to eat my words. I guess I would have to give the following answer the question, “can a lizard be happy?”

Yes. Kind of.

The thing is: it’s not as easy to detect a happy lizard as it is to detect a happy dog. Let’s compare these animals:

HUMAN        DOG         COW           BIRD         LIZARD         WORM

This list is roughly ordered by how similar the animal is to humans in terms of intelligent body language. Dogs share a great deal of the body language that we associate with emotions. Dogs are especially good at expressing shame. (Do cats feel less shame than dogs? They don’t appear to show it as much as dogs, but we shouldn’t immediately jump to conclusions because we can’t see it in terms of familiar body language signals).

3009107.largeOn the surface, a cow may appear placid and relaxed…in that characteristic bovine way. But an experienced veterinarian or rancher can easily detect a stressed-out cow. As we move farther away from humans in this list of animals, the body language cues become harder and harder to detect. In the simpler animals, do we even know if these emotions exist at all? Again…that may be the wrong question to ask.

happy-worm

It would be wrong of me to assume that there are no emotional signals being generated by an insect, just because I can’t see them.

ants communicating via touch

Ant body language is just not something I am familiar with. The more foreign the animal, the more difficult it is for us humans to attribute “intelligence” or “emotion” to it.

Zoosemiotics may help to disambiguate these problematic definitions, and place the gaze where it may be more productive.

I would conclude that we need to continue to remove those anthropocentric biases that have gotten in the way of science throughout our history.

8212f1d8d4ab1d159c6e0837439524c3When we have adequately removed those biases regarding intelligence and emotion, we may more easily see the rich signaling that goes on between all animals on this planet. We will begin to see more clearly a kind of super-intelligence that permeates the biosphere. Our paltry words will step aside to reveal a bigger vista.

Dinosaur_615I have never taken LSD or ayahuasca, but I’ve heard from those that have that they have seen this super-intelligence. Perhaps these chemicals are one way of removing that bias, and taking a peek at that which binds us with all of nature.

But short of using chemicals….I guess some good unbiased science, an open mind, and a lot of compassion for our non-human friends can help us see farther – to see beyond our own body language.