This post is not just for software developers. It is intended for a wider readership; we all encounter complexity in life, and we all strive to achieve goals, to grow, to become more resilient, and to be more efficient in our work.
Here’s what I’m finding: when I am dealing with a complicated software problem – a problem that has a lot of moving parts – many dimensions, I can easily get overwhelmed with the complexity. Most programmers have this experience when a wicked problem arises, or when a nasty bug is found that requires delving into unknown territories or parts of the code that you’d rather just forget about.
Dealing with complexity is a fact of life in general. What’s a good rule of thumb?
Externalize
We can only hold so many variables in our minds at once. I have heard figures like “about 7”. But of course, this begs the question of what a “thing” is. Let’s just say that there are only so many threads of a conversation, only so many computer variables, only so many aspects to a system that can be held in the mind at once. It’s like juggling.
Most of us are not circus clowns.
Externalizing is a way of taking parts of a problem that you are working on and manifesting them in some physical place outside of your mind. This exercise can free the mind to explore a few variables at a time…and not drop all the balls.
Dude, Your Brain is Too Big
I have met several programmers in my career who have an uncanny ability to hold many variables in their heads at once. These guys are amazing. And they deserve all the respect that is often given to them. But here’s the problem:
People who can hold many things in their minds at once can write code, think about code, and refactor code in a way that most of us mortals could never do. While these people should be admired, they should not set the standard for how programming should be done. Their uncanny genius does not equate with good engineering.
This subject is touched-upon in this article by Levi Notik:
who says:
A common narrative is that these freak geniuses are “ideal for the job of programming”. In some cases, this may be true. But software has a tendency to become complex in the same way that rain water has a tendency to flow into rivers and eventually into the ocean. People with a high tolerance for complexity or a savant-like ability to hold many things in their minds are not (I contend) the agents of good software design.
I propose that people who cannot hold more than a few variables in their minds at once have something very valuable to contribute to the profession. We (and I’m taking about those of us with normal brains…but who are very resourceful) have built a lifetime’s worth of tools (mental, procedural, and physical) that allow us to build complexity – without limit – that has lasting value, and which other professionals can use. It’s about building robust tools that can outlive our brains – which gradually replace memory with wisdom.
My Fun Fun Fun Job Interview
I remember being in a job interview many years ago. The guy interviewing me was a young cocky brogrammer who was determined to show me how amazingly clever and cocky he could be, and to determine how amazingly clever and cocky I was willing to be. He asked me how I would write a certain algorithm (doesn’t matter what it was – your typical low-level routine).
Well, I was stumped. I had nothing. I knew I had written one of these algorithms before but I couldn’t for the life of me remember how I did it.
Why could I not remember how I had written the algorithm?
Because I did such a good job at writing it, testing it, and optimizing it, that I was able to wrap it up in a bow, tuck it away in a toolbox, and use it for eternity – and NEVER THINK ABOUT HOW IT WORKED ANY MORE.
Hello.
“Forgetting” is not only a trick we use to un-clutter our lives – it actually allows us to build complex, useful things.
Memory is way too precious to be cluttered with nuts and bolts.
Consider a 25-year-old brogrammer who relies on his quick wit and multitaskery. He will not be the same brogrammer 25 years later. His nimble facility for details will gradually give way to wisdom – or at least one would hope.
I personally think it is tragic that programming is a profession dominated by young men with athletic synapses. (At least that is the case here in the San Francisco Bay area). The brains of these guys do not represent the brains of most of the people who use software.
Over time, the tools of software development will – out of necessity – rely less and less on athletic synapses and clever juggling, and more on plain good design.
At the right is a graph I drew which shows how an Alife simulation (or any emergent system) creates novelty, creativity, adaptation, and emergent behavior. This emergence grows out of the base level inputs into the system. At the bottom are atoms, molecules, and bio-chemistry. Simulated 
Earth...Brains...Technology...Design 