Bruce Schneier has written an interesting piece on how form follows function in any architecture, be it physical or electronic. My thanks to Kevin for pointing it out to me, and to Cory for making sure I didn’t forget about it …. I’ve been rushed off my feet lately….

Schneier’s arguments are simple, brought to life with eloquent examples and anecdotes:

• [Security-driven] changes were expensive. The problem is that architecture tends toward permanence, while security threats change much faster. Something that seemed a good idea when a building was designed might make little sense a century — or even a decade — later. But by then it’s hard to undo those architectural decisions.
• The same thing can be seen in cyberspace as well. In his book, Code and Other Laws of Cyberspace, Lawrence Lessig describes how decisions about technological infrastructure — the architecture of the internet — become embedded and then impracticable to change. Whether it’s technologies to prevent file copying, limit anonymity, record our digital habits for later investigation or reduce interoperability and strengthen monopoly positions, once technologies based on these security concerns become standard it will take decades to undo them.
• It’s dangerously shortsighted to make architectural decisions based on the threat of the moment without regard to the long-term consequences of those decisions.

All this made me think of the QWERTY keyboard. I grew up in a journalist family, with typewriters (the old Remington heavy-enought-to-give-you-a-hernia kind) outnumbering flowerpots at home. And when I first visited our printing press (I must have been nine at the time) I was quite surprised to see an ETAOIN SHRDLU keyboard. And my father explained to me that the QWERTY layout was designed to ensure that adjacent typebars didn’t jam, by separating high-frequency letters; that the layout had the additional “benefit” of slowing typing speeds down as a result. The linotype keyboard, on the other hand, was designed for speed, and therefore followed letter frequency distributions.

Form follows function. Just look how long QWERTY’s lasted. [An aside: It’s always amused me that the longest word you can form using the letters of the first line of the QWERTY keyboard is …. TYPEWRITER. What an unintended consequence. or was it? Maybe Grassy Knoll designed it]

We live in a world of many many cyber threats, some real, many perceived. I like the points that Schneier and Lessig make, particularly the pace-of-change one. There is always a temptation to take corrective action against security threats, both real and perceived; it is best to avoid that temptation altogether; but if we do give in, what we must ensure is that the corrective actions we take are designed to be as temporary as the threats; that we take care to make the response easily reversible, dismantlable, removable.

Imagine what would have happened if the recent ban on liquids on airplanes was enacted as law. Stupider things have been known to happen. In fact some part of me is actually surprised that the No Liquids rule didn’t become law.

Imagine what we’ve been doing to ourselves in building walls around our own information, within our own information. Actually paying people to build the walls, then paying people to drill through them, then paying people to fill the holes in…..

Now they know how many holes it takes to fill the Albert Hall.

Let’s keep those paths unpolluted.

For most of my adult life, I’ve been bemused, perplexed, sometimes irritated and occasionally completely taken aback by the error messages spewed out by the applications we build and use. Over the last twenty-five years or so, I’ve watched them improve, but at speeds that make glaciers look agile.

Today, while looking at technorati, I saw this:

Something is wrong! We know about it, and are working furiously to fix it. Please check back later and probably everything will be back up and running.

Great stuff. You may not think it’s perfect, given that it doesn’t actually say how long it will take to get fixed. But I like it. It is open, conversational, simple, honest and brief. I tend to think that the uncertainty implied in the message is actually a good thing; it made me think … could we expect to see error messages that are Agile in nature, improving by iteration as better information emerges/is discovered?

Who knows, this may be a leading indicator of the paradigm shift taking place in the applications and services space today.
My thanks to all at Technorati.

James Governor pointed me at this Kathy Sierra post before I’d trundled my way there (Thanks, James!).

The picture below says it all:

Kathy’s one of those people who creates extreme reactions amongst her readers, and I’ve seen raging arguments about some of her earlier posts. I can’t help but feel that at least some of the reactions are because she touches raw nerves. Which is why I love reading her stuff.
Three quotes stand out for me:

“If that person shakes us up, gets us to rethink, creates a little tension, well that’s a Good Thing”, the CEO says. riiiiiiiiiight. While I believe most CEOs do think this way, wow, that attitude reverses itself quite dramatically the futher you reach down the org chart. There’s a canyon-sized gap between what company heads say they want (brave, bold, innovative) and what their own middle management seems to prefer (yes-men, worker bees, team players). “

The management-middle management gap/reversion is something that has been commented on in depth before, and is by itself nothing new. What makes it new is that we have three new(ish, anyway) factors acting on the enterprise: a real war for talent; a real move from hierarchy to network; a real battle between professions as historical lines continue to blur. On to quote 2:

Of course some argue that exuberance on the job is not necessarily a good thing. That too much passion leads to problems. I say BS on that one. Real passion means you love the profession, the craft, the domain you’re in.

And I guess this is where the problem becomes more acute, as Abbott’s System of Professions evolves into its not-so-subtle conflicts. No single profession has an inalienable right to passionate people; middle management tends to be full of “professional” people; the passion for their profession creates considerable conflict, and can result in their looking for robotic slaves out of sheer frustration. And so to quote 3:

If you knock out exuberance, you knock out curiosity, and curiosity is the single most important attribute in a world that requires continuous learning and unlearning just to keep up. If we knock out their exuberance, we’ve also killed their desire to learn, grow, adapt, innovate, and care. So why do we do it?

Maybe we do it because we can’t find a way out of the professional conflicts and tensions Abbott refers to.
All is not lost. Kathy herself provides the answer, as much in her blog as in her post. Create passionate users.
We need to find a way to unite the professions. And there is a way. We’re just not very good at it, and keep getting in our own way.

Don’t focus on the profession. Don’t even focus on the firm. Focus on the customer. Focus on the customer. Focus on the customer.

A firm that unites around the customer unites all it does.  And becomes a formidable force.

My two previous posts, looking at the reasons for deviance between management and engineer views of projects, attracted a number of comments, primarily in two camps. One camp spoke of Management-in-Denial, going into the reasons and contexts why an enterprise would behave that way, travelling into shareholder-versus-customer expectations along the way. A second camp looked orthogonally at the same shareholder-versus-customer contention, suggesting that the expectations diverged right at the start, and that we need to get better at reducing that initial divergence by concentrating on pragmatic realities. The arguments then morphed into a classic marketing-versus-engineering debate, what we used to call vapourware or markitecture or, more recently, software-by-powerpoint.

It is hard to summarise a series of comments, you can’t edit snowballs. My apologies to any and all who feel I’ve misrepresented the arguments. If you want to know what they really said, read the comments.

I get the feeling that both camps concentrate on what happens after fan and excrement are in simultaneous play, rather than in the prevention of said occurrence.

I’ve had to manage projects in some form or shape for a couple of decades now, and so far nobody in management has ever asked me to lie, even if I’ve been that “management”. Sometimes I’ve been unhappy with the techniques used to present “reality”. Sometimes I’ve been unhappy with late changes to schedules and deliveries presented to me as a fait accompli. And sometimes we’ve just got it wrong. But it hasn’t been about lying or being in denial.

How can we get better? Here are some personal musings:

1. Let the deadline influence the design
When you ask someone to do something, one of the design criteria, an important one, is time. Once you agree on time, you can negotiate the functionality and performance characteristics of the deliverable. Often you have to negotiate with a set of “incomplete contracts” but that’s life. Emotion can be taken out of the negotiating process; agile techniques can be used to learn more about detail aspects of functions and performance; there is a continuous improvement process explicit in all this, and sometimes it results in a continuous migration process, when you have yesterday, today and tomorrow systems in parallel. Time-boxing and time-placing techniques are based on this, yet for some reason get little airplay.

How come everyone got ready for things like EMU and Year 2000 and Sarbanes-Oxley and all that jazz? Because there were two projects in each case. One did the paperwork. One did the real work. The ones that did the real work knew that the dates couldn’t change, so they negotiated on the What and the How rather than the By When. Which brings me to my second point.

2. Get the signal-to-noise ratio right

More than once, I’ve been involved in projects where the governance mechanism is larger than the project team, and where the pressure placed by such mechanisms creates more paperwork than code. This does not create a divergence between management and engineer; it’s fairer to say there are two projects, one producing paper and one producing code. And sometimes there is only one team having to do both, with predictable results.

As long as we have an Armaggedon approaching for the battle between professions, and as long as organisations haven’t completed their painful trip from hierarchy to network, we’re going to have intermediaries. And there is a price to pay for having them. The only solutions I can see are twofold: One, try and make the doer-to-checker ratio an integral part of the reporting pack, so that the cost becomes visible. Two, use the remaining checkers wisely, they can help in many ways. Which brings me to my third point.

3. Pass ownership to the real customer as soon as possible

We use terms like fast iteration, early testing, agile development, fail-fast, whatever. Hitherto I’d been worried about the tension between such approaches and the classical waterfall or cascade approaches. But the more I think about it, the more I feel that the approach-conflict is but a symptom. The root cause is a blame culture. Which sadly pervades too many organisations. So what we need to do is to find ways of reducing the impact of this blame culture. As far as I can make out, the best (perhaps the only) solution is to place something that works in the hands of the real customer, that can be used to create or derive value. Once you do that, all the dynamics change. Even if the value is marginal to begin with. Which brings me to my final point.

4. Get legacy skin into the game as soon as possible

Don’t talk parallel run. Don’t talk big bang switchover. Carry a big stick, move low-volatility information from Old to New as soon as humanly possible. Expose what you’ve done to the customer, and soon the old will atrophy and the new will thrive. [Note: There is an exception to this, an important one. Sometimes you get involved in building systems that are expected to do precisely what the ones they are meant to replace do in the first place. To the letter. No point getting legacy skin into this game, the customer doesn’t want the system. Get out quick.]

John Dodds, Lars Plougmann and Stephen Smoliar (sorry Stephen, haven’t found a simple way to link to your blog) were kind enough to share their views with me following my earlier post on the subject. There I quoted from Richard Feynman’sRogers Commission looking into the Challenger disaster. submission to the

The comments touched on three aspects of the communication problem. John felt that we need to focus on “accentuating the negative” in order to avoid everyone building castles of sand in the air, suggesting that both “clients” as well as “IT people” have a problem with understanding what is to be built and what can be built. Lars inferred that project managers wilt under pressure and agree to unachievable timescales. Stephen felt that the willingness for self-deceit came from a postmodern “withdrawal into fictions”.

As with many of these things, I have some sympathy with all three views. I posted what I posted primarily to learn more about the issue. Let me try and take the argument further, quoting again from Feynman’s submission:

Official management, on the other hand, claims to believe the
probability of failure is a thousand times less. One reason for this
may be an attempt to assure the government of NASA perfection and
success in order to ensure the supply of funds. The other may be that
they sincerely believed it to be true, demonstrating an almost
incredible lack of communication between themselves and their working
engineers.

In my previous post, I was trying to understand the root cause of the deviance between the “manager” view and the “engineer” view, why and where the management self-deceit was happening. My gut feel is that it has to do with biases we bring in, biases related to incentives, measurement, even the tools we use for managing projects. Let me try and explain.

When John raised the question of accentuating the negative, he was trying to ensure that people build things that can be built, that solve a real problem, that allow value to be delivered in small measurable increments. Of course I agree with all this. But I have a nagging worry that we have tried this and failed. When we accentuate the negative, quite often we seek to solve yesterday’s problem. Sometimes that is the right thing to do. Too often it is “paving the cowpaths”. Even if it makes me sound more idealistic than I want to sound, I feel driven towards avoiding this accentuation of the negative. Yes we should build things that can be built. Yes we should try and deliver value in small increments. But not necessarily by focusing on the negatives.

Similarly, of course I agree with with what Lars says. There is no dearth of project managers accepting unachievable deadlines as a result of some external pressure. But then there is also no dearth of project managers failing to deliver without any such pressure…. So where does it go wrong? How does the management view start deviating so much from the engineer view? This is not just about schedules, it’s about functionality and performance as well. When the rot sets in, it affects many things. And maybe Stephen’s right, but then I have to give up hope for a better way of doing things, and this I am not yet prepared to do.

In a previous post I have already spoken of waterfall/cascade development styles versus agile styles, and accepted that there are horses for courses rather than just one sledgehammer way. So maybe I need to understand all this within the context of the two styles.

In a waterfall style project, most of what we need to know is known from the start. At least that’s what the pro-waterfallers argue. In which case bad estimates are primarily due to bad estimation, either by the engineers themselves or by those who “edit” the engineering view en route to management. Bad execution is due to bad planning, bad quality is due to bad testing, and so on and so forth. Waterfalls are good when there is nothing to discover, nothing to learn, so we have to look for the faults elsewhere. They are not in the definition of the problem, but caused by corruptions to the original data relating to effort and timeline and functionality. Or corruptions to the resources applied, in terms of availability and skillset. Or corruptions to the processes applied, in terms of quality of test cases, the testing process itself, and often the quality of end-to-end-testing.

In an agile project the story is different. There is much that is unknown, and there is a considerable risk that corruption can occur between the engineer and management views. Engineers make best guesses based on the information they have, and quite often management, without a deep understanding of the “guess” aspects, convert them into hard deadlines and functionality and cost. And then we have a period of House-Of-Cards-meets-Emperor’s-New-Clothes, until we have a Yeatsian Second Coming:

Things fall apart; the centre cannot hold;

Mere anarchy is loosed upon the world,

The blood-dimmed tide is loosed, and everywhere

The ceremony of innocence is drowned;

The best lack all conviction, while the worst

Are full of passionate intensity.

When this happens (and it happens often enough) we have a traditional wailing and gnashing of teeth, and blame cultures revel in firing some and promoting others. Such is life. But rarely is all this done as sins of commission, as malevolent acts. More often than not, this road to hell, like most others, is paved with good intentions. I think Feynman’s right on both his points quoted above. Often there is “an incredible lack of communication between [management] and their working engineers”. Often this is to “assure perfection and success …[…]… to ensure the supply of funds”. And this is where I think the incentive bias, the measurement bias and the corruptions caused by the project management tools come in. And I think we do have new ways of working, new tools to use, that can get around these biases.

To paraphrase Einstein, engineers don’t fail, they find ways that don’t work. And then they find ways that do work. In bits. And then they have to test the whole thing, end-to-end, to ensure that the bits work together, to create the experience that the customer wants to have and own. The opensource community can teach us a lot about testing the bits and testing the end-to-end. Given enough eyeballs the bugs are shallow. Web 2.0 companies can teach us a lot about ensuring that the bits work together to provide the customer with the requisite experience. When they say Beta they mean “It works, but we’re still ironing stuff out”. They don’t ship until it works, and when they ship, they listen to customers who do the real end-to-end tests, who exercise the code and at the same time suggest ways to improve the overall experience. These approaches, covering community development and real customer functionally-integral Beta releases, they’re often intertwined. And they tend to rely on social software to improve communications throughout the process.

Success in project delivery takes place, as Kathy Sierra suggested, when the customer says “My software isn’t working” rather than “Your software isn’t working”. This emotional shift, when the customer takes ownership of the output, is absolutely critical. And it doesn’t take place unless there is something for the customer to experience in totality. Nobody takes a car to a garage and says “Your car isn’t working”. We have to let the customer experience the product as quickly as possible, so that we can pick up the bugs and deficiencies that cloud that experience. This requires agile mindsets, community development approaches, a release-quick-and-release-often schedule, good social-software based communications processes, and a willingness to ship Beta.

But all this is difficult to describe, plan for, estimate and monitor using traditional tools and methodologies. Agile development is likely to become more common as we continue to deal with uncertainties. Unless we solve this, we are likely to see increasing divergence between engineering views and those of management, as waterfall approaches get used to govern agile practices.