Sunday, April 30, 2006

Failure - Part 2 - Strategy

Sherlock Holmes, the omniscient, begat Dashiell Hammett whose grandchildren solve problems simply by being themselves, often clueless, but of the right social order. Once the villain could be anyone in the reader’s closest social set; now it is the butler recast as the outsider, the parvenu, the upstart who threatens the status quo with guile.

The cultural transition from the hope for discovering the solution to the pragmatic acceptance of the possible is reflected in data processing. When I began in the 1970s, programmers were supposed to collect all the requirements for a project, then implement them. Every contingency was supposed to be considered. Now, companies buy packages, maybe with consultant services, and bumble through.

The goal of the perfect application written by employees was well neigh impossible, and often produced systems that were far more complicated than necessary. One critical problem was that one person was supposed to talk to users about their needs; design, write and test code; write documentation and do training.

An equally serious problem was that it assumed users could articulate what they wanted. Since many found it difficult to extract the general from the day-to-day, one either found people who talked grand rhetoric with no details, or found people could not be found. Words like "blue sky" and "bells and whistles" became common, and were to be avoided.

Programmers were either trained in math departments or community colleges. Both focused on teaching how to code, the one with more emphasis on theory, elegance and efficiency. Few taught students to do user documentation or training. And certainly, no one offered classes in interviewing techniques, not even for anthropologists or social workers.

Companies began to talk about how to manage failure. At my last job, our customer sent us a video that told us more than 80% of the information systems projects were never completed. The rest weren’t what was needed, came in late and/or were over budget. I don’t remember if the sample was government projects or projects in general, local or national in scope.

The perception that managers were handling failure prone projects caused by creative individuals like Holmes has persisted. My last supervisor loved to tell me the problem for GM was that engineers would spend too much money creating the perfect vehicle if left alone. It was the duty of a good manager to keep them under control. He, of course, had never worked in Detroit, and was passing on received wisdom. The memorat probably goes back to Frederick Donner who was chairman in the 1960s.

In that last shop, my supervisor’s managers believed a new release of an application should be installed exactly as it already existed. Then, in a later phase, users could request some of the added features. He couldn’t make them understand that not planning for those changes meant implementation decisions were made that made some enhancements impossible.

Users were unhappy in the 1970s and are still unhappy, but the costs in the last shop were exactly as predicted. Higher level managers who look at budgets were happy, even as they blamed their subordinates who couldn’t produce desired reports. The costs for the future change were the problem of the next manager. A new phrase appeared, "foul up, move up."

In the planning phase, my supervisor created a set of specifications which I was supposed to read to the users. He modeled it on the list of requirements provided by the purchased package he hoped would be chosen. He told me I could not add anything, even if it was what someone needed, because the salesman would get too creative and use any modifications as an excuse to raise the price.

I persisted in the old-fashioned methods and asked the man who ran the warehouses how he wanted to run his operation. I told him, nothing he wanted was likely to be in the application, but if we didn’t know what he wanted, it would be difficult to plan a future transition path. My underlying purpose was to get him to talk about his department in concrete terms so he would pay attention to the predigested specification. I was taken off the project.

Once the application was installed, the inventory manager wanted to know why he couldn’t change his costing method, and discovered what he wanted was not available. The already contracted vendor was more than happy to quote a price to write the code, and he somehow managed to convince whoever controlled the budget to sign the contract addendum. Then he left before it was available. The supervisor had already gone, and the IS managers had been replaced.

Failure was transformed from a problem into a marketing strategy. The software company would charge us the full cost of development. If other companies asked for the same thing, they could make minor modifications and recharge them for development costs. If the modifications were useful they could introduce it in a future release with no development costs.

Failure became an ad hoc means of defining user requirements. Programmers were told to convert all existing reports, but not find out if they were used.. The assumption was that if something was important, users would tell them when it was missing, and that would set the priorities.

Needless to say, report problems were never discovered until they were needed, and the programmers lived in constant crisis. When they stayed late to get something ready for the next morning, the success of the department was highly visible. Managers hoped their responsiveness would disguise their failure to plan; if not, replacing a programmer would show their willingness to improve.

All that happened in the years between the time I worked for Shatterproof and my last shop is expectations for programmers were lowered, and, hopefully expectations by users were lowered to an equal level. Once the goal was set at the attainable, success was guaranteed, especially if the initial costs were kept within budget.

No comments:

Post a Comment