Trying to Think Like Fred Brooks

1. Ordinary Genius

Dr. Brooks is by many regarded as what may be called an "ordinary genius".

"There are two kinds of geniuses: the 'ordinary' and the 'magicians'. An ordinary genius is a fellow whom you and I would be just as good as, if we were only many times better. There is no mystery as to how his mind works. Once we understand what they've done, we feel certain that we, too, could have done it. It is different with the magicians. Even after we understand what they have done it is completely dark. Richard Feynman is a magician of the highest calibre." - Mark Kac.

Feynman Diagrams are sometimes given as an example of a product of magician genius. Instead of performing difficult mathematical derivations in Quantum Field Theory to calculate infinite particle interactions, one draws Feynman Diagrams and then simply writes down the answer:

The problem that Feynman Diagrams make easy to solve is calculating infinite numbers of quantum paths. The trail leads through the Feynman path integral formulation, the principle of least action, and then the probability amplitudes, some of which cancel out, but infinitely many do not. Worse, infinitely many things may happen along these quantum paths, posing a calculation problem. Feynman Diagrams are introduced as doodles that tame the infinities, with details how in the videos following this one. Video: Problem statement: "Feynman's Infinite Quantum Paths", PBS Space Time series.
To a first approximation of understanding Feynman Diagrams one draws the top few levels of an infinitely-deep tree of Feynman Diagrams, one labels components of the diagrams with appropriate simple terms, and then one combines the terms to form complex equations. Video: Beginner level: "Solving the Impossible in Quantum Field Theory", PBS Space Time series.
To a second approximation of understanding Feynman Diagrams six different particle interaction scenarios are covered by just one Feynman Diagram fragment depending on how it is rotated, and with that understanding one may be able to complete a homework problem given at the end of the video. Video: Intermediate level: "The Secrets of Feynman Diagrams", PBS Space Time series.
The description in Feynman Diagrams of antimatter as ordinary matter traveling backwards in time originated from John Wheeler's One-Electron Universe idea, which never worked out in whole, but within it the time reversal idea was sound and picked up by Feynman for his Feynman Diagrams. The video that discusses this time reversal idea, Backwards time: "One-Electron Universe", PBS Space Time series, also gives the answer to the homework problem from step 2 above, except that the answer here to the homework problem is incomplete, missing 2 diagrams. Video: Extraterrestrial Superstorms, gives the missing 2 diagrams at time 13:12.

Returning to the ordinary genius as described in the quote above, "There is no mystery as to how his mind works." In some cases perhaps. Mystery is in surplus in the case of Dr. Brooks. Dr. Brooks's ordinary genius of insight and decision making seem to be simple, common sense, the essence, something that any one of us could have thought of. But we didn't, and figuring out why is vexatious. The ordinary genius is no less worthy of study than the magician genius, perhaps more so.

Of all the geniuses that live among us, none fascinate more than the ordinary geniuses, because they transform unexpectedly from the familiar into the wise. For those, one seeks nuggets of insight that can magically lay the mystery to rest. The familiar difficult decision, at least as seen by those unschooled in righteous judgment, has something of this character; the question is usually innocent and straightforward, but is capable of becoming a monster of alternatives missed or of uncertain value, as to yield only to exceptional insight. Yes, I adapted this paragraph from that other one ....

"Of all the monsters that fill the nightmares of our folklore, none terrify more than werewolves, because they transform unexpectedly from the familiar into horrors. For these, one seeks bullets of silver that can magically lay them to rest. The familiar software project, at least as seen by the nontechnical manager, has something of this character; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products." [Brooks1987]

Some have tried to lay this mystery to rest. On two occasions a graduate student walked up and announced something like, "I want to be like Dr. Brooks. I'm going to try to be like Dr. Brooks.", and then walked away without me saying anything. Why did they say this and why to me? Weirds me out. But I did watch them, and, over time, no change. Lets us try now.

2. Heavy Cross, Times Two

Presented before the faculty was a matter for decision. It went back and forth, but alternatives seemed equal. Then, one faculty member spoke (approximate quote): "Let's wait for Dr. Brooks to get back in town. When he hears this, he will say something. We don't know what he will say, but it will make the correct choice clear and obvious to all of us." Nods all around. Grown men looking up to Dr. Brooks as if as children. I noticed especially, "We don't know what he will say," There is no Brooks-like decision. Wisdom seems to not have a pattern.

Consequences:

Being the smartest person around might be a comfortable place. One's genius may be called upon, expected, demanded to be performed on command.

It may too be lonely, surrounded by all who do not get it and must seem to have little common sense.

Each must be a heavy cross to bear, even if cheerfully.

3. Fred Brooks Think-alike Contest

Round 1

Dr. Steve Weiss was up for his tenure decision. He had asked not teach numerical analysis, because he did not know it and intensely hated it. Other than that, he was, some say, the best teacher (besides Dr. Brooks) in the department. Some students chose their electives only by his teaching schedule. The tenure rules were that a candidate must score "medium" or "high" on both teaching and on research. The man scored "high" on teaching but "low" on research. The rules are not flexible.

Please pause to think of what you might do.

A lesser man might have denied tenure. But Dr. Brooks considered the purpose, the essence, of the rule, that the rule existed to weed out the great teacher of today, who in a few years would still be a great teacher, but teaching things that nobody cared about anymore. Good research can prevent this. But Dr. Brooks hewed to the essence of the rule and assigned the candidate the task of teaching the hated numerical analysis, and by all accounts, with struggle and anguish, the candidate did a good job. He could learn anything. Tenure approved. And righteously so.

Dr. Brooks did not give the candidate any special breaks really. He had to meet at its essence the same requirement as did anyone else, and in so doing not a single person from any point of view could be dissatisfied.

Round 2

A terminal Master's student, who was terminal because he had flunked not just out of the degree program but out of graduate school itself and then re-admitted as terminal, nevertheless wrote a Master's thesis that read to Dr. Brooks, and to at least one other member of the Master's committee, more like a dissertation, and the student even called it so.

Please pause to think of what you might do.

A lesser man might have just signed-off the student and closed the book on him, as it were. Dr. Brooks called a meeting with the student acknowledging to the student that the purpose of the Ph.D. course work additional beyond the Masters, of the Ph.D. Written, and of the Ph.D. Oral was to prepare the student to pursue a dissertation, and that the student having written a dissertation it made no sense to require them on that basis. Nevertheless, then, Dr, Brooks proposed to go forward on two principles (approximate quote): (A) "I don't want anyone to say that Jim Lipscomb got away with something, so you are going to have to pass the extra courses, the Written, and the Oral just like everyone else, and (B) if you don't, then we shall not have another conversation." Times being what they were, I accepted the job. And I was righteously tasked.

A lesser man might have been cowed by considerations of personal or professional risk in sticking one's neck out for a student whose subsequent failure, should that happen, might raise questions as to the professional way to move students along. But for Dr. Brooks such thinking is what to set aside on the way to the essence, and then at the essence such issues are moot. Questions could be answered.

Dr. Brooks did not give me any special breaks really. I had to meet at its essence the same requirement as did anyone else, and in so doing not a single person from any point of view could be dissatisfied.

4. No Way as a Way of Rational Thinking

Fred Brooks's thinking appears to be driven rationally in a specific, technical sense. I suppose that we all think ourselves rational enough, at least at work. But there is a discipline of Rational Thinking whose proponents go beyond the usual. They seek the essence, following no set plan.

I regret that I cannot locate either the video I saw about Rational Thinking or the few books about it. So, for an example I shall call upon the highest available authority, if one may excuse the topic, Bruce Lee and his No Way system of fighting.

Bruce Lee, besides being a star of martial arts movies, was also a serious student of the martial arts. He came from a tough neighborhood, in contests where to determine the winner a referee was not necessary. In the martial arts there are different Arts and Ways, the Way of the Crane for example. Each is a complete, consistent system. Bruce Lee studied these Ways with an eye towards which might be best. He saw that none would win a street fight, because there the one who delivered the first blow was usually the winner. He therefore created his own Way, the "Way of the Intercepting Fist", featuring the most rapid, direct attack. An example of the opposite of the Way of the Intercepting Fist may be found in the Judo Punch, in which the fist is first cocked at the waist before attempting to launch the punch - after the fight is over.

But then Bruce Lee realized that his Way of the Intercepting Fist, the shortest, most direct attack, could not always be best. Two steps sometimes really are better, as exemplified by go no sen karate, which employs dodging and faking to sucker the opponent out of position, opening a vulnerability.

Bruce Lee's conclusion was that the best Way is "No Way", that rather than following a set plan or set teaching one must flow like water, not in the meek sense, but rapidly, vigorously, and forcefully to instantly take the shape of the container (the situation). No Way is showcased in his unfinished movie, Game of Death (Wikipedia, YouTube), in which he climbs the stairs of a pagoda, defeating at each level a master of a particular Way by exploiting in weaknesses in their Ways, arriving at the top to battle in No Way a master at the highest level who also fights in No Way.

It takes a double dose of rational thinking in which one follows no set plan or set teaching but rather the imperative essence of the moment, first to come up with the idea of No Way, and then to fight using it. Still though, learning set plans and teachings of the Ways may be important so that in the moment one may select bits and pieces from these practiced Ways to act quickly and more appropriately, rather than pausing to make everything up from nothing on the spot.

In waters seemingly calmer Dr. Brooks nevertheless flows rapidly, vigorously, and forcefully to fill the container (the imperative, the essence) of each situation, apparently following no set plan. Indeed, recall from the Heavy Cross section above, "We don't know what he will say, ....", meaning that the speaker saw no predictable pattern to Dr. Brooks's decisions. Decisions come from somewhere else, not from a pattern. Dr. Brooks also dismisses distractions (previous section) that can dominate the thinking of the average. Dr. Brooks seems to solve problems in No Way, his solutions sometimes novel, always fitting the situation. But perhaps we may discern a little of a system from examples.

5. No Extraneous Agenda

In shaping decisions to fit the container, not any container will do. Willpower as well as reason are required to find and to fill the one container of the essence of the problem.

It is often considered wise to take everything into account when making a decision. Wrong, as least to the point that some items must be identified as personal or professional agendas or considerations, and then brushed off the table. To think rationally is to leave nothing left to consider but the essence of the thing. But ignoring the distractions is hard. They scream importance.

Starting late, finishing first

Perhaps a dozen chemistry labs around the world had already started on building interactive molecular graphics systems to enable crystallographers to manually assemble bits of large protein structure models into views of experimental electron-density data and thereby build a model of the protein with full atomic detail. Dr. Brooks decided, with the coaxing of a university Provost (or Dean?), that he too would assemble a team to build such a molecular graphics system, GRIP-75. According to the crystallographers who used it, it was the first to succeed, documented in their published paper at the time. There is also a recent undocumented claim of priority made for Stan Swanson's FIT program.

At the other labs the thinking, I was told, was as follows, with extraneous agendas at every point:

The electron density data, although large, was uniform and could be broken down into smaller cubes to work with sequentially. The fitting of the molecular structures to this data could be done only in a series of small scale manipulations anyway, so the whole process could be fit into a 12-bit or 16-bit minicomputer of the day, the DEC PDP-11 or equivalent. One tiny computer could do much of what was needed, in the convenience of the laboratory setting.
The labs hated spending money at the large university computing centers, money they saw leaving the lab for elsewhere, which also onerously forced researchers to budget how computing on a project-by-project and run-by-run basis. Such money is hard to find in the compartmentalized structure of grants, hardly matching the natural, veering course of research. If the lab had its own minicomputer, it could compute, at min-computer speed and size of course, for the project needs of the moment. Building a protein structure fitting system was seen as a way to get funding for a lab minicomputer, to be used principally for other computation.
The slow displays at the time made it debatable that enough of a scene to allow for work could be drawn, but by zooming in tightly on the data the researchers hoped that the fewer lines to display might possibly do. There was one exception: At Washington University at St. Louis a team did recognize that displays of the day were insufficient, and so they undertook to build one good enough, but they failed to recognize first, that industry would quickly outpace them and second, that molecular-specific functions they thought were necessary (analog adjustment the level of electron density contouring) were unnecessary. They filled the wrong container and so were years late in their success.
Only a graduate student or two was assigned to write code, the graphics system being secondary to the primary purpose of obtaining a minicomputer for lab computations. To succeed this student need only be proficient in all skills.

Extraneous agendas have the seductive power of a siren song, drawing the ship up on the rocks, solving problems additional to the problem at hand, and thereby solving the problem at hand badly, which is to say that some of these efforts did succeed, but years late.

By contrast Dr. Brooks's mind travels light, without the baggage of extraneous agendas, shaping decisions to fit the containers of the essences of the problem at hand:

He evaluated that the large electron density data was inherently a 24-bit size issue, which would be best addressed by a 24-bit computer, which would have to be the one at the computer center. He likens the attempts of the other labs to try to squeeze this 24-bit problem onto a 16-bit computer to a hurdle on a circular racetrack. One can be off to a quick start and be almost up to speed when the hurdle appears and one must stop to clamber over it, and then quickly be off again. But as this racetrack is circular, before one is back up to speed what soon appears in front yet again is the very same hurdle, which one must stop to climb over again, and again, the racer forever slow. So, Dr. Brooks made arrangements with the computer center for the 24-bit part of the problem and for them to link to a lab minicomputer for the 16-bit part of the problem, the user joystick interaction.
He solved the problem of cost per hour at the university computer center by buying a bank of memory dedicated to this project for the university computer, but also partly available for the computer center's use, in exchange for no hourly charge.
He evaluated the drawing speed required for the display and held to that spec, even at one point returning to the manufacturer a display that did not meet the spec. To fit a graphics rendering problem of a certain size onto a slower display would have put the project on a circular racetrack similar to the one above.
He set up a software development structure staffed to critical mass and based on three best practices:

Surgical team: Building on a graphics system built by one student (W. Wright), three "surgeons" (E. Britton, J. Lipscomb, M. Pique) wrote the new application program and only that, each responsible for a specialty. The small number of surgeons, still larger than the entire team at other labs, maximized conceptual integrity and minimized training and communications. Working in the days of punched cards, development at its peak was done around the clock with each surgeon doing an 8-hour shift with the card deck. Overnight shift for me.

Surrounding the surgeons was the surgical support team, which built application program add-ons (W. Siddall, D. Tolle, J. Hermans, J. McQueen, S. Wei) a machine-to-machine communications system (P.J. Kilpatrick, W. Kerr), an extended-memory driver (S. Bellovin, D. Tolle, R. Motely), a graphics language (G. Hamlin, R. Hogan, P. Mullen), a joystick analog-to-digital driver (R. Hogan), command line interface to the mainframe (D. Kehs), hardware support (P. Nichols, J. Ross, P. Reintjes), and even a compiler (D. Kehs, T. Dunnigan) for a subset of PL/I that focused the surgeon's work by letting them program in the same language on the two computers. There was also a software library and data library management team (R. Motley, T. Dineen, W. Siddall, J. Crawford, L. Brown, G. Kennedy, L. Nackman, T. Williams). At any given time the support team was about 1/5 this large, its members being brought on temporarily for specific jobs.

Iterative development: Cycles of plan-implement-test-evaluate with collaborative users (S. Kim, J. Sussman, J. Richardson, D. Richardson, J Hermans).

Egoless programming: Well, a form of it anyway. Each of the three surgeons co-piloted the other two, even to the extent that there was an informal majority-rules discipline that was invoked when the lead surgeon was sure that the wrong plan was right. In one case two surgeons waited for when the third was out of town and changed his code. Obvious that it was better there was no going back. In the other case the lead surgeon was convinced to at least try the alternative he did not like. Users loved it, so no going back then either. The co-piloting part of this comes from Dr. Brooks's surgical team plan combined with his pressure-cooker plan (throw them together, clamp down the lid, and turn up the heat). The majority rule part is something that the surgeons worked out themselves.

Willpower

Such focus is not easy or cheap. Willpower is in abundance in each of the four main decisions above. Those decisions greatly increased the effort, expense, and social challenges, as well as delaying the project start by extra years beyond the original years-late start. In no other lab was there the willpower to do even one of them, except for the troubled effort at Washington University at St. Louis as described above.

Given the willpower to make these four decisions, they sped the date of accomplishment with Dr. Brooks's team finishing first, as documented above.

6. Best Among Alternatives

Dr. Brooks reliably chooses the best among alternatives. An example:

We help our friends, but do we really? In my time in the department one professor was noted for getting his Masters students through in good time. He helped them. He ensured that there was a schedule, had regular meetings, milestones, and shepparded the students through hands-on. But there is the echo here of the helicopter parent who props up the child into overachievement. Our friends too, if they see that we have chosen a path into trouble, will rush to help. But when we help someone to be better than they are, then they are someone that they are not. Oversimplifying for easy reference: We are their greatest friends, and by being so are not their friends.

By way of contrast, I heard Dr. Brooks say that if a student is sinking and is too stupid to call out for a line, then he does not deserve to graduate from this university. Dr. Brooks will help one down the path one has chosen, and if that is to silently sink out of sight, then he will help one to so do. But if one does cry for help Dr. Brooks will give righteous help. Dr. Brooks does not make asking hard. Help is the default. Anyone who has walked into Dr. Brooks's office knows that he speaks first, "How can (may?) I help you?" For that you need only walk yourself there, which you must do. Oversimplifying for easy reference: Dr. Brooks is not your friend, and by being so is your greatest friend.

Neither choice above is very bad, but Dr. Brooks's treatment may give benefit more long-term. I got similar tough-love from my father, who helped me when I asked but otherwise left me to to define my own projects and to figure them out myself. A result in the long term? This web page.

7. Hypothesis

In summary Dr. Brooks seems to dive down into a problem to find an essence in the abstract for the kind of problem to be solved, somehow finding the best of alternative candidates for what might be the essence. Then he surfaces back up to the specifics, somehow finding an innovative way to apply that essence to them.

Powering all from underneath is righteous judgment (morally right, justifiable, virtuous, excellent) as exemplified in The Bible, which Dr, Brooks has studied as a lay preacher. When those who are wrong he challenges with inconvenient questions, the force of them recalls the rhetoric of Jesus. In righteous judgment there is an undertone of an impersonal essence, what is inherently right without any special consideration for the specific problem. What emerges from that starting point is then something seen as right by all and from every point of view. Absent from the list above in parentheses are good enough, cuddly, soothing. What is righteous may seem harsh, perhaps in a good way.

Take directly opposing points of view simultaneously into account to ensure the important but uncomfortable is not omitted when assembling all pertinent points of view to take into account. This can be a first step towards finding an answer with which not a single person starting from any point of view could be dissatisfied.

The solving of problems in human relations through the impersonal side of the essence and of righteous judgment at the foundation may be akin to the solving in science and engineering problems through the impersonal, logical analysis of the practitioner, and indeed Dr. Brooks is both a scientist and an engineer. Perhaps the solving of problems in these two domains by these two means is similar enough that similar thinking is involved, and that developed skills in one domain might help in the other. Mixing in learnings from the Bible of righteous decisions and selecting for long-term benefit as one does for teaching children, all mentioned above, plus experience in formal debating, shall we go further to suppose that all these Ways of judgment can involve thinking to draw upon to reach decisions in No Way?

This search for answers starting from a surplus of mystery ends with a surplus of questions.

An analysis of a simpler case of structured problem solving applied to the everyday is the subject of my essay, Scientific Aggression as a Way of Life.

Reference

[Brooks1987] Brooks, F. P. , J. (1987). "No Silver Bullet—Essence and Accidents of Software Engineering". Computer 20 (4): 10. doi:10.1109/MC.1987.1663532. See also: http://en.wikipedia.org/wiki/No_Silver_Bullet