For novel ideas about building embedded systems (both hardware and firmware), join the 28,000+ engineers who subscribe to The Embedded Muse, a free biweekly newsletter. The Muse has no hype and no vendor PR. Click here to subscribe.
By Jack Ganssle
Lessons Your Mom Should Have Taught You
Momism ( m“m iz' em) n. 1. A brief statement of a principle passed maternally. 2. A tersely worded statement of an observation of truth: APHORISM
I recently saw a commercial in which a middle-aged couple are sitting in their car, in the dark, apparently in front of their son's apartment. Neither speaks a word, but at first we hear dad's thoughts: "What's wrong with that kid? He's 22 and it's time he grew up and took care of his own problems." Then mom's mental musings fade in: "What do we have money for, anyway, if not to help him?"
Those words could have come from my wife and I. Or from so many other parents we know who have teenagers and 20-somethings. Fathers let go; mothers keep an empathic connection to their kids that's truly remarkable, admirable, and, for dads, sometimes frustrating.
Biographies often paint a harsh picture of dad as distant or the disciplinarian. Mom was always a saint. All of us remember mom's words of wisdom and advice, usually gently given in counterpoint to dad's imperative directives. My mother offered lots of advice, some of which still guides my work. Here are a few of her thoughts about creating schedules for embedded projects.
Be honest in scheduling - Mom thought that the devil drives scheduling in most engineering teams, which is why most projects are late. She often quoted the IEEE Code of Ethics, which, among other things, requires us to "be honest and realistic in stating claims or estimates."
Few of us do this, of course.
One of the problems with modern politics is that it's entirely focused on the practical rather than the ethical. I wish we started all policy debates with a search for the ethical, and then derive a sensible strategy from that. Similarly, the phrase: "The boss is gonna cut whatever schedule we give him in half, so let's double our estimate" while both practical and logical, it's mendacious. If the boss will cut the schedule in half, that's a different problem unrelated to that of creating a realistic schedule. Deal with it separately from scheduling.
There will always be a fundamental disconnect between the very real needs of management and those of engineering. Capricious deadlines aside, a Mars mission, for instance cannot slip. There's a launch window only every two years. You can't negotiate with planetary geometry. Other schedules may appear more fluid but very real business drivers, like limited engineering funding, market windows and customer needs legitimately compromise the, also very real, needs of the development team. Strategies promoted by the agile community do help bridge this gap.
Mom always told us to manage complexity - Code complexity grows faster than code size. Robert Glass (Facts and Fallacies of Software Engineering, Addison-Wesley, 2002, ISBN 978-0321174253) claims that for every 25% increase in a software problem's requirements, the complexity of the solution doubles. Perhaps this explains why firmware sizes are growing 46% per year (Rich Nass quoted VDC in the webinar here: http://www.embedded.com/showArticle.jhtml?articleID=199903515). Couple that with the typical $20 to $40 per line cost of firmware and it's clear that a little feature creep can cost astronomical amounts of time and money.
Complexity is the devil's workshop - complex code is buggy code. Data I've collected, to be shared in a future article, unsurprisingly shows that complex functions are buggy functions. So measure complexity using the Mccabe metrics and recode those with high scores.
Never create a calendar schedule - Scheduling is the process of estimating how much work is required to complete a project. Your calendar represents a snapshot of time allocated to numerous disparate and always changing projects, meetings, support and needs. The two are measures of very different things, and are not equivalent.
Don't overload the CPU - Ah, this is one my mother whispered to us nightly, just before sending all five of us off to our beds in our Dr. Dentons. Her words still echo in my mind: "Dear, a 90% loaded system doubles development time; at 95% loading the schedule triples." (Davis, Alan: 201 Principles of Software Development, McGraw-Hill, New York, NY, 1995). Though hardly surprising it's a fact too many of us neglect. Finding a few extra CPU cycles is just as difficult as squeezing one little feature into a system with little spare ROM or flash.
Never provide a single-point estimate - One of mom's pet peeves. She'd tell us: "An estimate is a probability distribution function. Only a fool believes any distribution given with zero variance!" Provide min/max estimates specified to some standard deviation. I like to use 2s, giving a probability of being right 95% of the time. Karl Wiegers has good advice here: http://www.processimpact.com/articles/delphi.html
Use your best people on the small projects - Superprogrammers, the best developers on the team, are six times more productive than the worst team member, but only when both are measured on small projects. As project size grows the best and the worst become equally non-productive. Big projects have vast amounts of overhead - meetings, email and the like. A superprogrammer can attend a meeting no faster than a sluggard.
Your boss is right about the show - Engineers have always been infuriated when reasonable estimates get tossed because the show - there's always a show - is three months away. That's a rotten way to schedule a project. But it may be the correct way. The show is important. It's a chance to see customers, demo to the press, frighten the competition. Engineering serves the business and its needs, including important events like the show.
If a real deadline and the show deadline aren't congruent, create enough features to make a fantastic demo. Attendees don't dig through every menu and option; they rarely look at a product for more than 10 or 20 seconds. How much needs to work? Again, the agile precepts express much that's wise when it comes to building a system to meet an early deadline.
Where's your $%#W$ report card? - Mom is no dummy now and was just as sharp when I was in high school. The Jesuits foolishly mailed our report cards home. For some reason they always turned up on a Friday afternoon. Till we were caught, my siblings and I would pluck our reports from the mailbox and reinsert them Monday, so the weekend wouldn't be ruined.
Engineering teams sometimes do the same thing when scheduling. Hunched over Microsoft Project, everyone jiggles triangles to create something that's, well, believable if not accurate. We're really just trying to postpone the inevitable day of reckoning. Maybe something will change for the better. Maybe the project will be cancelled. But the instinct to defer pain is a strong one. As mom often said: "Fess up now!"
Separate R and D - There's no such thing as R&D. I have no idea why we routinely lump "research" with "development." That's about as accurate as linking "science" and "astrology."
Development is the process of solving a known problem with known technology. Research is all about discovering new things. If research were schedulable like development, then we could expect scientists to find a cure for cancer by April 22, 2008. If that's too aggressive, add resources: more scientists.
A lot of embedded applications depend on science, say the response of a sample to various wavelengths of light. If the science isn't right or completely understood, the schedule is toast.
But mom had plenty to say about resources. In fact, I discovered that Fred Brookes secreted microphones in our house to pick up mom's homespun wisdom (National Inquirier, May 20, 1978, page 28 right next to the story about the 600 pound cat), which he later plagiarized into his "The Mythical Man-Month" (Addison-Wesley, Second edition 1995, ISBN 978-0201835953). He ripped off mom's dinner-table throwaway phrase children, remember that adding people to a late project makes it later. For more people means more meetings, more reports, and less productive work.
A churchgoing lady, mom never let us forget one of her prime values: don't take the Lord's name in vain. It took us a while to discover that this rule covered an entire host of words in addition to the Lord's name, most of which were comprised of four letters. Since she doesn't follow ESD (she prefers IEEE Transactions on Software Engineering) I can get away with using one of those here: hack. Oh, how she detests that curse! I remember getting whacked for hacking some Perl code, till I explained that no one knows how to write clean Perl. (She's an old-school kind of girl and mostly works in assembly and SNOBOL).
But mom was wrong. Hack is a perfectly acceptable word even in mixed company (OO developers and procedural programmers, that is). Hacking is a perfect tool for doing research. When the hardware team misdocuments I/O control registers, the firmware folks must use either bribes or focused hacking to elicit the devices' actual operation. But once the research is done, switch to a more disciplined model and create awesome code.
Overtime takes engineers away from their mothers. Mom sure nailed this! A 2005 study by Circadian Technologies (http://www.circadian.com/publications/swp2005.html) documented what we all know: consistent overtime is a Bad Idea. Productivity can decrease by as much as 25%when people work 60 or more hours a week for weeks on end. It also increases turnover by as much as 300%, while nearly doubling absenteeism.
Once I remember she said: Honey, someday folks will invent something called eXtreme Programming. Those XPers will say some funny things; some of those fellows will need their mouths washed with soap. And where was their mother when they learned to capitalize? But they will be right when they say "never work overtime two weeks in a row."
Mom was quite clairvoyant, which is probably why she was so good at scheduling. Why, all five of us shipped in about 9 months! When I was a small lad hiding under a desk during the Cuban Missile Crisis she warned: Someday you'll have a wireless phone so you can call me anytime those godless commies threaten you. Just don't yak on the phone while driving, unless you need to call me. I think she was trying to warn me of the perils of multitasking. Researchers (Clark, Kim and Steven Wheelwright, 1994. Managing New Product and Process Development: Text and Cases. The Free Press) have found a 20% efficiency hit when multitasking between two activities. That jumps to 65% when managing four. So balancing multiple activities, though often unavoidable, destroys schedules.
Finally, here's a bit of wisdom for all of us: Call your mother.