Go here to sign up for The Embedded Muse.



TEM Logo The Embedded Muse
Issue Number 344, February 19, 2018
Copyright 2018 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Editor's Notes

Wouldn't you want to be in the top tier of developers? That is part of what my one-day Better Firmware Faster seminar is all about: giving your team the tools they need to operate at a measurably world-class level, producing code with far fewer bugs in less time. It's fast-paced, fun, and uniquely covers the issues faced by embedded developers. Information here shows how your team can benefit by having this seminar presented at your facility.

I'll be wandering the halls of Embedded World (Feb 27 to Mar 1); feel free to stop me and chat.

Quotes and Thoughts

Andrzej Telszewski sent this quote: "Perfection is reached, not when there is no longer anything to add, but when there is no longer anything to take away." - Antoine de Saint-Exupery

Tools and Tips


Here's a site full of all sorts of calculators.

Long ago we were taught to match input resistors on an op-amp. Turns out, that advice might be as antiquated as a 741. Interesting article here.

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

Is it a Bug or an Error?

In Muse 340 I included a quote from E. W. Dijkstra where he suggested we abandon the word "bug" in favor of "error." Readers had a range of opinions, but it did spawn a discussion among a small email group of people who write about embedded systems.

Mike Barr's posting on the subject is here. Chris Hills take is here. Jacob Beningo posted this.

Here's my take:

I've never had a bug in my code.

I wrote my first computer program 49 years ago. This was via a time-sharing service at a friend's high school using a Model ASR-33 teletypewriter, connected to a remote Honeywell mainframe over a 110-baud modem. Storage was paper tape.

At the time I was making minimum wage, $1.60/hour, but computer time cost $4/hour, so instead of a practice "hello world" bit of code, it was a FORTRAN simulation of an AI robot. It didn't work at first, but eventually I removed my mistakes and got the results I'd hoped for.

In the succeeding half century and thousands of programs I have never had a bug in my code. Not one.

Admittedly, I've made lots of mistakes. Errors. Goof ups. Brain-dead dumb stuff. Code that was so flawed one wonders what I was thinking. Or subtle mistakes that consumed weeks of troubleshooting.

In some cases the problems stemmed from uncontrollable events, like chips that didn't function as the datasheets claimed. But most were self-induced errors I injected through my own blunders.

"Bug" implies that something bad snuck into the code when I wasn't looking. I left the window open, went to get a cup of coffee, and of its own volition that nasty bug slipped into the code.


When I ran the Dijkstra quote advocating for the term "error" over "bug" a few readers were outraged. One wrote that he has dyslexia and he relies on his compiler to find those sorts of bugs. I don't know much about that affliction so can't comment. In the spirited email discussion mentioned above, one of the participants note that he, too, has dyslexia.

Other readers thought "bug" insulated them from the boss's ire. "If we call these errors," one reader opined, "then the boss will be all over us. 'Error' makes us accountable."


Practicing engineers call these goof-ups "bugs." In the literature they're referred to as "defects," a better term as that encompasses everything from outright mistakes that cause the system to malfunction to minor spelling or documentation problems.

I think Dijkstra is correct, and we need a better moniker that holds us more accountable than "bug." It could be "error," or even "damn-fool screwup," but let us be honest: the blame lies with we who write the code. "I made that mistake" is much more honest than "uh, this code seems to be buggy." (Admittedly, there's often plenty of blame to go around, like when we're given incorrect requirements).

Section 1.3 of the ACM Code of Ethics requires us to "Be honest and trustworthy." The IEEE Code of Ethics is even better, in part commanding us "to seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors."

Those are excellent aspirational philosophies to guide our work.

Aspirational, because we will always make mistakes. Debugging, uh, de-erroring, is simply a fact of life in this business. But let's own up to the source of the errors. Once we own the problem, we can start looking for solutions... or at least ways to drastically reduce our defect rate.

Sound impossible? The SPARK community doesn't think so. Their motto is "Correctness by Construction." Sure, SPARK users, like the rest of us, make mistakes, but their tools and processes ensure that these rarely make it into the code (even in initial testing an extremely low, or even zero, defect rate occurs). The late Peter Amey wrote "In my 10-plus years of using SPARK, I have never needed to use a debuggger. I have become so used to things working the first time that my debugging skills have almost completely atrophied." (The next article in this Muse has links about this).

Most of us can't, or won't, use SPARK or other languages that are less error-prone than C. But the literature is full of ways to reduce our defect rate regardless of language.

Did you know the average developer injects 5 to 10 errors per 100 lines of code, and removes only 85% of these before shipping? High quality teams make only half to a third as many mistakes and remove 96% during de-erroring. Best-in-class safety-critical groups do even better.

Oh, but the cost of getting it right! Turns out, teams working with languages like C and C++ can get great code for no incremental cost if they improve their processes (Safety Critical Software and Development Productivity by O. Benediktsson).

The trick is for us to own up to our inevitable errors, and to work harder and smarter to avoid them.

Riffing on an AA theme, perhaps we should stand up and say "Hi, I'm a firmware developer and I make mistakes."

More on SPARK and Ada

Nick P sent links to some presentations and studies about the efficacy of the Ada and SPARK languages:

I haven't read your stuff in a while since I've been quite busy. So, I enjoyed going through a lot of back issues. :) I noticed you brought up Ada and software assurance again. I have several links for you to consider for your newsletter.

First, most comparisons of programming languages are anything but apples to apples. Too many variables not considered. The study below (1995) is the best one I've ever seen at accounting for all the variables. Spoiler: Ada wins. 


Far as use of SPARK, I have three, case studies: an older one that does a whole DNS server in SPARK; a new one on a glider; a new one on a spacecraft. They show it can handle real-world use. The caveats look like they'd be easy for embedded folks to handle since they usually deal with bigger problems. 




For software assurance, the most practical method was probably Meyer's Design-by-Contract he promoted in the 1980's based on Dijkstra's work. Aside from documenting assumptions, one can use so-called "property-based testing" to generate tests directly from the specs. Meyer's Eiffel supports that with "theft" being an example for C language. If DbC contracts become runtime checks, you can use fuzz testing tools such as AFL to find errors with the contract checks taking you directly to the problem with laser focus. I often recommend combining all three for a low-effort, high-ROI form of verification. Also, leave whatever runtime checks in that you can if it doesn't hurt performance. Goes double for complex module with lots of state or interactions.

Below is an introduction from a Lockheed-Martin employee that project managers might find as approachable as engineers. It shows the conflicting goals one wants to achieve, how various verification methods faired at the time, and how Design-by-Contract balances things out nicely. I'll also drop a well-designed study done for game development since their requirements overlap a little with real-time embedded.





On Engineering

In the last issue I claimed engineering is about:

  • Predict what is going to happen.
  • Make it happen.
  • Measure to see if it really happened the way you expected.

Harold Kraus had a more complete view:

With some refinement, this is the Correctness/Quality/Safety/Security Management loop:

  • Determine how much Correctness/Quality/Safety/Security is appropriate for the product (Planning)
  • Define what you want to happen, including Correctness/Quality/Safety/Security. (Requirement Specification)
  • Make it happen. (Design and Implementation)
  • Measure to see if it really happened the way you expected. (Verification)
  • Revise and repeat if it didn't happen the way you expected (Corrective Action/Continual Improvement)
  • Prove you have accomplished the needed level of Correctness/Quality/Safety/Security (Validation).
No Degree? No Problem

Michael Linden came across something surprising recently:

I recently read an article about how some employers are considering lesser-educated candidates in order to fill their openings in the tightening labor market. Here's a link to that article.

No degree? No problem as tight labor market changes the game


Then, I ran across the following Embedded Systems job posting this weekend. Yup, no degree, no problem!  ;^)

Embedded Device Developer



Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

Note: These jokes are archived at www.ganssle.com/jokes.htm.

There are only two really difficult things in programming:

- Naming
- Cache invalidation
- Off-by-one errors

Advertise With Us

Advertise in The Embedded Muse! Over 27,000 embedded developers get this twice-monthly publication. .

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.