Go here to sign up for The Embedded Muse.
TEM Logo The Embedded Muse
Issue Number 314, October 3, 2016
Copyright 2016 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com.

Editor's Notes
I've decided to stop writing for embedded.com. After 27 years of cranking out a weekly on-line article, and a monthly one before that for the print publication, it's time to slow down a little - just a little. The Embedded Muse will continue and I'm looking forward to putting more effort into it, as will my other activities in the embedded arena. As always, I welcome your input for the Muse.

Better Firmware Faster classes in Denmark and Minneapolis

One phrase I repeat in my firmware seminars is that a focus on fixing bugs will not lead to a quality product. This is the entire thesis of the quality movement, which revolutionized manufacturing but has somehow largely missed software engineering. This is what my one-day Better Firmware Faster seminar is all about: giving your team the tools they need to operate at a measurably world-class level, producing code with far fewer bugs in less time. It's fast-paced, fun, and uniquely covers the issues faced by embedded developers.

I'm holding public versions of this seminar in two cities in Denmark - Roskilde (near Copenhagen) and Aarhus October 24 and 26. More details here.

I'm also holding a public version of the class in partnership with Microchip Technology in Minneapolis October 17. More info is here.

Quotes and Thoughts

"'Everything takes a week' was the mantra of a seasoned engineer friend of mine. At first we didn't believe it, but when you factor in all the details that need to get done to get a change into production, even the smallest tweak takes a week." - Source unknown

Tools and Tips

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

Chris Hemingway submitted this:

Ragel (http://www.colm.net/open-source/ragel/) is a tool for generating finite state machines to parse a language described by regular expressions. It allows you to embed blocks of code when a given word is matched, and makes debugging very easy by generating state diagrams automatically using Graphviz. It outputs C code with no runtime dependencies or dynamic allocation and hence is ideal for writing robust text parsers quickly in embedded systems. I have used it myself to write a parser for a simple "key=value" protocol and found it very easy to add new features such as hexadecimal values to the protocol without affecting existing code. Undefined states were also avoided entirely, a problem I often encounter when writing parsers by hand.

Wavedrom (http://wavedrom.com) draws digital timing diagrams from a simple JSON based syntax. It features an incredibly wide range of different types of notation and annotations, and exports beautifully rendered output to both SVG and PNG. Once I had spent a few minutes learning the quite terse syntax, I found creating and editing diagrams much faster than using a generic drawing tool such as Illustrator or Visio. There is a live editor online (http://wavedrom.com/editor.html) to try without needing to download anything.

Freebies and Discounts

I've had one of Digilent's Analog Discovery USB oscilloscopes around here for some time and will be giving it to a lucky winner at the end of October. It's a nice unit - 5 MHz, two analog channels, 16 digital, and a pair of waveform generators. Note: The little black Analog Discovery is this month's giveaway, not the Agilent scope in the background! (I reviewed the Analog Discovery here).

Enter via this link.

Serious About Security?

Last month I attended the Cybersecurity Summit in Farmville, VA at Longwood University. The dean of the business school is a friend; he invited my wife and me to the Summit and a reception afterwards.

Longwood, founded in 1839, is a small school in a very pretty town. The first vice-presidential debate of this agonizing campaign season will be held there tomorrow, October 4th. Dean Barrett was brought in some years ago to jazz up the college and to bring in some new thinking. He founded one of the first cybersecurity programs in the country, which was a natural fit for the Summit.

The panel had some heavy-hitters, including retired General James Cartwright, formerly the number two person in the army. The moderator, Roben Farzad of NPR was witty and asked some great questions. Participants stressed the importance of security, issues with passwords, privacy and more. They touched on the IoT though didn't delve deeply into that subject which is so near and dear to our embedded hearts. Plenty was said about how important security will be to the massively connected world that's coming.

But one point was not addressed, and I wonder if it's something only practitioners grok. Security has no value, especially in the embedded space. Everyone wants it, of course. But no one – to a first approximation – is willing to pay for it.

How many extra months is your boss willing to add to the schedule to ensure the product can't be hacked? I bet the number is exactly zero. Does he simply assume the team will build a robust product that's safe in the awful threat environment that exists today, and that will get much worse in the future?

Does your boss even know what the Common Criteria is? Is the phrase "evaluation assurance level" foreign to your team's work?

Today you can buy connected toothbrushes that monitor your brushing habits (next year: one that videos your mouth), and I've even seen a smart Bluetooth spoon that counts how much food gets shoveled into a customer's mouth. These are potential attack surfaces and vectors. We all know about the vast number of automotive hacks, and even pacemakers have been compromised (though in artificial lab environments… for now).

When everything is connected, everything is at risk of compromise.

It's not unreasonable to think that no one would have the slightest interest in hacking your niche product, and indeed security through the sheer banality of a smart device might be effective. Who is going to figure out how to break into that MSP430, anyway?

But many products do use common software platforms. The worst thing a team could do is write their own TCP/IP stack; instead it makes much more sense to use a COTS or FOSS solution. But when everyone uses a relatively small set of common platforms the attack surface gets much broader. While the recent proliferation of Cortex-M MCUs very cool, a common instruction set running common platforms opens the door to the Bad Guys.

Then there are patches. One of my oscilloscopes runs Windows. It has never had a software upgrade, though Microsoft has released innumerable security fixes over the years it has graced my lab bench. (For a variety of reasons I never put it on the network.) Linux, too, changes quickly. The truth is, though, that most embedded systems never get upgrades. The sheer nuisance of doing them would infuriate our customers. We tolerate these on our PCs, but just barely.

I know of an embedded device running Windows that automatically did an OS upgrade – over a satellite link. The data charge was tens of thousands of dollars.

Even with the best intentions and funding, managing security updates is a daunting prospect for many embedded products.

Unfortunately, I've come to the conclusion that when it comes to embedded devices, most talk about security is simply BS – there's no polite way to put it. Yes, pundits gush lots of words and digital ink, but until the boss is willing to pay for it security will be mostly hot air.

There have been plenty of high-profile attacks on and through embedded systems, like Target's POS terminals. Or the Akamai DDoS attack which is thought to have come from a million or so IoT devices. Or the recent OVH attack, also from IoT units.

Unfortunately, it seems security is valuable only after a breech.

How much does your team spend on making your products secure?

Tips for Aspiring Developers

Reader Michael James asks:

Say you are a master embedded programmer, and a young hardworking student comes to you for sage advice. And the student asks... "What are the disciplines of becoming a great embedded programmer?" What would they reply?

Program daily?

Study Algorithms?

Comment well?

Know the hardware?

Frequently read other peoples' code?

I'd say:

  • Study widely
  • Think hard about the problem before writing code.
  • Think hard about what you're doing while writing the code.
  • Think hard about what you just wrote before testing.
  • Have your mentor review that code before testing.

Some writers call the last point "baby killing." Great novelists subject their work to review by editors, friends and associates and accept the fact that their "baby" - the prose - may need to be greatly revised and even completely rewritten. Code may need the same treatment before it becomes Great Code.

What would you recommend to an aspiring developer?

How to Measure Good Design?

In the last issue Charles Manning posed a question for the readership: How can one measure, hopefully quantitatively, whether a software design is good or not?

Jack Bombeeck wrote:

> the ability to add new functionality and/or features

The less effort it takes to do it, the better the design was in the first place. This is actually something that can be measured (though how would you distinguish between adding a new feature poorly to well- designed code vs. adding a new feature well to poorly designed code...).

Then there is the old adage of the elegance of a solution. Another factor that is very difficult to measure.

Steohen Bernhoeft had a nice short list:

1. Clarity (code and comments combined, deduct marks for excessive verbosity or terseness)

2. Generality, re-usablity, extendability. Can it be re-coded in another language easily for instance.

3. 'Connectivity' (how many parameters in or out)

4. Performance (where applicable).

5. Robustness, graceful handling of illegal situations.

6. Consistency of style amongst modules.

Paul Bennett wrote:

Tracking reported bugs, from the first instance they are noticed, through to their eventual elimination is an important part of managing a systems development. However, many bugs appear well before coding ever starts and these early bug introductions never seem to be adequately tracked.

Having a development process that automatically monitors and builds the numbers case for each module (how many bugs encountered, times revisited, time to elimination and log of confirmation testing) so that the management have an overview of how the project is really progressing (or not).

The place to begin eliminating complexity is in the development of the requirements specification. Doing a McCabe style analysis of your requirements should prove illuminating. How many have a cyclomatic complexity factor above seven? Complexity at this level in your initial requirements will lead to that complexity getting greater in the code (I know it shouldn't but it will always seem to. One of Murphy's laws?).

Above a cyclomatic complexity factor of seven in your requirements expect to spend a lot of time in running the test cases.

Lazy code seems to come from the "bulk buying" of library functions that indicate they will handle your specific case along with all the cases you really don't care about. A component oriented approach would create a spec for the particular requirements of a component. You then shop for the very specific component you need to fulfill your task. Components are very single minded in that they do one thing very effectively. They are not burdened with a whole host of decision trees to decide specific side cases (make the programmer do that work, he will grow by it).

No single measure will ever be adequate by itself. You need a whole raft of things you can enumerate and measure within your development process and related to all the components you use, design and build. Having a simple, very well defined development process that helps you kick out the required numbers should always be deployed. Part of that process should include a plenitude of review stages where each component is fully assessed and not permitted to escape without that it has proven correctly aligned with the requirements it was generated from.

As for bugs in delivered software. You should aim for the position of eagerness to see one as it is a rarity. Sadly, the current truth for many systems at the moment is that they are all too commonplace.

If someone would like to pay me to divulge my process and methods I am quite happy to do so. Otherwise look to the likes of Phil Koopman, Jack Ganssle, Marion Kelly and Freidman and Weinberg for clues on how you might develop your own process and methods. When you get to be able to certify your software with a CofC that has a voracity as robust as that for a mechanical nut and bolt then you may get closer to perfection.

Paul Carpenter makes a good point about copied code:

Trouble is there is more lazy code these days because you just spec a system with faster clock and more memory to get over bad code. Let alone the copy/paste nightmares and these days mainly from Stack Overflow instead of doing homework.

More on Upgrading Windows

David Trierweiler had a final comment on upgrading Windows:

For the last year I have been using http://wu.krelay.de/en/ as a reference whenever I do a clean Windows 7 install. So far it has been a huge help.

Not only do Updates happen in a speedy amount of time now but the Windows Update and related System and library issues that various tools had been finding but not fixing have all gone away.


Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

Note: These jokes are archived at www.ganssle.com/jokes.htm.

Remember the old FDIV bug in the Pentium? Here are the top ten slogans for the part:

9.9999973251   It's a FLAW, not a Bug
8.9999163362   It's Close Enough
7.9999414610   Nearly 300 Correct Opcodes
6.9999831538   You Don't Need to Know What's Inside
5.9999835137   Redefining the PC -- and Mathematics As Well
4.9999999021   We Fixed It, Really
3.9998245917   Division Considered Harmful
2.9991523619   Why Do You Think They Call It *Floating* Point?
1.9999103517   We're Looking for a Few Good Flaws
0.9999999998   Errata Inside

Advertise With Us

Advertise in The Embedded Muse! Over 27,000 embedded developers get this twice-monthly publication. .

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.