Go here to sign up for The Embedded Muse.
The Embedded Muse Logo The Embedded Muse
Issue Number 478, November 20, 2023
Copyright 2023 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Editor's Notes

SEGGER Gang Programming Solution

Tip for sending me email: My email filters are super aggressive and I no longer look at the spam mailbox. If you include the phrase "embedded muse" in the subject line your email will wend its weighty way to me.

Tip for sending me email: My email filters are super aggressive and I no longer look at the spam mailbox. If you include the phrase "embedded" in the subject line your email will wend its weighty way to me.

Quotes and Thoughts

When Herbert Hoover told a lady he was an engineer, she replied, “Why, I thought you were a gentleman!”

Tools and Tips

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

Here's a good article about memory safety.

Freebies and Discounts

This month's Cool Product is the H32 development board. Christos Apostolidis has kindly made one available to Muse readers.

Enter via this link.

Designing Ultra-Low Power Systems

I'm getting a lot of email about designing systems that must run for years from a coin cell. If you see claims that a particular MCU can run for decades from a CR2032, well, apply some engineering sense.

This is a tough problem that spans both hardware and software issues. Neglect either one and the system's performance will be very disappointing. I've studied this at length, going so far as to discharge over 100 coin cells while monitoring their characteristics, and the results are surprising. A full (and long!) report is here, but here are the highlights:

Though MCU vendors make a big fuss about their incredibly-low sleep currents (often measured in nA), for all practical purposes those are irrelevant. If under a microamp or so, sleep currents matter little.

What does matter is how much time your system sleeps.

And even more important is how much work you can do when the system is awake.

The MCU is often the least important coulomb consumer. Other components leak, sometimes a lot. Even the lowly decoupling capacitor can be a significant resistive load.

As a coin cells discharges its internal resistance goes up - a lot. There could be a ton of capacity left, but a high resistance may mean it is unusable, depending on how much current your system needs when it is awake. Here are my experimental results, which shows how the battery's internal resistance means a large drop in voltage delivered to the circuitry:

(The horizontal axis is time, but is unlabeled as it could be in years, for a really low-power system, or hours if it is a current hog.)

Four loads are shown: 1 uA, 0.5 mA, 10 mA and 30 mA. These represent the current the system needs when awake. At 30 mA less than 2 volts is delivered while the battery is still far from discharged.

This implies it's a bad idea to wake up running at full speed. A fast clock pulls a lot of power, and if the battery's resistance is high that could result in a voltage drop large enough to crash the MCU. Better, wake up with a slow clock, and ramp up while monitoring the voltage.

On-board brown-out circuits are pretty much worthless. See the paper for details.

Some advocate using a capacitor to provide some boost to the power supply for a few milliseconds. Though the math works out, in the real world it won't work. An analysis is in the paper.

Though there's much said and advertised about running a system for a decade from a coin cell, that is so technically challenging few will achieve it. Perhaps one can take consolation in thinking one may be retired, or at another job, when the systems start failing, but that's poor engineering practice.

Software Entropy

I needed a light bulb for my car, so went to Rock Auto, once a sponsor of my favorite radio show (Car Talk) and found the right part. The URL is 690 characters long! Comprised of numbers and upper and lower case letters, some experimenting found that it is indeed case sensitive. So that's a set of 62 distinct characters, representing 101237 distinct possible part numbers. Now, I'm sure Rock Auto does stock a lot of parts, but given that there are only about 1080 protons in the universe, one wonders just what the developers were smoking.

Later in the week I surfed over to Bank of America and clicked on a 927 character link, which could select one of 101662 web pages.

No doubt these links are conglomerations of selectors and such, probably derived from a lot of history. They remind me of old software that accumulates lots of cruft that nobody really understands anymore.

Software is like life: untended, it tends to rot. Little bits of nastiness fester and accumulate. A quick hack to get the thing out the door never gets cleaned up. The next release also gets some added nastiness. Perhaps one realizes that a method could really be improved... but isn't.

Luke Hohmann preaches "post-release entropy reduction", the idea that the cost of maintenance increases non-linearly with each release as these defects increase development costs as time goes on. "Entropy" is the disorder in the code stemming from the ugliness. He advocates that, at the beginning of each release cycle, we clean up the poor parts of the software.

Though I'm not sure I agree with the math, it is thought-provoking. If M is the mess in a release, and N is the number of releases, he thinks the cost of a release is something like:

(1+M)(1+M)+... (1+M) = (1+M)N

He goes on to posit that if the cost of refactoring the mess out is R, then the cost goes to:

1+R+R+R... = 1+R*N

which is linear, rather than exponential.

However suspect the math might be, experience suggests that he is on to something. All to often a project will grow into an unmanageable mess until the team, inevitably, tells management that "this code has become too expensive to maintain." His prescription to clean it up before adding features seems good advice.

More on Crappy Code

Tom von Dein took issue with my thoughts in Muse 477:

I need to disagree with the sentiments about crappy code made in the last Muse.

Nobody questions the fact, that these days almost every part of our lives is
being maintained by software. But that is not the statement J.Kwak made.

Just because most software works most of the time doesn't mean it's good
software. Look at all the CVE's published everyday for so many software
products in widespread use. Microsoft products alone like Windows, Active
Directory and Exchange run the offices of the better part of the whole
planet. Yet there are thousands of critical bugs. And the bugs we know about
are only the ones someone has discovered and published. There are even more
so called zero days almost no body knows about them.

How many times have read about these "data breaches" when millions or even
billions of customer records have been stolen and put up for sale on the
darknet? What do you think is the cause of this mess (cluster fuck would be
the correct term though)? It's crappy software! Of course. Release fast, let
the customer test the code, improve iteratively and let's make as much money
in as little time as possible - this is the way the software industry works.

And don't you dare to think that embedded software is any better! That would
be the classical confirmation bias at work :) Yeah, most devices just work.
Unless some security engineer or hacker takes a deeper look at it. Then the
poor thing falls into pieces. I am working in the IT industry and we are
plagued every other week by DDOS attacks. Gues where the attack traffic
comes from? IOT devices, that is, embedded software. Exploited, owned and
put to another (or additional) use without the knowledge of the owners.

Especially the embedded marked is bad in that respect because most devices,
particularly the cheap ones, can't be updated. And even IF there is a vendor
here and there who provides updates and a procedure to actually do it - the
majority of customers just don't. They don't even have a clue that they
could update their light bulb.

Or just look at this Japanese probe which failed lastly (I can't remember
its name, sorry) because of a bug?

So, from my point of view as a software engineer who has worked more than 20
years in the security department of a large company, it's exactly the other
way around: software is not just crap. It's a security nightmare, a tar pit
for future generations. And that holds true for at least 99% of all code
running out there. Come on, there are tanks running kubernetes on them!

Failure of the Week

Shannon Spencer sent this:

And this is from Rob Wehrli:

Have you submitted a Failure of the Week? I'm getting a ton of these and yours was added to the queue.


Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

These jokes are archived here.

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.