Go here to sign up for The Embedded Muse.
The Embedded Muse Logo The Embedded Muse
Issue Number 400, June 15, 2020
Copyright 2020 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact info@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Editor's Notes

SEGGER SystemView analyzing embedded systems

Dear readers: I truly enjoy and encourage your emails. Please understand that this week (and hopefully only this week) due to a detached retina I will likely be very slow in responding. Keep 'em coming! But please be patient.

Jack's latest blog: Humility

Quotes and Thoughts

Software is free!  But, correctly placing all the ones and zeros is priceless. Ron Brombach

Tools and Tips

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

François Baldassari wrote:

On the Logic Cube Pro: This Logic Analyzer, alongside many others, is supported by Sigrok. Sigrok is an open source logic analyzer UI with support for a long list of protocols on a longer list of hardware devices. More and more often I turn to Sigrok rather than whatever proprietary UI ships with the device. Sigrok is cross platform (Linux, MacOS, Windows, ... you name it), open source, and constantly improving. Best of all, it comes with a library, so you can script it to your heart's content.

Freebies and Discounts

Cy Clarke won the CodeBot in last month's contest.

This month's giveaway is one of Zeroplus's new Logic Cube Pro logic analyzers which I reviewed here. This is their top-of-the-line unit with 32 channels, 256 Msa/channel memory, and 2 GHz sample rate. It lists for $3500.

Logic Cube Pro giveaway

Enter via this link.

What Does Firmware Cost?

I've been collecting this data for some years, and my numbers peg the cost of firmware at $20 to $40 per line of code (LOC). That's not just coding; this includes the entire life-cycle, from requirements elicitation to final delivery. And that does not include costs like certification when required for avionics or medical devices.

It's pretty astonishing to interview developers and hear about their code production rates. Typically they tell me they can crank 100 to 300 LOC/day. That may be true in the pure coding phase, but is simply not relevant. The real question is: what does it cost to build a product? That includes requirements, design, coding, debug, test, integration, documentation, and release. These are all real costs that must be accounted for.

In a 1996 paper by Thomas Drake (Measuring Software Quality: A Case Study, IEEE Computer Nov, 1996) NSA (you know, those folks who are listening to you at this very minute) figures they pay $70/LOC. That's $116/LOC in today's deflated greenbacks.

The Space Shuttle code cost $1000/LOC. Now, this was the best code ever written, with but a single bug per 400,000 lines of code.

VDC's 2017 Embedded Engineer Survey Results, once you boil down all of their data, suggests the number is about $43/LOC.

Toyota has the coveted "most expensive code every written" award. Their engine control code caused the Feds to hit them with a $1.2 billion fine, which works out to over $1200/LOC. Oh, and then you have to add in those pesky actual engineering costs.

Rockwell Collins, in a 2009 report ( Certification Cost Estimates for Future Communication Radio Platforms) figures for low-assurance levels (Design Assurance Level D), an experienced engineer produces 65 LOC/day:

Now, that's just for documentation, production and validation - these figures do not include the other aspects of building a product. Still, it's instructive to play with the numbers. An experienced developer in the USA will cost around $150k to $200k/year (including overhead). At 65 LOC/day that's $12/LOC, or probably twice that once all of the other, required, engineering activities are included.

At 12 LOC/day for great code (DAL-A), figure on five times the price.

Though inexperienced developers are a bit cheaper than old salts they are considerably less productive. Here's some more data from NASA's Software Manager's Handbook:

NASA software experience multipliers

Now, the costs cited above assume one is working 40 hours per week on the code. An article in the May/June 2014 issue of Crosstalk claims the average developer is 50% utilized on the project at hand; other sources cite 55%. So the real costs are twice what they've cited.

I recently blogged about a complaint we constantly hear: Why Does Software Cost So Much? A better question might be: Who would have imagined what a steal software is? Consider Microsoft Excel. I bet that cost billions. But what would it cost if implemented with gears and cams? Trillions? Quadrillions?

More on State Machines v. RTOSes

Peter Mueler wrote two interesting articles about state machines and RTOSes:

Using State-Machines in Low-Power Embedded Systems. I.e. using state machines in a simple foreground / background system

Using Statemachines with the embOS Real-Time Operating System (RTOS). This article is about the integration of state machines into a RTOS

Luca Matteini also sent some very useful article links:

Finite state machines with the classic Moore and Mealy algorithms aren't enough when you're designing a complex user interface. Pushdown automatons and Harel statecharts are a great resource

What I never used are coroutines, as I find them an intriguing method to explore C language behavior, but more a hack than a safe procedure

Time triggered patterns are a nice solution, where maybe you think that software can be more like hardware

Stephen Adels adds:

Jack Crenshaw said this:

Quite frankly, I've never been a big fan of the RTOS.  It usually seems to me that they try to offer too many options.  I mean, you can use a semaphore system, or you can use all interrupt-driven event handlers, or you can use purely timed tasks, perhaps counted down to get sub-frequencies.  But you don't need to use _ALL_ of them!

Yes, but you don't have to throw the baby out with the bath water.  What you get with the RTOS is complicated code you don't have to create, fix, or maintain -- quite an expense, if you have to do it yourself.  So, for your company, take a framework-worthy developer willing to be a guardian of the framework, and create a shim (decorator) -- an interface that only uses the subset of the RTOS that your company has targeted as what they want to use.  Many languages would benefit from being able to constrain portions of the code to subsets of the language (or library, or API, etc.).  What is critical is being able to require that the developers only use your interface to the RTOS, and put any requests to expand that interface through the "guardian" who is supposed to say "no" unless the need is truly critical.  Having automated builds helps to implement this process by keeping control.

François Baldassari wrote:

On state machines vs. RTOSes. This seems like a false dichotomy to me. Embedded software engineers should be fluent with both approaches, and should select the one that best matches their use case. Here's an example: at Pebble, we needed to sandbox 3rd party applications that could be loaded on the watch. There was no MMU of course, we ran on an STM32F4xx. Using FreeRTOS, we were able to create an unprivileged task with only a few APIs accessible to it. It worked famously, and would have been much harder to do with a state-machine-based system. Another option: coroutines (aka protothreads).

Free Micros - Pretty Much

"There is no reason anyone would want a computer in their home." Ken Olsen, founder of Digital Equipment Corporation, in 1977

I sure wanted a computer in my home in 1977, though "home" was a sailboat at the time. Though the word "computer" invoked an image of a massive building-sized machine in most peoples' minds, the little-known technology of embedded systems was emerging. In 1974 an 8 bit processor chip cost about $2k in today's dollars. A complete, working computer was out of reach for most people.

Today, not so much. A desktop or laptop is cheap. And embedded microcontrollers are in some cases so cheap as to be virtually free. Looking at Digi-Key's website Rochester Electronics (though the datasheet says "Renesas") sells a handful of parts for under a dime, in small quantities. You won't get much memory, but timers and an ADC are included. If you can spring for two bits ($0.25 - my dad used the term "two bits" a lot, but I wonder if anyone does anymore) more parts from other vendors are available. For a pair of quarters ($0.50) there's quite a selection of 32-bit MCUs available. In singles.

But these are incredibly pricey parts compared the three-cent MCUs from Padauk. Yep, $0.03 for an 8-bit MCU. Apparently these even have hardware instructions to wait and delay a specified number of cycles. Ever build a bit-banging UART? It's not much fun as you've got to pick instructions based on the number of t-states they require. An interrupt or DMA request would corrupt comm. The Padauk's wait and delay would make this pretty trivial, even more so because they have a tool that will write all of that code for you.

C support is minimal, but with just 1KB of ROM and 64 bytes of RAM most of us would stick with assembly.

At these prices the parts could compete with a variety of I/O devices. Need a not-blazing-fast USB chip? Save a few bucks using one of Padauk's parts. Need a UART? The cheapest one listed on Digi-Key's site is $0.76. You could implement 25 using the Padauk for the same price.

There's an excellent and detailed write-up here. I don't know where the company's name came from but padauk is a species of beautiful bright red hardwood that turns disappointingly brown with exposure to light.

At $11/board-foot this 12" x 6" x 1" padauk board costs almost 200 times more than a Padauk MCU Who would have thought an MCU would be cheaper than wood?

It's hard to guess where the semiconductor industry will go in a few years. But of one thing I'm sure: In 1977 none of us would have dreamed of a $0.03 computer.

Failure of the Week

Funny software failure

730488 days is two thousand years!

This Week's Cool Product

Attila Kalinka sent a link to TI's TPS546D24A is a 2.95-V to 16-V input, 40-A synchronous buck converter. That's right, 40A, and without external transistors. 40A in a 5 x 7 mm package. And you can parallel four of them to get 160A from a single output. I didn't find anything in the 175 page (!) datasheet about heat sink requirements, but the FETs are spec'd at 4.5mΩ. There's a thermal pad on the bottom of the device that must be soldered to the PCB.

I count 20 capacitors in this typical application from the datasheet:

Note that five pins carry the 40A out of the device. That's 8A per pin, each of which is pretty tiny on an LQFN.

Ten bucks in singles. Half that in quantity.

Note: This section is about something I personally find cool, interesting or important and want to pass along to readers. It is not influenced by vendors.


Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

These jokes are archived here.

From Jon Titus an oldy but goody:

Definition of recursion: Recursion (n), see recursion

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster. We offer seminars at your site offering hard-hitting ideas - and action - you can take now to improve firmware quality and decrease development time. Contact us at info@ganssle.com for more information.