Go here to sign up for The Embedded Muse.
TEM Logo The Embedded Muse
Issue Number 362, November 19, 2018
Copyright 2018 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Editor's Notes

After over 40 years in this field I've learned that "shortcuts make for long delays" (an aphorism attributed to J.R.R Tolkien). The data is stark: doing software right means fewer bugs and earlier deliveries. Adopt best practices and your code will be better and cheaper. This is the entire thesis of the quality movement, which revolutionized manufacturing but has somehow largely missed software engineering. Studies have even shown that safety-critical code need be no more expensive than the usual stuff if the right processes are followed.

This is what my one-day Better Firmware Faster seminar is all about: giving your team the tools they need to operate at a measurably world-class level, producing code with far fewer bugs in less time. It's fast-paced, fun, and uniquely covers the issues faced by embedded developers. Information here shows how your team can benefit by having this seminar presented at your facility.

According to Electronic Design, the average salary for EEs in the USA rose from $105k to $114k in the last year.

Latest blog: Voice of the Crystal, and Instruments of Amplification - two books about building your own components. Including vacuum tubes and transistors.

Quotes and Thoughts

"It's not what you make, it's what you save." Eric Hoellen quoted by Studs Terkel in "Working."

Tools and Tips

SEGGER emPack The complete OS for embedded systems

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

All About Circuits has a nice on-line basic electronics resource here.

Need to destroy 3000 hard drives per hour? There's an app for that. And there's a 64 page NIST spec on ways to "sanitize" various forms of electronic media.

Martin Thompson suggested:

You and your readers might find this an interesting tool for quickly performing experimental parsing of binary data: http://triq.net/bitbench for quickly performing experimental parsing of binary data. There's more info here.

Freebies and Discounts

Five years ago I reviewed Voltagestandard's DMMCheck, a nifty and very accurate voltage and resistance standard for checking the accuracy of a DMM. Alas, this product is no longer available (though they have other, better, options). However, there's a Chinese version on eBay, which I review later in this Muse.

Enter via this link.

A Nice, Very Inexpensive, Voltage Reference

Micheal Salem sent me a link to a Chinese voltage standard one could use to calibrate DMMs (see picture above) . That's an eBay link; if it goes away, search the site for a "Voltage Reference Module AD584kH." For $16 I just had to check it out.

The unit generates one of four precise voltages: 2.500, 5.000, 7.500 and 10.000. It came without a stitch of documentation, but the eBay listing claims it's lithium battery can be charged with a USB cable. There is a USB connector on the top of the board.

You can see four red LEDs along the bottom of the PCB. Pressing the "on" switch causes the leftmost one to come on, indicating 2.500V is selected. Pressing the "on" button again advances the unit to the 5.000, then 7.500 and finally 10.000V settings. A long press turns it off.

It's hard to see in the picture, but the "on" switch is in the middle of the LEDs. It's inside the plastic case. You can just make out that the case is routed near the switch. You press that routed plastic, which bends, and activates the switch.

Old-timers will remember when the Altair 8800 sold for $400, yet the 8080 CPU chip ran $360 alone. That puzzled many. Well, there's a similar conundrum with this voltage standard. It's $16, but uses an Analog Devices AD584KH precision voltage reference. That part is $20 in 500 unit lots (the highest quantity listed on Mouser). Perhaps they are buying these in vast quantities. Perhaps they don't quite meet specs. Or, maybe they get them cheap as Analog says they are not recommended for new designs.

Apparently the manufacturer checks each one against an Agilent 34401A 6.5 digit DMM. The actual outputs are listed:

My 5.5 digit DMM found the voltages were exactly the ones listed on the calibration sticker (minus the last digit).

Putting a 1k resistor on the 10V setting dropped the output 2 mV.

Interestingly, there are large holes drilled in the PCB all around the AD584KH, probably to isolate that part from the rest of the board:

Not a bad deal for $16.

My experience with this device and the DMMCheck is that my meters are spot-on, at least at the few voltages these standards produce, yet they haven't been calibrated in years. Many years. But it's nice to know they're measuring correctly.

Advice to a Young Firmware Developer

Probably the biggest source of emails I get is from people struggling to get into this field, and young folks looking for career advice. Most are primarily interested in firmware development. I wrote a long article about this some years ago, and have updated it many times since. But there are some meta-musings that are really important. Here's a few thoughts:

Learn, in detail, how a computer works. You should be able to draw a detailed block diagram of one. Even if you have no interest in the hardware, it's impossible to understand assembly language and other important aspects of creating firmware without understanding program counters, registers, busses and the like.

Learn an assembly language. Write real programs in it, and make them work. Absent a grounding in assembly much of the operation of a computer will be mysterious. In real life you'll have to delve into the assembly at least occasionally, at least to work on the startup code, and to find some classes of bugs.

But do learn some other languages. Python can be useful for scripting. Ada gives a discipline I wish more had.

Work in a cross-development environment with an embedded target board. It's very different from using Visual Studio.

Get comfortable with a Linux shell. With sed, awk, and a hundred other tools you can do incredible things without writing any code.

Take the time to think through what you're building. It's tempting to start coding early. Design is critically important. Remember the old saying: "if you think good design is expensive, consider the cost of bad design."

Monitor your bug rates. Forever. Skip this and you'll never know two things: if you're improving, and how you compare to the industry. We all think we're great developers, but metrics can be a cold shower.

Always be on the prowl for tools. Some are free, others expensive, but great tools will make you more productive and less error-prone. These change all the time so figure on constantly refining your toolbox.

Did you know the average firmware person reads one technical book a year? Yet this field evolves at the pace of Moore's Law. Constantly study software engineering. We do have a Body of Knowledge. Every year new ideas appear. Some are brilliant, others whacky, but they all make you think.

Learn about the hardware. At least get a general understanding. An engineer who can use an oscilloscope and logic analyzer for troubleshooting code is a valuable addition to a software team. Digital and analog hardware is cool and fascinating.

The final and probably most important bit of advice I give to young folks: Pay off your credit cards every month. And save money. Nearly half of Americans can't deal with a $400 emergency. Money in the bank gives you freedom. Debt is bondage. Save $250/month at today's CD rates and you'll be a millionaire around age 40.

On Legacy Code

Write functions using a coding standard. Inspect all new code. Lint your C before testing. Following the precepts of eXtreme Programming, develop tests in parallel with writing the code itself.

All sound advice. Yet worthless for many.

A lot of developers are slaves to an extant code base. They spend most of their time tweaking a few lines here and there to slip in a bit of new functionality or fix a bug. It may be rare to start a new project with a clean slate.

Firmware is the most expensive thing in the universe. A boss managing hundreds of thousands of lines of legacy code will be loathe to chuck the lot and start over unless there's an utterly compelling business need. It's likely we'll hear: "Yeah, I know it's all crap, but can't you just tune things a bit to keep marketing off my back?" So developers slave away at a code base whose entropic chaos escalates maintenance to a new Olympic sport, wishing they could start over, but bound by the inescapable chains of a ship date.

I suspect this problem will grow worse with time. More developers will become mired in maintenance. Fewer new products will start with an entirely new set of code; it'll be more common to torture working software into a different application.

Every year the inventory of legacy code grows; those working for established companies who are both cursed and blessed with lots of working software will be forced to find ways to use the legacy programs as the base for new products. Like the US automakers who labor under the penalties of old pension liabilities, established high-tech companies with big software inventories may start to mirror the rust-belt industries of yesterday. Instead of being saddled with too much antiquated capital equipment, the very technology that established their success will start to become the millstone of failure.

Various authorities claim that firmware doubles every ten months to two years. Where will all that code come from? When a company has millions invested in their legacy products, the pressure to use that code will be irresistible, even when that code might be simply awful.

What do you think? When was the last time you started a project from scratch?

This Week's Cool Product

The forward voltage drop of a simple silicon diode is in the neighborhood of 0.7 volts, depending on current and temperature. That's a lot to give up when designing solar chargers, reverse-current protection and the like. Some engineers prefer a Schottky diode as they offer a much lower drop. Anecdotal tales suggest this is on the order of 100 mV, but the truth is less pleasant. Here's a typical Schottky diode:

Shottky diode Vf

At low currents the forward drop is tiny, but even at 10 mA and room temperature we're approaching half a volt.

Analog Devices just announced an "ideal" diode, one that is close to perfect. The same graph for their LTC4376 looks like (though the axes are reversed):

The vertical axis is the forward voltage drop (ΔVSD - you'll see why they call it this and not VF in a moment). At a 3A load there's only 40 mV across the diode! (The curve is a "typical" rating, so be a bit wary. Unless I missed it the datasheet does not list a worst-case ΔVSD. The closest max number is the RDS(ON) at 7A, which is 30 mΩ. RDS(ON) is the resistance across the device, so at 7 A the worst-case forward voltage drop is 210 mV. I'm sure that's significantly lower at lower currents, but wish it had been parameterized.)

It's available in a 5x4mm 17-pin plastic package because, with such a small VF, uh, ΔVSD, there's practically no power dissipation in the device. 17 pins? Weird, until you look at the suggested PCB layout. There are 8 inputs and one monster output to handle seven amps:

How do they achieve such a low forward voltage drop? Instead of an anode and cathode, current flows through a low RDS FET driven by some smart analog. So ΔVSD is the difference between the FET's source and drain voltage.

The diode has a number of controls and modes; the datasheet has a lot of fascinating applications and circuits.

Downsides? Well, it needs at least 4V to do anything. And at two and a half bucks in thousands, they're pricey. But it is interesting to see how adding smart analog can make a simple normally imperfect component nearly ideal.

Note: This section is about something I personally find cool, interesting or important and want to pass along to readers. It is not influenced by vendors.


Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

Note: These jokes are archived here.

This is an old story I wrote in 2004, which seems appropriate as the elections settle down in the USA:

News Analysis
By Daniel Shorrt

President-elect Bubba "the can man" Jones expressed surprise at his unexpected win in last night's quadrennial elections. With 100% of the precincts reporting in, Jones overwhelmed the incumbent by acquiring an astonishing 65.536% of the vote. His opponent garnered just 1.024%, with the rest strangely going to an as-yet unidentified candidate named "hckergrrl".

Seeming confused by the crush of reporters and well-wishers Mr. Jones continued to hover protectively over his shopping cart. Suspiciously eying members of his new Secret Service detail he was heard complaining about attempts to take his collection of aluminum cans and old clothes. "Nobody ain't gonna rip off my stuff," he muttered. "Dem cans worth 2, mebbee 3 bucks."

Hustled into the waiting limo he expressed delight at the prospect of sampling the car's mini-bar. Tonight the president-elect remains in seclusion in the Blair House, hastily-recruited aides telling reporters he was "sleeping off the effects of a trying campaign."

Though it remains unclear just how Mr. Jones won the presidential election despite his not having been on the ballot, this reporter has seldom seen an election run in such an efficient and forthright manner. Contrast last night's speedy decision with the rancorous climate of just four years ago, where slim margins and problems with paper ballots led to a court's decision that displeased many.

Modern technology has eliminated the age of hand-counted ballots. "Hanging chad" will be nothing more than a colorful phrase recalled in history books. Recounts now take seconds. Never again will the courts decide an election. Call 2004 "The Year of the Electronic Vote."

Thank the citizens of Florida for our electronic elections. Misplaced ballots, poorly-punched selections, and perhaps even outright corruption held the 2000 presidential race in suspense for weeks. Determined to avoid scandal, the Sunshine State replaced thousands of antiquated manual machines with the latest of electronic vote counters. High tech touch screens instantly record each voter's decision, transmitting the results to a national database when the polls close. Recounts involve nothing more than a retransmission of the data, since we know the computers themselves are deterministic, immune from fraud, and cannot make mistakes.

Initial trials in the 2002 Florida primaries seemed less-than-promising. System crashes, locked up touch screens, and confusing instructions held the McBride/Reno contest in abeyance. Yet a winner did emerge, to tackle the incumbent in November's general election. Call 2002 a prototype of success, one that blazed a trail to this new and better form of e-Democracy that our children will inherit.

Luddites suggest that we rely too much on software for critical systems, sometimes referring to the Navy's unfortunate Smart Ship program. Recall, though, that after the sixth fleet attacked Palau in Micronesia earlier this year, the review board showed the logic of that autonomous decision: the computer realized that Palau lacked the ability to shoot back. "This clearly shows the efficacy of a system designed to win wars," Admiral Decobocker noted, "the system created a war that it could not lose. The subsequent divide overflow that crashed the flagship's propulsion system was entirely unanticipated."

(Reports indicating the ship was overwhelmed by an unruly band of Caroline Islanders who turned it into a floating casino bar remain unconfirmed).

Others mentioned last month's recall of 600,000 Internet-ready automobiles when hackers found they could deflate the cars' tires, reprogram the engine controllers to emit constant backfires, and lock the windows up and doors closed with the heat on "high". Detroit reluctantly admitted to a buffer overflow problem but stated these were minor problems blown out of proportion by the press. Spokesman Graeme Kirchner once again urged passage of the National Anti-Hacking Act, saying "these bloody unsupervised latch-key kids run rampant through the computer networks. Parents should keep them bloody well away from the bloody home computer."

To get to the truth of software risks I interviewed Tom Thorten, lead software engineer of the SmartyVote system employed so successfully in this week's elections.

Me: "Mr. Thorton, isn't this device what you folks call a safety-critical system?"

TT: "Nah, this isn't a safety-critical app. That's for avionics and nuke plants. No one's gonna die if this thing crashes. Hey, if there's a problem with this box ya just reset and carry on. Pretty much like any chunk a code."

Me: "How can you ensure the vote is tabulated correctly?"

TT: "Vote? Is that what this thing does? They just told me what to put on the LCD and how to log the data. The spec was pretty light, know what I mean?"

Me: "No."

TT: "Well, it's like any system. The boss is wrapped up with making promises to customers. Her boss is usually off testifying somewhere, and the Big Boss is still awaiting extradition from Barbados. It's pretty much up to me what this puppy does."

Me: "How was this tested?"

TT: "Oh, the usual, ya know? We pounded on it some, then Bob's kid (ya know the one? Cute little tyke?) played with it for a while. It was hell getting the ice cream outa the PCB. Then we stuck it down in Florida in 2002. Worked pretty good, what I hear. Few probs, but as soon as we get some more developers we'll work out those kinks. We figured most of those old folks wouldn't hit the screen too fast, but when they did, man that ole priority inversion thing hit bad, know what I mean?"

Me: "No. What happens to the data?"

TT: "Oh, we toss it into one of those embedded DBs, a cool relational thing. Sure wish it was reentrant; those corruption problems are killing us. Then at the end of the day we TCP it to Republican National Headquarters. Or was it the Dems? One of them, they keep changing the IP address on us. Little do they know - we stuck in a back door that lets us set a new IP at will. Saves a recompile every time they change their freakin' minds. Those dudes then dump it all into the National Database. But our box logs the data and ships it out. After that it's not my problem."

Me: "So if these things are on the net, what about security?"

TT: "Well, I guess some folks might want to hack their way in, but it's pretty unlikely. We're running Embedded Win eXtraP, the most secure Win OS ever. It's so good it keeps downloading patches and fixes; hey, the system we ship evolves to something else over the course of a year. It's like, organic, man. No one really even knows what the code base looks like now. Pretty awesome, know what I mean?"

Me: "So no one can break in?"

TT: "Nah, probably not. Well, not at least if the users had a brain. I can't figure out why our customers never seem to set up a password like the read.me tells them to. Half them systems are wide-open! You know users, though. It's like my cable modem, you know? I see all sorts of open systems on the cable. My kid even ripped off some lawyer's Amex numbers from a Word doc on this dude's exposed C drive. Sure am proud of that little rug rat."

Me: "In Florida some people struggled with system lockups. What happens then?"

TT: "Oh, jeez, customers complain about every little thing. Just reset it, for Pete's sake."

Me: "But what happens to all of that data?"

TT: "Bit bucket, I guess."

Me: "Huh?"

TT: "/dev/null. You know."

Well, the arcane language of the computer professional was something we English majors didn't learn in Princeton's class of 1938. But I feel it's only proper to delegate the complexities of encryption, computers, and corporate accounting to those professionals who know these things best. Proper government oversight, as mandated in the Public Security and Trusted Computing Act of 2003, insures the nation's interest will be met. Mr. Thorten assured me that the Transportation Security Administration will start examining his code base "real soon now".

This reporter would argue that every vote counts, just as it did when the Founding Fathers brought forth this great nation more than 200 years ago. I see no reason why our scientists can't employ the same technology that so accurately guided the Mars Climate Observer to the surface of Mars, or that enabled Armstrong's triumphant Apollo 13 landing, to ensure every vote counts.

The new technology of electronic voting insures every person is properly represented. Your touch-screen selection instantly tips the balance in favor of your candidate. The margin for error, due to the supreme number-crunching power of the computer, is surely zero. I look forward to the day I can vote from home using the power of the Internet and the security inherent in the most popular operating systems.

This is Daniel Shorrt. Thank you.

Advertise With Us

Advertise in The Embedded Muse! Over 28,000 embedded developers get this twice-monthly publication. .

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.