Go here to sign up for The Embedded Muse.
The Embedded Muse Logo The Embedded Muse
Issue Number 368, February 18, 2019
Copyright 2019 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact info@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Editor's Notes

Express Logic

Over 400 companies and more than 7000 engineers have benefited from my Better Firmware Faster seminar held on-site, at their companies. Want to crank up your productivity and decrease shipped bugs? Spend a day with me learning how to debug your development processes.

Attendees have blogged about the seminar, for example, here, here and here.

Jack's latest blog: In April, GPS Will Have its Own Y2K Problem

I'm on Twitter.

Quotes and Thoughts

Cost estimating deals with truly complex numbers: each having a real and an imaginary part.

Tools and Tips

SEGGER emCompress Real Time Compression

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

Mike Skrtic sent this link to a video featuring Grace Hopper handing out nss (sorry, I don't know what the acronym plural of ns is!). She wants to hang a coil of µs (300 m of wire) around the neck of every developer who says "oh, it's just a µs, no big deal."

Microsoft reports that 70% of their bugs or security issues (the article is somewhat confused about this) are memory-safety errors. Now, as embedded folk we're not all that interested in Windows's woes, but what strikes me is how this shows the importance of quantifying bugs. If you know the bulk of problems stem from issue X, then wise teams will develop strategies to proactively look for X. Conversely, if you don't quantify bugs, and thus don't know how errors in your project cluster, you'll have to find every last one using the least efficient debugging strategy: debugging.

Jeremey Overesh, replying to a trick in the last Muse, wrote:

I just want to bring up a word of caution on the helpful shadow registers for 16-bit timers on an 8-bit device. A significant number of Atmel AVR processors have a similar thing, where a 16-bit timer has a shadow register for copying data. Unfortunately, this register is shared among all capture and compare registers in that timer. Therefore, say that you read the first half of the Timer0 Compare register, then get an interrupt, which reads the Timer0 CAP0 register, you get garbage in the second half of your Compare read! These 16-bit read/writes *must* have critical code segment wrappers if you're using that hardware in more than one operating context.

Freebies and Discounts

Courtesy of the fine folks at Keysight, this month's giveaway is one of their brand new $2200 DSOX-1204G bench 200 MHz 4-channel scopes:

Keysight DSOX1204G oscilloscope

They sent two; I'll give the other away next month.

Enter via this link.

The Last Word (For Now!) on AI

A number of readers wrote about autonomous cars in response to the last two issues' comments on AI. The thinking is that drivers will rely on these driver-assist features to the point where any failure, or even unexpected action, by them may cause havoc.

Maybe the best perspective on this goes back a half century to the days of the Apollo program. I highly recommend Digital Apollo, by David Mindell. In it he examines how designers of the spacecraft didn't think in terms of the pilots by themselves in control, nor did they consider complete automation. The answer was a very clever bit of integration of the astronauts and the digital control systems. One could not function without the other; the final system was a meld of computer and human. Maybe that's the near-term future of smart cars.

David Reynolds has a fascinating perspective:

I have enjoyed your musings on AI. However I see something lacking in almost all articles and papers I read on AI these days: Save turning it off entirely, it can be really hard to correct errant AI. The consequences of this can be dire.

Consider this hypothetical...

You've spend years and enormous amounts of money developing your AI (teaching it and/or waiting for it to learn). Now you depend on the AI for your very existence (a very rewarding existence indeed, thanks to the AI). But one day you discover the AI is making a mistake - a mistake everyone agrees is intolerable. What do you do?

You try to make the AI unlearn the mistake of course. But it turns out that for complex AI constructs, unlearning mistakes can be far more difficult than learning them, and unlearning something can affect other things in unpredictable ways.

So now the AI mistake is intolerable, but you cannot easily and/or safely correct it. You've painted yourself into a corner. The only course of action remaining is to pull the plug; a very costly act indeed, especially if society has come to depend on the AI.

These complex AI constructs are like humans in a lot of ways, but they're also different in very dangerous ways: Individual humans who make errors or have bad beliefs are small, numerous, yet distributed. That (usually) makes them easier to control. A complex AI construct is big, unique, yet omnipotent. It is very hard to control without simply turning it off. [This is not to say that bad humans when organized into large groups cannot be very dangerous too. History bears evidence of that.]

So like everyone else these days, ponder all the wonders and benefits of AI. But unlike everyone else, give equal weight to considering what happens when AI goes wrong.

Dual Sweep in the Olden Days

The last Muse had a review of Keysight's new DSOX1204G scope. I wanted to include the following but ran out of room.

A modern digital scope's display is merely a computer's output; a monitor on which anything can be drawn. How did scopes work in the pre-processor age?

One interesting challenge was displaying two channels at a time, something we take for granted today. There used to be three options:

  • Use a CRT with two beams. The CRT had two independent electron guns shooting beams at a common phosphor screen. While not really rare, you didn't see a lot of these around.
  • Set the vertical mode to "Alternate." A single beam CRT would display channel 1 first, then add a DC offset to show channel 2 at a different place on the screen. This happened so fast it looked like there were indeed two individual beams.
  • Go to "Chop" mode. The scope would display a square wave, but not trigger on it. The waveform and its associated rise/fall time was so fast it looked like two traces. Then, the scope would modulate the positive step of the square wave with one channel and the negative step with the other. The result looked like two traces, but triggering mishaps could reveal the underlying square wave.

Chop mode didn't behave well at high sweep rates, and alternate was epilepsy-inducing at low rates, so the user selected the mode that matched the measurement.

According to http://rtellason.com/catalogs/Tektronix-1959.pdf Tektronix's very popular 545 scope, plus the vertical plug-in, cost about $2700 in 1959 dollars, or over $23k today. That 70 pound monster offered 30 MHz bandwidth. In 1970 their 7704 150 MHz instrument http://bitsavers.trailing-edge.com/pdf/tektronix/catalog/Tektronix_Catalog_1970.pdf was found everywhere. With four decent plug-ins it ran around $50k in today's degraded dollars.

We truly live in a golden age of test equipment.

Joulescope Review - An Energy-Measuring Instrument

The folks at Jetperch are firing off a Kickstarter campaign for their new Joulescope energy-measuring tool. Matt Liberty of Jetperch sent me an early unit for testing. Production units will be available in June.

The Joulescope is another entrant in the fray of tools that measure energy consumption of IoT-type devices. (See this for my reviews of competing products). When comparing DMMs or oscilloscopes feature sets are more or less the same; in the energy-profiling world they vary tremendously. First, many don't measure energy at all; they often log current. Remember that power is current times voltage, and energy is the integral of power. Energy is what's important when looking at what a battery can give before it needs to be recharged or replaced.

The unit is powered from a USB connection to a host computer (its application will run on Windows, Linux and Macs). There are four banana jacks: two for the IoT device's power, and two to supply power to the device. The instrument is inserted in the device's power lines where it measures voltage and current, deriving power and energy from these numbers.

The Joulescope

I would have preferred binding posts that accept wires as well as banana plugs, like power supplies have, as these would give more connection options. Interestingly, though, the front panel can be swapped to permit different kinds of connections. The unit I received also had a panel which used USB connections rather than binding posts. And it even came with a torx screwdriver to change panels.

First, some specs:

  • Range: just over a nA of resolution to 3A, with very short bursts up to 10A possible
  • Sampling: 2m Sa/s, with an analog bandwidth of 250 KHz (more on this later)
  • Burden voltage: 25 mV at 1 A. (This is the drop internal to the instrument)
  • Measurements: V, A, W and joules

Interesting factoid 1: the host USB side is electrically isolated from the measurement side, so there are no ground loops. I couldn't resist the lure of dismantling the thing, and sure enough, there appears to be optical or other isolation electronics separating the two sides of the PCB. (I do miss the good old days when components were big enough one could read the part numbers!)

Factoid 2: It comes with an Arduino test board and power supply that runs a current profile so a new user can get familiar with its operation.

The Joulescope covers a phenomenal range of some 10 orders of magnitude. That's not practical using a single range, and, indeed, the unit has seven. It switches ranges automatically - and for all intents instantaneously. I couldn't trick it into showing a range-switching glitch.

The bandwidth is rated at 250 KHz, but as every analog designer knows the gain-bandwidth product of op amps limit what's achievable for reasonable prices. The Joulescope's bandwidth depends on the range, with the three most sensitive having degraded numbers:

Bandwidth figures for the Joulescope

I don't think the lower bandwidth at very low currents is an issue because these mostly represent sleeping modes for an MCU. Generally, sleep times are long and there's not enough interesting activity to monitor.

When powered up the application defaults to a DMM mode which updates at 1/2 second intervals:

Joulescope DMM display

The unit appears to use a 14-bit ADC and, depending on displayed voltage or current, the last digit or two appear to be mostly meaningless. Disregarding that, the numbers are spot-on according to my 5.5 digit DMM. Notice the blizzard of statistics!

I've heard the argument that there's no need to measure voltage for these sorts of instruments as V is a constant. But that simply isn't true. Operating off a battery? The voltage declines with charge level. A battery has internal resistance, so the voltage delivered to the load depends on load current. And some systems (like your mobile phone) vary V depending on computational needs.

Much more fun than DMM mode is the oscilloscope display which continuously streams data, storing the last 30 seconds worth. The update rate is blazing, faster than any USB scope I've used.

Joulescope scope display

This picture is a bit hard to read as I had to shrink it to fit the Muse. First, the current current is displayed in a DMM-like window (I could have selected voltage, watts or joules). The top display is current vs time, the bottom voltage vs time. Statistics at the cursor are shown (min, max, current value, standard deviation and time).

Here's where it gets a bit hard to understand, though it's worth digging in a bit as the apparent fog is really showing the power of the instrument. The Joulescope acquires 2m samples/second. In this example the display shows 20 seconds of data, with that data "binned": multiple acquisitions constitute each displayed point. Three traces each, for current and voltage, show the max, average, and min value in each bin. This thing is acquiring and displaying a lot of data.

As you zoom in to the maximum resolution min and max go away and only the actual acquired current or voltage at that point is shown.

Why show min and max? It's so very short events, such as servicing an interrupt, don't get lost. Sans min/max, you'd have to go to max resolution and pan like mad to go through all 2m points to find short events.

I mentioned the huge dynamic range of the unit. The following two pictures are portions of the oscilloscope display. I drove a transistor with a pulse generator to alternate between a tiny and a heavy load:


Though this experiment doesn't begin to tax the Joulescope's range, it shows the "target system" going from just a few µA to 63 mA (four orders of magnitude) in a handful of microseconds. Wide dynamic range and decent resolution are critical for profiling systems that sleep, wake up, and do coulombly-expensive things like comm or actuating something mechanical.

Two issues arose in my testing. First, the Joulescope buttons buttons are used for scrolling and panning, and I found them baffling. There's probably an obvious trick I missed.

Second, and more important, I feel the unit needs a trigger of some sort so one can use another tool to correlate current with what the code is doing. Matt tells me early users typically capture 30 seconds worth of data and then hunt around in the buffer looking for interesting events or places to optimize power use.

As I wrote, the Joulescope won't be on the market till June, and a number of features will be added. The current list is here . Turns out, all of the complaints I had are on that list. Included: Doing something about the buttons, and adding some sort of trigger mode.

In June the Joulescope will be priced at $799. Kickstarter offerings are much less.

The embedded world is ever changing, and one of the memes today is running devices from small batteries for years or even powering electronics via energy harvesting. We will need tools like the Joulescope to understand where the coulombs are going, and I suspect this will be a popular tool when it hits the market.

Because engineering without numbers isn't engineering. It's art.

RIP Opportunity

NASA regrets to announce that Mars Opportunity has passed away. After falling into a coma in June 2018 it succumbed to energy loss and hypothermia and was pronounced dead February 12, 2019.

Opportunity had an unusually long life, exceeding the anticipated four score and ten days by fifteen years. A peripatetic wanderer, the 185 kg Opportunity ambled vast distances, covering 45 km despite a slow pace of about 1 cm/s. Often cold, Opportunity's heaters kept it comfortable most days. An intellectual lightweight, its 20 MHz brain proved more than adequate to assist Opportunity in its leisurely promenade.

Late in life Opportunity suffered from progressive dementia which no medical ministrations could halt. It was eventually only able to keep short-term memories when engineers found flash failures and redirected experiences into RAM.

Opportunity received many accolades, perhaps the most enduring was the naming of asteroid 39382 Opportunity in its honor.

Opportunity was predeceased by sibling Spirit in 2011. It is survived by Mars Curiosity.

NASA and the engineering community mourns its passing but acknowledges the great science Opportunity beamed down for so many years. Donations for successor rovers will be sent to the Internal Revenue Service.


Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

Note: These jokes are archived here.

From Tomasz Kawala:

The era of 8-bit embedded systems might be half-way gone, but for us humans, 8-bit capacity is going to last a long time, with one field in reserve.

I realised that eight bits is enough to flag progress through the years of human life, and my flags are going as follows:

  • 1 - walking
  • 2 - talking
  • 4 - socialising
  • 8 - reading/writing
  • 16 - 1st declaration of independence
  • 32 - taking some responsibility
  • 64 - recognising accountability
  • 128 - reserved for future use 

128 ... see you there! 

Advertise With Us

Advertise in The Embedded Muse! Over 28,000 embedded developers get this twice-monthly publication. For more information email us at info@ganssle.com.

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster. We offer seminars at your site offering hard-hitting ideas - and action - you can take now to improve firmware quality and decrease development time. Contact us at info@ganssle.com for more information.