By Jack Ganssle

Get Some Rocks

Published in Embedded Systems Design, June, 2009.

For thousands of years the moon has beckoned. But four decades ago a dozen men walked on its surface. They left, and we lost interest in sending people into deep space.

What happened to those guys?

This month I read Andrew Smith's Moondust (Harper Collins, 2005, NY, NY), which purports to be the story of the later lives of the moonwalkers. In truth, the book is a well-written but ultimately disappointing narrative of the author's search for what he thinks the astronauts should feel after waltzing on another planetoid. An unabashedly unspiritual man, Smith chases down these aging men in an effort to find how their journeys through space could instill some glimmer of happiness into his apparently barren life. Not recommended.

But he does paint some interesting portraits of the post-Apollo lives of some of the moonwalkers. I have often wondered what became of the others, the lonely six-pack left in lunar orbit, who quietly circled the moon in relative anonymity while their more famous colleagues left their footprints in the surface's dust.

Just weeks after I finished the book, Ken Mattingly, one of those Apollo Command Module Pilots, gave the keynote speech at this year's Embedded Systems Conference. I arrived a bit early. Patrick Mannion, who works for the Conference, sat down next to me and we chatted. "Do you want to meet him?" he asked? Well, of course! But I declined, not wanting to be another in a four decade line of admiring people who undoubtedly ask the poor man the same questions.

Admiral Mattingly strode onto the stage, fit and hale despite his 73 years. He spoke brilliantly, engagingly, and captivated the audience.

As anyone who reads space history - or watched Tom Hank's movie - knows, Mattingly was on the prime crew of Apollo 13, the mission almost lost due to the explosion of an oxygen tank. Possibly exposed to measles pre-launch he was bumped from the flight. Before the catastrophe he was pretty bitter about losing his slot on that mission, and said: "When I grow up I want to be Gary Sinise [Sinise played Mattingly in the movie], who gave an amateur performance of being miserable. I gave a truly Oscar-worthy performance!"

His talk skillfully blended the movie and the reality, giving a gripping description of what it was like to sit atop the Saturn V's 7.5 million pounds of thrust and of the subsequent coast to Luna and return to Earth.

How did he glean such a graphic understanding of such a mission? After all, his apparent claim to fame is that he did not fly on 13. Neither did I.

Well, there were the simulations, which were orchestrated by the sim supervisor, who: "Had no morals or scruples." Each run was more difficult than the last, till that wretch tossed so many failures into a simulation the astronauts would finish up drenched in sweat.

Was the movie accurate? Pretty much so, it seems. He feels the movie "missed the majesty of what was done." Instead of the relatively few key characters Hanks portrayed there were thousands of people involved in rescuing the crew. And Mattingly didn't work for days in the simulator trying to find a power-up sequence that worked: the engineers spent many, many hours devising a procedure which he then quickly simulated. It worked. That's rather like programming: they got it right, and used the test to prove it was right. Uh, actually, a better comparison is the one Hanks depicts: debug like mad, hack it together, if it sort of works, ship it.

Mattingly is an engineer and he talks like one of us. "The anomaly investigation" (i.e., what exploded), "compute trajectory to go around the moon and meet reentry parameters at the earth" (head home) and "tape lithium hydroxide cylinders to the suit loop on the LM."

Everyone was worried about bringing up the command module's inertial platform after days of soaking in the deep freeze of an unheated spacecraft. The unit was designed for a temperature range of 70 to 70.5 degrees, though had been tested over extremes of 68 to 72. But a young engineer admitted that many months earlier he had removed a platform and was carrying it his station wagon to the Cape. An unexpected ice storm forced him home; the platform, forgotten, sat in the car in the cold overnight. The next day, covering his tracks, he reinstalled it without telling anyone, and this now-revealed secret gave the team confidence the system would work.

That young engineer reflected the average age of those who built Apollo: 25 to 26. Managers were in their early 30s. The astronauts, in their late 30s and early 40s, were the old men of the project. And now they are indeed old men.

I felt like I was along for the ride, due to Ken Mattingly's wonderfully descriptive narrative fueled by his extensive training for Apollo 13. It probably helped that he did indeed get to the moon, as CMP on Apollo 16, and into space on two Shuttle missions. Oddly, he never mentioned any of those three space flights. The careful listener would conclude his service ended with 13, while I suspect those later flights seed the memories he must have when looking skyward. How much did the failure, made famous by a fabulous movie, shape his life. or at least public perception of his life? That, alas, is yet another story that neither Moondust author Smith nor Mattingly explored.

We remember JFK's speech that started America on the path to the moon. Mattingly said it better: "Get some rocks. Come back. Before 1970."

Now those are clear and measurable requirements!

Ken Mattingly wasn't the only attraction at the ESC, of course. There were so many other things going on I could only surf a few of the highlights.

One overarching them on the show floor was low power. A number of booths featured technology for energy harvesting, for instance. One, Cymbet, was so swamped by fascinated engineers I was never able to get close enough to get a sense of their technology. But their website (http://www.cymbet.com/) describes a battery-on-a-chip in a 5x5 mm package. Way cool.

Microchip introduced several lines of 8 and 16 bit PICs sporting "eXtreme Low Power" (XLP) technology. These parts have a deep sleep mode which consumes a mere 20 nanoamps. That's so little power that some battery-operated systems (assuming they mostly sleep) can operate for up to 20 years on a single battery. It's not your average AA cell, of course; these are lithium thyionyl chloride batteries, but not having to swap power sources for an entire human generation can be extremely cost-effective.

Deep sleep turns basically everything off on the PIC. Even the SRAM loses power, and thus all variables disappear (unless written to non-volatile memory). But that's not a problem in many instances: a smoke detector, for instance, does not need to maintain any state information.

Other sleep modes use more power but leave more components running. For instance, if you need to keep a real-time clock active add about half a microamp.

There's more about the XLP technology at microchip.com/xlp. The company promises to enhance that site with many design articles about power management, though as of this writing it's a bit thin on content.

Xilinx, too, sees opportunities in both the recession and in power reduction. ASICs are just too risky and expensive for many of us today, making FPGAs ever more attractive to many designers. Their new Spartan-6 and Virtex-6 FPGAs offer a 65% reduction in power over the previous generation devices, and the company offers a wide line of these devices that fit both lower-cost applications as well as the most demanding mega-logic needs.

Then there's Cypress, long a provider of innovative chips like their PSoCs, which are FPGA-like in that they offer programmable logic, but also include programmable analog. But they're now into the power business as well, though their new PowerPSoC parts are designed for high power applications. How much can your logic drive? A few milliamps? The PowerPSoC devices can drive up to 4 amps! That's right out of the IC, no additional MOSFETs required.

I had an interesting talk with the folks at AdaCore, a provider of Ada tools. Though Ada's market share in the embedded space seems perpetually stuck in the single digits, the language really does make a lot more sense than C. Ada just doesn't let one make stupid mistakes, while C encourages them. Sort of like the sim scenes in Apollo 13 where Mattingly is hacking together a LM power-up sequence.

One reason Ada hasn't dominated is that it doesn't work on smaller systems. Except that AdaCore now has a toolchain for Atmel's 8 bit AVR microcontrollers. Their "zero-footprint" runtime eliminates some Ada features (like exception handling and tasking), which means there's no size or performance hit compared to C.

AdaCore has formed a relationship with Praxis, the company behind the SPARK language. SPARK is a subset of Ada - any Ada toolchain will compile SPARK. But it ensures a "correctness by construction" approach to programming. One annotates the source files with formal descriptions of the code's intent, which a Praxis-supplied tool analyzes to ensure the code does what it is supposed to do. Bottom line: SPARK code's bug rate is so low, even before debugging starts, that it's difficult to measure.

But wait: there's more. All of the SPARK tools are now integrated with AdaCore's development environment. And they're all open source.

I sat through a Microsoft presentation about real-time issues with Windows CE version 6.0. Windows and real-time? Well, the talk was a bit muddled but as I understand it Microsoft claims that, on a 200 MHz ARM, CE starts an ISR 1.2 to 13.3 microseconds after an interrupt. That presumably means 13.3 usec, since wise engineers always design for worst-case scenarios. In CE an ISR can do real work, as in any interrupt-handling environment, but it's generally preferred that the ISR initiates an Interrupt Service Thread (IST). On the same platform ISTs start between 31.7 to 103 usec after an interrupt occurs. That's not going to win any NASCAR awards but is fast enough for many applications. But I got confused when the speaker quoted interrupt jitter figures for CE on a 250 MHz ARM of between 200 usec and 2 msec.

The RTOS community had its share of announcements, too. Green Hills has reported a record $130m in sales last year, with an after tax profit of $30m. Those are astonishing numbers and a testament to the company's continued health and the demand for high-reliability operating systems.

Wind River, the giant of this industry, announced, well, nothing. The company didn't have a booth.

Express Logic (makers of the ThreadX RTOS) had a huge booth where they demonstrated the latest in their X-series of products: StackX. StackX analyzes your .elf file and computes exactly how much stack space your application will require. There are some limitations, of course. It can't handle recursion or a few other oddball techniques. But StackX does change the standard way off calculating stack size (take a wild guess and sacrifice a goat) to nearly a science. It works best in conjunction with ThreadX, but will analyze .elf files from any toolchain.

Finally, Micrium has released a completely new version of their uC/OS. uC/OS-III includes features like round robin scheduling and an unlimited number of tasks and priority levels. Interrupts are never disabled by the OS; instead, the RTOS accesses internal critical regions by locking the scheduler for very brief periods of time. uC/OS-II will continue to be sold and supported, and the new version works with all of the company's other products like their file system, USB stack, TCP/IP, etc.

The Best of Times
Dickens wrote about the worst of times and the best of times. I always enjoy the ESC, but this year's event, despite the recession, had more going on than ever. Unhappily I had to leave after just three days. There were plenty of other vendors I wanted to talk to.

In bad times good companies get innovative, and innovation at the ESC showed times are indeed really bad. And that will be good for these companies when the economy inevitably begins to recover.

Free Embedded Muse newsletter - twice/monthly, hard-hitting technical info that goes to 48,000+ engineers. Click here to sign up or enter your email here: