Hardware for Better Firmware

For hints, tricks and ideas about better ways to build embedded systems, subscribe to The Embedded Muse, a free biweekly e-newsletter. No hype, just down to earth embedded talk. 23,000 other engineers subscribe. It takes just a few seconds (all we need is your email address, which is shared with absolutely no one) to subscribe to the Embedded Muse.

Hardware for Better Firmware

In 1971 Intel shocked the industry by announcing their 4004, the first microprocessor. This little memory company (at the time) created the embedded industry more or less out of thin air.

Embedded Systems Programming magazine didn't exist; neither did Dr. Dobbs or any of the many publications now targeted at the microprocessor world. Publications like the now defunct Electronics magazine and EDN carried word of this new technology to their readers, most of whom were electronic engineers, not software people. The word "firmware" hadn't been invented.

EEs, though, quickly grasped the possibilities created by this new invention. It wasn't long before products were developed using the 4004 and its successor, the 8008. Virtually all of the development, both hardware and software, was done by EEs. It was common for a single engineer to design the board and write the code.

Today various agile methods stress the benefits of working in small teams. Back in the 70s one additional benefit was apparent: the designer, being so intimate with both the hardware and the firmware, could, and did, optimize both for best system performance. No hardware decision was made without considering its impact on the code, and where a bit of extra logic could drastically simplify firmware, well, that hardware was added without a second thought.

I remember working on scanning monochrometer designed by Dick Hayden, the most brilliant logic designer I ever met. Flat interference filters rotated in a beam of light to create a quite non-linear scan of IR frequencies beamed onto test sample. Our 1.3 MHz 8008 was, to say the least, limited in its ability to keep up with the A/D converter that sampled the photosensor. An analysis showed that we could probably write code that could keep up, but the system would be so loaded other housekeeping would suffer. Dick created a stunning state machine so intricate in its complexity that no other engineer really ever understood all of its implications. Yet in a synchronous design comprising but a handful of chips he created a circuit that modeled the system's mechanical motion, initiating A/D reads when appropriate, and parking the data into a FIFO made from RAM chips. That clever bit of hardware allowed us to write well-organized firmware, code that wasn't an abysmal mess tuned explicitly for sucking in data at the processor's max capabilities. It was a perfect example of designing hardware simply for the sake of improving the quality of the code.

This industry changed in the 80s. Bigger processors and cheap memory led to products bloated with features. Embedded development largely fragmented into discrete hardware and software groups. As the wall between these engineers grew the firmware folks often found they had little say in the hardware design, and were forced to accept designs that were practically impossible to debug, and which in some cases led to overly complex code.

So I'd like to make some suggestions for the hardware people, ideas that are simple to incorporate into the circuit board but which can greatly ease the task of building the code.

Diagnostics

In the non-embedded world a favorite debugging trick is to seed print statements into the code. These tell the programmer if the execution stream ever got to the point of the print. But firmware people rarely have this option. So, add a handful of unassigned parallel I/O bits. The firmware people desperately need these as a cheap way to instrument their code. Seeding I/O instructions into the code that drives these outputs is a simple and fast way to see what the program is doing.

Developers can assert a bit when entering a routine or ISR, then drive it low when exiting. A scope or logic analyzer then immediately shows the code snippet's execution time.

Another trick is to cycle an output bit high when the system is busy, and low when idle. Connect a voltmeter to the pin, one of the old fashioned units with an analog needle. The meter will integrate the binary pulse stream, so the displayed voltage will be proportional to system loading.

If space and costs permit, include an entire 8 bit register connected to a row of .1 inch spaced vias or headers. Software state-machines can output their current "state" to this port. A logic analyzer captures the data and shows all of the sequencing, with nearly zero impact on the code's execution time.

At least one LED is needed to signal the developer- and perhaps even customers - that the system is alive and working. It's a confidence indicator driven by a low priority task or idle loop, which shows the system is alive and not stuck somewhere in an infinite loop. A lot of embedded systems have no user interface; a blinking LED can be a simple "system OK" indication.

Highly integrated CPUs now offer a lot of on-chip peripherals, sometimes more than we need in a particular system. If there's an extra UART connect the pins to an RS-232 level shifting chip (e.g., MAX232 or similar). There's no need to actually load the chip onto the board except for prototyping. The firmware developers may find themselves in a corner where their tools just aren't adequate, and will then want to add a software monitor (see www.simtel.com) to the code. The RS-232 port makes this possible and easy.

If PCB real estate is so limited that there's no room for the level shifter, then at least bring Tx, Rx, and ground to accessible vias so it's possible to suspend a MAX232 on green wires above the circuit board.

(Attention developers: if you do use this port, don't be in such a panic to implement the monitor that you implement the RS-232 drivers with polled I/O. Take the time to create decent interrupt-driven code. In my experience, polled I/O on a monitor leads to missed characters, an unreliable tool, and massive frustration.)

Bring the reset line to a switch or jumper, so engineers can assert the signal independently of the normal power-up reset. Power-up problems can sometimes be isolated by connecting reset to a pulse generator, creating a repeatable scenario that's easy to study with an oscilloscope.

Connecting Tools

Orient the CPU chip so it's possible to connect an emulator, if you're using one. Sometimes the target board is so buried inside of a cabinet that access is limited at best.  Most emulator pods have form factors that favor a particular direction of insertion.

Watch out for vertical clearance, too! A pod stacked atop a large SMT adaptor might need 4 to 6 inches of space above the board. Be sure there's nothing over top of the board that will interfere with the pod.

Don't use a "clip-on" adaptor on a SMT package. They are just not reliable (the one exception is PLCC packages, which have a large lead pitch). A butterfly waving its wings in Brazil creates enough air movement to topple the thing over. Better, remove the CPU and install a soldered-down adaptor. The PCB will be a prototype forever, but at least it will be a reliable prototype.

Leave margin in the system's timing. If every nanosecond is accounted for no emulator will work reliably. An extra 5 nsec or so in the read and write cycle - and especially in wait state circuits - does not impact most designs.

If your processor has a BDM or JTAG debug port, be sure to add the appropriate connector on the PCB. Even if you're planning to use a full-blown emulator or some other development tool, at least add PCB pads and wiring for the BDM connector. The connector's cost approaches zero, and may save a project suffering from tool-woes.

A logic analyzer is a fantastic debugging tool, yet is always a source of tremendous frustration. By the time you've finished connecting 100 clip leads the first 50 have popped off. There's a better solution: surround your CPU with AMP's Mictor connectors. These are high-density, controlled impedance parts that can propagate the system's address, data and control busses off-board. Both Tektronix and Agilent support the Mictor. Both companies sell cables that lead directly from the logic analyzer to a Mictor. No clip leads, no need to make custom cables, and a guaranteed reliable connection in just seconds. Remove the connectors from production versions of the board, or just leave the PCB pads without loading the parts.

Some signals are especially prone to distortion when we connect tools. ALE (address latch enable), also known as AS (address strobe) on Motorola parts, distinguishes address from data on multiplexed busses. The tiniest bit of noise induced from an emulator or even a probe on this signal will cause the system to crash. Ditto for any edge-triggered interrupt input (like NMI on many CPUs). Terminate these signals with a twin-resistor network. Though your design may be perfect without the terminations, connecting tools and probing signals may corrupt the signals.

Add test points! Unless its ground connection is very short a scope cannot accurately display the high-speed signals endemic to our modern designs. In the good old days it was easy to solder a bit of wire to a logic device's pins to create an instant ground connection With SMT this is either difficult or impossible, so distribute plenty of accessible ground points around the board.

Other signals we'll probe a lot, and which must be accessible, include clock, read, write, and all interrupt inputs. Make sure these either have test points, or each has a via of sufficient size a developer can solder a wire (usually a resistor lead) to the signal.

Do add a Vcc test point. Logic probes are old but still very useful tools. Most need a power connection.

Other Thoughts

Make all output ports readable. This is especially true for control registers in ASICs as there's no way to probe these.

Be careful with bit ordering. If reading from an A/D, for instance, a bad design that flips bit 7 to input bit zero, 6 to 1, etc, is a nightmare. Sure, the firmware folks can write code to fix the mix-up, but most processors aren't good at this. The code will be slow and ugly.

Use many narrow I/O ports rather than a few wide ones. When a single port controls 3 LEDs, two interrupt masks, and a stepper motor, changing any output means managing every output. The code becomes a convoluted mess of ANDs/ORs. Any small hardware change requires a lot of software tuning. Wide ports do minimize part counts when implemented using discrete logic; but inside a PLD or FPGA there's no cost advantage.

Avoid tying unused digital inputs directly to Vcc. In the olden days this practice was verboten, since 74LS inputs were more susceptible to transients than the Vcc pin. All unused inputs went to Vcc via resistor pull-ups. That's no longer needed with logic devices, but is still a good practice. It's much easier to probe and change a node that's not hardwired to power.

However, if you must connect power directly to these unused inputs, be very careful with the PCB layout. Don't run power through a pin; that is, don't use the pin as a convenient way to get the supply to the other pins, or to the other side of the board. It's much better to carefully run all power and ground connections to input signals as tracks on the PCB's outside layers, that are visible when the IC is soldered in place. Then developers can easily cut the tracks with an X-Acto knife and make changes.

Pullup resistors bring their own challenges. Many debugging tools have their own pull-ups which can bias nodes oddly. It's best to use lower values rather than the high ones permitted by CMOS (say 10k instead of 100k).

PCB silkscreens are oft-neglected debugging aids. Label switches and jumpers. Always denote pin 1 as there's no standard pin one position in the SMT world. And add tick-marks every 5 or 10 pins around big SMT packages, and indicate if pin numbers increase in a CW or CCW direction. Otherwise, finding pin 139 is a nightmare, especially for bifocal-wearing developers suffering from caffeine-induced tremors.

Key connectors so there's is no guessing about which way the cable is supposed to go.

Please add comments to your schematic diagrams! For all off-page routes, indicate what page the route goes to. Don't hide the pin numbers associated with power and ground - explicitly label these.

When the design is complete, check every input to every device and make absolutely sure each is connected to something - even if it's not used. I have seen hundreds of systems fail in the field because an unused input drifted to an asserted state. You may expect the software folks to mask these off in the code, but that's not always possible, and even when it is, it's often forgotten.

Try to avoid hardware state machines. They're hard to debug and are often quite closely coupled to the firmware, making that, too, debug-unfriendly. It's easier to implement these completely in the code. Tools (e.g., VisualState from IAR) can automatically generate the state machine code.

Summary

Embedded systems are a blend of hardware and software. Each must complement the other. Hardware people can make the firmware easier to implement.

Many of the suggestions above will make a system easier to debug. Remember that a good design works; a great design is also one that's easy to debug.