Follow @jack_ganssle
Go here to sign up for The Embedded Muse.
TEM Logo The Embedded Muse
Issue Number 346, March 19, 2018
Copyright 2018 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com

   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact info@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Contents
Editor's Notes

Wouldn't you want to be in the top tier of developers? That is part of what my one-day Better Firmware Faster seminar is all about: giving your team the tools they need to operate at a measurably world-class level, producing code with far fewer bugs in less time. It's fast-paced, fun, and uniquely covers the issues faced by embedded developers. Information here shows how your team can benefit by having this seminar presented at your facility.

I'm on Twitter.

Quotes and Thoughts

You must be the change you wish to see in the world. - Gandhi

Tools and Tips

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

Chris Gates sent this in response to last issue's take on the fire code.

Freebies and Discounts

Gary Strunc was the lucky winner of the ZeroPlus LAP-F1 logic analyzer.

This month's giveaway is a copy of Jean Labrosse's excellent book "uC/OS-III, The Real-Time Kernel for the Kinetis ARM Cortex-M4."

It will close at the end of March, 2018. Enter via this link.

Handling UARTs

Simon Gilbert had a suggesting for handling UARTs:

I would like to submit a technique I have found useful quite a few times now, it is useful when battling transfers of data from a peripheral where you don't know the data length and there's no or limited FIFO.

I have used this solution mostly on STM32's where there is no FIFO for modules like the UART and the UART is most likely to receive indeterminate length data strings. This solution works well with other peripherals as well. The solution does require some form of DMA.

Problem:

  • limited or no FIFO
  • unknown data length

The solution comes in 2 parts:

  • DMA based double buffer
  • software circular buffer or similar

As we don't know the data length to count in, we have to read each character in. We could poll the UART or use an interrupt service routine to pull the data in, both of these options have draw backs polling can be time resource consuming and the ISR could potentially not be serviced quickly enough and data could be lost (unlikely for UART but something like SPI may be an issue). However, there's a useful feature of the DMA engine on the STM32 range where an interrupt occurs on a half complete transfer and a full transfer complete. The DMA engine can also be set to run in circular mode. By having a buffer 2 characters wide and the DMA engine set to circular mode, the DMA continuously transfers the data out of the UART module as soon as its available preventing overrun conditions of the peripheral and then raises an interrupt on every transfer where it can be handled at a later time. By double buffering it reduces the risk of data being overwritten before it can be handled.

Step 2 is to have some form of receptacle for the data like a software FIFO buffer that can have its status polled at a leisurely pace e.g. a task. I prefer circular buffers as they offer flexibility as they can be used in all sorts of arrangements without taking up too much code space

This method can be modified to deal with large fast data sets where you have a rough idea of data packet size i.e. SPI or memory access where the double buffering gives breathing room to process the data, just increase the size of the buffer used by the DMA engine and make sure its a factor of 2.

Sample code below lifted directly from a project with variable names changed. The base code was generated using STM32CubeMX giving the HAL functions.

#define ARRAY_LEN(x) (sizeof(x) / sizeof(x[0]))
#define WIFI_UART_PORT			huart4
#define TERMINAL_UART_PORT       huart1

void startDataInHandler(void const * argument)
{
  static uint8_t rxDataArray[2];    //   UART receive data buffer, data is read 
                                    //  in to here to prevent 
                                    // UART over run errors, array is static to make sure it's
                                    //  persistant 
                                    // and declared here to reduce ability to access it
  static elemType_t replyBuf[128];  //   the data buffer for the circular buffer, defined here 
                                    // to make sure 
                                    // the memory is allocated but inaccessible except via the 
                                    // struct and cb functions

	/* initialise a circular buffer for receiving characters from a source.
	* Configure the DMA to read 2 bytes using DMA. Note, using DMA, there are
	* 2 handlers, 1 for RX half complete and 1 for full complete. This allows a
	* form of double buffing to prevent data loss during high data rates.
	*/
  cbInit(&dataRXCB, replyBuf, sizeof(replyBuf));   // pairs replyBuf with dataRXCB circular 
                                                   // buffer struct
  HAL_UART_Receive_DMA(&TERMINAL_UART_PORT, rxDataArray, ARRAY_LEN(rxDataArray)); // enable the 
                                                   // DMA transfer
							
  /* Infinite loop */
  for(;;)
  {
  /* if there's data, parse it */
    if(!cbIsEmpty(&dataRXCB))
  {
    dataParser();
  }
  osDelay(1);
  }
}

/**
  * @brief  Rx Transfer completed callbacks.
  * @param  huart pointer to a UART_HandleTypeDef structure that contains
  *         the configuration information for the specified UART module.
  * @retval None
  */
void HAL_UART_RxCpltCallback(UART_HandleTypeDef *huart)
{
  if(&WIFI_UART_PORT == huart)
  {
    (void)cbWrite(&CBWiFiRX, (elemType_t*)&huart->pRxBuffPtr[1], 1, NO_LOCK);
  }
  else if (&TERMINAL_UART_PORT == huart)
  {
    (void)cbWrite(&dataRXCB, (elemType_t*)&huart->pRxBuffPtr[1], 1, NO_LOCK);
	}
  else
  {
  }
}

/**
  * @brief  Rx Half Transfer completed callbacks.
  * @param  huart pointer to a UART_HandleTypeDef structure that contains
  *                the configuration information for the specified UART module.
  * @retval None
  */
void HAL_UART_RxHalfCpltCallback(UART_HandleTypeDef *huart)
{
  if(&WIFI_UART_PORT == huart)
  {
	  (void)cbWrite(&CBWiFiRX, (elemType_t*)&huart->pRxBuffPtr[0], 1, NO_LOCK);
  }
  else if (&TERMINAL_UART_PORT == huart)
  {
    (void)cbWrite(&dataRXCB, (elemType_t*)&huart->pRxBuffPtr[0], 1, NO_LOCK);
  }
  else
  {
  }
}	
Notes from Embedded World

There's an old saying: "See Embedded World and die." This year's Embedded World (EW) in Nuremberg (February 27-March 1) was the largest embedded show on Earth. While shows in the USA are in decline, EW was bigger than ever. How big? It covered six buildings. There were 1100 vendors. Over 30k attendees.

Though the vendors displayed a vast array of products and services, a few themes seemed to dominate the show:

  • Automotive. Now, being in the heart of Germany, it's hardly surprising that the car biz would be so important, but the automotive meme pervaded many, many offerings.
  • Standards. For decades we've been wrestling with a variety of international and ad hoc standards for building embedded systems. But EW 2018 attendees couldn't walk ten meters without seeing 26262, 62304, Autosar, 61508 or any of a dozen other standards being pushed by vendors. It felt like nearly all of the tool vendors had some standards story; either their products conformed to one or more standards, or their tools would help customer achieve compliance.
  • Security. No surprise there. But what was interesting, to me, was that few of the vendors I questioned had a quantifiable measure of the security of their products. Admittedly, we really don't have a handle on this; the Common Criteria is a start but hardly the end-all metric. But I felt there was more hand-waving than substance. On a positive note, at least the industry is trying to grapple with this issue.
  • Vision. To a lesser extent, embedded vision occupied part of the EW zeitgeist.

Here are some observations from a few of the companies I talked to.

GrammaTech is one of a small number of companies that make "static analysis tools." Unfortunately, the phrase "static analysis" (SA) is not well understood, and is in my opinion, a poor choice of words. Many developers include Lint in this taxonomy, but Lint is really a checker on speed. SA tools, instead, while not executing the code, do a mathematical analysis of the code's probable behavior at runtime to find errors like buffer overflows, null point dereferences and much more.

Now, just to confuse the issue, some SA tools do find things like MISRA violations, which Lint does as well. Why this is so baffles me; a wise developer will use a series of tools as a set of filters to find every possible problem prior to testing. Use Lint in conjunction with these tools.

GrammaTech told me that they consider three aspects of these tools:

  • Recall - The percentage of real-world problems that the tool can identify
  • Precision -The percentage of results that are true positives
  • Speed - The time it takes to get results

These three are in conflict. High speed means less recall and less precision.

GrammaTech introduced CodeSonar/Libraries at the show. Most SA tools more or less toss up their hands when encountering a library call, as the internals of that library aren't known. CodeSonar/Libraries has a priori knowledge of a number of libraries, so can use that knowledge to create better Recall and Precision in its analysis.

But there's another aspect of CodeSonar I found most interesting. It can do a SA of the binary code. The tool includes a disassembler (invisible to the user) that converts the binary to the same intermediate representation used when it parses source code. It can then look for likely runtime errors without the source.

But give it the source as well, and the tool will then look for miscompilations. Even more interesting, it can find likely errors due to C's unspecified, undefined behavior and implementation-defined behaviors (Annex J of the C11 specification).

According to a paper from GrammaTech (Making Safety-Critical Software Development Affordable With Static Analysis), 25% of the "capital costs" of new automobiles are electronics and software. The same paper implies that now, today, avionics is unaffordable due to software engineering costs. The graph that accompanies this is somewhat muddy, but seems to imply that we can figure on 2018-2020 avionics needing well over 100 M lines of code at a cost of tens of billions of dollars. I suspect airframe manufacturers, who have thousands of large planes on backlog, might disagree.

I have no personal experience with CodeSonar so will neither endorse nor condemn it. However, I've worked with a lot of developers using SA tools, and most (not all!) report somewhere between reasonable satisfaction with them, to outright delight.

Other SA vendors, like Parasoft, were present as well, but I didn't have time to talk to them.

An ublox module

I came across a press release for ublox before the show, and stopped by their booth. It seems the company is well-known on the Eastern side of the Pond, though was new to me. These folks make a huge range of modules meant to be mounted on a customer's PCB for positioning and wireless comm.

Do you need wifi or 4G LTE? Bluetooth or RPMA? GNSS or GPS, or even GNSS with a dead-reckoning addition? None of this is particularly new, but what is novel is the incredibly broad range of products offered, and they are all pin-compatible. In other words, you could design a board that could handle pretty much any of the common comm protocols or positioning options, and customize it for a particular option just be swapping ublox boards. Interesting concept.

Oscilloscopes were everywhere. Here's a medley:


<

Keysight's latest is not this:

That was on display to show they support their really old stuff.

If there is a success story in the USB-connected instruments business, it's Pico. I was stunned by how many employees they have - and won't repeat it here as I fear that number may be proprietary. I had figured maybe, at most, dozens, but that estimate is quite low.

Their new product for the show is a vector network analyzer. The Tek equivalent is about twice the price. I was impressed by the specs and the speed: 6 GHz with a 118 dB dynamic range. But I don't use a VNA, and have never needed one.

Their scopes, though, blow me away. In the USB scope/logic analyzer market most companies offer a few models, most with credible but limited specs. Maybe a few hundred MHz max. Pico has those, for example, a two channel 10 MHz model for a hundred bucks or so. But they also offer high-performance products like a 25 GHz sampling scope with 16 bit ADCs for over a hundred times as much. A bunch of bucks, but those sorts of speeds are mostly offered in bench instruments from the majors, at eye-popping prices.

They also offer a wide range of probes. At higher speeds conventional scope probes don't measure up. That little 10 cm ground lead becomes a complex circuit guaranteed to distort, often wildly, the measured signal. High-speed probes can be much less convenient to use, requiring a ground connection within mm of the node being probed, but such is life in the fast lane.

PMK is another company that sells probes. I wasn't familiar with them, but they offer a huge range of solutions for dealing with all sorts of probing problems. With high-speed logic the ground lead needs to be very close to the signal node, so everyone's solution is cumbersome. But PMK's probes are much cheaper than those from the big scope vendors. How cheap? It's hard to tell as their web site seems to lack any sort of ordering information, but I was told on the order of $1k. The new Sonic probe, pictured, has a 123 ns rise time and is rated to 4 GHz at -3 dB. With 0.6 pF of tip capacitance it's far better than your normal passive probe, but I calculate that's 66 ohms, representing 45 mA of load at 4 GHz with a 3 volt target supply. Good luck finding a logic gate that can drive it. Not to knock DMK, with whom I was impressed; this is just a problem with any sort of high-speed probes.

DMK's Sonic probe

I mentioned the automotive meme; autonomous vehicles will require a network of sensors, including lots of RADAR. In the olden days RADAR needed magnetrons, rotating antennas, and even a rotating yoke around the CRT. No longer. TI and Infineon showed single-chip (how cool is that?) solutions operating at 77 GHz. Infineon's demo board had small horn antennas:

Infineon's RADAR eval board. Check out those transmit horns!

TI's demo board simply used etched tracks on the PCB. Though both companies promised to contact me with more details, neither has. However, I imagine that, given these are for automotive applications, the cost in volume is minimal. The range is on the order of meters, rather than AEGIS-like over-the-horizon units, of course.

Hitex demoed one of the more intriguing products at the show. TESSY, as I understand it, will automatically generate all of your unit tests. Based on the Classification Tree Method of building tests, TESSY analyzes the code to generate complete test coverage. The tool can show traceability from requirements to code, so can be used to meet various safety-critical certifications (e.g., 61508). As I understand it, TESSY generates the test conditions from the code (rather like LDRA's Unit). Using the latter tool I found a 10,000 line program produced 50k lines of test code - an astonishingly complete set of tests far exceeding any sort of manual test generation we're likely to create. The one caveat, of course, is that a tool that produces tests from the code can be foiled by incorrect code, so engineering judgment is keenly needed.

As mentioned, standards compliance was a major theme at Embedded World. Express Logic, purveyor of ThreadX and associated middleware, now has many of their products (and more coming) qualified to SIL (Safety Integrity Level) 4. They're also going after the Common Criteria EAL (Evaluation Assurance Level) 4+ using an independent lab to qualify the products. This attempt to quantify security is to be applauded. (To really understand the EAL level one needs to know the protection profile, which is the type of threats the EAL quantifies.)

Express Logic is also supporting a number of cloud providers; the cloud was a lower-case meme at the show. None of the vendors I talked to had many customers actually using that resource now; they are all forward-looking anticipating future cloud uses.

Years ago Express Logic snagged the best URL around - RTOS.com.

LDRA also sees standards as the future. The company sells a variety of tools to help developers manage/track requirements, implement and manage tests, and insure code quality. They are supporting the automotive cybersecurity standard SAE J3061. 26262 and Autosar compliance has long been their forte; they tell me a car can have a GB of code or more! That that is a big attack surface.

HCC-Embedded, whom I have long admired for their utter focus on quality, (I interviewed them here in 2015) has had various crypto solutions for some time. They've partitioned that out as a separate product. While being MISRA-compliant, that product, and their others, are available with additional life-cycle evidence to show both quality, and to ease getting certification in safety-critical applications.

HCC has a broad line of middleware products. Looking to write a flash driver? Please don't! HCC, and others, will sell you complete, tested, working code for far less than you can write it yourself. In the last few months alone I've visited several companies writing their own drivers for various components when cost-effective solutions are already available.

Despite the fact that Embedded World is a European show, and despite the new EU General Data Protection Regulations which will be in effect in May, I didn't see a single company offering any sort of solution to meet these new regs. These new rules are going to hit the industry hard, yet there appears to be little panic about them as yet. Break the rules and the company can be subject to up to 4% of worldwide sales in fines - potentially billions of dollars for a big multinational. Did you know Paypal shares your data with some 600 companies? I wonder how they, Amazon, Google, and the other behemoths will comply, given their hoovering of every scrap of data that traverses the ether.

Percepio's booth highlighted their Tracealyzer debugging tool (full disclosure - in the past they have advertised in the Muse). The tool graphically shows what your system is doing, especially with regards to multitasking. Segger's SystemView and Micrium's uC/Probe are somewhat similar tools. I recommend Percepio's recent RTOS 101 article that demonstrates the value of using the product to peer into the opaque goings-ons inside of a multitasking system.

Priority inversion bug shown in Percepio's Tracealyzer

Not all was fun and games: in a lecture on security I was chilled when a representative of the German government said "I trust the security of my smartphone because I get regular updates from the vendor." The same speaker said "the main problem with security is the liability issue." Uh, maybe that's true for leaky companies, but I think the main problem is the utter lack of security. And frequent updates may indicate poor security practices in general rather than trustworthy products.

There was a lot of interest in very low-power systems. One lecture, given by the Nile god Hapy, was about achieving three millenia of sleep time on a coin cell. Hapy's slow and inaudible presentation made the Ents seem garrulous; seemingly turned to stone at times, he mirrored the deep sleep states needed to achieve femtoamp-levels of power consumption:

Finally, I had some business in England, and spent a night in a "Hub by Premier Inn" hotel in London. The room, though small, was very high-tech, and sort of felt like being in a spaceship. The hotel has a very innovative design and cleverly uses limited space. All room functions are controlled by a bedstead switch panel. The next morning, as is my wont, I got up early. The panel had crashed. I couldn't turn any lights on.

Reception had to reboot the room.

A Response to Fire Code for Software

Lars Pötter (and many others) responded to the "Fire Code for Software" article in the last Muse. I think Lars does a great job of stating most people's points. Several readers disagreed with me, writing that software is unlike fires, in that - to date - it hasn't been responsible for huge numbers of deaths. And, in an interesting twist, Marybeth and I escorted my nonogenarian mom to Boston for a family reunion a week ago, where I learned one of her aunts was badly injured, and her husband killed, in the 1942 Coconut Grove Fire in Boston.

Lars wrote:

That was a really nice writeup to make the point for software quality.

The sad part about it is that judging from the similarities we need thousands of people to die until the software quality gets better.

And there is a fact that puts software in an even worse situation. If I build a house I can choose to install sprinklers/Fire detectors/.. and once the house is built I can inspect it and can see the sprinklers. I can test the alarm system and I know that this house is well equipped.

On the software side I can ask my vendor to give me the best/the safest/the most secure software. I can also pay more to get "better" software. But I can not inspect it, I can not test if the CMMI procedures that the vendor executed have been effective and increase the quality of the software I payed for, or if that vendor also "bribed the fire brigade".

It is easy to call for open source software. And I'm not against it. I would love to see that happen. But can the software industry do that? At the Embedded World show I asked a vendor why the Tool to create code/projects for their chips that the vendor developed was not open source. I reported that I had issues with it and had no way of reporting the issue. All I got was that they were thinking of maybe having a bugzilla sometime in the future and that they couldn't open the code because they need to protect their technology and open sourcing would increase their support workload. I told them that I would not touch a tool that they would not support. They give the tool away for free to look people into their chips. The worst thing is that all vendors do that. The company I talked to isn't even the worst (in my opinion).

But open source alone also isn't the solution. I started looking into open source software that I like and use. And the reaction is either "WTF" or "OMG". Maybe my expectations are to high, but there is a lot of very old code out there. After Heartbleed more people realized the "code quality" of openssl, and we had a reaction.

On a 8 bit MCU it is easy to write high quality software. You basically can hand craft each bit and polish it until it shines. As the CPUs get bigger and the feature sets get bigger that gets harder. More and more preexisting code needs to be used. More and more complex tool chains need to be used and as we saw with openssl and others not all of those "working" tools and libraries have the code quality we want. And the hand crafted secure software becomes the locked door right next to the open window.

Maybe as good craftsmen we should take more responsibility for the quality of our work and for the quality of our tools. The woodworker doesn't get free tools when he buys the wood. He would not work using bad quality material. He would not build on a rotten base. He would also not accept that a special wood can only be crafted with special tools.

But in our world the vendors provide IDE's combined with proprietary debug interfaces that only their tool implements. And embedded engineers that talk exchange the list of bugs found in vendor tools and vendor code. I had a really interesting discussion about the different issues in a vendor supplied USB stack, we both didn't like the code quality, and fixed several bugs already. But we both used it. And we both had no way of reporting the bugs back to the vendor. (Note from Jack - that's what the Tools and Tips section of this newsletter is for).

And this will not change if we wait for the vendors to help us. This will only change if we change it. I know how to write good code. So please Jack tell me: How can we create an environment, a tool set, a work ethic that makes at least "low bug count" code the default? How can we create an environment that says no to bad coding? That communicates publicly, clearly and understandably why bad code is dangerous for the users. How can we do that? Because we need the fire brigade and the inventors of fire safety systems ready once people start dying.

Who thinks that dying is to hard a word? Please think of hacked self driving cars, misbehaving factories, hacked hospitals (the CCC had a very interesting talk about hacking pacemakers) and hacked military equipment,...

PS: A few years back hacker talks were all fun. One learned a lot and could make things to new stuff. Now the talks have disclaimers to please not exploit the obvious issues to not harm the people that need to use them. Back the we expected the vendors to fix their bugs, have we now accepted that they will not fix their mess?

Jobs!

Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Joke For The Week

Note: These jokes are archived at www.ganssle.com/jokes.htm.

Flying back from Embedded World, the display at my seat showed the outside air temperature:

Maybe the sensor failed. Who knows? But one of the oldest rules of software engineering is to check your goesintas and goesoutas. This is just sloppy work.

Advertise With Us

Advertise in The Embedded Muse! Over 27,000 embedded developers get this twice-monthly publication. For more information email us at info@ganssle.com.

About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster. We offer seminars at your site offering hard-hitting ideas - and action - you can take now to improve firmware quality and decrease development time. Contact us at info@ganssle.com for more information.