Wednesday, March 12, 2014

Toyota, Where Are Ya? Your Customers Are Dyin'!

New findings include defective software that contains bugs, and -- in the 2005 Camry -- an electronic throttle control system with inadequate safety architecture, whose design created a single point of failure with no redundancy in place.

At this point, EE Times does not have access to the 800-page report, which concluded that misbehavior by Toyota's electronic throttle control system was a cause of unintended acceleration, filed by Michael Barr, CTO of Barr Group. Barr also served as an expert witness in Oklahoma.
(The full report is in the hands of several lawyers. A redacted version of the report was filed in US District Court in Santa Ana, Calif., in St. John v Toyota on April 12, 2013, according to Barr.)

FROM: What Toyota Did

Toyota Camry L4 case: the single bit flip that killed

Junko Yoshida, EETimes
October 28, 2013

MADISON, Wis. — Could bad code kill a person? It could, and it apparently did.

The Bookout v Toyota Motor Corp. case, which blamed sudden acceleration in a Toyota Camry for a wrongful death, touches the issue directly.

This case -- one of several hundred contending that Toyota's vehicles inadvertently accelerated -- was the first in which a jury heard the plaintiffs' attorneys supporting their argument with extensive testimony from embedded systems experts. That testimony focused on Toyota's electronic throttle control system -- specifically, its source code.

The plaintiffs' attorneys closed their argument by saying that the electronics throttle control system caused the sudden acceleration of a 2005 Camry in a September 2007 accident that killed one woman and seriously injured another on an Oklahoma highway off-ramp. It wasn't loose floor mats, a sticky pedal, or driver error.

An Oklahoma judge announced that a settlement to avoid punitive damages had been reached Thursday evening. This was announced shortly after an Oklahoma County jury found Toyota liable for the crash and awarded $1.5 million of compensation to Jean Bookout, the driver, who was injured in the crash, and $1.5 million to the family of Barbara Schwarz, who died.

During the trial, embedded systems experts who reviewed Toyota's electronic throttle source code testified that they found Toyota's source code defective, and that it contains bugs -- including bugs that can cause unintended acceleration.

"We've demonstrated how as little as a single bit flip can cause the driver to lose control of the engine speed in real cars due to software malfunction that is not reliably detected by any fail-safe," Michael Barr, CTO and co-founder of Barr Group, told us in an exclusive interview. Barr served as an expert witness in this case.

A core group of seven experts, including four from Barr Group, analyzed the Toyota case. Their analysis ultimately resulted in Barr's 800-plus-page report.

In Toyota's own view, though, the automaker had been already exonerated when the National Highway Traffic Safety Administration closed its probe of Toyota models in February 2011. The NHTSA decision came after NASA investigated Toyota's electronic throttle control system and found no electronic causes of unintended acceleration during a 10-month review.

But not everyone in the embedded systems industry thinks NASA had enough time to come up with a complete report. Perhaps more significantly, in its report, NASA itself did not rule out the possibility of software having caused unintended acceleration.

The group of seven experts was given the task of picking up where the NASA investigation left off.
To read more of this article, go to “What NASA didn’t have time to do.

Other resources on on this topic include:
Total recall,” by Jack Ganssle
Unintended acceleration,” by Ron Wilson
Unintended acceleration and other embedded software bugs,” by Michael Barr
Toyota’s accelerator stuck on a software bug,” by Michael Barr
Firmware forensics: best practices in embedded software,” by Michael Barr, and,
Safety-critical software – more not less certification ahead.”

Behind the scenes, Toyota played hardball with critics. A public relations manager named Masami Doi had spelled out the approach in a December email. "There are at most around 10 people who are the sources of negative tone communications. If they can be suppressed, I think we will be able to manage it somehow. Like you said, let's go with an intention of destroying each individual person's ability to oppose us, one by one…."
– David Hechler, Is Toyota Telling The Truth About Sudden Acceleration (emphasis supplied)

According to this story, some of the NASA scientists who worked on the February 2011 report that DOT Secretary Ray LaHood proclaimed an exoneration of Toyota electronics were so disturbed by the way they were forced to “investigate,” they refused to sign the final product.