功能安全的代码覆盖范围

概述

Speed Up Safety Certification & Improve Code Quality with TRACE32®-based Code Coverage

Code coverage measurement is a requirement for certification to evaluate the completeness of test cases and to demonstrate that there is no unintended functionality. However, measurements are also made to increase code quality, e.g. to rule out the existence of dead code that is never executed.

When using traditional test tools, integration and system tests often generate considerable time and personnel ex-penditure due to the necessary code instrumentation, which can be reduced dramatically with trace-based code cov-erage measurements using our TRACE32® tools.

Code instrumentation is either not necessary at all, or only to a very limited extent. You need fewer build and test runs for a complete code coverage measurement, which leads to a saving as much as 50% of the total effort in real world examples. When you consider that these tests often take days or weeks to complete, you can estimate the time and cost benefits of utilizing TRACE32® for your code coverage measurements.

概述

Speed up Safety Certification & Improve Code Quality with TRACE32®-Based Code Coverage

Code coverage measurement is a requirement for certification to evaluate the completeness of test cases and to demonstrate that there is no unintended functionality. However, measurements are also made to increase code quality, e.g. to rule out the existence of dead code that is never executed.

When using traditional test tools, integration and system tests often generate considerable time and personnel expenditure due to the necessary code instrumentation, which can be reduced dramatically with trace-based code coverage measurements using our TRACE32® tools.

Code instrumentation is either not necessary at all, or only to a very limited extent. You need fewer build and test runs for a complete code coverage measurement, which leads to a saving of 50% of the total effort in real world examples. When you consider that these tests often take days or weeks to complete, you can estimate the time and cost benefits of utilizing TRACE32® for your code coverage measurements.

Benefit from Hundreds of Supported Architectures and Compilers

Get Test Results Faster during Integration & System Test

Measure all Metrics from Object Code Coverage to MC/DC

Measure Code Coverage in Real-Time

Use Qualifiable Simulators for Unit Testing

Easily Integrate with 3rd Party Tools

DATA COLLECTION

Select the Optimum Data Collection Mode According to Your Requirements

Depending on which architectures you use, which trace interfaces your chip implements - off-chip trace, on-chip trace or none at all - and which code coverage metrics you want to cover, you can select one of four different modes for data acquisition. These cover all conceivable use cases from maximum compatibility to maximum performance.

Use Real-Time Trace Capture to Minimize Your Time Expenditure

Our TRACE32® can process the trace data during recording in the operation mode RTS (Real Time Profiling). The biggest advantage of RTS mode code coverage is that all necessary steps run in parallel and large amounts of trace data can be processed quickly, therefore your code coverage measurement results become available immediately. RTS is currently supported for Arm® ETMv3/v4 and PTM, Nexus for MPC5xxx/QorIQ, and AURIX MCDS trace prococols. RTS requires one of our TRACE32® trace extensions such as PowerTrace, CombiProbe or µTrace and supports the code coverage metrics statement coverage, function coverage, and call coverage. The raw trace data are streamed to the host computer and decoded to reconstruct the program flow before the program flow is added to the code coverage system and the code coverage results are updated. All these steps are performed concurrently to minimize your time expenditure.

Stream Trace Data for All Code Coverage Metrics and All Instruction Set Architectures

Our TRACE32® tools support processing of trace data while being recorded for all architectures in the operation mode SPY. SPY mode achieves slightly lower processing speeds than RTS mode, but supports all code coverage metrics and all chip architectures. For SPY mode code coverage, trace streaming is periodically suspended in order to decode the raw trace data and to process it for code coverage measurements. Your Code coverage measurement becomes available rapidly.

Benefit from Incremental Mode on Chips Providing On-Chip-Trace Only

Incremental code coverage is supported by all processor architectures which provide information about program flow. It also supports all code coverage metrics. It is a reliable fallback method for chips implementing on-chip trace only and can be used in the vast majority of situations.

A characteristic feature of incremental code coverage is that the individual steps are executed one by one. Trace information is recorded while the program is running. After the program has been stopped, the raw trace data is uploaded to the host computer and decoded to reconstruct the complete program flow. After that, the program flow is finally added to the code coverage system and the code coverage results are updated.

Benefit from Incremental Mode on Chips Providing On-Chip-Trace Only

Incremental code coverage is supported by all processor architectures which provide information about program flow. It also supports all code coverage metrics. It is a reliable fallback method for chips implementing on-chip-trace only and can be used in the vast majority of situations.

A characteristic feature of incremental code coverage is that the individual steps are executed one by one. Trace information is recorded while the program is running. After the program has been stopped, the raw trace data is uploaded to the host computer and decoded to reconstruct the complete program flow. After that, the program flow is finally added to the code coverage system and and the code coverage results are updated.

Collect Trace Data Even on Chips without A Trace Interface

Our special data collection mode ART(Advanced Register Trace) operates by single stepping on assembler level. After each step, the contents of the CPU registers are uploaded to our TRACE32® PowerView software and stored in a similar fashion to a program flow trace. This pseudo-trace data can be used for code coverage. You can use trace data recorded with ART for all code coverage metrics.

ART allows the collection of data even on chips without any trace interface, however, it has a significant impact on the real-time performance of the target. To improve the performance, we recommend to use our TRACE32® Instruction Set Simulator (ISS), which supports all code coverage metrics as well. The ISS simulates the instruction set and provides a bus trace so that code coverage is easy to perform.

EVALUATION

Covering All Coverage Metrics

Test coverage and test progress can be quantitatively determined using code coverage measurements. Depending on the criticality of your embedded system and the individual application-specific standards for functional safety, you have to apply different levels of coverage, usually called code coverage metrics. Essentially, the individual levels of code coverage are assigned as follows: function level, instruction level, branch level, path level and condition level. The different levels of coverage supported by our TRACE32® trace tools fulfill all requirements as defined in the standards ISO 26262 (Automotive), IEC 62304 (Medical Devices), DO 178C (Avionics), EN 50128 (Railway) and IEC 61508 (the Umbrella Standard for Industrial Projects).

Object Code Coverage

Object code coverage measures whether each object code instruction was executed at least once and all conditional instructions (e.g. conditional branches) have evaluated to both true and false.

Function Coverage

Function coverage measures whether all functions of the program have been reached at least once. Complete function coverage is achieved when every function of the program has been called during the test. 100% call coverage doesn’t imply 100% function coverage.

Call Coverage

Call coverage is a measure of the number of calls to functions that have been exercised during the execution of a program, as a percentage of the number of calls that exist in that program. 100% function coverage doesn’t imply 100% call coverage.

Statement Coverage

Statement coverage measures how high the proportion of tested statements is compared to all statements. At this level, it is therefore possible to find dead code or statements for which there are no tests yet.

Decision Coverage / Branch Coverage

At this coverage level, each decision must be tested at least once as “true” and “false”. For normal “if statements”, this corresponds to branch coverage, where every branch must have been executed (branch coverage). To achieve decision or branch coverage, branches without statements must also be run through. For an “if statement” without an else block, the case that the associated decision is incorrect must still be tested. Full statement coverage follows from (full) decision coverage, in this sense it is the stronger measure.

Condition Coverage

In addition to the requirements of decision coverage, condition coverage considers composite decisions in detail. In the case of decisions that consist of several atomic conditions composed via Boolean operators, each of these conditions must be tested individually as “true” and “false”. Condition coverage is essentially only relevant for understanding multicondition coverage and, in particular, modified condition/decision coverage (MC/DC).

Modified Condition/Decision Coverage (MC/DC)

With multicondition coverage, all possible true/false combinations must be checked for composite decisions. If there are several conditions within a decision, this usually requires an impractically high number of test cases, as this number grows exponentially with the number of conditions in the decision. For this reason, multicondition coverage is not required in any safety standard, but rather modified condition/decision coverage (MC/DC). With it, the number of test cases only increases linearly with the number of conditions, but at the same time it is more difficult to understand and also more difficult to determine.

Typical Automotive Use Case

Meet Automotive Safety Integrity Level (ASIL) Requirements According to ISO 26262





ISO 26262, where the Automotive Safety Integrity Level (ASIL) is introduced, is to be used for safety-relevant electrical/electronic systems in motor vehicles. The standard also recommends code coverage measurements being used at function and architecture level. In the tables on the right you can find the structural coverage metrics at the software unit level (ISO 26262-6:2018, Table 9) as well as structural coverage metrics at the software architectural level (ISO 26262-6:2018, Table 12).

The fields marked with “++” indicate that the specified measure is particularly recommended. If, for a specific reason, a decision is made not to comply with this requirement and to dispense with the measure described, a detailed explanation must be provided as to why this decision was made and what effects it has. Recommended measures are marked with a simple “+” and indicate that the corresponding measure results in an improvement in quality or safety.

Since you can measure all the metrics listed in the tables on the right with our TRACE32® trace tools, you can fully meet the code coverage related requirements up to the highest level ASIL-D.

CODE COVERAGE ANALYSIS

Analyze Your Results in PowerView and Beyond

We provide you with several reporting options for the evaluated code coverage measurements. This includes the display within our PowerView software, the export into several human-readable and machine-readable formats as well as our TRACE32® Code Coverage Merging and Reporting Tool which you can use to merge results from different test runs and different target platforms.

Display Code Coverage Measurements in PowerView

In a PowerView window, you can immediately display the coverage results of all functions or modules under consideration for the measured metrics.

Generate Human-Readable Reports in XML

With the help of our TRACE32® Report Utility you can display the results of your code coverage measurements in the internet browser of your choice.

Generate Machine-Readable Reports in CSV, JSON or VectorCAST CBA

After concluding the test run, you can export the outcomes for all source code metrics, stored within our TRACE32's internal code coverage system, to a file. Supported source code metrics include statement, decision and condition coverage, MC/DC, as well as call and function coverage. You can choose between the open file formats CSV or JSON (JavaScript Object Notation) and the proprietary CBA format for further processing of the data in VectorCAST.

Generate Machine-Readable Reports with the TRACE32® Code Coverage Merging and Reporting Tool

Our TRACE32® Code Coverage Merging and Reporting Tool  allows you to consolidate the results of multiple code coverage test runs performed with different ELFs based on different suites of test cases, or using different target configurations. The tool is compatible with Windows and Linux operating systems and uses exported JSON files to merge the TRACE32® code coverage results from various test runs. At the same time, these results are translated into a visually appealing HTML report for analysis and documentation purposes. The open and extendable JSON format permits processing of the merged code coverage measurements by many third-party tools.

FUNCTIONAL SAFETY CERTIFICATION

Save Time and Effort with Our Tool Qualification Support Kit

The qualification of safety-critical embedded systems also includes the development tools used and their integration into the project environment. Our certified Tool Qualification Support Kits (TQSK) provide everything you need to qualify our TRACE32® solutions for use in avionics, medical, automotive, railroad, or general industrial projects and reduce your time-to-market, effort, and costs.

Besides many other artifacts our TQSK includes the Test Suite – Coverage which provides test cases for statement and branch coverage, MC/DC, condition, function and call coverage. It supports source code coverage for C, assembly level coverage for executable object code, and Incremental as well as RTS and SPY code coverage modes.


合作伙伴

Easily Integrate with Third Party Tools

When using traditional test tools, integration and system tests often generate considerable time and personnel expenditure due to the necessary code instrumentation, which can be reduced dramatically with trace-based code coverage measurements using our TRACE32® tools. Nevertheless traditional test tools and their code coverage measurement capabilities including their sophisticated test case and requirement management, can play to their strengths particularly in unit testing.

The combination of traditional test tools and TRACE32® provides you the best possible customer experience for efficient and time saving code coverage measurements. Currently you can import TRACE32® code coverage measurements into three widely distributed and well-known third party tools.

  • The LDRA tool suite®

    The LDRA tool suite provides requirements traceability, test management, coding standards compliance, code quality reviews, code coverage analysis, data-flow and control-flow analysis, unit/integration/target testing to achieve certification in accordance with rigorous industry standards.

    了解更多
  • RapiCover/RapiCoverZero

    Both RapiCover and RapiCoverZero support on-target structural coverage analysis for critical software. With automated coverage reporting up to MC/DC, support for testing decisions with up to 1000 conditions, and qualification support, RVS solutions ensure that you get the most from your testing.

    了解更多
  • VectorCAST

    The VectorCAST embedded software testing platform is a family of products that automates testing activities across the software development lifecycle. The VectorCAST product family is used in the fields of avionics, medical device, automotive, industrial controls, railway and financial industries.

    了解更多

有什么问题吗?

凭借 40 多年的行业领先地位,我们的专业工程师随时为您提供帮助。如果您对我们的产品或哪种配置最适合您需要咨询,请联系我们的销售工程师。如果您需要劳特巴赫系统的使用帮助,请联系我们的工程支持团队。