Cantata is seamlessly integrated with VxWorks® 7, providing C/C++ developers with unit and integration testing and integrated code coverage. Regulatory safety standards compliance for DO-178B/C (Avionics), IEC 61508 (Industrial), ISO 26262 (Automotive), IEC 62304 (Medical), EN 50128 (Railways) and IEC 60880 (Nuclear) is achieved faster and more cheaply.
Cantata uses the build specs and target connection framework within Wind River® Workbench to integrate all aspects of testing into the project environment, allowing users to build, run, and obtain their results with minimal effort.
Cantata installs easily into Workbench using the powerful Eclipse P2 provisioning system and the built-in configurations for VxWorks Simulator, or deployment for any VxWorks target.
Cantata also supports earlier (3.x) versions of Workbench—please see “Cantata for Wind River Workbench 3.x” under Reference Documents.
Static analysis generates over 300 source code metrics (procedural and object oriented), in csv format, for maintainability and complexity aid in estimating test effort.
Requirements-based testing is supported through tagging of test cases for requirements traceability into reports.
- Out-of-the-box configurations for VxWorks Simulator: A fully integrated deployment wizard for custom build specs allows end users to configure Cantata for any VxWorks target.
- Test script generation by parsing source code to derive parameter, calls, and data information with call interface control (stubs, isolates, and wrappers): Optional initial values for variables and returns enable tests to run immediately.
- Test script manager: Graphical development, editing, and management of tests are synchronized with the C/C++ editor.
- Reuse of xUnit tests within Cantata tests: Retaining investment in open source scripts, xUnit tests can be reused in and extended with Cantata features.
- Intuitive test directives: Manual scripting is reduced by calling a library of helper functions to provide structured, readable, independent, repeatable tests with clear and unambiguous results.
Cantata tests are fully integrated into Wind River® Workbench projects, which allows Cantata to automatically create build targets for the Workbench build system, and build and run tests with a single click. Test results are automatically obtained during test execution and stored within the specific test folder.
All calls (external and internal to the compilation unit boundary) may be simulated by stubs or isolates, or intercepted using the real call via wrappers.
- Interface control selection for the method used on each call at test script creation (with defaults), and script editing, with the call list refreshed on each code build
- Optional automatic checks of all parameters/returns passed over call interfaces
- Programmable test control points available at each call interface control through named instances to set and check data, raise exceptions, and so on
- Automated stubs for replacement simulation of the called object interface
- Automated isolates for simulation of the linked called object interface, support system, and variadic functions
- Automated wrappers for interception of the linked called object interface with Before/After and Before/Replace modes to intercept or simulate, with more powerful and flexible control over interfaces than stubs or isolates
- Call sequence verification for full control over the call order and use of stub, isolate, or wrapper instances in each test case, with exact sequence or any time matches
Black-Box and White-Box Testing
Cantata provides a high degree of automation at unit and integration levels for both black-box testing and the more efficient and thorough white-box testing.
- Per-function testing that generates a complete template test case for each function or method in the code with all parameters, accessible data, and calls to control identified
- Table-driven testing with multiple input value ranges, combinatorial effect calculator, and CSV import/export for large data set black-box verification
- Robustness testing with pre-defined values for basic data types, in table-driven test cases
- Automated global data checks to verify expected and unexpected changes to all global data accessible to the software under test
- Automatic white-box access to efficiently call functions or methods and set/check data directly from the test script without conditional compilation:
- Static functions and private methods
- Private data and data static to the file
- Automated C++ exception verification of multiple expected and unexpected exceptions
Object Oriented Testing
Cantata unit and integration testing supports C++ functions, templates, classes, or clusters of classes in single or multiple source files. Test scripts for classes are written in C++ as a test class.
- Automated test case reuse via a parallel inheritance hierarchy retains the benefit of code reuse for testing, and allows you to verify that a derived or specialized class has the same semantics as a base or template class (the Liskov Substitution Principle).
- Automatic test class inheritance structure created for inherited classes allows child class tests to inherit from parents.
- Automatic implementation of abstract base classes (ABCs) or pure virtual methods, with generation of appropriate stubs in the script, supports use of the factory method.
Coverage analysis of C/C++ and Java provides objective measurement of how effective testing has been in executing the source code.
Code coverage metrics supported:
- Entry points
- Call returns
- Basic blocks
- Relational operators
- Decisions (branches)
- MC/DC (masking and unique cause)
Coverage by context provides coverage by:
- Each Cantata test case
- Derived inheritance contexts
- User-defined context for multiple states, threads, and data contexts (for DO-178B/C data coupling analysis)
- Any non-Cantata test run
Configurable coverage requirements are easily defined in simple rule sets. Predefined rule sets can be used for standard compliance.
Coverage target checks on metrics are integrated into dynamic tests, resulting in Pass/Fail for coverage requirements.
Automatic test case optimization provides the option to remove or disable test cases that do not contribute additional coverage.
Project code coverage trees for multiple projects include filters for coverage types and fully covered code items and drill down to highlighted source code.
Source code coverage views highlight individual code constructs (not just by line), with additional diagnostics by test case, test run, and metric type.
Automatically Test Legacy Code
Cantata AutoTest automatically generates complete passing unit test scripts to:
- Reduce reliance on system tests
- Support continuous integration
- Automatically close gaps in coverage
- Identify testability problems in code
- Easily change unit testing tools
Test depth can be configured, with code paths determined by selecting the metric types in a code coverage rule set.
Automatic verification can be configured using standard workspace preferences for passing checks on function(s) in each file of:
- Return value from function
- Output parameters from function
- Accessible global data values
- Order of function calls made
- Parameter values to function calls
Cantata provides IDE test generation for selected source files, or directly into existing test scripts, as well as CLI test generation for larger code base source directories, files, and functions.
The separate standard license feature allows controlled use of AutoTest.
Testability issues are identified with warning messages for problems in code preventing the generation of complete passing tests:
- Dynamically unreachable code
- Crash scenarios
- Data uninitialized or function static
- Implicit function declarations
- Compiler type truncation
An AutoTest Generation Report is created as an HTML report for fully, partially, or untested files and functions.
An automatic regression suite of AutoTests is created using Cantata makefiles for CLI invocation.
Ongoing maintainability of AutoTest is simplified as test scripts are standard Cantata style, with reuse of call instances and a detailed path-solving description for each generated test case.
Diagnostics and Reports
Cantata offers powerful filterable diagnostics of test and code coverage results within the UI, and flexible user configurable reports. Detailed test diagnostics are provided for all checks of expected against actual results by test case for:
- Global data
- Parameters and returns
- Call order
- Code coverage targets
Cantata also provides:
- Printable views from within the Cantata UI for all test and coverage results displays
- Test summary reports for test build and execution results of all tests with makefiles
- Configurable XML reports with summary or full test details consolidated across multiple projects
- Project-level tree views of test Pass/Fail results with drill-down hyperlinked navigation to individual test cases and checks
- ASCII Text Report available with passes and fails highlighted inside Cantata with an outline view, or as a plain text results file for high-integrity certification needs