Embedded Platform Testing & Diagnostics Tool – Adept Chips

 

Overview of Adept’s Diagnostic Framework

Embedded systems are growing in their nature of complexity. As a result, their reliable real-time capabilities are in higher demand. This has given rise to a proportionate increase in the need for support methods for designers and testers at various stages of the development cycle. At Adept Chips, we have developed our own Embedded Platform Testing & Diagnostics Tool with proven capabilities in identifying hardware and software level issues and bugs, significantly reducing revision cycles in the development process. In this post, we will share details of this tool, our learnings while in the process and how we envision its future.

 

How We Develop OEM specific test frameworks

Over the years, we have gained a fair amount of experience, depth and breadth in developing custom test frameworks. We have delivered solutions to customers looking for efficient and manageable test suites that are helpful throughout the SDLC. To start with, we develop test cases by taking acceptance test plan documents as a reference and use them for deriving parameters of specific test cases along with tolerance limits per customer’s requirements. Next, we identify potential conflicts or issues that may arise during testing the system as a unit. This helped us define certain design decisions to build robust test frameworks.

 

Maintenance of test frameworks to accommodate future module additions.

Adept Chip’s test suite is designed with a scalable architecture that makes it easier to incorporate additional modules into the test framework. We adopt a modular approach to test case development because it reduces the changes needed for the entire framework, thus reducing valuable time in the testing and validation phases. In our experience, this flexibility helps the customer keep up the speed during rapid development cycles.

 

Performance Analysis of software through diagnostic framework test run logs and reports.

Identifying unintended bugs early in the development process has helped us avoid reworks on a previously developed module. To address this, our test suite comes with a database for storing regression, and periodic test runs with timestamps that help to identify the bugs introduced in previous releases. This feature allows the development team to identify specific changes that have introduced the errors and resolve them accordingly. Our test reports include the following information:

• Code coverage
• Protocol information particular to the test
• Statistical data of the result
• Marker functionality to select specific tests from the test input file.

TIP – Knowing when a bug has been introduced after a known working test case starts failing is crucial in efficient bug fixing.

 

Test tool development for testing of AutoSAR Communication Platform

Adept Chips provides end-to-end testing of hardware, software and integration testing in the framework. We offer an option to expand this because testing requirements evolve through the development cycle.

Adept Chip’s framework uses Pytest, a testing-focused python module in its framework that is designed to reduce the run time of the tests along with the ability to add test inputs in a parameterized format helping in testing multiple scenarios of tests at once. The test logging, with report generation, supports JSON format that can be used for test data analysis and is extendable to any mobile device.

Adept Chips complies with the standard acceptance test for AutoSAR to help validate the interoperability of different AutoSAR stacks within the network. The test cases involve RTE requirements, essential software services, bus behaviour, protocol management,  and diagnostic communication. The test suite also complies with AutoSAR testing methodology for testing extended functionalities for user-specific features.

 

Some of the Challenges Faced:

We faced numerous challenges and unique problems during the development of the test suite. Some include connecting to the command prompts of multiple controllers on board over serial, telnet and ssh. This flexibility is vital for a customer because of its easy-to-use provisioning scripts that can configure the communication interface as needed. The test suite also offers the ability to set up test environments. For example, a test that requires a certain file, set of conditions to be configured within the device under test, or needs a precondition that needs to be checked before running the tests are handled with automated scripts designed around the test suite along with customer’s inputs.

Here are a few tips for better testing and diagnostics:

• Writing unit tests for individual components in a module helps with modularity and identifying specific issues in the system.
• Testing the components, together in a module helps identify potential conflicts in interoperability that respective design modifications can resolve.
• Performing regression activity regularly helps identify new bugs in the development process and fix them at the earliest.
• Designing test cases with a modular approach. For example, parameterized test inputs are ideal for scaling up instead of hardcoded test parameters. This helps reusability of the test code for similar modules in the system.
• When the system under test has test cases in thousands, it is critical to design the test cases execution with the total time taken by individual test cases. This can be achieved by defining minimal delays and incorporating time-efficient coding practices.

 

A Peek into Adept Chips’ Testing & Diagnostics Tool

Adept Chips’ platform covers the testing and diagnostic needs of the system under test. The testing part of the tool includes the functional level of component testing that comprises the module, along with feature-level testing to ensure 100% coverage. The diagnostics include preboot testing that takes care of memory initialization, IO initializations, and module readiness across the system to ensure the system is up and running as expected. This helps in reducing false failures in test reports. Post boot, the diagnostic tool gathers required data from all modules and stores it with timestamps for historical data analysis helpful for observing system behaviour in specific conditions. The tool also has real-time diagnostics to help identify defects in certain hardware and software by connecting to the device. Adept Chips’ diagnostic tool supports signal-level testing that lets customers test for individual pin outputs to get more granularity while hardware debugging for any issues.

Our tool helps production units having multiple test configurations depending on the versions. We also intend to integrate an alert system via text or email of reports of the tests to all the stakeholders once completed to ensure everyone is updated on the latest developments. Our diagnostic tools’ ability to connect to modules via telnet helps combine multiple modules simultaneously by assigning different IPs to modules. This feature helps to organize data collected from development centres distributed in various locations into a singular framework making it easier to track development progress from various system modules.

Our vision for our Embedded Platform Testing & Diagnostics Tool is to evolve continuously to include support for multiple testing standards. We will continue to provide the modularity and flexibility of test case design to our customers to assist them in their development process.