Background: The customer is a global leading wireless communication device company. They need to test a mesh network wireless routers family. The test of each product model/version is hard to regress, in addition to the poor productivity, so the automation test must be imported to increase efficiency.

A traditional automation test hardware environment, or testbed, is usually composed of a Windows PC (acting as server) preinstalled with Radius or other service software, two Linux PCs with WiFi NICs as wireless clients, one Ixia test device, two or more wireless routers as DUTs (device under test) that make up of a mesh network. One of the routers is used to control DUT’s terminal server (TS), and one controls the switch to connect tester’s client PC with all DUTs, composing a control network. All devices’ network interfaces are interconnected by a switch to build a test network. The DUT provides a command line console accessible by serial ports or the Ethernet network. The test client PC can be Telneted to the TS/test PC with Tcl/Expect languages, which can automate the DUT and wireless client. Then we’ll install Ixia client on the test client PC to control the Ixia test equipment and obtain the associated Tcl test script.

The automation test scripts run on tester’s client PC; the tester manages/executes these scripts and makes sure the devices for such scripts to run on don’t conflict with each other. The testbed is configured as below:

Sigma-RT provided a SigmationTF-based full automation test solution to address the company’s wireless product tests. In this solution, a master controller (the “MC”) is used to control testbeds in replacement of test engineers, and test Jobs are scheduled by their priority setting and the testbed state. The solution is designed to secure the communication products quality in a reliable, effective way.

When an automation test Job is scheduled by its priority level, the system will choose a testbed with enough available resources and occupy the device(s) the testcase requires, then control the switch to create logic topology connections according to the testcase topology configuration, set up device initiation parameters, and then execute the test/verification steps before logging. A statistic report will be given after all test cases of the automation test Job have been accomplished. Multiple automation test Jobs can be executed concurrently under qualified resource conditions.

SigmationTF provides a PC proxy and PC control libraries for PC controlling, as well as device control libraries/expansion framework for the control of different DUTs (device under test) by inheritance to device control libraries and self-defined expansion; it also provides plug-ins with various applications for regular test processes and the control of popular test tools/equipment.

In this solution, the Ixia Tcl server has been installed on PC 3. SigmationTF’s MC directly invokes Ixia API on the PC 3’s Ixia plugin and controls Ixia Chassis instrument to generate test traffic. The SigmationTF testbed is configured as below:

Comparatively, a conventional automation test solution faces the following problems:

First, the device resource conflict between different test Jobs. For instance, once a test Job is already running on DUT through the TS (terminal server), any other engineer’s actions such as accessing the DUT, modifying parameters or rebooting will disturb the test execution, further, you’ll have to modify connection parameters whenever converting to other devices.

Second, changes of various testcase topologies require physical connections consequently, which is ineffective.

Third, it is hard to orchestrate all devices by scripts because of the large coding backlog for communication, interoperation and log statistics.

So the 24/7 unattended automation test scheduling and execution become a mission-impossible because of the foregoing difficulties.

The SigmationTF solution copes with problems that hamper the test progressing in traditional test solutions, and successfully implements a full-range automation from test Job planning, scheduling, resource appointment, topology building by controlling test switches, device communication, logging to statistics reporting.

SigmationTF provides APIs and supports device inheritance/expansion, free engineers from tedious Expect device control script writing, inter-device communication coding, test data collection and library entry coding. The reuse of test code becomes easier, which relieve your engineers from heavy workload of automation test coding.

What your testing engineers need to do is focus on the test execution itself, which greatly increases test efficiency and catch bugs before they escape to your customers.