As we were reminded while watching the Olympics in Rio de Janeiro this summer, nearly every event involves one or more qualifying rounds. Each athlete or team has to be successful in qualifying in order to move onto the next round, and eventually, strive for the gold. While it might be a bit of a stretch, one could apply a similar analogy to IoT product development, with Thread certification being one of many events a product needs to "win" to make it to market.
From its inception, the Thread Group has held a rigorous testing and certification program (an Olympics, of sorts) as a cornerstone of delivering products that connect effortlessly, reliably and securely.
In its effort to drive adoption of the Thread networking protocol in the places where we live and work, the Thread Group this week opened its test tools and test lab to its members. In support of this "semi-final" round, the Thread Group announced that ARM, NXP and Silicon Labs have released the first conforming stacks that have successfully passed testing based on the Thread 1.1 technical specification and are now undergoing interoperability evaluation to achieve the Gold medal - the Thread certification logo.
As the final round approaches and certification is in sight, developers, retailers and consumers alike can feel confident that solutions featuring the Thread Certified Component or Built on Thread mark provide a solid; secure networking foundation to usher in a new era of connected products.
We asked The Thread Group's Certification Director, Tom Sciorilli to share more details about this announcement and how it will benefit our members and the industry:
Q: What does this announcement really mean?
A: This announcement means that the independent test house, UL, is open for initial product testing to help members build confidence that their solution will meet the ultimate certification goal of ensuring Thread-based products work together seamlessly and securely out of the box. All members may now submit their product to UL for debug and testing.
Q: Please tell us more about the test resources and what value each provides?
A: Availability of the test tools and opening of the test lab is the final milestone on the path towards offering Thread product certification. Since releasing the Thread 1.0 technical specification in July 2015, the Thread Group's engineering team has worked tirelessly to evolve and mature the specification to a 1.1 release, adding critical channel agility and master key change capabilities, plus enhancing core foundational operation to make Thread networks even more secure and robust. The 1.1 specification is the standard against which all Thread-enabled products will be evaluated to achieve certification.
In concert with the specification, Thread Group has defined a reference test bed comprised of Thread stacks developed by ARM, NXP and Silicon Labs. This test bed comprises the set of reference implementations against which all Thread products shall be tested for achieving certification. The test bed is used to execute a comprehensive test suite that has been vetted for coverage by Thread experts. To make a repeatable and efficient certification test process, Thread Group built an automated test platform which directs all test bed devices in the performance of each test, collects all the test output data, and analyzes the output data for correctness.
To help members progress through their own rounds of "qualification" for certification, the reference test bed and automated test platform are available for all Contributor and Sponsor members to acquire, set up and run internal debug/validation testing in their own environment. Members who don't have the resources or desire to use the test environment internally are welcome to book time at UL to accomplish the same debug/validation work, but with UL's resources and assistance.
The early testing involves exercising a Thread product against one or more reference stack implementations in a single-vendor test environment, e.g., a single vendor's device is used exclusively in a full pre-certification test run. Testing using a single reference stack provides substantial Thread specification conformance coverage, while testing with more than one reference stack provides excellent insight into the looming challenges of passing the multi-stack interoperability certification testing.
Q: Will companies receive a certification logo at this phase?
A: No, just like in the semi-finals of the Olympics, no medal or logo is given out until completing the final round of the certification program.
Q: So, why should companies put their effort into testing before the certification program is live?
Companies developing their own Thread stacks or those using the lead validated stacks for their product can use the Thread test environment to start evaluating the robustness of their product. Once product certification is released, they will be highly confident of having a short path to certification, and market launch.
Q: What happens if companies and their products successfully complete the Thread certification program? When do you expect to see Thread-certified products in the market?
A: Products, which successfully complete the certification program, receive the Thread certification logo. Thread is planning to open certification testing by the end of 2016, and we could see certified products in the market as early as CES 2017.
Q: Why is interoperability testing so critical for product manufacturers?
A: Manufacturers are very good at testing their own devices thoroughly within their own environment, but the ability to test their device against the sea of other manufacturers' devices is a tightrope between coverage and cost effectiveness. In a perfect world, a new device would be tested exhaustively and repeatedly against every other Thread device on the market. Thread certification aims to provide substantially similar coverage to its members within reasonable cost and time.