Keyword-Driven IoT Testing for a Leading US Manufacturer’s Smart Thermostat App
Our client is one of the largest manufacturers of residential and commercial water heaters, boilers, and HVAC equipment in the United States — with products sold globally. As part of their connected product strategy, the company developed a smart thermostat device that allows homeowners and facility managers to control room temperature, water heating, and HVAC systems remotely through a companion mobile application.
The thermostat device communicates over Wi-Fi and integrates a water management system, Wi-Fi module, and control system into a single hardware kit. The mobile app — available on both iOS and Android — lets users adjust temperature, set schedules, check device status, and manage connected systems from anywhere.
With the product ready for broad market release, the client needed a comprehensive testing partner to validate the full mobile experience across device types, network conditions, and security requirements — and to build an automation framework capable of sustaining regression coverage across future product updates.
IoT mobile testing presents a unique class of challenges that standard UI automation doesn’t address. The app’s behavior depends not just on the software, but on the physical hardware state — the thermostat device, its network connectivity, the control system response, and real-time synchronization between the mobile client and the embedded device.
The client needed testing coverage across three dimensions simultaneously:
Full functional testing of the mobile app across both iOS and Android — including temperature control, schedule management, status monitoring, and device pairing flows
Network environment testing under 2G, 3G, 4G, and Wi-Fi conditions — ensuring reliable connectivity and graceful degradation on slower networks
Security validation — verifying data transmission, authentication, and protection of home network credentials stored or transmitted by the app
Device compatibility validation across 100+ real iOS and Android devices spanning different manufacturers, screen sizes, and OS versions
A reusable automation framework capable of running regression suites for future releases without rebuilding from scratch
SDT deployed a Keyword-driven automation approach using the Robot Automation Framework — specifically chosen for its ability to bridge the gap between the business-language test cases domain experts can write and the technical execution layer required to interact with real devices and IoT hardware.
The client provided the hardware kit — including the water management system, Wi-Fi module, and integrated control system device — which was used as the live test environment throughout the engagement. Testing was conducted on physical devices, not simulators, to ensure real-world fidelity.
The Keyword approach ensured that test cases were maintainable independently of the automation layer — meaning the regression suite can be extended by test designers without requiring deep technical knowledge of the underlying framework.
Test Management Tool
Robot Automation Framework
Keyword automation approach
100+ iOS and Android devices
2G/3G/4G/Wi-Fi environments
Hardware kit components
"SDT successfully tested an exclusive and complex business app on multiple iOS and Android devices under 2G/3G/4G/Wi-Fi environments — ensuring the client's connected product reached market with confidence across every device and network condition their customers would encounter."
— Rheem · VP of Engineering
Testing was conducted across a broad matrix of real devices and network conditions — not emulators — to validate behavior at each level of connectivity fidelity.
| Category | Test Item | Coverage Detail | Status |
|---|---|---|---|
| Network Environments |
Wi-Fi | Full-speed baseline — primary home and commercial environment; used as reference for all feature testing | Tested |
| 4G LTE | Mobile broadband — standard cellular conditions; tested all real-time thermostat control flows | Tested | |
| 3G | Variable latency — tested app responsiveness and timeout handling under degraded connection | Tested | |
| 2G | Low-bandwidth edge conditions — validated graceful degradation and error messaging | Tested | |
| Mobile Platforms |
iOS | iPhone and iPad — multiple OS versions (iOS 14+); tested pairing, control, scheduling, status on all major form factors | ✓ Compatible |
| Android | 100+ device models across Samsung, Google, OnePlus, and others — multiple Android OS versions and screen sizes | ✓ Compatible | |
| Functional Coverage |
Temperature Control | Set, adjust, and confirm temperature changes in real time across all network conditions | ✓ Passed |
| Schedule Management | Create, edit, and delete schedules; verify device sync and execution on hardware kit | ✓ Passed | |
| Device Pairing | Initial setup flow, re-pairing, and error recovery tested across 2G–Wi-Fi environments | ✓ Passed | |
| Status Monitoring | Real-time device status updates, connectivity indicators, and notification handling | ✓ Passed | |
| Security Validation |
Authentication | Login, session management, token handling, and logout flows across iOS and Android | Validated |
| Data Transmission | Verified encrypted transmission of thermostat commands and status data between app and device | Validated | |
| Credential Handling | Home Wi-Fi credentials stored and transmitted by the app verified against security standards | Validated | |
| IoT Hardware Kit |
Wi-Fi Module | Connectivity, reconnection behavior, and signal handling under all four network conditions | ✓ Passed |
| Water Management System | Command receipt and execution from app; status reporting back to mobile client | ✓ Passed | |
| Control System Device | End-to-end integration — app command → device response latency tested across all network environments | ✓ Passed |
Let’s discuss how we can help you achieve similar results with our proven methodologies and expert consulting services.
