Lockheed Martin’s GTN Program Successfully Improves its Testing Process and Implements Third Generation Test Automation
Lockheed Martin is one of the nation’s premier Government contracting companies. They develop many kinds of systems for our nation, as well as states, communities, and the private sector. They believe strongly in process-driven software development and are always open to new ideas and technology innovation. Lockheed Martin’s Mission Systems Company awarded a subcontract to Software Development Technologies (SDT) to assist with the improvement of their software testing processes and implementation of software test automation. SDT worked with the Global Transportation Network (GTN) program, a large data integration and query system used within the DoD community that is managed and operated by the United States Transportation Command (USTRANSCOM). Recently, the Lockheed Martin GTN Team was awarded a 100% award fee score from the Government. One of the noted areas of the award was the improvement in the test process and the successful implementation of a test automation capability.
For several years the GTN program was finding a significant number of issues during the final testing phase of each software release. These issues, found so late in the development process, were causing cost and schedule overruns on most releases.
Recent changes in the GTN architecture, coupled with constant change in industry web browser functionality, resulted in a requirement to repeatedly ensure that the GTN system operated correctly in this volatile environment. The Lockheed Martin team agreed that a set of web regression tests should be run with each new software release.
Working with the LM-MS team, SDT conducted an evaluation of the software development and test processes. The resulting recommendations included the need to increase focus on effective Peer Reviews of requirements and design documents, the need to involve the test organization in creating test deliverables earlier in the development cycle, the need to update processes for both peer review and test processes, and the need for training on the updated processes across the entire organization. The main goal of all of these recommendations was to detect and fix defects early in the software development lifecycle.
Due to the time and cost of the manual regression test effort (300 labor hours and 10 shifts of test lab time), a pilot activity was initiated to determine if there was a test automation capability available that could provide a more efficient solution. The pilot selected two industry-leading products to attempt to make this regression testing effort more efficient – Mercury Interactive’s WinRunner™, the leading capture/playback tool on the market and Software Development Technologies’ Unified TestPro (UTP).
Because of this Pilot success, the Government requested an investigation of the feasibility of automating more testing of the GTN architecture. This request led to a scenario-based approach that includes “dropping” and loading data files followed by a series of queries, and then “dropping” additional data files and executing more queries to simulate the real-world GTN scenarios. This effort was also successful and the scenario is currently being enhanced to accurately simulate a 10-day scenario to be executed unattended.
Because of the success of this second pilot the Government requested the LM-MS GTN test automation team implement a complete set of automated functional and scenario-based tests that will provide robust test coverage of the GTN system. This will result in a library of automated test cases available to be run whenever a change is made to the GTN system. The test cases will consist of a core set of regression tests run overnight as a confidence test, and a regression test library used selectively to perform additional regression tests based on where specific changes occurred in the GTN system.
Like many other programs, GTN had previously unsuccessfully attempted to automate testing. The current success was due to the use of the WinRunner and UTP test tools, and the involvement of an experienced test automation specialist able to resolve tool and application interface issues. These issues included applications built with non-standard GUI objects, application and tool timing issues, and Web Page naming convention issues. The UTP tool provided two major benefits:
Significantly fewer test scripts are required which reduces test maintenance costs.
Most testers do not need to know the test tool script language since test cases are recorded in excel spreadsheets using a specific vocabulary known as "Keywords".
The GTN test process enhancements and automation solution have successfully improved the quality of the delivered software and increased the amount of testing without cost and schedule growth. The following quote is evidence of this success…
"Our Lockheed Martin Global Transportation Network team has been on an aggressive plan to improve program and product performance. This included improving our product test processes and adding test automation capabilities. Since our initiative began, our process improvements have given us the ability to detect defects much earlier in the lifecycle resulting in increased value to the customer and delivering superior products. At the heart of our process improvement initiatives was SDT's testing methodology and 3rd generation approach to test automation. The SDT team provided the vision, required leadership and persistence to implement best-of-breed test processes and test automation tools".
Let’s discuss how we can help you achieve similar results with our proven methodologies and expert consulting services.
