BUSINESS WHITE PAPER Maximize QA Resources with TestDriven Development and Automation Find and fix defects earlier and reduce the risk of customer-facing defects with a development strategy driven by automated testing. Overview TABLE OF CONTENTS Overview........................................1 The Reality of Manual Testing....2 QA and Project Realities..............2 Automated Customer Experience Testing.............................................3 Results of Automation.................3 Benefits of Automated Customer Experience Testing........................4 Conclusion.....................................5 About Cyara...................................5 Quality Assurance (QA) is a critical role in any technology project. QA professionals ensure that when new systems are rolled out, the customer experience works as intended without defects that are visible to customers. Not getting it right costs you money, customers, and reputation as increasingly companies are penalized for customer experience issues. You can make sure you get the customer experience right by effectively using your QA resources. The challenge for QA in any project is two-fold. First, development delays lead to pressure to “make up time” in QA, which compromises the thoroughness of testing. Second, critical defects that are identified during testing further compress the time available to QA. Do you ever wonder how products are released with critical defects? Automated testing can help resolve the time crunch by reducing the time needed for testing, decreasing the cost of testing, and testing more of the software. But there is still a problem if automated testing is not deployed until the end of the project. A poor customer experience, a drop in customer loyalty, and direct or indirect financial losses are often the result of a project methodology where testing follows development, instead of being an integral part of development. This white paper explains how a test-driven strategy, one where test cases are created using automated testing software in parallel with development, can help you find and fix defects earlier, thus saving you both time and money and reducing the risk of customer-facing defects. You will also learn how this strategy better prepares your QA team for the quick changes and rapid regression cycles that occur when defects are discovered. An example of time saved by a real Cyara customer is also included to demonstrate the advantage of automated testing over traditional manual customer experience testing. BUSINESS WHITE PAPER QA and Project Realities / Page 2 Benefits of automated testing: • • • • • • Testing starts earlier Criteria is more objective Saves time, up to 80% Eliminates human error Fix defect faster Better defect documentation • Minimizes regression testing • Lowers risk of defects in production The Reality of Manual Testing Many companies have embraced automated testing tools across the majority of their IT infrastructure. Automated regression and load testing tools are available in virtually any environment. But, this trend has been slow to take hold in customer experience testing and many companies continue to rely on traditional QA practices, particularly manual testing. In most software development methodologies, QA’s work comes at the end of the cycle, when the development is complete, and project timelines are established up front before development begins. At that early point, it is difficult to estimate how much time is needed for testing because the number of defects and their severity is unknown. With manual regression testing, the project timelines are blown up if there are many defects or many re-runs of the entire testing suite that can be required for effective regression testing. As a consequence, projects extend beyond the original delivery date. This additional time causes a delay and leads to frustration. This is because the realities of cost and time narrow the scope of regression testing. In a typical manual testing scenario, another risk factor is that when defects are corrected, QA teams usually retest only the fix. Likewise, when existing products are enhanced or changes are made, typically just that change or enhancement is tested. If the “fix” or change affects any other systems or processes that weren’t tested, there is a high risk of a customer-facing defect that will be discovered in production, potentially causing downtime and service outages that hit your bottom line and damage your reputation with your customers. Relying on manual testing not only jeopardizes project timelines, it also puts quality at risk. Performing QA in this traditional way near the end of the project also leads to inefficient use of QA resources. In a typical project there is a waiting game. QA has little to do while waiting on development to finish their work. Then, QA is overwhelmed once development is done and there is a mad rush to the finish line for the project. An ironic consequence is that often after QA has identified defects and communicated them to development, QA can then end up playing the waiting game again, while they wait for the development team to produce the fix. QA and Project Realities For many companies, product release dates are driven by factors far outside of development and QA. Often things such as a promise to a customer, the need for something new to drive sales, or pressure to keep up with competitors leads to setting unrealistic project delivery dates that are unmovable. When reality hits with a deadline that can’t be moved, project timelines become too short, and the release date is not always able to accommodate all of the tasks in the project lifecycle. When QA cycles are reserved for the end of the project, management looks to QA to make up the time. Eliminating planned testing cycles essentially puts the entire project at a heightened risk for errors that could be found by customers. BUSINESS WHITE PAPER QA and Project Realities / Page 3 Mid-project adjustments to the testing plan that reduce QA time in response to schedule pressures are the major cause of critical defects being released into production. This scenario is all too common in customer experience projects. By not carrying out complete regression testing of the overall customer experience, the team is making a decision to accept the risk of unanticipated impacts caused by the latest enhancement. Typically, the QA team is fully aware of these risks, but there is no other option. In many customer experience projects, the clock just runs out. As systems age over time, the situation does not improve. With every project to enhance or change existing systems, budget pressure continue to dictate that testing be performed only on that enhancement. Manual regression testing is just too slow and costly to thoroughly test the entire system, and, since it comes right before release, there’s just not enough time for a complete testing cycle. Over time, the limited, manual testing process all but assures that critical defects are released to production and found by customers. Automated Customer Experience Testing There is an answer. Automated testing reduces the overall time needed to ensure robust test coverage, and tests can be run throughout the entire project lifecycle. Cyara offers a comprehensive automated customer experience testing platform to support companies that want to implement capabilities more rapidly, ending the time crunch and ensuring that the customer experience that is designed is the customer experience that is delivered. Recently, Cyara was approached by a major US airline that wanted to replace their old voice self-service system with a new speech-enabled, sophisticated IVR to deliver a uniquely pleasant and efficient customer experience. Anticipating the need for multiple and ongoing regression testing cycles, the airline investigated automated customer experience testing software to better utilize their QA team and ultimately release a better product. Results of Automation During the first phase of the four-phase project, automation was not yet available and the test cycle took two testers two weeks to complete. Automated customer experience testing software was then introduced, and the phase one test cycle was automated as a regression campaign including all of the functional test cases. Running the regression campaign, it took one tester just four hours to complete and validate the campaign. That’s four hours using automation instead of 160 hours using manual testing as shown in the chart below. Manual Regression Testing Cyara Automated Regression Testing 2 Testers 160 hours 1 Tester 4 hours for identical task BUSINESS WHITE PAPER QA and Project Realities / Page 4 Thorough regression testing of the customer experience uncovers defects that will not be found by testing only the “fix”. Developing the regression test cases was an investment, but it paid dividends not only over the balance of the project, but also for future QA testing changes as well. The airline now has a full set of regression test cases, which they have subsequently added to, to run against the product whenever new functionality is added or changed. This thorough regression testing of the customer experience has uncovered defects that would not have been caught if they had continued testing only specific changes. Benefits of Automated Customer Experience Testing The major benefit of automated testing for the airline has been the ability to implement test-driven development. The airline’s QA team prepares and adjusts test cases in parallel with development, which provides the following benefits: • Functional tests are performed earlier in the project using exactly the same objective criteria that will be used in final testing • Automated testing is 80% faster than manual testing • Defects are determined by the testing software based on agreed criteria • Human error and subjectivity is eliminated in determining errors • Time needed to fix defects is reduced because QA has more time to document information on the defect to pass to development • Time required to thoroughly regression-test the fix is minimal due to automation • Lower probability of releasing a critical defect to production In addition, the airline is now in a position to move rapidly to adapt their customer experience as business requirements and customer expectations change in the future. The airline’s QA resources are now spending their time building test cases that effectively ensure the design specifications are met, working with development to ensure testing is available when capabilities are complete, and designing incremental testing that can drive business results even higher. The majority of defects are now found during development because those test cases are available sooner, resulting in dramatically cleaner releases to user acceptance testing and periodic load testing cycles. BUSINESS WHITE PAPER QA and Project Realities / Page 5 Conclusion Automated customer experience testing supporting a test-driven development strategy has resulted in tremendous benefits for this major US airline helping them transform the IVR experience for their customers. It also enabled their critical need to continue to evolve the customer experience to keep pace with business needs while ensuring that they are releasing a higher quality product. They are now setting the benchmark in the travel industry for delivering the finest caller experience, helping them to achieve and maintain a leadership position in their industry. About Cyara: Cyara is the leading provider of customer experience testing for organizations around the world. The Cyara Platform enables contact centers and online channels to optimize the customer experience rapidly and decrease the risk of exposing customers to a poor experience. Every day, millions of flawless customer interactions are delivered by organizations utilizing the Cyara Platform. Many Fortune 100 companies trust Cyara and automated testing for their contact centers. If you are planning a significant transformation of your contact center or to quickly bring customer experience improvements to your customers, Cyara is here to help. www.Cyara.com Corporate Headquarters: Level 1 / 1 Cook Street Hawthorn, VIC 3122 | AUSTRALIA Tel. +61 3 9093 0815 | www.cyara.com Cyara and the Cyara logo are trademarks or registered trademarks of Cyara Solutions, Pty Ltd. or its affiliates in the United States of America and/or other countries. All other company names or product names may be the trademarks of their respective owners. © 2014 Cyara. All rights reserved.
© Copyright 2019