STARWEST 2017 - Test & Release Automation

When we discover—often late in an automation effort—that the automated tests are cumbersome and costly to maintain, we often view this as a technical problem for the automator to solve. However, an often-overlooked cause is the role that testers who designed these tests play in making automation scalable and maintainable. In this interactive tutorial for both testers and automation engineers, Hans Buwalda explores how better test designs not only will result in much improved test automation but also can make the difference between automation success and failure. See why successful...

You've seen this before. The project requirement states that You will be responsible for developing test automation that covers ______. And it is rarely as simple as it first seems. With upstream and downstream considerations, tool changes, and course correction at every juncture, how can you possibly improve efficiency and quality, and still plan for and stay on target during the process of developing and implementing a robust test automation harness? Dawn Code will guide you through the steps of building test automation that can suit a variety of needs. Following a Choose Your Own...

Many organizations find that test automation does not work as well as they thought it would. In many cases, these failures are due to generic technical reasons, which can be fixed with relative ease. Solutions that have worked well for others are patterns; these test automation patterns are common to automation efforts at any level with whatever tools you are using. Dot Graham focuses on often-neglected technical issues—i.e., non-management issues—and the patterns that help solve them. These are not development or code patterns—this is a code-free tutorial. Using a set of patterns...

In many organizations, agile development processes are driving the pursuit of faster software releases, which has spawned a set of new practices called DevOps. DevOps stresses communications and integration between development and operations, including rapid deployment, continuous integration, and continuous delivery. Because DevOps practices require confidence that changes made to the code base will function as expected, automated testing is essential. Join Jeffery Payne as he discusses the unique challenges associated with integrating automated testing into continuous integration/...

The DevOps movement is here. Now, companies with mature systems are breaking down siloed IT departments and federating them into product development teams and departments. Testing and its practices are at the heart of these changes. Traditionally, development organizations have been filled with mostly manual testers and a limited number of automation and performance engineers. Adam Auerbach says this has to change. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay...

In chess, the word blunder means a very bad move by someone who should know better. Even though functional test automation has been around for a long time, people still make some very bad moves and serious blunders. The most common misconception in automation is thinking that manual testing is the same as automated testing. And this thinking accounts for most of the blunders in system level test automation. Dorothy Graham takes us on a tour of these blunders, including: the Stable-Application Myth (you can’t start automating until the application is stable), Inside-the-Box Thinking (...

Creating automated tests for your team stories, integration, or regression test cycles within agile sprints is almost every tester’s top challenge. Usually it consumes many hours and requires a great deal of effort to achieve, especially in complex and large agile projects. Teams need to deliver software as quickly as they can while producing the best possible product quality. Talal Ibdah shows how you can achieve these goals and automate your API functional and performance tests; define test environments and configuration files; make chaining requests; continuously deliver your...

A lot of folks doing testing (QAs, BAs, and Devs alike) are experienced with testing applications through the front end—a graphical user interface or a mobile app. However, Hilary Weaver-Robb says that with this type of testing we often miss the internal web services and APIs that power those applications. Integration or web service tests are right in the middle of the Testing Pyramid, so to ensure adequate coverage it’s vital for testers to know how to test at that level. Thankfully, to test web services we can apply many of the same principles we already know. Hilary focuses on...

You may have heard the term “continuous testing” and thought it was just the DevOps flavor of the month … or that it isn’t part of DevOps … or that it isn’t for cloud-based applications. Marianne Hollier says that continuous testing means adopting the right set of automated tests along with service virtualization, which allows the team to simulate missing dependencies and to start testing earlier and more frequently. She shares how the right combination of best practices and tools can help software development and testing teams adopt a continuous testing approach. Since you can’t test...

Automation is a powerful tool to help testing but too often it is used to replicate existing manual tests. This leads organizations to spend large amounts of time and money constantly updating flaky automated tests and test teams to suffer frustration from having to focus on activities that are not truly testing. This cost and frustration can be avoided by using automation as a tool to assist testing—not to replace tests. Jeffrey Martin shares some real-world examples of using automation to supplement testing by leveraging its true value—the replication and repetition of tasks...

Selenium has an industry reputation of being a “flaky” tool where individual tests pass, then fail—sometimes with no production changes at all. Such flakiness in your test suites can be extremely difficult, time consuming, and frustrating to debug. The vast majority of these issues stem from using either bad locators or bad wait conditions. But both of these root causes can be addressed by implementing the right framework for your Selenium tests. Craig Schwarzwald shares the most important concepts in creating such a Selenium framework. He has proved that using a framework...

Walk the Expo, and you will see all kinds of test automation tools. Some run scripts. Some communicate with the system under test. Some virtualize system components. Some do interesting things that you may never have considered. Yet, none gives you a complete recipe for testing your product and synthesizing the results. That is not their job. It's yours. Mike Duskis says an effective test automation program will reflect the unique nuances of your product and your business. However, unique nuances need not add up to radically different architectures. In fact, effective automation...

Is your company spending a lot of time and effort on an automation strategy while your customers believe that product quality has not improved? Does management see automation as a silver bullet that will save money, increase coverage, and reduce headcount? Do you work for a company where the goal is (almost) 100 percent test automation? Paul Holland discusses issues and problems with these approaches and perceptions about test automation. He provides strong arguments why the “automate everything” approach is not likely to be successful and provides details of an alternative, balanced...