STARWEST 2016 - Test Automation

The Internet of Things (IoT) is the next big technology challenge for software testing. IoT testing uses concepts from traditional and mobile environments but has new testing problems and new patterns. Jon Hagar begins by examining how to use data analytics from error profiles and social media to discover the new error patterns in IoT systems. Usage data on IoT devices is growing rapidly and becoming a big data issue. Through hands-on exercises, Jon explains how teams can use data analytics to improve development and testing. Next, he uses the analytics to define...

With the adoption of agile practices in many organizations, the test automation landscape has changed. Bob Galen explores current disruptors to traditional automation strategies, and discusses relevant and current adjustments you need to make when developing your automation business case. Open source tools are becoming incredibly viable and beat their commercial equivalents in many ways―not only in cost but also in functionality, creativity, evolutionary speed, and developer acceptance. Agile methods have fundamentally challenged our traditional automation...

Knowledge of Selenium, the industry-standard tool for testing web applications, is a much sought after skill in today’s world of test automation. Many believe it is a must-have skill for test engineers. If you want to learn Selenium, then this tutorial is a great start. Cheezy Morgan shows you how to build test automation using Selenium. But he doesn’t stop there. He uses his years of experience to show you how to build automation that is clean and easy to maintain. Cheezy introduces other tools that work with Selenium to help manage the data used to drive your...

Let’s face it—agile testing is different. Challenges exist in successfully integrating within the teams themselves. Scrummerfall continues to run rampant. The dichotomy of testing vs. quality and balancing both the team’s and your focus still exists. Delivering value is both an imperative and a challenge. In this dynamic workshop, join agile coaches Mary Thorn and Bob Galen as they explore the tools, techniques, and mindset you need to bring to the table to successfully test in agile contexts. Mary and Bob examine risk-based testing, iterative test planning,...

Today, many agile organizations are making a terrible error. They are assuming there is no place for test management and leadership in agile, self-directed team contexts. We beg to differ with this view and believe a strong need exists for testing leadership in agile organizations—just not the way we’ve typically approached it. Join Mary Thorn and Bob Galen as they explore what excellent test team leadership looks like in agile contexts. Explore the aspects of self-directed teams and the implications to your previous leadership styles. Look under the covers of...

The continuous delivery pipeline is the process of taking new or changed features from developers, and getting features deployed into production and delivered quickly to the customer. Gene Gotimer says testing within continuous delivery pipelines should be designed so the earliest tests are the quickest and easiest to run, giving developers the fastest feedback. Successive rounds of testing lead to increased confidence that the code is a viable candidate for production and that more expensive tests—time, effort, cost—are justified. Manual testing is performed...

Are you tired of flaky automation scripts that fail without apparent cause? Are most script failures due to bugs in the test script rather than bugs in the product? Do you simply want to get more out of your automation framework? Many testers and organizations are frustrated with the high cost and low return of their automation frameworks. Scott Miles shares a range of techniques to convert your existing framework from brittle to adaptive. Learn how to improve the quality of your automation framework while reducing the required maintenance. Adaptive automation...

Capital One’s highly integrated environment creates many interdependencies for its agile teams. Because these dependencies were not being completed until late in their sprints, Adam Auerbach says that Capital One faced prolonged integration and regression testing phases and did not realize expected improvements in quality or time-to-market. As technology leaders pushed for continuous delivery (CD), testing needed to shift left and occur simultaneously with development. To shift left, the testing community needed to learn basic development skills, including Ruby...

Much of the success of agile adoptions is due to the automated testing approach used in agile projects. Because many of these techniques were pioneered in the development of web applications, it can be difficult to see how these techniques can be leveraged for a project where software is being built for an embedded or Internet of Things (IoT) application. Thomas Stiehm describes ways to leverage agile testing techniques for embedded systems. Whether you are building a medical device, embedded controller, or IoT device, learn how to leverage these testing...

Behavior-driven development (BDD) is a hot topic in the development community. Not only does a properly implemented BDD process help drive increased automation and quicker development cycles, it also facilitates better collaboration between departments and reduces siloed communication. An ideal partner of continuous integration/delivery, BDD can help solve many testing bottlenecks associated with DevOps. For all its benefits, BDD is underadopted. Only 10–25 percent of development organizations have implemented or are experimenting with a BDD process....

Mobile testing is getting harder—more devices, multiple operating systems, higher quality expectations, and shorter development cycles. How do you deal with these demands? In order to align mobile testing with product strategies and market goals, Tom Chavez says you first need to (1) know your users and how they will use your app. (2) Knowing the app and how users may actually be using it differently are key to testing to satisfy users—not the designers. (3) With test case matrices vastly larger than ever, prioritizing tests into “must-haves” and “nice-to-haves”...

If you don’t proactively manage and control your application test data, you may be missing out on a big process improvement that will save money, improve test coverage, and may save your systems from embarrassing failures. Paul Merrill describes the advantages and disadvantages of four data strategies—Elementary, Refresh Data Source, Selfish Data Generation, and Selfish Data Generation with Batch Cleanup—to help gain control of test data. Explore the two components of a data strategy—the Creational Strategy and the Cleanup Strategy—and their value to effective...

The expression “garbage, garbage out” emphasizes the need for thorough testing in any Big Data and analytics implementation. Big Data testing means ensuring the correctness and completeness of voluminous, often heterogeneous, data as it moves across different stages—ingestion, storage, analytics, and visualization—producing actionable insights. What should be our testing focus? Which of the 4 V’s—variety, volume, velocity, and veracity—are most important at which stage? For example, in the ingestion stage, testing needs to focus on variety of data rather than...

Although most test managers and testing staff know how to cope with new and difficult testing challenges, Martin Pol and Jeroen Mengerink maintain that the cloud phenomenon requires special attention. Testing must evolve and innovate to address the newly introduced risks combined with the changing emphasis for classical quality requirements. How do you test resource elasticity and the “bring your own device” philosophy with their cloud-specific security and performance risks? How do you understand and then cope with local and international privacy legislation for...

Big Data, one of the latest buzzwords in our industry, involves working with petabytes of data captured by various systems and making sense of that data in some way. Maryam Umar has found that testing systems like Hadoop is very challenging because of the frequency with which the data arrives in the system, the number of jobs that run to process that data, and the interdependency of the data. Maryam describes some of the projects at Hotels.com which involve identifying multiple users and using that data to make recommendations of hotels. Testing this is fairly...

Security, data privacy, reliability, and regulatory compliance are critical factors when evaluating whether to move business applications from in-house, client-hosted environments to a cloud platform. Quality assurance plays a vital role in ensuring that the appropriate level of risk assessment, verification, and validation takes place to ensure business continuity during the migration to a new cloud platform. Vandana Viswanathan shares an introduction to the five components of a sound cloud application validation strategy: cloud provider qualification process,...