STARWEST 2016 - Test Automation Engineer

When we discover—often late in an automation effort—that the automated tests are cumbersome and costly to maintain, we often view this as a technical problem for the automator to solve. However, an often-overlooked cause is the role that testers who designed these tests play in making automation scalable and maintainable. In this interactive tutorial for both testers and automation engineers, Hans Buwalda explores how better test designs will result in much improved test automation and can make the difference between automation success and failure. See why...

We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. But help is at hand. Enter behavior-driven development (BDD) and Cucumber and SpecFlow, two tools for running automated acceptance tests and facilitating BDD. Mary Thorn explores the nuances of Cucumber and SpecFlow, and...

With the adoption of agile practices in many organizations, the test automation landscape has changed. Bob Galen explores current disruptors to traditional automation strategies, and discusses relevant and current adjustments you need to make when developing your automation business case. Open source tools are becoming incredibly viable and beat their commercial equivalents in many ways―not only in cost but also in functionality, creativity, evolutionary speed, and developer acceptance. Agile methods have fundamentally challenged our traditional automation...

Do you want to take your testing skills to the next level and help your organization reap the full benefits of testing earlier? Are you trying to stay relevant on an agile team where testing is shifting to the left of the project timeline? Join Tariq King as he explains the fundamentals of white box testing so you can find bugs as soon as they happen, and do more thorough, targeted testing during software development. Learn how to apply white box testing techniques such as testing by looking, automated unit testing, mocking, and code coverage analysis. This...

Let’s face it—agile testing is different. Challenges exist in successfully integrating within the teams themselves. Scrummerfall continues to run rampant. The dichotomy of testing vs. quality and balancing both the team’s and your focus still exists. Delivering value is both an imperative and a challenge. In this dynamic workshop, join agile coaches Mary Thorn and Bob Galen as they explore the tools, techniques, and mindset you need to bring to the table to successfully test in agile contexts. Mary and Bob examine risk-based testing, iterative test planning,...

Today, many agile organizations are making a terrible error. They are assuming there is no place for test management and leadership in agile, self-directed team contexts. We beg to differ with this view and believe a strong need exists for testing leadership in agile organizations—just not the way we’ve typically approached it. Join Mary Thorn and Bob Galen as they explore what excellent test team leadership looks like in agile contexts. Explore the aspects of self-directed teams and the implications to your previous leadership styles. Look under the covers of...

To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Mike Sowers as he addresses common metrics—...

You've been through some initial automation bootstrapping and training. You're now three to six months down the road and spending too much time chasing intermittent test failures and maintaining your scripts. You're dealing with frustration and trust issues in the automated tests. What you're not doing is adding a lot of value around new feature work. Sound familiar? Jim Holmes says that pattern is common across nearly every team that is new to building significant automation suites. Jim helps you with a few approaches—all based on his years of experience and...

The continuous delivery pipeline is the process of taking new or changed features from developers, and getting features deployed into production and delivered quickly to the customer. Gene Gotimer says testing within continuous delivery pipelines should be designed so the earliest tests are the quickest and easiest to run, giving developers the fastest feedback. Successive rounds of testing lead to increased confidence that the code is a viable candidate for production and that more expensive tests—time, effort, cost—are justified. Manual testing is performed...

Are you tired of flaky automation scripts that fail without apparent cause? Are most script failures due to bugs in the test script rather than bugs in the product? Do you simply want to get more out of your automation framework? Many testers and organizations are frustrated with the high cost and low return of their automation frameworks. Scott Miles shares a range of techniques to convert your existing framework from brittle to adaptive. Learn how to improve the quality of your automation framework while reducing the required maintenance. Adaptive automation...

Capital One’s highly integrated environment creates many interdependencies for its agile teams. Because these dependencies were not being completed until late in their sprints, Adam Auerbach says that Capital One faced prolonged integration and regression testing phases and did not realize expected improvements in quality or time-to-market. As technology leaders pushed for continuous delivery (CD), testing needed to shift left and occur simultaneously with development. To shift left, the testing community needed to learn basic development skills, including Ruby...

Much of the success of agile adoptions is due to the automated testing approach used in agile projects. Because many of these techniques were pioneered in the development of web applications, it can be difficult to see how these techniques can be leveraged for a project where software is being built for an embedded or Internet of Things (IoT) application. Thomas Stiehm describes ways to leverage agile testing techniques for embedded systems. Whether you are building a medical device, embedded controller, or IoT device, learn how to leverage these testing...

Exploratory testing (ET) is a great complement to traditional test approaches. Testing professionals often see ET as an approach for manual testing. However, trying to organize ET from a test process perspective can be a challenge. There is an important question to ask—and answer: Is exploratory testing compatible with test automation? Jim Trentadue explores how automated test cases can be designed to include provisions for exploratory testing. Examples of this are data used in input fields, navigating various paths in your application, encountering unexpected...

Behavior-driven development (BDD) is a hot topic in the development community. Not only does a properly implemented BDD process help drive increased automation and quicker development cycles, it also facilitates better collaboration between departments and reduces siloed communication. An ideal partner of continuous integration/delivery, BDD can help solve many testing bottlenecks associated with DevOps. For all its benefits, BDD is underadopted. Only 10–25 percent of development organizations have implemented or are experimenting with a BDD process....

Mobile testing is getting harder—more devices, multiple operating systems, higher quality expectations, and shorter development cycles. How do you deal with these demands? In order to align mobile testing with product strategies and market goals, Tom Chavez says you first need to (1) know your users and how they will use your app. (2) Knowing the app and how users may actually be using it differently are key to testing to satisfy users—not the designers. (3) With test case matrices vastly larger than ever, prioritizing tests into “must-haves” and “nice-to-haves”...

If you don’t proactively manage and control your application test data, you may be missing out on a big process improvement that will save money, improve test coverage, and may save your systems from embarrassing failures. Paul Merrill describes the advantages and disadvantages of four data strategies—Elementary, Refresh Data Source, Selfish Data Generation, and Selfish Data Generation with Batch Cleanup—to help gain control of test data. Explore the two components of a data strategy—the Creational Strategy and the Cleanup Strategy—and their value to effective...

Although the idea of doing performance testing throughout the software lifecycle sounds simple enough, as soon as you try to combine the concepts of “always testing” (in dev, pre-prod, and production) with “limited time and resources” and throw in the word “comprehensive,” the challenges can be monumental. Quickly the “how” of it emerges as the most important question—and one worth focusing on. Brad Stoner tackles this topic by explaining how he has been able to solve this seemingly impossible puzzle by applying various approaches such as early and often,...

Although Lee Hawkins stumbled into testing—in 1999 after migrating from the UK to Australia amid a tech boom time—he has since become a passionate member of the worldwide testing community and currently holds the title of principal test architect. So, what does that really mean? A test architect at Dell Software provides technical leadership and strategic direction for testing, and Lee describes what that means in his day-to-day work. His position involves advocacy for great new testing ideas gleaned from the wider testing community, mentoring new testers, and...

Andreas Grabner maintains that most performance and scalability problems don’t need a large or long running performance test or the expertise of a performance engineering guru. Don’t let anybody tell you that performance is too hard to practice because it actually is not. You can take the initiative and find these often serious defects. Andreas analyzed and spotted the performance and scalability issues in more than 200 applications last year. He shares his performance testing approaches and explores the top problem patterns that you can learn to spot in your...