STARWEST 2018 - Test Techniques

Skill with Git is a prerequisite for most software jobs today. This is because the vast majority of software developed is stored in Git-based repositories such as GitHub, GitLab, and BitBucket. GitHub's domination of social coding makes it the new business card to demonstrate your creativity, popularity, capability, and tenacity. “Configuration as code” is a standard DevOps practice so testers must know how to set up and use Git to obtain and update versions of their infrastructures. In this hands-on tutorial, create your own website and learn the GitHub markdown used to format documents....

We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. Mary Thorn says help is at hand. Enter behavior-driven development (BDD), and Cucumber and SpecFlow, tools for running automated acceptance tests and facilitating BDD. Mary explores the nuances of Cucumber and SpecFlow, and shows you how to...

The authors of the Agile Manifesto wanted to emphasize lightweight and collaborative ways to deliver good quality software, not create a bunch of procedures and rituals to blindly follow. If you find that your project considers testing to be only about making sure you have unit tests for every method or a user story for every requirement, you may be missing out on a lot of bugs. Join Jon Bach to learn about session-based exploration, a lightweight and fun way to find new and emerging issues while also managing your testing focus and increasing your effectiveness as a quality engineer....

Understanding when and how to apply test design techniques is fundamental to your testing. Join Isabel Evans to learn some basic techniques that she uses in her own work, and add these vital test design techniques to your personal toolbox of methods. Isabel will demonstrate how to use test design techniques not just to decide what tests to conduct, but also to find potential defects and missing requirements before coding starts, to derive a test basis when requirements are unwritten, to drive the direction for nonscripted approaches, and to provide input to automation. Different techniques...

Many organizations find that test automation does not work as well as they thought it would. In many cases, these failures are due to generic technical reasons, which can be fixed with relative ease. Other solutions that have worked well are patterns, common to automation efforts at any level with whatever tools you are using. Seretta Gamba focuses on often-neglected technical issues—i.e., non-management issues—and the patterns that help solve them. These are not development or code patterns—this is a code-free tutorial. Using a set of patterns developed with Dot Graham, Seretta...

Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking,...

Have you ever worked on a project where you felt testing was thorough and complete—all features were covered and all tests passed—yet in the first week in production the software had serious issues and problems? Join Dawn Haynes to learn how to inject robustness testing into your projects to uncover those issues before release. Robustness—an important and often overlooked area of testing—is the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. Dawn shows you how—by expanding basic tests and incorporating specific...

The cognitive skills of testing are being threatened by two major forces: the assumption that automation can replace all other forms of testing, and the acceptance of lower quality by consumers. You might be feeling like you’re living on an eroding island, but there is a way to adapt and even thrive using your testing skills. Your project still needs to have someone who will question assumptions, examine design, create experiments, analyze data, and report meaningful metrics. It needs your social skill to connect teams who work in silos, bring customer insights to light, and track risks...

Isn’t it amazing? Stakeholders drop software on our desks and expect us to test it—without any requirements, design, or product knowledge whatsoever. About the only clear thing is the absurd and unrealistic deadline. We are expected to bend over backward, spread magic pixie dust, and heroically test quality into a product we have never heard of before. But testing in the dark is not impossible, and as Rob Sabourin shows, it can even be a very valuable and fun experience. Learn strategies to emerge from a murky fog into clear, meaningful quality insights and leverage unlikely sources about...

Software testing is sometimes described as “verification and validation”—or, according to Wikipedia, “the process of checking that a software system meets specifications and that it fulfills its intended purpose.” Yet, renowned tester and teacher Michael Bolton argues, if we examine the concept and logic of verification, we quickly recognize that there are serious limitations to what can and cannot be checked and verified. This is not to say that checking is a bad thing—on the contrary; checking can be very valuable. Still, it’s important for testers and their clients to recognize the...

Reality is a great source of inspiration. Real-life situations can present complexities that are not always anticipated—and, as a consequence, not always handled well. Business functional tests should try to present situations that are routed in reality but also aren’t too obvious. Testing and automation pioneer Hans Buwalda came up with a concept for test design called "soap opera testing" based on this concept. It is a style of writing tests where one writes as if they were episodes in an imaginary soap opera on television. Soap opera episodes are based on real life, but usually they are...

When Isabel Evans learned to drive a car, she also learned how to check, clean, and change spark plugs, mend the fan belt with a stocking, and indicate speed and direction changes with arm and hand signals. Now, we don’t expect to have to do any of those things; we just drive the car. That’s how test tools and automation could be. Just drive and concentrate on the journey of delivering software continuously—concentrate on engineering the solutions, not on the automation. To be effective engineers, we need the support of a powerful toolset that we understand. Is that what we have? Or do we...

Today’s IT systems communicate with customers through multiple points of engagement and various interfaces, ranging from web, mobile, and voice to BOTs and apps like Alexa and Siri. Sanil Pillai says these systems need to provide seamless handoffs between different points of interaction—while at the same time providing relevant and contextual information quickly. To accomplish this, a team must be able to successfully pair device hardware capabilities and intelligent software technologies such as location intelligence, biometric sensing, and Bluetooth. Sanil shows that testing...

You may have heard the saying “The only constant on any project is change.” Yet the prospect of change is rarely welcomed—either personally or professionally. How is it that we still believe that these changes apply to others but not to us? Julie Gardiner says that now is the time to re-evaluate and transform how we do testing in order to deliver more value to organizations—from a people, processes, and tools perspective. Join Julie as she shares current experiences of transformations and lessons learned within different organizations. She discusses an automation framework that ended up...

The most common requests Dawn Haynes gets as a consultant these days is to help testers transition to an agile development process, or to help testers be more effective in “agile-ish” environments. But Dawn recognises that transforming the process and the environment is not enough. Interestingly, the core answer to these questions starts with forgetting the process for a moment and focusing on yourself and what you’re trying to accomplish. Being agile starts with a mindset and an attitude that drive focus, approaches, and solutions. When you start there, the path to improvement can almost...

API-based applications have become ubiquitous in the past few years. Yet many of us still really don't know what an API is from the inside out. Moreover, many of us don't know much about how to comprehensively test them! Join Paul Merrill to learn what everyone is doing and talking about, related to APIs: microservices, REST, endpoints, requests, request methods, and responses. Paul demonstrates four free tools you can use to directly interact with a simple API: curl, the browser, Postman, and Java (with Rest Assured). Learn techniques for testing APIs at the service level and how to test...

Performance issues substantially impact quality, cost, and customer confidence. Agile teams are challenged to build in performance processes throughout the lifecycle, but it is critical to incorporate performance into your CI/CD pipeline. Join Amit Patel as he shares his recent project experiences and the steps his team took to change processes, leverage different technologies, and align internal stakeholders. He explains how they use production-monitoring solutions to create a real-world production feedback loop in order to ensure they can analyze data and turn the information...

Successful agile testers collaborate with programmers as code is written, isolating problems, troubleshooting defects, and debugging code all along the way to getting the product to done. But modern systems are scaling beyond what traditional teams are able to understand using familiar tools. New appreciation for systems and complexity theory, as well as disciplines and tools around emerging areas such as observability and resilience engineering, are offering solutions that allow teams to actively debug their systems and explore properties and patterns they have not defined in advance....

When the atmosphere is hostile to QA, and yet the demands on the QA Team are increasing, how do you transform a team where everything is tested and deployed manually, to an organization that delivers great software multiple times a day? Where do you start and how do you create the strategy for implementing Continuous Testing? Join David Lumpkin as he shares his company's journey to answer these questions and the team's evolution along the way. Over a three-year period, Craftsy went from an environment hostile towards QA, to one that embraces automation and exploratory testing,...