STARWEST 2017 - Managing Risk

Critical thinking is the kind of thinking that specifically looks for problems and mistakes. Regular people don't do a lot of it. However, if you want to be a great tester, you need to be a great critical thinker. Critically-thinking testers save projects from dangerous assumptions and ultimately from disasters. The good news is that critical thinking is not just innate intelligence or a talent—it's a learnable and improvable skill you can master. Michael Bolton shares the specific techniques and heuristics of critical thinking and presents realistic testing puzzles that help you practice...

To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Mike Sowers as he addresses common metrics—measures of product...

Many organizations find that test automation does not work as well as they thought it would. In many cases, these failures are due to generic technical reasons, which can be fixed with relative ease. Solutions that have worked well for others are patterns; these test automation patterns are common to automation efforts at any level with whatever tools you are using. Dot Graham focuses on often-neglected technical issues—i.e., non-management issues—and the patterns that help solve them. These are not development or code patterns—this is a code-free tutorial. Using a set of patterns...

You have just been assigned a new testing project. Where do you start? How do you develop a plan and begin testing? How will you report on your progress? Paul Holland shares new test project approaches that enable you to plan, test, and report effectively. Paul demonstrates ideas, based on the Heuristic Software Test Model from Rapid Software Testing that can be directly applied or adapted to your environment. In this hands-on tutorial, you’ll be given a product to test. Start by creating three raw lists—product coverage outline, potential risks, and test ideas—that help ensure...

In test automation, we must often use several tools that have been developed or acquired over time with little consideration of an overall plan or architecture and no consideration for how to integrate the tools. As a result, productivity suffers and frustrations increase. Join Mike Sowers as he shares experiences from multiple organizations in creating an integrated test automation plan and developing a test automation architecture. Mike discusses both the good (engaging the technical architecture team) and the bad (too much isolation between test automators and test designers) on...

In chess, the word blunder means a very bad move by someone who should know better. Even though functional test automation has been around for a long time, people still make some very bad moves and serious blunders. The most common misconception in automation is thinking that manual testing is the same as automated testing. And this thinking accounts for most of the blunders in system level test automation. Dorothy Graham takes us on a tour of these blunders, including: the Stable-Application Myth (you can’t start automating until the application is stable), Inside-the-Box Thinking (...

Think of manually executed test scripts—like pulling a wagon without wheels. Eventually the wagon will make it to the final destination, but the journey itself will be long and painful. Many people think test scripts are outdated because of the long, painful process of writing and running them. Andrea Fox says that their analytics team shared this way of thinking. To make things worse, the team also was dealing with defects being constantly introduced because the restrictive scripts were not catching the issues. Change was vital to provide more efficiency in manual testing, as well...

Modern software products tend to have a rich UI that supports many user workflows, all of which need to be covered in testing. Agile organizations quickly discover that manual end-to-end testing neither supports their velocity nor provides respectable regression coverage. A common progression is to move from fully manual testing to record/replay, then to Selenium IDE style tests, then to automation based on Selenium WebDriver, perhaps with a BDD overlay. Daria Mehra has practiced this approach and shares her experience. She compares the Selenium style of automation to an...

Automation is a powerful tool to help testing but too often it is used to replicate existing manual tests. This leads organizations to spend large amounts of time and money constantly updating flaky automated tests and test teams to suffer frustration from having to focus on activities that are not truly testing. This cost and frustration can be avoided by using automation as a tool to assist testing—not to replace tests. Jeffrey Martin shares some real-world examples of using automation to supplement testing by leveraging its true value—the replication and repetition of tasks...

Although requirements provide valuable information that informs and shapes testing, sometimes the information provided is incomplete or unclear. Join Julie Lebo as she shares her experience with requirements engineering and how she has integrated her testing group into the requirement process. She believes testers are a valuable asset to the requirement process, and their involvement can improve quality in many ways including finding bugs earlier in the lifecycle, gaining a deeper understanding of their project and product, and writing tests that provide more value to the...

All testers have users with unique needs. Are these needs included in your requirements? Lisa denDekker-Redemann says that was not always the case at UPS. Were we testing the mobile systems that our crew members use like we should? Sometimes to get it right, we have to go out into the wild—or in our case, out into the wild blue yonder. To get the whole story (and awesome test coverage), we boarded an aircraft to design scripts and test live 18,000 feet in the air. We found real conditions our users face that didn’t make the user stories and test conditions. Lisa shares why she went into...

Ensuring a website will scale with excellent performance under peak levels of load is no easy task. Any number of problems can occur—from switch hardware failure to third party service outages, to a poor choice of algorithms or memory use in the code. Melissa Chawla describes Shutterfly's three-tiered approach to prevent site outages during peak load. First, check the development team's designs for scalability by holding performance design reviews for each project including identifying throughput requirements for all down-stream resources. Second, automate continuous load testing...

Traditionally, test management has focused on two areas—test planning and test execution. Test planning creates the test strategy and prepares test cases. Test execution focuses on who is responsible for and assigned to executing the respective test cases and logging defects. These views, however, are not inclusive of everything a tester does. For example, the work of team members must be coordinated, environments made ready, and test data prepared. For this reason, Jim Trentadue says the testing effort should be managed by dependencies—not activities. Jim shares logical models for...

Let's face it. As leaders, we often struggle to create and nurture the sort of communication and conversation that our organizations and teams need. For many of us it's not a comfort zone or a strength. And frankly, these conversations are often uncomfortable and take a great deal of energy. That being said, today's leaders need to face this shortcoming and become much more skilled and comfortable initiating and executing the sorts of conversations that will engage their teams and deliver value for their customers. In this session Bob Galen will explore the central aspects of having...