STARWEST 2017 - Software Tester

Critical thinking is the kind of thinking that specifically looks for problems and mistakes. Regular people don't do a lot of it. However, if you want to be a great tester, you need to be a great critical thinker. Critically-thinking testers save projects from dangerous assumptions and ultimately from disasters. The good news is that critical thinking is not just innate intelligence or a talent—it's a learnable and improvable skill you can master. Michael Bolton shares the specific techniques and heuristics of critical thinking and presents realistic testing puzzles that help you practice...

When we discover—often late in an automation effort—that the automated tests are cumbersome and costly to maintain, we often view this as a technical problem for the automator to solve. However, an often-overlooked cause is the role that testers who designed these tests play in making automation scalable and maintainable. In this interactive tutorial for both testers and automation engineers, Hans Buwalda explores how better test designs not only will result in much improved test automation but also can make the difference between automation success and failure. See why successful...

We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. But Mary Thorn says help is at hand. Enter behavior-driven development (BDD) and Cucumber and SpecFlow, two tools for running automated acceptance tests and facilitating BDD. Mary explores the nuances of Cucumber and SpecFlow, and shows you how to...

To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Mike Sowers as he addresses common metrics—measures of product...

Influence diagrams provide a simple-to-create and easy-to-understand approach to address the complexities of real-life problems. As testers, we may want to find more bugs, but this may have an unintended consequence for developers. Developers now have more defects to debug, which affects their capacity to deliver new functionality. Isabel Evans has found that influence diagrams provide a means of understanding and managing the complexities of key interactions among testers, developers, and business stakeholders. In the past few years, Isabel has used influence diagrams as a tool to...

With Git and GitHub, testers today have unprecedented visibility into both development and DevOps code. GitHub provides powerful online collaboration, code review, code management, and version control services. GitHub's domination of social coding makes it the professionals’ new business card, indicating their creativity, popularity, capability, and tenacity. Today’s testers need to learn about and use Git and GitHub. In this hands-on tutorial Wilson Mar presents material available nowhere else and a unique explanation of tools, repositories, and GUI programs used by professional...

No longer just a futuristic concept, the Internet of Things (IoT) has a strong presence in our world even today. If your business is not prepared for it, you’re already behind. With the proliferation of connected “things”—devices, appliances, cars, and even clothes—Jennifer Bonine says that the stage is set. IoT apps are here to stay. Testing, product management, and development teams must address developing and testing in this paradigm. Testers, accustomed to traditional platforms, are now asked to test on more complex devices and more advanced platforms. Testers must keep up with the...

Many organizations find that test automation does not work as well as they thought it would. In many cases, these failures are due to generic technical reasons, which can be fixed with relative ease. Solutions that have worked well for others are patterns; these test automation patterns are common to automation efforts at any level with whatever tools you are using. Dot Graham focuses on often-neglected technical issues—i.e., non-management issues—and the patterns that help solve them. These are not development or code patterns—this is a code-free tutorial. Using a set of patterns...

You have just been assigned a new testing project. Where do you start? How do you develop a plan and begin testing? How will you report on your progress? Paul Holland shares new test project approaches that enable you to plan, test, and report effectively. Paul demonstrates ideas, based on the Heuristic Software Test Model from Rapid Software Testing that can be directly applied or adapted to your environment. In this hands-on tutorial, you’ll be given a product to test. Start by creating three raw lists—product coverage outline, potential risks, and test ideas—that help ensure...

Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking,...

In many organizations, agile development processes are driving the pursuit of faster software releases, which has spawned a set of new practices called DevOps. DevOps stresses communications and integration between development and operations, including rapid deployment, continuous integration, and continuous delivery. Because DevOps practices require confidence that changes made to the code base will function as expected, automated testing is essential. Join Jeffery Payne as he discusses the unique challenges associated with integrating automated testing into continuous integration/...

Are you a tester, developer, or quality engineer who participates in code reviews or unit testing activities? Can you write test scripts to cover all program statements but would like to learn more advanced code-based testing techniques? Join Tariq King as he goes beyond the basics of unit testing and shows you how to design tests that cover multiple data conditions, independent paths, and loops within code. Learn how to apply these techniques to different application tiers, ranging from the user interface to the data access layer. Discover strategies for integration testing based on...

In today’s environment, the user experience (UX) is overwhelmingly important—and is not just about the product. UX describes all facets of a person’s interactions with and reactions to the product, the organization that supplies it, and the environment in which it is experienced. Isabel Evans says that in order to focus our tests appropriately, it is vital that we testers understand our users’ experiences. We need to explore and measure human, business, and societal impacts of products we develop, and how those are underpinned by technical qualities. Unless we “shift left” as...

The DevOps movement is here. Now, companies with mature systems are breaking down siloed IT departments and federating them into product development teams and departments. Testing and its practices are at the heart of these changes. Traditionally, development organizations have been filled with mostly manual testers and a limited number of automation and performance engineers. Adam Auerbach says this has to change. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay...

Mike Cohn’s Test Pyramid describes a test automation strategy consisting of a wide base of unit tests, service-oriented acceptance tests for business logic, and a thin layer of tests exercising the user interface. Tests that provide the quickest feedback and fault precision serve as the testing foundation. So, how does this work in practice? How does a team achieve this level of test automation and maintain it over time? How can the team avoid redundancy in the various test layers? Jim Weaver demonstrates the different types of tests for a real feature—enforcing business rules for...

Substantial confusion exists about the roles and responsibilities of test managers in an agile software development process. Agile seeks to streamline project management and leadership under the role of a ScrumMaster. So, what does this mean for test managers? How do they stay involved in the process? What role do they fill? Is it possible that test managers are no longer needed? Join Jeffery Payne for a collaborative dialog on the challenges test managers face in an agile model. Learn the pros and cons of a variety of test management models he has seen used by organizations that have...

Think of manually executed test scripts—like pulling a wagon without wheels. Eventually the wagon will make it to the final destination, but the journey itself will be long and painful. Many people think test scripts are outdated because of the long, painful process of writing and running them. Andrea Fox says that their analytics team shared this way of thinking. To make things worse, the team also was dealing with defects being constantly introduced because the restrictive scripts were not catching the issues. Change was vital to provide more efficiency in manual testing, as well...

Are you being asked to shorten your testing timelines? Do you feel pressured to increase your test automation coverage but don’t have the time, staff, or budget? How do you as a leader upgrade your existing teams’ programming skills and technical abilities without bringing in external resources—and still meet your daily release deliverables? Join Jennifer Scandariato as she shares her journey in transforming the QA department at iCIMS into a Test Engineering Center of Excellence, where manual testers are now automation engineers who apply appropriate automation technologies to...

Creating automated tests for your team stories, integration, or regression test cycles within agile sprints is almost every tester’s top challenge. Usually it consumes many hours and requires a great deal of effort to achieve, especially in complex and large agile projects. Teams need to deliver software as quickly as they can while producing the best possible product quality. Talal Ibdah shows how you can achieve these goals and automate your API functional and performance tests; define test environments and configuration files; make chaining requests; continuously deliver your...

Although most software companies have adopted agile development, many still treat quality assurance as something that gets handled when coding is finished and ready for test. However, practicing this reactive approach to quality costs teams in rework, context switching, slower code release cycles, growing bug queues, and the release of defects into production. Join Oscar Gracia and Todd Albers as they share techniques you can use to help change this “ready for test” mindset. Learn how to focus on testing and quality from the start by using a pre-grooming approach to ensure stories...

Test automation, one of several key technical enablement practices, allows teams to be more successful in their agile journeys. Although there are many test practices and automation tools available for software development teams to leverage, few data-centric testing tools are targeted to data-related development and testing, leaving data warehousing and business intelligence teams thinking they can't possibly automate their tests. Cher Fox explores why test automation is important to agile data teams, discusses why they aren’t automating their tests today, and investigates the path to test...

You may have heard the term “continuous testing” and thought it was just the DevOps flavor of the month … or that it isn’t part of DevOps … or that it isn’t for cloud-based applications. Marianne Hollier says that continuous testing means adopting the right set of automated tests along with service virtualization, which allows the team to simulate missing dependencies and to start testing earlier and more frequently. She shares how the right combination of best practices and tools can help software development and testing teams adopt a continuous testing approach. Since you can’t test...

Sabermetrics turned the baseball world upside down by challenging decades-old measures of individual performance and their perceived linkage to team success. After cementing their legacy as the Lovable Losers for 108 years, the Chicago Cubs were able to leverage a data-driven approach to finally win a World Series. A high-school football coach, devoted to statistical analysis, has won three state championships—by never punting. Formula 1 racing teams collect staggering amounts of telemetry data from their race cars for the purpose of eking out seconds during the course of a race....

Automation is a powerful tool to help testing but too often it is used to replicate existing manual tests. This leads organizations to spend large amounts of time and money constantly updating flaky automated tests and test teams to suffer frustration from having to focus on activities that are not truly testing. This cost and frustration can be avoided by using automation as a tool to assist testing—not to replace tests. Jeffrey Martin shares some real-world examples of using automation to supplement testing by leveraging its true value—the replication and repetition of tasks...

Although requirements provide valuable information that informs and shapes testing, sometimes the information provided is incomplete or unclear. Join Julie Lebo as she shares her experience with requirements engineering and how she has integrated her testing group into the requirement process. She believes testers are a valuable asset to the requirement process, and their involvement can improve quality in many ways including finding bugs earlier in the lifecycle, gaining a deeper understanding of their project and product, and writing tests that provide more value to the...

The concept of “shifting testing left” in the software development lifecycle is not new. Shifting testing from manual to automated and then upstream into engineering is a driving factor in DevOps and agile software development. However, Michael Nauman wonders why test automation, DevOps, and agile software development still frequently fail to deliver on their promises? Aligning and hardening your DevOps and test automation—along with streamlining your agile processes—is critical to your project. Michael shares how AutoCAD’s shifting testing left enabled improvements within their...

Selenium has an industry reputation of being a “flaky” tool where individual tests pass, then fail—sometimes with no production changes at all. Such flakiness in your test suites can be extremely difficult, time consuming, and frustrating to debug. The vast majority of these issues stem from using either bad locators or bad wait conditions. But both of these root causes can be addressed by implementing the right framework for your Selenium tests. Craig Schwarzwald shares the most important concepts in creating such a Selenium framework. He has proved that using a framework...

Communication breakdowns are a primary cause of IT project failure. Marcia Buzzella believes increasing the success rate of IT projects across waterfall, Agile, and DevOps methods requires a balance of social (soft) and technical (hard) capabilities to improve team performance. Social interactions among team members facilitate knowledge sharing, build relationships, promote trust, and—perhaps most importantly—align expectations. Unfortunately, the fast pace of technological change prompts many software test professionals to prioritize the improvement of technical capabilities over...

Competition is driving our business and IT partners to be ever more nimble. And Byron Glick and Jithesh Ramachandran say that the growing agility is eroding the old foundations of testing—test management, project governance, and centers of excellence. An organization pursuing lean startup approaches may reduce or eliminate traditional planning cycles and the related test management and governance. But all is not lost. Understanding why those foundations were effective in their time points to new foundations that will carry us forward into the new world of testing, technology, and...

Walk the Expo, and you will see all kinds of test automation tools. Some run scripts. Some communicate with the system under test. Some virtualize system components. Some do interesting things that you may never have considered. Yet, none gives you a complete recipe for testing your product and synthesizing the results. That is not their job. It's yours. Mike Duskis says an effective test automation program will reflect the unique nuances of your product and your business. However, unique nuances need not add up to radically different architectures. In fact, effective automation...

When people hear about my past jobs, my career, and the many places I “work” (at foreign conferences and even on ski lifts), I often get the question “How do I get your job?” However, when people hear some of the details of my career, their reaction is “Gee, that’s a lot of work!” Well, yes, but that does not mean that hard work is not fun. Jon Hagar presents a career map that contains his secrets to getting, keeping, and growing in the fun and exciting career of software testing. These include getting started with learning, seeking challenges, sustaining passion, finding...

Traditionally, test management has focused on two areas—test planning and test execution. Test planning creates the test strategy and prepares test cases. Test execution focuses on who is responsible for and assigned to executing the respective test cases and logging defects. These views, however, are not inclusive of everything a tester does. For example, the work of team members must be coordinated, environments made ready, and test data prepared. For this reason, Jim Trentadue says the testing effort should be managed by dependencies—not activities. Jim shares logical models for...

Is your company spending a lot of time and effort on an automation strategy while your customers believe that product quality has not improved? Does management see automation as a silver bullet that will save money, increase coverage, and reduce headcount? Do you work for a company where the goal is (almost) 100 percent test automation? Paul Holland discusses issues and problems with these approaches and perceptions about test automation. He provides strong arguments why the “automate everything” approach is not likely to be successful and provides details of an alternative, balanced...

To be a well-rounded and analytical tester, Adam Satterfield says you should learn to code. Learning to code empowers a tester to be more self-reliant and less dependent on someone else to write their test scripts, which can take valuable time from the critical and time-crunched testing process. Learning to code positions a tester for long-term career growth, opens up new professional opportunities, and gives them the perspective to be a better tester with a deeper understanding of what questions to ask and how to approach a meaningful testing plan to gain better insights. However...

The key to creating high-performing teams is psychological safety—the ability to be vulnerable in front of others even when they hold diverse viewpoints, and the opportunity to take risks and trust that everything will be OK. However, creating this safety is easier said than done. Maaret Pyhäjärvi shares her story of working with software development and test teams to enable them to be awesome. She explains how to reinforce the positive while enabling great software product development by empowering others in your team. Maaret explores how to be brave when others are not, and how to care...