Test Management

To improve as a skilled tester, you don't necessarily have to attend a class or read a book. James and Jon Bach show how you can grow by recording and then dissecting as few as five minutes of your test process. Using a Testopsy, you build your skills of observation, narration, and test framing. And if you do it with a colleague or as a group, it stimulates discussion on test design and test strategy. James and Jon demonstrate a live Testopsy to underscore these points and give you specific guidelines for conducting your own. Testing is a performance—not an artifact. Much like a medical examiner narrates his autopsies into a tape recorder, you can look very carefully at what you actually do and identify your own heuristics. By putting that process into descriptive, evocative words, you can discover surprising depths in each act of testing you perform.

Many development and test organizations must work within the confines of compressed release cycles, various agile methodologies, and cloud and mobile environments for their business applications. So, how can test organizations keep up with the pace of development and increase the quality of their applications under test? Clint Sprauve and Todd DeCapua describe how service virtualization and network virtualization can help your team improve speed and increase quality. Learn how to use service virtualization to simulate third-party or internal web services to remove wait times and reduce the need for high-cost testing infrastructures. Take back techniques to incorporate network virtualization into the testing environment to simulate real-world network conditions. Learn from Clint and Todd how the combination of service and network virtualization allows teams to implement a robust and consistent continuous testing strategy to reduce defects in production applications.

Classic testing methods and tools can’t keep pace with agile development practices and emerging continuous delivery models. Melvin Laguren describes how Macy's Ecommerce Merchant System uses a variety of tools to support its rapid delivery pipeline. A code coverage tool such as Karma determines if they need additional tests or if the existing tests provide satisfactory coverage. Using data analysis tools such as Code Maat with source code control, they have a robust approach to improve the code review process and easily identify necessary file changes to prevent issues in production. By incorporating these two tools, Melvin’s team members get real-time informative feedback that helps them make better testing decisions within their continuous delivery pipeline. Testers, developers, and all stakeholders have easy access to a dashboard reporting tool that showcases findings from unit, functional, and performance testing. Join Melvin to explore new tools and practices you can implement to smooth the road to continuous delivery in your systems.

“How absurd! She swallowed a bird. She swallowed the bird to catch the spider. She swallowed the spider to catch the fly. I don't know why she swallowed the fly. Perhaps she'll die.” The silly nursery rhyme teaches a serious lesson. Because software products are complex, we seek to manage them by spinning a complex web of processes and tools. Thankfully, not all complex problems demand complex solutions. Join Mike Duskis as he demonstrates how his test team employed kanban practices to manage the test work of a multi-national, multi-project department with a system of index cards on the wall. With the kanban system in place and lean practices driving decisions, the team simplified the prioritization process, improved test visibility, which led to better testing choices. Lean-kanban produced strong results at Mike's shop and could do the same for yours. Join Mike to learn how you can clarify and communicate your test effort—without swallowing any spiders.

Organizing testing activities under a Test Center of Excellence (TCoE) is a common practice, now seen as a logical step in the growth toward a more mature testing organization. Although it appears straightforward, the efforts of many companies soon plateau and excellence proves elusive. What can be done to reach a level of service delivery that customers will value? Wim Demey describes how the ITIL model can help in the journey to improve your TCoE. Based on a lifecycle model, ITIL is a widely accepted service management framework, consisting of five stages—service strategy, service design, service transition, service operation, and continual service improvement. From the ITIL perspective, you can consider test activities as services that need to be aligned with the needs of the business. Using an example, Wim describes the ITIL lifecycle and explains how its concepts are applied to the TCoE. Discover how one Test Center of Excellence fully integrated ITIL into its existing test organization.

At one time or another, every tester hears the dreaded question, “Why didn’t you guys catch these bugs?” We all have some standard responses, and they are most likely true). But what can we learn about our testing when we look beyond the easy answers? Pamela Gillaspie proposes that the key to improving your testing is determining the areas where bugs are slipping past your defenses. For her team, the practice is a lot like basketball. If you group the bugs into zones, you can devise a strategy to cover those zones more effectively. Some zones need a different testing approach than you’ve used; others might reveal a need for closer communication. Join Pamela as she shares her experience as defensive coordinator, addressing the developers’ playbook (What kinds of recurring problems do we see?), trick plays (The user is doing what?), and penalties (That wasn’t in the requirements!).

Finding major design issues late in development—and the resulting unpredictability at a project’s end—cause many projects to suffer. The use of graphical test planning (GTP) helps eliminate this unpredictability. GTP is a revolutionary test analysis method that uses behavior modeling to capture the system design, anticipate product bugs before coding, and develop test suites before code is ready. GTP is lightweight, yet covers boundary conditions, class equivalence, domain analysis, and combinations. David Bradley shows how to use your skills and experiences to identify and remove design issues from the beginning. Learn how to capture detailed technical information that elicits input and reviews from many sources, making testing a valuable and integral part of the whole project lifecycle. Discover how GTP provides the flexibility, efficiency, and agility to fit in with today’s development processes. Explore how you can use GTP to successfully test your next project—no matter how challenging.

Before starting an exploratory testing session, you should create a charter that defines the range and the goal of your effort. Because a good charter usually results in a productive testing session, charter design is an important and creative skill in session-based exploratory testing. During her years of coaching, Xiaomei Tai has found that many testers have difficulty creating high-quality charters, so she likes to share the way of using Edward de Bono’s Six Thinking Hats approach in charter design. For example, use the blue hat to generate initial ideas for intake sessions; use the white hat to create collecting-information charters for survey sessions; use the black hat and the green hat to get some negative or creative testing charters for deep coverage sessions; and so on. In this session you will practice using the six thinking hats in an interesting exercise, and then compare and discuss the designs of the other delegates. Take away the benefits of applying the six thinking hats in charter design.

Testing enterprise software requires effective resource management to prevent costly delays, budget overruns, and failed projects. In many software projects, more than 50 percent of development costs are attributed to software testing activities. With testing accounting for such a large portion of development efforts, it is critical for software engineering teams to avoid and eliminate wasteful testing tasks. Tariq King applies lean and agile principles to test management as a way of reducing waste in the testing process. Join Tariq as he describes an integrated approach for identifying and eliminating waste in test planning, automation, and execution. He demonstrates how to combine tools and techniques for test case management, exploratory testing, test automation, continuous integration, and impact analysis to keep testing activities lean and lightweight. Learn how to avoid common sources of waste such as over-documentation, redundant tests, brittle automation, and testing unimportant or unaffected parts of the system.

Test automation is not an exact science. As we drive toward an automation solution that validates accurately, the application or data undergoes changes, making it challenging to maintain the tests. The test automation professional must plan to make each test as predictive as possible. Jim Trentadue introduces a framework called S.M.A.R.T.E.R that can be used to accomplish this. The acronym is defined as: Strategy for understanding application behavior and variations, Methodology for implementing the logical model, Adaptable across different platforms or browsers, Roadmap for outlining the sequence and priority of those tests to address first, Toolbox to work with technologies available, Experienced personnel requirements for those to support such changes, and a Repeatable design for future success. By implementing this framework, your automation will be structured for ongoing success by making your test cases SMARTER!

Testing a PC-based game poses interesting challenges. With different OS platforms and many different GPUs, the number of combinations to test grows quickly. The large number of hardware configurations used by the players of EVE Online makes it impossible to test them all. This can lead to the situation where reproducing defects in the test lab is not possible. Björgvin Reynisson shares how CCP Games got fed up with the situation and decided to make a crowd-testing app that tests the game’s graphics engine. He describes the app, the framework for deploying the app, and how they handled the test results coming in from testers. Björgvin shares how the app has become both a development tool and an automated testing tool. Discover how this approach has helped identify hard-to-catch bugs, and discover some of the challenges of crowd testing. See how this project took an unexpected turn and led the team to change how the game content is distributed.tes

Mobile users, driven by their actions as consumers, have high expectations. They also expect to be able to use their own mobile devices for work. All this change brings uncertainty. Nearly half of enterprises have concerns with IT’s ability to keep up with the rapid pace of mobile. Success depends on your enterprise’s ability to fully embrace and leverage these changes for optimal user experiences, making functional and performance testing critical to your strategy. Ron Anderson shares practical guidance on how to create testing processes that keep the user top of mind, fueled by best practices within an enterprise mobile environment. He presents several use cases to serve as the backdrop, giving you practical insight into the methods presented. Leave with significantly increased clarity on how to understand and address user expectations, plan for functional testing mapped to user characteristics, and develop strategies to ensure responsiveness.