Project Management

Test automation effectiveness can be significantly compromised by over abstraction, improper object recognition, ineffective synchronization, poor data strategies, and more. In fact, Greg Paskal has identified more than a dozen common issues in test automation efforts that can rob you of potential value. Greg shares how your test automation development and maintenance are adversely impacted by these issues. Many automation frameworks are difficult to scale or expensive to maintain, resulting in little to no return on investment (ROI). Learn about the test automation N-Curve effect and how it is directly impacted by ROI robbers, resulting in an unsettling ride on the “ROI Roller Coaster.” Whether you're new to test automation or have years of experience, it's smart to consider what can hinder your results. Learn about the most common mistakes made in test automation and how to avoid—or overcome—these common ROI robbers.

Do you struggle to find the best words to explain testing coverage and status to your stakeholders? Do numbers and metrics make your stakeholders' eyes glaze over? Do you feel dirty giving metrics that you know are going to be abused? Words and numbers are powerful, but Thomas Vaniotis says that good visualizations can amplify their power and effectively communicate to busy or visually-oriented stakeholders. Learn a number of simple visualizations that you—even with few artistic skills—can create inexpensively. Take back visual tools for communicating testing status and product quality, working through test planning and management, discussing risk, and ideas to combine them into compound visualizations. Thomas demonstrates the use of these snappy visuals in real-world contexts and warns of some of their pitfalls. In most cases, the tools won't be special software but regular office supplies like markers, sticky notes, graph paper, walls—and a bit of creativity.

Organizing testing activities under a Test Center of Excellence (TCoE) is a common practice, now seen as a logical step in the growth toward a more mature testing organization. Although it appears straightforward, the efforts of many companies soon plateau and excellence proves elusive. What can be done to reach a level of service delivery that customers will value? Wim Demey describes how the ITIL model can help in the journey to improve your TCoE. Based on a lifecycle model, ITIL is a widely accepted service management framework, consisting of five stages—service strategy, service design, service transition, service operation, and continual service improvement. From the ITIL perspective, you can consider test activities as services that need to be aligned with the needs of the business. Using an example, Wim describes the ITIL lifecycle and explains how its concepts are applied to the TCoE. Discover how one Test Center of Excellence fully integrated ITIL into its existing test organization.

Each day thousands of mobile apps are built, and many are released with poor quality. Pressure mounts exponentially on organizations to test mobile apps with shorter time-to-market cycles. Mobile app testing becomes overwhelming due to multiple platforms, varying OS versions, numerous device manufacturers, wide range of screen resolutions, and more. Parimala Hariprasad presents an approach to designing test strategies for mobile apps. She addresses such questions as: What devices should we test? How do we select them? Can we use simulators/emulators? How do we handle fragmentation challenges? Which platforms are good enough? Parimala shares her experiences and highlights how analytics and user reviews can facilitate the creation of an effective test strategy that evolves over time and balances tradeoffs between cost, quality, and time-to-market in the constantly changing mobile market. Key takeaways include learning about fragmentation, the shotgun approach, mobile personas, and use of analytics to fine-tune the test strategy.

Classical test process improvement models no longer fit in organizations adopting the newest development approaches. Instead, a more flexible approach is required today. Solutions like SOA, virtualization, web technology, cloud computing, mobile, and the application of social media have dramatically changed the IT landscape. In addition, we are innovating the way we develop, test, and manage. Many organizations are moving toward a combination of agile/scrum, context-driven testing, continuous integration and delivery, DevOps, and TestOps. Effective test automation has become a prerequisite for success. All of this requires a different way of improving testing, an adaptable way that responds to innovations in both technology and development. Martin shares a roadmap that enables you to translate the triggers and objectives for test improvement into actions that can be implemented immediately. Learn how to achieve continuous test improvement in any situation, and take away a practical set of guidelines to enable a quick start.

Finding major design issues late in development—and the resulting unpredictability at a project’s end—cause many projects to suffer. The use of graphical test planning (GTP) helps eliminate this unpredictability. GTP is a revolutionary test analysis method that uses behavior modeling to capture the system design, anticipate product bugs before coding, and develop test suites before code is ready. GTP is lightweight, yet covers boundary conditions, class equivalence, domain analysis, and combinations. David Bradley shows how to use your skills and experiences to identify and remove design issues from the beginning. Learn how to capture detailed technical information that elicits input and reviews from many sources, making testing a valuable and integral part of the whole project lifecycle. Discover how GTP provides the flexibility, efficiency, and agility to fit in with today’s development processes. Explore how you can use GTP to successfully test your next project—no matter how challenging.

Before starting an exploratory testing session, you should create a charter that defines the range and the goal of your effort. Because a good charter usually results in a productive testing session, charter design is an important and creative skill in session-based exploratory testing. During her years of coaching, Xiaomei Tai has found that many testers have difficulty creating high-quality charters, so she likes to share the way of using Edward de Bono’s Six Thinking Hats approach in charter design. For example, use the blue hat to generate initial ideas for intake sessions; use the white hat to create collecting-information charters for survey sessions; use the black hat and the green hat to get some negative or creative testing charters for deep coverage sessions; and so on. In this session you will practice using the six thinking hats in an interesting exercise, and then compare and discuss the designs of the other delegates. Take away the benefits of applying the six thinking hats in charter design.

Testing enterprise software requires effective resource management to prevent costly delays, budget overruns, and failed projects. In many software projects, more than 50 percent of development costs are attributed to software testing activities. With testing accounting for such a large portion of development efforts, it is critical for software engineering teams to avoid and eliminate wasteful testing tasks. Tariq King applies lean and agile principles to test management as a way of reducing waste in the testing process. Join Tariq as he describes an integrated approach for identifying and eliminating waste in test planning, automation, and execution. He demonstrates how to combine tools and techniques for test case management, exploratory testing, test automation, continuous integration, and impact analysis to keep testing activities lean and lightweight. Learn how to avoid common sources of waste such as over-documentation, redundant tests, brittle automation, and testing unimportant or unaffected parts of the system.

Test automation is not an exact science. As we drive toward an automation solution that validates accurately, the application or data undergoes changes, making it challenging to maintain the tests. The test automation professional must plan to make each test as predictive as possible. Jim Trentadue introduces a framework called S.M.A.R.T.E.R that can be used to accomplish this. The acronym is defined as: Strategy for understanding application behavior and variations, Methodology for implementing the logical model, Adaptable across different platforms or browsers, Roadmap for outlining the sequence and priority of those tests to address first, Toolbox to work with technologies available, Experienced personnel requirements for those to support such changes, and a Repeatable design for future success. By implementing this framework, your automation will be structured for ongoing success by making your test cases SMARTER!

Testing is on the schedule. Your title is Tester. It is time to test. The team is waiting for you. Everybody thinks you spend your time testing. So, why does it seem that you spend so little time actually testing? Michael Bolton and James Bach will show you. They have developed a training and research tool to visually animate the progress of testing. Through the use of testopsies and session-based test management data, Michael and James have collected patterns of how context-driven testers use their time—from the micro to the macro level.

Testing a PC-based game poses interesting challenges. With different OS platforms and many different GPUs, the number of combinations to test grows quickly. The large number of hardware configurations used by the players of EVE Online makes it impossible to test them all. This can lead to the situation where reproducing defects in the test lab is not possible. Björgvin Reynisson shares how CCP Games got fed up with the situation and decided to make a crowd-testing app that tests the game’s graphics engine. He describes the app, the framework for deploying the app, and how they handled the test results coming in from testers. Björgvin shares how the app has become both a development tool and an automated testing tool. Discover how this approach has helped identify hard-to-catch bugs, and discover some of the challenges of crowd testing. See how this project took an unexpected turn and led the team to change how the game content is distributed.tes