Concurrent Sessions

Software testing is a highly technical, logical, rational effort. There's no place for squishy emotional stuff here. Not among professional testers. Or is there? Because of commitment, risk, schedule, and money, emotions can run high in software development and testing. It is easy to become frustrated, confused, or bored; angry, impatient, and overwhelmed. However, Michael Bolton says that, if we choose to be aware of our emotions and are open to them, feelings can be a powerful source of information for testers, alerting us to problems in the product and in our approaches to our work. People don't decide things based on the numbers; they decide based on how they feel about the numbers. Our ideas about quality and bugs are rooted in our desires, which in turn are rooted in our feelings. You'll laugh, you'll cry...and you may be surprised as Michael discusses the important role that emotions play in excellent testing.

To create better test cases, Koray Yitmen says you must know your users. And the path to better test case creation in usability testing starts with the segmentation and definition of users, a concept known as personas. Contrary to common market-wise segmentation that focuses on users' demographic information, personas focus on users’ behavioral characteristics, animating them in the minds of designers, developers, and testers. Put these personas “on stage” and let them play their roles in user scenarios. Then, turn these scenarios into use cases and turn use cases into test cases—and you have created better test cases. Koray shares stories from his usability testing projects for multinational clients. Learn how to define personas and scenarios, and convert them into use cases and test cases. Using a few concepts and skills from engineering, psychology, sociology, and art, this is no ordinary test case creation session.

The best software development teams find ways for programmers and testers to work closely together. These teams recognize that programmers and testers each bring their own unique strengths and perspectives to the project. However, working in agile teams requires us to unlearn many of the patterns that traditional development taught us. In this interactive session with Nate Oster, learn how to use the agile practice of concurrent testing to overcome common testing dysfunctions by having programmers and testers work together—rather than against each other—to deliver quality results throughout an iteration. Join Nate and practice concurrent testing with games that demonstrate just how powerfully the wrong approaches can act against your best efforts and how agile techniques can help you escape the cycle of poor quality and late delivery. Bring your other team members to this session and get the full effect of these revealing and inspiring games!

Many cloud-based performance and load testing tools claim to offer “cost-effective, flexible, pay-as-you-go pricing.” However, the reality is often neither cost-effective nor flexible. With many vendors, you will be charged whether or not you use the time (not cost effective), and you must pre-schedule test time (not always when you want and not always flexible). In addition, many roadblocks are thrown up—from locked-down environments that make it impossible to load test anything other than straight-forward applications, to firewall, security, and IP spoofing issues. Join Leslie Segal to discover when it makes sense to set up your own cloud-based performance test lab, either as a stand-alone or as a supplement to your current lab. Learn about the differences in licensing tools, running load generators on virtual machines, the real costs, and data about various cloud providers. Take home a road map for setting up your own performance test lab—in less than twenty-four hours.

A tester’s toolbox today contains a number of test case design techniques—classification trees, pairwise testing, design of experiments-based methods, and combinatorial testing. Each of these methods is supported by automated tools. Tools provide consistency in test case design, which can increase the all-important test coverage in software testing. Cause-effect graphing, another test design technique, is superior from a test coverage perspective, reducing the number of test cases needed to provide excellent coverage. Gary Mogyorodi describes these black box test case design techniques, summarizes the advantages and disadvantages of each technique, and provides a comparison of the features of the tools that support them. Using an example problem, he compares the number of test cases derived and the test coverage obtained using each technique, highlighting the advantages of cause-effect graphing. Join Gary to see what new techniques you might want to add to your toolbox.

In today’s competitive world, more and more HTML5 applications are being developed for mobile and desktop platforms. Spotify has partnered with world-renowned organizations to create high quality apps to enrich the user experience. Testing a single application within a few months can be a challenge. But it's a totally different beast to test multiple world-class music discovery apps every week. Alexander Andelkovic shares insights into the challenges they face coordinating all aspects of app testing to meet their stringent testing requirements. Alexander describes an agile way to use the Kanban process to help out. He shares lessons learned including the need for management of acceptable levels of quality, support, smoke tests, and development guidelines. If you are thinking of starting agile app development or want to streamline your current app development process, Alexander’s experience gives you an excellent starting point.

How can testers spend more time doing productive testing and waste less time preparing "useless" project documentation? Rob Sabourin employs whiteboarding techniques to enable faster, easier, and more powerful communication and collaboration—without all the paperwork. Rob uses whiteboarding to help identify technical risks, understand user needs, and focus testing on what really matters to business stakeholders. Whiteboard block diagrams visualize technical risk to stakeholders. Whiteboard fault models highlight failure modes to developers and testers. Testers can elicit usage scenarios directly from customers using storyboard diagrams. Rob shows how simple whiteboarding strategies help testers learn new concepts, design better tests, and estimate their activities. Rob shares his experiences whiteboarding all kinds of visual models: time sequences, block diagrams, storyboards, state models, control flows, data flows, and mind maps. Save time and avoid the pain of back-and-forth or written document reviews that testers, developers, customers, and users often come to despise.

Many organizations use software testing metrics extensively to determine the status of their projects and whether or not their products are ready to ship. Unfortunately most, if not all, of the metrics in use are so flawed that they are not only useless but possibly dangerous—misleading decision makers, inadvertently encouraging unwanted behavior, or providing overly simplistic summaries out of context. Paul Holland reviews Goodhart’s Law and its applicability to software testing metrics. Paul identifies four characteristics that will enable you to recognize the bad metrics in your organization. Despite showing how the majority of metrics used today are bad, all is not lost as Paul shares the collection of information he has developed that is more effective. Learn how to create status reports that provide details sought after by upper management—and avoid the problems that bad metrics cause.

Your test strategy is the design behind your plan—the set of big-picture ideas that embodies the overarching direction of your test effort. It captures the stakeholders’ values that will inspire, influence, and ultimately drive your testing. It guides your overall decisions about the ways and means of delivering on those values. The weighty test strategy template mandated in many organizations is not conducive to thinking through the important elements of a test strategy and then communicating its essentials to your stakeholders. A lightweight medium like a mindmap is far more flexible and direct. In this interactive session Fiona Charles works with you to develop your own strategic ideas in a mindmap, exploring along the way what really matters in a test strategy and how best to capture it using a mindmap. Fiona shares tips on how to use your mindmap to engage your stakeholders’ interest, understanding, and buy-in to your strategy.

Stakeholders always want to release when they think we’ve “finished testing”. They believe we have revealed “all of the important problems” and “verified all of the fixes,” and now it’s time to reap the rewards. However, as testers we still can assist in improving software by learning about problems after code has rolled “live-to-site”—especially if it’s a website. At eBay we have a post-ship “site quality” mindset in which testers continue to learn from A/B testing, operational issues, customer sentiment analysis, discussion forums, and customer call patterns—just to name a few. Jon Bach explains how and what eBay’s Live Site Quality team learns every day about what they just released to production. Take away some ideas on what you can do to test and improve value—even after you’ve shipped.

During the past ten years, static analysis tools have become a vital part of software development for many organizations. However, the question arises, “Can we quantify the benefits of static analysis?” William Oliver presents the results of a Lawrence Livermore National Laboratory study that first measured the cost of finding software defects using formal testing on a system without static analysis; then, they integrated a static analysis tool into the process and, over a period of time, recalculated the cost of finding software defects. Join William as he shares the results of their study and discusses the value and benefits of static testing. Learn how commercial and open source analysis tools can perform sophisticated source code analysis over large code bases. Take back proof that employing static analysis can not only reduce the time and cost of finding defects and their subsequent debugging but ultimately can reduce the number of defects making their way into your releases.

The software industry knows that the majority of software defects have their root cause in poor requirements. So how can testers help improve requirements? Richard Bender asserts that requirements quality significantly improves when testers systematically validate the requirements as they are developed. Applying scenario-driven reviews ensures that the requirements have the proper focus and scope. Ambiguity reviews quantitatively identify unclear areas of the specification leading to early defect detection and defect avoidance. Modeling the requirements via cause-effect graphing helps find missing requirements and identifies logical inconsistencies. Going further with this approach, domain experts and developers should review the tests derived from the requirements models to find additional defects. Join Richard to learn how testers—applying these processes in partnership with analysts—can reduce the percentage of defects caused by poor requirements from the usual 55–60 percent down to low single digits.

Jamie Mitchell explores an underused and often forgotten test technique—white-box testing. Also known as structural testing, this technique requires some programming expertise and access to the code. Using only black-box testing, you could easily ship a system having tested only 50 percent or less of the code base. Are you comfortable with that? For mission-critical systems, such low test code coverage is clearly insufficient. Although you might believe that the developers have performed sufficient unit and integration testing, how do you know that they have achieved the level of coverage that your project requires? Jamie describes the levels of code coverage that the business and your customers may need—from statement coverage to modified condition/decision coverage. He explains when you should strive to achieve different code coverage target levels and leads you through examples of pseudocode. Even if you have no personal programming experience, understanding structural testing will make you a better tester. So, join Jamie in this code-diving session.