Concurrent Sessions

Test automation effectiveness can be significantly compromised by over abstraction, improper object recognition, ineffective synchronization, poor data strategies, and more. In fact, Greg Paskal has identified more than a dozen common issues in test automation efforts that can rob you of potential value. Greg shares how your test automation development and maintenance are adversely impacted by these issues. Many automation frameworks are difficult to scale or expensive to maintain, resulting in little to no return on investment (ROI). Learn about the test automation N-Curve effect and how it is directly impacted by ROI robbers, resulting in an unsettling ride on the “ROI Roller Coaster.” Whether you're new to test automation or have years of experience, it's smart to consider what can hinder your results. Learn about the most common mistakes made in test automation and how to avoid—or overcome—these common ROI robbers.

The promise of DevOps is that we can push new ideas out to market faster while avoiding delivering serious defects into production. Andreas Grabner explains that testers are no longer measured by the number of defect reports they enter, nor are developers measured by the lines of code they write. As a team, you are measured by how fast you can deploy high quality functionality to the end user. Achieving this goal requires testers to increase their skills. It’s all about finding solutions—not just problems. Testers must transition from reporting “app crashes” to providing details such as “memory leak caused by bad cache implementation.” Instead of reporting “it’s slow,” testers must discover “wrong hibernate configuration causes too much traffic from the database.” Using three real-life examples, Andreas illustrates what it takes for testing teams to become part of the DevOps transformation—bringing more value to the entire organization.

Do you struggle to find the best words to explain testing coverage and status to your stakeholders? Do numbers and metrics make your stakeholders' eyes glaze over? Do you feel dirty giving metrics that you know are going to be abused? Words and numbers are powerful, but Thomas Vaniotis says that good visualizations can amplify their power and effectively communicate to busy or visually-oriented stakeholders. Learn a number of simple visualizations that you—even with few artistic skills—can create inexpensively. Take back visual tools for communicating testing status and product quality, working through test planning and management, discussing risk, and ideas to combine them into compound visualizations. Thomas demonstrates the use of these snappy visuals in real-world contexts and warns of some of their pitfalls. In most cases, the tools won't be special software but regular office supplies like markers, sticky notes, graph paper, walls—and a bit of creativity.

Can your test automation actually find defects? Are there situations where your software will fail, but your automation will still report “pass”? How can you be certain that your automation does what it’s supposed to do? Jay Kremer asked himself all these questions when taking over as test manager for Zoosk, an international dating site with 27 million monthly customers. In searching for answers to these questions, Jay examined several different approaches and landed on chaos testing, the Netflix-developed approach that involves intentionally breaking the service to simulate failure scenarios. Learn how Jay adapted chaos testing for pre-production environments, identified and fixed faults in his own automation, and demonstrated the success of Zoosk’s automation solution to his management and engineering as a whole. If you want to prove to yourself and your team how reliable your automation is—and you are always looking for ways to respond faster—then this session is for you.

Classic testing methods and tools can’t keep pace with agile development practices and emerging continuous delivery models. Melvin Laguren describes how Macy's Ecommerce Merchant System uses a variety of tools to support its rapid delivery pipeline. A code coverage tool such as Karma determines if they need additional tests or if the existing tests provide satisfactory coverage. Using data analysis tools such as Code Maat with source code control, they have a robust approach to improve the code review process and easily identify necessary file changes to prevent issues in production. By incorporating these two tools, Melvin’s team members get real-time informative feedback that helps them make better testing decisions within their continuous delivery pipeline. Testers, developers, and all stakeholders have easy access to a dashboard reporting tool that showcases findings from unit, functional, and performance testing. Join Melvin to explore new tools and practices you can implement to smooth the road to continuous delivery in your systems.

“How absurd! She swallowed a bird. She swallowed the bird to catch the spider. She swallowed the spider to catch the fly. I don't know why she swallowed the fly. Perhaps she'll die.” The silly nursery rhyme teaches a serious lesson. Because software products are complex, we seek to manage them by spinning a complex web of processes and tools. Thankfully, not all complex problems demand complex solutions. Join Mike Duskis as he demonstrates how his test team employed kanban practices to manage the test work of a multi-national, multi-project department with a system of index cards on the wall. With the kanban system in place and lean practices driving decisions, the team simplified the prioritization process, improved test visibility, which led to better testing choices. Lean-kanban produced strong results at Mike's shop and could do the same for yours. Join Mike to learn how you can clarify and communicate your test effort—without swallowing any spiders.

You know you need to automate your tests. You’ve attended countless sessions and witnessed many talented experts explaining how they’ve built their custom test automation frameworks. You’ve thought, “I wish one of those experts would just give me their framework so I could get down to the business of writing automated tests.” If you’re responsible for creating diverse, scalable automated tests but don’t have the time, budget, or a skilled-enough team to create yet another custom test automation framework, then Bryan Lamb says you need to know about the Robot Framework. He reveals how his team uses this powerful, free, open source, generic framework to create continuous enterprise automated regression tests. Join Bryan to get a look at Robot Framework—where to find, how to install it, how to create and run test cases locally or via a browser cloud, how to use pre-built or custom libraries, and even how to integrate it into a Jenkins build and deployment pipeline.

Capital One has a highly integrated environment, which creates many dependencies for its agile teams. Because these dependencies are often not completed until late in their sprints, Capital One faced prolonged integration and regression testing phases, and did not realize the expected improvements in quality or time to market. As technology leaders pushed for continuous delivery, testing needed to “shift left” and execute test in real time concurrently with development. Adam Auerbach shares Capital One’s experience implementing continuous testing. He explains the core principles of continuous testing, service virtualization, and the continuous integration/continuous delivery pipeline—and why testers need to understand and leverage these important concepts. Adam believes that testers need to learn basic development skills, including Ruby and Java, so they can take advantage of advanced automation practices. Because continuous testing is not easy and many companies have large populations of manual testers, Adam will provide a learning map to help you plan your personal and team’s transition.

Many organizations spend a great deal of time and effort acquiring and learning to use the latest techniques and technology, but they make little or no attempt to train or mentor their staff to be better leaders. It is true that technology is important, but test teams without able leaders will struggle to be successful. Rick Craig shares some lessons he has learned in his roles as test manager, military leader, and entrepreneur. Rick discusses some classic leadership topics―leadership traits and styles, the cornerstones of leadership, and principles of leadership. Explore the importance of influence leaders and how to identify and encourage them. Discover the positive and negative indicators of morale and how to maintain high morale within a team. Learn how to give direction without being a micromanager. Discuss what motivates and what de-motivates testers. Rick encourages you to bring your leadership challenges to serve as points of discussion.

Already using Selenium but have some unanswered questions? Want to learn how to use Selenium like a pro? Join Andrew Krug as he shares the best and most useful tips and tricks. Topics covered include headless test execution, testing HTTP status codes, blacklisting third-party content with a proxy server, repurposing your Selenium scripts to build an initial load testing suite, various ways to perform broken image checking, testing “forgot password” end-to-end, working with A/B testing (most notably, how to opt-out of it), testing file downloads (both the easy way and the hard way), how to add robust debugging output to your tests, and adding visual testing to your existing Selenium tests. If you're already using Selenium and looking for a way to take your automated testing practice to the next level, then this session is for you.

It is quite possible that you are spending a considerable amount of your time as a QA manager making sense of the multitude of metrics reported by your teams, connecting the facts, understanding the underlying reality, and articulating it to your peers and leadership. Still, others in the organization may not interpret the message correctly, rendering most of your efforts futile. Nirav Patel and Sutharson Veeravalli share insights to help you resolve this challenge through a composite measure called Quality Index. By aligning metrics to business outcomes and using Quality Index as a tool of articulation, disparate interpretation of data can be eliminated and a cohesive message delivered to stakeholders. Learn how QA can acquire a voice across the senior forums by articulating succinct, contextual, and actionable information to speed up executive decisions in the course of programs and projects.

Throughout her career, Iris Trout has uncovered gaps in testing that prevent organizations from experiencing high efficiency and high quality test results. Join Iris as she shares her journey to improve QA practices. Learn what QA transformation means and discover how to set realistic timelines, prioritize what you need to work on, and measure and report on what you are improving. Exploring testing artifacts, metrics, reporting, outsourcing, roles, organization change management, and several others areas where most QA organizations fall short, Iris offers practical and easy-to-implement solutions for building a strong QA organization. She shares helpful information on how to engage a vendor in your transformation journey and how to make the vendor relationship strong and collaborative. With her passion and candor for quality delivery processes, Iris provides successful solutions that she has implemented several times over.

Many browser, UI, and Java-based tools and frameworks can help you test REST services. However, in the world of continuous integration and delivery, manual UI- or browser-based tools typically fall short in many aspects—from early test development to developer support. When using Java-based libraries such as HttpClient, much code has to be written for all aspects of a web service call. These extensions or wrappers tend to be complex, hard to read, and difficult to maintain. This is where RestAssured comes in. RestAssured is an open source Java DSL for testing REST-based services, making test code more readable, easier to write, and cheaper to maintain. Learn how easily you can write HTTP get and post requests as well as more complex scenarios involving session management, authentication, and (de)serialization of objects. Take back good practices and an open source command line tool that can help you jumpstart your RestAssured testing.

Before starting an exploratory testing session, you should create a charter that defines the range and the goal of your effort. Because a good charter usually results in a productive testing session, charter design is an important and creative skill in session-based exploratory testing. During her years of coaching, Xiaomei Tai has found that many testers have difficulty creating high-quality charters, so she likes to share the way of using Edward de Bono’s Six Thinking Hats approach in charter design. For example, use the blue hat to generate initial ideas for intake sessions; use the white hat to create collecting-information charters for survey sessions; use the black hat and the green hat to get some negative or creative testing charters for deep coverage sessions; and so on. In this session you will practice using the six thinking hats in an interesting exercise, and then compare and discuss the designs of the other delegates. Take away the benefits of applying the six thinking hats in charter design.

Testing enterprise software requires effective resource management to prevent costly delays, budget overruns, and failed projects. In many software projects, more than 50 percent of development costs are attributed to software testing activities. With testing accounting for such a large portion of development efforts, it is critical for software engineering teams to avoid and eliminate wasteful testing tasks. Tariq King applies lean and agile principles to test management as a way of reducing waste in the testing process. Join Tariq as he describes an integrated approach for identifying and eliminating waste in test planning, automation, and execution. He demonstrates how to combine tools and techniques for test case management, exploratory testing, test automation, continuous integration, and impact analysis to keep testing activities lean and lightweight. Learn how to avoid common sources of waste such as over-documentation, redundant tests, brittle automation, and testing unimportant or unaffected parts of the system.

Test automation is not an exact science. As we drive toward an automation solution that validates accurately, the application or data undergoes changes, making it challenging to maintain the tests. The test automation professional must plan to make each test as predictive as possible. Jim Trentadue introduces a framework called S.M.A.R.T.E.R that can be used to accomplish this. The acronym is defined as: Strategy for understanding application behavior and variations, Methodology for implementing the logical model, Adaptable across different platforms or browsers, Roadmap for outlining the sequence and priority of those tests to address first, Toolbox to work with technologies available, Experienced personnel requirements for those to support such changes, and a Repeatable design for future success. By implementing this framework, your automation will be structured for ongoing success by making your test cases SMARTER!

Testing is on the schedule. Your title is Tester. It is time to test. The team is waiting for you. Everybody thinks you spend your time testing. So, why does it seem that you spend so little time actually testing? Michael Bolton and James Bach will show you. They have developed a training and research tool to visually animate the progress of testing. Through the use of testopsies and session-based test management data, Michael and James have collected patterns of how context-driven testers use their time—from the micro to the macro level.

When done right, testing is more than test plans, test scripts, and executing tests. In fact a test leader should consider testing a sub-project of the larger development project. By applying the same techniques project managers use to plan and manage the overall project, test leaders can improve testing and greatly influence the entire project’s success. Ricki Henry explores project management processes that test leaders need to master—risk management, human resources, stakeholder communications, and scope management. Even though you understand that the scope of testing cannot be “everything tested with zero defects,” the customer does not have this same understanding. To prevent this disconnect, test leaders need to determine the scope of what can be tested and then articulate that to the stakeholders. Join Ricki to learn new ways to improve testing while contributing to overall project success through project management processes that test leaders need to master.

A huge gap exists between conventional results in test automation and the potential value that automation can bring to software quality. Matt Griscom presents an effective way to achieve that value with the MetaAutomation pattern language—fast, scalable, and reliable automation. MetaAutomation’s artifacts contain focused, structured information to make results immediately actionable, informative for the entire product team, and available for robust analysis. With these patterns, quality information automatically goes to the people who need to know. With a complete implementation, false positives and false negatives don’t interrupt people’s work. Manual testing is accelerated and focused—not confused by the false promise of automated manual tests. Learn the five patterns of MetaAutomation—Atomic Check, Precondition Pool, Parallel Run, Smart Retry, and Automated Triage. Join Matt to explore the nature and importance of MetaAutomation to software quality, and take away tips for quickly improving automation projects.

Computer hacking isn’t a new thing, but the threat is real and growing even today. It is always the attacker’s advantage and the defender’s dilemma. How do you keep your secrets safe and your data protected? In today’s ever-changing technology landscape, the fundamentals of producing secure code and systems are more important than ever. Exploring the psyche of hackers, Mike Benkovich exposes how they think, reveals common areas where they find weakness, and identifies novel ways to test your defenses against their threats. From injection attacks and cross-site scripting to security mis-configuring and broken session management, Mike examines the top exploits, shows you how they work, explores ways to test for them, and then shares what you can do to help your team build more secure software in the future. Join Mike and help your company avoid being at the center of the next media frenzy over lost or compromised data.