Better Software East 2016 - Testing & QA
Tuesday, November 15
Measurement and Metrics for Test Managers
To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Mike Sowers as he addresses common metrics—...
Thinking Inside the Box – Root Cause Analysis with The Six Boxes
Do you want to improve business and user value delivery, quality, efficiency, and productivity of your software engineering team? OK, that’s a stupid question because who doesn’t? Poor productivity problems, quality issues, failing to meet commitments, and general team inefficiencies are, unfortunately, still commonplace. And what is at the root of most problems? James Waletzky says the answer is those highly imperfect creatures—humans. So how do we go about fixing the problems? First, we must discover the root causes, not just the symptoms, and those are not...
The Tester’s (New) Role in Agile Development
Avoiding siloed development is a tricky business. It’s easy for agile teams to fall into the rut in which testers only do traditional testing activities and programmers strictly do their time-worn coding activities. Rob Sabourin shares a number of examples of how testing skills can be applied to a wide variety of activities in an agile project. Testers are among the most skilled team members in story grooming, elicitation, and exploration. Risk analysis in self-organized agile teams empowers testers to drive design decisions. A tester’s affinity analysis skills...
Plan, Architect, and Implement Test Automation within the Lifecycle
In test automation, we often must use several tools that have been developed or acquired over time with little consideration of an overall plan or architecture, and without considering the need for integration. As a result, productivity suffers and frustrations increase. Join Mike Sowers as he shares experiences from multiple organizations in creating an integrated test automation plan and developing a test automation architecture. Mike discusses both the good (engaging the technical architecture team) and the bad (too much isolation between test automators and...
Wednesday, November 16
A New Approach to Software Safety, Risk, and Vulnerability Analysis
Software has found its way into almost every part of our lives. Increased automation in the cars we drive can lead to failures which could result in physical injury, unacceptable risk, or cyber security vulnerabilities. In order to prevent accidents, identification of hazards, risks, and security vulnerabilities is required during development. The problem is the traditional hazard analysis techniques—failure effects and modes analysis, fault tree analysis, and root cause analysis—were developed for simplistic hardware controllers and are based on single point...
Tools and Techniques for Testing Mobile Apps
With hundreds of mobiles, tablets, and wearables running different versions of iOS and Android, automated continuous testing is the only sane way to address mobile testing. Tom Chavez reviews automated testing tools for mobile apps, including tools that require writing tests in a programming language and tools that record and playback mobile app actions for replay requiring no programming at all. See tools that show your real user data including location, OS, connection type, device battery level, and data I/O. Tom shares how these affect application performance...
SEBTE: Simple Effective Experience-Based Test Estimation
Testers are challenged to provide accurate estimates. Later, magically, test estimates become commitments cast in concrete even in the most turbulent contexts. In fact, most test estimates are based on shallow project insights. Estimators are often ignorant of key design, usage, and environmental factors. Many traditional project management estimation techniques fail when applied to testing projects. After all, what you test next depends on what you learned before, and knowledge evolves as the project continues. Rob Sabourin shares his SEBTE test estimation model...
Leading a Testing Center of Excellence
As companies look to drive down the cost of quality, they often leap to the conclusion that centralizing QA into a Testing Center of Excellence (TCoE) will magically result in increased quality and lower costs—without understanding the path required to get there. Change requires defining mission and goals, and putting into place a plan to move forward. Equally important is the need to establish cost of quality metrics to shift the focus from the cost of testing to the cost of building quality applications. Susan Schanta shares her experience transitioning...
Artificial Intelligence (AI) in Testing: The Future Is Now
When building new content for a game such as Candy Crush Saga that has more than 1000 levels, it is extremely important to make sure that the level of difficulty is balanced and that the user does not experience crashes or problems through some unforeseen type of play. Alexander Andelkovic shows you how King is training AI-programs (bots) to test its games by mimicking human interactions. Alex explains how King is taking testing to the next level by employing Monte Carlo tree search, automatic heuristic construction, and NeuroEvolution of Augmenting Topologies (...
Thursday, November 17
Continuous Delivery of Microservices: Patterns and Processes
Tight coupling and interdependence between dev teams slows down development, and environment drift causes deployment problems when promoting applications between environments. The introduction of the microservices architecture expands on the role of SOA by allowing for the looser coupling of endpoints and environments, with more predictability, flexibility, and agility. Anders Wallgren says that microservices have gained attention for their ability to facilitate innovation among dev teams. By architecting applications into sets of smaller services that are more...
Exploratory Security Testing in the Browser
Much of the time, security testing is relegated to the end of development and then is assumed to be someone else’s problem. However, late security fixes can have a huge impact on your delivery dates. Although automated scanners offer some help by inspecting your application’s HTTP traffic, most common attacks are best performed in the browser, which should be the focus of your exploratory security testing. Using familiar automation practices, open source tools (Webdriver/Selenium), and a browser (Chrome/Firefox/Edge), Abraham Marin-Perez shows you how to drive a...
Optimizing Performance across the DevOps Lifecycle
The world of DevOps moves fast and substantially impacts application development quality, costs, and customer confidence. Giving your operations and development teams an effective performance process from an early stage means quality is a priority throughout the development lifecycle. A “test early and often” approach improves your chances of on time/on budget application release and meeting your customer’s performance and quality expectations. Successful performance across DevOps relies on the use of proper processes, performance tools and solutions, and...
Simplify Performance Testing with Production Data
When creating models of system performance, the primary challenges are where and how to start. Whatever the performance characteristics to be estimated or modeled, we need to adopt methods that are repeatable and able to address both the business and technical needs of the project's delivery. All too often, performance tests inadvertently mix scenarios with little regard for how transaction rates can confound technical recommendations and business decisions. Join Will Hurley as he presents a real-world case study that demonstrates the impact on business decisions...
Continuous Integration of Innovation and Quality
High-performing DevOps organizations deploy hundreds of times more frequently than lower performers, so it is no surprise that high performers are known for their innovation and quality. Low-performing DevOps teams fail to achieve continuous delivery due to limitations in their testing process and strategy and thus miss the opportunity to continuously test their ideas and integrate feedback. Stacy Kirk describes how to start or optimize quality in your DevOps lifecycle for continuous delivery of innovative applications. Stacy explains how to foster a culture of...
Conquer the Murky Waters of Test Automation
A solid test automation implementation is key to any foray into continuous delivery. Although the test automation waters may look peaceful and pristine, anyone who has jumped in to automate testing complex systems has found the waters to be troubled and murky. Tests take too long to run, they periodically break for mysterious reasons, or they fail to catch business critical problems. Over time this leads to automated tests becoming outdated and irrelevant. Brian Saylor has studied a number of test automation projects—and far too many lines of test code—to...
Build Fail-Proof Tests in Any Browser with Selenium
What happens when you have thousands of tests that run beautifully in Chrome but many of them fail in Internet Explorer? Unfortunately, this scenario is all too common for testers and remains a major sore point for teams tasked with getting software to work in any browser. Kevin Berg highlights how access to cloud-based Selenium Grids makes it easier than ever to run functional test suites in every imaginable operating system and browser combination. The result is less time and hassle adapting testing suites to each individual browser. Join Kevin as he shares...
Friday, November 18
Six Things to Avoid in Software Testing on Agile Projects
With the emphasis on cross-functional teams, are testers still valuable on agile teams? The answer: Absolutely! Testing helps consider the full spectrum of quality attributes by bringing a unique perspective of evaluating various user interactions with the application. Insufficient testing may allow severe defects to pass into production, causing financial loss or even bankruptcy. Knowing how to leverage testing to gain the greatest rewards can be tricky as there are many things you can do that appear to be equally important—and likewise many things to avoid. For...