STAREAST 2017 - Testing & QA

Whether you are new to testing or looking for a better way to organize your test practices, understanding risk is essential to successful testing. Dale Perry describes a general risk-based framework—applicable to any development lifecycle model—to help you make critical testing decisions earlier and with more confidence. Learn how to focus your testing effort, what elements to test, and how to organize test designs and documentation. Review the fundamentals of risk identification, analysis, and the role that testing plays in risk mitigation. Develop an inventory of test objectives to help...

In the tradition of James Whittaker’s book series, How to Break Software, Jon Hagar applies the testing “attack” concept to mobile and embedded software systems. First, Jon defines the environments of mobile and embedded software. He then examines the issues of software product failures caused by defects found in these types of software. Next, Jon shares a set of attacks against mobile and embedded software based on common modes of failure that teams can direct against their software. Like different kinds of software design patterns, attacks are test design patterns that must be...

Influence diagrams provide a simple-to-create and easy-to-understand approach to address the complexities of real-life problems. As testers, we may want to find more bugs, but this may have an unintended consequence for developers. Developers now have more defects to debug, which affects their capacity to deliver new functionality. Isabel Evans has found that influence diagrams provide a means of understanding and managing the complexities of key interactions among testers, developers, and business stakeholders. In the past few years, Isabel has used influence diagrams as a practical tool...

In test automation, we must often use several tools that have been developed or acquired over time with little consideration of an overall plan or architecture and no consideration for how to integrate those tools. As a result, productivity suffers and frustrations increase. Join Mike Sowers as he shares experiences from multiple organizations in creating an integrated test automation plan and developing a test automation architecture. Mike discusses both the good (engaging the technical architecture team) and the bad (too much isolation between test automators and test designers)...

With the adoption of agile practices in many organizations, the test automation landscape has changed. Bob Galen explores current disruptors to traditional automation strategies and discusses relevant and current adjustments you need to make when developing your automation business case. Open source tools are becoming incredibly viable and often best their commercial equivalents in many ways―not only in cost but also in functionality, creativity, evolutionary speed, and developer acceptance. Agile methods have fundamentally challenged our traditional automation strategies. Now we must keep...

Cloud computing is rapidly changing the way systems are developed, tested, and deployed. New system hosting capabilities—software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS)—are forcing us to review and revise our testing processes. At the same time, cloud computing is affording us opportunities to employ new test tooling solutions, which we call testing as a service (TaaS). In this technical session, Jeroen Mengerink focus on testing SaaS systems, linking relevant IaaS and PaaS capabilities along the way. He will discuss how to test...

As testers, we know that we can define many more test cases than we will ever have time to design, execute, and report. The key problem in testing is choosing a small, “smart” subset—from the almost infinite number of tests available—that will find a large percentage of the defects. Join Lee Copeland to discover how to design test cases using formal black-box techniques, including equivalence class testing, boundary value testing, decision tables, and state-transition diagrams. Explore examples of each of these techniques in action. Don’t just pick test cases randomly. Learn to selectively...

No longer just a futuristic concept, the Internet of Things (IoT) has a strong presence in our world even today. If your business is not prepared for it, you’re already behind. With the proliferation of connected “things”—devices, appliances, cars, and even clothes—Jennifer Bonine says that the stage is set. IoT apps are here to stay. Testing, product management, and development teams must address developing and testing in this paradigm. Testers, accustomed to traditional platforms, are now asked to test on more complex devices and more advanced platforms. Testers must keep up with the...

Many organizations invest a lot of effort in test automation at the system level but then have serious problems later on. As a manager, how can you ensure that your new automation efforts will get off to a good start? What can you do to ensure that your automation work provides continuing value? Dot Graham describes the most important management issues you must address for test automation success, particularly when you are new to automation. Dot helps you understand and choose the best approaches for your organization—no matter which automation tools you use. Focusing on system...

You have just been assigned a new testing project. Where do you start? How do you develop a plan and begin testing? How will you report on your progress? Paul Holland shares new test project approaches that enable you to plan, test, and report effectively. Paul demonstrates ideas, based on the Heuristic Software Test Model from Rapid Software Testing, that can be directly applied or adapted to your environment. In this hands-on tutorial, you’ll be given a product to test. Start by creating three raw lists (Product Coverage Outline, Potential Risks, and Test Ideas) that help ensure...

Have you ever worked on a project where you felt testing was thorough and complete—all features were covered and all tests passed—yet in the first week in production the software had serious issues and problems? Join Dawn Haynes to learn how to inject robustness testing into your projects to uncover those issues before release. Robustness—an important and often overlooked area of testing—is the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. By expanding basic tests and incorporating specific robustness attacks, Dawn...

To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Mike Sowers as he addresses common metrics—measures of product...

The current trend in testing is to move away from traditional manual testing and toward test automation. To keep up with the rapid pace of continuous delivery today, test teams are urgently reskilling and reshaping themselves. Sally Goble shares her experience of what happened to the test team at the Guardian when the engineering team began the process of delivering software fast and at will—deploying software up to 400 times a day. Sally shares why her team reevaluated the role of testing in the software development cycle, what prompted the Guardian’s controversial decision to...

Acceptance testing is often thought of as the little brother of system testing and, in many projects, it ends up as a little phase at the end. Having worked in system testing for most of her testing career, Bettina Faldborg found it was a bigger jump than you might think to move to acceptance testing. She had this overwhelming urge to test it ALL. Like most testers, she did not assume that what has been tested before was done well enough, but that is one of the preconditions to accept going into an acceptance test. With regard to acceptance testing Bettina presents four basic...

Finding time to learn test techniques, mentor other testers, grow application knowledge, and cross-train your team members is a daunting task with a complicated recipe. What if you could do these things while testing and finding bugs? Enter Pair Testing. What’s that? Well, maybe you’ve heard of pair programming. It’s like that—only you’re testing rather than programming. And it’s the secret sauce of agile testing because it makes your routine, bland testing so much more fun! Testers on Jess Lancaster’s team use pair testing not only to make better software but also to foster better...

Artificial intelligence (AI) is the most important technology for software testers to understand today. All software will soon have AI-powered components, and they are unlike anything you’ve ever tested before. As risky as AI can be, it is a powerful weapon for testers to solve some of their most painful testing challenges today. The web was great, mobile is interesting, but AI will truly change the way you build and test all software. Jason Arbon gives a brief introduction to AI and machine learning (ML) so you can nod your head knowingly when the topics come up. Explore how products that...

Unrestrained access to a trustworthy and realistic test environment—including the application under test and all of its dependent components—is essential for achieving “quality @ speed” with agile, DevOps, and continuous delivery. Service virtualization is an emerging technology that provides teams access to a complete test environment by simulating the dependent components that are beyond their control, still evolving, or too complex to configure in a test lab. Arthur Hicken covers the ABCs of service virtualization—what it is and how it impacts Access, Behavior, Cost, and Speed....

Over the past twenty years, Mary Thorn has had the opportunity to work at many startups, creating several QA/test departments from scratch. For the past ten years, she has done this in agile software companies. Recently Mary moved from leading small agile test organizations to leading a large agile test organization. She has learned how to lead agile testers and agile testing in large contexts. Mary takes you through what she has learned, identifies the keys to transitioning your test organization as it grows, and discusses the techniques required to lead it through the changes. Agile...

Classical test process improvement is often not today’s best solution. With virtualization, SOA, web, cloud, mobile, and integration with social media, the way we develop, test, and manage has drastically changed. Jeroen explores why Agile, context-driven testing, SCRUM, continuous integration/development and DevOps require a flexible and pragmatic context-driven approach to test improvement. Context-driven test improvement is organized on two levels. [1] The improvement architecture level starts by clarifying the goal, scope, and context. This results in the improvement approach, a...

Software development is trending toward building systems using small, autonomous, independently deployable services called microservices. Leveraging microservices makes it easier to add and modify system behavior with minimal or no service interruption. Because they facilitate releasing software early, frequently, and continuously, microservices are especially popular in DevOps. But how do microservices affect software testing and testability? Are there new testing challenges that arise from this paradigm? Or are these simply old challenges disguised as new ones? Join Tariq King as he...

We often talk about agile in terms of development of new systems, sometimes called greenfield projects. These scenarios present us with a “cleaner” version of agile. But Jim Peers is asking the question: What about agile methodologies when applied to the maintenance of a mature product? A mature product brings into play many more variables—addition or upgrading of features, addressing technical debt, bug fixing, security and accessibility concerns, and working with an aging codebase. And each variable brings its own challenges. By necessity, these scenarios can force us to adapt our work...

Aboriginal people across Canada face an unemployment rate more than twice the national average. Meanwhile, vacant IT positions continue to rise across the country, with an additional 218,000 new openings expected by 2020. Denis Carignan shares how PLATO Testing is building a network of Aboriginal software testers across Canada to help solve these problems. Students begin with intensive lab-based training, followed by an immersive internship, and end with full-time employment at PLATO Testing as professional software testers. As this initiative grows, so does the ability to capture large...

The testing community is caught between the devil and the deep blue sea when it comes to exploratory testing. Although exploratory testing has been around for ages, it often leads to more confusion than clarity. Is exploratory testing an activity—something that you do? Or is it an approach—a way or a style of doing something? Isn’t all testing exploratory? When do you do it? How do you do it properly? How does it relate to the entire software lifecycle? To answer these questions, Ingo Philipp outlines the most common confusions and controversies on this topic. He explains what exploratory...

All organizations doing mobile development have critical decisions in common—selecting which devices to support and how to test their products across the growing range of devices. With the segregation of device brands, organizations are forced to support at least three code bases (Apple, Android, and Windows). This makes it three times more challenging for testing organizations to keep up with the minimum required testing efforts for their products. Ardy Signey says that test leadership has to look beyond conventional testing strategies and become more innovative. Companies that already...

DevOps and regulatory compliance are two critically important ingredients in today’s connected organization. The first—DevOps—enables you to move quickly and respond to change in an era where change is increasing at an exponential rate with no sign of slowing down. The second—regulatory compliance—ensures that your organization takes the appropriate steps to follow relevant laws surrounding your software development lifecycle and appears to require adding burdensome processes and controls. At first glance, these two ideas seem to be incompatible, but they actually go together like peanut...

Have you ever been dragged into a production emergency that severely drains deadlines, budgets, morale, and weekends only to find out—yet again—that the problem is related to application stability and performance under load? Building a culture that embraces proactive and well-planned performance testing is no small feat. Tom Poirier shares a journey that begins with clarifying enterprise performance requirements, performing competitive analysis, cost-justifying purchases, and closing the deal with senior management. Tom walks through practical challenges including technical difficulties,...

James Koch demonstrates how to apply test-driven development and continuous integration using Appium, an industry leading open-source automation tool. Appium was inspired by Selenium for web browser testing. Their philosophy—Why do anything different for mobile? Appium allows you to run tests without having to add any third party framework or extension into your application, so you are testing the same application you are shipping! With faster release cycles, the need for quality testing is more necessary than ever. Tools like Appium leverage continuous integration and test-driven...

The challenge for test teams in organizations that want to use blockchain technologies is to not only test chaincode that manages the execution of transactions on their blockchain networks but also to test the provision and scalability of the blockchain networks themselves. James Hunter presents new approaches that are emerging to test blockchain technology, supply-chain business processes and transactions, and APIs from multiple perspectives; to simulate the behavior of consensus within a blockchain network; and to test integrations at scale without affecting the distributed ledger. James...

To maintain optimal product quality of large-scale enterprise systems, the regression test suite usually increases in size over time. Whether using automated or manual regression, this brings an additional maintenance and infrastructure cost that tends to get way out of hand, often defeating the purpose of regression suites. Sharing a case study, Ashish Mehta and Sohail Farooqui provide insights into analyzing enterprise application defects with associated gaps in relevancy and effectiveness of an existing regression suite. Ashish and Sohail implemented an automated solution to test...

In her more than thirty years in the IT industry, Isabel Evans says she has learned more from her failures than she has from her successes. Why is this? And what has she learned? That making mistakes is the way to learn, and that allowing yourself to be wrong allows you to grow. Join Isabel to enjoy her greatest failures, and learn not to make the same mistakes she has made. Recently, someone described Isabel as unusual in the technology industries as she is an “elderly woman,” so she has taken as her motto Bob Dylan’s line: “I was so much older then—I’m younger than that now.” Isabel...

Scapegoats, spin-doctors, white knights, and sycophants—have you found your test team playing these roles? Organizations, both large and small, often have distinct cultures and power structures with significant but insidious impact on how individual testers and teams are expected to operate. Sometimes the difference between doing what sponsors and stakeholders request and doing what is really needed becomes blurred. John Hazel helps you learn how to recognize the cultural characteristics of different types of software development teams, and how they drive expectations for the test team....

The most critical component in capacity planning and performance engineering is the Workload Model, which defines the workflows, throughputs, and target performance your system must support at peak loads. As critical as it is, it can be difficult and particularly challenging to predict loads for new applications, features, or events. A typical approach starts with a wild-guess worst-case scenario—but overestimates waste time and money, and force you to engineer applications and infrastructure to support unrealistic loads. Low estimates can result in terrible customer experiences, lost...

You ran all the tests you planned for your team, you reported all the bugs with clear and to the point descriptions, and you sent a weekly email with a professional PowerPoint presentation including graphs and statistics pointing out the risk areas and project issues. However, you still feel the organization is not taking your testing seriously, management is unaware of what your team is achieving—and apparently no one is actually reading your reports. Sound familiar? Everyone else is not the problem; the way you are communicating your testing information is! Join Joel Montvelisky to...

Software testers face a major challenge when designing test strategies and their related tests that effectively find software failures. With many techniques available, it is sometimes difficult to immediately apply them to real-life situations. What initially seems like a straightforward effort often turns into something resembling a winding road with no end. A test strategy may seem good at first, but once the testing begins, it may fail. Randy Rice believes there are ways to define a workable and flexible test strategy that will lead to tests that are creative and effective. Test...

Tania Katan knows a little something about sneaking principles from art into the tech space in order to start social media revolution! As the Brand Evangelist for Axosoft she co-created the internationally viral campaign #ItWasNeverADress and now she has a map for you to use to find your creativity too! She will share uniquely developed exercises for flexing your creative muscles, funny stories from the front-lines of an everyday disruptor and enough empowering quotes and inspirational shouting (GO get ‘em, Creative Tiger!) to make your high school soccer coach crack a smile. You will...

Are women testers valued in the software industry? Are women leaders making a mark in the software industry? Absolutely! Join Madhu Das as she discusses her personal and career journeys—from a child growing up in India, to becoming a software professional, her journey from India to the U.S. and her growth as a leader in the male-dominated world of software testing. She speaks of her experiences, the successes she discovered as a woman leader in the world of agile software testing, including the challenges she faced and how to surmount them. Through stories from the trenches at multiple...

Let’s discuss and define the most important career superpowers for women.  Hear what industry professionals, leaders, and idea disruptors think the important career superpowers are for women in 2017 and going forward.