STARWEST 2018 - Test Strategy, Planning, Metrics

Software continues to revolutionize the world, impacting nearly every aspect of our work, family, and personal life. Artificial intelligence (AI) and machine learning (ML) are playing key roles in this revolution through improvements in search results, recommendations, forecasts, and other predictions. AI and ML technologies are being used in platforms for digital assistants, home entertainment, medical diagnosis, customer support, and autonomous vehicles. Testing practitioners are recognizing the potential for advances in AI and ML to be leveraged for automated testing—an area that still...

We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. Mary Thorn says help is at hand. Enter behavior-driven development (BDD), and Cucumber and SpecFlow, tools for running automated acceptance tests and facilitating BDD. Mary explores the nuances of Cucumber and SpecFlow, and shows you how to...

Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking,...

Artificial Intelligence (AI) is making it possible for computers to diagnose some medical diseases more accurately than doctors. Such systems analyze millions of patient records, recognize underlying data patterns, and generalize them for diagnosing previously unseen patients. A key challenge is determining whether a patient's symptoms and history are attributed to a known disease or other factors. Software testers face a similar problem when triaging automation failures. They investigate questions like, Is the failure due to a defect, environmental issue, or nondeterministic test script?...

Isn’t it amazing? Stakeholders drop software on our desks and expect us to test it—without any requirements, design, or product knowledge whatsoever. About the only clear thing is the absurd and unrealistic deadline. We are expected to bend over backward, spread magic pixie dust, and heroically test quality into a product we have never heard of before. But testing in the dark is not impossible, and as Rob Sabourin shows, it can even be a very valuable and fun experience. Learn strategies to emerge from a murky fog into clear, meaningful quality insights and leverage unlikely sources about...

Although processes and tools play an important role in software testing, the most important testing tool is the mind. Like scientists, testers search for new knowledge and share discoveries—hopefully for the betterment of people’s lives. More than sixty years ago, William I.B. Beveridge reframed discussion of scientific research in his classic book The Art of Scientific Investigation. Rather than add to the many texts on the scientific method, he focused on the mind of the scientist. Join Ben Simo as he applies Beveridge’s principles and techniques for scientific investigation to software...

Emerging technologies such as the internet of things (IoT) and cloud computing have introduced a significant software variety and complexity. Wendy Siew Wen Chin and Heng Kar Lau explain that testers are challenged to support a wide product portfolio within harsh time, resource and budget constraints. More test automation may seem to be a solution to test efficiency, however there are many inefficient hot spots throughout the test automation life cycle. Join Wendy and Heng Kar as they share their experiences from the Intel IoT team. They share how to make use of artificial intelligence (AI...

Behavior-driven development (BDD) has been around for a while and is here to stay. However, the added abstraction levels pose a technical problem for writing and managing tests. While BDD does a great job of marrying the nontechnical aspect of test writing to the technical flow of an application under test, keeping this information under source control becomes problematic. Frameworks such as JBehave, Cucumber, or Robot give subject matter experts that additional ability to write tests, but they are often restricted access from them; because people treat test cases as code, they get stored...

Teams are sometimes asked to turn a mess of undocumented, poorly structured legacy code into a robust product under impossible deadlines. Test strategies blending automation, exploration, and refactoring can help focus development efforts and converge even the most chaotic projects. But, where do you start? Join Jonathan Solórzano-Hamilton as he shows how automation can help drive products into a state of release readiness. Learn how refactoring, test-driven development, SOLID principles, dependency injection, and mocking frameworks help break down complex development problems into...

Testing is shifting left, moving closer to testing the code itself before the full product is ready for release. While the backend world already has established methodologies for testing, frontend developers and testers are still trying to figure out how to work together to effectively test the code. Gil Tayar suggests testers need to communicate with the frontend developers to understand the framework by which frontend code is tested, the various kinds of testing that can be performed on frontend code, and which tools can be used. During this session, Gil helps ease your fear of the...

Each of us has personal weaknesses that are often perceived to have negative impacts on performance and capability in the workplace. When this weakness is prominent to yourself or others, are you capable of benefiting IT teams and qualified to do the job? In dealing with negative impacts of her Generalized Anxiety Disorder, Julene Johnson falls into this category and has experienced personal doubt in her ability to perform her job well. Over time, however, she’s noticed that habits and techniques developed while dealing with anxiety can be applied in a manner that improves her life...

API-based applications have become ubiquitous in the past few years. Yet many of us still really don't know what an API is from the inside out. Moreover, many of us don't know much about how to comprehensively test them! Join Paul Merrill to learn what everyone is doing and talking about, related to APIs: microservices, REST, endpoints, requests, request methods, and responses. Paul demonstrates four free tools you can use to directly interact with a simple API: curl, the browser, Postman, and Java (with Rest Assured). Learn techniques for testing APIs at the service level and how to test...

Everybody wants to have their projects tested so that they can deliver top-notch products to their clients. But with multiple projects all running in parallel, all with tight deadlines and all needing testing, simply allocating a tester on a project for some time isn’t going to give you quality. This, in turn, can sometimes make our stakeholders question the value of testing. Join Nimesh Patel as he shares his experience in building a test practice and team from the ground up into valued, credible members of a project, in a digital agency where the focus is often on delivering...

Performance issues substantially impact quality, cost, and customer confidence. Agile teams are challenged to build in performance processes throughout the lifecycle, but it is critical to incorporate performance into your CI/CD pipeline. Join Amit Patel as he shares his recent project experiences and the steps his team took to change processes, leverage different technologies, and align internal stakeholders. He explains how they use production-monitoring solutions to create a real-world production feedback loop in order to ensure they can analyze data and turn the information...

Articles on abc.net and elsewhere reported that Volvo has recently discovered a non-traditional requirement: Any self-driving vehicle approved for use outside Australian cities must recognize kangaroos on or near the roadway and take proper actions. The kangaroo’s bounce confused the large animal detector! In this session, industry expert David Gelperin shares a new perspective on the value of imprecise requirements and explores the nature of testing them. Excess precision may hamper the development of optimal solutions by excluding effective designs. Imprecise statements reduce the risk...

Does test automation need traditional test cases? Mehmet Duran says no. Using the open source tool TESTAR, he devised an innovative approach to automated testing. Join Mehmet as he shares the research he conducted to solve this challenge and how he confirmed his findings by comparing test approaches using a new framework with the quality attributes of learnability and usability. Mehmet will also describe he worked with Hatim Chahim of Prorail on a real-world railroad project to try out his theory. After applying the framework, results showed that within this setting, the test automation...

Risk-based testing is essential to focus our testing, but it is not always easy to apply to our projects. Risk management tends to focus more on project and process risks (i.e., Will we make the deadline? Do we follow our processes?) and less on the product risks that can act as a foundation for a risk-based approach to test. Including this aspect of risk in your test coverage will give you a solid foundation for defining a test strategy that implements and executes the right tests with the right intensity to mitigate the most critical product risks. In this presentation, Gitte...

Organizations are consistently looking to deliver high quality products in a quick, repeatable and predictable manner. Deliver more, deliver faster, and deliver quality! Often QA leaders are challenged to do just this while keeping the lights on, addressing process impediments, and dealing with pressure from sales and/or clients to continue standard operating procedure. In this session, Leigh will share her journey in transforming a testing organization from manual to leveraging test automation. She will discuss best practices in transforming testing teams, the challenges and pitfalls she...

We live in a time of rapid change when products and services often become obsolete overnight, and competition includes startups and companies in adjacent industries — where the traditional formula for leadership may no longer hold relevance. A new value system is beginning to emerge for the leaders of the future which will continue to grow with the rise of tools like robotics, automation, and artificial intelligence. The leaders who thrive are those who work well in unknown conditions, and who continuously inspire into their culture the ability to...

Join your peers in an engaging and highly interactive discussion about the issues that concern you most. Using answers to the question—"As a Leader, What Is Keeping You Up at Night?"—posed at Thursday’s evening reception, participants will form small groups to work on finding solutions to pressing test management issues. Discussions will review identified issues and barriers to change, and focus on innovative strategies and practical next steps. At the end of the think tank discussion, all feedback will be collected and posted online to encourage further collaboration....

Let’s discuss and define the most important career superpowers for women. Hear what industry professionals, leaders, and idea disruptors think the important career superpowers are for women in 2018 and going forward.