STARWEST 2023 - Test Techniques
Monday, October 2
Test Automation: How to Start and Succeed
Many organizations invest a lot of effort in test automation at the system level but then have serious problems as their product matures and changes over time. As a leader, how can you ensure that your new automation efforts will get off to a good start? What can you do to ensure that your automation work provides continuing added value? Chris Loder will explain the critical issues you need to know to get a good start, and he will share his extensive experience in building great automation. He covers the most important management issues you should address for test automation success,...
Tuesday, October 3
Testing in the Cloud From All Four Corners of the World!
This is a workshop for those of you that have done some sort of scripting or automation before. It’s a workshop where we’ll write a simple test agent using Python for sending traffic to a centralized REST API. You’ll follow up by creating 10,000 items of random test objects for offline test data. You’ll spin up server instances in the AWS cloud using Python. And finally, to top it all off, you’ll deploy your test agent and test data in cloud instances spread out over the four corners of the world, all of them generating simultaneous test traffic toward a single REST API where you’ll all...
Getting Smart on API Testing & Automation
API testing has become more and more popular as service-oriented architectures have become common. In addition, testing at the API level can be effectively automated to provide maintainable regression tests that work well in a DevOps process. In this tutorial, Jeffery Payne discusses what API testing is all about and how it is effectively done. Techniques and tools are discussed that highlight where in the testing process API testing makes the most sense to perform. Various open source and commercial tools will be demonstrated and the pros and cons of various solutions discussed. Effective...
Web Security Testing: The Basics and More
Web applications are often security critical or serve as front-ends for security critical applications, making web testing for vulnerabilities an essential part of software testing. Unfortunately, most software testers have not been taught how to identify web security issues while testing applications. Join Tom Stiehm as he shares what you need to know to security test web-based applications as part of your overall testing process. Learn about the most common web security vulnerabilities and how they are introduced into web code and exploited by hackers. Explore test techniques for...
.jpg%3Fitok=PIWiwpDq&sc=8bd2262f52177530ca62ad8be334afca "Chris Blain")
Exploratory Testing in the Heat of the Sprint
Agile teams are burdened with the challenge of delivering working product increments after short iterations of development. Getting software from an ambiguous terse, incomplete requirement–to a done, working, solid, valuable, high-quality code requires testers to continuously adapt to change in a turbulent context and deliver actionable results. Chris Blain will illustrate how charter-driven session-based exploratory testing techniques can empower agile teams and help them learn quickly and adapt based on what really matters. Testers can design and execute tests on the fly as they explore...
Test Estimation in the Face of Uncertainty
Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking,...
Smarter Test Design with Classification Trees and Pairwise Techniques
In many teams, the total number of possible combinations of inputs, outputs, browsers, and devices for the software we need to test has grown to an alarming number. As testers, we need to choose the most important tests first, but how do we do that without understanding the potential scope in the first place? In this tutorial, Julie Gardiner will share two powerful testing techniques that can help us be more efficient and effective with our testing. Classification trees are a structured, visual approach to identifying test objects and documenting test ideas and data in a way that allows...
Wednesday, October 4
What Can ChatGPT Do for Your Testing Organization?
PreviewChatGPT can assist a testing organization by automating repetitive and mundane tasks, such as answering FAQs, generating test cases, and providing relevant information on testing processes. Additionally, it can provide quick and accurate responses to inquiries, freeing up time for human testers to focus on more complex tasks. ChatGPT can also be integrated into an organization's testing tools to improve efficiency and reduce errors in testing processes. How? Through test case generation, ChatGPT can be used to generate test cases automatically by analyzing the language patterns and...
Exploratory Testing As Code
What if testing followed the lead of revolutions in infrastructure and configuration management. What if testers treated exploratory testing like developers do code? Exploratory testing is a powerful and effective technique. However, its acceptance is inhibited because there isn’t a common structured process that enables the traceability and transparency to make it a standard across the software industry. In this session, Brendan Connolly will introduce the tooling and process needed to implement a light-weight, scalable, transparent, and traceable approach to managing exploratory testing...
An AI-Partnered Testing Future—It’s Here!
Let’s be real, we are quite a ways out from AI taking over the world. AI doomsday apologists stay home! On the other hand, AI is changing everything about software creation. In this presentation, Kevin Pyles will take you on a walk down memory lane through waterfall, agile and DevOps practices, and will then introduce you to the future of software creation known as AI-Partnered Development. Let’s get practical with real-world examples of AI assisting designers, solving coding challenges and testing software right next to human testers. We will explore how AI is also involved today in...
The Theatre of Exploratory Testing: Going Off Script
PreviewAsk a handful of testers to define exploratory testing. Some will respond that it’s simply poking around—try something, see what breaks, and then try something else. Many will identify it as an iterative process of learning and experimentation. But ask about what makes exploratory testing hard, and you find that practitioners are challenged in knowing where to start, what to test, when to stop, and how to know if they’ve found a bug. What makes for good exploratory testing? Join Philip as he examines the techniques of the actors of the Commedia dell'Arte, the 16th century approach...
JavaScript Mutation Testing: Improving Confidence in Your Tests
PreviewJavaScript code requires effective testing strategies to avoid bugs in production. Code coverage metrics alone can be misleading and don't necessarily indicate that the code is actually working as intended. It can be easy to get high code coverage metrics, but not actually test anything meaningfully. To address this problem, this session will explore the concept of JavaScript mutation testing. Mutation testing introduces small changes or "mutations" to the code and then expects tests to fail. This technique helps identify areas of the code that may need improvement and strengthens...
Lady on the Block: A Tester’s Perspective on Blockchain Technology
How do you go about testing the immutability of deployed code? What control does one have over a decentralized test environment? How does one overcome the complexity of smart contracts and associated potential vulnerabilities? What can be done to ensure seamless integration and interoperability between the blockchain application and external components? As a tester of blockchain applications, Gomathi Ramalingam has encountered and faced many of these challenges, which are inherently due to the unique nature of blockchain technology. She believes that having a good understanding of these...
Thursday, October 5
How My Testing Skills Served My Personal Life
Kristoffer Nordström faced one of the most challenging and emotional times of his life when his daughter was diagnosed with a rare and life-threatening disease. As a caregiver, father, and primary project manager, he had to learn everything he could about her illness, treatment options, and potential outcomes. The skills he acquired as a tester helped him manage this overwhelming amount of information, make sense of it, and develop a plan of action while still being a loving and ever-present parent. He asked questions, researched, and analyzed data to make informed decisions about his...
A Realistic Approach to Scalable Testing for Cross-Team Distributed Systems
Let’s face it. Building tests tends to take more time than developing new features themselves. It is tedious to build sufficient coverage and cumbersome to maintain long-term. However, on an enterprise systems-level, tests are crucial in ensuring bugs or defects are caught before they reach clients. As engineers, we pour our hearts and souls into trying to identify defects before we release our code to clients. But when rolling out big features, it is rare to successfully complete a software deployment process without some errors popping up along the way. This challenge becomes...
Coded Automation Isn't Dead—A Deep Dive on the Page Object Pattern with Some Super Cool New Twists
Software Development Engineers in Test know that page objects have long been used to make test automation more resilient. This talk evolves the page object pattern in ways that provide some really strong advantages over the traditional approach. Timothy Heck will demonstrate how to make page objects isolated from the browser automation technology, making it possible to switch from Selenium to Playwright (or the next big thing) without changing the page objects. This talk will also look at how to make self-narrating automation scripts where the simple act of interacting with a page object...
Harnessing the Power of Ephemeral Environments
Integration testing in the staging environment is a pain. One bad commit from a dependency, and all your automated integration tests start failing. Consequently, developers rightly don't want to gate their code pushes on successful integration tests, and integration test results are quickly dismissed as unreliable, and ignored completely. Ephemeral environments change all of that. By using ephemeral environments as a key part of test strategies, not only has John Jenkins succeeded in moving staging dependent integration tests into the build pipeline, but he has also caught and prevented...
Are We Done Testing? A Metrics Dashboard
In many cases, it takes multiple metrics to get a complete picture of the status of a process, product, or service. A metrics dashboard accumulates a set of metrics associated with a decision. A dashboard allows the metric's customer to see, in a single collection, the relevant information. In this session, Linda Westfall will use the question, "are we done testing?" to illustrate an example of a metrics dashboard. She will also discuss each individual metric that makes up the dashboard and how they contribute information to help answer this question. These metrics will include testing...
Software Test Case Selection Based on Comparative Cost and Risk-Based Value
PreviewThe testing challenge is being able to select an efficient and effective set of test cases within the limited resources available for testing. The term “efficient” refers to the realization that the number of test cases is inherently limited by the testing resources available. The term “effective” refers to the ability of the selected test cases to find code defects that exist. Mark Gillenson addressed this problem by interviewing a set of testing professionals in a corporate partner and applying qualitative research techniques to reach their conclusions. This work was conducted in...
How I Spend More on Coffee Than DevOps
PreviewJacob Orshalick and focus.dev solutions are a development shop that provides Continuous Delivery for every client product they build. Even so, with the cloud solutions available today, they just don’t need a DevOps team. Maybe after this session you won’t need one either. Automating your infrastructure—you need an environment where your software product can be deployed for testing. Attendees will learn how to use AWS CloudFormation with AWS Serverless Access Model (SAM) to automate the creation of their deployment environment stack. They will see how they can then make changes to...