STARWEST 2023 - Test Strategy, Planning, Metrics

To be most effective, leaders—including development and testing managers, ScrumMasters, product owners, and IT managers—need metrics to help direct their efforts and make informed recommendations about the software’s release readiness and associated risks. Because one important evaluation activity is to “measure” the quality of the software, the progress and results of both development and testing must be measured. Collecting, analyzing, and using metrics are complicated because developers and testers often are concerned that the metrics will be used against them. Join Jeff Pierce as he...

With the sudden rise of ChatGPT and large language models (LLMs), professionals have been attempting to use these types of tools to improve productivity. Building off prior momentum in AI for testing, software quality professionals are leveraging LLMs for creating tests, generating test scripts, automatically analyzing test results, and more. However, if LLM's are not fed good prompts describing the task that the AI is supposed to perform, their responses can be inaccurate and unreliable, thereby diminishing productivity gains. Join Tariq King as he teaches you how to craft high-quality AI...

Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and a bit of wishful thinking,...

Do you find it difficult to showcase the value of quality engineering initiatives to your business and C-suite? Are you curious about how to measure the true cost of defects on productivity, customer satisfaction, and profitability? Quality engineering contributes to multiple IT processes by identifying overlooked requirements, assessing code quality and deployment issues, and ensuring adequate test coverage prior to a release. However, knowing how to quantify, coalesce, manage, and report on defects is critical for success. Join Siva Kopparapu on a journey to discover how to shift the...

Maintaining API test cases across multiple environments, for different load levels, and types of tests (eg. smoke, functional, regression) traditionally required separate tests be written and maintained. Test maintenance is expensive. If you have three lower environments that tests are run for, then you usually have at least three different tests you are trying to keep in sync. If you have a new API added for a microservice, then you have to modify and validate across all three tests at a minimum. If you are running different load levels for build tests verses load tests, for example, that...

JavaScript code requires effective testing strategies to avoid bugs in production. Code coverage metrics alone can be misleading and don't necessarily indicate that the code is actually working as intended. It can be easy to get high code coverage metrics, but not actually test anything meaningfully. To address this problem, this session will explore the concept of JavaScript mutation testing. Mutation testing introduces small changes or "mutations" to the code and then expects tests to fail. This technique helps identify areas of the code that may need improvement and strengthens...

How do you go about testing the immutability of deployed code? What control does one have over a decentralized test environment? How does one overcome the complexity of smart contracts and associated potential vulnerabilities? What can be done to ensure seamless integration and interoperability between the blockchain application and external components? As a tester of blockchain applications, Gomathi Ramalingam has encountered and faced many of these challenges, which are inherently due to the unique nature of blockchain technology. She believes that having a good understanding of these...

In this exhaustive testing world, it can be daunting to measure multiple metrics to achieve the end goal of higher product quality. Measuring metrics can quickly become a daunting task that can pose several challenges on the time and efficiency front. What are these two simple metrics that can get you to your finish line? How can teams use these metrics to bring about a process change to set high quality standards? In this session, attendees will take away simple, but effective, ideas that can help their teams to clearly define their product quality goals, and attach a quantifiable...

Technical implementations often span across multiple companies. How can a test architect drive better results in quality and continuous integration in their own organization while working through challenging, complex, multi-company implementations? By creating a holistic test strategy, including evaluating major areas of testing early in the process, a test architect can enable not only their own organization, but all of the organizations involved in the program. Learn specific techniques at every stage including identifying commonalities in test environments, understanding implications of...

#A11y, aka Accessibility is a hot topic nowadays. It is not only an important business priority for each digitally transformed company to increase user retention and sales revenue, but it also ensures that people with disabilities can use any digital product without any barriers. Many organizations started to focus on digital accessibility by investing tremendous amounts of budget in vendor or user testing post product deployment. However, quite a portion of this budget can be eliminated by educating their product teams, especially engineers to adopt accessibility automated test in their...

New approaches in robotics are changing the way companies build and test software as it becomes increasingly device dependent. Companies like Apple, Google, and T-Mobile already rely on in-house robotics fleets to automate manual testing of mobile apps on devices. And that is just the beginning. Robots can perform all of the expected actions that a human would make on a device, including finger swiping, switching between various apps, toggling the notification center, and more. Using computer vision, a field of AI that teaches computers how to parse digital images and videos, robots can...

Many times accessibility activities occur late in a project’s release when it takes longer to figure out problems and costs more to fix them. Typically all you can do at this point is patch the problems rather than fix them properly. CGI Federal approaches accessibility with an agile mindset, integrating it throughout the iterative design, user story creation, development, and testing efforts. During this session, Pavani Gonuguntla and Stephen Omojuyigbe will focus on case studies and recommended best practices from where they worked with management and cross-functional teams to get...