STARWEST 2017 - Performance Testing & Monitoring
Sunday, October 1
Mobile Application Testing (2-Day)
Mastering HP LoadRunner® for Performance Testing (2-Day)
Performance, Load, and Stress Testing (2-Day)
Monday, October 2
Measurement and Metrics for Test Management
PreviewTo be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them. Join Mike Sowers as he addresses common metrics—measures of product...
Influence Diagrams: A New Way to Understand Testing
PreviewInfluence diagrams provide a simple-to-create and easy-to-understand approach to address the complexities of real-life problems. As testers, we may want to find more bugs, but this may have an unintended consequence for developers. Developers now have more defects to debug, which affects their capacity to deliver new functionality. Isabel Evans has found that influence diagrams provide a means of understanding and managing the complexities of key interactions among testers, developers, and business stakeholders. In the past few years, Isabel has used influence diagrams as a tool to...
Tuesday, October 3
How to Break Software: Robustness Edition
Have you ever worked on a project where you felt testing was thorough and complete—all features were covered and all tests passed—yet in the first week in production the software had serious issues and problems? Join Dawn Haynes to learn how to inject robustness testing into your projects to uncover those issues before release. Robustness—an important and often overlooked area of testing—is the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. Dawn shows you how—by expanding basic tests and incorporating specific...
Wednesday, October 4
Rise of the Machines: Can Artificial Intelligence Terminate Manual Testing?
The state of the art in automated software testing is far from being a replacement for human-guided testing. There is more to testing than setting up preconditions, applying inputs, verifying outputs, and logging the results. Testing requires significant planning, exploring, learning, modeling, inferencing, experimenting, and more. Therefore, before we can truly automate testing, we must bridge the gap between the testing capabilities of humans and machines. Tariq King says that breakthroughs in artificial intelligence (AI) and machine learning (ML) are challenging our thinking about the...
Testing and DevOps: Organizations and Their Culture Must Change
The DevOps movement is here. Now, companies with mature systems are breaking down siloed IT departments and federating them into product development teams and departments. Testing and its practices are at the heart of these changes. Traditionally, development organizations have been filled with mostly manual testers and a limited number of automation and performance engineers. Adam Auerbach says this has to change. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay...
Blunders in Test Automation
In chess, the word blunder means a very bad move by someone who should know better. Even though functional test automation has been around for a long time, people still make some very bad moves and serious blunders. The most common misconception in automation is thinking that manual testing is the same as automated testing. And this thinking accounts for most of the blunders in system level test automation. Dorothy Graham takes us on a tour of these blunders, including: the Stable-Application Myth (you can’t start automating until the application is stable), Inside-the-Box Thinking (...
Service Virtualization: What, Who, When, and How
PreviewService virtualization provides many benefits for both development and test teams. For testers, service virtualization empowers them to work in parallel with their development counterparts and take control of their own schedules. They no longer have to wait for development to complete their work or to get access to a restricted system such as a mainframe or a third party API. Test teams can get the basic details from dev and/or use a sample request and response pair to create a virtual service themselves. With no need to wait on others to start testing, testing can start at...
Test Automation for Data-Centric Applications
Test automation, one of several key technical enablement practices, allows teams to be more successful in their agile journeys. Although there are many test practices and automation tools available for software development teams to leverage, few data-centric testing tools are targeted to data-related development and testing, leaving data warehousing and business intelligence teams thinking they can't possibly automate their tests. Cher Fox explores why test automation is important to agile data teams, discusses why they aren’t automating their tests today, and investigates the path to test...
What Does Continuous Testing Really Mean?
You may have heard the term “continuous testing” and thought it was just the DevOps flavor of the month … or that it isn’t part of DevOps … or that it isn’t for cloud-based applications. Marianne Hollier says that continuous testing means adopting the right set of automated tests along with service virtualization, which allows the team to simulate missing dependencies and to start testing earlier and more frequently. She shares how the right combination of best practices and tools can help software development and testing teams adopt a continuous testing approach. Since you can’t test...
Thursday, October 5
Elegant Dev and Test Processes for a More Civilized Age
Software engineering as a discipline has come a long way. For some teams, months-long cycles of dev-test-build-release have shrunk down to mere days—or even hours. In the fastest, leanest organizations, most testing happens in parallel with development as part of a slick, continuous integration pipeline. Come along with Melissa Benua as she explores just how quickly and safely a product can be released using ultramodern engineering technologies. All testers know how to file a bug, but more testers should know how to configure a continuous integration (CI) pipeline tool, how to wrest code...
Leverage Big Data and Analytics for Testing
PreviewSabermetrics turned the baseball world upside down by challenging decades-old measures of individual performance and their perceived linkage to team success. After cementing their legacy as the Lovable Losers for 108 years, the Chicago Cubs were able to leverage a data-driven approach to finally win a World Series. A high-school football coach, devoted to statistical analysis, has won three state championships—by never punting. Formula 1 racing teams collect staggering amounts of telemetry data from their race cars for the purpose of eking out seconds during the course of a race....
A Three-Tier Load Testing Program Saved Our Bacon
PreviewEnsuring a website will scale with excellent performance under peak levels of load is no easy task. Any number of problems can occur—from switch hardware failure to third party service outages, to a poor choice of algorithms or memory use in the code. Melissa Chawla describes Shutterfly's three-tiered approach to prevent site outages during peak load. First, check the development team's designs for scalability by holding performance design reviews for each project including identifying throughput requirements for all down-stream resources. Second, automate continuous load testing...
Integrate Your Test Automation Tools for More Power
PreviewWalk the Expo, and you will see all kinds of test automation tools. Some run scripts. Some communicate with the system under test. Some virtualize system components. Some do interesting things that you may never have considered. Yet, none gives you a complete recipe for testing your product and synthesizing the results. That is not their job. It's yours. Mike Duskis says an effective test automation program will reflect the unique nuances of your product and your business. However, unique nuances need not add up to radically different architectures. In fact, effective automation...
Use Model-Based Testing to Navigate the Software Forest
PreviewEven seemingly simple software systems can be a dense forest of intersecting logical pathways which may leave you wondering if your testing was robust enough. Traditional test cases are flawed since they only execute the pathways the tester considered at the time the test case was written, and they will execute the same way—every time and without variation. Jon Fetrow shows how, using model-based testing, you can create a map of your software forest and answer the question “Did you test enough?” Jon discusses the use of models to catch defects in the requirements and design phase...
Performance Testing in a Containerized World
PreviewContainer-based and microservices architectures have become the ideal setting for faster development cycles and more robust applications. As companies shift to these technologies, an integral part of the solution is the development of a continuous performance testing pipeline. Adopting a containerized architecture presents a variety of challenges. There are concerns about introducing additional overhead into the application performance. At the same time, a new paradigm implies defining new testing strategies, new metrics, and new tools that can better adapt to these architectures....
Get Ready for Cloud Testing
PreviewIn the past few years, deployment of applications in the cloud has become an industry standard. Meher Nori believes that it is very important for QA/testing organizations to understand the impact the cloud may have on them and prepare accordingly. The impact primarily involves a change in the testing strategy, and two items become very important. (1) Security testing and elasticity/scalability testing—new types of tests which previously were not so important—need to be created and executed once an application is hosted in the cloud. (2) Some traditional tests—availability testing...
On Becoming a Tester Who Codes
PreviewTo be a well-rounded and analytical tester, Adam Satterfield says you should learn to code. Learning to code empowers a tester to be more self-reliant and less dependent on someone else to write their test scripts, which can take valuable time from the critical and time-crunched testing process. Learning to code positions a tester for long-term career growth, opens up new professional opportunities, and gives them the perspective to be a better tester with a deeper understanding of what questions to ask and how to approach a meaningful testing plan to gain better insights. However...