STARWEST 2016 - Project Manager

When we discover—often late in an automation effort—that the automated tests are cumbersome and costly to maintain, we often view this as a technical problem for the automator to solve. However, an often-overlooked cause is the role that testers who designed these tests play in making automation scalable and maintainable. In this interactive tutorial for both testers and automation engineers, Hans Buwalda explores how better test designs will result in much improved test automation and can make the difference between automation success and failure. See why...

As test managers and test professionals, we're constantly dealing with fragile egos, highly-charged situations, and pressured people playing a high-stakes game under conditions of massive uncertainty. We're often the bearers of bad news and are sometimes perceived as critics, activating people's primal fear of being judged. The concept of emotional intelligence (EI), popularized by Harvard psychologist and science writer Daniel Goleman, has much to offer test managers and testers. Key EI skills include self-awareness, self-management, social awareness, and...

Testers often participate in requirement reviews, review requirements for testability, and then use requirements—in the form of user stories—as the basis of test cases. In an agile environment, most testers contribute to the development of user stories and acceptance criteria. Erik van Veenendaal says that unfortunately many testers have little knowledge or skill in requirements engineering. What level of quality and detail is realistic to expect for requirements and user stories? What does testability really mean? How can testers help improve requirements? Erik...

In the tradition of James Whittaker’s book series, How to Break Software, Jon Hagar applies the testing “attack” concept to the domain of mobile and embedded software systems. First, Jon defines the environments of mobile and embedded software. He then examines the issues of software product failures caused by defects found in these types of software. Next, Jon shares a set of attacks against mobile and embedded software based on common modes of failure that teams can direct against their software. Like different kinds of software design patterns,...

Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and...

Within software and test engineering, two new disciplines—Chaos Engineering and Intuition Engineering—provide avenues to address trust in complex systems. Chaos Engineering is a methodology for test engineers to validate a system’s behavior and establish empirical trust metrics. Intuition Engineering provides new interfaces to navigate complexity, filling in gaps in understanding that classical methods cannot address.

Casey Rosenthal explains how these disciplines are used to improve quality on Netflix.com, one of the largest scaled deployments on the...

With billions of devices containing new software connected to the Internet, the Internet of Things (IoT) is poised to become the next growth area for software development and testing. Although many traditional test techniques and strategies remain viable, challenges in IoT testing include huge amounts of data, multiple communication channels, device protocols, resource limitations (battery or memory), addressing sensors and controllers, cloud-hardware-device integration, and security concerns. Jon Hagar says that for IoT testers to be successful, they must...

Are you tired of flaky automation scripts that fail without apparent cause? Are most script failures due to bugs in the test script rather than bugs in the product? Do you simply want to get more out of your automation framework? Many testers and organizations are frustrated with the high cost and low return of their automation frameworks. Scott Miles shares a range of techniques to convert your existing framework from brittle to adaptive. Learn how to improve the quality of your automation framework while reducing the required maintenance. Adaptive automation...

As a newly promoted quality assurance manager, Willie Smith assumed responsibility for a large application that allows UPS customers to ship packages worldwide. His team was distributed worldwide and had varying levels of automation experience, application knowledge, and QA processes in place. With the next deployment date only a few months away, Willie found that the testing side of the project was in trouble. To align the test team with project and company goals, he had to develop virtual training plans and implement new practices to deal with the host of...

Much of the success of agile adoptions is due to the automated testing approach used in agile projects. Because many of these techniques were pioneered in the development of web applications, it can be difficult to see how these techniques can be leveraged for a project where software is being built for an embedded or Internet of Things (IoT) application. Thomas Stiehm describes ways to leverage agile testing techniques for embedded systems. Whether you are building a medical device, embedded controller, or IoT device, learn how to leverage these testing...

Cheaper … faster … smarter—words familiar to modern QA test team leaders everywhere. Although many organizations recognize that a globally-distributed QA model will achieve some of these objectives, it comes with obvious challenges. Lack of immediate communication, not knowing what the other is doing, and even cultural clashes can result in a lack of trust within the team, an absence of team spirit, and high attrition. Elizabeth Wisdom identifies the key to mitigating the risks associated with each challenge. And it’s not a technical solution (like advanced...

In performance testing we run a suite of tests, modify the system in an attempt to improve its performance, and then repeat the tests. We want to know if the performance is “better.” However, no single performance measure exists; we must consider and evaluate many variables. Although viewing a full histogram of various test runs is more helpful, quantifying the change between the tests becomes the challenge. Parvez Ahammad introduces the relative divergence index (RDI), a multidimensional statistical method to compare differences between two sets of performance...

Congratulations on making the jump to testing on real mobile devices. But are you also testing under the actual conditions experienced by your users? Do your tests include scenarios of varying location, interrupts (call, SMS), backgrounds, and varying networks in motion? Join Carlo Cadet as he describes modeling user condition testing scenarios with the same personas used by marketing teams. Carlo focuses on three key areas of user-condition testing: 1) defining personas, 2) implementing tests, and 3) analyzing results. He demonstrates how user-condition testing...

Those of us in the business analyst role are encouraged to communicate with clients, while other roles in an organization may not have the same opportunity. However, talking to clients must not be limited to business analysts. Jyothi Rangaiah believes that testers need to talk to the clients, too, because by the time the product information trickles down from a business analyst to a programmer to a tester, valuable information has been lost. Testers are often blamed for defects in the product, but this can be avoided by fostering effective communication between...

Although the idea of doing performance testing throughout the software lifecycle sounds simple enough, as soon as you try to combine the concepts of “always testing” (in dev, pre-prod, and production) with “limited time and resources” and throw in the word “comprehensive,” the challenges can be monumental. Quickly the “how” of it emerges as the most important question—and one worth focusing on. Brad Stoner tackles this topic by explaining how he has been able to solve this seemingly impossible puzzle by applying various approaches such as early and often,...

Stories help us learn. They can be fun or scary, exciting or relaxing. People worldwide tell and listen to stories. We access them through books, film, TV, and computers. But direct, face-to-face storytelling is still a powerful experience. When Isabel Evans was young, there was a program on the radio called Listen with Mother. For fifteen minutes, mothers and children across the land would sit and listen to a story. Join Isabel and become your younger child, bring your testing parent, and listen to her stories. In fifteen minute sections, Isabel recounts stories...

Big Data, one of the latest buzzwords in our industry, involves working with petabytes of data captured by various systems and making sense of that data in some way. Maryam Umar has found that testing systems like Hadoop is very challenging because of the frequency with which the data arrives in the system, the number of jobs that run to process that data, and the interdependency of the data. Maryam describes some of the projects at Hotels.com which involve identifying multiple users and using that data to make recommendations of hotels. Testing this is fairly...

Security, data privacy, reliability, and regulatory compliance are critical factors when evaluating whether to move business applications from in-house, client-hosted environments to a cloud platform. Quality assurance plays a vital role in ensuring that the appropriate level of risk assessment, verification, and validation takes place to ensure business continuity during the migration to a new cloud platform. Vandana Viswanathan shares an introduction to the five components of a sound cloud application validation strategy: cloud provider qualification process,...