Explore all that STARWEST has to offer in the conference schedule. Filter by job function or interest area to find the topics that matter the most to you.
Large-scale and complex testing projects can stress the testing and automation practices we have learned through the years, resulting in less than optimal outcomes. However, a number of innovative ideas and concepts are emerging to better support industrial-strength testing for big projects. Hans Buwalda shares his experiences and presents strategies for organizing and managing testing on large projects. Learn how to design tests specifically for automation, including how to incorporate keyword testing and other techniques.
You're under tight time pressure and have barely enough information to proceed with testing. How do you test quickly and inexpensively, yet still produce informative, credible, and accountable results? Rapid Software Testing, adopted by context-driven testers worldwide, offers a field-proven answer to this all-too-common dilemma. In this one-day sampler of the approach, Michael Bolton introduces you to the skills and practice of Rapid Software Testing through stories, discussions, and "minds-on" exercises that simulate important aspects of real testing problems.
Whether you are new to testing or looking for a better way to organize your test practices, understanding risk is essential to successful testing. Dale Perry describes a general risk-based framework—applicable to any development lifecycle model—to help you make critical testing decisions earlier and with more confidence. Learn how to focus your testing effort, what elements to test, and how to organize test designs and documentation. Review the fundamentals of risk identification, analysis, and the role testing plays in risk mitigation.
Selenium is an open source automation tool for test driving browser-based applications. WebDriver, the newly-introduced API for Selenium against which tests are written in Java, contains classes including ChromeDriver, AndroidDriver, and iPhoneDriver. Sometimes test authors find the API daunting and their initial automation code brittle and poorly structured. In this introduction, Alan Richardson provides hints and tips gained from his years of experience both using WebDriver and helping others improve their use of the tool.
Have you ever needed a way to measure your leadership IQ? Or been in a performance review where the majority of time was spent discussing your need to improve as a leader? If you have ever wondered what your core leadership competencies are and how to build on and improve them, Jennifer Bonine shares a toolkit to help you do just that.
The practice of agile software development requires a clear understanding of business needs. Misunderstanding requirements causes waste, slipped schedules, and mistrust within the organization. Jared Richardson shows how good acceptance tests can reduce misunderstanding of requirements. A testable requirement provides a single source that serves as the analysis document, acceptance criteria, regression test suite, and progress-tracker for any given feature. Jared explores the creation, evaluation, and use of testable requirements by the business and developers.
You’ve “gone agile” and have been relatively successful. So, how do you know how well your team is really doing? And how do you continuously improve your practices? When things get rocky, how do you handle the challenges without reverting to old habits? You realize that the path to high-performance agile testing isn’t easy or quick. It also helps to have a guide. So consider this workshop your guide to ongoing, improved, and sustained high-performance. Join Bob Galen and Mary Thorn as they share lessons from their most successful agile testing transitions.
As test managers and test professionals we can have an enormous emotional impact on others. We're constantly dealing with fragile egos, highly charged situations, and pressured people playing a high-stakes game under conditions of massive uncertainty. We're often the bearers of bad news and are sometimes perceived as critics, activating people's primal fear of being judged. Emotional intelligence (EI), the concept popularized by Harvard psychologist and science writer Daniel Goleman, has much to offer test managers and testers.
In response to increasing market demand for high performance applications, many organizations implement performance testing projects, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. With specific examples from recent client implementations, Scott Barber shares the fundamentals of implementing T4APM™, a simple and universal approach that is valuable independently or as an extension of existing performance testing programs.
To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics are complicated because many developers and testers are concerned that the metrics will be used against them.
Testers want to be responsible and professional. However, they often come under pressure to comply with rules, standards, and processes that aren't always helpful. It's the price of keeping your auditors happy. But do you really know what auditors want? Are they all simply rule-obsessed, pedantic “little dictators”? James Christie shows why good auditors worry about risk—not rules. They want to explain the important risks to the people who lose sleep over them. James explains auditors' and regulators' attitudes toward risk and evidence.
Automating system level test execution can result in many problems. It is surprising to find that many people encounter the same problems yet are unaware of common solutions that worked well for others. These problem/solution pairs are called “patterns.” Seretta Gamba recognized the commonality of these test automation issues and their solutions and, together with Mark Fewster, has organized them into Test Automation Patterns. Although unit test patterns are well known, Seretta’s and Mark’s patterns address more general issues.
Jon Bach has been in testing for twenty years. Is testing in 2014 different from testing in 1994? If you’ve been in the business that long, maybe you’ve seen it move from a little bit of automation and tooling to almost total automation and tooling. Maybe you’ve seen lab setups go from hours of loading OS images on “boat anchor” boxes to virtual, on-demand, scalable cloud provisioning in seconds. Maybe you think testing is dead because we live in a DevOps world where it’s good enough to run a happy path checklist.
The reality is that technology is complicated. As testers, we are challenged with complicated problems that need solving. Andy Glover presents a hands-on workshop that describes a new way of looking at testing problems and ideas. Andy demonstrates how thinking with pictures can help testers discover and develop new ideas, solve problems in unexpected ways, and dramatically improve their ability to share their insights with others.
Innovation is a word frequently tossed around in organizations today. The standard clichés are do more with less and be creative. Companies want to be innovative but often struggle with how to define, implement, prioritize, and track their innovation efforts. Using the Innovation to Types model, Jennifer Bonine will help you transform your thinking regarding innovation and understand if your team and company goals match their innovation efforts. Learn how to classify your activities as "core" (to the business) or "context" (essential, but non-revenue generating).
Exploratory testing is an approach to testing that emphasizes the freedom and responsibility of testers to continually optimize the value of their work. Exploratory testing is the process of three mutually supportive activities—learning, test design, and test execution—done in parallel. With skill and practice, exploratory testers typically uncover an order of magnitude more problems than when the same amount of effort is spent on procedurally-scripted testing.
Critical thinking is the kind of thinking that specifically looks for problems and mistakes. Regular people don't do a lot of it. However, if you want to be a great tester, you need to be a great critical thinker. Critically thinking testers save projects from dangerous assumptions and ultimately from disasters. The good news is that critical thinking is not just innate intelligence or a talent—it's a learnable and improvable skill you can master.
Many organizations invest substantial time and effort in test automation but do not achieve the significant returns they expected. Some blame the tool they used; others conclude test automation just doesn't work in their situation. The truth, however, is often very different. These organizations are typically doing many of the right things but they are not addressing key issues that are vital to long term test automation success.
As testers, we know that we can define many more test cases than we will ever have time to design, execute, and report. The key problem in testing is choosing a small, “smart” subset from the almost infinite number of tests available that will find a large percentage of the defects. Join Lee Copeland to discover how to design test cases using formal black-box techniques, including equivalence class testing, boundary value testing, decision tables, and state-transition diagrams. Explore examples of each of these techniques in action. Don’t just pick test cases at random.
The nature of exploration, coupled with the ability of testers to rapidly apply their skills and experience, make exploratory testing a widely used test approach—especially when time is short. Unfortunately, exploratory testing often is dismissed by project managers who assume that it is not reproducible, measurable, or accountable. If you have these concerns, you may find a solution in a technique called session-based test management (SBTM), developed by Jon Bach and his brother James to specifically address these issues.
Cloud computing is rapidly changing the way systems are developed, tested, and deployed. New system hosting capabilities—software as a service (SaaS), platform as a service (PaaS), infrastructure as a service (IaaS)—are forcing us to review and revise our testing processes. At the same time, cloud computing is affording us opportunities to employ new test tooling solutions, which we call testing as a service (TaaS). In this technical session, Martin Pol and Jeroen Mengerink focus on testing SaaS systems, describing relevant IaaS and PaaS capabilities along the way.
Running a test project can be a challenge. Running a number of test projects as part of a portfolio can be even more challenging. However, most challenging of all can be running a group of projects in which every project needs to merge at a single end point. Geoff Horne considers: How does a program test manager (PTM) slice up the testing work packages and then group them by “like” types into discrete projects? How does the PTM determine the best approach for each project while maintaining the most advantageous approach for the overall program?
It’s not enough to verify that software conforms to requirements by passing established acceptance tests. Successful software products engage, entertain, and support the users' experience. Goals vary from project to project, but no matter how robust and reliable your software is, if your users do not embrace it, business can slip from your hands. Rob Sabourin shares how to elicit effective usability requirements with techniques such as storyboarding and task analysis.
In automation, we often use several different tools that are not well integrated. These tools have been developed or acquired over time with little consideration of an overall plan or architecture, and without considering the need for integration. As a result, both efficiency and effectiveness suffer, and additional time and money are spent. Ensuring that tools we currently have, or the tools we develop or acquire in the future, work well with other application lifecycle tools is critical to our testing team’s success.
In many organizations, agile development processes are driving the pursuit of faster software releases, which has spawned a set of new practices called DevOps. DevOps stresses communications and integration between development and operations, including continuous integration, continuous delivery, and rapid deployments. Because DevOps practices require confidence that changes made to the code base will function as expected.
The key to successful testing is effective and timely planning. Rick Craig introduces proven test planning methods and techniques, including the Master Test Plan and level-specific test plans for acceptance, system, integration, and unit testing. Rick explains how to customize an IEEE-829-style test plan and test summary report to fit your organization’s needs. Learn how to manage test activities, estimate test efforts, and achieve buy-in. Discover a practical risk analysis technique to prioritize your testing and become more effective with limited resources.
Many software systems are required to process huge combinations of input data, all of which deserve to be tested. Since we rarely have time to create and execute test cases for all combinations, the fundamental problem in testing is how to choose a reasonably-sized subset that will find a large percentage of the defects and can be performed within the limited time and budget available. Pairwise testing, the most effective test design technique to deal with this problem, is unfortunately not understood by many testers.
Today’s software applications are often security critical, making security testing an essential part of a software quality program. Unfortunately, most testers have not been taught how to effectively test the security of the software applications they validate. Join Jeff Payne as he shares what you need to know to integrate effective security testing into your everyday software testing activities. Learn how software vulnerabilities are introduced into code and exploited by hackers. Discover how to define and validate security requirements.
You have just been assigned a new testing project. Where do you start? How do you develop a plan and begin testing? How will you report on your progress? Paul Holland shares new test project approaches that enable you to plan, test, and report effectively. Paul demonstrates ideas, based on the Heuristic Software Test Model from Rapid Software Testing, that can be directly applied or adapted to your environment. In this hands-on tutorial, you’ll be given a product to test.
With the adoption of agile practices in many organizations, the test automation landscape has changed. Bob Galen explores current disruptors to traditional automation strategies, and discusses relevant and current adjustments you need to make when developing your automation business case. Open source tools are becoming incredibly viable and beat their commercial equivalents in many ways―not only in cost, but also in functionality, creativity, evolutionary speed, and developer acceptance. Agile methods have fundamentally challenged our traditional automation strategies.
In both agile and traditional projects, keyword-driven testing—when done correctly—has proven to be a powerful way to attain a high level of automation. Many testing organizations use keyword-driven testing but aren't realizing the full benefits of scalability and maintainability that are essential to keep up with the demands of testing today's software. Hans Buwalda describes the keyword approach, and how you use it to can meet the very aggressive goal that he calls the "5 percent challenge"―automate 95 percent of your tests with no more than 5 percent of your total testing effort.
Data warehouses have become a popular mechanism for collecting, organizing, and making information readily available for strategic decision making. The ability to review historical trends and monitor near real-time operational data has become a key competitive advantage for many organizations. Yet the methods for assuring the quality of these valuable assets are quite different from those of transactional systems. Ensuring that the appropriate testing is performed is a major challenge for many enterprises.
Communication is at the heart of our profession. No matter how advanced our testing capabilities are, if we can’t convey our concerns in ways that connect with key members of the project team, our contribution is likely to be ignored. Because we act solely in an advisory capacity, rather than being in command, our power to exert influence is almost entirely based on our communication skills. With people suffering information overload and deluged with emails, it is more important than ever that we craft succinct and effective messages, using a range of communication modalities.
Anyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The key to good estimates is to understand the primary variables, compare them to known standards, and normalize the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when very little is known about the project and what is known is constantly changing.
Kick off the conference with a welcome reception!Mingle with experts and colleagues while enjoying complimentary food and beverages.
Are you a new star speaker or aspiring to be one in the future? Join us at this workshop on making effective conference presentations. Learn the secrets of developing content, identifying the Big Message, preparing slides with just the right words and images, presenting your message, handling questions from the audience, and being ready when things go wrong. Lee Copeland, a professional speaker since birth, shares ideas that will help you be a better speaker, no matter what the occasion.
Join this unique breakfast bonus session featuring speakers Kannan Subramaniam, VP, TQM, Comcast, and Premkumar Balasubramanian, Sr. Director, E&A, Cognizant Technologies.
In the past, developers knew every line of code in their applications. They designed it, wrote it, tested it, and controlled it. Today’s applications are far different. Rather than written, they are often assembled―from program language libraries, third-party frameworks, encapsulated web services, and even entire external systems—and glued together with small amounts of code. Before your organization committed to using these external pieces of software, were testers part of the evaluation process? Was the software thoroughly tested before betting your organization’s success on it?
In his journeys, Bob Galen has discovered that testing takes on many forms. Some organizations have no automated tests and struggle to run massive manual regression tests within very short iterative releases. Other organizations are going “all in”―writing thousands of acceptance tests in Gherkin. The resulting imbalance in their testing approaches undermines an organization’s efficiency, effectiveness, and delivery nimbleness. Bob shares ideas to bring balance to testing. He explores the choices: manual vs. automated testing, designed and scripted test cases vs.
Network with colleagues and speakers in the Expo...Read More
This presentation will be streaming LIVE at the STARWEST Virtual Conference.Click here to register for free and for more information on this session.
• Why the quantity of tests that you have is far less important than the quality of tests• How you can leverage crowdsourcing to help find bugs that your team would never discover• How to write automated tests in your own lingo so they bridge gaps across all phases of development
• How big companies select test tools• See how it fits together in an agile environment• Examples: desktop browsers, iOS, and Android native apps
When trying to improve the development process, agile and lean transformations often start by focusing on engineering. Product management and development get a lot of attention; however, tester is not one of the defined Scrum roles. Despite the attention given to automated tests in agile, many transformations seem lost in knowing how to engage testers—and testers struggle to find their place.
Rob Sabourin has discovered testing lessons in the Simpsons, the Looney Tunes gang, Great Detectives, Dr. Seuss, and other unlikely places, but this year he journeys to Sesame Street. Sesame Street teaches basic life skills in a safe, entertaining, memorable style. Rob uses them to solve stubborn technical, management, and people-related testing problems. Oscar the Grouch guides us through failure mode analysis. Ernie and Bert help us tackle problems from different perspectives. Big Bird and Mr. Snuffleupagus teach about persistence, rhetoric, and bug advocacy.
The role of the Product Owner in Scrum is only vaguely defined—owning the Product Backlog and representing the “customer.” In many organizations, Product Owners go it alone, trying their best to represent business needs to their teams.
Pete Walen is not going to tell you how to be a good test manager. Instead, Pete shares ideas on becoming a true leader. While some managers certainly are leaders, testers of all varieties and experience levels can become leaders. Developing technical leadership skills, regardless of job title, involves overcoming our own uncertainties, self-doubts, and perceptions. Learning to foster relationships while perfecting our craft is a challenge for everyone, particularly when others look to us to be an expert—even when we don’t feel like one.
Currently, 2.4 billion people use the Internet, and about 10 percent of the world’s population has some form of disability. This means millions of potential users will have difficulty accessing the Internet. Thus, accessibility testing should not be ignored. Anish Krishnan discusses the importance of accessibility testing, reasons for considering accessibility issues while designing, and international Web accessibility laws.
Testers face common challenges in automation. Unfortunately, these challenges often lead to subsequent failures. Jim Trentadue explains a variety of automation perceptions and myths―the perception that a significant increase in time and people is needed to implement automation; the myth that, once automation is achieved, testers will not be needed; the myth that scripted automation will serve all the testing needs for an application; the perception that developers and testers can add automation to a project without additional time, resources, or training; the belief that anyone can implement automation. The testing organization must ramp up quickly on the test automation process and the prep-work analysis that needs to be done including when to start, how to structure the tests, and what system to start with. Learn how to respond to these common challenges by developing a solid business case for increased automation adoption by engaging manual testers in the testing organization, being technology agnostic, and stabilizing test scripts regardless of applications changes.
• Creating requirement traceability• Test case reuse• Leveraging results in test planning• Increasing efficiency for test execution
The first step in delivering quality mobile applications is finding a way to efficiently manage the Big 4 of mobile testing: apps, automation, devices and security. This is increasingly important as mobile continues to bring new testing challenges to enterprise test and QA teams.
• What’s and Why’s of QA Metrics—Significance of QA metrics in driving “End-to-End Quality of IT Implementation”• How’s of QA Metrics—Leverage “QA Analytics” to predict outcomes of IT implementation• Build Quality rather than Test for Quality—Drive discipline throughout IT with Shift Left QA and Shift Right QA metrics
• Form a stronger strategy that considers sufficient device and OS coverage a crucial component• Improve velocity and support continuous, unattended testing by implementing a reliable automated testing system• Approach test automation with re-use in mind
An underwear advertisement in 1985 featured the dedicated and thorough Inspector 12 saying, “They don't say Hanes until I say they say Hanes.” Historically, software testers have been called on to perform a similar role―preventing defective products from reaching customers. However, software development is not underwear manufacturing. The specifications are less clear and the acceptance criteria more complex. Why then do organizations continue to place acceptance decisions in the hands of testers? Because they lack the information required to make a sound decision.
Many development and test organizations must work within the confines of compressed release cycles, various agile methodologies, and cloud and mobile environments for their business applications. So, how can test organizations keep up with the pace of development and increase the quality of their applications under test? Clint Sprauve describes how service virtualization and network virtualization can help your team improve speed and increase quality.
If we know that good co-worker relationships can positively impact our success, why don’t we take a systematic approach to relationship building? Martin Nilsson shares how building personal relationships has helped develop his personal competency. Even though Martin’s technical skills are high, his greatest successes as a tester have come from his ability to build relationships. He shares how a focused effort at building rapport resulted in greater cooperation. When he was mistaken for a test lead during a project, Martin learned that having coffee with someone can trump an email.
In organizations adopting the newest development approaches, classical test process improvement models no longer fit. A more flexible approach is required today. Solutions like SOA, virtualization, web technology, cloud computing, mobile, and the application of social media have changed the IT landscape. In addition, we are innovating the way we develop, test, and manage. Many organizations are moving toward a combination of agile/Scrum, context-driven testing, continuous integration and delivery, DevOps, and TestOps. Effective test automation has become a prerequisite for success.
One of the Agile Principles states that working software is the primary measurement of success―generally measured by the level of customer satisfaction. So, how do you measure “customer satisfaction” when it is based on successful surgical outcomes? Join John Pasko as he takes you through a case study of the design, development, testing, and release of a complex system—integrating embedded software with hardware—for a surgical product which met stringent FDA standards and regulations.
Developing high-quality software requires effective communication among various project stakeholders. Business analysts must elicit customer needs and capture them as requirements, which developers then transform into working software. Software test engineers collaborate with business analysts, domain experts, developers, and other testers to validate whether the software meets the customer’s expectations. Misunderstandings between different stakeholders can introduce defects into software, reducing its overall quality and threatening the project’s success.
• What is the percentage of budget allocated for the testing function?• How is the testing function structured within your organization?• Does your organization currently tests mobile applications and devices?• What portion of your testing work is carried out in Agile projects?• What portion of your testing occurs in Cloud-based test environment?
• Transforming the traditional data industry with Big Data• The role of QA in the new era of Big Data world• Diving into Big Data with an effective QA and testing strategy
What do test managers do? In traditional organizations, they assign people to projects, oversee the testers’ progress, provide feedback, and perhaps offer coaching to people who want it. Test managers are the go-to people when you don't know how to do something—not because they know, but because they know who does know. How does that change with a transition to agile? Do we still need test managers? As one who has successfully made the transition from traditional to agile test manager, Mary Thorn shares keys to the transition.
At Walt Disney Parks and Resorts Technology, we provide the applications and infrastructure our online guests use to plan, book, explore, and enjoy their stay at our parks and resorts. With millions of page views per day and a multi-billion dollar ecommerce booking engine, we face a unique set of challenges. Join Les Honniball for insights into how they work with Product Owners and development teams to design tests, both manual and automated for these challenges.
As organizations continue to adopt agile methodologies, testers are getting involved earlier in product testing. They need tools that empower them to manage varied test automation needs for web services, web APIs, and web and mobile applications. Open source solutions are available in abundance. However, most of these solutions are independent and not integrated, significantly increasing the tester’s work around test automation development. Ongoing test automation suite evolution and building a robust regression test suite have become cumbersome.
Many organizations adopt agile software development processes, yet they do not adopt agile testing processes. Then they fall into the trap of having development sprints that are just a set of mini-waterfall cycles. Some software developers still feel they can work more quickly if they let QA test after code is completed. Jeanne Schmidt identifies simple ways to get your team to adopt agile testing methods. Embracing agile testing requires you to change processes, responsibilities, and team organization.
Ever feel like your manager, development manager, product manager, product owner, or (you fill in the blank) is not listening to you or your team? Are you struggling to make an impact with your messages? Are you “pushing a wet rope uphill” in championing product quality? Are you talking, but no one is listening? Mike Sowers shares practical examples of how to more effectively speak like a test manager and offers concrete advice based on his experiences in the technology, financial, transportation, and professional services sectors.
Outsourcing test services are all the rage today. But are they really faster, better, and cheaper? Shelley Rueger shares how you can improve the efficiency and effectiveness of your test process using a third-party test service. She provides guidance on how to determine if your product is a good candidate for testing services, how to select the right vendor, and how to avoid common pitfalls. Shelley discusses her team's experience as they made the transition from in-house testing to using external testing services. She addresses questions including: When should you outsource testing?
Throughout the years, Lightning Talks have been a popular part of the STAR conferences. If you’re not familiar with the concept, Lightning Talks consists of a series of five-minute talks by different speakers within one presentation period. Lightning Talks are the opportunity for speakers to deliver their single biggest bang-for-the-buck idea in a rapid-fire presentation. And now, lightning has struck the STAR keynotes.
Like millions of other Americans, Ben Simo visited HealthCare.gov in search of health insurance and found a frustratingly buggy website that was failing to fulfill its purpose―to educate people on the new health insurance law and help them purchase health insurance. After failing to create an account, Ben put on his tester hat and turned on his web developer tools. In addition to many functional and performance issues, Ben soon discovered a chain of security vulnerabilities that exposed users to unnecessary risk. Finding HealthCare.gov customer service unequipped to receive reports of security vulnerabilities, he blogged his discoveries, spawning a storm of public attention which hailed Ben as a “web expert,” “methodical IT guru," “folk hero”—and “not too bright.” His reports even came up in congressional hearings, where the Secretary of Health and Human Services referred to Ben as “a sort of skilled hacker.” Ben’s reports helped bring attention to problems that suggested a systematic lack of care and understanding of system design and information security. Join Ben as he shares his experience, the issues he found, and lessons testers can learn from HealthCare.gov.
• Essentially a four-layered platform: Development (construction layer), Build (release layer), Testing layer (Mindtree’s Anywhere Automation Solution + Requirements for automation), and Analytics (reporting layer)• Offers an integrated environment for all testing activities and enhances test coverage—Continuous Integration Validation, Automated Environment Provisioning, Requirements Automation & Execution, Dynamic Code coverage, and Reporting• Shrinks application testing lifecycle time by 30 percent, thus accelerating time to market
• Learn how to use any tool you want in even the most complicated and rigorous environments all while increasing collaboration and visibility across existing assets.• See how Borland Connect, the industry’s first integration technology that brings change management to any system of record, can link requirements, development artifacts, popular distributed version control systems like Git and SVN, and quality tools with little to no disruption.
To aid development in the mobile and smartphone app world, testers must do more than simply test against requirements; they should include attack-based testing to find common errors. In the tradition of James Whittaker’s How to Break Software books, Jon Hagar applies the testing “attack” concept to mobile app software, defines the domain of mobile app software, and examines common industry patterns of product failures. Jon then shares a set of ten software test attacks, based on the most common modes of failure in native, web-based, and hybrid apps.
DevOps is gaining popularity as a way to quickly and successfully deploy new software. With all the emphasis on deployment, software quality can sometimes be overlooked. In order to understand how DevOps and software testing mesh, Jeff Payne demonstrates a fully implemented continuous integration/continuous delivery (CI/CD) stack. After describing the internals of how CI/CD works, Jeff identifies the touch points in the stack that are important for testing organizations.
For thousands of years, human language has provided us with beautiful and complex ways of sharing important ideas. At the same time, language can derail attempts to communicate even the most basic pieces of critical information. We testers are the heralds of vast amounts of data, and it is our responsibility to use that data wisely—or not at all.
Test automation is difficult to get right. Working under FDA regulation presents its own challenges. Combining the two is a scary proposition because the FDA requires—and will scrutinize—the validation of any test automation used. Despite this, working in a regulated environment only magnifies the value of test automation. Aware that automation is a driver of quality and consistency, the FDA welcomes automated tests as part of an audit submission. The key to success is demonstrating quality in a way that the FDA recognizes.
The dictionary defines a rainmaker as “an executive (or lawyer) in the unsentimental world of business with an exceptional ability to attract clients, use political connections, increase profits, etc.” Simply put, a rainmaker is someone who gets things done. Is this relevant to testing? Absolutely! It is too easy to get stuck in the status quo and to avoid trying something new because everything works well as it is. But what we do can always be made better.
The cloud is all about redundancy and fault tolerance. Since no single component can guarantee 100 percent uptime, we have to design architectures where individual components can fail without affecting the availability of the entire system. But just designing a fault tolerant architecture is not enough. We have to constantly test our ability to actually survive these “once in a blue moon” failures. And the best way is to test in an environment that matches production as closely as possible or, ideally, actually in production.
• Why does automation fail to deliver the promised ROI?• Understanding the pitfalls and how BDD can help• Open source automation challenges: What should you fix with a framework? • Sneak preview of Qmetry Automation Studio—makes BDD effective through open source tools like Selenium
Learn how:• To leverage scriptless test automation within the HP ALM/UFT toolset to auto-create and auto-maintain tests for both desktop and mobile applications—without any technical scripting or coding• A true data-driven architecture can help you increase test coverage of your business processes and user stories, while significantly reducing the number of test cases you need to create and maintain• Framework extensibility can be used to support a much broader range of applications and environments than with native HP QTP/UFT alone
Are you employing your offshore test team to its best advantage—gaining the cost savings and test coverage you expected? Unless correct management methodologies are in place, you will lose rather than gain both time and money with internationally-distributed testers. If you are thinking you can go offshore with minimal effort, think again. Distributed test leadership and management issues apply when working with third-party firms, a subsidiary, or even your own employees. Don’t let unrealistic expectations impact your career or your company’s goals.
When you start writing automation for your projects, you quickly realize that you need to organize and design the code. You will write far more than “test” code; you also will write abstraction code because you want to make tests easier to read and maintain. But how do you design all this code? How do you organize and structure it? Should you use a domain specific language? Should you go keyword driven or use Gherkin? Should you use page objects with POJO or Factories? Do you create DOM level abstractions? Where do domain models fit in?
Do you consider the performance impact when adding a new JavaScript file, a single AJAX call, or a new database query to your app? Negligible, you say? I disagree―and so should you. Andreas Grabner demonstrates the severe impact small changes can have on performance and scalability. Many small changes will have an even bigger impact so it is important to catch them early.
Imagine you’re a test manager starting a new assignment. On the first day of work, you’re presented with a list of metrics you are to report. Soon, you realize that most of the metrics are not really connected to what should be measured. Or, consider the situation where you’re told that there is no value collecting metrics because “we’re agile.” In either situation, what would be your next step be? Join Pablo Garcia as he shares his experience with the dangers of poor metrics.
Today, an alarming 65 percent of mobile apps—more than 1.3 million—have a 1-star rating or less. Why? The majority of development organizations have neither the right processes nor access to the devices required to properly test mobile applications. If not addressed, these deficiencies will have a major impact on the quality of the apps the organization develops. In addition, users are intolerant of problems and quick to switch to competing apps. Costa Avradopoulos explores how to address the unique challenges of mobile testing, starting with adopting the right test strategy.
Being a software tester has its own unique set of challenges. To help testers overcome these challenges, it is vital to set up a system where employees have available a number of development opportunities, including on-the-job mentorship, coaching, classroom training, and a defined career path. Nate Shapiro outlines how the quality assurance department at Blizzard Entertainment is investing in its employees by implementing a program to help create and sustain long-term careers in software testing.
• Migrate from waterfall to Agile with extreme automation from the first sprint• Build intelligent, data-driven test cases to cut workload by 50–85 percent and guarantee risk coverage• Accelerate time to market by eliminating script maintenance to enable continuous integration
• This session explains how a holistic Continuous Testing process can help you balance speed with quality• Learn how to reduce the number of defects being injected into the application• Mitigate existing risks before they progress to the next stage of the SDLC.
As a test manager, James Christie experienced two divergent views of a single project. The official version claimed that planning and documentation were excellent, with problems discovered during test execution being managed effectively. In fact, the project had no useful plans, so testers improvised test execution. Creating standardized documentation took priority over preparing for the specific problems testers would actually face during testing. The required documentation standards didn't assist testing; they actually hindered by distracting from relevant, detailed preparation.
The software development lifecycle is a pretty complex process in many organizations. However, by using monitoring tools and methodologies, you can accelerate testing and release higher quality code―the cornerstone of rapid software delivery. These tools provide immediate feedback with actionable information so you can address problems as they are detected instead of waiting until the end of a testing cycle. Earlier detection, combined with tests that are a better representation of production workloads, are key to releasing better code, faster.
Business analysts, developers, and testers are sometimes not on the same page when it comes to test automation. When there is no transparency in test cases, execution, coverage, and data, review of automation by all stakeholders is difficult. Making automation scripts easily readable and writable allows stakeholders to better participate. Subodh Parulekar describes how his team dealt with these issues.
Performance testing requires knowledge of systems architecture, techniques to simulate the load equivalent of sometimes millions of transactions per day, and tools to monitor/report runtime statistics. With the evolution from desktop to web and now the cloud, performance testing involves an unparalleled combination of different workloads and technologies. There is no one tool available—either commercial or open source—that meets all performance testing needs.
Computer hacking isn’t a new thing, but the threat is real and growing even today. It is always the attacker’s advantage and the defender’s dilemma. How do you keep your secrets safe and your data protected? In today’s ever-changing technology landscape, the fundamentals of producing secure code and systems are more important than ever. Exploring the psyche of hackers, Mike Benkovich exposes how they think, reveals common areas where they find weakness, and identifies novel ways to test your defenses against their threats.
Applications today are accessed over myriad network configurations—wired, wireless, and mobile networks. Deployed applications may deliver different content and functionality depending on whether the user is accessing it via a browser, smartphone, or tablet. Steve Weisfeldt explains how these combinations significantly impact the performance of applications, creating a previously unseen set of testing challenges.
• Roles of typical QA vs expectations• Shift left approach—how, when, and why• Need of an hour—purpose over process
Do you find yourself discussing with your peers what you think the system you’re building should do? Do you argue over what the users want? Do discussions wind up in a heated debate? This result indicates that no shared understanding exists about the system. With a lack of shared understanding, it’s easy to fall into the trap of making assumptions about system functionality, who the users will be, and how to build the system. These assumptions introduce errors into the requirements and design—long before a single line of code is written.
Centralization of testing services into a testing center of excellence (TCoE) for system testing is common in IT shops today. To make this transformation mature, the next logical step is to incorporate the user acceptance testing (UAT) function into the TCoE. This poses unique challenges for the TCoE and mandates the testing team develop a combination of business process knowledge coupled with technology and test process expertise. Deepika Mamnani shares her experiences in implementing a UAT TCoE and best practices—from inception to planning to execution.
When doing test automation, you sometimes notice that things are not working out as expected, but it’s not clear why. You are so caught up in the day-to-day work that you don't see the bigger picture. It’s like when you get sick―you know something is wrong, but you don’t know what. That’s the time to visit a doctor. Doctors diagnose diseases mainly by asking questions. First, they get a general idea of what’s wrong; then the questions become more and more specific; and in the end, they identify the disease and prescribe the appropriate cure. This method also works well for test automation.
Few technology shifts have impacted the way we do business as much as mobile. The new and exciting functionality delivered by mobile apps, the pace at which they are being developed, and their emergence as the “face of the business” requires that organizations deliver unprecedented quality in these software systems. Join Dennis Schultz to learn how leading enterprises are approaching their mobile application testing challenges and how they have integrated mobile into their existing processes.
When creating system performance models, the primary challenges are where and how to start. Whatever the performance characteristics being estimated or modeled, we need a solid approach that addresses both business and system needs. All too often performance tests inadvertently mix load and stress scenarios with little regard for how this will confound recommendations and business decisions. If you are a test manager, a business process owner, or you simply want to better understand performance testing, you will be interested in William Hurley’s case studies.
As we've seen in recurring events in the past year, web services APIs are a primary target for security attacks—and the consequences can be catastrophic for both API providers and end users. Stolen passwords, leaked credit card numbers, and revealed private messages and photos are just some of the headaches awaiting those who have been compromised. Ole Lensmar puts on his hacker-cloak to show how attackers break systems via web service APIs with fuzzing, session spoofing, injection attacks, cross-site scripting, and other methods.
Security testing is often shrouded in jargon and mystique. Security conjurers perform arcane rites using supposed “black hat” techniques and would have us believe that we cannot do the same. The fact is that security testing “magic” is little more than specialized application of exploratory test techniques we already understand. In this Defense against the Black Hats, Paco Hope dispels the myth that security testing is a magical art. By deconstructing security activities into techniques we already know well, we expand our testing.
More information coming soon!
Join us at The Workshop on Regulated Software Testing (WREST)—a free, full-day bonus session held on Friday after the conference concludes. A unique peer workshop, WREST is dedicated to improving the practice of testing regulated systems. We define regulated software as any system that is subject to an internal or external review.
