Explore all that STARCANADA has to offer.
Many organizations never achieve the significant benefits that are promised from automated test execution. Surprisingly often, this is due not to technical factors but to management issues, especially at system testing level. Surprisingly often, this is due not to technical factors but to management issues. Dot Graham describes the most important management concerns the test manager must address for test automation success, and helps you understand and choose the best approaches for your organization—no matter which automation tools you use or your current state of automation.
So you’ve “gone agile” and have been relatively successful for a year or so. But how do you know how well you’re really doing? And how do you continuously improve your practices? When things get rocky, how do you handle the challenges without reverting to old habits? You realize that the path to high-performance agile testing isn’t easy or quick. It also helps to have a guide. So consider this workshop your guide to ongoing, improved, and sustained high-performance. Join seasoned agile testing coach Bob Galen as he share lessons from his most successful agile testing transitions.
The key to successful testing is effective and timely planning. Rick Craig introduces proven test planning methods and techniques, including the Master Test Plan and level-specific test plans for acceptance, system, integration, and unit testing. Rick explains how to customize an IEEE-829-style test plan and test summary report to fit your organization’s needs. Learn how to manage test activities, estimate test efforts, and achieve buy-in. Discover a practical risk analysis technique to prioritize your testing and become more effective with limited resources.
Today’s software applications are often security-critical, making security testing an essential part of a software quality program. Unfortunately, most testers have not been taught how to effectively test the security of the software applications they validate. Join Jeff Payne as he shares what you need to know to integrate effective security testing into your everyday software testing activities. Learn how software vulnerabilities are introduced into code and exploited by hackers. Discover how to define and validate security requirements.
Critical thinking is the kind of thinking that specifically looks for problems and mistakes. Regular people don't do a lot of it. However, if you want to be a great tester, you need to be a great critical thinker. Critically thinking testers save projects from dangerous assumptions and ultimately from disasters. The good news is that critical thinking is not just innate intelligence or a talent—it's a learnable and improvable skill you can master.
As applications for smartphones and tablets become incredibly popular, organizations encounter increasing pressure to quickly and successfully deliver testing for these devices. When faced with a mobile testing project, many testers find it tempting to apply the same methods and techniques used for desktop applications. Although some of these concepts transfer directly, testing mobile applications presents its own special challenges. Max Saperstone says if you follow the same practices and techniques as you have before, you will miss critical defects.
To be most effective, test managers must develop and use metrics to help direct the testing effort and make informed recommendations about the software’s release readiness and associated risks. Because one important testing activity is to “measure” the quality of the software, test managers must measure the results of both the development and testing processes. Collecting, analyzing, and using metrics is complicated because many developers and testers are concerned that the metrics will be used against them.
In response to increasing market demand for well-performing applications, many organizations implement performance testing programs, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. Scott Barber shares the fundamentals of implementing T4APM™ including specific examples from recent client implementations. T4APM™ is a simple and universal approach that is valuable independently or as an extension of existing performance testing programs.
In both agile and traditional projects, keyword-driven testing has proven to be a powerful way to attain a high level of automation—when it is done correctly. Many testing organizations use keyword testing but aren’t realizing the full benefits of scalability and maintainability that are essential to keep up with the demands of testing today’s software. Hans Buwalda outlines how you can meet what he calls the 5 percent challenges—automating 95 percent of your tests with no more than 5 percent of your total testing effort—using the proven, keyword-driven test method he uses.
It used to be that your work and results spoke for themselves. No longer is that the case. Today you need to be a better collaborator, communicator, and facilitator so that you focus your teams on delivering value. Join Bob Galen to explore the power of the story, one of the most effective communication paradigms. You can tell stories that create powerful collaboration. You can tell stories that communicate product requirements and customer needs. You can tell stories that inspire teams to deliver results.
At the STARCANADA Expo, you can visit the top industry providers offering the latest in testing solutions.
Some people insist that machines will replace humans in many different ways; others disagree, but without explaining why. You could say a smartphone app replaces an admin assistant by managing your schedule. However, to make the app work, you must modify your usual behavior by providing input in a formal, explicit way, and then interpreting and correcting its output to make up for the app's inflexibility. But storing, transforming, and displaying data are only part of the picture.
• Outline key trends and organizational needs driving testing evolution• Review new approaches and innovations that addresses change• Identify best practices to implementing these new strategies and techno0logies that drive ROI, quality at lower risk
Not all testers are lucky enough to get a good foundation for their testing—detailed requirements and system specifications. That, combined with the harsh reality of not having enough time, presents a challenge for the tester—How do I test to such an extent that I will be able to identify defects as early as possible and properly inform stakeholders about the quality of the product?
As more and more organizations transition to agile, many still do not understand how testing fits into agile teams. Does it simply mean placing a tester on every team? Or does it mean doing away with the role of testers? Pradeepa Narayanaswamy explains the importance of working in cross-functional teams that integrate development and testing. Pradeepa shares her insights into the keys of agile testing including understanding the agile testing mindset and goals. She discusses the responsibilities of a tester in an agile team and describes the diverse skill sets required in those teams.
The world has become increasingly dependent on computer-based models to make informed decisions. These models may be financial or engineering based and often are built with spreadsheets. We may not think of spreadsheets as software and therefore tend to overlook applying robust testing techniques to them. However, spreadsheet error rates are often ten times those found in rigorously tested commercial software.
Managing a software project is challenging. Managers often need to deal with overwhelming amounts of data and detail but they must also be able to see the big picture at a glance. What can testers do to help? Excellent testing involves composing, editing, narrating, and justifying a story about the product, about the testing, and about the quality of the testing. However, telling a good story doesn't just depend on the raw data; it depends on representing, organizing, and displaying information to illuminate key points, highlight trends, and show variation.
Good software testers often interact more with people than with software, especially when reporting on the results of testing. Yet our industry provides little training or guidance on the social and psychological aspects of our jobs. There are times when the results testers deliver to project teams and stakeholders can be difficult to accept.
The role of the Product Owner in Scrum is only vaguely defined—owning the Product Backlog and representing the “customer.” In many organizations, Product Owners go it alone, trying their best to represent business needs to their teams.
Many believe that regression testing an application with minimal data is sufficient. However, the data testing methodology becomes far more complex with big data applications. Testing can now be done within the data fabrication process as well as in the data delivery process. Today, comprehensive testing is often mandated by regulatory agencies—and more importantly by customers. Finding issues before deployment and saving your company’s reputation—and in some cases preventing litigation—is critical.
Often, usability testing does not receive the attention it deserves. A common argument is that usability issues are merely “training issues” and can be dealt with through the product's training or user manuals. If your product is only for internal staff use, this may be a valid response. However, the market now demands easy-to-use products—whether your users are internal or external. David Greenlees shares a tool he has developed to generate test ideas for usability testing. His mnemonic—CAN I USE THIS?—provides a solid starting point for testing any product.
Whether you have dedicated test teams or testers distributed over Scrum teams, you have the challenge of planning, tracking, and reporting their testing not only in a meaningful way but also in a way that can adapt to the rapidly changing environment of software development projects. Many commonly used planning methods do not allow for flexibility, and reporting often relies on horribly flawed metrics including number of test cases executed or test pass percentage.
The demand to accelerate software delivery and for teams to continuously test and release high quality software sooner has never been greater. However, whether your release strategy is based on schedule or quality, the entire delivery process hits the wall when agility stops at testing. When software or services that are part of the delivered system, or required environments are unavailable for testing, the entire team suffers. Al Wagner explains how to remove these testing interruptions, decrease project risk, and release higher quality software sooner.
Transitioning to agile development isn't any easier for the test team than it is for the development team. New features and fixes appear daily, priorities change fluidly as customer feedback shapes the backlog, and design documents are frequently sparse as developers use iterations to zero in on design solutions. On top of all this, with agile testing there is rarely time to build the giant spreadsheets of test cases that management loves to use for tracking testing progress.
How many bugs have you missed that were obvious to others? We all approach testing hampered by our own biases. Understanding our biases—preconceived notions and the ability to focus our attention—is key to effective test design, test execution, and defect detection. Gerie Owen and Peter Varhol share an understanding of how the testers’ mindsets and cognitive biases influence their testing. Using principles from the social sciences, Gerie and Peter demonstrate that you aren’t as smart as you think you are.
Throughout the years, Lightning Talks have been a popular part of the STAR conferences. If you’re not familiar with the concept, Lightning Talks consists of a series of five-minute talks by different speakers within one presentation period. Lightning Talks are the opportunity for speakers to deliver their single biggest bang-for-the-buck idea in a rapid-fire presentation. And now, lightning has struck the STAR keynotes.
It’s one week after your product’s launch, and everyone is happy. After all, for the first time in years, your product development exceeded expectations. Coding was completed on time with very few defects. Suddenly, the report of a major usability and security flaw destroys the euphoria and sends everything into chaos. Unfortunately, this is not uncommon in our industry. So, how can we mitigate such things from happening?
Networking break.
When you're responsible for testing, it's almost a given that you will find yourself in a situation in which you feel alone and out in the cold. Management commitment for testing might be lacking, your colleagues in the project might be ignoring you, your team members might lack motivation, or the automated testing you had planned is more complicated and difficult than you anticipated. You feel you can't test enough, and you will be blamed for post-release quality problems.
The term “cross-browser functional testing” usually means some variation of automated or manual testing of a web-based application on different mobile or desktop browsers. The aim of the testing might be to ensure that the application under test behaves or looks the same way on different browsers. Another meaning could be to verify that the application works with two or more browsers simultaneously. Malcolm Isaacs examines these different interpretations of cross-browser functional testing and clarifies what each means in practice.
When implementing test automation, many people encounter problems: where to start with automation, high maintenance costs for the automated tests, or unrealistic management expectations. The good news is that solutions to these problems exist and have been effectively used by many. A “pattern” is a general reusable solution to a commonly occurring problem. Patterns have been popular in software development for many years, but they are not commonly recognized in system-level test automation.
What we really know about other people is their behavior, their words, and their body language. But we assume a great deal more about what's going on in their heads. We behave as though our assumptions are both Valid and True. This frequently leads not only to misunderstandings but also to friction, frustration, and falling out. However, our assumptions could be INvalid and UNtrue! “Doc” List leads you through the basics of an approach called IAAM ("It's All About Me!"), and the implications that IAAM has for you in your everyday professional and personal life.
The classic software testing team is becoming increasingly obsolete. Traditional processes and tools just don’t meet today’s testing challenges. With the introduction of methodologies such as agile, testing processes with a "test last" philosophy cannot succeed in a rapid deployment environment. To exacerbate our testing difficulties, we now have to deal with "big data" which introduces an entirely new set of problems.
Due to the sensitive nature of the personal information often stored on mobile phones, security testing is vital when building mobile applications. Jeff Payne discusses some of the characteristics that make testing mobile applications unique and challenging. These characteristics include how mobile devices store data, fluid trust boundaries due to untrusted applications installed on the device, different and unique aspects of device security models, and differences in the types of threats one must be concerned with. Jeff shares hints and tips for effectively testing mobile applications.
Testers are often evaluated by metrics that don’t really quantify the value of their work. Metrics such as tests planned, tests executed, coverage achieved, and defects reported all measure effort rather than results. Since people generally want to meet metrics goals, measurements that focus on activity rather than effectiveness often encourage unintended behaviors. Since the true value of testers lies in their ability to analyze and communicate risks and impacts, we must change the focus of metrics from numbers to insights.
Organizations implement automated testing in hopes of reducing time and cost while creating higher quality products. They invest in tools, training, and development; identify candidates for automation; and then develop and execute test scripts. Unfortunately, some of these projects are considered failures because the promised savings are not realized. Funding for future automation efforts may be reduced, or automation projects may be cancelled outright.
To be successful, test managers must keep pace with the constantly changing world of software development. The test manager’s job description—planning, supervising, and reporting the activities of the test process, with a focus on functional black-box tests—has remained virtually unchanged over the past decade. Meanwhile, the job requirements have changed dramatically—fast release cycles leading to widespread adoption of agile methodologies; increased security, performance, and usability requirements; and myriad smart devices in users' hands. And these are just the most obvious.
In the past several years the software development lifecycle has changed significantly with high-speed software releases, shared application services, and platform virtualization. The traditional performance assurance approach of pre-release testing does not address these innovations. To maintain confidence in acceptable performance in production, pre-release testing must be augmented with in-production performance monitoring.
In the not-too-distant past, the only viable options for testing complicated web-based applications were commercial (i.e., expensive) tools. These tools were well designed but difficult to scale without significant investment in both human capital and licensing costs. Now, a number of open-source tools are available, allowing for rich, robust, expressive testing against applications as complicated as any in the world—and they’re free.
Today’s IT plans tend to be defined as much by constraints as by requirements. Conventionally, the testing phase starts after the design and build phase. In today’s competitive and cost-conscious business environment, companies look to adopt different test strategies to balance the four key constraints of cost, quality, speed, and risk. In the process, quick often wins, and short-term solutions end up causing more problems than they solve.
As organizations mature in their testing processes, the perennial quest to achieve ultimate excellence has led many to establish a Testing Center of Excellence (TCoE). Many of these initiatives have been plagued with issues ranging from partial implementation to complete abandonment midway. Additionally, most TCoE initiatives meet heavy resistance and inertia from testing teams who perceive them as a threat to their independence and way of working.
The performance of your application affects your business more than you might think. Top engineering organizations think of performance as not just a nice-to-have feature but as a crucial feature of their product. Those organizations understand that performance has a direct impact on user experience and, ultimately, their bottom line. Unfortunately, most engineering teams do not regularly test the performance and scalability of their infrastructure.
The world of information technology is undergoing revolutionary changes. Advancements in mobile computing, fueled by mobile applications, are playing an important role in driving these changes. While developers build their technical skills to accommodate these evolving trends, it is equally important for testers to understand what it takes to test mobile applications. Testers must understand the scope of mobile device applications testing, whether automation is feasible, and what challenges will face the test team.
The IEEE 829 Test Documentation standard is thirty years old this year. Boris Beizer’s first book on software testing also turned thirty. Testing Computer Software, the best selling book on software testing, is twenty-five. During the last three decades, hardware platforms have evolved from mainframes to minis to desktops to laptops to tablets to smartphones. Development paradigms have shifted from waterfall to agile. Consumers expect more functionality, demand higher quality, and are less loyal to brands. The world has changed dramatically and testing must change to match it.
