STARWEST 2019 - Experienced Tester
Monday, September 30
Test Design for Fully Automated Build Architecture
Imagine this … As soon as any developed functionality is submitted into the code repository, it is automatically subjected to the appropriate battery of tests and then released straight into production. Setting up the pipeline capable of doing just that is becoming more and more common and something you need to know about. But most organizations hit the same stumbling block—just what IS the appropriate battery of tests? Automated build architectures don't always lend themselves well to the traditional stages of testing. In this hands-on tutorial, Melissa Benua introduces you to key test...
Unit Testing: What Every Developer and Tester Should Know
Do you want to take your testing skills to the next level? Are you trying to stay relevant on an agile team where testing is shifting to the left on the project timeline? Do you want to help your organization reap the full benefits of testing earlier? Then join Tariq King to explore the fundamentals of unit testing so you can find bugs as soon as they happen and do more thorough, targeted testing during software development. This introductory session is for everyone—of all programming skill levels. Learn how to apply program-based techniques such as testing by looking, automated unit...
Test Data: Mining, Morphing, Managing and Maintaining It!
NewAccording to the 2018/2019 World Quality Report, the number one challenge in applying testing to agile development is overcoming the challenges of creating, managing, and maintaining test environments and test data. Over 48% of respondents had issues with test data. As our systems complexity and time to market demands have increased, the appetite for resolving the test data issue can be diminished or be viewed as test data doesn’t really matter. Join Julie Gardiner as she shares the good, bad and ugly of test environments and data, defines an approach to establish where you are in terms of...
Testing Today's Web Applications: Tools You Can Use
NewIf you are a software tester today, chances are you're testing a website or web application. Whether you are testing in a browser, within a web “wrapper” framework like Electron, or even in a web app wrapped in a mobile application, your customers are using web apps and websites constantly—and they want them to work well. Our challenge in testing, of course, is to test as thoroughly and efficiently as we can, given our time and resource constraints. Chris Kenst will demonstrate some of his favorite free and inexpensive tools for testing web apps and share stories of using these tools in...
Continuous Testing Using Containers
NewContainers. Every manager thinks they want them, but few teams have experience in knowing what to DO with them. Used thoughtfully, containerization of your services can transform the way your organization thinks about testing. Gone can be the days of maintaining X different compute environments with Y different configurations. Imagine instead spinning up just the code you need, on the machine type it needs, and only for as long as you need it. In this technical training, Melissa will walk through what containerization means for a legacy code base attempting to practice continuous...
Artificial Intelligence and Machine Learning Skills for the Testing World
PreviewSoftware continues to revolutionize the world, impacting nearly every aspect of our work, family, and personal life. Artificial intelligence (AI) and machine learning (ML) are playing key roles in this revolution through improvements in search results, recommendations, forecasts, and other predictions. AI and ML technologies are being used in platforms for digital assistants, home entertainment, medical diagnosis, customer support, and autonomous vehicles. Testing practitioners are recognizing the potential for advances in AI and ML to be leveraged for automated testing—an area...
Instill a DevOps Testing Culture in Your Team and Organization
The DevOps movement is here. Companies across many industries are breaking down siloed IT departments and federating them into product development teams. Testing and its practices are at the heart of these changes. Traditionally, IT organizations have been staffed with mostly manual testers and a limited number of automation and performance engineers. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay relevant and add greater value to the business. DevOps really...
Get Your Message Across: Communications Skills for Testers
Communication is at the heart of our profession. If we can’t convey our concerns in ways that connect with key members of the project team, our contribution is likely to be ignored—no matter how advanced our testing capabilities are. Because we act in an advisory capacity rather than being in command, our power to exert influence is almost entirely based on our communication skills. Join Julie Gardiner as she draws on techniques from psychology and marketing to help you get your message across. With people suffering information overload and deluged with emails, it is more important than...
How to Break Software: Robustness Edition
Have you ever worked on a project where you felt testing was thorough and complete—all features were covered and all tests passed—yet in the first week in production the software had serious issues and problems? Join Dawn Haynes to learn how to inject robustness testing into your projects to uncover those issues before release. Robustness—an important and often overlooked area of testing—is the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. Dawn shows you how—by expanding basic tests and incorporating specific...
Data Analytics and Machine Learning
Do you have access to lots and lots of test, development, app, and service data—really big data—from client and cloud service log files, test execution results, and more? Then, you have a great opportunity to begin using data analytics and machine learning (ML) to gain new product quality insights. Bring your laptops and your sense of discovery as Eun Chang introduce analysis techniques and ML tools to help you develop new and potentially groundbreaking insights. First, she will present a fast-paced statistics primer for those with no prior data exploration experience and others looking...
Tuesday, October 1
Testing Transformation: The Art and Science for Success
Technologies, testing processes, and the role of the tester have evolved significantly in the past few years with the advent of agile, DevOps, and other new technologies. It is critical that we testing professionals evaluate ourselves and continue to add tangible value to our organizations. In your work, are you focused on the trivial or on real game changers? Jennifer Bonine describes critical elements that help you artfully blend people, process, and technology to create a synergistic relationship that adds value. Jennifer shares ideas on mastering politics, maneuvering core vs. context...
Testing Strategies for Microservices
Software development is trending toward building systems using small, autonomous, independently deployable services called microservices. Leveraging microservices makes it easier to add and modify system behavior with minimal or no service interruption. Because they facilitate releasing software early, frequently, and continuously, microservices are especially popular in DevOps. But how do microservices affect software testing and testability? Are there new testing challenges that arise from this paradigm? Or are these simply old challenges disguised as new ones? Join Tariq King as he...
Shifting Our Testing to Embrace Customer Experience
In today’s environment, customer and user experience (CX/UX) is overwhelmingly important—and is not just about the product. CX/UX describes all facets of a person’s interactions with and reactions to the product, the organization that supplies it, and the environment in which it is experienced. Isabel Evans says that in order to focus our tests appropriately, it is vital that we testers understand our users’ experiences. We need to explore and measure human, business, and societal impacts of products we develop, and how those aspects are underpinned by technical qualities. We need to shift...
Test Estimation in the Face of Uncertainty
PreviewAnyone who has ever attempted to estimate software testing effort realizes just how difficult the task can be. The number of factors that can affect the estimate is virtually unlimited. The keys to good estimates are understanding the primary variables, comparing them to known standards, and normalizing the estimates based on their differences. This is easy to say but difficult to accomplish because estimates are frequently required even when we know very little about the project—and what we do know is constantly changing. Throw in a healthy dose of politics and a bit of wishful...
A Hands-on Intro to Docker for Testers
Docker is one of the hottest technologies in our industry today. It lets you create and manage containers—like virtual machines but smaller and faster. They can provide on-demand, disposable test environments that start quickly, repeatably, locally or in the cloud. When you want, you can tear down the environment and recreate a pristine copy within seconds. And since the process is automated, anyone on the team can use the same commands to get their own systems. When you finish testing, those containers can be deployed directly into production. At each stage, the environments for...
Integrating Automated Testing into DevOps
In many organizations, agile development processes are driving the pursuit of faster software releases, which has spawned a set of new practices—DevOps. DevOps stresses communications and integration between development and operations, including rapid deployment, continuous integration, and continuous delivery. Because DevOps practices require confidence that changes made to the code base will function as expected, automated testing is essential. Join Jeffery Payne as he discusses the unique challenges associated with integrating automated testing into continuous integration/continuous...
Holistic Performance Testing for Modern Applications
NewWith the advent of frameworks like Angular, React, and Vue, the landscape of application performance has changed significantly in terms of testing and measurement. Gone are the days of measuring response time as a single value based on back-end performance. In modern web and mobile applications, additional layers need to be peeled apart at the front end to truly understand its performance characteristics. Traditional approaches to performance testing are no longer sufficient to provide a delightfully responsive user experience. Join Kaushal Dalvi as he details new developments in the...
Develop and Refine Your Company Testing Strategy
NewEvery quality director wants to implement a modern, forward-looking testing strategy, and many test managers look at other companies to try and figure out that "silver bullet" so they can achieve excellent quality, quickly and reliably. But the truth is everything you need to know about your best test strategy is already in your company. In this workshop, you will use your quality priorities, strategies, and code base as the foundation for defining and improving your testing strategy. You’ll work individually and in small groups to examine your current testing structure and methodologies,...
Wednesday, October 2
QADevSecOps: Leading a Quality-Driven DevOps Transformation
Have you wondered where QA professionals fit into a DevSecOps transformation? Stacy Kirk thinks they should champion the transformation. Regardless of where your company is on its journey to DevSecOps, quality must be at the forefront for optimal effectiveness and customer value. This means promoting feedback loops that use monitoring and reporting tools effectively, and most importantly, it means creating a culture of collaborative communication and continuous improvement. The role of the QADevSecOps practitioner must evolve from ensuring the quality of software to assessing the...
Keynote - Making the Career Transition from Software Testing to Data Science
A decade ago Microsoft had over twelve thousand full-time testers, and when you added up all the contract and outsourced testers too, there were more software test engineers than developers. The test automation solutions alone had more than a hundred million lines of code. However, that process was built for a company that would release a new version of a monopoly-scale product once every three years and ship it on a CD. That world had already begun to change, and Microsoft was missing the boat. When Microsoft tester Ken Johnston first encountered agile development and DevOps, he realized...
Conquering the Testing Challenges of Serverless Applications
PreviewServerless cloud applications are rapidly moving into the mainstream. In this model, teams focus on developing and deploying code on a known technology stack and runtime, with fixed interfaces for application, database, and network, and they offer the advantages of lower costs, faster development, and elastic growth. But testing serverless applications brings significant challenges to testers. Because the stack is typically maintained by the cloud provider, it is updated with new versions and security patches on a regular basis. Testers have to continuously test the stack...
Fuzz Testing for Fun and Profit
A software test is no better than the data that drives it. Fuzz testing is a great way to find buggy, exploitable, or otherwise bad code – and if you’re working with a native application that operates on file input, it’s a solved problem. Grab AFL or some other all-in-one suite, hit go, and profit! But what about web services? What about managed applications? For a service-based developer, fuzz testing your HTTP serving layer isn’t all that interesting if you’re running on the latest version of Apache. Fuzz testing your application along its interface boundaries, though? Very interesting...
From Zero to AI Hero
PreviewAI is here. Will it take over your job? Is it possible to make it beneficial, not detrimental to your career? Kevin Pyles and his team jumped right into the AI universe. Untrained and inexperienced, they realized immediately that they knew nothing, but they were motivated to learn and experiment so they could be part of this revolution. Kevin will show you how his team went from little knowledge of machine learning algorithms, artificial intelligence, and, in some cases, even programming and automation to implementing custom automation frameworks, data analytics, and machine...
Using Design Thinking to Create Better Test Cases
Designing good test cases can be described as an art. With test cases being written with a focus on business, testers should be part of the discovery and design phase of the project, and the business drivers should dictate test case design. But how can we ensure we are focusing on the user and bringing the biggest value possible in this phase? Larissa Rosochansky will describe what design thinking is, how it relates to the testing methodology, and how to use it in the design phase of your project. She will also show you how to better frame the business drivers and select the persona and...
You, Inc.: Building Your Life's Development Plan
PreviewWe all spend enough time thinking about the next feature and its development plan. Days, months, and sometimes years go by working on things that are a priority for our team and company. But how much time do we spend on discovering what is crucial for us and our own development? We can give more when we are at our best. This is not limited to career aspirations, but also applies to work-life balance, as it takes a family and a team to make anyone successful. Aprajita Mathur will work with you in a live session to build your own development plan. She will discuss six strategies that...
What’s Our Job When the Machines Do Testing?
Members of the engineering community are beginning to explore the exciting capabilities of artificial intelligence in order to remove even more mundane and manual tasks from our jobs. The next generation of automation within the software development lifecycle comes to us in the form of AI-inspired approaches: analytics, machine learning, and natural language processing. Geoff Meyer refers to this as cognitive automation, and it offers the promise of automating tasks that up until now could only be performed by humans. However, engineering practitioners going down the path of cognitive...
Finding Performance Issues Early with JMeter
Nonfunctional tests like performance tests are often left until the end of the delivery cycle because they can be expensive in terms of time, resources, and effort. However, performance issues can be difficult to resolve when found late in the software development lifecycle. Apache JMeter is an open source tool designed to load test functional behavior and measure performance. JMeter can be used to develop, manage, and execute load and performance tests while the code is being developed, rather than waiting to do performance testing until just before release. By using it early in your...
AI in Testing: A Moderated Panel Discussion
Artificial intelligence is the newest trend in software testing. But what is it, and how will it impact the tester's role, both today and in the future? What do you need to do to embrace this emerging technology? Adam Auerbach will moderate this panel discussion—which will include Jason Arbon, Jennifer Bonine, and Tariq King—to give you an opportunity to hear the opinions of industry leaders about AI in testing. You will have a chance to drive the debate, so come prepared with all your AI questions.
What's That Smell? Tidying Up Our Test Code
We are often reminded by those experienced in writing test automation that code is code. The sentiment being conveyed is that test code should be written with the same care and rigor that production code is written with. However, many people who write test code may not have experience writing production code, so it’s not exactly clear what is meant. And even those who write production code find that there are unique design patterns and code smells that are specific to test code. Join Angie Jones as she presents a smelly test automation code base littered with several bad coding practices...
Testing AI-Based Systems: A Gray-Box Approach
Testing artificial intelligence- and machine learning-based systems presents two key challenges. First, the same input can trigger different responses as the system learns and adapts to new conditions. Second, it tends to be difficult to determine exactly what the correct response of the system should be. Such system characteristics make test scenarios difficult to set up and reproduce and can cause us to lose confidence in test results. Yury Makedonov will explain how to test AI/ML-based systems by combining black box and white box testing techniques. His "gray box" testing approach...
Enterprise DevOps: Reducing Big-Bang Integrations in Global Organizations
PreviewSix or seven years ago, the norm for update rollouts was a single Microsoft Windows OS release every three years being validated with an annual tick-tock cadence of Intel CPUs. About three years ago, new OSes started to be released twice a year, with new platforms developed several times a year. For us at Intel, this meant we had to increase the speed of our integration and test processes up to tenfold. At the time, our scaling challenges included slow software delivery mechanisms, inefficient testing strategies, and lengthy times to find and fix bugs. If we were going to meet the...
Privacy in a Time of Rich Telemetry
Most modern testing, especially in a DevOps model, uses a lot of telemetry to evaluate and monitor quality of experience for apps and services. In this interconnected world, there is power and risk in data. Ken Johnston will share his personal experiences dealing with US and European Union privacy regulations and the methods he and his team have implemented to mitigate the potential of significant penalties for the misuse of data. He will cover privacy-preserving techniques such as differential privacy and private enclave, what constitutes primary versus secondary uses of data, and how you...
Thursday, October 3
Fun as a Productivity Tool
We should all just be professionals and do our jobs, right? There's no need for fun at work as long as everyone comes in and does their hours, right? As long as we have our processes to cover the work needed, we're good, right? Wrong! Kristoffer Nordström will show you why we need to have fun at work, why simply following the processes isn't always right, and, yes, why we sometimes need to bend the rules just a little to get things done. New research and insights into humanistic approaches have illuminated concepts such as grit and drive, which attempt to explain what motivates people. We'...
Getting to Continuous Testing
Max Saperstone tells the story of how a health care company striving to get to continuous releases built up their automation to secure confidence in regular releases. Initially, as no test automation existed, Max was able to take an opportunity for greenfield test automation and, in the span of twelve months, develop over two thousand test cases. A pipeline was created to verify the integrity of the automated tests and build Docker containers for simplified test execution. These containers could be easily reused by developers and the DevOps team to verify the application. Join Max as he...
Internet of Things: Changing the Way We Test
The internet of things (IoT) brings connectivity to a range of previously non-internet-enabled physical devices and real-world objects. This shift has an impact testing—changing what we test, when we test, and the way we test. For one thing, once you’re in the real world, the number of possible issues explodes due to environmental conditions. Just like a race car must adjust its tires for different track conditions, IoT devices must account for environmental factors such as temperature and humidity to prevent unanticipated failures. Jane Fraser believes that for the IoT to be successful,...
Testers: The Unsung Data Heroes
PreviewConnor Dodge believes that data is the most valuable commodity in the world, and that testers generate some of the most valuable data in product development organizations. Test data can inform release schedules, aid in decision-making, and shape the direction of the product. However, such benefits can only be realized if test data is effectively tracked and presented, and if testers learn how to present that data in compelling and persuasive ways. Join Connor as he talks about how testers can create and use real-time dashboards to analyze the status of agile projects and...
The Game of Continuous Delivery
The holy grail of a modern QA professional is to achieve high quality while continuously delivering features to customers, and the only way to achieve that mission is via continuous testing and delivery. The most critical step in agile transformation and continuous delivery adoption is identifying the bottlenecks in the product development cycle. Tanya Kravtsov will describe tools and methods that facilitate that discovery process and explore ways you can use mind maps, innovation games, and more to identify, prioritize, and resolve bottlenecks. You'll learn ways to deal with the most...
Large-Scale DevSecOps: Bringing Security Confidence to Chaotic Development
PreviewImplementing application security (AppSec) programs on a large scale can often seem chaotic and unwieldy. Without the proper knowledge to implement robust AppSec tools, DevSecOps on a large scale can be overwhelming. And with the countless number of customizable build, task-tracking, and CI integration systems available, many companies don’t know where to begin implementation of DevSecOps. Join Dennis Hurst as he shares the knowledge necessary to wield powerful AppSec tools based on his experience with a variety of large corporate clients. Dennis will discuss common pitfalls and...
.jpg%3Fitok=nZDxuHau&sc=f115d40c31058eaede6022afb22f47ce "Christine_Fisher")
Conquering BDD: How One Team Is Making It Work
PreviewBehavior-driven development has been a buzzworthy term in the testing and development community for several years. At first glance, the elements of BDD seem simple: testing scenarios, living documentation, automation, and reports. That all sounds great, but then why isn't everyone doing it? Upon a deeper dive, it becomes obvious that the implementation of BDD needs a lot of forethought and planning, and teams must approach it for the right reasons. Christine Fisher will follow the evolution that one team is currently experiencing in their shift to BDD. BDD was selected to help them...
Creating Self-Documenting, Reportable, DevOps-Driven Tests
For applications that have fully integrated service middleware, application servers, and backend databases, validation and verification of the entire application ecosystem is critical and complex. The challenge of testing these types of systems is compounded when each integrated application or component is developed by a different team or third-party vendor. Furthermore, application teams with diverse development practices generally require quick responses to system validation checks. Mimi Balcom Meng and her team have developed a testing framework that enables quality engineering teams to...
Mainframe Continuous Delivery: A Journey to the Left Side
PreviewDevOps and continuous delivery have produced exceptional results in the industry, and most of this success has been in the area of distributed systems. Tim Wheeler will discuss identifying problems delivering products in a large, integrated, legacy systems environment with PL/I, COBOL, IMS, and large-batch testing. Tim will talk about the journey his company, State Farm, took toward development and implementation of a plan to address inefficiencies in their delivery mechanism and to adopt modern DevOps practices and tools. You'll leave with a greater understanding of mainframe...
Evolve Your Testing the Pokémon Way
How can you know that your services will handle the requests of millions of users a day? Or that making a fundamental change to one of your technologies won’t break your user experience? The answer: By having your entire team use and build on a phased approach to testing the right pieces at the right time. The Pokémon Company International services group develops and updates the services used for logging into the Pokemon.com website and applications like Pokémon Go and Pokémon TV. Since the launch of Pokémon Go in 2016, their quality-focused team has worked to develop strategies that...
Smart Testing with AI Using Data Mining
PreviewThe approaches to testing are continuously evolving as we try to keep up with the application needs of today’s users. Our industry is facing a new paradigm where AI is helping achieve scale, coverage, and business impact for many organizations that have been primarily focused on finding defects and refactoring test cases. Are your teams struggling to figure out how to leverage AI to improve their testing? To successfully do this, you must first understand one of AI’s underpinning foundations: data mining. Lorna Smyth will explain the fundamentals of data mining and how it is the...
Before and After You Say ‘I Do’ to Docker
PreviewDocker is the most popular containerized solution being used in the software industry for development. Like any other test automation tool, however, implementation can get complicated, tricky, and unmaintainable if all you understand is the record-and-playback features or think it is the same as using a virtual machine. Whether you are already using Docker for your automation needs or plan to start using it soon, join Aprajita Mathur as she shares her journey of using Docker for testing complex and constantly evolving data analysis pipelines. She will discuss basic concepts of...
Continuous Performance Testing: A Tale of Two Worlds
PreviewOnce upon a time, in a land far away, application performance was a differentiator in the market. But in today's competitive landscape, having a highly responsive and scalable application is a basic expectation. Performance no longer lives on the fringe of the quality world, and it is becoming more common for organizations to include performance testing as part of their CI/CD pipelines. However, engineers are faced with a common dilemma: Do they implement small and simple performance tests that may not be very realistic but are fast and reliable in the build pipeline, or do they...
De-Risking the Deployment Process
PreviewIn a world of continuous delivery, deployments happen faster than ever. But while every organization wants to move faster, many teams struggle to keep up with digital consumers’ expectations for quality. Join Philip Soffer to learn how smaller companies can reduce risk in their deployment processes by combining automation with human insight and performing functional and nonfunctional tests in a synchronized process. Using real-world examples, Philip will discuss the interactions among continuous integration, automated functional testing, crowd testing, feature flagging, and...
The Impact of the IoT: What’s in Store for Testing
No longer just a futuristic concept, the internet of things (IoT) has a strong presence in our world. If your business is not prepared for it, you’re already behind. With the proliferation of connected “things”—devices, appliances, cars, and even clothes—Jennifer Bonine says that the stage is set and IoT mobile apps are here to stay. Testers are now being asked to test on more complex devices and more advanced platforms, so they must keep up with the demand for new skills, new strategies, and an entirely new set of knowledge for testing IoT apps. Jennifer will review where we are today and...
Test Data as a Service: Fueling Quality at Speed
PreviewAny business that uses technology to interface with or provide service to their customers, suppliers, and partners requires customer-facing applications that have exceptional quality. The assurance of business outcomes must become an imperative for IT professionals, which means we must learn how to continually engineer quality into every product and process to provide confidence that our technology solutions protect the company brand. However, delivery within the required time frames is continually constrained by the quality, relevance, quantity, and security of test data. Join...
Continuous Testing: A Fishbowl Discussion
Many people confuse continuous testing with test automation, but continuous testing is much more. It requires that all tests to certify a feature are automated and part of the continuous integration pipeline, and the results of those tests determine automatically when the next step should be run. We will discuss how companies have enabled this process, common pitfalls and how to avoid them, and how to move from automation to continuous testing. In a fishbowl discussion, the audience members sit in a circle of chairs in the middle of the room. Several brave souls will fill all but one of...
Are You Ready for Your Digital Transformation?
Many organizations have embarked on a digital transformation in order to stay competitive and gain market share. Whether it is enabling access to legacy applications via a web front end or a migration to the cloud, the journey is complex and requires comprehensive testing. To begin with, most systems use responsive web design to accommodate for the myriad mobile devices retail customers use. How do you test for interoperability across these multiple digital footprints? What about UI testing? ADA compliance? And many digital transformations make use of a microservices architectures using...
Storytelling in the Age of AI
Around the world, citizens are actively engaging in civic tech, social robots are tweeting, and veteran storytellers are capturing and curating stories in new and dynamic ways using virtual and augmented reality. This explosion of tools, sources, voices, and data is indicative of a new, more collaborative era for storytelling. But is the public getting quality stories? Join award-winning storyteller and entrepreneur Davar Ardalan for a dynamic look at the intersection of emerging technology, transparency, and inclusivity in media. The founder and storyteller-in-chief of IVOW, a cultural...