Conference archive

SEE PRICING & PACKAGES

Thursday, October 6, 2016 - 3:00pm to 4:00pm

Big Data, Big Trouble: Getting into the Flow of Hadoop Testing

Add to calendar

Big Data, one of the latest buzzwords in our industry, involves working with petabytes of data captured by various systems and making sense of that data in some way. Maryam Umar has found that testing systems like Hadoop is very challenging because of the frequency with which the data arrives in the system, the number of jobs that run to process that data, and the interdependency of the data. Maryam describes some of the projects at Hotels.com which involve identifying multiple users and using that data to make recommendations of hotels. Testing this is fairly difficult as we need an ability to represent the jobs being executed in the Hadoop ecosystem with an appropriate test tool. Maryam presents a few examples of how she has been able to overcome this challenge using the Oozie workflow coordinator as a test tool that works with the Hadoop file system (HDFS). She demonstrates how test code can be written in a non-testing tool to help gain confidence in the data produced as a result of running a job processor.

Hotels.com

Maryam Umar works in London at Hotels.com, an Expedia, Inc. company. She started her career nine years ago as a QA test engineer in the finance and mobile industry. After transitioning to the eCommerce sector, Maryam performed QA in various capacities for online restaurant and travel services. She continues to work in QA since she strongly feels that software testing is critical to getting products to meet the customer’s desire. In the past few years, Maryam has been passionately promoting and developing testing automation techniques, which can streamline and fortify the QA processes. Her mantra is to reduce any repetitive tasks which can be automated for testing.