Past Details

Self-Tuning Hadoop for Big Data Analytics

Date / Time:
24/02/2014 Download ICS File
Location:
Carnegie Mellon Qatar, room 1202
Category:
Lectures and Activities
QCRI-Logo-new-2016.jpg Dr. Ashraf Aboulnaga is a Principal Scientist in the Qatar Computing Research Institute, where he heads the Distributed Systems group. His current research interests are in the area of data intensive cloud computing, with a current focus on platforms for Big Data, distributed storage systems, and data integration on the web. Dr. Aboulnaga obtained M.S. and Ph.D. degrees from the University of Wisconsin - Madison, and B.S. and M.S. degrees from Alexandria University, Egypt. He holds an Associate Professor position at the University of Waterloo in Canada, where he has worked since 2004.


Hadoop has become one of the most popular platforms for Big Data analytics, enabling flexible and powerful analysis of massive data sets on commodity computing clusters. However, tuning and administering Hadoop clusters is still not a well-studied area. In this talk, I will present two projects that aim to make Hadoop more self-tuning. The first is ReStore, a system for reusing MapReduce results. ReStore focuses on queries that are expressed in high-level query languages such as Pig Latin or Hive. The compilers of these languages translate queries into workflows of MapReduce jobs. Each job in these workflows reads its input from the Hadoop file system (HDFS) and produces output that is stored in HDFS.  The current practice is to delete all intermediate results from HDFS at the end of executing the workflow. ReStore keeps these intermediate results and reuses them for future queries, resulting in significant performance improvement. I will also present PStorM, a system for tuning Hadoop configuration parameters based on the history of previously observed MapReduce jobs. The two projects presented in the talk share the goal of automatically improving the performance of Hadoop based on workload characteristics.

Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

CSAIl image october 2017 story.JPG

CSAIL hosts annual meeting highlighting innovative collaboration with QCRI

10/10/2017

This year CSAIL celebrates five years of collaboration with the Qatar Computing Research Institute (QCRI), an esteemed research institute that’s part of Hamad Bin Khalifa University in Doha. This ...

Read More

Poynter fact-checking story.JPG

Study: On Twitter, you're better off fact-checking your crazy uncle than a complete stranger

12/09/2017

A new study has found what many of us have always thought to be true: We are more likely to accept correction from people we know than strangers. The  study , conducted by researchers at Cornell, ...

Read More

ingredients CSAIL.jpg

Artificial intelligence suggests recipes based on food photos

25/07/2017

There are few things social media users love more than flooding their feeds with photos of food. Yet we seldom use these images for much more than a quick scroll on our cellphones. Researchers from ...

Read More

Events

2017

after school pic.JPG

QCRI's Creative Space launches free after-school computing courses for teenagers

Download ICS File 01/11/2017  - 20/12/2018 ,

We offer an App Inventor Course in Arabic for students aged 13-15 and an Arduino Programming Course in English for students aged 14-18. Courses are free. Please register quickly as places are limited.

Read More

Past Events

Summer Camp 2.jpg

QCRI conducts first summer computing camps for kids

Download ICS File 16/07/2017  - 27/07/2017 ,

Children and teenagers have been given a rare chance to develop their computing skills with world-class computing scientists at the first summer computing camp conducted by the Qatar Computing ...

Read More

CS 1.jpg

QCRI’s Creative Space holds Open House event for kids

Download ICS File 20/05/2017 ,

The Qatar Computing Research Institute’s new Creative Space, which conducts fun activities to teach children computing skills, has successfully held its first Open House event. About 100 children ...

Read More

News

Ingmar for award story.JPG

QCRI's Ingmar Weber wins UN award for joint gender equality research

21/09/2017

Joint research undertaken by Dr. Ingmar Weber of Qatar Computing Research Institute, part of Hamad Bin Khalifa University, along with scientists from Oxford and Princeton universities, has won a ...

Read More

Food serving - boys (2).jpg

Qatar girls embrace technology to tackle obesity

11/09/2017

Mothers are also more likely to use social media to learn about a healthy lifestyle than fathers, new study finds.

Read More

Majd interns ex-irina (2).jpg

QCRI’s summer internship program kicks off

07/05/2017

The Qatar Computing Research Institute’s 2017 summer internship program is underway. This year’s program involves computer science and computer engineering students from Carnegie Mellon University, ...

Read More