LiveZilla Live Chat Software

iShuffle: Improving Hadoop Performance with Shuffle-on-Write

iShuffle: Improving Hadoop Performance with Shuffle-on-Write

Starting at: Rs.4,500.00

4500 reward points

iShuffle: Improving Hadoop Performance with Shuffle-on-Write

Hadoop is a popular implementation of the MapReduce framework for running data-intensive jobs on clusters of commodity servers. Shuffle, the all-to-all input data fetching phase between the map and reduce phase can significantly affect job performance. However, the shuffle phase and reduce phase are coupled together in Hadoop and the shuffle can only be performed by running the reduce tasks. This leaves the potential parallelism between multiple waves of map and reduce unexploited and resource wastage in multi-tenant Hadoop clusters, which significantly delays the completion of jobs in a multi-tenant Hadoop cluster. More importantly, Hadoop lacks the ability to schedule task efficiently and mitigate the data distribution skew among reduce tasks, which leads to further degradation of job performance. In this work, we propose to decouple shuffle from reduce tasks and convert it into a platform service provided by Hadoop. We present iShuffle, a user-transparent shuffle service that pro-actively pushes map output data to nodes via a novel shuffle-on-write operation and flexibly schedules reduce tasks considering workload balance. Experimental results with representative workloads and Facebook workload trace show that iShuffle reduces job completion time by as much as 29.6 and 34 percent in single-user and multi-user clusters, respectively



ClickMyProject Specifications
Including Packages
  * Supporting Softwares   * 24/7 Support
  * Complete Source Code   * Ticketing System
  * Complete Documentation   * Voice Conference
  * Complete Presentation Slides   * Video On Demand *
  * Flow Diagram   * Remote Connectivity *
  * Database File   * Code Customization **
  * Screenshots   * Document Customization **
  * Execution Procedure   * Live Chat Support
  * Readme File   * Toll Free Support *
  * Addons    
  * Video Tutorials    

*- PremiumSupport Service (Based on Service Hours) ** - Premium Development Service (Based on Requirements)

Add to Cart:

  • Model: PROJ8152
  • 999 Units in Stock
  • Manufactured by: ClickMyProjects

Please Choose:


This product was added to our catalog on Friday 15 June, 2018.