Description

In cloud Computing, MapReduce is a current computing standard for important data processing. But those existing techniques in the process are not providing the better performance because of the un-improved resource allocation. To overcome this issue, and to provide the performance improved resource allocation with the proposed technique called DynamicMR. In DynamicMR, we have three major steps they are Dynamic Hadoop Slot Allocation it is used to overcome the slot allocation restraint, Speculative Execution Performance Balancing it is to steadiness the performance in cluster of jobs, Slot Prescheduling it is to advance the data vicinity. Finally it improves the performance of MapReduce workloads. We improve the performance of a MapReduce cluster via optimizing the slot utilization primarily from two perspectives. First, we can classify the slots into two types, namely, busy slots (i.e., with running tasks) and idle slots (i.e., no running tasks). Given the total number of map and reduce slots configured by users, one optimization approach (i.e., macro-level optimization) is to improve the slot utilization by maximizing the number of busy slots and reducing the number of idle slots. Second, it is worth noting that not every busy slot can be efficiently utilized. Thus, our optimization approach (i.e., micro-level optimization) is to improve the utilization efficiency of busy slots after the macrolevel optimization. Particularly, we identify two main affecting factors: Speculative tasks based on these, we propose DynamicMR, a dynamic utilization optimization framework for MapReduce, to improve the performance of a shared Hadoop cluster under a fair scheduling between users. In the dynamic MR process apart from the three concepts present in the paper we are going to introduce clustering approach. In addition to the multi data center processing we are going to add clustering concept. Because we are going to split the data and process the data in multiple datacenters. If we combine the similar data’s into clusters using k-means algorithm. By clustering the data we can able to process the data in short execution time. After preprocessing, we split our process into multiple files and apply clustering process. Here we are going to use k-means clustering algorithm which is a standard algorithm, which helps to process the data in a short execution time.