An Automatic Alignment and Grouping of Hadoop Cluster

Dr V. Govindasamy,P. Siva Kumar, K. Puviarasu, K. Vijiya Kumar

The MapReduce Framework and Hadoop is the platform for scalable analysis on large Datasets in recent years. The primary concern in the Hadoop is to minimize the completion length (i.e., makespan) and fixed number of MapReduce jobs. This makes performance low and causes low resource utilization. To overcome this, we propose a system which dynamically allocates the map and reduce jobs, thus leading to high resource utilization and reduced completion length. The dynamic allocation of MapReduce jobs is achieved by implementing Combiner Interface in MapReduce Framework. The Proposed solution is implemented in the Amazon EC2 Cluster in both Homogeneous and Heterogeneous Clusters. The experimental results show the effectiveness and robustness of our proposed system under simple workloads.