การเปรียบเทียบเทคนิคการเพิ่มประสิทธิภาพของการประมวลผลแมพรีดิวซ์สำหรับข้อมูลที่มีความเบ้

Abstract

Nowadays, data is large in terms of volume, variety of format and changes rapidly. Therefore, it is necessary to rely on processing to make data accuracy and precision. To ensure real-time utilization and maximum benefit. MapReduce is a framework for processing big data, including 2 functions: Map function and Reduce function. Map function processes input data set as key/value pairs and generates intermediate key/value pairs with the same key as output. Reduce function merges all intermediate values in each set. In this process, the reducer must wait for all maps to finish before the reducer starts. Due to non-uniform distribution of big data. MapReduce process will be delayed because data that split to all nodes is not same size in map stages. As a result, each node complete processing at different times. Nodes that finish earlier must wait for the final mapper node to complete before they can proceed with reducing. The results show that all of the algorithms studied in this paper can improve the execution time of MapReduce with skewed data. However, there are some limitations to improvement, especially when data is not heavily skewed; the overhead of the algorithms might overcome their benefits.