Author(s): Jia-Ke Lv; Yang Li; Xuan Wang

Linked Author(s):

Keywords: No keywords

Abstract: The log data real-time processing platform which is built using Storm On YARN integrated MapReduce and Storm that use MapReduce to complete large-scale off-line data global knowledge extraction, sudden knowledge extraction of small-scale data in Kafka buffers through Storm, and continuous real-time calculation of streaming data in combination with global knowledge. We tested our technique with the well-known KDD99 CUP data set. The experimentation results prove the system to be effective and efficient.

DOI: https://doi.org/10.1051/matecconf/201824603009

Year: 2018