Description

With the explosion of big data, processing and analyzing large numbers of continuous data streams in real-time, such as social media stream, sensor data streams, log streams, stock exchanges streams, etc., has become a crucial requirement for many scientific and industrial applications in recent years. Increased volume of streaming data as well as the demand for more complex real-time analytics require for execution of processing pipelines among heterogeneous event processing engines as a workflow. In this paper, we propose a workflow transformation for cost minimization in real-time big data processing on the heterogeneous systems. We first give the definition of stream-based workflow; and then we define eight different patterns as rules for workflow transformation; next, we give our workflow transformation algorithm based on our designed rules. Finally, our experi ment shows that our proposed workflow transformation method can reduce the communication and computation cost effectively.