![]() ![]() ![]() ![]() Streams from sources such as Kafka, and Kinesis, or by applying high-level DStreams can be created either from input data Which represents a continuous stream of data. Spark Streaming provides a high-level abstraction called discretized stream or DStream, The data into batches, which are then processed by the Spark engine to generate the final Spark Streaming receives live input data streams and divides Graph processing algorithms on data streams. Like Kafka, Kinesis, or TCP sockets, and can be processed using complexĪlgorithms expressed with high-level functions like map, reduce, join and window.įinally, processed data can be pushed out to filesystems, databases,Īnd live dashboards. Spark Streaming is an extension of the core Spark API that enables scalable, high-throughput,įault-tolerant stream processing of live data streams. Accumulators, Broadcast Variables, and Checkpoints.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |