FAQ

What is Stream Processing?

In short, modern stream processing is the parallel processing of events - as the data arrives. Various streaming data technologies exist (Kafka, Kinesis, Pravega), but stream processing is the practice of creating computations (often stateful) on these event streams for some business purpose.

There are a handful of technologies that provide proper stream processing (Flink, Samza, Storm, KStreams), each with various tradeoffs.

Eventador has embraced Apache Flink because of it’s poweful stateful capabilties, rich API, deep community, maturity, and simple cluster/scaling model.

Apache Flink is an open source stream processing framework developed by the Apache Software Foundation. The core of Apache Flink is a distributed streaming dataflow engine written in Java and Scala. Flink executes arbitrary dataflow programs in a data-parallel and pipelined manner. More from Wikipedia. Check out the documentation here.

No. It’s common to use Flink and Kafka together. Flink has a number of built in integrations with Kafka to use it as a source and sink for data.

Can SQLStreamBuilder use AWS Managed Kafka (MSK) for a data source and/or data sink?

Yes. Use the data providers option in the Eventador.io console.

What is the difference between SQLStreamBuilder and Confluent KSQL?

SQLStreamBuilder was designed to both be a simple and feature-rich streaming SQL editor and job manager. SQLStreamBuilder has built in scalability, fault tolerance, and high performance recovery of jobs. SQLStreamBuilder allows for creation of sources and sinks using many popular technologies like databases, NOSQL sources, S3, Kafka, Kinesis, etc. SQLStreamBuilder has rich SQL grammar based on Apache Calcite, and a schema definition component built in. SQLStreamBuilder allows for simple and powerful streaming SQL job management.

KSQL is a kafka-centric tool for using SQL to query Kafka topics. It runs in distinct “editor” mode and “headless” mode.