Don't miss the upcoming webinar: Building Real-Time Data Pipelines with a 3rd Generation Stream Processing Engine - sign up now!

Streaming Connectors

Avro Sample code on Github Docs

Source: Yes Sink: Yes Batch & Streaming

Jet can read and write Avro-serialized data from self-contained files (Avro Object Container format), HDFS and Kafka. Kafka connector can be configured to use the schema registry.

Custom Connector Sample code on Github Docs

Source: Yes Sink: Yes Batch & Streaming

Jet provides a programming interface allowing you to write your own Connectors for both batch and streaming.

File watcher (Streaming) Sample code on Github Docs

Source: Yes Sink: Yes Streaming

Watches directory and streams the new lines of text coming from the files in the watched directory. The same directory has to be available to all cluster members (using shared network file system for example).

Hazelcast ICache (Streaming) Sample code on Github Docs

Source: Yes Sink: Yes Streaming

Reads the change stream of a Hazelcast ICache. Supports a predicate and projection pushdown. The connector makes use of data locality when reading from an embedded Hazelcast IMDG.

Hazelcast IMap (Streaming) Sample code on Github Docs

Source: Yes Sink: Yes Streaming

Reads the change stream of a Hazelcast IMap. Supports a predicate and projection pushdown. The connector makes use of data locality when reading from an embedded Hazelcast IMDG.

JMS Sample code on Github Docs

Source: Yes Sink: Yes Streaming

Streams messages from/to a JMS queue or a JMS topic using a JMS Client on a classpath. Reading from the queue can be parallelized.

Kafka Sample code on Github Docs

Source: Yes Sink: Yes Streaming

Consumes one or more Apache Kafka topic. Reader is based on Kafka Consumer. Jet assigns Kafka partitions evenly to the reader instances to align the parallelism of Kafka and Jet. Sink publishes messages to an Apache Kafka topic using Kafka Producer.

TCP Socket Sample code on Github Docs

Source: Yes Sink: Yes Streaming

Connects to the specified socket, reads (source) or writes (sink) lines of text received from it.

Hazelcast Jet

Main Menu