Hazelcast Jet is an in-memory data processing framework combined with IMDG to provide in-memory storage layer to be used during processing and to publish results of the computation. The breakthrough speed of Jet-based applications is achieved by keeping both the computation and data in the memory.
Jet provides Hazelcast users with tools to aggregate, sort, join, filter and transform the data streams in the parallel, distributed and consistent way (speed at scale!). The speed of Jet is to be read not just as “high throughput”, but also as “low-latency”. When processing real-time data, then processing 100GB per minute is irrelevant if the information is to be available in seconds at most (examples: ad placement in on-line system, detecting anomalies based on events from industry sensors, detecting fraudulent transactions).
The vision of Jet is to allow data-intensive applications to operate at near real-time speeds.
Hazelcast Jet is a standalone product with separate use-cases, roadmap and lifecycle. Hazelcast Jet introduces a new product to the Hazelcast company portfolio.
The full-blown Hazelcast IMDG is embedded (OEMed) in Jet to allow data processing jobs to benefit from embedded in-memory storage. All the services provided by IMDG are available without restriction. As the IMDG is the embedded, supportive structure for Jet, the IMDG is fully controlled by Jet (start, shutdown, scaling etc.). Also, the version of underlying IMDG is defined by Jet. For example Jet 0.3 comes with IMDG 3.8 as it benefits from the specific features of IMDG 3.8. So the IMDG JAR cannot be replaced with different version.
If you need more control over your Hazelcast IMDG cluster — use separate, remote Hazelcast IMDG cluster. See the features to compare benefits of both embedded and remote deployment.
To get closer to the vision of Jet, we’re giving a high priority to those features : robust streaming (fault tolerance, exactly once processing guarantees), high level streaming API and off-heap tiered storage.
Influence Jet. Become part of the Hazelcast community and tell us your opinion on Jet roadmap.
We decided to do several non-public releases of Jet before the public launch. That gave us more flexibility to internally iterate over the fundamental concepts of Jet. We consider Jet 0.3 to provide enough value to be shared with the Hazelcast community.
However, prior to 1.x release, Jet and it’s APIs are still a subject of potential change.Jet in 5 minutes