Distributed java.util.stream

Use Java 8 Stream API

Java 8 streams-based applications can be boosted by making the computation distributed using Hazelcast Jet.

The popular Java 8 java.util.stream is a set of classes to support functional-style operations on streams of elements, mostly on collections. Hazelcast Jet implements java.util.stream API in the distributed way.

Hazelcast Jet has an implementation of java.util.stream for Hazelcast IMDG’s IMap and IList. java.util.stream operations are mapped to a DAG and then executed, and the result returned to the user. The computation is run as distributed and parallelized. Jet takes a benefit of data locality – if the data source and Jet are co-located Jet will read local data partitions and process data in-place to avoid unnecessary shuffling and sending over the wire.

Due to it’s simplicity, the Streams API is a great way to get started with Hazelcast Jet:

        IStreamMap<String, String> lines = instance1.getMap("lines");
        IMap<String, Long> counts = lines
                .flatMap(m -> Stream.of(PATTERN.split(m.getValue().toLowerCase())))
                .collect(DistributedCollectors.toIMap(w -> w, w -> 1L, (left, right) -> left + right));
        System.out.println("Counts=" + counts);

Visit code samples for distributed java.util.stream.

Jet in 5 minutes


Main Menu