Don't miss the upcoming webinar: Building Real-Time Data Pipelines with a 3rd Generation Stream Processing Engine - sign up now!

Get Started with Hazelcast Jet

We’ll show you how to build and run a distributed Word Count application using Hazelcast Jet. Please follow the instructions below or watch the video.

Introduction to Hazelcast Jet

Requirements

In the good tradition of Hazelcast products, Jet is distributed as a JAR with no other dependencies. It requires JRE version 8 or higher to run.

A Jet instance can live in the application JVM in the embedded member mode. We’ll make use of this setup and we’ll start and control the Jet cluster from our application. This simplifies the packaging and deployment of our application — it will remain self-contained, no other service needs to be installed or started.

Get Hazelcast Jet

The easiest way to start using Hazelcast Jet is to add it as a dependency to your project.

Hazelcast Jet is published on the Maven repositories. Add the following lines to your pom.xml:

<dependencies>
  <dependency>
    <groupId>com.hazelcast.jet</groupId>
    <artifactId>hazelcast-jet</artifactId>
    <version>3.0</version>
  </dependency>
</dependencies>

If you prefer to use Gradle, execute the following command:

compile 'com.hazelcast.jet:hazelcast-jet:3.0'

Alternatively you can download the latest distribution package of Hazelcast Jet and add the hazelcast-jet-<version>.jar file to your classpath.

Run a Word Count Example

You can verify your setup by running Word Count. This simple program finds the number of occurrences of each word in the input text.

This program defines the computation using the Pipeline API, starts an in-process instance of Hazelcast Jet and submits the pipeline for execution.

p>In a distributed computation job the input and output cannot be simple in-memory structures like a Java List; they must be accessible to any member of the computing cluster and must persist after a job ends. Therefore we use the distributed data structures available in the Hazelcast Jet cluster: Hazelcast IList contains lines of text (our input), the results are stored in a Hazelcast IMap.

import com.hazelcast.jet.Jet;
import com.hazelcast.jet.JetInstance;
import com.hazelcast.jet.pipeline.Pipeline;
import com.hazelcast.jet.pipeline.Sinks;
import com.hazelcast.jet.pipeline.Sources;

import java.util.List;
import java.util.Map;

import static com.hazelcast.jet.Traversers.traverseArray;
import static com.hazelcast.jet.aggregate.AggregateOperations.counting;
import static com.hazelcast.jet.function.Functions.wholeItem;

public class HelloWorld {
    public static void main(String[] args) {
        // Create the specification of the computation pipeline. Note
        // it's a pure POJO: no instance of Jet needed to create it.
        Pipeline p = Pipeline.create();
        p.drawFrom(Sources.<String>list("text"))
         .flatMap(word ->
                 traverseArray(word.toLowerCase().split("\\W+")))
         .filter(word -> !word.isEmpty())
         .groupingKey(wholeItem())
         .aggregate(counting())
         .drainTo(Sinks.map("counts"));

        // Start Jet, populate the input list
        JetInstance jet = Jet.newJetInstance();
        try {
            List<String> text = jet.getList("text");
            text.add("hello world hello hello world");
            text.add("world world hello world");

            // Perform the computation
            jet.newJob(p).join();

            // Check the results
            Map<String, Long> counts = jet.getMap("counts");
            System.out.println("Count of hello: "
                    + counts.get("hello"));
            System.out.println("Count of world: "
                    + counts.get("world"));
        } finally {
            Jet.shutdownAll();
        }
    }
}

You should expect to see a lot of logging output from Jet (sent to stderr) and two lines on stdout:

Count of hello: 4
Count of world: 5

What’s Next

Get Hands-on Training

Sign up for hands-on training from Hazelcast experts, online or in the classroom.

Learn more at Hazelcast University

Hazelcast 3.6 Deployment and Operations Guide

Read It Online Now

Hazelcast Jet

Main Menu