I first used Netty to build a load test driver to simulate thousands of mobile clients accessing REST services and responding to real-time push notifications over XMPP. These days, I’m using WebSockets more than XMPP and have revisited Netty to push updates asynchronously to multiple clients. I’ll show how I configured the Netty pipeline to support this scenario.

Let’s dig right in to the factory, in particular the method used to create the pipeline.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

  @Override
  public ChannelPipeline getPipeline() throws Exception {
    // Create a default pipeline implementation.
    ChannelPipeline pipeline = pipeline();
    pipeline.addLast("decoder", new HttpRequestDecoder());
    pipeline.addLast("aggregator", new HttpChunkAggregator(65536));
    pipeline.addLast("encoder", new HttpResponseEncoder());
    pipeline.addLast("handler", new WebSocketServerHandler());
    pipeline.addLast("jsondecoder", jsonDecoder);
    pipeline.addLast("jsonencoder", jsonEncoder);
    pipeline.addLast("builderencoder", builderEncoder);
    pipeline.addLast("georegionencoder", regionEncoder);
    pipeline.addLast("presencefilter", new PresenceFilter(bus));
    return pipeline;
  }
}

So one of the first things to understand about Netty is that a pipeline is created for each session. This matters as you look at the mix of objects that are added to the pipeline. Some are newly allocated for each pipeline (e.g. encoder) and some are common to all (e.g. georegionencoder). The newly-allocated ones encapsulate session-specific behavior, while the common ones don’t have any session-specific processing. Netty actually provides a way to access per-session information from a common handler (the ChannelHandler documentation provides a good discussion of the various options), but I prefer to use new objects for all session-specific processing.

Besides the distinction between per-session and common behavior in the pipeline, the order of items matters. While the Netty documentation is really good (see ChannelPipeline), I still find myself messing up. For incoming packets, the handlers are executed from the top down, while for outgoing packets, they are executed from the bottom up. Not all handlers process information in both directions, so this can be a little confusing.

For our pipeline, the incoming packets go through the handlers in the following order

• wsdecoder
• handler
• jsondecoder
• presencefilter

Wait, wsdecoder? We didn’t set up a handler with that name. The WebSocketServerHandler is responsible for detecting the WebSocket handshake and, once successful, replaces the HTTP decoder and encoder handlers with WebSocket handlers named wsdecoder and wsencoder. One of the powerful aspects of Netty pipelines are the ability to query and change them. So technically the list above is for sessions that have negotiated a WebSocket connection. For those that remain plain HTTP, the order is

• decoder
• aggregator
• handler
• jsondecoder
• presencefilter

So what do these handlers do, exactly? In our server, it allows clients to register for push notifications based on spatial information. The client can register for updates regarding people or devices within certain regions and with desired dwell or behavioral characteristics. It registers using the WebSocket connection and then receives the updates over that same connection. So for this registration path, the processing is:

That processing chain explains why jsondecoder can be common to all sessions while presencefilter is unique per session. The filter needs to apply the specific registration behavior for that user.

The outgoing path is a little longer

This path is triggered by external presence and location updates matching the filters configured for the session. Those updates originate from processing on other pipelines. The pipelines are connected by a Guava EventBus. Guava is just so useful in so many ways, you should definitely check it out. Here, we use EventBus to distribute presence and location updates to the relevant sessions. The georegionencoder handler performs functionality similar to a previous post to identify and label regions. The builderencoder handler supports my internal messaging infrastructure. It uses Protocol Buffers as the canonical message representation. Before they can be serialized, Protocol Buffer messages must be built. This encoder simply ensures any Protocol Buffer Message is built before passing it downstream. Depending on the client capabilities, they could receive and process the messages in the binary format. That isn’t a good option for browser-based clients, though. The jsonencoder converts the Protocol Buffer to JSON format and then the wsencoder handles the WebSocket framing as before.

The great thing about this setup is how modular the handlers are and how easily they can be reused. It doesn’t matter if the final endpoint is HTTP, WebSocket, JMS or another type of message broker, or whether the client can handle binary Protocol Buffers or needs an alternate representation like JSON or XML. The handler stack can be composed to support all these options.

The downside of Netty is that it is somewhat low level. For my load testing work, I had to build my own multipart/form-data support, for example. But you get a really powerful framework for building messaging apps. It is similar to Rack’s middleware concept, but more general in that it can be used for any TCP or UDP protocol.

Netty includes WebSocket client and server examples to get started with and now you should have some background to be able to modify the sample pipeline to add your own processing.