Blogroll

Clustered Idempotent Consumer Pattern with Infinispan

I've created a small project that shows how to use JBoss Infinispan with Apache Camel and the Idempotent Consumer Pattern to guarantee a message will not be processed twice in a clustered environment.
Imagine you have an application that has to scale out easily by deploying it on multiple containers. But the application has to process each unique request only once across the cluster.
The solution is simple: use Idempotent Consumer Pattern in Camel with a repository that can scale out easily. This is where Infinispan comes into play. Infinispan is extremely scalable, highly available key/value store and data grid. If you use InfinispanIdempotentRepository with an idempotent consumer, it will create an in-memory cache to store the requests, and the moment you start another instance of the application, the cache instances will sync and the idempotent consumers in all applications will not process existing requests any longer.
With this project (idempotent consumer demo5) you can start as many containers as you want, each container will start a rest endpoint on a new port starting from 8080 (http://localhost:8080/idempotent/KEY), and if you perform a GET request with a key, the subsequent requests with the same key to any other container will be rejected. Behind the scene Infinispan is replicating all the processed keys across the cluster of Camel applications and ensuring consistency.
The core of the application is the following route definition that finds a new free port number for each instance of the application:
Simple, isn't it.

Why bother contributing a Camel component?

Camel has a staggering number of connectors and that is thanks to you - the community. May be you also know a cool library that still doesn't have a Camel connector and wonder whether you should create a connector and contribute it to Apache Camel? Hopefully this article will give you the answer why you should do so.

The learning experience - There are great books about Apache Camel (mine is not bad either), but nothing can teach you better than banging your head while trying to write your first component. Understanding how Camel components works equals to understanding half of the project (the other half is the framework lifecycle and EIPs which are similar to components). So if you want to learn Camel and do more than than hello world, try writing a component.

A sense of accomplishment - the moment your code is pushed to Apache repository by a committer it becomes available to everyone and it is owned by the Apache foundation. But your name remains in the commit history forever, and no matter how much the code has changed, you can always claim: "Oh yeah, I've created that Camel component".

Becoming a Camel contributor - you get the right to add your name to the Camel team list. It is one of the most popular open source projects in the Java world and being part of something great is ... great. And who knows, may be this small contribution will spark a new passion and you will follow up with many other contributions and become a Camel committer in a short time.

Sharing can only bring you good - may be you are luckier than the average developer and you work for a company that let's you hack whatever you want for a day every fortnights. Or may be you are even luckier and you get paid for contributing to open source projects as my colleagues at Red Hat do (here is a list with Camel jobs at Red Hat for those interested). If you are neither of those groups, you can still work on something that is interesting to you and contribute to open source projects. All you need is couple of hours in a train or Starbucks.

Why a component? Writing a component is the easiest way to contribute something to Camel. It doesn't require deep Camel or EIP knowledge. You need good understanding of Camel components, good understand of the library you want to create connector for, and a mindset open for sharing.

This article is only the appetizer, if you liked it, the next article will explain how to create a Camel component.

10 Good Reasons for using Apache Camel to create Microservices

Microservices is an architectural style for distributed applications that is taking over the developer community in high pace. There are tens of blog posts on the net every day, conferences and even books on the topic already.
To contribute to all the buzz around it I've written a post where I gave 10 good reasons why Apache Camel and Fabric8 are great tools for creating microservices applications. If you are interested in microservices topic, the article is published on jaxenter.com.

Circuit Breaker Pattern in Apache Camel

Camel is very often used in distributed environments for accessing remote resources. Remote services may fail for various reasons and periods. For services that are temporarily unavailable and recoverable after short period of time, a retry strategy may help. But some services can fail or hang for longer period of time making the calling application unresponsive and slow. A good strategy to prevent from cascading failures and exhaustion of critical resources is the Circuit Breaker pattern described by Michael Nygard in the Release It! book.
Circuit Breaker is a stateful pattern that wraps the failure-prone resource and monitors for errors. Initially the Circuit Breaker is in closed state and passes all calls to the wrapped resource. When the failures reaches a certain threshold, the circuit moves to open state where it returns error to the caller without actually calling the wrapped resource. This prevents from overloading the already failing resource. While at this state, we need a mechanism to detect whether the failures are over and start calling the protected resource. This is where the third state called half-open comes into play. This state is reached after a certain time following the last failure. At this state, the calls are passed through to the protected resource, but the result of the call is important. If the call is successful, it is assumed that the protected resource has recovered and the circuit is moved into closed state, and if the call fails, the timeout is reset, and the circuit is moved back to open state where all calls are rejected. Here is the state diagram of Circuit Breaker from Martin Fowler's post:

How Circuit Breaker is implemented in Camel?

Circuit Breaker is available in the latest snapshot version of Camel as a Load balancer policy. Camel Load Balancer already has policies for Round Robin, Random, Failover, etc. and now also CircuiBreaker policy.
Here is an example load balancer that uses Circuit Breaker policy with threshold of 2 errors and halfOpenAfter timeout of 1 second. Notice also that this policy applies only to errors caused by MyCustomException
And here is the same example using Spring XML DSL:

A Docker Maven Plugin for Integration Testing

What is Docker?
Docker is the buzzword that is taking the DevOps world. If you don't know yet what is Docker, be warned, you will find yourself using it one way or another very soon. The rest of this post assumes some basic understanding of Docker, but if you are not familiar with it now, I'm sure you will come back later and read this.
Docker is ideal for integration testing, complex demos of distributed systems or even running production systems. It is an open source software container. You can imagine it as a very lightweight and ultra fast virtual machine.
An example
Inspired by the "Integration testing with Maven and Docker" article and using Docker Java API I've created a simple Docker Maven Plugin that can manage Docker containers. Given a Docker image, the plugin will create a container from it and start it as part of maven build process and stop and remove the container when the build process is finished. If the image is not available locally it will pull it down from the public Docker registry before creating a container from it.

The following integration test in Apache Camel is ignored because it requires a running Redis instance:

To make it pass, we can use docker-maven-plugin with a Redis image and 6379 port accessible for the test:

The plugin will start a Docker container at compile phase that has a running Redis instance and shut it down at post-integration-test phase.
This is a very simple example, but the plugin can support more advanced scenarios with multiple images configured differently and started/stopped at different phases. Enjoy.