Blogroll

Olympics image loader powered by Camel

Live #Olympics Image Stream
This is a very short post about a very simple application. Inspired by the London 2012 Olympics and based on Apache Camel examples, I've created an application that displays twitter images related to the Olympics in real time. It listens for tweets containing images, filters out duplicates, and sends the images to the browser using websockets every 5 seconds. See how simple is the application on my github account.

This is an end-to-end push application: once a user pushes an image to twitter, twitter pushes the image to the camel application, and then the camel application pushes the image to all the clients. It is built using only "free stuff": Twitter's free streaming API, Apache Camel framework, Amazon's free micro instance and done during my free time. Here is the essence:

Line by line explanation
Twitter pushes tweets containing the #Olympics and #London2012 tags to the app:
from("twitter://streaming/filter?type=event&keywords=" + searchTerm)
Log statistical information about the number of messages every minute. Not visible the users:
.to("log:tweetStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")
Extract images from the tweets which has media associated with:
.process(new ImageExtractor())
Put the current number of tweets and images in the message:
.process(new Statistics())
Filter out all the tweets which doesn't contain images:
.filter(body().isInstanceOf(Tweet.class))
Filter out duplicated images, identified by their url:
.idempotentConsumer(header(UNIQUE_IMAGE_URL), MemoryIdempotentRepository.memoryIdempotentRepository(10000))
Log again, the messages which reached this far in the route:
.to("log:imageStream?level=INFO&groupInterval=60000&groupDelay=60000&groupActiveOnly=false")
Let images go with 5 seconds difference, so the user can enjoy it. Also important - don't block the twitter listener by using callerRunsWhenRejected if the image buffer fills up, Twitter will block you:
.throttle(1).timePeriodMillis(5000).asyncDelayed().callerRunsWhenRejected(false)
Serialize into json:
.marshal().json(JsonLibrary.Jackson)
Push it to the users:
.to("websocket:camelympics?sendToAll=true");
The application can be run also locally and allows filtering images not only for the Olympics but for any keywords passed as argument. Don't forget to use your own twitter oath tokens when using locally though.

Enjoy the Olympics.

11 comments:

I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post.
Custom Thesis Writing || Custom Dissertation Writing

Justin Time Example to show case Camel power during London Olympics..Bravo!!

Dude, really. This was the best example I've seen about Camel Twitter... I started this component years ago and I feel so glad to see part of my work being used to build great things.

Well done!

Thanks guys. I've played more with the app and updated the UI a little more. You can have a look at the live version again.

This comment has been removed by a blog administrator.

Excellent example, but I'm confused on two points

1) .marshal().json(JsonLibrary.Jackson) How exactly are you marshalling an image to JSON for the client?

2) you say "Log statistical information about the number of messages every minute. Not visible the users" but the log statement doesn't contain any reference to stats..I'm guess it's just a general log statement for this post.

Nevermind...Read the source. You're extracting the URL only...I was assuming there was a binary involved.

Glad you figured it all out :)

This was the best example I've seen about Camel Twitter...
Fall Protection Systems

It was very useful for me. Keep sharing such ideas in the future as well. This was actually what I was looking for, and I am glad to.

Post a Comment

About Me