Service Bus Live Monitoring

I explained in the last articles how I tested the Play Framework, Web sockets and how I integrated all this nice stuff with a real example based on a Service Bus, Web services Notifications, etc…

This time, let’s go one step further. We have a Service Bus which is Web service notification enabled like last time. We can bind services to the bus, expose service endpoints as Web services, blahblahblah… But, this time, I am interested on having some real time monitoring of service invocations. It means that each time a message goes through the service bus (a service invocation in fact), I want to know (almmost) immediatly the service response time.
Hopefuly, the PetalsLink Distributed Service Bus I develop and use provides many extension points. One is the capability to add modules to the routing engine ie the software module each message must be able to go through on service request and response. So adding some router module which catch all the messages, timestamp them and then send this monitoring data to someone is quite easy. At the implementation level, this monitoring router module publishes monitoring reports to the service bus notification engine topic dedicated to monitoring stuff.

So, a client interested in monitoring data just has to register itself as subscriber to the monitoring notification topic. Every time a message is published in the topic, it will be delivered to all the subscribers. Up to the subscriber to display data as soon as it can. This is where Play, Web sockets and some cool javascript library came in. Since I never developed javascript stuff, I tried to find an easy to integrate solution to create some moving plots, asking twitter. I finally found the Smoothie Chart library which is really easy to use and updates graph in real time.

The high level architecture of the system can be defined as

The following video shows the result of the complete stack: Each time a message a service is invoked with SOAPUI, a Web service notification is sent to a Play application which subscribed to the monitoring topic, the Play application then pushes the data to the client by using a Web socket. Finally, the javascript code on the client side feeds the Smoothie chart which updates automatically. At the end, it is quite simple and efficient.

Oh, I forgot to say something: This took me 2 or 3 hours to create all this stuff… The code has been published on github in the dsbmanager-webapp project.

Some Play Framework, Service Bus, WS Notification and Web Sockets…

The Context

In the previous post I was introducing some tests I did with Play Framework and Web sockets. To summarize, it was just ‘about’ receiving messages on the Play! application and pushing them to the browser. This time, let’s go one step forward: Let’s add some infrastructure stuff to do something more real…

In the current case, I want to introduce a service bus which allows to create a real integration between service consumers and producers (I need to write another article in response to this nice videocast about integration from Zenexity guys ‘EAI & ESB n’apportent rien si les applications ne sont pas intégrables et interopérables’, but I really need more time to explain my thoughs…).
Using a service bus in the current case (and not for this case in fact…) must bring some added value. Here I choose to show that a service bus, even if it is not so lightweight at all, can provide some real cool features that you can have out of the box. Now, let’s add some event stuff to switch to an event-based world where we can have tons of event producers and let’s say thousands of event consumers. Since I can not setup such hude amount of actors, let’s say that we have one event producer and two event consumers:

• The event producer wants to publish some stuff somewhere. To illustrate, let’s say that we have a weather sensor connected to our platform.
• The platform provides a list of topics which producers can use to publish data. One is the weather topic which will be used by the producer above.
• The event consumers want to be notified on new weather data i.e. as soon as the weather sensor publish new data. To keep it simple, they need to subscribe to the weather topic provided by the platform.

To recap, in the event context, the event producer only knows the service he has to push weather data to, the event consumers just have to subscribe to a topic they are interested in. All the knowledge stuff about producers, consumers, topics and all the mapping is delgated at the service bus level. Yes, true it is exactly like in some topic-based messaging stuff because at the end of the day, it is topic-based messaging stuff…

The Stuff

Now we can speak about the stuff we are going to use in the software point of view for notificaiton actors…

• The event producer will not be a sensor but a Play! application. The application sends Web service notifications message to the service bus on a given topic.
• The service bus is (of course) the Petals Distributed Service Bus with some Web service notification modules inside.
• The event consumers are 1/A Play! application exposing a Web service to receive notification it subscribes to and 2/ A local java application displaying OS X notifications using Growl (let’s use JavaGrowl I published some days ago…). Note that the Play! application which have subscribed to notifications pushes them to their clients (browsers) using the funky Websocket stuff.

Let’s look at what really happens in a short video:

• I use the Play! powered application play-soap-wsnclient to subscribe to notification on behalf of the play-soap-websocket Web application.
• In Eclipse, I start a Web service notification powered Web service which subscribes to the same notification topic. Its listener is configured to use JavaGrowl to display incoming notifications.
• I use play-soap-wsnclient to send notification messages to the notification service hosted on the service bus.
• Once received, the service bus forwards the notification to all the subscribers using internal routing and WSN stuff.
• The play-soap-websocket Web application receives a notification and push it to the client browser using Websocket.
• At the same time, the Java application also receives a notification and display it using Growl.

One (or more) step(s) further…

And what if we have something which is not a Web service which subscribes to notifications? With the help of a service bus like Petals ESB/DSB, we just have to add a component which knows how to speak with the subscriber. For example, let’s say that SOAP is bad and that REST is the best thing ever. Can we have REST services receiving notifications? Yes, we can! Let’s just add the REST connector to the service bus. Another protocol/transport/format? Develop and add the new one. This is where the service bus can also help you. Hopefully, there are also other things which are possible with a service bus, we will see it later in other posts if I have time (as usual): Let’s think about business processes, service orchestration, transformation…
Next time we will have a look on what we can do if we use the distributed feature of the service bus, for example, receiving some notifications on one node and be able to notify subscribers which are bound to other nodes…

Source Code

• The event producer Play! application is on GitHub at play-soap-wsnclient
• The event consumer Play! application is also on GitHub at play-soap-websockets
• The Distributed Service Bus is not on GitHub but is open sourced on PetalsLinkLabs

L’acheminement de messages dans Petals ESB : #2 Le routage

Intro

Dans l’article précédent sur l’acheminement de messages dans Petals ESB, j’introduisais les notions, aujourd’hui nous allons regarder en détails la couche de routage du bus. Cette couche est largement appelée Router par les développeurs du Bus, et le sera donc dans cette article et dans la suite de la série.

L’archi simplifiée

L’architecture du Router utilise la notion très répandue et  fort utile de modules ie le Router invoque séquentiellement une liste de modules pour élire les services à appeller. A la fin de la traversée on se retrouve avec toutes les informations nécessaires pour appeler les services. On peut modifier cette liste en ajoutant ou supprimant des modules par configuration. Un schéma simplifié donne un Router avec cette tête :

Plus de détails sur l’utilisation des modules dans Petals et leur injection dans un vieil article (à non pas si vieux…).

L’implémentation

L’implémentation de base de Petals ESB est composée de 2 modules en émission :

1. Le module qui interroge le registre de services et retourne une liste de point d’accès (endpoint dans le jargon service) en fonction des informations contenues dans le message d’entrée.
2. Le module de résolution de la couche de transport. En fonction de la localisation des endpoints trouvés par le module précédent, un contexte est créé pour définir la couche de transport à utiliser (cette couche de transport sera détaillée dans le prochain article de la série).

On peut imaginer un grand nombre d’utilisations de ces modules. Personnellement, je les utilise à outrance par exemple pour :

• Ajouter des informations de timestamp sur les messages
• Notifier une couche de monitoring que des messages sont échangés entre consommateurs et fournisseurs de service
• Logger les appels
• Mettre à jour le choix de la couche de transport avec ma propre implémentation du transport de messages inter container
• …

Outro

Vous en savez un peu plus sur le routage dans le bus de service Petals. il faut vraiment retenir que cette couche est vraiment extensible et customisable sans grand effort, en effet pas besoin de recompiler le coeur de Petals pour ajouter des fonctionnalités pour la résolution des endpoints. Dans le prochain article de la série, nous regarderons de plus prêt comment sont échangés les messages entre les instances de Petals.