What’s new (and not new) with Petals ESB?


Here is the video of a quick talk I gave two weeks ago at OW2Con 2012 dealing with Petals Enterprise Service Bus.

He gives some background about what is Petals, how it can be used, why it is a distributed runtime, what is really new in the last version and finally what is planned in the next version.

I already spent many time explaining what we have in mind to push Petals in the cloud and this will be a reality in the next months as we are currently working on that exiting feature. More information soon, for real this time!

Comparing comparable things


Last week at OW2Con, Talend CTO talked about their data and service integration solution. This sentence impressed me (almost this one, not sure it was so short):

We have more than 500 connectors!

Wow! Great, let’s have a look to that! What is a connector? In the Petals ESB context connectors are the bindings represented in the upper part of this well-known image

Petals ESB

Petals ESB

So, we have almost 8 connectors. Looks that we are poor… So let’s look in details what is a Talend connector: http://www.talendforge.org/components/.

Can you see that? A Talend component is almost an operation in a service, or let’s say that it is an input/output from a data source. So a service is not as generic as a Petals service but it is a specialized service i.e. we can also have tons of ‘components’ for Petals ESB, it just means that we have to provide configuration artifacts for all the services that we find: Salesforce, Amazon EC2, S3, all the databases in the world, etc, … Oh no, wait! Don’t you see the Talend component on the bottom of the figure just above? Yes we have a Talend connector so we have 500+ components in Petals ESB 🙂

No really, Talend guys really do an incredible work and their recent press releases are really impressive. Congrats!

 

Playing in Brussels


Last week was the first annual review meeting of the Play project I work on since one year. I am involved at several levels in this project: from the architecture point of view, to the software integration and quality ones. On my side, my goal is to provide the efficient software infrastructure for events actors, or how to build an Event Driven Architecture based on Petals Distributed Service Bus. There are others points which have to be developed, especially all the platform governance stuff and Service Level Agreement for events, what we call Event Level Agreement.

We showed several things to the European Commission reviewers and we were also able to show an early prototype (this one was originally planned to be show at mid project ie in 6 months…). I made a video capture of the demo, which really needs to be explained…

  • There is the idea of a market place for events : The event marketplace. From there users are able to subscribe to things they are interested in. For now it is just a simple Web application which subscribe on behalf of the user to events through topics. This subscription is sent to the Play paltform using Web standards. Here we use OASIS Web service Notification to create this communication link.
  • Events are collected from several sources by events adapters. In the video above, we can see the user setting this FB status which is collected by the Play system and transformed into a notification which is published to the platform. There are also some Twitter adapters and Pachube ones which collects data in real time and publish them to the platform. This time again, we use Web standards for adapters communication.
  • Now, what happens in the video? The user logs in to event marketplace and subscribes to FB events. The user then publishes something to its FB wall (note that it is not mandatory to have the same user in the event marketplace and in FB. A user can subscribe to FB events and receive events from all the FB statuses collected from the FB application users). After the event propagation delay, the event marketplace display the event to the user.

So do we need such machinery to do things like that? No, if you want to do some other simple mashup portal. There are several components which are under active development: Storage and processing. Yes we store all events in the Play platform. This storage will be huge, but it is one of the project goal: Providing efficient and elastic storage in some P2P way. The need for this storage comes with the other important aspect of the project: Complex Event Processing. We will soon be able to create complex rules on top of events and be able to generate notifications based on past and real time notifications because we have efficient storage and real time stuff inside the platform. I am not an expert of this domain, so I can not give more details about that point but capabilities are huge! For example, we can express something like « Hey Play, can you send me a SMS me when there is my favorite punk rock band playing just around and I am not on a business trip and X and !Y or Z ». All of this intelligence coming from the processing of various sources I push since months in the platform coming from Twitter, FB, last.fm and other data providers.

Now let’s take some time to work on my OW2Con talk. The session name is pretty cool : Open Cloud Summit Session.

Petals DSB is not Petals ESB


Almost true… In fact Petals DSB uses and extends Petals ESB in several ways. When I started to think about extending the Enterprise Service Bus, it was just to avoid all the JBI stuff at the management level i.e. use a real, simple and efficient API. So I added many management stuff exposed as Web services : Bind external services, expose internal services (oh yes bind + expose = proxy), get endpoints, activate things, etc…
One other goal was to avoid to use closed protocols for inter node communication: The ESB uses at least three ports to create inter node communications for JMX, NIO and SOAP. So why not, just using an open protocol like SOAP and just one port? This is what I did, I changed some implementations to use the same port and the same protocol for all services.

All this stuff has been developed focusing on extensibility and easier development. This is mainly because the ESB can be hard to extend for newbies, there are so many things inside… Today the DSB is not only a Distributed Service Bus, it is also a framework so that developers can easily extend the DSB without the need to know how it works inside. Some examples? Want to expose a kernel service : Add the JAXWS @WebService annotation. Want to subscribe to Web service notifications : Annotate you Java method with @Notify. Want to be notified about new endpoints : Add the @RegistryListener annotation. That’s all, you do not have to search for the Web service server to expose your service, nor read all the WSN documentation to receive notifications, … Simple, efficient.

There are other things you can do, mostly all is detailed in the DSB page.

Let’s talk at OW2Con 2011


This year again, I submitted a talk proposal to the OW2 annual conference and it has just been approved by the OW2 management office.

While last year I spoke about some conceptual things around the Distributed Service Bus and the Cloud, this year I will go one step further with some live demonstrations not only dealing with service bus stuff, but also with some BPM tools and the Cloud stack we actively develop in the research team at PetalsLink. Here is my talk proposal:

All the services are moving to the Cloud, so are business processes. In this talk, we will show how to create collaborative business processes using an open source SaaS BPMN Editor. But designing business processes is not enough, why not running them in the Cloud? We will see that we can rely on a completely Cloud-aware SOA software infrastructure combining several open sources solutions such as a Service Bus and IaaS framework. The resulting ‘Cloud Service Bus’ allows the integration of in-house services in order to benefit from Cloud-based features such as elasticity, load balancing, service clustering and migration. This Cloud Service Bus will serve as the runtime basis of the business processes producing a Petals Cloud Stack solution. All in the Cloud, all open source!

I really hope to have time to work on some cool things to add more foggy-cloudy stuff and have things running on a real cloud infrastructure. I have many ideas in my mind these days and it is really really really cool.

Oh and I will also talk a bit about what we are currently building in the Play FP7 project. We have to show things in two weeks at the European Commission and these things are really interesting to share with OW2 attendees and staff.

BTW, I think that there is some free beer social event this year at OW2Con, see you there of course!