Monday, July 27, 2015

Red Hat JBoss Fuse - Getting Started Home Loan Demo version 6.2 Part 1

This is a update to my existing demo for people that are getting started to know Fuse, and wants to get their hands dirty and develop a JBoss Fuse project for the first time.

There will be a series of blogs with video coming after this one, it is focus on playing with the integration framework in JBoss Fuse, which base on the open source project Apache Camel. And in the last video of the series I will deploy the integration projects onto the OSGi container and manage it with Fuse Fabric.

The story behind the home loan demo is to prepare housing appraisals base on the content given real estate vendor. The appraisal needs to talk to local database, external google map API and also provide information through a web Restful API.

For those of you have been following my work may know, I have made some changes to the demo, the old demo has place all integration functions in a single project and deploy on single container. But when I started to maintain, because it's been a while from when I developed this project, took me sometime to remember and figure what which part of the application was doing what. Then I started to think it's a better idea if I break it down into separate project, each handles single functions. Now the architecture of this integration project is broken down into 4 separate independent projects and then deploy to containers distributed in the fabric. Using messaging broker as the medium between these distributed integration parts.




In part one, we will start taking in XML files from different vendor's home loan application, they will place their customer input into a directory listening by JBoss Fuse. The XML files are in 2 different formats, one with it's customer data, the other contains housing details. Both files will be place into same folder either by FTP or Batch generated overnight.


To begin, make sure you have download the JBoss Developer Studio to follow along the video:


In this demo, you will learn

File Endpoint
Which provides access to file systems, allowing files to be processed by any other Apache Camel Components or messages from other components to be saved to disk.

Content Base Routing
From the EIP patterns allows you to route messages to the correct destination based on the contents of the message exchanges.

xPath Expression
Allow an Expression or Predicate to be used by parsing in the DSL or Xml Configuration.

Set up the activemq for messaging broker.
The ActiveMQ component allows messages to be sent to a JMS Queue or Topic; or messages to be consumed from a JMS Queue or Topic using Apache ActiveMQ.

  <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
      <property name="brokerURL" value="tcp://localhost:61616"/>
      <property name="userName" value="admin"/>
      <property name="password" value="admin"/>

  </bean>

Hope you find this useful. The source code of the entire project can be found here:
https://github.com/jbossdemocentral/jboss-fuse-homeloan

Tuesday, July 21, 2015

JBoss Fuse - Enterprise application architecture with JBoss Fuse and JBoss BPM suite

Modern enterprise application consists of cross-communication between departments, dealing with multiple type of technologies from different era. Having a complete well design application in today's enterprise requires more then just tying up all the nots, that is, connecting all the systems, application together. One simple missed decision can lead to disastrous results of having to adjust every single application for every small changes. So it is important to define different role and responsibility of what each individual module of the application should do, and making sure they are independent form each other, so we can limit the impact of changes needed, but also flexible enough to cope with enhancements.

With JBoss BPM suite and JBoss Fuse,they are great when it comes to helping developer who wants to build an enterprise application. JBoss BPM Suite allows developers to create and monitor business process and rules. And JBoss Fuse can help with the application integration. To achieve a better software architecture we want to make sure the business aspect of the application are separated with the other. When combining these two middleware together, I found sometime it's pretty hard to define what belongs to business layer and what should go with integration? So the steps I normally goes through to define what goes where is

  • Define the business process and the people or parties involve in it. 
  • Define the services needs to provide for each step and where they belong. 

For instance, when our inventory department produces a report of old stock and ready for clearance, it require co-ordination between departments, such as finance for cost analysis to set how much lower the price can go down to and create a report, ask manager to approve the clearance base on the cost analysis. It might also need marketing for campaign on television, web advertisement. Lastly we need to make sure all shops are aware of the clearance and ready for this by automatically updating labels, and make rooms for the dispatch inventories … etc.



First thing first, we need to find out where are the business processes, and are there any existing applications providing services, as well as, are there any human interactions allowed?  Do I need to build services for this process, what are the granularity of these services?  We want to automate the process of product clearance, therefore a process is created to deal with product clearance. In the process, we need automatically calls a services that does cost analysis by accessing databases, applying business rules to calculate costs, generate reports, and let managers to review. It is then passed onto marketing team for them to organize advertising, and then a service kicks off, so each shops with the information they need to support the clearance. Here we have identify a process, it require human task interaction in both Finance and Marketing department.

For me process should be a description that even a non-IT personal should understand, this can be done by either a computer application service(either embedded or external services), or needs to pass to a human for further interactions. A process should not have transaction, no rollbacks, transactions should be capsulate in a service. You need to specify in the process to handle exception or error occurred in the process.

We can then start implement this process in JBoss BPM Suite.

Let's see the services we need for this process, few thing to watch out for on the services, look for existing services and reuse them as possible. Avoid building complex business algorithm into integration service, make it simple and small (microservices?), if really need to include these logic, use Drools, or java bean to encapsulate and package them, so whenever if we need to change it, it minimize the impact to integration layer by simply replace the package. In our example case here are the services I defined:

A. A automatic polling services from the inventory that kickstart the clearance process. This polling services pulls


B. Cost analysis that calculate the cost and generates report.


C. Generates and print labels for the clearance.


D. Print out shipping schedule.

E. Handle situation when the clearance was rejected by managers.
All the above services can be easily done by utilizing the Enterprise Integration Pattern in Apache Camel of JBoss Fuse and then deploy onto distributed lightweight ESB container, which can be provision, manage and monitor with JBoss Fuse too.

And put all these together, it'll become an application architecture for this particular case.


As for later monitoring of the application in maintenance phase, in JBoss Fuse, by going to Fuse Console, you will see how each one of the service are doing and manage it through the console accordingly. As for processes, inside BPM suite, there is a BAM, which allows you to see the statistics of processes, tasks and even business related data by configuring it.


Thursday, July 16, 2015

JBoss Fuse/A-MQ - Achieve fault tolerant messaging with MQ Gateway

Sometime ago I posted a blog about achieve fault tolerant messaging, in the article I mention using the fabric registry discovery to hide all the IP address and port detail in the client side. With new Fuse 6.2 you will not be able to find this anymore.  So how do we achieve fault tolerant messaging in the new Fuse?

In JBoss Fuse 6.2, we can do it by starting up a MQ Gateway. What it does is it provides a single IP and port to accept connections from clients.  All the individual IPs and Port of the broker are hidden away tothe client, the client only needs to know the IP and port of the MQ Gateway (Default Port: 61616). The gateway will discover all the brokers in the fabric, and no matter what the incoming protocol is, it can be OpenWire, MQTT, STOMP and AMQP. The gateway will see which broker is available in the host specified for the protocol, and connect the client to the broker. If multiple brokers are available in the host group, gateway can dispatch request to them, by this we can achieve load balance. There are 3 different way to do load balance, Random, Round Robin and Sticky.

If any thing goes wrong with the broker a client is connecting, and got disconnected, the gateway will look for another available broker in the group. This gives you high availability to your service.

To startup a MQ Gateway, simply choose the gateway-mq profile to the container.

It will automatically find brokers in fabric, 



I updated my failover demo instead of using fabric registry discovery, I change my application and connect it to MQ Gateway.  In this demo, you will first need to create a broker failoverMS, choose Master/Slave type and use blogdemo as Group.





Click on the red triangle to start provision the brokers. It will take you yo the page to create container for the broker profile.


And will start create number of container you specified in the configuration.


Then we can startup the MQ-Gateway, before we do that, go to the gateway-mq profile and edit the io.fabric8.gateway.detecting.properties file. Change port to 8888 and defaultVirtualHost to blogdemo.



And add the gateway-mq profile to a container, as previously stated.
Let's take a look at the client application, it's written with Camel. Look at the configurations in the Camel route, here you see no matter if it's an OpenWire or MQTT protocol, it is set to tcp://localhost:8888, which is the IP and port of MQ Gateway.

This application send a message to the broker, through OpenWire and MQTT protocol every 5 secs. Deploy the profile "" to a new container.



Go to container console of testcon, you will find the camel routes are running



In the MQ Gateway, you can see the request was dispatched to the broker we have created.
And the messages was sent to broker.






Video can be found here.


The application code can be found here.
https://github.com/jbossdemocentral/jboss-fuse-mqgateway-failoverdemo

Enjoy!

Tuesday, July 14, 2015

JBoss Fuse - Tips, Deploying Profile without Fabric8 maven plugin


If you want to take advantage of the Fuse Fabric deploy plugin, with new JBoss Fuse 6.2 requires you need to have Apache Maven 3.1.1 - 3.2.4. Otherwise, you may bump into error. The reason is because there has been a major API change in maven after 3.2.4. 

Here is the exception you might see if you don't have the correct maven version. 


No implementation for org.eclipse.aether.connector.wagon.WagonProvider was bound.
while locating org.eclipse.aether.connector.wagon.WagonRepositoryConnectorFactory

1 error
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1018)
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1044)
at org.eclipse.sisu.space.AbstractDeferredClass.get(AbstractDeferredClass.java:48)
at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:86)
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:54)
at com.google.inject.internal.ProviderInternalFactory$1.call(ProviderInternalFactory.java:70)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:115)
at org.eclipse.sisu.bean.BeanScheduler$Activator.onProvision(BeanScheduler.java:176)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:126)
at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:68)
at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:68)
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:46)
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1066)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:36)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1009)
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1059)
at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1005)
at org.eclipse.sisu.inject.LazyBeanEntry.getValue(LazyBeanEntry.java:81)
at org.eclipse.sisu.wire.EntrySetAdapter$ValueIterator.next(EntrySetAdapter.java:102)
at org.eclipse.aether.internal.impl.DefaultRepositoryConnectorProvider.newRepositoryConnector(DefaultRepositoryConnectorProvider.java:103)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run(DefaultMetadataResolver.java:603)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

So there are two things you can do, 

1. Change you maven to 3.1.1 ~ 3.2.4 and you are good to go.

2. If for some reason, you need specific version in maven, probably the best way is not to use the fabric maven deploy plugin. So we have to do everything manually. What it basically does, 

  • A. Compile and package the application into maven repository.
  • B. Create a profile to deploy into Fuse Fabric.
  • C. Assign repositories, features and bundles into profile. 

For example, in my Travel agency demo, with the maven plugin, these are the configurations I need to provide. 

        <fabric8.profile>demo-travelagency-bookingservice</fabric8.profile>
<fabric8.parentProfiles>feature-camel mq-client</fabric8.parentProfiles>                                                       
<fabric8.bundles>mvn:com.h2database/h2/1.4.181</fabric8.bundles>
<fabric8.features>camel-jpa activemq-camel hibernate jdbc</fabric8.features>  

With the plugin, it will create a profile named "demo-travelagency-bookingservice" and have feature-camel and mq-client as the parent profile. Add camel-jpa , activemq-camel, hibernate, jdbc as features into profile. and lastly add two bundles, h2 and the application itself, bookingservice

Before I turn this into manual step, there are some settings I want to create beforehand, so it's easier to manage. Create a feature project, I added a new feature repository with a feature name "booking-service" that contains all the depended features and bundles in it.

<features name="TravelAgencyRepo" xmlns="http://karaf.apache.org/xmlns/features/v1.0.0">

  <repository>mvn:org.apache.camel.karaf/apache-camel/2.15.1.redhat-620133/xml/features</repository>
  <repository>mvn:org.apache.activemq/activemq-karaf/5.11.0.redhat-620133/xml/features</repository>
  <repository>mvn:org.apache.activemq/activemq-karaf/5.11.0.redhat-620133/xml/features-core</repository>
  <repository>mvn:org.apache.cxf.karaf/apache-cxf/3.0.4.redhat-620133/xml/features</repository>
  <repository>mvn:io.fabric8/fabric8-karaf/1.2.0.redhat-133/xml/features</repository>
  <repository>mvn:org.apache.karaf.assemblies.features/enterprise/2.4.0.redhat-620133/xml/features</repository>

  <feature name="booking-service" resolver="(obr)">
    <bundle start-level="20">mvn:com.h2database/h2/1.4.181</bundle>
    <feature version="2.15.1.redhat-620133">camel-jpa</feature>
    <feature version="5.11.0.redhat-620133">activemq-camel</feature>
    <feature version="4.2.19.Final-redhat-1">hibernate</feature>
    <feature version="2.4.0.redhat-620133">jdbc</feature>
  </feature>

</features>

Now we can start creating the profile, (make sure you have Fuse installed with Fabric created too). We first create the profile with two parent profiles

fabric:profile-create --parents feature-camel --parents mq-client demo-travelagency-webendpoint

And then add the repository we create previously to the profile.

fabric:profile-edit --repository mvn:org.blogdemo.travelagency/features/1.0/xml/features demo-travelagency-webendpoint

Add "booking-service" feature from the repository we just added to the profile.

fabric:profile-edit --feature webendpoint-service/0.0.0 demo-travelagency-webendpoint

Lastly add our application to the profile and we are done.

fabric:profile-edit --bundle mvn:org.blogdemo.travelagency/webendpoint/1.0 demo-travelagency-webendpoint


That's all you have to do :)

Thursday, July 9, 2015

JBoss Fuse - Tips, Display Camel counter in Fuse 6.2



This is a frequently asked questions, in previous version of JBoss Fuse, when you have Camel route deployed in the Fabric container, in the container you will see a complete diagram of your camel route and on top of it you will also find the number of messages flow through it. But when JBoss Fuse 6.2 comes out, you will find the counter in the route are missing.


Here is what you have to do to turn the counter back on again, under etc/auth/jmx.acl.whitelist.cfg add the following

#Camel Counter
org.apache.camel;getIndex=bypass
org.apache.camel;getRouteId=bypass
org.apache.camel;getState=bypass
org.apache.camel;dumpStatsAsXml=bypass
org.apache.camel;getExchangesInflight=bypass
org.apache.camel;getProcessorId=bypass

This will work if you do not turn your fabric on, but once the fabric is create, you will need to update the jmx.acl.whitelist.cfg under your default profile.




Here is the result,



This problem will go away by Fuse 6.2.1.


Monday, June 29, 2015

JBoss Fuse - What's new in JBoss Fuse 6.2

Have been busy for a while, other then working on my Lab for Summit, I am also working on migrating my early Home Loan getting started demo to JBoss Fuse 6.2 which is going launch later in the month.

What is going on JBoss Fuse 6.2.

First of all, MY PERSONAL FAVORITE, which is the huge improvement on the Development tool, JBoss Developer Studio. One of the thing I always wants to have is DEBUGGING. Now we can actually set the debug point on every endpoint of the applications, and then through the debug view, we can now see the stack frame for the suspended threads for each route. What I found the most helpful is the view that allow you to trace or even change the header and body in Camel message. The way to start the debugging is just like other Java application. In JBoss Developer Studio, click on "Run as Debug..". Other really cool improvement is the addition of commonly use component available on Camel canvas. The properties now provides the out of box built in configuration field that is going to help developer on what can be configure in that particular components rather then going back and forward to the fuse documentation. To improve development experience, one thing I notice from the tooling, when you drag n drop a component in the canvas, the dependency in the pom.xml file is automatically added.

There are also improvement on the camel components (version 2.15.1) and quite a few new connector added. We have improve the salesforce, Olingo component, added the connectors to Dropbox, Box.com  and Google Drive. To me, because I need to interact Database very often, now I can finally use Simple expressions as parameters in the SQL components. I have not play with all of them yet, but be sure to check them out. Another really useful feature in this release is the REST DSL, this allow developer to  establish the REST web endpoint with few lines of setting versus creating and configure in POJO then exposes as the old ways.

Another important new feature is the Role Base Access Control. Powered by JAAS implemented. Many enterprise needed this in real production environment, what this does, it authorized user's right on access the Fuse console, JMX, command line mode and the service allowed in OSGi.

Here are main update themes in JBoss Fuse 6.2 ,

 * Improved tooling & Usability
 * Foundation for creating and sharing API services (REST DSL, Swagger support)
 * Additional connectivity (Box, Dropbox, GoogleDrive, OData, SalesForce APEX API)
 * Enhanced IoT support (Camel-Kura, MQTT improvements)
 * Messaging enhancements..


You can find out more details in Jon's blog.
http://janstey.blogspot.com/2015/06/jboss-fuse-62-is-out.html

Friday, May 22, 2015

Integration Project- Micro Services Migration Story with JBoss BPM Travel Agency - Part Five

Part Five- Migrating Web Services.


This post is the fifth part of a JBoss Travel Agency Integration Demo series, and we are going to talk about how to migrate from an existing web service to JBoss Fuse. And it is so much easier then you can ever imagine. When migrating web services, we know there will be a contract already exists, it is the WSDL file. So all we have to do is just to implement this contract. If you want to know more about the micro service architecture of the demo, please see this post for more details. For now, all you have to know, it that now we are going to migrate from an already existing web service to JBoss Fuse, and then dispatch these request to difference message queue in broker, so later on the backend service can retrieve and process it.

After receiving hotel request from client, the web service will be dispatched to messaging queues. And because the result needs to go back to the client, we need to make sure the request comes back, there for we need to set the JMS to in-out to make sure it gets the return message back. Also, we need to transform the data from client to the format backend service understand, which is JSON.


OK, first things first, from the WSDL file we have, by adding the cxf-codegen maven plugin to generate the related object and services, and direct it to read the wsdl file locate in the project.
      
      
        org.apache.cxf
        cxf-codegen-plugin
        2.7.0.redhat-610379
        
          
            generate-sources
            generate-sources
            
              ${basedir}/target/generated/src/main/java
              
                
                  ${basedir}/src/main/resources/wsdl/HotelWS.wsdl
                  
                    -impl
                  
                
              
            
            
              wsdl2java
            
          
        


And then we can start setting up the web service endpoint



Rather then just a simple connection to the messaging queue, this small application needs interact with messaging a lot, so we are going to create a pool to the broker.

    
    
    


    
    


    
    


    


Take a look at the WSDL, here you see it has three services, with different inputs, so we are going to implement these services.

The first route is the universal endpoint, where it will take in the entire request, and dispatch them to the routes that actually implements the web service.


Then we go on can create the route that handles booking and cancel booking, it simply converts the input into whatever format is needed.

Booking Route

Cancel Booking Route


For available hotel request it is a bit more complex, because the web service requires a specific object to return, so here we are going to do map JSON content and turn it into the specific object.
package org.blogdemo.travelagency.hotelwsendpoint;

import java.util.Map;

import acme.service.soap.hotelws.Resort;

public class ListHotelBean {
 
 public Resort getResort(Map promotedResorts){
  Resort resort = new Resort();
  resort.setHotelId((Integer)promotedResorts.get("hotelId"));
  resort.setHotelName((String)promotedResorts.get("hotelName"));
  resort.setHotelCity((String)promotedResorts.get("hotelCity"));
  resort.setAvailableFrom((String)promotedResorts.get("availableFrom"));
  resort.setAvailableTo((String)promotedResorts.get("availableTo"));
  resort.setRatePerPerson(((Double)promotedResorts.get("ratePerPerson")));
  
  return resort;
 }
}

Register the mapping bean in camel context,
    

And lastly, the available route.


And that's it. More detail instructions are available in the slides below.



 For more detail side in BPM Suite, please click here to Eric Schabell's post.

Micro Services Migration Story with JBoss BPM Travel Agency

  Part One: Introduction and Installation
  Part Two: Moving towards Microservices
  Part Three: Retrieving Data with JDBC 
  Part Four: Persistence with JPA
  Part Five: Migrating Web Service