Tuesday, October 6, 2015

RHTE - Improve business process with microservice integration

The Red Hat Tech Exchange take place every year in Asia Pacific, this year is in Vietnam, Ho Chi Minh city. This year's theme is Train (gain and apply knowledge and skills), Share (best practices and experiences) and Enable (your personal and customer success).  I sure had a great time there.

In the event I have share 2 sessions, the first one I did with Thomas Qvarnström, it about handling large data and how JBoss Data Grid can help. You can find the slides in my previous post.  

My second session talks about the difference between BPM process and Camel route, when to use what and share my thoughts on the best practices of when to use it. The agenda of my talk :

Sometimes, building enterprise class business processes is difficult, it involves  complex application integration, many requires co-work between cross functions. See how JBoss BPM suite and JBoss Fuse can do! In this talk, we are going to talk about possible architectures, go through a live demo and see how it's done.

And here is the slide:

Here are some note about the transaction, when I say BPM has no transaction, I mean the automatic transaction. In BPM the transaction boundaries are between persistence nodes, so if you have 2 human tasks in the process, your process will be automatically splits into 2 or 3 transactions. If something goes wrong, BPM engine will auto roll back all nodes between persistence node. But what was done was done, say you have already called a Java bean to deduct values from databases. Although the process will start over and redo the process again, the value of data still need to handle specifically to return to original state in all the databases. But for Fuse, it can automatically rollback as it supports transactions and XA transactions for Database and JMS. It would be a great practice if we can wrap the transaction into a specific service in Fuse and allow process to call the services. If anything goes wrong, in the transaction, it will automatically row back. And Process can then be start over again, as it will also restart by BPM engine.

I hope to see you in the event next year :) 

Friday, October 2, 2015

Red Hat JBoss Fuse - Tools make your world go round, the demo

Few days ago I posted the slides of what suppose to my vJBug presentation, for the talk, I have planned to showcase the tooling by showing what a developer would normally do when they would develop application with JBoss Developer Studio.  Here are the things I go over,
  • How to debug Camel application using the tooling provides.
  • Data mapping between POJO and XML, JSON with Dozer with drag and drops only. 
  • Deploy and package an OSGi ready Applications. 
The story of the demo was pretty simple, it's a bookstore integration application, takes in order in XML format, in the xml, contains multiple orders, each order will have the customer information, and has the list of books the customer would like to order. 

<?xml version="1.0" encoding="UTF-8"?>
<order product="electronics" id="2012_0001">
<customer id="A0001">
            <name>Antwerp Zoo</name>
                <article id="A0001">
                <article id="A0011">
<order product="books" id="2012_0002">
<customer id="A0002">
            <name>Antwerp Zoa</name>
                <article id="B0002">
                <article id="B0202">

We are going to process the order, if the customer is VIP by convert each orderline (book) into Java POJO and print the POJO, for non-VIP customers, we are simply going to print out the XML part of the entire order.  Here is the video of the demo, please enjoy it. 

Tuesday, September 29, 2015

Red Hat JBoss Fuse - Tools make your world go round, the slides and basics

This is suppose to be my vJBUG topic, it's about better tooling saves you valuable time when developing! During the talk, I have develop a JBoss Fuse Application using the latest tooling available in JBoss Fuse! Here are the things I go over,
  • How to debug Camel application using the tooling provides.
  • Data mapping between POJO and XML, JSON with Dozer with drag and drops only. 
  • Deploy and package an OSGi ready Applications. 
Before the demo, here are some basics. This video goes through and tells you all about the tools available for JBoss Fuse.

The slide for the presentation is here:

demos will be on next post.

Thursday, September 24, 2015

RHTE - Supercharge your integration services

Red Hat Tech Exchange has taken place in Vietnam, Ho Chi Minh city two weeks ago, it is a great event held by Red Hat in Asia Pacific Region. It is open to all Red Hat partners who are interested in learning what Red Hat is doing recently, see what the trend of the open source world, basically it is a great event to share your knowledge and experience, to meet other enthusiastic people.

I am very fortunate to talk in this great event, to talk about the things I have been working on and even discuss it with many. Also got lots of great ideas too. So here are the slide.

My first talk was with Thomas Qvarnström about how to handle large size data in JBoss Fuse and how JBoss Data Grid can help in the situation.

Here is the agenda of the talk, we will be talk about this in the up coming webinar on 24th Sept.

Integration often involves storing, retrieving, and transforming data. Using a traditional database in your integration is likely to becomes a bottleneck that is expensive and hard to scale. By combining the agile, lightweight, Enterprise Integration pattern (EIP) based integrations using open source technology with scalable in-memory storage, it’s possible to achieve near linear scalability and boost performance of your integration services.

Slides of the talk,

There are some interesting questions,

Why do you suggest JBoss Data Grid as the medium for claim check pattern, why do you suggest starting up a local JDG? 

Because in JDG, you can set the eviction and expiration policy, so the it avoid running into OUTOFMEMEORYEXCEPTION, also it manages and remove the unwanted entry after use.

When do I use Messaging Topic and when do I use JDG? 

For people who is concern about the sequence of when the data arrive, and if the data seems to have the nature of "fire and forget", that is, when client only get the data when it is subscribe to the broker. And JDG is best if you have client that will keep asking for repetitive data. Or if the data need extra processing we can utilized the Map-Reduce and Distributed calculation in JDG for it.

Monday, September 21, 2015

New top menu

OK, I have been saying this for a long time, updating my menu list in my blog, so it's easier to find the content. Was really busy working and traveling, so never actually go the time to sit down and does the menu and categories them as I wanted it. So, in this lazy sunday afternoon, I decided to finally do it.

So, on the top of my blog, you will find four new tabs,

  • Fuse/AMQ & Demo 
  • Tips & Tricks
  • Multi-Product & Fuse
  • Workshops & Getting Started Demo

Under Fuse/AMQ & Demo you will find all the component demo and technical blog related to Fuse and AMQ. Also about the slide of different events, and new capabilities of different versions.

Under Tips & Tricks, there are some tips I collect or found when developing applications in JBoss Fuse.

Multi-Product & Fuse will have all the posts related integrating between Fuse and other middleware, such as JBoss BPM Suite, JBoss Data Grid and OpenShift.

The last tab contains all the workshops post and the getting started demo I did to get people started with the technology.

I hope by categorizing the post will allow you to find the information you need :), and please let me know if you have any better idea on how to categorize it. 

For those of you wondering about my Blog name "Christina のJ老闆" it's simply means Christina's JBoss.. That's all. :)

Friday, September 18, 2015

Red Hat JBoss Fuse - Load Balance Quartz Cron Jobs

In the beginning of the year I posted a post on how to do cron jobs with Apache Camel in JBoss Fuse, the reason I am doing it is because sometimes we need to off-load the integration jobs and schedule them to run in less time during busy hours. Cron Jobs allows you to run the integration application at the certain period of time you set it to run, it will be mostly in the night time. We use the Quartz2 component in camel to achieve this.

Just to recap a little on the scenario I had. Basically it starts up two instance that runs the same cron jobs, the job simply writes a file to an FTP server. The job is implemented by Quartz2 and setting it to run twice every minute,both instances are connected to a database (In my code I used H2 for easy installation, but there is another option to use PostgreSQL database, see my previous blog). The reason we want to connect them to database is to make sure these two instance will share the loading of the the job. This is very useful if you have a long running job, and can divide them into different parts and share the loading among distributed resources.
Don't forget to setup the database beforehand, setting the database table schema, download the Quartz library, you will find the scripts to setup tables are under docs/dbTables, find the sql file of the database you are using. Run the sql file to create all the tables needed in your database.

And let's go back to the camel project itself. The setting of the database are in the properties.

<bean id="quartz2" class="org.apache.camel.component.quartz2.QuartzComponent">
    <property name="propertiesFile" value="org/blogdemo/demojob/myquartz.properties"/>


Since we are going to deploy this with fabric, we will place the properties file under main/fabric

Inside myquartz.properties contains detail about the shared database such as driver, ID , password and also the setup detail like, number of threads, driverDelegateClass used and most importantly if this setting is clustered.

# Main Quartz configuration
org.quartz.scheduler.skipUpdateCheck = true
org.quartz.scheduler.instanceName = DatabaseClusteredScheduler
org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.jobFactory.class = org.quartz.simpl.SimpleJobFactory
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.dataSource = quartzDataSource
org.quartz.jobStore.tablePrefix = QRTZ_
org.quartz.jobStore.isClustered = true
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.jobStore.clusterCheckinInterval = 20000

# JobStore: JDBC jobStoreTX
org.quartz.dataSource.quartzDataSource.driver = org.h2.Driver
org.quartz.dataSource.quartzDataSource.URL = jdbc:h2:file:~/h2/cronjob;AUTO_SERVER=TRUE
org.quartz.dataSource.quartzDataSource.user = sa
org.quartz.dataSource.quartzDataSource.password = 

org.quartz.dataSource.quartzDataSource.maxConnections = 5

The camel route is very simple and easy, 

<route id="quartzRoute">
        <from uri="quartz2://myGroup/myTimer?cron=0/45+*+*+*+*+?"/>
        <setHeader headerName="CamelFileName">
            <simple>MyJob-${date:now:yyyyMMdd HH:mm:ss}.txt</simple>
            <simple>${date:now:yyyyMMdd HH:mm:ss} Hello This is the content from Quartz2 - {{instancename}}</simple>
        <to uri="sftp://demo@localhost?password=ZAQ!2wsx"/>
Basically, it is kicked of by the cron job, on every 0 and 45 second, and then creates a file under name MyJob-YYYYMMDDHHMMSS.txt with content Hello This is the content from Quartz2 in the file. 

And to deploy the application to JBoss Fuse, I need to create a fragment bundle for connection pooling, here we use c3p0 and the JDBC driver. And then deploy both onto the same instance. 

Deploy another container with the same job profile and fragment bundle. 

Add another profile,  that prints the content received in FTP server. 

If you encounter any problem when deploying, it's probably OSGi not loading the bundles correctly, you can simply restart the container or make sure the following bundles are activated in the container. (Check by using osgi:list
[ 157] [Active     ] [            ] [       ] [   80] Apache ServiceMix :: Bundles :: c3p0 (, Fragments: 148
[ 167] [Active     ] [            ] [       ] [   80] H2 Database Engine (1.4.181)
[ 172] [Active     ] [Created     ] [       ] [   80] Quartz Two Demo (1.0.0.SNAPSHOT)

Take a look at the log running quartzprint profile, you will find the jobs are clustered and loads are shared between two nodes. 

And detail of the job can also be found in console of each container.

You can find the Demo code and detail instructions here

Tuesday, September 15, 2015

Red Hat JBoss Fuse - Tips, Driver not found in JBoss Fuse

I have bumped into ClassNotFound Exception problem several times when I need to add Database driver or migrate non-OSGi libraries into my JBoss Fuse applications.

Could not load driverClass org.postgresql.Driver

java.lang.ClassNotFoundException: org.postgresql.Driver not found by org.apache.servicemix.bundles.c3p0 [135]
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1556)[org.apache.felix.framework-4.4.1.jar:]
at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:77)[org.apache.felix.framework-4.4.1.jar:]
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1993)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)[:1.7.0_60-ea]
at java.lang.Class.forName0(Native Method)[:1.7.0_60-ea]
at java.lang.Class.forName(Class.java:190)[:1.7.0_60-ea]

The problem is mostly due to the class loading algorithm in OSGi. As each module need to specified what package it needs setting these packages in "Import-Package" inside META-INF/MANIFEST.MF of the bundle. But sometime your application load the driver dynamically in runtime, or the 3rd party library did not specified the import in their settings.

What I normally do when I am in the middle of development is to temporary resolve this problem in the JBoss Fuse Karaf container, for instance this example, I am using the c3p0 pool in servicemix, but it needs to dynamically lookup the postgresql database. What I will do is to install both servicemix c3p0 bundle and postgresql bundle

osgi:install mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.c3p0/

Bundle ID: 148

osgi:install wrap:mvn:postgresql/postgresql/9.0-801.jdbc4

Bundle ID: 148

then do a osgi:list to list all available bundles, make sure they are all active

[ 100] [Active     ] [            ] [   80] Fabric8 :: Web (1.2.0.redhat-133)
[ 102] [Active     ] [            ] [   80] Bean Validation API (1.1.0.Final)
[ 123] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: jackson-module-scala_2.10 (
[ 124] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: javassist (3.12.1.GA_3)
[ 127] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: json4s (
[ 128] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: reflections (
[ 129] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: swagger-annotations (
[ 130] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: swagger-core_2.10 (
[ 131] [Active     ] [            ] [   80] Apache ServiceMix :: Bundles :: swagger-jaxrs_2.10 (
[ 145] [Resolved   ] [            ] [   80] Scala Standard Library (2.10.4.v20140209-180020-VFINAL-b66a39653b)
[ 148] [Installed  ] [            ] [   80] Apache ServiceMix :: Bundles :: c3p0 (
[ 149] [Installed  ] [            ] [   80] wrap_mvn_postgresql_postgresql_9.0-801.jdbc4 (0)

and then use dynamical loading for servicemix c3p0 to look up the postgresql

dev:dynamic-import 148
Enabling dynamic imports on bundle org.apache.servicemix.bundles.c3p0 [148]

But this is not a good idea when you want to package the application, I used the dynamical import when developing because I am the only person running the container, and the dependency may change rapidly when I am developing, but this is not the best way to do in OSGi, instead of a strict modularized system, it will search the public exported packages to find a match instead of the careful normal calculation. Not a good practice. That's when I want to deploy my application into production, is I will create a fragment bundle. There is no big secret about this  "fragment bundle", all you have to do is to specify a Fragment Host. And all the content in your fragment bundle will be made available to that host. From my example, all I have to do it to create an empty bundle, and have my servicemix c3p0 bundle as the fragment host and import postgresql host.

Fragment-Host: org.apache.servicemix.bundles.c3p0
Import-Package: org.postgresql 

To be more specific, create a JBoss Fuse blueprint project, remove all the unnecessary blueprint.xml or java classes (Basically an empty project), Then in your pom.xml, find the maven plugin, add your fragmented Host and package you need in Import-package tag.

      <!-- to generate the MANIFEST-FILE of the bundle -->


And then deploy this bundle first along with your application. This is my MANIFEST.MF look like after packaging it.                

Manifest-Version: 1.0
Bnd-LastModified: 1441500854071
Build-Jdk: 1.7.0_60-ea
Built-By: christina
Bundle-ManifestVersion: 2
Bundle-Name: Camel Blueprint Route
Bundle-SymbolicName: demojobfrag
Bundle-Version: 1.0.0.SNAPSHOT
Created-By: Apache Maven Bundle Plugin
Export-Package: org.blogdemo.demojobfrag;version="1.0.0.SNAPSHOT"
Fragment-Host: org.apache.servicemix.bundles.c3p0
Import-Package: org.osgi.service.blueprint;version="[1.0.0,2.0.0)",org.p
Tool: Bnd-1.50.0

The problem should go away if you done it correctly.