Glassfish ESB, easier than you think. Part I

Over the last 2 years I got to learn, use and enjoy OpenESB, we switched to GlassfishESB when it was released with the support of Sun. Unfortunately I am not using it anymore at work and I missed the 2.2 release in which there seems to have been a couple nice changes, but to get to the point. The concept of a ESB (Enterpise Service Bus) will seem daunting or intimidating to some people, this post is to get rid of that and show how developer friendly GlassfishESB can be. Another huge benefit people do not often mentioned is working on very large distributed applications a tool like GlassfishESB with it's visual representation of your entire deployment makes maintenance and system structural updates a lot simpler.

Anyways to begin...
Go download GlassfishESB / Netbeans / Glassfish and any other components you need here

Install it, start it up and you are probably half way done, it is that simple.

Ok so first thing you want to do is create a standard Java project.. and create some classes, for this demo I am just creating a simple "message" with a Header and Body:


Now the next bit is my way of "cheating" to actually have anything connect to the ESB process you need to have an webservice, you get that by defining a WSDL and XSD and adding them to the BPEL, but who wants to type out all that boring XML...

So now create an EJB Project:


Click next..



Click next... Finish

Add the Java project to the libs of the EJB project.



Create a new webservice...



And make sure that ... the EJBs' code looks like



Now for the cheat ...



This will generate the WSDL and XSD...

Now we create the BPEL Project


Copy the WSDL and XSD from the EJB Project..
Open the bpel... Drag the WSDL onto the little orange circle on the left, the following will popup:



Now your BPEL looks like:




Add a Receive, Assign and Reply to the BPEL, the edit the Receive:



Edit the Reply:



Click on the Assign and then on "mapper" in the bpel editor window, drag the contents of the In variable to the contents of the Out variable



Save, and done...

Now just add a composite application.



Open the Service Assmbly and drag the BPEL project onto the JBI module section:

:

Add some Endpoints from the Palette, connect the arrows:



Deploy.. and you are done.

If you like you can add a test to the Composite Application, very simple, On the Test folder, right click add, choose your main WSDL, choose the method you wanna test, it generates test xml based on the XSD. On the properties of the test set the port in ${HttpDefaultPort} in the properties of the test...and test...

In Part II , I will add some more EJBs, so that this actually becomes a functioning application and some more awesome GlassfishESB functionality.

So your POM is quite nice.. now what?

Last week, I described some of the Maven lessons and configurations we had used in the last while: Some useful Maven configs...

So now that you have all these Maven projects, local repos, well managed versions, unit tests, code coverage and static code analysis what do you do next?...
You run it all the time and you let everyone know how it is going...
Continuous Integration (CI)...
I am not going to go into too much detail about the concept of CI, Martin Fowler and countless others have already done that.
Martin Fowler - Continuous Integration

I will however ramble out some personal opinions of the advantages and challenges that come with CI and mention some of the tools out there. I say "challenges" because I don't feel there are actually any disadvantages with this approach, but rather it just illuminates some difficulties that always existed in software development.

The Advantages:
CI gives you an early warning of any major broken or conflicting code changes with continuous unit testing of all the changes, it enables better project visibility. Depending on thoroughness of your CI environment there can be immediate feedback to developers on the quality, functionality and state of the application. Bugs, systems issues and the time needed to correct them gets exponentially worse the longer you they go unseen and are left unattended. Another benefit is that there is always a "current" build, that can be used for anything like testing, demos or interim releases, so not only are the unit tests running all the time, functional testing can also happen on more current code.
After the initial setup and with having all the SCM integration, building, deploying and reporting automated it allows people to focus on what they actually should be and not constant "environment" issues.

The Challenges:
If CI is something new to the environment there will be some initial setup time / cost, including maybe some extra hardware as build machines.
I feel there need to be well developed, comprehensive unit test suites these are crucial if there is to be any benefit gotten out of the automated testing. With this need for comprehensive unit test suites there is a requirement for all parties involved clients, management and developers to understand that more upfront development time will be required and there will be long term benefits. What happens all too often as soon a deadline looms... all standards, tests and processes go flying out the window, and as soon as that happens there is generally never time to go back and fix it and those holes in testing and quality will remain in the system.
There is also challenges with large changes can be hindered by the constant builds although this can be be handled in a couple ways, depending mainly on how you handle your Revision control / SCM (source configuration management), but that is a subject for another day.

The Tools:
I am only listing the ones that I feel are probably the most common (in the java environment), and obviously all are free and open source. I was tempted to create a very scaled down & up to date version of: CI Feature Matrix, but when you get down to the details, the 3 tools below all do very similar things, and have very similar features. They all can run, schedule, build, notify, integrate with SCMs (CVS, Subversion, Git, Mercurial, StarTeam, ClearCase), graph and have plugins for pretty much everything you will ever need.

Hudson
Cruise Control
Apache Continuum

We have used Hudson for the last couple years, and a couple practices that have served us well are:
Ensure that a build can always be reproduced, the build must be a clean build from scratch and by that I mean all the code must code from your SCM tool.
Set up email notifications to all the developers, the architect/s, and the project manager or development manager of the project. That way everyone has an idea of what is going on... and sometimes us developers need a little "motivation". There are also RSS feeds available for all builds, failed builds etc. I personally use those rather than the emails... but that is purely personal preference.
Hudson needs a fair amount of disk space on large projects, to avoid issues midway in a project ensure you have ample space and keep an eye on the version / archiving settings.
There are plugins for all major IDEs that allow you monitor and control Hudson without opening a browser, I personally like having the Hudson build information available within my IDE.
Something I would have liked to have set up, is integration with a issue tracking system like bugzilla or something, but alas my little corporate doesn't work that way.

MyEclipseIDE Maven4MyEclipse Null Pointer bug

I use MyEclipseIDE generally as my main IDE, I recently switched development teams at work, while importing all their existing maven projects with the Maven4MyEclipse plugin into my new workspace I had quite a few of these little annoying popups:


I went and checked the Log:

java.lang.NullPointerException
at com.genuitec.eclipse.maven.project.MyEclipseProjectSupport.findWebRoot (MyEclipseProjectSupport.java:204)
at com.genuitec.eclipse.maven.project.MyEclipseProjectSupport.configureWebProject (MyEclipseProjectSupport.java:71)
at com.genuitec.eclipse.maven.project.MyEclipseProjectSupport.configure (MyEclipseProjectSupport.java:57)
at org.maven.ide.eclipse.project.configurator.AbstractLifecycleMapping.configure (AbstractLifecycleMapping.java:57)
at org.maven.ide.eclipse.internal.project.GenericLifecycleMapping.configure (GenericLifecycleMapping.java:48)
at org.maven.ide.eclipse.internal.project.ProjectConfigurationManager.updateProjectConfiguration (ProjectConfigurationManager.java:332)
at org.maven.ide.eclipse.internal.project.ProjectConfigurationManager.updateProjectConfiguration (ProjectConfigurationManager.java:318)
at org.maven.ide.eclipse.actions.UpdateSourcesAction$1.runInWorkspace (UpdateSourcesAction.java:93)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run
(InternalWorkspaceJob.java:38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)


So I figure it is moaning about the WebRoot... let me try:

Right click on project go to MyEclipse Menu and Add Web Project Capabilities.



Worked like a charm.

Some useful Maven configurations, practices and lessons learnt

After using Maven for a while, my colleagues and I have configured a couple useful things and learnt a few lessons.

Edit: as per comments
Local Repository Managers
On the maven site here: Repository Management there is a brief description of the major repo managers out there
and a very good breakdown on why they are a good idea. We had a couple issues with Artifactory, (this was a couple years back) and in the end went with Nexus, which has proven to be very stable and offers all we need.

Parent POMs
Define common things, like versions, compiler settings, reporting settings and common build plugins in a parent POM to allow this configuration to be used throughout the applications' components and sub projects. If you have multiple projects with different requirements you can create another layer of Parent. To give an example, we had a legacy project, and the new re-write. For the legacy system we could not enforce checkstyle or code coverage but we did want to enforce it for the new project. So we created a base parent pom that had some general configuration and version numbers in it and then another parent pom for the new system that had the base parent as it's parent and then all the specific code coverage and style checks.

Versioning(See update at end of post)
In a parent POM define variables for all out internal component versions, so that you would have 1 central place to change release versions / snapshots etc.
We didn't include the versions of common 3rd party libraries like Spring, The Apache commons projects, JUnit etc, which in hindsight was a mistake. This is now really quite a pain 3 years and 30ish POMs down the line when trying to upgrade a specific library, and invariably one gets missed and then you have 2 / 3 versions of Spring or Hibernate sneaking into the dependencies.

Parent POM:


Project POM:


Compiler Settings
Just to ensure that it is the same accross all the projects using the parent.


Common Build Settings
1. Unit Testing
Actually running and reporting on the unit tests and results.
1.a. In the report plugin I have included command line arguments to increase the VM size, we found that on our large projects with tons of unit tests it did occasionally ran out of memory.
1.b. In the build plugin, there is a "parallel" config, that i recently saw in John Smarts blog, that allows you to run the test classes concurrently.
2. Cobertura Code Coverge, This is great way to ensure that there are atleast a certian amount of test cases and that code is being run.
It unfortunately can't check the validility of the test cases so developers will be developers and code "useless" tests just to pass the coverage requirement, but it is still worth putting the plugin in, and "useless" tests can be caught with code reviews.

1. Surefire - Running and Reporting



2. Cobertura Code Coverage
We found that in our predetermined deadline environment going for more that 50% coverage really extended the development time and effort for components more than we could afford. There were a couple smaller components that off the bat had 80%+, but we couldn't enforce it on all due to the time constraints.We are now slowly increasing that percentage when ever there is new work on a component but that is also quite a slow and painful process.
The "totalLineRate" is the setting that determines the coverage % required.




Checkstyle
We defined our basic coding standards within Checkstyle, and to ensure that they are followed we would fail the build if the code doesn't comply. I have also snuck in the "taglist-maven-plugin" plugin under this topic as it was a pet peeve on one of the developers, and with it in the reporting section it will show up when using 'mvn site'.




Profiles
One of our .EARs needed to be deployed on both Glassfish and Weblogic and there were unfortunate compatibility issues between the 2 so we couldn't actually deploy the same ear.
We configured that specific POM with 2 profiles. The Weblogic on was set to be the default, and when we required a new deployment onto Glassfish we would run maven with -Dglassfish-build=true.



Edit 2: Dependency Management and Versioning
Again, thanks to the people that commented. I am always happy to learn something new. So a couple people pointed out that I completely missed this. When writing this I based it on personal and team experiences and we somehow missed this section of the maven docs. I state in "Versioning" it is now a bit of a pain in our lives, so to help out others, below is the link to how Maven says you should handle dependencies.

Dependency Management

Spring 3: RESTful Web service Support... Take 2 with JPA and on Glassfish 3

In my original example. I just used a horrible static list in a controller as state for the CRUD methods. I couldn't leave it like that so I have now updated that version to use JPA to a MySQL DB.

This updated source code, config and POM files are available in SVN

I am not going to go through all the code like in my first example, but just go through a couple things I changed and new things picked up while making the change

1.
This thing, that actually cost me quite a bit of time...
The exception:
"A JDBC Driver or DataSource class name must be specified in the ConnectionDriverName property"

Which actually meant in my example
"I couldn't find your persistence.xml" imagine that... obviously very clear.

When you have a web application you can not just put your JPA persistence.xml in the META-INF folder like you would with a EJB application. Sad thing is I actually (after struggling for quite a while) remembered I had read this while studying for the SCBCD exam like 2 months ago. Doh!

So you need it in a META-INF in with your source, so you can either put in in a jar or if you lazy just have a META-INF in java/main/src.

2.
Trying to get Hibernate to run on Glassfish 3 was more effort that I thought it would be, and plain straight JPA was quick and simple.

3.
I used MySQL many years back, and it was alright. This latest version 5.2 Rocks... Seriously, go download the DB server, the divers and the Workbench.. The workbench is the winner... simple easy admin, almost instant EER / ERD diagrams, nice SQL query interface too, for my own home development I doubt I will look any further.

4.
For the last project I worked on we used Hibernate, and we used the generation tools to create our mappings and DAOs, which left us will a ton of Abstract and Implementation classes. I was never a fan of all those so went searching while doing this example and came across the Java Generic DAO implementation, which is very nice.

So you create a base DAO with the Key and Entity defined as generic parameters, just for ease of reading i went with "key" and "entity" instead of the usual K and E.


You then just have the following very small implementations just defining the actual Class type.


Very neat.

5.
In my original controller I had:
@RequestMapping(value="/users/", method=RequestMethod.POST) on each method.

Much nicer to have that on Controller level and just the following on the methods:
@RequestMapping(method=RequestMethod.POST)


6.
I mapped my dto purely through JPA annotations:

Gah...This was annoying... Glassfish 3.0.1 on Vista...

I have some time off... So I decide to check out Glassfish 3.0.1 (yes that is very geeky), it should be a nice new toy even with and evil red logo on it now. I am quite fond of glassfish, it was a pleasure to work with as a developer. I had spent a good almost 2 years using and fighting for the use of Glassfish 2.1 / OpenESB or Glassfish ESB... and lost the month I decided to get married and go on honeymoon...
That is probably lesson in itself, with some kind of motto:Never go on leave

.... anyway ....

Back to my current annoying discovery:
I try start up Glassfish after the install... (in Netbeans 6.9) and it fails with a silly little error:
"Address family not supported by protocol family: bind: 7676"
and then proceeds to shutdown..

So off to google...
Read quite a few useless forum posts...
Eventually find the below... pay it no attention of course... and carry on reading useless posts...
"
Hi,

Try to change configuration of localhost in file
C:\Windows\System32\drivers\etc\hosts
from
::1 localhost
to
127.0.0.1 localhost
"

After another couple minutes, I decide "what the hell", and I go find the host file, make the change...big surprise... Windows doesn't let me.
Download "Unlocker", pretty neat utility if you want to overpower the Windows file locks, but it picks up no lock.
Another trip to google.... Only to find you need to run the edit application as an administrator obviously...

Make the change, and Tah Dah! Nice new Glassfish 3.0.1 server up and running.

Ehcache 2.1 Monitoring

In my previous post Spring, AspectJ, Ehcache Method Caching Aspect

I demo'ed and described how and why to cache the results of methods. Now to enhance that, I did some more investigation into the latest release of ehcache available at Terracotta

With 2.1 they released a monitoring api, to allow you to monitor cache's real time within your runtime environment, this will allow to fine tune and manage any cache you may have... be that custom caching with something like my MethodCacheAspect or your applications' Hibernate caches. To setup and use the monitoring you will need to do the following (I am setting this up on weblogic 10.3)

I placed the following 4 jars into my domain/lib, this will be different for every application server, but as long as the following are in the relative classpath you should be ok.

• ehcache-core-2.1.0.jar
• ehcache-probe-1.0.0.jar
• slf4j-api-1.5.11.jar
• slf4j-jdk14-1.5.11.jar

So now for the configuration:

The ehcache.xml, needs to be in your classpath and have the following details:


Take note of the following that is added to the cache you want to monitor.



In the start up of the application server... in my case weblogic 10.3 you will see the following:
(it is a licensed product from terracotta, and I am only using in a development environment to tune my cache settings)

INFO: Initializing EHCache CacheManager
04 Jul 2010 6:18:24 PM net.sf.ehcache.distribution.RMICacheManagerPeerListener
WARNING: Explicitly setting the listener hostname to 'localhost' is not recommended. It will only work if all CacheManager peers are on the same machine.
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.monitor.probe.ProbeDxService startBackend
INFO: Started probe at http://localhost:51115/monitor/list
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.license.LicenseResolver resolveLicense
WARNING: No Ehcache Monitor license key found. This monitoring probe software is not licensed for production usage, and is only licensed for development usage. See LICENSE.txt for details. A temporary key will be generated for development usage. When the temporary key expires, the Ehcache monitoring probe capability will be suspended but your ability to continue to use Ehcache will not be affected. Please contact sales@terracottatech.com to request a license.
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.license.LicenseResolver logLicense
INFO:
--------- Ehcache Monitor license key ---------
License type = DevOnly
License number = 0
Licensee = DevOnly User
Product = Ehcache Monitor
Capabilities = monitor
Expiration date = 2010-07-07
-----------------------------------------------
04 Jul 2010 6:18:25 PM org.terracotta.ehcachedx.monitor.probe.ProbeDxService$RegisterWithServer run
INFO: Null license registered


Now that it is up and running, you can browse to:
http://localhost:51115/monitor/list

There you will get a list of methods that allow you to query you live caches, or even check the status of the CacheManager with, "probeGetCacheManagerStatus( )" very handy if you want to monitor the health of your live application. We have had a couple occasions where we we getting exceptions calling another teams' services and it took a while to discover that the "Cache Manager is not alive" exception.

The Ehcache API is REST based service. You Should be able to to use a simple RestTemplate like I used in my Spring 3 RESTFul example to manipulate and query the API, or you can just use a standard web browser:

http://localhost:51115/monitor/probeGetCacheManagerStatus
Result:
STATUS_ALIVE

http://localhost:51115/monitor/probeGetCacheNames
Result:
testCache
restCache

http://localhost:51115/monitor/probeGetCacheElementMemorySize?cache=testCache
Result:
584

The full list of functions are:

AmountOfHistoryStoredOnMonitor( )
getSampledStatisticNames( )
getSamplingHistory( )
getSamplingSeconds( )
getVersion( )
probeEvictExpiredElements( cache )
probeFlushCache( cache )
probeGetAllCachesConfigAllValues( )
probeGetAllCachesSampledStatisticValues( name )
probeGetCacheConfigAllValues( cache )
probeGetCacheConfigNames( )
probeGetCacheConfigValue( cache, name )
probeGetCacheConfigValues( cache, name )
probeGetCacheCount( )
probeGetCacheElementCount( cache )
probeGetCacheElementCountDisk( cache )
probeGetCacheElementCountMemory( cache )
probeGetCacheElementMemorySize( cache )
probeGetCacheElementMetaDataAllValues( cache, key )
probeGetCacheElementMetaDataNames( )
probeGetCacheElementMetaDataValue( cache, key, name )
probeGetCacheElementMetaDataValues( cache, key, name )
probeGetCacheKeys( cache, count, [query] )
probeGetCacheManagerName( )
probeGetCacheManagerStatus( )
probeGetCacheNames( )
probeGetReplicationMode( )
probeGetSampledStatisticAllValues( cache )
probeGetSampledStatisticHistory( cache, name, [fromDateTime], [timeInterval], [offset] )
probeGetSampledStatisticValue( cache, name )
probeGetSampledStatisticValues( cache, name )
probeRemoveAllFromCache( cache )
probeRemoveFromCache( cache, key )
probeRemoveQueryFromCache( cache, query )


Some of these could be very valuable to fine tune you caching strategy. Knowing things like "probeGetCacheElementCountDisk", "probeGetCacheElementMemorySize" and being able to monitor them may allow you to tailor your Ehcache config for optimal usage and consequently better application performance.

Also "probeFlushCache" or "probeRemoveFromCache" allow you to clear out the cache in the case of a data change without having to reinitialize / restart your application.

The tale of a Calendar... a Date... and a Object reference bug..

To set the scene, there is a software release 4 days away and everyone is in mid-pre-release-panic... There is one of those nasty bug's only happens sometimes, and was missed by testing until 2 hours before final signoff, and is "absolutely necessary" for the release.

I rush off go through the date calculations done by our system, adding and changing To and From dates into a sequence depending on some items, not overly complicated but just enough to have quite a few Calendar and Date objects flying around.

So I figure it has to be an object reference being used of one the Date objects, because 2 of the dates set are to the same value. This obviously only happens on the actual deployment and my unit tests pass, and debugs print out the right values..

First thing I do is look for shared Date objects (all the Date objects get created via Calendars or set back to the Calendar like below:



The shared Date objects are then used on a internal Period class that has From and To dates and some convenience methods that is used all around our system and it was those date values that were incorrect.

So I think... easy.. I'll just add ".clone()" the Date because it must be their object references.... I put in 2x .clone() compile, deploy ... test... fail... 2 Dates still the same.

Go back to the code, get a second pair of eyes, a work colleague. Together we slap together this completely immutable class, that makes the calculations less complicated and ensures that there can't be any object reference sharing between that dates and the calendars or anything, and at the same time create a new Period object to hold these new dates from our new class that we made sure that could not be messed with by reference.

build, deploy ... test ... works like a charm.. notify the testers, they sign off ... all is good.

A hour or so later, the same work colleague pops me an IM on MSN Messenger... says that this could happen somewhere else... we get the code... at roughly the same time we both happen to go into Calendar.getTime() source code and to our surprise (and amusement) we discover a little gem... both can't believe that we had not actually seen it before..

Calendar source code below:



The standard java Calendar had ensured a long time ago that you would not have this object reference scenario and that it is safe by using the "TimeInMillis". So we had wasted a good 30 minutes of our lives, when the only issue was our own Period object that was the bad reference.

So I guess the point of this is....

1. Using the Calendar.getTime() and setTime() is safe and won't let you pollute object references.. and...
2. I had a similar discovery with Spring a little while back (...) So it is fair to say if you have the source code of classes you are using, its probably an excellent idea to have a look at it from time to time.

Spring 3: RESTful Web service Support

UPDATE: This has been updated to use JPA and a more real life type object with Id's and the like... Spring 3: RESTful Web service Support Take 2 with JPA

So in between some really long days at work, the usual before release panic and regression defects. I finally completed a simple RESTFul example, before getting into the details, here are a couple facts about RESTFul services.

• REST stands for Representational State Transfer.
• Each unique URL is a representation of some object.
• You can get the contents of that object using an HTTP GET.
• You may use a POST, PUT, or DELETE to modify the object.
• It is not necessary to use XML as a data format, so for small amounts of data it can be significantly lighter on bandwidth.
• REST as a protocol does not define any form of message envelope.
• REST takes advantage of HTTP caches. (A cache can't to do anything a POST; but they can cache GETs and expire those entries based on PUTs and DELETEs.).
• The Sun specification for RESTFul webservices can be found atJAX-RS
• Spring does not directly support or implement the JAX-RS spec but instead RESTful functionality is added to feature Spring MVC.

As with my other examples all the source code, config and POM files are available in SVN

As we are exposing information via HTTP this is going to be a web project. I won't be going into details of compiling and deploying a web application, but will go through all the required parts
Starting with the web.xml:




The DispatcherServlet will receive the request it will then, based on the servlet mapping
go looking for "Spring3RESTFul-servlet.xml" which in turn defines the controller and view (which are dynamically searched for and created):


The component-scan in the application context above will be used to automatically pick up our @Controller. If you go to /app/users/Bob, the getUser method is executed and their respective name is passed through as a parameter. The Spring JaxB 2 marshaller will be used to marshall the User object in and out of XML. Any of the OXM marshallers can be used, I just simply prefer JaxB.

Note: You really should not have a horrible static map containing state within a controller (like I do below), I am only doing it now because this is an example, and I want to keep it simple and not digress into a full blown application. If I have time over the next week or so, I'll remove that map of data and just create an embedded DB or something.


You can view the results in your browser by simply going to the url. I am using weblogic, so mine looks like "http://localhost:7001/Spring3RESTFul-1/app/users/Mary" but depending on your deployment it could differ as long as you add "/app/users/Mary" to your context root you should get the following displayed. (if you are using firefox)


However, chances are you are not writing this to view some XML in a browser but rather to get or manipulate information from one system or component to another. For this I created a little client, this client uses Springs' RestTemplate which is autowired and lets me access the HTTP functions:

Quote from Spring blog:


DELETE delete(String, String...)
GET getForObject(String, Class, String...)
HEAD headForHeaders(String, String...)
OPTIONS optionsForAllow(String, String...)
POST postForLocation(String, Object, String...)
PUT put(String, Object, String...)


This example only uses getForObject, postForObject, put and delete for the moment, so just the normal CRUD operations.


Now to use the above client I have a test application context injecting the RestTemplate and a test case preforming the create, retrieve, update and delete.



Note:Because I just wanted to create 1 project for this example I have included the following, just so that you can compile and deploy and then run the test case... but ever adding this to a actual project POM file should be grounds for a good beating.



Lastly actual test case. Deploy the war file created by the POM to the web / application server of your choice before running the below:




And that is it... a Simple CRUD RESTFul Webservice example with Spring 3.