LSNED35: Testing JPA with JUnit, Spring and Maven

Too lazy to type it myself when someone else has done such a good job: http://www.bewareofthebear.com/java/testing-jpa-with-junit-spring-and-maven/
That’s helped me get unit tests for my OpenJPA domain model working.

For my purposes listing the classes in the persistence.xml is necessary, so the bit about persistenceUnitPostProcessors wasn’t much help.

The bit that was most helpful was getting the right attributes onto the test class:

@RunWith( SpringJUnit4ClassRunner.class )
@ContextConfiguration( locations =
{
    "classpath:testContext.xml"
} )
@TransactionConfiguration(transactionManager="transactionManager", defaultRollback=false)
@Transactional

LSNED34: Accessing the CXF Exchange object from within a service implementation.

If you have a method that provides the implementation for a CXF web service operation and, for some reason, that method needs to access the CXF Exchange object, this is how to do it.

Firstly add an annotated member of type javax.xml.ws.WebServiceContext to your implementation class:

    @Resource
    private WebServiceContext webServiceContext;

The CXF runtime will set that member for you automatically.

Then, in order to get to the Exchange object:

            MessageContext context = webServiceContext.getMessageContext();
            WrappedMessageContext wmc = (WrappedMessageContext)context;
            Message m = wmc.getWrappedMessage();
            Exchange exchange = m.getExchange();

The WebServiceContext and MessageContext interfaces are part of the JAX-WS specification, but the WrappedMessageContext is CXF only (but then, if you are bothered by that then you wouldn’t be trying to access the CXF Exchange object).

I used this technique to pass error information from the service implementation to a logging interceptor.

LSNED 33: OpenJPA in Apache Felix

For Routemaster ESB I need to have some form of ORM working in an OSGi container. Given that the only ORM I really know is JPA, that was my starting point. I spent a long time trying to persuade Hibernate to work, before eventually giving up and moving to OpenJPA istead. And the good news is that getting OpenJPA working in felix is quite straightforward (though a little long winded, this is a big post!).

Executive Summary

Tips for solving problems with OpenJPA:

  • Do not enable any kind of LoadTimeWeaver.
    Be aware that if you have OpenJPA logging on it will report an exception, but it’s just a warning and can be safely ignored.
  • Do enable a build time enhancer, via the openjpa-maven-plugin.
  • Disable validation in your persistence.xml.

Step by step

I began by creating a project and a module:

pax-create-project -g org.yaytay -a jpa-in-osgi
cd jpa-in-osgi
pax-create-bundle -p org.yaytay.jpauser

File Structure

After using Pax to create the basic layout I added the remaining files manually (I don’t really like things that create files for me). The basic file layout is:


The Root pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.yaytay</groupId>
  <artifactId>jpa-in-osgi</artifactId>
  <version>1.0-SNAPSHOT</version>

  <name>org.yaytay.jpa-in-osgi (OSGi project)</name>

  <description>Generated using Pax-Construct</description>

  <properties>
    <!-- Change the source encoding, I wish UTF-8 was the default -->
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <packaging>pom</packaging>

  <modules>
    <module>poms</module>
    <module>provision</module>
    <module>org.yaytay.jpauser</module>
  </modules>

  <build>
    <plugins>
      <plugin>
        <!-- Change the compiler to use JDK >= 1.5 for annotations -->
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.ops4j</groupId>
        <artifactId>maven-pax-plugin</artifactId>
        <version>1.4</version>
        <configuration>
          <provision>
            <param>--platform=felix</param>
            <param>--profiles=log,web</param>
            <param>--features=war,webconsole,ssh,management</param>
            <param>scan-bundle:mvn:org.apache.geronimo.specs/geronimo-jta_1.1_spec/1.1.1</param>
            <param>scan-bundle:mvn:org.apache.felix/org.apache.felix.fileinstall/2.0.8</param>
            <param>scan-bundle:mvn:org.ops4j.pax.url/pax-url-mvn/1.1.2</param>
            <param>scan-bundle:mvn:mysql/mysql-connector-java/5.1.12</param>
            <!-- There are bugs in spring.orm prior to 3.0.2 that prevent working with JPA 2.0 -->
            <param>scan-features:file:spring-3.0.2.RELEASE-features.xml!/spring,spring-dm</param>
            <param>scan-features:file:openjpa-features.xml!/openjpa</param>
            <param>--shell=karaf.shell/1.2.0</param>
            <param>--vmOptions=-Dfelix.fileinstall.dir=../etc -Dorg.osgi.framework.system.packages=javax.accessibility,javax.activity,javax.annotation,javax.annotation.processing,javax.crypto,javax.crypto.interfaces,javax.crypto.spec,javax.imageio,javax.imageio.event,javax.imageio.metadata,javax.imageio.plugins.bmp,javax.imageio.plugins.jpeg,javax.imageio.spi,javax.imageio.stream,javax.lang.model,javax.lang.model.element,javax.lang.model.type,javax.lang.model.util,javax.management,javax.management.loading,javax.management.modelmbean,javax.management.monitor,javax.management.openmbean,javax.management.relation,javax.management.remote,javax.management.remote.rmi,javax.management.timer,javax.naming,javax.naming.directory,javax.naming.event,javax.naming.ldap,javax.naming.spi,javax.net,javax.net.ssl,javax.print,javax.print.attribute,javax.print.attribute.standard,javax.print.event,javax.rmi,javax.rmi.CORBA,javax.rmi.ssl,javax.script,javax.security.auth,javax.security.auth.callback,javax.security.auth.kerberos,javax.security.auth.login,javax.security.auth.spi,javax.security.auth.x500,javax.security.cert,javax.security.sasl,javax.sound.midi,javax.sound.midi.spi,javax.sound.sampled,javax.sound.sampled.spi,javax.sql,javax.sql.rowset,javax.sql.rowset.serial,javax.sql.rowset.spi,javax.swing,javax.swing.border,javax.swing.colorchooser,javax.swing.event,javax.swing.filechooser,javax.swing.plaf,javax.swing.plaf.basic,javax.swing.plaf.metal,javax.swing.plaf.multi,javax.swing.plaf.synth,javax.swing.table,javax.swing.text,javax.swing.text.html,javax.swing.text.html.parser,javax.swing.text.rtf,javax.swing.tree,javax.swing.undo,javax.tools,javax.xml,javax.xml.crypto,javax.xml.crypto.dom,javax.xml.crypto.dsig,javax.xml.crypto.dsig.dom,javax.xml.crypto.dsig.keyinfo,javax.xml.crypto.dsig.spec,javax.xml.datatype,javax.xml.namespace,javax.xml.parsers,javax.xml.stream,javax.xml.transform,javax.xml.transform.dom,javax.xml.transform.sax,javax.xml.transform.stax,javax.xml.transform.stream,javax.xml.validation,javax.xml.xpath,org.ietf.jgss,org.omg.CORBA,org.omg.CORBA.DynAnyPackage,org.omg.CORBA.ORBPackage,org.omg.CORBA.TypeCodePackage,org.omg.CORBA.portable,org.omg.CORBA_2_3,org.omg.CORBA_2_3.portable,org.omg.CosNaming,org.omg.CosNaming.NamingContextExtPackage,org.omg.CosNaming.NamingContextPackage,org.omg.Dynamic,org.omg.DynamicAny,org.omg.DynamicAny.DynAnyFactoryPackage,org.omg.DynamicAny.DynAnyPackage,org.omg.IOP,org.omg.IOP.CodecFactoryPackage,org.omg.IOP.CodecPackage,org.omg.Messaging,org.omg.PortableInterceptor,org.omg.PortableInterceptor.ORBInitInfoPackage,org.omg.PortableServer,org.omg.PortableServer.CurrentPackage,org.omg.PortableServer.POAManagerPackage,org.omg.PortableServer.POAPackage,org.omg.PortableServer.ServantLocatorPackage,org.omg.PortableServer.portable,org.omg.SendingContext,org.omg.stub.java.rmi,org.w3c.dom,org.w3c.dom.bootstrap,org.w3c.dom.css,org.w3c.dom.events,org.w3c.dom.html,org.w3c.dom.ls,org.w3c.dom.ranges,org.w3c.dom.stylesheets,org.w3c.dom.traversal,org.w3c.dom.views,org.xml.sax,org.xml.sax.ext,org.xml.sax.helpers,org.osgi.framework;version=1.5.0,org.osgi.framework.launch;version=1.0.0,org.osgi.framework.hooks.service;version=1.0.0,org.osgi.service.packageadmin;version=1.2.0,org.osgi.service.startlevel;version=1.1.0,org.osgi.service.url;version=1.0.0,org.osgi.util.tracker;version=1.4.0</param>
          </provision>
        </configuration>
        <executions>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

There are three things to note here:

  • Spring and OpenJPA are pulled in via separate Karaf features files (described next).
    JPA 2.0 does not work reliably in Spring 3.0.1, so 3.0.2 is required, and that is not described in any features files that I could find.
  • The default system packages for JDK 1.6 on felix include javax.transaction with no version number, but it is required with a version number for OpenJPA.
    To get around this I add in the geronimo JTA package and remove javax.transaction from the system packages by manually specifying all the other packages.
  • The –vmOptions argument is also used to specify a directory for the felix fileinstall bundle to monitor.
    The fileinstall bundle is used for setting up the config properties that will be used to identify the database connection properties.

  • openjpa-features.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <features>
        <feature name="openjpa">
            <bundle>mvn:javax.persistence/com.springsource.javax.persistence/2.0.0</bundle>
            <bundle>mvn:commons-collections/commons-collections/3.2.1</bundle>
            <bundle>mvn:commons-lang/commons-lang/2.4</bundle>
            <bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.commons-pool/1.5.4_1</bundle>
            <bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.commons-dbcp/1.2.2_5</bundle>
            <bundle>mvn:net.sourceforge.serp/com.springsource.serp</bundle>
            <bundle>mvn:org.jboss.javassist/com.springsource.javassist/3.9.0.GA</bundle>
            <bundle>mvn:org.apache.openjpa/openjpa/2.0.0</bundle>
        </feature>
    </features>
    


    spring-3.0.2.RELEASE-features.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <features name="spring-3.0.2">
        <feature name="spring" version="3.0.2.RELEASE">
            <bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.aopalliance/1.0_3</bundle>
            <bundle>mvn:org.springframework/org.springframework.core/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.asm/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.expression/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.transaction/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.beans/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.aop/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.orm/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.jdbc/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.context/3.0.2.RELEASE</bundle>
            <bundle>mvn:org.springframework/org.springframework.context.support/3.0.2.RELEASE</bundle>
        </feature>
        <feature name="spring-dm" version="1.2.1">
            <feature version="3.0.2.RELEASE">spring</feature>
            <bundle>mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.cglib/2.1_3_4</bundle>
            <bundle>mvn:org.springframework.osgi/spring-osgi-io/1.2.1</bundle>
            <bundle>mvn:org.springframework.osgi/spring-osgi-core/1.2.1</bundle>
            <bundle>mvn:org.springframework.osgi/spring-osgi-extender/1.2.1</bundle>
            <bundle>mvn:org.springframework.osgi/spring-osgi-annotation/1.2.1</bundle>
            <bundle>mvn:org.apache.felix.karaf.deployer/org.apache.felix.karaf.deployer.spring/1.4.0</bundle>
        </feature>
    </features>
    


    The Code pom.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    
      <parent>
        <relativePath>../poms/compiled/</relativePath>
        <groupId>org.yaytay.jpa-in-osgi.build</groupId>
        <artifactId>compiled-bundle-settings</artifactId>
        <version>1.0-SNAPSHOT</version>
      </parent>
    
      <properties>
        <bundle.symbolicName>org.yaytay.jpauser</bundle.symbolicName>
        <bundle.namespace>org.yaytay.jpauser</bundle.namespace>
      </properties>
    
      <modelVersion>4.0.0</modelVersion>
      <groupId>org.yaytay.jpa-in-osgi</groupId>
      <artifactId>org.yaytay.jpauser</artifactId>
      <version>1.0-SNAPSHOT</version>
    
      <name>${bundle.symbolicName}</name>
    
      <packaging>bundle</packaging>
    
      <dependencies>
        <dependency>
          <groupId>org.osgi</groupId>
          <artifactId>osgi_R4_core</artifactId>
          <optional>true</optional>
        </dependency>
        <dependency>
          <groupId>org.osgi</groupId>
          <artifactId>osgi_R4_compendium</artifactId>
          <optional>true</optional>
        </dependency>
        <dependency>
          <groupId>org.apache.openjpa</groupId>
          <artifactId>openjpa</artifactId>
          <version>2.0.0</version>
          <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>3.0.2.RELEASE</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-tx</artifactId>
            <version>3.0.2.RELEASE</version>
            <scope>provided</scope>
        </dependency>
      </dependencies>
    
      <build>
          <plugins>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>openjpa-maven-plugin</artifactId>
                    <version>1.1</version>
                    <configuration>
                        <connectionDriverName>com.mysql.jdbc.Driver</connectionDriverName>
                        <includes>**/model/jpaimpl/*.class</includes>
                        <excludes>**/model/jpaimpl/XML*.class</excludes>
                        <addDefaultConstructor>true</addDefaultConstructor>
                        <enforcePropertyRestrictions>true</enforcePropertyRestrictions>
                    </configuration>
                    <executions>
                        <execution>
                            <id>enhancer</id>
                            <phase>process-classes</phase>
                            <goals>
                                <goal>enhance</goal>
                                <goal>sql</goal>
                            </goals>
                        </execution>
                    </executions>
                    <dependencies>
                        <dependency>
                            <groupId>org.apache.openjpa</groupId>
                            <artifactId>openjpa</artifactId>
                            <version>2.0.0</version> <!-- set the version to be the same as the level in your runtime -->
                        </dependency>
                    </dependencies>
                </plugin>
                <plugin>
                    <!-- I prefer using the maven-bundle-plugin to using an osgi.bnd file -->
                    <groupId>org.apache.felix</groupId>
                    <artifactId>maven-bundle-plugin</artifactId>
                    <version>2.0.1</version>
                    <extensions>true</extensions>
                    <configuration>
                        <instructions>
                            <Export-Package>org.yaytay.* </Export-Package>
                            <!--
                            Import packages are always a trial and error job, depending on what BND discovers,
                            in a real application you may need a different set.
                            -->
                            <Import-Package>*,
                                org.apache.commons.dbcp,
                                com.mysql.jdbc,
                                org.apache.openjpa.kernel,
                                javax.transaction,
                                javax.persistence.criteria,
                                javax.persistence.metamodel,
                                javax.persistence.spi,
                                javax.sql,
                            </Import-Package>
                        </instructions>
                    </configuration>
                </plugin>
          </plugins>
      </build>
    </project>
    

    Notes:

    • Any dependencies required by your code need to be added here, but remember to put them as “provided”.
    • The OpenJPA plugin serves two roles:
      • It generates a SQL script to build your database, the script is put in target\database.sql.
      • It enhances the Java class to work with JPA.
    • The bundle plugin use BND to prepare the OSGi manifest for the bundle, including any customisation you choose to add.
      For our purposes it is necessary to add a number of ImportPackages that BND would otherwise not realise we use.
      Unfortunately I don’t know way to work out which packages to add except trial and error.


    DataItem.java


    A very simple interface to the JPA class.

    package org.yaytay.jpauser.model;
    
    public interface DataItem {
        public long getId();
        public String getName();
        public void setName( String name );
    }
    


    “DataItem_JpaImpl.java


    The simplest JPA class I could come up with.

    package org.yaytay.jpauser.model.jpaimpl;
    
    import javax.persistence.Column;
    import javax.persistence.Entity;
    import javax.persistence.GeneratedValue;
    import javax.persistence.GenerationType;
    import javax.persistence.Id;
    import javax.persistence.SequenceGenerator;
    import javax.persistence.Table;
    import org.yaytay.jpauser.model.DataItem;
    
    @Entity
    @Table( name = "DataItems" )
    public class DataItem_JpaImpl implements DataItem {
        @Id
        @GeneratedValue(strategy=GenerationType.SEQUENCE, generator="DataItem_Seq")
        @SequenceGenerator(name="DataItem_Seq", sequenceName="table()")
        private long id;
    
        @Column( nullable = true, length = 512 )
        private String name;
    
        @Override
        public long getId() {
            return id;
        }
    
        @Override
        public String getName() {
            return name;
        }
    
        @Override
        public void setName(String name) {
            this.name = name;
        }
    }
    


    JpaTester.java


    This class is nasty, but I wanted to demonstrate JPA working without adding in extra things like an interface, JUnit or Pax exam.

    package org.yaytay.jpauser;
    
    import javax.persistence.EntityManager;
    import javax.persistence.EntityManagerFactory;
    import javax.persistence.EntityTransaction;
    import org.springframework.context.ApplicationEvent;
    import org.springframework.context.ApplicationListener;
    import org.yaytay.jpauser.model.DataItem;
    import org.yaytay.jpauser.model.jpaimpl.DataItem_JpaImpl;
    
    public class JpaTester implements ApplicationListener  {
        EntityManager entityManager;
    
        @Override
        public void onApplicationEvent(ApplicationEvent e) {
            if( e instanceof org.springframework.context.event.ContextRefreshedEvent ) {
                this.applicationStarted();
            }
        }
    
        public void setFactory(EntityManagerFactory factory) {
            entityManager = factory.createEntityManager();
        }
    
        private DataItem getNewDataItem(String name) {
            DataItem item;
            EntityTransaction transaction = entityManager.getTransaction();
            transaction.begin();
            try {
                item = new DataItem_JpaImpl();
                item.setName(name);
                entityManager.persist(item);
            } finally {
                transaction.commit();
            }
            return item;
    
        }
    
        private DataItem getOldDataItem(long id) {
            return (DataItem)entityManager.find(DataItem_JpaImpl.class, id);
        }
    
        private void applicationStarted() {
            System.err.println( "\n\nStarted.\n\n");
    
            DataItem item = getNewDataItem( "Test Name" );
            long id = item.getId();
            System.err.println( "DataItem " + id + " = " + item.getName() );
            
            DataItem newItem = getOldDataItem(id);
            System.err.println( "NewDataItem " + newItem.getId() + " = " + item.getName() );
        }
    }
    

    When Spring has finished initialising the singleton instance of this class it simply creates a JPA entity, writes it to the database and then queries for it.


    applicationContext.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:context="http://www.springframework.org/schema/context"
           xmlns:tx="http://www.springframework.org/schema/tx"
           xmlns:osgi="http://www.springframework.org/schema/osgi"
           xmlns:osgix="http://www.springframework.org/schema/osgi-compendium"
           xsi:schemaLocation="
           http://www.springframework.org/schema/beans                 http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
           http://www.springframework.org/schema/context               http://www.springframework.org/schema/context/spring-context-2.5.xsd
           http://www.springframework.org/schema/tx                    http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
           http://www.springframework.org/schema/osgi                  http://www.springframework.org/schema/osgi/spring-osgi.xsd
           http://www.springframework.org/schema/osgi-compendium       http://www.springframework.org/schema/osgi-compendium/spring-osgi-compendium.xsd
           ">
        <context:annotation-config />
    
        <osgix:cm-properties id="cmProps" persistent-id="org.yaytay.jpauser" />
    
        <context:property-placeholder properties-ref="cmProps" />
    
        <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
            <property name="driverClassName" value="${org.yaytay.jpauser.dataSource.driverClassName}" />
            <property name="url"             value="${org.yaytay.jpauser.dataSource.url}" />
            <property name="username"        value="${org.yaytay.jpauser.dataSource.username}" />
            <property name="password"        value="${org.yaytay.jpauser.dataSource.password}" />
            <property name="initialSize"     value="10" />
        </bean>
    
        <bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
            <property name="dataSource" ref="dataSource"/>
            <property name="persistenceUnitName" value="JpaUser"/>
            <property name="jpaVendorAdapter">
                <bean class="org.springframework.orm.jpa.vendor.OpenJpaVendorAdapter">
                    <property name="databasePlatform" value="org.apache.openjpa.jdbc.sql.MySQLDictionary" />
                </bean>
            </property>
        </bean>
    
        <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
            <property name="entityManagerFactory" ref="entityManagerFactory" />
        </bean>
    
        <tx:annotation-driven transaction-manager="transactionManager" />
    
        <bean class="org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor"/>
        <bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor"/>
    
        <!-- Test classes -->
        <bean class="org.yaytay.jpauser.JpaTester">
            <property name="factory" ref="entityManagerFactory" />
        </bean>
    
    </beans>
    

    Most of this file is standard Spring/JPA stuff. One thing to note is the osgix:cm-properties element, which brings in a map from the OSGi configuration store, this map is then used by the property-placeholder. This is only effective when the bundle is started, changes are not passed on dynamically.

    Note that there is no LoadTimeWeaver or ValidationBean – I cannot get these two features working (the LoadTimeWeaver shouldn’t be necessary anyway, but validation would be nice).


    persistence.xml


    A simple persistence file that does not contain the details of the data source (which are set via the Spring context file).

    <?xml version="1.0" encoding="UTF-8"?>
    <persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence">
      <persistence-unit name="JpaUser">
        <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
        <class>org.yaytay.jpauser.model.jpaimpl.DataItem_JpaImpl</class>
        <exclude-unlisted-classes>true</exclude-unlisted-classes>
        <validation-mode>NONE</validation-mode>
        <properties>
          <property name="openjpa.Log" value="DefaultLevel=INFO,SQL=TRACE,Runtime=TRACE"/>
          <property name="openjpa.RuntimeUnenhancedClasses" value="unsupported"/>
        </properties>
      </persistence-unit>
    </persistence>
    


    org.yaytay.jpauser.cfg


    The configuration file that fileinstall picks up and puts into the OSGi configuration store, from where Spring picks it up.

    org.yaytay.jpauser.dataSource.driverClassName = com.mysql.jdbc.Driver
    org.yaytay.jpauser.dataSource.url                   = jdbc:mysql://localhost:3306/jpauser
    org.yaytay.jpauser.dataSource.username          = jpauser
    org.yaytay.jpauser.dataSource.password          = TopSecret
    

    I am kinda hoping that you’ll use a better password than that one!

    Putting it All Together

    The first thing to do is to run “mvn clean install” and to find the database script (org.yaytay.jpauser/target/database.sql) and to run it in your database.

    Then run “mvn clean install pax:provision” to run the OSGi container. With a bit of luck, somewhere near the end of the spew of output you will see something like:

    DataItem 201 = Test Name
    NewDataItem 201 = Test Name
    

    then look in your database to see the data item there.

    And that’s it.

    You can download the source code here if you want: jpa-in-osgi.tar.gz (but that’s a nasty download site and there’s nothing there that isn’t in this post).

LSNED 32: Getting the MySQL Connector/J working in Apache Felix under pax:provision with JDK 1.6

It ought to be easy to get the MySQL Connector/J (aka mvn:mysql/mysql.connector.java/5.1.12) working under Apache Felix: after all it is provided as an OSGi bundle. Unfortunately if you try to load that bundle in Apache Felix under pax:provision using JDK 1.6 you will have problems.
This is because the manifest for MySQL Connector/J has an import-packages like this (formatted to be readable):

Import-Package: 
  javax.net,javax.net.ssl;version="[1.0.1, 2.0.0)";resolution:=optional,javax.xml.parsers, 
  javax.xml.stream,
  javax.xml.transform,
  javax.xml.transform.dom,
  javax.xml.transform.sax,
  javax.xml.transform.stax,
  javax.xml.transform.stream,
  org.w3c.dom,
  org.xml.sax,
  org.xml.sax.helpers;resolution:=optional,
  javax.naming,
  javax.naming.spi,
  javax.sql,
  javax.transaction.xa;version="[1.0.1, 2.0.0)";resolution:=optional,
  org.apache.commons.logging;version="[1.1.1,2.0.0)";
  resolution:=optional,
  org.apache.log4j;version="[1.2.15, 2.0.0)";resolution:=optional,
  com.mchange.v2.c3p0;version="[0.9.1.2, 1.0.0)";resolution:=optional,
  org.jboss.resource.adapter.jdbc;resolution:=optional,
  org.jboss.resource.adapter.jdbc.vendor;resolution:=optional

The problem one is javax.transaction.xa, because, by default, pax:provision with apache felix and JDK 1.6 has javax.transaction listed as a system package, but with no version details.

The camel and spring features depend on javax.activation 1.1.0, and pull in the servicemix implementation (with a version number).
That gives two versions of javax.activation and a problem with spring-context-support, which fails to run with the mystic “constraint violation” error – because spring-context-support takes an optional export with a “use” clause for javax.activation with no version.
So spring-context-support resolves against the system javax.activation, but camel resolves to the servicemix copy, and then camel can’t resolve against spring-context-support.

The frustrating thing is that camel and spring features both work, until MySQL is added.

Anyway, the solution is to remove the unnumbered javax.transaction.xa from the system packages, which isn’t quite as easy as it ought to be. It seems that this (within your pom.xml) works:

<param>--vmOptions=-Dorg.osgi.framework.system.packages=javax.accessibility,javax.activity,javax.annotation, ...</param>

but this does not:

<param>--systemPackages=javax.accessibility,javax.activity,javax.annotation, ...</param>

The –systemPackages appends system packages, it doesn’t replace them (so “headers 0” shows the complete original set, plus the custom set, with most entries being duplicates).

To find the list of packages that should be in your system packages look in the file runner\felix\config.ini (created afresh ever time you run pax:runner), it will contain a huge list of system packages (not formatted nicely as with a proper felix/karaf installation). Just copy that list and remove the packages you don’t want made available as system packages.
The set that I’m currently using is:

            <param>--vmOptions=-Dfelix.fileinstall.dir=../etc -Dorg.osgi.framework.system.packages=javax.accessibility,javax.activity,javax.annotation.processing,javax.crypto,javax.crypto.interfaces,javax.crypto.spec,javax.imageio,javax.imageio.event,javax.imageio.metadata,javax.imageio.plugins.bmp,javax.imageio.plugins.jpeg,javax.imageio.spi,javax.imageio.stream,javax.lang.model,javax.lang.model.element,javax.lang.model.type,javax.lang.model.util,javax.management,javax.management.loading,javax.management.modelmbean,javax.management.monitor,javax.management.openmbean,javax.management.relation,javax.management.remote,javax.management.remote.rmi,javax.management.timer,javax.naming,javax.naming.directory,javax.naming.event,javax.naming.ldap,javax.naming.spi,javax.net,javax.net.ssl,javax.print,javax.print.attribute,javax.print.attribute.standard,javax.print.event,javax.rmi,javax.rmi.CORBA,javax.rmi.ssl,javax.script,javax.security.auth,javax.security.auth.callback,javax.security.auth.kerberos,javax.security.auth.login,javax.security.auth.spi,javax.security.auth.x500,javax.security.cert,javax.security.sasl,javax.sound.midi,javax.sound.midi.spi,javax.sound.sampled,javax.sound.sampled.spi,javax.sql,javax.sql.rowset,javax.sql.rowset.serial,javax.sql.rowset.spi,javax.swing,javax.swing.border,javax.swing.colorchooser,javax.swing.event,javax.swing.filechooser,javax.swing.plaf,javax.swing.plaf.basic,javax.swing.plaf.metal,javax.swing.plaf.multi,javax.swing.plaf.synth,javax.swing.table,javax.swing.text,javax.swing.text.html,javax.swing.text.html.parser,javax.swing.text.rtf,javax.swing.tree,javax.swing.undo,javax.tools,javax.xml,javax.xml.crypto,javax.xml.crypto.dom,javax.xml.crypto.dsig,javax.xml.crypto.dsig.dom,javax.xml.crypto.dsig.keyinfo,javax.xml.crypto.dsig.spec,javax.xml.datatype,javax.xml.namespace,javax.xml.parsers,javax.xml.transform,javax.xml.transform.dom,javax.xml.transform.sax,javax.xml.transform.stax,javax.xml.transform.stream,javax.xml.validation,javax.xml.xpath,org.ietf.jgss,org.omg.CORBA,org.omg.CORBA.DynAnyPackage,org.omg.CORBA.ORBPackage,org.omg.CORBA.TypeCodePackage,org.omg.CORBA.portable,org.omg.CORBA_2_3,org.omg.CORBA_2_3.portable,org.omg.CosNaming,org.omg.CosNaming.NamingContextExtPackage,org.omg.CosNaming.NamingContextPackage,org.omg.Dynamic,org.omg.DynamicAny,org.omg.DynamicAny.DynAnyFactoryPackage,org.omg.DynamicAny.DynAnyPackage,org.omg.IOP,org.omg.IOP.CodecFactoryPackage,org.omg.IOP.CodecPackage,org.omg.Messaging,org.omg.PortableInterceptor,org.omg.PortableInterceptor.ORBInitInfoPackage,org.omg.PortableServer,org.omg.PortableServer.CurrentPackage,org.omg.PortableServer.POAManagerPackage,org.omg.PortableServer.POAPackage,org.omg.PortableServer.ServantLocatorPackage,org.omg.PortableServer.portable,org.omg.SendingContext,org.omg.stub.java.rmi,org.w3c.dom,org.w3c.dom.bootstrap,org.w3c.dom.css,org.w3c.dom.events,org.w3c.dom.html,org.w3c.dom.ls,org.w3c.dom.ranges,org.w3c.dom.stylesheets,org.w3c.dom.traversal,org.w3c.dom.views,org.xml.sax,org.xml.sax.ext,org.xml.sax.helpers,org.osgi.framework;version=1.5.0,org.osgi.framework.launch;version=1.0.0,org.osgi.framework.hooks.service;version=1.0.0,org.osgi.service.packageadmin;version=1.2.0,org.osgi.service.startlevel;version=1.1.0,org.osgi.service.url;version=1.0.0,org.osgi.util.tracker;version=1.4.0</param>

(that doesn’t display nicely in wordpress, but the copy and viewsource buttons do work correctly)

I hope that helps someone, figuring out those constraint violations is never much fun.

The Routemaster ESB: Introduction

My day job is for a company that produces recruitment systems, primarily aimed at new graduates. Within the company we have four different codebases:

  • Drupal
  • .Net MVC
  • Classic ASP (VBScript and horribly complex stored procedures, almost no compiled code at all)
  • Java (Spring MVC)

There is, of course, quite a lot of crossover in the tasks that these systems do, but currently there is very little functionality that is shared between them. That’s where I come in, my job is to extract functionality from these systems and reimplement it as a shared component, enabling the systems themselves to shrink and become more managable.

This is a kind of SOA, though the functionality that can be extracted tends to be more transactional rather than message-queue based, due to the nature of the systems. It would be nice to, for example, put in a system that manages “job descriptions” for all the other systems, but that’s just a step too far for now; rather I am looking at tasks such as scheduling, emails, PDF conversion and I also have ownership for public web services.

ESBs – The State of the Nation

When I took up this, role about a year ago, I began by reviewing the state of ESBs that were available. It was clear that my work would involve web services and I wanted to provide a platform for my work that would enable me to centralise (in a distributed manner :)) such things as security, logging, routing, etc.

I tried most of the commercial and open-source ESBs, but found them sorely lacking. In the end I chose glassfish-esb, and used it in earnest for some months, before eventually throwing it out too, for two main reasons:

  • glassfish-esb provided two options for tracking: track every action taken by a BPEL file, or track nothing.
    Tracking is vital for me, so the latter is not an option, and the former slowed things down to such an extent that it simply wasn’t viable.
  • Preparing a new route for the ESB was tortuous.
    A route should be a simple thing, it’s the bread and butter for an ESB, it shouldn’t require multiple maven projects to be set up for each route.
    That it does is really the fault of the JBI specification (implemented by a number of ESB products).

A third nightmare for working with glassfish-esb is that the netbeans plugins that it uses are not at all svn-friendly – they keep things that need to be rebuilt in directories that need to be managed by svn, but then delete (and recreate) those directories with no thought to svn (feel free to blame svn’s requirement of a .svn directory inside every directory it manages).

I also spent some time trying to use servicemix (3.X) as an alternative, but found it’s maven archetypes and depencies in such a mess that I never succeeded in actually getting it working.

There is a quote that is often used in relation to SOA/ESBs, and it is: Configuration over coding/customisation. BPEL is held up as an example of this, but I disagree, BPEL has no advantages over Java, and significant disadvantages:

  • Tranformations in BPEL are horrendously complex to write (compared with Java)
  • The source of the BPEL still has to be managed within a version control system
  • BPEL is not well known (and never will be as well known as Java)
  • Callouts from BPEL to something else increase complexity further

BPEL is, in fact, just a very poor language, not configuration at all.

There is one potential advantage of BPEL – as a standard it ought to make it easy to move between different ESB implementations. However the steps necessary to make that BPEL useful to an ESB are not standardized, so much of the benefit is theoretical (and noone wants to change ESB implementation very often).

The non-ESB

So, after ditching glassfish, and BPEL, I went for simplicity: I just implemented each route as a separate Java web application to be hosted within tomcat.
To make that manageable I wrote a separate jar library to do all the heavy lifting, so each route contained little more than a Spring configuration file and a single Java class implementing the external interface and calling the internal one.
This works, and is simple, but has a few significant limitations and one killer:

  • There is no failover for internal endpoints.
  • There is no way to dynamically create an endpoint.
  • Configuration of routes is via JNDI, which requires me to restart tomcat to pick up a change (this was a mistake).
  • It only supports web services, and only transactional ones at that (it’s a web service proxy, not an ESB).

I could fix all of those limitations by just writing some code, so what’s the killer? It’s still too much work to deploy a route. Each web application can really only support a single external endpoint, so providing separate routes for test, UAT and live requires complely independent web applications (i.e. more maven projects).

Routemaster ESB

The first port of call on my search for a replacement ESB was Apache Camel. Which provides a dynamically configurable infrastructure that does everything I want (and that is used at the heart of many open source ESBs).

Camel alone isn’t enough though, I remain of the opinion that the best language for handling complex logic (or even just transformations) within a route is Java – where, thanks to CXF/JAXB I can get compile time checking of everything – and using Java means I need to be able to dynamically load, unload and reload compiled java classes, which means OSGi.
I looked at a few of the OSGi containers available before settling on felix/karaf. It’s arguable that, in some ways, the eclipse equinox container is better, but I’m just prejudiced against eclipse – it shouldn’t matter which you choose.

With camel and karaf installing a route can be as simple as placing a Spring context file in the correct location. Any resources used by the context file (particularly wsdl files) will also have to be deployed to the server, and any java that is required in the route will need to be packaged up as an OSGi service using Spring-DM.

I will need to provide some standard services used by all my routes – tracing, error translation, and CXF basic authentication being three examples – these will all be implemented as OSGi services. I will also need to provide some web-based UI features for accessing configuration and trace information – though I hope to minimise the effort here by providing access to the Camel and Felix web interfaces.

I want to use as much of my existing code as possible, so I’m going to need to get JPA working from within felix.

Routemaster is not a product, you cannot (and never will be able to) download it. What it is, is a demonstration of how you can build your own ESB using open source components, to do precisely what you want.
Over the coming weeks I’ll be adding blogs on the development of Routemaster, eventually leading to a number of downloadables so you’ve got the same base as me.
On top of that base you will be able to develop you own components to make your ESB do what you want.

I don’t claim that I am the only one to have concluded that OSGi is important to ESBs: many of the open source ESBs have versions based on OSGi due real soon now, but I think I’ve lost faith in them (certainly in servicemix and glassfish). Also these ESBs tend to be trying to provide JBI on top of OSGi, and JBI is a dead end for me. I sincerely hope that the open source ESBs will, eventually, offer a much better solution than Routemaster ESB, but for now much of what I’m using is bleeding edge and the ESBs simply aren’t there yet.

So where is it? Well, it’ll be coming out bit by bit as a series of “Learn Something New Every Day” posts:

LSNED 31: pax-provision with camel-spring-osgi and karaf

I am working on a Camel OSGi bundle for a simple ESB in my day job.
I have made the decision, relatively arbitrarily, to use felix/karaf as the container, and I’ve had camel-spring-osgi running inside that quite happily.

As part of my development I’m clearly going to be needing to set up a clean container and load the appropriate bundles into it – which means I’m going to be using pax-provision.
And just for added fun I’m working on what will be camel 2.3, so I need to provision the 2.3-SNAPSHOT bundles, not the live 2.2.0 ones.

I do my development on Windows, but the live environment will be linux (centos, which I don’t recommend at all).

Getting all of this running has been somewhat painful (taking up most of the last week and all of the last two days), so here’s what I went through.

pax-provision

First step was to install the pax-construct scripts and use them to create a project and a few bundles.
The starting point for this blog is that you’ve already done that, so I’m not going to rehash the command here.

karaf

The default pax-provision/pax-runner environment is a raw felix shell, but I want to be using karaf for my live deployment and don’t want to be developing in a different environment, so I had to get karaf working in pax-provision.
Fortunately this is very simple, in the pom.xml file for the project simply edit the plugin configuration to this:

      <plugin>
        <groupId>org.ops4j</groupId>
        <artifactId>maven-pax-plugin</artifactId>
        <version>1.4</version>
        <configuration>
          <provision>
            <param>--platform=felix</param>
            <param>--profiles=log,web</param>
            <param>--shell=karaf.shell/1.2.0</param>
          </provision>
        </configuration>
      </plugin>

Note that the latest release of karaf is 1.4.0, but this is using 1.2.0.
I’m just going to live with that!

Unfortunately, after doing this on my dev box (Windows) I found that the karaf shell did nothing, it simply would not accept any commands.
I could ssh into karaf running on the windows box, but I can’t find out how to set the password for the account used under pax-provision (where is the etc dir that karaf is looking for?) so that wasn’t much use.
Trying the same on a linux box worked, so I’ve installed a couple of VMs and I’m doing all my testing on the linux VMs (ubuntu 10.4 beta).
I think this is a bug in the karaf shell (possibly something that 1.4.0 would fix) but I couldn’t find anyone else talking about it so I’m just working on linux instead.

karaf features

Unfortunately the basic karaf shell does not provide all the facilities found in a basic karaf download.
We can get around this by installing a couple of karaf features, and fortunately pax-runner makes this easy:

      <plugin>
        <groupId>org.ops4j</groupId>
        <artifactId>maven-pax-plugin</artifactId>
        <version>1.4</version>
        <configuration>
          <provision>
            <param>--platform=felix</param>
            <param>--profiles=log,web</param>
            <param>scan-features:mvn:org.apache.felix.karaf/apache-felix-karaf/1.4.0/xml/features!/wrapper,war,webconsole,ssh,management</param>
            <param>--shell=karaf.shell/1.2.0</param>
          </provision>
        </configuration>
      </plugin>

You can find the features URL by using features:listUrl on a running download of karaf (i.e. not a pax-provision version).
Note that the karaf.shell is v1.2.0, but the features URL must be 1.4.0 – trying to use karaf 1.2.0 caused problems with the camel-cxf bundle (felix never completed resolution).

karaf management

One of the features usually found in a standard karaf installation is the management feature.
Unfortunately enabling that feature caused pax-provision/pax-runner/maven to fall over, due to a dependency on mvn:org.osgi/org.osgi.impl.bundle.jmx/4.2.0.200907080519.
I’m pretty sure that something about either maven on my VM, or maven as used by pax-runner is configured incorrectly, however getting that bundle into my local repository solved the problem.
And how did I get that bundle into my local repository?
Simple, I downloaded the source for karaf, untarred it and ran mvn from the src directory!
Much simpler than manually hunting for dependencies, though whether karaf management was worth it I don’t know :).

camel

Getting camel into karaf is easy, just add the features URL to karaf and install the necessary features.
Unfortunately I cannot find any publicly available features file for camel that provide the 2.3-SNAPSHOT versions.
Not a big problem, the features file is easily found within the camel svn: camel/platforms/karaf/features/target/classes/features.xml
So my final plugins configuration is:

      <plugin>
        <groupId>org.ops4j</groupId>
        <artifactId>maven-pax-plugin</artifactId>
        <version>1.4</version>
        <configuration>
          <provision>
            <param>--platform=felix</param>
            <param>--profiles=log,web</param>
            <param>scan-features:mvn:org.apache.felix.karaf/apache-felix-karaf/1.2.0/xml/features!/obr,wrapper,ssh,management</param>
            <param>scan-features:file:/home/me/Work/Camel/camel/platforms/karaf/features/target/classes/features.xml!/camel-core,camel-spring-osgi,camel-cxf</param>
            <param>--shell=karaf.shell/1.2.0</param>
          </provision>
        </configuration>
      </plugin>

But that’s still not quite there: camel now hits the same problem jmx had – mvn within felix/karaf/pax-runner/pax-provision/my VM is not willing to go out and download dependencies.
So I have to get the dependencies for camel into the local repository manually.
And, as with karaf, the easiest way to do that is to just build it, only this time the source is found via svn rather than download: svn co https://svn.apache.org/repos/asf/camel/trunk camel

Maven Proxies

If you, like us, have a maven proxy then both karaf and camel builds will cause lots of build failures because they use a lot or components from non-standard repositories (particularly when building SNAPSHOT builds).
You have two choices: either manually track down each dependency and add it to your proxy; or, somewhat simpler, just disable the proxy for those builds by changing your settings.xml file (don’t forget to change it back again afterwards).

Summary

I hope camel 2.3.0 gets released soon (though at the time of writing I get build and test failures with every update on linux)!

It would also be nice if pax-runner karaf.shell was updated to 1.4.0.

LSNED 30: Instantiating a CXF web service without access to the WSDL file

When you use the cxf-codegen-plugin maven plugin to generate your web service client code from a WSDL file it, by default, hardcodes the location of the WSDL file and tries to read that file when you instantiate the client.

The simple trick to avoid this is to tell wsdl2java a null wsdlLocation in the pom.xml:

            <plugin>
                <groupId>org.apache.cxf</groupId>
                <artifactId>cxf-codegen-plugin</artifactId>
                <version>2.2.6</version>
                <executions>
                    <execution>
                        <id>generate-hasher-client</id>
                        <phase>generate-sources</phase>
                        <configuration>
                            <wsdlOptions>
                                <wsdlOption>
                                    <wsdl>src/main/resources/MyService.wsdl</wsdl>
                                    <extraargs>
                                        <extraarg>-client</extraarg>
                                        <!-- Note that the next two are a pair, without them the path to the WSDL file gets hardcoded in the Service -->
                                        <extraarg>-wsdlLocation</extraarg>
                                        <extraarg></extraarg>
                                    </extraargs>
                                </wsdlOption>
                            </wsdlOptions>
                        </configuration>
                        <goals>
                            <goal>wsdl2java</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Unfortunately, by itself this will produce an error like this:

javax.xml.ws.WebServiceException: Port {http://MyServiceNamespace}MyServiceBindingPort not found.

To avoid this you need to excplicitly configure the port when creating the client:

        public SmsWebService( String url, String userName, String password )
        {
            this.targetService = new MyService( null );
            this.targetService.addPort( MyService.MyServicePortTypeBindingPort, SOAPBinding.SOAP11HTTP_BINDING, url );
            this.port = targetService.getMyServicePortTypeBindingPort();

            BindingProvider bindingProvider = ( BindingProvider ) port;
            bindingProvider.getRequestContext().put( BindingProvider.ENDPOINT_ADDRESS_PROPERTY, url );
            if ( (userName != null) && (!userName.isEmpty()) && (password != null) && (!password.isEmpty()) )
            {
                bindingProvider.getRequestContext().put( BindingProvider.USERNAME_PROPERTY, userName );
                bindingProvider.getRequestContext().put( BindingProvider.PASSWORD_PROPERTY, password );
            }
        }
Follow

Get every new post delivered to your Inbox.