NoSQL Inside SQL with Java, Spring, Hibernate, and PostgreSQL

There are many benefits to schema-less NoSQL datastores, but there are always trade-offs. The primary gift the NoSQL movement has given us is the variety of options we now have for data persistence. With NoSQL we no longer must try to shoehorn everything into a relational model. Now the challenge is in deciding which persistence model fits best with each domain in a system and then combining those models in a cohesive way. The general term to describe this is Polyglot Persistence and there are many ways to accomplish it. Lets walk through how you can combine a regular SQL model with a key-value NoSQL model using Java, Spring, Hibernate, and PostgreSQL.

This article covers the pieces of a simple web application which uses regular SQL and PostgreSQL’s hstore for key value pairs. This method is a mix of NoSQL inside SQL. One benefit of this approach is that the same datastore can be used for both the SQL and the NoSQL data.

In this example the server technologies will be Java, Spring, and Hibernate. (The same thing can also be done with Rails, Django, and many other technologies.) To add Hibernate support for hstore I found a fantastic blog about “Storing sets of key/value pairs in a single db column with Hibernate using PostgreSQL hstore type“. I won’t go through that code here but you can find everything in the GitHub repo for my demo project.

This demo app uses Maven to define the dependencies. Embedded Jetty is started via a plain ‘ole Java application that sets up Spring MVC. Spring is configured via Java Config for the main stuff, the web stuff, and the database stuff.

The client technologies will be jQuery and Bootstrap and there is a strict seperation between the client and server via RESTful JSON services. The whole client-side is in a plain ‘ole HTML file. Via jQuery / Ajax the client communicates to JSON services exposed via a Spring MVC Controller.

Ok. Now onto the NoSQL inside SQL stuff. This application stores “Contacts” that have a name but also can have many “Contact Methods” (e.g. phone numbers and email addresses). The “Contact Methods” are a good use of a schema-less, key-value pair column because it avoids the cumbersome alternatives: putting that information into a separate table or trying to create a model object that has all of the possible “Contact Methods”. So lets take a look at the simple Contact Entity:

package com.jamesward.model;
 
import net.backtothefront.HstoreUserType;
import org.hibernate.annotations.Type;
import org.hibernate.annotations.TypeDef;
 
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import java.util.HashMap;
import java.util.Map;
 
@Entity
@TypeDef(name = "hstore", typeClass = HstoreUserType.class)
public class Contact {
 
    @Id
    @GeneratedValue
    public Integer id;
 
    @Column(nullable = false)
    public String name;
 
    @Type(type = "hstore")
    @Column(columnDefinition = "hstore")
    public Map<String, String> contactMethods = new HashMap<String, String>();
 
}

If you are familiar with Hibernate / JPA then most of this should look pretty familiar to you. The new / interesting stuff is the contactMethods property. It is a Map<String, String> and it uses PostgreSQL’s hstore datatype. In order for that to work, the type has to be defined and the columnDefinition set. Thanks again to Jakub Głuszecki for putting together the HstoreHelper and HstoreUserType that make this possible.

Now the rest is simple because it’s just plain Hibernate / JPA. Here is the ContactService that does the basic query and updates:

package com.jamesward.service;
 
import com.jamesward.model.Contact;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
 
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.criteria.CriteriaQuery;
 
import java.util.List;
 
@Service
@Transactional
public class ContactServiceImpl implements ContactService {
 
    @PersistenceContext
    EntityManager em;
 
    @Override
    public void addContact(Contact contact) {
        em.persist(contact);
    }
 
    @Override
    public List<Contact> getAllContacts() {
        CriteriaQuery<Contact> c = em.getCriteriaBuilder().createQuery(Contact.class);
        c.from(Contact.class);
        return em.createQuery(c).getResultList();
    }
 
    public Contact getContact(Integer id) {
        return em.find(Contact.class, id);
    }
 
    @Override
    public void addContactMethod(Integer contactId, String name, String value) {
        Contact contact = getContact(contactId);
        contact.contactMethods.put(name, value);
    }
 
}

Now that you understand how it all works, check out a live demo on Heroku.

If you want to run this app locally or on Heroku, then first you need to grab the source code and continue working inside the newly created spring_hibernate_hstore_demo directory:

$ git clone https://github.com/jamesward/spring_hibernate_hstore_demo.git
$ cd spring_hibernate_hstore_demo

To run locally:

  1. Setup your PostgreSQL database to support hstore by opening a psql connection to it:
    $ psql -U username -W -h localhost database
  2. Then enable hstore:
    => create extension hstore;
    => \q
  3. Build the app (depends on having Maven installed):
    $ mvn package
  4. Set the DATABASE_URL environment variable to point to your PostgreSQL server:
    $ export DATABASE_URL=postgres://username:password@localhost/databasename
  5. Start the app:
    $ java -cp target/classes:target/dependency/* com.jamesward.Webapp
  6. Try it out

Cool! Now you can run it on the cloud with Heroku. Here is what you need to do:

  1. Install the Heroku Toolbelt
  2. Login to Heroku:
    $ heroku login
  3. Create a new app:
    $ heroku create
  4. Add Heroku Postgres:
    $ heroku addons:add heroku-postgresql:dev
  5. Tell Heroku to set the DATABASE_URL environment variable based on the database that was just added (replace YOUR_HEROKU_POSTGRESQL_COLOR_URL with your own):
    $ heroku pg:promote YOUR_HEROKU_POSTGRESQL_COLOR_URL
  6. Open a psql connection to the database:
    $ heroku pg:psql
  7. Enable hstore support in your database:
    => create extension hstore;
    => \q
  8. Deploy the app:
    $ git push heroku master
  9. View the app on the cloud:
    $ heroku open

Fantastic! Let me know if you have any questions.

Containerless Spring MVC

Many of the new JVM-based web frameworks are ditching containers and WAR files and instead using a WAR-less / Containerless approach. But that doesn’t mean you have to ditch your favorite Java web framework. A while back I posted about going containerless with Tapestry. Now lets do the same with Spring MVC. You can grab the full source code from GitHub.

First we need a build that defines the dependencies. Here is the build.gradle file for my Gradle build:

apply plugin:'java'
apply plugin:'application'
 
version = '0.0.1-SNAPSHOT'
 
mainClassName = "com.jamesward.Webapp"
applicationName = "webapp"
 
repositories {
    mavenCentral()
}
 
dependencies {
    compile 'org.springframework:spring-webmvc:3.1.2.RELEASE'
    compile 'cglib:cglib:2.2.2'
    compile 'org.eclipse.jetty:jetty-webapp:8.1.5.v20120716'
}

There isn’t much to this build except a few dependencies: Spring MVC, CGLib, and Jetty.

The src/main/resources/assets/index.html file just contains simple HTML:

<!doctype html>
<html>
<body>
hello, world
</body>
</html>

The src/main/java/com/jamesward/WebConfig.java file uses Spring annotations to configure Spring MVC:

package com.jamesward;
 
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.*;
 
@Configuration
@EnableWebMvc
public class WebConfig extends WebMvcConfigurerAdapter {
 
    @Override
    public void addResourceHandlers(ResourceHandlerRegistry registry) {
        registry.addResourceHandler("/**").addResourceLocations("classpath:/assets/");
    }
 
    @Override
    public void addViewControllers(ViewControllerRegistry registry) {
        registry.addViewController("/").setViewName("redirect:index.html");
    }
 
}

Finally, a simple “static void main” Java class is used to start Jetty. The src/main/java/com/jamesward/Webapp.java file just sets up the HTTP listener and starts it:

package com.jamesward;
 
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.springframework.web.context.support.AnnotationConfigWebApplicationContext;
import org.springframework.web.servlet.DispatcherServlet;
 
 
public class Webapp {
 
    public static void main(String[] args) throws Exception {
 
        final AnnotationConfigWebApplicationContext applicationContext = new AnnotationConfigWebApplicationContext();
        applicationContext.register(WebConfig.class);
 
        final ServletHolder servletHolder = new ServletHolder(new DispatcherServlet(applicationContext));
        final ServletContextHandler context = new ServletContextHandler();
        context.setContextPath("/");
        context.addServlet(servletHolder, "/*");
 
        String webPort = System.getenv("PORT");
        if (webPort == null || webPort.isEmpty()) {
            webPort = "8080";
        }
 
        final Server server = new Server(Integer.valueOf(webPort));
 
        server.setHandler(context);
 
        server.start();
        server.join();
    }
 
}

That’s it! To build and run this project locally you can simple run:

./gradlew run

(Note: Run “gradlew.bat” on Windows.)

So simple it’s hard to believe it works. :) Let me know if you have any questions.

Integrating Java Spring Apps on Heroku with Force.com REST APIs

Recently I co-presented a webinar about how to integrate Java Spring Apps on Heroku with the Force.com / Salesforce.com REST APIs. Check out the recording:

I’ve also created an in-depth walk through of the code example and step-by-step instructions for setting up and deploying the example Java Spring app on Heroku. Check it out and let me know how it goes.

Graphs in the Cloud: Spring + Neo4j on Heroku

Last week I hosted a webinar about running Java apps on Heroku that use the Spring Framework and the Neo4j graph database. Here is the recording of that webinar:

In the webinar I began by deploying a copy of the Spring MVC + Hibernate template app from heroku.com/java on Heroku. Then I made a few modifications to the app to switch the persistence from Hibernate / JPA to Neo4j. You can get the full source code on GitHub.

Here is a quick recap of what I did to switch the template app to use Neo4j:

  1. Added the Neo4j Heroku Add-on:
    heroku addons:add neo4j
  2. Added the Spring Data Neo4j dependencies (optionally you can remove the unused JPA dependencies) to the “pom.xml” Maven build descriptor:
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-neo4j-rest</artifactId>
        <version>2.0.1.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-validator</artifactId>
        <version>4.2.0.Final</version>
    </dependency>
  3. Modified the “src/main/java/com/example/service/PersonService.java” interface to use the Neo4j GraphRepository:
    package com.example.service;
     
    import com.example.model.Person;
    import org.springframework.data.neo4j.repository.GraphRepository;
     
    public interface PersonService extends GraphRepository<Person> {
     
     
    }
  4. Removed the unneeded “src/main/java/com/example/service/PersonServiceImpl.java” DAO.
  5. Modified the “src/main/java/com/example/model/Person.java” POJO to be a @NodeEntity (instead of JPA Entity) and switched the “id” primary key property to be a Long annotated as a @GraphId:
    package com.example.model;
     
    import org.springframework.data.neo4j.annotation.GraphId;
    import org.springframework.data.neo4j.annotation.NodeEntity;
     
    @NodeEntity
    public class Person {
     
        @GraphId
        private Long id;
     
        // the rest is omitted
  6. Modified the “src/main/java/com/example/controller/PersonController.java” Spring MVC controller to use the new “PersonService”, take a Long parameter in the “deletePerson” method, and make the “deletePerson” and “addPerson” methods transactional:
    package com.example.controller;
     
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.stereotype.Controller;
    import org.springframework.transaction.annotation.Transactional;
    import org.springframework.web.bind.annotation.ModelAttribute;
    import org.springframework.web.bind.annotation.PathVariable;
    import org.springframework.web.bind.annotation.RequestMapping;
    import org.springframework.web.bind.annotation.RequestMethod;
     
    import com.example.model.Person;
    import com.example.service.PersonService;
     
    import java.util.Map;
     
    @Controller
    public class PersonController {
     
        @Autowired
        private PersonService personService;
     
        @RequestMapping("/")
        public String listPeople(Map<String, Object> map) {
            map.put("person", new Person());
            map.put("peopleList", personService.findAll().iterator());
            return "people";
        }
     
        @RequestMapping(value = "/add", method = RequestMethod.POST)
        @Transactional
        public String addPerson(@ModelAttribute("person") Person person) {
            personService.save(person);
            return "redirect:/people/";
        }
     
        @RequestMapping("/delete/{personId}")
        @Transactional
        public String deletePerson(@PathVariable("personId") Long personId) {
            personService.delete(personId);
            return "redirect:/people/";
        }
    }
  7. Then I modified the “src/main/resources/applicationContext.xml” Spring config file to use a file for local Neo4j storage in the “default” profile and then in the “prod” profile the “NEO4J_REST_URL”, “NEO4J_LOGIN”, and “NEO4J_PASSWORD” environment variables are used to connect to the Neo4j Heroku add-on service:
    <?xml  version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:context="http://www.springframework.org/schema/context"
           xmlns:tx="http://www.springframework.org/schema/tx"
           xmlns:mvc="http://www.springframework.org/schema/mvc"
           xmlns:neo4j="http://www.springframework.org/schema/data/neo4j"
           xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
                               http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
                               http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd
                               http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd
                               http://www.springframework.org/schema/data/neo4j http://www.springframework.org/schema/data/neo4j/spring-neo4j-2.0.xsd">
     
        <context:annotation-config />
        <context:spring-configured />
        <context:component-scan base-package="com.example" />
     
        <neo4j:repositories base-package="com.example.service"/>
     
        <mvc:annotation-driven/>
     
        <bean id="jspViewResolver" class="org.springframework.web.servlet.view.InternalResourceViewResolver">
            <property name="viewClass" value="org.springframework.web.servlet.view.JstlView" />
            <property name="prefix" value="/WEB-INF/jsp/" />
            <property name="suffix" value=".jsp" />
        </bean>
     
        <tx:annotation-driven />
     
        <beans profile="default">
            <neo4j:config storeDirectory="target/neo4j-db"/>
        </beans>
     
        <beans profile="prod">
            <bean class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase" id="graphDatabaseService">
                <constructor-arg index="0" value="#{systemEnvironment['NEO4J_REST_URL']}"/>
                <constructor-arg index="1" value="#{systemEnvironment['NEO4J_LOGIN']}"/>
                <constructor-arg index="2" value="#{systemEnvironment['NEO4J_PASSWORD']}"/>
            </bean>
     
            <neo4j:config graphDatabaseService="graphDatabaseService"/>
        </beans>
     
    </beans>
  8. After testing my changes locally (which actually didn’t work in my webinar due to a problem with Eclipse) I committed my changes to the git repo and pushed them to Heroku:
    git push heroku master

If you want to just skip to a working example on the cloud, simply follow the instructions in the project README.

Hopefully that helps you get started with Neo4j and Java applications on the cloud!

BTW: If you watched the webinar, you probably noticed that my Maven and Eclipse were misbehaving. Turns out that M2E didn’t read my Maven config and all I had to do was right-click on the project, select Maven, and then Update Project Configuration. That got everything back in sync. My excuse for not being able to figure that out during the demo… I usually use IntelliJ IDEA. :)

WebJars in Spring MVC

Last week I announced the WebJars project that allows you to specify your web libraries (JavaScript, CSS, etc) as dependencies in your Java web applications. With some help from Jeremy Grelle I was able to get a simple WebJars Spring MVC example working.

First you will need to add the WebJars repository to your build. For Maven, just add the following to your “pom.xml” build file:

    <repositories>
        <repository>
            <id>webjars</id>
            <url>http://webjars.github.com/m2</url>
        </repository>
    </repositories>

Then add a WebJar dependency, like Twitter Bootstrap:

        <dependency>
            <groupId>com.github.twitter</groupId>
            <artifactId>bootstrap</artifactId>
            <version>2.0.2</version>
        </dependency>

Then you need to add a resource handler to Spring MVC that maps requests from a given path to files in the classpath. The WebJars are all inside of a “public” directory. If you are using Java configuration in Spring then you would do the following:

@Configuration
@EnableWebMvc
public class WebConfig extends WebMvcConfigurerAdapter {
 
  @Override
  public void addResourceHandlers(ResourceHandlerRegistry registry) {
    registry.addResourceHandler("/public/**").addResourceLocations("classpath:/public/");
  }
}

Or if you are using XML config then add the following to your Spring config:

<mvc:resources mapping="/public/**" location="classpath:/public/"/>

Then you can reference a WebJar’s assets with something like:

<link rel='stylesheet' media='screen' href='/public/stylesheets/bootstrap.min.css'>

How easy is that? Now you are managing your web libraries as versioned dependencies and you even get transitive dependencies! Check out the full source code for this example on GitHub: https://github.com/jamesward/spring_webjars_demo

Right now there are only a couple of WebJars so if you need something else then simply request a new one via a new issue on GitHub.

Please let me know what you think about this. Thanks!

Video: Spring Roo and Grails Apps on the Cloud

At SpringOne 2GX I did a presentation about running Spring Roo (really Spring anything) and Grails apps on the cloud with Heroku. A video recording of my session has been posted on InfoQ.

If you want to try out Spring Roo or Grails on Heroku then one way is to deploy a copy of one of the demos I showed. I’ve created a little app that makes that easy. Just go to java.herokuapp.com and select a demo then enter your email address (your Heroku username) and click Go! A copy of the demo will be deployed for you on Heroku and then you will see instructions on how to get started with it. Give it a try and let me know what you think.

My Upcoming Flex Sessions in Atlanta – DevNexus and Atlanta Flex User Group

In a few weeks I’ll be speaking in Atlanta at DevNexus and at the Atlanta Flex User Group. On March 21, 2011 at DevNexus I have two sessions:

On March 22, 2011 at the Atlanta Flex User Group I will be presenting about Flex 4.5 – The Non-Mobile New Features.

I hope to see you at DevNexus and at the Atlanta Flex User Group!

Building Cross-Device Apps with Flex and Spring at the Detroit JUG

On February 15, 2011 I’ll be presenting at the Detroit Java User Group about Building Cross-Device Apps with Flex and Spring. If you live in the Detroit area then I hope to see you there! If not, then talk to your local Java User Group leader about having me come and speak.

Here is the abstract for the talk:
Today, users want apps in the browser, on their desktop, phone, tablet, and TV. Reusing code across all of these devices is now crucial for developers. By combining Flex for the UI with Spring and Hibernate for the back-end, developers can build apps that work on a variety of devices including PCs, Android phones / tablets, the BlackBerry PlayBook tablet, iPhones, and iPads. This session will walk developers through the steps for creating these cross-device apps with Flex and Java.

Go register now! See you there! :)



My JavaOne 2010 Sessions

Next week at JavaOne 2010 I’ll be co-presenting two sessions! On Monday Jeremy Grelle and I will present “End-to-End Richness: Integrating Java EE Services to Create Engaging RIAs” at the Hilton San Francisco. Here is the session information:
Session ID: S313939
Abstract: Flex is one of the most widely used tools for building RIAs. But how do you efficiently connect applications to diverse back-end services built on Java EE? The Spring BlazeDS Integration project has emerged as an effective solution for building a lightweight connective tissue for exposing rich Java EE services to a Flex client from any Java Servlet container. Aimed at experienced Java developers who want to make maximum use of their knowledge of Spring and Java EE to build a compelling integrated experience, this session covers:

  • Connecting services built with Spring and Java EE APIs (JPA, JMS, JAX-WS, and JAX-RS) to a Flex client
  • Using DI techniques to configure and expose RPC and pub/sub endpoints
  • Applying a rich security model

Time and Location: Monday, September 20, 16:00 | Hilton San Francisco, Golden Gate 3

On Thursday Greg Wilson and I will be present “Making Real-Time Data Come Alive“. Here is the session information:
Session ID: S313543
Abstract: Using a combination of Java, JMS, and Flash, data can be visualized in real time on a client. Real-time visualizations can be in dashboards, heat maps, charts, and numerous other interactive controls. Intended for developers looking to create applications with real-time data visualizations, this session will show how to set up a system that pushes real-time messages to a client from a Java EE server and how to create a rich internet application that renders the data. It will demonstrate how to:

  • Configure a Java EE system that can push JMS messages to a client in real-time or near real-time
  • Build a Flash client application that renders data from the server
  • Connect a client to messages using HTTP Streaming and HTTP Long Polling

Time and Location: Thursday, September 23, 15:30 | Parc 55, Cyril Magnin II

If you are at JavaOne then I hope to see you at one of these sessions!