Java and Play Framework on the Cloud at JavaZone

By James Ward | Published: August 31, 2011

Next week I am speaking in Oslo at JavaZone about deploying Java and Play! Framework apps on the Cloud. I submitted the session before Heroku for Java was available so I had to obfuscate the title and description. If you are at JavaZone then I hope to see you at my “Deploying Apps on Heroku” session!

Posted in Heroku, Java, Play Framework | 2 Comments

Getting Started with Play Framework on Heroku

By James Ward | Published: August 29, 2011

Last week Heroku announced that you can now run Java apps on Heroku. Today Heroku announced that you can also easily run Play Framework apps on Heroku! Here’s a quick guide to getting started with Play! on Heroku:

  1. Install the heroku command line client on Linux, Mac, or Windows.
  2. Install git and setup your SSH key
  3. Install Play! version 1.2.3
  4. Login to Heroku from the command line:

    heroku auth:login
  5. Create a new Play! app: 
    play new play_hello_world
cd play_hello_world
  6. Run the app locally to test it: 
    play run --%production
  7. Create a git repo, add the files, and commit: 
    git init
git add app conf lib public test
git commit -m init
  8. Create a new app on Heroku: 
    heroku create -s cedar
  9. Push the app to Heroku: 
    git push heroku master
  10. Open the app in your browser: 
    heroku open

That’s it! If you want to learn more about Heroku, check out the Heroku for Java Workbook and the Heroku Dev Center. And if you are at Dreamforce 2011 then check out Felipe Oliveira’s session on “Introducing Play! Framework: Painless Java and Scala Web Applications” on Wednesday at 3:30pm.

Let me know what you think and if you have any questions.

Posted in Heroku, Play Framework | 34 Comments

Heroku Adds Java Support

By James Ward | Published: August 25, 2011

Today Heroku announced that Java is now supported on the Heroku Cloud Application Platform! This is incredibly exciting news and I’m very lucky to be a Heroku for Java Developer Evangelist!

Joining salesforce.com and jumping into the the Java Cloud space holds some nostalgia for me. When I began using Java in 1997 I was working at an ISP in Denver. We did the regular web hosting thing, but when the first Java Servlet engines (like Java Web Server 1.0) came out, I created the “wantjava.com” hosting service. Things were really nasty at first. We could only run one instance of the JWS on a server so I came up with a really bad way to do “multi-tenancy”. I setup a cron job to rsync the customers’ .class files into the server’s webapp and then restart the server. Customers had to email me to get a servlet added to the web.xml file. Uggg… I feel like I need to go to confession for this. But it worked and as the Servlet containers improved we quickly migrated to a more sustainable model.

Now thirteen years later I am privileged to once again be part of Java on the Cloud. But this time around things are so much easier, better, and sexier! Heroku is a leading the way in a new generation of application deployment that is making things much better for us Java developers.

What is Heroku?

Shortly I will dive into how you can run Java on Heroku, but first, what is Heroku? From my perspective, Heroku is a Polyglot Cloud Application Platform. Heroku provides us a way to run Ruby, Node.js, Clojure, and Java applications on a managed, scalable, and multi-tenant system. Heroku also provides numerous add-ons that help us make the shift from monolithic middleware to Cloud Components. Another way to say it is:

Heroku = Polyglot + Platform as a Service (PaaS) + Cloud Components

It is very exciting to see these three things coming together! With Polyglot I can choose the right tool for the job. With PaaS I don’t have to think about managing operating systems, scalability, failover, etc. And with the Cloud Component Architecture I can keep my app thin and focused on what is unique to the problem it needs to solve. Heroku brings these models together as a cloud application platform.

Running Java Apps on Heroku

Heroku can run any Java app that runs in OpenJDK 6. Today Heroku uses Maven to create a “slug” for Java apps. That slug can then be loaded onto one or more “dynos“. You can tell a dyno to execute / start a Java app from the command line and you can also use a “Procfile” to provide a command that will auto-start for each instance of a specific dyno type. Web dynos are able to listen on a port and will receive HTTP traffic through a load balancer that is automatically setup for each app. With that background knowledge, lets dive into code!

For Dreamforce 2011, I (with the help of a few co-workers) put together a Heroku for Java Workbook. The Workbook provides detailed instructions on how to create web apps, connect to a database, setup worker processes, use the Redis to Go Heroku add-on, and use Spring Roo on Heroku. But if you are anxious to get started and don’t need as much hand-holding, here is a quick and very simple walk through of how to run Java on Heroku:

  1. Install the heroku command line client on Linux, Mac, or Windows.
  2. Install git and setup your ssh key
  3. Install Maven
  4. Login to Heroku from the command line:

    heroku auth:login
  5. Create a new project directory and move into it:

    mkdir helloherokujava
cd helloherokujava
  6. Create a Maven build file named pom.xml containing:

    <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>foo</groupId>
    <version>1.0-SNAPSHOT</version>
    <name>helloherokujava</name>
    <artifactId>helloherokujava</artifactId>
</project>
  7. Create a Java source directory:

    mkdir -p src/main/java
  8. Create a new Java class in the src/main/java directory named Hello.java containing:

    public class Hello
{
  public static void main(String[] args)
  {
    System.out.println("hello, world");
  }
}
  9. Compile the class:

    mvn compile
  10. Run the class locally:

    java -cp target/classes Hello
  11. Create a local git repo, add the pom.xml file & src dir, and commit the files:

    git init
git add pom.xml src
git commit -m init
  12. Create a new app on Heroku using the Cedar stack:

    heroku create -s cedar
  13. Upload your app to Heroku:

    git push heroku master

    Heroku will create a slug for your app.

  14. Run the app on Heroku:

    heroku run "java -cp target/classes Hello"

    Heroku will start a new dyno with your slug and then run the specified command.

You just ran Java on the cloud! Obviously this is a very simple example. But I like to start new things with the simplest thing that could possibly work. Now that you have that working there is more to learn and much more power to harness!

Next Steps

Have fun and please let me know if you have any questions about Heroku.

Posted in Heroku, Java | 6 Comments

WAR-less Java Web Apps

By James Ward | Published: August 23, 2011

Have you ever thought about why in Java we package up web apps into WAR files (or WAR directory structures)? It certainly is a convenient way to move an application and its dependencies from one place to another. But wouldn’t it be nice if everything could just stay in its original location and there wouldn’t be any moving of files around? Wouldn’t it also be nice if you specified your required version of Jetty or Tomcat just like you do with every other dependency? The WAR-less approach is one that is catching on as emerging Java web frameworks like Play! ditch the WAR files. With standard Java web apps we can also ditch the WAR files by simply launching an embedded Jetty or Tomcat server. Let’s give this a try and see how it goes.

For this experiment I’m going to use Maven and Jetty. This will still use the same standard source structure for a WAR file (src/main/java, src/main/webapp, etc). The major difference is that I will actually startup Jetty using a good-old static void main. This is similar to using the jetty:run goal but will allow us to have the same exact setup in development and in production. The static stuff will be in src/main/webapp, the compiled classes will be in target/classes, and the dependencies will be right were Maven downloaded them to. First, here is a little Java class (src/main/java/foo/Main.java) that sets up a Jetty server and starts it:

package foo;
 
import java.io.File;
import java.net.URL;
import java.util.jar.Attributes;
import java.util.jar.JarFile;
 
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.handler.*;
import org.eclipse.jetty.util.StringUtil;
import org.eclipse.jetty.webapp.WebAppContext;
 
public class Main
{
 
  public static void main(String[] args) throws Exception
  {
    String webappDirLocation = "src/main/webapp/";
 
    Server server = new Server(8080);
    WebAppContext root = new WebAppContext();
 
    root.setContextPath("/");
    root.setDescriptor(webappDirLocation + "/WEB-INF/web.xml");
    root.setResourceBase(webappDirLocation);
 
    root.setParentLoaderPriority(true);
 
    server.setHandler(root);
 
    server.start();
    server.join();
  }
}

As you can see, Main just references the webapp directory so I don’t have to copy the stuff from there to another place. Next I have a little test servlet (src/main/java/foo/HelloServlet.java):

package foo;
 
import java.io.IOException;
import java.io.PrintWriter;
 
import javax.servlet.ServletException;
import javax.servlet.http.*;
 
public class HelloServlet extends HttpServlet
{   
 
  @Override
  protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException
  {
    PrintWriter out = resp.getWriter();
    out.println("hello, world");
    out.close();
  }
}

And now the web.xml file (src/main/webapp/WEB-INF/web.xml):

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
  version="2.5">
  <servlet>
    <servlet-name>HelloServlet</servlet-name>
    <servlet-class>foo.HelloServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>HelloServlet</servlet-name>
    <url-pattern>/</url-pattern>
  </servlet-mapping>
</web-app>

And finally a pom.xml file that specifies Jetty as a dependency and provides an easy way to run the Main class:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
 
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.jamesward</groupId>
  <version>1.0-SNAPSHOT</version>
  <name>warless_java_web_app</name>
  <artifactId>warless_java_web_app</artifactId>
  <packaging>jar</packaging>
 
  <properties>
    <jettyVersion>7.3.1.v20110307</jettyVersion>
  </properties>
 
  <dependencies>
    <dependency>
      <groupId>org.eclipse.jetty</groupId>
      <artifactId>jetty-server</artifactId>
      <version>${jettyVersion}</version>
    </dependency>
    <dependency>
      <groupId>org.eclipse.jetty</groupId>
      <artifactId>jetty-webapp</artifactId>
      <version>${jettyVersion}</version>
    </dependency>
    <dependency>
      <groupId>org.eclipse.jetty</groupId>
      <artifactId>jetty-servlet</artifactId>
      <version>${jettyVersion}</version>
    </dependency>
  </dependencies>
 
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.2</version>
        <configuration>
          <mainClass>foo.Main</mainClass>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

And now simply run:

mvn compile exec:java

Maven compiles my Java classes into target/classes and then the exec:java goal runs the Main which finds the other WAR assets in the src/main/webapp directory. If you have been following along, make a request to http://localhost:8080/ to verify that it works (which it should).

There are two alternatives to running Jetty from Maven. You can use the Maven appassembler plugin to create start scripts containing the correct CLASSPATH references and then launch Main class using the generated scripts. Or you can use the Maven assembly or shade plugin to create a JAR containing the application and all of its dependencies.

Here is an example section of a pom.xml file for using the appassembler plugin:

  <plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>appassembler-maven-plugin</artifactId>
    <version>1.1.1</version>
    <configuration>
      <assembleDirectory>target</assembleDirectory> 
      <generateRepository>false</generateRepository>
      <programs>
        <program>
          <mainClass>foo.Main</mainClass>
          <name>main</name>
        </program>
      </programs>
    </configuration>
    <executions>
      <execution>
        <phase>package</phase>
        <goals>
          <goal>assemble</goal>
        </goals>
      </execution>          
    </executions>
  </plugin>

To generate the start scripts simply run:

mvn install

Then to run the script set the REPO environment variable to your Maven repository:

export REPO=~/.m2/repository

And then simply run the script:

sh target/bin/main

All of the code for this example is on github.com:
https://github.com/jamesward/warless_java_web_apps

To make all of this even easier, Jetty has a Maven archetype for generating everything for you. To create a new project containing this setup run:

mvn archetype:generate -DarchetypeGroupId=org.mortbay.jetty.archetype -DarchetypeArtifactId=jetty-archetype-assembler -DarchetypeVersion=7.5.0-SNAPSHOT

And now you are ready to build a WAR-less Java web app!

This setup is really the bare minimum required to handle web resource and servlet requests you will need to do a little more work if you want to add JSP support. Find out more about this in the Jetty documentation.

So… What do you think about Java Web apps without WAR files & WAR packaging? I’d love to hear your thoughts!

Posted in Java | 6 Comments

Dreamforce 2011

By James Ward | Published: August 10, 2011

I’m very excited to be presenting at Dreamforce (salesforce.com’s anual conference) this year! On Thursday, September 1, from 1:15 pm to 2:15 pm I will be presenting:

Developing Java Cloud Apps
The cloud makes it easy to deploy highly scalable apps in an instant. This session will walk you through the steps to build your first Java app for the cloud. You’ll also learn best practices for building mission-critical and horizontally scalable Java cloud apps.

Then on Friday, September 2, from 10:00 am to 11:00 am I will be hosting a panel discussion:

Fireside Chat: Java on the Cloud
Come join the Java on the cloud product managers, architects, and experts for a casual, unscripted chat to find out how Java developers can best take advantage of the cloud. The session will be a mix of preselected and audience-provided questions. So bring all your tough, interesting, and quirky questions to this Fireside Chat.

I hope to see you there!

Posted in Cloud, Conferences, Java | Leave a comment

Setup Play Framework with Scala in IntelliJ

By James Ward | Published: July 28, 2011

Yesterday at the Programming Summer Camp a group of us were working together to learn the Play Framework. Once we were able to get the basics working we wanted to get everything working in IntelliJ IDEA. Due to a lack of good documentation on the subject things did not go smoothly until we figured out the right “magical incantations”. We did eventually get it working so I wanted to document the steps we took.

  1. Add Scala support to Play: 
    play install scala
  2. Create a new Play project with Scala support: 
    play new foo --with scala
  3. Have Play create an IntelliJ Module Descriptor: 
    play idealize foo
  4. Run the app: 
    play run foo
  5. Access the app in a browser to generate some source files we will use later:
    http://localhost:9000
  6. Shutdown the Play server.
  7. Currently the Play Scala Module (version 0.9.1) only supports Scala 2.8.1 so download and extract that version.
  8. Create a new project (from scratch) in IntelliJ. The location should be the same as the directory where the Play project was created. Un-check the “Create module” option.
  9. In the Project Structure window, with “Modules” selected, click the “+” button to add a new module to the project.
  10. Select the “Import existing module” option and then point it to the generated .iml file.
  11. With the module now selected, select the content root block for the module (mine is /home/jamesw/projects/foo). Then select the “tmp/generated” directory in the tree on the right and press the “Sources” button. The HTML template pages are converted to .scala source files by Play and we need these source files to be included in the project along with the regular source files in the app directory.
  12. Select “Global Libraries” in the Platform Settings on the left and add a new Java Library named “scala-compiler-2.8.1″. Then press the “Attach Classes…” button and navigate to the “lib” directory in your Scala 2.8.1 directory. Then select the “scala-compiler.jar” and “scala-library.jar” files.
  13. Add another Global Library named “scala-library-2.8.1″ containing the “scala-dbc.jar”, “scala-library.jar”, and “scala-swing.jar” files.
  14. Press the “Apply” button to save the Global Libraries configuration.
  15. Select “Modules” and then the “Dependencies” tab. Press the “Add…” button and select “Library” to add a new library to the module. Select “scala-library-2.8.1″ and then press the “Add Selected” button to add it to the Dependencies.
  16. Select the “Scala” Facet in the module and set the Compiler library to the “scala-compiler-2.8.1″ option.
  17. Save the Project Structure by pressing the “Ok” button. Make sure the project now builds without any errors.
  18. To run the Play server from IntelliJ a new Run Configuration must be configured. To create a new Run Configuration select “Run” from the main IntelliJ menu and then select “Edit Configurations”.
  19. Press the “+” button and select “Application” from the list of Run Configuration types.
  20. Set the name to “Play Server”.
  21. Set the Main class to “play.server.Server”.
  22. Set the VM Parameters to: 
    -Dapplication.path="."
  23. De-select the “Make” option in the “Before Launch” section.
  24. Press the “Ok” button to save the configuration.
  25. Run the Play Server by selecting “Run” from the “Run” IntelliJ menu and verify that the application still works by opening the application in a browser.

It is so much easier to figure this stuff out with a group of people. That is just one of the many reasons why the Programming Summer Camp is a great event!

Let me know if you have any questions or problems.

Posted in Play Framework, Scala | 22 Comments

Architectural Evolution: From Middleware to The Cloud

By James Ward | Published: July 12, 2011

You’ve heard it said that “all things old are new again.” That statement can certainly be applied to the current Cloud hype. But each time the old becomes new it gets a bit better because of what was learned the last time around. If we look back ten years at enterprise application development in Java things were quite different than they are today. EJB was “the way” to build scalable systems from a vast abundance of components. But things didn’t work out as well as the vendors planned.

EJB Component Architecture

I remember back in the early days of enterprise Java everyone was talking about “Components.” Application complexity would be greatly reduced because there would be components for everything! Need to connect your app to Exchange? Well, there’s a component for that. Does your app need to send email? No problem, there are twenty components for that! Component marketplaces flourished with VC funding galore.

The official way to build reusable Java components became standardized as Enterprise Java Beans (EJB). These “beans” could be accessed either locally or remotely! Vendors led us to believe this was the panacea of Lego-style application development. Just grab pieces from every-which place and hook them together. Hooking the components together required a heavyweight “Middleware” server. Here is what Monolithic Middleware with EJBs looks like:

But the EJB Component Architecture didn’t work. Billions of dollars were spent on components and the middleware to tie them all together. And now I bet you can’t find a single person that doesn’t regret going that route. Why? Three primary reasons…

  1. The programming model was too hard. The EJB programming model consisted of too much boilerplate code (“solved” through code-gen tools like xdoclet). EJB’s also required configuration which was often middleware server-specific. The EJB Component Architecture creates too many layers of indirection (Core J2EE Patterns anyone?).
  2. Scalability was too hard. EJBs can either run inside your container (using what is called a “Local Interface”) or somewhere else (a “Remote Interface”). Using Local Interfaces is fast but causes middleware to run into memory limits and scaling bloated app servers is challenging. Using Remote Interfaces leads to massive serialization and routing overhead and whatever is on the remote end of the wire is still a pain to scale.
  3. Deployment was too hard. Remember the days when starting up an app server / middleware container took minutes not seconds?

If you need further proof that the middleware model didn’t work then just try to name one place you can still go to buy an EJB component today. Obviously we needed another way to compose the parts of an application.

POJO Component Architecture

SpringSource deserves a lot of credit for pulling us out of the EJB muck. They created a model where the application pieces are Plain Old Java Objects (POJOs) injected into an application. This led to better testability, much easier deployment, and a much better programming model. Essentially the revolution of Spring was to make all those app pieces injectable dependencies. This was a huge step forward. But there are still some limitations with this model that are currently being addressed by the next revolution. The three primary challenges with the POJO Component Architecture are:

  1. Isolation is too hard. It is now very easy to throw a bunch of components together into a single Web application ARchive (WAR). But at some point all of these pieces being stacked on top of each other make our application brittle and difficult to piece together. What do you do when the version of Hibernate you want to use requires a different version of an Apache Commons library than the version of XFire that you want to use? Or when two libraries that your app needs actually require conflicting dependencies. Sometimes isolating the pieces of an application is actually simpler than injecting them. And unfortunately with POJOs you may not be able to easily switch from using a “Local Interface” to an external “Remote Interface” like you can with EJBs.
  2. Polyglot is too hard. The POJO components we use today in our systems are not inherently supportive of a Polyglot world where different parts of a system may be built using different technologies. Suppose your system has a rules engine and you want to access it from a Java-based application and a Ruby-based application. Today the only way to do that is to proxy that component and expose it through an easily serialization protocol (likely XML or JSON over HTTP). This will likely add unnecessary complexity to your system. When the high-level functional pieces of a system are technology-specific the entire system may be forced to use that technology or those pieces may exist multiple times to support the Polyglot nature of today’s systems.
  3. Scaling is still too hard. As we continue to stack more pieces on top of each other it becomes harder to stick with simple, lightweight share-nothing architectures where each piece is individually horizontally scalable.

Cloud Component Architecture

The emerging solution to the challenges we have faced with the EJB and POJO Component Architectures is the Cloud Component Architecture. Instead of bundling components for things like search indexing, distributed caching, SMTP, and NoSQL data storage into your application those high level functions can instead be used as Cloud Components. There are already numerous vendors providing “Component as a Service” products like MongoDB, Redis, CouchDB, Lucene Search, SMTP, and Memcache.

SMTP / outbound email is a simple example where the Cloud Component Architecture makes a lot of sense. With the EJB and POJO Component Architectures I’d find a SMTP component that simply sends email. Then configure my server to be able to send emails that aren’t considered spam. I’d also need to deal with constant blacklisting challenges and a larger management surface. Or in a Cloud Component Architecture I could simply sign-up with one of the SMTP as a Service providers like AuthSMTP or SendGrid and then just use the Component as a Service.

Here is what the new Cloud Component Architecture for application composition looks like:

The top six benefits of the Cloud Component Architecture are:

  1. Simple scalability. By making each functional piece of an application an independent and lightweight service they can each be horizontally scaled without impacting the overall application architecture or configuration. If you chose to use a vendor’s Component as a Service then they will handle the scalability of those pieces. Then you only need to scale a very thin web layer. Composing Cloud Components also makes it easier to stick with a share-nothing architecture that is much easier to scale than the traditional architectures.
  2. Rapid composition. Cloud Components are flourishing! Most of the basic building blocks that applications need are now provided “as a Service” by vendors who maintain and enhance them. This is a much more erosion-resistant way to assemble applications when compared to the typical abandon-ware which is prevalent with many Java components. Many of the emerging Cloud Components also provide client libraries for multiple platforms and RESTful APIs to support easy composition in Polyglot systems.
  3. Reduced management surface. With Cloud Components you can reduce the number of pieces you must manage down to only the stuff that is unique to your app. Each Cloud Component you add doesn’t enlarge the management surface like it does in typical component models where you own the implementation of the component.
  4. Simple Deployment. One of the biggest benefits of using the Cloud is the ease of deployment. Partitioning the functional pieces of an application makes it thinner and easier to deploy. With Cloud Components you can also setup development and staging instances that make it easy to simulate the production environment. Then moving from one environment to another is simply a matter of configuration.
  5. Better Security. In most application architectures today there is one layer of security. This would be like a bank without a vault. There are a few ways into the bank that are wrapped with security (doors with locks) but as soon as someone has found a way in, they have access to everything. With Cloud Components security can be more easily distributed to provide multiple layers of security.
  6. Manageable costs. With Cloud Components your costs can scale with your usage. This means it’s easy to get started and grow rather than make large up-front investments.

The Cloud Component Architecture may seem similar in ways to the old EJB and POJO Component Architectures because it is similar! The wheel has not been reinvented, just improved. The dream of Lego-style application assembly is now being realized because we’ve come full circle on some old ideas from twenty years ago (CORBA anyone?). This time around those ideas are reality thanks to the evolution of many independent pieces like REST, Polyglot, and the Share-Nothing pattern. Cloud Components are the foundation of a new era of application development. My only question is… How long before we see the UDDI idea again? ;)

Posted in Cloud, Java | 9 Comments

Programming Summer Camp 2011

By James Ward | Published: June 29, 2011

Over the past few years I’ve attended a bunch of geek/coder events in the Colorado Mountains. It always surprises me how much I can learn by getting away, being around some really smart people, and writing code with them. So I’m incredibly excited to be attending The Programming Summer Camp 2011 this summer in Crested Butte, Colorado!

At the summer camp you can choose which campsite to hangout in. Or if you want, you can create your own campsite! I’ll probably spend most of my time in the Scala campsite. But I look forward to mingling in some of the other campsites as well.

It’s like I’m a kid again – getting all excited about going to camp! But I really hope you all can join me July 25 – 29 in Crested Butte, Colorado for The Programming Summer Camp 2011!

Posted in Conferences | Leave a comment

Getting Started with Node.js on The Cloud

By James Ward | Published: June 21, 2011

In my new job at salesforce.com I’m incredibly exited about getting into Heroku, a Platform as a Service provider / Cloud Application Platform. In a future blog post I’ll provide more details on what Heroku is and how it works. But if you are like me the first thing you want to do when learning a new technology is to take it for a test drive. I decided to take my Heroku test drive using the recently announced Node.js support. I’m new to Node.js, but at least I know JavaScript. Heroku also offers Ruby / Rails support but I don’t know Ruby – yet. So let me walk you through the steps I took (and that you can follow) to get started with Node.js on the Heroku Cloud.

(If you have already signed up for Heroku, installed the heroku command line client, and installed git then skip ahead to Step 6.)

Step 1) Sign up for Heroku

Step 2) Install the heroku command line client

All of the Heroku management tasks are exposed through a RESTful API. The easiest way to call those APIs is using the heroku open source command line Ruby app. To install the heroku command line I first had to install Ruby. I’m on Ubuntu Linux so this process will be slightly different if you are on Windows or Mac but the Heroku Dev Center provides more information on how to do this on Windows and Mac. On Ubuntu you can install Ruby with apt-get (or various other tools):

sudo apt-get install ruby

Now download RubyGems, unpack, and then install it:

sudo ruby setup.rb

This installs the gem utility at /usr/bin/gem1.8 but I also created a symlink to it so I can run it with just the “gem” command:

sudo ln -s /usr/bin/gem1.8 /usr/bin/gem

Now the heroku gem can be installed:

sudo gem install heroku

Heroku should now run from the command line:

heroku

You should see something like:

Usage: heroku COMMAND [--app APP] [command-specific-options]
 
Primary help topics, type "heroku help TOPIC" for more details:
 
  auth      # authentication (login, logout)
  apps      # manage apps (create, destroy)
  ps        # manage processes (dynos, workers)
  run       # run one-off commands (console, rake)
  addons    # manage addon resources
  config    # manage app config vars
  releases  # view release history of an app
  domains   # manage custom domains
  logs      # display logs for an app
  sharing   # manage collaborators on an app
 
Additional topics:
 
  account      # manage heroku account options
  db           # manage the database for an app
  help         # list commands and display help
  keys         # manage authentication keys
  maintenance  # toggle maintenance mode
  pg           # manage heroku postgresql databases
  pgbackups    # manage backups of heroku postgresql databases
  plugins      # manage plugins to the heroku gem
  ssl          # manage ssl certificates for an app
  stack        # manage the stack for an app
  version      # display version

Step 3) Login to Heroku via the command line

You can verify that everything is setup correctly by logging into Heroku through the heroku command line. This will save an API key into a ~/.heroku/credentials file. That key will be used for authenticating you on subsequent requests. Just run the following command and enter your Heroku credentials:

heroku auth:login

Step 4) Install git

The git tool is used to transfer apps to Heroku. On Ubuntu I installed it by doing:

sudo apt-get install git

Step 5) Setup your SSH key

Heroku uses SSH keys to authenticate you when you push files through git. If you don’t already have a SSH key then you will need to generate one (I used ssh-keygen).


Step 6) Create an app on Heroku

A new app needs to be provisioned on Heroku. Since Heroku supports multiple application provisioning stacks you will need to tell it the stack you want to use, unless it’s the default. For Node.js we need to use the “cedar” stack which is not the default since it’s still in beta. To do that run:

heroku create -s cedar

A default / random app name is automatically assigned to your app. It will be somethingunique.herokuapp.com. You can change the name either through the Heroku web admin or via the command line:

heroku apps:rename --app somethingunique hellofromnodejs

When the app was created your SSH key should have also been uploaded to Heroku for git access. You can manage the keys associated with an app using the “heroku keys” commands. Check out “heroku help keys” for more details.

Now that the app is provisioned it needs something to actually run! So lets build a Node.js app and then upload it to Heroku.

Step 7) Install Node.js

On Ubuntu I installed Node.js through apt-get. But first I had to add a PPA so that I could get the latest version.

sudo apt-add-repository ppa:jerome-etienne/neoip
sudo apt-get update
sudo apt-get install nodejs

For other platforms, check out the Node.js Download page.

Step 8) Create a Node.js app

I started by building a very simple “hello, world” Node.js app. In a new project directory I created two new files. First is the package.json file which specifies the app metadata and dependencies:

{
  "name": "heroku_hello_world",
  "version": "0.0.1",
  "dependencies": {
    "express": "2.2.0"
  }
}

Then the actual app itself contained in a file named web.js:

var express = require('express');
 
var app = express.createServer(express.logger());
 
app.get('/', function(request, response) {
  response.send('hello, world');
});
 
var port = process.env.PORT || 3000;
console.log("Listening on " + port);
 
app.listen(port);

This app simply maps requests to “/” to a function that sends a simple string back in the response. You will notice that the port to listen on will first try to see if it has been specified through an environment variable and then fallback to port 3000. This is important because Heroku can tell our app to run on a different port just by giving it an environment variable.

Step 9) Install the Node.js app dependencies

My simple Node.js app requires the Express Node.js library. In order to install Express, the Node Package Manager (npm) is required. Installing npm on Ubuntu was a bit trickey because I didn’t feel the regular method followed good security practices. So I followed the alternative install instructions by just cloning npm from github and then installed it from source:

git clone git://github.com/isaacs/npm.git
cd npm
sudo make install

Now we can install the node dependencies into the local project directory. Just run:

npm install .

This uses the package.json to figure out what dependencies the app needs and then copies them into a “node_modules” directory.

Step 10) Try to run the app locally

From the command line run:

node web.js

You should see “Listening on 3000″ to indicate that the Node.js app is running! Try to open it in your browser:
http://localhost:3000/

Hopefully you will see “hello, world”.

Step 11) Create a Procfile

Heroku uses a “Procfile” to determine how to actually run your app. Here I will just use a Procfile to tell Heroku what to run in the “web” process. But the Procfile is really the foundation for telling Heroku how to run your stuff. I won’t go into detail here since Adam Wiggins has done a great blog post about the purpose and use of a Procfile. Create a file named “Procfile” in the project directory with the following contents:

web: node web.js

This will instruct Heroku to run the web app using the node command and the web.js file as the main app. Heroku can also run workers (non-web apps) but for now we will just deal with web processes.

Note: Once you have a Procfile you can run your application locally using Foreman. This allows you to simulate locally how Heroku will run your app based on your Procfile.

Step 12) Store the project files in a local git repo

In order to send the app to Heroku the files must be in a local git repository. Of course you can also put them in a remote git repo (like github.com). To create the local git repo run the following inside of your project directory:

git init

Now add the three files you’ve created to the git repo:

git add package.json Procfile web.js

Note: Make sure you don’t add the node_modules directory to the git repo! You can have git ignore it by creating a .gitignore file containing just “node_modules”.

And commit the files to the local repo:

git commit -m "initial commit"

Step 13) Push the project files to Heroku

Now we need to tell git about the remote repository on Heroku which we will push the app to. When you provisioned the app on Heroku it gave you a web URL and a git URL. If you don’t have the git URL anymore you can determine it either by running the “heroku apps” command or by navigating to the app on heroku.com. The git URL will be something like “git@heroku.com:somethingunique.git” where the “somethingunique” is your app’s name on Heroku. Once you have the git URL add the remote repo:

git remote add heroku git@heroku.com:somethingunique.git

Note: If we had created the git repo before creating the Heroku app then the heroku command line client would have automatically added the remote repo to your git configuration.

Now you can push your app to Heroku! Just run:

git push heroku master

You should see something like:

Counting objects: 6, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (6/6), 617 bytes, done.
Total 6 (delta 0), reused 6 (delta 0)
 
-----> Heroku receiving push
-----> Node.js app detected
-----> Vendoring node 0.4.7
-----> Installing dependencies with npm 1.0.8
       express@2.2.0 ./node_modules/express 
       ├── mime@1.2.2
       ├── connect@1.4.4
       └── qs@0.1.0
       Dependencies installed
-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size is 3.1MB
-----> Launching... done, v4
       http://somethingunique.herokuapp.com deployed to Heroku
 
To git@heroku.com:somethingunique.git
 * [new branch]      master -> master

Now you should be able to connect to your app in the browser! You can also get some diagnostic information out of the heroku command line. To see your app logs (provisioning, management, scaling, and system out messages) run:

heroku logs

To see your app processes run:

heroku ps

And best of all, if you want to add more Dynos* just run:

heroku scale web=2

* Dynos are the isolated containers that run your web and other processes. They are managed by the Heroku Dyno Manifold. Learn more about Dynos.

That increases the number of Dynos running the app from one to two. Automatically Heroku will distribute the load across those two Dynos, detect dead Dynos, restart them, etc! That is seriously easy app scalability!

There is much more to Heroku and I’ll be continuing to write about it here. But in the meantime, check out all of the great docs in the Heroku Dev Center. And please let me know if you have any questions or problems. Thanks!

Posted in Cloud, Heroku, JavaScript, Node.js | 15 Comments