Heroku is Hiring

It’s been four months since I started working at Heroku as a Developer Evangelist. Now it is clear to me that I got really lucky. Heroku is a top-notch place to work. The product is sexy. The people are all rock stars. Heroku is owned by Salesforce.com so there is the stability of a large company but the start-up culture remains in place.

Heroku is looking to hire lots of people (engineers, marketing, etc) but of particular interest to my readers might be the Java Developer Evangelist position. Come work with me to help educate Java developers about Heroku!

BTW: Check out how the application asks for your GitHub URL. I love this place!

Getting Started with Scala on Heroku

Over the past year I’ve been gradually learning Scala and I think it’s fantastic! So I’m incredibly excited that Scala now runs on Heroku! Of course you can use the standard Java on Heroku / Maven method of running Scala on Heroku. But as of today you can also use sbt (the Scala Build Tool) to run Scala apps on Heroku. If you are new to Heroku, it is a Polyglot Cloud Application Platform. Put very simply:

Heroku = Polyglot + Platform as a Service (PaaS) + Cloud Components

If you want to try out Scala on Heroku, here are a few quick steps to get you started:

  1. Create a Heroku account
  2. Install the Heroku command line client on Linux, Mac, or Windows.
  3. Install git and setup your SSH key
  4. Install sbt 0.11.0
  5. Login to Heroku from the command line:

    heroku auth:login
  6. Create a file named build.sbt containing:
    scalaVersion := "2.9.1"
     
    {
      val stage = TaskKey[Unit]("stage", "Prepares the project to be run, in environments that deploy source trees rather than packages.")
      stage in Compile := {}
    }

    This adds the “stage” task which is used for the build on Heroku.

  7. Create a project/build.properties file containing:
    sbt.version=0.11.0

    This tells sbt which version of sbt to use.

  8. Create a very simple Scala app in src/main/scala/Hello.scala containing:
    object Hello extends App {
      println("hello, world")
    }
  9. Test the app locally by running:
    sbt run

    You should see something like:

    [info] Set current project to default-0c17d0 (in build file:/home/jamesw/projects/helloscala/)
    [info] Running Hello 
    hello, world
    [success] Total time: 1 s, completed Sep 7, 2011 4:17:01 AM
  10. Create a .gitignore file containing:
    target
    project/boot
    project/target
  11. Create a git repo, add the files to it, and commit them:
    git init
    git add .
    git commit -m init
  12. Create a new app on Heroku using the Cedar stack:
    heroku create -s cedar
  13. Upload your app to Heroku using git:
    git push heroku master
  14. Run the app on Heroku:
    heroku run "sbt run"

    Voilà! You just ran Scala on the Cloud!

You can get the full code for this project on github:
https://github.com/jamesward/helloscala

That was a very simple example to get you started. Visit the Heroku Dev Center to continue learning about how to use Scala on Heroku. And let me know if you have any questions.

Sending Play Framework File Uploads to Amazon S3

UPDATE: I’ve released a S3 Play Module based on this project.

A couple of questions [1, 2] on StackOverflow.com led me to look into how we can send file uploads in a Play Framework application to Amazon S3 instead of the local disk. For applications running on Heroku this is especially important because the local disk is not persistent. Persistent disk storage makes it hard to scale apps. Instead of using the file system, it’s better to use an external service which is independent of the web tier.

While at JavaZone I sat down with Peter Hilton and Nicolas Leroux to come up with a way to handle this. It only took us 30 minutes to get something working – start to finish – including setup time. This is what is so compelling about Play Framework. I’ve built many Java web apps and it always seems like I spend too much time setting up builds, IDEs, and plumbing. With Play we were setup and working on the actual app in less than a minute. After getting everything working locally it took another minute to actually run it on the cloud with Heroku. The combination of Play Framework and Heroku is a developer’s dream for fast-paced development and deployment.

All of the code for the sample application is on github:
https://github.com/jamesward/plays3upload

The basics of what we did was this:

public static void doUpload(String comment, File attachment)
{
    AWSCredentials awsCredentials = new BasicAWSCredentials(System.getenv("AWS_ACCESS_KEY"), System.getenv("AWS_SECRET_KEY"));
    AmazonS3 s3Client = new AmazonS3Client(awsCredentials);
    s3Client.createBucket(BUCKET_NAME);
    String s3Key = UUID.randomUUID().toString();
    s3Client.putObject(BUCKET_NAME, s3Key, attachment);
    Document doc = new Document(comment, s3Key, attachment.getName());
    doc.save();
    listUploads();
}

This uses a JPA Entity to persist the metadata about the file upload (for some reason we named it ‘Document’) and a reference to the file’s key in S3. But there was a sexier way, so my co-worker Tim Kral added a new S3Blob type that could be used directly in the JPA Entity. Tim also cleaned up the configuration to make it more Play Framework friendly. So lets walk through the entire app so you can see the pieces.

The app/models/Document.java JPA Entity has three fields – the file being of type S3Blob:

package models;
 
import javax.persistence.Entity;
 
import play.db.jpa.Model;
import s3.storage.S3Blob;
 
@Entity
public class Document extends Model
{
    public String fileName;
    public S3Blob file;
    public String comment;
}

The S3Blob is now doing all of the work to talk to the Amazon S3 APIs to persist and fetch the actual file.

Configuration of S3 is done by adding a plugin to the conf/play.plugins file:

0: s3.storage.S3Plugin

The S3Plugin handles reading the AWS credentials from the conf/application.conf file, setting up the S3Client, and creating the S3 Bucket – if necessary.

In the conf/application.conf file, environment variables are mapped to the configuration parameters in the Play application:

aws.access.key=${AWS_ACCESS_KEY}
aws.secret.key=${AWS_SECRET_KEY}
s3.bucket=${S3_BUCKET}

The values could be entered into the conf file directly but I used environment variables so they would be easier to change when running on Heroku.

The Amazon AWS API must be added to the conf/dependencies.yml file:

require:
    - play
    - com.amazonaws -> aws-java-sdk 1.2.7

The sample application has a new controller in app/controllers/Files.java that can display the upload form, handle the file upload, display the list of uploads, and handle the file download:

package controllers;
 
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.util.List;
 
import models.Document;
import play.libs.MimeTypes;
import play.mvc.Controller;
import s3.storage.S3Blob;
 
public class Files extends Controller
{
 
  public static void uploadForm()
  {
    render();
  }
 
  public static void doUpload(File file, String comment) throws FileNotFoundException
  {
    final Document doc = new Document();
    doc.fileName = file.getName();
    doc.comment = comment;
    doc.file = new S3Blob();
    doc.file.set(new FileInputStream(file), MimeTypes.getContentType(file.getName()));
 
    doc.save();
    listUploads();
  }
 
  public static void listUploads()
  {
    List<Document> docs = Document.findAll();
    render(docs);
  }
 
  public static void downloadFile(long id)
  {
    final Document doc = Document.findById(id);
    notFoundIfNull(doc);
    response.setContentTypeIfNotSet(doc.file.type());
    renderBinary(doc.file.get(), doc.fileName);
  }
 
}

The uploadForm() method just causes the app/views/Files/uploadForm.html page to be displayed.

The doUpload() method handles the file upload and creates a new Document object that stores the file in S3 and the comment in a database. After storing the file and comment it runs the listUploads() method. Of-course a database must be configured in the conf/application.conf file. For running on Heroku the database is provided and just needs to be configured with the following values:

db=${DATABASE_URL}
jpa.dialect=org.hibernate.dialect.PostgreSQLDialect
jpa.ddl=update

The listUploads() method fetches all Document objects out of the database and then displays the apps/views/files/listUploads.html page.

If a user selects a file from the list then the downloadFile() method is called which finds the file in S3 and sends it back to the client as a binary stream. An alternative to this would be to get the file directly from Amazon using either the S3 generatePresignedUrl() method or via CloudFront.

Finally in the conf/routes file, requests to “/” have been mapped to the Files.uploadForm() method:

GET     /                                       Files.uploadForm

That’s it! Now we have an easy way to persist file uploads in an external system!

Running the Play! app on Heroku

If you’d like to run this example on Heroku, here is what you need to do:

Install the heroku command line client on Linux, Mac, or Windows.

Login to Heroku via the command line:

heroku auth:login

Clone the git repo:

git clone git@github.com:jamesward/plays3upload.git

Move to the project dir:

cd plays3upload

Create the app on Heroku:

heroku create -s cedar

Set the AWS environment vars on Heroku:

heroku config:add AWS_ACCESS_KEY="YOUR_AWS_ACCESS_KEY" AWS_SECRET_KEY="YOUR_AWS_SECRET_KEY" S3_BUCKET="AN_AWS_UNIQUE_BUCKET_ID"

Upload the app to Heroku:

git push heroku master

Open the app in the browser:

heroku open

Let me know if you have any questions or problems. And thanks to Peter, Nicolas, and Tim for helping with this!

Getting Started with Play Framework on Heroku

UPDATE: This tutorial is for Play 1.x (an old version). Check out my Play 2 Tutorial if you want to get started with the latest stuff.

Last week Heroku announced that you can now run Java apps on Heroku. Today Heroku announced that you can also easily run Play Framework apps on Heroku! Here’s a quick guide to getting started with Play! on Heroku:

  1. Install the heroku command line client on Linux, Mac, or Windows.
  2. Install git and setup your SSH key
  3. Install Play! version 1.2.3
  4. Login to Heroku from the command line:

    heroku auth:login
  5. Create a new Play! app:
    play new play_hello_world
    cd play_hello_world
  6. Run the app locally to test it:
    play run --%production
  7. Create a git repo, add the files, and commit:
    git init
    git add app conf lib public test
    git commit -m init
  8. Create a new app on Heroku:
    heroku create -s cedar
  9. Push the app to Heroku:
    git push heroku master
  10. Open the app in your browser:
    heroku open

That’s it! If you want to learn more about Heroku, check out the Heroku for Java Workbook and the Heroku Dev Center. And if you are at Dreamforce 2011 then check out Felipe Oliveira’s session on “Introducing Play! Framework: Painless Java and Scala Web Applications” on Wednesday at 3:30pm.

Let me know what you think and if you have any questions.

Heroku Adds Java Support

Today Heroku announced that Java is now supported on the Heroku Cloud Application Platform! This is incredibly exciting news and I’m very lucky to be a Heroku for Java Developer Evangelist!

Joining salesforce.com and jumping into the the Java Cloud space holds some nostalgia for me. When I began using Java in 1997 I was working at an ISP in Denver. We did the regular web hosting thing, but when the first Java Servlet engines (like Java Web Server 1.0) came out, I created the “wantjava.com” hosting service. Things were really nasty at first. We could only run one instance of the JWS on a server so I came up with a really bad way to do “multi-tenancy”. I setup a cron job to rsync the customers’ .class files into the server’s webapp and then restart the server. Customers had to email me to get a servlet added to the web.xml file. Uggg… I feel like I need to go to confession for this. But it worked and as the Servlet containers improved we quickly migrated to a more sustainable model.

Now thirteen years later I am privileged to once again be part of Java on the Cloud. But this time around things are so much easier, better, and sexier! Heroku is a leading the way in a new generation of application deployment that is making things much better for us Java developers.

What is Heroku?

Shortly I will dive into how you can run Java on Heroku, but first, what is Heroku? From my perspective, Heroku is a Polyglot Cloud Application Platform. Heroku provides us a way to run Ruby, Node.js, Clojure, and Java applications on a managed, scalable, and multi-tenant system. Heroku also provides numerous add-ons that help us make the shift from monolithic middleware to Cloud Components. Another way to say it is:

Heroku = Polyglot + Platform as a Service (PaaS) + Cloud Components

It is very exciting to see these three things coming together! With Polyglot I can choose the right tool for the job. With PaaS I don’t have to think about managing operating systems, scalability, failover, etc. And with the Cloud Component Architecture I can keep my app thin and focused on what is unique to the problem it needs to solve. Heroku brings these models together as a cloud application platform.

Running Java Apps on Heroku

Heroku can run any Java app that runs in OpenJDK 6. Today Heroku uses Maven to create a “slug” for Java apps. That slug can then be loaded onto one or more “dynos“. You can tell a dyno to execute / start a Java app from the command line and you can also use a “Procfile” to provide a command that will auto-start for each instance of a specific dyno type. Web dynos are able to listen on a port and will receive HTTP traffic through a load balancer that is automatically setup for each app. With that background knowledge, lets dive into code!

For Dreamforce 2011, I (with the help of a few co-workers) put together a Heroku for Java Workbook. The Workbook provides detailed instructions on how to create web apps, connect to a database, setup worker processes, use the Redis to Go Heroku add-on, and use Spring Roo on Heroku. But if you are anxious to get started and don’t need as much hand-holding, here is a quick and very simple walk through of how to run Java on Heroku:

  1. Install the heroku command line client on Linux, Mac, or Windows.
  2. Install git and setup your ssh key
  3. Install Maven
  4. Login to Heroku from the command line:

    heroku auth:login
  5. Create a new project directory and move into it:

    mkdir helloherokujava
    cd helloherokujava
  6. Create a Maven build file named pom.xml containing:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <groupId>foo</groupId>
        <version>1.0-SNAPSHOT</version>
        <name>helloherokujava</name>
        <artifactId>helloherokujava</artifactId>
    </project>
  7. Create a Java source directory:

    mkdir -p src/main/java
  8. Create a new Java class in the src/main/java directory named Hello.java containing:

    public class Hello
    {
      public static void main(String[] args)
      {
        System.out.println("hello, world");
      }
    }
  9. Compile the class:

    mvn compile
  10. Run the class locally:

    java -cp target/classes Hello
  11. Create a local git repo, add the pom.xml file & src dir, and commit the files:

    git init
    git add pom.xml src
    git commit -m init
  12. Create a new app on Heroku using the Cedar stack:

    heroku create -s cedar
  13. Upload your app to Heroku:

    git push heroku master

    Heroku will create a slug for your app.

  14. Run the app on Heroku:

    heroku run "java -cp target/classes Hello"

    Heroku will start a new dyno with your slug and then run the specified command.

You just ran Java on the cloud! Obviously this is a very simple example. But I like to start new things with the simplest thing that could possibly work. Now that you have that working there is more to learn and much more power to harness!

Next Steps

Have fun and please let me know if you have any questions about Heroku.

Getting Started with Node.js on The Cloud

In my new job at salesforce.com I’m incredibly exited about getting into Heroku, a Platform as a Service provider / Cloud Application Platform. In a future blog post I’ll provide more details on what Heroku is and how it works. But if you are like me the first thing you want to do when learning a new technology is to take it for a test drive. I decided to take my Heroku test drive using the recently announced Node.js support. I’m new to Node.js, but at least I know JavaScript. Heroku also offers Ruby / Rails support but I don’t know Ruby – yet. So let me walk you through the steps I took (and that you can follow) to get started with Node.js on the Heroku Cloud.

Step 1) Sign up for Heroku

Step 2) Install the Heroku command line client

Step 3) Login to Heroku via the command line:

heroku auth:login

Step 4) Install git

Step 5) Install Node.js

Step 6) Create a Node.js app

I started by building a very simple “hello, world” Node.js app. In a new project directory I created two new files. First is the package.json file which specifies the app metadata and dependencies:

{
  "name": "hellonode",
  "version": "0.0.1",
  "dependencies": {
    "express": "2.5.11"
  },
  "engines": {
    "node": "0.8.4",
    "npm": "1.1.45"
  }
}

Then the actual app itself contained in a file named web.js:

var express = require('express');
 
var app = express.createServer(express.logger());
 
app.get('/', function(request, response) {
  response.send('hello, world');
});
 
var port = process.env.PORT || 3000;
console.log("Listening on " + port);
 
app.listen(port);

This app simply maps requests to “/” to a function that sends a simple string back in the response. You will notice that the port to listen on will first try to see if it has been specified through an environment variable and then fallback to port 3000. This is important because Heroku can tell our app to run on a different port just by giving it an environment variable.

Step 7) Install the app dependencies with npm:

npm install .

This uses the package.json file to figure out what dependencies the app needs and then copies them into a “node_modules” directory.

Step 8) Try to run the app locally:

node web.js

You should see “Listening on 3000” to indicate that the Node.js app is running! Try to open it in your browser:
http://localhost:3000/

Hopefully you will see “hello, world”.

Step 9) Heroku uses a “Procfile” to determine how to actually run your app. Here I will just use a Procfile to tell Heroku what to run in the “web” process. But the Procfile is really the foundation for telling Heroku how to run your stuff. I won’t go into detail here since Adam Wiggins has done a great blog post about the purpose and use of a Procfile. Create a file named “Procfile” in the project directory with the following contents:

web: node web.js

This will instruct Heroku to run the web app using the node command and the web.js file as the main app. Heroku can also run workers (non-web apps) but for now we will just deal with web processes.

Step 10) In order to send the app to Heroku the files must be in a local git repository. To create the local git repo, run the following inside of your project directory:

git init

Now add the three files you’ve created to the git repo:

git add package.json Procfile web.js

Note: Make sure you don’t add the node_modules directory to the git repo! You can have git ignore it by creating a .gitignore file containing just “node_modules”.

Commit the files to the local repo:

git commit -m "initial commit"

Step 11) Create an app on Heroku:

heroku create

A default / random app name is automatically assigned to your app.

Step 12) Now you can push your app to Heroku! Just run:

git push heroku master

You should see something like:

Counting objects: 8, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 729 bytes, done.
Total 6 (delta 2), reused 0 (delta 0)
 
-----> Heroku receiving push
-----> Node.js app detected
-----> Resolving engine versions
       Using Node.js version: 0.8.4
       Using npm version: 1.1.41
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
       npm WARN package.json hellonode@0.0.1 No README.md file found!
       npm http GET https://registry.npmjs.org/express/2.5.11
       npm http 200 https://registry.npmjs.org/express/2.5.11
       npm http GET https://registry.npmjs.org/express/-/express-2.5.11.tgz
       npm http 200 https://registry.npmjs.org/express/-/express-2.5.11.tgz
       npm http GET https://registry.npmjs.org/connect
       npm http GET https://registry.npmjs.org/mime/1.2.4
       npm http GET https://registry.npmjs.org/qs
       npm http GET https://registry.npmjs.org/mkdirp/0.3.0
       npm http 200 https://registry.npmjs.org/qs
       npm http 200 https://registry.npmjs.org/mkdirp/0.3.0
       npm http 200 https://registry.npmjs.org/mime/1.2.4
       npm http 200 https://registry.npmjs.org/connect
       npm http GET https://registry.npmjs.org/qs/-/qs-0.4.2.tgz
       npm http GET https://registry.npmjs.org/connect/-/connect-1.9.2.tgz
       npm http GET https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.0.tgz
       npm http GET https://registry.npmjs.org/mime/-/mime-1.2.4.tgz
       npm http 200 https://registry.npmjs.org/qs/-/qs-0.4.2.tgz
       npm http 200 https://registry.npmjs.org/mime/-/mime-1.2.4.tgz
       npm http 200 https://registry.npmjs.org/connect/-/connect-1.9.2.tgz
       npm http 200 https://registry.npmjs.org/mkdirp/-/mkdirp-0.3.0.tgz
       npm WARN package.json connect@1.9.2 No README.md file found!
       npm http GET https://registry.npmjs.org/formidable
       npm http 200 https://registry.npmjs.org/formidable
       npm http GET https://registry.npmjs.org/formidable/-/formidable-1.0.11.tgz
       npm http 200 https://registry.npmjs.org/formidable/-/formidable-1.0.11.tgz
       express@2.5.11 node_modules/express
       ├── qs@0.4.2
       ├── mime@1.2.4
       ├── mkdirp@0.3.0
       └── connect@1.9.2 (formidable@1.0.11)
       npm WARN package.json hellonode@0.0.1 No README.md file found!
       npm WARN package.json connect@1.9.2 No README.md file found!
       express@2.5.11 /tmp/build_h2gd35fmkgzm/node_modules/express
       connect@1.9.2 /tmp/build_h2gd35fmkgzm/node_modules/express/node_modules/connect
       qs@0.4.2 /tmp/build_h2gd35fmkgzm/node_modules/express/node_modules/qs
       mime@1.2.4 /tmp/build_h2gd35fmkgzm/node_modules/express/node_modules/mime
       formidable@1.0.11 /tmp/build_h2gd35fmkgzm/node_modules/express/node_modules/connect/node_modules/formidable
       mkdirp@0.3.0 /tmp/build_h2gd35fmkgzm/node_modules/express/node_modules/mkdirp
       Dependencies installed
-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size is 4.0MB
-----> Launching... done, v5
       http://fast-plateau-8288.herokuapp.com deployed to Heroku
 
To git@heroku.com:fast-plateau-8288.git
   26b4efc..6e333a2  master -> master

Now you should be able to connect to your app in the browser! An easy way to open the app in your browser is to run:

heroku open

To see your app logs (provisioning, management, scaling, and system out messages) run:

heroku logs

To see your app processes run:

heroku ps

And best of all, if you want to add more Dynos* just run:

heroku scale web=2

* Dynos are the isolated containers that run your web and other processes. They are managed by the Heroku Dyno Manifold. Learn more about Dynos.

That increases the number of Dynos running the app from one to two. Automatically Heroku will distribute the load across those two Dynos, detect dead Dynos, restart them, etc! That is seriously easy app scalability!

There is much more to Heroku and I’ll be continuing to write about it here. But in the meantime, check out all of the great docs in the Heroku Dev Center. And please let me know if you have any questions or problems. Thanks!