Tuesday, August 5, 2008

Ruby on Rails on REST on Java

(or Rails can Scales!)

In this 'very special issue' of Coderoshi, I'll go over how to create a scalable REST service in Java using CXF that can be consumed by Rails via ActiveResource. I do this based on the following assumptions:

* Rails sucks at scaling, but rocks at creating quick, slick presentations.
* Java sucks at writing presentation code, but scales like a mofo.

If you disagree, and really do believe Rails can scale with ActiveRecord's simple abstraction of an RDBMS, or that Java is awesome for presentation, best to stop reading now. But if you know enough about these technologies to muster a knowing nod, then hold onto your face - it's about to be rocked off.

What's all this, then?

Let us begin with an overview: Ruby on Rails ActiveResource consuming Java REST via CXF defined by JAXB. Whaa??

ActiveResource (not to be confused with ActiveRecord, Ruby's ORM) is a simple REST consumer. That's it. It wraps structured REST actions and managed data (xml or json) in active objects. Just like ActiveRecord, executing methods on an ARes object triggers backend actions or accessed data. For example:
@customer = Customer.find( 1 )
will translate into executing some defined REST service to populate @customer, internally constructing a URL like "http://server/customers/1.xml", getting the xml returned:
<customer>
<id>1</id>
<name>Joe</name>
</customer>
And making the value accessible by the Ruby program
#outputs: Joe
puts @customer.name
It's probably the drop-dead easiest method of consuming a remote service - assuming the remote service conforms to ActiveResource's standards (there's always a catch).

Happily, this isn't too difficult to do with the magic ingredient of the server-side: CXF. CXF is the evolution of Dan Diephouse's XFire (merged with IONA's Celtix). It's primarily considered to be a simple annotated WS-* impl, but also has some mean REST hooks. We'll be using the JSR 311 implementation rather than CXF's custom annotations.

Getting Started
Although I suggest you follow along, if you just plain hate writing code, then download both projects here. The Ruby on Rails project is called "ror" and the REST on Java project is "roj".

I hear it time and time again: Ruby on Rails can't scale. This is, of course, an absurd statement. If your software gets to the point that this is really a problem, well, then take your millions and rearchitect it. You can dry your eyes on hundred dollar bills while your Java counterparts are still fighting to get GWT to work. Most projects fail anyway - and I don't have enough lives to waste on them.

People complain that Java sucks at presentation, and for good reason. Sure, you can write webservices that scale out to thousands of servers, but try and write JSF and you have two choices: drink the koolaid, or pray for death.

With those points in mind, I'm going to go over how to create a scalable REST service in Java using CXF that can be consumed by Rails new ActiveResource.

Ingredients

Like a good cook, I like to gather my ingredients up front. Here is what we'll need to bake our little cake (I'm bolding the versions in case you can't read:

* Ruby >= 1.8.6
* Ruby on Rails >= 2.1 (2.0 is required for ActiveResource, but 2.1 has tons of bug fixes)
* Java >= 6 (or v5 - I haven't tried it)
* Maven >= 2.0.9 (you need the new graph resolution stuff, or dep overload may not work)

I presume you understand these technologies, at least in a cursory way. If not, there are plenty of good guides to get you started. Go ahead. I'll wait...

Oh, you're back? Then let's begin.

REST on Java

Although this is about scalability, the focus here isn't about handling multiple concurrent nodes. If you need a super-scalable data-management system check out HBase. If you want to cluster thousands of JVMs, use Terracotta. Wire it all up with Spring, and deploy using Maven. There's your scalability, buddy.

Let's begin with the Java service side. In a nutshell, we're going to use Maven to manage our dependencies (naturally), Spring to wire our components (of course), JAXB to define our XML (what else?), CXF as our REST service and Jetty as our web server. The easy way to start (besides downloading the finished product) is use a Maven archetype to generate a basic WAR.
mvn archetype:generate
Choose "maven-archetype-webapp" (mine was #18), make up a groupId, artifactId and version, and "packaging: war". Next, replace the generated pom with this one:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.coderoshi</groupId>
<artifactId>roj</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>war</packaging>

<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>maven-jetty-plugin</artifactId>
<version>6.1.11</version>
<configuration>
<contextPath>/</contextPath>
</configuration>
</plugin>
</plugins>
</build>

<dependencies>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-jaxrs</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http-jetty</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-bindings-http</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>2.5.5</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>2.5.5</version>
</dependency>
</dependencies>
</project>

This pom does two important things: gives us the spring and CXF dependencies that we need, and configures the plugins that we require. Namely, that we assume JDK 1.6, and that we want to use the Jetty plugin to run our war (eventually).

Next we'll set up our Spring's context file. This describes to spring how much we love CXF and really want to use it. Spring replies by constructing our components correctly for us. Isn't that nice? Create a new file under "src/main/webapp/WEB-INF/" named "beans.xml", and paste the following:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jaxrs="http://cxf.apache.org/jaxrs"
xmlns:cxf="http://cxf.apache.org/core"
xsi:schemaLocation="
http://cxf.apache.org/core http://cxf.apache.org/schemas/core.xsd
http://cxf.apache.org/jaxrs http://cxf.apache.org/schemas/jaxrs.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/cxf/cxf-extension-jaxrs-binding.xml" />
<import resource="classpath:META-INF/cxf/cxf-servlet.xml" />

<jaxrs:server id="campaignREST" address="/">
<jaxrs:serviceBeans>
<bean class="com.coderoshi.service.CampaignService" />
</jaxrs:serviceBeans>
<jaxrs:entityProviders>
<bean class="com.coderoshi.service.JAXBCollectionProvider" />
</jaxrs:entityProviders>
<jaxrs:features><cxf:logging /></jaxrs:features>
</jaxrs:server>
</beans>
Cryptic? Well, no, not if you already know Spring. If not, here's the skinny: We are importing some built-in CXF Spring configurations (thanks CXF!) and using them to define our own REST server (we haven't written it yet) named "campaignREST". The "jaxrs:serviceBeans" element contains a list of services we define, in this case, one we are about to write called "com.coderoshi.service.CampaignService". The block between "jaxrs:features" turns on logging, so we can view incomming/outgoing messages and errors. Finally, the "jaxrs:entityProviders" contains a bean we are about to write to convert a JAXB Collection into an XML form consumable by Rails.

Finally, we need to tell the web server (Jetty) that it needs to load up Spring and also the CXF servlet (which handles requests and passes them off to our REST service).
<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>WEB-INF/beans.xml</param-value>
</context-param>

<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>

<servlet>
<servlet-name>CXFServlet</servlet-name>
<servlet-class>org.apache.cxf.transport.servlet.CXFServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>CXFServlet</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
</web-app>
Whew! With configuration out of the way, let's get to the fun part... coding!

First, we'll start with the JAXB bean we eventually want to serialize. Just to keep it simple, we'll make public fields and forgo getters and setters. We're using Maven to build, so you'll need to create the package under src/main/java (where "java" is a sibling of the "webapp" directory.
package com.coderoshi.service;

@javax.xml.bind.annotation.XmlRootElement( name = "campaign" )
public class Campaign
{
public long id;
public String name;
public long budget;

public Campaign() {}
public Campaign( long i, String n, long b ) { id=i; name=n; budget=b; }
}
Ah, the magic of JAXB! That's all you need to create an XML-marshallable object. Eat it, Rails (and you're about to...). Now we get to make our REST server. Create a new class named "com.coderoshi.service.CampaignService". Next we need to annotate the class with two annotations:
@javax.ws.rs.Path( "/campaignService" )
@javax.ws.rs.ProduceMime( "application/xml" )
These tell the server that this is a service available at the HTTP path "/campaignService", and that by default, calling this service will return XML (it sets the HTTP header's mime-type on response).

Let's write our standard CRUD operations (which Rails' ActiveResource will presume exist by default).

It's important to note that we're faking data and operations. If you downloaded the zip provided, then you'll see I made the service database-backed (HSQLDB), but that's just for fun, and really is beyond the code here.

Create (addCampaign)
@Path( "/campaignService" )
@ProduceMime( "application/xml" )
public class CampaignService
{
// ... define actions ...
@POST
@Path("/campaigns.xml")
@ConsumeMime( "application/xml" )
public Response addCampaign( Campaign c )
{
System.out.println( "Adding Campaign: " + c.name );
c.id = 99;
// Adding header Location=/{id} is required so Rails can extract an id
return Response.status( 200 ).header( "Location", "/" + c.id ).build();
}
Rails presumes all adds are "POST" operations. It also assumes the path is the plural name of the resource ("campaigns") and post-fixed by the produced mime-type. In our case XML. On a successful update ActiveResource will assume a 200 (successful) response. We can play with error handling at a later time. Since we are expecting data is in XML form, we explicitly enforce it to be so with @ConsumeMime. Note that all of these method annotations are in the JSR 311 as the "javax.ws.rs.*" package.

Read (getCampaign, getCampaigns)
  @GET
@Path("/campaigns.xml")
public List<Campaign> getCampaigns()
{
return Arrays.asList( new Campaign(1, "campaign #1", 500), new Campaign(2, "campaign #2", 600) );
}

@GET
@Path("/campaigns/{id}.xml")
public Campaign getCampaign( @PathParam( "id" ) long id )
{
return new Campaign(id, "campaign #"+id, 900);
}
In Rails REST reads are GET operations. It also presumes the pathnames begin with "campaigns", but notice this time the singular "getCampaign" has a param called "id". In CXF/JSR311 you can define that you expect a path to contain a variable - in this case, we expect an ID. However, since Java doesn't keep track of parameter names after compilation, the best thing you can do is name the parameter you want CXF to populate for you. Most basic types are supported, like String or int - in our case long. So when someone "GETs" our url "http://localhost:8080/campaignService/campaigns/35.xml", the "getCampaign" method is called and the value "35" is passed in as parameter "id".

Also notice that our getCampaigns path is the same as addCampaign. How can this be? Because although the URLs are the same, different operations take place on them (POST versus GET). This is how CXF differentiates between the operations and knows when to expect XML that conforms to our Campaign JAXB, or no arguments at all.

Update (updateCampaign)
  @PUT
@Path("/campaigns/{id}.xml")
@ConsumeMime( "application/xml" )
public Response updateCampaign( @PathParam( "id" ) long id, Campaign c )
{
System.out.println( "Updating Campaign: " + id );
return Response.status( 200 ).build();
}
Here we PUT Campaign XML to the server, along with the ID. Although technically we shouldn't require the ID (since it should already be in the Campaign data), Rails builds URLs that way, so we need to also. Don't ask me why - I don't know.

Delete (deleteCampaign)
  @DELETE
@Path("/campaigns/{id}.xml")
public Response deleteCampaign( @PathParam( "id" ) long id )
{
System.out.println( "Deleting Campaign: " + id );
return Response.status( 200 ).build();
}
}
Finally we handle delete. To delete an object you merely need to pass in the ID as a DELETE operation. Here we don't really do anything, just print to the console so you know it worked.

Yeah yeah, I know REST is not necessarily CRUD, but for our purposes the mapping is close enough, and we're going to run with it.

Yay! We wrote our Java REST server - but we have a problem. Our "getCampaigns" method returns a "List" (list of JAXB Campaigns) - but CXF doesn't know how to handle a list like this. And it certainly wouldn't handle a JAXB list in a way friendly to ActiveResource, which has it's own expectations.

Luckily for us JSR-311/CXF allows you to create your own custom providers. Providers are responsible for translating to and from one kind of data (an JAXB object, for example) from and to another (an XML stream). The one we're about to write has already been defined in the Spring context above under "jaxrs:entityProvider". Without too much ado, here it is in it's entirety:
package com.coderoshi.service;

import java.io.*;
import java.util.*;
import javax.ws.rs.*;
import javax.ws.rs.core.*;
import javax.ws.rs.ext.*;
import javax.xml.bind.*;

import org.apache.cxf.binding.http.strategy.EnglishInflector;

@Provider
@ProduceMime( "application/xml" )
public class JAXBCollectionProvider
implements MessageBodyWriter<Collection<?>>
{
public boolean isWriteable( Class<?> type )
{
return Collection.class.isAssignableFrom( type );
}

public long getSize( Collection<?> l )
{
return l == null ? 0 : l.size();
}

public void writeTo( Collection<?> li, MediaType mt, MultivaluedMap<String, Object> headers, OutputStream os )
throws IOException
{
try
{
if( li == null || li.isEmpty() )
{
os.write( "<nil type=\"array\" />".getBytes() );
return;
}

Marshaller marshaller = null;
String envelopeName = null;
for ( Object object : li )
{
if ( object != null )
{
if ( marshaller == null )
{
JAXBContext context = JAXBContext.newInstance( object.getClass() );
marshaller = context.createMarshaller();
marshaller.setProperty( Marshaller.JAXB_FRAGMENT, true );

envelopeName = context.createJAXBIntrospector().getElementName( object ).getLocalPart();

// pluralize the collection
envelopeName = new EnglishInflector().pluralize( envelopeName );

os.write( "<".getBytes() );
os.write( envelopeName.getBytes() );
os.write( " type=\"array\">".getBytes() );
}
marshaller.marshal( object, os );
}
}
if ( envelopeName != null )
{
os.write( "</".getBytes() );
os.write( envelopeName.getBytes() );
os.write( ">".getBytes() );
}
}
catch ( JAXBException e )
{
throw new IOException( "broke", e );
}
}
}
Whew! That's some chunk of code, where to begin? Let's start at the top.

Notice that we annotate our class with
@Provider
@ProduceMime( "application/xml" )
These tell the framework that this is an available provider implementation and that it produces XML. We implement "MessageBodyWriter", which contains hook methods for the REST container. "isWriteable" checks if outbound object classes can utilize this Provider. In our case, we can use anything that is a Collection (which, of course, our List<Campaign> most certainly is).

The "writeTo" method does the actual work of marshaling any approved Collection object directly into the output stream. Here is where things get weird. First we ensure that the collection contains objects. If not, we just output "<nil type=\"array\" />". This is a flag to Rails' ActiveResource that we would like to return an array, but we don't have any data to give it. We call the element "nil" since we don't have any objects, we can't exactly know what types they are.

Assuming our collection contains value, we first wrap the collection within a pluralized version of the elements we envelope. For example, since our List contains two Campaign JAXB objects named "campaign", we wrap them in a root element named "campaigns". How can we know how to correctly pluralize words? We use a built-in CXF class called "EnglishInflector". We then define the type as "array" for the benefit of the ActiveResource consumer.

That's it! If it seems like a lot, don't worry. When it's all together and chugging along, it's amazing how little code there really is (considering we're dealing with Java here).

Running the REST Service

Assuming no typos or environment problems, you can now run the server using the Maven Jetty plugin. Don't worry about building it, the plugin will ensure we are built before launching. So go to the project base dir, and type:
mvn clean jetty:run-war
Navigate your browser to "http://localhost:8080/campaignService/campaigns.xml", and you should be treated to a list of your campaigns:
<campaigns type="array">
<campaign>
<id>1</id>
<name>campaign #1</name>
<budget>500</budget>
</campaign>
<campaign>
<id>2</id>
<name>campaign #2</name>
<budget>600</budget>
</campaign>
</campaigns>


Ruby on Rails

Now on to the Ruby on Rails frontend, which gladly, is simpler. Would you expect less from Rails? Like the Java backend, we can build the project from scratch, or you can download the completed project here.
rails ror
cd ror
Next, create the model file "app/models/campaign.rb" with the contents (assuming your server is at port 8080:
class Campaign < ActiveResource::Base
self.site = "http://localhost:8080/campaignService"
end
That's it for the ActiveResource. Rails presumes the rest (REST, get it? ha ha). It points the ActiveResource to the given site as a base URL, and from there constructs URLs based upon a pluralized version of the classname. For each of the following ActiveResource actions:
Campaign.find(:all)        # GET http://localhost:8080/campaignService/campaigns.xml
Campaign.find(7) # GET http://localhost:8080/campaignService/campaigns/7.xml
Campaign.create(data).save # POST http://localhost:8080/campaignService/campaigns.xml
Campaign.find(7).save # PUT http://localhost:8080/campaignService/campaigns/7.xml
Campaign.delete(7) # DELETE http://localhost:8080/campaignService/campaigns.xml
This covers it for the ActiveResource, but we need something to show the user. Luckily Rails makes this simple with scaffold generation.
./script/generate scaffold Campaign name:string budget:integer
This generates a controller, resources in the routing table, test cases, and attempt to generate a model in the form of an activerecord object. But since we already created our campain ActiveResource there, it just skips it.

Next, we need to set up the server. Since we don't actually need Rails to run on a database anymore, we can configure the server to skip it. Open up your "config/environment.rb" file, and within "RailsInitializer" add the line:
  config.frameworks -= [ :active_record ]
To stop active_record from loading (and thus any database requirements).

Now, start up the rails server:
./script/server
Did that work? Probably not. Unless you are running edge Rails, there is a current bug (gasp!) that forces you to comment out the lines from "new_rails_defaults.rb" (4th line in the server startup stack trace):
# ActiveRecord::Base.include_root_in_json = true
# ActiveRecord::Base.store_full_sti_class = true
Try and start the server again. It should work this time. If you visit "http://localhost:3000/campaign", it will probably work. This is just a coincidence.

Next make a few tweaks to your new rails CampaignController. This is an issue with Rails scaffold
generation. It generated a Controller for an ActiveRecord, not an ActiveResource, so you
need to modify for using ActiveResource objects.

Under "def update", replace the AR.update_attributes method with the ARes.load and save methods. Like so:
      if @campaign.update_attributes(params[:campaign])
with
      @campaign.load(params[:campaign])
if @campaign.save
and replace
    @campaign = Campaign.find(params[:id])
@campaign.destroy
with
    Campaign.delete(params[:id])
Better yet, let's just rip out all the stuff we don't need. Here's the complete Controller:
class CampaignsController < ApplicationController
# GET /campaigns
def index
@campaigns = Campaign.find(:all)
end

# GET /campaigns/1
def show
@campaign = Campaign.find(params[:id])
end

# GET /campaigns/new
def new
@campaign = Campaign.new()
end

# GET /campaigns/1/edit
def edit
@campaign = Campaign.find(params[:id])
end

# POST /campaigns
def create
@campaign = Campaign.new(params[:campaign])
if @campaign.save
flash[:notice] = 'Campaign was successfully created.'
redirect_to(@campaign)
else
render :action => "new"
end
end

# PUT /campaigns/1
def update
@campaign = Campaign.find(params[:id])
@campaign.load(params[:campaign])
if @campaign.save
flash[:notice] = 'Campaign was successfully updated.'
redirect_to(@campaign)
else
render :action => "edit"
end
end

# DELETE /campaigns/1
def destroy
Campaign.delete(params[:id])
redirect_to(campaigns_url)
end
end
You'll also need to make a slight change to the generated "new.html.erb" file. Again, this is because the scaffold generator we used presumes it's dealing with an ActiveRecord object, and we provide an ActiveResource. (more info if you care: The "form_for(@campaign)" line generated, given an ARec object, will generate a hidden "_method=put" field in the form. This tells the controller that you are going to update an object - which isn't true - you are creating one. So you really want to POST, not PUT)

So, as long as I'm giving away free answers, put this in place of the new.html.rb form generated (don't forget your authenticity token):
<form action="/campaigns/" method="post">
<%= hidden_field_tag "authenticity_token", form_authenticity_token %>

<p>
<label for="campaign_name">Name</label><br />
<%= text_field_tag "campaign[name]", "" %>
</p>
<p>
<label for="campaign_budget">Budget</label><br />
<%= text_field_tag "campaign[budget]", 0 %>
</p>
<p>
<%= submit_tag "Create" %>
</p>
</form>
Navigate to "http://localhost:3000/campaigns" and check out your list of two campaigns. click "New campaign" to add one. Fill out the form and "Create". This then forwards you to "show" the newly created campaign, which is retrieved from the Java's REST service. Click "back" to return to the list of campaigns, and you should see your new one sitting there. Click "Edit" to make a change to the campaign, and "Destroy" to delete it. These actions don't actually do anything since we hard-coded our REST service.

However, if you downloaded the sample projects and run them, then the REST service is backed by an HSQLDB - so adding/editing/removing values actually works.

That's it! This convergence of technologies is precisely what ActiveResource was created to deal with. Simple REST serviced is why JSR311 (aka, JAX-RS) exists for. Simplicity of domains - what more could you want?

RoRoRoJ? Ro3J?

5 comments:

artied said...

interesting post and i would like to read more but am getting this error when i try to subscribe

"feed/http://feeds.feedburner.com/coderoshi/rss.xml" has no items.

my error or yours??

Eric said...

I believe yours. The correct URL for the feed is: http://feeds.feedburner.com/coderoshi/ (without the rss.xml)

Anonymous said...

Hello Eric,

I have been all over the Internet trying to find you. You made the iBall app, correct? Please track me down, I have an interesting proposition to offer you. Webb Nelson, 800-678-8697, webbn@playvisions.com . Thanks.

Mikael said...

Great article. I'm trying to expand the example to work with complex value objects. Imagine a Campaign that contains a List of Event objects for example. How best customize the xml in a situation like this? I've managed to get close by using JAXB annotations, but the "type=array" part is missing, which Rails doesn't like. It would be nice if the CXF provider could handle collections inside the value objects as well, but apparently it isn't applied recursively. Any ideas?

Eric Redmond said...

@Mikael

I'd alter the JAXBCollectionProvider to be more general, like a JAXBModelProvider - which can reflect an object, and generate the annotation you desire for collections, but let leaf objects (any not containing a collection) to just pass to other providers.

Just a thought. I'd be interested to know if you get it to work.