Isomorphic Web Design with React and Scala

Isomorphic Web Design with React and Scala

Isomorphic web design is an approach to web development that is gathering momentum. The concept of Isomorphic web design is simple, dynamically generate HTML using either server or client rendering based on which approach yields the best experience for our audience. Server rendered content is very fast if built correctly; however, client rendered Single Page Applications (SPA's) have shown that they can offer faster page transitions leading to a more engaging user experience. Users typically equate the experience of using a SPA with mobile apps or desktop applications. Popular web-based e-mail clients like Gmail and Outlook 365 are textbook examples of well written SPA's. Users of these systems can fluidly navigate between reading, composing, searching, and managing e-mail without the jarring page flashes and loading delays associated with a typical server rendered web application.

Isomorphic web design is a great architecture option on its own, but it's recent growth is driven by the success of SPAs and their two primary disadvantages, Search Engine Optimization (SEO) and initial page load performance. These two challenges are the principal reason many SPA developers are looking to Isomorphic web design for solutions. Today we'll discuss how React can solve these challenges. We'll also review a simple Scala website sample that demonstrates how React can be integrated into the server to accomplish true Isomorphic web design. The code presnted in this article is posted on github if you want to follow along.

Single Page Application SEO Challenges

SPAs face a difficult challenge when it comes to SEO. Until recently, Google's search index crawler would not execute JavaScript when analyzing web pages. As of June 2015, Bing (and Yahoo) have not publicly indicated similar support for JavaScript. Although Google changed their JavaScript limitation, it's still unclear how much JavaScript the Google crawler executes and how effective or accurate the crawler is in determining the true intent of the JavaScript generated content. Any website that carefully manages its SEO would have reservations about using a SPA due to the risk that Google might incorrectly score their content and adversely impact their domain or page rank. The best practice for an SEO dependent site is to follow traditional web design using server generated HTML, allowing businesses to carefully control their SEO content while ensuring their intent is properly understood by Google. Websites that want to utilize SPAs to provide an engaging user experience without impacting their SEO are left in a difficult situation. Some enterprising developers have developed solutions using PhantomJS based on a proposal by Google but this solution is far from ideal as well as being difficult to setup and maintain.

Single Page Application SEO Performance Challenges

The second challenge faced by SPAs is how to provide an optimized experience during the initial page load. Because SPAs utilize JavaScript to render on the client, they must load and execute all the included script files, possibly make a second call for data, and then render the results. All of these steps must be completed before a user can be shown content. SPA developers are familiar with the dreaded FOUC (flash of un-styled content) that can occur while a SPA page is being loaded and scripts have yet to execute. A second concern with page speed is also related to a Google decision. In 2014, Google made headlines when they announced that page load performance would be a factor in their algorithm to determine page rank. This policy change meant that quickly loading websites would be preferred to websites that load more slowly. Today, Google provides a handy page speed test for designers to analyze performance as seen by the Google crawler.

Unfortunately SPAs are heavily penalized in this new scoring algorithm. A traditional website renders on a server and returns HTML suitable for indexing by Google. A SPA works entirely differently and must first load and execute a significant amount of JavaScript in the user's browser, make a second request to the server for data to render, and use client processing power and resources to produce HTML. Google's crawler counts all this extra work and delay against the page load time and scores SPAs lower than traditional websites. For comparison, a SPA might take 800 ms or more to render, compared to a traditional web server that might render the same content in 200 ms or less. Obviously, once the SPA is loaded, it will be much faster than a traditional server rendered website when navigating to other pages, but Google only considers the initial page load performance, not the user's observed performance when browsing multiple pages.

Isomorphic Solution for SEO and Page Speed

SPA developers understand that a new approach is necessary to utilize a SPA architecture while maintaining SEO and initial page load performance. Isomorphic web development is a popular solution that many developers are embracing. In an Isomorphic web app, the developer writes a SPA that renders HTML on the client, but is also capable of executing on the server to render content for the initial page load. Most isomorphic web applications accomplish this by using a shared code base between the server and client. Usually a server-side JavaScript engine like Node.js is used to pre-render the page for the client. In an Isomorphic website, server rendered HTML is returned to the client on the first page request. The HTML contains special markup that allows an asynchronously loaded client-side SPA to attach and handle future rendering on the client. As the user changes views or pages, the client-side SPA completely takes over navigation and rendering, only using the server for API requests as needed. In our sample today, we're going to use React for both client and server rendering. The following illustration demonstrates how our Isomorphic React application compares to a traditional server rendered application and a SPA like AngularJS or Ember.js.

Our Sample Project

In this article we're going to use React, Node, and Scala to demonstrate an Isomorphic application. All three technologies will be used on the server, and we'll use React and react-router on the client for client-side rendering and routing. We decided to include Scala on the server because we want to show how node can be integrated for rendering content in an existing solution, even if it's in a different language. We'll also be demonstrating a Scala JSON API that could be easily imagined as the back-end of an existing SPA application. We could just as easily have built the entire server solution in Node.js or many other server stacks.

Scala JSON WEB API

For our server we decided to use Scalatra, a micro-framework for Scala inspired by Ruby's Sinatra framework. There are several other great Scala framework options like the MVC-based Play Framework, but for our solution we wanted something small and lightweight that would be familiar to other developers coming from node. Scalatra's routing system feels very similar to node. We also wanted the Scala application to utilize the same asynchronous, event-based programming style that is one of node's main strengths. Scalatra's asynchronous servlets provide similar functionality using Futures and AsyncResult.

Scalatra Getting Started

Getting started with Scalatra is very simple using this guide. You only need a JDK and a few steps to be up and running quickly. After getting installed, you can follow a second guide to create your first project. Scalatra uses SBT (simple build tool) which provides dependency management, builds, and automatic code re-loading. You'll also want to read the instructions on setting up the ActorSystem, which manages concurrency for our asynchronous actors. The actor we create will be responsible for making asynchronous calls to node and rendering using React.

Building a Website with Scalatra

After getting your template project up and running, make sure you have the following dependencies listed in your build.scala file:

libraryDependencies ++= Seq(
        "org.scalatra" %% "scalatra" % ScalatraVersion,
        "org.scalatra" %% "scalatra-scalate" % ScalatraVersion,
        "org.scalatra" %% "scalatra-specs2" % ScalatraVersion % "test",
        "org.scala-lang" %  "scala-reflect"  % scalaVersion.value,
        "org.eclipse.jetty" % "jetty-webapp" % "9.1.5.v20140505" % "container",
        "org.eclipse.jetty" % "jetty-plus" % "9.1.5.v20140505" % "container",
        "javax.servlet" % "javax.servlet-api" % "3.1.0",
        "org.scalikejdbc" %% "scalikejdbc" % "2.2.5",
        "org.scalikejdbc" %% "scalikejdbc-interpolation" % "2.2.5",
        "org.postgresql" % "postgresql" % "9.4-1200-jdbc41",
        "org.json4s" %% "json4s-jackson" % "3.2.11",
        "org.json4s" %% "json4s-native" % "3.2.11",
        "org.scalatra" %% "scalatra-json" % ScalatraVersion,
        "net.databinder.dispatch" %% "dispatch-core" % "0.11.2",
        "com.typesafe.akka" %% "akka-actor" % "2.3.4",
        "org.scala-lang.modules" %% "scala-xml" % "1.0.3"
      )

also check that your plugins.sbt file contains the following dependencies.

addSbtPlugin("com.mojolly.scalate" % "xsbt-scalate-generator" % "0.5.0")
addSbtPlugin("org.scalatra.sbt" % "scalatra-sbt" % "0.3.5")
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.7.4")
addSbtPlugin("org.scalastyle" %% "scalastyle-sbt-plugin" % "0.6.0")
resolvers += "sonatype-releases" at "https://oss.sonatype.org/content/repositories/releases/"

Next we'll setup the ActorSystem in our ScalatraBootstrap.scala file below.

//other imports...
import _root_.akka.actor.{ActorSystem, Props}
import org.scalatra._

class ScalatraBootstrap extends LifeCycle {
  // Get a handle to an ActorSystem and a reference to one of your actors
  val system = ActorSystem()

  override def init(context: ServletContext) {
    context.mount(new DemoServlet(system), "/*")
 }

  // Make sure you shut down
  override def destroy(context:ServletContext) {
    system.shutdown()
  }
}

Now we'll move to the main code created by the giter8 template project. This file will be located in a subdirectory of /src/main/scala and will end with "Servlet.scala". We selected the name "Demo", so our file is DemoServlet.scala.

First, make sure you have these imports:

import org.scalatra._
import scalate.ScalateSupport
import org.json4s._
import org.json4s.jackson.JsonMethods._
import org.slf4j.{Logger, LoggerFactory}
import scalikejdbc._
import org.json4s.{DefaultFormats, Formats}
import org.scalatra.json._
import org.json4s.native.Serialization.{ read, write, writePretty }

import dispatch._, Defaults._
import scala.concurrent.{ExecutionContext, Future, Promise}
import _root_.akka.actor.{Actor, ActorRef, ActorSystem, Props}
import _root_.akka.dispatch._
import org.scalatra.{ScalatraServlet, FutureSupport, AsyncResult}

Next we're going to setup a case class that represents our data structure we'll use in rendering. For our data we're going to continue our work with a datasource we created based on the World Cup 2014. Our case class:

case class CupResult(goals_against: Int, _id: String, goals_for: Int, _rev: String, points: Int, team: String,
 goal_diff: Int, draw: Int, loss: Int, games: Int, win: Int)

With these items in place, we can start building our sample solution. We selected the name DemoServlet, so you'll need to replace with what you selected in the template step.

class DemoServlet(system:ActorSystem) extends DemoStack with JacksonJsonSupport with FutureSupport {
  protected implicit def executor: ExecutionContext = system.dispatcher
  protected implicit val jsonFormats: Formats = DefaultFormats

  //setup the logger
  val logger =  LoggerFactory.getLogger(getClass)

  //create the postgres connection pool
  ConnectionPool.singleton("jdbc:postgresql://localhost:5432/worldcup", "webdemo", "test")

  //return list of all CupResults in the database
  def teams():List[CupResult] = {
    val results: List[CupResult] = DB readOnly { implicit session =>
      sql"select * from result"
        .map(rs => CupResult(rs.int("goals_against"), rs.string("id"), rs.int("goals_for"),
          rs.string("rev"), rs.int("points"), rs.string("team"), rs.int("goal_diff"), rs.int("draw"),
          rs.int("loss"), rs.int("games"), rs.int("win") )).list.apply()
    }
    results
  }

  case class RenderRequest(view: String, url: String, content: List[CupResult])

  get("/standings"){
    contentType = "text/html"
    new AsyncResult{
      val is =  Future {
        //build our node request
        val render = RenderRequest("team-list", "/standings", teams)

        //build json string of our RenderRequest
        val json = write(render)

        //dispatch the rendering async to Akka and node
        DispatchAkka.renderPage(json)
      }
    }
  }

 import _root_.akka.pattern.ask

Stepping through this code, we first establish our logger and postgres connection pool. After that, we create a method called teams() that returns a List of the case class CupResult we mentioned earlier. Next we create a new case class RenderRequest. We'll use a JSON serializer to convert the RenderRequest class to POST data we send to node. RenderRequest contains the data we want to show, as well as the desired template name and the current url for routing. Finally, we get to our Scalatra GET route get("/standings"). This route asynchronously calls our node server via DispatchAkka and returns the HTML rendered by React. There are other ways to interact with React from Scala, such as creating node as a process or using a JavaScript engine for the JVM, but both have drawbacks in either data size limitations, poor JavaScript performance, or limited ECMAScript support.

Because we want our server to scale we decided to call node asynchronously. In Scala, we use a Future that becomes available after node returns our results. This approach to concurrency allows non-blocking, callback-based asynchronous programming for Scalatra, just like node. The call to DispatchAkka.renderPage returns the result of our call to node. With this result we can now return the final HTML to the client.

Server Rendering in Node.js with React

Many SPA developers are already familiar with node. In the code below we are using the koa web framework along with Node v0.12 features like generators and the new asynchronous yield keyword. If you haven't used these new ECMAScript 6 features, the code might be a little different than you expect.

The code below creates a new POST endpoint called render. Any data passed to render is sent to the view requested by the client along with the data context built in the Scala code. React renders the complete HTML and we return that from our method call.

function AttachReact(router){
  return new Promise((resolve,reject) => {
    try{
      //how does error surface?
      router.run( (Handler, state) => {
        const body = React.renderToString(React.createElement(Handler, null));
        resolve(body);
      });
    } catch(e){
      reject(e);
    }
  });
}

app.use(router(app))
.get('/test', function *(){
  //provide easy route for testing to make sure node is listening
  this.body = 'node running...';
})
.post('/', function *(){
  //the endpoint called from scala
  const data = this.request.body.content;
  const view = this.request.body.view;
  const url = this.request.body.url;

  const router = Router.create({
    location: url,
    routes: AppRoutes.getRoutes(data)
  });

  //const that = this;
  try {
    const body = yield AttachReact(router);
    yield this.render(view, {
      body: body,
      cacheBuster: uuid.v4(),
      teams: JSON.stringify(data)
    });
  } catch (e) {
    console.log('router.run exception', e);
  }
});

Isomorphic React Components

Our node API method renders a template supplied by the view property of the JSON POST request. The code also passes the content JSON property to React as state for the renderToString React method. React's renderToString method is responsible for generating HTML using a class we created, Table. Table is a React custom component we authored that is composed with other React classes to render our content. Here are the contents of the Table.jsx file:

const React = require('react/addons');
const _ = require('lodash');
const Row = require('./row');

const Table = React.createClass({
  mixins: [React.addons.PureRenderMixin],

  propTypes: {
    initialTeams: React.PropTypes.arrayOf(React.PropTypes.object).isRequired
  },

  getInitialState: function() {
    return {
      teams: this.props.initialTeams.map( team => {
        team.goalsPerGame = this.goalsPerGame(team.games, team.goals_for);
        return team;
      })
    };
  },

  columns: [
    'goals_against',
    'goals_for',
    'points',
    'team',
    'goal_diff',
    'draw',
    'loss',
    'games',
    'win',
    'gpg'
  ],
  calcTotalGames: function() {
    return _.reduce(this.state.teams, function(total, team){
                      if( total ){
                        return parseInt(total) + parseInt(team.games);
                      } else {
                        return parseInt(team.games);
                      } });
  },

  goalsPerGame: function(games, goals) {
    return goals / games;
  },

  handleAllClick: function() {
    const teamsClone = _.cloneDeep(this.state.teams);
    teamsClone.map(team => {
      team.games++;
      team.goalsPerGame = this.goalsPerGame(team.games, team.goals_for);
      return team;
    });
    this.setState({teams: teamsClone});
  },

  render: function() {
    const sortedTeams = _.sortByOrder(this.state.teams, 'goalsPerGame', [false]);
    return (
      <div>
        <button onClick={this.handleAllClick}>All</button>
        <Link to={`/about`}>About</Link>

        <div>{this.calcTotalGames()}</div>
        <table>
          <thead>
            <tr>
              <th>rank</th>
              {this.columns.map(name =>
                <th key={'th-' + name}>{name}</th>
              )}
            </tr>
          </thead>
          <tbody>
            {sortedTeams.map((team, outerIndex) =>
              <Row key={'row-' + outerIndex}
                data={sortedTeams[outerIndex]}
                index={outerIndex}
                columns={this.columns} />
            )}
          </tbody>
        </table>
      </div>
    );
  }
});

module.exports = Table;
if (typeof window !== 'undefined') {
  window.Table = Table;
}

In the above code, Table sets it initial state to the team list passed in from node's call to React.renderToString. In an Isomorphic web application, we use a shared code base to render on the client as well. When Table is used by React in server rendering, we pass the state via the call to React.renderToString, but if Table is used in the client, we pass the data in slightly differently, which we'll show in the HTML section later. How we pass the initial state to our Table class doesn't matter. React will just render whatever was given to it by the caller. This shared code base approach is very convenient for server and client rendering.

Earlier we mentioned that Table uses composition to render with another small React class, Row. Let's look at row.jsx:

const React = require('react/addons');

const Row = React.createClass({
  mixins: [React.addons.PureRenderMixin],

  propTypes: {
    data: React.PropTypes.object.isRequired,
    columns: React.PropTypes.arrayOf(React.PropTypes.string).isRequired,
    index: React.PropTypes.object.isRequired
  },

  render() {
    return (
      <tr>
        <td className={this.props.index % 2 === 0 ? '' : 'odd'}>{this.props.index + 1}</td>
        <td className={this.props.data.goals_against > 6 ? 'red' : ''}>{this.props.data.goals_against}</td>
        <td className={this.props.data.goals_for > 10 ? 'green' : ''}>{this.props.data.goals_for}</td>
        <td className={this.props.data.goals_points > 15 ? 'blue' : ''}>{this.props.data.points}</td>
        <td className={this.props.index % 2 === 0 ? '' : 'odd'}>{this.props.data.team}</td>
        <td className={this.props.data.goal_diff < 0 ? 'red' : ''}>{this.props.data.goal_diff}</td>
        <td className={this.props.data.draw > 2 ? 'blue' : ''}>{this.props.data.draw}</td>
        <td className={this.props.data.loss > 3 ? 'red' : ''}>{this.props.data.loss}</td>
        <td className={this.props.data.games > 10 ? 'blue' : ''}>{this.props.data.games}</td>
        <td className={this.props.data.win > 3 ? 'green' : ''}>{this.props.data.win}</td>
        <td>{this.props.data.goalsPerGame}</td>
      </tr>
    );
  }
});

module.exports = Row;
if (typeof window !== 'undefined') {
  window.Row = Row;
}

In the code above, the Row component simply renders a single row from our team data, and Table maps the teams to Row for rendering each one. Row is a much simpler React component than Table because it only implements the render method. React applications typically use the composition pattern like this to build their view from reusable components.

Isomorphic Web Design with HTML5

A great way to improve page speed is to load JavaScript asynchronously. HTML 5 added the async keyword for the script tag just for this purpose. We'll make use of this tag in the HTML below to improve our page speed performance. Our node API method renders a view using React.renderToString; however, in the client we use React.createElement along with React.render instead. Consider the following template HTML we use in our application:

<!DOCTYPE html>
<html>
<head>
  <title>Scala-NodeJS</title>
  <link rel="stylesheet" type="text/css" href="css/highlight.css">
</head>
<body>
  <div id="app">
  <%= body %>
  <div>

  <script id="team-content" type="application/json"><%= teams %></script>
  <script type="text/javascript">
    function init(){
      var elem = document.getElementById("app");
      var data = JSON.parse(document.getElementById("team-content").innerHTML);

      Router.attach(elem, data);
    }
  </script>
  <script async src="js/bundle.js" onload="init()" type="text/javascript"></script>
</body>
</html>

On the server-side, node render's this template by replacing <%= body %> with the React rendered component. On the client the <div id="app"> element is already populated from the server-rendered content, and React simply finds the rendered HTML and re-attaches to manage the DOM. Node also replaces the contents of our script tag <%= teams %> with a copy of the JSON model data from the server. We send this copy of the data in the initial request so that we don't have to make an additional XHR request to load the same data later. After React loads, it will create an observer on this data and re-render DOM elements in the <div> if the model data is changed, just like a typical SPA.

Also note that React, react-router, our views, components, and dependent libraries are all loaded asnychronously via the script <script src="js/bundle.js"></script>. We build the bundle.js using webpack and gulp. After the client loads and executes this script, React starts client-side rendering to continue DOM management. lodash and our webpack JS file should be placed in a subdirectory of webapp called js. Scalatra will look for public static files under webapp. We accomplish all of this via gulp tasks.

Client DOM manipulation

In our React code you probably noticed several calculated fields including calcTotalGames and goalsPerGame. We added these fields to the Table component to demonstrate that they can be server rendered and continue to be maintained as React renders on the client. In Table.jsx we added a handleAllClick method and wired it up to a button onClick event. If the button is pressed in the client, React will call handleAllClick which alters the model, causing React's virtual DOM to change and React to re-render the row. This button press changes the underlying state which causes React's observable pattern to notice a data difference. React intelligently finds only the row or rows that have changed and re-renders those elements to synchronize the HTML DOM with our client state.

Routing with React

At this point our Isomorphic web application will deliver a SPA while ensuring SEO performance for the initial page speed and content. Now we'll show how other routes and views on the website can be requested directly from the server. We'll also show how navigating to other views or pages will still benefit from the client rendering advantages of a SPA. A quick note, we're using v0.13.x of React router, but future plans for react-router v1.0 may have significant changes. Let's discuss the two routes, /standings and /about.

In an Isomorphic website, we want server requests and client routes to either URL to work and render the same HTML. We also want a user to seamlessly transition to client routing and rendering after making a request for a server rendered page. We'll setup this scenario by adding a new route to our Scalatra app, /about.

  get("/about"){
    contentType = "text/html"
    new AsyncResult{
      val is =  Future {
        //build our node request
        val render = RenderRequest("team-list", "/about", teams)

        //build json string of our RenderRequest
        val json = write(render)

        //dispatch the rendering async to Akka and node
        DispatchAkka.renderPage(json)
      }
    }
  }

This endpoint will asynchronously request node to render our React SPA from the path /about. Previously we were calling React from the url /standings. Our node code correctly identifies the path difference and uses a different React component to render the view, resulting in completely different HTML than the /standings route we used earlier. React knows which component to use because of the wire up in our router.jsx file.

  ...
  getRoutes(data){
    const routes = (
      <Route hander={App}>
        <DefaultRoute handler={this.wrapComponent(Table, {initialTeams: data})}/>
        <NotFoundRoute handler={NotFound}/>
        <Route path="standings" handler={this.wrapComponent(Table, {initialTeams: data})}/>
        <Route path="node" handler={this.wrapComponent(Table, {initialTeams: data})}/>
        <Route path="about" handler={About}/>
      </Route>
    );
    return routes;
  },
  ...

Client routing and rendering allows us to provide fast transitions between views. Here, a user requesting /about is shown a small about page with a link to /standings. With react-router, when the user clicks either of these links their browser doesn't make a second GET request to the server. Instead, React router uses the client view and JSON data embedded in the page to render the standings table locally. Try this yourself by clicking back and forth using the links to "standings" and "about". You will see that there are no additional server requests and the rendering is almost instant, even with a large data grid. If you perform a hard-refresh in your browser on either of the links, forcing a server request, you will see the correct content for your current URL. React will still attach and then route client-side for any other clicks on these links.

Conclusion

In this article we've shown how to use React to render content on the server or client in an Isomorphic web application. Isomorphic web applications are a great way to address some of the biggest SEO challenges faced by Single Page Applications. If you are interested in learning more about this topic, please check out our blog for other articles on Single Page Applications. The entire source for our article is also published on github. Feel free to check it out and send us your thoughts and suggestions. If you need help with your website or mobile app development, please let us know!