Caucho maker of Resin Server | Application Server (Java EE Certified) and Web Server


home
 

Resin Documentation

home company blog docs 
app server 		 Resin Server | Application Server (Java EE Certified) and Web Server
 

resin load balancing

As traffic increases beyond a single server, Resin's load-balancing lets you add new machines to handle the load and simultaneously improves uptime and reliability by failing over requests from a downed or maintenance server to a backup transparently.

  1. Load Balancing
    1. Two Server Configuration
    2. Socket Pooling, Timeouts, and Failover
    3. Dispatching

Load Balancing

When your traffic requires multiple servers, your site will naturally split into two clusters: an app-tier of identical servers to handle the content and a web-tier of HTTP servers talking to the browsers, caching the content, and distributing the load to the app-tier servers.

Each app-tier server is configured identically, because servers produces the same content: the app-tier servers have the same virtual hosts, web-applications and servlets, and use the same resin.xml. Adding a new machine just requires adding a new <server> tag to the cluster.

web-tier(80:web-a) -> app-tier(6800:app-a(foo.com,bar.com), 6801:app-b(foo.com,bar.com))

The new server has a unique name like "app-b" and a TCP cluster-port consisting of an <IP,port>, so the other servers can communicate with it. Although you can start multiple Resin servers on the same machine, TCP requires the <IP,port> must be unique, so you might need to assign unique ports like 6801, 6802 for servers on the same machine. On different machines, you'll use unique IP addresses. Because the cluster-port is for Resin servers to communicate with each other, they'll typically be private IP addresses like 192.168.1.10 and not public IP addresses. In particular, the load balancer on the web-tier uses the cluster-port of the app-tier to forward HTTP requests.

The load balancer on the web-tier forwards requests to the app-tier, distributing the load evenly and skipping any app-tier server that's down for maintenance or restarting due to a crash. This failover capability increases reliability and improves the customer's experiency by making your site look like every server is always up. The load balancer also steers traffic from a user session to the same app-tier server, improving caching and session performance, i.e. it supports sticky sessions.

Two Server Configuration

For example, a site running Drupal on Resin might now need two app-tier servers to handle additional load as it grows traffic. In the following configuration, the two app-tier servers "app-a" and "app-b" both serve the Drupal content, while the web-tier server "web-a" handles the HTTP, caches content, and balances the load to the app-tier cluster.

The web-tier is configured with a <cache> tag for the caching, and uses <resin:LoadBalance> for the dispatching. In this case, we send all content to the app-tier. <resin:LoadBalance> is part of Resin's rewrite capabilities, load Resin's equivalent of the Apache mod_rewrite module, providing a powerful and detailed URL matching and decoding, so more complicated sites might load-balance based on the virtual host or URL.

Example: resin.xml for load balancing
<resin xmlns="http://caucho.com/ns/resin"
       xmlns:resin="urn:java:com.caucho.resin">

<cluster-default>
   <resin:import path="${__DIR__}/app-default.xml"/>
   <development-mode-error-page/>
</cluster-default>

<cluster id="app-tier">
  <server id="app-a" address="192.168.0.10" port="6800"/>
  <server id="app-b" address="192.168.0.11" port="6800"/>

  <host id="">
    <web-app id="" root-directory="/var/resin/htdocs"/>
  </host>
</cluster>

<cluster id="web-tier">
  <server id="web-a" address="192.168.0.1" port="6800">
    <http port="80"/>
  </server>

  <cache memory-size="256M"/>

  <host id="">
    
      <resin:LoadBalance regexp="" cluster="app-tier"/>

  </host>
</cluster>

</resin>

All three servers will use the same resin.xml, which makes managing multiple servers easier. The servers are name by the server id attribute, which must be unique. When you start Resin, you'll use the server-id as part of the command line

Example: starting servers
192.168.0.10> bin/resin.sh -server app-a start

192.168.0.11> bin/resin.sh -server app-b start

192.168.0.1> bin/resin.sh -server web-a start

Since Resin lets you start multiple servers on the same machine, a small site might start the web-tier server and one of the app-tier servers on one machine, and start the second server on a second machine. You can even start all three servers on the same machine, increasing reliability and easing maintenance, without addressing the additional load. If you do put multiple servers on the same machine, remember to change the port to something like 6801, so the TCP bind doesn't conflict.

In the /resin-admin management page, you can manage all three servers at once, gathering statistics and load and watching for any errors. When setting up /resin-admin on a web-tier server, you'll want to remember to add a separate <web-app> for resin-admin to make sure the <rewrite-dispatch> doesn't inadvertantly send the management request to the app-tier.

Socket Pooling, Timeouts, and Failover

For efficiency, Resin's load balancer manages a pool of sockets connecting to the app-tier servers. If Resin forwards a new request to an app-tier server and it has an idle socket available, it will reuse that socket, improving performance an minimizing network load. Resin uses a set of timeout values to manage those idle sockets and to handle any failures or freezes of the backend servers. The following diagram illustrates the main timeout values:

web-a:connecty-timeout,app-a:socket-timeout,web-a:recover-time,app-a:keepalive-timeout,web-a:idle-time
  • load-balance-connect-timeout: the load balancer timeout for the connect() system call to complete to the app-tier (5s).
  • load-balance-idle-time: load balancer timeout for an idle socket before closing it automatically (5s).
  • load-balance-recover-time: the load balancer connection failure wait time before trying a new connection (15s).
  • load-balance-socket-timeout: the load balancer timeout for a valid request to complete (665s).
  • keepalive-timeout: the app-tier timeout for a keepalive connection (15s)
  • socket-timeout: the app-tier timeout for a read or write (65s)

When an app-tier server is down due to maintenance or a crash, Resin will use the load-balance-recover-time as a delay before retrying the downed server. With the failover and recover timeout, the load balancer reduces the cost of a failed server to almost no time at all. Every recover-time, Resin will try a new connection and wait for load-balance-connect-timeout for the server to respond. At most, one request every 15 seconds might wait an extra 5 seconds to connect to the backend server. All other requests will automatically go to the other servers.

The socket-timeout values tell Resin when a socket connection is dead and should be dropped. The web-tier timeout load-balance-socket-timeout is much larger than the app-tier timeout socket-timeout because the web-tier needs to wait for the application to generate the response. If your application has some very slow pages, like a complicated nightly report, you may need to increase the load-balance-socket-timeout to avoid the web-tier disconnecting it.

Likewise, the load-balance-idle-time and keepalive-timeout are a matching pair for the socket idle pool. The idle-time tells the web-tier how long it can keep an idle socket before closing it. The keepalive-timeout tells the app-tier how long it should listen for new requests on the socket. The keepalive-timeout must be significantly larger than the load-balance-idle-time so the app-tier doesn't close its sockets too soon. The keepalive timeout can be large since the app-tier can use the keepalive-select manager to efficiently wait for many connections at once.

Dispatching

In most cases, the web-tier will dispatch everything to the app-tier servers. Because of Resin's proxy cache, the web-tier servers will serve static pages as fast as if they were local pages.

In some cases, though, it may be important to send different requests to different backend clusters. The <resin:LoadBalance> tag can choose clusters based on URL patterns.

The following rewrite keeps all *.png, *.gif, and *.jpg files on the web-tier, sends everything in /foo/* to the foo-tier cluster, everything in /bar/* to the bar-tier cluster, and keeps anything else on the web-tier.

Example: resin.xml split dispatching
<resin xmlns="http://caucho.com/ns/resin"
      xmlns:resin="urn:java:com.caucho.resin">

  <cluster-default>
     <resin:import path="${__DIR__}/app-default.xml"/>
     <development-mode-error-page/>
  </cluster-default>
      
  <cluster id="web-tier">
    <server id="web-a">
      <http port="80"/>
    </server>

    <cache memory-size="64m"/>

    <host id="">
      <web-app id="/">

        <resin:Dispatch regexp="(\.png|\.gif|\.jpg)"/>

        <resin:LoadBalance regexp="^/foo" cluster="foo-tier"/>

        <resin:LoadBalance regexp="^/bar" cluster="bar-tier"/>

      </web-app>
    </host>
  </cluster>

  <cluster id="foo-tier">
    ...
  </cluster>

  <cluster id="bar-tier">
    ...
  </cluster>
</resin>
Copyright © 1998-2015 Caucho Technology, Inc. All rights reserved. Resin ® is a registered trademark. Quercustm, and Hessiantm are trademarks of Caucho Technology.