This blog post was originally posted in the ATG Developer Community by Spark::red’s co-founding partner Russell Moore in September, 2008.

Abstract

mod_jk works well, but is more complicated to set up and does not come built-in with apache. This is a step-by-step guide to setting up load balancing using mod_proxy_ajp and mod_proxy_balancer. mod_proxy_ajp gives you the same fast ajp connectivity as mod_jk, and mod_proxy_balancer gives you load balancing with sticky sessions and automatic failover.

Assumptions

This document assumes you have a working knowledge of how to set up jBoss and configure apache. You should have installed the version of jBoss you are going to use (4.0.5.GA for ATG 2007.1) and apache 2.2, with the following modules either compiled in or compiled as usable modules:

  • mod_proxy
  • mod_proxy_ajp
  • mod_proxy_balancer
  • For the examples below, the environment is assumed to look like this:
    web01 & web02 are 2 physical hosts, each with apache 2.2.9 installed, with the above required modules compiled in.

    app01 & app02 are 2 physical hosts, each with 2 instances of jBoss running on them. For this example, they will use bindings from $JBOSS_HOME/docs/examples/binding-manager/sample-bindings.xml. Instances on each host will be prd1a on ports-default, prd1b on ports-01 on app01, and prd2a on ports-default and prd2b on ports-01 on app02. Here is what that looks like:
    conf/jboss-service.xml

    >...
    <mbean code="org.jboss.services.binding.ServiceBindingManager" name="jboss.system:service=ServiceBindingManager">
         <attribute name="ServerName">ports-01</attribute>
    ...

    Note that we are not configuring servers that will not participate in load balancing, such as a fulfillment instance.

    It is assumed you will have a hardware load balancer (or some other load balancer) out in front for load balancing across the web servers.

    Configure jBoss Instances

    For each of the jBoss instances, you need to do a couple of things. First you have to turn on JK. THis is done by editing the the file under each of the jBoss servers’ deploy directory: deploy/jbossweb-tomcat55.sar/META-INF/jboss-service.xml
    jboss-service.xml

    ...

    <attribute name="UseJK">true</attribute>
    ...

    by default this value is set to false.

    Second, you need to configure the jvmRoute. This is done by editing deploy/jbossweb-tomcat55.sar/server.xml:

    server.xml
    ...
    <Engine name="jboss.web" defaultHost="localhost" jvmRoute="node1">
    ...

    Important : each jvmRoute must be unique! So in our example, prd1a will be “node1”, prd1b will be “node2”, prd2a will be “node3”, and prd2b will be “node4”.

    Configure Apache

    Now on to the web servers – you need to edit the httpd.conf for your apache install. This is in the conf directory of wherever you told apache to build to, or where apt or rpm put it, usually under /etc/httpd/conf/ or /etc/apache2/conf/.

    Here is what it looks like, and I will explain below what each line does:

    httpd.conf
    ...
    #
    #Set up jBoss-specific proxying
    #
        ProxyRequests Off
        ProxyVia On
        ProxyPass /favicon.ico !
        ProxyPass /dyn/admin !
        ProxyPass /web-console !
        ProxyPass / balancer://ajpCluster/ stickysession=JSESSIONID nofailover=Off
        <Proxy balancer://ajpCluster>
           BalancerMember ajp://app01:8009 route=node1
           BalancerMember ajp://app01:8109 route=node2
           BalancerMember ajp://app02:8009 route=node3
           BalancerMember ajp://app02:8109 route=node4
        </Proxy>
    ...

    Line 1 – ProxyRequests Off: This is for security. If you do not set this you will become an internet proxy, and people can route through you to do all kinds of nasty stuff. Make sure you have this set correctly.

    Line 2 – ProxyVia On: this turns on reverse proxying, which is what we want to do. We want to act as a proxy for requests to internal resources.

    Line 3 – ProxyPass /favicon.ico ! : The ProxyPass directive usually means pass everything under this to the back, unless, like in this case,you put a ! (not!) at the end, meaning even if this URI falls under a later ProxyPass directive, do not pass it back. You may want to use this line for everything you are not going to be serving out of jBoss, such as /images. You can set up as many ProxyPass exceptions as you need here for your particular installation.

    Lines 4 & 5 – ProxyPass /dyn/admin! & /web-console ! : Similar to the previous one, you want to make sure that you are not exposing your administrative interfaces to the world. You may not need this, but in the primary directive (below) we are defaulting “/” to go to jBoss.

    Line 6 – ProxyPass / balancer://ajpCluster/ : This is the primary ProxyPass directive. It says pass everything under “/” to the balancer named ajpCluster “/”. Additional lines say stickysession-JSESSIONID, nofailover=Off. Stickysession keeps users on the jBoss instance that their session is on. You will see additional information appended to the sessionid in the cookies (more on that later). nofailover=Off is the default setting, but wanted to show it here. Basically, nofailover Off means if the instance that the session is associated with goes down/becomes unreachable, then redirect the user to the next available instance. The user will lose their session information (unless you have set up session replication in jBoss- that’s another topic) but will not get a white page. If you set this to On, individual users will get a 503 error if their server goes down. You could use this and write other directives to notify the user, but for most eCommerce sites that is an unnecessary step (banking or something else it may be more critical to handle).

    Again, you do not have to pass everything under “/” to jBoss. If you have a context root set for a particular application, you can change both of these to /context-root; for example if your ATG application is under /estore, it would look like:
    ProxyPass /estore/ balancer://ajpCluster/estore/ stickysession=JSESSIONID
    you can even map to different context roots (assuming your jsps do not hardcode values).

    Remaining block – Proxy balancer : This si the section where you list all of your instances in the balancer named ajpCluster.For each member participating in the balancer, you have a BalancerMember entry, with the ajp URL (including the ajp port you set up) and the additional jvmRoute information. This is why it is important to create jvmRoutes in jBoss – if you look at the session cookies, in our case both JSESSIONID and ATG_SESSION_ID in a balanced cluster such as this, you will see it looks something like: A6EB7C5A5E14B6222C543B9E321B29E8.node1

    The .node1 is how it keeps track of where the session goes, and even without sticky sessions set up on the front-end hardware load balancer (the one balancing the web servers) users should get directed to the correct jBoss instance by the virtue of this value being on the end of their session id.