Converting Apache/Tomcat Access Logs to CSV

Recently, I had to analyze the Apache / Tomcat access log files, and so I needed to convert the log files into CSV in order to have a chance to use other tools such as spreadsheet.
The conversion shouldn't be hard. I found some scripts (in PHP, AWK, Perl or Ruby) on the internet, but those didn't fit my needs quite well. I didn't want to lose any data such as http method, byte size sent in response, etc. Also, the CSV should contain spreadsheet friendly data format. For example, "2012-07-10 22:30:03" instead of "10/Jul/2012:22:30:03".
So, I ended up writing yet another one by myself. Why not? ;-)
Here's the link to the source:

• https://github.com/woonsan/accesslog2csv/blob/master/accesslog2csv.pl

The script can be executed like the following:

$ perl accesslog2csv.pl access_log_file > csv_output_file.csv

Or, you can redirect STDIN like the following examples:

$ perl accesslog2csv.pl < access_log_file > csv_output_file.csv

$ cat access_log_file | perl accesslog2csv.pl > csv_output_file.csv

Also, you can check invalid log lines by redirecting STDERR, too:

$ perl accesslog2csv.pl < access_log_file > csv_output_file.csv 2> invalid_log_lines.txt

Hope it helps somewhere! :-)

Generating Reports from Web Logs with AWStats

When you want to analyze the web access pattern from the web access logs, AWStats (http://awstats.sourceforge.net) is a handy solution. In my case, I needed to collect summary data from Tomcat access log files and build proper sample data for load testing.
Here's how to generate reports with AWStats from an access log file:

1. Prerequisites

• Your system should be able to run Perl scripts. If not, get Perl now: http://www.perl.org/get.html
• If you want to generate PDF report files, then you have HTMLDOC installed: http://www.htmldoc.org/.
  Let's assume the htmldoc command is available as `/usr/bin/htmldoc` after installation.
• If you want to run AWStats on Web, you should have an Apache2 Web Server and enable CGI. In this article, I'll skip this. See the online documentation if you're interested in: http://awstats.sourceforge.net/docs/awstats_setup.html

2. Install AWStats

If you extract the compressed AWStats distribution file, then you can find the `awstats_configure.pl' script under `tools' directory. You can start from the script like the following example.
 

$ perl ./awstats_configure.pl

<SNIP>

Do you want to continue setup from this NON standard directory [yN] ? y

<SNIP>

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile file (required if first install) [y/N] ? y

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> demo


<SNIP>

Press ENTER to continue... 

<SNIP>


Press ENTER to finish...

In the above example, I just installed AWStats just to generate reports offline from access log files without installing onto Apache Web Server for simplicity.
In the second prompt, I just typed 'demo' for a demo analysis task.
The above execution will generate the configuration file for the demo into the `../wwwroot/cgi-bin/awstats.demo.conf' file.

3. Setting the configuration file

Let's open and edit the configuration file for the 'demo' analysis task.
Assuming you're going to analyze a Tomcat access log file, which is in Apache Common Log format.
Here are what you need to edit at least in the configuration file (e.g., `../wwwroot/cgi-bin/awstats.demo.conf'):

# <SNIP>

# Set the access log file path here
LogFile="/var/log/tomcat/access.log"

# <SNIP>

# Examples for Apache combined logs (following two examples are equivalent):
# LogFormat = 1
# <SNIP/>
# For Apache Common Log Format (e.g., Tomcat access log), set it to 4.
LogFormat=4

# <SNIP>

# Set the data directory where AWStats internal data files are stored.
DirData="/var/log/data"

# <SNIP>

With the above configuration (the name of which is 'demo' as shown earlier), this analysis task will analyze the log file configured by 'LogFile' directive, and the internal data will be stored in the directory configured by 'DirData' directive.

4. Update Log Data

Now, you can run AWStats. Go to the `../wwwroot/cgi-bin/' directory and run the following command to update the data from the configured log file:

$ cd ../wwwroot/cgi-bin/
$ perl awstats.pl -config=demo -update

Create/Update database for config "./awstats.demo.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/var/log/tomcat/access.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 44217
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 1 corrupted records,
 Found 0 old records,
 Found 44216 new qualified records.
 

By the above command, AWStats will reads all the data from the configured log file and update the internal data files.
If you want to delete the data and re-update from the log files, then you can simply delete all the `*.txt' files in the data directory (which was configured by DirData directive above) and run `perl awstats.pl -config=demo -update` again.

5. Generate Reports

Finally, you can generate a report from the updated data by the following command:

#
# First copy the awstats_buildstaticpages.pl script from tools directory 
# if not exists here.
#
$ cp ../../tools/awstats_buildstaticpages.pl ./

$ perl awstats_buildstaticpages.pl -config=demo -month=all -year=2012 -dir=/tmp -awstatsprog=./awstats.pl -buildpdf=/usr/bin/htmldoc

or

$ perl awstats_buildstaticpages.pl -config=demo -month=all -year=2012 -dir=/tmp -awstatsprog=./awstats.pl

Main HTML page is 'awstats.demo.html'.
PDF file is 'awstats.demo.pdf'.

$



Now, the report file is generated into either html files or /tmp/awstats.demo.pdf!

You can skip `-buildpdf ...' option if you do not have HTMLDOC installed.  

Open the pdf file or the main html page now. It contains nice reports!

Spring Web MVC framework support in HST-2

(This article was migrated from http://blogs.onehippo.org/woonsan/2009/06/spring_web_mvc_framework_suppo_1.html, originally written on June 5, 2009.)


HST-2 has provided a basic support to enable developers to utilize Spring Framework IoC container for HST components. [1]
Now, HST-2 provides even more. It supports Spring Web MVC Framework based applications under HST-2 environment! Using Spring Web MVC Framework in HST-2 based application development, developers can make use of very useful features that Spring Web MVC Framework is providing, such as clear separation of roles (controller, validator, command object, form object, model object, handler mapping, view resolver, etc.), high configurability, customizability and flexibility.

Acknowledgement: I wrote and tested this Spring Web MVC Framework bridging solution with Spring Framework 2.5.6. However, I think this bridging solution would work with Spring Framework 1.1.5 or later version because the bridging solution depends on the followings only:

• The bridging solution's extended DispatcherServlet needs to override protected void render(ModelAndView mv, HttpServletRequest request, HttpServletResponse response), which was added since Spring Framework 1.1.5.
• The bridging solution is using simple URL dispatching to spring managed URLs, which has been already in the core part of Spring Web MVC Framework since its origination.

1. A Simple Form Controller Example: Contact-SpringMVC

You can build and run a Spring Web MVC Framework integration example. This example is available since HST-2.03.07.

• Build all:
  

  $ mvn clean install -DskipTests

• Run a testsuite's cms application:
  

  $ cd testsuite/cms

  $ mvn jetty:run-war

• Run a testsuite's site application:
  

  $ cd testsuite/site

  $ mvn jetty:run

• Visit http://localhost:8080/site/preview/news

Now, click the "Contact-SpringMVC" link on the left menu. You can see a page like the following:

If you enter some invalid information, e.g., "wicky" as email, the page will show some validation errors which were generated by the Spring Web MVC Framework like the following:

Now, let's fill valid information and it will show a success view which is defined in the Spring Web MVC configurations.

Here's the simplified configuration for the above Spring Web MVC Framework based application.

<?xml version="1.0" ?>
<!-- /WEB-INF/applicationContext.xml -->

<beans>
  <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">

    <property name="viewClass" value="org.springframework.web.servlet.view.JstlView"/>
    <property name="prefix" value="/WEB-INF/jsp/"/>
    <property name="suffix" value=".jsp"/>
  </bean>

  <bean
    class="org.springframework.web.servlet.handler.SimpleUrlHandlerMapping">
    <property name="mappings">
      <value>
        /spring/contactspringmvc.do = contactFormController
      </value>
    </property>
  </bean>

  <!-- Spring Web MVC Integration Example -->
  <bean id="contactFormController" class="org.hippoecm.hst.springmvc.ContactFormController">
    <property name="mailSender" ref="mailSender" />
    <property name="templateMessage" ref="templateMessage" />
    <property name="formView" value="/spring/contactspringmvc-form"/>
    <property name="successView" value="/spring/contactspringmvc-success"/>
    <property name="commandName" value="contactMessage"/>
    <property name="commandClass" value="org.hippoecm.hst.springmvc.ContactMessageBean"/>
    <property name="validateOnBinding" value="true"/>
    <property name="validators">
      <list>
        <bean class="org.hippoecm.hst.springmvc.ContactMessageValidator" />
      </list>
    </property>
  </bean>

<beans>

There's nothing new. Every beans in the applicationContext.xml are just normal beans which can be found in just a normal Spring Web MVC Framework applications.
The only connection point from HST-2 container is the following component configurations in the repository:

     <sv:node sv:name="contactspringmvcform">
        <sv:property sv:name="jcr:primaryType" sv:type="Name">
            <sv:value>hst:component</sv:value>
        </sv:property>
        <sv:property sv:name="hst:template" sv:type="String">
            <sv:value>contactspringmvc</sv:value>
        </sv:property>
        <sv:property sv:name="hst:componentclassname" sv:type="String">
            <sv:value>org.hippoecm.hst.component.support.SimpleDispatcherHstComponent</sv:value>
        </sv:property>
        <sv:property sv:name="hst:parameternames" sv:type="String">
            <sv:value>action-path</sv:value>
        </sv:property>
        <sv:property sv:name="hst:parametervalues" sv:type="String">
            <sv:value>/spring/contactspringmvc.do</sv:value>
        </sv:property>
    </sv:node>

The Contact-SpringMVC example has one component, "contactspringmvcform", which component class should be set to "org.hippoecm.hst.component.support.SimpleDispatcherHstComponent" to enable bridging to a pure Spring Web MVC Framework application.
Please note that this bridge component can have additional parameters as follows:

Name	Description
dispatch-path	The default dispatch path, to which the container dispatches on each invocation.
action-path	The dispatch path for doAction() invocation of the component. If this is not configured, then 'dispatch-path' would be used instead.
before-render-path	The dispatch path for doBeforeRender() invocation of the component. If this is not configured, then 'dispatch-path' would be used instead.
render-path	The dispatch path for rendering phase. If this is not configured, then 'dispatch-path' would be used instead.
before-resource-path	The dispatch path for doBeforeServeResource() invocation of the component. If this is not configured, then 'dispatch-path' would be used instead.
resource-path	The dispatch path for resource serving phase. If this is not configured, then 'dispatch-path' would be used instead.

The bridge component, "org.hippoecm.hst.component.support.SimpleDispatcherHstComponent", delegates all invocations by dispatching to configured paths.

Finally, you need to configure an extended Dispatcher Servlet in web.xml to run this example:

  <servlet>
    <servlet-name>HstDispatcherServlet</servlet-name>
    <servlet-class>org.hippoecm.hst.component.support.spring.mvc.HstDispatcherServlet</servlet-class>
    <init-param>
      <param-name>contextConfigLocation</param-name>
      <param-value>/WEB-INF/applicationContext.xml</param-value>
    </init-param>
  </servlet>

  <servlet-mapping>
    <servlet-name>HstDispatcherServlet</servlet-name>
    <url-pattern>*.do</url-pattern>
  </servlet-mapping>

The only difference is that the extended Dispatcher Servlet, "org.hippoecm.hst.component.support.spring.mvc.HstDispatcherServlet" should be used instead of the default "org.springframework.web.servlet.DispatcherServlet" of Spring Web MVC Framework.
The reason why this is necessary is explained in the next section.

In summary, you can use Spring Web MVC based application for HST-2 component development.

• To enable this bridging from HST-2 container, you need to use the delegator component, "org.hippoecm.hst.component.support.SimpleDispatcherHstComponent".
• To allow seamless bridging from HST-2 container, you need to use the "org.hippoecm.hst.component.support.spring.mvc.HstDispatcherServlet" in your web.xml.
• You need to configure some parameters such as "action-path" for the delegator comopnent, "org.hippoecm.hst.component.support.SimpleDispatcherHstComponent".
• You can now make use of all features provided by Spring Web MVC Framework such as validating, form controller, etc.!

2. The Internal with Architectural Explanation

2.1. Introduction to HST-2 request processing

I think it is a good time to explain briefly about the request processing architecture here because it is fundamental to understand the bridging solution.
For simplicity, I'd like to show an interaction between the HST container and each HST component here instead of explaining all details.
The basic interactions can be depicted as follows.

In the above diagram, the followings are assumed:

• The client is requesting a page which maps to a page configuration which is composed of a root HstComponents, "Parent"
• The "Parent" component has two child components, "LeftChild" and "RightChild". These two child components are siblings.
• At the time, the client is submitting a form included in the HstComponent, "RightChild".

The interaction sequences would be like the following in this case:

1. Client requests to HST-2 container.
2. Because the client is submitting a form by an action URL, the container invokes doAction() of "RightChild".
3. The container redirects to a render page.
   (Because the container aggregates multiple components in a page, the action phase should be separated from the render phase of all components. HST-2 container's aggregation implies the PRG pattern. [2])
4. Client requests to the render page.
5. The container invokes doBeforeRender() of each component. The invocation order of doBeforeRender() is from parent to child. The invocation order between siblings is not specified.
6. The container dispatches to the render path of each component. The dispatch order of render page of component is from child to parent. The invocation order between siblings is not specified.
7. A parent component's render page can include the rendered content of a child component.
8. The container writes the aggregated content to the client.

Here are important things to note as a bridge solution developer:

• Because the action phase request and render phase request are separated in HST-2 request processing, the web application framework should not assume that the request would be shared between the two phases.
  For example, when you use SimpleFormController of Spring Web MVC Framework with a form view page and a validator, if a user enters invalid information in the form, the dispatcher would render the form view again with some validation error information. Internally, this information is stored in a ModelAndView object to be rendered in the render phase. This cannot work in HST-2 request processing because the requests are not shared between action phase and render phase.
  Therefore, that kind of shared information between action phase and render phase should be passed correctly between two separate request processing phases by bridging solutions.
• By the way, because HST request and response objects are just extended objects to the default HttpServletRequest and HttpServletResponse, the other bridging integration stuffs could be easier than expected in other technologies such as Apache Portals Bridge. [3]

Because HST request and response objects are just HttpServletRequest and HttpServletResponse objects, we can think of a very simple bridging solution. We can create a HstComponent which dispatches all invocation to the specified dispatch path. In this case, all necessary handlings should be done by the dispatched servlet or JSP page.
This is covered in the next section.

2.2. A very simple bridging solution: SimpleDispatcherHstComponent

This component is the simplest bridging solution to native servlet-based applications.
Here's the simplified source:

package org.hippoecm.hst.component.support;

public class SimpleDispatcherHstComponent extends GenericHstComponent {
    public static final String LIFECYCLE_PHASE_ATTRIBUTE = SimpleDispatcherHstComponent.class.getName() + ".lifecycle.phase";
    public static final String BEFORE_RENDER_PHASE = "BEFORE_RENDER_PHASE";
    public static final String DISPATCH_PATH_PARAM_NAME = "dispatch-path";
    public static final String BEFORE_RENDER_PATH_PARAM_NAME = "before-render-path";
    public static final String RENDER_PATH_PARAM_NAME = "render-path";
    public static final String ACTION_PATH_PARAM_NAME = "action-path";

    @Override
    public void doAction(HstRequest request, HstResponse response) throws HstComponentException {
        doDispatch(getDispatchPathParameter(request, request.getLifecyclePhase()), request, response);
    }

    @Override
    public void doBeforeRender(HstRequest request, HstResponse response) throws HstComponentException {
        request.setAttribute(LIFECYCLE_PHASE_ATTRIBUTE, BEFORE_RENDER_PHASE);
        String dispatchPath = getDispatchPathParameter(request, request.getLifecyclePhase());
    
        if (dispatchPath != null) {
            response.setRenderPath(dispatchPath);
        }

        try {
            doDispatch(getDispatchPathParameter(request, BEFORE_RENDER_PHASE), request, response);

        } finally {
            request.removeAttribute(LIFECYCLE_PHASE_ATTRIBUTE);
        }
    }

    protected void doDispatch(String dispatchPath, HstRequest request, HstResponse response) throws HstComponentException {
        if (dispatchPath != null) {
            try {
                getServletConfig().getServletContext().getRequestDispatcher(dispatchPath).include(request, response);
            } catch (ServletException e) {
                throw new HstComponentException(e);
            } catch (IOException e) {
                throw new HstComponentException(e);
            }
        }
    }

    protected String getDispatchPathParameter(HstRequest request, String lifecyclePhase) {
        String dispatchPath = null;
    
        if (BEFORE_RENDER_PHASE.equals(lifecyclePhase)) {
            dispatchPath = getParameter(BEFORE_RENDER_PATH_PARAM_NAME, request, null);
        } else if (HstRequest.RENDER_PHASE.equals(lifecyclePhase)) {
            dispatchPath = getParameter(RENDER_PATH_PARAM_NAME, request, null);
        } else if (HstRequest.ACTION_PHASE.equals(lifecyclePhase)) {
            dispatchPath = getParameter(ACTION_PATH_PARAM_NAME, request, null);
        }
    
        if (dispatchPath == null) {
            dispatchPath = getParameter(DISPATCH_PATH_PARAM_NAME, request, null);
        }
    
        if (dispatchPath != null) {
            if (dispatchPath.charAt(0) != '/') {
                dispatchPath = new StringBuilder(dispatchPath.length() + 1).append('/').append(dispatchPath).toString();
            }
        }
    
        return dispatchPath;
    }

    protected String getParameter(String name, HstRequest request, String defaultValue) {
        String value = (String) this.getComponentConfiguration().getParameter(name, request.getRequestContext().getResolvedSiteMapItem());
        return (value != null ? value : defaultValue);
    }
}

In the above component, doAction() just dispatches to a dispatch path, which is configured by 'action-path' or falled back to 'dispatch-path' if 'action-path' is not specified in the repository configuration.
And, doBeforeRender() just dispatches to a dispatch path, which is configured by 'before-render-path' or falled back to 'dispatch-path' if 'before-render-path' is not specified in the repository configuration. Also, it sets the render path dynamically by the configuration value for 'render-path', which can be falled back to 'dispatch-path' if not configured.
So, when the container invokes doAction() or doBeforeRender() of this component, it actually dispatches to the native servlet or JSP pages. Also, the container would invoke the render path dynamically set by this component.
The remaining thing is to write the dispatched servlet or JSP page to handle all the invocation correctly.

In most web application framework, the frontend controller should be a servlet, but I'd like to use a simple JSP page for simplicity here.
The above component should have a paramter 'dispatch-url' set to 'jsp/components/contactdispatch.jsp'.
Here is an example native JSP page to handle those (as a simplified version):

<%-- contactdispatch.jsp --%>
<%!
private static String[] formFields = {"name","email","textarea"};

private void doBeforeRender(HstRequest request, HstResponse response) throws HstComponentException {
    HttpSession session = request.getSession(true);
    FormMap formMap = (FormMap) session.getAttribute("contactdispatch:formMap");
    if (formMap == null) {
        formMap = new FormMap();
        session.setAttribute("contactdispatch:formMap", formMap);
    }
    request.setAttribute("form", formMap);
}

private void doAction(HstRequest request, HstResponse response) throws HstComponentException {
    HttpSession session = request.getSession(true);
    FormMap formMap = new FormMap(request, formFields);
    session.setAttribute("contactdispatch:formMap", formMap);
    // Do a really simple validation:
    if (formMap.getField("email") != null && formMap.getField("email").contains("@")) {
        // success
        // do your business logic
        // possible do a redirect to a thankyou page: do not use directly response.sendRedirect;
        HstSiteMapItem item = request.getRequestContext().getResolvedSiteMapItem().getHstSiteMapItem().getChild("thankyou");
        if (item != null) {
            sendRedirect(request, response, item.getId());
        } else {
            log.warn("Cannot redirect because siteMapItem not found. ");
        }
    } else {
        // validation failed. Persist form map, and add possible error messages to the formMap
        formMap.addMessage("email", "Email address must contain '@'");
    }
}

private void sendRedirect(HstRequest request, HstResponse response, String redirectToSiteMapItemId) {
    HstLinkCreator linkCreator = request.getRequestContext().getHstLinkCreator();
    HstSiteMap siteMap = request.getRequestContext().getResolvedSiteMapItem().getHstSiteMapItem().getHstSiteMap();
    HstLink link = linkCreator.create(siteMap.getSiteMapItemById(redirectToSiteMapItemId));
    StringBuffer url = new StringBuffer();
    for (String elem : link.getPathElements()) {
        String enc = response.encodeURL(elem);
        url.append("/").append(enc);
    }
    String urlString = ((HstResponse) response).createNavigationalURL(url.toString()).toString();
    try {
        response.sendRedirect(urlString);
    } catch (IOException e) {
        throw new HstComponentException("Could not redirect. ",e);
    }
}
%>

<%
HstRequest hstRequest = (HstRequest) request;
HstResponse hstResponse = (HstResponse) response;

String hstRequestLifecyclePhase = hstRequest.getLifecyclePhase();
String dispatchLifecyclePhase = (String) hstRequest.getAttribute(SimpleDispatcherHstComponent.LIFECYCLE_PHASE_ATTRIBUTE);

if (HstRequest.ACTION_PHASE.equals(hstRequestLifecyclePhase)) {
    doAction(hstRequest, hstResponse);
} else if (SimpleDispatcherHstComponent.BEFORE_RENDER_PHASE.equals(dispatchLifecyclePhase)) {
    doBeforeRender(hstRequest, hstResponse);
} else if (HstRequest.RENDER_PHASE.equals(hstRequestLifecyclePhase)) {
%>
<div>
    <form method="POST" name="myform" action="<hst:actionURL/>">
    <input type="hidden" name="previous" value="${form.previous}"/>
    <br/>
    <table>
        <tr>
            <td>Name</td>
            <td><input type="text" name="name" value="${form.value['name']}" /></td>
            <td><font style="color:red">${form.message['name']}</font></td>
        </tr>
        <tr>
            <td>Email</td>
            <td><input type="text" name="email" value="${form.value['email']}"/></td>
            <td><font style="color:red">${form.message['email']}</font></td>
        </tr>
        <tr>
            <td>Text</td>
            <td><textarea name="textarea">${form.value['textarea']}</textarea></td>
            <td><font style="color:red">${form.message['textarea']}</font></td>
        </tr>
        <tr>
            <td>
                <c:if test="${form.previous != null}">
                  <input type="submit" name="prev" value="prev"/>
                </c:if>
            </td>
            <td><input type="submit" value="send"/></td>
        </tr>
    </table>
    </form>
</div>
<% } %>

Because the component just dispatches each invocation to a dispatch path, the above JSP pages should handle everything correctly.
The following JSP scriptlets detect the request process lifecycle phases and invoke the proper methods, which were just copied from the existing Contact component example.

<%
if (HstRequest.ACTION_PHASE.equals(hstRequestLifecyclePhase)) {
    doAction(hstRequest, hstResponse);
} else if (SimpleDispatcherHstComponent.BEFORE_RENDER_PHASE.equals(dispatchLifecyclePhase)) {
    doBeforeRender(hstRequest, hstResponse);
} else if (HstRequest.RENDER_PHASE.equals(hstRequestLifecyclePhase)) {
%>
//...
<%
}
%>

So, any kind of servlet based application can control everything by using this kind of technique.

2.3. An extended DispatcherServlet: HstDispatcherServlet

In the Spring Web MVC Framework bridging solution, the simplest bridging component, "SimpleDispatcherHstComponent", is used, and the 'action-path' parameter is just set to a spring managed URL like '/spring/contactspringmvc.do'.
So, we can say that the frontend controller should handle everything.
For this reason, we provide a dispatcher servlet, "HstDispatcherServlet", which extends the default DispatcherServlet.
The responsibility of HstDispatcherServlet is very simple. It should pass the ModelAndView object from the action request phase to render request phase:

• After completing action phase, it should store the ModelAndView object into session attributes temporarily.
• Before doing render phase, it should restore the ModelAndView object from the session attributes if available.

HstDispatcherServlet just overrides the method, render(ModelAndView mv, HttpServletRequest request, HttpServletResponse response) of the default DispatcherServlet to accomplish this.

References

[1] http://woonsanko.blogspot.com/2012/07/spring-framework-support-in-hst-2.html
[2] http://en.wikipedia.org/wiki/Post/Redirect/Get
[3] http://portals.apache.org/bridges/

Spring framework support in HST-2

(This article was migrated from http://blogs.onehippo.org/woonsan/2009/04/spring_framework_support_in_hs_1.html, originally written on April 20, 2009.)

I'd like to explain the current status to support Spring Framework-based developer community.
HST-2 (Hippo Site Toolkit - 2) now supports Spring Framework more than the earlier versions because there are a lot of developers utilizing the framework.

1. Introduction

It is very desirable to use spring web framework if the development team is familiar with Spring and they could make better productivity with that.
HST-2 now provides a bridge HstComponent to HstComponent beans which are managed by spring web application context.
So, you can define your HstComponent beans in your spring context configuration and inject your existing component to them. You can see an example in the HST-2 source already if you download the source from the trunk for now.
In the Hippo Test Suite project, 'contact-spring' menu will show the example which uses the bridge component. The actual bean is defined in /WEB-INF/applicationContext.xml like the following:

  <!-- HST Component Beans -->
  <bean id="contactBean" class="org.hippoecm.hst.components.ContactSpring">
    <property name="mailSender" ref="mailSender" />
    <property name="templateMessage" ref="templateMessage" />
  </bean>

  <!-- The existing components as an example -->
  <bean id="templateMessage" class="org.springframework.mail.SimpleMailMessage">
    <property name="to" value="contact@mycompany.com" >
    <property name="subject" value="My opinion" >
  </bean>

  <bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
    <property name="host" value="mail.mycompany.com"/>
  </bean><br>

2. How it integrates with HST 2.3?

HST container manages, invokes and aggregates only components based on HstComponent interface for some reasons: to support transparent page aggregation, seamless portal integration, etc.
A generic HstComponent was developed to play a role as a bridge to a bean managed by spring framework.
So, the generic bridge HstComponent will delegate all invocation to the actual HstComponent bean managed by spring framework.

3. What the benefits are? What the drawbacks are?

Spring-based developers can use full cool functionalities of spring framework like Dependency Injection, out-of-box spring components to support enterprise computing like JDBC template, transaction management and enterprise messaging, AOP techniques, Web Service support, Various view technologies, etc. HstComponent can be fully integrated with these rich spring supports.
The Spring Bridge HstComponent invokes directly the actual bean, so there's no performance degrade and functional shortage here. 
Of course, it looks a little bit different from the original spring web mvc pattern. The differences you can think of are originally from the differences of goals: HST-2 is to support transparent page aggregation and seamless portal integration, etc. Because HST-2 container aggregates multiple components in one page, the action phase should be separated from the rendering phase of all components. This means that HST container's aggregation should imply the PRG pattern internally. [1] Therefore, in a HST component, the *controlling* logic should be separated in doAction() and in doBeforeRender(). Also, those two separate phase should have separate request life cycles. 
Anyway, once you get accustomed to HstComponent request lifecycles, you can easily use the pattern with HstComponent in the spring framework best practices. For example, spring developers can write controller logics in HstComponents and use any kinds of view technologies to build view pages. 
On the other hand, somebody can ask me, 'why can't spring webmvc manage the request handling directly itself? why HST container controls the request at the frontend and it dispatches to spring web mvc later?'  The answers can be like the following:

• HST manages the page aggregation like portal. If you want to use that kind of page aggregation and you don't want to care its detail except of focusing on making each component block, you need to hand it over to HST container. HST container promises that it would provide transparent page aggregation, sitemap/sitemenu and seamless portal integration. 
• If you don't want that kind of page aggregation, you can still use some HST components as out-of-box components such as JCR session pooling repository, event listener (observation), search helper, etc. by definining those components in your spring configuration.

4. What's next?

I think this spring bean bridging solution is very useful for many use cases. However, some Spring Web MVC framework-proficient developers can complain that they cannot use some kind of handler mappings for each controller.
I agree that this simple bridging solution could not be enough for the developers.
I think we can go further by providing *HST component phase parameter handler mapping*. :-) This can be regarded as *HST Spring Web MVC*.
Sounds like org.springframework.web.portlet.handler.PortletModeParameterHandlerMapping of Spring Portlet Web MVC?
Yes, because the request life cycle of HST component is very similar to portlet, almost similar design can be adopted.
However, I think there are some better features in HST Spring Web MVC than Spring Portlet Web MVC. One is the fact that Hst Component controller in HST Spring Web MVC might not be tied to portlet api directly.
Anyway, I'd like to look into this improvement soon. Please stay tuned!