Kualigan

Kuali Parameterized ValueFinder Implementation

2011-12-21T21:16:00.000-08:00

Overview

Sometimes it is necessary to dynamically build a values finder based on some outside influence (sometimes another values finder). This is how to create a value finder that uses input from another source. I am going to use an example where we want to give a list of available mileages for a user to select from. We only want to show mileages to the user based on the date their travel arrangements are made during. If we don't limit in this way, the number of available mileages to choose from will be infinite and indistinguishable. What we need is a list of mileages based on a date determined by the trip information already in the form. This is how we do that.

Steps

1. Create a JstlFunctions class with static methods

package org.kuali.kfs.module.tem.web;

import static java.lang.Class.forName;

import java.util.List;
import java.util.Map;

import org.apache.commons.beanutils.BeanUtils;
import org.kuali.rice.kns.lookup.keyvalues.KeyValuesFinder;

/**
 * Full of static methods for JSTL function access.
 * 
 */
public final class JstlFunctions {
    private static final String SETTING_PARAMS_PROLOG = "Setting params ";
    private static final String PROPERTY_SETTING_EXC_PROLOG = "Could not set property ";
    private static final String IN_PREPOSITION = " in ";
    private static final String VALUES_FINDER_CLASS_EXC_PROLOG = "Could not find valuesFinder class ";
    private static final org.apache.commons.logging.Log LOG = org.apache.commons.logging.LogFactory.getLog(JstlFunctions.class);

    private JstlFunctions() {}
    
    /**
     * Returns a list of key/value pairs for displaying in an HTML option for a select list. This is a customized approach to retrieving
     * key/value data from database based on criteria specified in the <code>params {@link Map}</code><br/>
     * <br/>
     * Here is an example of how the code is used from a JSP:<br/>
     * <code>
     * <jsp:useBean id="paramMap" class="java.util.HashMap"/>
                    <c:set target="${paramMap}" property="forAddedPerson" value="true" />
                    <kul:checkErrors keyMatch="${proposalPerson}.proposalPersonRoleId" auditMatch="${proposalPerson}.proposalPersonRoleId"/>  
                    <c:set var="roleStyle" value=""/>
                    <c:if test="${hasErrors==true}">
                        <c:set var="roleStyle" value="background-color:#FFD5D5"/>
                    </c:if>
                    <html:select property="${proposalPerson}.proposalPersonRoleId" tabindex="0" style="${roleStyle}">
                    <c:forEach items="${krafn:getOptionList('org.kuali.kra.proposaldevelopment.lookup.keyvalue.ProposalPersonRoleValuesFinder', paramMap)}" var="option">
                    <c:choose>
                        <c:when test="${KualiForm.document.proposalPersons[personIndex].proposalPersonRoleId == option.key}">
                        <option value="${option.key}" selected>${option.label}</option>
                        </c:when>
                        <c:otherwise>
                        <option value="${option.key}">${option.label}</option>
                        </c:otherwise>
                    </c:choose>
                    </c:forEach>
                    </html:select>
       </code>
     * 
     * 
     * @param valuesFinderClassName
     * @param params mapped parameters
     * @return List of key values
     */
    @SuppressWarnings("unchecked")
    public static List getOptionList(String valuesFinderClassName, Map params) {
        return setupValuesFinder(valuesFinderClassName, (Map<String, Object>) params).getKeyValues();
    }
    
    /**
     * Initiates the values finder by its <code>valuesFinderClassName</code>. First locates the class in the class path. Then, 
     * creates an instance of it. A <code>{@link Map}</code> of key/values <code>{@link String}</code> instances a is used
     * to set properties on the values finder instance. Uses the apache <code>{@link PropertyUtils}</code> class to set properties
     * by the name of the key in the <code>{@link Map}</code>.<br/>
     * <br/>
     * Basically, a new values finder is created. the <code>params</code> parameter is a <code>{@link Map}</code> of arbitrary values
     * mapped to properties of the values finder class.<br/>
     * <br/>
     * Since this is so flexible and the ambiguity of properties referenced in the <code>{@link Map}</code>, a number of exceptions are caught
     * if a property cannot be set or if the values finder cannot be instantiated. All of these exceptions are handled within the method. None
     * of these exceptions are thrown back.
     * 
     * 
     * @param valuesFinderClassName
     * @param params
     * @return KeyValuesFinder
     * @see PropertyUtils#setProperty(Object, String, Object)
     */
    private static KeyValuesFinder setupValuesFinder(String valuesFinderClassName, Map<String, Object> params) {
        KeyValuesFinder retval = getKeyFinder(valuesFinderClassName);
        
        if(LOG.isDebugEnabled()) {
            LOG.debug(SETTING_PARAMS_PROLOG + params);
        }
        
        addParametersToFinder(params, retval);

        return retval;
    }

    private static void addParametersToFinder(Map<String, Object> params, KeyValuesFinder finder) {
        if (finder != null && params != null) {
            for (Map.Entry<String, Object> entry : params.entrySet()) {
                try {
                    BeanUtils.setProperty(finder, entry.getKey(), entry.getValue());
//                    setProperty(finder, entry.getKey(), entry.getValue());
                } catch (Exception e) {
                    warn(PROPERTY_SETTING_EXC_PROLOG + entry.getKey(), e);
                    e.printStackTrace();
                }
            }
        }
    }

    private static KeyValuesFinder getKeyFinder(String valuesFinderClassName) {
        KeyValuesFinder retval = null;
        try {
            retval = (KeyValuesFinder) forName(valuesFinderClassName).newInstance();                        
        } catch (ClassNotFoundException e) {
            warnAboutValueFinderClassExceptions(valuesFinderClassName, e);
        } catch (InstantiationException e) {
            warnAboutValueFinderClassExceptions(valuesFinderClassName, e);
        } catch (IllegalAccessException e) {
            warnAboutValueFinderClassExceptions(valuesFinderClassName, e);
        }
        return retval;
    }

    private static void warnAboutValueFinderClassExceptions(String valuesFinderClassName, Exception e) {
        warn(VALUES_FINDER_CLASS_EXC_PROLOG + valuesFinderClassName, e);
    }

    private static void warn(String message, Exception e) {
        if (LOG.isWarnEnabled()) {
            LOG.warn(new StringBuilder(message).append(IN_PREPOSITION).append(buildTraceMessage(e)));
        }
    }

    /**
     * Get the stack trace from a <code>{@link Throwable}</code> and create a log message from it for tracing purposes
     * 
     * @param thrownObj 
     * @return String log message
     */
    private static String buildTraceMessage(Throwable thrownObj) {
        StackTraceElement stackTraceElement = thrownObj.getStackTrace()[0];
        return new StringBuilder(stackTraceElement.getClassName())
                        .append("#") 
                        .append(stackTraceElement.getMethodName())
                        .append(":") 
                        .append(stackTraceElement.getLineNumber())
                        .append(" ")
                        .append(thrownObj.getClass().getSimpleName())
                        .append("\n")
                        .append(thrownObj.getMessage())
                        .toString();
    }
}

getOptionsList is the only public method in the class.

@SuppressWarnings("unchecked")
    public static List getOptionList(String valuesFinderClassName, Map params) {
        return setupValuesFinder(valuesFinderClassName, (Map<String, Object>) params).getKeyValues();
    }

You can see it returns a List because this will eventually be used with a JSP or JSTL tag file which do not support parameterized types. You can see that parameters are passed in as a Map called params. This is used via the addParametersToFinder method here:

private static void addParametersToFinder(Map<String, Object> params, KeyValuesFinder finder) {
        if (finder != null && params != null) {
            for (Map.Entry<String, Object> entry : params.entrySet()) {
                try {
                    BeanUtils.setProperty(finder, entry.getKey(), entry.getValue());
//                    setProperty(finder, entry.getKey(), entry.getValue());
                } catch (Exception e) {
                    warn(PROPERTY_SETTING_EXC_PROLOG + entry.getKey(), e);
                    e.printStackTrace();
                }
            }
        }
    }

2. Create a JSTL TLD

Now that we have our class and our public method that will create our finder for us, we will need some way to invoke it. The best way to do this is by using the static method through a tag. We can create this tag easily with a TLD.

<?xml version="1.0" encoding="UTF-8" ?>
<taglib xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-jsptaglibrary_2_0.xsd" version="2.0">

    <description>TEM functions library</description>
    <display-name>TEM functions</display-name>
    <tlib-version>1.0</tlib-version>
    <short-name>fn</short-name>
    <uri>http://www.kuali.org/jsp/jstl/functions</uri>

    <function>
        <description>Parameterized Values Finder List!!</description>
        <name>getOptionList</name>
        <function-class>org.kuali.kfs.module.tem.web.JstlFunctions</function-class>
        <function-signature>java.util.List getOptionList(java.lang.String, java.util.Map)</function-signature>
        <example><c:forEach items="${tem-fn:getOptionList()}">></example>
    </function>
</taglib>

3. Use it in a JSP

In our JSP, we will need to define a Map to add parameters to. You saw earlier that our getOptionsList method takes a Map with parameters in it.

<jsp:useBean id="paramMap" class="java.util.HashMap" />

Then, we need to assign some value to the Map:

<c:set target="${paramMap}" property="queryDate"
                                        value="${perDiemExpense.mileageDate}" />

Above we have assigned ${perDiemExpense.mileageDate} to the queryDate parameter in our Map! Now we just need our dropdown.

<c:forEach items="${temfunc:getOptionList('org.kuali.kfs.module.tem.businessobject.options.MileageRateValuesFinder', paramMap)}" var="option">
  <c:set var="mileageSelected" value="" />
  <c:if test="${option.key} == KualiForm.document.perDiemExpenses[perDiemIndex.count - 1].mileageRateId}">
    <c:set var="mileageSelected" value="selected" />
  </c:if>
  <option value="${option.key}"${mileageSelected}>${option.label}</option>
</c:forEach>

Now you can see where getOptionList is actually used. Now we have a dropdown list of mileage rates based on the date of the mileage since mileage rates can vary by date.

SEVERE: Error listenerStart

2011-12-01T11:52:00.001-08:00

Overview

I saw on the rice mailinglist recently someone posted a question where essentially, the application dies before a logger can be initialized. Here is the result:

INFO: Deploying web application archive kr-dev.war
log4j:WARN No appenders could be found for logger (org.kuali.rice.core.web.listener.KualiInitializeListener).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Dec 1, 2011 1:35:19 PM org.apache.catalina.core.StandardContext start
SEVERE: Error listenerStart
Dec 1, 2011 1:35:19 PM org.apache.catalina.core.StandardContext start
SEVERE: Context [/kr-dev] startup failed due to previous errors

What has happened here is the listener failed to start. The listener is what starts up the application (in this case, Rice). If Rice does not start, there is no logger to tell you what is wrong. Thus, you get no error. Usually, what causes this is a dependency is not included in tomcat.

So far, I have not seen a case where the problem is caused by anything but a missing library. It is usually the database driver, because that is the only thing not included in the rice war by default. It can possibly be something else though. I have seen KFS not start because of a missing log4j file. Since KFS actually requires the application to deploy additional libraries to the application server before startup, it is the only exceptional case.

Solution

Of course, the solution is to make sure your database driver is in the tomcat/lib folder, right? What if that doesn't do the job? What if it's another library? Read on.

Alternate

Sometimes you need to figure out what library is missing. This is almost never the case. Nearly always it is the database driver that is missing. Almost, right? If this happens and the database driver is loading correctly, then you need to figure out what the problem is. This means consulting the tomcat logger. I am going to assume that you have installed the rice war file in your tomcat installation. The war file is now unpacked and there's a webapp directory somewhere. Within the WEB-INF/classes path of that webapp directory, you can create a logging.properties with the following contents.

org.apache.catalina.core.ContainerBase.[Catalina].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].handlers = java.util.logging.ConsoleHandler

Why? The reason is that rice uses commons-logging and log4j while tomcat uses java.util.logging. Essentially, separate things. Also, since Kuali did not start, it did not start the logging configuration. What we can do is add a logging configuration that tomcat will pick up and use when it tries to start Rice. That is essentially what we are doing here.

Now try to restart the server. You should see some kind of NoClassDefFoundError somewhere explaining what library you are missing.

Getting Started with Rice 2.0

2011-11-15T12:50:00.001-08:00

This is my "Getting Started with Rice 2.0" presentation

Enjoy

Current KIM Refactoring Documentation

2011-11-09T07:24:00.000-08:00

Overview

There are 2 ways to add objects to KIM.

Rice User Interface
SQL insert statements when building the application

. Since Kuali is venturing into the use of liquibase, I have put together custom refactorings that simplify adding objects to KIM and other Rice modules.

Using SQL

This is how one would normally create an attribute, create a responsibility, add the responsibility to a role, and add an attribute to a responsibility.

Example

-- International Travel Reviewer Node
insert into krim_rsp_t (rsp_id, nmspc_cd, nm, actv_ind, rsp_tmpl_id, ver_nbr, obj_id)
values(krim_rsp_id_s.nextval, 'KFS-TEM', 'Review', 'Y', 1, 1, sys_guid());


insert into krim_role_t (role_id, obj_id, ver_nbr, nmspc_cd, role_nm, kim_typ_id, actv_ind, last_updt_dt)
   values (krim_role_id_s.nextval, sys_guid(), 1, 'KFS-TEM', 'International Travel Reviewer',
   (select kim_typ_id from krim_typ_t where nm = 'Default' and nmspc_cd = 'KUALI' and actv_ind = 'Y'),
    'Y', SYSDATE);

insert into krim_rsp_attr_data_t (attr_data_id, obj_id, ver_nbr, rsp_id, kim_typ_id, kim_attr_defn_id, attr_val)
values(krim_rsp_rqrd_attr_id_s.nextval, sys_guid(), 1, krim_rsp_id_s.currval, 7, 16, 'InternationalTravelReviewer');

insert into krim_rsp_attr_data_t (attr_data_id, obj_id, ver_nbr, rsp_id, kim_typ_id, kim_attr_defn_id, attr_val)
values(krim_rsp_rqrd_attr_id_s.nextval, sys_guid(), 1, krim_rsp_id_s.currval, 7, 13, 'TA');

insert into krim_rsp_attr_data_t (attr_data_id, obj_id, ver_nbr, rsp_id, kim_typ_id, kim_attr_defn_id, attr_val)
values(krim_rsp_rqrd_attr_id_s.nextval, sys_guid(), 1, krim_rsp_id_s.currval, 7, 41, 'false');

insert into krim_rsp_attr_data_t (attr_data_id, obj_id, ver_nbr, rsp_id, kim_typ_id, kim_attr_defn_id, attr_val)
values(krim_rsp_rqrd_attr_id_s.nextval, sys_guid(), 1, krim_rsp_id_s.currval, 7, 40, 'false');

insert into krim_role_rsp_t (role_rsp_id, obj_id, ver_nbr, role_id, rsp_id, actv_ind)
values (krim_role_rsp_id_s.nextval, sys_guid(), 1, (select role_id from krim_role_t where role_nm = 'International Travel Reviewer' and nmspc_cd= 'KFS-TEM'), krim_rsp_id_s.currval, 'Y');

insert into krim_role_rsp_actn_t (role_rsp_actn_id, obj_id, ver_nbr, actn_typ_cd, priority_nbr, actn_plcy_cd, role_mbr_id, role_rsp_id, frc_actn)
   values (krim_role_rsp_actn_id_s.nextval, sys_guid(), 1, 'A', 1, 'F', '*',
   (select role_rsp_id from krim_role_rsp_t where role_id = (select role_id from krim_role_t where role_nm = 'International Travel Reviewer' and nmspc_cd = 'KFS-TEM')),
    'N');

New Liquibase Refactoring Method

I have documented a better way on the Rice Liquibase Extensions page.

Project Updates and the Coming Onslaught of Documentation

2011-11-08T07:53:00.000-08:00

I have been working on improving the documentation in the projects surrounding liquibase. That's right. The changes aren't all just for looks. A lot of documentation that is going out to wikis will be updated on the official project pages first. Also, among the documentation improvements are:

javadoc APIs
Usage instructions
Screencasts (most importantly is that they're better and more informative screencasts)
Working example (examples taken directly from implemented scenarios

Updated Projects

Here is a list of projects recently updated:

rice-lb-extensions

This is a new project that is going to focus on custom refactorings. Instead of having a bunch of SQL inserts, database specific sql, and having to keep track of your previous ID, there will be meaningful code behind your database object changes. Here are some up and coming KIM Refactorings.

New GitHub Project

2011-11-07T15:14:00.000-08:00

Overview

I've created a new project on GitHub. It is called project. What it is, is a maven skin that will make your maven site look just like the Kuali site.

Sites Used

I am currently using this skin for my rsmart-lb-ant project. Which brings me to my next update. I am currently working on improving my documentation on rsmart-lb-ant and rsmart-lb-extensions. I will be providing better examples and videos on how to use these too. Examples, are from real, working implementations.

I hope you enjoy it.

LDAP KIM Integration Documentation

2011-09-28T12:24:00.001-07:00

I have added documentation in a permanent location that will be updated and maintained for KIM LDAP Integration

Testing with lb-maven-plugin

2011-09-21T22:04:00.000-07:00

Overview

These are some instructions for developers using the lb-maven-plugin on how to test your changelogs to verify they work on varying database platforms before committing changes to your VCS.

Setup

I'm going to assume you have already installed the lb-maven-plugin. Here are some preliminary setups we need to do before actual testing:

Add lb-maven-plugin to the pom.xml
Create connection information for each database to test.
Create an actual changelog

Setup the lb-maven-plugin

This is what I added to my pom.xml. This ensures my changelogs are run whenever I do any kind of testing. Since I am testing both mysql and oracle databases, it requires the drivers during testing.

  <plugin>
    <groupId>com.rsmart.kuali.tools</groupId>
    <artifactId>lb-maven-plugin</artifactId>
    <version>0.0.1</version>
    <configuration>
      <changeLogTagUrl>https://svn.rsmart.com/svn/kuali/contribution/community/travel_module/tags/</changeLogTagUrl>
    </configuration>
    <executions>
      <execution>
        <id>test-liquibase-changelogs</id>
        <phase>test</phase>
        <goals>
          <goal>test</goal>
        </goals>
      </execution>
    </executions>
    <dependencies>
      <dependency>
        <groupId>com.rsmart.kuali.tools.liquibase</groupId>
        <artifactId>rsmart-lb-extensions</artifactId>
        <version>1.0.0</version>
        <scope>runtime</scope>
      </dependency>
      <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>${mysql.version}</version>
        <scope>runtime</scope>
      </dependency>
      <dependency>
        <groupId>com.oracle</groupId>
        <artifactId>ojdbc14</artifactId>
        <version>${oracle.version}</version>
        <scope>runtime</scope>
      </dependency>
    </dependencies>
  </plugin>

This may seem a little weird, but I also modified the resources plugin

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-resources-plugin</artifactId>
    <version>2.5</version>
    <executions>
      <execution>
        <id>copy-test-changelogs</id>
        <!-- here the phase you need -->
        <phase>validate</phase>
        <goals>
          <goal>copy-resources</goal>
        </goals>
        <configuration>
          <outputDirectory>${basedir}/target/changelogs/update</outputDirectory>
          <resources>          
            <resource>
              <directory>src/main/changelogs/update</directory>
            </resource>
          </resources>              
        </configuration>            
      </execution>
    </executions>
  </plugin>

The reason is because the lb-maven-plugin assumes by convention changelogs are located in src/main/changelogs and your updates are within the update directory. Further, the target location of the changelogs is in target/changelogs/update. The resources plugin needs to be modified to recognize this.

Setup the Liquibase Properties Files

The lb-maven-plugin regards any properties files in target/test-classes/liquibase to be considered a liquibase properties file. It will iterate over each one and run the changelog in target/changelogs/update for each one. In order to get properties files into target/test-classes/liquibase, I created a directory called src/test/resources/liquibase. These are already moved to the appropriate location. All I need to do is create the files:

driver: oracle.jdbc.driver.OracleDriver
url: jdbc:oracle:thin:@localhost:1521:KFS
username: TEM
password: TEMPORARY

driver: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/TEM
username: TEM
password: TEMPORARY

Create a Changelog

Next, I just need to make a change to test out. I'm just going to do something simple:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd">	
  <changeSet author="kuali (generated)" id="CM-156-1">
    <comment>Adding System Parameter for testing</comment>
    <sql><![CDATA[
insert into KRNS_PARM_T (NMSPC_CD, PARM_DTL_TYP_CD, PARM_NM, OBJ_ID, VER_NBR, PARM_TYP_CD, TXT, PARM_DESC_TXT, CONS_CD, APPL_NMSPC_CD) 
values ('KFS-TEM','TravelReimbursement','TEST_PARAMETER',sys_guid(),1,'CONFG','Y','System parameter to test update and rollback','A','KFS');
    
    ]]></sql>
    <rollback><![CDATA[
    delete from KRNS_PARM_T where PARM_DTL_TYP_CD = 'TravelReimbursement' AND PARM_NM = 'TEST_PARAMETER';
    ]]></rollback>
  </changeSet>
</databaseChangeLog>

I'm just adding your basic system parameter. Testing requires that I rollback the change when I'm done, so I'm forced into providing a rollback. Imagine that. Testing forcing good practices on developers. That's the way it should be.

Prepare for Testing

Now that I have created these files, I simply run the following:

mvn validate testResources

. You may recall that the changelogs are copied during the validate goal. I run testResources to copy my properties files to the appropriate locations. After doing that, I should see this in target/changelogs/update/

leo@behemoth~/.workspace/kfs/release-4-0-overlay
(21:38:19) [540] ls target/changelogs/update
CM-156.xml

I see the changelog I created. Good. I should also see my properties in target/test-classes/liquibase

leo@behemoth~/.workspace/kfs/release-4-0-overlay
(22:02:22) [541] ls target/test-classes/liquibase/
TEM.properties			TEMNIGHTLY.properties		liquibase.properties.template

There you have it. Now we're ready to test.

leo@behemoth~/.workspace/kfs/release-4-0-overlay
(21:29:10) [537] mvn validate lb:test
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building kfs 4.0M2
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.5:copy-resources (copy-test-changelogs) @ kfs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- lb-maven-plugin:0.0.1:test (default-cli) @ kfs ---
[WARNING] Artifact with no actual file, 'org.kuali.kfs:kfs'
[WARNING] Artifact with no actual file, 'commons-lang:commons-lang'
[WARNING] Artifact with no actual file, 'com.lowagie:itext'
[WARNING] Artifact with no actual file, 'jasperreports:jasperreports'
[WARNING] Artifact with no actual file, 'org.kuali.kfs:kfs'
[WARNING] Artifact with no actual file, 'mysql:mysql-connector-java'
[WARNING] Artifact with no actual file, 'junit:junit'
[WARNING] Artifact with no actual file, 'javax.servlet:servlet-api'
[WARNING] Artifact with no actual file, 'javax.servlet:jstl'
[WARNING] Artifact with no actual file, 'taglibs:standard'
[WARNING] Artifact with no actual file, 'javax.servlet:jsp-api'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-deploy'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-jsp-2.1'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-server'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-webapp'
[WARNING] Artifact with no actual file, 'org.hamcrest:hamcrest-library'
[WARNING] Artifact with no actual file, 'org.springframework:spring-beans'
[WARNING] Artifact with no actual file, 'org.springframework:spring-context'
[WARNING] Artifact with no actual file, 'org.springframework:spring-context-support'
[WARNING] Artifact with no actual file, 'org.springframework:spring-core'
[WARNING] Artifact with no actual file, 'org.springframework:spring-jdbc'
[WARNING] Artifact with no actual file, 'org.springframework:spring-tx'
[WARNING] Artifact with no actual file, 'org.springmodules:spring-modules-ojb'
[INFO] Parsing Liquibase Properties File
[INFO]   File: /Users/leo/.workspace/kfs/release-4-0-overlay/target/test-classes/liquibase/TEM.properties
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[WARNING] Artifact with no actual file, 'org.kuali.kfs:kfs'
[WARNING] Artifact with no actual file, 'commons-lang:commons-lang'
[WARNING] Artifact with no actual file, 'com.lowagie:itext'
[WARNING] Artifact with no actual file, 'jasperreports:jasperreports'
[WARNING] Artifact with no actual file, 'org.kuali.kfs:kfs'
[WARNING] Artifact with no actual file, 'mysql:mysql-connector-java'
[WARNING] Artifact with no actual file, 'junit:junit'
[WARNING] Artifact with no actual file, 'javax.servlet:servlet-api'
[WARNING] Artifact with no actual file, 'javax.servlet:jstl'
[WARNING] Artifact with no actual file, 'taglibs:standard'
[WARNING] Artifact with no actual file, 'javax.servlet:jsp-api'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-deploy'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-jsp-2.1'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-server'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-webapp'
[WARNING] Artifact with no actual file, 'org.hamcrest:hamcrest-library'
[WARNING] Artifact with no actual file, 'org.springframework:spring-beans'
[WARNING] Artifact with no actual file, 'org.springframework:spring-context'
[WARNING] Artifact with no actual file, 'org.springframework:spring-context-support'
[WARNING] Artifact with no actual file, 'org.springframework:spring-core'
[WARNING] Artifact with no actual file, 'org.springframework:spring-jdbc'
[WARNING] Artifact with no actual file, 'org.springframework:spring-tx'
[WARNING] Artifact with no actual file, 'org.springmodules:spring-modules-ojb'
[INFO] Parsing Liquibase Properties File
[INFO]   File: /Users/leo/.workspace/kfs/release-4-0-overlay/target/test-classes/liquibase/TEM.properties
[INFO] ------------------------------------------------------------------------
[INFO] Executing on Database: jdbc:mysql://localhost:3306/TEM
[INFO] Tagging the database
INFO 9/21/11 9:29 PM:liquibase: Successfully acquired change log lock
INFO 9/21/11 9:29 PM:liquibase: Reading from `DATABASECHANGELOG`
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
[INFO] Doing update
INFO 9/21/11 9:29 PM:liquibase: Successfully acquired change log lock
INFO 9/21/11 9:29 PM:liquibase: Reading from `DATABASECHANGELOG`
INFO 9/21/11 9:29 PM:liquibase: ChangeSet /Users/leo/.workspace/kfs/release-4-0-overlay/target/changelogs/update/CM-156.xml::CM-156-1::kuali (generated) ran successfully in 37ms
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
[INFO] Doing rollback
INFO 9/21/11 9:29 PM:liquibase: Successfully acquired change log lock
INFO 9/21/11 9:29 PM:liquibase: Rolling Back Changeset:/Users/leo/.workspace/kfs/release-4-0-overlay/target/changelogs/update/CM-156.xml::CM-156-1::kuali (generated)::(Checksum: 3:85a5e658332342fba8b60df3a29fc393)
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] Parsing Liquibase Properties File
[INFO]   File: /Users/leo/.workspace/kfs/release-4-0-overlay/target/test-classes/liquibase/TEMNIGHTLY.properties
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[WARNING] Artifact with no actual file, 'org.kuali.kfs:kfs'
[WARNING] Artifact with no actual file, 'commons-lang:commons-lang'
[WARNING] Artifact with no actual file, 'com.lowagie:itext'
[WARNING] Artifact with no actual file, 'jasperreports:jasperreports'
[WARNING] Artifact with no actual file, 'org.kuali.kfs:kfs'
[WARNING] Artifact with no actual file, 'mysql:mysql-connector-java'
[WARNING] Artifact with no actual file, 'junit:junit'
[WARNING] Artifact with no actual file, 'javax.servlet:servlet-api'
[WARNING] Artifact with no actual file, 'javax.servlet:jstl'
[WARNING] Artifact with no actual file, 'taglibs:standard'
[WARNING] Artifact with no actual file, 'javax.servlet:jsp-api'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-deploy'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-jsp-2.1'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-server'
[WARNING] Artifact with no actual file, 'org.eclipse.jetty:jetty-webapp'
[WARNING] Artifact with no actual file, 'org.hamcrest:hamcrest-library'
[WARNING] Artifact with no actual file, 'org.springframework:spring-beans'
[WARNING] Artifact with no actual file, 'org.springframework:spring-context'
[WARNING] Artifact with no actual file, 'org.springframework:spring-context-support'
[WARNING] Artifact with no actual file, 'org.springframework:spring-core'
[WARNING] Artifact with no actual file, 'org.springframework:spring-jdbc'
[WARNING] Artifact with no actual file, 'org.springframework:spring-tx'
[WARNING] Artifact with no actual file, 'org.springmodules:spring-modules-ojb'
[INFO] Parsing Liquibase Properties File
[INFO]   File: /Users/leo/.workspace/kfs/release-4-0-overlay/target/test-classes/liquibase/TEMNIGHTLY.properties
[INFO] ------------------------------------------------------------------------
[INFO] Executing on Database: jdbc:oracle:thin:@heisenberg.rsmart.com:1521:KFS
[INFO] Tagging the database
INFO 9/21/11 9:29 PM:liquibase: Successfully acquired change log lock
INFO 9/21/11 9:29 PM:liquibase: Reading from DATABASECHANGELOG
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
[INFO] Doing update
INFO 9/21/11 9:29 PM:liquibase: Successfully acquired change log lock
INFO 9/21/11 9:29 PM:liquibase: Reading from DATABASECHANGELOG
INFO 9/21/11 9:29 PM:liquibase: ChangeSet /Users/leo/.workspace/kfs/release-4-0-overlay/target/changelogs/update/CM-156.xml::CM-156-1::kuali (generated) ran successfully in 205ms
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
[INFO] Doing rollback
INFO 9/21/11 9:29 PM:liquibase: Successfully acquired change log lock
INFO 9/21/11 9:29 PM:liquibase: Rolling Back Changeset:/Users/leo/.workspace/kfs/release-4-0-overlay/target/changelogs/update/CM-156.xml::CM-156-1::kuali (generated)::(Checksum: 3:85a5e658332342fba8b60df3a29fc393)
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
INFO 9/21/11 9:29 PM:liquibase: Successfully released change log lock
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28.920s
[INFO] Finished at: Wed Sep 21 21:29:49 MST 2011
[INFO] Final Memory: 11M/262M
[INFO] ------------------------------------------------------------------------
leo@behemoth~/.workspace/kfs/release-4-0-overlay
(21:29:49) [538]

Conclusion

You can see that it ran against two separate databases relatively quickly. It's a good way to test your changes and make sure they work everywhere you need them to.

lb-maven-plugin: Why Another Liquibase Maven Plugin?

2011-09-20T10:33:00.000-07:00

Overview

I've started a project at github for an extension of the liquibase maven plugin. It's true that liquibase already has a maven plugin, but liquibase is so configuration intensive. I believe there needs to be a more standardized approach to developing with liquibase. Maven is a paragon for the methodology of "convention over configuration". I sincerely believe this methodology is a simpler way to go as far as configuration is concerned. With that in mind, my goal is to make development with liquibase more constrained and simpler for developers. This is especially the case in Kuali.

The Problem

You may recall from a previous blog post the structure and style of Liquibase implementation I recommend. To summarize, I recommend only publishing updates to your VCS. This makes it easier on developers and also eliminates confusion when making essentially the same change to multiple files. This approach does have its complications though. For example, it raises the question, "How do you create a database to start with?" Also, "What if an environment or a developer gets several revisions behind? How do we bring them back up to the current version?" The maven plugin basically answers those questions. Especially, for developers.

lb-maven-plugin

This plugin is an extension of the liquibase-maven-plugin, so it has all the same configuration and goals. In addition, it has 2 more goals: migrate and test.

migrate

You may recall that there used to be a migrate task in the liquibase-maven-plugin. I think they renamed it to be consistent with the CLI and Ant. I have brought it back to be more consistent with RoR migrations. The objective here is to not just update the database, but to bring the database up to the current version. Right now, liquibase update just runs whatever changelogs you want to update with. They might not be the ones that will get you to the right version. Liquibase doesn't know what changelogs to run to get you to the right version. This goal takes into consideration the structure outlined in a previous blog post, and retrieves from the SCM the relevant changelogs to update your database.

test

There's already an updateAndRollback goal. Unfortunately, this tests the update and rollback, but does another update. This is great if you want to test your script while updating your database. In development, this isn't as realistic though. Sometimes, you want to group your changes. Developers are likely to continually add to a change log, then commit it to the VCS. This is especially the case if the changes are related to the same Jira issue. If you update, you can't really keep updating. You could get around this by being really hacky with your changelogs, but it would just be better if the tool did what you expected: run the test, and then rollback to a state prior to do more testing. That's exactly what this goal does. It basically tests your changelogs out. You can define multiple databases to run it against. If you are concerned about changelog compatibility, you can run the changes against as many databases as you want (local or remote).Continue to follow this blog, and I'll post shortly an example on testing your changelogs with this tool.

Liquibase: Semi-Database Agnostic Changelogs

2011-09-16T20:44:00.000-07:00

Overview

Changelogs in Liquibase are typically RDBMS specific. For a database change management system that advertises being database agnostic, it is pretty weird to still be writing changelogs that are database specific. For example, check this out:

Above is a changelog specific to MySQL. You can tell because of the VARCHAR type used. If it were Oracle, that would probably be a VARCHAR2

This makes it a little difficult to reuse changelogs across database platforms. End up locked into one database platform which is a bit troublesome for projects that use more than one (or support more than one like Kuali).

Semi-Database Agnostic Changelogs

Not all RDBMS-specific issues can be corrected. Liquibase suggests using contexts in these kinds of scenarios. However, it is possible to directly address the field type issue. Liquibase supports generic SQL Types defined in the java.sql.Types class. Here's an example:

You start seeing java.sql.Types.VARCHAR(30) where the 30 is the length of the field. This will work in Oracle as well. The java.sql.Types.VARCHAR refers to the field in the Java API. During processing, Liquibase will substitute the native Oracle type for the java.sql type.

Generating Semi-Database Agnostic Changelogs

You may be asking right away, "Why do you want to generate changelogs? Why not just have developers maintain them?" I've tried this method before, and I found it distracting and cumbersome for developers to duplicate work on changelogs. Whenever a database is modified, then the install changelog and the update changelogs need to be modified. The install changelog is to recreate the database, and the update change log is for updating existing databases. Indeed, this does eliminate the "Master database" antipattern, but it forces some extra effort and testing. I've found this to be unnecessary. I much rather prefer and recommend sticking with the "Master database" antipattern. Just apply update scripts to your "Master" and run generate changelogs against it. Since liquibase is database agnostic, we should get the same changelog from any database and be able to apply it to any other database; therefore, it doesn't really matter which database is used as the master.

The Problem

Of course, anyone can start doing their changelogs differently to support the new types, but what about when you generate your changelogs? Well, Liquibase will just use the standard RDBMS targeted changelogs. You could just do post-processing on your changelogs and cleanup after generating.

The Solution

I have put together some Liquibase extensions to get around this. They are in a Git Hub repository. Anyone can download or check them out. You simply put them into your classpath when you run Liquibase, and it will load the extensions automagically. Among the extensions are numerous fixes for generalizing changelogs. One very prominent one is the lack of sequence support in MySQL databases. Even though MySQL does not require sequences, Kuali does use sequences. This causes conflict. It is especially the case when generating changelogs.

For examples on how to use the extensions, you can look at my other github project lbcopy. it's a database copy tool based on Liquibase. It can export/import databases as well as directly migrate from one to another.

Screencast: Database Export with Liquibase

2011-09-10T21:35:00.000-07:00

Screencast

This screencast is of me demoing a tool called lbcopy. It is a tool built with liquibase and a few liquibase extensions. The idea is a fast and simple database migration. One of the greatest advantages of this tool is its database agnostic personality. You can export from MySQL and import the schema and content directly into an Oracle database without any modification. The same goes for just about any database.

Another distinguishing concept between this tool and others is that data is exported in its native environment. Not SQL, but a database. When exporting data, an H2 or HSQLDB jar file read-only database is produced. The data can then be observed and queried using a normal database query tool like Aqua Data Studio. This gives users a much better visualization of the data vs. observing raw SQL or a CSV.

That's what I demo here. I demo how to export a database and gain access to the exported data.

Screencast: Database Migration with Liquibase

2011-09-10T15:49:00.000-07:00

Screencast

This screencast is of me demoing a tool called lbcopy. It is a tool built with liquibase and a few liquibase extensions. The idea is a fast and simple database migration. One of the greatest advantages of this tool is its database agnostic personality. You can export from MySQL and import the schema and content directly into an Oracle database without any modification. The same goes for just about any database.

Another distinguishing concept between this tool and others is that data is exported in its native environment. Not SQL, but a database. When exporting data, an H2 or HSQLDB jar file read-only database is produced. The data can then be observed and queried using a normal database query tool like Aqua Data Studio. This gives users a much better visualization of the data vs. observing raw SQL or a CSV.

There are 3 primary uses for this tool: Export, Import, and Migrate. This demo is on Migrate. Migrate is direct copy of the database from one location to another without export or import. It is just one step.

UPDATE: Kuali Days 2011 Presentation Proposals 2

2011-08-12T14:55:00.000-07:00

The following presentation is on the bubble and may possibly be dropped.

Abstact
One of the necessities implementers have is to remotely execute batch processes. Many institutions already have existing enterprise scheduling systems that are far more robust than Quartz which is shipped with KFS. The cases for this are when implementing institutions use a third-party scheduling system (Peoplesoft or BMC). Institutions certainly will want to put in place something more robust or may even already have a campus-wide scheduling system.

This is an informative talk on how to connect an enterprise scheduling system with KFS. Attendees will see examples of how to integrate using shell-scripting and web-services.
Objectives
Learn about caveats of normal shell-scripting of batch processes.
Learn of creating a simple web service that isn't published to Kuali Service Bus (KSB) for executing isolated batch processes.
Learn how to create a client for communicating with the batch Web Services Definition Language (WSDL).
Batch processes and WS-SEC with Rice and KFS

Audience
Developers on a technical

I received an email today from the Kuali Foundation letting me know there are indeed a large number of proposals this year. Some very good ones will be dropped. If you have a favorite presentation or a presentation you really want to see, please let them know at the Kuali Foundation

Struts 1, Spring, PropertyChangeEvent, and the Observer Pattern in Kuali Part 3

2011-08-12T04:01:00.000-07:00

How the DateChangedListener looks to a 3-year old with a box of crayons

Struts 1, Spring, PropertyChangeEvent, and the Observer Pattern in Kuali Part 2

2011-08-10T21:18:00.000-07:00

Overview

I am continuing the saga on this. In Part 1, I described the scenario:

For example, checkbox toggling. What if when a check box is selected 1, 2, or even 3 other boxes are deselected. Maybe a select list is modified. Perhaps a set of checkboxes are disabled after checking a box somewhere on the form. How do we handle this? The instinctive thing to do is to write some rather messy code in the Action class.

I want to revisit this. Before I do though, a little diagram on the aforementioned Observer Pattern.

Events and Listeners

Here is the problem I ran into. I needed to add a note to my document whenever the start or end date changed. Not when it was updated, but when it was changed. This is troublesome because I will have to use Spring to access my DocumentService and the change will occur via event from the UI. Further, the properties I want to watch are on the Document. All around, this spells one huge mess. Probably means I will be either calling the SpringContext.getBean() method from the Document or from my Struts class. How about I do neither?

Here's my plan. In my Struts Action, there's already a createDocument() method that calls the DocumentService. How about I reuse this in some way? Maybe instead of using the generic DocumentService, I use my own TravelDocumentService that creates the Document for me with a newDocument() method, right? How about in this newDocument() method, I inject some services as PropertyChangeListeners? That way, my Document is making calls to transient services that are injected at creation. Sound awesome? Let's make it happen.

TravelReimbursementAction

Here's my new createDocument() method.

/*

 * Copyright 2010 The Kuali Foundation.

 * 

 * Licensed under the Educational Community License, Version 1.0 (the "License");

 * you may not use this file except in compliance with the License.

 * You may obtain a copy of the License at

 * 

 * http://www.opensource.org/licenses/ecl1.php

 * 

 * Unless required by applicable law or agreed to in writing, software

 * distributed under the License is distributed on an "AS IS" BASIS,

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 * See the License for the specific language governing permissions and

 * limitations under the License.

 */

package org.kuali.kfs.module.tem.document.web.struts;

...

...

/***

 * Action methods for the {@link TravelReimbursementDocument}

 *

 * @author Leo Przybylski (leo [at] rsmart.com)

 */

public class TravelReimbursementAction extends TravelActionBase {

    ...

    ...

    /**

     * Do initialization for a new {@link TravelReimbursementDocument}

     * 

     * @see org.kuali.rice.kns.web.struts.action.KualiDocumentActionBase#createDocument(org.kuali.rice.kns.web.struts.form.KualiDocumentFormBase)

     */

    @Override

    protected void createDocument(KualiDocumentFormBase kualiDocumentFormBase) throws WorkflowException {

        super.createDocument(kualiDocumentFormBase);

        final TravelReimbursementForm travelForm = (TravelReimbursementForm) kualiDocumentFormBase;

        final TravelReimbursementDocument document = (TravelReimbursementDocument) travelForm.getDocument();

        getTravelDocumentService().addListenersTo(document);

        addContactInformation(document);

    }

    ...

    ...

}

TravelDocumentService

Ok, let's see how I inject those listeners.

/*

 * Copyright 2010 The Kuali Foundation

 * 

 * Licensed under the Educational Community License, Version 2.0 (the "License");

 * you may not use this file except in compliance with the License.

 * You may obtain a copy of the License at

 * 

 * http://www.opensource.org/licenses/ecl2.php

 * 

 * Unless required by applicable law or agreed to in writing, software

 * distributed under the License is distributed on an "AS IS" BASIS,

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 * See the License for the specific language governing permissions and

 * limitations under the License.

 */

package org.kuali.kfs.module.tem.service.impl;

...

...

/**

 * Travel Reimbursement Service Implementation

 * 

 */

public class TravelReimbursementServiceImpl implements TravelReimbursementService {

    ...

    ...

    public void addListenersTo(final TravelReimbursementDocument reimbursement) {

        reimbursement.setPropertyChangeListeners(getPropertyChangeListeners());

    }

    ...

    ...

    /**

     * Sets the propertyChangeListener attribute value.

     * 

     * @param propertyChangeListener The propertyChangeListener to set.

     */

    public void setPropertyChangeListeners(final List<PropertyChangeListener> propertyChangeListeners) {

        this.propertyChangeListeners = propertyChangeListeners;

    }



    /**

     * Gets the propertyChangeListeners attribute.

     * 

     * @return Returns the propertyChangeListenerDetailId.

     */

    public List<PropertyChangeListener> getPropertyChangeListeners() {

        return this.propertyChangeListeners;

    }

    ...

    ...

}

So where do these PropertyChangeListeners get set? Obviously, they're injected into the service, right? But how? Let's look.

spring-tem.xml

This is where we setup our TravelDocumentService to be injected with our property change listeners.

<?xml version="1.0" encoding="UTF-8"?>

<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 

	Community License, Version 2.0 (the "License"); you may not use this file 

	except in compliance with the License. You may obtain a copy of the License 

	at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 

	law or agreed to in writing, software distributed under the License is distributed 

	on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 

	express or implied. See the License for the specific language governing permissions 

	and limitations under the License. -->

<beans xmlns="http://www.springframework.org/schema/beans"

	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"

	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"

	xsi:schemaLocation="http://www.springframework.org/schema/beans

                           http://www.springframework.org/schema/beans/spring-beans-2.0.xsd

                           http://www.springframework.org/schema/tx

                           http://www.springframework.org/schema/tx/spring-tx-2.0.xsd

                           http://www.springframework.org/schema/aop

                           http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">

    ...

    ...

    <!-- Property Listeners -->

    <bean id="TravelReimbursementDocument-dateChangedListener" class="org.kuali.kfs.module.tem.document.listener.DateChangedListener">

        <property name="documentService" ref="documentService" />

    </bean>

    ...

    ...

    <bean id="temTravelDocumentService" parent="temTravelDocumentService-parentBean" />

    <bean   id="temTravelDocumentService-parentBean" 

         class="org.kuali.kfs.module.tem.service.impl.TravelDocumentServiceImpl" 

      abstract="true">

      <property name="propertyChangeListeners">

        <list>

          <ref bean="TravelReimbursementDocument-dateChangedListener" />

        </list>

      </property>

    </bean>

    ...

    ...

</beans>

You can see that there's a dateChangedEvent and it is added as a listener to the document. Great! But wait! How does it get called?

PropertyChangeEvent

In my TravelReimbursementDocument, I'm going to trigger a PropertyChangeEvent on the listener.

/*

 * Copyright 2010 The Kuali Foundation.

 * 

 * Licensed under the Educational Community License, Version 1.0 (the "License");

 * you may not use this file except in compliance with the License.

 * You may obtain a copy of the License at

 * 

 * http://www.opensource.org/licenses/ecl1.php

 * 

 * Unless required by applicable law or agreed to in writing, software

 * distributed under the License is distributed on an "AS IS" BASIS,

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 * See the License for the specific language governing permissions and

 * limitations under the License.

 */

package org.kuali.kfs.module.tem.document;

...

...

import java.beans.PropertyChangeEvent;

...

...

/**

 * Abstract Travel Document Base

 */

public abstract class TravelDocumentBase extends AccountingDocumentBase implements TravelDocument, Copyable {

    ...

    ...

    @Transient

    private List<PropertyChangeListener> propertyChangeListeners;

    ...

    ...

    /**

     * Sets the propertyChangeListener attribute value.

     * 

     * @param propertyChangeListener The propertyChangeListener to set.

     */

    public void setPropertyChangeListeners(final List<PropertyChangeListener> propertyChangeListeners) {

        this.propertyChangeListeners = propertyChangeListeners;

    }



    /**

     * Gets the propertyChangeListeners attribute.

     * 

     * @return Returns the propertyChangeListenerDetailId.

     */

    public List<PropertyChangeListener> getPropertyChangeListeners() {

        return this.propertyChangeListeners;

    }

    ...

    ...

    /**

     * This method sets the trip begin date for this request

     * 

     * @param tripBegin

     */

    public void setTripBegin(Date tripBegin) {

        notifyChangeListeners(new PropertyChangeEvent(this, "tripBegin", this.tripBegin, tripBegin));

        this.tripBegin = tripBegin;

    }

    ...

    ...

}

This is just showing the trip begin change, but the end date is pretty much identical except it is for the tripEnd. You can see that before the change occurs, we notify the listeners. The reason for this is because I will lose the changed information after the change is made, so I have to notify my listeners before. You can see I am passing in the object being modified, the old value and the new value!

DateChangedListener

Now let's look at what happens when the date is changed

/*

 * Copyright 2010 The Kuali Foundation

 * 

 * Licensed under the Educational Community License, Version 2.0 (the "License");

 * you may not use this file except in compliance with the License.

 * You may obtain a copy of the License at

 * 

 * http://www.opensource.org/licenses/ecl2.php

 * 

 * Unless required by applicable law or agreed to in writing, software

 * distributed under the License is distributed on an "AS IS" BASIS,

 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

 * See the License for the specific language governing permissions and

 * limitations under the License.

 */

package org.kuali.kfs.module.tem.document.listener;

...

...

/**

 * Executed when a trip date is modified

 * 

 * @author Leo Przybylski (leo [at] rsmart.com

 */

public class DateChangedListener implements PropertyChangeListener, java.io.Serializable {

    ...

    ...

    public void propertyChange(final PropertyChangeEvent event) {

        final TravelReimbursementDocument reimbursement = (TravelReimbursementDocument) event.getSource();

        final Date oldDate = (Date) event.getOldValue();

        final Date newDate = (Date) event.getNewValue();



        if (hasDateChanged(oldDate, newDate)) {

            notifyDateChangedOn(reimbursement, newDate, "tripBegin".equals(event.getPropertyName()));

        }

    }



    

    protected Boolean hasDateChanged(final Date oldDate, final Date newDate) {

        final SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yyyy");

        final String oldDateStr = formatter.format(oldDate);

        final String newDateStr = formatter.format(newDate);

        return oldDateStr.equals(newDateStr);

    }



    public void notifyDateChangedOn(final TravelReimbursementDocument reimbursement, final Date newDate, boolean start) throws Exception {

        final SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yyyy");

        final String origStartDateStr  = formatter.format(reimbursement.getTripBegin());

        final String origEndDateStr    = formatter.format(reimbursement.getTripEnd());

        String newStartDateStr = origStartDateStr;

        String newEndDateStr   = origEndDateStr;



        if (start) {

            newStartDateStr = formatter.format(newDate);

        }

        else {

            newEndDateStr = formatter.format(newDate);

        }



        final String noteText = String.format(DATE_CHANGED_MESSAGE, origStartDateStr, origEndDateStr, newStartDateStr, newEndDateStr);

        

        final Note noteToAdd = getDocumentService().createNoteFromDocument(reimbursement, noteText);

        getDocumentService().addNoteToDocument(reimbursement, noteToAdd);

    }



    public void setDocumentService(final DocumentService documentService) {

        this.documentService = documentService;

    }



    protected DocumentService getDocumentService() {

        return documentService;

    }

}

Conclusion

That's how you can inject services into documents that react on modifications to the Document.

Lightning IMPEX

2011-07-23T22:57:00.000-07:00

Screencast

I put together a screencast that shows how a certain branch of the impex tool is really really fast and resource friendly for KFS 3.0 installations. Here it is.

Steps

1 Get kfs-cfg-dbs

$ svn co https://test.kuali.org/svn/kfs-cfg-dbs/branches/release-3-0-1/ kfs-cfg-dbs-3-0-1

2 Get rice-cfg-dbs

$ svn co https://test.kuali.org/svn/rice-cfg-dbs/branches/rice-release-1-0-1-1-br rice-cfg-dbs-1-0-1-1

3 Get kul-cfg-dbs

$ svn co https://test.kuali.org/svn/kul-cfg-dbs/branches/kul-handle_dollar_signs_in_tables-br kul-cfg-dbs

4 Copy sample properties file

$ cp impex/impex-build.properties.sample $HOME/rice-impex-build.properties 
$ cp impex/impex-build.properties.sample $HOME/kfs-impex-build.properties

5 Copy library file

$ cd kul-cfg-dbs/impex;
$ cp kuali-impextasks.jar lib

6 Run import in rice-cfg-dbs

$ cd rice-cfg-dbs-1-0-1-1;
$ export ANT_OPTS="-Xmx2048m -XX:MaxPermSize=256m"
$ ant -f ../kul-cfg-dbs/build.xml -Dimpex.build.properties=$HOME/rice-impex-build.properties import

7 Run import in kfs-cfg-dbs

$ cd kfs-cfg-dbs-3-0-1;
$ ant -f ../kul-cfg-dbs/build.xml -Dimpex.build.properties=$HOME/kfs-impex-build.properties import

Struts 1, Spring, PropertyChangeEvent, and the Observer Pattern in Kuali

2011-07-23T22:51:00.000-07:00

Overview

There are a couple things that have plagued me as a developer on Kuali.

No abstraction layer between struts and spring
No subsystem for reacting to business object changes other than validation

There is this fantastic validation framework in KFS/Kuali software. However, validation is a passive action. It simply checks for inconsistencies and reports on them. A validation can never change the state in any way whatsoever. For example, checkbox toggling. What if when a check box is selected 1, 2, or even 3 other boxes are deselected. Maybe a select list is modified. Perhaps a set of checkboxes are disabled after checking a box somewhere on the form. How do we handle this? The instinctive thing to do is to write some rather messy code in the Action class.

I'll come back to that later. Speaking of the Action class, I have noticed that it is very difficult for me as a developer to resist putting business logic in the Action class. The Action and Form classes are intended to be part of an abstraction layer between the Struts M-V-C components. With Spring providing an IOC with services, it isn't very clear where certain logic should live. What do I do if I want as little logic in my Action and Form as possible, so that the services are doing most of the work?

The Observer Pattern

Below is how I implemented the Observer pattern for Struts to communicate with Spring and not have any coupling between SOA and MVC. Here's how I did it:

1 Define Spring Service Beans

It is easiest to just build your framework first with SOA. That way everything else can fit into place around it. I'm going to create all my Observables and Observers in Spring.

<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
 Community License, Version 2.0 (the "License"); you may not use this file 
 except in compliance with the License. You may obtain a copy of the License 
 at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
 law or agreed to in writing, software distributed under the License is distributed 
 on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
 express or implied. See the License for the specific language governing permissions 
 and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
                           http://www.springframework.org/schema/tx
                           http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
                           http://www.springframework.org/schema/aop
                           http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
    <!-- Struts Events -->
    <bean id="addOtherExpenseEvent" class="org.kuali.kfs.module.tem.document.web.struts.AddOtherExpenseEvent">
      <property name="travelReimbursementService" ref="temTravelReimbursementService" />
      <property name="ruleService"                ref="kualiRuleService" />
    </bean>
    <bean id="removeOtherExpenseEvent" class="org.kuali.kfs.module.tem.document.web.struts.RemoveOtherExpenseEvent">
    </bean>
    <bean id="addExpenseDetailEvent" class="org.kuali.kfs.module.tem.document.web.struts.AddExpenseDetailEvent">
      <property name="travelReimbursementService" ref="temTravelReimbursementService" />
      <property name="ruleService"                ref="kualiRuleService" />
    </bean>
    <bean id="removeExpenseDetail" class="org.kuali.kfs.module.tem.document.web.struts.AddExpenseDetailEvent">
    </bean>


    <!-- Struts Observable Pattern -->
    <bean class="org.kuali.kfs.module.tem.document.web.struts.TravelStrutsObservable">
      <property name="observers">
        <map>
          <entry key="addOtherExpenseLine">
            <list>
              <ref bean="addOtherExpenseEvent" />
            </list>
          </entry>
          <entry key="deleteOtherExpenseLine">
            <list>
              <ref bean="removeOtherExpenseEvent" />
            </list>
          </entry>
          <entry key="addOtherExpenseDetailLine">
            <list>
              <ref bean="addExpenseDetailEvent" />
            </list>
          </entry>
          <entry key="deleteOtherExpenseDetailLine">
            <list>
              <ref bean="removeExpenseDetailEvent" />
            </list>
          </entry>
        </map>
      </property>
    </bean>
...
...
</beans>

Each event is an Observer. The Observer is basically waiting for the Observable (by observing) to notify it. For example,

...
...
    
      
      
    
...
...

is an Observer.

The Observable is just another Spring bean/service. It has a Map of observers. The MVC action or methodToCall is mapped to the Observer. When certain button clicks happen (addOtherExpenseLine for example), the Observable will notify the appropriate Observer/Event.

2 Create the Observable

This is the interesting part. Not only do we need to create an Observable, but it needs to be flexible or dynamic enough that we aren't actually creating more work for ourselves by trying to solve this problem. Here's how I did it. I added the Observable to the TravelFormBase class. Now TravelFormBase is my base Form class that I use to store common code for all the TEM documents. This ensures that all the TEM documents are going to get the same Observable.

...
...
import java.util.Observable;
...
...
public abstract class TravelFormBase extends KualiAccountingDocumentFormBase implements TravelMvcWrapperBean {
...
...
    private Observable observable;
...
...

    /**
     * Gets the observable attribute.
     * 
     * @return Returns the observable.
     */
    public Observable getObservable() {
        return SpringContext.getBean(TravelStrutsObservable.class);
    }
...
...
}

I cut out a lot, but notice a couple things. My TravelFormBase is NOT the Observable. Just as I showed before in my Spring configuration, the TravelStrutsObservable is my Observable. I am getting that through my handy dandy SpringContext class. This is great because now I passively rely on the Observable to manage the Spring work. I only have to ever lookup one Spring bean and that is the Observable. The rest is handled through Dependency Injection!

Now let's look at that Observable.

/*
 * Copyright 2011 The Kuali Foundation
 * 
 * Licensed under the Educational Community License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 * 
 * http://www.opensource.org/licenses/ecl2.php
 * 
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.kuali.kfs.module.tem.document.web.struts;

import java.util.List;
import java.util.Map;
import java.util.Observable;
import java.util.Observer;

import org.kuali.kfs.module.tem.document.web.bean.TravelMvcWrapperBean;

/**
 *
 * @author Leo Przybylski (leo [at] rsmart.com)
 */
public class TravelStrutsObservable extends Observable {
    public Map<String, List<Observer>> observers;
 
    /**
     * deprecating this since the best practice is to use Spring
     */
    @Deprecated
    public void addObserver(final Observer observer) {      
        super.addObserver(observer);
    }

    public void notifyObservers(final Object arg) {
        TravelMvcWrapperBean wrapper = null;
        if (arg instanceof TravelMvcWrapperBean) {
            wrapper = (TravelMvcWrapperBean) arg;
        }

        final String eventName = wrapper.getMethodToCall();
        for (final Observer observer : getObservers().get(eventName)) {
            observer.update(this, wrapper);
        }
        clearChanged();
    }

    /**
     * Gets the observers attribute.
     * 
     * @return Returns the observers.
     */
    public Map<String, List<Observer>> getObservers() {
        return observers;
    }

    /**
     * Sets the observers attribute value.
     * 
     * @param observers The observers to set.
     */
    public void setObservers(final Map<String,List<Observer>> observers) {
        this.observers = observers;
    }
}

What? That's it?! Yeah, that's it. You can see that notifyObservers() is overridden to iterate through the observer map and notifies the Observer mapped to the methodToCall on the wrapper.

3 TravelMvcWrapperBean

You may be wondering what in the world that TravelMvcWrapperBean is. You may recall that one of the goals is to abstract the MVC layer. I don't want further coupling to Struts. Even though I have a TravelStrutsObservable, that's about as deep as the coupling gets. Actually, you can probably tell from the Observable code that besides the name, nothing really couples that to Struts. Not even the getMethodToCall(). Everything is abstracted by this new TravelMvcWrapperBean which is just an interface.

/*
 * Copyright 2010 The Kuali Foundation.
 * 
 * Licensed under the Educational Community License, Version 1.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 * 
 * http://www.opensource.org/licenses/ecl1.php
 * 
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.kuali.kfs.module.tem.document.web.bean;

import java.util.Collection;
import java.util.Map;
import java.util.List;

import org.kuali.rice.kns.bo.Note;
import org.kuali.rice.kns.document.Document;
import org.kuali.rice.kns.web.ui.ExtraButton;
import org.kuali.kfs.module.tem.document.TravelDocument;

public interface TravelMvcWrapperBean {

    Integer getTravelerId();

    TravelDocument getTravelDocument();


    void setTravelerId(Integer travelerId);


    Integer getTempTravelerId();


    void setTempTravelerId(Integer tempTravelerId);


    /**
     * Gets the empPrincipalId attribute.
     * 
     * @return Returns the empPrincipalId.
     */
    String getEmpPrincipalId();


    /**
     * Sets the empPrincipalId attribute value.
     * 
     * @param empPrincipalId The empPrincipalId to set.
     */
    void setEmpPrincipalId(String empPrincipalId);


    /**
     * Gets the tempEmpPrincipalId attribute.
     * 
     * @return Returns the tempEmpPrincipalId.
     */
    String getTempEmpPrincipalId();


    /**
     * Sets the tempEmpPrincipalId attribute value.
     * 
     * @param tempEmpPrincipalId The tempEmpPrincipalId to set.
     */
    void setTempEmpPrincipalId(String tempEmpPrincipalId);


    Map<String, String> getModesOfTransportation();

    /**
     * Gets the showLodging attribute.
     * 
     * @return Returns the showLodging.
     */
    boolean isShowLodging();


    /**
     * Sets the showLodging attribute value.
     * 
     * @param showLodging The showLodging to set.
     */
    void setShowLodging(boolean showLodging);


    /**
     * Gets the showMileage attribute.
     * 
     * @return Returns the showMileage.
     */
    boolean isShowMileage();


    /**
     * Sets the showMileage attribute value.
     * 
     * @param showMileage The showMileage to set.
     */
    void setShowMileage(boolean showMileage);


    /**
     * Gets the showPerDiem attribute.
     * 
     * @return Returns the showPerDiem.
     */
    boolean isShowPerDiem();


    /**
     * Gets the canReturn attribute value.
     * 
     * @return canReturn The canReturn to set.
     */
    boolean canReturn();

    /**
     * Sets the canReturn attribute value.
     * 
     * @param canReturn The canReturn to set.
     */
    void setCanReturn(final boolean canReturn);

    /**
     * Sets the showPerDiem attribute value.
     * 
     * @param showPerDiem The showPerDiem to set.
     */
    void setShowPerDiem(boolean showPerDiem);

    boolean isShowAllPerDiemCategories();


    /**
     * This method takes a string parameter from the db and converts it to an int suitable for using in our calculations
     * 
     * @param perDiemPercentage
     */
    void setPerDiemPercentage(String perDiemPercentage);


    /**
     * Gets the perDiemPercentage attribute.
     * 
     * @return Returns the perDiemPercentage.
     */
    int getPerDiemPercentage();


    /**
     * Sets the perDiemPercentage attribute value.
     * 
     * @param perDiemPercentage The perDiemPercentage to set.
     */
    void setPerDiemPercentage(int perDiemPercentage);

    Map<String, List<Document>> getRelatedDocuments();

    void setRelatedDocuments(Map<String, List<Document>> relatedDocuments);


    /**
     * Gets the relatedDocumentNotes attribute.
     * 
     * @return Returns the relatedDocumentNotes.
     */
    Map<String, List<Note>> getRelatedDocumentNotes();


    /**
     * Sets the relatedDocumentNotes attribute value.
     * 
     * @param relatedDocumentNotes The relatedDocumentNotes to set.
     */
    void setRelatedDocumentNotes(Map<String, List<Note>> relatedDocumentNotes);

    boolean isCalculated();

    void setCalculated(boolean calculated);

    List<ExtraButton> getExtraButtons();

    String getMethodToCall();
}

The interface is implemented by my TravelFormBase that I showed you earlier.

...
...
public abstract class TravelFormBase extends KualiAccountingDocumentFormBase implements TravelMvcWrapperBean {
...
...
}

This means that when you see the TravelMvcWrapperBean referenced, it's really the struts Form class. Keep that in mind as we delve further in.

4 Observers

Now we discuss what those events were all about in my Spring configuration. Those events are my Observers. They get triggered by the Observable. I will show you how. In a normal request, the Controller hands the Form off to the Action and calls the execute() method. It has already set the methodToCall on the form, so execute() will use this to delegate further in the Action. If my methodToCall is addOtherExpenseLine, then the addOtherExpenseLine() method is called. Here

...
...
    /**
     * Action method for adding an {@link OtherExpense} instance to the {@link TravelReimbursementDocument}
     * 
     * @param mapping
     * @param form
     * @param request
     * @param response
     * @return
     * @throws Exception
     */
    public ActionForward addOtherExpenseLine(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception {
        final TravelReimbursementForm reimbForm = (TravelReimbursementForm) form;
        final TravelReimbursementMvcWrapperBean mvcWrapper = newMvcDelegate(form);
        reimbForm.getObservable().notifyObservers(mvcWrapper);

        return mapping.findForward(KFSConstants.MAPPING_BASIC);
    }
...
...

Here you can see that it gets the Observable from the Form and then passes in the TravelReimbursementMvcWrapperBean which is really the Form. Why not just pass in the Form? Well, that would couple us to Struts. I will illustrate that later on. From what we already know, the Observable will examine the wrapper for its methodToCall, and grab the appropriate Observer(s) for it. It then calls update on them. This is what happens for the AddOtherExpenseEvent

...
...
    public void update(final Observable observable, Object arg) { 
        if (!(arg instanceof TravelReimbursementMvcWrapperBean)) {
            return;
        }
        final TravelReimbursementMvcWrapperBean wrapper = (TravelReimbursementMvcWrapperBean) arg;

        final TravelReimbursementDocument document = wrapper.getTravelReimbursementDocument();
        final ReimbursementOtherExpense newOtherExpenseLine = wrapper.getNewOtherExpenseLine();
        newOtherExpenseLine.refreshReferenceObject("travelExpenseTypeCode");

        getTravelReimbursementService().handleNewOtherExpense(newOtherExpenseLine);            

        boolean rulePassed = true;

        // check any business rules
        rulePassed &= getRuleService().applyRules(new AddOtherExpenseLineEvent(NEW_OTHER_EXPENSE_LINE, document, newOtherExpenseLine));

        if (rulePassed) {
            document.addOtherExpense(newOtherExpenseLine);
            final OtherExpenseDetail newDetail = new OtherExpenseDetail();
            newDetail.setExpenseDate(newOtherExpenseLine.getExpenseDate());
            wrapper.getNewOtherExpenseDetailLines().add(newDetail);
            wrapper.setNewOtherExpenseLine(new ReimbursementOtherExpense());
            wrapper.getNewOtherExpenseLine().setDetails(new ArrayList());
        }

        wrapper.setCalculated(false);
    }
...
...

You can see there is no mention of Struts anywhere in this Spring bean. It uses 2 services that are injected (KualiRuleService and TravelReimbursementService), so it never has to call SpringContext to get to those. You can see it uses the TravelReimbursementMvcWrapperBean as wrapper to communicate with the Form object without knowing or caring that is a Form.

5 Proxying the Form

Why not just use the Form? Why does the Action class have to lookup an MVC Delegate and pass that as the wrapper? This is to further abstract away from Struts. Instead of calling the Struts object directly, a proxy is made on it. I added the following method to my TravelActionBase so that all my Action classes would have access to it

...
...
    public <T> T newMvcDelegate(final ActionForm form) throws Exception {
        T retval = (T) Proxy.newProxyInstance(getClass().getClassLoader(),
                                              new Class[] { getMvcWrapperInterface() },
                                              new TravelMvcWrapperInvocationHandler(form));
        return retval;
    }
...
...
    /**
     * Just a passthru {@link InvocationHandler}. It's used when creating a proxy, to access methods in a class
     * without knowing what that class really is. This allows us to put a facade layer on top of whatever MVC we use;
     * hence, the name {@link TravelMvcWrapperInvocationHandler}
     *
     * @author Leo Przybylski leo [at] rsmart.com
     */
    class TravelMvcWrapperInvocationHandler<MvcClass> implements InvocationHandler {
        private MvcClass mvcObj;

        public TravelMvcWrapperInvocationHandler(final MvcClass mvcObj) {
            this.mvcObj = mvcObj;
        }        
        
        public Object invoke(final Object proxy, final Method method, final Object[] args) throws Exception {
            return method.invoke(mvcObj, args);
        }
    }
...
...

This will allow me to make changes to my Form that normally would not be acceptable or whatever.

PropertyChangeEvent

I think I will save this for another post.

Conclusion

That's how you can abstract your code out of Struts and add more to your SOA. This will of course make your code easier to unit test. It will also strip any non-UI related logic from your Action.

UPDATE: Kuali Days 2011 Presentation Proposals

2011-07-14T17:06:00.001-07:00

"The Status is Not Quo!" JSR-286 and Rice has been curbed. I and a few others may try to put together an informal meetup regarding this, but it was not accepted as a presentation. I will definitely add it again for next year.

Thanks everyone for the feedback!

Kuali Days 2011 Presentation Proposals

2011-07-13T10:09:00.001-07:00

Overview

Below are proposals I am submitting for Kuali Days. Please vote in a comment!

The Impact of Dev Ops on Kuali Development and Hosting

Abstract

There is a movement occurring acrossed the OSS universe. It is DEVOPS. The concept is where developers are also the operational staff. They become basically swiss-army knives of software. In organizations spread out remotely, it can be difficult for developers to interact with support staff. Timezones can produce bottlenecks that delay projects. DEVOPS basically come out of necessity.

Objectives

Implementations will see how to utilize DEVOPS in their organization.
Developers will be discover how to use their skills at their organization to improve development productivity.
SAAS systems hugely benefit from DEVOPS to provide the best possible uptime for your hosted software. Implementors and will see how SAAS solutions are more reliable, cost-effective, and secure because of utilizing DEVOPS.

Audience

Implementation Track
Developers
Project Managers

The Status is Not Quo! JSR-286 and Rice

Abstract

JSR-286 is the 2.0 portlet specification. This is a walkthrough of implementing web services as a portlet.

Objectives

Developers will see what goes into producing a portlet implementation
Developers will learn/understand how to create a web services client in java to communicate with Kuali Rice
Developers will learn how to deploy a portlet to a portlet container

Audience

Technical track for developers

Rice 1.1.1(2.0) Project: From the Cradle to the Grave

Abstract

A major version update for Kuali Rice is released, but not much is known about implementing it. There will undoubtably be numerous panels and presentations on taking advantage of the new version’s greatest features; however, this will be the one presentation that shows how to set up a new project, prepare it for Continuous Integration and deployment with use cases and best practices. This presentation overviews the new version project setup start to end.

Objectives

Best practices for setting up your SVN project to pull in your rice codebase
Best practices for structuring your rice project
Setting up a strong Continuous Integration environment for development
Developing with overlays
Deploying to tomcat
Deploying to jetty
Maven best practices

Audience

Technical Rice track for developers and configuration managers

Scripting Batch Processing with Web Services

Abstact

One of the necessities implementers have is to remotely execute batch processes. Many institutions already have existing enterprise scheduling systems that are far more robust than Quartz which is shipped with KFS. The cases for this are when implementing institutions use a third-party scheduling system (Peoplesoft or BMC). Institutions certainly will want to put in place something more robust or may even already have a campus-wide scheduling system.

This is an informative talk on how to connect an enterprise scheduling system with KFS. Attendees will see examples of how to integrate using shell-scripting and web-services.

Objectives

Learn about caveats of normal shell-scripting of batch processes.
Learn of creating a simple web service that isn't published to Kuali Service Bus (KSB) for executing isolated batch processes.
Learn how to create a client for communicating with the batch Web Services Definition Language (WSDL).
Batch processes and WS-SEC with Rice and KFS

Audience

Developers on a technical

Database Change Migrations with Liquibase

Abstact

Database changes via DDL/DML can be difficult to manage. Much of the time, DDL changes will be unreversable. For example, a table rename may require a table drop and recreate. This is usually favorable because then developers can ignore the proprietary language nature of SQL across RDBMS. Liquibase provides a common language and method supporting almost every database there is. Liquibase can also provide a common methodology for applying updates from the Kuali Foundation across different software systems. This can be very useful in apply upgrades from the Kuali Foundation or managing changes for your local institution.

This session covers intuitive and simple database change management process and how to integrate it with existing data migration, change management, and foundation KC, KFS and Rice updates.

Objectives

Implementors will learn best practices for structuring their projects
Implementors will learn how to accept database changes from the foundation and structure theirs around
Developers will learn how to test changes and the effects of change management within their development process
Implementors will learn how to rollback software versions including database changes
Implementors will learn how to play/fast-forward changes acrossed several revisions to update to a later version of database and source code
Business Intelligence Analysts will learn how to integrate database change management and ETL processes so that source code changes have minimal impact upon their conversion processes
Introduce a common methodology for apply upgrades across projects (KC, KFS, and Rice) using Maven and Liquibase

Audience

Technical track for DBA’s, developers, and DevOps

UPDATE (ATTN: Eclipse Users!): Hot Code Replacement for KFS Development With Tomcat

2011-06-27T16:52:00.000-07:00

Ok. A lot of Eclipse users out there are having a lot of trouble. They are encountering a lot of errors like this

"Access restriction: Class is not accessible due to restriction on required library"

. The best way to get around this is to go to Window->Preferences->Java->Compiler. A dialog will appear. Make sure yours looks like this Forbidden reference (access rules) needs to be set to Warning. That will solve your problem and you will be able to use Hot Code Replacement for KFS

Hot Code Replacement for KFS Development With Tomcat

2011-06-26T21:51:00.000-07:00

Overview

This is not hot deployment as in reloadable="true" like here:

<Context path="/kfs-dev" reloadable="true"  docBase="/Users/leo/.workspace/kfs/release-4-0-overlay/target/kfs-dev">

No no. That will force the webapp context to reload which is the trouble with KFS in tomcat. When the context reloads there are often memory problems. Worst of all, Spring has to reload everything. That's just a joy and a pleasure to experience for all developers too. Not only do you wait forever and a day for Spring to reload your DD, but then it finishes to remind you there's a memory leak with context redeployment. Pleasant, isn't it?

Solution 1: Use Maven

Honestly, the best thing you can do is this Convert Your KFS Distribution to Maven. You can then use the much coveted Tomcat Maven Plugin oooor the lesser Jetty Maven Plugin. These will support not only hot redeploys, but Hot Code Replacement. You may even avoid Spring reloading.

Solution 2: JDI (Java Debugging Interface)

There may be some of you that would rather not be bothered with converting to maven. You may just not be ready yet, but you love the idea of Hot Code Replacement because you're a developer and developers hate 2 things

Waiting around
Waiting around when they don't need to.

What's a developer to do? Brilliantly, before Sun was overtaken by the evil empire (Oracle), they left us with a few magical treasures to save our dying world from the evil that is upon us poised to deliver its reign of destruction and bind and afflict us. One is JDI (Java Debugging Interface) or JPDA (Java Platform Debugger Architecture) as it is sometimes referred. A veritable sword of omens as you will soon find out.

This is actually how Eclipse handles its Hot Code Replacement in debug mode. It's actually using JDI/JPDA. Given is that this will run slower than normal applications. It is also Debug mode. That is, you would never ever ever ever use this in production...ever. This is a development tool. The great part is that since Eclipse is also using JDI, that means that attaching a debugger or starting Eclipse in debug mode works seamlessly with this technique.

In a Nutshell

The simple explanation for my solution is:

Create a Quartz Job (WebappWatchJob) with a step (WebappWatchStep)
Set the job to run every 30 seconds
Add necessary configuration to directory.properties and configuration.properties so this only shows up when dev.mode is true
Implement WebappWatchJob to traverse your webapp installation path for .class files and check for last modified updates to reload the class on change

...in a nutshell.

The Details and Steps

Deep breath...now go!

1. Setup Properties

In configuration.properties I added

##############################################################################################################
## Properties from institutional.configuration.file (${institutional.configuration.file}) are appended after this point.
##############################################################################################################
webapp.classes.directory=${webapp.classes.directory}

I then set in directory.properties

# Application server directories - these assume the Tomcat 5.5 structure
tomcat.version=5
#appserver.lib.dir=${appserver.home}/common/lib
#appserver.classes.dir=${appserver.home}/common/classes
appserver.deploy.dir=${appserver.home}/webapps
appserver.config.dir=${appserver.home}/conf
appserver.localhost.dir=${appserver.config.dir}/Catalina/localhost
appserver.work.dir=${appserver.home}/work/Catalina/localhost
webapp.classes.directory=${appserver.deploy.dir}/${project.name}-${build.environment}/WEB-INF/classes

The above defines a property we will eventually use to set which path to watch for changes in.

2. Create WebappWatchJob

I add the following to my spring configuration. In KFS, my module is tem, so I add it to my spring-tem.xml:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
 Community License, Version 2.0 (the "License"); you may not use this file 
 except in compliance with the License. You may obtain a copy of the License 
 at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
 law or agreed to in writing, software distributed under the License is distributed 
 on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
 express or implied. See the License for the specific language governing permissions 
 and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
                           http://www.springframework.org/schema/tx
                           http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
                           http://www.springframework.org/schema/aop
                           http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<property name="jobNames">
 <list>
   <value>webappWatchJob</value>
 </list>
</property>
<property name="triggerNames">
  <list>
 <value>webappWatchTrigger</value>
  </list>
</property>
...
...
</beans>

Clearly, I have declared a webappWatchJob and a webappWatchTrigger. I later define those in the same file this way:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
 Community License, Version 2.0 (the "License"); you may not use this file 
 except in compliance with the License. You may obtain a copy of the License 
 at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
 law or agreed to in writing, software distributed under the License is distributed 
 on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
 express or implied. See the License for the specific language governing permissions 
 and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
                           http://www.springframework.org/schema/tx
                           http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
                           http://www.springframework.org/schema/aop
                           http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<bean id="webappWatchStep" class="org.kuali.kfs.module.tem.batch.WebappWatchStep" parent="step">
  <property name="webappPath" value="${webapp.classes.directory}" />
</bean>

<bean id="webappWatchJob" parent="scheduledJobDescriptor">
  <property name="steps">
    <list>
      <ref bean="webappWatchStep" />
    </list>
  </property>
</bean>

<bean id="webappWatchTrigger" parent="cronTrigger">
  <property name="jobName" value="webappWatchJob" />
  <property name="cronExpression" value="0/30 * * * * ?" />
</bean>
</beans>

Notice that our property we created before is used here. It is injected into the step so that it may be watched. Also, the cronExpression is set for every 0 and 30th second or every 30 seconds.

3. Implement WebappWatchStep

Now that we have the spring definitions setup, we need to implement the WebappWatchStep. I will first explain its pieces and then paste the full class implementation.

Properties

There are just 2 properties.

...
... 
    /**
     * Gets the value of WebappPath
     *
     * @return the value of WebappPath
     */
    public String getWebappPath() {
        return this.webappPath;
    }
    
    /**
     * Sets the value of WebappPath
     *
     * @param argWebappPath Value to assign to this.WebappPath
     */
    public void setWebappPath(final String argWebappPath) {
        this.webappPath = argWebappPath;
    }

    /**
     * Gets the value of LastUpdated
     *
     * @return the value of LastUpdated
     */
    public Date getLastUpdated() {
        return this.lastUpdated;
    }
    
    /**
     * Sets the value of LastUpdated
     *
     * @param argLastUpdated Value to assign to this.LastUpdated
     */
    public void setLastUpdated(final Date argLastUpdated) {
        this.lastUpdated = argLastUpdated;
    }
...
...

webappPath - is the path injected to define what directory to recurse and watch
lastUpdated - is a cached date of the last run. This is compared to the file lastModified property to determine whether the file has changed or not

Traversing webappPath

...
...
    /**
     * @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
     */
    public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
        info("WebappWatch triggered. I'm going in");
        final Collection<File> classFiles = new ArrayList<File>();
        traverseForClasses(new File(getWebappPath()), classFiles);
    ...
    ...
    }
...
...
    /**
     * Recursively searches for classes. If <code>file</code> is a directory, it traverses recursively. If it is a
     * file, then it compares the <code>lastModified</code> date with the cached {@link #getLastModified()} value.
     * 
     * @param file {@link File} instance for traversing
     * @param classFiles is a {@link Collection} of {@link File} instances where the {@link File#lastUpdated()} value is
     * before {@link #getLastUpdated()}
     */
    protected void traverseForClasses(final File file, final Collection<File> classFiles) {
        if (file.isDirectory()) {
            final String[] fileNames = file.list();
            
            for (final String fileName : fileNames) {
                traverseForClasses(new File(file, fileName), classFiles);
            }
        }
        else if (new Date(file.lastModified()).after(lastUpdated) && file.getName().endsWith(".class")) {
            classFiles.add(file);
        }
    }
...    
...

A classFiles Collection is passed into a method that recursively adds class names to that collection. It is later iterated over in the execute method like this:

...
...
        try {
            for (final File classFile : classFiles) {
                final byte[] classData = new byte[new Long(classFile.length()).intValue()];
                FileInputStream classStream = null;
                try {
                    classStream = new FileInputStream(classFile);
                    
                    classStream.read(classData, 0, classData.length);
                }
                catch (Exception e) {
                }
                finally {
                    if (classStream != null) {
                        try {
                            classStream.close();
                        }
                        catch (Exception e) { } // failure to close. Do nothing.
                    }
                }
                
                final String className = classFromFileName(classFile.getAbsolutePath()); 
                boolean classExists = false;
                try {
                    Class.forName(className);
                    classExists = true;
                }
                catch (ClassNotFoundException cnfe) {
                }
                
                if (classExists) {
                    addRedefineClass(toRedefineMap, className, classData);
                }
                else {
                    defineClass(className, classData, 0, classData.length);
                }
            }
            
            debug("Delivering the redefinition map");
            vm.redefineClasses(toRedefineMap);
        }
        catch (Exception e) {
            warn(e.getMessage());
            e.printStackTrace();
            Throwable cause = e.getCause();
            while (cause != null) {
                cause.printStackTrace();
                warn("Caused by: ");
                cause = cause.getCause();
            }
        }
        finally {
            try {
                vm.resume();
                if (conn != null && conn.isOpen()) {
                    debug("Closing the JPDA connection...");
                    conn.close();
                    debug("Connection closed");
                }
            }
            catch (Exception e) {
                e.printStackTrace();
            }
        }
...
...

The Juicy Part

So what is this stuff we saw in the execute method?

...
...
    /**
     * @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
     */
    public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
    ...
    ...
        try {
            conn = openJdiConnection();
            vm = Bootstrap.virtualMachineManager().createVirtualMachine(conn);
        }
        catch (Exception e) {
            return false;
        }
    ...
    ...
            try {
                vm.resume();
                if (conn != null && conn.isOpen()) {
                    debug("Closing the JPDA connection...");
                    conn.close();
                    debug("Connection closed");
                }
            }
            catch (Exception e) {
                e.printStackTrace();
            }
    ...
    ...
    }
...
...

Near the beginning of the execute method, we establish a connection with JPDA using the openJdiConnection described here

...
...
    /**
     * Creates a {@link SocketTransportService} instance and uses that to attach to localhost:8080 and return a {@link Connection} to the 
     * current running JVM instance. This is used by bootstrap to create a {@link VirtualMachine} instance.
     * 
     * @throws Exception when there's a problem attaching to localhost:8080 or if it cannot create an instance of {@link SocketTransportService}
     * @return an open {@link Connection} to the JPDA instance of the current running JVM
     * @see com.sun.jdi.VirtualMachineManager#createVirtualMachine(Connection)
     * @see com.sun.tools.jdi.SocketTransportService
     * @see com.sun.jdi.Bootstrap#virtualMachineManager()
     */
    protected Connection openJdiConnection() throws Exception {
        TransportService ts = null;
        try {
            Class c = Class.forName("com.sun.tools.jdi.SocketTransportService");
            ts = (TransportService)c.newInstance();
        } catch (Exception x) {
            throw new Error(x);
        }
        
        return ts.attach("localhost:8000", 5000, 5000);
    }
...
...

The builtin com.sun.tools.jdi.SocketTransportService is used to connect to the JPDA. I've set it to default to port 8000. I believe that's a standard, but you may want to customize this further. Tomcat uses the following environment variables to define this. It may be leveraged here.

#   JPDA_TRANSPORT  (Optional) JPDA transport used when the "jpda start"
#                   command is executed. The default is "dt_socket".
#
#   JPDA_ADDRESS    (Optional) Java runtime options used when the "jpda start"
#                   command is executed. The default is 8000.
#
#   JPDA_SUSPEND    (Optional) Java runtime options used when the "jpda start"
#                   command is executed. Specifies whether JVM should suspend
#                   execution immediately after startup. Default is "n".
#
#   JPDA_OPTS       (Optional) Java runtime options used when the "jpda start"
#                   command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,
#                   and JPDA_SUSPEND are ignored. Thus, all required jpda
#                   options MUST be specified. The default is:
#
#                   -Xdebug -Xrunjdwp:transport=$JPDA_TRANSPORT,
#                       address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND
#

Once the connection is made, it eventually also needs to be closed

...
...
            try {
                vm.resume();
                if (conn != null && conn.isOpen()) {
                    debug("Closing the JPDA connection...");
                    conn.close();
                    debug("Connection closed");
                }
            }
            catch (Exception e) {
                e.printStackTrace();
            }
...
...

You can see that right before the connection is closed, the virtual machine is resumed with the resume method. This is in case some process caused the vm to become suspended. It resumes all threads before disconnecting.

While looping over the modified class names, we ran into this

...
...
                final String className = classFromFileName(classFile.getAbsolutePath()); 
                boolean classExists = false;
                try {
                    Class.forName(className);
                    classExists = true;
                }
                catch (ClassNotFoundException cnfe) {
                }
                
                if (classExists) {
                    addRedefineClass(toRedefineMap, className, classData);
                }
                else {
                    defineClass(className, classData, 0, classData.length);
                }
...
...

Simply put, we use a method defineClass for classes that are new and addRedefineClass for methods that are already loaded with the current ClassLoader. Loading a class is easy reflection with the ClassLoader, so I am going to skip that. The tough part is handling classes that are already loaded. This is where JPDA comes in. Normally, you can only define a class once per ClassLoader, but with JDPA, you can use the redefineClasses method in the VirtualMachine. This redefineClasses method takes a Map though. This means we do not typically redefine a class at a time. We have to round them all up into a Map, and then call redefineClasses on the map. To do this, we created a addRedefineClass method which takes our already created Map, and puts classes into it. Let's have a look


...
...
    /**
     * Populates a map of {@link ReferenceType} and {@link byte[]} data. This {@link Map} is used by JDI for 
     * redefining classes.
     *
     * @param toRedefineMap {@link Map} of {@link byte[]} mapped by {@link ReferenceType}. This map is populated by the method, so it shouldn't be null.
     * @see com.sun.jdi.VirtualMachine#redefineClasses(Map, byte[])
     */
    protected void addRedefineClass(final Map<ReferenceType,byte[]> toRedefineMap, final String className, final byte[] b) throws Exception { 
        debug("Found change in ", className);
        for (ReferenceType ref : vm.classesByName(className)) {
            if (ref.name().equals(className) && className.indexOf("WebappWatch") < 0) {
                debug("redefining ", className);
                toRedefineMap.put(ref, b);
                return;
            }
        }
    }
...
...

Since I made vm a local variable in the WebappWatchStep, I can call it from addRedefineClass. I use it for its nifty classesByName method. I pass it my fully qualified class name, and it returns a list of ReferenceType instances. Typically, this will return only one class. Actually, I cannot think of any case where it wouldn't, but just to be sure, I double check that. I also omit the WebappWatchStep class. We wouldn't want to reload the class we're currently accessing VirtualMachine from. That would be bad. We simply add the ReferenceType as a key to the byte array representing the actual class. This will get used later by the VirtualMachine#redefineClasses method. For more information on ReferenceType, check out http://download.oracle.com/javase/6/docs/jdk/api/jpda/jdi/com/sun/jdi/ReferenceType.html

As promised

Here is the full source code

/*
 * Copyright 2007 The Kuali Foundation
 * 
 * Licensed under the Educational Community License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 * 
 * http://www.opensource.org/licenses/ecl2.php
 * 
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package org.kuali.kfs.module.tem.batch;

import java.lang.reflect.Method;

import java.io.File;
import java.io.FileInputStream;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

import com.sun.jdi.Bootstrap;
import com.sun.jdi.ArrayType;
import com.sun.jdi.ClassType;
import com.sun.jdi.ReferenceType;
import com.sun.jdi.InterfaceType;
import com.sun.jdi.VirtualMachine;
import com.sun.jdi.connect.spi.Connection;
import com.sun.jdi.connect.spi.TransportService;

import org.kuali.kfs.sys.batch.AbstractStep;

import static org.kuali.kfs.module.tem.util.BufferedLogger.*;
import static org.apache.commons.lang.StringUtils.replace;

/**
 * Batch step that traverses the webapp for changed classes. It then reloads the classes into the current {@link ClassLoader}
 * this gives to appearance of hot redeploys.
 *
 * @author leo [at] rsmart.com
 */
public class WebappWatchStep extends AbstractStep {
    private Date lastUpdated;
    private String webappPath;
    private Connection conn;
    private VirtualMachine vm;

    public WebappWatchStep() throws Throwable {
        super();
        lastUpdated = new Date();
    }

    /**
     * @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
     */
    public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
        info("WebappWatch triggered. I'm going in");
        final Collection<File> classFiles = new ArrayList<File>();
        traverseForClasses(new File(getWebappPath()), classFiles);
        final Map<ReferenceType,byte[]> toRedefineMap = new HashMap<ReferenceType,byte[]>();

        try {
            conn = openJdiConnection();
            vm = Bootstrap.virtualMachineManager().createVirtualMachine(conn);
        }
        catch (Exception e) {
            return false;
        }

        try {
            for (final File classFile : classFiles) {
                final byte[] classData = new byte[new Long(classFile.length()).intValue()];
                FileInputStream classStream = null;
                try {
                    classStream = new FileInputStream(classFile);
                    
                    classStream.read(classData, 0, classData.length);
                }
                catch (Exception e) {
                }
                finally {
                    if (classStream != null) {
                        try {
                            classStream.close();
                        }
                        catch (Exception e) { } // failure to close. Do nothing.
                    }
                }
                
                final String className = classFromFileName(classFile.getAbsolutePath()); 
                boolean classExists = false;
                try {
                    Class.forName(className);
                    classExists = true;
                }
                catch (ClassNotFoundException cnfe) {
                }
                
                if (classExists) {
                    addRedefineClass(toRedefineMap, className, classData);
                }
                else {
                    defineClass(className, classData, 0, classData.length);
                }
            }
            
            debug("Delivering the redefinition map");
            vm.redefineClasses(toRedefineMap);
        }
        catch (Exception e) {
            warn(e.getMessage());
            e.printStackTrace();
            Throwable cause = e.getCause();
            while (cause != null) {
                cause.printStackTrace();
                warn("Caused by: ");
                cause = cause.getCause();
            }
        }
        finally {
            try {
                vm.resume();
                if (conn != null && conn.isOpen()) {
                    debug("Closing the JPDA connection...");
                    conn.close();
                    debug("Connection closed");
                }
            }
            catch (Exception e) {
                e.printStackTrace();
            }
        }
        
        setLastUpdated(new Date());
        return false;
    }

    /**
     * Defines a class. Only used for new classes that don't already exist in the {@link ClassLoader}
     * @param className name of the {@link Class} to define
     * @param b byte array of data
     * @param offset where in the byte array to start
     * @param len size of the byte array to read
     * @throws Exception <code>defineClass()</code> on the {@link ClassLoader} is protected. It cannot normally be called.
     * an exception can be thrown if something goes wrong.
     */
    protected void defineClass(final String className, final byte[] b, final int offset, final int len) throws Exception {
        info("Defining class ", className);
        final ClassLoader classLoader = getClass().getClassLoader();

        final Method method = ClassLoader.class.getDeclaredMethod("defineClass", 
                                                                  new Class[] { String.class, byte[].class, int.class, int.class });
        method.setAccessible(true);
        final Class definedClass = (Class) method.invoke(classLoader, className, b, offset, len);
        info("Defined class ", definedClass);
        // info("Loaded class ", getClass().getClassLoader().loadClass(className));
    }

    /**
     * Populates a map of {@link ReferenceType} and {@link byte[]} data. This {@link Map} is used by JDI for 
     * redefining classes.
     *
     * @param toRedefineMap {@link Map} of {@link byte[]} mapped by {@link ReferenceType}. This map is populated by the method, so it shouldn't be null.
     * @see com.sun.jdi.VirtualMachine#redefineClasses(Map, byte[])
     */
    protected void addRedefineClass(final Map<ReferenceType,byte[]> toRedefineMap, final String className, final byte[] b) throws Exception { 
        debug("Found change in ", className);
        for (ReferenceType ref : vm.classesByName(className)) {
            if (ref.name().equals(className) && className.indexOf("WebappWatch") < 0) {
                debug("redefining ", className);
                toRedefineMap.put(ref, b);
                return;
            }
        }
    }

    /**
     * Creates a {@link SocketTransportService} instance and uses that to attach to localhost:8080 and return a {@link Connection} to the 
     * current running JVM instance. This is used by bootstrap to create a {@link VirtualMachine} instance.
     * 
     * @throws Exception when there's a problem attaching to localhost:8080 or if it cannot create an instance of {@link SocketTransportService}
     * @return an open {@link Connection} to the JPDA instance of the current running JVM
     * @see com.sun.jdi.VirtualMachineManager#createVirtualMachine(Connection)
     * @see com.sun.tools.jdi.SocketTransportService
     * @see com.sun.jdi.Bootstrap#virtualMachineManager()
     */
    protected Connection openJdiConnection() throws Exception {
        TransportService ts = null;
        try {
            Class c = Class.forName("com.sun.tools.jdi.SocketTransportService");
            ts = (TransportService)c.newInstance();
        } catch (Exception x) {
            throw new Error(x);
        }
        
        return ts.attach("localhost:8000", 5000, 5000);
    }

    /**
     * Tries to create a canonical class name from a .class file.
     *
     * @param fileName is the fully qualified path of a .class file
     * @return a canonical class name (java.lang.String)
     */
    protected String classFromFileName(final String fileName) {
        String retval = fileName.substring(0, fileName.indexOf(".class"));
        retval = retval.substring(getWebappPath().length() + 1);
        retval = replace(retval, File.separator, ".");
        retval = replace(retval, "/", ".");
        return retval;
    }

    /**
     * Recursively searches for classes. If <code>file</code> is a directory, it traverses recursively. If it is a
     * file, then it compares the <code>lastModified</code> date with the cached {@link #getLastModified()} value.
     * 
     * @param file {@link File} instance for traversing
     * @param classFiles is a {@link Collection} of {@link File} instances where the {@link File#lastUpdated()} value is
     * before {@link #getLastUpdated()}
     */
    protected void traverseForClasses(final File file, final Collection<File> classFiles) {
        if (file.isDirectory()) {
            final String[] fileNames = file.list();
            
            for (final String fileName : fileNames) {
                traverseForClasses(new File(file, fileName), classFiles);
            }
        }
        else if (new Date(file.lastModified()).after(lastUpdated) && file.getName().endsWith(".class")) {
            classFiles.add(file);
        }
    }
    
    /**
     * Gets the value of WebappPath
     *
     * @return the value of WebappPath
     */
    public String getWebappPath() {
        return this.webappPath;
    }
    
    /**
     * Sets the value of WebappPath
     *
     * @param argWebappPath Value to assign to this.WebappPath
     */
    public void setWebappPath(final String argWebappPath) {
        this.webappPath = argWebappPath;
    }

    /**
     * Gets the value of LastUpdated
     *
     * @return the value of LastUpdated
     */
    public Date getLastUpdated() {
        return this.lastUpdated;
    }
    
    /**
     * Sets the value of LastUpdated
     *
     * @param argLastUpdated Value to assign to this.LastUpdated
     */
    public void setLastUpdated(final Date argLastUpdated) {
        this.lastUpdated = argLastUpdated;
    }
}

That's it. Use this and you should have no problem modifying classes as you are developing without needing to restart Spring or Tomcat. Enjoy.

Convert Your KFS Distribution to Maven

2011-06-14T15:33:00.000-07:00

Overview

I am going to cover the process/steps for converting your KFS distribution to use maven. The two greatest reasons for doing this is for development and building/releasing the software. Using maven will make your distribution more consistent with other Kuali projects, get your transitive dependency management, simplify your build process, and simplify your development process with convention over configuration.

At first, this may seem to be a daunting task to move from ant to maven, but it is actually pretty simple. The properties file conventions in KFS do not make this any more difficult as one would think. We can still use maven.

Steps

Before I begin, I want to point out that this assumes you are working with a fresh checkout of the KFS from http://test.kuali.org/svn/kfs/branches/release-4-0-br. You can use 4.1.1 as well. The important thing to note is that it's KFS 4.x.

1 Create path structure

In the directory where KFS is checked out, I suggest first creating all the necessary directories for maven. On Mac or Linux, this is as simple as the following command(s):


% mkdir -p src/main/java src/main/resources src/main/webapp src/main/config src/test/resources src/test/config

2 Copy build components to necessary project configuration locations

build components are typically located in your build/ directory in you KFS project. To do this, I like to use rsync because then I can exclude .svn and .git metadata.

% rsync -avC distribution/ external/ helper-scripts/ project/ properties/ tomcat/ upgrades/ src/main/config/

I have left out the library directories because maven handles our dependencies for us. There is no longer any reason to include the jars in the project. I am not completely sure if tomcat is necessary either, but I left it until later when I can decide if I really need it or not.

3 Move any other project configuration

I consider workflow doctypes, ddl, and dml to be project configuration metadata. These are typically in work/db and work/workflow respectively. I want them to be in src/main/config with the rest of my project configuration.

% mv work/workflow work/db src/main/config

4 Relocate java sources and project resources

Project resources are really anything that isn't a java source. These are going into src/main/resources. Java sources will go into src/main/java. Currently, the KFS project lumps these up into work/src. I am going to split this up.

First, I want to copy the source.

% rsync -avC work/src src2

Now I can do with it what I want which is to remove all the java source from it.

% find src2/ -name \*.java -exec {} \;

Copy project resources

Now that the java code is gone I copy the resources

% rsync -avC src2/* src/main/resources
% rm -rf src2

Copy java sources

I got all the resources, now I will move all my java source.

% find work/src \! -name \*.java -type f -exec rm {} \
% rsync -avC work/src/* src/main/java

5 Relocate tests

I like the way KFS has separated tests. They're interdependent still, but maven also differentiates between test types. I'll keep this separation and move the tests over to src/test/java

% rm -rf test/lib
% mv test/* src/test/java

6 Clean up a little

Now I get rid of all the unnecessary stuff

% rm -rf test work build *.xml

You should now have something that looks like this

leo@behemoth~/.workspace/kfs/release-4-0-mvn
(15:59:41) [6] ls
cnv-build.properties ptd-build.properties src
dev-build.properties reg-build.properties
leo@behemoth~/.workspace/kfs/release-4-0-mvn
(15:59:42) [7]

Crazy, right?

7 Get the pom.xml

To make things easy. I have a pom already. You can get it like this:

% svn export https://svn.rsmart.com/svn/kuali/contribution/community/travel_module/branches/release-4-0-mvn/pom.xml

I'm not going to paste it in here. It's too long.
For the most part, you will not need to change the pom.xml. The groupId, artifactid, or version may be properties about the project you will want to change. Other than that, everything should be peachy and ready to go (with the pom.xml at least).

8 Fix some properties in KFS

There are a couple things that do not jive with maven's conventions in the old KFS. Some are directory structure related. Mostly is the use of ant.project.name allover the place. This property is specific to ant only and not very generic. Therefore, it cannot be used with maven. Maven had the right idea by going with generic property name like project.name

Fix ant.project.name

First, thing to do is clean this up. I used grep to determine where al it's used

leo@behemoth~/.workspace/kfs/travel_module/build/properties
(16:07:09) [5] grep -l ant.project.name *
build-foundation.properties
directory.properties
email.properties
url.properties

I just went through those files and replaced ant. with nothing.

Fix directory properties

Next, is to fix all the properties that point to non-conventional places.


webroot.directory=src/main/webapp
build.directory=src/main/config

Maven Overlay!

The following are more about setting up an overlay project.

9 Install the war

We install the war in our local repository

% mvn -Dmaven.test.skip=true install

10 Create a jar and install it

We need both jar and war to do the overlay, so let's create one and install it.

% mvn jar:jar
% mvn install:install-file -Dpackaging=jar -DgroupId=org.kuali.kfs -DartifactId=kfs -Dversion=4.0 -DgeneratePom=true -Dfile=target/kfs-dev.jar

11 Setup your overlay

Now you pretty much just repeat steps 1-3. This is mostly to get the build configuration going again. Everything will be overlaid on top of the existing war, so we don't need to add resources our java sources.

12 Copy overlay pom.xml

Just like with the KFS project pom.xml, I have a good overlay pom with the dependencies setup just right!

%  svn export https://svn.rsmart.com/svn/kuali/contribution/community/travel_module/branches/release-4-0-overlay/pom.xml

That's it!!! You now have mavenized KFS and you're using an overlay to boot!

Want to see how I did it? Checkout the Mavenize KFS Screencasts

Another Game Changer: Mavenize KFS Screencasts

2011-06-06T14:29:00.001-07:00

Overview

I have put together what I believe to be as a reproducable methodology to converting a KFS project to Maven. There will be a formal blog post detailing the adventure, but for now wet your appetite with these screencasts.

Screencasts

Mavenize KFS Part 1

Mavenize KFS Part 2

Mavenize KFS Part 3: Create an Overlay Project

Constants Done with Spring Part 2: Inheriting Constants

2011-06-06T11:58:00.000-07:00

Motivation

In another chapter, I wrote about constants implemented through Spring. In one of my reasons for doing so, I illustrated that in Kuali, constraints are difficult to manipulate because of their static final nature. I called them an antipattern. Here's an example again:

Notice again that these constants cannot be manipulated. In the other chapter, I approached the use case that one would want to modify an existing constant and change it for their institution. In this chapter, I am going to approach the use case of adding a new constant. Why would you do this? Why not just make a new constants class, right? Well, you could do that. I find it to be more elegant to reuse your constants interface instead of having more than one.

The Pattern

Since we are using Spring, let's assume that we are using the aforementioned Spring constants implementation instead of the antipattern. Now that we know we are using Spring, there are a couple assumptions we can make.

Inversion of Control - we can assume that we can overwrite any known beans with the name we want to occupy. That is, if there is an existing constants implementation, we can not only reuse it, but replace it as well.
Dependency Injection - we can assume that we can inject any known constants we may already have into our new constants implementation.
We can reuse the existing KFS parentBean pattern.

With that, our methodology here is to first inject our existing constants into a new bean, then overwrite our old bean with it.

The Screencast

The Steps

1 Create a new Constants bean to be the delegate.

I created the following as my delegate.

<!--
  Copyright 2007 The Kuali Foundation. Licensed under the Educational
  Community License, Version 1.0 (the "License"); you may not use this
  file except in compliance with the License. You may obtain a copy of
  the License at http://www.opensource.org/licenses/ecl1.php Unless
  required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
  implied. See the License for the specific language governing
  permissions and limitations under the License.
 -->
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

    <bean id="kimDelegateConstants" class="org.kuali.rice.kim.util.ConstantsImpl">
      <property name="kimLdapIdProperty"         value="uaid" />
      <property name="kimLdapNameProperty"       value="uid" />
      <property name="snLdapProperty"            value="sn" />
      <property name="givenNameLdapProperty"     value="givenName" />
      <property name="entityIdKimProperty"       value="entityId" />
      <property name="employeeMailLdapProperty"  value="mail" />
      <property name="employeePhoneLdapProperty" value="employeePhone" />
      <property name="defaultCountryCode"        value="1" />
      <property name="mappedParameterName"       value="KIM_TO_LDAP_FIELD_MAPPINGS" />
      <property name="mappedValuesName"          value="KIM_TO_LDAP_VALUE_MAPPINGS" />
      <property name="parameterNamespaceCode"    value="KR-SYS" />
      <property name="parameterDetailTypeCode"   value="Config" />
      <property name="personEntityTypeCode"      value="PERSON" />
      <property name="employeeIdProperty"        value="emplId" />
      <property name="departmentLdapProperty"    value="employeePrimaryDept" />  
      <property name="employeeTypeProperty"      value="employeeType" />
      <property name="employeeStatusProperty"    value="employeeStatus" />
      <property name="affiliationLdapProperty"   value="affiliationProperty" />
      <property name="primaryAffiliationLdapProperty"   value="eduPersonPrimaryAffiliation" />
      <property name="defaultCampusCode"         value="MC" />
      <property name="defaultChartCode"          value="UA" />
      <property name="taxExternalIdTypeCode"     value="TAX" />
      <property name="externalIdProperty"        value="externalIdentifiers.externalId" />
      <property name="externalIdTypeProperty"    value="externalIdentifiers.externalIdentifierTypeCode" />
      <property name="affiliationMappings"       value="staff=STAFF,faculty=FCLTY,employee=STAFF,student=STDNT,affilate=AFLT"/>
      <property name="employeeAffiliationCodes"  value="STAFF,FCLTY" />
    </bean>
...
...
</beans>

The important thing to notice is that it's in a new bean and not kimConstants any longer.

2 Override the original kimConstants bean

Now you need to create a new kimConstants bean that uses your implementation and delegates to the delegate created in Step 1

<?xml version="1.0" encoding="UTF-8"?>
 <!--
  Copyright 2007 The Kuali Foundation. Licensed under the Educational
  Community License, Version 1.0 (the "License"); you may not use this
  file except in compliance with the License. You may obtain a copy of
  the License at http://www.opensource.org/licenses/ecl1.php Unless
  required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
  implied. See the License for the specific language governing
  permissions and limitations under the License.
 -->
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
...
...
    <bean id="kimConstants" class="edu.arizona.kim.util.ConstantsImpl">
        <property name="phoneNumberMask"           value="###-###-####" />
        <property name="delegate"                  value-ref="kimDelegateConstants" />
    </bean>
</beans>

Here a new kimConstants is created. It only has 2 properties. One is for the delegate and the other is the field I wanted to add.

3. Create a new interface for your constants that includes your new field

/*
 * Copyright 2007-2009 The Kuali Foundation
 *
 * Licensed under the Educational Community License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.opensource.org/licenses/ecl2.php
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package edu.arizona.kim.util;


/**
 * Extension interface for constants
 */
public interface Constants extends org.kuali.rice.kim.util.Constants {
    
    /**
     * Retrieve the phoneNumberMask constant
     *
     * @return String instance of the phoneNumberMask
     */
    String getPhoneNumberMask();
}

You can see it extends the original constants interface and thereby gaining all its abstract methods for all the delegate method's properties.

4 Create the implementation class for the new bean

/*
 * Copyright 2007-2009 The Kuali Foundation
 *
 * Licensed under the Educational Community License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.opensource.org/licenses/ecl2.php
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package edu.arizona.kim.util;

import java.util.Collection;

import org.kuali.rice.kim.bo.entity.dto.KimEntityInfo;

/**
 * Extension implementation for constants
 *
 */
class ConstantsImpl implements Constants {
    private String phoneNumberMask;
    private org.kuali.rice.kim.util.Constants delegate;

    public void setDelegate(final org.kuali.rice.kim.util.Constants delegate) {
        this.delegate = delegate;
    }

    public org.kuali.rice.kim.util.Constants getDelegate() {
        return delegate;
    }

    public Collection getTestPrincipalNames() {
        return getDelegate().getTestPrincipalNames();
    }


    /**
     * Gets the value of entityPrototype
     *
     * @return the value of entityPrototype
     */
    public KimEntityInfo getEntityPrototype() {
        return getDelegate().getEntityPrototype();
    }

    /**
     * Gets the value of externalIdTypeProperty
     *
     * @return the value of externalIdTypeProperty
     */
    public String getExternalIdTypeProperty() {
        return getDelegate().getExternalIdTypeProperty();
    }

    /**
     * Gets the value of taxExternalIdTypeCode
     *
     * @return the value of taxExternalIdTypeCode
     */
    public String getTaxExternalIdTypeCode() {
        return getDelegate().getTaxExternalIdTypeCode();
    }


    /**
     * Gets the value of externalIdProperty
     *
     * @return the value of externalIdProperty
     */
    public String getExternalIdProperty() {
        return getDelegate().getExternalIdProperty();
    }

    /**
     * Gets the value of employeePhoneLdapProperty
     *
     * @return the value of employeePhoneLdapProperty
     */
    public String getEmployeePhoneLdapProperty() {
        return getDelegate().getEmployeePhoneLdapProperty();
    }

    /**
     * Gets the value of employeeMailLdapProperty
     *
     * @return the value of employeeMailLdapProperty
     */
    public String getEmployeeMailLdapProperty() {
        return getDelegate().getEmployeeMailLdapProperty();
    }

    /**
     * Gets the value of defaultCountryCode
     *
     * @return the value of defaultCountryCode
     */
    public String getDefaultCountryCode() {
        return getDelegate().getDefaultCountryCode();
    }

    /**
     * Gets the value of personEntityTypeCode
     *
     * @return the value of personEntityTypeCode
     */
    public String getPersonEntityTypeCode() {
        return getDelegate().getPersonEntityTypeCode();
    }

    /**
     * Gets the value of kimLdapIdProperty
     *
     * @return the value of kimLdapIdProperty
     */
    public String getKimLdapIdProperty() {
        return getDelegate().getKimLdapIdProperty();
    }

    /**
     * Gets the value of kimLdapNameProperty
     *
     * @return the value of kimLdapNameProperty
     */
    public String getKimLdapNameProperty() {
        return getDelegate().getKimLdapNameProperty();
    }

    /**
     * Gets the value of snLdapProperty
     *
     * @return the value of snLdapProperty
     */
    public String getSnLdapProperty() {
        return getDelegate().getSnLdapProperty();
    }

    /**
     * Gets the value of givenNameLdapProperty
     *
     * @return the value of givenNameLdapProperty
     */
    public String getGivenNameLdapProperty() {
        return getDelegate().getGivenNameLdapProperty();
    }

    /**
     * Gets the value of entityIdKimProperty
     *
     * @return the value of entityIdKimProperty
     */
    public String getEntityIdKimProperty() {
        return getDelegate().getEntityIdKimProperty();
    }

    /**
     * Gets the value of parameterNamespaceCode
     *
     * @return the value of parameterNamespaceCode
     */
    public String getParameterNamespaceCode() {
        return getDelegate().getParameterNamespaceCode();
    }

    /**
     * Gets the value of parameterDetailTypeCode
     *
     * @return the value of parameterDetailTypeCode
     */
    public String getParameterDetailTypeCode() {
        return getDelegate().getParameterDetailTypeCode();
    }

    /**
     * Gets the value of mappedParameterName
     *
     * @return the value of mappedParameterName
     */
    public String getMappedParameterName() {
        return getDelegate().getMappedParameterName();
    }

    /**
     * Gets the value of unmappedParameterName
     *
     * @return the value of unmappedParameterName
     */
    public String getUnmappedParameterName() {
        return getDelegate().getUnmappedParameterName();
    }

    /**
     * Gets the value of mappedValuesName
     *
     * @return the value of mappedValuesName
     */
    public String getMappedValuesName() {
        return getDelegate().getMappedValuesName();
    }

    /**
     * Gets the value of employeeIdProperty
     *
     * @return the value of employeeIdProperty
     */
    public String getEmployeeIdProperty() {
        return getDelegate().getEmployeeIdProperty();
    }

    /**
     * Gets the value of departmentLdapProperty
     *
     * @return the value of departmentLdapProperty
     */
    public String getDepartmentLdapProperty() {
        return getDelegate().getDepartmentLdapProperty();
    }

    /**
     * Gets the value of employeeTypeProperty
     *
     * @return the value of employeeTypeProperty
     */
    public String getEmployeeTypeProperty() {
        return getDelegate().getEmployeeTypeProperty();
    }

    /**
     * Gets the value of employeeStatusProperty
     *
     * @return the value of employeeStatusProperty
     */
    public String getEmployeeStatusProperty() {
        return getDelegate().getEmployeeStatusProperty();
    }

    /**
     * Gets the value of defaultCampusCode
     *
     * @return the value of defaultCampusCode
     */
    public String getDefaultCampusCode() {
        return getDelegate().getDefaultCampusCode();
    }

    public String getDefaultChartCode() {
        return getDelegate().getDefaultChartCode();
    }

    public String getEmployeeAffiliationCodes() {
        return getDelegate().getEmployeeAffiliationCodes();
    }
    public String getAffiliationMappings() {
        return getDelegate().getAffiliationMappings();
    }

    /**
     * Gets the mappings for the affiliation ldap property
     * @return mapping for KIM affiliation and LDAP
     */
    public String getAffiliationLdapProperty() {
        return getDelegate().getAffiliationLdapProperty();
    }

    /**
     * Gets the mappings for the primary affiliation ldap property
     * @return mapping for KIM primary affiliation and LDAP
     */
    public String getPrimaryAffiliationLdapProperty() {
        return getDelegate().getPrimaryAffiliationLdapProperty();
    }

    /**
     * @see edu.arizona.kim.util.Constants#getPhoneNumberMask()
     */
    public String getPhoneNumberMask() {
        return this.phoneNumberMask;
    }

    /**
     * Sets the phone number mask constant
     *
     * @param phoneNumberMask value to set for the phoneNumberMask constant
     */
    public void setPhoneNumberMask(final String phoneNumberMask) {
        this.phoneNumberMask = phoneNumberMask;
    }
}

Notice the following allover the place

    /**
     * Gets the mappings for the primary affiliation ldap property
     * @return mapping for KIM primary affiliation and LDAP
     */
    public String getPrimaryAffiliationLdapProperty() {
        return getDelegate().getPrimaryAffiliationLdapProperty();
    }

The getDelegate() call is accessing the delegate we setup in spring. The dependency injection takes care of everything for us, so we don't need to worry about it being a package level class either. We also only override the getters because this is a constant. We're not supposed to change it.

Conclusion

There you have it. Another reason to use constants in spring. Not only are they easy to define, but now you can use object composition to inherit constants from other beans.

Setting up CAS on KC/Rice

2011-04-10T18:25:00.000-07:00

Screencast

Just a screencast to show where to get the files and how to set it up.

Instructions

1 Download Source from rSmart

% svn co  https://svn.rsmart.com/svn/kuali/rice/rsmart_rice_core/trunk rsmart_rice_core

2 Copy Example Config

% cp web/src/main/config/example-config/rice-config.xml $HOME/kuali/main/dev/

3 Copy Contents of LDAP Example Config

% cat ldap/src/main/config/example-config.xml

You should see something that looks like:

<!--
 Copyright 2008-2009 The Kuali Foundation
 
 Licensed under the Educational Community License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
 http://www.opensource.org/licenses/ecl2.php
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->
<config>
  <param name="cas.url">http://localhost:8080/${cas.context.name}</param>
  <param name="cas.require.https">false</param>
  <param name="cas.validate.password">true</param>
  <param name="cas.rice.server.name">${appserver.url}</param>
  
  <param name="filter.login.class">org.jasig.cas.client.authentication.AuthenticationFilter</param>
  <param name="filter.login.casServerLoginUrl">${cas.url}/login</param>
  <param name="filter.login.serverName">${appserver.url}</param>
  <param name="filtermapping.login.1">/</param>
  
  <param name="filter.validation.class">org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter</param>
  <param name="filter.validation.casServerUrlPrefix">${cas.url}</param>
  <param name="filter.validation.serverName">${appserver.url}</param>
  <param name="filtermapping.validation.2">/</param>
  
  <param name="filter.caswrapper.class">org.jasig.cas.client.util.HttpServletRequestWrapperFilter</param>
  <param name="filtermapping.caswrapper.3">/</param>
  
  <param name="rice.ldap.username">uid=user,ou=Ldap Users,dc=localhost</param>
  <param name="rice.ldap.password">[secret]</param>
  <param name="rice.ldap.url">ldaps://localhost:636</param>
  <param name="rice.ldap.base">ou=People,dc=localhost</param>
  <param name="rice.additionalSpringFiles">org/kuali/rice/kim/config/KIMLdapSpringBeans.xml</param>
</config>

When you finish, your config should look like:

<!--
 Copyright 2008-2009 The Kuali Foundation
 
 Licensed under the Educational Community License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
 http://www.opensource.org/licenses/ecl2.php
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->
<config>

 <!-- Please fill in a value for this parameter! -->
 <param name="http.port">8080</param>
 <param name="application.host">http://yourserver</param>
 <param name="application.url">${application.host}:${http.port}/${app.context.name}</param>

 <param name="plugin.dir">/usr/local/rice/plugins</param> 

 <!-- set some datasource defaults -->
 <param name="datasource.username">rice</param>
 <param name="datasource.password">*** password ***</param>
 
 <!-- MySQL example -->
 <param name="datasource.ojb.platform">MySQL</param>
 <param name="datasource.platform">org.kuali.rice.core.database.platform.MySQLDatabasePlatform</param>
 <param name="datasource.url">jdbc:mysql://localhost:3306/${datasource.username}</param>
 <param name="datasource.driver.name">com.mysql.jdbc.Driver</param>
 <param name="datasource.pool.validationQuery">select 1</param>
 
 <!-- Oracle example 
 <param name="datasource.ojb.platform">Oracle9i</param>
 <param name="datasource.platform">org.kuali.rice.core.database.platform.OracleDatabasePlatform</param>
 <param name="datasource.url">jdbc:oracle:thin:@localhost:1521:XE</param>
 <param name="datasource.driver.name">oracle.jdbc.driver.OracleDriver</param>
 <param name="datasource.pool.validationQuery">select 1 from dual</param>
 -->
 
 <param name="attachment.dir.location">/usr/local/rice/kew_attachments</param>
 <param name="data.xml.root.location">/usr/local/rice/kew/xml</param> 

 <!-- log4j settings -->
 <param name="log4j.settings.path">/usr/local/rice/log4j.properties</param>
 <param name="log4j.settings.reloadInterval">5</param>

 <!-- Keystore Configuration -->
 <param name="keystore.file">/usr/local/rice/rice.keystore</param>
 <param name="keystore.alias">*** key alias ***</param>
 <param name="keystore.password">*** password ***</param>
 
 <param name="mail.relay.server">localhost</param>
    <param name="mailing.list.batch">mailing.list.batch</param>
    <param name="encryption.key">*** encryption key ***</param> 

  <param name="cas.url">http://localhost:8080/${cas.context.name}</param>
  <param name="cas.require.https">false</param>
  <param name="cas.validate.password">true</param>
  <param name="cas.rice.server.name">${appserver.url}</param>
  
  <param name="filter.login.class">org.jasig.cas.client.authentication.AuthenticationFilter</param>
  <param name="filter.login.casServerLoginUrl">${cas.url}/login</param>
  <param name="filter.login.serverName">${appserver.url}</param>
  <param name="filtermapping.login.1">/</param>
  
  <param name="filter.validation.class">org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter</param>
  <param name="filter.validation.casServerUrlPrefix">${cas.url}</param>
  <param name="filter.validation.serverName">${appserver.url}</param>
  <param name="filtermapping.validation.2">/</param>
  
  <param name="filter.caswrapper.class">org.jasig.cas.client.util.HttpServletRequestWrapperFilter</param>
  <param name="filtermapping.caswrapper.3">/</param>
  
  <param name="rice.ldap.username">uid=user,ou=Ldap Users,dc=localhost</param>
  <param name="rice.ldap.password">[secret]</param>
  <param name="rice.ldap.url">ldaps://localhost:636</param>
  <param name="rice.ldap.base">ou=People,dc=localhost</param>
  <param name="rice.additionalSpringFiles">com/rsmart/kuali/rice/ldap/KIMLdapSpringBeans.xml</param>

    <!-- Sample Application Flag -->
    <param name="sample.enabled">false</param>

    <param name="dev.mode">false</param> 
</config>

Notice that the DummyLoginFilter is not to be found. This is important. This handles logins by default. You don't want it around when you're configuring CAS.

4 Make Changes to Config

Right now, CAS is configured for localhost. At your institution, you will want to point this to your REAL CAS server. Also, make sure to set the cas.context.name. It is probably better to use https as well.

    <param name="cas.context.name">webauth</param>
  <param name="cas.url">https://webauth.arizona.edu/${cas.context.name}</param>
  <param name="cas.require.https">false</param>
  <param name="cas.validate.password">true</param>
  <param name="cas.rice.server.name">${appserver.url}</param>
  
  <param name="filter.login.class">org.jasig.cas.client.authentication.AuthenticationFilter</param>
  <param name="filter.login.casServerLoginUrl">${cas.url}/login</param>
  <param name="filter.login.serverName">${appserver.url}</param>
  <param name="filtermapping.login.1">/</param>
  
  <param name="filter.validation.class">org.jasig.cas.client.validation.Cas20ProxyReceivingTicketValidationFilter</param>
  <param name="filter.validation.casServerUrlPrefix">${cas.url}</param>
  <param name="filter.validation.serverName">${appserver.url}</param>
  <param name="filtermapping.validation.2">/</param>
  
  <param name="filter.caswrapper.class">org.jasig.cas.client.util.HttpServletRequestWrapperFilter</param>
  <param name="filtermapping.caswrapper.3">/</param>
  
  <param name="rice.ldap.username">uid=user,ou=Ldap Users,dc=eds,dc=arizona,dc=edu</param>
  <param name="rice.ldap.password">[secret]</param>
  <param name="rice.ldap.url">ldaps://eds.arizona.edu:636</param>
  <param name="rice.ldap.base">ou=People,dc=eds,dc=arizona,dc=edu</param>
  <param name="rice.additionalSpringFiles">com/rsmart/kuali/rice/ldap/KIMLdapSpringBeans.xml</param>

5 Configure LDAP

At UA, we setup there is an institutional directory service called EDS. Your institution may also have one. I configured it like this,

  <param name="rice.ldap.username">uid=user,ou=Ldap Users,dc=eds,dc=arizona,dc=edu</param>
  <param name="rice.ldap.password">[secret]</param>
  <param name="rice.ldap.url">ldaps://eds.arizona.edu:636</param>
  <param name="rice.ldap.base">ou=People,dc=eds,dc=arizona,dc=edu</param>
  <param name="rice.additionalSpringFiles">com/rsmart/kuali/rice/ldap/KIMLdapSpringBeans.xml</param>

6 Love

That's it. This is a runtime configuration, so simply restarting my application server will reload this configuration and make the changes live.

The Only Logger You'll Ever Need

2011-04-10T11:18:00.000-07:00

The Status Quo for Logging in Kuali

One of the most frustrating things about developing with Kuali Foundation Software is the logging. Setup aside, just adding the logging is frustrating. Mostly because of the immense amounts of copy/paste that is encouraged by it. Here's an example of some of the boiler plate.

...
...
import org.apache.log4j.Logger;
import org.apache.ojb.broker.query.Criteria;
import org.apache.ojb.broker.query.QueryByCriteria;
import org.kuali.kfs.coa.dataaccess.impl.ChartDaoOjb;
import org.kuali.kfs.fp.businessobject.TravelMileageRate;
import org.kuali.kfs.fp.document.dataaccess.TravelMileageRateDao;
import org.kuali.rice.kns.dao.impl.PlatformAwareDaoBaseOjb;

/**
 * This class is the OJB implementation of the TravelMileageRate interface.
 */
public class TravelMileageRateDaoOjb extends PlatformAwareDaoBaseOjb implements TravelMileageRateDao {
    private static Logger LOG = Logger.getLogger(ChartDaoOjb.class);
...
...

Let's go through this. First, examine the import:


import org.apache.log4j.Logger;

Every single class you write with logging will require this weird, foreign class that really has nothing to do with the functionality of your software. It's awkward, and it's boilerplate. It's everywhere needlessly, and in some cases can cause you to neglect that it's there. Next, is my favorite part. We declare the logger on top of having to import it:


public class TravelMileageRateDaoOjb extends PlatformAwareDaoBaseOjb implements TravelMileageRateDao {
    private static Logger LOG = Logger.getLogger(ChartDaoOjb.class);

See anything unusual? This is really what I was getting at behind the copy/paste. To my knowledge, this still exists in the KFS source code. It's misleading. This TravelMileageRateDao is logging as the ChartDaoOjb I doubt this is on purpose. Rather, it is the result of copy/pasting the logger declaration from another class. Many do this because it is tedious. As a result, many forget to change the class name.

I am not going to blame the developer for this. In my mind, it shouldn't even be necessary to do this. Shouldn't the framework just know what class I'm logging from? Is that really so hard?

Researching the Performance of Logging

After putting together this post on another blog, I became determined to devise a simpler way to handle logging. Here were my goals.

Limit Logger boilerplate to the import statement
Efficient logging where the full log message is not concatenated until it is determined whether the message would be used or not
printf style formatting if possible.

Back to the Question

Shouldn't the framework just know what class I'm logging from? Is that really so hard?

No. It's not.

I have created two classes called BufferedLogger and FormattedLogger. These are "The Only Loggers You'll Ever Need".

How to use them

It's easy. Before now, you probably thought static imports are pretty useless. Think again.


import static org.kuali.kra.logging.BufferedLogger.*;

That's your boilerplate. Next, let's use it:


    public ActionForward insertProposalPerson(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception {
...
...
        // if the rule evaluation passed, let's add it
        if (rulePassed) {
            document.getDevelopmentProposal().addProposalPerson(pdform.getNewProposalPerson());
            info(ADDED_PERSON_MSG, pdform.getNewProposalPerson().getProposalNumber(), pdform.getNewProposalPerson().getProposalPersonNumber());
            // handle lead unit for investigators respective to coi or pi
            if (getKeyPersonnelService().isPrincipalInvestigator(pdform.getNewProposalPerson())) {
                getKeyPersonnelService().assignLeadUnit(pdform.getNewProposalPerson(), document.getDevelopmentProposal().getOwnedByUnitNumber());
            }
...
...

Very easy stuff. Notice that there are multiple objects being passed to the info method. They are not concatenated yet. The info method first checks if the message will be used before concatenating. This is actually a huge timesaver if you consider that the '+' and '+=' concatenation is pretty time consuming.

What about printf style logging? Here's another example:


    void prepare(ActionForm form, HttpServletRequest request) {
        ProposalDevelopmentForm pdform = (ProposalDevelopmentForm) form;
        request.setAttribute(NEW_PERSON_LOOKUP_FLAG, EMPTY_STRING);
        ProposalDevelopmentDocument document=pdform.getDocument();
        List proposalpersons=document.getDevelopmentProposal().getProposalPersons();
        for (Iterator iter = proposalpersons.iterator(); iter.hasNext();) {
            ProposalPerson person=(ProposalPerson) iter.next();
            if (person.getRole() != null) {
                person.getRole().setReadOnly(getKeyPersonnelService().isRoleReadOnly(person.getRole()));
            }
        }
        
        pdform.populatePersonEditableFields();
        handleRoleChangeEvents(pdform.getDocument());
        
        debug(INV_SIZE_MSG, pdform.getDocument().getDevelopmentProposal().getInvestigators().size());
    
        try {
            boolean creditSplitEnabled = this.getParameterService().getIndicatorParameter(ProposalDevelopmentDocument.class, CREDIT_SPLIT_ENABLED_RULE_NAME)
                && pdform.getDocument().getDevelopmentProposal().getInvestigators().size() > 0;
            request.setAttribute(CREDIT_SPLIT_ENABLED_FLAG, new Boolean(creditSplitEnabled));
            pdform.setCreditSplitEnabled(creditSplitEnabled);
        }
        catch (Exception e) {
            warn(MISSING_PARAM_MSG, CREDIT_SPLIT_ENABLED_RULE_NAME);
            warn(e.getMessage());
        }        
    }

You can see that this looks no different than the info illustrated earlier. There is one difference though. Examine the warn statement:


            warn(MISSING_PARAM_MSG, CREDIT_SPLIT_ENABLED_RULE_NAME);

It uses a constant called MISSING_PARAM_MSG. This is actually a format string that looks like:


    private static final String MISSING_PARAM_MSG = "Couldn't find parameter '%s'";

Which is better? `BufferedLogger` or `FormattedLogger`

FormattedLogger has its downside. Formatting actually takes more clock cycles than concatenation. It's friendlier to developers though. With a rather large number of parameters to format, it can be relatively fast. I fall on BufferedLogger the most, but FormattedLogger has its uses.

There you have it. No more copy paste. printf style logging.

KIS Me Kate - RPM Packaging KFS Part 1

2011-04-10T10:30:00.000-07:00

The default packaging option from the Kuali Foundation is typically JAR or WAR packaging. These are my observations from modifying typically deployment of KFS at the University of Arizona, to using RPMs. This is fine for libraries and/or web applications. It is platform independent and follows the standards of software deployment. So

Why Repackage KFS?

There are shortcomings tow WAR and JAR packaging. WAR packaging was created with the intent that the application is actually WebSphere, Weblogic, JBOSS, etc..., and the WAR is actually a webapp that is deployed within. With that intent comes the concept that everything is contained within the WAR. What is lacking is:

There is no notion of pre/post processing at installation and deployment
There is no verification of dependencies or requisites.
There is no workflow for software installation
No maintenance over documentation vs. configuration files.
No platform-specific task hooks.
Upgrade software management and configuration management

All of these things may not make sense. For example, "No platform-specific task hooks". WAR is platform independent. Why would you want that, right? Well, that's just it. I think it's great that WAR is platform-independent. It let's you independently define your own packaging around it. Again, why would you want to do that? Double packaging? Let's approach each of these.

Pre/Post-Processing at Installation and Deployment

There may be some actual server information that the application needs to be configured at the point of installation and deployment. For example, server name database configuration, ssh key generation, certificate authority verification, ssl configuration, etc... These are normally configured manually by the system administrator manually after installing a WAR. What if this needs to be installed on several servers in a cloud? Some of this information can be automated. It does not require interaction or input from a user, so why do we require it to be done manually? That shouldn't be necessary.

Verification of Dependencies or Requisites

Sure, WAR files can contain all required libraries. There's no guarantee of this. Further, what if the server has a configuration that supercedes the WAR configuration. There's no way to know that either. Libraries aren't the only requisites that a WAR can have. You can ship with the BSF (Bean Scripting Framework), but that does you absolutely no good whatsoever if you have no native scripting languages installed on your server. What about the application server? The WAR doesn't come with that. Wouldn't it be nice if installing the application meant that even the appserver was installed with it and any software (libraries or not) it depends on? Yes it would, that's why proper software packaging starts to look pretty good.

Workflow for Software Installation

Like your development process, installation itself can have its own phases. A good installation infrastructure allows you to augment or even create and define your own phases. For example, pre/post-processing scripts (described earlier), patching, build, file installation, permission assignments, documentation handling, cleanup, etc...

Maintenance Over Documentation vs. Configuration Files

Documentation and Configuration files are very delicate items in your deployment. Any configuration deployed with the application is usually reference implementation. That is, it is typically replaced manually or at the first deployment. Upon upgrading, configuration files are not something you want to override. You typically want to back these up with each upgrade and identify changes in configuration formats between versions. For example, new configuration entries can be added to a configuration file with each version. Some may become obsolete. It is undesirable to keep these, but you typically do not want to sacrifice the rest of your configuration for this. Kuali software is not a stranger to this situation. Kuali Foundation projects have the concept of an "external" configuration directory that exists outside of the WAR. The purpose of this is to exclude sensitive information from the webapp itself like passwords, any uploaded financial information batch files, or even log files. Such information is kept out of reach for security purposes within the external configuration. The external configuration files will not likely change, so when deploying upgrades, these should remain unchanged.

Platform-specific Task Hooks

Trouble with WAR files is that their plaform independent. This is their greatest strength, but also a huge weakness. WAR files know nothing about the system your are deploying too, this is a hassle for system administrators because the software has to be treated separately from other installations. It is difficult to observe changes and possible security threats. For example, if there is an exploit in the version of bouncy castle that KFS uses, there is no way for a system administrator to know. Further, if installation can be simplified by using platform-specific knowledge about the installed system at installation time, this knowledge can be used to automate the process more. For example, knowing what software and what version is installed can help determine what arguments to pass to utilities at install time.

Upgrade Software Management and Configuration Management

Basically, this is software intelligence. Having metadata about your software before and after it's installed for dependency management (illustrated earlier), configuration file handling (illustrated earlier), bug reporting, vulnerability observing (illustrated earlier), etc... One huge use is when a system administrator needs to manage multiple software installations across several servers. Knowing what version, build number, and configurations are on servers should be as easy as checking the software database of that system. Each platform has one. If a webapp is packaged and installed using that system, the software database will know about it and be able to disseminate that information back to the system administrators. With further scripting and automation tools, system administrators can have much better control over the systems they maintain.

Why is Packaging Important to KFS?

KFS is victim to all of the above illustrated.

At the University of Arizona, there are application sanity and maintenance tasks that run before packaging and before package installation. Packaging is done on a system separate from installation because there are actually several installation systems. To provide a repeatable installation process, it was decided that if any were going to be on a remote system then all should. For security purposes I cannot go into much detail about what pre/post-processing the University of Arizona does, but I can say that I would expect just about any university to require it.
Since building, packaging, and installation happen on separate systems a workflow has to be maintained. Further, the software is not the only thing that is installed for some implementing institutions. Some may couple data or even database schema information to the software. Therefore, this information may be distributed and deployed with the software. It cannot happen all at once. If there is a problem anywhere during the installation, a fallover path must exist. Installation workflows help this work out.
KFS makes use of something called an "external" settings directory. This directory is created and populated with reference information at installation. However, this information is overwritten at each installation by default. It would be good to not have to do this each time. When dealing with configurations on several servers, it can be tedious and problematic to rebuild each time and reconfigure each time. Mistakes are made. It is best to just configure once, and then make modifications only when necessary.
I have noticed that some institutions like to give access to the shared files directory via setacls on RedHat and other linux systems. Such capabilities are not available by dropping in a war.
Currently KFS does not install all other required software with it. Also, there are no reports or observations on vulnerabilities of libraries distributed with KFS. When upgrading from one release to another, there is no verification of version or software compatibility.

How can KFS Further Benefit from Software Packaging?

I used RPM packaging of KFS at the University of Arizona. When I did, it greatly streamlined installation. System administrators were familiar already with RPM. They were adept with installing packages, handling logs and dependencies, and found it easy to modify and maintain configuration. We were able to integrate RPM building into our CI (continuous integration). Whenever a release was due, a new RPM was created. This RPM could then be manually or automatically installed by the package management system. RPM information could quickly be verified with

% rpm -qa | grep kuali
kuali-coeus-2.0-10
kuali-coeus-settings-dev-2.0-10
kuali-coeus-kittdb-2.0-10

% rpm -qi kuali-coeus-settings-dev-2.0-10
Name : kuali-coeus-settings-dev Relocations: (not relocatable)
Version : 2.0 Vendor: (none)
Release : 10 Build Date: Tue 28 Dec 2010 07:19:51 AM MST
Install Date: Tue 11 Jan 2011 01:23:25 AM MST Build Host: uaz-kr-a02.mosaic.arizona.edu
Group : System/Base Source RPM: kuali-coeus-2.0-10.src.rpm
Size : 200685086 License: EPL
Signature : (none)
Packager : leo [at] rsmart.com
Summary : External configuration settings for Kuali Coeus
Description :
Mosaic Kuali Coeus external configuration and settings. These files are located
in /home/tomcat/app

. With dependencies management, they are able to make sure that all the necessary tools exist in the system before the software is installed/updated. Database upgrades are streamlined and integrated into the installation process.

Package Repositories

I mentioned that UA has a package management repository. Right now, it is a crude CIFS share. It is possible through YUM to be more elegant with updates. I plan to in the future create my own YUM repository for managing Kuali software upgrades.

Another package management and build system is APT which uses DEB packages. A package maintainer can create a project on LaunchPad which is a portal for Ubuntu Package/Build management. These packages are then present on a package repository (PPA or Personal Package Archive) where a system administrator can then point a server to this PPA and gain updates to all the software on it. The software and repository are verified against a PGP (Pretty Good Privacy) key. See my launchpad. My goal is to eventually have a working packaged KFS distribution by Summer 2011.

Between YUM and APT, it is possible to get automatic updates and patches to your Kuali Software. This is one of the biggest reasons to use software packaging for KFS.

Implementing KC 2.0 as an Overlay

2011-04-02T21:43:00.000-07:00

Overview

Maven overlays are a way to take 2 web application projects and combine them with one overlaying the other. It sounds like it would be interesting if you want to use another project as the base for your own, right? Almost as if it is slightly wrong. This only makes sense when your project builds on that of another. Literally building upon another project as a foundation. You would only make another project the foundation when you are in absolute control of that other project. Kuali is community source, so the community runs the foundation project in this case. The other project is intended to be an implementation of Kuali Coeus where the implementing institution is a member of the community. Then technically, an overlay makes sense. You wouldn't want to make an overlay of just any project you found that you thought was cool and you wanted to change. It would be much better to fork that project instead. If you overlay, you run the risk of the foundation changing from underneath you. When implementing Kuali software, this is fine because you have the community supporting you and the software.

The Problem with Enterprise Software Maintenance at a University

To truly understand why overlays are a good idea at institutions and particularly universities, you need to understand the problem universities have had implementing software pretty much throughout history.

Universities are used to getting this software implemented, but as time goes business processes and practices change. Business is changing. Universities are changing. Why shouldn't their business practices change? It makes sense. The trouble is that they bankrupt their budgets on implementing the software and have nothing left to maintain it. All they budget for maintaining it is fixing leaks and bandaiding. Eventually, these institutions are left with failing systems on the brink of demise. It all comes down to maintenance. These systems can't just be good now. They need the potential to be good 10 years from now.

Kuali is no different in that respect. Compared to other enterprise software, the source code is very extensive. Making bug fixes to it will have effects on upgrading. Every change an institution adds to the source code, that does not get back to the trunk will cause problems because the institution's code base differs that much from that which is upgraded. The mindset is change as little of the original codebase as possible. Only build upon it if possible. If you're going to change something, find a way to do it without modifying the original distribution. No matter what you do, keep a record between versions so we know what changed. Among all of these, the consistent idea is to modify the distribution as little as possible.

That is what overlays allow. Modifying and customizing the distribution by overlaying it affords institutions the ability to

make changes without making patching or upgrading difficult in the future
track what changes you made
simplify your local distribution

The Screencast

This is a screencast based on the instructions laid out in KC 2.0 Customization.

Instructions

Written instructions for following along with the screencast.

1 Checkout the KC Project

First, you need to download the full KC project. I created a path in my workspace to store all this.

% mkdir -p .workspace/rsmart

Then checkout the source code from it. I used export because eventually, I want to import this into my own svn repository

% cd .workspace/rsmart
% svn export https://test.kuali.org/svn/kc_project/tags/kc-release-2_0-tag

The above creates a new kc-release-2_0-tag directory.

2 Install KC WAR and JAR Files

To install the WAR file in our maven repository, we use

% mvn -Dmaven.test.skip=true install

To create the JAR file, we use

% mvn jar:jar

Installing the JAR is a little different.

% mvn install:install-file -Dpackaging=jar -DgroupId=org.kuali.kra -DartifactId=kc_project -Dversion=2.0 -DgeneratePom=true -Dfile=target/kc_project-2.0.jar

That should be the end of our work with the kc_project.

3 Setup kc_custom

Create the directory structure and pom.xml.

3.1 Create Directory Structure

% mkdir kc
% mkdir -p kc/src/main/java/com/rsmart/kuali/kc
% mkdir -p kc/src/main/java/org/kuali/kra/infrastructure
% mkdir -p kc/src/main/config
% mkdir -p kc/src/main/resources/com/rsmart/kuali/kc
% mkdir -p kc/src/main/webapp/WEB-INF/

3.1 Copy Some Files Over

Might as well copy a couple files from the kc_project.

% cp kc-release-2_0-tag/src/main/webapp/WEB-INF/web.xml kc/src/main/webapp/WEB-INF/
% cp kc-release-2_0-tag/src/main/java/org/kuali/kra/infrastructure/KraServiceLocator.java kc/src/main/java/org/kuali/kra/infrastructure

3.2 Create pom.xml

The new overlay project needs its own pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.rsmart.kuali.kc</groupId>
  <artifactId>kc</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>kc</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.kuali.kra</groupId>
      <artifactId>kc_project</artifactId>
      <version>2.0</version>
      <type>war</type>
    </dependency>
    <dependency>
      <groupId>org.kuali.kra</groupId>
      <artifactId>kc_project</artifactId>
      <version>2.0</version>
      <scope>provided</scope>
      <type>jar</type>
    </dependency>
    <dependency>
          <groupId>org.kuali.rice</groupId>
          <artifactId>rice-kns</artifactId>
      <version>1.0.2.1</version>                               
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
 <repositories>
  <repository>
   <id>kuali</id>
   <name>Kuali Repository</name>
   <url>https://test.kuali.org/maven</url>
   <snapshots><enabled>true</enabled></snapshots>
  </repository>
  <repository>
   <id>codehaus</id>
   <name>Codehaus</name>
   <url>http://dist.codehaus.org</url>
  </repository>
  <repository>
   <id>apache</id>
   <name>apache</name>
   <url>http://people.apache.org/repo/m2-ibiblio-rsync-repository</url>
  </repository>
  <repository>
   <id>jboss</id>
   <name>jboss</name>
   <url>http://repository.jboss.com/maven2</url>
  </repository>
  <repository>
   <id>atlassian</id>
   <name>atlassian</name>
   <url>http://maven.atlassian.com/repository/public</url>
  </repository>
  <repository>
     <snapshots />
     <id>maven-repo1</id>
     <name>maven2 repo</name>
     <url>http://repo1.maven.org/maven2</url>
  </repository> 

 </repositories>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <configuration>
          <overlays>
            <overlay>
              <groupId>org.kuali.kra</groupId>
              <artifactId>kc_project</artifactId>
            </overlay>
          </overlays>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

The important parts to notice are the

dependencies - there are 2 kc_project dependencies. One is for the JAR we installed and the other is for the WAR.
```
    <dependencies>
    <dependency>
      <groupId>org.kuali.kra</groupId>
      <artifactId>kc_project</artifactId>
      <version>2.0</version>
      <type>war</type>
    </dependency>
    <dependency>
      <groupId>org.kuali.kra</groupId>
      <artifactId>kc_project</artifactId>
      <version>2.0</version>
      <scope>provided</scope>
      <type>jar</type>
    </dependency>
```
Notice that the JAR dependency uses the provided flag. This states how to use the jars in classpath when building. Here is an excerpt from Maven POM Reference
provided - this is much like compile, but indicates you expect the JDK or a container to provide it at runtime. It is only available on the compilation and test classpath, and is not transitive.

repositories - I added the rice repository to pick up all the Rice dependencies at build time

 <repositories>
  <repository>
   <id>kuali</id>
   <name>Kuali Repository</name>
   <url>https://test.kuali.org/maven</url>
   <snapshots><enabled>true</enabled></snapshots>
  </repository>
    </repositories>

plugins - here's what actually does the overlay

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <configuration>
          <overlays>
            <overlay>
              <groupId>org.kuali.kra</groupId>
              <artifactId>kc_project</artifactId>
            </overlay>
          </overlays>
        </configuration>
      </plugin>
    </plugins>
  </build>

4 Add a Custom O/R Mapping File

Create a file in src/main/resources/com/rsmart/kuali/kc which is my institution's module path. I call it rsmart-repository.xml

<?xml version="1.0" encoding="UTF-8"?>
<descriptor-repository version="1.0">
</descriptor-repository>

5 Add a Custom Spring Beans File

To load our O/R mapping, we'll need to wire it up with Spring. Kuali has a facility to handle this. We just create a CustomSpringBeans.xml file

<?xml version="1.0" encoding="UTF-8"?>
<!--
 Copyright 2005-2010 The Kuali Foundation.

 Licensed under the Educational Community License, Version 1.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at

 http://www.opensource.org/licenses/ecl1.php

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
       xmlns:p="http://www.springframework.org/schema/p" 
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
                           http://www.springframework.org/schema/tx
                           http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
                           http://www.springframework.org/schema/aop
                           http://www.springframework.org/schema/aop/spring-aop-2.0.xsd"> 
  <bean id="customModuleConfiguration-parentBean" class="org.kuali.rice.kns.bo.ModuleConfiguration" abstract="true">
    <property name="databaseRepositoryFilePaths">
      <list>
        <value>com/rsmart/kuali/kc/rsmart-repository.xml</value>
      </list>
    </property>
  </bean>
</beans>

6 Add Custom Struts Config XML File

Struts has this concept of a context specific configuration where you can have more than one configuration. This is very helpful, but we need to list ours in the web.xml and create it. This is a bare one fresh for putting new forms, actions, forwards, etc... into src/main/webapp/WEB-INF/struts-custom-config.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE struts-config PUBLIC "-//Apache Software Foundation//DTD Struts Configuration 1.2//EN" "http://struts.apache.org/dtds/struts-config_1_2.dtd" [
 <!ENTITY protocol_forwards SYSTEM "struts_protocol_forwards.xml">
 ]>
<struts-config>
  <data-sources>
  </data-sources>
  
  <form-beans>
  </form-beans>
  
  <global-exceptions>
  </global-exceptions>

  <global-forwards>
  </global-forwards>

  <action-mappings>
  </action-mappings>

  <controller processorClass="org.kuali.kra.web.struts.action.KraRequestProcessor" />
  <message-resources factory="org.kuali.rice.kns.web.struts.action.KualiPropertyMessageResourcesFactory" parameter="" />
  <plug-in className="org.kuali.kra.web.struts.action.GlobalFormatterRegistry" />
</struts-config>

Of course, I change the web.xml from

>servlet>
        >servlet-name>action>/servlet-name>
        >servlet-class>org.kuali.rice.kns.web.struts.action.KualiActionServlet>/servlet-class>
>init-param>
  >param-name>config>/param-name>
  >param-value>/WEB-INF/struts-config.xml>/param-value>
>/init-param>
        >init-param>
                >param-name>debug>/param-name>
                >param-value>3>/param-value>
        >/init-param>
        >init-param>
                >param-name>detail>/param-name>
                >param-value>3>/param-value>
        >/init-param>
        >load-on-startup>0>/load-on-startup>
>/servlet>

>servlet>
        >servlet-name>action>/servlet-name>
        >servlet-class>org.kuali.rice.kns.web.struts.action.KualiActionServlet>/servlet-class>
>init-param>
  >param-name>config>/param-name>
  >param-value>/WEB-INF/struts-config.xml,/WEB-INF/struts-custom-config.xml>/param-value>
>/init-param>
        >init-param>
                >param-name>debug>/param-name>
                >param-value>3>/param-value>
        >/init-param>
        >init-param>
                >param-name>detail>/param-name>
                >param-value>3>/param-value>
        >/init-param>
        >load-on-startup>0>/load-on-startup>
>/servlet>

I just added ,/WEB-INF/struts-custom-config.xml to the parameter.

7 Replace the KraServiceLocator.java

This is a little strange because you are not extending, but overriding the KraServiceLocator class. That means that if there are any changes made to it, you will probably not pick those up unless you explicitly know. This is of course, a maintenance issue, but we're only making minor changes. Remaking them is not a hassle.

7.1 Add the Custom Spring Beans as a Constant

First, we create a constant in KraServiceLocator called CUSTOM_SPRING_BEANS

private static final String CUSTOM_SPRING_BEANS = "com/rsmart/kuali/kc/CustomSpringBeans.xml";

7.2 Add the Constant to the springFiles Array

Now we make use of the constant.

...
...
    private static final class ContextHolder {
        
        static String[] springFiles = new String[] {COMMON_SPRING_BEANS,BUDGET_SPRING_BEANS, AWARD_SPRING_BEANS, IRB_SPRING_BEANS, COMMITTEE_SPRING_BEANS, 
                                                    INSTITUTIONAL_PROPOSAL_SPRING_BEANS, QUESTIONNAIRE_SPRING_BEANS, TIME_AND_MONEY_SPRING_BEANS, CUSTOM_SPRING_BEANS};
...
...
    }

8 Love

Now you have your overlay project. Just run the following to create your war.

% mvn -Dmaven.test.skip=true package

You will see a new WAR file in target

leo@behemoth~/.workspace/rsmart/kc
(18:54:16) [48] ls target/
classes   kc-1.0-SNAPSHOT.war war
kc-1.0-SNAPSHOT  maven-archiver

KIS Me Kate - RPM Packaging KFS Part 4

2011-04-02T12:29:00.000-07:00

KC Packaging

Packaging KC is much the same as KFS. The only difference is that when you build, you are using maven instead of ant.

Setting up Hudson

Therefore, to get around adding my own hooks to the build, all I did was use Hudson. First, I setup a maven build.
It is the equivalent to

% mvn -Dmaven.test.skip=true package

Then, I created an invoke shell command

TARFILE=kuali-coeus-2.0-$(cut -d= -f 2 kc-2.0/version.properties).tar.gz
rm -rf kuali-coeus-2.0
mkdir kuali-coeus-2.0
cp -rf kc-2.0/target/kc_custom-2.0 kuali-coeus-2.0

for x in $HOME/kuali/main/[a-z]*; do 
    mkdir -p kuali-coeus-2.0/kuali/main/$(basename $x)/
    cp -rf $HOME/kuali/main/$(basename $x)/kc-config.xml kuali-coeus-2.0/kuali/main/$(basename $x)
done
cp $HOME/lib/ojdbc* kuali-coeus-2.0/kc_custom-2.0/WEB-INF/lib
cp $HOME/*.properties kuali-coeus-2.0
tar -czf $TARFILE kuali-coeus-2.0 kc-cfg-dbs/
mv $TARFILE /mosaic/data/KITT/SOURCES

You can see that I am using a version.properties. It looks like

leo@behemoth~/.workspace/kc
(11:02:12) [21] cat version.properties 
release=11

Finally, I create the package by calling the build-rpm.xml using ant.
The build-rpm.xml looks like

<?xml version="1.0" encoding="UTF-8"?>
<project             name="kc" 
                  default="build" 
         xmlns:kitt-tools="urn:com.rsmart.ant">
  <target name="build" depends="filter-spec">
    <exec executable="rpmbuild">
      <arg value="-bb" />
      <arg value="kc.spec" />
    </exec>
  </target>

  <target name="filter-spec">
    <property file="version.properties" />
    <kitt-tools:filter srcfile="kc.spec.template"
                       filename="kc.spec" />
    
  </target>
  
  <macrodef uri="urn:com.rsmart.ant" name="filter">
    <attribute name="srcfile" />
    <attribute name="filename" />
    <sequential>
      <loadfile property="buildroot.filter.template"
                srcfile="@{srcfile}">
        <filterchain>
          <expandproperties/>
        </filterchain>
      </loadfile>

      <echo file="@{filename}">${buildroot.filter.template}</echo>
    </sequential>
  </macrodef>
</project>

A lot like our original build.xml for KFS. The differences here are

No vendor path, so we use build-rpm.xml instead of build.xml
no workflow
no changelogs
Building doesn't happen here since hudson did that for us
No importing the other build.xml

You can see we use a kc.spec.template and filter that just as we did with kfs.

KC spec file

Here is the spec file I used


%define __os_install_post %{nil}
%define debug_package %{nil} 

Summary: Kuali Coeus
Name: kuali-coeus
Version: 2.0
Release: %release
Provides: kuali-coeus
License: EPL
BuildArch: noarch
Source0: kuali-coeus-2.0-%release.tar.gz
BuildRoot: /tmp/kc
Requires: ant
Group: Development/Tools
Packager: przybyls@arizona.edu

%package settings-${build.environment}
Summary: External configuration settings for Kuali Coeus
Group: System/Base
Requires: kuali-coeus

%package changelogs
Summary: Kuali Coeus KITT Customization Schema
Group: System/Base
Requires: kc,liquibase,wget

%description
The Kuali Foundation research administration software

%description changelogs
Mosaic Kuali Coeus Environment Database Schema for KITT customizations based on KITT
modification set %release

%description settings-${build.environment}
Mosaic Kuali Coeus external configuration and settings. These files are located
in /home/tomcat/app

%prep
%setup -q

%install
TOOLSDIR=%{_builddir}/kuali-coeus-2.0/kitt-tools-1.0
mkdir -p %{buildroot}/home/tomcat/kitt-tools
mkdir -p %{buildroot}/home/tomcat/kitt-tools/bin
mkdir -p %{buildroot}/home/tomcat/kitt-tools/config
mkdir -p %{buildroot}/home/tomcat/kitt-tools/lib
mkdir -p %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF

cp ${build.environment}.properties %{buildroot}/home/tomcat/kitt-tools
cp $HOME/apache-ant/lib/ant.jar %{buildroot}/home/tomcat/kitt-tools/lib
cp $HOME/apache-ant/lib/ant-launcher.jar %{buildroot}/home/tomcat/kitt-tools/lib

cd kc_custom-2.0/WEB-INF/classes/META-INF/
sed -e 's/\(.*build.environment.*"false">\).*/\1${build.environment}<\/param>/' kc-config-defaults.xml | sed -e 's/\(.*build.version.*"false">\).*/\12.0-%release<\/param>/' > /tmp/kc-config-defaults.xml
mv /tmp/kc-config-defaults.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF
cd -

cp kc_custom-2.0/WEB-INF/web-${build.environment}.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/web.xml
cp -rf kuali/main/${build.environment} %{buildroot}/home/tomcat/kuali/main/


cat <<EOF > %{buildroot}/home/tomcat/.rpmmacros
%%_topdir /home/tomcat/.workspace/redhat
%%_dbpath /home/tomcat/rpm
EOF

cat <<EOF > %{buildroot}/home/tomcat/kitt-tools/.envrc
${build.environment}
EOF

mv %{_builddir}/kuali-coeus-2.0/kc_custom-2.0 %{buildroot}/usr/share/tomcat5/webapps/kra/
mv %{_builddir}/kc-cfg-dbs/update* %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/

set -x 

ant -f /dev/stdin -Dbuild.environment=${build.environment}<<EOF
<?xml version="1.0"?>
<project name="kitt-tools" default="run" basedir="." xmlns:kitt-tools="urn:com.rsmart.ant">
    <target name="run">
        <property file="$TOOLSDIR/\${build.environment}.properties" />
        <echo file="%{buildroot}/home/tomcat/kitt-tools/credentials.properties">
source.driver=\${dbcopy.default.driver}
source.url=jdbc:oracle:thin:@uaz-kf-d02.mosaic.arizona.edu:1521:UAZKRDEV
source.username=sandbox
source.password=kulowner
source.schema=SANDBOX
target.driver=\${dbcopy.default.driver}
target.url=\${oracle.datasource.url}
target.username=\${datasource.username}
target.schema=KULOWNER
encrypted.password=\${encrypted.password}
        </echo>
    </target>
</project>
EOF

%clean
rm -rf %{buildroot}

%files settings-${build.environment}
%defattr(2770,tomcat,kuali)
/home/tomcat/.rpmmacros
/home/tomcat/kitt-tools
/home/tomcat/kuali/main/${build.environment}/kc-config.xml
/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/web.xml
/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF/kc-config-defaults.xml


%files changelogs
%defattr(-,tomcat,kuali)
%config /home/tomcat/kuali/main/2.0-%release/changesets/update.xml
%config /home/tomcat/kuali/main/2.0-%release/changesets/update/

%files
%defattr(2770,tomcat,kuali)
/usr/share/tomcat5/webapps/kra/

%pre
rm -rf /usr/share/tomcat5/webapps/kra-*

%post
set -x

MYPWD=$PWD
cd /usr/share/tomcat5/webapps
mv kra kra-$(cat ~tomcat/kitt-tools/.envrc)
cd $MYPWD

%post changelogs 
set -x

VERSION=%release
CURRENT=$(ls -t ~tomcat/kuali/main/|grep -v %release|head -1|cut -d- -f 2)
REPO_URL=
KC_VERSION=2.0

for x in $(seq $(expr $CURRENT + 1) $VERSION);
do
    if [ ! -e ~tomcat/kuali/main/$KC_VERSION-$x ];
    then
        cd ~tomcat/kuali/main
        wget -r -l 2 --no-parent -nH --cut-dirs=5 $REPO_URL/$KC_VERSION-$x/
    fi

    cd ~tomcat/kuali/main/$KC_VERSION-$x/changesets
    liquibase --changeLogFile=update.xml --logLevel=finest update
    liquibase --logLevel=finest tag $KC_VERSION.$x
    cd 
done

cd -

if [ -e ~tomcat/kuali/main/$KC_VERSION-$(expr %release + 1) ];
then
    echo '%release' > /tmp/lquninst
fi

%postun changelogs
if [ -e /tmp/lquninst ];
then

    START=%release
    END=$(cat /tmp/lquninst)
    for x in $(seq $START -1 $END); 
    do 
        cd /home/tomcat/kuali/main/2.0-$x/changesets
        liquibase --changeLogFile=update.xml rollback 2.0.$(expr $x - 1)

        cd -;
    done
    rm /tmp/lquninst
fi 
exit

The script is pretty crazy looking, so I will take a jab at explaining it. It will help to explain the major differences in project organization and structure between KFS and KC.

New XML Configuration for KC/Rice in kuali/main/dev
no more security.properties (normally where database passwords are stored)
No build.properties since we're not using ant. Configuration is now in kc-config.xml in kuali/main/<environment>

Handle PreProcessing

Keeping that in mind, let's have a look at the %install section which is where most of the work that is going on is. Since we can't bootstrap or hook into the regular maven packaging for the WAR, what I've done instead is to handle that preprocessing here.

Keep in mind that since this is the %install directive, this is not packaging yet. This is just creating the build sandbox we're going to package later. So first we need to prepare the structure

TOOLSDIR=%{_builddir}/kuali-coeus-2.0/kitt-tools-1.0
mkdir -p %{buildroot}/home/tomcat/kitt-tools
mkdir -p %{buildroot}/home/tomcat/kitt-tools/bin
mkdir -p %{buildroot}/home/tomcat/kitt-tools/config
mkdir -p %{buildroot}/home/tomcat/kitt-tools/lib
mkdir -p %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF

You can see this is building out the path for the webapp to live. There is also this new concept of a kitt-tools path. These are actually environment utilities for administrating KC. They are outside the scope of this explanation, but we do have to add to them a few things during installation. One of these is the credential.properties

ant -f /dev/stdin -Dbuild.environment=${build.environment}<<EOF
<?xml version="1.0"?>
<project name="kitt-tools" default="run" basedir="." xmlns:kitt-tools="urn:com.rsmart.ant">
    <target name="run">
        <property file="$TOOLSDIR/\${build.environment}.properties" />
        <echo file="%{buildroot}/home/tomcat/kitt-tools/credentials.properties">
source.driver=\${dbcopy.default.driver}
source.url=jdbc:oracle:thin:@uaz-kf-d02.mosaic.arizona.edu:1521:UAZKRDEV
source.username=sandbox
source.password=kulowner
source.schema=SANDBOX
target.driver=\${dbcopy.default.driver}
target.url=\${oracle.datasource.url}
target.username=\${datasource.username}
target.schema=KULOWNER
encrypted.password=\${encrypted.password}
        </echo>
    </target>
</project>
EOF

Ant is used to populate this based on values set in the configuration.

Each environment is going to get its own kc-config.xml. For example, TST would be kuali/main/tst/kc-config.xml. Usually, the there are very slight differences between each environment configuration. Usually, the only difference is the database password. For most of the common settings, we use a kc-config-defaults.xml. These files are maintained on the build system (where hudson is living). As a result our build version is usually clobbered by the default. I do a little hacking to get around that. The following takes care of that for us.

cd kc_custom-2.0/WEB-INF/classes/META-INF/
sed -e 's/\(.*build.environment.*"false">\).*/\1${build.environment}<\/param>/' kc-config-defaults.xml | sed -e 's/\(.*build.version.*"false">\).*/\12.0-%release<\/param>/' > /tmp/kc-config-defaults.xml
mv /tmp/kc-config-defaults.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF

We have environment specific web.xml files for any weird changes that are by environment

cp kc_custom-2.0/WEB-INF/web-${build.environment}.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/web.xml
cp -rf kuali/main/${build.environment} %{buildroot}/home/tomcat/kuali/main/

Once a package is installed, tools that we install take advantage of knowing what environment they're on. We handle this by creating an .envrc file on the environment.

cat <<EOF > %{buildroot}/home/tomcat/kitt-tools/.envrc
${build.environment}
EOF

The last thing done is to move the changelogs and the webapp to it's environment agnostic locations. Remember from Part 2 we move these files into their appropriate locations during post processing

mv %{_builddir}/kuali-coeus-2.0/kc_custom-2.0 %{buildroot}/usr/share/tomcat5/webapps/kra/
mv %{_builddir}/kc-cfg-dbs/update* %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/

New packages

Aside from the %install, the spec file and packages are very much the same or at least familiar to the KFS packages. For KC, it was decided that workflow would be handled manually instead of automatically, so the packages are:

kuali-coeus-2.0-1.noarch.rpm
kuali-coeus-settings-2.0-1.noarch.rpm
kuali-coeus-changelogs-2.0-1.noarch.rpm

KIS Me Kate - RPM Packaging KFS Part 3

2011-04-02T11:35:00.000-07:00

Overview/Recap

This is the last part of a 3 part series on packaging KFS with RPM. Just when you thought there wasn't anything left to say about the subject, there's more. What didn't we cover last time?

Workflow packaging - this is actually useful to separate from the main package. Sometimes, you do not want to install/upgrade/reinstall your workflow definitions
Database Upgrade/Installation - if you use liquibase, this is a very useful thing to split out from the main package. It is a useful thing to on-demand upgrade your database.
KC Setup/Packaging (Part 4) - backtracking a little and it doesn't really have anything to do with KFS, but you have to admit that if you're interested in packaging KFS, you're also interested in packaging KC.

Workflow Packaging

This refers to workflow as in the workflow XML that one ingests (usually manually). Sometimes workflow changes are couple to java source code changes in Rice. As a project manager/release manager, you want your project to deploy with as little hiccups as possible. This includes all of your changes that are interdependent to be deployed. If you deploy documents that require workflow changes, your application may not work unless you get those changes in somehow. I wouldn't trust a person to do it, so how do you get this done automatically?

In a previous post, I showed some configuration source code

rice.dev.mode=false
rice.standalone=false
rice.kew.xml.pipeline.lifecycle.enabled=true

These are important. I'll explain:

rice.kew.xml.pipeline.lifecycle.enabled - turns on a thread that runs periodically to ingest KEW xml (it does not use quartz, but an internal scheduler)
rice.dev.mode - you want to set to false because unless, it is in dev, the xml pipeline will not run (regardless of the previous setting). There's a good reason for this. You generally do not want this running in any kind of production environment.
rice.standalone - for now we are building KFS with rice running bundled/embedded. If this were set to false, then our rice would run separately, and we wouldn't be ingesting workflow through the KFS application.

Now, let's move from theory to practice. It appears that things are mostly setup through our configuration. What we need next is

Copy workflow xml to the appropriate ingestion directory during the build process
Setup the spec file so that the files are included in their correct locations at packaging

Modify Build to Move Workflow XML

We now need to modify our build.xml in vendor/<your institution>/. In a previous post, there is a target called dist-rpm. It looks like,

<target        name="dist-war" 
          description="Kuali distribution plus post processing."
              depends="init-classpath,dist">
    <fail unless="build.environment">Need the build.environment to build</fail>
    
  </target>

  <target name="dist-rpm" depends="prepare-rpm,dist-war" />

We add a new target dist-workflow

 <target        name="dist-war" 
          description="Kuali distribution plus post processing."
              depends="init-classpath,dist">
    <fail unless="build.environment">Need the build.environment to build</fail>
    
  </target>

<target        name="dist-workflow" 
          description="Kuali post processing for KEW XML."
              depends="init-classpath">
    <fail unless="build.environment">Need the build.environment to build</fail>
    
    <deploy:workflow-sieve release="${build.version}" kfspath="${basedir}" />
   
   <mkdir dir="${rpm.ingestion.directory}" />
    
    <copy todir="${rpm.ingestion.directory}" flatten="true">
      <fileset dir="${work.directory}/src/com" erroronmissingdir="false">
        <include name="**/workflow/*.xml" />
      </fileset>
      <fileset dir="${work.directory}/src/edu" erroronmissingdir="false">
        <include name="**/workflow/*.xml" />
      </fileset>

    </copy>
  </target>

  <target name="dist-rpm" depends="prepare-rpm,dist-war" />

Obviously, we want to find the workflow xml and copy it to our desired location which is

rpm.ingestion.directory=${rpm.external.work.directory}/staging/workflow/pending/

set in the rpm.properties file mentioned in KIS Me Kate - RPM Packaging KFS Part 2. There is a caveat though. In recent versions of Rice, a new ingestion of a workflow document type does NOT replace the old document type. It creates a new one. The old type still exists. This means that with subsequent ingestion, new document types will be created regardless of their differences. If your institution fancies having daily building/packaging, you could find yourself with a rather large list of document types with very little different from each other. How do we get around this? What I did was create a workflow-sieve task in the macros-rpm.xml that determines whether the workflow XML had any changes. This is what it looks like:

<project  xmlns:deploy="urn:com.rsmart.kuali">
...
...
   <macrodef uri="urn:edu.arizona.kitt" name="workflow-sieve">
    <attribute name="release" />
    <attribute name="kfsPath" />
    <sequential>
      <echo file="/tmp/workflow-sieve.py"><![CDATA[
#!/usr/bin/env python

import os.path
import re
import sys
from subprocess import *

svnpath = "https://subversion.uits.arizona.edu/kitt-anon/kitt/financial-system/kfs/branches"
trunkpath = "https://subversion.uits.arizona.edu/kitt-anon/kitt/financial-system/kfs/trunk"

def findWorkflowFiles(basedir):
    retval = []
    for root, dirs, files in os.walk(basedir):
        for file in (files):
            if (re.match(".*workflow$", root)):
                newroot = root.split('/work/')[1]
                retval.append('/'.join(['work', newroot, file]))

    return retval

def getLastReleaseRevision(release):
    releaseLoc = svnpath + "/3.0-" + str(release)
    return getRevisionFor(releaseLoc)

def getRevisionFor(path):
    retval = Popen(["svn", "info", path], stdout=PIPE).communicate()[0]
    retval = int(retval.split("Last Changed Rev: ")[1].split("\n")[0])
    return retval

def command(command):
    print 'Executing: ' + command
    os.system(command)


release = int("@{release}".split("-")[1]) - 1
workflowFiles = findWorkflowFiles('@{kfsPath}/work/src/edu/')
workflowFiles.extend(findWorkflowFiles('@{kfsPath}/work/src/com/'))
revision = getLastReleaseRevision(release)

for workflow in workflowFiles:
    filename = trunkpath + "/" + workflow
    print "Checking revision on " + filename
    fileRev = getRevisionFor(filename)
    if (revision > fileRev):
        print "Removing " + workflow + " from package"
        os.remove(workflow)
      ]]>
      </echo>
      <exec executable="${user.home}/python/bin/python">
        <arg value="/tmp/workflow-sieve.py" />
      </exec>
      <delete file="/tmp/workflow-sieve.py" />
    </sequential>
  </macrodef>
...
...
</project>

Above is a simply python script that gets run as part of the workflow-sieve task which occurs during the dist-workflow target! This is great! Now our build is altered sufficiently to handle the workflow XML.

Define Workflow Package

Now I will add workflow to the kfs.spec.template

%define __os_install_post %{nil}

Summary: Kuali Financial System
Name: kfs
Version: ${version}
Release: ${release}
Provides: kfs
License: EPL
BuildArch: noarch
Requires: tomcat5
BuildRoot: /tmp/kfs/
Source0: kfs-${build.version}.tar.gz
Group: Development/Tools
Packager: leo [at] rsmart.com

%package workflow
Summary: Kuali Financial System Workflow Document Types
Group: System/Base
Requires: kfs
...
...

We set our requirement for KFS. this will ensure that all our KFS prerequisites exist before this is installed.
After that, I add my description

%description  workflow
Workflow XML for %release

Notice that after %description, I give the string "workflow". Normally, there is no qualifier for %description. That indicates that it's going into the default package. We qualify with "workflow". This means that there will be a workflow qualifier added to the package name. The resulting package will be called kfs-workflow-4.0-1.noarch.rpm.

Next, we specify our %files just like we did with the default package.

%files workflow
%defattr(-,tomcat,tomcat)
/home/tomcat/app/work/kfs/staging/workflow/

Now when you rerun your packaging, you will have a workflow package that can be installed with your KFS application as well as all your workflow customizations for that release!

Packaging Database Changes

In particular, packaging your liquibase changelogs and having them run on installation! This is actually, a really good idea. Just like workflow changes, your database changes are coupled to your java source code. This is especially the case thanks to ORM. KFS may not even start correctly if you do not have tables mentioned in your mappings. The best way to make sure they exist is to add your liquibase change logs to installation. As an added bonus, you can now have visibility on all changes made to your database from release to release.

Update build.xml

Just like we did with our workflow packaging, we will need to change the build to include the liquibase change logs.

<target name="dist-ddl" depends="init-classpath">
  <mkdir dir="${rpm.ddl.directory}" />
      
  <copy todir="${rpm.ddl.directory}">
    <fileset dir="${work.directory}/db/">
     <include name="changesets/**/*" />
     <include name="scripts/**/*" />
    </fileset>
  </copy>

</target>

<target name="dist-rpm" depends="prepare-rpm,dist-war,dist-ddl,dist-workflow" />

I have added now dist-ddl to the dist-rpm dependencies. I have also created the target for it. All it does is copy files out of kfs-4.0/work/db/changets and kfs-4.0/work/db/scripts into my build directory for packaging.

Add Spec Information

Just like I did with workflow, I now add the %package for changelogs

%package changelogs
Summary: Kuali Financial System KITT Customization Schema
Group: System/Base
Requires: liquibase,kfs,wget

This will create a new package with the name kfs-changelogs-4.0-1.noarch.rpm. Also, notice that my dependency is liquibase. This means that liquibase will required as well as KFS before the changelogs can be run! A program called wget is required too. I will explain why later.

Now I add %description and files.

%description changelogs
Liquibase change logs for %release
...
...
%files changelogs
%defattr(-,tomcat,tomcat)
/home/tomcat/app/ddl/${build.version}/changesets/latest
/home/tomcat/app/ddl/${build.version}/changesets/install.xml
/home/tomcat/app/ddl/${build.version}/changesets/constraints.xml
%config /home/tomcat/app/ddl/${build.version}/changesets/update.xml
%config /home/tomcat/app/ddl/${build.version}/changesets/update/
...
...

The above describes that a new changeset path is created for each ${build.version}. The reason for this is because we want to keep all the changelogs from previous builds. This allows us to quickly revert back in case we need to undo packages. One of the advantages of package management is being able to cleanly and systematically remove the software change without any evidence that it ever happened. The update.xml is listed as a configuration item because we never want this overwritten.

Installation Post processing Script

Unlike with workflow, we now have some pre/post processing to do. Until now, just files are being dropped in for installation. We don't do anything with these files. That is, liquibase is required, but it never runs. The database isn't actually changed yet. We still need to run liquibase on the changelogs.

%post changelogs 
set -x

VERSION=%release
CURRENT=$(ls -t ~tomcat/app/ddl/|grep -v %release|head -1|cut -d- -f 2)
REPO_URL=<your institutions' SVN repo accessible via http>
KFS_VERSION=%version

for x in $(seq $(expr $CURRENT + 1) $VERSION);
do
    if [ ! -e ~tomcat/app/ddl/$KFS_VERSION-$x ];
    then
        cd ~tomcat/app/ddl
        wget -r -l 2 --no-parent -nH --cut-dirs=5 $REPO_URL/$KFS_VERSION-$x/
    fi

    cd ~tomcat/app/ddl/$KFS_VERSION-$x/changesets
    liquibase --changeLogFile=update.xml --logLevel=finest update
    liquibase --logLevel=finest tag %version.$x
    cd 
done

if [ -e ~tomcat/app/ddl/%version-$(expr %release + 1) ];
then
    echo '%release' > /tmp/lquninst
fi

The above script is running liquibase on the current installation. This is pretty tricky which is why it's a shell script. When we upgrade/install our liquibase changelogs, we need to run all of the changes subsequently between the last and current version. For example, if we are upgrading from release 3 - release 15, we need to run all the scripts in between in order. The caveat is that when we upgrade directly (going from package for release 3 to package for release 15), we're skipping packages that contain the changelogs. This means we cannot assume the system has the changelogs installed. This is where wget comes in. We actually setup an anonymous, read-only accessible SVN rep url in the script

REPO_URL=<your institutions' SVN repo accessible via http>

. We now use wget to retrieve from SVN the missing changelogs and install them before proceeding to process them through liquibase.

   liquibase --changeLogFile=update.xml --logLevel=finest update
    liquibase --logLevel=finest tag %version.$x

it is important to note that liquibase is tagging a version after each update. We will use this in a moment.

Uninstallation Post-processing Script

That handles installation/upgrades, but ... what about uninstallations. Well, an uninstallation is pretty much reverting liquibase to the state of the previous changelog. The way we know the state of the previous changelog is that the state was tagged on installation (mentioned earlier). Now we know where to downgrade to. To handle uninstallations, RPM has a directive called %postun. We use that here

%postun kittdb
if [ -e /tmp/lquninst ];
then

    START=%release
    END=$(cat /tmp/lquninst)
    for x in $(seq $START -1 $END); 
    do 
        cd /home/tomcat/app/ddl/%version}-$x/changesets
        liquibase --changeLogFile=update.xml rollback %version.$(expr $x - 1)

        cd -;
    done
    rm /tmp/lquninst
fi 
exit

Just as before, the uninstallation script determines what versions it needs to uninstall, then it loops through each one doing a "rollback" through liquibase.

Part 4 will be packaging KC which is really interesting compared to KFS because it uses maven and the concept of xml configuration by environment instead of property-based configuration.

Constants Done with Spring

2011-04-01T23:41:00.000-07:00

Kuali's Antipattern

The Rice pattern for handling constants in Kuali software is really an antipattern. It is basically an interface with public static final's. They chose an interface because it cannot be instantiated. This is an antipattern because it's not considered OO. An interface/class is created without the prospect of ever being instantiated or used for polymorphism. It's also inconsistent because as an interface is used at first, inner classes are used. Here's an example:

public class KFSConstants extends JSTLConstants implements ParameterKeyConstants {
    private static final long serialVersionUID = 2882277719647128949L;

    public static final String APPLICATION_NAMESPACE_CODE = "KFS";    
    
    public static class ParameterNamespaces {
        public static final String KFS = "KFS-SYS";
        public static final String CHART = "KFS-COA";
        public static final String FINANCIAL = "KFS-FP";
        public static final String GL = "KFS-GL";
    ...
    ...
    }
...
...

Now look at

public interface DisbursementVoucherConstants extends ParameterKeyConstants {

    // Text limits
    public static final int MAX_NOTE_LINE_SIZE = 90;
    
    // payment methods
    public static String PAYMENT_METHOD_CHECK = "P";
    public static String PAYMENT_METHOD_WIRE = "W";
    public static String PAYMENT_METHOD_DRAFT = "F";

    // payee types
    public static final String DV_PAYEE_TYPE_EMPLOYEE = "E";
    public static final String DV_PAYEE_TYPE_VENDOR = "V";
    public static final String DV_PAYEE_TYPE_CUSTOMER = "C";
    public static final String DV_PAYEE_TYPE_SUBJECT_PAYMENT_VENDOR = "VSP";
    public static final String DV_PAYEE_TYPE_REVOLVING_FUND_VENDOR = "VRF";
    
    public static final List VENDOR_PAYEE_TYPE_CODES = Arrays.asList(DV_PAYEE_TYPE_VENDOR, DV_PAYEE_TYPE_SUBJECT_PAYMENT_VENDOR, DV_PAYEE_TYPE_REVOLVING_FUND_VENDOR);

    // document location
    public static final String NO_DOCUMENTATION_LOCATION = "N";

    public static final String TAX_CONTROL_CODE_ALLOWS_EMPLOYEES = "A";
    public static final String TAX_CONTROL_CODE_BEGIN_WITHHOLDING = "B";
    public static final String TAX_CONTROL_CODE_HOLD_PAYMENT = "H";

    public static class DocumentStatusCodes {
        public static final String APPROVED = "A";
        public static final String EXTRACTED = "E";
    }
...
...
}

One is an class with another inner class. The other is an interface with an inner class. It's really inconsistent. Everyone does it differently.

Use Spring for Constants

I propose getting constants from Spring. Here is an example I used in the KIM Ldap Integration:

Create Constants Interface

package org.kuali.rice.kim.util;

import java.util.Collection;

import org.kuali.rice.kim.bo.entity.dto.KimEntityInfo;

/**
 *
 * @author Leo Przybylski (leo [at] rsmart.com)
 */ 
public interface Constants {    
    Collection getTestPrincipalNames();

    String getDefaultChartCode();

    /**
     * Gets the value of entityPrototype
     *
     * @return the value of entityPrototype
     */
    KimEntityInfo getEntityPrototype();

    /**
     * Gets the value of externalIdTypeProperty
     *
     * @return the value of externalIdTypeProperty
     */
    String getExternalIdTypeProperty();

    /**
     * Gets the value of taxExternalIdTypeCode
     *
     * @return the value of taxExternalIdTypeCode
     */
    String getTaxExternalIdTypeCode();

    /**
     * Gets the value of externalIdProperty
     *
     * @return the value of externalIdProperty
     */
    String getExternalIdProperty();

    /**
     * Gets the value of employeePhoneLdapProperty
     *
     * @return the value of employeePhoneLdapProperty
     */
    String getEmployeePhoneLdapProperty();

    /**
     * Gets the value of employeeMailLdapProperty
     *
     * @return the value of employeeMailLdapProperty
     */
    String getEmployeeMailLdapProperty();

    /**
     * Gets the value of defaultCountryCode
     *
     * @return the value of defaultCountryCode
     */
    String getDefaultCountryCode();

    /**
     * Gets the value of personEntityTypeCode
     *
     * @return the value of personEntityTypeCode
     */
    String getPersonEntityTypeCode();

    /**
     * Gets the value of uaidLdapProperty
     *
     * @return the value of uaidLdapProperty
     */
    String getKimLdapIdProperty();

    /**
     * Gets the value of uidLdapProperty
     *
     * @return the value of uidLdapProperty
     */
    String getKimLdapNameProperty();

    /**
     * Gets the value of snLdapProperty
     *
     * @return the value of snLdapProperty
     */
    String getSnLdapProperty();

    /**
     * Gets the value of givenNameLdapProperty
     *
     * @return the value of givenNameLdapProperty
     */
    String getGivenNameLdapProperty();

    /**
     * Gets the value of entityIdKimProperty
     *
     * @return the value of entityIdKimProperty
     */
    String getEntityIdKimProperty();

    /**
     * Gets the value of parameterNamespaceCode
     *
     * @return the value of parameterNamespaceCode
     */
    String getParameterNamespaceCode();

    /**
     * Gets the value of parameterDetailTypeCode
     *
     * @return the value of parameterDetailTypeCode
     */
    String getParameterDetailTypeCode();

    /**
     * Gets the value of mappedParameterName
     *
     * @return the value of mappedParameterName
     */
    String getMappedParameterName();

    /**
     * Gets the value of unmappedParameterName
     *
     * @return the value of unmappedParameterName
     */
    String getUnmappedParameterName();

    /**
     * Gets the value of mappedValuesName
     *
     * @return the value of mappedValuesName
     */
    String getMappedValuesName();

    /**
     * Gets the value of employeeIdProperty
     *
     * @return the value of employeeIdProperty
     */
    String getEmployeeIdProperty();

    /**
     * Gets the value of departmentLdapProperty
     *
     * @return the value of departmentLdapProperty
     */
    String getDepartmentLdapProperty();

    /**
     * Gets the value of employeeTypeProperty
     *
     * @return the value of employeeTypeProperty
     */
    String getEmployeeTypeProperty();

    /**
     * Sets the value of employeeTypeProperty
     *
     * @param argEmployeeTypeProperty Value to assign to this.employeeTypeProperty
     */
    void setEmployeeTypeProperty(String argEmployeeTypeProperty);

    /**
     * Gets the value of employeeStatusProperty
     *
     * @return the value of employeeStatusProperty
     */
    String getEmployeeStatusProperty();

    /**
     * Sets the value of employeeStatusProperty
     *
     * @param argEmployeeStatusProperty Value to assign to this.employeeStatusProperty
     */
    void setEmployeeStatusProperty(String argEmployeeStatusProperty);

    /**
     * Gets the value of defaultCampusCode
     *
     * @return the value of defaultCampusCode
     */
    String getDefaultCampusCode();
    
    /**
     * Sets the value of defaultCampusCode
     *
     * @param argDefaultCampusCode Value to assign to this.defaultCampusCode
     */
    void setDefaultCampusCode(String argDefaultCampusCode);
    
    /**
     * Gets the value of the employee affiliation code
     * 
     * @return the value of employeeAffiliationCode
     */
    String getEmployeeAffiliationCodes();

    /** 
     * Gets the mappings between LDAP and KIM affiliations
     * @return mappings of the form "staff=STAFF,affiliate=AFLT"
     */
    String getAffiliationMappings();
}

Notice, there are only get methods. You'll see easier in a second, but this is how we get Spring beans to be read-only and constants.

Create the Constants Implementation

It's just going to be a pojo. We can easily put this within the the original interface as an inner class since it's just spring that's going to use it.

/**
 * KIM Related Constants Implementation
 *
 * @author Leo Przybylski (leo [at] rsmart.com)
 */ 
static class ConstantsImpl implements Constants {    
    private Collection testPrincipalNames;
    private KimEntityInfo entityPrototype;
    private String externalIdTypeProperty;
    private String taxExternalIdTypeCode;
    private String externalIdProperty;
    private String employeePhoneLdapProperty;
    private String employeeMailLdapProperty;
    private String defaultCountryCode;
    private String personEntityTypeCode;
    private String kimLdapIdProperty;
    private String kimLdapNameProperty;
    private String snLdapProperty;
    private String givenNameLdapProperty;
    private String entityIdKimProperty;
    private String parameterNamespaceCode;
    private String parameterDetailTypeCode;
    private String mappedParameterName;
    private String unmappedParameterName;
    private String mappedValuesName;
    private String employeeIdProperty;
    private String departmentLdapProperty;
    private String employeeTypeProperty;
    private String employeeStatusProperty;
    private String defaultCampusCode;
    private String defaultChartCode;
    private String employeeAffiliationCodes;
    private String affiliationMappings;

    public Collection getTestPrincipalNames() {
        return testPrincipalNames;
    }

    public void setTestPrincipalNames(Collection testPrincipalNames) {
        this.testPrincipalNames = testPrincipalNames;
    }

    /**
     * Gets the value of entityPrototype
     *
     * @return the value of entityPrototype
     */
    public KimEntityInfo getEntityPrototype() {
        // return this.entityPrototype;
        return (KimEntityInfo) getService("entityPrototype");
        // return (KimEntityDefaultInfo) getService("entityPrototype");
    }

    /**
     * Sets the value of entityPrototype
     *
     * @param argEntityPrototype Value to assign to this.entityPrototype
     */
    public void setEntityPrototype(KimEntityInfo argEntityPrototype) {
        this.entityPrototype = argEntityPrototype;
    }

    /**
     * Gets the value of externalIdTypeProperty
     *
     * @return the value of externalIdTypeProperty
     */
    public String getExternalIdTypeProperty() {
        return this.externalIdTypeProperty;
    }

    /**
     * Sets the value of externalIdTypeProperty
     *
     * @param argExternalIdTypeProperty Value to assign to this.externalIdTypeProperty
     */
    public void setExternalIdTypeProperty(String argExternalIdTypeProperty) {
        this.externalIdTypeProperty = argExternalIdTypeProperty;
    }

    /**
     * Gets the value of taxExternalIdTypeCode
     *
     * @return the value of taxExternalIdTypeCode
     */
    public String getTaxExternalIdTypeCode() {
        return this.taxExternalIdTypeCode;
    }

    /**
     * Sets the value of taxExternalIdTypeCode
     *
     * @param argTaxExternalIdTypeCode Value to assign to this.taxExternalIdTypeCode
     */
    public void setTaxExternalIdTypeCode(String argTaxExternalIdTypeCode) {
        this.taxExternalIdTypeCode = argTaxExternalIdTypeCode;
    }

    /**
     * Gets the value of externalIdProperty
     *
     * @return the value of externalIdProperty
     */
    public String getExternalIdProperty() {
        return this.externalIdProperty;
    }

    /**
     * Sets the value of externalIdProperty
     *
     * @param argExternalIdProperty Value to assign to this.externalIdProperty
     */
    public void setExternalIdProperty(String argExternalIdProperty) {
        this.externalIdProperty = argExternalIdProperty;
    }

    /**
     * Gets the value of employeePhoneLdapProperty
     *
     * @return the value of employeePhoneLdapProperty
     */
    public String getEmployeePhoneLdapProperty() {
        return this.employeePhoneLdapProperty;
    }

    /**
     * Sets the value of employeePhoneLdapProperty
     *
     * @param argEmployeePhoneLdapProperty Value to assign to this.employeePhoneLdapProperty
     */
    public void setEmployeePhoneLdapProperty(String argEmployeePhoneLdapProperty) {
        this.employeePhoneLdapProperty = argEmployeePhoneLdapProperty;
    }

    /**
     * Gets the value of employeeMailLdapProperty
     *
     * @return the value of employeeMailLdapProperty
     */
    public String getEmployeeMailLdapProperty() {
        return this.employeeMailLdapProperty;
    }

    /**
     * Sets the value of employeeMailLdapProperty
     *
     * @param argEmployeeMailLdapProperty Value to assign to this.employeeMailLdapProperty
     */
    public void setEmployeeMailLdapProperty(String argEmployeeMailLdapProperty) {
        this.employeeMailLdapProperty = argEmployeeMailLdapProperty;
    }

    /**
     * Gets the value of defaultCountryCode
     *
     * @return the value of defaultCountryCode
     */
    public String getDefaultCountryCode() {
        return this.defaultCountryCode;
    }

    /**
     * Sets the value of defaultCountryCode
     *
     * @param argDefaultCountryCode Value to assign to this.defaultCountryCode
     */
    public void setDefaultCountryCode(String argDefaultCountryCode) {
        this.defaultCountryCode = argDefaultCountryCode;
    }

    /**
     * Gets the value of personEntityTypeCode
     *
     * @return the value of personEntityTypeCode
     */
    public String getPersonEntityTypeCode() {
        return this.personEntityTypeCode;
    }

    /**
     * Sets the value of personEntityTypeCode
     *
     * @param argPersonEntityTypeCode Value to assign to this.personEntityTypeCode
     */
    public void setPersonEntityTypeCode(String argPersonEntityTypeCode) {
        this.personEntityTypeCode = argPersonEntityTypeCode;
    }

    /**
     * Sets the value of kimLdapIdProperty
     *
     * @param kimLdapIdProperty value to set
     */
    public void setKimLdapIdProperty(String kimLdapIdProperty) {
        this.kimLdapIdProperty = kimLdapIdProperty;
    }

    /**
     * Gets the value of kimLdapIdProperty
     *
     * @return the value of kimLdapIdProperty
     */
    public String getKimLdapIdProperty() {
        return kimLdapIdProperty;
    }

    /**
     * Gets the value of kimLdapNameProperty
     *
     * @param kimLdapNameProperty the value of kimLdapNameProperty
     */
    public void getKimLdapNameProperty(String kimLdapNameProperty) {
        this.kimLdapNameProperty = kimLdapNameProperty;
    }

    /**
     * Gets the value of kimLdapNameProperty
     *
     * @return the value of kimLdapNameProperty
     */
    public String getKimLdapNameProperty() {
        return kimLdapNameProperty;
    }

    /**
     * Gets the value of snLdapProperty
     *
     * @return the value of snLdapProperty
     */
    public String getSnLdapProperty() {
        return this.snLdapProperty;
    }

    /**
     * Sets the value of snLdapProperty
     *
     * @param argSnLdapProperty Value to assign to this.snLdapProperty
     */
    public void setSnLdapProperty(String argSnLdapProperty) {
        this.snLdapProperty = argSnLdapProperty;
    }

    /**
     * Gets the value of givenNameLdapProperty
     *
     * @return the value of givenNameLdapProperty
     */
    public String getGivenNameLdapProperty() {
        return this.givenNameLdapProperty;
    }

    /**
     * Sets the value of givenNameLdapProperty
     *
     * @param argGivenNameLdapProperty Value to assign to this.givenNameLdapProperty
     */
    public void setGivenNameLdapProperty(String argGivenNameLdapProperty) {
        this.givenNameLdapProperty = argGivenNameLdapProperty;
    }

    /**
     * Gets the value of entityIdKimProperty
     *
     * @return the value of entityIdKimProperty
     */
    public String getEntityIdKimProperty() {
        return this.entityIdKimProperty;
    }

    /**
     * Sets the value of entityIdKimProperty
     *
     * @param argEntityIdKimProperty Value to assign to this.entityIdKimProperty
     */
    public void setEntityIdKimProperty(String argEntityIdKimProperty) {
        this.entityIdKimProperty = argEntityIdKimProperty;
    }

    /**
     * Gets the value of parameterNamespaceCode
     *
     * @return the value of parameterNamespaceCode
     */
    public String getParameterNamespaceCode() {
        return this.parameterNamespaceCode;
    }

    /**
     * Sets the value of parameterNamespaceCode
     *
     * @param argParameterNamespaceCode Value to assign to this.parameterNamespaceCode
     */
    public void setParameterNamespaceCode(String argParameterNamespaceCode) {
        this.parameterNamespaceCode = argParameterNamespaceCode;
    }

    /**
     * Gets the value of parameterDetailTypeCode
     *
     * @return the value of parameterDetailTypeCode
     */
    public String getParameterDetailTypeCode() {
        return this.parameterDetailTypeCode;
    }

    /**
     * Sets the value of parameterDetailTypeCode
     *
     * @param argParameterDetailTypeCode Value to assign to this.parameterDetailTypeCode
     */
    public void setParameterDetailTypeCode(String argParameterDetailTypeCode) {
        this.parameterDetailTypeCode = argParameterDetailTypeCode;
    }

    /**
     * Gets the value of mappedParameterName
     *
     * @return the value of mappedParameterName
     */
    public String getMappedParameterName() {
        return this.mappedParameterName;
    }

    /**
     * Sets the value of mappedParameterName
     *
     * @param argMappedParameterName Value to assign to this.mappedParameterName
     */
    public void setMappedParameterName(String argMappedParameterName) {
        this.mappedParameterName = argMappedParameterName;
    }

    /**
     * Gets the value of unmappedParameterName
     *
     * @return the value of unmappedParameterName
     */
    public String getUnmappedParameterName() {
        return this.unmappedParameterName;
    }

    /**
     * Sets the value of unmappedParameterName
     *
     * @param argUnmappedParameterName Value to assign to this.unmappedParameterName
     */
    public void setUnmappedParameterName(String argUnmappedParameterName) {
        this.unmappedParameterName = argUnmappedParameterName;
    }

    /**
     * Gets the value of mappedValuesName
     *
     * @return the value of mappedValuesName
     */
    public String getMappedValuesName() {
        return this.mappedValuesName;
    }

    /**
     * Sets the value of mappedValuesName
     *
     * @param argMappedValuesName Value to assign to this.mappedValuesName
     */
    public void setMappedValuesName(String argMappedValuesName) {
        this.mappedValuesName = argMappedValuesName;
    }
    
    /**
     * Gets the value of employeeIdProperty
     *
     * @return the value of employeeIdProperty
     */
    public String getEmployeeIdProperty() {
        return this.employeeIdProperty;
    }

    /**
     * Sets the value of employeeIdProperty
     *
     * @param argEmployeeIdProperty Value to assign to this.employeeIdProperty
     */
    public void setEmployeeIdProperty(String argEmployeeIdProperty) {
        this.employeeIdProperty = argEmployeeIdProperty;
    }

    /**
     * Gets the value of departmentLdapProperty
     *
     * @return the value of departmentLdapProperty
     */
    public String getDepartmentLdapProperty() {
        return this.departmentLdapProperty;
    }

    /**
     * Sets the value of departmentLdapProperty
     *
     * @param argDepartmentLdapProperty Value to assign to this.departmentLdapProperty
     */
    public void setDepartmentLdapProperty(String argDepartmentLdapProperty) {
        this.departmentLdapProperty = argDepartmentLdapProperty;
    }

    /**
     * Gets the value of employeeTypeProperty
     *
     * @return the value of employeeTypeProperty
     */
    public String getEmployeeTypeProperty() {
        return this.employeeTypeProperty;
    }

    /**
     * Sets the value of employeeTypeProperty
     *
     * @param argEmployeeTypeProperty Value to assign to this.employeeTypeProperty
     */
    public void setEmployeeTypeProperty(String argEmployeeTypeProperty) {
        this.employeeTypeProperty = argEmployeeTypeProperty;
    }

    /**
     * Gets the value of employeeStatusProperty
     *
     * @return the value of employeeStatusProperty
     */
    public String getEmployeeStatusProperty() {
        return this.employeeStatusProperty;
    }

    /**
     * Sets the value of employeeStatusProperty
     *
     * @param argEmployeeStatusProperty Value to assign to this.employeeStatusProperty
     */
    public void setEmployeeStatusProperty(String argEmployeeStatusProperty) {
        this.employeeStatusProperty = argEmployeeStatusProperty;
    }

    /**
     * Gets the value of defaultCampusCode
     *
     * @return the value of defaultCampusCode
     */
    public String getDefaultCampusCode() {
        return this.defaultCampusCode;
    }

    /**
     * Sets the value of defaultCampusCode
     *
     * @param argDefaultCampusCode Value to assign to this.defaultCampusCode
     */
    public void setDefaultCampusCode(String argDefaultCampusCode) {
        this.defaultCampusCode = argDefaultCampusCode;
    }


    public void setDefaultChartCode(String chartCode) {
        this.defaultChartCode = chartCode;
    }

    public String getDefaultChartCode() {
        return defaultChartCode;
    }

    public String getEmployeeAffiliationCodes() {
        return employeeAffiliationCodes;
    }

    public void setEmployeeAffiliationCodes(String employeeAffiliationCodes) {
        this.employeeAffiliationCodes = employeeAffiliationCodes;
    }

    public String getAffiliationMappings() {
        return affiliationMappings;
    }

    public void setAffiliationMappings(String affiliationMappings) {
        this.affiliationMappings = affiliationMappings;
    }
}

It is static, so it can be instantiated, but it's also package level. This is important. Now, normal classes cannot just instantiate a Constants instance and override spring programmatically. Spring can still instantiate it and set it at startup though, so we are safe. Spring can instantiate and spring only.

Define the Constants Bean in Spring

Now let us set our constants


 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
...
...
    
 
    
      
      
      
      
      
      
      
      
      
      
      
      
      
      
        
      
      
      
      
      
      
      
      
      
  
...
...

Inject the Constants Bean into a Service

There, now the Constants is instantiated and stored in Spring as a bean. We can now easily inject this into other classes that need it.

Now use it

Here is an example of the usage:

        String affiliationCode = getAffiliationTypeCodeForName(primaryAffiliation);
        KimEntityAffiliationInfo aff1 = new KimEntityAffiliationInfo();
        aff1.setAffiliationTypeCode(affiliationCode == null ? "AFLT" : affiliationCode);
        aff1.setCampusCode(getConstants().getDefaultCampusCode());
        aff1.setEntityAffiliationId("" + affiliationId++);
        aff1.setDefault(true);
        aff1.setActive(true);

Use Without Injection


...
        String affiliationCode = getAffiliationTypeCodeForName(primaryAffiliation);
        KimEntityAffiliationInfo aff1 = new KimEntityAffiliationInfo();
        aff1.setAffiliationTypeCode(affiliationCode == null ? "AFLT" : affiliationCode);
        aff1.setCampusCode(getConstants().getDefaultCampusCode());
        aff1.setEntityAffiliationId("" + affiliationId++);
        aff1.setDefault(true);
        aff1.setActive(true);
...
public Constants getConstants() {
    return SpringContext.getService("kimConstants");
}

That's it! I found this very useful when needing to change constants or finding what the values of constants are. The public static final boiler plate always makes it difficult to read, and the inner classes really make things a mess.

KIS Me Kate - RPM Packaging KFS Part 2

2011-03-24T12:59:00.001-07:00

Now that we know packaging is a pretty good idea, let's have a look at what needs to be done to actually create the package. I'm going to take a "city building" approach. I'll start small with a no frills RPM distribution of KFS. I will slowly add more complexity until there is a fully robust package building system in place.

Let's Get Started

What do we need for a bare-bones installation of KFS?

The source code (can't forget that).
Prebuilt local installation of KFS
RPM spec file
A repository to store packages in
rpm-build on the packaging machine
A machine setup for packaging
An environment ready to install to

1 Prebuilt local installation of KFS

Keep in mind that when you're packaging software, it is best to have the software built as much as possible. Compiling while installing is not the best thing. It requires more tools (development tools) on the machine you're installing on, it can complicate installation with further dependencies. Development dependencies are the worst to handle. You don't want those on your installation machine.

You want to get this right. Let me show you what I did to create an installation ready for packaging. In the interest of time, I'm going to skip getting the source code and setting up your tools. I am going to assume that as a developer or configuration manager, you're already familiar with the tools and how to get the source code. Let's just pretend you just checked out the KFS source code from the Kuali Foundation as is.

Once you checkout KFS, you should have something that looks like this:

build cnv-build.properties reg-build.properties
build-foundation.xml dev-build.properties test
build.xml ptd-build.properties work

1.1 Setup Pre-processing

Here we can add our pre-processing hooks. Create a directory called vendor/<yourinstitution>/deployment. This is where all the processing code is going to go, so we do not disrupt the original KFS codebase. Within, I add a build.xm:


<?xml version="1.0" encoding="UTF-8"?>
<project          name="kfs" 
               default="dist-rpm" basedir="../../../"
          xmlns:deploy="urn:com.rsmart.kuali"
         xmlns:twitter="urn:com.rsmart.kuali.twitter">

  <property file="${tools.directory}/home/${ant.project.name}-build.properties" 
/>
  <import file="${basedir}/build.xml"/>
  <import file="${tools.directory}/macros-rpm.xml" />

  <target      name="init-classpath" 
          description="Initializes classpath for normal execution. Classpath consists of the normal kuali classpath plus build dependencies."
              depends="init-properties">
    <property  name="build.temp.directory" 
              value="${deploy.working.directory}/deploy-${build.environment}" />
    <property file="${basedir}/vendor/<yourinstitution>/deployment/rpm.properties" />    
  </target>

  <target name="prepare-rpm" depends="init-classpath">
    <mkdir dir="${rpm.appserver.deploy.dir}" />
    <mkdir dir="${rpm.external.config.directory}" />
    <mkdir dir="${rpm.appserver.localhost.dir}" />
    <mkdir dir="${rpm.external.config.directory}" />
    <mkdir dir="${rpm.settings.directory}" />
    <mkdir dir="${rpm.security.directory}" />
    <mkdir dir="${rpm.logs.directory}" />
    <mkdir dir="${rpm.reports.directory}" />
    <mkdir dir="${rpm.external.work.directory}" />
  </target>

  <target        name="dist-war" 
          description="Kuali distribution plus post processing."
              depends="init-classpath,dist">
    <fail unless="build.environment">Need the build.environment to build</fail>
    
  </target>

  <target name="dist-rpm" depends="prepare-rpm,dist-war" />
</project>

You probably noticed right away the following line:


<project          name="kfs" 
               default="dist-rpm" basedir="../../../"
          xmlns:deploy="urn:com.rsmart.kuali"
         xmlns:twitter="urn:com.rsmart.kuali.twitter">

  <property file="${tools.directory}/home/${ant.project.name}-build.properties" />
  <import file="${basedir}/build.xml"/>
  <import file="${tools.directory}/macros-rpm.xml" />
...
...
</project>

The answer is, "Yes"! We're actually importing the original build.xml file. The reason for this is we want to execute the normal build and add hooks to it from within. This build.xml is executed via:

% ant -f vendor/<yourinstitution>/deployment/build.xml dist-war

You can see from the init-classpath ant target the above script uses the following rpm.properties file:


rpm.appserver.deploy.dir=${rpm.build.root}${appserver.deploy.dir}/
rpm.appserver.localhost.dir=${rpm.build.root}${appserver.localhost.dir}/
rpm.external.config.directory=${rpm.build.root}${external.config.directory}/
rpm.settings.directory=${rpm.build.root}${settings.directory}/
rpm.security.directory=${rpm.build.root}${security.directory}/
rpm.logs.directory=${rpm.build.root}${logs.directory}/
rpm.reports.directory=${rpm.build.root}${reports.directory}/
rpm.staging.directory=${rpm.build.root}${staging.directory}/
rpm.external.work.directory=${rpm.build.root}${external.work.directory}/
rpm.log4j.settings.file=${rpm.build.root}${log4j.settings.file}/
rpm.security.property.file=${rpm.build.root}${security.property.file}/
rpm.ingestion.directory=${rpm.external.work.directory}/staging/workflow/ingestion/
rpm.ddl.directory=${rpm.build.root}${external.config.directory}/ddl/${build.version}/
rpm.install.command.dev=
rpm.install.command.tst=
rpm.install.command.cnv=
rpm.install.command.cfg=
rpm.install.command.trn=
rpm.install.command.dmo=touch work/db/changesets/configuration.sql; \
touch work/db/changesets/conversion.sql;
rpm.install.command.stg=

1.2 Build Configuration

Notice that these are pretty much the original properties set in the kfs-build.properties file. The new properties are:

rpm.ingestion.directory=${rpm.external.work.directory}/staging/workflow/ingestion/
rpm.ddl.directory=${rpm.build.root}${external.config.directory}/ddl/${build.version}/
rpm.install.command.dev=
rpm.install.command.tst=
rpm.install.command.cnv=
rpm.install.command.cfg=
rpm.install.command.trn=
rpm.install.command.dmo=touch work/db/changesets/configuration.sql; \
touch work/db/changesets/conversion.sql;
rpm.install.command.stg=

I will explain these later. This is how you configure what the filesystem layout of your package is going to look like. The above illustrated properties are all paths. Each one defines where files are going to go on the installed system. If you are unsure about how you want your system laid out, just use mine. I will provide a diagram of where everything ends up later.

Now, let's have a look at the kfs-build.properties

do.filter.project.help
# TEMPORARILY CUSTOMIZE A GIVEN EXECUTION OF THE BUILD
appserver.home=/Library/Tomcat/Home
drivers.directory=${user.home}/kuali/drivers
appserver.name=localhost:8080
appserver.url=http://${appserver.name}/
external.config.directory=${user.home}/kuali/main/dev/
base.security.directory=${external.config.directory}/security
base.settings.directory=${external.config.directory}/settings
is.local.build=true
rice.dev.mode=false
# this property can be used to turn the batch schedule on and off
use.quartz.scheduling=true
periodic.thread.dump=false
rice.standalone=false
rice.kew.xml.pipeline.lifecycle.enabled=true

You can see that I change the appserver.home because I want it to point the RHEL (RedHat Enterprise Linux) default tomcat5 location. I am going to run this build as the tomcat user on my packaging machien, so I also changed the external.config.directory to be /home/tomcat5/app. This will insure that on my installation machine, the /home/tomcat/app will be the path consistently. I set is.local.build, rice.dev.mode as false , and rice.kew.xml.pipeline.lifecycle.enabled to true because I want automatice XML ingestion on my development server. These properties in combination will enable automatic XML ingestion at the path ${external.config.directory}/settings/work/kfs/dev/staging/workflow/pending. Next, I set periodic.thread.dump=false because I don't want the annoying log messages and rice.standalone is set to false as well. For simplicity, I am not going to complicate this with a Rice installation. We're going to run bundled to keep it simple.

That's it for communication. Now we will look at what happens during building and what the actual preprocessing is.

1.3 Build the WAR

  <target        name="dist-war" 
          description="Kuali distribution plus post processing."
              depends="init-classpath,dist">
    <fail unless="build.environment">Need the build.environment to build</fail>
    
  </target>

  <target name="dist-rpm" depends="prepare-rpm,dist-war" />

From the above, you can probably get a basic understanding of what is happening. So far, there isn't much going on other than creating a war file. "We could have done that with the original KFS code", right? True. We've just setup for preprocessing. We'll do some actual preprocessing later. Now let's create the spec file.

2 RPM Spec File

This is the part most people have been wondering about. How do we tie RPM in with building KFS? How does that work? Here we go.

Before moving on, first check to make sure your packaging system has rpm-build installed. Without this, the spec file is just a txt file. It's no more useful to building KFS than mindless scribbling on a bathroom stall.

[tomcat@uaz-kf-a00 ~]$ rpm -qi rpm-build
Name        : rpm-build                    Relocations: (not relocatable)
Version     : 4.4.2.3                           Vendor: Red Hat, Inc.
Release     : 18.el5                        Build Date: Thu 23 Jul 2009 10:58:22 PM MST
Install Date: Mon 04 Jan 2010 08:57:28 AM MST      Build Host: hs20-bc1-7.build.redhat.com
Group       : Development/Tools             Source RPM: rpm-4.4.2.3-18.el5.src.rpm
Size        : 1582544                          License: GPLv2+
Signature   : DSA/SHA1, Wed 29 Jul 2009 07:39:44 AM MST, Key ID 5326810137017186
Packager    : Red Hat, Inc. 
URL         : http://www.rpm.org/
Summary     : Scripts and executable programs used to build packages.
Description :
The rpm-build package contains the scripts and executable programs
that are used to build packages using the RPM Package Manager.
[tomcat@uaz-kf-a00 ~]$

2.1 Setup Package Home

Add this to the .rpmmacros file

%_topdir /home/tomcat/.workspace/redhat
%_dbpath /home/tomcat/rpm

The above sets up the tomcat user rpm environment so its database resides in /home/tomcat/rpm and the rpm environment location to /home/tomcat/.workspace/redhat. RHEL users will be familiar with the path for rpms

/usr/src/redhat/
/usr/src/redhat/BUILD
/usr/src/redhat/RPMS
/usr/src/redhat/SOURCES
/usr/src/redhat/SPECS
/usr/src/redhat/SRPMS

2.2 Metadata

%define __os_install_post %{nil}

Summary: Kuali Financial System
Name: kfs
Version: ${version}
Release: ${release}
Provides: kfs
License: EPL
BuildArch: noarch
Requires: tomcat5
BuildRoot: /tmp/kfs/
Source0: kfs-${build.version}.tar.gz
Group: Development/Tools
Packager: leo [at] rsmart.com

Given the above metadata, let's try to predict what the package will look like. The name is kfs, so that means we can start with:

kfs-

There's a version and release. The version is the official KFS version. In our case 4.0. The release is what I usually use to determine what my local release is by my institution's strategy. Let's just start with 1. That will make it look like:

kfs-4.0-1

. Our buildarch is noarch. Arch is for architecture. If there is a specific architecture (darwin, i386, i686, amd64, etc...), that gets declared here. KFS is platform independent and architecture free by virtue of Java; therefore, we choose noarch. Now it is

kfs-4.0-1.noarch.rpm

Observe that ${version}, ${release}, and ${build.version} look like they're ant tokens. They are ant tokens. This file is intended to be filtered through ant. These values are passed in. That allows us to store this spec file in version control as a template, and repopulate it as needed. The tokens we are going to use are:

version
release
build.version
build.environment

Before getting into how the spec file is filtered, I want to show the rest of the spec file.

%description
Kuali Financial Sytem Environment based on rSmart Vendor build ${version}
with release ${release}

%prep
%setup -q
%build

%install
BUILDDIR=%{_builddir}/kfs-%{version}
rm -rf %{buildroot}
${rpm.install.command}
ant -f vendor/kitt/deployment/build-rpm.xml -Dbasedir=$BUILDDIR -Drpm.build.root=%{buildroot} -Dbuild.environment=${build.environment} -Dbuild.version=${build.version} dist-rpm

%files
%defattr(2770,tomcat,tomcat)
/usr/share/tomcat5/webapps/kfs/

%def(-,tomcat,tomcat)
/home/tomcat/app/work/kfs/staging/workflow/pending/

%attr(2770,tomcat,tomcat)
/usr/share/tomcat5/webapps/kfs-${build.environment}/WEB-INF/web.xml
/usr/share/tomcat5/webapps/kfs-${build.environment}/WEB-INF/classes/configuration.properties
/usr/share/tomcat5/webapps/kfs-${build.environment}/static/
/home/tomcat/app/sa_forms/java/kfs/
%config /home/tomcat/app/work/kfs/archive/
%config /home/tomcat/app/work/kfs/attachments/
%config /home/tomcat/app/work/kfs/reports/
%config /home/tomcat/app/work/kfs/skel.zip
%config /home/tomcat/app/work/kfs/staging/1099
%config /home/tomcat/app/work/kfs/staging/ar
%config /home/tomcat/app/work/kfs/staging/cm
%config /home/tomcat/app/work/kfs/staging/cr
%config /home/tomcat/app/work/kfs/staging/fp
%config /home/tomcat/app/work/kfs/staging/gl
%config /home/tomcat/app/work/kfs/staging/ld
%config /home/tomcat/app/work/kfs/staging/pdp
%config /home/tomcat/app/work/kfs/staging/purap
%config /home/tomcat/app/work/kfs/staging/recon
%config /home/tomcat/app/work/kfs/staging/sys
%config /home/tomcat/app/work/kfs/staging/tax
%config /home/tomcat/app/work/kfs/staging/WEB-INF
%config /home/tomcat/app/work/kfs/temp/
%config /home/tomcat/app/logs/kfs/WEB-INF/web.xml
%config /home/tomcat/app/j2ee/kfs/log4j.properties

%pre
rm -rf /usr/share/tomcat5/webapps/kfs-*

%post
MYPWD=$PWD
cd /usr/share/tomcat5/webapps
mv kfs kfs-$(cat ~tomcat/kitt-tools/.envrc)
cd $MYPWD

setfacl -R -d -m g:kfs:rwx /home/tomcat/app/work/${build.environment}/kfs/ #modify directory defaults for new files 
setfacl -R -m g:kfs:rwx /home/tomcat/app/work/${build.environment}/kfs/ #modify existing files & directories, x for directories 

%clean
rm -rf %{buildroot}

I probably should have mentioned this earlier, but this is not an RPM tutorial. I am not going to go over all of RPM has to offer. There is just way too much to discuss. If you would like to study or know more about RPM, read the online book instead. I am simply going to review how it works for our purposes. Speaking of which! The spec file is describes two different phases. It is only used in one phase (packaging), but it describes both packaging and installation. That is, when the package is installed, it is still obeying instructions from the spec file. Keep that in mind or things can be pretty confusing. It's like working in two different universes that stare at each other.

When the spec file is processed, the first thing it will try to do is build. That's what these directives are doing:

%prep
%setup -q
%build

%install
BUILDDIR=%{_builddir}/kfs-%{version}
rm -rf %{buildroot}
${rpm.install.command}
ant -f vendor/kitt/deployment/build.xml -Dbasedir=$BUILDDIR -Drpm.build.root=%{buildroot} -Dbuild.environment=${build.environment} -Dbuild.version=${build.version} dist-rpm

The first few steps are merely uncompressing the source into the temporary build location. RPM likes to uncompress source files into %_topdir/BUILD which is in our case /home/tomcat/.workspace/redhat/BUILD. We should expect to find a /home/tomcat/.workspace/redhat/BUILD/kfs-4.0 directory after files have been uncompressed.

Next, ant is called providing basedir, rpm.build.root, build.environment and build.version. The target is dist-rpm which is what we looked at before. That means that by processing the spec file after the source code has been checked out, it will create the WAR file and necessary external configuration files. Thereby, it is essentially creating our application inside of /home/tomcat/.workspace/redhat/BUILD/kfs-4.0. That's just what we need.

2.3 Spec File Re: Packaging

Now we need to tell RPM how the files are laid out and what the permissions are. RPM is very unforgiving when it comes to files not being where they are expected as well as finding unexpected files. Either case is considered a failure. Wouldn't want unexpected files getting into your package, right? Also wouldn't want to create a package without all the files you expect. It makes sense. This is how we do that:

%files
%defattr(2770,tomcat,tomcat)
/usr/share/tomcat5/webapps/kfs/

%attr(-,tomcat,tomcat)
/home/tomcat/app/work/kfs/staging/workflow/pending/

%attr(2770,tomcat,tomcat)
/usr/share/tomcat5/webapps/kfs-${build.environment}/WEB-INF/web.xml
/usr/share/tomcat5/webapps/kfs-${build.environment}/WEB-INF/classes/configuration.properties
/usr/share/tomcat5/webapps/kfs-${build.environment}/static/
/home/tomcat/app/sa_forms/java/kfs/
%config /home/tomcat/app/work/kfs/archive/
%config /home/tomcat/app/work/kfs/attachments/
%config /home/tomcat/app/work/kfs/reports/
%config /home/tomcat/app/work/kfs/skel.zip
%config /home/tomcat/app/work/kfs/staging/1099
%config /home/tomcat/app/work/kfs/staging/ar
%config /home/tomcat/app/work/kfs/staging/cm
%config /home/tomcat/app/work/kfs/staging/cr
%config /home/tomcat/app/work/kfs/staging/fp
%config /home/tomcat/app/work/kfs/staging/gl
%config /home/tomcat/app/work/kfs/staging/ld
%config /home/tomcat/app/work/kfs/staging/pdp
%config /home/tomcat/app/work/kfs/staging/purap
%config /home/tomcat/app/work/kfs/staging/recon
%config /home/tomcat/app/work/kfs/staging/sys
%config /home/tomcat/app/work/kfs/staging/tax
%config /home/tomcat/app/work/kfs/staging/WEB-INF
%config /home/tomcat/app/work/kfs/temp/
%config /home/tomcat/app/logs/kfs/WEB-INF/web.xml
%config /home/tomcat/app/j2ee/kfs/log4j.properties

You can see it has both individual files and directories. In the case of directories, these are recursive. That means, that it assumes to include all files therein. The %config directive denotes paths that are considered configuration paths. These paths are only created/updated on installation. They are not touched on upgrade. This is how we preserve any configuration customizations we have made. Once they have been installed, only system administrators may/can manually modify them.

Permissions are set using

%attr(2770,tomcat,tomcat)

which means the tomcat user and group have access with the mask 2770.

But what is this /usr/share/tomcat5/webapps/kfs/? That is the location of the webapp, but there is no environment designation. This is because when the application is built using the spec file, it is environment agnostic. It is built generally, so that environment specific information does not interfere. Only when the package is created and then installed does the environment actually matter. Not during building. During installation it will create a path /usr/share/tomcat5/webapps/kfs/, but during post-processing, this will change to the appropriate environment. See below:

%pre
rm -rf /usr/share/tomcat5/webapps/kfs-*

%post
MYPWD=$PWD
cd /usr/share/tomcat5/webapps
mv kfs kfs-$(cat ~tomcat/.envrc)
cd $MYPWD

setfacl -R -d -m g:kfs:rwx /home/tomcat/app/work/${build.environment}/kfs/ #modify directory defaults for new files 
setfacl -R -m g:kfs:rwx /home/tomcat/app/work/${build.environment}/kfs/ #modify existing files & directories, x for directories

During pre-processing of the package, cleanup is done to make sure any files that are intended to be overwritten are removed first. Then in post processing the path kfs is moved to the environment of the machine.

$(cat ~tomcat/.envrc)

is a file that is part of environment preparation that I will discuss prior to installation of the package. It is fair to assume though, that this contains the environment name of application.

2.4 Processing the Spec File

I explained earlier that the spec file we have been looking at so far is actually just a template (let's call it kfs.spec.template) that is filtered through ant to create the real spec file. Here's how I handle it. Earlier, I showed from the build.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project          name="kfs" 
               default="dist-rpm" basedir="../../../"
          xmlns:deploy="urn:com.rsmart.kuali"
         xmlns:twitter="urn:com.rsmart.kuali.twitter">

  <property file="${tools.directory}/home/${ant.project.name}-build.properties" 
/>
  <import file="${basedir}/build.xml"/>
  <import file="${tools.directory}/macros-rpm.xml" />
...
...
</project>

You can see that there is an import of macros-rpm.xml. Within this macros file, I define:


<project  xmlns:deploy="urn:com.rsmart.kuali">
  <macrodef uri="urn:com.rsmart.kuali" name="filter">
    <attribute name="property" />
    <attribute name="srcfile" />
    <attribute name="filename" />
    <sequential>
      <loadfile property="@{property}"
                srcfile="@{srcfile}">
        <filterchain>
          <expandproperties/>
        </filterchain>
      </loadfile>

      <echo file="@{filename}">${@{property}}</echo>
    </sequential>
  </macrodef>
...
...
</project>

The filter task processes a file with the currently loaded ant properties. This is where the the aforementioned properties will be populated in the spec file. In order to use this task, I add the following to the build.xml

  <target name="package" depends="init-classpath">
    <!-- create spec file -->
    <deploy:filter   property="kfs.spec.template" 
                      srcfile="kfs.spec.template"
                     filename="kfs.spec" />

    <exec executable="rpmbuild">
      <arg value="-bb" />
      <arg value="kfs.spec" />
    </exec>
  </target>

Two things are happening here. The first thing that happens is that the spec file is filtered and output to the current working directory. Then, rpmbuild is run against it producing the rpm file in /home/tomcat/.workspace/redhat/RPMS

2.5 Execution

We run this simply by

ant -Dversion=4.0 -Drelease=1 -Dbuild.environment=dev package

3 Installation

Now that we have a package, let's install it.

3.1 Setup the Environment First

Before we actually do an installation, there are a couple things that need to be setup on the environment.

3.1.1 .envrc

First, if you recall, the .envrc file contains the environment designation. We just need to create one for DEV

% echo dev > $HOME/.envrc

3.1.2 RPM Database

RPM normally requires root access to run because it requires root to be able to read/write the RPM database located at /var/lib/rpm. What we're going to do is copy the database to retain all the information in it to /home/tomcat/app/rpm.

% mkdir $HOME/app
% cp -rf /var/lib/rpm $HOME/app/

3.1.3 Setup .rpmmacros

Now that the database has been copied. RPM needs to be told which database to use. We do this by adding the following line to the .rpmmacros file

%_dbpath /home/tomcat/app/rpm

3.2 Installation

To finally install the new RPM package I use

% rpm -Uvh --force $HOME/.workspace/redhat/RPMS/kfs-4.0-1.noarch.rpm

That is part 2. Part 3 will add workflow, automatic database upgrades/downgrades, and multi-packages with dependency management.

Please Leave Comments

2011-02-07T06:37:00.001-08:00

Again, more stuff related to the blog.

Why Should I?

Well, I couldn't understand why I never get comments. I thought to myself, why would people not want to comment? I checked out my settings and found that you must be a registered user. I thought to myself, "Why would something like this stop someone from commenting?" Answer: "Maybe people want to remain anonymous on my blog." Solution: I've allowed anonymous users to post. Now please, let me know what you think of the screencasts. I've been experimenting with xtranormal, so I want to know what people think of it and if it's worth continuing to pursue.

Screencasts Coming Out Slowly

2011-02-07T06:29:00.000-08:00

What's Goin' On?

I know this isn't a kuali-related post, but it is related to the blog. I just started using Gnome 3 (with Gnome Shell) and KdenLive!!! This pair is incredible. Actually, KdenLive alone is incredible. It's available for macusers through macports, but it's hands down the best non-linear editor I've ever seen. This is the combintation I used for my most recent screencasts; however, I do not think I will be able to continue the trend. I just switched to Mac because recent events with my linux system started to really hamper my development. I started to cry out for help and found a welcome tone with OS X. I don't want this to become a Linux v. OS X blog post, but I will say this: I finally got around some of the bugs I'd experienced in OS X that the team refuses to fix and millions continue to complain about. 2 things have always been my limiting factor for switching to OS X. No thanks to the team, I found my own solutions and am now comfortable with my decision.

What's next?

What am I doing then? Well, I downloaded a new application that seems to give me what I've been looking for with both KdenLive and Gnome-Shell. That's Voila. Wish me luck and feel free to comment on any of my work to let me know how the switch is turning out.

The KIS Identity - LDAP Development Setup

2011-01-31T12:12:00.000-08:00

Abstract

Before you can use the LDAP/Entity integration into your Kuali Rice-based application, you have to first build it into Kuali Rice. To do that, you first need to setup a development environment for building and packaging rice.

Video Instructions

I've put together a video for instruction how to not just setup your rice development environment, but also modularize the LDAP Integration source code distribution with Kuali Rice to be packaged and distributed together.

If you're an institution developing custom rice applications or just running rice-based kuali applications, you will most likely have your own rice release to add customizations too. LDAP integration in its current incarnation is pretty much that. It's like a third-party module/customization that is added to existing rice source.

This screencast is also using the Eclipse IDE. Again, if you have another IDE
or some other method. Please substitute whatever you would normally do.

Here it is. Enjoy:

Steps

These are written instructions for those that found the video to be difficult to follow.

First, let's checkout the rice source code. Here, I am going to checkout the rice 1.0.3 branch from rsmart. Our trunk currently has an ldap integration. I created this branch specifically for this screencast. You should use whatever URL you have at your institution
Now that it's checked out, I am simply going to copy it to my Eclipse project. I
then refresh my project.
The next thing we want to do is to add our ldap integration as a module or subproject within the rice project. What we need for this is to first create a path for it.
Now that the source code is checked out, we want to tell Eclipse that this is a Maven project. Go to your project context menu. Then, select the Maven submenu. Finally, Enable Dependencies Management.

All that needs to be done is to update the rice pom.xml and the rice-web pom.xml. I now open the rice pom.xml. I add to the modules section that ldap is a module.




 4.0.0
 org.kuali.rice
 rice
 pom
 Kuali Rice
 1.0.3
 The Kuali Rice effort provides an enterprise class
  middleware suite of integrated products that allows both Kuali and
  non-Kuali applications to be built in an agile fashion, such that
 

 
  api
  impl
  ldap
  web
  sampleapp
  ksb
  kcb
  kns
  kim
  kew
  ken
 
...

Next, open the rice-web pom.xml. In here we set rice-ldap as a dependency. I will just copy the rice-impl dependency and change impl to ldap.




 Kuali Rice Web
 4.0.0
 
  org.kuali.rice
  rice
  1.0.3
 
 rice-web
 war

 
  
   ${project.groupId}
   rice-impl
   ${project.version}
  
  
   ${project.groupId}
   rice-ldap
   ${project.version}
  
...
...

There, it's that easy. Now, when rice is packaged, the rice-ldap jar will be created with it. Let's do that.

leo@leviathan~/.workspace/rice-1.0.3
(02:15:03) [13] mvn -Dmaven.test.skip=true package
...
...
leo@leviathan~/.workspace/rice-1.0.3
(02:20:06) [14] ls ldap/target/
classes/ maven-archiver/ rice-ldap-1.0.3.jar
leo@leviathan~/.workspace/rice-1.0.3
(02:20:11) [15]

LDAP Development + Xtranormal

2011-01-20T16:13:00.000-08:00

Video that I overlaid with an xtranormal as an experiment in instruction.

Liquibase Testing with SVN Keywords

2010-06-05T07:53:00.000-07:00

Abstract

In KIS of the Dragon, I documented using the liquibase rollback functionality for testing. Since then, I have found that liquibase has a small bug with rollbacks. Some of you may have encountered it. It uses regex substitution when rolling back (not sure what exactly). Part of defining the group in regex substitution is the '$', so that makes '$' kind of a reserved character that isn't very well documented in liquibase. It doesn't effect SQL at all, but it does mean using '$' in changeSet attributes is forbidden. Just to give an example of what will happen, when you rollback in this scenario, you get an exception:

Jun 5, 2010 8:07:24 AM liquibase.commandline.Main main
SEVERE: Illegal group reference
java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:725)
at java.util.regex.Matcher.replaceFirst(Matcher.java:872)
at java.lang.String.replaceFirst(String.java:2158)
at liquibase.database.AbstractDatabase.removeRanStatus(AbstractDatabase.java:1328)
at liquibase.parser.visitor.RollbackVisitor.visit(RollbackVisitor.java:23)
at liquibase.parser.ChangeLogIterator.run(ChangeLogIterator.java:41)
at liquibase.Liquibase.rollback(Liquibase.java:273)
at liquibase.Liquibase.rollback(Liquibase.java:248)
at liquibase.commandline.Main.doMigration(Main.java:682)
at liquibase.commandline.Main.main(Main.java:97)

Below, we have the offending XML responsible for this.

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd"&gh;
 
  <changeSet id="$Revision$" author="$Author$">

Problem

You probably thought the above was the problem. No no no no no. That was just background on it. Such a thing is actually easily fixed. The real problem is testing in your IDE. The reason we have '$' is for the Subversion keywords. We want these in our changelogs so we can rollback in our other environments. Subversion doesn't actually add this information until after the svn:keywords property and the file are committed to the VCS. When it does, you get something that looks like this:

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd"&gh;
 
  <changeSet id="$Revision: 1$" author="$Author: przybyls$">

There are still '$' in there.

What does this have to do with the IDE?

At UA, we use a liquibase template, so all developers are consistent with their usage. It looks something like:


<?xml version="1.0" encoding="UTF-8"?>
 
<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">
 
  <changeSet id="$Revision$" author="$Author$">
    <sql splitStatements="false" endDelimiter=""><![CDATA[
    DECLARE
    BEGIN
    END;
]]>
    </sql>
    <rollback>
      <sql><![CDATA[
      DECLARE
      BEGIN
      END;
]]>
      </sql>
    </rollback>
  </changeSet>
</databaseChangeLog>

The first thing you should notice is the '$' in the id and author attributes. This actually makes the problem part of our IDE. Now developers have to remember to change this. That makes this template pretty much...useless.

Solution (it's not as cool as you think)

At UA, we decided to take liquibase testing out of our IDE and instead move it into continuous integration with Ad-Hoc builds. In the end, developers still use the template. Only now, they don't need to install liquibase at all! It's one of those, "Why didn't I think of this before!?" moments.

Instructions for Hudson

I hope you're using Hudson. If you're not, you have my sympathies. I am truly sorry that you will have to find your own way of doing this.

Step 1: Create an ad-hoc build

To do this, you simply setup a build parameter in your project/build configuration. When you do this, Hudson will want to know about the parameter you are adding. I used a file. This way developers simply upload their file to Hudson. I called mine test.xml and you will see that come up in a second.

Step 2: Setup the Execution Script

I just used shell with the following contents:

/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties tag before
sed -e 's/\$Revision\$/test/g' test.xml > /tmp/test.xml;mv /tmp/test.xml test.xml
sed -e 's/\$Author\$/test/g' test.xml > /tmp/test.xml;mv /tmp/test.xml test.xml
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml updateSQL
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml update
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml rollbackSQL before
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml rollback before

Notice the sed commands that do the replace on test.xml. It is then simply doing an update and rollback just like in KIS of the Dragon.

Conclusion

That's it. Now developers can all test their liquibase changelogs in a consistent manner without any modification to their IDE. Hudson is also really effective here because changelog testing is really really fast. It takes a few seconds and won't hang up Hudson. Hudson also can facilitate multiple executors. I will include in another post on liquibase automation how to integrate this into configuration management and automated build processes.

Apologies

I am very sorry for the content in KIS of the Dragon because it was somewhat incorrect. I'm glad to remedy it here.

I also apologize for having not put out a screencast in awhile. My goal was one a week. I've had some family crisis to deal with lately and I made the mistake of working on a few at the same time. In the coming week, expect a torrent of posts.

Setting up Eclipse to use Jetty for KFS Development

2010-04-30T13:29:00.000-07:00

Sometimes, you just want to use a straight, no-nonsense appserver for development. You want it to be fast, simple, and integrate with Eclipse and its debugger. Sounds like a tall order, but it's not. It's possible to setup Jetty to run KFS within Eclipse and hook in the debugger.

Move Required Libraries into Your KFS Project

The following need to be copied to the .classpath of your eclipse project

jetty-6.1.15.rc5.jar
jetty-util-6.1.15.rc5.jar
connector-1_5.jar

Create a Startup Server Class in Your KFS Project

Here's ours


package edu.arizona.jetty;

import org.mortbay.jetty.*;
import org.mortbay.jetty.webapp.*;
import org.mortbay.jetty.nio.*;

import static java.lang.System.getProperty;


/**
 *
 * @author Leo Przybylski (przybyls@arizona.edu)
 */
public class HappyMeal {
    public static void main(String[] args) {
        final String WEBROOT        = getProperty("webroot.directory");
        final int    SERVER_PORT    = Integer.parseInt(getProperty("jetty.port"));
        final String CONTEXT_PATH   = "/kfs-dev";

        Server server = new Server();
        
        Connector[] connectors = new Connector[1];
        connectors[0] = new SelectChannelConnector();
        connectors[0].setPort(SERVER_PORT);
        server.setConnectors(connectors);
        
        WebAppContext webapp = new WebAppContext();
        webapp.setContextPath(CONTEXT_PATH);
        webapp.setWar(WEBROOT);
        server.setHandler(webapp);
        
        try {
            server.start();
            server.join();
        }
        catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Create a .launch for Jetty Server


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.jdt.launching.localJavaApplication">
<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_PATHS">
<listEntry value="/kfs"/>
</listAttribute>
<listAttribute key="org.eclipse.debug.core.MAPPED_RESOURCE_TYPES">
<listEntry value="4"/>
</listAttribute>
<booleanAttribute key="org.eclipse.debug.core.appendEnvironmentVariables" value="true"/>
<listAttribute key="org.eclipse.debug.ui.favoriteGroups">
<listEntry value="org.eclipse.debug.ui.launchGroup.debug"/>
<listEntry value="org.eclipse.debug.ui.launchGroup.run"/>
</listAttribute>
<stringAttribute key="org.eclipse.jdt.launching.MAIN_TYPE" value="edu.arizona.jetty.HappyMeal"/>
<stringAttribute key="org.eclipse.jdt.launching.PROJECT_ATTR" value="kfs"/>
<stringAttribute key="org.eclipse.jdt.launching.VM_ARGUMENTS" value="-Dwebroot.directory=work/web-root/ -Djetty.port=8080 -Xmx2g -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -Denvironment=dev -DlogDefault=WARN"/>
</launchConfiguration>

The parts to pay attention to are the org.eclipse.jdt.launching.MAIN_TYPE, org.eclipse.jdt.launching.PROJECT_ATTR, and org.eclipse.jdt.launching.VM_ARGUMENTS attributes. You'll need to adjust yours to your preferences.

Test it out

This will add jetty to the tools menu

Hooking up the Debugger

The jetty selection should now be available under the debug menu.

Love

REPOST KFS Inheritance/Composition with Private Methods

2010-04-30T13:12:00.001-07:00

Why the Repost?

I got some feedback that I

Never resolved the namespace issue
Didn't show original code to explain the problem

Both are very good points, so I'm going to do that now.

The Problem Scenario

Suppose an instance arrives where you need to change the functionality of KFS slightly, but the method you want to override is declared private. Need an example? I have one. Read on.

Digester and Namespace Validation

By default, Digester within KFS has namespace validation off. Suppose you want to turn it on. Now you run into exactly such issue. Below is the original code from the XmlBatchInputFileTypeBase

...
...
public class XmlBatchInputFileTypeBase extends BatchInputFileTypeBase {
    /**
     * @see org.kuali.kfs.sys.batch.BatchInputFileType#parse(byte[])
     */
    public Object parse(byte[] fileByteContent) throws ParseException {
        if (fileByteContent == null) {
            LOG.error("an invalid(null) argument was given");
            throw new IllegalArgumentException("an invalid(null) argument was given");
        }

        // handle zero byte contents, xml parsers don't deal with them well
        if (fileByteContent.length == 0) {
            LOG.error("an invalid argument was given, empty input stream");
            throw new IllegalArgumentException("an invalid argument was given, empty input stream");
        }

        // validate contents against schema
        ByteArrayInputStream validateFileContents = new ByteArrayInputStream(fileByteContent);
        validateContentsAgainstSchema(getSchemaLocation(), validateFileContents);

        // setup digester for parsing the xml file
        
        Digester digester = buildDigester(getSchemaLocation());

        Object parsedContents = null;
        try {
            ByteArrayInputStream parseFileContents = new ByteArrayInputStream(fileByteContent);
            parsedContents = digester.parse(validateFileContents);
        }
        catch (Exception e) {
            LOG.error("Error parsing xml contents", e);
            throw new ParseException("Error parsing xml contents: " + e.getMessage(), e);
        }

        return parsedContents;    
    }

}

The problem is with buildDigester(getSchemaLocation()). XmlBatchInputFileTypeBase#buildDigester() is declared private. That is, if one tries to extend XmlBatchInputFileTypeBase, you not only cannot override the buildDigester() method, but you can't call it from an inheriting class either.

We would love to do

        Digester digester = buildDigester(getSchemaLocation());

But we can't. The alternative is to use reflection and call the private buildDigester method anyway.

Here's an example:

package edu.arizona.kfs.sys.batch;
...
...
public class XmlBatchInputFileTypeBase extends org.kuali.kfs.sys.batch.XmlBatchInputFileTypeBase {
    /**
     * @see org.kuali.kfs.sys.batch.BatchInputFileType#parse(byte[])
     */
    public Object parse(byte[] fileByteContent) throws ParseException {
        if (fileByteContent == null) {
            LOG.error("an invalid(null) argument was given");
            throw new IllegalArgumentException("an invalid(null) argument was given");
        }

        // handle zero byte contents, xml parsers don't deal with them well
        if (fileByteContent.length == 0) {
            LOG.error("an invalid argument was given, empty input stream");
            throw new IllegalArgumentException("an invalid argument was given, empty input stream");
        }

        // validate contents against schema
        ByteArrayInputStream validateFileContents = new ByteArrayInputStream(fileByteContent);
        validateContentsAgainstSchema(getSchemaLocation(), validateFileContents);

        // setup digester for parsing the xml file
        
        Digester digester = null;
        try {
            Method digesterMethod = getClass().getDeclaredMethod("buildDigester", String.class, String.class);
            m.setAccessible(true); // Private? Who cares?! I sure don't.
            m.invoke(this, getSchemaLocation(), getDigestorRulesFileName());
        }
        catch(IllegalAccessExption e) {
            // Not since it's accessible now.
        }
        catch(InvocationTargetException e) {
            // Something else naughty happened
        }

        if (digester == null) {
        // throw some exception because this is very bad
        }

        digester.setNamespaceAware(true); // Enabling the namespace checking! Yeay!


        Object parsedContents = null;
        try {
            ByteArrayInputStream parseFileContents = new ByteArrayInputStream(fileByteContent);
            parsedContents = digester.parse(validateFileContents);
        }
        catch (Exception e) {
            LOG.error("Error parsing xml contents", e);
            throw new ParseException("Error parsing xml contents: " + e.getMessage(), e);
        }

        return parsedContents;    
    }

}

The trick is in


        Digester digester = null;
        try {
            Method digesterMethod = getClass().getDeclaredMethod("buildDigester", String.class, String.class);
            m.setAccessible(true); // Private? Who cares?! I sure don't.
            m.invoke(this, getSchemaLocation(), getDigestorRulesFileName());
        }
        catch(IllegalAccessExption e) {
            // Not since it's accessible now.
        }

By calling m.setAccessible(true), we can still call the private method. many call this a hack because it ignores the private access. I think many people are confused in what that means. The control is rather just for managing scope and maintaining OO encapsulation. These directives are not intended for security. They are intended to enforce OO. If we wanted to restrict method access, a SecurityManager implementation would be more appropriate. IMHO, this is an entirely acceptable thing to do in this situation. That being that we are being restricted by a flaw in the API from doing something we should be allowed to do. This happens often, and is a much better alternative than duplicating code.

Then later, we append our changes to the digester with...


        digester.setNamespaceAware(true); // Enabling the namespace checking! Yeay!

With this, we can get away with extending code and overriding behavior (setting namespace awareness).

KFS Inheritance/Composition with Private Methods

2010-04-27T23:15:00.000-07:00

The Problem Scenario

Suppose an instance arrives where you need to change the functionality of KFS slightly, but the method you want to override is declared private. Need an example? I have one. Read on.

Digester and Namespace Validation

By default, Digester within KFS has namespace validation off. Suppose you want to turn it on. Now you run into exactly such issue.

package edu.arizona.kfs.sys.batch;
...
...
public class XmlBatchInputFileTypeBase extends org.kuali.kfs.sys.batch.XmlBatchInputFileTypeBase {
    /**
     * @see org.kuali.kfs.sys.batch.BatchInputFileType#parse(byte[])
     */
    public Object parse(byte[] fileByteContent) throws ParseException {
        if (fileByteContent == null) {
            LOG.error("an invalid(null) argument was given");
            throw new IllegalArgumentException("an invalid(null) argument was given");
        }

        // handle zero byte contents, xml parsers don't deal with them well
        if (fileByteContent.length == 0) {
            LOG.error("an invalid argument was given, empty input stream");
            throw new IllegalArgumentException("an invalid argument was given, empty input stream");
        }

        // validate contents against schema
        ByteArrayInputStream validateFileContents = new ByteArrayInputStream(fileByteContent);
        validateContentsAgainstSchema(getSchemaLocation(), validateFileContents);

        // setup digester for parsing the xml file
        
        Digester digester = null;
        try {
            Method digesterMethod = getClass().getDeclaredMethod("buildDigester", String.class, String.class);
            m.setAccessible(true); // Private? Who cares?! I sure don't.
            m.invoke(this, getSchemaLocation(), getDigestorRulesFileName());
        }
        catch(IllegalAccessExption e) {
            // Not since it's accessible now.
        }
        catch(InvocationTargetException e) {
            // Something else naughty happened
        }


        Object parsedContents = null;
        try {
            ByteArrayInputStream parseFileContents = new ByteArrayInputStream(fileByteContent);
            parsedContents = digester.parse(validateFileContents);
        }
        catch (Exception e) {
            LOG.error("Error parsing xml contents", e);
            throw new ParseException("Error parsing xml contents: " + e.getMessage(), e);
        }

        return parsedContents;    
    }

}

The trick is in


        Digester digester = null;
        try {
            Method digesterMethod = getClass().getDeclaredMethod("buildDigester", String.class, String.class);
            m.setAccessible(true); // Private? Who cares?! I sure don't.
            m.invoke(this, getSchemaLocation(), getDigestorRulesFileName());
        }
        catch(IllegalAccessExption e) {
            // Not since it's accessible now.
        }

By calling m.setAccessible(true), we can still call the private method. many call this a hack because it ignores the private access. I think many people are confused in what that means. The control is rather just for managing scope and maintaining OO encapsulation. These directives are not intended for security. They are intended to enforce OO. If we wanted to restrict method access, a SecurityManager implementation would be more appropriate. IMHO, this is an entirely acceptable thing to do in this situation. That being that we are being restricted by a flaw in the API from doing something we should be allowed to do. This happens often, and is a much better alternative than duplicating code.

With this, we can get away with extending code and overriding behavior.

KIS of the Dragon - Testing Liquibase Changelogs

2010-04-27T21:01:00.001-07:00

Here is a screencast on how to test changelogs with Liquibase tagging and rollback commands.

The Breakdown

Tag the database
You want to tag before changes are made. This is for rolling back later.
% liquibase --defaultsFile=liquibase.properties tag
Run your update
% liquibase --defaultsFile=liquibase.properties --changeLogFile=your changelog update <your tag name>
Rollback
You can rollback if you make a mistake and run your update again, but to make sure you leave the place the same as when you came in, cleanup with rollback.
% liquibase --defaultsFile=liquibase.properties rollback <tag you used earlier>

KIS and Tell - Screencasts on the Horizon

2010-04-26T01:26:00.000-07:00

I have started a screencast series on developing for Kuali. Entries that are part of the screencast series follow the naming convention where KIS stands out as a popular movie title that is used in the blog entry title. Here are names of upcoming screencasts.

KIS of the Dragon - Testing Liquibase Changelogs
KIS'ing Katie Holmes - Migrating an Oracle Database to Mysql with Liquibase and hsqldb
KIS KIS Bang Bang - Liquibase Automation
The Long KIS Goodnight - Automating Selenium Testing with Hudson CI and Amazon EC2
KIS Me Kate - RPM Packaging KFS
Pythagoras KIS - Exposing Quartz Scheduled Jobs via Web Service w/o KSB
The KIS Identity - University of Arizona LDAP Integration
The KIS Supremacy - Amazon EC2 and Launchpad for KFS
A KIS to Build a Dream On - Configuration Management of KFS at University of Arizona

Stay tuned. It is my sincere hope that these screencasts empower implementing institutions to build strong sustainable implementations of Kuali Foundation projects.

KISmet - Game Changer: Structuring the Project to Manage Database Changes with Liquibase

2010-04-26T00:53:00.000-07:00

Structuring your project to handle configuration management of your project RDBMS can probably be the most difficult part of managing your project. If you plan to use Liquibase to manage your database migrations, then this is even more the case. At the University of Arizona, we first followed the instructions laid out by this tutorial (Tutorial Using Oracle) since we're using Oracle.

We retain in our methodology much from that article, but rather than explain the differences, I'll just explain what we did.

The University of Arizona Liquibase Methodology in a Nutshell

The goals we wanted to solve with simplified data migration using Liquibase are:

Structure for isolating database changes.
Integrates with a process that versions a database from SVN, Jira, and Continuous Integration.
Coupled database version with application version.
Integrates with a process that facilitates rollback, update, and complete schema rebuilds.

Structure for Isolating Database Changes

A project was created at University of Arizona called kfs-cfg-dbs. Within that project is where we created the structure to manage out database migrations. We followed the example outlined in Tutorial Using Oracle. We found that we can create two paths. One path is for update (update/), and the other path is for building the latest schema entirely (latest/).

latest/

Here, we followed the convention of using 3-character paths according to the changelog content.

Pathname	Content
cst	constraint-related changelog
dat	table-related changelog
idx	index-related changelog
seq	sequence-related changelog
tab	table-related changelog
vw	view-related changelog

constraints.xml

During our database migrations, we load schema changes and data changes. These data changes can sometimes effect constraints. For full schema rebuilds, we load constraints last to allow data loads to process faster. Therefore, we separate our constraint changelog information into its own file to run last.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">
    <include file="latest/cst/LD_EXP_TRNFR_DOC_T.xml" />
    <include file="latest/cst/FP_PMT_MTHD_T.xml" />
    <include file="latest/cst/FP_PMT_MTHD_CHART_T.xml" />
</databaseChangeLog>

Notice that entries are simply includes that point to files associated by table name in the cst/ directory within the latest/ path. For example, latest/cst/LD_EXP_TRNFR_DOC_T.xml refers to constraints on the LD_EXP_TRNFR_DOC_T table. Because cst/ is taken from latest/ we know that this file relates to new schema migrations. 100% of the time, includes in constraints.xml will point to latest. That is our convention.

data.xml

Similarly to constraints.xml, data.xml has entries to data by table-name in latest/ for new schema migrations. Here is ours:


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">
    <include file="latest/dat/FP_PMT_MTHD_T.xml" />
    <include file="latest/dat/FP_PMT_MTHD_CHART_T.xml" />
    <include file="latest/dat/KREW.xml" />
    <include file="latest/dat/KRIM.xml" />
    <include file="latest/dat/KRIM3.xml" />
    <include file="latest/dat/GL_OFFSET_DEFN_T.xml" />
    <include file="latest/dat/KRIM2.xml" />
    <include file="latest/dat/KRNS_PARM_T.xml" />

install.xml

Anything that needs to be migrated before data and constraints is added to the install.xml. It follows exactly the same convention as the previous two:


    <include file="latest/tab/FP_PRCRMNT_LVL3_ADD_ITEM_T.xml" />
    <include file="latest/tab/FP_PRCRMNT_LVL3_FUEL_T.xml" />
    <include file="latest/tab/FP_PRCRMNT_CARD_HLDR_DTL_T.xml" />
    <include file="latest/tab/FP_PRCRMNT_CARD_TRN_T.xml" />
    <include file="latest/tab/FP_PRCRMNT_CARD_HLDR_LD_T.xml" />
    <include file="latest/tab/PDP_SHIPPING_INV_TRACKING_T.xml" />
    <include file="latest/seq/ERROR_CERT_ID_SEQ.xml" />
    <include file="latest/seq/CM_CPTLAST_AWARD_HIST_NBR_SEQ.xml" />
    <include file="latest/seq/PUR_PDF_LANG_ID.xml" />

update/

The update changelog is responsible for database migrations on existing schemas. It simply updates a schema already in use, so these are all changes.


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">
    <include file="update/KITT-958.xml" />
</databaseChangeLog>

The update.xml refers to files in the update/ path which contains files associated by Jira Issue #. By this convention, we know that update/KITT-958.xml contains a change in this version for KITT-958. This is how we get our Jira coupling. Using this convention, we can migrate perpetually and let Jira handle linkage with our data migration/issue management.

Integrates with a process that versions a database from SVN, Jira, and Continuous Integration.

I have already shown how this works with Jira, but how do we get exclusive database versions? We use SVN to handle changelog ids for us. Changelog ids are what Liquibase uses to identify which changes are to be run. Each change gets its own id, to identify it apart from others. In order to lessen developer overhead, we simply use the $Revision$ keyword from SVN.


<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">
    <changeSet id="$Revision$" author="$Author" >
        <comment>Adding a new kim type of patisserie and a new eclair baker role. Yum!!</comment>

This will put a new revision id with each checkin. We also use $Author$ to identify the person that made the change. This convention falls apart when a person commits more than one changelog at a time. Then two changelogs have the same id. This will cause problems.

Part of our methodology is the concept of one-change-per-checkin. That is, when creating a changelog for update/, we put all changes relating to KITT-958 for this release into a single changeset in our KITT-958.xml. If we have changes for another issue (KITT-959), then we put changes in KITT-959.xml. That is fine. It is crucial to understand that only one of these files gets checked in at a time though. That is, first, commit KITT-958.xml, then KITT-959.xml. The reason is that when one gets committed, it gets a revision number. Then, when the next one is committed, it gets a different revision number. This helps us keep a consistent set of changelog ids, and also prevent changelogs from stepping on each other.

Integrates with a process that facilitates rollback, update, and complete schema rebuilds.

I will explain this in more detail when I discuss testing changes to data migration. I will say though that Liquibase supports rollback of changes. Using our configuration management system, this allows us to couple data migrations with a release of KFS. We can suddenly move between versions very easily by undoing what we have done. In most cases, Liquibase can automatically rollback changes by analyzing different patterns based on the refactoring. Some cases, this is very difficult though (consult the Liquibase manual for more details on what does and doesn't auto-rollback). One example is data-related migrations. When inserting data, liquibase cannot understand how to undo an insert or an update of a record. For this, changelogs have a rollback directive to explicitly define a rollback pattern.


<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
                   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">
    <changeSet id="$Revision$" author="$Author" >
        <comment>Adding a new kim type of patisserie and a new eclair baker role. Yum!!</comment>
    <sql splitStatements="false" endDelimiter=""><![CDATA[
declare 
ktyp_id krim_typ_t.kim_typ_id%TYPE;
BEGIN
    INSERT INTO KRIM_TYP_T 
    (KIM_TYP_ID, OBJ_ID, VER_NBR, NM, SRVC_NM, ACTV_IND, NMSPC_CD) VALUES 
    (KRIM_TYP_ID_S.NEXTVAL,'SYS_GUID()', 1,'Patisserie',null,'Y','KUALI')
    RETURNING kim_typ_id into ktyp_id;

    INSERT INTO KRIM_ROLE_T 
    (ROLE_ID,OBJ_ID,VER_NBR,ROLE_NM,NMSPC_CD,DESC_TXT, KIM_TYP_ID, ACTV_IND) VALUES 
    (KRIM_ROLE_ID_S.NEXTVAL,SYS_GUID(),1,'Eclair Baker','KUALI',null,ktyp_id,'Y');
END;
]]>
    </sql>
    <rollback>
      <sql><![CDATA[
delete from KRIM_TYP_T where NM = 'Patisserie';

delete from KRIM_ROLE_T where ROLE_NM = 'Eclair Baker';
]]>
      </sql>
    </rollback>

Case Study: Adding a Data-Only Change

Here is a screencast on how to use liquibase to migrate database changes using the University of Arizona's methodology for change management. Includes an example on making a data related change.

Looking Ahead

Be sure to read my next blog entry which will describe how to test this change against a database using rollbacks.

Prelude to a KIS - Setting up and Getting Started with Mylyn

2010-04-25T01:25:00.000-07:00

This is my first screencast. I am going to discuss how to setup and get started using mylyn since both Jira and Eclipse are prominent Kuali development tools.

My reasons for using Mylyn

Enforces one-issue-at-a-time practice. Many times, complex check-ins from working on multiple issues at a time is the cause of bugs.

Organizes files to an issue. This helps you focus on the task by blocking out files that don't matter and helping you remember what you were working on.

Issue management integration connects what you're working on with ... what you're working on. Issue metadata only available in Jira is something that can be lost when developing. Mylyn keeps your work connected with your tools

Tuning the Garbage Collection in KFS

2010-04-23T18:29:00.001-07:00

KFS does not ship with JVM configuration that optimizes the application. Implementing institutions are expected to do that themselves. At University of Arizona, we have spent some time tuning KFS garbage collection for what seems to be pretty decent performance. Our results came mostly from trial-and-error, research from literature, and Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine

This is what we've come up with.

-Xms2g -Xmx2g -XX:MaxPermSize=512m -Doracle.jdbc.Trace=true -Djava.util.logging.config.file=/usr/share/tomcat5/conf/ojdbclog.conf -server -XX:+UseParNewGC -XX:MaxNewSize=256m -XX:NewSize=256m -XX:SurvivorRatio=128 -XX:MaxTenuringThreshold=0 -XX:+UseTLAB -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled

Concurrent Low Pause Collector

What we decided to use is the Concurrent Low Pause Collector. We run our application out of a 64-bit Redhat Enterprise Linux VM running from Vmware ESX Server with 2 processors and 4 gb of RAM. Now that's a mouthful. You may be wondering, "With 2 processors, why don't you use the parallel GC?" Well, we tried both parallel and concurrent low pause gc. Really, the only reason why you would use a parallel is not because you have an extra processor, but rather that sacrificing that processor during runtime is better than bringing the system to its knees. Well, there's got to be a way that you run the garbage collector without sacrificing processing power at what might be a crucial time. That's where the concurrent low pause collector comes in. Concurrent means it runs in a separate thread. You may sacrifice a processor here, yes. Low Pause means that unlike a Parallel Collector, this runs in a separate thread for a short amount of time. Win-win!! So what's this other stuff? Read on.

Heap Memory Requirements

We set the min and max to be the same. This way the JVM doesn't have to reallocate memory. It's greedy and takes everything it can right away. This saves time. In a server environment, you don't want to be conservative. Greedy is a better way to go.

Optimizing for Caching

The -XX:MaxNewSize=256m -XX:NewSize=256m properties grow the Eden gen space to 256 mb immediately. Just like with the heap, we don't want to reallocate on a server platform. Let's be as greedy as possible. We've optimized to 256mb because we expect to have a large amount of cache rewriting. Tuning JVM Garbage Collection for Production Deployments recommends setting it to 32mb for caching systems, but we've found that for larger retained caches like what we want, maybe 256mb is more about what we want to use. This is especially the case with the number and size of HashMap instances in use in KFS between Spring and Rice. We also optimize the survivor space for the young generation to be 128th the size of the eden space. MaxTenuringThreshold is turned off so that a new NewSize space becomes reusable with each collection. This is actually really really small, and works well for caching.

Optimizing for Concurrency

CMSClassUnloadingEnabled, CMSPermGenSweepingEnabled, and UseConcMarkSweepGC give us our concurrentcy collection. Together, they force different algorithms for the GC that are optimal for multiprocessor system like forcing class unloading to prevent the GC from being intrusive on the application. UseTLAB "uses thread-local object allocation blocks. This improves concurrency by reducing contention on the shared heap lock." from Tuning JVM Garbage Collection for Production Deployments.

Conclusion

We had tried numerous configurations, and this worked out the best for us. It gave us a 4x improvement when processing large batch jobs during the day. In most cases, we won't process large batch jobs while users are in the system; however, for testing we are limited on servers and sometimes we try to get more out of our systems than they can give.

Turning on Oracle Driver Tracing

2010-04-23T17:44:00.001-07:00

You are probably asking right about now, "Why would I do this if I have p6spy?" Well, here are my reasons.

P6Spy is a proxy on the driver. It watches SQL that goes in and prints it.
OJDBC in debug mode prints what goes in (not just SQL).
OJDBC in debug mode reports what comes out (including exceptions!)

Basically, if there's an exception or any kind of warning handled by the driver, you would find out about it. This is useful for too many reasons to count. Further, it doesn't just spit out the SQL you send it. It spits out any requests to recover connections or API level communication happening in the backend. Really, anything over Net8 happens will get reported. That's huge if you're working with Oracle.

Without further ado, here it is. Pass the following in when starting your JVM.

-Doracle.jdbc.Trace=true -Djava.util.logging.config.file=<location of your java logging configuration file>

Note: You must use the ojbdcX_g.jar. Oracle optimizes its driver library for each JVM. ojdbc14.jar is optimized for jdk 1.4. ojdbc5.jar is optimized for java 5. ojdbc6.jar is optimized for java 6. _g is appended for the debugging enabled jar. For example, ojdbc14_g.jar is the jdk1.4 jar with debugging enabled.

ojdbc debug mode uses java.util.logging framework to define its logging. You simply configure it. Here is what our config looks like:

.level=SEVERE
oracle.jdbc.level=WARNING
oracle.jdbc.handlers=java.util.logging.FileHandler
java.util.logging.FileHandler.level=WARNING
java.util.logging.FileHandler.pattern=/usr/share/tomcat5/logs/jdbc.log
java.util.logging.FileHandler.count=3
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter

We use a java.util.logging.FileHandler, but you can use the ConsoleHandler which is useful for development; however, this merges all your logs to file which may not always be ideal

That's it. If you're working with tomcat, just set this in your CATALINA_OPTS environment variable. You can also set this in your Eclipse JRE settings for your Kuali project. If you're using eclipse, you may want to look at the Eclipse Log Viewer plugin which will allow you to follow external logs from within eclipse.

CSV Export in Internet Explorer

2010-04-22T11:18:00.000-07:00

At the University of Arizona, we have had a problem for some time now when opening exported CSV using Internet Explorer. We knew the problem stemmed from security settings in Internet Explorer. Until now, we have not had a solution that did not involve weakening security settings in Internet Explorer. This was important to us because our institution has security policies that are enforced internally for applications on our domain like Internet Explorer. Weakening security settings just was not an option we could use.

Just the other day, Andrew Hollamon, discovered a solution to this problem that had been plaguing us. He discovered that a setting in Internet Explorer would disable caching on pages rendered through HTTPS (details here http://support.microsoft.com/kb/812935). Basically, what happens is the security settings in Internet Explorer require it to submit a header to the server requesting a page without caching. Caching is necessary because the file is cached before it can be opened. If the page is not cached, then there is nothing to be opened. Andrew's solution was to add some custom code to the Rice WebUtils class that changes the headers only for requests of reports in CSV.

We plan to contribute this back, but any institution can implement a solution easily enough by simply modifying the necessary headers in the servlet response according to this document http://support.microsoft.com/kb/812935)