Fork me on GitHub

n. Slang a rough lawless young Kuali developer.
[perhaps variant of Houlihan, Irish surname]
kualiganism n

Blog of an rSmart Java Developer. Full of code examples, solutions, best practices, et al.

Saturday, July 23, 2011

Lightning IMPEX

Screencast

I put together a screencast that shows how a certain branch of the impex tool is really really fast and resource friendly for KFS 3.0 installations. Here it is.



Steps


1 Get kfs-cfg-dbs

$ svn co https://test.kuali.org/svn/kfs-cfg-dbs/branches/release-3-0-1/ kfs-cfg-dbs-3-0-1

2 Get rice-cfg-dbs

$ svn co https://test.kuali.org/svn/rice-cfg-dbs/branches/rice-release-1-0-1-1-br rice-cfg-dbs-1-0-1-1

3 Get kul-cfg-dbs

$ svn co https://test.kuali.org/svn/kul-cfg-dbs/branches/kul-handle_dollar_signs_in_tables-br kul-cfg-dbs

4 Copy sample properties file

$ cp impex/impex-build.properties.sample $HOME/rice-impex-build.properties 
$ cp impex/impex-build.properties.sample $HOME/kfs-impex-build.properties

5 Copy library file

$ cd kul-cfg-dbs/impex;
$ cp kuali-impextasks.jar lib

6 Run import in rice-cfg-dbs

$ cd rice-cfg-dbs-1-0-1-1;
$ export ANT_OPTS="-Xmx2048m -XX:MaxPermSize=256m"
$ ant -f ../kul-cfg-dbs/build.xml -Dimpex.build.properties=$HOME/rice-impex-build.properties import

7 Run import in kfs-cfg-dbs

$ cd kfs-cfg-dbs-3-0-1;
$ ant -f ../kul-cfg-dbs/build.xml -Dimpex.build.properties=$HOME/kfs-impex-build.properties import

Struts 1, Spring, PropertyChangeEvent, and the Observer Pattern in Kuali

Overview

There are a couple things that have plagued me as a developer on Kuali.
  • No abstraction layer between struts and spring
  • No subsystem for reacting to business object changes other than validation


There is this fantastic validation framework in KFS/Kuali software. However, validation is a passive action. It simply checks for inconsistencies and reports on them. A validation can never change the state in any way whatsoever. For example, checkbox toggling. What if when a check box is selected 1, 2, or even 3 other boxes are deselected. Maybe a select list is modified. Perhaps a set of checkboxes are disabled after checking a box somewhere on the form. How do we handle this? The instinctive thing to do is to write some rather messy code in the Action class.

I'll come back to that later. Speaking of the Action class, I have noticed that it is very difficult for me as a developer to resist putting business logic in the Action class. The Action and Form classes are intended to be part of an abstraction layer between the Struts M-V-C components. With Spring providing an IOC with services, it isn't very clear where certain logic should live. What do I do if I want as little logic in my Action and Form as possible, so that the services are doing most of the work?

The Observer Pattern

Below is how I implemented the Observer pattern for Struts to communicate with Spring and not have any coupling between SOA and MVC. Here's how I did it:

1 Define Spring Service Beans

It is easiest to just build your framework first with SOA. That way everything else can fit into place around it. I'm going to create all my Observables and Observers in Spring.
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
Community License, Version 2.0 (the "License"); you may not use this file 
except in compliance with the License. You may obtain a copy of the License 
at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
law or agreed to in writing, software distributed under the License is distributed 
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
express or implied. See the License for the specific language governing permissions 
and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<!-- Struts Events -->
<bean id="addOtherExpenseEvent" class="org.kuali.kfs.module.tem.document.web.struts.AddOtherExpenseEvent">
<property name="travelReimbursementService" ref="temTravelReimbursementService" />
<property name="ruleService"                ref="kualiRuleService" />
</bean>
<bean id="removeOtherExpenseEvent" class="org.kuali.kfs.module.tem.document.web.struts.RemoveOtherExpenseEvent">
</bean>
<bean id="addExpenseDetailEvent" class="org.kuali.kfs.module.tem.document.web.struts.AddExpenseDetailEvent">
<property name="travelReimbursementService" ref="temTravelReimbursementService" />
<property name="ruleService"                ref="kualiRuleService" />
</bean>
<bean id="removeExpenseDetail" class="org.kuali.kfs.module.tem.document.web.struts.AddExpenseDetailEvent">
</bean>


<!-- Struts Observable Pattern -->
<bean class="org.kuali.kfs.module.tem.document.web.struts.TravelStrutsObservable">
<property name="observers">
<map>
<entry key="addOtherExpenseLine">
<list>
<ref bean="addOtherExpenseEvent" />
</list>
</entry>
<entry key="deleteOtherExpenseLine">
<list>
<ref bean="removeOtherExpenseEvent" />
</list>
</entry>
<entry key="addOtherExpenseDetailLine">
<list>
<ref bean="addExpenseDetailEvent" />
</list>
</entry>
<entry key="deleteOtherExpenseDetailLine">
<list>
<ref bean="removeExpenseDetailEvent" />
</list>
</entry>
</map>
</property>
</bean>
...
...
</beans>


Each event is an Observer. The Observer is basically waiting for the Observable (by observing) to notify it. For example,
...
...
<bean id="addOtherExpenseEvent" class="org.kuali.kfs.module.tem.document.web.struts.AddOtherExpenseEvent">
<property name="travelReimbursementService" ref="temTravelReimbursementService" />
<property name="ruleService"                ref="kualiRuleService" />
</bean>
...
...
is an Observer.

The Observable is just another Spring bean/service. It has a Map of observers. The MVC action or methodToCall is mapped to the Observer. When certain button clicks happen (addOtherExpenseLine for example), the Observable will notify the appropriate Observer/Event.

2 Create the Observable

This is the interesting part. Not only do we need to create an Observable, but it needs to be flexible or dynamic enough that we aren't actually creating more work for ourselves by trying to solve this problem. Here's how I did it. I added the Observable to the TravelFormBase class. Now TravelFormBase is my base Form class that I use to store common code for all the TEM documents. This ensures that all the TEM documents are going to get the same Observable.
...
...
import java.util.Observable;
...
...
public abstract class TravelFormBase extends KualiAccountingDocumentFormBase implements TravelMvcWrapperBean {
...
...
private Observable observable;
...
...

/**
* Gets the observable attribute.
* 
* @return Returns the observable.
*/
public Observable getObservable() {
return SpringContext.getBean(TravelStrutsObservable.class);
}
...
...
}

I cut out a lot, but notice a couple things. My TravelFormBase is NOT the Observable. Just as I showed before in my Spring configuration, the TravelStrutsObservable is my Observable. I am getting that through my handy dandy SpringContext class. This is great because now I passively rely on the Observable to manage the Spring work. I only have to ever lookup one Spring bean and that is the Observable. The rest is handled through Dependency Injection!

Now let's look at that Observable.
/*
* Copyright 2011 The Kuali Foundation
* 
* Licensed under the Educational Community License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* 
* http://www.opensource.org/licenses/ecl2.php
* 
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.kuali.kfs.module.tem.document.web.struts;

import java.util.List;
import java.util.Map;
import java.util.Observable;
import java.util.Observer;

import org.kuali.kfs.module.tem.document.web.bean.TravelMvcWrapperBean;

/**
*
* @author Leo Przybylski (leo [at] rsmart.com)
*/
public class TravelStrutsObservable extends Observable {
public Map<String, List<Observer>> observers;

/**
* deprecating this since the best practice is to use Spring
*/
@Deprecated
public void addObserver(final Observer observer) {      
super.addObserver(observer);
}

public void notifyObservers(final Object arg) {
TravelMvcWrapperBean wrapper = null;
if (arg instanceof TravelMvcWrapperBean) {
wrapper = (TravelMvcWrapperBean) arg;
}

final String eventName = wrapper.getMethodToCall();
for (final Observer observer : getObservers().get(eventName)) {
observer.update(this, wrapper);
}
clearChanged();
}

/**
* Gets the observers attribute.
* 
* @return Returns the observers.
*/
public Map<String, List<Observer>> getObservers() {
return observers;
}

/**
* Sets the observers attribute value.
* 
* @param observers The observers to set.
*/
public void setObservers(final Map<String,List<Observer>> observers) {
this.observers = observers;
}
}

What? That's it?! Yeah, that's it. You can see that notifyObservers() is overridden to iterate through the observer map and notifies the Observer mapped to the methodToCall on the wrapper.

3 TravelMvcWrapperBean

You may be wondering what in the world that TravelMvcWrapperBean is. You may recall that one of the goals is to abstract the MVC layer. I don't want further coupling to Struts. Even though I have a TravelStrutsObservable, that's about as deep as the coupling gets. Actually, you can probably tell from the Observable code that besides the name, nothing really couples that to Struts. Not even the getMethodToCall(). Everything is abstracted by this new TravelMvcWrapperBean which is just an interface.
/*
* Copyright 2010 The Kuali Foundation.
* 
* Licensed under the Educational Community License, Version 1.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* 
* http://www.opensource.org/licenses/ecl1.php
* 
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.kuali.kfs.module.tem.document.web.bean;

import java.util.Collection;
import java.util.Map;
import java.util.List;

import org.kuali.rice.kns.bo.Note;
import org.kuali.rice.kns.document.Document;
import org.kuali.rice.kns.web.ui.ExtraButton;
import org.kuali.kfs.module.tem.document.TravelDocument;

public interface TravelMvcWrapperBean {

Integer getTravelerId();

TravelDocument getTravelDocument();


void setTravelerId(Integer travelerId);


Integer getTempTravelerId();


void setTempTravelerId(Integer tempTravelerId);


/**
* Gets the empPrincipalId attribute.
* 
* @return Returns the empPrincipalId.
*/
String getEmpPrincipalId();


/**
* Sets the empPrincipalId attribute value.
* 
* @param empPrincipalId The empPrincipalId to set.
*/
void setEmpPrincipalId(String empPrincipalId);


/**
* Gets the tempEmpPrincipalId attribute.
* 
* @return Returns the tempEmpPrincipalId.
*/
String getTempEmpPrincipalId();


/**
* Sets the tempEmpPrincipalId attribute value.
* 
* @param tempEmpPrincipalId The tempEmpPrincipalId to set.
*/
void setTempEmpPrincipalId(String tempEmpPrincipalId);


Map<String, String> getModesOfTransportation();

/**
* Gets the showLodging attribute.
* 
* @return Returns the showLodging.
*/
boolean isShowLodging();


/**
* Sets the showLodging attribute value.
* 
* @param showLodging The showLodging to set.
*/
void setShowLodging(boolean showLodging);


/**
* Gets the showMileage attribute.
* 
* @return Returns the showMileage.
*/
boolean isShowMileage();


/**
* Sets the showMileage attribute value.
* 
* @param showMileage The showMileage to set.
*/
void setShowMileage(boolean showMileage);


/**
* Gets the showPerDiem attribute.
* 
* @return Returns the showPerDiem.
*/
boolean isShowPerDiem();


/**
* Gets the canReturn attribute value.
* 
* @return canReturn The canReturn to set.
*/
boolean canReturn();

/**
* Sets the canReturn attribute value.
* 
* @param canReturn The canReturn to set.
*/
void setCanReturn(final boolean canReturn);

/**
* Sets the showPerDiem attribute value.
* 
* @param showPerDiem The showPerDiem to set.
*/
void setShowPerDiem(boolean showPerDiem);

boolean isShowAllPerDiemCategories();


/**
* This method takes a string parameter from the db and converts it to an int suitable for using in our calculations
* 
* @param perDiemPercentage
*/
void setPerDiemPercentage(String perDiemPercentage);


/**
* Gets the perDiemPercentage attribute.
* 
* @return Returns the perDiemPercentage.
*/
int getPerDiemPercentage();


/**
* Sets the perDiemPercentage attribute value.
* 
* @param perDiemPercentage The perDiemPercentage to set.
*/
void setPerDiemPercentage(int perDiemPercentage);

Map<String, List<Document>> getRelatedDocuments();

void setRelatedDocuments(Map<String, List<Document>> relatedDocuments);


/**
* Gets the relatedDocumentNotes attribute.
* 
* @return Returns the relatedDocumentNotes.
*/
Map<String, List<Note>> getRelatedDocumentNotes();


/**
* Sets the relatedDocumentNotes attribute value.
* 
* @param relatedDocumentNotes The relatedDocumentNotes to set.
*/
void setRelatedDocumentNotes(Map<String, List<Note>> relatedDocumentNotes);

boolean isCalculated();

void setCalculated(boolean calculated);

List<ExtraButton> getExtraButtons();

String getMethodToCall();
}

The interface is implemented by my TravelFormBase that I showed you earlier.
...
...
public abstract class TravelFormBase extends KualiAccountingDocumentFormBase implements TravelMvcWrapperBean {
...
...
}
This means that when you see the TravelMvcWrapperBean referenced, it's really the struts Form class. Keep that in mind as we delve further in.

4 Observers

Now we discuss what those events were all about in my Spring configuration. Those events are my Observers. They get triggered by the Observable. I will show you how. In a normal request, the Controller hands the Form off to the Action and calls the execute() method. It has already set the methodToCall on the form, so execute() will use this to delegate further in the Action. If my methodToCall is addOtherExpenseLine, then the addOtherExpenseLine() method is called. Here
...
...
/**
* Action method for adding an {@link OtherExpense} instance to the {@link TravelReimbursementDocument}
* 
* @param mapping
* @param form
* @param request
* @param response
* @return
* @throws Exception
*/
public ActionForward addOtherExpenseLine(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception {
final TravelReimbursementForm reimbForm = (TravelReimbursementForm) form;
final TravelReimbursementMvcWrapperBean mvcWrapper = newMvcDelegate(form);
reimbForm.getObservable().notifyObservers(mvcWrapper);

return mapping.findForward(KFSConstants.MAPPING_BASIC);
}
...
...

Here you can see that it gets the Observable from the Form and then passes in the TravelReimbursementMvcWrapperBean which is really the Form. Why not just pass in the Form? Well, that would couple us to Struts. I will illustrate that later on. From what we already know, the Observable will examine the wrapper for its methodToCall, and grab the appropriate Observer(s) for it. It then calls update on them. This is what happens for the AddOtherExpenseEvent
...
...
public void update(final Observable observable, Object arg) { 
if (!(arg instanceof TravelReimbursementMvcWrapperBean)) {
return;
}
final TravelReimbursementMvcWrapperBean wrapper = (TravelReimbursementMvcWrapperBean) arg;

final TravelReimbursementDocument document = wrapper.getTravelReimbursementDocument();
final ReimbursementOtherExpense newOtherExpenseLine = wrapper.getNewOtherExpenseLine();
newOtherExpenseLine.refreshReferenceObject("travelExpenseTypeCode");

getTravelReimbursementService().handleNewOtherExpense(newOtherExpenseLine);            

boolean rulePassed = true;

// check any business rules
rulePassed &= getRuleService().applyRules(new AddOtherExpenseLineEvent(NEW_OTHER_EXPENSE_LINE, document, newOtherExpenseLine));

if (rulePassed) {
document.addOtherExpense(newOtherExpenseLine);
final OtherExpenseDetail newDetail = new OtherExpenseDetail();
newDetail.setExpenseDate(newOtherExpenseLine.getExpenseDate());
wrapper.getNewOtherExpenseDetailLines().add(newDetail);
wrapper.setNewOtherExpenseLine(new ReimbursementOtherExpense());
wrapper.getNewOtherExpenseLine().setDetails(new ArrayList());
}

wrapper.setCalculated(false);
}
...
...

You can see there is no mention of Struts anywhere in this Spring bean. It uses 2 services that are injected (KualiRuleService and TravelReimbursementService), so it never has to call SpringContext to get to those. You can see it uses the TravelReimbursementMvcWrapperBean as wrapper to communicate with the Form object without knowing or caring that is a Form.

5 Proxying the Form

Why not just use the Form? Why does the Action class have to lookup an MVC Delegate and pass that as the wrapper? This is to further abstract away from Struts. Instead of calling the Struts object directly, a proxy is made on it. I added the following method to my TravelActionBase so that all my Action classes would have access to it
...
...
public <T> T newMvcDelegate(final ActionForm form) throws Exception {
T retval = (T) Proxy.newProxyInstance(getClass().getClassLoader(),
new Class[] { getMvcWrapperInterface() },
new TravelMvcWrapperInvocationHandler(form));
return retval;
}
...
...
/**
* Just a passthru {@link InvocationHandler}. It's used when creating a proxy, to access methods in a class
* without knowing what that class really is. This allows us to put a facade layer on top of whatever MVC we use;
* hence, the name {@link TravelMvcWrapperInvocationHandler}
*
* @author Leo Przybylski leo [at] rsmart.com
*/
class TravelMvcWrapperInvocationHandler<MvcClass> implements InvocationHandler {
private MvcClass mvcObj;

public TravelMvcWrapperInvocationHandler(final MvcClass mvcObj) {
this.mvcObj = mvcObj;
}        

public Object invoke(final Object proxy, final Method method, final Object[] args) throws Exception {
return method.invoke(mvcObj, args);
}
}
...
...
This will allow me to make changes to my Form that normally would not be acceptable or whatever.

PropertyChangeEvent

I think I will save this for another post.

Conclusion

That's how you can abstract your code out of Struts and add more to your SOA. This will of course make your code easier to unit test. It will also strip any non-UI related logic from your Action.

Thursday, July 14, 2011

UPDATE: Kuali Days 2011 Presentation Proposals

"The Status is Not Quo!" JSR-286 and Rice has been curbed. I and a few others may try to put together an informal meetup regarding this, but it was not accepted as a presentation. I will definitely add it again for next year.

Thanks everyone for the feedback!

Wednesday, July 13, 2011

Kuali Days 2011 Presentation Proposals

Overview

Below are proposals I am submitting for Kuali Days. Please vote in a comment!

The Impact of Dev Ops on Kuali Development and Hosting


Abstract

There is a movement occurring acrossed the OSS universe. It is DEVOPS. The concept is where developers are also the operational staff. They become basically swiss-army knives of software. In organizations spread out remotely, it can be difficult for developers to interact with support staff. Timezones can produce bottlenecks that delay projects. DEVOPS basically come out of necessity.

Objectives

  • Implementations will see how to utilize DEVOPS in their organization.
  • Developers will be discover how to use their skills at their organization to improve development productivity.
  • SAAS systems hugely benefit from DEVOPS to provide the best possible uptime for your hosted software. Implementors and will see how SAAS solutions are more reliable, cost-effective, and secure because of utilizing DEVOPS.

Audience

  • Implementation Track
  • Developers
  • Project Managers

The Status is Not Quo! JSR-286 and Rice


Abstract

JSR-286 is the 2.0 portlet specification. This is a walkthrough of implementing web services as a portlet.

Objectives

  • Developers will see what goes into producing a portlet implementation
  • Developers will learn/understand how to create a web services client in java to communicate with Kuali Rice
  • Developers will learn how to deploy a portlet to a portlet container

Audience

  • Technical track for developers

Rice 1.1.1(2.0) Project: From the Cradle to the Grave


Abstract

A major version update for Kuali Rice is released, but not much is known about implementing it. There will undoubtably be numerous panels and presentations on taking advantage of the new version’s greatest features; however, this will be the one presentation that shows how to set up a new project, prepare it for Continuous Integration and deployment with use cases and best practices. This presentation overviews the new version project setup start to end.

Objectives

  • Best practices for setting up your SVN project to pull in your rice codebase
  • Best practices for structuring your rice project
  • Setting up a strong Continuous Integration environment for development
  • Developing with overlays
  • Deploying to tomcat
  • Deploying to jetty
  • Maven best practices

Audience

  • Technical Rice track for developers and configuration managers

Scripting Batch Processing with Web Services


Abstact

One of the necessities implementers have is to remotely execute batch processes. Many institutions already have existing enterprise scheduling systems that are far more robust than Quartz which is shipped with KFS. The cases for this are when implementing institutions use a third-party scheduling system (Peoplesoft or BMC). Institutions certainly will want to put in place something more robust or may even already have a campus-wide scheduling system.

This is an informative talk on how to connect an enterprise scheduling system with KFS. Attendees will see examples of how to integrate using shell-scripting and web-services.

Objectives

  • Learn about caveats of normal shell-scripting of batch processes.
  • Learn of creating a simple web service that isn't published to Kuali Service Bus (KSB) for executing isolated batch processes.
  • Learn how to create a client for communicating with the batch Web Services Definition Language (WSDL).
  • Batch processes and WS-SEC with Rice and KFS

Audience

  • Developers on a technical

Database Change Migrations with Liquibase


Abstact

Database changes via DDL/DML can be difficult to manage. Much of the time, DDL changes will be unreversable. For example, a table rename may require a table drop and recreate. This is usually favorable because then developers can ignore the proprietary language nature of SQL across RDBMS. Liquibase provides a common language and method supporting almost every database there is. Liquibase can also provide a common methodology for applying updates from the Kuali Foundation across different software systems. This can be very useful in apply upgrades from the Kuali Foundation or managing changes for your local institution.

This session covers intuitive and simple database change management process and how to integrate it with existing data migration, change management, and foundation KC, KFS and Rice updates.

Objectives

  • Implementors will learn best practices for structuring their projects
  • Implementors will learn how to accept database changes from the foundation and structure theirs around
  • Developers will learn how to test changes and the effects of change management within their development process
  • Implementors will learn how to rollback software versions including database changes
  • Implementors will learn how to play/fast-forward changes acrossed several revisions to update to a later version of database and source code
  • Business Intelligence Analysts will learn how to integrate database change management and ETL processes so that source code changes have minimal impact upon their conversion processes
  • Introduce a common methodology for apply upgrades across projects (KC, KFS, and Rice) using Maven and Liquibase

Audience

  • Technical track for DBA’s, developers, and DevOps