Grails Webdav Plugin with Apache Shiro causes Hibernate LazyInitializationException

We use Grails 2.1 with Apache Shiro 1.1.3 as security layer which handles the user authentication such as login via web app or webdav. To provide an easy and flexible file service we use webdav plugin 3.0.1. It hides the complexity of the HTTP file protocol webdav and gives a simple interface to work with virtual network filesystem structure.

When a user tries to login, our SystemDBRealm authenticates the given user with its password against the database and does other checks as well. These checks include the validation of the user role. The user role is modeled via SystemUserRoleRel within our grails domain model. SystemUserRolRel has two fields: user and role which links the SystemUser and SystemRole domain models together. Hibernate loads these models lazy in special hibernate proxies. These model proxies are loaded on demand to proper domain model instances which requires a valid hibernate session.

To check that given user has a valid role we execute following code (the code is simplified to depict the problem):

def user = SystemUser.findByUsername(username)
def standardRole = SystemRole.findByName('Standard')
def hasStandardRole = SystemUserRolRel.findAllByUser(user).find { it.role == standardRole } != null

On a normal login via browser SystemDBRealm has a hibernate session and can resolve the it.role hibernate proxy within the find closure find { it.role == standardRole }.

In case of a login via Webdav (e.g. through cadaver, a command line webdav client), the user is authenticated via BasicHttpAuthenticationFilter of shiro. As described in the Basic HTTP Auth with Shiro in Grails this basic authentication filter is configured in config.groovy like this:

security.shiro {
        authc.required = false
        filter.config = """\
[main]
authcBasic = org.apache.shiro.web.filter.authc.BasicHttpAuthenticationFilter
authcBasic.applicationName = Rewoo World

[urls]
/webdav/** = authcBasic
"""
}

The BasicHttpAuthenticationFilter extracts the user password token and returns it to shiro. Shiro processes this authentication token in the SystemDBRealm like the browser based login. But now the hibernate session is missing and the find closure find { it.role == standardRole } throws a hibernate LazyInitializationException: it.role can not be resolved.

To solve this issue we use the withNewSession closure of an arbitrary domain model class to wrap the authentication code defined above (in our case we choose the SystemUser class but another class should be ok as well):

SystemUser.withNewSession {
  def user = SystemUser.findByUsername(username)
  def standardRole = SystemRole.findByName('Standard')
  def hasStandardRole = SystemUserRolRel.findAllByUser(user).find { it.role == standardRole } != null
}

Now a hibernate session is bound to the closure and it.role hibernate proxy can be resolved again.

PS: A ticket is filed at GPWEBDAV-18 for this issue. Comments are welcome.

Advertisements

How to use gorm after a http session timeout

In the previous part we’ve already discussed how to get informed about “regular” logout- and timeout-events in order to maintain a list of currently logged in users. But in our use-case we not only want to maintain a simple list, but also want to do some database cleanup queries whenever a user logs out (in our case: we have to remove some “locking” entries, identifiying resources exclusively locked by the current user).

We’ve already identified the two different places where we have to handle the logout event:

  1. The “onLogout” method of the RewooAuthenticationListener (informed by shiro when a regular logout occurres)
  2. The “sessionDestroyed” method of the RewooHttpSessionListener (informed by the servlet container when a timeout occurres)

The first scenario can be handled quite easily: because we are informed about the event by the shiro plugin we will have a fully initialized hibernate session which can be used to clean up your tables in the usual way:

public class RewooAuthenticationListener implements AuthenticationListener {
...
    public void onLogout(PrincipalCollection principalCollection) {
        // Maintain your user list as described in the previous post
        // ...
        // Do the clean up now
        SystemUser user = SystemUser.findByName(userName)
        ExclusiveLocks.findAllByUser(user) {
            it.delete()
        }
    }
...
}

The second one is the tricky part: because the method is called by the servlet container we will not have any valid hibernate session bound to the SessionHolder of the thread-context. So, the Gorm calls will break with a “No hibernate session bound to thread” exception The first solution which came into our minds was, to retrieve a new session by wrapping the routine in a simple withNewSession-closure call, like this:

public class RewooHttpSessionListener implements HttpSessionListener {
...
    public void sessionDestroyed(HttpSessionEvent event) {
        // Maintain your user list as described in the previous post
        // ...
        // Do the clean up now
        ExclusiveLocks.withNewSession {
            SystemUser user = SystemUser.findByName(userName)
            ExclusiveLocks.findAllByUser(user) {
                it.delete()
            }
        }
    }
    ...
}

In fact this solution will work for the FIRST timeout event. But afterwards you will get the following exception each time a new timeout occurres:

org.springframework.dao.DataAccessResourceFailureException: Could not open Hibernate Session; nested exception is org.hibernate.SessionException: Session is closed!

After some research and digging into the grails / hibernate code we discovered that after leaving the withNewSession closure (which resides within a grails class called “HibernatePluginSupport”) the session is closed but not removed from the SessionHolder. When the next timeout occurres the old session will be found by hibernate and the exception will be thrown. Interestingly enough the clean up code used by grails 1.3.5 is significantly different to the new cleanup algorithm used by HibernateGormEnhancer (which replaces HibernatePluginSupport in newer grails versions). Because of some compatibility issues we can not upgrade to a newer version of grails at the moment so we can not say if the problems described here will vanish with a newer grails release.

We’ve searched around for a while to find a possibility to remove the outdated session from the session holder, but we did not found any proper way to accomplish this:

  • you can not access the underlying sessionMap directly, because it is a private variable and for encapsulation purposes there is no getter
  • you can not use the removeSession method, because you have to pass in the key of the session to remove: unfortunately this key is not the session itself but a private object created by SessionHolder during instantiation of the holder and there is no way to retrieve it
  • even calling the clean method of the SessionHolder will not clean the internal map

Finally we found a solution: by calling the SessionHolder#getValidatedSession you will only get the current session, if the session is not already closed. In case it is closed, it will be removed from the internal map as a side effect. So, that is exactly what we want to have in our case but be aware that this is a very ugly workaround. It depends upon an implementation detail of a class which is not under your control.

Our solution leads us to the following method which can be used to clean up the hibernate session:

private void cleanUpHibernateSession() {
    SessionHolder sessionHolder = (SessionHolder) TransactionSynchronizationManager.getResource(SessionGrabber.getSessionFactory())
    if (sessionHolder != null && !sessionHolder.isEmpty()) {
        // Calling this method will "remove" already closed sessions as a side-effect (so we don't care about the return-value)
        sessionHolder.getValidatedSession()
        if (sessionHolder.isEmpty()) {
            sessionHolder.clear()
        }
    }
}

The method will be called after the withNewSession closure call like this:

public class RewooHttpSessionListener implements HttpSessionListener {
...
    public void sessionDestroyed(HttpSessionEvent event) {
        // Maintain your user list as described in the previous post
        // ...
        // Do the clean up now
        ExclusiveLocks.withNewSession {
            SystemUser user = SystemUser.findByName(userName)
            ExclusiveLocks.findAllByUser(user) {
                it.delete()
            }
        }
        cleanUpHibernateSession()
    }
    ...
}

This will make sure that the next call of withNewSession will not be disturbed by an already closed session.

Conclusion

The solution we’ve presented here give you the possibility to clean up parts of your database each time a user logs out. We don’t feel very comfortable with our workaround for the SessionHolder problem: if you know a better solution we would be glad to hear about it!

Count prepared statements within your grails application

Sometimes it is not so easy to find hotspots within your grails application. Even commercial profilers like JProfiler seem to have their problems generating useful “output” of the artifacts generated by the grails/groovy stack. For lack of alternatives we use a “primitve” closure to measure the time-consumption of a code-block, just to get some hint if we increased the performance of a block of code (e.g. a grails-service) or not.

But sometimes you need more than just a time-measurement closure. A good point to start performance-optimizations in our application seems to be to watch out for blocks doing a lot of sql-statements. With all that ORM and GORM magic you can run very fast in a situation that hundreds of selects and updates are fired within one service but you don’t know which individual methods within the service are causing them.

To “count” the number of prepared statements used within a code-block, we use the following closure within a “Measurement” class:

public class Measurement {
    // ...
    public static def countSQLStatements = { String blockIdentifier, Closure toExecute ->
        SessionFactory sessionFactory = ApplicationHolder.application.mainContext.sessionFactory
        boolean oldStatisticsState = enableStatistics(sessionFactory)

        long start = System.nanoTime()
        long queriesBeforeCall = sessionFactory.getStatistics().prepareStatementCount
        toExecute.call()
        long queriesOfCall =  sessionFactory.getStatistics().prepareStatementCount - queriesBeforeCall
        println "${blockIdentifier} caused the execution of ${queriesOfCall} prepared statements and took ${getTimeDeltaInMilliSeconds(start, System.nanoTime())} ms."

        resetStatisticsState(oldStatisticsState, sessionFactory)
        return queriesOfCall
    }
    //...
}

The method “getTimeDeltaInMilliSeconds” is implemented like this:

private static long getTimeDeltaInMilliSeconds(long nanoTimeStart, long nanoTimeEnd) {
    return  (nanoTimeEnd - nanoTimeStart) / 1E6
}

So, whenever you want to know how expensive a method call or other block within your code is, you can just write:

Measurement.countSQLStatements("Some block description",
  { methodCallToMeasure1()
     methodCallToMeasure2()
     // ...
})

Do you have any more tricks how to profile a grails-app? We would love to hear about them.

How to fix Hibernate Query Cache for Grails 1.3.5

To speed up our Grails-application we’ve decided to use hibernates’ second-level- and query-cache. To do so, we performed the following steps:

  • Added appropriate caching-xml-elements to our ehcache.xml (resides within grails-app/conf)
  • Enabled query-cache (“cache.use_query_cache=true” within DataSource.groovy)
  • Enabled the 2nd level cache for the relevant entities
  • Enabled the Query-Cache for a specific query via (SomeClass.findAllByX(someX, [cache: true]))

To test the cache, we implemented an integration-test. Whenever we created an entity and asked for it multiple times the cache worked as expected. The first retrieval-attempt lead to a cache-miss, the following calls to cache-hits. In a next step we adapted the test a little: we changed the entity AFTER it was added to the query-cache and tried afterwards to retrieve it again and again. But the cache just produced cache-misses. After some debugging and reading we discovered that this behaviour was the result of a bug of the hibernate-version used by Grails 1.3.5. You can find details about  this here. In addition to the linked blog-post we will describe here in more detail what you have to do to fix the bug so that you can really use the hibernate-cache with this grails-version.

  1. Download hibernate-distribution-3.3.1.GA-dist.zip from http://sourceforge.net/projects/hibernate/files/
  2. You are just interested in the “project”-folder, so unzip it to a separate project-director
  3. Open the pom with your favourite java-ide (we are using IntelliJ for this)
  4. Download the latest 1.5 jdk and install it. Adapt the project to use this jdk
  5. Open org.hibernate.cache.StandardQueryCache and go to the “put” method. There you will find the following line (around line 90):

    ...
    Long ts = Long.valueOf( session.getTimestamp() );
    ...

    Replace it with the following code:

    ...
    Long ts = Long.valueOf(cacheRegion.nextTimestamp());
    ...
  6. Use the “package” target of “Hibernate Core” to create your customized jar. When Maven have problems to download some of the necessary dependencies (e.g. JAAS jar), you have to download them manually and add them to the appropriate folder within /youruserhome/.m2/repository/
  7. Finally you have your own version of hibernate-core.jar -> add it to your grails installation by copying it to GRAILS_HOME/lib. You will have to overwrite the existing hibernate-core-3.3.1.GA.jar with your customized jar
  8. There is just one step left: don’t forget to also replace the hibernate-core-3.3.1.GA.jar from your local ivy cache. You can find it under/youruserhome/.ivy2/cache. If you don’t replace this jar, Grails will replace your custom  hibernate version with the old one and you will see no effect.

Conclusion

To create a “custom build” of hibernate is no big deal, after you found the sources and all the dependencies you need. But from my point of view you shouldn’t need to do this for an “ordinary feature” like query-caching. I’m not quite sure why Grails ships such an outdated version of hibernate: the current version is 3.6.0 and the version which comes with Grails is 3.3.1.