Appengine – improving cold startup with spring enabled apps

First of all, in 2013 I refuse to work like 1983. Most of the issues with cold startup seem to be related with appengine being not able to guarantee fast disk operations at all times. There are many times, when their servers have an oversubscribed i/o, that booting time become a bottleneck if you’re doing classpath scanning and things like that.

Below are the bottlenecks for appengine cold startup with spring:

1. At the very beginning, before spring is even called, appengine do a nice classpath scanning searching for taglib stuff. Even if you don’t use taglib, jsp or any other 1997 stuff.
2. context:component-scan is slow. Again, because of classpath scanning.
3. WebApplicationContext takes forever to initialize.
4. reduce classpath scanning in general.

Easy fixes:

Set default-lazy-init=”true” as global beans level.

You can read https://developers.google.com/appengine/articles/spring_optimization, but just ignore where it says about component-scan, as that one is easily solvable by caching at maven time. (explained below)

Solving Taglib Classpath Scanning (TLD search of file)

Yes, this happens even if you don’t use taglib at all. Here are the symptoms:

INFO] Jul 11, 2013 11:47:51 PM com.google.apphosting.utils.jetty.JettyLogger debug
[INFO] FINE: TLD search of file:/Users/mufumbo/workspace-sts/allthecooks-gae/target/allthecooks-gae-1/WEB-INF/lib/lucene-core-3.6.2.jar
[INFO] Jul 11, 2013 11:47:51 PM com.google.apphosting.utils.jetty.JettyLogger debug
[INFO] FINE: TLD search of file:/Users/mufumbo/workspace-sts/allthecooks-gae/target/allthecooks-gae-1/WEB-INF/lib/appengine-api-1.0-sdk-1.8.1.jar
[INFO] Jul 11, 2013 11:47:51 PM com.google.apphosting.utils.jetty.JettyLogger debug
[INFO] FINE: TLD search of file:/Users/mufumbo/.m2/repository/com/google/appengine/appengine-java-sdk/1.8.1/appengine-java-sdk/appengine-java-sdk-1.8.1/lib/impl/agent/appengine-agentruntime.jar

…….. 500 more of these lines later…

This issue is not solvable. This is my stackoverflow thread about this issue. By the way, God forbid myself from using taglib, jsp, struts or anything related to that ugly technology. Also, if you still use those kind of technologies, take the time to read how to make yourself free from 1997 programming with: spring-mvc + velocity + dcvm.

Solving context:component-scan slowness

There’s a nifty library called reflections that enables you to pre-cache the classpath scan for your annotations with a maven plugin.

Here’s my spring configuration, that replaces :

<reflections:component-scan base-package="com.yumyumlabs" collect="true" save="false" parallel="false">
		<reflections:exclude-filter type="regex" expression="org.springframework.(?!stereotype).*"/>
    </reflections:component-scan>

Attention to the parameter “collect“, it basically says that it will enable you to use the serialized information generated by maven at packaging time. Here’s my maven conf for the plugin:

<plugin>
				<groupId>org.reflections</groupId>
				<artifactId>reflections-maven</artifactId>
				<version>0.9.9-RC1</version>
				<executions>
					<execution>
						<id>reflections-generator</id>
						<goals>
							<goal>reflections</goal>
						</goals>
						<phase>process-classes</phase>
					</execution>
				</executions>
				<configuration>
					<!-- Set com.yumyumlabs because it's the basePath for spring scanning! -->
					<destinations>${project.build.outputDirectory}/META-INF/reflections/com.yumyumlabs-reflections.xml</destinations>
					<includeExclude>+com.yumyumlabs.*</includeExclude>
					<parallel>true</parallel>
					<!-- <serializer>JavaCodeSerializer</serializer> -->
				</configuration>
			</plugin>

Here’s my maven conf for the packages:

<dependency>
			<groupId>org.reflections</groupId>
			<artifactId>reflections</artifactId>
			<version>0.9.9-RC1</version>
		</dependency>
		<dependency>
			<groupId>org.reflections</groupId>
			<artifactId>reflections-spring</artifactId>
			<version>0.9.9-RC1</version>
		</dependency>

How to initialize WebApplicationContext faster?

This is where spring spends most of the booting time.

As a science fiction technique, we should be able to serialize and save it in memcache. Unfortunately this is a mess.

I have tried the approach of ContextLoaderListerner and serialize the ApplicationContext to memcache after it’s properly initialized.

Unfortunately this solution didn’t worked because the beanFactory isn’t serialized together, so I had to refresh the AbstractRefreshableApplicationContext, which cause the complete boot to happen.

This is possible at testing time, but unfortunately it’s just an static cache: http://static.springsource.org/spring/docs/3.2.x/spring-framework-reference/html/testing.html#testcontext-ctx-management-caching

Use proguard to shrink classes and reduce classpath scanning

This topic is complex enough for generating another blog thread. I will post as soon as I have some time. We used the maven plugin net.sf.proguard to do this while in the packaging lifecycle.

It’s a rather complex configuration, but it shrunk our package from 50mb to 23mb.

Our maven config is available here: https://gist.github.com/mufumbo/6406033

Our proguard configuration is available here: https://gist.github.com/mufumbo/6406055