Shaun Abram
Technology and Leadership Blog
Subscribe to RSS FeedGroovy: Cool but slow
I have been learning Groovy recently (as part of Project Euler, a series of Maths programming problems). For the most part, I love it. I particularly like
the easy syntax. Java without the crud!
e.g.
System.out.printlnbecomes simplyprintlnfor (int i-0; i<10; i++)becomesfor ( i in 0..9 )- Accessor methods are autogenerated and used
Sure, the poor IDE support is a little annoying, especially the lack luster code completion and debugging capabilities, but I am sure it will improve soon (especially with SpringSource on the case).
But then I started to notice some slowness when using it. So, I performed a simple performance comparison and the results shocked me.
I wrote an identical loop in both Java and Groovy (see the code below) which simply loops a million times.
- The Java version took ~5ms
- The Groovy version took ~500ms
I was shocked! Could Groovy really be a 100 times slower? I had heard rumors about Groovy being a bit slower than Java, but I hadn't expected it to be this dramatic. Surely this is drastic enough to stop Groovy being used in serious, production quality projects?
I discussed this with a friend who is a Groovy advocate. He made the following points:
1. I was running with an older, slower version of Groovy
This is true. I ran the tests in Eclipse 3.5, with v1 of the Eclipse Groovy plugin, which uses v1.5.7 of the Groovy compiler.
However, when I downloaded the latest version of Groovy (Version: 1.6.3 with JVM: 1.6.0_13) and ran from the command line, it still took the Groovy code ~200 ms. A huge improvement, but still ~40 times slower than Java.
2. It is an artificial, unrealistic test, as these kind of huge loops are not normal in most programs
This is also true. But although artificial, it is still a straight comparison. Also, these kind of loops aren't so unusual in Project Euler type problems (at least in my crude brute force solutions!).
There are also some more detailed Java/Groovy performance test results here and here.
So, overall, I think the performance is a big issue. But not enough to stop me using it for pet projects. I am even going to keep trying to use it for Project Euler - it will be more motivation to avoid the brute force solutions. It is however, enough to stop me using it in any serious production apps.
For the record, here is the exact code I ran:
Java version
long max = 1000000;
long start = Calendar.getInstance().getTimeInMillis();
for(int i=0;i
Groovy version
long max = 1000000;
long start = Calendar.getInstance().getTimeInMillis();
for(int i=0;i
It is also worth pointing out that when I changed the Groovy code to use proper Groovy syntax, as in for (i in 1..1000000) is was substantially faster (although still significantly slower than Java), but I have left it as is, for a apples vs apples comparison.
Tags: groovy
You should actually do something in your loop to stop the influence of micro optimizations performed by the compiler.
Hey – I’m a big fan of Groovy too and especially Grails, which I’ve been using for some pet projects.
Was intrigued by your results – I also expect something like Groovy to be slower than the Java equivalent, which is the price you have to pay for the syntactical niceness the dynamic language brings you, but was surprised at how slow you found things. I guess any tests like this are always going to be subjective, and with something like Grails you do have the option of falling back to Java for any areas that are proving to be performance bottlenecks in your running app.
Tried a few tests myself, and with a few tweaks, the results end up being a lot closer:
1. modify your test so that there is some activity within the loop (just instantiating an empty String)
2. Run your test loop multiple times and discard the timings of the first few runs – whenever we run performance tests for a production webapp we always give the JVM a chance to ‘warm up’ before starting the test statistics, thereby giving the JVM a chance to work its internal optimzation magic.
3. Run with a 1.6 rather than a 1.5 JVM (I note that you’re already doing this, but initially I wasn’t and it made a massive impact to the Groovy timings). This gave about a 50x perf boost to the Groovy test for me!
Now running the tests, the Java version took ~30ms and the Groovy version took ~70ms
For the record, here are the Groovy and Java versions I’m using
monaco:Temp james$ groovy -version
Groovy Version: 1.6.4 JVM: 1.6.0_13
monaco:Temp james$ java -version
java version “1.6.0_13”
Java(TM) SE Runtime Environment (build 1.6.0_13-b03-211)
Java HotSpot(TM) 64-Bit Server VM (build 11.3-b02-83, mixed mode)
… and here are the modified test files
for (outer in 1..20){
long max = 1000000;
long start = Calendar.getInstance().getTimeInMillis();
for(int i=0;i<max; i++) {
String s=new String();
}
long end = Calendar.getInstance().getTimeInMillis();
long duration = end – start;
println(duration);
}
… and Java…
import java.util.*;
public class Test{
public static void main(String[] args){
for (int outer=1; outer<20; outer++){
long max = 1000000;
long start = Calendar.getInstance().getTimeInMillis();
for(int i=0;i<max; i++) {
String s=new String();
}
//for (i in 1..max){}
long end = Calendar.getInstance().getTimeInMillis();
long duration = end – start;
System.out.println(duration);
}
}
}
You should also use Long in java version of this test, since groovy doesn’t have primitives.
For the record, IDE support for Groovy is not poor. The article linked is from 2008/01/23, and it may have been poor at that time, but IDE support has greatly improved since then. From my experience in Groovy development over the past year, support for it in IntelliJ IDEA is excellent, including code completion and debugging support. I’ve also heard that SpringSource’s STS has good support for Groovy, but haven’t used it myself.
Josh,
Thanks for the comments. I stand by my post and I think IDE support for Groovy has still a long way to come before matching Java – my benchmark, as unfair as that may be. But I think we can definitely agree that great progress is being made.
There was an interesting article on Groovy Eclipse work from Spring a ago (http://groovy.dzone.com/news/interview-meet-groovy-eclipse).
Shaun