Robust Java benchmarking

Doing Micro benchmarks on the JVM is very hard, due to the Hotspot compiler (assuming you are on a SUN/SAP JVM) doing very advanced optimizations at runtime.

I usually recommend to be very careful about doing any micro benchmarks without having a real JVM hacker nearby ;)

Brent Boyer has written an article series at developerworks that explains how to do it correctly and he also presents a framework to make it easier to write robust microbenchmarks :

Robust Java benchmarking, Part 1

Robust Java benchmarking, Part 2


Also I have not yet tried the framework, it looks promising.