• jmh, the Java Microbenchmark Harness takes care of statistical probability calculations to take meaningful results.
    • Trial
    • Setup (at trial, iteration, invocation levels)
    • Teardown
    • Blackhole ensures values are "used".
  • Load generators:
    • JMeter
    • Gatling
    • Faban
    • MicroFocus LoadRunner

Where to start

  • High CPU usage: profiler.
  • High GC time: heap profiler.
  • CPU time decrease and GC time decrease: may be a sign of lock contention, look for synchronisation bottlenecks in stack data.
  • Application latencies: JFR.
  • Data store latency: slow query logs, CPU and I/O metrics.

When benchmarking

  • Ensure the code cache is adequately warmed up: be sure to run warm-up iterations in each JVM before starting measurement.
  • Be sure to use values, else you risk the code that generates them being optimised out by the compiler.
  • Make use of different values (either random or predetermined) to avoid pre-calculation of values.
  • TODO: use the volatile keyword.
  • Tighter iterations in smaller benchmarks will encounter more lock contention due to synchronisation issues not present in the context of a broader application.
  • The GC profile matters: short-lived objects may be discarded in minor GCs during benchmarking but in an application executing many threads may be promoted, requiring expensive full GCs.