Do you get Just-in-time compilation?

October 31, 2012 by Nikita Salnikov-Tarnovski

Remember the last time when you were laughed at by C-developers? That the Java is so slooooow that they would never even consider using a language like this? In many ways, the concept still holds. But for it’s typical usage – in the backbones of a large enterprise – Java performance can definitely stand against many contestants. And this is possible mostly thanks to the magical JIT. Before jumping into explaining Just-In-Time compilation tricks, lets dig into background a bit.

As you might remember – Java is an interpreted language. Java compiler known by most users, javac, does not compile java source files directly into processor instructions like C compilers do. Instead it produces bytecode, machine independent binary format governed by specification. This bytecode is interpreted during runtime by JVM.This is the main reason why Java is so successful in cross-platform – you can write and build the program in one platform and run it on several others.

On the other hand – it does introduce some negative aspects. Out of which one of the most severe is the fact that interpreted code is usually slower than code compiled directly to platform-specific native binaries. Sun realized the severity already at the end of the nineties, when it hired dr Cliff Click to provide a solution.

Welcome – HotSpot. The name derives from the ability of JVM to identify “hot spots” in your application’s – chunks of bytecode that are frequently executed. They are then targeted for the extensive optimization and compilation into processor specific instructions. The optimizations lead to high performance execution with a minimum of overhead for less performance-critical code. In some cases, it is possible for adaptive optimization of a JVM to exceed the performance of hand-coded C++ or C code.

The component in JVM responsible for those optimizations is called Just in Time compiler (JIT). It takes advantage of an interesting program property. Virtually all programs spend the majority of their time executing a minority of their code. Rather than compiling all of your code, just in time, the Java HotSpot VM immediately runs the program using an interpreter, and analyzes the code as it runs to detect the critical hot spots in the program. Then it focuses the attention of a global native-code optimizer on the hot spots. By avoiding compilation of infrequently executed code, the Java HotSpot compiler can devote more attention to the performance-critical parts of the program. This means that your compilation time does not increase overall. This hot spot monitoring is continued dynamically as the program runs, so that it adapts its performance on the fly according to the usage patterns of your application.

JIT achieves the performance benefits by several techniques, such as eliminating dead code, bypassing boundary condition checks, removing redundant loads, inlining methods, etc.

Following samples illustrates those techniques used by JIT to achieve better performance. In the first section there is the code written by a developer. In the second code snippet is the code executed after hotspot has detected it to be “hot” and applied it’s optimization magic:

    1. Unoptimized code.
class Calculator {
   Wrapper wrapper;
   public void calculate() {
      y = wrapper.get();
      z = wrapper.get();
      sum = y + z;
   }
}

class Wrapper {
   final int value;
   final int get() {
      return value;
   }
}
    1. Optimized code
class Calculator {
   Wrapper wrapper;
   public void calculate() {
      y = wrapper.value;
      sum = y + y;
   }
}

class Wrapper {
   final int value;
   final int get() {
      return value;
   }
}

First class described in the small sample above is a class a developer has written and the second is a sample after JIT has finished it’s work. The sample contains several optimization techniques applied. Lets try to look how the final result is achieved:

    1. Unoptimized code. This is the code being run before it is detected as a hot spot:
public void calculate() {
   y = wrapper.get();
   z = wrapper.get();
   sum = y + z;
}
    1. Inlining a method. wrapper.get() has been replaced by b.value as latencies are reduced by accessing wrapper.value directly instead of through a function call.
public void calculate() {
   y = wrapper.value;
   z = wrapper.value;
   sum = y + z;
}
    1. Removing redundant loads. z = wrapper.value has been replaced with z = y so that latencies will be reduced by accessing the local value instead of wrapper.value.
public void calculate() {
   y = wrapper.value;
   z = y;
   sum = y + z;
}
    1. Copy propagation. z = y has been replaced by y = y since there is no use for an extra variable z as the value of z and y will be equal.
public void calculate() {
   y = wrapper.value;
   y = y;
   sum = y + y;
}
    1. Eliminating dead code. y = y is unnecessary and can be eliminated.
public void calculate() {
   y = wrapper.value;
   sum = y + y;
}

The small sample contains several powerful techniques used by JIT to increase performance of the code. Hopefully it proved beneficial in understanding this powerful concept.

Enjoyed the post? We have a lot more under our belt. Subscribe to either our RSS feed or Twitter stream and enjoy.

The following references were used for this article (besides two angry C developers):

Can't figure out what causes your OutOfMemoryError? Read more

ADD COMMENT

COMMENTS

not enjoyed at all. trivial and it is obvious that exactly this technic will not seed up the code.

yuraminsk

why doesn’t java expose this optimization functionailty (menu item in eclipse or netbeans) so that users can click it and optimize all of thier code or as needed?n

Guest

It is often runtime specific. I mean it depends upon particular execution flow.

iNikem

Because you’d loose all platform independence.

Locked-spamcollect6

If the value field would be marked as volatile, the redundant calls are not allowed to be optimized.

Peter Veentjer

Volatile is a special breed indeed as they involve memory barriers.

iNikem

Hello! Optimized code is not equal to original one due to multi-threading issues. In other words, wrapper.value state may be changed between two subsequent calls so optimizer must not replace two calls with single one

Alexander Pavlov

In the strict sense you are right. Without further insurance JIT cannot make exactly those optimisations. But this serves as a good enough example IMO :)

iNikem

I believe “final int value;” will make it perfect example :)

Alexander Pavlov

Thanks, I’ve changed the article :)

iNikem

AFAIK final is not needed. As long as within the thread the reordering of instructions are not visible (so within-thread as-if-serial semantics), then everything is allowed. Compiler optimizations could be seen as instruction reorderingsnnnThis helps to make cpu’s fast.. (also cpu’s and caches can do funky stuff with code). All this can be seen as instruction reorderings. nIf other thread would look at this code, than they are able to observe these changes. This is allowed according to the JVM. If you do not want them to observe these changes, you need to explicitly take the correct actions (e.g. making stuff volatile, final, synchronized blocks etc etc).

Peter Veentjer

The JIT is allowed to do so. nnnThe compiler/cpu can go as nuts with the code as they want, as long as the within-thread as-if-serial are not violated, and in this case they are not. So the optimizations are allowed.

Peter Veentjer

Could you please give some links to read more about that?

Alexander Pavlov

I would go for:nn- Java Concurrency in Practice (there is a chapter about the JMM)n- Specification for JSR-133.n- Have a quick peek here:nhttp://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html

Peter Veentjer

Can't figure out what causes your OutOfMemoryError? Read more

Latest
Recommended
You cannot predict the way you die
When debugging a situation where systems are failing due to the lack of resources, you can no longer count on anything. Seemingly unrelated changes can trigger completely different messages and control flows within the JVM. Read more
Tuning GC - it does not have to be that hard
Solving GC pauses is a complex task. If you do not believe our words, check out the recent LinkedIn experience in garbage collection optimization. It is a complex and tedious task, so we are glad to report we have a whole lot simpler solution in mind Read more
Building a nirvana
We have invested a lot into our continuous integration / delivery infrastructure. As of now we can say that the Jenkins-orchestrated gang consisting of Ansible, Vagrant, Gradle, LiveRebel and TestNG is something an engineer can call a nirvana. Read more
Creative way to handle OutOfMemoryError
Wish to spend a day troubleshooting? Or make enemies among sysops? Registering pkill java to OutOfMemoryError events is one darn good way to achieve those goals. Read more