Debugging a dying Eclipse instance

November 12, 2012 by Vladimir Šor

The story starts with our decision to go to Devoxx.  With a booth. By a caravan. Driving 2,200 kilometers. But this is not going to be a travel story, do not worry.

During the travel we taught our marketing guys to be prepared for doing demos in our booth.  After all, what’s the benefit of dragging all the droids to Antwerp if they are not even good for a demo or two.

As we had ~30 hours of driving time ahead of us, we thought we have enough time to get the droids up to speed.

First stop – verifying the required infrastructure. One of the droids had managed to install a JDK and Eclipse all by himself. So far so good. But when firing up the Eclipse and adding our demo project resulted in CPU usage jumping to 100% and Eclipse becoming completely unresponsive.

So what is going on here? After all, we are hardcore Java troubleshooters, so  this should be a task suitable just for us. No problems, lets just attach jconsole to Eclipse and find out what is going on. First setback – jconsole fails to attach to Eclipse and displays the following error message instead:

Connection Failed

Well, apparently the process we are trying to connect to is so exhausted it cannot even create the JMX connection anymore. Next tool in our belt to the rescue is jstat.

Turning to the jstat –gcutil PID we see that something bad is indeed going on – the permanent generation is filled up to it’s maximum capacity and indeed it might be tough for Eclipse to load additional classes.

my-macbook:demo$ jstat -gcutil 21742 1s
S0     S1     E      O      P     YGC     YGCT    FGC    FGCT     GCT
0.00   0.00   3.24  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.24  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.24  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.24  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.24  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.27  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.27  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.27  59.20 100.00     45    0.346   499  169.383  169.729
0.00   0.00   3.27  59.20 100.00     45    0.346   499  169.383  169.729

Next step – how much memory has the unlucky droid given to his Eclipse. Our friend jstat comes to help us out again here:

my-macbook:demo $ jstat -gccapacity 21742
NGCMN    NGCMX    ...  PGCMN    PGCMX     PGC       PC     YGC    FGC
16384.0  16384.0  ...  12288.0  65536.0  65536.0  65536.0     39   122

For some reason, the droid is running Eclipse with just 64MB available for permanent generation (PGCMX). And all of this available space is already being used (PGC). Now it becomes clear why the CPU is 100% stuck – JVM makes its last desperate efforts to free up some memory.

Now when searched for the configuration which should be located in eclipse.ini file we are left with bedazzling looks in our faces. Apparently its not where it is supposed to be:

my-macbook:juno$ pwd
/Applications/eclipse/juno
my-macbook:juno$ find . -name eclipse.ini
my-macbook:juno$

On whatever reason he had managed to download an Eclipse bundle without the eclipse.ini file in it. I guess it must require some skill. Or Eclipse has been distributed without the configuration. But at least we have found our culprit – without the settings.ini in place Eclipse fails back to JVM’s defaults for setting up heap and permgen sizes. Which are nowhere near to what this IDE needs for a normal run.

Fast forward three minutes and we have a working eclipse.ini copy in place and the IDE resumes it’s normal behaviour. So we can continue educating our droids.

Lessons learned. In order of importance:

  • Do not let marketroids near technical infrastructure. They mess up even with Eclipse installation. Which requires the exclusive knowledge about downloading and unzipping a ZIP archive.
  • Know thy tools. A bunch of them – when the first one fails on you, you need to have an alternative under your belt.

Enjoyed the post? We have got more coming, subscribe to our RSS or twitter feed and be notified on time!

Can't figure out what causes your OutOfMemoryError? Read more

ADD COMMENT

COMMENTS

Can't figure out what causes your OutOfMemoryError? Read more

Latest
Recommended
You cannot predict the way you die
When debugging a situation where systems are failing due to the lack of resources, you can no longer count on anything. Seemingly unrelated changes can trigger completely different messages and control flows within the JVM. Read more
Tuning GC - it does not have to be that hard
Solving GC pauses is a complex task. If you do not believe our words, check out the recent LinkedIn experience in garbage collection optimization. It is a complex and tedious task, so we are glad to report we have a whole lot simpler solution in mind Read more
Building a nirvana
We have invested a lot into our continuous integration / delivery infrastructure. As of now we can say that the Jenkins-orchestrated gang consisting of Ansible, Vagrant, Gradle, LiveRebel and TestNG is something an engineer can call a nirvana. Read more
Creative way to handle OutOfMemoryError
Wish to spend a day troubleshooting? Or make enemies among sysops? Registering pkill java to OutOfMemoryError events is one darn good way to achieve those goals. Read more