java.lang.OutOfMemoryError: Requested array size exceeds VM limit

Java has got a limit on the maximum array size your program can allocate. The exact limit is platform-specific but is generally somewhere between 1 and 2.1 billion elements.

outofmemoryerror

So when you face the java.lang.OutOfMemoryError: Requested array size exceeds VM limit, the application at hand is trying to allocate an array larger than your Java Virtual Machine can support.

Cause of java.lang.OutOfMemoryError

The error is thrown by the native code within the JVM. It happens before allocating memory for an array when JVM performs a platform-specific check: whether the allocated data structure is addressable in this platform. This error is less common than you might initially think.

The reason why you only seldom face this error is based on the fact that Java arrays are indexed by int. If you recall, the maximum positive int in Java is 2^31 – 1 = 2,147,483,647. And the platform-specific limits can be really close to this number – for example on my 64bit MB Pro on Java 1.7 I can happily initialize arrays with up to 2,147,483,645 or Integer.MAX_VALUE-2 elements.

Increasing the length of the array by one to Integer.MAX_VALUE-1 results in the familiar OutOfMemoryError being thrown:

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit

But the limit might not be that high – on 32-bit Linux with OpenJDK 6, you will hit the “java.lang.OutOfMemoryError: Requested array size exceeds VM limit” already when allocating an array with ~1.1 billion elements. To understand the limits of your specific environments run a small test to find out the limit.

Can’t figure out what causes your OutOfMemoryError?
Read more.

Example of java.lang.OutOfMemoryError

When trying to recreate the java.lang.OutOfMemoryError: Requested array size exceeds VM limit error, let’s look at the following code:


for (int i = 3; i >= 0; i--) {
	try {
		int[] arr = new int[Integer.MAX_VALUE-i];
		System.out.format("Successfully initialized an array with %,d elements.\n", Integer.MAX_VALUE-i);
	} catch (Throwable t) {
		t.printStackTrace();
	}
}

The example is iterating four times and is initializing an array of long primitives on each turn. The size of the array this program is trying to initialize grows by one and reaches the Integer.MAX_VALUE. Now, when launching the code snippet on 64-bit Mac OS X with Hotspot 7, you should get the output similar to the following:

java.lang.OutOfMemoryError: Java heap space
	at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)
java.lang.OutOfMemoryError: Java heap space
	at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at eu.plumbr.demo.ArraySize.main(ArraySize.java:8)

Note that before facing java.lang.OutOfMemoryError: Requested array size exceeds VM limit on the last two attempts, the allocations failed with a lot more familiar java.lang.OutOfMemoryError: Java heap space message. It happens because the 2^31-1 int primitives you are trying to make room for require 8G of memory which is less than the defaults used by the JVM.

This example also demonstrates the rarity of the error – in order to see the VM limit on array size being hit, you need to allocate an array with the size right in between the platform limit and Integer.MAX_INT. When our example is run on 64bit Mac OS X with Hotspot 7, there are only two such array lengths: Integer.MAX_INT-1 and Integer.MAX_INT.

Solution for java.lang.OutOfMemoryError

When facing the situation, there are two different situations you can experience:

  • You are facing the case where your arrays are growing to be in the range between the platform limit and the Integer.MAX_INT
  • You are deliberately trying to allocate arrays larger than 2^31-1 elements and are experimenting with the limits.

In the first case, check your code base to see whether you really need arrays that large. Maybe you could reduce the size of the arrays and be done with it. Or divide the array into smaller bulks and load the data you need to work with in batches fitting into your platform limit.

In the second case – remember that Java arrays are indexed by int. So you cannot go beyond 2^31-1 elements in your arrays when using the standard data structures within the platform. In fact, in this case you are already blocked by the compiler announcing “error: integer number too large” during compilation.

But if you really do work with truly large data sets, you need to rethink your options. You can either work in smaller bulk and load the data you need to work with in smaller batches and still use standard Java tools. Or you might go beyond the standard utities. One way to achieve this is to look into sun.misc.Unsafe class. This allows you to allocate memory directly like you would in C’s malloc.

Can’t figure out what causes your OutOfMemoryError?
Read more.