How to allocate an off-heap byte buffer in a portable way?

Issue

Is there a way to allocate a simple off-heap byte buffer while targeting both jdk-8 and jdk-11 (i.e it's impossible to use Unsafe or VarHandle?

Current solution relies on RandomAccessFile, but the goal is to use only direct memory.

I'm trying ByteBuffer.allocateDirect(), but it looks like it's still limited by the heap size.

 * $JDK_16/bin/jshell -R-Xmx32m 
jshell> java.nio.ByteBuffer.allocateDirect(33 * 1024 * 1024)
|  Exception java.lang.OutOfMemoryError: Cannot reserve 34603008 bytes of direct buffer memory (allocated: 141, limit: 33554432)
|        at Bits.reserveMemory (Bits.java:178)
|        at DirectByteBuffer.<init> (DirectByteBuffer.java:121)
|        at ByteBuffer.allocateDirect (ByteBuffer.java:330)
|        at (#1:1)

Solution

How to allocate an off-heap byte buffer in a portable way?

Use ByteBuffer.allocateDirect(capacity); see the javadoc for details. (Also read the section at the top that explains the difference between normal and direct buffers and lists some of the caveats of using direct buffers.)

I'm trying ByteBuffer.allocateDirect(), but it looks like it's still limited by the heap size.

It is not limited by the heap size (-Xmx). It is actually limited by:

• an explicit -XX:MaxDirectMemorySize=size option or the default MaxDirectMemorySize value, and
• how much memory the operating system is prepared to allocate to the JVM when it asks for it.

The java manual page says this about the above option:

"By default, the size is set to 0, meaning that the JVM chooses the size for NIO direct-buffer allocations automatically."

How it makes that choice is liable to be Java version and OS specific, and may possibly take account of the dimensions of your hardware (or virtual machine) platform.

Answered By - Stephen C