Heap problems with different amounts of memory
Java and heap allocation isn’t really an issue when developing programs with known hardware limitations. If it’s a server application you know that there is probably enough RAM and it’s safe to set big values for -Xms and -Xmx.
When client applications come to play anything is possible. We initially gave -Xmx 1024 Mb, at the start and this wasn’t a problem as most computers nowadays have more than 1Gb memory (well at least in the field the program is used). We thought that we were safe but we were wrong. What we found out was that the amount of RAM isn’t the only factor that can give troubles. Major problems come from different OSes. For example I could give 1Gb as max heap in my Ubuntu box and yet I have 3.2 Gb available with ~10% usage (4 Gb of RAM but not all is available due to addressing issues). On one Win XP box with 2Gb of memory setting max heap to 1 Gb gave the usual:
Error occurred during initialization of VM
Could not reserve enough space for object heap
Now this leads to a difficult problem. How to set the limit so that those OS-es/computers could run with high max heap size that have it available and those with low max heap size could also do it. There are 3 different options:
- Simple configuration file where the user tells how much max heap he wants to use. The problem is that most of our users don’t know how or want to configure this, so this option is out.
- Second option would be running a little script that would test the limits of the system, but it’s really unreliable.
- Search a way to make heap allocation dynamic.
After some digging I found from Sun Java forums a thread that solved this problem. The solution would be using -XX:+AggressiveHeap option. It allowed the program to use ~1.5 Gb of heap. I don’t know what the Windows machines result would be, as I haven’t received the results yet (I’ll update later when the results come in).
But this solution isn’t perfect either. From one part of JVM hotspot documentation you can find following:
This option should be used with caution. Because it makes the JVM greedy with memory resources, it is not appropriate for many programs and makes it easier to run out of memory. For example, by using most of the 4GB address space for heap, some programs will run out of space for stack memory. In such cases -Xss may sometimes help by reducing stack requirements.
At the moment we will use this option and see if it causes some of the problems it mentioned earlier. But what I’d like to see is some flexibility in allocating heap. For example why can’t there be a soft maximum where we just tell that JVM would try to allocate our given maximum amount of heap but when it’s not available to use whats available. And when the hard limit is used then give the initialization error.
As I have no prior experience with this aggressive heap I don’t know what to expect. If anyone has some good suggestions I’d like to hear.
4 commentsTPTP and Ubuntu
Installing TPTP under Eclipse isn’t as straightforward as other Eclipse plugins. Installing the plugin itself is easy, it can be done using Eclipse Update manager. But the getting the Agent Controller to work needs some tweaking.
The first part is really well documented under TPTP installation guide in http://www.eclipse.org/tptp/home/downloads/installguide/InstallGuide44.htm so I won’t cover that. Also most of the Agent Controllers installation instructions are covered in http://www.eclipse.org/tptp/home/downloads/installguide/agentcontroller_44/linux/getting_started.html. Unfortunately under Ubuntu you need a bit more than what those guides cover so I reuse some parts from there to maintain the readability.
Main part in installing the Agent Controller is easy, get the appropriate runtime package from http://www.eclipse.org/tptp/home/downloads/?ver=4.4.0#rac. Unpack it to your preferred location, considering that you will start running it from there.
First stumbling block is libstdc++-libc6.2-2.so.3 . There is no package called compat-libstdc++ in Ubuntu, instead you have to
sudo apt-get install libstdc++2.10-glibc2.2
Next step would be setting up the Agent Controller itself. The original guide says it all:
- Ensure that the files in the <install-dir>/bin and <install-dir>/lib directory are executable.
- Run SetConfig.sh script from a command shell in the <install-dir>/bin directory to generate the configuration file for the Agent Controller. The script requires that a Java Virtual Machine (JVM) be present in the PATH environment variable. Follow the prompts and enter the required parameters as appropriate. The configuration file <install-dir>/config/serviceconfig.xml will be generated based on your inputs.
If you want all users to have access to Agent Controller then you need to set following variables in bash.bashrc in /etc or if you are satisfied with that only your user can run Agent Controller then in ~/.bashrc
export TPTP_AC_HOME=/opt/agntctrl.linux_ia32-TPTP-4.4.0/
export JAVA_PROFILER_HOME=${TPTP_AC_HOME}/plugins/org.eclipse.tptp.javaprofiler
export LD_LIBRARY_PATH=${JAVA_PROFILER_HOME}:${TPTP_AC_HOME}/lib
PATH=”${PATH}”:${TPTP_AC_HOME}/bin
export TEMP=/tmp
For some reason I had to explicitly export TEMP as it wasn’t in environment variables. Of course you need TPTP_AC_HOME to point in the right direction.
And all thats left is re-login so your new environment variables would take effect.
To start profiling in eclipse you first have to start the Agent Controller by running ACStart.sh and if you are a heavy profiler then it might be a good idea to have it start automatically when ever the computer starts, but for others remembering to start it manually is all you need.
18 commentsHello World!
Hi! As you can see, this blog is really new. There is a lot of work still ahead so bare with me until I finish updating the looks and add some other widgets you can play with. After all major updates are done, I can start writing so no content before that :)
No comments