ObjectOutputSteam -reset() - OutOfMemory Issues

Have you ever faced heap memory issue with ObjectOutputSteam?  Then article would be very much useful to understand the internals and resolve the issue.

When you construct an ObjectOutputStream and an ObjectInputStream, they each contain a cache of objects that have already been sent across this stream. The cache relies on object identity, rather than the traditional hashing function. It is more similar to a java.util.IdentityHashMap than a normal java.util.HashMap. So, if you resend the same object, only a pointer to the object is sent across the network. This is very clever, and saves network bandwidth. However, the ObjectOutputStream cannot detect whether your object was changed internally or not, resulting in the Receiver just seeing the same object over and over again

In other way, If you send the different object every time, this will keep on storing in the cache handle and it will be keep growing, and at one point of time we might come across the heap memory issue since cache is huge and never cleared cache on the both the ends.

To avoid these kind of issues, we need to reset() the ObjectOutputStream, and this will clear the cache on the both ends. But, again this is a costly operation so we need to do it with some constrains. That will be mentioned below.

  ObjectOutputStream oos = new ObjectOutputStream(s.getOutputStream());
  oos.writeObject("my object data");
  oos.reset();

This will discard the state of any objects already written to the stream. The state is reset to be the same as a new ObjectOutputStream

Please go through the below java specialist article on this. Considering they might modify/delete from there site and so I have coped entire content here for my reference.

Article @ http://www.javaspecialists.eu/archive/Issue088.html

Same article content below.

Resetting ObjectOutputStream

A class with many mysteries is java.io.ObjectOutputStream. For instance, when and why should you reset the stream?

Let's look at an example. First we have class Person, which is the class that we want to send over the network:

public class Person implements java.io.Serializable {
  private final String firstName;
  private final String surname;
  private int age;

  public Person(String firstName, String surname, int age) {
    this.firstName = firstName;
    this.surname = surname;
    this.age = age;
  }

  public String toString() {
    return firstName + " " + surname + ", " + age;
  }

  public void setAge(int age) {
    this.age = age;
  }
}
  

Next we have the code that Receives lots of Person objects and code that Sends them:

import java.net.*;
import java.io.*;

public class Receiver {
  public static void main(String[] args) throws Exception {
    ServerSocket ss = new ServerSocket(7000);
    Socket socket = ss.accept();
    ObjectInputStream ois = new ObjectInputStream(
        socket.getInputStream());
    int count=0;
    while(true) {
      Person p = (Person) ois.readObject();
      if (count++ % 1000 == 0) {
        System.out.println(p);
      }
    }
  }
}


import java.net.Socket;
import java.io.*;

public class Sender {
  public static void main(String[] args) throws IOException {
    long start = System.currentTimeMillis();
    Socket s = new Socket("localhost", 7000);
    ObjectOutputStream oos = new ObjectOutputStream(
        s.getOutputStream());
    Person p = new Person("Heinz", "Kabutz", 0);
    for (int age=0; age < 1500 * 1000; age++) {
      p.setAge(age);
      oos.writeObject(p);
    }
    long end = System.currentTimeMillis();
    System.out.println("That took " + (end-start) + "ms");
  }
}

The output was:

java Receiver:
  *snip*
  Heinz Kabutz, 0
  Heinz Kabutz, 0
  Heinz Kabutz, 0
  Heinz Kabutz, 0
  Heinz Kabutz, 0
  Heinz Kabutz, 0

java Sender:
  That took 19548ms
  

When we run this, we will see lots of People objects on the Receiver side, but all the age values will be 0, even though we changed the age on the Sender side. Why is this?

When you construct an ObjectOutputStream and an ObjectInputStream, they each contain a cache of objects that have already been sent across this stream. The cache relies on object identity, rather than the traditional hashing function. It is more similar to a java.util.IdentityHashMap than a normal java.util.HashMap. So, if you resend the same object, only a pointer to the object is sent across the network. This is very clever, and saves network bandwidth. However, the ObjectOutputStream cannot detect whether your object was changed internally, resulting in the Receiver just seeing the same object over and over again. You will notice that this was quite fast. We sent 1'500'000 objects in 19548ms (on my machine). (well, we only sent one object, and 1'499'999 pointers to that object).

There seemed to be some problem with sending the same Person object many times, especially if the contents of that Person changed. Due to the optimisation in ObjectOutputStream, only the pointer to the Person would be sent each time. So, what would happen if we simply sent a new Person each time? Let's try it out...

import java.net.Socket;
import java.io.*;

public class Sender2 {
  public static void main(String[] args) throws IOException {
    long start = System.currentTimeMillis();
    Socket s = new Socket("localhost", 7000);
    ObjectOutputStream oos = new ObjectOutputStream(
        s.getOutputStream());
    for (int age=0; age < 1500 * 1000; age++) {
      oos.writeObject(new Person("Heinz", "Kabutz", age));
    }
    long end = System.currentTimeMillis();
    System.out.println("That took " + (end-start) + "ms");
  }
}

This seems to run fine for a while, until we all of a sudden see an OutOfMemory error on both the Receiver and the Sender2. Someone once challenged regarding the pathetic speed of Java. They claimed that Java was so slow that the Garbage Collector could not even keep up with objects that were being read over the network. It sounded strange to me that Java should run out of memory so after some questioning, we traced the problem to the object cache growing in the Receiver and never being cleared. Since the Person objects are always distinct, they are put into the cache on both sides of the ObjectOutputStream. The Receiver's side cannot clear entries from the table, since it does not know which entries the Sender might send again. It then keeps on growing until the JVM runs out of memory.

Resetting ObjectOutputStream

One hack^H^H^H^Hsolution to the OutOfMemory problem is to every time that you send an object also reset the cache on both sides. Let's try out what that does to our performance:

import java.net.Socket;
import java.io.*;

public class Sender3 {
  public static void main(String[] args) throws IOException {
    long start = System.currentTimeMillis();
    Socket s = new Socket("localhost", 7000);
    ObjectOutputStream oos = new ObjectOutputStream(
        s.getOutputStream());
    for (int age=0; age < 1500 * 1000; age++) {
      oos.writeObject(new Person("Heinz", "Kabutz", age));
      oos.reset();
    }
    long end = System.currentTimeMillis();
    System.out.println("That took " + (end-start) + "ms");
  }
}

When I ran that, it worked without causing any OutOfMemory Errors, so I should be happy. But am I happy? I am old, after having to wait for 314242ms for it to complete, i.e. 16 times longer than with Sender. Sender was fast, but incorrect. Sender2 ran out of memory. Sender3 was correct, but slow. Is there no better way?

The problem with reset() is that it clears the cache of ALL objects, even constants such as the Strings "Heinz" and "Kabutz". So, we end up sending these constants over the network time and time again! Unfortunately the reset() is an all-or-nothing approach, so the entire cache will be lost. But perhaps, if we don't clear it all the time, we can get the advantage of speed and correctness? Let's try that out:

import java.net.Socket;
import java.io.*;

public class Sender4 {
  public static void main(String[] args) throws IOException {
    long start = System.currentTimeMillis();
    Socket s = new Socket("localhost", 7000);
    ObjectOutputStream oos = new ObjectOutputStream(
        s.getOutputStream());
    for (int age=0; age < 1500 * 1000; age++) {
      oos.writeObject(new Person("Heinz", "Kabutz", age));
      if (age % 1000 == 0) oos.reset();
    }
    long end = System.currentTimeMillis();
    System.out.println("That took " + (end-start) + "ms");
  }
}

Because I don't reset the cache on every call, Sender4 can avoid sending the Strings "Heinz" and "Kabutz" over the network 1'500'000 times in just 66015ms. Infact, it only has to send these Strings 1'500 times. If we reset the ObjectOutputStream too frequently, we will increase the network bandwidth, and if we do not reset it often enough, we will increase the burden of our Garbage Collector. Like all things in Java Performance Tuning, you have to set it to the correct number, not too big and not too little.

Requesting Garbage Collector(GC) to free up the memory

This would be useful, if you are working on memory critical applications and your application/process want to call GC explicitly to acquire some free memory based on the need.


public class MemoryTest {

public static void callGC() {
long before = Runtime.getRuntime().freeMemory();

/* Let the finilizer finish its work and remove objects from its queue */
System.gc(); /* asyncronous garbage collector might already run */
System.gc(); /* to make sure it does a full gc call it twice */
System.runFinalization(); //Runs the finalization methods of any objects pending finalization. 

try {
Thread.sleep(100);
} catch (InterruptedException e) {
// do nothing
}

long after = Runtime.getRuntime().freeMemory();
System.out.print("Total Memory:: ");
System.out.println(String.valueOf(Runtime.getRuntime().totalMemory()));
System.out.print("Free Memory Before GC:: ");
System.out.println(String.valueOf(before));
System.out.print("Free Memory After GC:: ");
System.out.println(String.valueOf(after));
System.out.print("Memory gained after GC:: ");
System.out.println(String.valueOf(after - before));
}

public static void main(String[] args) {
callGC();
}
}

Write heap dump on OutOfMemoryError in Eclipse

Add below parameter to vm arguments of JVM.

-XX:+HeapDumpOnOutOfMemoryError

This writes heap dump on OutOfMemoryError to your eclipse directory.

Say for example, if your working eclipse directly is "D:\Work\eclipse", there can find hprof extension file. It looks like below.

ex: java_pid1864.hprof

1864 represents running javaw process id.

It would be useful to analyze the cause using using memory profiler tools ex: Eclipse MAT, JProfiler and YourKit 

Eclipse dialog/wizard size issues in extended monitor

If you have a use case, where you want to compute dialog/wizard size based upon the system client area(system resolution),then the following is the general approach.

Rectangle clientArea = Display.getDefault().getClientArea();

int x_hint = (int) (0.7 * clientArea.width); //% based on the required size

int y_hint = (int) (0.7 * clientArea.height));//% based on the required size

GridDataFactory.fillDefaults().grab(true, true).hint(x_hint, y_hint).applyTo(this);

The problem comes with above approach is, if your primary monitor is connected with another monitor in extended mode display, then the dialog/wizard may stretch to the extended monitor also, considering system calculates the both monitors width as a display area.

To avoid this kind of issues, you can get primary monitor display area and compute based on that.

Monitor primaryMonitor = Display.getDefault().getPrimaryMonitor();

Rectangle clientArea = primaryMonitor.getClientArea(); //Now compute based on this.

Heap Memory Analysis # SHALLOW HEAP

HOW MUCH MEMORY DO I NEED  – WHAT IS SHALLOW HEAP?

In the first post we pushed a whole lot of complexity away by stating that calculating shallow heap size is easy – it consists of only the heap occupied by the object itself. But how do you calculate how much memory does the object “itself” require?  Apparently there is a formula for it:

Shallow Heap Size = [reference to the class definition] + space for superclass fields + space for instance fields + [alignment]

Does not seem too helpful, eh? Let’s try to apply the formula using the following sample code:

class X {
   int a;
   byte b;
   java.lang.Integer c = new java.lang.Integer();
}

class Y extends X {
   java.util.List d;
   java.util.Date e;
}

Now, the question we strive to answer is – how much shallow heap size does an instance of a Y require? Lets start calculating it, assuming that we are on a 32-bit x86 architecture:

As a starting point – Y is a subclass of X, so its size includes “something” from the superclass. Thus, before calculating the size of Y, we look into calculating the shallow size of X.

Jumping into the calculations on X, first 8 bytes are used to refer its class definition. This reference is always present in all Java objects and is used by JVM to define the memory layout of the following state. It also has three instance variables – an int, an Integer and a byte. Those instance variables require heap as follows:

• a byte is what it is supposed to be. 1 byte in a memory.
• an int in our 32bit architecture requires 4 bytes.
• a reference to the Integer requires also 4 bytes. Note that when calculating retained heap, we should also take into account the size of a primitive wrapped into the Integer object, but as we are calculating shallow heap here, we only use the reference size of 4 bytes in our calculations.

So – is that it? Shallow heap of X = 8 bytes from reference to the class definition + 1 byte (the byte) + 4 bytes (the int) + 4 bytes (reference to the Integer) = 17 bytes? In fact – no. What now comes into play is called alignment (also called padding). It means that the JVM allocates the memory in multiples of 8 bytes, so instead of 17 bytes we would allocate 24 bytes if we would create an instance of X.

If you could follow us until here, good, but now we try to get things even more complex. We are NOT creating an instance of X, but an instance of Y. What this means is – we can deduct the 8 bytes from the reference to the class definition and the alignment. It might not be too obvious at first place but – did you note that while calculating the shallow size of X we did not take into account that it also extends java.lang.Object as all classes do even if you do not explicitly state it in your source code? We do not have to take into account the header sizes of superclasses, because JVM is smart enough to check it from the class definitions itself, instead of having to copy it into the object headers all the time.

The same goes for alignment – when creating an object you only align once, not at the boundaries of superclass/subclass definitions. So we are safe to say that when creating a subclass to X you will only inherit 9 bytes from the instance variables.

Finally we can jump to the initial task and start calculating the size of Y. As we saw, we have already lost 9 bytes to the superclass fields. Let’s see what will be added when we actually construct an instance of Y.

• Y’s headers referring to its class definition consume 8 bytes. The same as with previous ones.
• The Date is a reference to an object. 4 bytes. Easy.
• The List is a reference to a collection. Again 4 bytes. Trivial.

So in addition to the 9 bytes from the superclass we have 8 bytes from the header, 2×4 bytes from the two references (the List and the Date). The total shallow size for the instance of Y would be 25 bytes, which get aligned to 32.

While writing this article, the following online resources were used for inspiration:

• http://memoryanalyzer.blogspot.com/2010/02/heap-dump-analysis-with-memory-analyzer.html
• http://www.javamex.com/tutorials/memory/object_memory_usage.shtml
• http://www.javamex.com/tutorials/memory/instrumentation.shtml
• http://kohlerm.blogspot.com/2008/12/how-much-memory-is-used-by-my-java.html
• http://www.javaspecialists.eu/archive/Issue142.html

- See more at: http://plumbr.eu/blog/how-much-memory-do-i-need-part-2-what-is-shallow-heap#sthash.LThS3oJX.dpuf

PS: I have added this topic here for my reference, you can find many more interesting articles by Nikita Salnikov-Tarnovski @ plumber.eu

Heap Memory Analysis # RETAINED HEAP

HOW MUCH MEMORY DO I NEED  – WHAT IS RETAINED HEAP?

How much memory will I need? This is a question you might have asked yourself (or others) when building a solution, creating a data structure or choosing an algorithm. Will this graph of mine fit in my 3G heap if it contains 1,000,000 edges and I use a HashMap to store it? Can I use the standard Collections API while building my custom caching solution.

Apparently, the answer to the simple question is a bit more complex. In this post we’ll take a first peek at it and see how deep the rabbit hole actually is.

The answer to the question in the headline comes in several parts. At first we need to understand whether you are interested in shallow or retained heap sizes.

The shallow heap is easy – it consists of only the heap occupied by the object itself. There are some nuances to how to calculate it, but for the scope of this article we leave it as is. Stay tuned for future posts on the same topic.

The retained heap is in many ways more interesting. Only rarely are you interested in the shallow heap, in most cases your actual question can be translated to “If I remove this object from the memory, how much memory can now be freed by the garbage collector”.

Now, as we all remember, all Java garbage collection (GC) algorithms follow this logic:

1. There are some objects which are considered “important” by the GC. These are called GC roots and are (almost) never discarded. They are, for example, currently executing method’s local variables and input parameters, application threads, references from native code and similar “global” objects.
2. Any objects referenced from those GC roots are assumed to be in use and hence not discarded by the GC. One object can reference another in different ways in Java, in the most common case an object A is stored in a field of an object B. In such case we say  “B references A”.
3. The process is repeated until all objects that can be transitively reached from GC roots are visited and marked as “in use”.
4. Everything else is unused and can be thrown away.

Now to illustrate how to calculate the retained heap, let’s follow the aforementioned algorithm with the following example objects:

 

To simplify the sample, let’s estimate that all the objects O1-O4 have the shallow heap of 1024B = 1kB. Lets start calculating the retained sizes of those objects.

• O4 has no references to other objects, so its retained size is equal to its shallow size of 1kB.
• O3 has a reference to O4. Garbage collecting O3 would thus mean O4 would also be eligible for garbage collection and so we can say that O3 retained heap is 2kB.
• O2 has a reference to O3. But it is now important to note that removing the pointer from O2 to O3 does not make O3 eligible for GC, as O1 still has got a pointer to it. So O2 retained heap is only 1kB.
• O1 on the other hand is the object keeping all the references in this small graph, so if we would remove O1, everything on this graph would be garbage collected. So O1 retained heap is 4kB.

Which implications does this have in practice? In fact, understanding the differences between shallow and retained heap sizes makes it possible to work with tools such as memory profilers and heap dump analyzers – for example digging into Eclipse MAT might prove to be impossible if you don’t know how to distinguish these two types of heap size measurements.

Full disclosure: this post was inspired by Patrick Dubroy’s talk on Google I/O, which you can watch in full length here.

Resources at: http://plumbr.eu/blog/how-much-memory-what-is-retained-heap#sthash.e5Swh4xR.dpuf

PS: I have added this article here for my reference, you can find many more interesting articles by Nikita Salnikov-Tarnovski @ plumber.eu

Heap Memory Analysis # Shallow heap vs Retained Heap

Shallow heap
The amount of memory that is consumed by one object. An object requires different amounts of memory depending on the operating system architecture. For example, 32 bits or 64 bits for a reference, 4 bytes for an integer, or 8 bytes for an object of type "Long". Depending on the heap dump format, the size might be adjusted to provide a more realistic consumption of the JVM.

Retained Set
One or more objects plus any objects that are referenced, directly or indirectly, only from those original objects. The retained set is the set of objects that would be removed by garbage collection when an object, or multiple objects, is garbage collected.

Retained set of X is the set of objects which would be removed by GC when X is garbage collected.

Retained heap of X is the sum of shallow sizes of all objects in the retained set of X, i.e. memory kept alive by X.

Retained heap, or retained size
The total heap size of all the objects in the retained set. This value is the amount of memory that is consumed by all the objects that are kept alive by the objects at the root of the retained set

Generally speaking, shallow heap of an object is its size in the heap and retained size of the same object is the amount of heap memory that will be freed when the object is garbage collected.

The retained set for a leading set of objects, such as all objects of a particular class or all objects of all classes loaded by a particular class loader or simply a bunch of arbitrary objects, is the set of objects that is released if all objects of that leading set become unaccessible. The retained set includes these objects as well as all other objects only accessible through these objects. The retained size is the total heap size of all objects contained in the retained set.






Example, if you remove C, then C, D, E, F, G, H are eligible for garbage collection.

GIT: Unlink of file 'com.x.xx/a1.jar' failed. Should I try again? (y/n) y during the pull

Unlink of file 'com.x.xx/MINIFY.jar' failed. Should I try again? (y/n) y

I was getting this issue, while pulling the code from the server using below command.

>git pull origin Dev

What I observed was, I kept it opened "build.properties' file, where all my dependent build jars are specified of a plug-in and that was causing the issue.

This could mean that another program is using the file, which is preventing git to sync the repository changes with the local changes. Sometimes the file may not be actually open in eclipse but may have been opened by a process run by eclipse. In this event, try closing the file in any applications that might have used it. If that doesn't work, completely exit any applications which may have opened the file.