Prefuse Out of Memory Error

In my recent coding with Prefuse, I got a severe OutOfMemoryException, which almost jeopardy my dissertation project. Here is the case I got.

I have a visualization that renders ~50MB table data in X-Y axis layout. I need to change the table data when users choose another data file. But the problem is that, even I used

m_vis.removeGroup("groupname");

m_vis.addTable("groupname", table);

m_vis.run("actionname1");

m_vis.run("actionname1"); ....

Or I create a new Visualization and Display, remove the previous Display from the common container, like a JPanel, and add this newly created Display.

These two method both work fine in terms of creating a nice image. BUT!!!!

After rendering a few data files, it has a out of memory problem. I watch the memory usage info. I found that the previous visualization and data are not cleaned by the garbage collection function. So each image and data are still in memory, and soon it claimed all memory reserved by JVM.

After digging into other threads in the Sorucefore' Prefuse forum, I think I find the answer which partially (90%) solve my problem. Here is the thread that discuss the nearly same problem.

https://sourceforge.net/projects/prefuse/forums/forum/343013/topic/1809861

The only difference is that they used graph as base data, while I am using Table as base data. The reason to cause the out of memory problem is that Display will NOT automatically destroy its buffered off-screen image and activities. Also unused data will NOT automatically lost its reference to the TupleSet and registered listeners. Since these data still has valid references, JVM cannot collect the memory they took.

In the beta version, Heer fix the above problems by adding a new method in Display, called reset(), which will set m_offscreen to NULL, and clean queue, and by adding a removeAllGraphListener() in Graph and removeAllTableListener() in Table to clean up all registered listeners to base data structure. So here is the solution to reclaim memory unused by a Display. You need to create a public method to do the cleanup, e.g. a method like

public void cleanup()

Then do the following in this method.

1. Display.setVisualization(null)

2. Display.reset();

For the base data used in Visualization in this Display. Do these.

1. Table( or Graph).removeAllTable(or Graph)Listener();

2. Table( or Graph).clear(); After call this cleanup method, do garbage collection, e.g. Runtime.getRuntime.gc() or System.gc(); The majority of unused memory will be relocated.

In my case, there is still a few un-release memory, ~10MB. That is why I said, it is partially (90%) solved.

The largest dimension of a prefuse Display

It is an interesting finding to answer the question in the title.

Today, I try put my software to its edge. I create a Display with W=1024 and H=180,000 pixels. Then an OutOfMemeoryException was thrown.

At first, I thought the data is too big to fit in the main memory. So I checked the memory usage, which showed that I didn't reach the maximum memory, and only 60% of the max memory was used. Then I tried to google the origins of the OutOfMemoryException. Some similar links popped up. It turns out to be an issue about the largest size of a BufferedImage that a JVM can located. Here is my findings.

• In Prefuse, Display is the "canvas" allowing a Visualization to paint on. Its source codes show that, a Display uses a BufferedImage to create the Graphics2D for Visualizations.
• So how large can a BufferedImage be? A BufferedImage is a combination of a 2D int array, which indicate the X-Y coordination, and a 1D of colors, which indicate the color in each X-Y pixels. Therefore,

the size of a BufferedImage = int[][] * a color

• Theoretically, the largest size of an int[][] = the largest size of an int[], which Max2D array = Max1D array = 2^32B in JAVA. An integer will take 32 bits = 4 Bytes.
• A color in Java2D is a 32 bits number, with each Byte represent 0-255, for RGB color and a an alpha value for transparency. So a color takes 32 bits = 4Bytes;

OK, now it is clear, the Max size (Dimension) of a BufferedImage (2^32 Bytes/4)/4Bytes = 2^27 for W*H. If we have width as 1024, the max height = 2^17 = 131072.

Notices: the Max2D array could be limited by the VM in your machine and the max memory you located for your application. So the final max dimension of a display could vary from one machine to another machine.

Working process of actions and renderers

Recently a prefuse user want to do some custom work. One of the tasks is to decide which items need to be shown. So I checked the TreeView demo writen by Heer.

There is a action, called fisheyefilter, which determines how many layers a tree structure will show from the node whose DOI is 0. Then I checked its source codes and finally figure out what the functions of the columns (_visible, _visible:start, _visible:end) in a VisualItem are. Here are some findings.

FishEyeTreeFilter class is one of the actions, determining the value of “_visible; _visible: start; _visible: end”

Renderers will check the _visible value (true or false) of a VisualItem to decide if they need to paint this VisualItem.

If need to paint it, the VisualItem will need to find actions that quantify its position, shape, colors, and orientation.

Renderers normally has to implement the render() method. And render() method will usually call GraphicsLib.paint() method to finally paint the item.

Summary

1. If you want to customize your visualization, work closely with actions and renders.
2. Override render() method carefully, and alway try to use GraphicsLib.paint() method if this can work for you.
3. If the GraphicsLib.paint() method cannot work, create your own paint() method and use it in the render() method.

Relation between actions and renderers

It is unclear what the relation between actions and renderers.

According to Heer's explanations, actions are supposed to assign visual values to a group of VisualItems, such as colors, shapes, positions, sizes, and etc. BUT the final appearance of an item should be rendered by renderers some of which will decide sizes, and shapes.

This relation could work fine if the position or size of an VisualItem have been pre-set. However, some problem will occur, e.g. the size of an item is change according to the rendering area, or the position of an item were decided by other items. In these cases, an action should tell its renderer an absolute value of size. The duty of renderers shift to actions.

Find a VisualItem directly

In order to find a VisualItem in prefuse Visualization, there are two ways.

#1 The general way is to get an Iterator by calling Visualization class' item(), getGroup(), or other methods, which will return an Iterator. Then in a while loop, query this Iterator one by one and search for the VisualItem with certain contents.

Iterator i= v.items(SOURCE);

VisualItem vitem, v1=null, v2=null;

while(i.hasNext()){

vitem=(VisualItem) i.next();

if (vitem instanceof VisualItem){

if (vitem.getInt("ID")==s){

v1=vitem;

}

if (vitem.getInt("ID")==t){

v2=vitem;

}

}

}

#2 A specific way is to cast the TupleSet into a VisualTable if your back up data is a Table. So in this way, the VisualItem can be directly queried by using the row number.

VisualTable ts=(VisualTable) v.getGroup(SOURCE);

VisualItem v1=ts.getItem(s-1);

VisualItem v2=ts.getItem(t-1);

The speed of #2 is much fast than #1. Using this solution, it is easy to handle ~10000 nodes and lines.

But #1 is safe, since if your backup data is not a Table, there may be an exception of casting TupleSet into VisualTable.

If your data nodes is small, ~1000, #1 solution would be safe to use. If you want speed, use #2 with careful data design.

Design your own actions and renderers

Prefuse offers a set of actions and renderers to create nice and neat visualization applications.

Based on three weeks digging into the source codes of prefuse and trail-and-fail in a tabluer data demo, my understanding of part of the Actions and Renderers are related blow:

The basic logic is to use actions to assign visual features, such as colors, shapes, sizes, and position. And use renderers to finally paint these visual units with above features.

In terms of colorAction, dataColorAction, the key component is the "process" method, which assign the visual features to a VisualItem.

In shapeRenderer class, the key component is the "getShape" method and "render" method. Particularly, the "render" method that initialize the final painting of a VisualItem. In general "render" method will call PrefuseLib's "paint" method to do the painting.

Therefore, if the prefuse's default actions cannot help your customized application, it is necessary to write customized actions and renders. Here are one example for my work.

Design idea: draw a visualitem with a sequence of color-coded rectangles whose width is also customized. See the image.

In prefuse's default setting, one visualitem only has one color and one shape assigned with it (seen in the previous post). To my best knowledge, default color and shape actions cannot fulfill my design idea because my design require one visualitem to have multiple colors and shapes.

In order to achieve my design idea, I have to add a column into the data table, named length, which will store an integer array for lengths of these rectangles. Also to give the rectangles different colors, I need to add another column, named colors, which is used to store an integer array assigned to each rectangle.

The default colorAction and shapeAction process each VisualItem with its colors and shape. Therefore I need to write my own coloraction and renderer to deal with the arrays.

For my own coloraction, I create a class, named CompositedColorAction, which extends EncoderAction. And override its

public void process(VisualItem vi, double frac)

method. Here are the method codes:

@Override

public void process(VisualItem vi, double d) {

List l=getTypeList(vi);

ArrayList colors=new ArrayList();

for (int i=0;i

if (l.get(i) instanceof Integer){

colors.add(m_palette[(Integer) l.get(i)]);

}else throw new IllegalArgumentException(

"Unrecognized data type");

}

vi.set(m_colorListField, colors);

}

In these codes add an array of colors to the column "color" in a visual item for render to use.

To render multiple rectangles, I write a CompositedShapeRenderer class, which extends ShapeRender, and override its

public void Render(Graphics2D g, VisualItem vi)

method.

Here are the source codes.

@Override

public void render(Graphics2D g, VisualItem vi){

Shape[] shapes=getRawShapes(vi);

int[] colors=getRawColors(vi);

if (shapes!=null){

GraphicsLibPlus.compositedPaint(g, vi, shapes, colors, vi.getStroke(), getRenderType(vi));

}

}

Here the getRawShapes() and getRawColors() method is as below:

protected Shape[] getRawShapes(VisualItem vi) {

// a temp shape array to store all rectganles

Shape[] tempShapes=null;

// create shapes

double x=vi.getX();

double y=vi.getY();

if ( Double.isNaN(x) || Double.isInfinite(x) )

x = 0;

if ( Double.isNaN(y) || Double.isInfinite(y) )

y = 0;

ArrayList sizes=(ArrayList) vi.get(m_type);

tempShapes=new Shape[sizes.size()];

for (int i=0;i

m_shape=new RoundRectangle2D.Double();

m_shape.setRoundRect(x, y, (Integer)sizes.get(i)*m_baseSize,

m_baseSize, m_arcWidth, m_arcHeight);

tempShapes[i]=m_shape;

x=x+(Integer)sizes.get(i)*m_baseSize;

}

return tempShapes;

}

protected int[] getRawColors(VisualItem vi){

int[] tempColors=null;

ArrayList colors=(ArrayList) vi.get(m_colors);

tempColors=new int[colors.size()];

for (int i=0;i

tempColors[i]=(Integer) colors.get(i);

}

return tempColors;

} //end of getRawColors method

The two methods read the colors and widths from the visual item's "types" and "colors" columns and ask the paint method to render the rectangles with different colors.

Here the render method use a new painting method, called

GraphicsLibPlus.compositedpait.nnt()

I rewrite this method from prefuse' original painting method, called GraphicsLib.paint(). Because the original method only paint one shape, I have create a new painting method.

Summary:

In order to design your own visualization, you need to do a lot on actions and renderers. Usually you specify some new columns in a data tuple (a data row), and use actions to change values of these columns, and ask renderers to paint this visual item by looking at these rows and paint accordingly. If the original GraphicsLib.paint() method does not work, you need to write a new class and new paint() method for your purposes.

Welcome to add comments and ask questions from my site (see upper left conner of this page).

The structure of a VisulItem

VisulItem is the basic unit for prefuse' renderers to deal with. Also it extends a row in a data table, adding many columns into the row with many features associated with visual representation.

For a simple data table like this: Test Table

protected final Schema tableschema=new Schema();

{

tableschema.addColumn(ID, int.class);

tableschema.addColumn(SIMscore, float.class);

tableschema.addColumn(LINES, float.class);

}

The table schema is same as above.

Schema[(ID, int, -1) (score, float, 0.0) (lines, float, 0.0)]

When this table is associated with a Visualization, a VisualTable is created and the schema of this table is:

Schema[(_validated, boolean, false) (_visible, boolean, true) (_visible:start, boolean, false) (_visible:end, boolean, true) (_interactive, boolean, true) (_expanded, boolean, true) (_fixed, boolean, false) (_highlight, boolean, false) (_hover, boolean, false) (_x, double, 0.0) (_x:start, double, 0.0) (_x:end, double, 0.0) (_y, double, 0.0) (_y:start, double, 0.0) (_y:end, double, 0.0) (_bounds, java.awt.geom.Rectangle2D, java.awt.geom.Rectangle2D$Double[x=0.0,y=0.0,w=0.0,h=0.0]) (_strokeColor, int, 0) (_strokeColor:start, int, 0) (_strokeColor:end, int, 0) (_fillColor, int, 0) (_fillColor:start, int, 0) (_fillColor:end, int, 0) (_textColor, int, 0) (_textColor:start, int, 0) (_textColor:end, int, 0) (_size, double, 1.0) (_size:start, double, 1.0) (_size:end, double, 1.0) (_shape, int, 0) (_stroke, java.awt.Stroke, java.awt.BasicStroke@d1a0003e) (_font, java.awt.Font, java.awt.Font[family=SansSerif,name=SansSerif,style=plain,size=10]) (_font:start, java.awt.Font, java.awt.Font[family=SansSerif,name=SansSerif,style=plain,size=10]) (_font:end, java.awt.Font, java.awt.Font[family=SansSerif,name=SansSerif,style=plain,size=10]) (_doi, double, 4.9E-324) (ID, int, -1) (score, float, 0.0) (lines, float, 0.0)]

See the image for clarity.

So now the structure of a VisualItem is clear. Besides the basic three data columns, a set of features are added in, which helps to set up the size, position, color, shape, bound, and etc. And these features can be changed when various EncodingActions are called. For example, ColorAction assigns the color of this item, which is the ROW!!!!!!! (remember? each visualitem represents a row in the table. So one row, one item, and one visual unit in a Visualization). ShapeAction assigns the shape of an item.

Each visual item has its position information, normally assigned by Layout actions, such as AxisLayout, and CircleLayout.

Eventually various Renderers will paint a visual item based on its features that stored in its corresponding columns.

TMHK, Renderers will use PrefuseLib.paint(graphic g, visualitem item) method to do the painting.

In next post, I tried to explain how the Actions and Renderers work, as these two classes need to collaborate to paint a visual item in the right place, with right color and shape.