RooFitExplorer

Posted on October 13, 2012. Filed under: Uncategorized | Tags: , , , , , , |

Here is a project which I started to more easily navigate RooFit workspaces:

https://bitbucket.org/andreh/roofitexplorer/wiki/Home

This is a ‘interactive, graphical viewer for RooFit workspaces’. It still in very alpha stage but maybe useful to others (use at your own risk). Constructive feedback and test cases are welcome !

Here is a screenshot:

Read Full Post | Make a Comment ( None so far )

Debugging unresponsive Java GUI applications

Posted on June 25, 2011. Filed under: Uncategorized | Tags: , , , , , , , , |

You probably heard about it: one should not run time-consuming operations on the event dispatch thread in Java GUI applications but use SwingWorker instead to run lengthy tasks, possibly showing a dialog box once the task is completed. One typical case of running tasks on the event dispatch thread is in a callback to the user’s action such as pressing a button or selecting a menu item from a pop-up menu.

I’m working on an application which runs external commands very often and the running time of such commands can be several minutes. Wherever possible, commands are run only on demand and the states of the objects in the application are updated according to the command’s output. While care was taken to use SwingWorker in obvious places the application still became unresponsive (i.e. the windows were not repainted for minutes) because the event dispatching thread was waiting to lock an object which was locked by another thread running a lengthy task (an external command in this case). In the following I describe how I found such locks with the standard Java debugger.

Starting the program and the debugger

The application I was working on was packaged into a single .jar file. So instead of starting the program with

java -jar myapp.jar

I started the Java virtual machine and my application and made them listen to connections from a debugger:

java -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=localhost:10101 -jar myapp.jar

This instructs the Java virtual machine to bind to socket 10101 on localhost to listen to a connection from jdb. The suspend=n part instructs it to start program execution immediately.

WARNING: as far as I understood is there no authentication involved when connecting with jdb (at least not when running it as described above). In other words: all users having access to this machine can also connect to the process you are debugging and control it !

On a second console, I started the Java debugger:

jdb -attach localhost:10101

The command help on the debugger console will give a short list of all commands understood by jdb.

Finding the locks

I waited until the GUI froze. At that point, I did:

suspend

to suspend all threads. I got a quick overview of the state of all threads by using the command threads:

Group system:
  (java.lang.ref.Reference$ReferenceHandler)0x832 Reference Handler           cond. waiting
  (java.lang.ref.Finalizer$FinalizerThread)0x831  Finalizer                   cond. waiting
  (java.lang.Thread)0x830                         Signal Dispatcher           running
  (java.lang.Thread)0x82f                         Java2D Disposer             cond. waiting
  (java.lang.Thread)0x901                         TimerQueue                  cond. waiting
Group main:
  (java.lang.Thread)0x82e                         AWT-XAWT                    running
  (java.lang.Thread)0x82d                         AWT-Shutdown                cond. waiting
  (java.awt.EventDispatchThread)0x82c             AWT-EventQueue-0            waiting in a monitor
  (java.lang.Thread)0x82b                         pool-2-thread-1             cond. waiting
  (java.lang.Thread)0x82a                         pool-1-thread-1             cond. waiting
  (java.lang.Thread)0x829                         pool-1-thread-2             cond. waiting
  (java.lang.Thread)0x828                         pool-1-thread-3             cond. waiting
  (java.lang.Thread)0x827                         pool-1-thread-4             cond. waiting
  (java.lang.Thread)0x826                         pool-1-thread-5             cond. waiting
  (java.lang.Thread)0x825                         pool-1-thread-6             cond. waiting
  (java.lang.Thread)0x824                         pool-1-thread-7             cond. waiting
  (java.lang.Thread)0x823                         pool-1-thread-8             cond. waiting
  (java.lang.Thread)0x8db                         DestroyJavaVM               running
  (java.lang.Thread)0x9cd                         SwingWorker-pool-3-thread-1 cond. waiting
  (java.lang.Thread)0x9d6                         SwingWorker-pool-3-thread-2 waiting in a monitor
  (java.lang.Thread)0x9d8                         SwingWorker-pool-3-thread-3 cond. waiting
  (java.lang.UNIXProcess$1$1)0x9d9                process reaper              running
  (java.lang.Thread)0x9da                         SwingWorker-pool-3-thread-4 waiting in a monitor
  (java.lang.Thread)0x9db                         SwingWorker-pool-3-thread-5 waiting in a monitor
  (java.lang.Thread)0x9dc                         SwingWorker-pool-3-thread-6 waiting in a monitor
  (java.lang.Thread)0x9dd                         SwingWorker-pool-3-thread-7 cond. waiting
  (java.lang.UNIXProcess$1$1)0x9de                process reaper              running
  (java.lang.Thread)0x9df                         SwingWorker-pool-3-thread-8 cond. waiting
  (java.lang.UNIXProcess$1$1)0x9e0                process reaper              running
  (java.lang.UNIXProcess$1$1)0x9e1                process reaper              running

The important thing to note is the line:

  (java.awt.EventDispatchThread)0x82c             AWT-EventQueue-0            waiting in a monitor

Which means that the event dispatch thread is actually waiting to obtain a lock on an object — but which one ? First of all, I wanted to know where (in the source code) the thread was waiting. I could print the stack trace of this thread by doing:

where 0x82c

(0x82c is the thread id obtained from the above output). This gave me something like:

[1] MyTableModel.getValueAt (MyTableModel.java:136)
[2] javax.swing.JTable.getValueAt (JTable.java:2,686)
[3] javax.swing.JTable.prepareRenderer (JTable.java:5,703)
...

So indeed, the thread was stuck somewhere in my code. Looking at the corresponding line of source code, I had to go up a few lines to find the variable (the field myList of the class MyTableModel) in the synchronized statement which I was locking on.

However, I needed also to find out which other thread currently holds the lock on this variable in order to solve the problem. I could do this by doing running the following command in jdb:

lock this.myList

(where the jdb prompt showed me that the current thread was AWT-EventQueue0 because of the previous commands and the current stack frame was 1, so no need to set the current thread or stack frame). The output of the above command was:

com.sun.tools.example.debug.expr.ParseException: Unable to complete expression. Thread not suspended for method invoke
Owned by: SwingWorker-pool-3-thread-1, entry count: 1
Waiting thread: SwingWorker-pool-3-thread-2
Waiting thread: AWT-EventQueue-0

again, from the output of the threads command (see above) I could find the thread id of the thread SwingWorker-pool-3-thread-1 which owned the lock on myList. A quick

where 0x9cd

revealed that indeed the lock on myList was held by a thread which was waiting for an external command to complete:

[1] java.lang.Object.wait (native method)
[2] java.lang.Object.wait (Object.java:485)
[3] java.lang.UNIXProcess.waitFor (UNIXProcess.java:165)
[4] Utils.runCommand (Utils.java:102)
...

and I could also easily find the synchronized statement which acquired the lock on myList.

Read Full Post | Make a Comment ( 1 so far )

Using Jython as a configuration language for Java programs

Posted on November 26, 2010. Filed under: Uncategorized | Tags: , , , , |

I recently tried using Jython as a configuration language for a Java program. In fact, this was inspired by the fact that the main C++ project I’m working with at work has switched to python as a configuration language some time ago. This has several advantages:

  • no need to invent a configuration language from scratch
  • no need to write a parser for such a language
  • some users might already know python and thus they will not need to learn a new language (and python/jython is also useful beyond writing configuration files for the program)
  • the power of a programming language:
    • conditional statements
    • looping
    • generate configurations depending on parameters, contents of external files, environment variables etc.

An example configuration file could then look like:

input_files = [ "/etc/fstab", "/etc/mtab" ]
output_file = "/tmp/test.txt"

but we can also use some more complex logic inside the configuration file:

import glob, time
input_files = glob.glob("/etc/*tab")
output_file = "/tmp/test-" + time.strftime("%Y-%m-%d") + ".txt"

This example should give you an idea how powerful the concept is. In fact, whatever is possible in Jython is also possible to do in the configuration file.

It is often natural to group related parameters into a class, such as InputFile in the following example configuration:

input_files = [
  InputFile(name = "/etc/fstab", startLine = 2, endLine = 5, commentChar = '#'),
  InputFile(name = "/etc/mtab", endLine = 3),
  InputFile(name = "/etc/crontab"),
]

The underlying Java implementation of InputFile could look as follows:

public class InputFile
{
  ...

  /** Keyword arguments constructor. */
  public InputFile(PyObject values[], String names[])
  {
    System.out.println("called kwargs constructor: names=" +
      Arrays.asList(names) + " values=" + Arrays.asList(values));
     // store the given values in fields
    ...
  }
}

Note that keyword arguments constructors (requested here) are only supported in recent Jython releases (since 2.5.2-b2 according to the release notes). They are a bit more complicated to implement on the Java side than classic constructors but the use of keyword arguments in the configuration file makes it much more readable.

In order to read a configuration from a Jython file one needs to embed a Jython interpreter into the application. For this, it is necessary to add jython.jar from the Jython installation to the corresponding Java project.

The code for reading the configuration might look like this:

public class JythonConfigReader
{

  public JythonConfigReader(String input_fname)
  {
    // instantiate a Jython interpreter
    PythonInterpreter interpreter = new PythonInterpreter();

    // import configuration objects package in the Jython interpreter
    interpreter.exec("from my.package.with.configuration.classes import InputFile");

    // read the input file with the Jython interpreter
    // note that this will throw an exception if the python
    // code is not valid
    interpreter.execfile(input_fname);

    // get local variables
    PyStringMap locals = (PyStringMap) interpreter.getLocals();

    if (! locals.has_key("input_files"))
      throw new IllegalArgumentException("parameter input_files not found");

    // get the list of input files
    PyObject input_files = locals.__getitem__("input_files");

    if (! (input_files instanceof PyList))
      throw new IllegalArgumentException("parameter input_files is not a list");

    PyList input_files_list = (PyList) input_files;
    Iterator iter = input_files_list.iterator();

    // loop over all elements of the list
    while (iter.hasNext())
    {
      Object obj = iter.next();

      if (! (obj instanceof InputFile))
        throw new IllegalArgumentException("input_files contains objects of type other than InputFile");

      System.out.println("got an input file");

      // process this input file
      ...

    } // loop over all elements of input_files
  }
}

One potential danger of using Jython as a configuration language is that if one only looks at the names of the known parameters in locals() (such as input_files in the above example), one might miss mis-spelled optional parameters. To prevent such problems to some extent one can define a top-level configuration class and add other configuration objects as members, e.g. like:

config = Configuration()
config.input_files = [ InputFile(...), InputFile(...) ]

and then insist that there is only one top level Configuration object.

Read Full Post | Make a Comment ( None so far )

An animation of adding elements to a heap

Posted on October 24, 2010. Filed under: Uncategorized | Tags: , , , , , , , |

Here is an animation of how elements are inserted into a heap (sometimes used to implement a prioirty queue). At the top you can see the elements arranged in a tree, at the bottom is the corresponding array representation of the heap. The correspondence between the tree nodes and the array is such that the children of array element j are stored in positions 2j+1 and 2j+2 (where the indexing starts at 0).

The algorithm corresponds to what is described in section 4.3.2 of in “The Algorithm Design Manual” by Steven Skiena. Each time an element is added, it is put into the first empty position in the array representation at the bottom. This corresponds to making the new element a child of one of the leaf nodes. Then, the element is ‘bubbled up’ (swapped with its parent) as long as it is smaller than the parent.

Running the example requires Java Webstart (and is at your own risk, the author does not take any responsibility for any negative effects caused by running this example). To start the demo, click here. This demo makes use of the Trident java animation library.

Comments and suggestions are welcome !

Read Full Post | Make a Comment ( None so far )

An animation of the Quicksort algorithm

Posted on October 17, 2010. Filed under: Uncategorized | Tags: , , , , , , , |

Here is a (preliminary) animation of the Quicksort algorithm. At the same time, this is a small project to try out the Trident java animation library.

Running the example requires Java Webstart (and is at your own risk, the author does not take any responsibility for any negative effects caused by running this example). To start the demo, click here.

A few explanations:

  • The example is modeled after the Quicksort implementation as given in “The Algorithm Design Manual” by Steven Skiena (Section 4.6)
  • Those boxes which are already at the final position are drawn in white (instead of gray)
  • The boxes with dark red background are those currently being worked on (between the variables l and h). The variables l and h themselves are shown by the boxes ‘L’ and ‘H’
  • The variable i is displayed by the box I
  • The pivot (against which the element at i is compared) is at position h in this implementation
  • The box F corresponds to the value of the variable ‘firsthigh’ in Skiena’s book

You’ll notice that:

  • after i reached the right end of the range currently being worked on (end of routine ‘partition’), the value originally at h will be put where ‘firsthigh’ is and is at its final position. The elements left of it (in the current range) will have a smaller value, the ones on the right a larger value.

Comments and suggestions are welcome !

Read Full Post | Make a Comment ( None so far )

Using json from Jython

Posted on July 31, 2010. Filed under: Uncategorized | Tags: , , , , , |

I recently tried to use json from Jython. In CPython, there is a json module which is part of the standard library. However, this was only introduced in python 2.6. The most recent stable version of Jython is 2.5.1 however and it does not know a json module out of the box.

Fortunately, I came across jyson (http://opensource.xhaus.com/projects/show/jyson) which is a pure Java implementation of a json reader/writer. Although the code lives in its own name space, it seems that one can get the basic functionality (json.loads and json.dumps) equivalent to the CPython json module by doing:

import com.xhaus.jyson.JysonCodec as json

Read Full Post | Make a Comment ( 2 so far )

How to make the bookmarks become visible when opening a PDFBox generated file

Posted on May 19, 2010. Filed under: Uncategorized | Tags: , , , , |

I was wondering how to make the PDF reader show the bookmarks when opening a PDF file I generated with Apache’s PDFBox java library.

It turns out that the following did the trick for me:

PDDocument document = new PDDocument();
...
document.getDocumentCatalog().setPageMode(PDDocumentCatalog.PAGE_MODE_USE_OUTLINES);
....

Read Full Post | Make a Comment ( 1 so far )

JSch and kerberos authentication

Posted on October 17, 2009. Filed under: Uncategorized | Tags: , , , , , |

For an application I’m writing I’m using JSch (a java implementation of the ssh protocol). Now I tried to use this with authentication using a kerberos token (which has the advantage that I don’t have to supply a password every time I run the program for testing).

After spending some time googling and digging into the source code of JSch (a definitive advantage of open source libraries !), putting breakpoints in various places, especially those where it catches another type of exception and rethrows them as JSchException.

On this page I saw that one has to provide the location of login a configuration file by setting a property. This can be done on the command line by adding an option like:

-Djava.security.auth.login.config=/.../mylogin.conf

I got a little further. However, I got another exception:

javax.security.auth.login.LoginException: No LoginModules configured for

This looked to me like somebody is putting an empty string as configuration name somewhere (yes, the error message ends after the word ‘for’). I downloaded the sources of OpenJDK and digged further (even though I was not using OpenJDK as runtime library I was hoping that the differences were not too large). By looking at the source code, I had the impression that indeed at some point in call hierarchy (GSSUtil.login(..) ), an empty string literal is passed as name to the constructor of LoginContext (which I thought is used to look up the corresponding entry in the login configuration file). How am I supposed to put an empty string as login configuration name in the file ? (Simply leaving out the name did not work…)

By chance I found a related post in on Sun’s forums. It turns out that the following configuration entry in the login configuration file made JSch work with authentication by kerberos token:

com.sun.security.jgss.krb5.initiate {
  com.sun.security.auth.module.Krb5LoginModule required doNotPrompt=true useTicketCache=true;
};
Read Full Post | Make a Comment ( None so far )

Visualizing trees in java with JUNG

Posted on October 11, 2009. Filed under: Uncategorized | Tags: , , , , |

I was looking for a way to visualize a tree (of probabilities) in java. I came across JUNG (on sourceforge). There is a nice example (java applet) which displays a tree and lets the user collapse the subtrees on any node, exactly what I was looking for ! (I found out later that one can even drag the nodes of the tree around with the mouse)

I downloaded the binary release (jung2-2_0.zip), this seems to contain a bunch of jar files however it looks there is no source code of the above mentioned example. From the source of the example page, I could figure out the class name. After having found out which module I had to checkout from CVS (jung2), I could find the source code for the tree collapsing example, see this link (at least that’s what I think it is) in the CVS browser. I added the necessary (binary) jar files and I got the tree collapsing demo to run.

Another interesting example demonstrating different layout algorithms is showlayouts and the corresponding source code in CVS seems to be here.

There is also a nice step-by-step tutorial (with code snippets) by Greg Bernstein (linked from JUNG’s documentation page) which shows how to create graphs in section 2 and has a visualization demo in section 4. Thanks to the bookmarks in the PDF (which are not opened by default), navigation in the whole tutorial is very easy. Within minutes, one can put together a simple example of a graph with three vertices and two edges and display it on the screen.

Read Full Post | Make a Comment ( None so far )

Liked it here?
Why not try sites on the blogroll...