HOWTO Use the ProcessTool Utility to Execute Binaries

What does ProcessTool do?

ProcessTool provides an interface for executing binary programs found on your server from within your Java application. It captures stdout and stderr into StringBuffers at the same time using separate threads.

StringBuffer stdout=new StringBuffer();
StringBuffer stderr=new StringBuffer();
Strint cmd="grep \"ERROR\" /logs/myapp/*";
ProcessTool.exec(cmd, stdout, stderr);
if (stderr.length()!=0) {
  throw new IAmSoShockedException("Gasp!");
}
System.out.println(stderr.toString());

Why not use the "Process" Object provided by Java?

There's nothing wrong with the Java "Process" object-- in fact, ProcessTool wraps that object. One value-add from using ProcessTool is safety from the rare (but insidious) instances in which the application being executed writes to both stdout and stderr. If you are just using "Process" in a single thread, and first capturing stdout, waiting for it to finish, and then capturing stderr and waiting for it to finish, then you run a risk of deadlock.

A key to understanding the risk is understanding that there is a buffer for each stream that could vary in size depending on configuration of the OS, but frequently consists of eight 512 byte blocks. A process can write to stdout or stderr without another process reading from it until either buffer is full. At that point, it will stop and wait for the next block to become available before it can continue writing. When another process reads a block from the same stream, it frees it to be refilled.

A process that writes less data than the buffer can resume other work without waiting for that buffer to be read. If nothing is reading, then it stops when the buffer is full and resumes when it can. If it is single threaded, then it will typically block and wait without doing any other task until another process reads from that stream.

Consider an external process, for example, that performs some action on a list of files before deleting them. An error of some sort is encountered, perhaps from a corrupted file or a permissions issue. Your process outputs an error message, throws in some additional details, and there is enough content (a stack trace, for example) to overfill a 4k buffer. The expected content is another 6k of text.

If that process prints out all of its good text, followed by all of its error messages, then you should be able to process it fine. The process would print its stdout into the same buffer you're initially reading, and would thus be able to output all it needs until it's done. You would read all of the stdout, then all of the stderr, in the same order in which it is written.

The problem arises when it doesn't finish writing to stdout before writing to stderr. If it needs to write 6k to stderr, then has even 1 more byte to write to stdout, then stdout will be held open. Your app downstream will patiently wait for the next block to be written to stdout, the upstream process will wait for a block to be read from stderr (because the 4k buffer is full) and your app will not read stderr until it is done with stdout. You need another thread to pull down stderr so the process will resume its work and write to stdout again.

Another value-add is convenience. Starting with version 2.6.0, ProcessTool can parse and run statements like,

  echo "a b" 'cd' "O'Sean"