Dec 07, 2013
Jason Dusek
D2iQ
5 min read
At Mesosphere, a fair number of applications that we write are Java or Scala applications, many of them ship with a companion shell script that handles passing the JAR and all command line arguments to Java, which then delegates to the main class for processing the arguments and launching the program. This is unfortunate, both because of the regrettable "magic" sometimes used in the script to find the JAR, and because of how much simpler it is for users to deal with a single, self-contained application. (Especially for internal applications, where multi-versioning is the order of the day.)
Trying to download and execute a vanilla JAR archive usually doesn't work, although a JAR file with a main class -- set with Main-Class: ... in Manifest.txt -- does have a well-defined entrypoint.
:; curl -sSfLO https://download.elasticsearch.org/logstash/logstash/logstash-1.2.2-flatjar.jar:; chmod ug+x logstash-1.2.2-flatjar.jar:; ./logstash-1.2.2-flatjar.jar web --help-bash: ./logstash-1.2.2-flatjar.jar: cannot execute binary file
In this article, we explore a simple and general approach to making JARs executable in the traditional UNIX sense -- one can mark them executable with chmod and run them directly. In contrast to Linux's binfmt_misc, this technique is applicable to all systems with a POSIX shell.
Elementary Executable JARs
It may come as a surprise that Zip files, of which JAR files are a subset, can be prepended with arbitrary, non-Zip data and yet remain interpretable to the Zip decompressor. This suggests prepending exec java -jar "$0" "$@" to a JAR to make it function as UNIX executable, because:
An executable file, not marked with file magic, is treated as a POSIX shell script by the kernel.
Java, when handling the same file as a JAR, will delegate to a Zip library that skips forward in the file until it finds the Zip header, bypassing the shell script.
Continuing with our example:
:; ( echo 'exec java -jar "$0" "$@"' cat logstash-1.2.2-flatjar.jar ) > logstash.jar:; chmod a+rx logstash.jar:; ./logstash.jar web --helpUsage: runner.class [options] -a, --address ADDRESS Address on which to start webserver. Default is 0.0.0.0. -p, --port PORT Port on which to start webserver. Default is 9292.
Intermediate Executable JARs
In the shell example given above, $0 refers to the path to the script, which also is the path to the JAR, and $@ refers to all the positional arguments. They are quoted for safety, ensuring that both the path and each individual argument are passed as is to the JAR.
exec java -jar "$0" "$@"
When wrapping an external application, one might like provide some JVM settings, a way to load configuration from a default location if it's present and some defaulting rules for setting environment variables (PORT and JAVA_HOME are two that come up a lot, here).
In principle, one can use any scripting language whatever; but not all interpreters are happy with loading a script that consists of a few valid statements and a lot of binary rubbish. Even in a language as permissive as the shell, the above example will trigger a warning about a parse error more often than not (it depends on which shell is actually functioning as the sh on the system in question). To skip this warning, we should add exit after the exec line:
exec java -jar "$0" "$@"exit
This exit is never called unless the exec fails (in which case, exit with no arguments will propagate the exit code from the failed exec call). Once the shell sees exit -- and this is true of many shells, including Bash -- it will stop parsing the file, ignoring the contents of the Zip archive.
To read values for the --address and --port options from files in /etc/logstash/web, if they are present, we need a slightly longer script:
#!/bin/bashset -o errexit -o pipefail -o nounsetexport LC_ALL=C #### Optional.
args=( -jar "$0" )for f in address portdo if [[ -f /etc/logstash/web/"$f" ]] then value="$(cat "$f")" args+=( --"$f" "$value" ) fidone
exec java "${args[@]}" "$@"exitNote that in the above example, we use errexit, pipefail and nounset to ensure that the script fails when files which are found can not be read or variables that are needed have not been set. Setting the locale to C -- the strings-as-bytestrings locale -- ensures the program will behave consistently across a wide variety of setups; although en_US.UTF-8 might be preferable for server-side applications (since it enables UTF-8 and is widely available).
Advanced Executable JARs
There is no practical limit to the length and complexity of the script that is prepended to a JAR. One can implement syslogging, fairly general file-to-options translation and a variety of other additions without changing the underlying program, in just a few dozen lines of shell script. Naturally, one should break up the script in to a few shell functions to make it both easier to understand and easier to modify. The Logstash wrapper script that we've put together in logstash-pkg illustrates a range of techniques applicable in these circumstances.
Executable JARs Via Marathon & Chronos
As long as a JAR wrapped in this way sets reasonable defaults, it's easy to treat it as a completely self-contained application, launchable via Marathon or Chronos on Mesos. Here's an example API request for Marathon:
http -v POST http://localhost:8080/v1/apps/start id=logstash cpus=2.0 mem=1024 instances=1 uris:='["https://assets.example.com/logstash.jar"]' cmd='chmod a+rx *.jar && ./*.jar web'
Further reading
To find out more about launching applications on Mesos, please see our helpful tutorials:
