Why Compile Java Into Native Code?

June 2011

This web page was originally written in 2000 and the last update, before I added this addendum, was in 2004. As Steven King's Gunslinger would say, "the World has moved on" since then. The Internet bubble burst long ago. Sun Microsystems has been purchased by Oracle. My thinking about the choice between C++ and Java for software development has also changed. I would use Java to develop software, unless I had a compelling reason incur the high costs of using C++.

One of my core complaints about Java was the overhead incurred by the Java virtual machine. Since I wrote this essay, Java execution has gotten a lot faster. Programs that are run once, like a compiler, will still experience the overhead of the Java virtual machine. However, the Hot Spot compiler will dynamically compile portions of long running servers so that they execute with close to native compile speed. Examples of long running servers include database access engines, market trading systems and document search systems hosted on a Java Servlet container like Apache Tomcat.

There are many reasons that Java should be the langauge of choice for implementing an application:

  1. The largest body of reusable software that mankind has ever created is available to Java applications. This includes the extensive class libraries available from Sun (now Oracle) and the Apache Foundation. A large number of specialized open source projects have made software available in Java. For example, for a document database I made use of Christian Schlichtherle's TrueZIP classes for accessing and creating zip files and CABAReT Solutions jPod software for reading and creating Adobe PDF formatted documents.

  2. Java has become the lingua franca of the computer science community. For example, the Stanford Natural Langauge Processing group has made the Stanford Named Entity Recognizer (SNER) available to the Java community. The SNER can be trained to recognize person, organization and location entities in natural language (e.g., english) text. This can be useful for building large document databases.

  3. The Java virtual machine is standardized. Java that is compiled on one platform (Windows) will almost always run on another platform (Linux).

  4. Compared to a language like C++, compiled into native code, memory errors are rare. Java has arrays (in contrast to C++'s pointers) and has array bounds checking. A buffer overflow error in Java will result in a runtime error, rather than silent corruption of memory. The Sun (Oracle) implementation of Java includes a sophisticated memory "garbage collector" that recovers unused memory, avoiding most (but not all) types of memory leaks.

  5. To write correct C++ the programmer have a relatively deep knowledge of the runtime behavior of the language. For example, in the statement below (which uses classes from the Qt QtCore class library), the toAscii() function will allocate a QByteArray object that will be deallocated at the end of the statement. As a result, the character pointer that is returned by the data() function will point to deallocated memory.

      const char *cStr = myQString.toAscii().data();
    

    Compared to C++, Java is a simpler language. This means that the programmer can think about the application, not the details language execution.

When the decision is made to implement an application is C++, a huge cost is incurred. Memory allocation and reference errors are a constant concern. Many class libraries that are available without cost in Java are either not available in C++ or must be purchased. As a result, developing in C++ will take longer and developing reliable code is more difficult.

There are valid reasons for choosing to develop software in C++. For example:

  • C++ remains the language of choice for embedded systems and software that must directly interface to hardware.

  • As I write this, the support for Java in parallel distributed systems (e.g., openMPI) was not as mature as the support for C++.

  • Image processing libraries for C++ are much more mature and available than the limited libraries available to Java.

Outside of areas like these, there are rarely good reasons to bear the huge costs incurred by C++ software. C++ software development is increasingly a backwater and those who use C++ without a compelling reason don't understand that far better options exist.

March 2000

The Java Kool Aid

Sometimes I feel that Java is not just a programming language, class library and runtime, but a religion. One of the books from Sunsoft Press is titled Not Just Java: A Technology Briefing, by Peter Van der Linden, Second Edition, Prentice Hall, October 1999. The "blurb" for this book reads, in part:

Not Just Java is an up-to-the-minute briefing for IT professionals who need an executive summary of Java's impact on the Web, intranets, and E-commerce, as well as the role it will play in lowering the cost of computing. Understand how Java changes everything and more importantly how it will affect your IT organization. [italics added]

So it's "Not Just Java" but a religion that "changes everything". No religion is complete without dogma. Part of the dogma of Java is that a Java application can be compiled once and run anywhere, on any platform and the same results will be obtained. The "compile once, run anywhere" part of the Java dogma depends, in theory, on Java running on the Java virtual machine which provides a software Java processor on every platform.

The Java Boat Anchor

The problem with the "Java is the virtual machine" dogma is that it ignores the fact that for many applications Java is really slow. Java is slow even for an interpreted language. Some of this is due to the Java execution model.

A Java program is executed by running the Java virtual machine with the main class as an argument. The virtual machine will load this class, verify the byte code and then dynamically load and resolve all the classes referenced by this main class. Since these classes will in turn be verified and dynamically resolved, this process recursively loads and resolves all classes referenced by the application. This places a significant overhead on Java startup.

For example, I have published the source for a Java class file disassembler on these web pages. I tested this code by unpacking Sun's rt.jar, which contains the class library for Java 2. I then used the UNIX find command to recursively walk this directory tree and run the class file disassembler on each class. This kind of chaining of utility programs is natural in the UNIX environment. However, each time a class file is processed, the Java virtual machine must be loaded and the class file disassembler application dynamically resolved. This makes the execution of this utility surprisingly slow.

Another example, from a posting in comp.compilers:

On the topic of a processor simulator in Java, I can give you fairly accurate empirical results: I've just completed a fairly large ISA (MIPS32/MIPS64) simulator written in ISO C++, to replace a MIPS32 simulator written in Java and used in our Operating Systems course (University of New South Wales). Even without doing just-in-time translation of MIPS machine code to the target architecture, the speed-up was over 30 times. Unlike the Java simulator, my C++ simulator does some substential runtime checking. If our OS students are a representative sample, I can tell you one thing for sure: by staying clear of Java for processor simulation you're save yourself a lot of grief and customer support calls in the long run.

I think the moral of the story is that Java isn't well suited for CPU-intensive (esp. bit manipulation-intensive) applications that do 99% of work in a single tight but large loop with longjumps all over the place (to handle exceptions and interrupts.) My guess is that, even with JIT JVM, the performance is killed by the required bounds checking on the register file.

Patryk Zadarnowski

I have not seen Java perform well for compiler like applications. For example, the performance of the ANTLR parser generator, which is written in Java, is poor. For a 5K line grammar ANTLR takes a noticable amount of time. This is not a real problem with ANTLR and I mention it only as an example of a compiler like application. Here is another example from personal correspondence, in response to my mention that Java is a poor choice for compilers.

I translated the Sun Java compiler itself into C++ and realized a 10x speed boost, so I know what you're talking about.

Java is a simpler language than C++ and has features like garbage collection that would make it desirable for implementing large applications. But the virtual machine limits Java to applications where Java's performance is not the bottleneck. Virtual memory usage in Java is also awkward. The virtual machine has its own "memory". Memory usage cannot grow as an application executes. Java's memory space must be allocated at startup. The performance handicap and lack of flexibility created by the Java virtual machine suggest that it is unlikely that Java will ever be used for processor and memory intensive applications like VLSI chip design and simulation.

The Java Just In Time Compiler and Other Ducks (ah, Canards)

The performance problem introduced by the JVM is hardly unknown to Sun. They seem to take two tacks:

  1. There is no performance problem. For GUI, networking (e.g., Web), or application with lots of database lookups, the application performance is not limited by the JVM but by I/O to the graphic display or the network or the speed of the database engine. Ultimately the Java class references that support the GUI or networking break down to calls to native methods. So in these cases Java's performance can be similar to that of C or C++.

  2. There is a performance problem, but new Java technology, like the Just In Time (JIT) compiler and the HotSpot optimizer allow Java to run as fast or faster than native compiled code.

The first claim is probably true for networking code, and I would have guessed that it was true for GUI code as well. But truth is hard to come by and guessing is certainly not the same as factual data. Java is a new language and the computer science community (both commercial and academic) only just starting to get experience with Java. Vitaly Mikheev, the Java Project Manager at Excelsior, which developed the JET native Java compiler, writes:

Our experience with compling many AWT samples shows that the compiled version works much faster. Acceleration of Swing samples is even more significant. These results have a simple explanation: the current version of the Java GUI class library (AWT/Swing) is large enough so the amount of Java code running before direct calls to the operating system API is quite significant.

The second claim is true, at best, for a very limited number of applications that run for a very long time, don't use increasing amounts of virtual memory and have lots of loops that can be optimized by the HotSpot optimizer.

Sun is attempting to compare technologies like JIT and HotSpot to static optimizing compilers (e.g., for C++). To state the obvious, the advantage of static compilation is that it is done once. When full optimization is turned on, the compiler slows down significantly. But production code is run many times, so this is a cost most users are willing to pay.

A good optimizing compiler can reduce the code size by half and increase performance by a factor of two over unoptimized code. If the same optimization algorithms that are used in an optimizing compiler were used in a Java JIT, the JIT would be huge and it would run slowly until its code generation was complete. For most applications this would result in an unacceptable runtime startup cost.

Even if JIT is only optimizing the "hot spots", a proportional cost will be paid. And not all programs have hot spots. The inner loop theory is only true for some applications, not all. Compilers and many tools used in the EDA industry (e.g., logic synthesis) are not loop intensive. Optimizing compilers produce improvement throughout the code, not just in loops.

And the benchmark will set you free

What I have described above is largely my intuition based on what I know of compiler design and software architecture. I could be wrong or Sun could be using techniques that I know nothing about. Clearly what is needed is some hard evidence. Understanding performance issues is a lot of work, but truth is never cheap. Sun has an compiler group that has put a lot of effort into optimizing the output of their C++ compiler. Java and C++ share many similarities so comparing Java's performance to C++ is a reasonable comparison. So if Sun really believes that Java performance issues are put to rest by JIT and/or HotSpot they should publish a set of benchmarks, preferably against their own C++ compiler. If HotSpot comes within twenty percent of the performance of optimized native code I am happy to concede the point.

Some Benchmark Data Points

The Java world is moving at an amazing pace. Keeping my Java web pages up to date with the various developments in native compilation is becoming time consuming. Material is starting to appear benchmarking native Java compilers. If you know of other benchmark data, please send me e-mail (iank@bearcave.com).

As I've noted above, I'm willing to believe that Java will do well in comparision to C++ on floating point or elementary function (sine, cosine, sqrt) intensive benchmarks. I am less willing to believe that this is the case for non-floating point benchmarks. What I would like to see is a benchmark built around data structures like linked lists, binary trees, and hash tables. It might include other algorithms like finding a sub-string within a string. If Java beats C++ on these benchmarks then the question is why? The JVM introduces overhead, as does JIT compilation. If Java is faster than C++ compiled into native code there should be a reason.

Going Native

Some people claim that an optimizing "HotSpot" compiler will generate code that is as good as a static optimizing native compiler. For example, see Java on Steroids: Sun's High-Performance Java Implementation, HotChips IX, August 1997. This is a set of slides, in PDF format, from a talk given by Professor Urs Hoelzle of UC Santa Barbara.

Let us assume for a moment that the HotSpot people are correct: a HotSpot compiler/optimizer will allow Java code to run as fast as optimized native code. Note that the HotSpot compiler is part of the Java virtual machine. The Java virtual machine already has a large memory "footprint". Adding HotSpot will only make it bigger. This makes a JVM with HotSpot impractical in many embedded environments.

Java does not have to be limited to running on a virtual machine. Java can be compiled to native code. Java applications compiled to native code would have performance that is comparable to compiled C or C++. By removing the performance limitations imposed by the JVM, Java could be used to write UNIX style utilities and processor intensive applications.

Only a few features in the class library make compiling Java to native code difficult: specificly the ability to dynamically load classes. Interestingly enough, this is one of the features that is not required for Java support in an embedded environment. So apparently Sun is willing to violate the Java dogma when it suits them.

When Java Performance Does Not Matter

I've talked to several people who have stated that their productivity was much higher in Java than in C++. I believe that much of this increased productivity is a result of the Java class library. Between the Java standard edition library, the Java "Enterprise" library and libraries available from sources like Apache, Java has the best class library I have ever seen, for any language.

The Java class library has some nice support for networking and many of the applications people mentioned where network related. Most networking applications are limited by the network bandwidth or the performance of the networking stack on the local system. Since these applications are I/O limited, Java's performance is not an issue.

Java also has good support for database operations (e.g., JDBC). Since most of the execution time will be gated by the speed of the database, not the language that calls the database, we can expect that there would be litte difference between a C++ and Java database application.

The Power of Java as an Interpreted Language

Java is a very nice language and is, in many ways, an improvement over C++. The performance penalty introduced by the JVM limits Java's applicability. The applications that would benefit from a native compiled version of Java are classic computer applications that read data from disk, process it and write it out to disk or to the display. However, this does not mean that interpretation is not a powerful feature for Java.

Although Java does not change everything, the World Wide Web does. The Web is a new environment for computation. For many Web based applications performance is limited by the network or by the speed that the user inputs data. In these applications the JVM overhead is not a problem and the interpreted nature of Java offers many advantages.

Of all of the Java technology developed by Sun, the most ground breaking is Sun's architecture for distributed computing, Jini. This is not the place to describe Jini technology (see W. Keith Edwards outstanding book Core Jini for an excellent, hype free, description of Jini), but it is only reasonable to note that Jini would not be possible without the JVM.

Although the JVM is a critical component for some Java applications, Java should not be limited to to execution on a JVM. Limiting Java to an interpreted system is marketing and dogma, not engineering.

Does Java Have Too Many Features for Good Native Performance?

I have had the good fortune to work for Mike B., who is not only a good manager and all around Mensch, but is also a brilliant engineer. Mike pointed out that when a language is interpreted, the temptation to add features to the virtual machine becomes irresistible. If these complex features are critical to supporting the language, it becomes increasingly difficult to compile the language into native code efficiently. Sun certainly has done nothing to preserve the ability to compile Java into native code.

Java has grown into more than just a language. Several important Java applications rely on features like dynamic object loading (via the Java class loader), serialization, and remote method invocation (RMI). Support for some of these features in native code is difficult. The native code versions of these features may not, in fact, be any faster than code running on a JVM. For example, in the case of RMI an object, consisting of code and data, can be passed as an argument over a network. The system receiving the RMI call may be architecturally different from the system making the call, so native code would cause RMI to fail. Even if the systems were architecturally compatible, the overhead of dynamic linkage may limit the advantages of native code.

So it is possible that Sun may be right. Interpreted Java may turn out to be as fast as native compiled Java. But if this is the case it will not be a result of any Sun technology. Rather, it will be that Sun has larded Java with so many features that the efficiency offered by native code is destroyed by the overhead of the Java environment.

Currently there are very few native Java compilers. Compiler designers are also still learning how to compile features like exceptions efficiently (exceptions wreck havoc with the control flow graph). So it is too early to tell whether this bleak view of native compiled Java is justified.

For the last four years I have been part of a group that has been working on native compilers (or simulators) for the Verilog and VHDL Hardware Design Languages. These languages were originally interpreted. Supporting them in a native environment is a huge challenge. But it can be done and native version is significantly faster than the interpreted version. So I still have faith that if Verilog can be compiled into native code to gain a performance advantage, the same is true for Java. But it may turn out that compiling features like RMI into native code yields no performance improvement.

As noted above, one of the most important applications of RMI is as a foundation for a Jini distributed computing network. The elements of a Jini network must be able to communicate with the Jini protocol. They may, and in many cases must, have a significant non-Java component (e.g., native code device drivers or other software). The fact that Jini code runs on a JVM would not be a performance issue in many cases.

Web References

Ian Kaplan, March 17, 2000
Updated: June 15, 2004
Addendum: June 19, 2011


back to Notes on Software and Software Engineering