J2EE applications are characterized by the continuous creation, consumption, and destruction of various types of application objects.
These objects may be product objects in e-commerce applications, session objects, or user profile objects, to name a few common examples. Creation and destruction of these objects is expensive - object creation usually requires accessing persistent storage in back-end systems (e.g., DBMSs and file systems), while object destruction requires releasing resources used by the object (e.g., memory, database connections, etc.).
A very popular solution to address the costs of object creation and destruction is to store these objects in the application process memory, often referred to as in-process caching. One of the greatest advantages of in-process caching is that it provides fast access to application objects, which can improve application performance. Unfortunately, there are drawbacks associated with in-process caching. By far, the greatest drawback is its impact on garbage collection (GC) overhead in the application process where the cache resides. In certain situations, in-process caching can significantly increase the costs of GC, resulting in severely increased CPU utilization and response times.
In this article, I demonstrate that in-process caching can be harmful to application performance and scale in memory-constrained application environments due to its adverse impact on GC performance. I then discuss an alternative caching approach, an external caching architecture, and demonstrate that it can provide significant performance benefits for such applications when compared to in-process caching. Before delving into the details, I first provide an overview of two key concepts that are central to understanding this article: object caching and Java memory management.
Object Caching Basics
Object caching refers to storing an object that has been generated for a particular request so it can be used to serve subsequent requests for the object. A commonly used type of object caching is in-process caching, in which objects are stored in the application process memory. By keeping objects in local memory, subsequent requests for the object can be satisfied directly from memory, reducing the overhead of object creation and destruction.
Figure 1 depicts an in-process caching architecture. Nearly all enterprise software systems (whether coded from scratch or running packaged applications) have multiple caches that run inside each application process. Most J2EE application servers offer in-process caching features. In fact, in-process caching has become so widely used that it is considered the de facto standard approach for optimizing application performance and scale. JCACHE (JSR #107), which proposes a standard Java API for in-process caching, is evidence of this.
Overview of Java Memory Management
This article is concerned with two key concepts in Java memory management: garbage collection and reference objects.
Garbage Collection
Garbage collection is the method by which memory is automatically reclaimed from unused objects. An object is eligible for GC when it can no longer be reached from any pointer in the running program. Most JVMs use a generational collection model, which takes advantage of the fact that, in most programs, the vast majority of objects are very short-lived (e.g., temporary data structures). In a two-generational collection scheme (as used in the HotSpot and JRockit JVMs), a young generation is maintained for short-lived objects and an old generation for long-lived objects. When the young generation fills up, a minor collection is invoked. When the old generation fills up, a major collection is invoked.
