The Cold Start Myth: Optimizing the JVM for Events
SchemaBridge Team · 2026-01-20 · Serverless, Performance, JVM, Java
Why Java's cold-start reputation is about FaaS, not the JVM. Thread pools, JIT warming, and the long-running worker model.
The Latency Tax of Modern Cloud
"Serverless" was supposed to save us. It promised infinite scaling and zero management. But for User-Facing Latency, generic Serverless functions (FaaS) have a dirty secret: the Cold Start.
When a request hits a Lambda that hasn't run in a while, the provider has to spin up a container, boot the runtime, and load your code. For complex applications, this can take seconds. In the world of modern web applications, 2 seconds is an eternity. It is the difference between a "Snappy" experience and a "Broken" one.
For years, the industry wisdom was: "Java is too heavy for Serverless. Use Go or Node.js."
At SchemaBridge, we challenge this wisdom. We rely on the robustness of the JVM (Java Virtual Machine), and we sidestep the cold-start problem not by abandoning Java, but by not running it as a short-lived function in the first place.
The JVM: A Beast of Burden, Not a Sprinter
The JVM was designed for long-running server processes, not ephemeral functions. It spends its early moments "warming up"—loading classes, interpreting bytecode, and optimizing hot paths (JIT compilation).
If you treat a Java application as a "Function" that dies after 100ms, you are fighting against the physics of the runtime. You are paying the startup cost on every request without reaping the benefits of the JIT optimization.
Usage: Long-Running Application Contexts
SchemaBridge does not run your workflow steps as isolated, ephemeral functions. Instead, steps run on long-running JVM workers deployed on Kubernetes.
1. Thread Pooling: Each worker keeps a fixed pool of warm worker threads consuming from Valkey Streams. When an event arrives, an existing thread picks it up. There is no OS process to spawn and no JVM to boot per event.
2. JIT Optimization: Because workers are long-running processes, the JVM's C2 compiler has time to optimize hot paths to native machine code. A worker gets faster as it warms up—and, to be fair to the comparison, a freshly deployed worker starts cold and pays that warm-up once, not on every event.
3. Connection Pooling: Managing database connections is expensive. In a FaaS model, connections live only as long as a warm execution environment; every scale-up or recycled environment has to establish them again. Our long-running workers keep pooled connections to PostgreSQL and Valkey, and reuse HTTP clients for external services, so no step pays to establish them from scratch.
What a warm worker does not remove is the cost of being a durable workflow engine. Each step crosses a stream hop, claims its state in PostgreSQL and fetches its input before your logic runs. That is the price of surviving a crash mid-step, and it puts step latency in the range of network and database round-trips—not microseconds.
Where GraalVM Fits
GraalVM is already part of the platform, but not as a way to start Java faster: it is the polyglot engine that runs the JavaScript and Python snippets authors write inside workflows, in a sandbox that denies them process creation and file or network I/O.
GraalVM Native Image—compiling a Java service ahead-of-time into a standalone binary—is a different feature, and we do not use it. It is widely reported to cut startup time and memory substantially, at the cost of build complexity and restrictions on reflection and dynamic class loading. For a fleet of long-running workers the startup win matters less than it does for short-lived functions, which is exactly why we chose long-running workers instead.
Illustrative Scenario: A Latency-Sensitive Pipeline
Consider a hypothetical analytics team processing a steady stream of events with a Java function on FaaS.
The Problem
Periodic cold starts produce latency spikes into the seconds whenever the platform scales up or recycles an idle function, and the tail latency is dominated by boots rather than by the team's own code.
The Long-Running-Worker Approach
Moving that processing onto warm, long-running workers would change the shape of the latency curve:
1. Warm Threads: Events run on threads that already exist, in a JIT-warmed process.
2. Steadier Tail: Without per-event container provisioning, the spikes caused by boots go away. The remaining latency is dominated by the work itself and the durable hops each step makes.
3. The Trade-off: You pay for workers that are running whether or not events arrive, and a hard real-time budget measured in single-digit milliseconds is still the wrong job for a durable workflow engine—that belongs in your application's own request path.
Comparison: Ephemeral FaaS vs. Long-Running Workers
| Feature | Standard FaaS (Lambda) | SchemaBridge Warm Workers |
| :--- | :--- | :--- |
| Cold Start | Process/runtime boot on scale-up | No per-event process boot |
| Runtime | Boot on scale-up / after idle | Long-running Process |
| Optimization | Limited (environments recycle) | Full JIT Compilation |
| Connections | Lost when environments recycle | Pooled & Reused |
| Per-step Overhead | Function invocation | Stream hop + durable state claim |
Expert Checklist for Java Performance
1. Reuse Resources: Never create a database client inside your handler function. Create it once (static/global) and reuse it.
2. Tune Your Heap: Ensure your JVM heap is sized correctly for your container to avoid aggressive garbage collection.
3. Match the GC to the CPU Limit: A container with a fractional CPU limit may not benefit from a parallel collector; check which GC the JVM actually selects for your limits.
4. Avoid Reflection: Heavily reflective libraries (like some older JSON parsers) are slow to warm up. Prefer compile-time generation where possible.
5. Monitor GC Pauses: Use tools to ensure that Garbage Collection isn't introducing latency spikes into your workflow.
// Create expensive clients once per process, not once per event.
private static final HttpClient CLIENT = HttpClient.newHttpClient();Conclusion: The Runtime Matters
"Java means slow cold starts" is a statement about how Java is deployed, not about Java. By understanding the underlying physics of the runtime—and choosing an architecture that respects those physics—you can keep the productivity of the Java ecosystem without paying a boot on every event. Just be honest about what remains: a durable engine trades some latency for the guarantee that a step survives a crash.
Part of our Building the Bridge series. See also: Compliance & Auditing: Evidence as a Side-Effect and Building Generic Connectors.