Runtime model

When ol.busker/start! returns, Busker has created its first runtime generation, one running copy of the server built from a configuration snapshot. The snapshot is your config map with Busker’s defaults applied. For as long as that generation runs, Busker keeps the same listeners, network workers, request executor, handlers, and TLS setup.

Startup validates the configuration, prepares TLS and certificate services, opens the sockets, and starts the network workers. If validation, listener binding, TLS setup, or another startup stage fails, start! cleans up the partial generation and throws instead of returning a server handle.

Each event loop runs on a dedicated platform thread, an ordinary OS-backed JVM thread, and passes network events to libh2o. When a client connects, one of the generation’s event loops notices it. The :n-workers setting controls how many event loops a generation has and defaults to one.

The event loops accept connections, negotiate TLS and HTTP versions, read available request data, and send response data when a socket is ready. They also enforce connection limits and track how many bytes each HTTP stream or connection may have in flight. Only these network threads work with sockets. They do not run Ring handlers or middleware.

Once the request headers are ready, the event loop creates a Ring request and submits the handler to the request executor. That executor starts a new virtual thread, a lightweight JVM thread that the JVM can park without tying up a platform thread, for each request. The Ring handler and its synchronous middleware run there, and Busker writes the returned response from the same request task.

Most web handlers spend much of their time waiting for request data, databases, or other services. The JVM can park a virtual thread during ordinary blocking Java I/O, leaving the event loops free to serve other connections. This is why a synchronous Ring handler can read an upload or wait for a database without moving to Ring’s callback API.

CPU intensive work does not park. A handler that computes for a long time occupies one of the platform threads that runs virtual-thread work for the whole computation. A long native call can keep that platform thread occupied too. Move this kind of sustained CPU work to an executor sized for that work, and avoid blocking native calls in handlers.

See Ring compatibility for the supported handler and middleware model.

Now suppose a handler produces a large response for a client on a slow connection. The handler can produce bytes much faster than the network can send them. Keeping every unsent byte would let one client consume memory until the process failed.

Backpressure means that a producer waits when the consumer has no room for more data. Busker applies backpressure with a byte-bounded queue for each response. The :output-buffer-size setting controls the response buffer size and defaults to 32 KiB. Busker also holds the response data currently in flight through the HTTP engine, but it does not keep growing the per-response queue while the client is stalled.

The event loop removes a response chunk from the queue when it can hand that chunk to the HTTP engine. It waits for the engine to report progress on the current send before handing over more. Once the queue fills, the next application write or flush waits until the network side releases enough space. Busker waits rather than dropping response chunks.

For an ordinary Ring body such as an InputStream or lazy sequence, the handler’s job ends when it returns the response map. The request task is not finished, though.

Busker reads a chunk from the body, puts it into the response’s bounded queue, and repeats until the body is exhausted. When the queue is full because the client is reading slowly, that put waits like an ordinary blocking write.

Underneath, the JVM parks the virtual thread and gives its platform thread other work while the event loop keeps serving other connections. The request task stays alive until it has read and queued the complete body.

Long-lived responses use Busker’s response emitter so application code can send chunks after the handler returns. Backpressure applies to the emitter too, so an emit! or flush call waits on the calling thread when the response queue is full. On a virtual thread the JVM parks it, and the wait is cheap. On a platform thread the application thread remains blocked for as long as the client is slow. Run an emitter from a thread you can afford to block, ideally a virtual thread started for that stream. If application code closes an emitter, a final response completes, or libh2o reports request termination, Busker dispatches registered close callbacks on virtual threads. The callbacks run after Busker releases blocked response writers, and one failing callback does not suppress later callbacks. See the ResponseEmitter API for callback registration and stream closure details.

Uploads are bounded in the other direction. Busker asks the network side for another request-body chunk as the handler consumes the current chunk instead of copying the entire upload into JVM memory first.

Suppose the configuration changes while that slow response is still open. ol.busker/reload! does not tear down the running generation and then try to replace it. It builds a candidate generation beside the active one while the active generation keeps serving traffic. An unchanged configuration snapshot returns :unchanged unless the caller requests a forced reload.

If the candidate cannot start, reload! cleans it up, throws an exception, and leaves the active generation serving traffic. The old generation has not begun shutting down at this point. A configuration that validates but cannot bind a listener or initialize TLS therefore does not take the working server with it.

A successful reload starts the candidate and makes it active before asking the old generation to stop accepting work. New requests use the new handlers and configuration. Requests and response streams already running on the old generation keep using its handler and configuration until they finish. A request never jumps to the new generation halfway through.

The old generation then drains in the background. reload! returns after the old generation has stopped accepting work, not after every old request has finished. Several generations may be draining after several quick reloads, each with the requests that began while it was active.

Draining means that a generation accepts no new work while its current requests and response streams finish and its open connections close. Reload uses draining to retire one generation without delaying the new one. Shutdown uses the same path but waits for it to complete.

ol.busker/stop! starts draining the active generation and waits for every active or already draining generation. Busker waits for their response streams, close callbacks, request executors, network workers, listeners, and runtime services before stop! returns. A slow request or open stream can therefore delay application shutdown.

Each generation gives its request executor up to 60 seconds to finish before asking remaining tasks to stop immediately. Application code should treat interruption as cancellation and release its own resources promptly. Call stop! from an application lifecycle thread or shutdown hook rather than from a request handler, because a handler belongs to the work that stop! waits for. Calling stop! again after the server has stopped is harmless.