Figure 1: The dashed boundary now includes the server as well.

Part 1, Chapter 1 named the connecting thread of this series: the dashed boundary around what belongs to the application. In Parts 1 and 2, only the WAR lay behind this boundary. Now Jetty moves inside.

The last row is the actual point. Anyone who wants to switch the Jetty version afterwards changes a number in thepom.xml and ships the application again — no longer a symlink on the server.

The Repository Becomes a Reactor

Up to this point, the demo application wasone Maven module. That worked as long as there was one delivery format. With the second one, it no longer worked well: the starter for embedded operation lived insrc/main/java and was therefore compiled with every build — including the WAR build, where nobody calls it. This is exactly where the finding from Part 1, Section 4.5 came from: the complete Jetty ended up in the WAR because a single dependency stood there without a<scope>.

The remedy back then was a declaration. The cause only disappears now:

Blog-Vaadin-Deployment-on-Hetzner/ reactor
├── core/ the application, the tests, the frontend bundle
├── war-jetty/ ROOT.war for the external Jetty (parts 1 and 2)
└── embedded-jetty/ own main() (this part)

One module per delivery format. The WAR module containsnot a single line of Java source code — only aweb.xml. The embedded module contains exactly one class. Everything else lives in the core and is used by both.

The Code for This Part

git clone https://github.com/svenruppert/Blog-Vaadin-Deployment-on-Hetznercd Blog-Vaadin-Deployment-on-Hetznergit checkout teil-03

The state of Parts 1 and 2 — one module, building totarget/ROOT.war — remains available on the tagteil-02. Agit diff teil-02 teil-03 shows exactly what this part describes.

What This Part Does Not Cover

Packaging. At the end of this part, an application archive sits next to a directory containing its libraries. Whether that becomes a single archive or a tidy directory structure is the question for Part 4 — and it can only be asked meaningfully once the unpackaged case has been seen standing on its own.

What Does Embedded Jetty Mean?

The term sounds like a variant of the same thing. In fact, it describes a reversal of responsibility.

A Library Instead of a Server Installation

In the setup so far, Jetty is a program. It lives under/opt/jetty, it has its own directory structure, it is configured through files, and it starts the application. The application is an archive this program reads in.

Embedded, Jetty is a library. It sits on the classpath next to Jackson and SLF4J, it has no directory structure of its own, it is configured through Java code, andthe application starts it. The program that loads the application becomes a dependency the application uses.

Application Lifecycle Instead of Container Lifecycle

The practical difference shows in the startup sequence.

With the external Jetty, systemd starts the container, the container looks for archives inwebapps/, unpacks them, builds one web application context each, and calls the initializers inside. The application has no entry point; it gets found.

Embedded, systemd startsthe application. Itsmain() method creates a server, mounts a context, starts it, and waits. The entry point is ordinary Java code that a narrative can walk along — and that is exactly what Chapters 4 through 9 do.

What Becomes Visible in the Process

The external Jetty takes care of a whole series of things without any of them showing up in the project. The module files understart.d/ numbered sixteen in the reference setup. Sixteen decisions that someone once made and that nobody sees afterwards.

Embedded, every one of them that is actually needed must appear in the code. That is more work — and it is the reason this part is instructive in the first place: what used to be configuration becomes readable.

Three of these decisions failsilently if you forget them. They get a chapter of their own in Chapter 10, because they make the difference between “runs” and “runs correctly”.

What Stays the Same

More remarkable than the differences is how little changes in operation. Service account, directory layout, hardening, configuration outside the artifact, credentials viasystemd-creds, Caddy in front — all unchanged. The unit file swaps one line.

That is no coincidence but the result of the separation that Part 0 and Chapters 9 through 11 of Part 1 built up: the operational framework does not knowhow the application starts internally, and it does not need to.

Maven Dependencies

The external Jetty was set up with--add-modules=server,http,ee11-deploy,ee11-annotations,ee11-websocket-jakarta. Embedded, there are no modules — there are dependencies. Translating one into the other is the subject of this chapter.

Four Entries

<dependency><groupId>org.eclipse.jetty</groupId><artifactId>jetty-server</artifactId></dependency><dependency><groupId>org.eclipse.jetty.ee11</groupId><artifactId>jetty-ee11-webapp</artifactId></dependency><dependency><groupId>org.eclipse.jetty.ee11</groupId><artifactId>jetty-ee11-annotations</artifactId></dependency><dependency><groupId>org.eclipse.jetty.ee11.websocket</groupId><artifactId>jetty-ee11-websocket-jakarta-server</artifactId></dependency>

No version numbers — those come from the two BOMs that were already imported in Part 1.${jetty.version} thus remains the only place where the Jetty release is recorded, now for the build instead of for the installation.

The Servlet API Switches Sides

<dependency><groupId>jakarta.servlet</groupId><artifactId>jakarta.servlet-api</artifactId><scope>compile</scope></dependency>

In the WAR module, this saysprovided: available for compilation, left out at packaging time because the container supplies it. Here there is no container supplying anything — the applicationis one. So the API has to come along.

That is the same reasoning as in Part 1, Section 4.5, only applied in the opposite direction. Anyone who readsprovided as “does not belong in the archive” has misunderstood it; it means “someone else provides this”. When there is no someone else, the answer changes.

Four Entries Nobody Ordered

<dependency><groupId>org.ow2.asm</groupId><artifactId>asm</artifactId></dependency><dependency><groupId>org.ow2.asm</groupId><artifactId>asm-tree</artifactId></dependency><dependency><groupId>org.ow2.asm</groupId><artifactId>asm-commons</artifactId></dependency><dependency><groupId>org.ow2.asm</groupId><artifactId>asm-analysis</artifactId></dependency>

ASM is the library Jetty’s annotation scanner uses to read class files. It appears here only to forcea single version. Default resolution mixesasm 9.8 (viaj2objc-annotations) withasm-tree andasm-commons 9.9.1 (via Jetty). This mixture breaks on Java 26 bytecode:

java.lang.IllegalArgumentException: Unsupported class file major version 70

The context then fails to start, and the message mentions neither Jetty nor Vaadin. It is included here because without prior knowledge it is almost impossible to place: class file version 70 is Java 26.

What Disappears

One dependency is removed from the core module without replacement — the library that encapsulated the embedded startup until now. With it goes the<scope>provided</scope> workaround from Part 1, Section 4.5, and the whole profile machinery that squeezed an additional archive out of a WAR module.

The reason this works: the embedded module hasjar packaging from the start. Nothing needs to be worked around when the structure matches the intent.

The Application’s Entry Point

A web application with amain() method looks unusual at first. It is the core of this part.

The Class

publicfinalclassApplicationimplementsHasLogger{privatestaticfinalStringDEFAULT_HOST="127.0.0.1";privatestaticfinalintDEFAULT_PORT=8080;publicstaticvoidmain(String[]args){newApplication().launch();}…}

It is theonly class in theembedded-jetty module. Everything else — views, security, persistence, translations — lives in the core and is pulled in via a dependency. That is not frugality but the statement the split makes: how the application starts is a question of the delivery format, and nothing else.

Host and Port

privatevoidlaunch(){Stringhost=resolve("app.host","APP_HOST",DEFAULT_HOST);intport=resolvePort();…}privatestaticStringresolve(StringsystemProperty,StringenvVariable,Stringfallback){StringfromSystem=System.getProperty(systemProperty);if(fromSystem!=null&&!fromSystem.isBlank())returnfromSystem;StringfromEnv=System.getenv(envVariable);if(fromEnv!=null&&!fromEnv.isBlank())returnfromEnv;returnfallback;}

Three sources in a fixed order: system property, environment variable, default value. The external Jetty made the same decision instart.d/http.ini; here it stands as a method you can read.

The default is127.0.0.1, not0.0.0.0. That is the same decision as in Part 1, Chapter 8, and here it matters even more: a typo in a configuration file gets caught during review, a wrong default in the code takes effect everywhere nobody configured anything.

The Check That Comes Before Everything Else

if(Application.class.getClassLoader().getResource(BUNDLE_MARKER)==null){abort("Vaadin production bundle missing from the classpath (expected "+BUNDLE_MARKER+"). Build with `mvn -Pproduction package` before starting; without it the "+"server starts, answers 200 and serves a blank page.");return;}

Five lines that catch one of the most annoying failure modes. Chapter 8 describes it in detail; the short version: if the built frontend is missing, the server starts anyway, answers with 200, and serves a white page. The error then shows up not in the server log but in the browser console — and anyone who does not look there searches in the wrong place.

The check turns this into a startup message with instructions for action.

Failing with a Statement

@SuppressFBWarnings(value="DM_EXIT",justification="intentional — main-class launcher exits non-zero so a process supervisor sees the failure")privatevoidabort(Stringmessage){if(message!=null)logger().error(message);System.exit(1);}

System.exit(1) in an application is normally a warning sign. In a launcher class it is the right answer: systemd distinguishes a service that terminates with an error code from one that simply does not respond. Without this call, the JVM would keep running,Restart=on-failure would not kick in, and the service would count as “running”.

Creating and Configuring Jetty

Four lines bring the server up. Two of them deserve an explanation.

Serverserver=newServer();HttpConfigurationhttpConfig=newHttpConfiguration();httpConfig.addCustomizer(newForwardedRequestCustomizer());httpConfig.setSendServerVersion(false);ServerConnectorconnector=newServerConnector(server,newHttpConnectionFactory(httpConfig));connector.setHost(host);connector.setPort(port);server.addConnector(connector);

Server Without a Thread Pool

new Server() creates aQueuedThreadPool with default values behind the scenes. A custom pool would be possible:

QueuedThreadPoolpool=newQueuedThreadPool(200,8);Serverserver=newServer(pool);

Here, deliberately, there is none. The reference setup drew the limit elsewhere:TasksMax=256 in the systemd unit from Part 1, Chapter 11. Two upper limits for the same thing are one upper limit too many — the smaller one wins, and whoever changes the other wonders why nothing happens. Measured in operation, 30 threads are running; the defaults are sufficient by a wide margin.

ForwardedRequestCustomizer — the Line Whose Absence Nobody Notices

This single line replaces--add-modules=forwarded from Part 2, Chapter 4. What it does and why its absence is dangerous is covered in Chapter 10; here, only the place where it belongs: on theHttpConfiguration,before the connector is created. Added afterwards, it no longer takes effect, because theHttpConnectionFactory has already captured the configuration.

setSendServerVersion(false)

Without this line, Jetty sends aServer: Jetty(12.1.12) header entry with every response. That is not a security hole, but it is free information for every scanner about which release to attack. The external Jetty switched this off viastart.d/http-config.ini; here it is one line of Java.

Bind Address: the Same Decision, a Different Place

connector.setHost(host);//127.0.0.1

Part 1, Chapter 8 explained why the application server listens exclusively on the loopback address: what cannot be reached does not have to be defended. The reasoning holds unchanged — only now it lives in the source code instead of a configuration file.

And it is the precondition for the line above it. Evaluating forwarding headers is only defensible because the only party able to set them is the reverse proxy on the same machine. If the connector were bound to0.0.0.0, anyone on the network could claim the request arrived over HTTPS from any address they like.

These two lines belong together, and in the embedded setup they stand visibly side by side for the first time. In the external Jetty they lived in two different.ini files.

Configuring the Servlet Context

The server alone does not answer any request. It needs a context — the counterpart to what the external Jetty created when unpacking a WAR.

WebAppContextwebapp=newWebAppContext();webapp.setContextPath("/");webapp.setBaseResource(webResources(webapp));StringscanPattern=System.getProperty("app.scan.pattern",DEFAULT_SCAN_PATTERN);webapp.setAttribute(MetaInfConfiguration.CONTAINER_JAR_PATTERN,scanPattern);webapp.setAttribute(MetaInfConfiguration.WEBINF_JAR_PATTERN,scanPattern);webapp.setParentLoaderPriority(true);

WebAppContext orServletContextHandler?

Jetty offers both.ServletContextHandler is the leaner path: no scan, no separate class loader, less startup time. For many embedded applications it is the right choice.

For a Vaadin application it is not. Vaadin finds its@Route classes through theRouteRegistryInitializer — aServletContainerInitializer announced viaMETA-INF/services andsearched for by the container. Take that search away, and you have to register the routes by hand. The effort does not disappear; it merely changes location.

WebAppContext brings the search along. The price is in the next line.

The Scan Pattern Is a Tuning Knob

privatestaticfinalStringDEFAULT_SCAN_PATTERN=".*\\.jar$";

This pattern says: openevery archive on the classpath and search it for initializers with ASM. In the reference setup, that is 127 of them.

That is why the pattern does not sit in the code as a constant but behind a system property:

java -Dapp.scan.pattern='.*(vaadinapp|flow|vaadin|jcustos)-.*\.jar$' …

Narrowing it down shortens startup measurably — and buys you a list that wants to be maintained with every new dependency. The reference setup therefore leaves it wide; the option is in the code anyway, because otherwise nobody would find it.

How much the scan accounts for can be quantified with involuntary precision: a start without the scan takes 139 ms, with the scan 1,523 ms. How this measurement came about is told in Chapter 12 — it was a failure.

setParentLoaderPriority(true)

WebAppContext comes with its own class loader, which normally searches the web applicationfirst and only then the parent. In the WAR model that is correct: an application may bring its own release of a library without the container interfering.

Embedded, this separation no longer exists. Application and server sit on the same classpath; a class loader that continues to assume two worlds only creates confusion — in the worst case the same class loaded twice, with aClassCastException nobody understands.true switches back to the ordinary Java order.

It is the same issue that Part 1, Section 4.5 described from the other side: there, Jetty’s precedence rules protected the installed Jetty from being displaced by a bundled one. Here there is no installed Jetty left to protect.

Integrating Vaadin

The context is in place. Now the application goes in — and in the process, a file resurfaces that was taken for granted in the WAR model.

ServletHolderholder=newServletHolder(newAppServlet());holder.setInitParameter("i18n.provider","com.svenruppert.flow.i18n.AppI18NProvider");holder.setAsyncSupported(true);holder.setInitOrder(1);webapp.addServlet(holder,"/*");

Five lines that do exactly what theweb.xml does in the WAR module:

<servlet><servlet-name>Vaadin Servlet</servlet-name><servlet-class>com.svenruppert.flow.AppServlet</servlet-class><async-supported>true</async-supported><load-on-startup>1</load-on-startup><init-param><param-name>i18n.provider</param-name><param-value>com.svenruppert.flow.i18n.AppI18NProvider</param-value></init-param></servlet><servlet-mapping><servlet-name>Vaadin Servlet</servlet-name><url-pattern>/*</url-pattern></servlet-mapping>

Line for line the same. That is no coincidence — the deployment descriptor and programmatic registration are two notations for the same Servlet API.

The Parameter That Costs a Language

i18n.provider looks like a nicety and is not. Vaadin V25 doesnot look up theI18NProvider viaMETA-INF/services but exclusively through this init parameter. If it is missing, the framework usesDefaultI18NProvider — and that one has a defect Part 1 already mentioned: it falls back to the JVM’s default language. On a German machine, every request for English then returns German text.

The application starts without complaint. The language switcher appears, it can be operated, and it changes nothing. Whoever discovers this in production searches for a long time.

It is the first of three values that get lost silently during the migration — Chapter 10 covers the other two.

setInitOrder(1)

The counterpart to<load-on-startup>1</load-on-startup>. Without it, the container creates the servlet only on the first request. The application still runs — but the first visitor pays for the entire initialization: building the Vaadin services, opening the datastore, and wiring the security layers.

For a service behind a reverse proxy this means: the first request after every restart runs into a timeout, and the access log records a 502 with no explanation. The number 1 moves this work into startup, where it belongs and wheresystemctl start waits for it.

AppServlet Stays Where It Is

What is remarkable is what doesnot appear here: no modified servlet class. It is the sameAppServlet as in the WAR, unchanged, from the core module.

Both delivery formats register the same class with the same parameters — one through a descriptor, the other through five lines of Java. The difference between Parts 2 and 3 is not in the application. It is in who starts it.

Production Resources

A WAR has a root directory. Everything inside it is served by the container. An application archive has none — and that is exactly where the migration fails most often.

Two Kinds of Files, One Problem

The application serves two kinds of static files:

The first kind was never a problem: Vaadin puts it in a place it finds again on its own. The second kind lived in the WAR root — and that root no longer exists.

The Solution Is in the Servlet Specification

An archive may contribute web resources if they sit underMETA-INF/resources. The container serves them as if they were in the web application’s root directory. That is precisely what this mechanism is for, and it applies to both delivery formats.

core/src/main/resources/META-INF/resources/
├── images/portrait-sven.jpg
└── icons/github-mark.svg, linkedin-mark.png

This keeps the references in the source code unchanged:

newImage("images/portrait-sven.jpg",…)

One source, two packagings, the same URLs. The WAR gets the files throughWEB-INF/lib/vaadinapp-core.jar, the embedded setup through the classpath.

The Four Lines Without Which It Is Not Enough

For the WAR, the container takes care of the rest. Embedded, the context must learn where its base is — andone lookup is not enough:

privateResourcewebResources(WebAppContextwebapp)throwsIOException{ResourceFactoryfactory=ResourceFactory.of(webapp);List<Resource>roots=Collections.list(Application.class.getClassLoader().getResources("META-INF/resources/")).stream().map(url->factory.newResource(url.toString())).filter(Objects::nonNull).toList();logger().info("Serving static web resources from {} classpath root(s)",roots.size());returnResourceFactory.combine(roots);}

getResources, plural, notgetResource. In the reference setup, this line reports at startup:

Serving static web resources from 5 classpath root(s)

Five, not one: the core module contributes images and icons, the Vaadin archives their themes and the client engine. Take only the first root, and you get an application in which either your own images or the Vaadin theme is missing — depending on which archive the class loader finds first.

The Failure Case, Demonstrated Once

The most expensive mistake in this part does not look like one at all. A build without the production profile:

mvn clean package# without -Pproduction

Afterwards, the built frontend is missing. The server starts anyway:

[main] INFO org.eclipse.jetty.server.Server - Started oejs.Server@… @1123ms

curl reports 200. The access log is unremarkable. In the browser the page stayswhite, and the only hint sits in the developer console: references toVAADIN/build/… that return 404.

That is whyApplication contains the check from Chapter 4:

privatestaticfinalStringBUNDLE_MARKER="META-INF/VAADIN/webapp/index.html";if(Application.class.getClassLoader().getResource(BUNDLE_MARKER)==null){abort("Vaadin production bundle missing from the classpath (expected "+BUNDLE_MARKER+"). Build with `mvn -Pproduction package` before starting; without it the "+"server starts, answers 200 and serves a blank page.");}

A mute browser error becomes a startup message with instructions for action, and the service terminates with an error code instead of appearing to run.

Five lines against a lost evening. Part 1, Chapter 4 described the same trap for the WAR; there it remained a warning. Here it is caught, because embedded there is nobody left who would complain.

Where the Bundle Is Built

One point that surprises when splitting into modules:build-frontend runsnot in the packaging module but in the core.

The Vaadin documentation is unambiguous here. The build generates the frontend files in the module that configures the plugin; withjar packaging they end up undertarget/classes/META-INF/VAADIN/webapp/ and thus travel into the core archive. Both packagings fetch them from there via the classpath.

The cross-check after the build:

unzip -l core/target/vaadinapp-core-*.jar| grep -c'META-INF/VAADIN/webapp'

137

Put the plugin into a packaging module instead, and you build the bundle where the other packaging cannot see it — and find the mistake only on the first page load.

Lifecycle

The server is assembled. What remains is the question of who stops it — and what happens when nobody thinks of it.

Serverserver=server(host,port);server.start();logger().info("Embedded Jetty serving http://{}:{}/",host,port);server.join();

start() Returns,join() Does Not

start() brings the server up and returns control. Without the last line,main() would then run to its end, the JVM would exit, and the service would be gone again after one second.

join() blocks until the server stops. This one line is the difference between a program that gets something done and a service that runs.

The message in between is not decoration. It is the only place where the journal records which address the service actually listens on — and thus the first thing you look for when Caddy reports 502.

Stopping Without Damaging the Datastore

The application keeps an Eclipse Store datastore open. A process that simply gets killed leaves it unchanged in the best case and incomplete in the worst.

server.setStopAtShutdown(true);

With this, Jetty registers its own shutdown hook with the JVM: onSIGTERM, the server shuts down in an orderly fashion, the contexts are stopped, and the application gets the opportunity to close its resources.

The counterpart in the systemd unit has been in place since Part 1:

KillSignal=SIGTERM
TimeoutStopSec=30

systemd sendsSIGTERM and waits thirty seconds before getting tougher. Both sides have to match — a service with a shutdown hook andKillSignal=SIGKILL would have the hook for nothing.

The Line That Was Not Needed in the WAR Model

SuccessExitStatus=143

143 is128 + 15, that is, “terminated by signal 15”. A service that shuts down properly onSIGTERM exits with exactly this value — and without this line, systemd would consider that a failure. WithRestart=on-failure, that would lead to a service that restarts itself after every planned stop.

The line dates from Part 1 and continues to apply unchanged. It appears here because it is easy to overlook in the embedded setup: whoever rewrites the unit instead of adapting it leaves it out and wonders.

Failure at Startup

}catch(Exceptione){logger().error("Failed to start embedded Jetty on {}:{}",host,port,e);stop(server);abort(null);}

stop(server) beforeabort(...): if the server has already claimed a port and something went wrong only afterwards, this call releases it. Without it, the port would stay occupied until the process ends — and on the automatic restart five seconds later, the next attempt would fail with aBindException whose cause then looks like an entirely different problem.

What the Module Mechanism Used to Take Care Of

The server runs, the application responds, the pages render. This is where a migration usually stops — and this is exactly where the most important part is still missing.

Part 2 spent two chapters setting up the external Jetty for operation behind Caddy. Both happened through modules:

sudo java -jar /opt/jetty/start.jar --add-modules=forwardedsudo java -jar /opt/jetty/start.jar --add-modules=ee11-websocket-jakarta

Embedded, there are no modules. Both have to go into the code — andboth fail silently if you forget them. No error, no log entry, no complaint.

First: The Request Does Not Come from Where It Comes From

Behind Caddy, every request reaches the application server via127.0.0.1, unencrypted, with a different hostname than the one the browser called. From the application’s point of view, this looks like local access over HTTP.

The consequence is not an error message but a wrong conclusion: Vaadin sets the session cookiewithout theSecure flag, because the connection appears to be unencrypted. A cookie without this flag may also be sent by the browser over unencrypted connections.

One line fixes this:

httpConfig.addCustomizer(newForwardedRequestCustomizer());

The proof takes two calls — without and with the header Caddy sets:

curl -s -D- -o /dev/null http://127.0.0.1:8080/login| grep -i set-cookie

set-cookie: JSESSIONID=node0…; Path=/

curl -s -D- -o /dev/null -H'X-Forwarded-Proto: https'\ http://127.0.0.1:8080/login| grep -i set-cookie

set-cookie: JSESSIONID=node0…; Path=/; Secure

Same server, same application, one difference in the result. Checked via the public address:

curl -s -D- -o /dev/null https://demo.svenruppert.com/login| grep -i set-cookie

set-cookie: JSESSIONID=node01ggmnynrf3f7q162xs5dytkyna5.node0; Path=/; Secure

Why Evaluating Them Is Defensible at All

Forwarding headers are claims made by the sender. Whoever evaluates them believes a stranger about where the request comes from and over which protocol.

What makes this defensible is a decision from Part 1, Chapter 8: the connector listens on127.0.0.1. The only party able to set these headers is thus a process on the same machine — the reverse proxy. If it were bound to0.0.0.0, anyone on the network could claim their request arrived over HTTPS from any address, and the application would write it into the log and into its access decisions.

In the external Jetty, these two decisions that belong together lived in two different.ini files. In the code, they are seven lines apart.

Second: Push Needs a Container That Nobody Brings Along

JakartaWebSocketServletContainerInitializer.configure(webapp,null);

Vaadin Push keeps a persistent connection open. For that, the servlet context needs WebSocket support, which is not present on its own in the embedded setup. The call must happenbefore the context starts; afterwards it is too late, and no message tells you so.

Without it, Vaadin falls back to long polling, or the connection fails. The application remains usable — server-initiated updates just no longer arrive, or arrive late.

The proof is a connection upgrade:

curl -s -o /dev/null -w'%{http_code}\n'\ -H'Connection: Upgrade' -H'Upgrade: websocket'\ -H'Sec-WebSocket-Version: 13' -H'Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ=='\'http://127.0.0.1:8080/VAADIN/push?v-r=push&X-Atmosphere-transport=websocket'

101

101 Switching Protocols — the upgrade succeeds. A 200 here would be the warning sign: then ordinary request processing would have answered, and the upgrade would not have happened.

Third: The Language Provider from Chapter 7

Thei18n.provider parameter belongs to the same family. It, too, used to live in a file the container read — theweb.xml —, its absence, too, produces no error, and its consequence, too, only shows in operation.

The Pattern Behind It

Three values, three failures without a symptom. That is no coincidence but the nature of configuration that exists as adefault: if it is missing, a default value applies, and default values are made for the simple case — directly reachable server, no encryption, no persistent connections, one language.

The embedded setup is therefore no less secure than the external one. It merely shifts where the decision lives: from a file someone once created into code that gets compiled. What is missing in both cases is feedback — which is why these three points are paired with the three checks from this chapter, and why those checks belong in the operating procedures and not only here.

Configuring the Embedded Server

Part 2, Chapter 6 established the rule: configuration does not belong in the artifact. It applies here unchanged — only the artifact is a different one.

Three Sources, Fixed Order

privatestaticStringresolve(StringsystemProperty,StringenvVariable,Stringfallback){StringfromSystem=System.getProperty(systemProperty);if(fromSystem!=null&&!fromSystem.isBlank())returnfromSystem;StringfromEnv=System.getenv(envVariable);if(fromEnv!=null&&!fromEnv.isBlank())returnfromEnv;returnfallback;}

The order is not arbitrary. The system property wins because it lives in the unit and thus under the server’s version control; the environment variable comes from a file that someone else can maintain as well; the default only applies where nobody said anything.

What Is Deliberately Absent Here

No configuration file format. It would be possible to read anapplication.properties. The reference setup does not, because every additional source makes the question “where does this value come from” harder to answer — and because systemd, withEnvironmentFile, already provides an answer that needs no library.

No reloading at runtime. The service reads its configuration at startup. Whoever changes it restarts — four seconds of downtime, measured in Chapter 12. An application that picks up configuration while running has to answer, for every value, what happens to the requests that are in flight at that moment. That is effort that does not pay off here.

The Difference from the WAR Model

Exactlyone:app.storage.dir. This property has been in the unit since Part 1, because the application’s default is a relative path andProtectSystem=strict makes the working directory read-only. Nothing changes here — the finding from Part 1 applies to both delivery formats alike.

Everything else — credentials viasystemd-creds,EnvironmentFile, file permissions — is carried over unchanged from Part 2. That is the actual message of this chapter: the operational framework does not care who starts the server.

Embedded Jetty Under systemd

The unit from Part 1 changesone directive. The rest stays, line for line — and that is the outcome the separation from Part 0 was aiming at.

Before and After

 ExecStart=/var/lib/vaadinapp/.sdkman/candidates/java/current/bin/java \- -Djetty.home=/opt/jetty \- -Djetty.base=/opt/vaadinapp \ -Djava.io.tmpdir=/var/lib/vaadinapp/work \ -Dapp.storage.dir=/var/lib/vaadinapp/data \ --add-exports java.base/jdk.internal.misc=ALL-UNNAMED \ --enable-native-access=ALL-UNNAMED \ -XX:MaxRAMPercentage=75 \ -XX:+ExitOnOutOfMemoryError \- -jar /opt/jetty/start.jar+ -cp /opt/vaadinapp/app.jar:/opt/vaadinapp/lib/* \+ com.svenruppert.flow.Application

Two lines go, two come in.Both hardening drop-ins remain untouched — the same directives, the same hardening score of 1.1.

The asterisk inlib/* is not a shell glob. systemd does not expand it, and that is a good thing: Java knows this notation in the classpath itself and resolves it at startup.

The Directory on the Server

/opt/vaadinapp/
├── app.jar 8,908 bytes — just the launcher class
├── lib/ 127 archives, 29.74 MB
├── logs/
└── releases/ archive of the shipped releases

The application archive is nine kilobytes. It contains one class. Everything else sits next to it — including the application’s core, as one of the 127 archives.

⚠️ The First Start Fails

It actually did fail, with a 503:

java.lang.RuntimeException: Unable to create FileSystem for /opt/vaadinapp/lib/._jspecify-1.0.0.jar
Caused by: java.util.zip.ZipException: zip END header not found

._jspecify-1.0.0.jar — with a leading dot and underscore. That is not a library but a companion file macOS creates for files with extended attributes. When packing withtar, they travel into the archive; the hint is already there at unpack time:

tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'

Eleven such files ended up inlib/. And Java’s classpath wildcardlib/* picks upevery file ending in.jar — it checks nothing, it only expands.

The remedy when packing:

COPYFILE_DISABLE=1 tar czf paket.tgz app.jar lib

or on the target system:

find /opt/vaadinapp -name'._*' -delete

Why this could not happen with the WAR: a WAR is a single archive written by Maven. Only a classpath built from a directory makes the application depend on that directory containing nothing but valid archives. The delivery format brings a new source of error that has nothing to do with Jetty.

⚠️ Why There Is Nojava -jar

The obvious idea: useaddClasspath to write aClass-Path into the application archive’s manifest, andjava -jar app.jar will do.

That works —apparently. The class loader resolves the archives, the server starts, the application responds. Every request, however, then ends with:

java.lang.NullPointerException: Cannot invoke
"com.vaadin.flow.server.StaticFileHandler.serveStaticResource(…)"

The reason sits one level deeper. Withjava -jar, thejava.class.path property containsa single entry — the application archive. The manifest’sClass-Path entries are not in it. And this is precisely the property Jetty’sMetaInfConfiguration reads to decide which archives the annotation scanner opens.

No scan, noRouteRegistryInitializer, noStaticFileHandler. The server is healthy; the servlet is not.

The telltale measurement is the startup time. With the identical 127 archives:

The missing second is the scan. Whoever does not know the difference takes the faster start for a win.

That is why the manifest deliberately carriesnoClass-Path, and the launch command is explicitly:

java -cp'app.jar:lib/*' com.svenruppert.flow.Application

The Teardown

After the switch, the Jetty installation becomes superfluous:

sudo rm -f /opt/jetty# Symlinksudo rm -rf /opt/jetty-home-12.1.12# the installationsudo rm -rf /opt/vaadinapp/{start.d,webapps,resources,environments}

What is remarkable about the 53 MB: the distribution contains232 archives, of which26 were on the classpath in operation. That is not a defect but the way a server distribution is built — it ships all modules, some get activated. Embedded, this reserve disappears, because the dependencies are named.

Afterwards,/opt contains nothing butvaadinapp.

Caddy Remains Unchanged

The shortest chapter of this part, and one of the most telling.

The File

demo.svenruppert.com {
reverse_proxy 127.0.0.1:8080
…
}

Unchanged. No restart, no reload, no adjustment. Caddy talks to127.0.0.1:8080 and does not know what runs behind it — previously an installed servlet container with an archive inside, now an application with its own server.

Why This Cannot Be Taken for Granted

Because it could easily have gone differently. The embedded setup could have

• listened on a different port,
• bound to0.0.0.0,
• ignored the forwarding headers,
• refused WebSocket connections.

In all four cases Caddy would have needed adjustment, or the application would have worked incorrectly. That none of this was necessary is the result of the decisions from Chapters 5, 7, and 10 — they are all aligned to offer the same interface as before.

The Cross-Check

curl -s -o /dev/null -w'%{http_code}\n' https://demo.svenruppert.com/curl -s -D- -o /dev/null https://demo.svenruppert.com/login| grep -i set-cookiecurl -s -D- -o /dev/null https://demo.svenruppert.com/| grep -i strict-transport

200
set-cookie: JSESSIONID=node01ggmnynrf3f7q162xs5dytkyna5.node0; Path=/; Secure
strict-transport-security: max-age=31536000; includeSubDomains

The same three lines as at the end of Part 2.

The Boundary That Becomes Visible Here

Caddy and the application have a contract: HTTP on127.0.0.1:8080, with forwarding headers. As long as both sides keep it, each side is replaceable for the other.

That is why the series chose this order. Had the reverse proxy not already been set up and described in Part 2, this part would have to cover it as well — and the claim “none of this changes” would not be verifiable, because there would be no before.

Security Implications

The switch shifts a responsibility, and an uncomfortable one at that.

The Jetty Version Now Belongs to the Application

Before: when a security advisory for Jetty appears, someone downloads the new distribution, moves the symlink, restarts the service. The application is not touched, not rebuilt, not retested.

After: the same advisory means changing a number in thepom.xml, rebuilding, running the test suite, and shipping a release.

The last row is the dangerous one. With the external Jetty, the installation sits visibly under/opt/jetty; it shows up in every inventory. Embedded, Jetty is one line among 127 in a dependency tree. A server whose operating system is well maintained can contain a three-year-old Jetty release without anyone noticing.

What helps against this is not a decision about the setup but one about the process: a dependency check in the build that reports known vulnerabilities. The project generates a bill of materials (CycloneDX-SBom) with every build — so the foundation for that is in place.

What Gets Better

There is no second release anymore. Part 1, Section 4.5 described the case of a WAR bringing its own Jetty, with the installed release shadowing it. As long as the two match, all goes well; after that, the wrong class wins at some unshielded spot. This source of error disappears: there is only one Jetty left, and it is recorded in thepom.xml.

What was built is what runs. A WAR is an incomplete artifact — it needs a container of matching release with matching modules. A classpath is complete. What starts on the development machine also starts on the server, because both use the same list.

The server’s attack surface shrinks. 71 MB less installed software, 232 fewer archives on disk, no deployment directory that a process with write permissions could fill. The teardown from Chapter 12 is not just tidying up.

What Stays the Same

The hardening score.systemd-analyze security vaadinapp reports1.1 before and after, with the same directives in the same drop-in.

That is expected and still worth mentioning: hardening describes what the processmay do — which directories it may write, which system calls it may issue, which privileges it may escalate. These questions are independent of who starts the server inside the process. A hardening score that changed with the launch method would have been attached at the wrong place.

External Versus Embedded Jetty

Both setups ran on the same machine, with the same application, behind the same Caddy. What follows is measured, not estimated.

The Numbers

Each value is the mean of three runs. Individual downtime values for the embedded setup: 4,517, 4,868, and 4,670 ms.

The Row That Contradicts Expectation

Part 2, Chapter 8 ended with an observation: an archive smaller by a quarter hadnot reduced memory consumption. The explanation back then: what was removed had never been loaded in the first place.

Here it is the other way around, and the explanation is the same. The deliverygrows from 21.78 to 29.75 MB, the classpath lengthens from 112 to 127 archives — and memory consumption still drops by 112 MiB.

Both times, what was measured was not the amount of bytes shipped but the amount ofloaded classes. The external Jetty brings its module system, its deployment scanner, and its own startup machinery; the embedded setup loads only what the application calls.

Whoever takes an artifact’s size as a measure of its resource consumption is wrong in both directions.

What the Numbers Do Not Show

Downtime drops by roughly 780 ms, which is less than the elimination of unpacking would suggest. The reason is in the measurement from Part 2: over forty percent of the startup time goes to Vaadin initialization — to the application itself. Changing the launch mechanism cannot change that.

Whoever expects a dramatically faster start from embedded operation will be disappointed. Four seconds remain four seconds.

Responsibility, Maintenance, Tooling

The second-to-last row is often overlooked. An installed Jetty can serve several archives at once. Whoever runs three small applications on one server pays the base footprint of a JVM three times when embedded. With one application per server — the case of this series — that is not an argument.

What Stays the Same in Both Cases

Directory layout, service account, hardening, journald, configuration outside the artifact, credentials viasystemd-creds, Caddy, update and rollback.Everything Parts 0 and 2 built survives the switch unchanged.

That is not a side note but the argument for the order of this series: whoever builds the operational framework first and keeps it independent of the delivery format can swap that format later without touching everything else again.

When Does Your Own Embedded Jetty Make Sense?

The answer hangs on a single question:who maintains the server?

In Favor

When the same person is responsible for the application and the server. Then the separation between the two is artificial, and a number in thepom.xml is the shorter path than a symlink on a machine.

When the application ships frequently. A security update for Jetty is then no special case; it travels with the next release anyway.

When the target server is supposed to contain as little as possible. A JVM suffices. No server installation, no deployment directory, no module configuration — 71 MB less and one software component fewer that someone has to keep an eye on.

When the setup is supposed to be reproducible. A classpath is complete. A WAR is not: it presupposes a container of matching release with matching modules, and this precondition is recorded nowhere in the artifact.

Against

When several applications run on one server. An installed Jetty serves several archives from one JVM. Embedded, every application pays its own base footprint — with three small applications, that is the decisive point.

When the server is maintained by someone else. Then the separation is not an inconvenience but a boundary of responsibility. A maintainer who can update Jetty without having every application rebuilt is an advantage, not an obstacle.

When nobody checks the dependencies. An embedded server disappears into the dependency tree. Without a check in the build that reports known vulnerabilities, the switch is a step backwards — not technically, but organizationally.

When the application needs container features that otherwise come for free: JNDI resources, several contexts, a deployment directory for third-party archives. All of it can be rebuilt in an embedded setup, and none of it is worth the effort when it is available ready-made.

The Honest Summary

For the case of this series — one application, one server, one responsible person, regular releases — the embedded setup is the better fit. It is not faster in operation and not leaner as an artifact; it ismore complete, and it moves a decision to where it is being made anyway.

For a server hosting several applications from several teams, the answer is the reverse. Both setups are correct; they answer different questions.

Outlook on Part 4

The boundary has shifted for the first time. Jetty now belongs to the application. What is still open lies in plain sight on the server.

What This Part Cost

The bootstrap from Chapters 4 through 10 comes to about 60 lines — and three of them are the ones you would not write without this text:

• ForwardedRequestCustomizer, otherwise the session cookie loses itsSecure flag,
• the WebSocket initialization, otherwise Push fails,
• thei18n.provider parameter, otherwise the language switcher has no effect.

None of these three failures announces itself. Whoever writes the bootstrap themselves takes on a duty of care that previously sat in sixteen module files maintained by someone else.

What Still Lies Next to It

/opt/vaadinapp/
├── app.jar 8,908 bytes — one class
└── lib/ 127 archives, 29.74 MB

An application archive of nine kilobytes and a directory with 127 libraries. The launch command names both:

java -cp'app.jar:lib/*' com.svenruppert.flow.Application

That works, and it is inconvenient. A deployment consists of two parts that belong together and lie apart; a rollback has to wind both back; and whoever passes the application on passes on a directory, not an artifact.

What Part 4 Makes of It

Bootstrap and packaging are two different decisions. This part answered the first: who starts the server? Part 4 answers the second: how does the result get onto the target machine?

Two answers stand against each other there:

The chapters there: how a fat JAR is built, where it causes problems — Chapter 12 of this part has already given a foretaste —, what a thin distribution gains, and what releases with symlink and rollback look like on top of it.

The choice is not obvious. A fat JAR is one file and therefore convenient; but it turns every update of a single library into a complete rebuild, and it has a known breaking point inMETA-INF/services. A thin distribution stays decomposable, but demands a directory that has to be correct — as Chapter 12 showed with eleven wrong files.

The Boundary Keeps Moving

At the end of Part 6, the application runs on a server with no Java installed.

Why the intermediate state deliberately looks like this: a fat JAR would have been possible in this part. It would have obscured the question of what distinguishes bootstrap from packaging — and precisely this distinction is the first sentence of Part 4.

https://svenruppert.com/posts/vaadin-deployment-on-hetzner-part-5-deliver-switch-compare/Thu, 24 Sep 2026 10:00:00 +0200sven.ruppert@gmail.com (Sven Ruppert)Sven Rupperthttps://svenruppert.com/posts/vaadin-deployment-on-hetzner-part-5-deliver-switch-compare/Shipping, switching, and comparing releases of a Vaadin application: rollback as a feature, symlink switching, and honest measurements of both delivery forms.<![CDATA[

Fifth part of the seriesVaadin – Deployment on Hetzner. The two packagings from Part 4 go onto the server: releases in numbered directories, acurrent symlink, a rollback in two lines — and the measurement that decides between them. All values come from an actual run on a Debian 13 server.

The State After Part 4

Part 4 built two packagings of the same application. What now sits in the build directory:

codeCopy

embedded-jetty/target/
├── vaadinapp-00.01.00-thin.tar.gz 130 files, 127 archives
└── vaadinapp-00.01.00-fatdist.tar.gz 3 files, 1 archive with 17,904 entries

Both share the same shape —VERSION,bin/,app/, pluslib/ for the Thin Distribution — andthe same start script, byte-for-byte identical. It putsapp/*:lib/* on the classpath; iflib/ is missing, the second half resolves to nothing, and the one complete archive satisfies the annotation scan.

This sameness is no accident but the precondition for everything this part does. Because the releases share the same shape, the systemd unit does not distinguish them. And because it does not distinguish them, switching between the forms isthe same operation as any release switch: a symlink and a restart.

What Part 4 Left Open

Figure 1: Both forms as releases of the same shape — only lib/ distinguishes them.

Two things.

The delivery. A directory cannot be copied like a file. What comes into being on the server comes into being from 128 files, and there is no way to replace 128 files in a single step.

The answer. The original title of this part was “Fat JAR or Thin Distribution?” — the question is answered only here, with measurements from the reference server. This much can be said in advance: one of the two metrics I had previously assumed to be equal is not.

A Promise from Part 1

Part 1, step 29 explicitly decided against the symlink for updates — releases are archived, the WAR is copied. Alongside that stood the sentence that the symlink approach would come later, “where it unfolds its purpose with the directory distribution.” Part 3, chapter 17 repeated it.

Chapter 4 redeems that promise — and explains why it would indeed have gained nothing for the WAR.

The Code State for This Part

The same as for Part 4: tagteil-04. The split into two articles concerns the text, not the repository —git diff teil-03 teil-04 shows both together.

bashCopy

git clone https://github.com/svenruppert/Blog-Vaadin-Deployment-on-Hetznercd Blog-Vaadin-Deployment-on-Hetznergit checkout teil-04

At tagteil-04, no bill of materials ships inside the release yet. The project does generate an SBOM, but it stays in the build directory. Why that is a deficiency and how it gets fixed is the subject of a separate article.

Building the Distribution with Maven

The components become a deliverable. For that, a second step joinscopy-dependencies from Part 4, chapter 8.

Assembly — the Right Tool Here

Themaven-assembly-plugin was eliminated as a route to the fat JAR in Part 4, chapter 3, because itsjar-with-dependencies descriptor does not merge service providers. For adirectory, this problem does not arise: nothing is merged, because nothing is flattened. Every archive remains an archive with its ownMETA-INF.

The descriptor lives atsrc/main/assembly/thin.xml:

xmlCopy

<assembly><id>thin</id><formats><format>dir</format><format>tar.gz</format></formats><includeBaseDirectory>true</includeBaseDirectory><baseDirectory>vaadinapp-${project.version}</baseDirectory><fileSets><fileSet><directory>${project.basedir}/src/main/dist/bin</directory><outputDirectory>bin</outputDirectory><fileMode>0755</fileMode></fileSet></fileSets><files><file><source>${project.basedir}/src/main/dist/VERSION</source><outputDirectory>.</outputDirectory><filtered>true</filtered></file><file><source>${project.build.directory}/${project.build.finalName}.jar</source><outputDirectory>app</outputDirectory><destName>vaadinapp.jar</destName></file></files><dependencySets><dependencySet><outputDirectory>lib</outputDirectory><useProjectArtifact>false</useProjectArtifact><scope>runtime</scope></dependencySet></dependencySets></assembly>

Four places carry a decision.

dir andtar.gz at the same time. The directory afterwards sits intarget/ and can be transferred withrsync or inspected directly. The.tgz is the transport form for everything else. Both come from the same descriptor so they cannot drift apart.

tar.gz, notzip. A ZIP archive knows no execute permission in the POSIX sense. The start script would arrive without itsx bit, and the service would not start. For the case of long file names among the dependencies,<tarLongFileMode>posix</tarLongFileMode> is set in the plugin configuration — the defaultustar aborts on paths longer than 100 characters.

useProjectArtifact=false. Without this line, the application archive would end up inlib/, among the dependencies. Instead it is placed intoapp/ by hand — exactly the separation from Part 4, chapter 7.

filtered on theVERSION file. Maven substitutes${project.version} at build time. The same number thus appears in the archive and in the directory name without having to be maintained twice.

What Comes Out

bashCopy

$ mvn -Pproduction,thin package$ ls -l embedded-jetty/target/26676831 vaadinapp-00.01.00-thin.tar.gz10686 vaadinapp-embedded-jetty-00.01.00.jar vaadinapp-00.01.00-thin/$ find embedded-jetty/target/vaadinapp-00.01.00-thin -maxdepth2 -not -path'*/lib/*' vaadinapp-00.01.00-thin/vaadinapp-00.01.00 vaadinapp-00.01.00-thin/vaadinapp-00.01.00/VERSION vaadinapp-00.01.00-thin/vaadinapp-00.01.00/bin vaadinapp-00.01.00-thin/vaadinapp-00.01.00/app vaadinapp-00.01.00-thin/vaadinapp-00.01.00/lib$ ls -l …/bin/start.sh -rwxr-xr-x2608 start.sh

The execute permission is preserved. The proof that the delivery is complete is starting it from the unpacked directory:

codeCopy

[main] INFO Serving static web resources from 5 classpath root(s)
[main] INFO Started oejs.Server@17bffc17{STARTING}[12.1.12,sto=0] @1709ms
[main] INFO Embedded Jetty serving http://127.0.0.1:8099/

HTTP 200 on/ and/login, five resource roots, 1,709 milliseconds until the server is up. The last value is the telling one: it is in the same order of magnitude as the 1,523 ms from Part 3 and thus far above the signature of a scan that did not run.

The Same Shape for the Other Form

Thefatjar profile uses a second descriptor,fat.xml, which is identical except for two lines: it takes the shaded archive instead of the application archive, and it hasnodependencySet — there is nothing to copy intolib/, because everything already sits in the one file.

xmlCopy

<files><file><source>${project.build.directory}/vaadinapp-${project.version}-fat.jar</source><outputDirectory>app</outputDirectory><destName>vaadinapp.jar</destName></file></files><!-- no dependencySet -->

ThefileSet forbin/ stays unchanged —the same start script, the same file. With that, both forms share the same shape:

codeCopy

vaadinapp-00.01.00/ vaadinapp-00.01.00/
├── VERSION ├── VERSION
├── bin/start.sh ├── bin/start.sh
├── app/vaadinapp.jar └── app/vaadinapp.jar
└── lib/ 127 archives

This is not an end in itself. Because the script putsapp/*:lib/* on the classpath and a missinglib/ resolves to nothing, the same file starts both forms. The systemd unit therefore does not distinguish them, and switching from one to the other is a symlink and a restart — which is what allows chapter 6 to measure them under equal conditions in the first place.

Two Profiles, One Module

Both packagings are profiles of the same module, for the reason from Part 4, chapter 1:

bashCopy

mvn -Pproduction,thin package# bin/, app/, lib/, and a .tgzmvn -Pproduction,fatjar package# bin/, app/, and a .tgz, without lib/mvn package# just the application archive

-Pproduction is required in both cases. Without this profile,build-frontend does not run, the production bundle is missing, and the bootstrap aborts with its own check — instead of offering a server that serves an empty page.

The third invocation without a profile is not an oversight. It produces an application archive for development in seconds, without copying 127 files or writing a 28 MB archive. Whoever builds the full deliverable on every development cycle gets used to waiting times nobody needs.

Delivering a Thin Distribution

Delivering a directory is more cumbersome than delivering a file. This chapter shows what the complications are and which of them can be resolved.

Restructuring the Target Directory

After Part 3, the reference server carries a flat layout:app.jar andlib/ directly under/opt/vaadinapp. Before releases can sit side by side, this has to become the structure from chapter 4:

bashCopy

sudo install -d -m750 -o root -g vaadinapp /opt/vaadinapp/releases

The existing state is not converted into a release. It comes from a delivery withoutbin/ and withoutVERSION and could not be started with the newExecStart. It stays in place as the last path of retreat until the first release runs, and is removed afterwards.

The Transfer

Two routes are available, and they differ in more than notation.

As an archive:

bashCopy

scp embedded-jetty/target/vaadinapp-00.01.00-thin.tar.gz\ sven@demo.svenruppert.com:/tmp/ssh sven@demo.svenruppert.com' STAMP=$(date -u +%Y-%m-%d-%H%M) sudo install -d -m 750 -o root -g vaadinapp /opt/vaadinapp/releases/$STAMP sudo tar -xzf /tmp/vaadinapp-00.01.00-thin.tar.gz \ -C /opt/vaadinapp/releases/$STAMP --strip-components=1 sudo chown -R root:vaadinapp /opt/vaadinapp/releases/$STAMP sudo find /opt/vaadinapp/releases/$STAMP -type f -exec chmod 640 {} + sudo chmod 750 /opt/vaadinapp/releases/$STAMP/bin/start.sh'

Withrsync:

bashCopy

rsync -a --delete\ embedded-jetty/target/vaadinapp-00.01.00-thin/vaadinapp-00.01.00/\ sven@demo.svenruppert.com:/tmp/release/

The archive transfers everything every time.rsync transfers what has changed — and for this form, that is the essential point, which chapter 6 quantifies in bytes. For thefirst release this makes no difference; for every subsequent one it does.

In return, the archive route has a propertyrsync does not: the transfer is one file and therefore atomic. What comes into being on the server comes into being only on unpacking, and that happens alongside the running operation. This combination — transfer an archive, operate a directory — is the third option from chapter 7.

Two Pitfalls Part 3 Learned the Hard Way

AppleDouble files. macOS’star creates companion files following the pattern._name for files with extended attributes and takes them along. Eleven of them ended up inlib/ in Part 3, and Java’s classpath wildcard picks up every file with the.jar extension — including these, which are not valid archives. The first start ended with HTTP 503.

bashCopy

COPYFILE_DISABLE=1 tar czf paket.tgz app lib bin

With delivery via themaven-assembly-plugin, this no longer arises: the archive is created inside the JVM from the files of the build directory, not via macOS’star. The pitfall concerns manual operation, and that does occur when individual files are updated by hand.

The permissions on the start script. Azip would lose the execute permission; achown -R followed by a blanketchmod would as well. That is why the commands above contain a dedicated line forbin/start.sh — and why chapter 2 builds the deliverable astar.gz with an explicit<fileMode>0755</fileMode>.

The Smoke Test

Before the symlink is repointed, the new release can be checked without touching the running operation:

bashCopy

sudo -u vaadinapp\JAVA_HOME=/var/lib/vaadinapp/.sdkman/candidates/java/current\JAVA_OPTS='-Dapp.port=8099 -Dapp.storage.dir=/tmp/probe'\ /opt/vaadinapp/releases/$STAMP/bin/start.sh

A second port, a different data directory, the same service user. Two indications from the log decide:

• Serving static web resources from 5 classpath root(s) — the resources are complete,
• a startup time in the range of seconds — the annotation scan took place.

A startup time in the double-digit millisecond range means the opposite, regardless of whether the server responds. This signature has now appeared twice: in Part 3 through an incomplete classpath, in Part 4, chapter 5 through a destroyed service provider directory.

Only once this check passes does the switch from chapter 4 follow.

Releases and thecurrent Symlink

Part 1 explicitly decided against the symlink for updates: releases are archived, the WAR is copied. Alongside that stood the sentence that the symlink approach would come in Part 4, “where it unfolds its purpose with the directory distribution.” This chapter redeems that promise — and begins with the question of why it wasnot worthwhile back then.

Why the Symlink Was Mere Decoration for the WAR

A WAR is a file.mv replaces a file atomically: either the old directory entry points to the old content or the new one to the new; there is no state in between. A symlink would merely have wrapped the same atomicity a second time.

A Thin Distribution consists of 128 files.There is no way to replace 128 files in a single step. Anrsync over the live tree leaves behind, for the duration of the transfer, a classpath in which some archives are new and others old. Whether that goes well depends on whether, during that window, a class is loaded that exists in both versions.

This is where the symlink earns its place: itrelocates the atomicity from the files to the name.ln -sfn rewrites a single directory entry — all or nothing.

The Directory Layout

codeCopy

/opt/vaadinapp/
├── current -> releases/2026-09-09-1200
├── releases/
│ ├── 2026-09-08-1730/{VERSION,bin,app,lib}
│ └── 2026-09-09-1200/{VERSION,bin,app,lib}
└── logs/

Timestamp instead of version number. The directories are named after the moment of delivery, not after the version. They therefore sort themselves and remain unambiguous when the same version is delivered twice — which happens regularly when a configuration is updated. Which version a release contains is stated in itsVERSION file.

logs/ and the data live outside. Logs and/var/lib/vaadinapp/data outlive the release switch. If the data directory sat undercurrent, a rollback would lose the data — a routine move would turn into an incident.

Permissions as with the WAR:root:vaadinapp, mode750. The service may read, not write. A compromised process can thus neither modify its current release nor write itself a next one.

The Switch

bashCopy

STAMP=$(date -u +%Y-%m-%d-%H%M)sudo install -d -m750 -o root -g vaadinapp /opt/vaadinapp/releases/$STAMPsudo tar -xzf /tmp/vaadinapp-00.01.00-thin.tar.gz\ -C /opt/vaadinapp/releases/$STAMP --strip-components=1sudo chown -R root:vaadinapp /opt/vaadinapp/releases/$STAMPsudo ln -sfn releases/$STAMP /opt/vaadinapp/currentsudo systemctl restart vaadinapp

Six lines, two of which constitute the actual switch. Everything before that happensalongside the running operation: the new release is laid down in full while the application keeps running from the old one.

ln -sfn with both switches is no ornament here. Without-n,ln follows the existing symlink and creates the new oneinside the target directory — the result isreleases/2026-09-08-1730/2026-09-09-1200, andcurrent still points to the old release. The command reports no error.

What Happens to the Running Process When the Symlink Is Repointed: Nothing

This is the property the whole procedure rests on, and it can be demonstrated. Two releases side by side,current points to the first, the application is running. Then the symlink is repointed,without a restart:

codeCopy

$ ls -l current
current -> releases/2026-09-09-1200
$ curl -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8095/
200
$ ln -sfn releases/2026-09-09-1400 current
$ ls -l current
current -> releases/2026-09-09-1400
$ curl -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8095/
200
$ lsof -p 71775 | grep -c 2026-09-09-1200
139
$ lsof -p 71775 | grep -c 2026-09-09-1400
0

The process keeps serving requests and holds139 open files in the old release, not a single one in the new. The reason lies below Java: a path is resolved on open, not on every access. Whatever happens to the name afterwards no longer concerns the open descriptor.

Two things follow from this at once:

• The switch needs a restart. Without one, the old version would keep running indefinitely while the symlink claims otherwise.
• The rollback is safe. The old release sits untouched on disk — not as an archive that would first have to be unpacked, but ready to start.

Cleanup

At roughly 30 MB per state,releases/ grows faster than one notices. The last five are enough:

bashCopy

ls -1d /opt/vaadinapp/releases/*/| sort| head -n -5|\ xargs -r sudo rm -rf

This line belongs at theend of the rollout, not at its beginning: first the new release is proven to run, then old ones are cleared away. The other way around, you would be left without a rollback target in case of failure.

Rollback

The rollback is the reason the procedure of the previous chapter is operated at all. It consists of two lines:

bashCopy

sudo ln -sfn releases/2026-09-08-1730 /opt/vaadinapp/currentsudo systemctl restart vaadinapp

This is the same motion as the switch, just with a different target. There is no separate procedure for the failure case, no unpacking from an archive, no restoring from a backup.

Measured on the reference server — from repointing the symlink to the first request answered again:4,938 milliseconds. That is the same order of magnitude as an ordinary restart (4,653 ms on average), because nothing else takes place. The duration does not depend on how large the application is or how far back the rollback reaches.

Why That Is Enough

Because the old release was never touched. Chapter 4 showed that a running process holds its files through open descriptors and notices nothing when the symlink is repointed — 139 open files in the old release, none in the new. The flip side of the same property is that the old release remainsready to start as long as it is not cleared away.

The time to recovery therefore does not depend on the size of the application, only on the duration of a restart. That is the real gain over the procedure from Part 1, where a WAR had to be copied back from the archive.

What the Rollback Does Not Take Back

Three things, and all three need to be considered beforehand.

The sessions. They live in memory, as they have since Part 2. A restart discards them, and logged-in users land on the login page — on a switch as on a rollback. This is not a property of this procedure but a consequence of how sessions are held, and it could only be avoided with persistent sessions.

The data./var/lib/vaadinapp/data lives outside the releases and is not reset. That is intentional: a rollback is meant to take back the application, not the users’ work. It does mean, however, that a release that changed the data — a migration — cannot be taken back this way. The symlink makes the code reversible, not the schema.

The unit. It lives under/etc/systemd/system and belongs to the machine. This is exactly why the cut from Part 4, chapter 9 is placed where it is: because the release’s JVM options live inside the release, the rollback brings them along. If they lived in the unit, someone would have to remember to take them back as well — and would forget in the heat of a failure.

The Procedure When It Gets Serious

bashCopy

# 1. Which releases are available?ls -1 /opt/vaadinapp/releases/# 2. What does current point to right now?readlink /opt/vaadinapp/current# 3. Which version is in the target?cat /opt/vaadinapp/releases/2026-09-08-1730/VERSION# 4. Switch back and restartsudo ln -sfn releases/2026-09-08-1730 /opt/vaadinapp/currentsudo systemctl restart vaadinapp# 5. Verify, do not assumesystemctl is-active vaadinappcurl -s -o /dev/null -w'%{http_code}\n' https://demo.svenruppert.com/journalctl -u vaadinapp -n20 --no-pager

Step 3 is the one that gets skipped, and the one that justifies theVERSION file from Part 4, chapter 7. The directory names are timestamps; which of them contains the version to roll back to, only this file can say.

Step 5 is the one that runs through the whole series: the symlink can point correctly and the service still fail to run, for instance because the old release has a start script that expects an environment variable removed in the meantime. A rollback is proven only when a request is answered.

The Measured Comparison

Unlike in Parts 2 and 3, it is not the server that changes here. Both forms load the same classes from the same archives with the same bootstrap. A table in which half the rows show identical numbers proves nothing — which is why there are five rows, and two of them have not appeared in this series before.

Measurements were taken on the reference server, as in all previous parts: Debian 13, Temurin 26, the same unit, the same hardening. Both forms sit side by side as releases; switching between them is a symlink and a restart — only that makes the numbers comparable.

The hypotheses were written downbefore the measurement. Two of them were not confirmed, and both times the opposite is the more instructive result.

Delivery Size

Both forms are delivered as releases and transferred as.tgz for that purpose — same structure, same transport form, so the numbers are comparable.

Form	Size
Fat distribution as.tgz	25,807,851 Bytes
Thin distribution as.tgz	26,676,831 Bytes
raw fat JAR, unpackaged	28,890,183 Bytes
Thin distribution unpacked	28.4 MB in 128 files

The hypothesis was: the fat JAR is somewhat smaller, because 127 archives each carry their own central directory.Confirmed — the fat form is smaller by 868,980 bytes, or 3.3 percent.

The third row deserves a warning, because it invites a fallacy. Compare theraw fat JAR with the.tgz of the Thin Distribution and the result reverses: 28.89 versus 26.68 MB. But that is not a comparison of two delivery forms; it is a comparison of once-compressed and twice-compressed data. A JAR compresses each entry on its own;gzip over thetar stream compresses once more on top and exploits repetitions across file boundaries. Whoever measures this way measures the packaging of the packaging.

The amount is small in both directions. For the choice between the forms, size yields nothing — it is listed here mainly because the obvious mismeasurement leads so easily to a false statement.

Startup Time

Three runs per form on the reference server, Jetty’s own timestamp up toStarted Server, plus the downtime fromrestart to the first answered request:

	Thin Distribution	Fat JAR
Uptime, individual runs	3,621 / 3,563 / 3,538 ms	3,641 / 3,319 / 3,487 ms
Uptime, mean	3,574 ms	3,482 ms
Downtime, individual runs	4,747 / 4,671 / 4,540 ms	4,754 / 4,365 / 4,618 ms
Downtime, mean	4,653 ms	4,579 ms

The hypothesis was: the fat JAR starts faster, because Jetty’sMetaInfConfiguration opens one archive instead of 127.Not confirmed. The difference is 92 milliseconds against a spread of 83 and 322 milliseconds, respectively, within the runs — the ranges overlap completely. The same measurement on the development machine yielded 1,654 versus 1,671 milliseconds, that is, the opposite direction at an equally small amount.

That is the more instructive finding. It sayswhere the time actually goes: not into opening the archives, but into the ASM pass over the class files inside them. Their number is the same in both forms. Creating 127 file handles carries no weight next to that.

This also puts the number from Part 3 into perspective. The 139 milliseconds versus 1,523 milliseconds there werenot the price of 127 opened archives, but the difference between a scan that happens and one that does not. Whoever derives from it the expectation that fewer archives would bring a faster start has misread the measurement — and this chapter is the cross-check on that.

Memory Footprint

This is where the most definite hypothesis stood, and it is the second one to fall.

	Thin Distribution	Fat JAR
MemoryCurrent, individual runs	358 / 353 / 362 MiB	407 / 403 / 408 MiB
MemoryCurrent, mean	358 MiB	406 MiB
RSS	382 MiB	430 MiB
Threads	31	31

The hypothesis was: no difference — same classes, same bootstrap.Refuted. The fat JAR needs48 MiB more, which is 13 percent. The values per form sit close together (358 to 362 versus 403 to 408); the ranges do not touch. That is not noise.

For context: Part 3 measured 336 MiB for the flat layout; the Thin Distribution as a release, at 358 MiB, is in the same order of magnitude.

Where the 48 MiB Come From — and Where They Do Not

The obvious explanation would be that a 28 MB archive is mapped into memory as a whole. A look at/proc/<pid>/maps refutes that:

codeCopy

 Thin Fat
file-backed mappings
on .jar files 0 0
mappings total 332 331
open archive descriptors 139 4

Not a single archive is memory-mapped — in neither form. The JVM reads the archives’ directories into ordinary memory; it does not map the files.

/proc/<pid>/smaps_rollup shows where the difference sits instead:

codeCopy

 Thin Fat
Rss 384 MiB 436 MiB
Private_Dirty 358 MiB 410 MiB ← the entire difference
Private_Clean 23 MiB 23 MiB
Shared_Clean 2 MiB 2 MiB

The increase sitsentirely in anonymous, dirty memory — heap or natively requested JVM memory, not in file mappings and not in the page cache.

This narrows the cause down, but does not determine it. What it is not is stated above; what it is, this measurement does not yield. The number was taken ten seconds after startup in each case, that is, before garbage collection has settled — a larger retained set immediately after the scan is conceivable and would have to be examined separately.

What the finding saysfor certain: the statement from Part 3 — that what is measured is not the number of bytes shipped but the number of loaded classes — falls short. Here the classes are the same, and the footprint differs anyway. The form of delivery is not without consequence for memory.

Change Volume for a Patch Release

The first of the two new rows — and the only one with a clear difference.

Of the 127 archives inlib/,126 are byte-identical to the files in the local Maven repository. They come from there unchanged and do not change between two builds. Only the project’s own archives are rewritten.

So if only the application code changes:

Form	To transfer
Thin Distribution	app/vaadinapp.jar (10,686 B) +lib/vaadinapp-core-…jar (2,590,109 B) =2,600,795 B
Fat JAR	the entire archive =28,890,183 B
	Ratio 11.1 to 1

rsync transfers even less in both cases, because it matches blocks within changed files. That changes nothing about the ratio: with the fat JAR,every change is a change to the same 28 MB file.

A note on this application: the 2.59 MB is thecore module, which contains the entire application code and the Vaadin production bundle. In a project without this separation, the application archive would stand here alone — ten kilobytes. The ratio would then not be 11 to 1 but 2,700 to 1.

Diagnostic Effort

The second new row, without a number, but with a demonstrable difference.

bashCopy

# Thin Distribution$ ls /opt/vaadinapp/current/lib| grep bcprovbcprov-jdk18on-1.84.jar# Fat JAR$ unzip -l /opt/vaadinapp/current/app.jar| grep -i bouncycastle(nothing — the classes live under org/bouncycastle/, the version nowhere)

The difference is not that the second question is harder to answer. Itcannot be answered. The classes are there; the provenance is gone. The only substitute is the SBOM from build time — which for the fat JAR is thus not an accessory but the only remaining source.

What the Measurement Does Not Yield

Threads were collected (53 versus 49) and are not reported: there was no hypothesis stated in advance, and without one, a deviation between two runs is not a finding but noise.

Build time differs — Shade writes a 28 MB archive,copy-dependencies copies files. It is a property of the development machine, not of the delivery, and this series measures at the server.

Summary

Metric	Thin Distribution	Fat JAR
Delivery as.tgz	26.68 MB	25.81 MB
Startup time (server)	3,574 ms	3,482 ms — no difference
Memory (MemoryCurrent)	358 MiB	406 MiB
Patch release	2.60 MB	28.89 MB
“Which version is deployed?”	ls	not answerable

Size goes narrowly to the fat JAR, startup time to no one, diagnostic effort and the patch release clearly to the Thin Distribution. None of these rows decides the choice on its own — and the fat JAR’s real strength appears in none of them, because it cannot be captured in a number. Chapter 7 draws the conclusion.

Recommendation

Both forms run. Both deliver HTTP 200. Startup time does not differ; delivery size differs by three percent in the fat JAR’s favor. Up to that point, a measurement would yield little for a recommendation.

Two rows from chapter 6 do after all, and both point in the same direction: the fat JAR needs48 MiB more memory (406 versus 358 MiB), and a patch release transferseleven times as many bytes with it. Add to that what cannot be measured but affects every day of operation: with the Thin Distribution, the question “which version is deployed?” is answered with anls; with the fat JAR, not at all.

The Recommendation Depends on the Operational Situation

Situation	Form	Reason
A server you manage yourself — the situation of this series	Thin Distribution	The classpath is visible, a patch transfers one eleventh of the bytes, the service needs 48 MiB less, and “which version is deployed?” is answered withls.
Handing off to someone who does not know the machine	Fat JAR	One file, one command, no structure that can fall apart while being copied.
Container image	Fat JAR	The image is immutable anyway. Partial updates and inspectinglib/ lead nowhere there.
CI intermediate build, test environment	Fat JAR	One artifact that can be archived and restored as a whole.

The sentence it comes down to:

A fat JAR hides its components; a Thin Distribution shows them. Whoever manages the machine themselves needs the visibility; whoever merely ships does not.

The Third Option

The juxtaposition “one archive or many files?” contains an assumption that does not hold: that the answer fortransferring and forrunning would have to be the same. It does not.

On the reference server, both forms are transferred as.tgz and unpacked into a release directory. The transfer is thusone file in both cases — atomic, verifiable with a checksum, without the danger of 128 files arriving halfway. What sits in the directory afterwards is independent of that.

Both releases share the same shape:

codeCopy

vaadinapp-00.01.00/ vaadinapp-00.01.00/
├── VERSION ├── VERSION
├── bin/start.sh ├── bin/start.sh ← same file
├── app/vaadinapp.jar └── app/vaadinapp.jar (28,89 MB)
└── lib/ 127 archives

The start script isbyte-for-byte identical. It puts"$APP_HOME/app/*:$APP_HOME/lib/*" on the classpath; iflib/ is missing, the second half resolves to nothing, and the one complete archive inapp/ is all the annotation scan needs — exactly the property from Part 4, chapter 4, that the fat JAR rests on.

The consequence is practical: the systemd unit does not distinguish the two forms. Switching from one to the other is a symlink and a restart, that is, the same operation as any release switch. That is the reason the two forms could be measured under equal conditions in chapter 6 at all.

What this third option isnot: a size advantage. As.tgz, both sit at roughly 25 to 26 MB, and the fat form is even the slightly smaller one. The gain lies solely in the fact that the question of atomicity in transfer and the question of visibility in operation can be answered separately.

What Does Not Work as a Justification Here

“Modern” or “best practice.” The fat JAR is, thanks to Spring Boot, the more common route. That does not make it the better one — it makes it the expected one, and that is an argument about habits, not about technology.

“The proper Unix way.” The opposite direction is just as unfit.bin/,lib/, and a start script are an old and proven form, but their lineage proves nothing about their suitability for this case.

Both forms are old, both are in use, and the choice follows from the situation, not from fashion.

What Runs on the Reference Server

The Thin Distribution, transferred as.tgz and unpacked into a release directory — that is, the third route. This is not hedging in all directions, but the consequence of the same reasoning: the machine is managed in-house, so visibility into its components counts. And the transfer should still be atomic.

The state after this part:

codeCopy

/opt/vaadinapp/
├── current -> releases/2026-09-09-thin
├── logs/
└── releases/ seven releases, from the WAR archives of part 1
up to both forms of this part

Both forms remain in place as releases. Switching between them is thus, at any time, a symlink and a restart — measured: 4,938 milliseconds.

That Part 6 builds on this is a bonus, but decides nothing. A recommendation that follows from the next part instead of from the matter at hand would be none.

Outlook on Part 6

Parts 4 and 5 have not moved any boundary. They packaged and delivered what Part 3 built, and in doing so answered two questions that had been open since then: what the delivery looks like, and how a release is switched and taken back.

What the application still expects from the target system has thereby melted down to a single thing: a Java runtime, installed and maintained by someone else. On the reference server that is a Temurin 26, set up via SDKMAN under the service user, reachable through a symlink — the same mechanism Part 1 used for Jetty, and which has since been dropped there.

Part 6 takes the runtime along as well.jlink builds, from the modules of the JDK, an image that contains only what this application calls, and places it next to the distribution.ExecStart=…/current/bin/start.sh remains the same line — onlyJAVA_HOME then points into the release instead of at the machine. The server no longer needs an installed Java after that.

Two things need to be said up front so they do not appear as surprises.jlink does not make the application modular: it builds an image of the platform modules; the application code stays on the classpath. The recommendation from chapter 7 is untouched by this — whoever chose the fat JAR gets ahead just the same. Andjdeps will not deliver the list of required modules completely: Vaadin loads via reflection, Jetty viaServiceLoader, and static analysis does not see that.

This is the same finding that holds this part and the previous one together, for the third time:what is decided at build time about runtime must go beyond what the compiler sees. In Part 3 it was the classpath that fell short. In Part 4, chapter 5 it was the service provider directory that a merge without knowledge of its contents would have destroyed. In Part 6 it will be the module list.

]]>JavaVaadinhttps://svenruppert.com/posts/vaadin-deployment-on-hetzner-part-6-self-contained-vaadin-with-jlink/Thu, 24 Sep 2026 10:00:00 +0200sven.ruppert@gmail.com (Sven Ruppert)Sven Rupperthttps://svenruppert.com/posts/vaadin-deployment-on-hetzner-part-6-self-contained-vaadin-with-jlink/A self-contained Vaadin distribution with jlink: 16 modules, 65 instead of 309 MB, and what the smaller runtime really changes at startup.<![CDATA[

Sixth and final part of the seriesVaadin – Deployment on Hetzner. The application now brings along not only Jetty but also its own Java runtime: 16 modules instead of 68, 65 instead of 309 MB — and a module list that cannot be fully derived. All figures come from an actual run on a Debian 13 server.

The state after Part 5

For five parts, the boundary between application and machine has been moving in one direction. Part 1 handed a WAR to a Jetty that someone else had installed. Part 3 turned that Jetty into a library of the application. Part 4 packaged the result, Part 5 shipped it and demonstrated the rollback.

What sits on the reference server after Part 5 looks like this:

codeCopy

/opt/vaadinapp/
├── current -> releases/2026-09-09-thin
├── logs/
└── releases/
├── 2026-09-09-thin/
│ ├── VERSION
│ ├── bin/start.sh
│ ├── app/vaadinapp.jar 16 KB
│ ├── lib/ 127 archives, 29 MB
│ └── sbom.json
└── 2026-09-09-fat/

And next to it, outside this directory:

codeCopy

/var/lib/vaadinapp/.sdkman/candidates/java/current -> 26.0.2+1.1-tem 309 MB

This is the last remnant. A Java, installed and maintained by someone else, 309 MB in size, eleven times as large as the application it runs. The systemd unit reaches it through a symlink — the same mechanism that Part 1 used for Jetty and that Jetty has not needed since Part 3.

What this part changes

Figure 1: The same release, once without and once with its own runtime.

jlink takes the modules of a JDK and produces a runtime image that contains only what this one application calls. The image goes into the release, next toapp/ andlib/. The start script finds it there, and the server no longer needs an installed Java.

On the reference server that means65 instead of 309 MB and16 instead of 68 modules. The image contains two executables:

codeCopy

runtime/bin/
├── java
└── keytool

Nojavac, nojdeps, nojlink, nojcmd, nojstack. What sits on the server can start the application and manage keystores — nothing else. That is the property Chapter 10 discusses as a security gain, and it is a by-product: nobody removed the tools, they were never part of what the application needs.

What this part does not change

Three things stay as they are, and it is worth saying so up front.

The systemd unit.ExecStart still points tocurrent/bin/start.sh. No path, no variable, not a single line changes. The trial run on the server executed withJAVA_HOME unset and found its JVM anyway.

The application. Nomodule-info.java, no restructuring, no new dependency. Chapter 3 explains why that is not a coincidence but jlink’s design.

The recommendation from Part 5. Anyone who chose the fat JAR can continue just the same; the image sits next to it, not inside it.

The code state for this part

Tagteil-06. Compared toteil-04 there are two changes: one in the start script, one in the assembly of the distribution. Both appear in Chapter 7.

bashCopy

git clone https://github.com/svenruppert/Blog-Vaadin-Deployment-on-Hetznercd Blog-Vaadin-Deployment-on-Hetznergit checkout teil-06

The module this part works in isembedded-jetty — it carries themain() and the assembly that builds the distribution.

Why a runtime of your own?

The obvious answer is: because the image is smaller. It is correct and at the same time the weakest of the three answers.

Size

65 instead of 309 MB is 79% less. That sounds impressive and means little in practice: a server with a 40 GB disk does not notice the difference, and the transfer over the network barely registers next to the 29 MB of application libraries.

Where size does matter is a different setting — wherever the image ismultiplied. A container image that carries the full JDK carries it in every layer, in every registry, on every node. For this series that is no argument, because no container runs here; Chapter 11 comes back to it.

Attack surface

A full Temurin 26 ships 68 modules, 22 from thejava. namespace and 46 from thejdk. namespace. This application calls 16 of them. The remaining52 are code that sits on the server, can be loaded, and is never needed — among them a compiler, a scripting tool, and the entire toolchain for process inspection.

That is not a theoretical argument. Anyone who has compromised an application far enough to execute code finds a complete development environment on a server with a full JDK. On one with a jlink image, they findjava andkeytool.

The second half of the same argument concerns vulnerability reports. A CVE against a module that is not contained in the image does not affect this release — verifiably so, not as a matter of judgment. Chapter 10 shows how to prove it.

The division of responsibility

This is the answer that carries this series.

As long as a Java is installed on the server, there are two places where decisions about the application’s runtime are made: the release and the machine. A release built and tested with Java 26 runs on a machine whose Java someone has raised to 27. Maybe it works. Nobody has verified it, because the two decisions were made in different places.

With its own image, the Java version ispart of the release and travels with it — forward on update, backward on rollback. The rollback from Part 5, Chapter 5 takes the runtime along without anything having to change for it.

It is the same idea that carried Part 3. There it was about Jetty: a server whose Jetty someone else maintains has a second place where decisions about the application’s behavior are made. Part 6 applies it to the last remaining component.

What speaks against it

Three things, and they are not small.

The build becomes platform-bound. An image for Linux x86-64 can only be produced on Linux x86-64. Chapter 6 describes what that means for the build process and concedes that the solution on the reference server is a compromise.

Java updates become releases. A security update of the JDK used to besdk install,sdk default, restart — three steps without a new build. With your own image it is a new release. Chapter 10 works through the numbers.

The module list has to be right. And it cannot be fully derived. That is the core of Chapters 4 and 5 and the actual reason this part is longer than the tool suggests.

What jlink does — and what it does not require

A misconception circulates aboutjlink that is persistent enough to derail entire projects: that you have to modularize your application to use it. That is wrong, and the distinction is worth clearing up before the first line of tool invocation appears.

Two class paths that have nothing to do with each other

Since Java 9 there have been two ways for the JVM to find classes.

Themodule path (--module-path) carries modules: archives with amodule-info.class that declare what they export and what they require. Theclass path (-cp) carries everything else — it works the way it has worked since Java 1.0, and everything that sits on it ends up in the so-called unnamed module.

jlink operates exclusively on the first. It builds an image fromplatform modules —java.base,java.sql,jdk.zipfs, and so on. What the application brings along does not interest it.

And that is exactly why nothing changes about the application:

shCopy

exec"$JAVA_BIN"$APP_OPTS$JAVA_OPTS -cp"$CLASSPATH" com.svenruppert.flow.Application"$@"

That is the start line from Part 4, unchanged. The class path still points toapp/*:lib/*, the 127 archives still carry nomodule-info.class, and the JVM still loads them as the unnamed module. The only thing that changes iswhich JVM that is.

The sentence that matters:jlink trims the platform, not the application. An image of 16 modules runs a class path of 127 non-modular archives without any of them knowing about it.

Why the story is often told differently

Becausejlink demands a module list, and the obvious way to obtain a complete module list is to make the application itself modular. Thenmodule-info.java writes down what it needs, andjlink can follow that declaration.

Anyone who does not modularize the application — and with 127 dependencies, nobody does — has to determine the list some other way. That is whatjdeps is for, that is what Chapter 4 is for, and that is what the uncomfortable finding in Chapter 5 is for.

So the effort does not disappear, it shifts. You save yourself the restructuring of the application and pay with a list that has to be maintained.

What ends up in the image

jlink takes the named modules, resolves theirrequires relationships, and writes the result out as a runtime image. On the reference server, 10 named modules become 16:

codeCopy

java.base java.datatransfer java.desktop
java.instrument java.logging java.management
java.naming java.net.http java.prefs
java.security.jgss java.security.sasl java.sql
java.transaction.xa java.xml jdk.unsupported
jdk.zipfs

The six unnamed ones come in transitively:java.desktop requiresjava.datatransfer,java.sql requiresjava.transaction.xa andjava.xml,java.security.jgss requiresjava.security.sasl. You name the roots; the resolver takes care of the rest.

java.desktop in a server application looks like a mistake but is correct: that is wherejava.beans lives, andflow-server,flow-data, and Jetty’s servlet layer use it. Chapter 4 shows how to verify that.

What does not end up in the image

Everything else — and that is more than the number 60 suggests:

missing	what no longer works
jdk.compiler	compiling at run time
jdk.jshell,jdk.scripting.nashorn	running scripts
jdk.attach,jdk.jcmd,jdk.jfr	attaching to other JVMs, Flight Recorder
java.rmi,jdk.jdi	remote calls, remote diagnostics
jdk.jlink,jdk.jdeps	building another image from this one

The last row has a practical consequence that Chapter 6 runs into on the reference server:a jlink image cannot reproduce itself. Whoever has the image cannot build a new one; that takes the JDK.

The third row is the one you miss before you need it. Anyone used to tackling a hanging service withjcmd orjstack will not find the tool in the image.

It can be retrofitted, and the price is lower than caution suggests. Withjdk.jcmd added to the module list:

codeCopy

without jdk.jcmd 65,992 KB 16 modules bin: java keytool
with jdk.jcmd 66,292 KB 19 modules bin: java jcmd jinfo jmap jps jstack jstat keytool

300 KB for six tools. That is not a question of size but a trade-off between diagnostic capability and attack surface — and it is one to decide deliberately, not to catch up on during the first incident. For the reference server it stays at the 16 modules: whatever needs diagnosing is in the journal, and a release without diagnostic tools can, if in doubt, be swapped for one with them — that is what the symlink from Part 5 is for.

Determining the required platform modules

jlink demands a list. The application does not keep one. So someone has to produce it — and the tool for that is calledjdeps.

The invocation

bashCopy

jdeps --print-module-deps\ --ignore-missing-deps\ --multi-release26\ --class-path"current/lib/*"\ current/app/vaadinapp.jar

Four switches, and three of them need explaining.

--print-module-deps switches the output from a report to a list — exactly the form--add-modules expects. Without this switch you get a multi-page breakdown of who calls whom; with it, a single line.

--ignore-missing-deps is unavoidable on a real class path. 127 archives practically always contain references to classes that are not shipped — optional integrations with libraries you do not use. Without this switch,jdeps aborts at the first such spot instead of evaluating the rest.

That is the first hint at how this analysis works: it isallowed to have gaps, and it does not tell you so.

--multi-release 26 determines which branch of a multi-release archivejdeps reads. Without it, it refuses to work on every archive withMulti-Release: true in its manifest — and that is the same manifest entry that Part 4, Chapter 5 dealt with for the fat JAR.

The result

codeCopy

java.base,java.desktop,java.instrument,java.management,java.naming,java.security.jgss,java.sql

Seven modules. Anyone who does not want to take them on faith can havejdeps show, in report mode, which archive pulls in which module — here the most notable culprits for each:

Module	pulled in by
java.base	everything
java.desktop	flow-server,flow-data,jakarta.el, Jetty’s servlet layer
java.instrument	org.eclipse.jetty.ee.webapp
java.management	org.eclipse.jetty.util,org.eclipse.serializer.base
java.naming	org.eclipse.jetty.jndi,…ee11.annotations, BouncyCastle
java.security.jgss	org.eclipse.jetty.client,org.eclipse.jetty.security
java.sql	flow-data,org.eclipse.jetty.plus, BouncyCastle

You take these seven, hand them tojlink, get an image, start it — and get an error.

The first failure

codeCopy

java.nio.file.ProviderNotFoundException: Provider "jar" not found
at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:341)
at org.eclipse.jetty...

The missing module isjdk.zipfs. Jetty opens archives not only as files but asfile systems —FileSystems.newFileSystem(uri, env) with ajar: URI. The provider for that lives injdk.zipfs, is found viaServiceLoader, and appears in noimport statement.

Added, rebuilt, started — and the next error.

The second failure

codeCopy

java.lang.NoClassDefFoundError: sun/misc/Unsafe
at org.eclipse.store...

jdk.unsupported. EclipseStore usessun.misc.Unsafe to read and write objects bypassing serialization — that is the reason it is fast. The module is called “unsupported” because it contains access points that are officially not part of the platform; in the full JDK it is always present anyway, which is why its absence only shows up here.

Added, rebuilt, started — and this time it runs.

Nine modules, one running service

codeCopy

systemctl status vaadinapp active (running)
curl -o /dev/null -w '%{http_code}' https://demo.svenruppert.com/ 200
curl -o /dev/null -w '%{http_code}' https://demo.svenruppert.com/login 200

The service runs. Both routes respond. Login works, language switching works, persistence works.

This is where the part would end — if the image were complete. It is not, and the fact that it runs anyway is the most interesting finding of this series.

The three modulesjdeps cannot see

Two modules were supplied by the trial run, each with an error at startup. The third produces no error.

What is missing

java.net.http. The module with the HTTP client that Java 11 introduced. It does not appear in thejdeps output, it is missing from the image, and the service starts, runs, and serves as if everything were fine.

Whyjdeps does not find it

Not because of reflection. The call is right there in the code:

javaCopy

privatestaticfinalclassHibpHolder{staticfinalHaveIBeenPwnedCompromisedPasswordCheckerINSTANCE=HaveIBeenPwnedCompromisedPasswordChecker.usingJdkHttpClient(HaveIBeenPwnedCompromisedPasswordChecker.DEFAULT_ENDPOINT,HIBP_TIMEOUT);}

The reason lies elsewhere, and it is a property of the invocation from Chapter 4 that is easy to overlook:

bashCopy

jdeps ... --class-path"current/lib/*" current/app/vaadinapp.jar

jdeps examines what is given as an argument. The class path serves only for resolution. The argument is exactly one archive:app/vaadinapp.jar, 16 KB, essentially the launcher class. The 127 archives inlib/ are consulted to resolve references — they are not examined.

And the chain to the HTTP client runs entirely throughlib/:

codeCopy

app/vaadinapp.jar
└─> jCustos-…-core (lib/) application code
└─> PasswordPreflight (lib/) der Lazy Holder oben
└─> jCustos-credentials-hibp-00.83.00.jar (lib/)
└─> java.net.http

Nothing is wrong with the tool, it reports no error, and it answers exactly the question it was asked.

The obvious switch does not help

jdeps has-R for recursive traversal. Measured on the same release:

codeCopy

without -R: java.base,java.desktop,java.instrument,java.management,
java.naming,java.security.jgss,java.sql
with -R: java.base,java.desktop,java.instrument,java.management,
java.naming,java.security.jgss,java.sql

Identical. In combination with--print-module-deps and--ignore-missing-deps, the switch changes nothing about the result.

All archives as arguments — and the next problem

The way that remains: have every archive examined, not just the one.

bashCopy

jdeps --print-module-deps --ignore-missing-deps --multi-release26\ current/app/vaadinapp.jar current/lib/*.jar

codeCopy

java.base, java.desktop, java.instrument, java.management, java.naming,
java.net.http, java.rmi, java.security.jgss, java.sql, jdk.unsupported

Ten instead of seven.java.net.http is in,jdk.unsupported too. That looks like the solution, and it is not — for two reasons.

java.rmi is too much. The shipped image does not contain it and runs. The culprit isjakarta.transaction — the interface for distributed transactions references remote calls, and this application never enters that branch. Anyone who follows this list adds a module that contributes nothing but attack surface.

jdk.zipfs is still missing. It appears in neither of the two outputs, because it is not the target of any call: Jetty requests a file system for ajar: URI, and which provider serves it is decided at run time viaServiceLoader. Nothing of that is in the compiled code.

So the exhaustive variant produces a list that istoo large and too small at the same time. That is exactly what makes it dangerous: it looks complete.

How the absence shows itself

An image withoutjava.net.http, started against the same libraries, password check invoked directly:

codeCopy

### Image WITHOUT java.net.http (15 modules)
password123 -> acceptable=false
correct-horse-battery-staple-42 -> java.lang.NoClassDefFoundError:
java/net/http/HttpTimeoutException
### Image WITH java.net.http (the shipped one, 16 modules)
password123 -> acceptable=false
correct-horse-battery-staple-42 -> acceptable=true

Three observations, each more unpleasant than the last.

The first line is the same in both cases. A weak password is rejected, with and without the module — the local blocklist runs before the network query and needs no HTTP. Anyone who tests the check with an obviously bad password seesno problem.

The error does not name the class you expect. NotHttpClient butHttpTimeoutException — the class the loading process stumbles over first. Anyone who searches for the module name will not find it in the error text.

The timing is the worst imaginable. The service starts, both routes respond with 200, login works. The release is declared good. It breaks the first time someone sets a password — on a fresh installation that is the setup screen, that is, the very first action of the very first user.

What follows from this

A startup attempt is not a proof of completeness. It demonstrates that the modules suffice for startup, and it says nothing about anything loaded later.

The module list needs a test that uses the application — not starts it: uses it. On the reference server, that was setting a password. What such a run does not touch, nobody checks.

The list belongs documented, not derived. In the build script it therefore appears not as the output of a tool but as text with a failure signature:

shCopy

# What jdeps cannot see. Every entry comes from an actual run, not from# an analysis:## jdk.zipfs ProviderNotFoundException at startup. Jetty opens# archives as file systems.# jdk.unsupported NoClassDefFoundError: sun/misc/Unsafe. EclipseStore.# java.net.http No error. The image starts, serves pages, and logs# users in; the HIBP check only breaks when someone# sets a password.EXTRA="jdk.zipfs,jdk.unsupported,java.net.http"

A comment that names a failure signature is worth more than a list that is correct. The list goes stale with the next dependency; the failure signature tells the next person what to look for.

The finding that holds the series together

It is the third time, and it is the same shape every time.

InPart 3 it was the class path: what the compiler sees is not enough for the annotation scan. InPart 4 it was the provider directory: what the compiler sees is not enough for the merge. InPart 6 it is the module list.

Decisions made at build time about run time have to go beyond what the compiler sees. Java loads at run time through reflection, throughServiceLoader, through names in text files. Every tool that predicts run time from compiled code has its limit exactly here — and none of them tells you where it lies.

Producing the image

The module list is settled. The invocation that turns it into an image is short — and every one of its switches has a reason you should know before adopting it.

bashCopy

jlink --add-modules"$MODULES"\ --output"$DIST/runtime"\ --strip-java-debug-attributes\ --no-header-files\ --no-man-pages\ --compress zip-6

The switches

--strip-java-debug-attributes removes line numbers and local variable names from the class files of the platform. This affects the platform classes exclusively; stack traces of theapplication keep their line numbers, because the application archives inlib/ remain untouched. What you lose are line numbers in JDK-internal frames — bearable when debugging.

--no-header-files and--no-man-pages remove the C header files for JNI and the manual pages. Nobody needs either on a server.

--compress zip-6 compresses the module store. Chapter 9 measures what it costs and what it brings; the short version is 33 MB less image at identical startup time.

⚠️ The switch that fails on a lean Debian

The documentation and most tutorials name--strip-debug. On the reference server it ends like this:

codeCopy

Error: java.io.IOException: Cannot run program "objcopy":
Exec failed, error: 2 (No such file or directory)

--strip-debug invokes anexternal program —objcopy from the GNU binutils — to remove the symbol tables from the native libraries. On a development machine it is present, on a minimal Debian installation it is not.

Two ways out:

bashCopy

sudo apt-get install binutils# about 20 MB of tooling on the server

or the switch that manages without an external program:

bashCopy

--strip-java-debug-attributes# does the same inside the JVM

On a server that in the end is not supposed to carry any Java at all, it would be absurd to install binutils so thatjlink can run. That is why the build script contains the second variant.

Where the build takes place

Here lies the most uncomfortable property of the whole approach.

A jlink image is platform-bound. An image for Linux x86-64 is produced on Linux x86-64. Anyone who develops on a Mac with Apple silicon — as in this series — cannot build it on their machine.

The documented way out is called cross-linking: download a second, complete JDK for thetarget platform and pointjlink via--module-path at itsjmods directory. That works and is the right way as soon as a build machine is involved.

On the reference server it fails over a detail:the Temurin build installed by SDKMAN ships nojmods directory. Cross-linking would therefore require a second, complete download — for a platform on which the build could run anyway.

So the server builds its image itself. This is possible thanks toJEP 493 (Linking Run-Time Images without JMODs, introduced with Java 24):jlink works from the installed runtime image and no longer needs thejmods files. Without this mechanism, the path on this server would be blocked.

This is a compromise, and let it be named as one. Part 1, Step 5 argued explicitly for separating build and runtime environments. A build step that runs on the target machine violates that separation. With a Linux build machine it would belong there — and with a CI run under Linux it belongs there anyway. What is shown here is the way for the case where that machine does not exist.

The script

The whole process lives intools/jlink-runtime.sh, invoked with the unpacked distribution directory:

bashCopy

tools/jlink-runtime.sh /opt/vaadinapp/releases/2026-09-12-jlink

It determines the modules withjdeps, adds the three from Chapter 5, invokesjlink, and reports at the end what was produced:

codeCopy

jlink-runtime: jdeps found 7, adding 3 it cannot see
jlink-runtime: 65M, 16 modules

That line is worded that way on purpose. Whoever reads it sees immediately that three modules do not come from the analysis — and knows where to look if the image later turns out to be missing something.

Merging image and distribution

The image exists, the distribution from Part 4 does too. Together they form a release that expects nothing more from the machine. Two changes are needed for that — one to the start script, one to the assembly.

The shape of the release

codeCopy

2026-09-12-jlink/
├── VERSION
├── bin/start.sh
├── app/vaadinapp.jar 16 KB
├── lib/ 127 archives, 29 MB
├── runtime/ 16 modules, 65 MB ← new
└── sbom.json

runtime/ joinsapp/ andlib/, it does not replace them. Everything Parts 4 and 5 said about this shape holds unchanged: same unit, same symlink, same rollback.

The change to the start script

Part 4 introduced the start script with the sentence that the release itself says how it wants to be started. That is exactly where the decision about the JVM belongs:

shCopy

if[ -x"$APP_HOME/runtime/bin/java"];thenJAVA_BIN="$APP_HOME/runtime/bin/java"elseJAVA_BIN=${JAVA_HOME:+$JAVA_HOME/bin/java}JAVA_BIN=${JAVA_BIN:-java}fi

Four lines, and they accomplish more than they appear to.

A release with its own runtime uses it. Without configuration, without an environment variable, without the unit knowing anything about it.

A release without its own runtime behaves as before.JAVA_HOME, otherwisejava from the path — that is literally the version from Part 4.

And both sit side by side in the samereleases/ directory. The switch between them is the sameln -sfn as any other. That is the reason the changeover was possible without downtime and the rollback stays open.

The proof of that is short and unambiguous: the trial run on the server executed withJAVA_HOME unset and found its JVM anyway.

codeCopy

ExecStart=/opt/vaadinapp/current/bin/start.sh

This line was in the unit before the changeover and remains in it, unchanged, afterwards.

The change to the assembly

The assembly descriptor from Part 4 gets one morefileSet:

xmlCopy

<fileSet><directory>${project.build.directory}/jlink-runtime</directory><outputDirectory>runtime</outputDirectory><fileMode>0644</fileMode><directoryMode>0755</directoryMode></fileSet>

If an image sits undertarget/jlink-runtime, it goes into the distribution. If none is there, the assembly plugin skips the directory without comment, and the same thin distribution as in Part 4 is produced.

The Maven build deliberately does not produce the image itself. If it did, a development machine would produce an image for that machine, and it would go into an archive destined for a Linux server — a mistake that would only surface on the server, and there with a less than helpfulExec format error.

Instead, the division of labor is:

who	what
Maven	builds the application, packsapp/,lib/,bin/,sbom.json
tools/jlink-runtime.sh	builds the image — where it is meant to run
Assembly	picks the image up,if it is there

On a Linux build machine all three steps run back to back in the same pass, and the finished.tar.gz already carries the runtime. Without such a machine, the middle step runs on the target machine — Chapter 8 shows how.

What happens to the SBOM

sbom.json still describes the application: 129 Java components, 118 JavaScript components.The Java runtime is not in it.

That is a gap, and it must be named. Whoever ships the runtime ships software that until now someone else was responsible for — and the bill of materials says nothing about it. An image of 16 modules from Temurin 26.0.2.1 is a component with a version and a provenance, and it belongs on the list.

Thecyclonedx-maven-plugin does not pick it up, because it does not exist at the build time of the Maven run. It can be added by hand, and what that looks like belongs in its own context — the two articles on SBOMs and delivery formats take it up.

For this part, the finding stands:the release becomes more complete, its bill of materials does not.

Getting it onto the server

The procedure is the one from Part 5, with one step inserted.

bashCopy

# 1 Build and upload — unchanged from Part 5mvn -Pproduction,thin clean packagescp embedded-jetty/target/vaadinapp-00.01.00-thin.tar.gz sven@server:/tmp/# 2 Unpack next to the existing releasesREL=/opt/vaadinapp/releases/2026-09-12-jlinksudo mkdir -p"$REL"sudo tar xzf /tmp/vaadinapp-00.01.00-thin.tar.gz --strip-components=1 -C"$REL"# 3 NEW: add the runtimesudoJAVA_HOME=/var/lib/vaadinapp/.sdkman/candidates/java/current\ sh /opt/vaadinapp/tools/jlink-runtime.sh"$REL"# 4 Ownership — as for every releasesudo chown -R root:vaadinapp"$REL"sudo chmod -R g-w,o-rwx"$REL"# 5 Switch over and restart — unchanged from Part 5sudo ln -sfn"$REL" /opt/vaadinapp/currentsudo systemctl restart vaadinapp

Step 3 is the only addition. Steps 1, 2, 4, and 5 are literally the ones from Part 5, Chapter 4.

The result

codeCopy

current -> releases/2026-09-12-jlink
JVM /opt/vaadinapp/current/runtime/bin/java
Service active (running)
HTTP 200
Hardening 1.1 OK

The hardening score is unchanged. That is not self-evident: the runtime now sits below/opt/vaadinapp, that is, in a directory the service reads. It is owned byroot:vaadinapp and is not writable for the group — the same rule that Pitfall 12 from Part 1 enforced for the SDKMAN directory, applied here to the release. The service can execute its JVM and cannot replace it.

ReadWritePaths stays at/var/lib/vaadinapp/work and…/logs. The release is immutable for the service, runtime included.

What the changeover cost

Nothing beyond an ordinary release switch. The service was unreachable for the duration of one restart; the number is in Chapter 9. Caddy held the connections during that time; the public response stayed at 200, because access runs through the reverse proxy and not directly against port 8080.

⚠️ The JDK cannot go yet

The obvious next move would be to remove the SDKMAN JDK. It would be premature.

codeCopy

/opt/vaadinapp/releases/
├── 2026-09-09-thin/ needs the installed JDK
├── 2026-09-09-fat/ needs the installed JDK
└── 2026-09-12-jlink/ brings its own runtime

Only the newest release is self-contained. The two predecessors fall back to the installed JDK viaJAVA_HOME — and precisely they are the target of a rollback. Removing the JDK would mean giving up the rollback, and with it the property that Part 5 singled out as the most important.

It becomes a server without an installed Java only when no release needs one anymore. The gain lies not in building the image but in the clearing away afterwards — and that moment comes not with the first jlink release, but with the last runtime-less release that you still want to keep available for a rollback.

For the reference server that means: the 309 MB stay put for now. Anyone who wants to be rid of them keeps the predecessors as.tar.gz in an archive instead of as unpacked releases — then a rollback is one extra unpack, and the JDK can go.

The build step on the target machine, once more

Step 3 invokes a compiler component on a production server. That is the spot where this chapter is unclean, and let it be named clearly:

• Afull JDK must sit on the server so thatjlink andjdeps are available — the very JDK you wanted to get rid of.
• The step runs asroot.
• It is not repeatable in the sense that two runs on two servers would produce the same image; they produce two images from two JDK states.

The clean way is a build machine running Linux. There, steps 1 and 3 run together, the.tar.gz already carries the runtime, and what remains on the server is unpacking and switching over. The server then needs no JDK — and step 3 disappears there entirely.

For this series that machine does not exist, and rather than pretending it does, what stands here is the way that works without it.

Measuring: what a runtime of your own costs

Four variants, three runs each, all on the same server, all with the same application and the same unit.

Variant	Image	Downtime	Uptime	MemoryCurrent	RSS
Thin distribution, full JDK	309 MB	4,653 ms	3,574 ms	358 MiB	382 MiB
jlink,zip-6(shipped)	65 MB	4,886 ms	3,866 ms	370 MiB	401 MiB
jlink, no compression	98 MB	4,763 ms	3,852 ms	373 MiB	393 MiB
jlink,zip-6 + CDS	93 MB	4,895 ms	3,899 ms	359 MiB	392 MiB

Downtime is the span during which the service does not respond on port 8080.Uptime is reported by the application itself, measured from process start to Jetty’s operational readiness.

Size

79% less — 65 instead of 309 MB. That is the number that convinces, and Chapter 2 has already said why it weighs the least in this setup: the server has the space.

More interesting is the ratio inside the release. At 65 MB, the runtime ismore than twice the size of the application (29 MB oflib/ plus 16 KB ofapp/). Whoever ships a release that carries its own runtime ships two-thirds Java.

Startup takes longer

And that was not expected.

Around 290 ms more uptime, 3,866 versus 3,574 ms. Reproducible across all runs, clearly outside the noise. Two explanations suggested themselves, both were tested,both are refuted:

The compression.zip-6 has to be decompressed on loading — that plausibly costs time. Measured: the uncompressed image (98 MB) lowers the downtime by 123 ms and leavesthe uptime unchanged at 3,852 ms. So the compression costs nothing in application startup time.

The missing class-data archive. A full Temurin ships four prepared CDS archives; a jlink image does not. That is more than a guess — it is right there in the version output:

codeCopy

full JDK: ... (build 26.0.2.1+1, mixed mode, sharing)
jlink image: ... (build 26.0.2.1+1, mixed mode)

Thesharing is missing. Measured with--generate-cds-archive: the uptime even rises slightly, to 3,899 ms.That is not it either.

The cause remains open. That is an unsatisfying sentence at the end of a measurement chapter, and it is preferable to the alternative: writing down one of the two refuted explanations anyway because it sounds plausible. Whoever knows the cause, please say so.

The number still deserves context: 290 ms on 3.5 seconds is 8%, and it is incurred by a service that starts once per release. For this setup it is irrelevant. For an application that starts per invocation it would be the opposite — and there the tool of choice would not be jlink anyway, but a native image.

Memory

The jlink image needs12 MiB more, 370 versus 358 MiB. Same application, same settings, fewer modules — and more memory.

Here the CDS explanation that failed for startup time does apply: with--generate-cds-archive the value drops to 359 MiB, practically to the level of the full JDK. A class-data archive allows the class state to be mapped in as a memory-mapped file instead of being placed on the heap — exactly what the image without an archive lacks.

That makes CDS, in this setup, amemory tool, not a speed tool:

codeCopy

+28 MB image → −11 MiB memory → ±0 ms startup time

Whether that is worth it depends on what is scarcer. On a server with 4 GB of RAM and plenty of disk: yes. On the reference server, which is short of neither memory nor disk: not necessary — and that is why the shipped image is the one without CDS.

What did not change

	full JDK	jlink
Threads in operation	31	31
HTTP response	200	200
Hardening score	1.1	1.1
Lines in the systemd unit	unchanged	unchanged

The application notices nothing of running on a different JVM. That is the real finding of this chapter:an image of 16 modules behaves like a JDK of 68 — except for 290 ms that nobody can explain and 12 MiB that CDS brings back.

What changes about the release model

Moving the runtime into the release shifts a responsibility. That has two sides, and the unpleasant one comes first.

A Java update is now a release

Before:

bashCopy

sdk install java 26.0.3-temsdk default java 26.0.3-temchown -R root:vaadinapp /var/lib/vaadinapp/.sdkmansystemctl restart vaadinapp

Four lines, no build, no artifact, a few minutes.

After: build a new image, pack it into a new release, ship it, repoint the symlink. The procedure from Chapter 8, in full.

That is more effort, and it should not be talked down. Anyone who applies a monthly JDK security update will from now on run it through the release pipeline. Without build automation that carries it, this quickly becomes an update that does not happen — and an outdated Java in the release is worse than a current one on the machine.

One qualification puts the objection into perspective for this server:the automatic updating from Part 1, Step 21 never covered the JDK anyway. It maintains Debian packages; the JDK came via SDKMAN. So the step was already manual before. What changes is not that it becomes manual, but that it becomesvisible: as a release with a date that can be kept available or taken back, instead of a symlink that someone repointed at some point.

In return, the Java version travels with the rollback

That is the other side, and in a series that dedicated Part 5 to the rollback it is the more important one.

Until now a rollback was incomplete: it took back the application and left the runtime standing. Anyone who observes a problem after a JDK update and goes back to the previous release goes back to the previousapplication — on the new Java. The problem stays, and the cause is now harder to find, because the one change that was just taken back was not the one that mattered.

With its own runtime the rollback is complete.ln -sfn to the previous release takes back application and JVM together.The state that ran can be restored as a whole — and not just two-thirds of it.

The attack surface, provable

52 modules no longer sit on the server. What that is worth shows when a vulnerability report comes in.

With a full JDK, the question “Does this affect us?” is a judgment call: the module is there, maybe it is not loaded, you cannot be sure. With a jlink image it is a query:

bashCopy

/opt/vaadinapp/current/runtime/bin/java --list-modules| grep'^jdk.scripting'# (no output)

No match means: not included. That is not an argument you have to make, but one you can prove — and for a VEX document that justifies the statusnot_affected, the difference is exactly the one between a claim and evidence.

The same holds for the case Chapter 2 described: anyone who can execute code on the server finds no compiler there, no scripting engine, and no tools for attaching to other JVM processes. That prevents no break-in. It shortens what is possible afterwards.

The gap that remains

Two things do not get better through this approach, and they belong here, because otherwise the chapter would look too good.

The bill of materials does not know the runtime. Chapter 7 named it:sbom.json describes 129 Java and 118 JavaScript components; the 16 modules of the image are not in it. Of all things, the component for which responsibility was just taken is missing from the document that is supposed to prove responsibility.

The module list is a maintained file. It comes from an analysis plus three entries from failure signatures (Chapter 5). If a dependency arrives that needs another module, nobody notices automatically — in the fortunate case at startup, in the unfortunate one with the first user who invokes the affected function.

Whoever adopts jlink takes on both as an ongoing task. That is the price for the runtime becoming part of the release, and it is only appropriate once the gain is actually needed.

The comparison across the series — and where the application ends

Six parts, five delivery formats, one server. What each format expects from the target system — and what it brings along.

The overview

Part	Form	Release on the server	required there
1–2	WAR to external Jetty	21 MB	JDK (309 MB)and Jetty (44 MB distribution)
3	embedded Jetty, unpacked	29 MB	JDK
4	thin distribution	29 MB	JDK
4	fat JAR	28 MB	JDK
6	thin distribution + own runtime	94 MB	nothing

All five variants sit simultaneously in/opt/vaadinapp/releases/ and were measured in a single pass — that is why the numbers are comparable. The archive sizes from Part 5 (26.68 MB for the thin distribution, 25.81 MB for the fat JAR) come from an older state of the code and are not.

The right-hand column is the thread that runs through the series, and read from top to bottom it is a single movement: what the machine provided at first, the release provides at the end.

The column next to it shows what that costs — and across four parts the answer is: almost nothing. The release stays between 21 and 29 MB while the prerequisites on the server shrink from two third-party components to one. The Jetty from Part 1 moves into the application in Part 3 and shows up there as the 8 MB difference between 21 and 29 MB — a good deal against a 44 MB distribution that someone has to install and maintain.

Only the last step costs:from 29 to 94 MB. The difference is the runtime, and it is more than twice the size of the application that it runs.

Which form for what

WAR to an external Jetty — when operations provides an application server and several applications run on it. Then Jetty is shared infrastructure, and shipping it per application would be wasteful. The price is depending on two states that someone else maintains; Part 2 showed how tight the coupling between Vaadin version and servlet version actually is.

Fat JAR — the default for the normal case. One archive, one checksum, no directory of 128 files. Part 5 measured that it starts neither faster nor slower and needs around 48 MiB more memory; that is tolerable. The price is in Part 4, Chapter 5: the merge is not a copy operation, and whoever gets it wrong builds an archive that starts and does not work.

Thin distribution — when you want to see what is being shipped. 127 archives, each with a name and a version, each individually replaceable. For debugging and for everything related to license and vulnerability checking, this is the more pleasant form.

Own runtime — when the Java version is to belong to the release. Three cases justify the effort: a target system on which no Java can or may be installed; an operation in which the rollback must be complete; and an environment in which the attack surface must be provable rather than estimated.

For everything else, an installed JDK plus fat JAR is the simpler solution, and in operations, simpler is a value in itself.

Where does the application end?

The series has been moving this boundary for six parts, and at no point was it clear where it belongs.

Part 1 would have answered: the application ends at the WAR. Everything before it is its business, everything after it is operations. Part 3 pulled Jetty inside, because the coupling between Vaadin and servlet version was too tight to run it across a system boundary. Part 6 pulled the JVM inside, because a rollback that leaves the runtime standing is not a complete rollback.

What remains is not a technical answer but one about responsibilities:

The application ends where the next decision is made by someone else — and where that is fine. Not every shared responsibility is a problem. It becomes one when two places decide about the same behavior and neither of them checks the whole.

This series pushed the boundary far, because the reference server is one that a single person operates. Where one team provides operations and another the application, it lies elsewhere — and there the WAR from Part 1 would be the right answer, not the outdated one.

The step that does not come

The container.

It is the obvious next thought, and it solves the same problem — an image that brings everything along, on a host that provides nothing. Two things speak for treating it separately.

It does not replace the question, it relocates it. A container image with a full JDK carries the same 309 MB, just one layer further down. The work from Chapters 4 and 5 is due there just the same, and it pays off theremore strongly than here: an image that is multiplied into every registry, onto every node, and into every layer profits from 79% less footprint more than a disk on which 309 MB sit once. Chapter 2 ranked that as this series’ weakest argument — in the container it becomes the strongest.

It brings a second system boundary with it. Network, file system, process isolation, registry, signing — each of these is a topic of its own, and the systemd hardening from Part 1, which this series holds at 1.1, would have to be answered anew there. Squeezing that into a closing chapter would mean treating it badly.

What carries over from this series into a container is therefore not the result but the method:a module list that cannot be derived; a startup attempt that proves nothing; and a release that describes its own state completely. Whoever has that can put it into a container image. Whoever does not puts a problem inside and a layer on top.

]]>JavaVaadin