Dropwizard 1.1 and Let's Encrypt with no DowntimePublished on
Dropwizard and Let’s Encrypt go together like hotels on a beach
As of writing this, the free no gimmicks SSL certificate service, Let’s Encrypt, has issued nearly 15 million certificates. I’ve written about Let’s Encrypt before – it’s taking over the world and that’s a good thing.
This time I want to talk about Let’s Encrypt in the context of a web framework that I’ve gotten quite familiar with over the years, Dropwizard (I’m not going to link to all the articles I’ve written about Dropwizard because there’s so many!). Up until recently, Dropwizard needed to be restarted to apply a newly issued certificate, which doesn’t sound too bad, but a big plus with Let’s Encrypt is certificate renewel is frequent and automated. Some people like to refresh their certificates every month (instead of the required 90 days) and having guaranteed downtime every month is not a thought I relish. So I did what I had to do: submit a pull request. It was accepted and will be released as part of Dropwizard 1.1 (which is not yet released yet).
I do, very quickly, want to mention that this may not affect many people. It’s common to deploy Dropwizard behind a TLS termination proxy (HAProxy, apache, nginx), so you should refer to one of those guides when integrating Let’s Encrypt.
In this article, I want to provide a start to finish approach to setting a box with a standalone Dropwizard application.
First we’re going to start by creating a blank Dropwizard project through Maven.
Since Dropwizard 1.1 is not released yet, an additional step is to clone the Dropwizard repo and
mvn install. Then swap out the pom dependency version for
We’ll add an endpoint that returns “Hello world”
There shouldn’t be anything too new at this point. What is new is registering the
SslReloadBundle in our
SslReloadBundle will register a
reload-ssl endpoint on the admin servlet that will loop through all registered HTTPS endpoints and reload certificate information. The reload uses the same information as the configuration that Dropwizard was started with (same keystore path, same keystore password, etc). A nice feature is that if reload fails due to an incorrect password, your application will continue using the last known good certificate, but make sure you fix it right away else the app will be unable to restart.
As an aside, I’ll be including the
dropwizard-http2 module for that sweet, sweet HTTP2 endpoint. It will complicate some bits later on with the class path, but I’ll walk through that section as well.
Time for deployment. I’ll be using DigitalOcean to host and using their lowest tier machine because I’m cheap. I’ll be getting the following for 16.8¢ per day:
- 1 core
- 20GB SSD
- Ubuntu 16.04 (but there are many other options as well)
Careful now, a gust of wind could knock this machine over.
When creating a droplet, DigitalOcean has the option to boot with ssh keys. I recommend using them – there’s even a decent guide!
After logging in we’ll download and install the latest java as well as the Unlimited Strength Jurisdiction Policy Files, which will allow greater than 128bit key cryptography. I’ve had clients (non-browsers) unable to connect to servers, due to them requiring 256bit encryption. Note that this will only affect non-HTTP2 connections, as will be discussed later.
Next we’re going to install the Let’s Encrypt client and use the embedded server to request certificate information. We’re going to pass in a command flag that specifies that we want the certificate negotation to occur over port 80 because when we will want to renew the certificate, port 443 (the other port that the embedded server can bind to and the default HTTPS port) will be in use by our application.
The downside is that we’re unable to redirect HTTP requests to HTTPS, but I am more than willing to make this compromise because I can’t think of a situation where a client would request a plain HTTP request from our service and expect to redirected (this isn’t apache or nginx here!)
The certificate information needs to be massaged into a native Java format. For that we’ll be using openssl and keytool app installed in the standard Java direct. I’m using a dummy password. The same password is necessary to be used throughout.
Now that we have our keystore tidied up, what does our config look like?
Couple things to note about this configuration:
Type of application server is
h2, an HTTPS2 connection. Same configuration could have used for a regular
httpstype as well.
Don’t be scared that
false. Unfortunately, the default is
true, certificate validation will fail unconditionally. For more information, see the following issue.
The administration port is over plain HTTP and is listening to only 127.0.0.1 connections. This means that only a request originating from the box can communicate with the administration port and subsequently reload certificate information.
In addition to binding to loopback, it also prudent to have properly configured firewall rules, so that no one can connect from the outside. See: how secure is binding to localhost in order to prevent remote connections
The connection is plain http because we’ll be communicating with it via curl, and curl doesn’t support HTTP2 connections out of the box. One needs to compile and build nghttp2 and curl. I sleep better at night by knowing no sensitive information going to the admin port (this is normally a bad excuse to not implement HTTPS). This is just for demonstration purposes (could have used the
httpstype as well).
Cloudflare goes very deep into compiling curl for HTTP 2 in their article. May be a bit long.
To run our application:
As promised earlier, we have to modify the boot classpath to include this jar. The gist is that Java8 doesn’t contain the necessary bits for Application-Layer Protocol Negotiation (ALPN), something required for HTTP 2. And since the required jar version changes for each JDK version see the Jetty guide for what version you need and overall usage.
I retrieved the version I needed by downloading straight from Maven Central
Even more important to getting our first Let’s Encrypt certificate is keeping it renewed! For our purposes we’ll attempt to renew certificates at 2:30am every monday using cron:
And the script itself:
Things to note:
renewcommand is new the conversion to the Java keystore is very familiar. The only difference is an avoidance to overriding the default keystore before the new keystore is created.
A curl command to our adminstration port on 127.0.0.1 to reload certificate information. The HTTP status code is captured.
If the status code is not a 200 then we send an email so that I can fix the issue. For more information, see Send email alerts using ssmtp
If you want to force certificate renewal, pass
--forceto the command.
140 requests per second is pretty measly for a Hello World application (see turning it up to eleven for performance tips). I saw the machine pegged at 100% CPU and memory usage. Hence this is why you shouldn’t skimp on resources.
Sslyze will let us know what cipher suites our server supports. When we just have HTTPS enabled (so no h2).
Notice the cipher suites with 256bit AES, and that is thanks to the unlimited strength crypto installed earlier.
Switching to h2, we see a slightly different story:
Very interesting. Looks like the only cipher suite available is only one that is required from the HTTP 2 spec. I made sure to confirm this with the ssllabs tester. I’m not sure if this the desired default for Jetty.
Last but not least, this post doesn’t go into registering your dropwizard application as a service or starting it on boot (but it’s as easy as adapting an init script template and
chkconfig on, respectively)