Docker Healthcheck¶
The Defense Information Systems Agency (DISA) guidance requires a Docker Healthcheck, which reveals the wellness of applications in your containers. The Game Warden team recommends adding code that aligns with this requirement. Alternatively, we also allow a Kubernetes Liveness probe as a Docker Healthcheck replacement. In this circumstance, you must perform specific actions in Scan Lab (as referenced in the Resolutions section below) to ensure compliance.
Healthcheck is a command that displays the status of applications in your containers. As a matter of information, Healthcheck can be a generic reference. Some software applications use this term (and enter a command by the same name) when checking the wellness of applications to ensure they are functioning properly. In our context, this reference is a Docker-specific term.
DISA Specification¶
DISA authored the Container Image Creation and Deployment Guide for the Department of Defense (DoD). Section 2.6 of this guide includes the specification noted below:
" 2.6 The Container Image Must Be Built With a Process Health Check
Ensuring a container service is still executing and handling workloads is necessary to validate service availability. Adding the health check instruction, HEALTHCHECK, within Docker, to the container image, provides a mechanism for the container platform to periodically check the running container instance against that instruction to ensure the instance is still working. Based on the reported health status, the container platform can then exit a non-working container and instantiate a new one bringing the service back to an available state. Short lived containers that do not require a health check can be submitted for a waiver.
IA Control: SC-5 CCI: CCI-002385 ”
Healthcheck Context¶
Note
First, a bit of clarity relative to Docker, Dockerfiles, images, and containers . . .
Docker, at its core, is a Platform as a Service (PaaS) that enables developers to build and deploy containers, which store applications and their dependencies. Since a container stores applications and all components that applications need to run successfully, containerized applications function as designed despite where you deploy them.
A Dockerfile is a text file that includes instructions on how to build a base image. A Docker image can have several layers. The first layer is the base image and represents the foundation upon which you can build other images. You must not modify the base image.
Multiple images combine to form a single application, with each image providing a specific function. For example, you might use rhel, commonly known as ubi, as your base image; ubi is a universal base image that functions as an operating system. From ubi , you might add nginx (pronounced “engine-x”) atop this unmodified base image. nginx, which has many uses, might function as your web server. You might add other images as well. These other images provide additional functionality, and each additional image aligns with a specific application need. Again, all images combine to form a single application that includes the functionality designated by each.
When you build Docker images, Docker – at one point – included Healthcheck, which provides the status of applications in your containers. Although Docker no longer supports its use, the DoD still requires this check.
Recently, some customers reported a sudden Healthcheck not found message when viewing Scan Results in the Game Warden Web App. Although our team has not provided any recent application implementations, we now display Anchore compliance results which identify DoD compliance issues within an application; hence, your ability to now see this message. Anchore compliance results are based on the National Institute of Standards and Technology (NIST) 800-53 compliance policies that are required for the DoD. For additional information, read Common Vulnerabilities and Exposures, and Compliance Best Practices. Anchore compliance scans images for Dockerfile policy configurations, producing compliance results.
It is important to note that Iron Bank images do not include a Healthcheck. This absence triggers the Healthcheck not found message.
Resolutions¶
There are countless ways to satisfy the DoD Healthcheck specification. As a point of emphasis, a Docker Healthcheck is required per DISA guidance. That noted, the Game Warden team recommends that you implement the Healthcheck command in your code. However, we realize some customers prefer to use Kubernetes Liveness probe as a Docker Healthcheck replacement. Our team will permit you to use this option as an acceptable resolution. If using Kubernetes Liveness probe, however, you must justify this use in Scan Lab because Anchore compliance will trigger the Healthcheck not found message. More specifically, in the Vulnerability drop-down list box in Scan Lab, you must select Policy N/A. In the Justification section, you must add: Using Kubernetes Liveness Probe.
The curl and wget commands function similarly. Not only may you use each to add the Healthcheck command, but you also may use them to access URL or port information. In addition, you can use these command-line statements to download files.
Below are common resolution methods.
Example 1: Add Healthcheck Command in Dockerfile Using curl¶
Below is a Dockerfile that includes a Healthcheck command using the curl command.
bash
# Use an official Python runtime as the base image
FROM python:3.8-slim
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
COPY . /app
# Install required packages
RUN pip install --no-cache-dir -r requirements.txt
# Define the healthcheck command
HEALTHCHECK --interval=10s --timeout=5s --start-period=15s \
CMD curl --fail localhost:8080/health || exit 1
# Expose port 8080 for the application to listen on
EXPOSE 8080
# Run the application
CMD ["python", "app.py"]
The HEALTHCHECK instruction in the Dockerfile specifies a curl command that runs periodically to check the health of the application running in the container. The --interval and --timeout options specify the interval between checks, and the amount of time to wait for a response (respectively). –start-period specifies the amount of time to wait before beginning the Healthcheck after the container starts.
The Example #1 code runs a curl command against itself (localhost on port 8080) every 10 seconds, timing out every 5 seconds. If the command returns a non-zero exit code, Docker marks the container as unhealthy and restarts the container.
Example 2: Add Healthcheck Command in Dockerfile Using wget¶
You can use wget to add the Healthcheck to the Dockerfile:
bash
# Dockerfile
FROM alpine:3.12
RUN apk update && \
apk add --no-cache wget
HEALTHCHECK --interval=10s --timeout=5s --start-period=15s \
CMD wget --spider http://localhost || exit 1
Example #2 begins with an Alpine Linux image as the base image. Next, we update the package list and install wget. The HEALTHCHECK instruction establishes a 5-second interval for the Healthcheck command, which is a simple wget command that sends a request to the localhost URL. The --spider option is used to make a request that only checks for the existence of the page; there is no content download. If the wget command returns a non-zero exit code, the Healthcheck fails and Docker considers the container unhealthy.
Since this is a basic example, you may need to modify the Healthcheck command contingent upon both your use case and the application running in the container.
The Example #2 code runs a curl command against itself every 10 seconds, timing out every 5 seconds. If the command returns a non-zero exit code, Docker marks the container as unhealthy and restarts the container.
Example 3: Add Liveness Probe in Kubernetes using curl¶
To use a Liveness probe in Kubernetes, you must define a pod that uses the image then configure a Liveness probe that runs the Healthcheck command.
yaml
apiVersion: v1
kind: Pod
metadata:
name: my-app-pod
spec:
containers:
- name: my-app-container
image: my-app-image
livenessProbe:
exec:
command:
- curl - localhost:8080/health
initialDelaySeconds: 15
periodSeconds: 10
The livenessProbe section specifies a curl command that runs inside the container to check the application’s health. The initialDelaySeconds field specifies the time to wait before performing the first probe, and the periodSeconds field specifies the interval between probes. Kubernetes runs this command periodically to determine if the container remains healthy.
The Example #3 code runs a curl command against itself (localhost on port 8080) every 10 seconds. If the command returns a non-zero exit code, Kubernetes marks the container as unhealthy and restarts the container.
Example 4: Add Liveness Probe in Kubernetes using wget¶
A Kubernetes Liveness probe checks the health of a container and determines whether or not it should be restarted. You can use the wget command for the Liveness probe.
Below is an example of a Liveness probe using wget:
yaml
apiVersion: v1
kind: Pod
metadata:
name: my-app
spec:
containers:
- name: my-container
image: my-image
command: ["my-command"]
livenessProbe:
exec:
command:
- wget
- --quiet
- --tries=1
- --timeout=1
- --spider
- http://localhost:8080/health
initialDelaySeconds: 5
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
In Example #4, the Liveness probe uses wget to send a request to the endpoint http://localhost:8080/health and checks if the request returns a success status code. The --quiet option suppresses the output, and --tries=1 specifies that only one attempt should be made. --timeout=1 sets a timeout at 1 second for the request. --spider tells wget to only check the availability of the endpoint; there is no content download.
The initialDelaySeconds field specifies a 5-second delay before the first Liveness probe is executed. periodSeconds sets the interval between subsequent probes to 10 seconds. successThreshold specifies that a single success is enough to declare the container healthy. failureThreshold specifies that if 3 consecutive failures occur, the container is considered to have failed the Liveness probe and may be restarted.
Healthchecks Using Java Distroless Containers¶
Distinctly smaller than most containers, a distroless container reduces the attack surface because of its minuscule size. This container does not comprise many of the components specific to most. For example, a distroless container does not include a shell (operating system interface) and, by extension, a command line. With no command line, the process of performing a Healthcheck when using a distroless container differs from the process of performing a Healthcheck using other container types.
Content Resource
The remaining content and examples originate, in large part, from the Java Healthcheck and Java-based Healthcheck for Docker articles. Some of this content has been fine-tuned and, where necessary, clarified with the inclusion of explanatory text.
This Java Healthcheck article references two examples of performing Healthchecks with distroless containers:
- Use a Mini Web Server
- Use the
wgetCommand
Example 5: Use a Mini Web Server¶
To perform a Healthcheck:
- Create a mini web server.
Similar to a standard web server, a mini web server displays website content using HTTP (or over the Internet). - Bundle the mini web server into a Java ARchive (JAR).
A JAR serves many purposes to include file compression, similar to .zip files. A JAR is particularly helpful if you have several features, for example, that you want to include in different applications. Instead of writing the same feature code multiple times, you can use a JAR. More specifically, in Java, you can add a feature to a class. You can add several classes to a JAR then compress its content. You can then use the JAR to run the same code in multiple applications. -
Build a container using the following commands:
bash javac TheComSunNetHttpServer.java jar -c -e TheComSunNetHttpServer -v -f TheComSunNetHttpServer.jar TheComSunNetHttpServer.class javac HealthCheck.java jar -c -e HealthCheck -v -f HealthCheck.jar HealthCheck.class docker build -f Dockerfile-java . -t java-healthcheck -
Start the container in the background – listening on Port 8080.
bash docker run -d -p 8080:8080 java-healthcheck sleep 2 curl http://localhost:8080 -
Verify the Healthcheck is working:
bash ❯ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 009a72a63438 java-healthcheck "/usr/bin/java -jar …" 52 seconds ago Up 51 seconds (healthy) 0.0.0.0:8080->8080/tcp friendly_cannon
Example 6: Use the wget Command¶
This example requires that you use a multistage Docker build and copy wget from busybox before using wget with Healthcheck. While multistage Docker enables you to separate the Docker build process into different steps or stages, busybox (known as the Swiss Army knife for Unix/Linux) is a software that allows you to include or add multiple Unix utilities to form a single executable. Busybox allows you to execute commands such as copy/delete/mkdir – as these Linux commands are combined and compressed into this single file. With a distroless container, these and other Linux commands otherwise would be unavailable.
-
Build a container using the following commands:
bash javac TheComSunNetHttpServer.java jar -c -e TheComSunNetHttpServer -v -f TheComSunNetHttpServer.jar TheComSunNetHttpServer.class docker build -f Dockerfile-wget . -t wget-healthcheck -
Start the container in the background – listening on Port 8080.
bash docker run -d -p 8080:8080 wget-healthcheck sleep 2 curl http://localhost:8080 -
Verify the Healthcheck is working:
bash ❯ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 009a72a63438 wget-healthcheck "/usr/bin/java -jar …" 52 seconds ago Up 51 seconds (healthy) 0.0.0.0:8080->8080/tcp friendly_cannon
This Java-based Healthcheck for Docker article provides general Docker Healthcheck insight. This article includes instructions on developing a single-file program using Java11 and writing then executing a Hello World program to perform the Healthcheck. A single-file program allows Java program execution of a single file without compiling code using javac. This article also provides information on Java HttpClient, which is used to send requests and receive responses.
yaml
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse.BodyHandlers;
public class HealthCheck {
public static void main(String[] args) throws InterruptedException, IOException {
var client = HttpClient.newHttpClient();
var request = HttpRequest.newBuilder()
.uri(URI.create("http://localhost:8080/actuator/health"))
.header("accept", "application/json")
.build();
var response = client.send(request, BodyHandlers.ofString());
if (response.statusCode() != 200 || !response.body().contains("UP")) {
throw new RuntimeException("Healthcheck failed");
}
}
}
The code block above:
- Makes a call to the actuator endpoint using the HttpClient.
- Verifies if the status code of the response is 200 (which indicates a request has succeeded), and if the response contains the string: UP. A Healthcheck Failed message will be triggered if the status code is not 200 and the UP string is not present. If both conditions are satisfied, this message will not display.
The article uses the single-file program to provide the Healthcheck in a Java distroless container. As a reminder, when using a distroless container, you must specify the ENTRYPOINT or CMD in the exact (JSON array) form because a distroless container does not contain a shell (operating system interface) to launch commands.
This approach eliminates the need for any utility (curl or wget, for example) and relies solely on the Java tooling available in the container.