We can set up the order of service startup and shutdown using the depends_on option. However, it won’t make a Docker container wait for another one to be ready. There are many situations when we need to be sure that the container on which our service depends is actually running.
What we’re going to build
For this example, we’re going to configure the
metricbeat serivce to start only after the
kibana container is ready. The result of this post is available as a project in the spring-boot-log4j-2-scaffolding repository. When we’re done the project directory tree will look as follows:
The reason behind this post was a Metricbeat error I got when I tried to run the Elastic Stack using Docker Compose. The error occurred because Metricbeat requires a working connection with Kibana and it takes a lot of time to start Kibana. This article complements the Monitor Elastic Stack post. We’re going to focus on configuring Metricbeat to use the
wait-for-kibana.sh script that we’re going to add to its configuration.
Docker Compose config
Below you’ll find the
docker-compose.yml file responsible for setting up the two containers:
Assume that we have a
metricbeat service built from this custom Dockerfile:
This configuration is based on the custom image configuration presented in the Elastic documentation. Later in the article, we’re going to edit this file to make the
metricbeat service utilize a bash script that will make it wait for Kibana.
Additionally, the default Elastic Stack version is specified in my .env file:
Chaos resulting from unmet dependencies
Using solely the
depends_on option in the
docker-compose.yml file ensures that
kibana is run first. However, launching a Kibana instance takes some time. Therefore,
metricbeat will try to connect to the service that is not yet ready. As a consequence, the monitoring service will exit with the following error:
In this case, we need to force
metricbeat to attempt a connection only when
kibana is accessible. In other words, we have to make one Docker container wait for another.
Override the default entrypoint to make Docker wait for another container
We’re going to create a bash script delaying
metricbeat connection to
kibana. Furthermore, the script will be used as a new entrypoint when starting the
metricbeat container. We will execute the default entrypoint only when the
kibana dependency is met.
How to write a bash script that will wait until a container is ready
You can learn more about scripts like this one in the Control startup and shutdown order in Compose documentation. The most important part is using the curl command to test whether the service on the given url is responding (line 4). We use the following options:
- output <file> – Write to a file instead of stdout, in our case we write to the Unix null device as we don’t need any output.
- silent – Silent mode, doesn’t show progress meter or error messages.
- fail – Fail silently (no output at all) on HTTP errors, since we expect many errors.
- head – Show document info only, often used for:
(…) testing hypertext links for validity, accessibility, and recent modification.https://tools.ietf.org/html/rfc7231#section-4.3.2
Return control to the original entrypoint
The last line of our script executes the original entrypoint, the
/usr/local/bin/docker-entrypoint file (line 9). Therefore, the original script will be executed only after the Kibana instance is ready. You need to explore the image documentation to find the actual path to the default entrypoint.
Pass the kibana url to the metricbeat container
Our script requires the KIBANA_URL environment variable. Remember to add it to the
Execute a custom script before the original entrypoint
To actually benefit from our script we have to execute the following actions while building the image:
- copy our script to the container;
- set it as the new entrypoint;
- reconstruct settings/commands that the custom entrypoint may have removed.
Below you’ll find the adjusted
Dockerfile (lines 5, 10, 11):
Verify that your custom entrypoint was executed
Start the services defined in your
docker-compose.yml file with the following command:
If you changed the
metricbeat settings, remember to rebuild the image with:
Examine logs printed in the
As we can see, the
metricbeat service doesn’t try connecting to the
kibana service until the latter one is up.
Learn more on how to make one container wait for another
- How to create a loop in bash that is waiting for a webserver to respond?
- Verify if a URL exists
- Post on writing bash commands