Keycloak in Docker #5 – How to export a realm with users and secrets

Running a Keycloak service in a Docker container allows us to share its configuration across multiple environments. However, we can also export an entire Keycloak realm in case we need any backups or data transfer between servers.

Prerequisites

• Docker Engine and Docker Compose installed on your machine.
• If this is your first attempt to run Keycloak in Docker, I recommend reading the post Keycloak in Docker # 1 – How to run Keycloak in a Docker container as I explained the basic configuration there.

For reference, below you’ll find the resulting directory tree for this example:

In short, my goal is to get the export directory containing realm resources (in this case, the keep-growing-realm.json file). Thanks to this, I’ll be able to backup or recreate a complete realm whenever I need to.

Why not export with Keycloak Admin console?

The Keycloak Admin Console provides an easy way to export a realm. However, as you can see in the screenshot below, not all resources can be exported with this method:

The realm-export.json file produced with this technique won’t contain user data. In addition, client secrets will be masked. Although this approach might be appropriate in some use cases, we won’t be able to recreate the same instance using only this file.

If we want to get realm data suitable for backups or cross server migration, we need to run a boot-time export.

My example Keycloak realm

I have my keep-growing realm with the following users:

Furthermore, the realm contains the spring-boot-example-app client. It has the confidential access type and a secret, as you can see in the screenshot below:

At the end of this article, I will verify that the users and the secret are properly exported.

How to export all resources from a Keycloak realm

I’m going to use a volume to map exported data between the container and my machine. Next, I will show you how to export the data with a command or automatically when running the container. Finally, I’m going to verify if the export was successful.

Add a volume for exported resources

First, we need to start our container with a volume that will hold exported data. Therefore, I’m going to map the ./keycloak/realms/export folder on my machine to the /tmp/export directory in the keycloak service. Below you’ll find my docker-compose.yml file:

version: '3.3'
services:
  keycloak:
    image: jboss/keycloak:${KEYCLOAK_VERSION}
    ports:
      - "8024:8080"
    environment:
      - KEYCLOAK_USER=${KEYCLOAK_USER}
      - KEYCLOAK_PASSWORD=${KEYCLOAK_PASSWORD}
    volumes:
      - ./keycloak/realms/export:/tmp/export

I’m using my default environmental variables defined in the following .env file:

KEYCLOAK_VERSION=16.1.1
KEYCLOAK_USER=keycloak
KEYCLOAK_PASSWORD=keycloak

Now I’m going to start the service with the docker-compose up -d command. As we can see in the following screenshot, the volume is properly created and mapped:

Obviously, the /tmp/export directory inside the dockerized Keycloak instance is empty as we still need to export our realm:

Keycloak export options explained

Let me introduce the properties that we’re going to use:

• Djboss.socket.binding.port-offset, we want to run the export on a different port than Keycloak itself. By default, the jboss server in my kecyloak instance uses the 9990 port. Therefore, providing e.g. 100 as a value here will result in the command using the 10090 port.
• Dkeycloak.migration.action, we specify what we want to do. Available options:
  • export,
  • import.
• Dkeycloak.migration.provider, we define how we want the data. Available options are:
  • singleFile, if we want to export data into a file
  • dir, if we want to export data into a directory.
• Dkeycloak.migration.realmName, realm for export. Don’t specify this parameter if you want to export all realms.
• Dkeycloak.migration.usersExportStrategy, how to export realm users. Available options are:
  • DIFFERENT_FILES, it’s the default option for this attribute, we will get a file for realm config and multiple files for users (depending on how many users we have and the number of users per one file (50 by default));
  • SKIP, won’t export users;
  • REALM_FILE, export users to the same file as realm configuration;
  • SAME_FILE, we will get one file for realm configuration and one for all users (keep-growing-realm.json and keep-growing-users.json).
• Dkeycloak.migration.file, the file where we want to save data. Remember to use the exact same value as you used for the volume mapping.

Run the export command

We’re going to execute the /opt/jboss/keycloak/bin/standalone.sh script included in the jboss/keycloak image to initiate the export process:

docker exec -it <CONTAINER NAME> /opt/jboss/keycloak/bin/standalone.sh \
… // required Keycloak export properties

To clarify the following examples, my example keycloak service runs in the keycloakspringboot_keycloak_1 container.

Export to a single file

When a realm doesn’t contain a lot of users it might be appropriate to export it into a single file. The actual command that I’m going to execute looks like in the snippet below:

docker exec -it keycloakspringboot_keycloak_1 /opt/jboss/keycloak/bin/standalone.sh \
-Djboss.socket.binding.port-offset=100 \
-Dkeycloak.migration.action=export \
-Dkeycloak.migration.provider=singleFile \
-Dkeycloak.migration.realmName=keep-growing \
-Dkeycloak.migration.usersExportStrategy=REALM_FILE \
-Dkeycloak.migration.file=/tmp/export/keep-growing-realm.json

If the command encounters no problems, you’ll see console output similar to this:

…
12:11:10,399 INFO  [org.keycloak.services] (ServerService Thread Pool -- 60) KC-SERVICES0034: Export of realm 'keep-growing' requested.
12:11:10,400 INFO  [org.keycloak.exportimport.singlefile.SingleFileExportProvider] (ServerService Thread Pool -- 60) Exporting realm 'keep-growing' into file /tmp/export/keep-growing-realm.json
12:11:10,790 INFO  [org.keycloak.services] (ServerService Thread Pool -- 60) KC-SERVICES0035: Export finished successfully
…
12:11:11,195 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:10090/management
12:11:11,196 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:10090

As a result, a new file appears in the keycloak/realms/export directory on my machine:

You can quit the process with Ctr+C.

If you want to import that realm, the Keycloak in Docker #2 – How to import a Keycloak realm article describes in detail importing a realm from a file.

Export to a directory

Keycloak recommends exporting data into a directory if your realm contains more than 500 users:

Exporting many users into a directory performs optimally as the directory provider uses a separate transaction for each “page” (a file of users).

https://www.keycloak.org/docs/16.1/server_admin/#assembly-exporting-importing_server_administration_guide

In this case, my command would look like this (notice changes in lines 4 and 6):

docker exec -it keycloakspringboot_keycloak_1 /opt/jboss/keycloak/bin/standalone.sh \
-Djboss.socket.binding.port-offset=100 \
-Dkeycloak.migration.action=export \
-Dkeycloak.migration.provider=dir \
-Dkeycloak.migration.realmName=keep-growing \
-Dkeycloak.migration.dir=/tmp/export

If the command encounters no problems, you’ll see console output similar to this:

…
12:29:31,726 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0034: Export of realm 'keep-growing' requested.
12:29:32,107 INFO  [org.keycloak.exportimport.dir.DirExportProvider] (ServerService Thread Pool -- 53) Realm 'keep-growing' - data exported
12:29:32,212 INFO  [org.keycloak.exportimport.dir.DirExportProvider] (ServerService Thread Pool -- 53) Users 0-3 exported
12:29:32,214 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0035: Export finished successfully
…

As a result, a group of new files appear in the keycloak/realms/export directory on my machine:

You can quit the process with Ctr+C.

If you want to import that realm, see the Keycloak in Docker #6 – How to import realms from a directory article as it describes in detail importing realms exported to a folder.

Export on a container startup

Additionally, we can configure our docker-compose.yml file to initiate the export when we start the container (by adding the command config). It can be useful when our docker image doesn’t provide an equivalent to the standalone.sh script or we just don’t want to use it. The modified compose file for a single file export looks like this:

version: '3.3'
services:
  keycloak:
    image: jboss/keycloak:${KEYCLOAK_VERSION}
    ports:
      - "8024:8080"
    environment:
      - KEYCLOAK_USER=${KEYCLOAK_USER}
      - KEYCLOAK_PASSWORD=${KEYCLOAK_PASSWORD}
    volumes:
      - ./keycloak/realms/export:/tmp/export
    command:
      - "-Dkeycloak.migration.action=export"
      - "-Dkeycloak.migration.provider=singleFile"
      - "-Dkeycloak.migration.realmName=keep-growing"
      - "-Dkeycloak.migration.usersExportStrategy=REALM_FILE"
      - "-Dkeycloak.migration.file=/tmp/export/keep-growing-realm.json"

We’ll see in logs that the container starts with the provided options:

The export will proceed as in the previously described methods.

Data included in the exported Keycloak realm resources

We’re going to find the secret generated for the spring-boot-example-app client in the realm config file:

Furthermore, the users are included in the exported assets as well. In case of the directory export the users are stored in the keep-growing-users-0.json file. For the single file export, the users are included directly in the keep-growing-realm.json file:

Troubleshooting

What to check when the export didn’t work as planned?

Null pointer exception

If you see the NullPointerException error in the logs make sure that the Dkeycloak.migration.realmName property value is consistent with the actual name of the realm you want to export. The following example error shows what happens if I provide non-existing to the command but my realm’s id is actually keep-growing:

11:42:20,795 INFO  [org.keycloak.services] (ServerService Thread Pool -- 54) KC-SERVICES0034: Export of realm 'non-existing' requested.
11:42:20,795 INFO  [org.keycloak.exportimport.singlefile.SingleFileExportProvider] (ServerService Thread Pool -- 54) Exporting realm 'non-existing' into file /tmp/export/keep-growing-realm.json
11:42:20,804 FATAL [org.keycloak.services] (ServerService Thread Pool -- 54) Error during startup: java.lang.NullPointerException

No such file or directory

If you see the No such file or directory message in the logs make sure that the Dkeycloak.migration.file property value points to a path that actually exists within the container. The following example error shows what happens if I provide /tmp/wrong/directory/path/keep-growing-realm.json to the command but Docker actually created a folder according to the mapped volume which points to /tmp/export:

Exporting realm 'keep-growing' into file /tmp/wrong/directory/path/keep-growing-realm.json
11:50:20,269 FATAL [org.keycloak.services] (ServerService Thread Pool -- 53) Error during startup: java.lang.RuntimeException: Error during export/import: /tmp/wrong/directory/path/keep-growing-realm.json (No such file or directory)

Verify that you actually restarted the container after adding the volume config to the docker-compose.yml file. Otherwise, the mapped directory won’t be created for you.

Missing export files

There are no errors in logs but you can’t find the exported files on your machine? Verify that the Dkeycloak.migration.file property value is consistent with the volume mapping. The following example shows what happens if I provide /tmp/keep-growing-realm-test.json instead of /tmp/export/keep-growing-realm-test.json:

You can always verify the location of the exported data in the command logs:

11:59:32,192 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0034: Export of realm 'keep-growing' requested.
11:59:32,192 INFO  [org.keycloak.exportimport.singlefile.SingleFileExportProvider] (ServerService Thread Pool -- 53) Exporting realm 'keep-growing' into file /tmp/keep-growing-realm-test.json
11:59:32,805 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0035: Export finished successfully

Read more on exporting Keycloak realm

• The Jboss image documentation on exporting a realm
• The Keycloak documentation on importing and exporting the database

Photo by RODNAE Productions from Pexels