Keycloak in Docker #5 – How to export a realm with users and secrets

featured image

Running a Keycloak service in a Docker container allows us to share its configuration across multiple environments. However, we can also export an entire Keycloak realm in case we need any backups or data transfer between servers.

Prerequisites

For reference, below you’ll find the resulting directory tree for this example:

directory tree for Keycloak volumes with exported realm

In short, my goal is to get the export directory containing realm resources (in this case, the keep-growing-realm.json file). Thanks to this, I’ll be able to backup or recreate a complete realm whenever I need to.

Why not export with Keycloak Admin console?

The Keycloak Admin Console provides an easy way to export a realm. However, as you can see in the screenshot below, not all resources can be exported with this method:

Keycloak realm export through Admin Console

The realm-export.json file produced with this technique won’t contain user data. In addition, client secrets will be masked. Although this approach might be appropriate in some use cases, we won’t be able to recreate the same instance using only this file.

If we want to get realm data suitable for backups or cross server migration, we need to run a boot-time export.

My example Keycloak realm

I have my keep-growing realm with the following users:

realm users that will be exported

Furthermore, the realm contains the spring-boot-example-app client. It has the confidential access type and a secret, as you can see in the screenshot below:

client secret that will be exported

At the end of this article, I will verify that the users and the secret are properly exported.

How to export all resources from a Keycloak realm

I’m going to use a volume to map exported data between the container and my machine. Next, I will show you how to export the data with a command or automatically when running the container. Finally, I’m going to verify if the export was successful.

Add a volume for exported resources

First, we need to start our container with a volume that will hold exported data. Therefore, I’m going to map the ./keycloak/realms/export folder on my machine to the /tmp/export directory in the keycloak service. Below you’ll find my docker-compose.yml file:

I’m using my default environmental variables defined in the following .env file:

Now I’m going to start the service with the docker-compose up -d command. As we can see in the following screenshot, the volume is properly created and mapped:

volume binding for exported Keycloak realm

Obviously, the /tmp/export directory inside the dockerized Keycloak instance is empty as we still need to export our realm:

empty export directory in the docker container

Keycloak export options explained

Let me introduce the properties that we’re going to use:

  • Djboss.socket.binding.port-offset, we want to run the export on a different port than Keycloak itself. By default, the jboss server in my kecyloak instance uses the 9990 port. Therefore, providing e.g. 100 as a value here will result in the command using the 10090 port.
  • Dkeycloak.migration.action, we specify what we want to do. Available options:
    • export,
    • import.
  • Dkeycloak.migration.provider, we define how we want the data. Available options are:
    • singleFile, if we want to export data into a file
    • dir, if we want to export data into a directory.
  • Dkeycloak.migration.realmName, realm for export. Don’t specify this parameter if you want to export all realms.
  • Dkeycloak.migration.usersExportStrategy, how to export realm users. Available options are:
    • DIFFERENT_FILES, it’s the default option for this attribute, we will get a file for realm config and multiple files for users (depending on how many users we have and the number of users per one file (50 by default));
    • SKIP, won’t export users;
    • REALM_FILE, export users to the same file as realm configuration;
    • SAME_FILE, we will get one file for realm configuration and one for all users (keep-growing-realm.json and keep-growing-users.json).
  • Dkeycloak.migration.file, the file where we want to save data. Remember to use the exact same value as you used for the volume mapping.

Run the export command

We’re going to execute the /opt/jboss/keycloak/bin/standalone.sh script included in the jboss/keycloak image to initiate the export process:

To clarify the following examples, my example keycloak service runs in the keycloakspringboot_keycloak_1 container.

Export to a single file

When a realm doesn’t contain a lot of users it might be appropriate to export it into a single file. The actual command that I’m going to execute looks like in the snippet below:

If the command encounters no problems, you’ll see console output similar to this:

As a result, a new file appears in the keycloak/realms/export directory on my machine:

exported Keycloak realm in a file

You can quit the process with Ctr+C.

If you want to import that realm, the Keycloak in Docker #2 – How to import a Keycloak realm article describes in detail importing a realm from a file.

Export to a directory

Keycloak recommends exporting data into a directory if your realm contains more than 500 users:

Exporting many users into a directory performs optimally as the directory provider uses a separate transaction for each “page” (a file of users).

https://www.keycloak.org/docs/16.1/server_admin/#assembly-exporting-importing_server_administration_guide

In this case, my command would look like this (notice changes in lines 4 and 6):

If the command encounters no problems, you’ll see console output similar to this:

As a result, a group of new files appear in the keycloak/realms/export directory on my machine:

exported Keycloak realm in a directory

You can quit the process with Ctr+C.

If you want to import that realm, see the Keycloak in Docker #6 – How to import realms from a directory article as it describes in detail importing realms exported to a folder.

Export on a container startup

Additionally, we can configure our docker-compose.yml file to initiate the export when we start the container (by adding the command config). It can be useful when our docker image doesn’t provide an equivalent to the standalone.sh script or we just don’t want to use it. The modified compose file for a single file export looks like this:

We’ll see in logs that the container starts with the provided options:

exporting Keycloak realm on container startup

The export will proceed as in the previously described methods.

Data included in the exported Keycloak realm resources

We’re going to find the secret generated for the spring-boot-example-app client in the realm config file:

exported client secret

Furthermore, the users are included in the exported assets as well. In case of the directory export the users are stored in the keep-growing-users-0.json file. For the single file export, the users are included directly in the keep-growing-realm.json file:

exported users

Troubleshooting

What to check when the export didn’t work as planned?

Null pointer exception

If you see the NullPointerException error in the logs make sure that the Dkeycloak.migration.realmName property value is consistent with the actual name of the realm you want to export. The following example error shows what happens if I provide non-existing to the command but my realm’s id is actually keep-growing:

No such file or directory

If you see the No such file or directory message in the logs make sure that the Dkeycloak.migration.file property value points to a path that actually exists within the container. The following example error shows what happens if I provide /tmp/wrong/directory/path/keep-growing-realm.json to the command but Docker actually created a folder according to the mapped volume which points to /tmp/export:

Verify that you actually restarted the container after adding the volume config to the docker-compose.yml file. Otherwise, the mapped directory won’t be created for you.

Missing export files

There are no errors in logs but you can’t find the exported files on your machine? Verify that the Dkeycloak.migration.file property value is consistent with the volume mapping. The following example shows what happens if I provide /tmp/keep-growing-realm-test.json instead of /tmp/export/keep-growing-realm-test.json:

realm exported to a wrong path

You can always verify the location of the exported data in the command logs:

Read more on exporting Keycloak realm

Photo by RODNAE Productions from Pexels

2 thoughts on “Keycloak in Docker #5 – How to export a realm with users and secrets

  1. Hi,
    While running command to export in folder, getting below error 
    /tmp/export/testing-realm-realm.json (Permission denied)
    3:41:45,821 INFO  [org.hibernate.orm.beans] (ServerService Thread Pool — 59) HHH10005002: No explicit CDI BeanManager reference was passed to Hibernate, but CDI is available on the Hibernate ClassLoader.
    13:41:46,076 INFO  [org.hibernate.validator.internal.util.Version] (ServerService Thread Pool — 59) HV000001: Hibernate Validator 6.0.22.Final
    13:41:47,433 INFO  [org.hibernate.hql.internal.QueryTranslatorFactoryInitiator] (ServerService Thread Pool — 59) HHH000397: Using ASTQueryTranslatorFactory
    13:41:48,485 INFO  [org.keycloak.services] (ServerService Thread Pool — 59) KC-SERVICES0034: Export of realm ‘testing-realm’ requested.
    13:41:49,063 FATAL [org.keycloak.services] (ServerService Thread Pool — 59) Error during startup: java.lang.RuntimeException: Error during export/import: /tmp/export/testing-realm-realm.json (Permission denied)
    at org.keycloak.keycloak-services@17.0.1//org.keycloak.exportimport.util.ExportImportSessionTask.run(ExportImportSessionTask.java:37)
    at org.keycloak.keycloak-server-spi-private@17.0.1//org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:239)
    at org.keycloak.keycloak-services@17.0.1//org.keycloak.exportimport.util.MultipleStepsExportProvider.exportRealmImpl(MultipleStepsExportProvider.java:74
     
     
    Kecloack container: 
    drwxr-xr-x 2 root  root 4096 Apr  5 11:13 export
    drwxr-xr-x 2 jboss root 4096 Apr  5 13:41 hsperfdata_jboss
    drwxr-xr-x 1 root  root 4096 Mar 30 13:42 hsperfdata_root
    -rwx—— 1 root  root  291 Mar 28 09:18 ks-script-c9hd6mir
    -rwx—— 1 root  root  701 Mar 28 09:18 ks-script-jrnjxyf4
     
     

    1. The permission problem seems to be due to the fact that the directory export belongs to the user root. In my configuration, the owner is jboss:

      drwxrwxr-x 2 jboss 1000 4096 Feb 3 12:52 export
      drwxr-xr-x 2 jboss root 4096 Apr 7 20:04 hsperfdata_jboss
      drwxr-xr-x 1 root root 4096 Jan 25 15:26 hsperfdata_root

      The similar exception will be thrown when the volume on your local machine is owned by the root.

Leave a Reply

Your email address will not be published.