Prerequsites

As an example for this article, I’m going to configure role-based access control. If you want to recreate the work described below, you’re going to need a Spring Boot project with:

• a REST controller with at least one endpoint,
• spring-boot-starter-security and spring-security-test dependencies.

You can find an example project in the spring-boot-user-roles-management repository.

Summarizing the work described below: first I will create an example role and add it to my default user via the application properties. Then I will restrict access to POST endpoints to only users with this role. Finally, I am going to include new access rules in my MVC tests.

Create Enum for user roles

I’m going to start with creating a class for user roles. In straightforward cases, a simple Enum will suffice. However, if we need to encapsulate a role name that does not follow the Enum naming convention, we need to add an additional field for the value. Below you’ll find solutions for both situations.

Roles in a simple Enum

For a simple use case, I’m going to create the following UserRole Enum with a single role:

package in.keepgrowing.springbootuserrolesmanagement.security.domain.model;

public enum UserRole {

    CHIEF_OPERATING_OFFICER
}

In my sample project, I defined a default user with the spring.security.user.* properties. Now, I have to add the role in my application.properties file:

# application.properties
…
spring.security.user.roles=CHIEF_OPERATING_OFFICER

Thanks to this, the application starts with a default user who already has the authority and I will be able to quickly test my configuration later:

Role name as Enum value

In case we don’t have control over the role names (e.g. they are created in an external authorization service like Keycloak and may look like chief-operation-officer), we can still provide a convenient mapping. We just need to encapsulate the external name in an Enum value and override the toString() method:

package in.keepgrowing.springbootuserrolesmanagement.security.domain.model;

public enum UserRole {

    CHIEF_OPERATING_OFFICER("chief-operating-officer");

    final String value;

    UserRole(String value) {
        this.value = value;
    }

    @Override
    public String toString() {
        return value;
    }
}

As before, I need to add the role to my default user defined with the spring.security.user.* properties:

# application.properties
…
spring.security.user.roles=chief-operating-officer

Security configuration

We can define role-based security rules in the HttpSecurity configuration or for each method / class. Below you’ll find example implementation for both cases.

Specify access rules in the HttpSecurity config

In the following HttpSecurity configuration, I’m going to allow only users with the required role to access POST endpoints:

package in.keepgrowing.springbootuserrolesmanagement.security.config;

import in.keepgrowing.springbootuserrolesmanagement.security.domain.model.UserRole;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
…
protected void configure(HttpSecurity http) throws Exception {
    …
    .authorizeHttpRequests()
    .antMatchers(HttpMethod.POST).hasRole(UserRole.CHIEF_OPERATING_OFFICER.toString())
    .anyRequest().authenticated()
…

As a result, POST endpoints will be inaccessible to anyone except a COO.

As a side note, I’m using the authorizeHttpRequests() method because it seems that autorizeRequests() will be deprecated. Furthermore, you can use mvcMatchers() instead of antMatchers() if you need a different approach to pattern matching, like in the snippet below:

…
.mvcMatchers(HttpMethod.POST, "/example").hasRole(UserRole.CHIEF_OPERATING_OFFICER.toString())

Verify user role at the method level

When we need more precise access control, we can apply method level security. First, I’m going to enable method security using the following class:

package in.keepgrowing.springbootuserrolesmanagement.security.config;

import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;

@EnableMethodSecurity
public class CustomMethodSecurityConfig {
}

Annotate your class with @EnableGlobalMethodSecurity if you use Spring Security version < 5.6.

Next, I’m going to add the @PreAuthorize annotation onto the example POST endpoint:

package in.keepgrowing.springbootuserrolesmanagement.example.adapters.driving.api.controllers;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ExampleController {

    @PostMapping
    @PreAuthorize("hasRole(T(in.keepgrowing.springbootuserrolesmanagement.security.domain.model.UserRole).CHIEF_OPERATING_OFFICER.toString())")
    public ResponseEntity<Void> post() {
        return ResponseEntity.ok().build();
    }
…

Unfortunately, we have to use the fully-qualified path in the SpEL expression. Obviously, I don’t want to clutter controllers with such low-level details. Therefore, I’m going to create a custom annotation.

Create a custom annotation to hide irrelevant details

Luckily, creating a meta annotation is simple:

package in.keepgrowing.springbootuserrolesmanagement.security.adapters.driving.spring.annotations;

import org.springframework.security.access.prepost.PreAuthorize;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target({ElementType.TYPE, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@PreAuthorize(value = "hasRole(T(in.keepgrowing.springbootuserrolesmanagement.security.domain.model.UserRole).CHIEF_OPERATING_OFFICER.toString())")
public @interface MustBeChiefOperatingOfficer {
}

Let’s take a look at the annotations I’m using here:

1. @Target – puts constrains on the usage of the annotation. The TYPE Element Type allows using it on a class (e.g. controller classes), interface, Enum or record declaration. I’m also providing the METHOD as a possible target to allow using this annotation on a single endpoint.
2. @Retention – specifies how long an annotation is to be retained. The RUNTIME Retention Policy enables the annotation to be retained by the VM at run time.
3. @PreAuthorize – the original annotation that I want to remove from the controller code.

As a result, all irrelevant details remain hidden from my controllers:

package in.keepgrowing.springbootuserrolesmanagement.example.adapters.driving.api.controllers;

import in.keepgrowing.springbootuserrolesmanagement.security.adapters.driving.spring.annotations.MustBeChiefOperatingOfficer;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ExampleController {

    @PostMapping
    @MustBeChiefOperatingOfficer
    public ResponseEntity<Void> post() {
        return ResponseEntity.ok().build();
    }
…

Summary

By keeping user roles in Enum and using them in annotations as described above, we can take advantage of our IDE when maintaining the authorities.

Tests

Unfortunately, I haven’t found a convenient method of using Enum when mocking a user role in tests. Therefore, as we can see in the snippet below, we still have to provide the required role as a simple String:

package in.keepgrowing.springbootuserrolesmanagement.example.adapters.driving.api.controllers;

import in.keepgrowing.springbootuserrolesmanagement.testing.annotations.WithMockChiefOperationOfficer;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.security.test.context.support.WithMockUser;
import org.springframework.test.web.servlet.MockMvc;

import static org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.csrf;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@WebMvcTest
class ExampleControllerTest {

    private static final String PATH = "/" + ExampleControllerPaths.EXAMPLE_PATH;

    @Autowired
    private MockMvc mvc;

    @Test
    @WithMockUser(roles = "CHIEF_OPERATING_OFFICER")
    void shouldCallPostEndpoint() throws Exception {
        mvc.perform(post(PATH)
                        .with(csrf()))
                .andExpect(status().isOk());
    }
…

To simplify maintenance of the tests, we can create our custom annotation to mock users with the CHIEF_OPERATING_OFFICER role:

package in.keepgrowing.springbootuserrolesmanagement.testing.annotations;

import org.springframework.security.test.context.support.WithMockUser;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target({ElementType.TYPE, ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@WithMockUser(roles = "CHIEF_OPERATING_OFFICER")
public @interface WithMockChiefOperationOfficer {
}

For more advanced mapping, when we keep the actual user role in the Enum value:

package in.keepgrowing.springbootuserrolesmanagement.testing.annotations;
…
@WithMockUser(roles = "chief-operating-officer")
public @interface WithMockChiefOperationOfficer {
}

As a result, we don’t have to provide user role as a string in every test class or test method:

package in.keepgrowing.springbootuserrolesmanagement.example.adapters.driving.api.controllers;

import in.keepgrowing.springbootuserrolesmanagement.testing.annotations.WithMockChiefOperationOfficer;
…
    @Test
    @WithMockChiefOperationOfficer
    void shouldDenyAccessToPostEndpoint() throws Exception {
        mvc.perform(post(PATH)
                        .with(csrf()))
                .andExpect(status().isForbidden());
    }
…

Read more on simplifying roles management in Spring

• How to create typesafe user roles for Spring Security?
• Spring Security @PreAuthorization pass enums in directly
• Keycloak with Spring Boot #4 – Simple guide for roles and authorities

Photo by Kindel Media from Pexels

The post Simplify the management of user roles in Spring Boot appeared first on keep_growing.

Prerequisites

As an example for this article, I’m going to create a custom annotation to configure controller tests. If you want to recreate the work described below, you will need a Spring Boot project with a REST controller and some MVC tests.

Why use a custom annotation in Spring Boot tests

One reason is to simply group all the annotations we need to apply across multiple test classes into convenient metadata. Moreover, it’s also useful when you use @Import to register additional components required for configuring the “slice” of the system to test, as it reduces a lot of copy-and-paste code.

Another possible solution would be to create a base class and making each test extend it. However, inheritance comes with a lot of issues that I don’t want to deal with just to provide a common configuration for some tests.

A custom annotation that encloses all the common configurations for tests is both an elegant and simple solution. In the following example, instead of annotating my tests with @WebMvcTest and @Import, I’m going to create @RestControllerIntegrationTest which will take care of all the configuration details. Thanks to this, I will avoid duplicating the code or complicating it with unnecessary inheritance.

Create a custom annotation for MVC tests

First, I’m going to replace the @WebMvcTest annotation with a custom one. I’m going to create the following RestControllerIntegrationTest annotation:

package in.keepgrowing.keycloakspringboot.testing.annotations;

import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest;
import org.springframework.core.annotation.AliasFor;
import org.springframework.test.context.ContextConfiguration;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
@WebMvcTest
public @interface RestControllerIntegrationTest {

    /**
     * @see WebMvcTest#value
     */
    @AliasFor(annotation = WebMvcTest.class, attribute = "value")
    Class<?>[] value() default {};

    /**
     * @see WebMvcTest#controllers
     */
    @AliasFor(annotation = WebMvcTest.class, attribute = "controllers")
    Class<?>[] controllers() default {};
}

Let’s take a look at the annotations I’m using here:

1. @Retention – specifies how long an annotation is to be retained. The RUNTIME Retention Policy enables the annotation to be retained by the VM at run time.
2. @Target – puts constrains on the usage of the annotation. The TYPE Element Type will allow using it on my test classes.
3. @WebMvcTest – since I am creating my own annotation for MVC tests, i need to include this Spring Boot annotation to still benefit from all the default features.

Thus, I can now replace @WebMvcTest with @RestControllerIntegrationTest and still have the same functionality:

package in.keepgrowing.keycloakspringboot.products.adapters.driving.api.http.controllers;

import in.keepgrowing.keycloakspringboot.testing.annotations.RestControllerIntegrationTest;
…
@RestControllerIntegrationTest(value = ProductController.class)
class ProductControllerTest {
…

Now, I can add any additional configuration my test “slice” requires to the custom annotation.

Provide additional configuration

In my example project, I’m using the @PreAuthorize annotation to restrict user access to the API endpoints. Therefore, I need to enable method-level security. Without it, the following test fails because it is missing the configuration required to apply the appropriate access control:

For that reason, I’m going to create a configuration class and annotate it with @EnableMethodSecurity:

package in.keepgrowing.keycloakspringboot.testing.config;

import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;

@TestConfiguration
@EnableMethodSecurity
public class ControllerIntegrationTestConfig {
}

Next, I’ll load it into ApplicationContext using @ContextConfiguration in my custom annotation:

package in.keepgrowing.keycloakspringboot.testing.annotations;

import in.keepgrowing.keycloakspringboot.testing.config.ControllerIntegrationTestConfig;
…
@ContextConfiguration(classes = ControllerIntegrationTestConfig.class)
public @interface RestControllerIntegrationTest {

Verify results

As a result, all tests in classes annotated with @RestControllerIntegrationTest will be able to use method-level security. The following screenshot shows that my tests are passing now:

Read more about setting up tests in Spring Boot

• Spring Boot test slices with custom annotations
• Document how to use this if Keycloak is on the classpath
• My other posts regarding testing

Photo by Georgia Maciel from Pexels

The post Create a custom annotation to configure Spring Boot tests appeared first on keep_growing.

Prerequisities

• The main principles of managing roles in Keycloak are described in the Keycloak in Docker #4 – How to define user privileges and roles post.
• Furthermore, you can follow the Keycloak with Spring Boot #1 – Configure Spring Security with Keycloak post to recreate the example configuration I use in this article.

First, I will describe some debugging techniques. Next, I’ll show you some sample role mapping setups.

Debugging role mapping between Keycloak and Spring Boot

I’m going to show you some useful methods for verifying role support in a Spring Boot app and Keycloak server. By seeing what is really going on under the hood, you can save a lot of time when an apparently correct configuration does not bring the expected results.

How to debug roles in Spring Security Context

We can look into the Spring Security Context to verify the actual permissions that are being assessed in two ways:

• Set the security logging level to DEBUG – add the logging.level.org.springframework.security=DEBUG property to the application.properties file. Thanks to this, after each API request you’ll see a list of Granted Authorities in the logs:

DEBUG --- w.c.HttpSessionSecurityContextRepository : Stored SecurityContextImpl [Authentication=KeycloakAuthenticationToken [Principal=…, Granted Authorities=[ROLE_chief-operating-officer, ROLE_user]]] to HttpSession

• Log Authorities manually – if you only want to log authorities for a single request, you can obtain Authentication from Security Context Holder and log GrantedAuthorities directly inside a selected endpoint:

package in.keepgrowing.keycloakspringboot.products.adapters.driving.api.http.controllers;

import in.keepgrowing.keycloakspringboot.products.adapters.driving.api.http.model.responses.ProductResponse;
import in.keepgrowing.keycloakspringboot.products.adapters.driving.api.http.services.ProductHttpApiFacade;
import lombok.extern.log4j.Log4j2;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.web.bind.annotation.*;
import java.util.List;
…
@RestController
@RequestMapping(value = ProductControllerPaths.PRODUCTS_PATH, produces = MediaType.APPLICATION_JSON_VALUE)
@Log4j2
public class ProductController {
    …
    @GetMapping
    public ResponseEntity<List<ProductResponse>> findAll() {
        Authentication auth = SecurityContextHolder.getContext().getAuthentication();
        auth.getAuthorities().forEach(a->log.info(String.valueOf(a)));

        return new ResponseEntity<>(apiFacade.findAll(), HttpStatus.OK);
    }
…

As a result, when you call this endpoint, you’ll see log entries similar to these:

INFO --- i.k.s.p.p.controllers.ProductController  : ROLE_CHIEF-OPERATING-OFFICER
INFO --- i.k.s.p.p.controllers.ProductController  : ROLE_USER

How to debug roles sent from Keycloak

In my example realm, there is a default user christina who has the roles of user and chief-operation-officer. We can verify her privileges by copying the token from Postman or from the developer tool in the browser as seen in the screenshots below:

Then, decode the token value in e.g. jwt.io tool to see the actual content:

Roles vs Authorities

Spring Security allows us to configure privileges in a very granular manner with Authorities (e.g. CAN_WRITE). However, we can also manage resource access in a more coarse fashion with Roles (e.g ROLE_EDITOR). In other words, the ROLE_ prefix is what differentiate these concepts in Spring.

Keycloak recognises this naming convention. Therefore, we can decide whether we want to map privileges defined in our Keycloak server as roles or authorities in Spring Boot.

Mapping Roles

To map my chief-operating-officer Keycloak role to ROLE_CHIEF-OPERATING-OFFICER in Spring Boot, I’m going to use SimpleAuthorityMapper:

package in.keepgrowing.keycloakspringboot.security.config;

import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.authentication.KeycloakAuthenticationProvider;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.core.authority.mapping.SimpleAuthorityMapper;

@KeycloakConfiguration
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
    …
    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) {
        auth.authenticationProvider(getKeycloakAuthenticationProvider());
    }

    private KeycloakAuthenticationProvider getKeycloakAuthenticationProvider() {
        KeycloakAuthenticationProvider authenticationProvider = keycloakAuthenticationProvider();
        var mapper = new SimpleAuthorityMapper();
        mapper.setConvertToUpperCase(true);
        authenticationProvider.setGrantedAuthoritiesMapper(mapper);

        return authenticationProvider;
    }
    …
}

As we can see in the SimpleAuthorityMapper documentation, the default prefix is ROLE_:

Therefore, we don’t need to set it manually.

As a result, I can impose access restrictions to selected resources. Below you can see the role used in the @PreAuthorize annotation:

package in.keepgrowing.springbootswaggeruikeycloak.products.presentation.controllers;

import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.web.bind.annotation.*;
…
public class ProductController {
    …
    @PostMapping()
    @PreAuthorize("hasRole('CHIEF-OPERATING-OFFICER')")
    public ResponseEntity<Product> save(@RequestBody Product productDetails) {
        …
    }
}

On the other hand, I can restrict access to all POST endpoints only to users with this role:

package in.keepgrowing.keycloakspringboot.security.config;

import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
…

@KeycloakConfiguration
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
…
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        super.configure(http);
        http
                …
                .antMatchers(HttpMethod.POST).hasRole("CHIEF-OPERATING-OFFICER")
                …
    }
…

In both cases, the hasRole expression will automatically add the ROLE_ prefix to the given CHIEF-OPERATING-OFFICER value and compare the result with the ROLE_CHIEF-OPERATING-OFFICER GrantedAuthority value which we have mapped in the Spring SecurityContext. This behaviour is described in the method docs:

Mapping Authorities

If you want to control access in more detail, you can map the permissions from Keycloak to Authorities in Spring Security. Remove the ROLE_ prefix when configuring the authority mapper:

package in.keepgrowing.keycloakspringboot.security.config;
…
    private KeycloakAuthenticationProvider getKeycloakAuthenticationProvider() {
        KeycloakAuthenticationProvider authenticationProvider = keycloakAuthenticationProvider();
        var mapper = new SimpleAuthorityMapper();
        mapper.setConvertToUpperCase(true);
        mapper.setPrefix("");
        authenticationProvider.setGrantedAuthoritiesMapper(mapper);

        return authenticationProvider;
    }
    …

This makes the sample can-write permission from Keycloak become CAN-WRITE authority in Spring SecurityContext.

Below you can see the authority used in the @PreAuthorize annotation:

package in.keepgrowing.springbootswaggeruikeycloak.products.presentation.controllers;

import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.web.bind.annotation.*;
…
public class ProductController {
    …
    @PostMapping()
    @PreAuthorize("hasAuthority('CAN-WRITE')")
    public ResponseEntity<Product> save(@RequestBody Product productDetails) {
        …
    }
}

Then again, I can restrict access to all POST endpoints only to users with this authority:

package in.keepgrowing.keycloakspringboot.security.config;

import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.http.HttpMethod;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
…

@KeycloakConfiguration
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
…
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        super.configure(http);
        http
                …
                .antMatchers(HttpMethod.POST).hasAuthority(("CAN-WRITE")
                …
    }
…

In both cases, the hasAuthority expression will compare the given CAN-WRITE value with the CAN-WRITE GrantedAuthority value which we have mapped in the Spring SecurityContext.

If you configure the authority mapper in a way that removes the ROLE_ prefix from the authorities, don’t use hasRole in security expressions and configuration.

Realm roles vs client roles

You can read about differences between the realm and client roles in the Keycloak in Docker #4 – How to define user privileges and roles article. Given that the Keycloak server contains properly defined user roles, we need to instruct our Spring Boot app which role set to evaluate. We’ll do so with the keycloak.use-resource-role-mappings property.

Evaluate realm roles in Spring Boot

A Spring Boot app will evaluate the realm roles if the keycloak.use-resource-role-mappings property is set to false, which is the default value. You can verify that the values from the realm-access field in the access token are present as GrantedAuthirities in the SecurityContext using the debug methods shown above.

Evaluate client roles in Spring Boot

In the keycloak.* properties, resource is the client. Therefore, by setting keycloak.use-resource-role-mappings to true, we’re telling Spring Boot that the client roles should be considered. You can verify that the values from the resource-access field in the access token are present as GrantedAuthirities in the SecurityContext using the debug methods shown above.

Handle user roles in Spring Boot tests

When we restrict endpoint access to a specific role, we need to update Spring MVC tests to keep them working properly.

Custom annotation for a mocked user

Fortunately, Spring provides us with the @WithMockUser annotation that allows us to test an endpoint for a given user role, e.g.:

package in.keepgrowing.keycloakspringboot.products.adapters.driving.api.http.controllers;

import org.springframework.security.test.context.support.WithMockUser;
…
@Test
@WithMockUser(roles="CHIEF-OPERATING-OFFICER")
void shouldDeleteProduct() throws Exception {
    mvc.perform(delete(BASE_PATH + "/" + TEST_UUID)
                    .contentType(MediaType.APPLICATION_JSON)
                    .with(csrf()))
            .andExpect(status().isNoContent());
}

In the same manner we can mock a test user with a given authority:

…
@WithMockUser(authorities="CAN-WRITE")
…

However, having to manually enter a role name for each test case or test class where that role matters would be cumbersome. Instead, we can create a custom meta annotation:

package in.keepgrowing.keycloakspringboot.testing.annotations;

import org.springframework.security.test.context.support.WithMockUser;

import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;

@Retention(RetentionPolicy.RUNTIME)
@WithMockUser(roles="CHIEF-OPERATING-OFFICER")
public @interface WithMockChiefOperationOfficer {
}

And then use it in test cases (or classes) like in the example snippet below:

package in.keepgrowing.keycloakspringboot.products.adapters.driving.api.http.controllers;

import in.keepgrowing.keycloakspringboot.testing.annotations.WithMockChiefOperationOfficer;
…
@Test
@WithMockChiefOperationOfficer
void shouldDeleteProduct() throws Exception {
…

Enable method-level security

If we’re using annotations like @PreAuthorize to restrict access to endpoints, we have to create an appropriate configuration class for our MVC tests:

package in.keepgrowing.keycloakspringboot.testing.config;
 
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.security.config.annotation.method.configuration.EnableMethodSecurity;
 
@TestConfiguration
@EnableMethodSecurity
public class ControllerIntegrationTestConfig {
}

Then we have to load it when running the tests. We can achieve it by adding @Import or a custom annotation to the test classes.

Read more on handling Keycloak roles in Spring Boot

• Difference between Role and GrantedAuthority in Spring Security
• Spring security added prefix “ROLE_” to all roles name?
• SimpleAuthorityMapper.java in the GitHub repo
• Testing With Spring Security

Photo by Robert Nagy from Pexels

The post Keycloak with Spring Boot #4 – Simple guide for roles and authorities appeared first on keep_growing.

Prerequisites

• The security configuration for Keycloak is described in the Keycloak with Spring Boot #1 – Configure Spring Security with Keycloak post.
• You’ll need Swagger UI in your project. I’m using the springdoc-openapi dependency in my example:

<!-- pom.xml -->
…	
<properties>
    …
    <springdoc.version>1.6.6</springdoc.version>
</properties>
…
<dependencies>
    …
    <dependency>
        <groupId>org.springdoc</groupId>
        <artifactId>springdoc-openapi-ui</artifactId>
        <version>${springdoc.version}</version>
    </dependency>
</dependencies>
…

As a result, I have Swagger UI at http://localhost:8080/swagger-ui.html and OpenAPI specification at http://localhost:8080/v3/api-docs. For more information on documenting Spring Boot applications using Springdoc, see the Easy OpenAPI 3 specification for your Spring Boot REST API article.

Authorize Swagger requests in Keycloak

I’ll demonstrate three authorization configurations:

• Authorization Code which is one of the OAuth 2.0 flows,
• OpenID Connect Discovery mechanism,
• Bearer Authentication.

Select the one that better suits the needs of your project. In the end, we want to end up with the Authorize button that will provide authorization options suitable for our API and Keycloak server:

Allow access to Swagger endpoints

I’m going to modify my SecurityConfig class to allow Swagger endpoints to be called without authentication:

package in.keepgrowing.keycloakspringboot.security.config;

public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {

    private static final String[] SWAGGER_WHITELIST = {
            "/v3/api-docs/**",
            "/swagger-ui/**",
            "/swagger-ui.html",
    };

    public static void configureApiSecurity(HttpSecurity http) throws Exception {
        http
                …
                .antMatchers(SWAGGER_WHITELIST).permitAll()
                .anyRequest().authenticated();
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        super.configure(http);
        configureApiSecurity(http);
    }
…

Provide Keycloak server details

Authorizatin Code and OpenID Connect Disvovery require realm name and the url of the Keycloak server for proper configuration. Although we can provide them as hardcoded strings for testing purposes, I’m going to reuse the data that’s already available in my application.properties file:

# application.properties
keycloak.realm=keep-growing
keycloak.auth-server-url=http://localhost:8024/auth
…

To access the properties in the code, I’m going to create the following short record:

package in.keepgrowing.keycloakspringboot.security.config;

import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "keycloak")
public record KeycloakProperties(
        String authServerUrl,
        String realm) {
}

That’s all we need to inject these properties into our Swagger config if our project uses a Java version with Record support. Alternatively, follow the Type-safe Configuration Properties::Constructor Binding section in the Spring docs to use a simple class (with getters and @ConstructorBinding annotation).

Thanks to this, I only provide Keycloak details in one place and I don’t have to painstakingly search my configuration classes to update the values. Additionally, the properties will always be in accordance with the selected application profile (dev, production, etc.).

Swagger authorization with the OAuth 2.0 protocol

The example implementation is available in the keycloak-spring-boot repository.

First of all, I’m going to consult the Swagger docs for a quick summary of the authorization flows available:

The flows (also called grant types) are scenarios an API client performs to get an access token from the authorization server. OAuth 2.0 provides several flows suitable for different types of API clients:
– Authorization code,
– Implicit,
– Resource owner password credentials,
– Client credentials.

https://swagger.io/docs/specification/authentication/oauth2/

I’m going to provide the configuration for the Authorization Code Grant as this is the most secure option for my Swagger API calls. The following quote contains a brief explanation of this flow:

When the user authorizes the application, they are redirected back to the application with a temporary code in the URL. The application exchanges that code for the access token. When the application makes the request for the access token, that request can be authenticated with the client secret, which reduces the risk of an attacker intercepting the authorization code and using it themselves.

This also means the access token is never visible to the user or their browser, so it is the most secure way to pass the token back to the application, reducing the risk of the token leaking to someone else.

https://www.oauth.com/oauth2-servers/server-side-apps/authorization-code/

Make sure that the Standard Flow is enabled in Keycloak as it allows us to use the Authorization Code flow:

Configuration class

Let’s look at the example configuration documented in Swagger:

components:
  securitySchemes:
    oAuth2AuthCode:
      type: oauth2
      description: For more information, see https://api.slack.com/docs/oauth
      flows: 
        authorizationCode:
          authorizationUrl: https://slack.com/oauth/authorize
          tokenUrl: https://slack.com/api/oauth.access
          scopes:
            users:read: Read user information
            …

In other words, in order to authorize requests in Keycloak, Swagger requires a component containing a security scheme with a particular flow. Based on this, I can provide my own setup in Java. Below you can see the full configuration class:

package in.keepgrowing.keycloakspringboot.documentation.config;

import in.keepgrowing.keycloakspringboot.security.config.KeycloakProperties;
import io.swagger.v3.oas.annotations.OpenAPIDefinition;
import io.swagger.v3.oas.models.Components;
import io.swagger.v3.oas.models.OpenAPI;
import io.swagger.v3.oas.models.security.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@OpenAPIDefinition
public class SwaggerConfig {

    private static final String OAUTH_SCHEME_NAME = "oAuth";
    private static final String PROTOCOL_URL_FORMAT = "%s/realms/%s/protocol/openid-connect";

    @Bean
    public OpenAPI customOpenApi(KeycloakProperties keycloakProperties) {
        return new OpenAPI()
                .components(new Components() //1
                        .addSecuritySchemes(OAUTH_SCHEME_NAME, createOAuthScheme(keycloakProperties)))
                .addSecurityItem(new SecurityRequirement().addList(OAUTH_SCHEME_NAME));
    }

    private SecurityScheme createOAuthScheme(KeycloakProperties properties) {
        OAuthFlows flows = createOAuthFlows(properties); //3a

        return new SecurityScheme() //2
                .type(SecurityScheme.Type.OAUTH2)
                .flows(flows);
    }

    private OAuthFlows createOAuthFlows(KeycloakProperties properties) {
        OAuthFlow flow = createAuthorizationCodeFlow(properties);

        return new OAuthFlows()
                .authorizationCode(flow);
    }

    private OAuthFlow createAuthorizationCodeFlow(KeycloakProperties properties) {
        var protocolUrl = String.format(PROTOCOL_URL_FORMAT, properties.authServerUrl(), properties.realm());

        return new OAuthFlow()//3b
                .authorizationUrl(protocolUrl + "/auth")
                .tokenUrl(protocolUrl + "/token")
                .scopes(new Scopes().addString("openid", ""));
    }
}

There are a few things to note:

1. Components – a collection of reusable objects that define various aspects of the Open API Specification.
2. SecuritySceme – here we can define our scheme type with the Authorization Code flow.
3. OAuthflow – an object for specifying the configuration details of the selected flow. We have to follow the list of the required fields for each auth type from the screenshot below:

As a result, we’ll see that the Authiorization Code Grant will be the only available auth option in Swagger UI:

Adding Scopes to the config

In the screenshot with required fields for the OAuth flow, we can read that scopes field is just a map between their names and descriptions:

Therefore, if you want to provide a scope, e.g. address, you need to add it when creating the authorization flow:

…
private OAuthFlow createAuthorizationCodeFlow(KeycloakProperties properties) {
    var protocolUrl = String.format(PROTOCOL_URL_FORMAT, properties.getAuthServerUrl(), properties.getRealm());

    return new OAuthFlow()
        .authorizationUrl(protocolUrl + "/auth")
        .tokenUrl(protocolUrl + "/token")
        .scopes(new Scopes().addString("address", ""));
}

As a consequence, the list of scopes will appear in the Swagger authorization window after restarting the application:

Another thing to remember is that if you don’t add the openid scope, you’ll get the following warning in the keycloak instance’s logs:

Request is missing scope 'openid' so it's not treated as OIDC, but just pure OAuth2 request.

As a result, the requests are still valid OAuth2 requests. However, the behaviour might be unpredictable if you rely on the OIDC standard.

Swagger authorization with the OpenID Connect Discovery mechanism

The example implementation is available in the spring-boot-swagger-ui-keycloak repository.

If you need more security schemes available in Swagger UI, you can easily achieve it with the OIDC discovery mechanism. It takes advantage of the fact that the OpenID server publishes its metadata to a well-known url:

This specification defines a mechanism for an OpenID Connect Relying Party to discover the End-User’s OpenID Provider and obtain information needed to interact with it, including its OAuth 2.0 endpoint locations.

https://openid.net/specs/openid-connect-discovery-1_0.html

In order to find the correct endpoint for your Keycloak realm, select the OpenID Endpoint Configuration option in the realm settings:

In other words, the http://localhost:8024/auth/realms/keep-growing/.well-known/openid-configuration url contains the data required by the discovery mechanism to identify all available authorization schemes:

Configuration class

Let’s look at the example configuration documented in Swagger:

components:
  securitySchemes:
    openId:   # <--- Arbitrary name for the security scheme. Used to refer to it from elsewhere.
      type: openIdConnect
      openIdConnectUrl: https://example.com/.well-known/openid-configuration
…

The config for this security scheme is very short because all necessary information will be automatically acquired form the OpenID configuration endpoint:

package in.keepgrowing.springbootswaggeruikeycloak.shared.infrastructure.config.swagger.authorization;

import in.keepgrowing.springbootswaggeruikeycloak.shared.infrastructure.config.security.KeycloakProperties;
import io.swagger.v3.oas.annotations.OpenAPIDefinition;
import io.swagger.v3.oas.models.Components;
import io.swagger.v3.oas.models.OpenAPI;
import io.swagger.v3.oas.models.security.SecurityRequirement;
import io.swagger.v3.oas.models.security.SecurityScheme;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@OpenAPIDefinition
public class SwaggerOpenIdConfig {

    private static final String OPEN_ID_SCHEME_NAME = "openId";
    private static final String OPENID_CONFIG_FORMAT = "%s/realms/%s/.well-known/openid-configuration";

    @Bean
    OpenAPI customOpenApi(KeycloakProperties keycloakProperties) {
        return new OpenAPI()
                .components(new Components()
                        .addSecuritySchemes(OPEN_ID_SCHEME_NAME, createOpenIdScheme(keycloakProperties)))
                .addSecurityItem(new SecurityRequirement().addList(OPEN_ID_SCHEME_NAME));
    }

    private SecurityScheme createOpenIdScheme(KeycloakProperties properties) {
        String connectUrl = String.format(OPENID_CONFIG_FORMAT, properties.getAuthServerUrl(), properties.getRealm());

        return new SecurityScheme()
                .type(SecurityScheme.Type.OPENIDCONNECT)
                .openIdConnectUrl(connectUrl);
    }
}

As a result, I will see all the available authorizations in my Swagger UI:

Swagger authorization with bearer token

The example implementation is available in the spring-boot-swagger-ui-keycloak repository.

What if we already have access tokens from Keycloak? We can configure Swagger UI to allow us to provide only the bearer token value. Remember to provide the recommended security features for this authorization approach:

Similarly to Basic authentication, Bearer authentication should only be used over HTTPS (SSL).

https://swagger.io/docs/specification/authentication/bearer-authentication/

Configuration class

Let’s look at the example configuration documented in Swagger:

components:
  securitySchemes:
    bearerAuth:            # arbitrary name for the security scheme
      type: http
      scheme: bearer
      bearerFormat: JWT
…

We’re going to create the security scheme with the http type and bearer scheme. The bearerFormat field is optional and mostly used for documentation purposes. You can see my Java configuration below:

package in.keepgrowing.springbootswaggeruikeycloak.shared.infrastructure.config.swagger.authorization;

import io.swagger.v3.oas.annotations.OpenAPIDefinition;
import io.swagger.v3.oas.models.Components;
import io.swagger.v3.oas.models.OpenAPI;
import io.swagger.v3.oas.models.security.SecurityRequirement;
import io.swagger.v3.oas.models.security.SecurityScheme;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@OpenAPIDefinition
public class SwaggerBearerConfig {

    private static final String SCHEME_NAME = "bearerAuth";
    private static final String SCHEME = "bearer";

    @Bean
    OpenAPI customOpenApi() {
        return new OpenAPI()
                .components(new Components()
                        .addSecuritySchemes(SCHEME_NAME, createBearerScheme()))
                .addSecurityItem(new SecurityRequirement().addList(SCHEME_NAME));
    }

    private SecurityScheme createBearerScheme() {
        return new SecurityScheme()
                .type(SecurityScheme.Type.HTTP)
                .scheme(SCHEME);
    }
}

As a result, I will see the Bearer Authorization in my Swagger UI:

Make sure that the token contains user roles that your API requires (realm_access or resource_access in the screenshot below):

Why not other flows

Both implicit and password flows increase security risks and are therefore deprecated:

Please note that as of 2020, the implicit flow is about to be deprecated by OAuth 2.0 Security Best Current Practice. Recommended for most use case is Authorization Code Grant flow with PKCE.

https://spec.openapis.org/oas/latest.html#security-scheme-object

Because the client application has to collect the user’s password and send it to the authorization server, it is not recommended that this grant [password] be used at all anymore.

https://oauth.net/2/grant-types/password/

In addition, you can find more information in the Why not other flows section of the Kecloak in Docker #7 – How to authorize requests via Postman article.

Troubleshooting

What to check when API calls don’t work as planned?

The “Available authorizations” list is empty

An empty list of Available authorizations means that Swagger was unable to retrieve configuration from the OpenID server. You can debug the request responsible for loading Swagger UI in the browser:

This failure was due to the XSRF header attached to the request. If you are using Springdoc with csrf protection enabled, be sure to update the library to at least version 1.6.6. I described this problem in the Is there a way to authorize users using OpenID Connect Discovery when POST endpoints require CSRF protection? issue.

CORS Error

“No Access-Control-Allow-Origin header is present on the requested resource” is an example CORS error that may appear in your browser if the Web Origins field is misconfigured in the Keycloak client:

Make sure to configure the Keycloak client to accept connections from the Swagger endpoint as shown in the following screenshot:

Another reason for this error is missing authorization data. You should see a lock icon next to a secured endpoint:

Furthermore, the request should contain a Bearer Token that you can see in the curl window:

If the endpoints are not secured, make sure that the proper scheme is provided as a SecurityRequirement:

…
.addSecurityItem(new SecurityRequirement().addList(SAME_NAME_AS_USED_FOR_SECURITY_SCHEME));
…

More on how to authorize requests from Swagger UI in Keycloak

• Additionally, you can find sample configurations for other Swagger authorizations in the spring-boot-swagger-ui-keycloak repository.
• What is the OAuth 2.0 Authorization Code Grant Type?
• Moreover, read the Keycloak Authentication Flows, SSO Protocols and Client Configuration article.
• Finally, check out the OpenApi 3 custom setup in Spring Boot using springdoc-openapi-ui example.

Photo by Yan Krukov from Pexels

The post Keycloak with Spring Boot #3 – How to authorize requests in Swagger UI appeared first on keep_growing.

Prerequisites

• A Spring Boot project with security configuration for Keycloak described in the Keycloak with Spring Boot #1 – Configure Spring Security with Keycloak post.
• Dependencies that provide support for Spring Security and testing. Below are the relevant dependencies from my project:

<!-- pom.xml -->
…
<dependencies>
    …
    <!-- Security -->
    <dependency>
        <groupId>org.keycloak</groupId>
        <artifactId>keycloak-spring-boot-starter</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
    </dependency>
    <!-- END of Security -->

    <!-- Tests -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.security</groupId>
        <artifactId>spring-security-test</artifactId>
        <scope>test</scope>
    </dependency>
    <!-- END Tests -->
    …
</dependencies>
…

• I’m working on a Spring Boot REST API that you can find in my keycloak-spring-boot repository. If you want to run the app locally, visit the project repository on GitHub and follow the directions in the README.md file.

Why we need to overwrite configuration in tests

For example, after we add Keycloak config to a project, Spring MVC tests will fail to load the ApplicationContext due to the following error:

java.io.FileNotFoundException: Unable to locate Keycloak configuration file: keycloak.json

Currently, there is no default way to disable Keycloak protection (note that the property keycloak.enabled=false is not supported).

While we could provide just enough Keycloak configuration to run the tests, it would introduce irrelevant details into the code. Therefore, we need to ensure a security config that keeps the test code clean and still allows API access rules to be verified.

Configure integration tests to use Spring Security instead of Keycloak

First, we’ll make the Keycloak setup dependent on a custom application property. Then we’ll configure our tests to use just Spring Security.

Make the Keycloak config load dependent on a custom property

We’re going to create a custom application property to control the loading of the security config. An important point to remember is that I want Keycloak to be used by default. In other words, I only want to turn it off in testing.

Create the application.properties file in the test/resources directory and copy the following property:

# src/test/resources/application.properties
security.config.use-keycloak=false

When executing tests, Spring Boot will only see the properties defined in this file. If you have any properties that should also be used in your tests, be sure to copy them to this file as well. The advantage of this approach is that it removes all irrelevant properties from the testing context and also provides a clear view of our testing setup.

Next, we need to configure our app to use Keycloak either when our custom property explicitly says so or when it hasn’t been specified at all (by default). Therefore, we’re going to aply the ConditionalOnProperty annotation to all Keycloak-related configuration classes (there are two in my project):

package in.keepgrowing.keycloakspringboot.security.config;
…
import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Configuration;

@KeycloakConfiguration
@ConditionalOnProperty(name = "security.config.use-keycloak", havingValue = "true", matchIfMissing = true)
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {
…

package in.keepgrowing.keycloakspringboot.security.config;
…
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Configuration;

@Configuration
@ConditionalOnProperty(name = "security.config.use-keycloak", havingValue = "true", matchIfMissing = true)
public class KeycloakConfig {
…

Thanks to specifying the matchIfMissing attribute, the Keycloak configuration is our default security configuration. However, it won’t be used when running tests because we explicitly disabled it in the test/resources/application.properties file.

Define common security rules

As you may have noticed, we don’t have any security setup for our testing at the moment. Ideally, we want to test the business rules for API security without duplicating it in the tests. Therefore, I’m going to extract the common configuration into a static method so that I can reuse it in my tests:

package in.keepgrowing.keycloakspringboot.security.config;

import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.web.csrf.CookieCsrfTokenRepository;

@KeycloakConfiguration
@ConditionalOnProperty(name = "security.config.use-keycloak", havingValue = "true", matchIfMissing = true)
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {

    public static void configureApiSecurity(HttpSecurity http) throws Exception {
        http
                .csrf().csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())
                .and()
                .authorizeRequests()
                …
                .anyRequest().authenticated();
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        super.configure(http);
        configureApiSecurity(http);
    }
    …
}

As a result, I can reuse the SecurityConfig::configureApiSecurity method outside of the Keycloak configuration class.

Apply security configuration in tests

Consequently, we can now provide a separate security configuration for the tests. Create a class in the test package that:

• uses the @Configuration annotation,
• is enabled when the security.conifg.use-keycloak property is set to false,
• uses the SecurityConfig::configureApiSecurity method we defined earlier.

You can see my conifg class below:

package in.keepgrowing.keycloakspringboot.security.config;

import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

@Configuration
@ConditionalOnProperty(name = "security.config.use-keycloak", havingValue = "false")
public class TestSecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        SecurityConfig.configureApiSecurity(http);
    }
}

This is all I need to apply my security config in tests without coupling it with an external authorization server.

Finally, by running tests from my ProductControllerTest class, I can verify the actual configuration in logs:

As we can see, a user password is generated automatically according to the default Spring Security specification and there is no mention of any Keycloak configuration.

More on using Spring Security instead of Keycloak in tests

• Response to How to disable Keycloak question on StackOverflow
• Document how to use this if Keycloak is on the classpath

Photo by ELEVATE from Pexels

The post Keycloak with Spring Boot #2 – Spring Security instead of Keycloak in tests appeared first on keep_growing.

Prerequisites

• Docker Engine and Docker Compose (for running an example Keycloak instance), JDK and Maven (for running an example Spring Boot project).
• I’m working on a Spring Boot REST API that you can find in my keycloak-spring-boot repository. If you want to run the app locally, visit the project repository on GitHub and follow the directions in the README.md file.
• My project contains Spring MVC tests that allow me to verify the security configuration. However, you can learn how to authorize Postman requests in Keycloak or configure Swagger UI to comply with the security setup to manually test your API.
• The solution that I’ll show works for the following sample client configuration:

Securing Spring Boot applications with Keycloak

I’m going to add maven dependencies and required properties for Keycloak integration. Then, before I actually restrict access to my API endpoints, I will define a basic security configuration and test that both tools work together properly.

Add dependencies

First, I’m going to add the required dependencies to my project. In addition to the Keycloak and Spring Security starters for Spring Boot, I’ll add the Keycloak bom for adapters:

<!-- pom.xml -->
…
<properties>
    …
    <keycloak-adapter.version>16.1.1</keycloak-adapter.version>
</properties>
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.keycloak.bom</groupId>
            <artifactId>keycloak-adapter-bom</artifactId>
            <version>${keycloak-adapter.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>
<dependencies>
…
<dependency>
    <groupId>org.keycloak</groupId>
    <artifactId>keycloak-spring-boot-starter</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>
…
</dependencies>
…

Add application properties

I’m running my Keycloak instance in a Docker container. For my sample client configuration, see the screenshot from the Prerequisites section.

I’m going to copy some information from my Keycloak client setup to the application.properties file:

# src/main/resources/application.properties
keycloak.realm=keep-growing
keycloak.resource=spring-boot-example-app
keycloak.auth-server-url=http://localhost:8024/auth
keycloak.credentials.secret=QjLCjk1I9sugcZSDFCsyAkoLOqAHDLKC

Below is a brief explanation of the properties:

• realm – my example realm name;
• resource – my example client name;
• auth-server-url – you can get the value from the Keycloak OIDC URI endpoint list by visiting the realm settings, clicking the OpenID Endpoint Configuration and copying the auth path:

• credentials.secret – my example client has the confidential access type. Therefore, I have to copy the secret value from the Credentials tab:

If you want to access user roles at the client level and not user roles at the realm level, add the following property:

# src/main/resources/application.properties
…
keycloak.use-resource-role-mappings=true

However, in my example I’m using realm roles for users.

Add security config

I’m going to create two configuration classes to handle my Keycloak setup.

The first one provides the ability to resolve my Keycloak config based on the application.properties file. I’m going to enclose it in a separate file to avoid the circular dependency issue (described in Javadoc in the snippet below):

package in.keepgrowing.keycloakspringboot.security.config;

import org.keycloak.adapters.KeycloakConfigResolver;
import org.keycloak.adapters.springboot.KeycloakSpringBootConfigResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * The {@code KeycloakConfigResolver} bean is not defined in a configuration class that extends
 * {@code KeycloakWebSecurityConfigurerAdapter} to avoid the {@code Circular References} problem in Spring Boot from version 2.6.0.
 *
 * @see <a href="https://github.com/keycloak/keycloak/issues/8857">Application don't start because of Circular Reference due to dependency injection</a>
 */
@Configuration
public class KeycloakConfig {

    @Bean
    public KeycloakConfigResolver keycloakConfigResolver() {
        return new KeycloakSpringBootConfigResolver();
    }
}

Next, I’m going to add the following class containing a Keycloak-based Spring security configuration:

package in.keepgrowing.keycloakspringboot.security.config;

import org.keycloak.adapters.springsecurity.KeycloakConfiguration;
import org.keycloak.adapters.springsecurity.authentication.KeycloakAuthenticationProvider;
import org.keycloak.adapters.springsecurity.config.KeycloakWebSecurityConfigurerAdapter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.security.config.annotation.authentication.builders.AuthenticationManagerBuilder;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.core.authority.mapping.SimpleAuthorityMapper;
import org.springframework.security.core.session.SessionRegistryImpl;
import org.springframework.security.web.authentication.session.RegisterSessionAuthenticationStrategy;
import org.springframework.security.web.authentication.session.SessionAuthenticationStrategy;
import org.springframework.security.web.csrf.CookieCsrfTokenRepository;

@KeycloakConfiguration //1
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter { //2

    @Override
    protected void configure(HttpSecurity http) throws Exception { //3
        super.configure(http);
        http
                .csrf().csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())
                .and()
                .authorizeRequests()
                .anyRequest().permitAll();
    }

    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) { //4
        auth.authenticationProvider(getKeycloakAuthenticationProvider());
    }

    private KeycloakAuthenticationProvider getKeycloakAuthenticationProvider() { //5
        KeycloakAuthenticationProvider authenticationProvider = keycloakAuthenticationProvider();
        var mapper = new SimpleAuthorityMapper();
        mapper.setConvertToUpperCase(true);
        authenticationProvider.setGrantedAuthoritiesMapper(mapper);

        return authenticationProvider;
    }

    @Bean
    @Override
    protected SessionAuthenticationStrategy sessionAuthenticationStrategy() { //6
        return new RegisterSessionAuthenticationStrategy(new SessionRegistryImpl());
    }
}

There are a few things to note:

1. @KeycloakConfiguration – this metadata annotation provides all annotations that are needed to integrate Keycloak in Spring Security (e.g. for enabling Spring Security or configuring component scanning);
2. KeycloakWebSecurityConfigurerAdapter – by extending this class we gain a useful base class for creating a WebSecurityConfigurer instance secured by Keycloak;
3. SecurityConfig::configure – we’re going to overwrite the basic Spring Security behaviour, first by calling the parent implementation and then by adding our own csrf and endpoint protection config;
4. SecurityConfig::configureGlobal – furthermore, we have to register the KeycloakAuthenticationProvider with the authentication manager;
5. SecurityConfig::getKeycloakAuthenticationProvider – my auxiliary method for customising the authentication provider. Namely, I’m providing a simple one-to-one GrantedAuthoritiesMapper which adds the ROLE_ prefix and converts the authority value to upper case (e.g. a chief-operating-officer role from Keycloak realm becomes ROLE_CHIEF-OPERATING-OFFICER in my Spring Boot app);
6. SecurityConfig::sessionAuthenticationStrategy – the session authentication strategy bean has to be of type RegisterSessionAuthenticationStrategy for public or confidential applications (NullAuthenticatedSessionStrategy for bearer-only applications).

Test basic Keycloak configuration for Spring Security

Although I haven’t restricted access to any endpoint in my API, I’m still going to test the current setup. I’m going to enable the DEBUG log lever for security by adding the following property to my application.properties file:

logging.level.org.springframework.security=DEBUG

As a result, I can verify the configuration details and see that Keycloak filters are present in the filter chain:

…
DEBUG 12469 --- [main] edFilterInvocationSecurityMetadataSource : Adding web access control expression [permitAll] for any request
INFO 12469 --- [main] o.s.s.web.DefaultSecurityFilterChain      : Will secure any request with […, org.keycloak.adapters.springsecurity.filter.KeycloakPreAuthActionsFilter@439e3cb4, org.keycloak.adapters.springsecurity.filter.KeycloakAuthenticationProcessingFilter@31e76a8d, …]
…

Moreover, Spring Security doesn’t create a user password which would be its default behaviour without the Keycloak configuration.

At this point, my Spring Boot application is delegating authentication and authorization processes to Keycloak. However, I am still free to call the endpoints as they are not explicitly secured.

Secure endpoints

Next, I’m going to restrict access to the API endpoints by replacing anyRequest().permitAll() with the anyRequest().authenticated() line:

…
public class SecurityConfig extends KeycloakWebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        super.configure(http);
        http
                …
                .authorizeRequests()
                .anyRequest().authenticated();
    }

After restarting the application, I can see in its logs that all endpoints are protected:

…
DEBUG 13361 --- [main] edFilterInvocationSecurityMetadataSource : Adding web access control expression [authenticated] for any request
…

As a result, my Postman setup and Swagger UI config require additional configuration to allow me to make API calls.

Update tests after configuring Spring Security to use Keycloak

Configuring security in a project will break existing Spring MVC tests. Therefore, below you will find some details that require updating.

Disable Keycloak in tests and use plain Spring Security

I don’t want to clutter tests with the configuration of an external authorization service. Therefore, I’m going to configure my Spring MVC tests to use Spring Security instead of Keycloak. For full instructions on how to apply a different security configuration in tests, see the Keycloak with Spring Boot #2 – Spring Security instead of Keycloak in tests post.

Mock an authenticated user

Fortunately, Spring Security provides the @WithMockUser annotation. We can apply it to a specific test or an entire class. In the following example test method, we can see how to use this annotation in a single test while keeping the default user data:

@Test
@WithMockUser
void shouldReturnAllProducts() throws Exception {
    ProductResponse productResponse = productResponseProvider.full();
    String expected = objectMapper.writeValueAsString(List.of(productResponse));

    when(apiFacade.findAll())
        .thenReturn(List.of(productResponse));

    mvc.perform(get("/api/products")
                    .contentType(MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andExpect(content().json(expected));
    }

Additionally, we can customise the username, password, roles and authorities (the latter two are exclusive) that our mocked user will receive:

@WithMockUser(username = "thomas", password = "test", roles = {"EDITOR"})

Include csrf protection

As you may have noticed, my security configuration uses csrf protection. Therefore, I have to add the SecurityMockMvcRequestPostProcessors::csrf method that automatically populates a valid CSRF token in my POST request:

@Test
@WithMockUser
void shouldSaveNewProduct() throws Exception {
    ProductRequest newProduct = productRequestProvider.full();
    ProductResponse savedProduct = productResponseProvider.full();
    String expected = objectMapper.writeValueAsString(savedProduct);

    when(apiFacade.save(newProduct))
            .thenReturn(savedProduct);

    mvc.perform(post("/api/products")
                    .contentType(MediaType.APPLICATION_JSON)
                    .content(objectMapper.writeValueAsString(newProduct))
                    .with(csrf()))
            .andExpect(status().isOk())
            .andExpect(content().json(expected));
}

Read more on integrating Spring Security with Keycloak

• Keycloak with Spring Boot #2 – Spring Security instead of Keycloak in tests
• Keycloak with Spring Boot #3 – How to authorize requests in Swagger UI
• Kecloak in Docker #7 – How to authorize requests via Postman
• Securing Spring Boot with Keycloak documentation for Keycloak 16.1.1 and Keycloak 17.0.0.
• Keycloak in Docker #1 – How to run Keycloak in a Docker container article.
• Spring Security and Keycloak to Secure a Spring Boot Application – A First Look

Photo by cottonbro from Pexels

The post Keycloak with Spring Boot #1 – Configure Spring Security with Keycloak appeared first on keep_growing.

Prerequisites

• Minimum Postman version 7.23.
• The sample collection I’m using for this article is in my public Postman workspace and is part of my keycloak-spring-boot application. If you want to run the project locally (it requires Docker for running a Keycloak instance), visit the project repository on GitHub and follow the directions in the README.md file.
• If this is your first attempt to run Keycloak in Docker, I recommend reading the post Keycloak in Docker # 1 – How to run Keycloak in a Docker container, as I explained the basic configuration there.
• The solution that I’ll show works for the following sample client configuration:

Why we should use the PKCE Grant Type to authorize Postman requests in Keycloak

I’ll show you how to get access tokens with the Proof Key for Code Exchange Grant Type (abbreviated PKCE, pronounced “pixie”) which is an extension to the Authorization Code Grant Type. The reason for using this particular flow in place of regular Authorization Code is because it provides additional protection against CSRF and authorization code injection attacks.

As we can read in the Authorization Code Grant Type documentation:

It is recommended that all clients use the PKCE extension with this flow as well to provide better security.

https://oauhttps://oauth.net/2/grant-types/authorization-code/

Furthermore, the PKCE description states that:

PKCE (RFC 7636) is an extension to the Authorization Code flow to prevent CSRF and authorization code injection attacks.

https://oauth.net/2/pkce/

In addition, my example Keycloak client is configured with the confidential Access Type. Consequently, it provides client secrets when exchanging temporary codes for tokens. This configuration may appear to provide sufficient protection. However, the documentation advocates using the PKCE extension even when we already apply client secrets:

PKCE is not a replacement for a client secret, and PKCE is recommended even if a client is using a client secret.

https://oauth.net/2/pkce/

Provide the data required for authorization in Keycloak to the Postman environment

In order to execute the authorization flow from Postman, I will have to enter some confidential and environment-sensitive details, e.g. Client Secret, Auth URL etc.:

I will store the sensitive data as environment variables. First, click the Environment quick look button (the eye icon) and click Edit for the selected environment. Then, you can start adding variables. Below you’ll see what information we need for the authorization flow and where to find the values.

Callback URL

Postman uses this url to extract the access token after a successful authorization in Keycloak. This is the Redirect URI value that I specified for my client in Keycloak:

I’m going to create the {{redirectUri}} variable with the http://localhost://8080/* value.

Auth URL and Access Token URL

I’m going to provide the authorization and token urls from the Keycloak OIDC URI endpoint list. To get the values, visit the general realm settings and click the OpenID Endpoint Configuration:

We’re going to see the authorization and token endpoints in the list below:

Make sure that the host is consistent with the security properties in your API. If you use 127.0.0.1 instead of localhost in the API security config, consequently use the 127.0.0.1 value for the urls in Postman. I’m going to create the following variables in Postman:

• {{authUrl}} with the http://localhost:8024/auth/realms/keep-growing/protocol/openid-connect/auth value
• {{accessTokenUrl}} with the http://localhost:8024/auth/realms/keep-growing/protocol/openid-connect/token value.

Client ID

It’s the identifier for our client:

I’m going to copy that value and put it in the {{clientId}} variable.

Client secret

As I mentioned before, my sample client has the confidential access type. Therefore, its configuration includes the Credentials tab, where I generated the following secret:

I’m going to copy that value and put it in the {{clientSecret}} variable.

Full variable list

To summarize, all the environment variables I have added to my Postman collection are below:

If your API uses csrf protection, read the How to add X-XSRF-TOKEN header to Postman requests article. Otherwise, you’ll get 403 errors even after completing this tutorial.

Configuration required to authorize Postman requests in Keycloak

Now I will set up authorization for my Postman collection using the variables mentioned above and providing some additional information. First, I’ll edit the collection:

Then, I’ll and go the Authorization tab and select the OAuth 2.0 authorization type to configure a new token. Below you’ll see what additional information I’m going to provide.

Add authorization data to

I’m going to select the Request Headers option.

Token Name

This is an arbitrary value to distinguish this particular token from others you keep in Postman. I’m going to call it spring-boot-example-app token.

Grant Type

I’m going to select the Authorization Code (With PKCE) option.

Scope

Verify what scopes are available for your client.:

I’m going to provide the profile email value here.

Client Authentication

I’m going to select the Send client credentials in body option.

Full configuration

You can see the complete configuration in the screenshot below:

Get a new Access Token

Finally, we can click the Get new Access Token button. We’re going to see the following Postman screen informing us that authentication will be done through a browser:

Meanwhile, our authorization server will provide the a login form. Below you can see my Keycloak login screen where I authenticate as an example user of my realm:

After successful authentication, we will see the following screen for a few seconds:

As a result, a new spring-boot-example token value is available in the Manage Access Token window. Click the Use Token button:

You’ll see the token value in the Current Token section:

Furthermore, we can verify the token payload by decoding it with jwt.io:

Make the token available in all requests

To make this token accessible in all requests in my collection, I’m going to select the Inherit auth from parent option in the request Authorization tab:

As a result, Keycloak will authorize all requests with this config from my Postman collection.

Update and manage Access Tokens

When the token expires, generate a new value. Just edit your collection, go to the Authorization tab, click the Generate New Access Token button, then click the Use Token button and finally click the Update button:

To update the token or to overwrite the collection config for only one request, go to the Authorization tab of that request and select the OAuth 2.0 for the authorization type. You’ll be able to generate a new token or redefine the configuration values:

Furthermore, you can select a specific token value from the Available Tokens dropdown:

Finally, you can remove some tokens by clicking the Manage Tokens option:

Why not other flows

You can still find tutorials on setting up other Authorization Flows. However, here are the documented reasons why I don’t use them in my application.

Why not Implicit flow

This is the easiest flow. Nonetheless, it comes with some severe security vulnerabilities:

The implicit grant (…) and other response types causing the authorization server to issue access tokens in the authorization response are vulnerable to access token leakage and access token replay (…).
Moreover, no viable mechanism exists to cryptographically bind access tokens issued in the authorization response to a certain client (…). This makes replay detection for such access tokens at resource servers impossible.
In order to avoid these issues, clients SHOULD NOT use the implicit grant (…), unless access token injection in the authorization response is prevented and the aforementioned token leakage vectors are mitigated.

https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics#section-2.1.2

The vulnerabilities were described in depth in the Implement the OAuth 2.0 Authorization Code with PKCE Flow post on the Okta blog:

Notice that after you authenticate, the Authorization Server (like Google) responds directly with tokens. This means that the tokens are in your browser’s address bar as a result of the redirect. That’s problematic since Google can’t definitively know that your browser (the intended recipient) actually received the response. It’s also problematic because modern browsers can do browser history syncing and they support browser extensions that could be actively scanning for tokens in the browser address bar. Leaking tokens is a big security risk.

https://developer.okta.com/blog/2019/08/22/okta-authjs-pkce#why-you-should-never-use-the-implicit-flow-again

As a result of these security concerns, the Implicit Flow is officially deprecated:

Please note that as of 2020, the implicit flow is about to be deprecated by OAuth 2.0 Security Best Current Practice. Recommended for most use case is Authorization Code Grant flow with PKCE.

https://spec.openapis.org/oas/latest.html#security-scheme-object

Why not Password flow

Another convenient flow that we can configure in Postman, the Password Credentials Grant Type, also comes with an increased risk of attack:

The resource owner password credentials grant MUST NOT be used. This grant type insecurely exposes the credentials of the resource owner to the client. Even if the client is benign, this results in an increased attack surface (credentials can leak in more places than just the authorization server) and users are trained to enter their credentials in places other than the authorization server.

https://datatracker.ietf.org/doc/html/draft-ietf-oauth-security-topics-13#section-3.4

Because the client application has to collect the user’s password and send it to the authorization server, it is not recommended that this grant be used at all anymore.

https://oauth.net/2/grant-types/password/

More on how to authorize Postman requests in Keycloak

• Authorizing requests, OAuth 2.0 authorization type, Requesting an OAuth 2.0 token in the Postman documentation and the example collection that documents a few OAuth 2.0 authorization flows.
• OAuth 2.0: Implicit Flow is Dead, Try PKCE Instead and OAuth 2 Simplified articles.
• Proof Key for Code Exchange by OAuth Public Clients

Photo by ELEVATE from Pexels

The post Kecloak in Docker #7 – How to authorize requests via Postman appeared first on keep_growing.

Debugging the “Invalid CSRF token” error

We get the Invalid CSRF token error when an API has csrf protection enabled and our request doesn’t contain the required data. The security configuration regarding the csrf protection in my example Spring Boot project looks like this:

public static void configureApiSecurity(HttpSecurity http) throws Exception {
        http
                .csrf().csrfTokenRepository(CookieCsrfTokenRepository.withHttpOnlyFalse())
…

The Spring documentation describes the CookieCsrfTokenRepository as follows:

A CsrfTokenRepository that persists the CSRF token in a cookie named “XSRF-TOKEN” and reads from the header “X-XSRF-TOKEN”

https://docs.spring.io/spring-security/site/docs/5.0.13.RELEASE/api/index.html?org/springframework/security/web/csrf/CookieCsrfTokenRepository.html

As a result, the token will be present in the API responses as seen in the screenshot below:

Consequently, when I’m sending a POST request with no value provided in the required header, I get the 403 Forbidden in response. The exact error is specified in the application logs:

DEBUG 11528 --- [nio-8080-exec-3] o.s.security.web.csrf.CsrfFilter         : Invalid CSRF token found for http://localhost:8080/api/products
DEBUG 11528 --- [nio-8080-exec-3] o.s.s.w.access.AccessDeniedHandlerImpl   : Responding with 403 status code

In a Spring Boot application, we can debug the actual value that is checked in the CsrfFilter class:

In my case, the actual value is null. So I need to add the X-XSRF-TOKEN header to my Postman POST requests.

Add XSRF token to Postman requests

I’m going to configure my Postman collection to get the current token from each request and save it as an environment variable. Then, I will use this variable as the header value for my POST calls.

Save the token value as an environment variable

To read the value of the cookie, we’re going to execute a short JavaScript code in Postman.

The sample collection I’m using for this article is in my public Postman workspace and is part of my keycloak-spring-boot project. If you want to test my code locally (it requires Docker for running a Keycloak instance), visit the project repository on GitHub and follow the directions in the README.md file.

For all requests

First, I’m going to place the js code in the Collection testing space. As we can read in the documentation:

A test script associated with a collection will run after every request in the collection.

https://learning.postman.com/docs/writing-scripts/test-scripts/#testing-collections-and-folders

This way, I won’t have to add the script to every POST request I may create in this collection in the future.

I’m going to edit my example keycloak-spring-boot collection:

Then, I’m going to add the following code to the Tests tab:

var xsrfCookie = pm.cookies.get("XSRF-TOKEN");
pm.environment.set('xsrf-token', xsrfCookie);

You can see the result in the screenshot below:

Finally, I’m going to save the changes to the collection by clicking the Update button.

For a single request

Alternatively, I can only read the cookie after a selected request. To achieve this, I will need to add the js code in the request-specific test space and click the Save button. In summary, the configuration of a single request will look like this:

In this case, we must remember to add the js code to any future POST request as well.

Add the header

First, I’m going to verify that the value is actually available as an environment variable in Postman after running my request. Therefore, I’m going to execute the request, click on the Environment quick look button (the eye icon) and look for the xsrf-token variable as shown in the screenshot below:

Now I’m going to add a new header to my request, with the following data:

• Key: X-XSRF-TOKEN,
• Value: {{xsrf-token}}.

We can see the result in the screenshot below:

Verify the configuration

Finally, I can make my POST request with the certainty that it will work successfully:

Troubleshooting

What to check when API calls don’t work as planned?

403 Forbidden response status

• The JavaScript code we add will only run after a request. Therefore, we will get 403 on the first API call since the cookie value is not yet set to a variable. However, subsequent requests will have the appropriate value in the header.
• Is the {{xsrf-token}} variable set in Postman environment? You can see its current value in the Environment quick look or by hovering over the variable. Select the proper environment (localhost in my example) and make sure that the value is not empty.
• Can you check API logs or debug the actual verified header value? If the value is null, the header is empty.
• Remember that even if we add the script to a collection-specific test space, we need to manually add the X-XSRF-TOKEN header to each POST request.

More on complying with the CSRF requirements

• How do I send spring csrf token from Postman rest client?
• Moreover, you can learn how to get a list of cookies associated with a request in the Scripting with request cookies docs.
• Read the Defining variables in scripts if want to set variables programmatically in your request scripts.
• Difference between CSRF and X-CSRF-Token.
• What is the difference between X-XSRF-TOKEN and X-CSRF-TOKEN?
• Spring Security Reference Guide on Cross Site Request Forgery.
• Fix “Invalid CSRF token” error – add the XSRF-TOKEN header in Angular.
• Postman Pre-Request script to append CSRF token in header for POST requests in Laravel Sanctum authenticated SPA.

Photo by Andrea Piacquadio from Pexels

The post How to add X-XSRF-TOKEN header to Postman requests appeared first on keep_growing.

Prerequsites

• Docker Engine and Docker Compose installed on your machine.
• If this is your first attempt to run Keycloak in Docker, I recommend reading the post Keycloak in Docker # 1 – How to run Keycloak in a Docker container, as I explained the basic configuration there.
• You’ll need at least one realm that was exported to a directory. You can learn how to do it in the Keycloak in Docker #5 – How to export a realm with users and secrets post.

Configure importing Keycloak relams from a directory

The example Docker Compose configuration used below is available in the https://gist.github.com/little-pinecone/6b52ccd1fc0296b267e810d43fd01f3b GitHub gist.

The importance of the naming convention

As we can read in the docs:

When importing from a directory, the filenames must follow this naming convention:

<REALM_NAME>-realm.json. For example, “acme-roadrunner-affairs-realm.json” for the realm named “acme-roadrunner-affairs”.

<REALM_NAME>-users-<INDEX>.json. For example, “acme-roadrunner-affairs-users-0.json” for the first user’s file of the realm named “acme-roadrunner-affairs”.

https://www.keycloak.org/docs/16.1/server_admin/#assembly-exporting-importing_server_administration_guide

Make sure the resources for your realms follow this naming convention. Otherwise, some or all of the files may be skipped completely from import.

Add a Docker volume for the imported resources

First, I’m going to create a Docker volume to make the import assets available in the /tmp/import directory in the container. Below you’ll find the relevant configuration from my docker-compose.yml file:

version: '3.3'
services:
  keycloak:
    …
    volumes:
      - ./keycloak/realms/import:/tmp/import
    …

Provide the required options when running the container

Next, I’m going to add the minimum configuration required to perform the directory import to my docker-compose.yml file:

services:
  keycloak:
    …
    volumes:
       …
    command:
      - "-Dkeycloak.migration.action=import"
      - "-Dkeycloak.migration.provider=dir"
      - "-Dkeycloak.migration.dir=/tmp/import"

This configuration will overwrite existing realms by default. For other keycloak.migration.X options, see the official Keycloak documentation on importing and exporting the database.

Import a Keycloak realm that has been exported in multiple files

Below you can see the starting point for importing my keep-growing realm:

To summarize, I have one file with the realm configuration and one with the associated users. With the Docker Compose configuration described in the previous section, I’m going to start the container using the docker-compose up -d command.

As a result, we can see in the screenshot below that the volume with realm resources was mapped properly:

Furthermore, the container logs contain entries documenting a successful import:

INFO  [org.keycloak.exportimport.dir.DirImportProvider] (ServerService Thread Pool -- 62) Importing from directory /tmp/import
…
INFO  [org.keycloak.services] (ServerService Thread Pool -- 62) KC-SERVICES0050: Initializing master realm
INFO  [org.keycloak.services] (ServerService Thread Pool -- 56) KC-SERVICES0030: Full model import requested. Strategy: OVERWRITE_EXISTING
INFO  [org.keycloak.exportimport.util.ImportUtils] (ServerService Thread Pool -- 56) Realm 'keep-growing' imported
INFO  [org.keycloak.exportimport.dir.DirImportProvider] (ServerService Thread Pool -- 56) Imported users from /tmp/import/keep-growing-users-0.json
INFO  [org.keycloak.services] (ServerService Thread Pool -- 56) KC-SERVICES0032: Import finished successfully

At last, we can examine the imported realm in the Keycloak Admin Console:

Import multiple Keycloak realms

Below you can see the starting point for importing my keep-growing and Example-Realm realms:

To summarize, I have two files, one for each realm. With the Docker Compose configuration described in the previous section, I’m going to start the container using the docker-compose up -d command.

Consequently, we can see in the screenshot below that the volume with realms is available in the container:

Furthermore, the container logs contain entries documenting successful imports:

INFO  [org.keycloak.exportimport.dir.DirImportProvider] (ServerService Thread Pool -- 62) Importing from directory /tmp/import
…
INFO  [org.keycloak.services] (ServerService Thread Pool -- 62) KC-SERVICES0050: Initializing master realm
INFO  [org.keycloak.services] (ServerService Thread Pool -- 62) KC-SERVICES0030: Full model import requested. Strategy: OVERWRITE_EXISTING
INFO  [org.keycloak.exportimport.util.ImportUtils] (ServerService Thread Pool -- 62) Realm 'keep-growing' imported
INFO  [org.keycloak.exportimport.util.ImportUtils] (ServerService Thread Pool -- 62) Realm 'Example-Realm' imported
INFO  [org.keycloak.services] (ServerService Thread Pool -- 62) KC-SERVICES0032: Import finished successfully

Finally, we can examine the imported realm in the Keycloak Admin Console:

Read more on Keycloak directory import

• In addition to this article, see the Keycloak documentation on importing and exporting the database.
• Additionally, you can visit the How to import Multiple realms in keycloak? and Missing users from Keycloak Realm after import issues on StackOverflow.
• In case you need to import realm from a single file, see the Keycloak in Docker #2 – How to import a Keycloak realm post.

Photo by Vlada Karpovich from Pexels

The post Keycloak in Docker #6 – How to import realms from a directory appeared first on keep_growing.

Prerequisites

• Docker Engine and Docker Compose installed on your machine.
• If this is your first attempt to run Keycloak in Docker, I recommend reading the post Keycloak in Docker # 1 – How to run Keycloak in a Docker container as I explained the basic configuration there.

For reference, below you’ll find the resulting directory tree for this example:

In short, my goal is to get the export directory containing realm resources (in this case, the keep-growing-realm.json file). Thanks to this, I’ll be able to backup or recreate a complete realm whenever I need to.

Why not export with Keycloak Admin console?

The Keycloak Admin Console provides an easy way to export a realm. However, as you can see in the screenshot below, not all resources can be exported with this method:

The realm-export.json file produced with this technique won’t contain user data. In addition, client secrets will be masked. Although this approach might be appropriate in some use cases, we won’t be able to recreate the same instance using only this file.

If we want to get realm data suitable for backups or cross server migration, we need to run a boot-time export.

My example Keycloak realm

I have my keep-growing realm with the following users:

Furthermore, the realm contains the spring-boot-example-app client. It has the confidential access type and a secret, as you can see in the screenshot below:

At the end of this article, I will verify that the users and the secret are properly exported.

How to export all resources from a Keycloak realm

I’m going to use a volume to map exported data between the container and my machine. Next, I will show you how to export the data with a command or automatically when running the container. Finally, I’m going to verify if the export was successful.

Add a volume for exported resources

First, we need to start our container with a volume that will hold exported data. Therefore, I’m going to map the ./keycloak/realms/export folder on my machine to the /tmp/export directory in the keycloak service. Below you’ll find my docker-compose.yml file:

version: '3.3'
services:
  keycloak:
    image: jboss/keycloak:${KEYCLOAK_VERSION}
    ports:
      - "8024:8080"
    environment:
      - KEYCLOAK_USER=${KEYCLOAK_USER}
      - KEYCLOAK_PASSWORD=${KEYCLOAK_PASSWORD}
    volumes:
      - ./keycloak/realms/export:/tmp/export

I’m using my default environmental variables defined in the following .env file:

KEYCLOAK_VERSION=16.1.1
KEYCLOAK_USER=keycloak
KEYCLOAK_PASSWORD=keycloak

Now I’m going to start the service with the docker-compose up -d command. As we can see in the following screenshot, the volume is properly created and mapped:

Obviously, the /tmp/export directory inside the dockerized Keycloak instance is empty as we still need to export our realm:

Keycloak export options explained

Let me introduce the properties that we’re going to use:

• Djboss.socket.binding.port-offset, we want to run the export on a different port than Keycloak itself. By default, the jboss server in my kecyloak instance uses the 9990 port. Therefore, providing e.g. 100 as a value here will result in the command using the 10090 port.
• Dkeycloak.migration.action, we specify what we want to do. Available options:
  • export,
  • import.
• Dkeycloak.migration.provider, we define how we want the data. Available options are:
  • singleFile, if we want to export data into a file
  • dir, if we want to export data into a directory.
• Dkeycloak.migration.realmName, realm for export. Don’t specify this parameter if you want to export all realms.
• Dkeycloak.migration.usersExportStrategy, how to export realm users. Available options are:
  • DIFFERENT_FILES, it’s the default option for this attribute, we will get a file for realm config and multiple files for users (depending on how many users we have and the number of users per one file (50 by default));
  • SKIP, won’t export users;
  • REALM_FILE, export users to the same file as realm configuration;
  • SAME_FILE, we will get one file for realm configuration and one for all users (keep-growing-realm.json and keep-growing-users.json).
• Dkeycloak.migration.file, the file where we want to save data. Remember to use the exact same value as you used for the volume mapping.

Run the export command

We’re going to execute the /opt/jboss/keycloak/bin/standalone.sh script included in the jboss/keycloak image to initiate the export process:

docker exec -it <CONTAINER NAME> /opt/jboss/keycloak/bin/standalone.sh \
… // required Keycloak export properties

To clarify the following examples, my example keycloak service runs in the keycloakspringboot_keycloak_1 container.

Export to a single file

When a realm doesn’t contain a lot of users it might be appropriate to export it into a single file. The actual command that I’m going to execute looks like in the snippet below:

docker exec -it keycloakspringboot_keycloak_1 /opt/jboss/keycloak/bin/standalone.sh \
-Djboss.socket.binding.port-offset=100 \
-Dkeycloak.migration.action=export \
-Dkeycloak.migration.provider=singleFile \
-Dkeycloak.migration.realmName=keep-growing \
-Dkeycloak.migration.usersExportStrategy=REALM_FILE \
-Dkeycloak.migration.file=/tmp/export/keep-growing-realm.json

If the command encounters no problems, you’ll see console output similar to this:

…
12:11:10,399 INFO  [org.keycloak.services] (ServerService Thread Pool -- 60) KC-SERVICES0034: Export of realm 'keep-growing' requested.
12:11:10,400 INFO  [org.keycloak.exportimport.singlefile.SingleFileExportProvider] (ServerService Thread Pool -- 60) Exporting realm 'keep-growing' into file /tmp/export/keep-growing-realm.json
12:11:10,790 INFO  [org.keycloak.services] (ServerService Thread Pool -- 60) KC-SERVICES0035: Export finished successfully
…
12:11:11,195 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:10090/management
12:11:11,196 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:10090

As a result, a new file appears in the keycloak/realms/export directory on my machine:

You can quit the process with Ctr+C.

If you want to import that realm, the Keycloak in Docker #2 – How to import a Keycloak realm article describes in detail importing a realm from a file.

Export to a directory

Keycloak recommends exporting data into a directory if your realm contains more than 500 users:

Exporting many users into a directory performs optimally as the directory provider uses a separate transaction for each “page” (a file of users).

https://www.keycloak.org/docs/16.1/server_admin/#assembly-exporting-importing_server_administration_guide

In this case, my command would look like this (notice changes in lines 4 and 6):

docker exec -it keycloakspringboot_keycloak_1 /opt/jboss/keycloak/bin/standalone.sh \
-Djboss.socket.binding.port-offset=100 \
-Dkeycloak.migration.action=export \
-Dkeycloak.migration.provider=dir \
-Dkeycloak.migration.realmName=keep-growing \
-Dkeycloak.migration.dir=/tmp/export

If the command encounters no problems, you’ll see console output similar to this:

…
12:29:31,726 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0034: Export of realm 'keep-growing' requested.
12:29:32,107 INFO  [org.keycloak.exportimport.dir.DirExportProvider] (ServerService Thread Pool -- 53) Realm 'keep-growing' - data exported
12:29:32,212 INFO  [org.keycloak.exportimport.dir.DirExportProvider] (ServerService Thread Pool -- 53) Users 0-3 exported
12:29:32,214 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0035: Export finished successfully
…

As a result, a group of new files appear in the keycloak/realms/export directory on my machine:

You can quit the process with Ctr+C.

If you want to import that realm, see the Keycloak in Docker #6 – How to import realms from a directory article as it describes in detail importing realms exported to a folder.

Export on a container startup

Additionally, we can configure our docker-compose.yml file to initiate the export when we start the container (by adding the command config). It can be useful when our docker image doesn’t provide an equivalent to the standalone.sh script or we just don’t want to use it. The modified compose file for a single file export looks like this:

version: '3.3'
services:
  keycloak:
    image: jboss/keycloak:${KEYCLOAK_VERSION}
    ports:
      - "8024:8080"
    environment:
      - KEYCLOAK_USER=${KEYCLOAK_USER}
      - KEYCLOAK_PASSWORD=${KEYCLOAK_PASSWORD}
    volumes:
      - ./keycloak/realms/export:/tmp/export
    command:
      - "-Dkeycloak.migration.action=export"
      - "-Dkeycloak.migration.provider=singleFile"
      - "-Dkeycloak.migration.realmName=keep-growing"
      - "-Dkeycloak.migration.usersExportStrategy=REALM_FILE"
      - "-Dkeycloak.migration.file=/tmp/export/keep-growing-realm.json"

We’ll see in logs that the container starts with the provided options:

The export will proceed as in the previously described methods.

Data included in the exported Keycloak realm resources

We’re going to find the secret generated for the spring-boot-example-app client in the realm config file:

Furthermore, the users are included in the exported assets as well. In case of the directory export the users are stored in the keep-growing-users-0.json file. For the single file export, the users are included directly in the keep-growing-realm.json file:

Troubleshooting

What to check when the export didn’t work as planned?

Null pointer exception

If you see the NullPointerException error in the logs make sure that the Dkeycloak.migration.realmName property value is consistent with the actual name of the realm you want to export. The following example error shows what happens if I provide non-existing to the command but my realm’s id is actually keep-growing:

11:42:20,795 INFO  [org.keycloak.services] (ServerService Thread Pool -- 54) KC-SERVICES0034: Export of realm 'non-existing' requested.
11:42:20,795 INFO  [org.keycloak.exportimport.singlefile.SingleFileExportProvider] (ServerService Thread Pool -- 54) Exporting realm 'non-existing' into file /tmp/export/keep-growing-realm.json
11:42:20,804 FATAL [org.keycloak.services] (ServerService Thread Pool -- 54) Error during startup: java.lang.NullPointerException

No such file or directory

If you see the No such file or directory message in the logs make sure that the Dkeycloak.migration.file property value points to a path that actually exists within the container. The following example error shows what happens if I provide /tmp/wrong/directory/path/keep-growing-realm.json to the command but Docker actually created a folder according to the mapped volume which points to /tmp/export:

Exporting realm 'keep-growing' into file /tmp/wrong/directory/path/keep-growing-realm.json
11:50:20,269 FATAL [org.keycloak.services] (ServerService Thread Pool -- 53) Error during startup: java.lang.RuntimeException: Error during export/import: /tmp/wrong/directory/path/keep-growing-realm.json (No such file or directory)

Verify that you actually restarted the container after adding the volume config to the docker-compose.yml file. Otherwise, the mapped directory won’t be created for you.

Missing export files

There are no errors in logs but you can’t find the exported files on your machine? Verify that the Dkeycloak.migration.file property value is consistent with the volume mapping. The following example shows what happens if I provide /tmp/keep-growing-realm-test.json instead of /tmp/export/keep-growing-realm-test.json:

You can always verify the location of the exported data in the command logs:

11:59:32,192 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0034: Export of realm 'keep-growing' requested.
11:59:32,192 INFO  [org.keycloak.exportimport.singlefile.SingleFileExportProvider] (ServerService Thread Pool -- 53) Exporting realm 'keep-growing' into file /tmp/keep-growing-realm-test.json
11:59:32,805 INFO  [org.keycloak.services] (ServerService Thread Pool -- 53) KC-SERVICES0035: Export finished successfully

Read more on exporting Keycloak realm

• The Jboss image documentation on exporting a realm
• The Keycloak documentation on importing and exporting the database

Photo by RODNAE Productions from Pexels

The post Keycloak in Docker #5 – How to export a realm with users and secrets appeared first on keep_growing.