Keycloak and JDBC Ping

Monday, August 12 2019, posted by Sebastian Łaskaiwec

A few months back, we had a great article about clustering using JDBC_PING protocol. Since then, we introduced some improvements for the Keycloak container image that can simplify the setup. So, before diving into this blog post, I highly encourage you to visit the Keycloak Cluster Setup article.

What has changed in our Container Image?

Probably the most important change is configuring the JGroups discovery protocol by using variables (see the Pull Request). Once the change got in, we could configure the JGroups discovery by setting two properties:

  • JGROUPS_DISCOVERY_PROTOCOL
  • JGROUPS_DISCOVERY_PROPERTIES

Let's apply the changes, shall we...

The JDBC_PING-based setup works fine in all scenarios, where we connect all Keyclaok instances to the same database. Since JDBC_PING can be configured to obtain a database connection using JNDI binding, it can easily connect to the Keycloak database. All we need to do is to add two parameters to our docker image:

  • JGROUPS_DISCOVERY_PROTOCOL=JDBC_PING
  • JGROUPS_DISCOVERY_PROPERTIES=datasource_jndi_name=java:jboss/datasources/KeycloakDS

You may find an end-to-end scenario here.

Additional configuration

In some scenarios, you may need additional configuration. All additional settings might be added to the JGROUPS_DISCOVERY_PROPERTIES. Here are some hints and common problems, that you may find:

Problem description Possible solution
The initialization SQL needs to be adjusted In this case, you might want to look at initialize_sql JDBC_PING property
When Keycloak crashes, the database is not cleared Turn remove_old_coords_on_view_change property on
When Keycloak crashes, the database is not cleared Also, when a cluster is not too large, you may turn the remove_all_data_on_view_change property on
Sometimes, Keycloak doesn't write its data into the database You may lower the info_writer_sleep_time and info_writer_max_writes_after_view property values


Haven fun and don't forget to let us know what you think about this blog post using the User Mailing List.
Sebastian Łaskawiec and the Keycloak Team

Keycloak Cluster Setup

Friday, May 10 2019, posted by 张立强 liqiang@fit2cloud.com

This post shares some solutions to setup Keycloak cluster in various scenarios (e.g. cross-DC, docker cross-host, Kubernetes).

If you'd like to setup Keycloak cluster, this blog may give you some reference.

Two cli script files are added to the Keycloak image as per the guide.

The Dockerfile is below and these two files are the most important matter for this blog, you can find them from TCPPING.cli and JDBC_PING.cli.

FROM jboss/keycloak:latest

ADD cli/TCPPING.cli /opt/jboss/tools/cli/jgroups/discovery/
ADD cli/JDBC_PING.cli /opt/jboss/tools/cli/jgroups/discovery/

First of all we should know that for a Keycloak cluster, all keycloak instances should use same database and this is very simple, another thing is about cache(generally there are two kinds of cache in Keycloaks, the 1st is persistent data cache read from database aim to improve performance like realm/client/user, the 2nd is the non-persistent data cache like sessions/clientSessions, the 2nd is very important for a cluster) which is a little bit complex to configure, we have to make sure the consistent of cache in a cluster view.

Totally here are 3 solutions for clustering, and all of the solutions are base on the discovery protocols of JGroups (Keycloak use Infinispan cache and Infinispan use JGroups to discover nodes).

1. PING

PING is the default enabled clustering solution of Keycloak using UDP protocol, and you don't need to do any configuration for this.

But PING is only available when multicast network is enabled and port 55200 should be exposed, e.g. bare metals, VMs, docker containers in the same host.

We tested this by two Keycloak containers in same host.

The logs show that the two Keycloak instances discovered each other and clustered.

2. TCPPING

TCPPING use TCP protocol with 7600 port. This can be used when multicast is not available, e.g. deployments cross DC, containers cross host.

We tested this by two Keycloak containers cross host.

And in this solution we need to set three below environment variables for containers.

#IP address of this host, please make sure this IP can be accessed by the other Keycloak instances
JGROUPS_DISCOVERY_EXTERNAL_IP=172.21.48.39
#protocol
JGROUPS_DISCOVERY_PROTOCOL=TCPPING
#IP and Port of all host
JGROUPS_DISCOVERY_PROPERTIES=initial_hosts="172.21.48.4[7600],172.21.48.39[7600]"

The logs show that the two Keycloak instances discovered each other and clustered.

3. JDBC_PING

JDBC_PING use TCP protocol with 7600 port which is similar as TCPPING, but the difference between them is, TCPPING requires you configure the IP and port of all instances, for JDBC_PING you just need to configure the IP and port of current instance, this is because in JDBC_PING solution each instance insert its own information into database and the instances discover peers by the ping data read from database.

We tested this by two Keycloak containers cross host.

And in this solution we need to set two below environment variables for containers.

#IP address of this host, please make sure this IP can be accessed by the other Keycloak instances
JGROUPS_DISCOVERY_EXTERNAL_IP=172.21.48.39
#protocol
JGROUPS_DISCOVERY_PROTOCOL=JDBC_PING

The ping data of all instances haven been saved in database after instances started.

The logs show that the two Keycloak instances discovered each other and clustered.

One more thing

The above solutions are available for most scenarios, but they are still not enough for some others, e.g.Kubernetes.

The typical deployment on Kubernetes is one Deployment/ReplicateSet/StatefulSet contains multi Keycloak Pods, the Pods are really dynamic as they can scale up and down or failover to another node, which requires the cluster to discover and remove these dynamic members.

On Kubernetes we can use DNS_PING or KUBE_PING which work quite well in practice.

Besides DNS_PING and KUBE_PING, JDBC_PING is another option for Kubernetes.

On Kubernetes multicast is available only for the containers in the same node and a pod has no static ip which can be used to configure TCPPING or JDBC_PING. But in the JDBC_PING.cli mentioned above we have handled this, if you don't set the JGROUPS_DISCOVERY_EXTERNAL_IP env, the pod ip will be used, that means on Kubernetes you can simply set JGROUPS_DISCOVERY_PROTOCOL=JDBC_PING then your keycloak cluster is ok.

Discussion

Suggestions and comments can be discussed via Keycloak User Mail List or this GitHub Repository.

Keycloak Community Newsletter #2

Monday, May 06 2019, posted by Sébastien Blanc

We have a lot of news to share in this second edition, so fasten your seatbelt and let's go!

News from the community

First of all, we would like to thank the whole community, which has contributed to this edition by sharing their links, tips, and so on.

An increasing number of API Management/Gateway solutions can now be integrated with Keycloak. Recently, Ambassador, an Open Source Kubernetes-Native API Gateway built on the Envoy Proxy, has added support for Keycloak; it has also published a quickstart to show how Keycloak can be used to add Github as Identity provider.

We have some good news for Python users. Akhil Lawrence has created a Python Keycloak Adapter Client. Be sure to check out the really nice documentation that comes with many usage examples.

Like any other project, setting up clustering can be somehow complex. Liqiang has shared with us his setup and configuration tips. Thanks again for sharing your knowledge with the community!

The existing Keycloak Helm Chart has now been deprecated and is now replaced by the one managed by CodeCentric.

Dmitry Telegin has created a really nice example on how you can dynamically brand your login theme for Keycloak. Check out the repository here.

Hayri Cicek has written a nice introductory article on how to use Keycloak and MicroProfile. It even shows the usage of the brand new Client Scope "microprofile-jwt," which has been added in Keycloak 6.0.0.

News from the project

Keycloak 6.0.1 has been released. We know that some people were a bit confused by our new versioning schema. We hope that this blog post will clear it up.

We started the proposal process for Keycloak to be accepted into the Cloud Native Computing Foundation (CNCF). We hope that this effort will significantly boost our community adoption with our ultimate goal of becoming the de facto solution for OAuth2/OpenID Connect within Open Source and Cloud Native. You can watch the presentation to the CNCF TOC (Technical Oversight Committee) here and the slides are also available here.

As mentioned in the previous newsletter, we will now discuss larger features openly on Github. It's community-focused and we have currently two open Pull Requests:

Conferences / Webinars

W-Jax has just posted a YouTube talk in German Secure Spring Applications with Keycloak delivered by Thomas Darimont.

In May, Sébastien Blanc will deliver a full-day workshop about Keycloak during VoxxedDays Minsk

Contributing to Keycloak

We always welcome contributions to Keycloak. If you would like to contribute and have a great idea, tell us about it on the developer mailing list. If you are unsure about what to work on, let us know and we can help!

As a first time contributor, you may have a simpler idea to start, such as contributing a bug fix. This type of contribution will allow you to get to know the code base, the test suite, and the mechanics of creating a pull request. You can find a list of open bugs here.

We also have a list of open issues that are awaiting contributions. Not all issues are properly reviewed, so we recommend that you start by sending an email to the developer mailing list before you begin.

For each newsletter, we will also highlight a few features for which we would especially like contributions. These features include:

Keycloak Releases and Versioning

Wednesday, April 24 2019, posted by Stian Thorgersen

We are aiming to achieve a continuous delivery model with Keycloak. By that we mean it should be seamless to upgrade between Keycloak releases and to keep up to date with the latest release.

This requires no breaking changes, but rather deprecating old APIs allowing time to migrate to new APIs.

Traditional semantic versioning does not fit very well with this model. By following the mantra of continuous delivery we would forever be stuck on a major version and only update the minor version, and you could argue whether or not it would be correct to update the major version when an API that has been deprecated for a long period of time is removed.

With this in mind, we have made some slight changes to our release cadence and versioning schema.

For now, we will have a new feature release roughly 4 times each year. Each release will bump the major version number. That doesn't mean there are breaking changes, but until we perfect our continuous delivery model there may be some, so always refer to the migration guide prior to upgrading!

We have also decided to drop the Final suffix from releases. That is simply because it is not needed as we have not done any beta or release candidates for a long time. In the spirit of continuous delivery, we will have individual features marked as preview rather than whole releases.

As a final note, with the reduced release cadence we are planning to do more micro releases. This will be focused on critical bugs and security vulnerabilities. However, we may accept contributions to less critical bugs given the fix is well tested and has low risk of regressions.

Keycloak 6.0.1 released

Wednesday, April 24 2019

To download the release go to Keycloak downloads.

For details on what is included in the release check out the Release notes. The full list of resolved issues are available in JIRA

Before you upgrade remember to backup your database and check the upgrade guide for anything that may have changed.

For older entries go here.