Keycloak

We are using Keycloak for web SSO (Single Sign-On). Clients may use Keycloak for authenticating users via SAML or OIDC (OpenID Connect).

• Admin login: https://keycloak.csclub.uwaterloo.ca/auth/admin
• Regular user login: https://keycloak.csclub.uwaterloo.ca/auth/realms/csc/account
• OIDC Auto Discovery URL: https://keycloak.csclub.uwaterloo.ca/auth/realms/csc/.well-known/openid-configuration

Prerequisites

OK so before we get started, there's this really useful feature in Keycloak called "Conditional user attribute" which allows you to create a flow which branches based on attributes a user may have. For some reason, this is enabled in the test suite for Keycloak, but is not available from the main application. So we're going to compile and inject it ourselves.

Clone https://git.csclub.uwaterloo.ca/public/keycloak-spi and run mvn clean package. This will create a JAR file called csc-keycloak-spi.jar in the target directory; upload this to somewhere where it can be easily downloaded, e.g. your www directory.

Database setup

Go to biloba or chamomile, run mysql, and run the following:

CREATE USER 'keycloak' IDENTIFIED BY 'replace_this_password';    
CREATE DATABASE keycloak CHARACTER SET utf8 COLLATE utf8_unicode_ci;    
GRANT ALL PRIVILEGES ON keycloak.* TO 'keycloak';    
FLUSH PRIVILEGES;

Kubernetes setup

We are running Keycloak on Kubernetes. This introduces some complications because it gets reverse proxied twice, and we also can't (or at least shouldn't) modify the filesystem of the Pod where it's running, since that Pod can get destroyed at any time. We still need to load that JAR file we just created, though, so we're going to place it into a PersistentVolume instead. We're going to do this by first creating a PersistentVolumeClaim, then claiming it in a temporary Pod which we'll use for shell access:

cat <<EOF | kubectl apply -f    
apiVersion: v1    
kind: PersistentVolumeClaim    
metadata:    
  namespace: syscom    
  name: keycloak-spi-pvc    
spec:    
  storageClassName: cloudstack-storage    
  accessModes:    
    - ReadWriteOnce    
  resources:    
    requests:    
      storage: 5Mi    
---    
apiVersion: v1    
kind: Pod    
metadata:    
  namespace: syscom    
  name: temp-pod    
spec:    
  containers:    
    - name: temp    
      image: alpine    
      volumeMounts:    
        - mountPath: "/data"    
          name: keycloak-spi-pv    
      stdin: true    
      stdinOnce: true    
      tty: true    
  volumes:    
    - name: keycloak-spi-pv    
      persistentVolumeClaim:    
        claimName: keycloak-spi-pvc
EOF

Run kubectl -n syscom get pods a few times to check if the pod is ready; once it is, attach to it:

kubectl -n syscom exec -it temp-pod -- sh
cd /data
mkdir keycloak-spi
chmod a+w keycloak-spi
cd keycloak-spi
wget https://csclub.uwaterloo.ca/~merenber/csc-keycloak-spi.jar
exit

Now delete the pod since we don't need it anymore:

kubectl -n syscom delete pod temp-pod

Create some secrets (use the MySQL password which you chose earlier):

kubectl -n syscom create secret generic keycloak-secret \
  --from-literal=DB_USER=some_user \
  --from-literal=DB_PASSWORD=some_password \
  --from-literal=KEYCLOAK_USER=some_user \
  --from-literal=KEYCLOAK_PASSWORD=some_password

Now apply the main manifest:

kubectl apply -f https://git.csclub.uwaterloo.ca/cloud/manifests/raw/branch/master/keycloak.yaml

DNS setup

From Infoblox, make keycloak.csclub.uwaterloo.ca a CNAME for rr-public-cloud.csclub.uwaterloo.ca; that record points to biloba and chamomile, which know how to reverse proxy requests to Kubernetes.

NGINX setup

Pretty standard stuff:

server {
  listen 80;
  listen [::]:80;
  server_name keycloak.csclub.uwaterloo.ca;
  return 301 https://$host$request_uri;
}

server {
  listen 443 ssl http2;
  listen [::]:443 ssl http2;
  server_name keycloak.csclub.uwaterloo.ca;
  ssl_certificate /etc/ssl/private/csclub.uwaterloo.ca.chain;
  ssl_certificate_key /etc/ssl/private/csclub.uwaterloo.ca.key;
  
  location / {
    proxy_pass http://k8s;
  }
  include proxy_params;

  access_log /var/log/nginx/keycloak-access.log;
  error_log /var/log/nginx/keycloak-error.log;
}

Also make sure you have the following snippet in /etc/nginx/proxy_params:

# Increase buffer size
# See https://ma.ttias.be/nginx-proxy-upstream-sent-big-header-reading-response-header-upstream/
proxy_buffer_size 128k;
proxy_buffers 4 256k;

Don't forget to enable the site and reload NGINX on both chamomile and biloba.

Web UI setup

If all went well, you should now be able to visit https://keycloak.csclub.uwaterloo.ca from your browser. Create a new realm called 'csc'. Set the Display Name to 'Computer Science Club'.

Realm settings

From the web UI, go to 'Realm Settings', click the Login tab, and disable 'Login with email'.

Now click on the 'Tokens' tab. Set 'SSO Session Idle' and 'SSO Session Max' to 120 days each. You can optionally do this for the Master realm as well.

Now click on the 'Email' tab.

• Set 'Host' to 'mail.csclub.uwaterloo.ca'.
• Set 'From Display Name' to 'Keycloak'.
• Set 'From' to 'no-reply@csclub.uwaterloo.ca'.
• Set 'Reply To Display Name' to 'Systems Committee'.
• Set 'Reply To' to 'syscom@csclub.uwaterloo.ca'.
• Set 'Envelope From' to 'keycloak@csclub.uwaterloo.ca'.
• Enable StartTLS.

Authentication Settings

Click on 'Authentication' from the panel on the left-hand side.

Flows

Click on the 'Flows' tab and select the 'Browser' flow from the dropdown. Create a copy called 'CSC Browser' and make it look like this:

Cookie (Alternative)
CSC Browser Forms (Alternative)
    Username Password Form (Required)
    CSC Membership Check (Conditional)
        Condition - User Attribute (Required)
        Deny Access (Required)
    OTP Form (Required)

The condition should be whether a user has the attribute 'shadowExpire' set to '1'. Set the error message in 'Deny Access' to something reasonable (this gets displayed to users).

Screenshot:

Select the 'First Broker Login' flow and create a copy called 'CSC First Broker Login'. This will be invoked when a user signs in with an Identity Provider, and the IdP user ID is not linked to a Keycloak user. Make the flow look like this:

Review Profile (Disabled)
User Creation or Linking (Required)
    Detect Existing Broker User (Required)
    Automatically Set Existing User (Required)
CSC Membership Check (Conditional)
    Condition - User Attribute (Required)
    Deny Access (Required)

Finally, create a new flow called 'CSC Browser - IdP post-login'. This will be invoked when a user signs in with an IdP, and the IdP user ID has already been linked to a Keycloak user. Make it look like this:

CSC Membership Check - IdP Post-login (Conditional)
    Condition - User Attribute (Membership expired - IdP post-login) (Required)
    Deny Access (Denied because membership expired - IdP post-login) (Required)
Allow Access

Note that the last step needs to be 'Allow Access'.

Bindings

Click on the 'Bindings' tab and set 'Browser Flow' to 'CSC Browser'.

Required Actions

Click on the 'Required Actions' tab and disable the following:

• Update Password
• Update Profile
• Delete Account

User Federation

Click on 'User Federation' from the left-hand panel and select 'ldap'. Configure it as follows:

• Edit Mode: READ_ONLY
• Vendor: Other
• Username LDAP attribute: uid
• RDN LDAP attribute: uid
• UUID LDAP attribute: entryUUID
• User Object Classes: member
• Connection URL: ldaps://ldap1.csclub.uwaterloo.ca ldaps://ldap2.csclub.uwaterloo.ca
• Users DN: ou=People,dc=csclub,dc=uwaterloo,dc=ca
• Custom User LDAP Filter: (objectClass=member)
• Search Scope: One Level
• Bind Type: none

Under Sync Settings, enable Changed Users Full Sync. Set the Changed Users Sync Period to something reasonable (should be at least once a day).

Click on Mappers, click 'email', and set the LDAP Attribute to 'mailLocalAddress'. Also set the LDAP Attribute for 'First Name' to 'givenName'.

Create a new mapper of type 'user-attribute-ldap-mapper'. Set 'User Model Attribute' and 'LDAP Attribute' to 'shadowExpire'. Enable 'Always Read Value From LDAP'.

Now click the 'Synchronize all users' button from the main LDAP configuration page. This will take a few minutes so be patient.

Identity Providers

We are going to use ADFS for identity brokering. The idea is to allow members to login using their CSC credentials or their UW credentials.

SAML Passthrough

I created a monstrosity called saml-passthrough. The idea is to protect a SAML IdP behind another SAML IdP (ADFS). So basically SAML-inside-SAML.

The reason why we need this is because we can't change the AssertionConsumerService endpoint URL which we originally submitted to IST (if we really wanted to, we could submit another form, but that requires assistance from a faculty member, and it's just a big PiTA). This also allows us to multiplex ADFS information to multiple Keycloak realms, if necessary.

Anywho, the instructions for setting it up are in the README (it's just a FastCGI application). It should be running on caffeine right now under /srv/saml-passthrough. It's configured to download SP metadata from Keycloak at startup, so if you update the SP certificate in Keycloak, make sure to restart the saml-passthrough service.

Now, back to the Keycloak UI. Click 'Identity Providers' from the left-hand side, and add a SAML provider. Set the following:

• Alias: adfs
• Display Name: University of Waterloo (ADFS)
• First Login Flow: CSC First Login Flow
• Post Login Flow: CSC Browser - IdP post-login
• Sync Mode: legacy
• Service Provider Entity ID: https://keycloak.csclub.uwaterloo.ca/auth/realms/csc
• Single Sign-On Service URL: https://csclub.uwaterloo.ca/keycloak/saml/sso
• Principal Type: Attribute [Friendly Name]
• Principal Attribute: uid
• HTTP-POST Binding Response: ON
• Want AuthnRequests Signed: ON
• Want Assertions Signed: ON
• Validate Signature: ON
• Validating X509 certificates: download the content of https://csclub.uwaterloo.ca/keycloak/saml/metadata and copy and paste the value inside the <X509Certificate> tag.

That X509 certificate will expire in January 2032; before then, make sure you do the following:

• create a new keypair in /srv/saml-passthrough on caffeine
• restart the saml-passthrough service
• update the 'Validating X509 certificates' field in Keycloak

Keycloak's SP certificate (which can be viewed from Realm Settings -> Keys -> Providers -> rsa-generated) will expire in December 2031; before then, make sure you upload a new keypair (just choose 'rsa-generated' from the 'Add keystore' dropdown), then restart the saml-passthrough service on caffeine.

Testing

Here's a test OIDC application which you can use to verify that everything is working: https://www.keycloak.org/app/#url=https://keycloak.csclub.uwaterloo.ca/auth&realm=csc&client=test