Kerberos

IF YOU ARE LOOKING FOR A PASSWORD RESET, PLEASE VISIT https://csclub.uwaterloo.ca/ceo/pwreset FOR A SELF-SERVICE PORTAL FOR PASSWORD RESETS! (If you experience any issues, please contact Syscom)

---

We use MIT Kerberos 5 for authentication. Our kerberos realm is CSCLUB.UWATERLOO.CA. This realm is CASE-SENSITIVE. KDCs run on auth1 (kdc1) and auth2 (kdc2).

Kerberos, the network authentication protocol

ehashman's guide to MIT Kerberos v5 on Debian

Preparatory Reading

1. Kerberos: A Dialogue in Four Scenes (definitely read this)
2. Explain Like I'm 5: Kerberos (less entertaining than the stage play)
3. A very practical configuration guide to Kerberos on Debian squeeze (things don't change much in the Debian world)
4. The official Kerberos documentation

Set up host records

• We will need host records to correspond to our Kerberos admin server and key distribution center, kadmin.wics.uwaterloo.ca and kdc1.wics.uwaterloo.ca. These can just be A records pointing to our auth server (currently 129.97.134.212).

• We can also set up special SRV records as well. This is recommended but not necessary. They look like this:

  _kerberos._udp.wics.uwaterloo.ca          SRV       0 0 88  kdc1.wics.uwaterloo.ca
  _kerberos-master._udp.wics.uwaterloo.ca   SRV       0 0 88  kdc1.wics.uwaterloo.ca
  _kpasswd._udp.wics.uwaterloo.ca           SRV       0 0 464 kdc1.wics.uwaterloo.ca

  You may have guessed that the third integer is the port the service runs on.

Install packages

• First, install some common system utils that may be missing from the fresh container:

  # apt-get install ssh ntpdate xinetd nmap

• Do NOT install ntp on the container. Install it on the host system instead. See NTP for info on NTP servers.

• Next, install the Kerberos server:

  apt-get install krb5-{admin-server,user}

• During the install process, dpkg will ask you for the following three values, specified below:

  Default Kerberos version 5 realm: WICS.UWATERLOO.CA
  Kerberos servers for your realm: kdc1.wics.uwaterloo.ca
  Administrative server for your Kerberos realm: kadmin.wics.uwaterloo.ca

• You'll encounter this lovely error, from xinetd:

  Note: xinetd currently is not fully supported by update-inetd.
    Please consult /usr/share/doc/xinetd/README.Debian and itox(8).

  To solve this, we create a file /etc/xinetd.d/krb_prop with the following contents:

  service krb_prop
  {
      disable     = no
      socket_type = stream
      protocol    = tcp
      user        = root
      wait        = no
      server      = /usr/sbin/kpropd
  }

  And then restart xinetd:

  # service xinetd restart

• You'll also note that the krb5-kdc service failed to start. This is okay. > This is because the realm, EXAMPLE.COM, or rather the database file for it (/var/lib/krb5kdc/principal), has not yet been created. – http://www.rjsystems.nl/en/2100-d6-kerberos-master.php

Configuring Kerberos

• The first thing we'll configure is the access control list. Edit /etc/krb5kdc/kadm5.acl and enable/add the following line:

  */admin *

  Our primary admin principal will be sysadmin/admin@WICS.UWATERLOO.CA, so there is no need to add a separate admin principal to the ACL.

• Let's configure Kerberos client-side in /etc/krb5.conf. Consulting with the CSC's config, our favoured setup guide, and the Kerberos krb5.conf manual, we'll mostly select default settings. Notable additions include

  [libdefaults]
          allow_weak_crypto = false  # default is currently false but hey
  
          # If DNS breaks we don't want auth to fail
          dns_lookup_kdc = false
          dns_lookup_realm = false
  
  [logging]
          kdc = FILE:/var/log/krb5kdc.log
          admin_server = FILE:/var/log/kadmin.log
          default = FILE:/var/log/krb5.log

• We also want to ensure we're using good crypto for our Key Distribution Center, so let's set that up next in /etc/krb5kdc/kdc.conf:

  [kdcdefaults]
      kdc_ports = 750,88
  
  [realms]
      WICS.UWATERLOO.CA = {
          database_name = /var/lib/krb5kdc/principal
          admin_keytab = FILE:/etc/krb5kdc/kadm5.keytab
          acl_file = /etc/krb5kdc/kadm5.acl
          key_stash_file = /etc/krb5kdc/stash
          kdc_ports = 750,88
          max_life = 12h 0m 0s
          max_renewable_life = 1d 0h 0m 0s
          master_key_type = aes256-cts-hmac-sha1-96
          supported_enctypes = aes256-cts-hmac-sha1-96:normal aes128-cts-hmac-sha1-96:normal
          default_principal_flags = +preauth
      }

• We didn't choose to create a new krb5 log directory but we should set up logrotate. Create a file /etc/logrotate.d/krb5 with three of the following entries (one for each log file):

  /var/log/FILENAME.log {
      weekly
      missingok
      rotate 8
      compress
      delaycompress
      notifempty
      postrotate
              /etc/init.d/SERVICENAME restart > /dev/null
      endscript
  }

• Make sure you also create those files so the service can write to them:

  # touch /var/log/{krb5,krb5kdc,kadmin}.log

Creating the Kerberos Realm

• Now we're going to create the realm:

  # krb5_newrealm
  This script should be run on the master KDC/admin server to initialize
  a Kerberos realm.  It will ask you to type in a master key password.
  This password will be used to generate a key that is stored in
  /etc/krb5kdc/stash.  You should try to remember this password, but it
  is much more important that it be a strong password than that it be
  remembered.  However, if you lose the password and /etc/krb5kdc/stash,
  you cannot decrypt your Kerberos database.
  Loading random data

  The script may pause at this point until there is sufficient available entropy to generate a key. Then it will prompt for a password. USE A LONG, RANDOM ONE. THIS PASSWORD IS VERY IMPORTANT.

  Initializing database '/var/lib/krb5kdc/principal' for realm
  'WICS.UWATERLOO.CA',
  master key name 'K/M@WICS.UWATERLOO.CA'
  You will be prompted for the database Master Password.
  It is important that you NOT FORGET this password.
  Enter KDC database master key:
  Re-enter KDC database master key to verify:
  
  
  Now that your realm is set up you may wish to create an administrative
  principal using the addprinc subcommand of the kadmin.local program.
  Then, this principal can be added to /etc/krb5kdc/kadm5.acl so that
  you can use the kadmin program on other computers.  Kerberos admin
  principals usually belong to a single user and end in /admin.  For
  example, if jruser is a Kerberos administrator, then in addition to
  the normal jruser principal, a jruser/admin principal should be
  created.
  
  Don't forget to set up DNS information so your clients can find your
  KDC and admin servers.  Doing so is documented in the administration
  guide.

• We'll now configure the default and maximum ticket life for the Kerberos Ticket Granting Ticket (krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA):

  # kadmin.local 
  Authenticating as principal root/admin@WICS.UWATERLOO.CA with password.
  kadmin.local:  getprinc krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA
  Principal: krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA
  Expiration date: [never]
  Last password change: [never]
  Password expiration date: [none]
  Maximum ticket life: 0 days 12:00:00
  Maximum renewable life: 1 day 00:00:00
  Last modified: Thu Dec 03 03:59:04 UTC 2015 (db_creation@WICS.UWATERLOO.CA)
  Last successful authentication: [never]
  Last failed authentication: [never]
  Failed password attempts: 0
  Number of keys: 2
  Key: vno 1, aes256-cts-hmac-sha1-96, no salt
  Key: vno 1, aes128-cts-hmac-sha1-96, no salt
  MKey: vno 1
  Attributes: REQUIRES_PRE_AUTH
  Policy: [none]

  Let's set the max life to 4 hours and the renewable life to 10 hours, for extra security.

  kadmin.local:  modprinc -maxlife "4 hour" -maxrenewlife "10 hour" krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA
  Principal "krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA" modified.

Adding Principals

• We need some root users in our system in order to bootstrap the rest, so let's create our sysadmin user, and give them our root password for authentication:

  # kadmin.local
  Authenticating as principal root/admin@WICS.UWATERLOO.CA with password.
  kadmin.local:  addprinc sysadmin/admin
  WARNING: no policy specified for sysadmin/admin@WICS.UWATERLOO.CA; defaulting to no policy
  Enter password for principal "sysadmin/admin@WICS.UWATERLOO.CA": 
  Re-enter password for principal "sysadmin/admin@WICS.UWATERLOO.CA": 
  Principal "sysadmin/admin@WICS.UWATERLOO.CA" created.

• Now we need to add a principal and keytab for our KDC host. While addprinc -randkey does add a key, we need to use ktadd to ensure it's copied over to the client host (in this case, auth1). (keytab: a key table file containing one or more keys. A host or service uses a keytab file in much the same way as a user uses his/her password.)

  $ kadmin -p sysadmin/admin
  Authenticating as principal sysadmin/admin with password.
  Password for sysadmin/admin@WICS.UWATERLOO.CA:
  kadmin:  addprinc -randkey host/auth1.wics.uwaterloo.ca
  WARNING: no policy specified for
  host/auth1.wics.uwaterloo.ca@WICS.UWATERLOO.CA; defaulting to no policy
  Principal "host/auth1.wics.uwaterloo.ca@WICS.UWATERLOO.CA" created.
  kadmin:  ktadd host/auth1.wics.uwaterloo.ca
  Entry for principal host/auth1.wics.uwaterloo.ca with kvno 2, encryption type
  aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
  Entry for principal host/auth1.wics.uwaterloo.ca with kvno 2, encryption type
  aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.

• Now we can test that the KDC can grant principals tickets:

  $ kinit sysadmin/admin
  Password for sysadmin/admin@WICS.UWATERLOO.CA: 
  $ klist
  Ticket cache: FILE:/tmp/krb5cc_0
  Default principal: sysadmin/admin@WICS.UWATERLOO.CA
  Valid starting       Expires              Service principal
  12/03/2015 05:31:38  12/03/2015 09:31:38  krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA
     renew until 12/03/2015 15:31:38
  $ kdestroy
  $ klist
  klist: Credentials cache file '/tmp/krb5cc_0' not found

• Next, we'll probably want to add principals for any users that we created in LDAP. We can do this in weo using the following command, and we can even test that principal after its creation:

  $ python weo.py --add-krb-princ --username=ehashman
  Okay, adding Kerberos principal ehashman@WICS.UWATERLOO.CA
  Enter Kerberos admin password: 
  Enter password for principal ehashman@WICS.UWATERLOO.CA: 
  Retype password: 
  Adding Kerberos principal...
  Principal ehashman@WICS.UWATERLOO.CA successfully added.
  $ kinit ehashman
  Password for ehashman@WICS.UWATERLOO.CA: 
  $ klist
  Ticket cache: FILE:/tmp/krb5cc_1000
  Default principal: ehashman@WICS.UWATERLOO.CA
  Valid starting     Expires            Service principal
  15-12-03 17:36:22  15-12-03 21:36:22  krbtgt/WICS.UWATERLOO.CA@WICS.UWATERLOO.CA
   renew until 15-12-04 03:36:22

• From now on, though, Kerberos principals will automatically be generated when we add new users! Like this:

  $ python weo.py --adduser --username=fhboxwal --fullname="Fatema Boxwala"
  Okay, adding user fhboxwal
  Please enter the new user's password: 
  Retype password: 
  Enter LDAP admin password: 
  Enter Kerberos admin password: 
  Locking LDAP database...
  Adding user...
  Unlocked database.
  Adding Kerberos principal...
  User fhboxwal successfully added.

• Awesome! Now we're ready to configure Kerberos for clients.

Setting Up Client Machines with SSSD

• On your machine of choice, install the Kerberos client packages:

  # apt-get install krb5-user

• Now copy over your Kerberos config, krb5.conf, into /etc/krb5.conf.

• Next, set up a host keytab for the local machine:

  # kadmin -p sysadmin/admin
  Authenticating as principal sysadmin/admin with password.
  Password for sysadmin/admin@WICS.UWATERLOO.CA: 
  kadmin:  addprinc -randkey host/mother-goose.wics.uwaterloo.ca
  WARNING: no policy specified for host/mother-goose.wics.uwaterloo.ca@WICS.UWATERLOO.CA; defaulting to no policy
  Principal "host/mother-goose.wics.uwaterloo.ca@WICS.UWATERLOO.CA" created.
  kadmin:  ktadd host/mother-goose.wics.uwaterloo.ca
  Entry for principal host/mother-goose.wics.uwaterloo.ca with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.
  Entry for principal host/mother-goose.wics.uwaterloo.ca with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab FILE:/etc/krb5.keytab.

• In order to configure authentication, we'll use a package called SSSD. (It has 234823840 dependencies.) Install it and its utilities:

  # apt-get install sssd sssd-tools

• Next, copy over the following configs:
• /etc/ldap/ldap.conf
• /etc/sssd/sssd.conf
• /etc/ssh/sshd_config
• /etc/ssh/ssh_config

• /etc/hosts (because what the heck)

• Restart sssd and sshd. The former can be very temperamental:

  # service ssh restart
  # service sssd restart

• Test that this all worked by attempting to log in:

  # Just try logging in
  $ ssh me@machine.wics.uwaterloo.ca
  
  # Try logging in using Kerberos
  $ kinit me
  $ ssh -o GSSAPIAuthentication=yes me@machine.wics.uwaterloo.ca
  
  # Test that sudo is working
  machine:~$ sudo -i

Tools for Debugging SSSD

• It turns out sssd is not the greatest at telling us things. If it starts breaking, stop it and start it in the foreground in debugging mode:

  # service sssd stop
  # sssd -d 5 -i

• Some problems with sssd may be cache-related, and restarting it does not clear the cache. If you need to invalidate the cache, run

  sss_cache -E

and then reboot (or restart sssd).

Password Resets

To change your own password you can run passwd on any of the club's machines.

Changing other user's passwords (new)

The ceo utility now has the ability to reset members' passwords, and it also takes care of expiring it. Usage:

ceo members pwreset <username>

Changing other users' passwords (old)

• ssh auth1

• sudo kadmin.local

• cpw username

• Enter new password and confirm

• modify_principal +needchange username

API Documentation. While not even close to enough to let you do most things that you'd want to do with Kerberos (and also being somewhat woefully out-of-date, considering it's from 1996), it's at least a start.

Expiring Passwords

If you are on syscom, you can force a user to change their password by doing this:

• ssh auth1

• sudo kadmin.local

• modify_principal +needchange [username]

Suspending an Account

If you are on syscom, you can prevent a user from logging with a Kerberos ticket by doing this:

• ssh auth1
• sudo kadmin.local
• modify_principal -allow_tix [username]

If you are seriously locking out an account, you may want to do some other things as well, including but not limited to changing the user's password (prevents password login) and changing the ownership and permissions on .ssh/authorized_keys* (prevents SSH key login). Don't do these things without a strong reason (but know how to do them when the time comes).

bofh's Kerberos5 cheat sheet, or "what does *that* error message mean, exactly?"

• If GSSAPI complains about "Wrong Principal in Request", make sure there's no clockskew on the machine trying to get the service ticket and the machine running the service that you are trying to get a GSS token to. This will cause this error for some insane reason, despite there being ANOTHER message for clockskew that specifically says "your clocks are off" - it just never seems to be used in the source code anywhere (as of MIT-KRB5 1.9, at least).
• There are some "generic" errors that are hard to debug. A few possible causes: unreadable krb5.keytab, reverse resolution of a host does not match its principal.

Replication

auth1 replicates to auth2 using kpropd. This requires additional setup that needs to be documented here.

Occasionally, the replication results in "ulog_replay failed (Cannot allocate memory), updates not registered". To correct this, run `kproplog -R` on auth1 and start the `kpropd` server again.

raymo's guide to keytabs

Adapted from https://stackoverflow.com/a/55826172/9206488 and https://sfu.teamdynamix.com/TDClient/255/ITServices/KB/ArticleDet?ID=3932

You can follow this guide to never need to kinit or enter your password for CEO again. Note that all caps is needed for the Kerberos realm, and replace <user> with your CSC username.

ktutil
addent -password -p <user>@CSCLUB.UWATERLOO.CA -k 1 -e aes256-cts-hmac-sha1-96
# enter password at the prompt
wkt <user>.keytab
quit

Then move your keytab to a secure directory:

mkdir -m700 ~/keytabs
mv ~/$USER.keytab ~/keytabs/
chmod 600 ~/keytabs/$USER.keytab

Finally, add this line to the TOP of your bashrc (or before any command to disable it for non-interactive shells, unless you don't want kinit to be triggered when running remote commands:

kinit -kt ~/keytabs/$USER.keytab $USER@CSCLUB.UWATERLOO.CA

Alternatively, use k5start so that your tickets automatically get renewed in the background:

 KEYTAB_FILE=~/keytabs/$USER.keytab
 if command -v k5start >/dev/null && ! pgrep -u $USER k5start >/dev/null && [ -f $KEYTAB_FILE ]; then
   k5start -K 60 -H 70 -b -f $KEYTAB_FILE
 fi
 unset KEYTAB_FILE

Tada! Now you'll automatically get a kerberos ticket on SSH/shell. You can test this by running

ssh hfcs@csclub.uwaterloo.ca klist

You should see a valid ticket from a second or two ago.