Systemd

This page contains some tips and tricks for writing systemd units on CSC machines.

Services

TODO

Timers

Unlike cron, systemd timers do not send email alerts if the job fails. However, you can create your own alerts using OnFailure=. Paste the following into /usr/local/bin/csc-systemd-email and make it executable:

#!/bin/bash

# Adapted from https://wiki.archlinux.org/title/systemd/Timers#MAILTO

set -e

if [[ $# -ne 2 ]]; then
  echo "Usage: $0 <address> <unit>" >&2
  exit 1
fi

FROM="Systemd <root@$HOSTNAME>"
TO="$1"
if ! [[ $TO =~ @ ]]; then
  TO="$TO@csclub.uwaterloo.ca"
fi
SUBJECT="Systemd <root@$HOSTNAME> Unit '$2' failed"
MESSAGE="$(systemctl status --full "$2" || true)"

# Don't use the Postfix sendmail. It creates a new spool file and also
# forks to the background, which we don't want.
if [[ -x /usr/sbin/ssmtp ]]; then
  /usr/sbin/ssmtp -t <<EOF
To: $TO
From: $FROM
Subject: $SUBJECT
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

$MESSAGE
EOF
elif [[ -x /usr/bin/mutt ]]; then
  EMAIL="$FROM" /usr/bin/mutt -F /dev/null -e "set copy=no" -s "$SUBJECT" -- "$TO" <<< "$MESSAGE"
else
  echo "Could not find program to email" >&2
  exit 1
fi

Next, paste the following into /etc/systemd/system/csc-email-on-failure@.service:

[Unit]
Description=Send email alert when %i fails

[Service]
Type=oneshot
ExecStart=/usr/local/bin/csc-systemd-email root@csclub.uwaterloo.ca %i
# Do not use DynamicUser=true until this issue gets fixed:
# https://github.com/systemd/systemd/issues/22737
User=nobody
# Need to be in the adm group to read journald logs
Group=adm

Then run systemctl daemon-reload. Now, all you need to do is add the following line to the [Unit] of any service for which you would like to receive email alerts:

OnFailure=csc-email-on-failure@%n.service

IMPORTANT: make sure you have the following setting in /etc/ssmtp/ssmtp.conf:

FromLineOverride=NO

Otherwise, Mailman 3 will reject the message because the Envelope From does not have a FQDN.