How I Get Notified of Failed systemd Timerschain link icon indicating a link to a heading

I have a few backup jobs that are run periodically by systemd timers. Because these are backups, I would like to be notified somehow if they fail. systemd has a configuration option for this, OnFailure=.

This is not a radical idea, and in researching while writing this post, I found what I think is the original ServerFault answer that led me to this technique. But, for what it’s worth, here’s my /etc/systemd/system/notify-failed@.service:

[Unit]
Description=Failure notification for %i

[Service]
Type=oneshot
ExecStart=/opt/pushover.sh/pushover.sh -p 1 -t 'Unit Failed' '%i'

And here’s an example of it in use in a service unit that uploads some files to cloud storage:

[Unit]
Description=Sync files to cloud
OnFailure=notify-failed@%n
AssertPathExists=%f

[Service]
Type=oneshot
ExecStart=/usr/bin/rclone '%f' 'remote:%i'

I use the pushover.sh script from jnwatts on Github to send a simple message based on the name of the failing unit. I’ve found this to be incredibly useful over the years. Pushover is rock-solid, and I’ve never missed a broken backup for long.