auditd answers the questions that arise after the event: who changed this file, when was this user created, where did this process start from. No other tool answers them — application logs do not hold that information, and shell history is edited by whoever left it.

The difficulty is that installing the package gives you nothing by itself. There are practically no default rules, the journal is written but empty in substance, and this is discovered at the worst possible moment — when the data is needed and is not there.

Installation

sudo apt install auditd audispd-plugins
sudo systemctl status auditd

One peculiarity: on some versions auditd cannot be restarted through systemctl restart — the service is managed by its own command. Rules are re-read like this:

sudo augenrules --load
sudo auditctl -l

The second command shows what is really loaded. Trust it, not the contents of the files.

The rule set

Rules go into /etc/audit/rules.d/ as a separate file. Below is a minimal set that pays for itself: it answers most of the questions raised during an incident without burying the disk.

sudo tee /etc/audit/rules.d/50-baseline.rules <<'EOF'
## Users and privileges
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/sudoers -p wa -k privileges
-w /etc/sudoers.d/ -p wa -k privileges

## SSH access
-w /etc/ssh/sshd_config -p wa -k sshd_config
-w /root/.ssh/ -p wa -k ssh_keys

## Scheduled jobs
-w /etc/crontab -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /var/spool/cron/ -p wa -k cron

## Kernel modules
-w /sbin/insmod -p x -k modules
-w /sbin/modprobe -p x -k modules
-a always,exit -F arch=b64 -S init_module -S delete_module -k modules

## Use of sudo and su
-w /usr/bin/sudo -p x -k privileged
-w /bin/su -p x -k privileged
EOF
sudo augenrules --load

The syntax is short: -w is a path to watch, -p wa means react to writes and attribute changes, -p x to execution, -k is the key to search by later. Those keys are the main convenience: without them, searching the journal turns into sifting through a gigabyte of text.

What is not on that list, and why

Guides often suggest recording every execve call — that is, every launch of every program. The temptation is understandable: a complete picture of what was done. In practice, on a working server that produces hundreds of megabytes a day, eats a noticeable share of the processor and leads to the journal being trimmed sooner than it is needed. If that level of detail is required, enable it narrowly — for one user, or for the duration of an investigation.

The same applies to watching the site's directory: writes into it are constant, and the rule turns the journal into a stream of meaningless events.

Reading it

Search by key, with numeric identifiers resolved into names:

sudo ausearch -k identity -i
sudo ausearch -k privileges -i -ts today

The -i flag is essential for readability: without it you get numbers instead of user names and system calls. -ts limits the period (today, recent, or a specific date).

A summary for a period:

sudo aureport --summary -i
sudo aureport --auth -i --failed

Three fields in an event record are interesting: auid, the identifier of the user who started the session, uid, the identity the action ran as, and exe, the program that performed it. The difference between auid and uid is the answer to "who exactly became root": uid=0 with the auid of a specific person names the culprit unambiguously, whatever they claimed to be inside the session.

Disk

The size settings live in /etc/audit/auditd.conf:

max_log_file = 50
num_logs = 5
max_log_file_action = ROTATE
space_left = 500
space_left_action = SYSLOG

Look separately at disk_full_action. Some configurations default to halting the system when the disk fills — behaviour that is justified on machines with audit requirements and entirely out of place on a web server. Check that value before the disk runs out.

Immutable rules

The line -e 2 at the end of the rules file forbids changing the rules until a reboot — including by whoever obtained root. That noticeably raises the value of the journal: an intruder cannot switch off the watching without rebooting the machine, and a reboot is conspicuous in itself.

The flip side is obvious: you cannot change your own rules without a reboot either. Enable it once the rule set has settled and you have lived with it for a month.

Why bother on an ordinary server

auditd prevents nothing. Its value shows exactly once — when the sequence of events has to be reconstructed, and either the data is there or it is not. It cannot be collected retroactively, which is why the rule set is put in place in advance and then left alone.

A practical sign that the setup is right: the rule list is not empty, the journal does not grow uncontrollably, and events with the identity and privileges keys appear rarely and can each be explained. What that looks like on one page is on the demo below.