Logs and detection

Prevention eventually fails somewhere, so defenders also watch. Logs are records that systems write about what happened: who signed in and from where, which files were opened, which programs started, which network connections were made. Without logs you can't tell what an attacker did, or whether they're still there.

A Linux server records SSH sign-ins in its auth log. Each line has a time, the machine, the program, and what happened:

Oct 30 02:14:07 web1 sshd[812]: Failed password for root from 203.0.113.7 port 52144 ssh2
Oct 30 02:14:09 web1 sshd[812]: Failed password for root from 203.0.113.7 port 52150 ssh2
Oct 30 02:14:12 web1 sshd[815]: Failed password for admin from 203.0.113.7 port 52161 ssh2
Oct 30 08:02:44 web1 sshd[901]: Accepted publickey for maya from 192.0.2.15 port 50322 ssh2

Read it as a story: one address tried three passwords in five seconds for two common account names, which is an automated guessing attack. Later, maya signed in with a key from a known office address, which is normal. Windows keeps similar records with event IDs: 4625 is a failed sign-in and 4624 a successful one.

A company has hundreds of systems, so logs are sent to one place: a SIEM (security information and event management). It keeps them, lets analysts search across everything, and runs detection rules that raise alerts. A rule might fire on many failed sign-ins followed by a success, a sign-in from two countries an hour apart (impossible travel), or a new admin account created at 3 a.m.

from collections import Counter
log = [
    "Failed password for root from 203.0.113.7",
    "Failed password for root from 203.0.113.7",
    "Accepted publickey for maya from 192.0.2.15",
    "Failed password for admin from 203.0.113.7",
]
fails = Counter(line.split(" from ")[1] for line in log if "Failed" in line)
print(fails.most_common(1))

Alerts aren't always attacks. A false positive is an alert on harmless activity (someone who forgot their password); a false negative is an attack nobody was alerted to, which is worse. Analysts in a SOC (security operations centre) check each alert, tune noisy rules, and keep logs long enough to look back, since many break-ins are found weeks later. Clocks on every system should agree (synced with NTP) so the timeline lines up.