Linux Server Hardening: What to Do First and Why It Matters

Linux server hardening guides often present a flat list of fifty things to do, which is not useful when you have a production deployment in two hours. This guide is structured by time: what to do in the first 30 minutes, what to add before the server goes live, and what ongoing maintenance looks like. The order reflects how attackers actually move, not alphabetical convention.

First 30 Minutes: SSH Is the Attack Surface That Kills Most Servers

An internet-facing server running SSH with password authentication will start receiving login attempts within minutes. Bots rotate through leaked credential lists and common passwords continuously. Linux server hardening begins here because no other single step closes as much exposure as fast.

Move to Key-Based Authentication Only

Generate an Ed25519 key pair on your local machine and install the public key on the server:

ssh-keygen -t ed25519
ssh-copy-id user@yourserver

Verify it works before proceeding, then set these in /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin no
AllowUsers youruser
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2

Restart with systemctl restart sshd while keeping an active session open as a safety net. PermitRootLogin no means a stolen private key does not immediately give an attacker full system control. AllowUsers is an explicit whitelist: system accounts not named there cannot authenticate, regardless of key material.

SSH hardening also includes daemon configuration, not just authentication method. The Terrapin attack demonstrated that weak cipher negotiation settings can be exploited even when credentials are airtight.

Update the System Before Anything Else Connects

Run a full update immediately:

apt update && apt upgrade -y     # Debian/Ubuntu
dnf update                        # RHEL-based

Enable automated security updates (Debian: unattended-upgrades). Then set a patch cadence you will actually keep: critical CVEs within 72 hours, high-priority patches within 14 days.

First Hour: Lock Down the Network Perimeter

The firewall stage of linux server hardening reduces the visible attack surface from the full TCP port range to exactly what your server needs. Configure a default-deny policy and add explicit allows:

ufw default deny incoming
ufw default allow outgoing
ufw limit ssh
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable

ufw limit ssh applies connection rate-limiting at the network level, complementing Fail2Ban which operates at the log level. Both belong in the stack; they catch different things.

Install and enable Fail2Ban:

apt install fail2ban
systemctl enable --now fail2ban

Create /etc/fail2ban/jail.local and tighten the SSH jail for production: three failures triggering a one-hour ban, rather than the five-failure, 10-minute default.

Audit the ports that are actually open:

ss -pnltu

Anything listening that has no place in your firewall allow-list should be stopped and removed. Use nmap from a separate host to verify what the server looks like from the outside. The external view is what attackers see.

Before Production: Harden User Accounts and the Kernel

Root should not be the working administration account. Create a dedicated non-root user with sudo:

adduser adminuser
usermod -aG sudo adminuser

Review all accounts on the system. Service accounts should not have interactive login shells. Remove or lock any account with no current purpose. Attackers who get in as an unprivileged user will next attempt privilege escalation to reach root. Keeping the account list trim reduces their options.

Apply kernel hardening parameters in /etc/sysctl.d/99-hardening.conf:

kernel.randomize_va_space = 2
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
kernel.sysrq = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0

randomize_va_space = 2 enables full ASLR, making memory-corruption exploits require a separate information-leak vulnerability to be reliable. tcp_syncookies = 1 absorbs SYN-flood attacks without dropping legitimate connections. dmesg_restrict = 1 prevents unprivileged users from reading kernel ring buffer output, which leaks memory addresses exploitable for local privilege escalation.

Apply immediately: sysctl --system

Set strict mount options for temporary directories. Add nodev,nosuid,noexec to the /tmp fstab entry. Setuid binaries and device files have no business being created in /tmp, and scripts run there should not execute.

Before Production: Mandatory Access Control and Audit Logging

Good linux server hardening includes a MAC layer beneath normal Unix permissions. Mandatory Access Control (MAC) constrains what each process can reach, even as root. On RHEL-based systems, SELinux ships enabled in enforcing mode: verify with getenforce. On Debian and Ubuntu, AppArmor provides equivalent protection: check with apparmor_status.

Both are commonly disabled to resolve policy violations quickly. This is the wrong response: adjust the profile instead. Disabling MAC is removing a containment layer that limits the damage radius of a successful exploit.

Configure auditd to monitor writes to /etc/passwd, /etc/shadow, /etc/sudoers, SSH configuration, cron directories, and kernel module loading. These are the files most frequently modified during post-exploitation. Add AIDE for file integrity monitoring against critical system directories, and schedule it weekly.

Ongoing: Scanning, Patching, and Backup Posture

Linux server hardening does not end at deployment. Every package upgrade, new service deployment, and configuration change can introduce drift. A realistic maintenance cadence:

• Weekly: review authentication logs, verify Fail2Ban is catching attempts, confirm patches applied
• Monthly: run Lynis (lynis audit system), review service inventory, check for accounts that should be removed
• Quarterly: review firewall rules, rotate credentials, test a backup restore

Rootkit scanners like rkhunter and chkrootkit catch common implants and should run monthly. Neither is foolproof against a sophisticated attacker with extended dwell time, but both reliably identify standard post-exploitation tooling.

Backups should assume the server is already compromised. At least one copy should be stored somewhere the production server cannot reach or modify: an offline destination, or a write-once target that refuses deletes during its retention window. Test restores. A backup you have never restored from is not a backup.

What Most Linux Server Hardening Guides Get Wrong

The most common mistake is treating linux server hardening as a checklist you complete once. The second most common is disabling security controls to resolve an inconvenient policy violation rather than understanding the violation. Both end in the same place: a server that looks hardened on paper but has real gaps in practice.

Priority matters more than completeness. An incomplete hardening pass that covers SSH, the firewall, and kernel parameters is far more protective than a full checklist applied once two years ago and never revisited.

Frequently Asked Questions

Should I move SSH off port 22?

It reduces automated scan noise in your logs. Bots that only probe port 22 will miss your service entirely. But this is not a real security control. Key-based authentication, Fail2Ban, and a strict firewall are what actually prevent unauthorised access. Move the port if log volume is a problem; do not treat it as a substitute for any of the controls above.

Is Fail2Ban still relevant with key-based SSH?

Yes. Even with password authentication disabled, attackers still probe SSH and consume resources attempting connection negotiation. Fail2Ban reduces this noise and protects against connection-exhaustion scenarios. It also applies to services beyond SSH, including web application login endpoints.

How do I know if SELinux is breaking something?

Check the audit log: ausearch -m avc -ts recent. Denials show which process, file, and operation was blocked. Use audit2allow to generate a targeted policy module that permits the specific operation, rather than switching SELinux to permissive mode or disabling it entirely.

What is AIDE and do I need it if I have auditd?

Auditd monitors events as they happen, in real time. AIDE compares the current state of files against a trusted baseline captured at install time. They answer different questions. Auditd tells you when a file was changed and by what process. AIDE tells you whether the current state matches the known-good state. Run both.

How long does initial hardening actually take?

SSH hardening and a basic firewall take about 30 minutes. Kernel parameters and account management add another hour. Full audit logging configuration, AIDE baseline, and MAC policy review add another two to four hours. An experienced administrator can complete an initial hardening pass in a working afternoon. The ongoing maintenance cost after that is measured in hours per month, not per week.