SSH key management best practices for teams

TL;DR

SSH keys are the gatekeepers of your infrastructure. Shared keys, stale authorized_keys entries for departed team members, and missing passphrases are the most common ways teams accidentally leave the front door open. This guide covers generating strong keys, distributing them safely, revoking access promptly, and scaling key management as your team grows.

Generate Ed25519 keys

Ed25519 is the recommended algorithm — it produces shorter keys with stronger security than RSA and is resistant to several classes of side-channel attacks:

ssh-keygen -t ed25519 -C "alice@company.com" -f ~/.ssh/id_ed25519_company

Always set a passphrase when prompted. If you must use RSA (legacy systems), use a minimum of 4096 bits:

ssh-keygen -t rsa -b 4096 -C "alice@company.com" -f ~/.ssh/id_rsa_company

One key per person — never share keys

Every team member generates their own keypair. Shared keys make it impossible to audit who accessed what, and revoking access means rotating the key for everyone:

# Each developer's public key is added individually
echo "ssh-ed25519 AAAA... alice@company.com" | sudo tee -a /home/deploy/.ssh/authorized_keys
echo "ssh-ed25519 AAAA... bob@company.com" | sudo tee -a /home/deploy/.ssh/authorized_keys

Set correct permissions:

sudo chmod 700 /home/deploy/.ssh
sudo chmod 600 /home/deploy/.ssh/authorized_keys
sudo chown -R deploy:deploy /home/deploy/.ssh

Revoking access when team members leave

When someone leaves the team, immediately remove their public key from every server:

# Remove a specific key by email comment
sudo sed -i '/alice@company.com/d' /home/deploy/.ssh/authorized_keys

For larger teams, automate this. A manual grep across dozens of servers is error-prone and slow.

SSH agent forwarding risks

Agent forwarding (ssh -A) is convenient but dangerous — any user with root access on the intermediate server can hijack your forwarded agent to authenticate as you. Prefer ProxyJump instead:

# ~/.ssh/config — safer than agent forwarding
Host production
    HostName 10.0.1.50
    User deploy
    ProxyJump bastion.company.com
    IdentityFile ~/.ssh/id_ed25519_company

Centralised key management

As your team scales beyond a handful of people, managing authorized_keys files manually becomes unsustainable.

AuthorizedKeysCommand lets sshd fetch keys dynamically from a central source:

# /etc/ssh/sshd_config
AuthorizedKeysCommand /usr/local/bin/fetch-keys.sh %u
AuthorizedKeysCommandUser nobody

The script queries an API, LDAP directory, or HashiCorp Vault for the user's current public keys. When someone leaves, you remove them from the central store and every server respects the change on the next login.

HashiCorp Vault SSH secrets engine takes this further by issuing short-lived signed certificates instead of long-lived keys:

vault write ssh-client-signer/sign/developer \
    public_key=@$HOME/.ssh/id_ed25519.pub \
    valid_principals="deploy" \
    ttl="8h"

Certificates expire automatically — no manual revocation required.

Audit logging SSH access

Enable verbose logging to track who connects and when:

# /etc/ssh/sshd_config
LogLevel VERBOSE

Review authentication logs:

sudo grep "Accepted publickey" /var/log/auth.log | tail -20

For compliance, ship these logs to a centralised aggregator with tamper-proof retention.

SSH key rotation policy

Establish a rotation schedule and enforce it:

• Rotate keys annually at minimum
• Rotate immediately when a team member leaves or a laptop is compromised
• Use ssh-keygen -y -f keyfile to verify key integrity periodically
• Maintain an inventory of which keys are deployed where

With Reflex

Reflex tracks SSH access across your fleet, alerts on unrecognised public keys appearing in authorized_keys files, and flags servers where departed team members still have access — closing the gap between your off-boarding checklist and reality. See How it works.