OpenSSH beyond the basics: ProxyJump, ControlMaster, and SSH certificates

~/.ssh/config: the file that should exist

Every option you'd pass on the command line can live in ~/.ssh/config per host:

Host github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_github
    IdentitiesOnly yes

Host bastion
    HostName bastion.example.com
    User ops
    Port 22
    IdentityFile ~/.ssh/id_ed25519_work

Host db-*
    User postgres
    IdentityFile ~/.ssh/id_ed25519_work
    ProxyJump bastion

Host web-*.prod
    HostName %h.lab.example.net
    User www-data
    IdentityFile ~/.ssh/id_ed25519_work

Host *
    # Defaults that apply unless overridden above
    ServerAliveInterval 60
    ServerAliveCountMax 3
    HashKnownHosts yes
    UpdateHostKeys yes
    AddKeysToAgent yes
    IdentityAgent ~/.gnupg/S.gpg-agent.ssh    # if using a YubiKey via gpg-agent

%h is the literal hostname from the command line, so ssh web-01.prod connects to web-01.prod.lab.example.net. Wildcards (db-*, web-*.prod) let one stanza apply to many hosts. The Host * stanza at the bottom catches everything not matched earlier.

ProxyJump: hop through bastions cleanly

To reach a host that's only accessible from another host:

Host db-prod
    HostName 10.0.5.10
    User postgres
    ProxyJump bastion

Now ssh db-prod opens a connection to bastion, then from there to 10.0.5.10. The TCP traffic flows through the bastion; the SSH session encryption is end-to-end (the bastion sees encrypted bytes, not your terminal).

For chained jumps: ProxyJump bastion-a,bastion-b goes A → B → target.

scp and rsync over the same path:

scp file.txt db-prod:/tmp/
rsync -av ./build/ db-prod:/var/www/

Both use ~/.ssh/config automatically; ProxyJump applies.

ControlMaster: open one connection, use it many times

Establishing an SSH session does multiple TCP round-trips plus the key exchange — perceptible latency, especially over slow links. ControlMaster opens one master connection and multiplexes subsequent sessions through it:

Host *
    ControlMaster auto
    ControlPath ~/.ssh/cm/%r@%h:%p
    ControlPersist 10m

mkdir -p ~/.ssh/cm
chmod 700 ~/.ssh/cm

Now the first ssh hostname establishes the master and the session. The second ssh hostname opens a new shell instantly — same TCP connection. scp, rsync, sftp, git push over ssh, all of them piggyback. After 10 minutes of idle the master closes (configurable).

Force-close all masters: ssh -O exit hostname.

SSH certificate authority

Per-host ~/.ssh/authorized_keys entries are operationally painful: adding a new user means updating every host. SSH certificates solve this by introducing a Certificate Authority that signs short-lived user certificates; servers trust the CA and accept any cert it issues.

Generate a CA key:

ssh-keygen -t ed25519 -f ~/.ssh/ca_user -C "user CA"

Sign a user's existing public key with the CA:

ssh-keygen -s ~/.ssh/ca_user \
    -I "amir@2025-05-24" \
    -n amir,root \
    -V +12h \
    /tmp/amir-id_ed25519.pub

This produces amir-id_ed25519-cert.pub — a certificate that:

• Authorizes the public key for SSH logins as user amir or root
• Identifies itself with key-ID amir@2025-05-24 (logged on every login)
• Expires in 12 hours

On every server, configure sshd to trust the CA:

# /etc/ssh/sshd_config
TrustedUserCAKeys /etc/ssh/ca_user.pub

(Put the CA's public key — not the private key — in /etc/ssh/ca_user.pub.)

Now any user with a valid cert signed by that CA can log in as amir or root, with no ~amir/.ssh/authorized_keys entry needed.

Step CA (see that tutorial) automates the entire issue-and-renew loop — users grab fresh 12-hour certs via SSO, never edit authorized_keys again.

Host certificates

The other half: instead of every user trusting every host's first-time-key-fingerprint via known_hosts, the SSH CA signs each host's host key:

# On the host
ssh-keygen -s ~/.ssh/ca_host \
    -I "web-01.prod-2025-05" \
    -h \
    -n web-01.example.com,10.0.5.20 \
    -V +365d \
    /etc/ssh/ssh_host_ed25519_key.pub

# In sshd_config
HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub
HostKey /etc/ssh/ssh_host_ed25519_key

On every client:

# ~/.ssh/known_hosts (or /etc/ssh/ssh_known_hosts)
@cert-authority *.example.com,*.lab.example.net ssh-ed25519 AAAA... host-CA-public-key

Clients now trust any host whose host-key cert is signed by the CA — no "are you sure you want to continue?" prompt on first connect.

Agent forwarding done safely

ForwardAgent yes on a host gives that host access to your local SSH agent — convenient for hopping to a second host using your laptop's keys, but a security hole if the bastion is ever compromised (the attacker can use your agent to log in elsewhere).

Two safer patterns:

1. Don't forward at all. Use ProxyJump (above) — the TCP traffic goes through the bastion but the agent stays on your laptop.
2. If you must forward, use the ssh-add -c confirmation prompt so every use of the agent on the remote side requires a local "OK" click. Pair with a strict ~/.ssh/agent with only the specific keys allowed for that session.

Port forwarding shorthand

# Local forwarding: localhost:8080 → remote service on db-prod:5432
ssh -L 5432:db-prod:5432 bastion

# In ~/.ssh/config
Host pg-tunnel
    HostName bastion
    LocalForward 5432 db-prod:5432
    ExitOnForwardFailure yes
    BatchMode yes

# Remote forwarding: expose a local service on the remote
ssh -R 9000:localhost:9000 dev-vm

# Dynamic SOCKS proxy — route browser traffic through a remote
ssh -D 1080 bastion

Best-practice defaults

# /etc/ssh/sshd_config on every server
PermitRootLogin prohibit-password   # or "no" if SSH cert isn't issuing root certs
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
TrustedUserCAKeys /etc/ssh/ca_user.pub
LogLevel VERBOSE                    # logs key fingerprint used — useful for audit
ClientAliveInterval 60
ClientAliveCountMax 10
AllowAgentForwarding no             # unless you need it

LogLevel VERBOSE is the most-underused setting in sshd_config: it makes journalctl -u ssh log the exact key fingerprint used for every login, which is gold during incident response.