Skip to content

SSH behind the scenes

Introduction

Secure Shell Protocol (SSH) is a cryptographic network protocol that enables secure remote access between systems. OpenSSH, the most popular implementation, has evolved significantly in recent years. In this article, I will dive into the most interesting code examples from the OpenSSH implementation. Some of this knowledge can be useful for troubleshooting why an SSH client cannot connect to an SSH server.

SSH Connection Diagram

How encryption is implemented

1. Key exchange and algorithm negotiation

The connection begins with protocol version and cryptographic algorithm negotiation.

Implementation: - kex_exchange_identification() in kex.c handles the initial protocol version exchange and negotiation.
- Cipher algorithm lists are defined in cipher.c, with a preference for modern cryptographic choices: - AEAD ciphers (preferred first): chacha20-poly1305@openssh.com (preferred on platforms without AES hardware acceleration), aes256-gcm@openssh.com, aes128-gcm@openssh.com - Fallback options: AES-CTR (aes256-ctr, aes192-ctr, aes128-ctr) with HMAC-SHA2
- Negotiated parameters are stored in the ssh->kex->newkeys structure during key exchange.

2. Diffie-Hellman key exchange

SSH uses elliptic curve Diffie-Hellman (ECDH) by default in modern implementations.

Current code: - Supports X25519 (Curve25519) for perfect forward secrecy - Legacy DH now disabled by default in OpenSSH 8.2+ - Curve NIST P-256 (ecdh-sha2-nistp256) available as fallback

3. Encryption key generation

The shared secret generates session keys using a specialized key derivation function.

Process: - Uses SSH-specific KDF based on HMAC (not standard HKDF) - Implemented in kex_derive_keys() in kex.c - Keys stored in struct ssh->newkeys[] array - Protocol employs unique key derivation with distinct session keys for each direction

4. Data encryption (Modern Methods)

Modern OpenSSH prefers AEAD (Authenticated Encryption) modes.

Implementation: - ChaCha20-Poly1305: Default on systems without AES hardware acceleration - May use OpenSSL's EVP_chacha20_poly1305() or custom implementation depending on platform - Optimized for software implementations - AES-GCM: Default on systems with hardware acceleration - Takes advantage of AES-NI instructions when available - Fallback to AES-CTR with HMAC-SHA2 for compatibility

Key functions: - cipher_init() in cipher.c sets up encryption context - cipher_crypt() handles both encryption/decryption via OpenSSL EVP or custom implementations

5. Data integrity protection

Modern AEAD ciphers handle integrity automatically.

For legacy ciphers: - HMAC verification occurs during decryption - mac.c contains legacy HMAC implementations - Uses SHA2 family (SHA256, SHA512) for message authentication

6. Encrypted data transmission

Packet handling has been optimized in recent versions.

Packet flow: 1. ssh_packet_read() in packet.c receives incoming data 2. cipher_crypt() decrypts and verifies integrity 3. ssh_dispatch_run() processes decrypted messages (in dispatch.c file) 4. Binary packet protocol handles framing and sequencing

Complete overview

Connection establishment

  1. Algorithm negotiation
  2. Client and server agree on strongest mutual algorithms
  3. Uses extension negotiation (RFC 8308) for modern features

  4. Protocol version and compatibility checks

  5. Key exchange

  6. X25519 ECDH by default
  7. Generates ephemeral session keys

  8. Server authentication through host key verification

  9. Authentication

  10. Public key (Ed25519 preferred)
  11. FIDO/U2F security key support (since OpenSSH 8.2)
  12. Certificate-based authentication (increasingly common in enterprise environments)
  13. SSH certificates provide scalable key management and fine-grained control

Data protection

  • Confidentiality
  • AEAD ciphers (ChaCha20-Poly1305/AES-GCM)
  • Both 128-bit and 256-bit encryption (depending on negotiated algorithm)

  • Separate encryption keys for each direction

  • Integrity

  • Built into AEAD modes
  • For legacy: HMAC-SHA256+

  • Protects against tampering and data corruption

  • Replay protection

  • Sequence numbers in packet headers
  • Strict message ordering enforcement
  • Protection from replay and out-of-order packet attacks

💡 OpenSSH source code is primarily hosted in the OpenBSD CVS repository, with an official mirror available on GitHub

Can SSH traffic be captured with tcpdump or Wireshark?

Answer:

Capturing SSH packets with tools like tcpdump or Wireshark does not allow an attacker to decrypt the communication without access to session keys or a fundamental cryptographic protocol failure. This is due to several key security mechanisms that SSH employs:

1. Perfect Forward Secrecy (PFS)

SSH uses the Diffie-Hellman key exchange (or ECDH) to ensure Perfect Forward Secrecy (PFS). This means that even if an attacker captures the entire communication and later obtains the private keys of either the client or server, they will still be unable to reconstruct the session keys used for encryption.

How does this work? - Session keys are ephemeral, generated for a single session, and discarded once the session ends.
- The shared secret for session key generation is never transmitted over the network, making it impossible to retrieve from captured traffic.

2. Strong encryption algorithms

OpenSSH employs modern encryption algorithms such as: - ChaCha20-Poly1305 (default on systems without AES hardware acceleration) - AES-GCM (for hardware-accelerated encryption) - AES-CTR with HMAC-SHA2 (as a fallback option)

Without access to the session key, decrypting SSH traffic would require breaking the encryption algorithm itself, which is currently infeasible.

3. Data integrity via HMAC

Every SSH packet includes a Message Authentication Code (HMAC), ensuring that data has not been altered during transmission.

  • Even if an attacker captures encrypted packets, they cannot modify them without invalidating the HMAC.
  • Any tampered packets will be detected and discarded by the receiving SSH endpoint.

4. No possibility of reconstructing session keys

Session keys are derived using Diffie-Hellman secrets, which are: - Independently computed by both the client and server.
- Never sent over the network, preventing passive eavesdroppers from reconstructing them.

When could decryption be possible?

While SSH encryption is highly secure, there are theoretical scenarios where decryption might be possible:

🔴 1. Weak SSH configuration

  • If an SSH server uses outdated algorithms (e.g., deprecated ciphers like CBC), attacks such as man-in-the-middle (MITM) or downgrade attacks could be feasible.
  • Modern OpenSSH versions (8.8+) enforce strong defaults, including not accepting RSA keys below 2048 bits and using RSA with SHA-2 signatures instead of SHA-1.
  • Modern OpenSSH versions enforce strong cryptographic defaults.

🔴 2. Compromised session keys

  • If an attacker gains physical access to the server or client and extracts the session keys from memory during an active session, they can decrypt the traffic.
  • This is difficult to achieve remotely unless the system is already compromised (e.g., via malware or privilege escalation attacks).

🔴 3. Vulnerabilities in SSH implementations

  • Side-channel attacks, memory leaks, or cryptographic weaknesses in SSH software could expose encryption keys.
  • OpenSSH is regularly audited and updated to mitigate such risks.

Conclusion

Captured SSH packets are virtually impossible to decrypt without access to the session keys or a critical protocol failure. SSH is designed with strong security mechanisms, including Perfect Forward Secrecy and robust encryption algorithms, making passive eavesdropping ineffective.

Key Takeaways:

✅ Capturing SSH traffic with tcpdump or Wireshark does not allow decryption. ✅ Session keys are ephemeral and never transmitted over the network. ✅ Strong encryption (AES-GCM, ChaCha20-Poly1305) protects confidentiality. ✅ Data integrity is enforced via HMAC to prevent tampering. ✅ Only compromised systems or weak configurations pose a risk.

🔍 If you need to analyze SSH traffic, you must have session key access or use application-level debugging tools to capture unencrypted data before encryption occurs. Otherwise, captured packets will remain encrypted and unreadable.

References

  • RFC 4251 - SSH Protocol Architecture
  • RFC 8308 - SSH Extension Negotiation
  • OpenSSH Release Notes (8.0+)
  • OpenSSH Source Code