How HTTPS Works?

This is my understanding of Why HTTPS is Required and How HTTPS Works. The exact facts like steps of particular algorithm, etc.s might differ but the general concept about HTTPS is almost similar (from the resources I have read).

About HTTP

Let’s talk in points:

• HTTP is a request-reponse application layer protocol
• HTTP is a plaintext protocol so it is limited to local development — anything user facing should be HTTPS
• Underlying transport can be TCP, UDP, QUIC anything
• HTTP is stateless so each request should carry enough information for the server to understand it

    sequenceDiagram
  autonumber
  Client->>Server: GET /order/123 HTTP/1.1
  Server-->>Client:HTTP/1.1 200 OK {response json}
  Note over Client, Server: Underlying Transport Layer Connection <br> (TCP, UDP or QUIC)
  

Why Need HTTPS

We need HTTPS because plaintext HTTP includes following problems of privacy, integrity and indentification which HTTPS solves – stay with me to see how it solves.

Privacy

HTTP is a plaintext protocol i.e. every information sent to or received from server is in plain text readable by anyone who is in the middle of this communication. You control only your host – you can’t control every network and networking devices involved between you and the server you are accessing for whatever reason.

This means every simple information or your password is carried in plaintext enabling anyone eavesdropping the communication to read and potentially use your personal information including your passwords, api keys for evil.

    sequenceDiagram
  participant You as Your Host
  participant Net as Networks & Devices<br/>(Outside Your Control)
  participant Server as HTTP Server
  participant Eve as Eavesdropper

  You->>Net: HTTP Request<br/>GET /login<br/>password=secret123
  Net->>Server: HTTP Request<br/>GET /login<br/>password=secret123

  Note over Net,Eve: Traffic is plaintext
  Eve-->>Net: Reads the HTTP traffic
  Eve-->>Eve: Sees password<br/>secret123
  Eve-->>Eve: Uses password for his benefit

  Server-->>Net: HTTP Response<br/>Account information
  Net-->>You: HTTP Response<br/>Account information
  

Integrity

Think of this as a letter sent in old days. When you are sending letter to your friend, the process is generally the following:

• You write a letter and post in on postbox
• Postman takes it from postoffice
• Postman delivers letter to your friend

Everything is okay if postman doesn’t do anything. But it doesn’t mean postman can’t do anything. Postman can easily open your mail, manipulate every word you wrote, forge it and give it to your friend and you have no idea about this at all. What happens next is that your friend is angry at you and … you know the rest. The same analogy applies to HTTP. You may have heard about man-in-the-middle attack. Well the postman is that man in the middle.

    sequenceDiagram
  participant You as You
  participant Postman as Postman<br/>(Man in the Middle)
  participant Friend as Friend

  You->>Postman: "Let's have dinner tonight!"
  Note over Postman: ✉ Opens the letter
  Postman->>Postman: ✏ Changes the message
  Postman->>Friend: "Don't ever talk to me again!"
  Note over Friend: Friend thinks<br/>this came from You
  Note over You,Friend: Neither You nor Friend<br/>knows the letter was altered
  

Identification

When you are browsing the web say it example.com, you want that the site you are visiting is indeed the one you think it is and it this case it is example.com. HTTPS does this by attaching some sort of signature from the server side which you (client) can verify and say it is valid or invalid. But in plaintext HTTP, there is no way you can attach the signature – even if attached, as seen previously, anyone in the middle could potentially forge it.

    sequenceDiagram
  participant C as Your Browser
  participant A as Attacker / Middleman
  participant S as example.com

  C->>A: HTTP request<br/>"I want example.com"

  Note over A: Attacker intercepts request

  A-->>C: HTTP response<br/>"I am example.com"

  Note over C: No cryptographic identity<br/>to verify who actually responded

  C->>A: Sends information
  A->>A: Reads / modifies information

  Note over C,A: ❌ Browser cannot reliably prove<br/>the server is really example.com
  

What HTTPS Does Differently

When HTTP is protected with TLS (Transport Layer Security), it becomes HTTPS. TLS encrypts the communication between the client and server, helping protect data from being read or modified while it travels across the network.

But to understand how HTTPS works and what it does differently, I am tying to provide rough step by step view of how HTTPS connection happens. Don’t take this as exact process rather take this as a general process which is only intended to make the concept of HTTPS more clear.

1. Establish Underlying Connection (Same For HTTP)

Even though HTTP/3 is the latest version, HTTP/2 remains the dominant protocol for actual web requests, and even legacy HTTP/1.1 still carries a significant portion of global web traffic. Hence, I am assuming the underlying protocol be TCP:

    sequenceDiagram
  autonumber
  participant C as Client<br/>(Browser)
  participant S as Server<br/>(example.com)

  Note over C,S: TCP Handshake
  C->>S: SYN
  S-->>C: SYN/ACK
  C->>S: ACK
  Note over C,S: Connection Established
  

2. Certificate/Signature Check

We can say the actual HTTPS process start from here because previous step was just to establish transport layer connection using TCP. The purpose of certificate/signature check is to verify server identity — is example.com you are going to browse really the example.com you think it is.

    sequenceDiagram
  autonumber

  participant C as Client<br/>(Browser)
  participant S as Server<br/>(example.com)

  Note over C,S: Certificate/Signature Check
  C->>S: Client Hello (Supported TLS versions, encryption algorithms AKA cipher suites + Client Random)
  S-->>C: Server Hello (Selected TLS versions, cipher suites + Server Random)
  S-->>C: Certificate + Signature (Certificate signed by Private key also includes Public key)
  C->>C: Is certificate valid?<br/>(asks to CA)
  C->>C: Is sinature valid?<br/>(verifies using Server's public key)
  Note over C,S: Check Done (if checks are okay)
  

After Client Hello and Server Hello, when example.com sends Certificate and Signature, it’s Client (browser, curl, etc.) responsibility to validate them.

Certificate Validation Process

I would phrase the third point like this, while slightly correcting point 2:

1. The browser downloads the certificate presented by the server. This certificate is digitally signed by the private key of the CA (or an intermediate CA) that issued it.

   A Certificate Authority (CA) is a trusted organization that verifies the identity of websites, people, or companies and issues digital certificates to secure online communications.

2. Browsers/operating systems come with a pre-installed trust store containing trusted root CA certificates. The browser uses the CA’s public key from the certificate chain to verify the digital signatures.

3. If the certificate is not directly signed by a trusted root CA, the browser follows the certificate chain upward. It finds the intermediate CA that signed the server’s certificate, then checks who signed that intermediate CA, continuing until it reaches a trusted root CA in its trust store. This is called the chain of trust.

example.com certificate
        │
        │ signed by
        ▼
Intermediate CA
        │
        │ signed by
        ▼
Trusted Root CA
        │
        ▼
Browser's Trust Store ✓

Signature Validation

This is relatively easy to understand. Think of server’s signature as a message signed by the server using its private key. Once the client receives that message, it can easily validate the message using server’s public key because it’s already extracted from the server’s certificate sent alongwith it.

3. Key Exchange/Change Cipher Sepc

Let the diagram talk first and I will talk later to avoid your confusion on step 1 through 5 — stay with me:

    sequenceDiagram
  autonumber
  participant C as Client<br/>(Browser)
  participant S as Server<br/>(example.com)

  opt Key Exchange
    C->>S: Client Key Exchange (`pre_master_secret` encrypted using Server's public key <br/> if calculated by algorithms other than DH)
    S->>S: Decrypts the secret<br/>using its private key
  end

  C->>C: Calculates `session_key`
  S->>S: Calculates `session_key`

  C->>S: Change Cipher Spec (Request to start using symmetric `session key` instead of asymmetric)
  C->>S: Finished (sends encrypted message using `session_key` — basically saying I am ready!)

  S-->>C: Change Cipher Spec (Confirmation of Request)
  S-->>C: Finished (sends encrypted message using `session_key` — basically saying I am ready too!)
  

First of all: Do you notice Client Key Exchange part is optional? This is because, if DH algorithm is used — both party already know the pre_master_secret whereas it may not the case when RSA or different key exchange algorithm is used. I know you are already wondering and probably saying What the hell are you talking about or probably yelling at me:

What is DH Algorithm? What is pre_master_secret? What is session_key? Well — you have all your answer next.

And Change Cipher Sepc is a fancy way of saying:

Let’s start using the session_key (symmetric key) we together have derived previously instead of asymmetric key.

But Why -> Because using symmetric key is faster compared to asymmetric key and also the session_key is subject to forward secerecy — hence cryptography is swithed here. Here is what Wikipedia says about forward secerecy:

In cryptography, forward secrecy (FS), also known as perfect forward secrecy (PFS), is a feature of specific key-agreement protocols that gives assurances that session keys will not be compromised even if long-term secrets used in the session key exchange are compromised, limiting damage.[1][2][3] For TLS, the long-term secret is typically the private key of the server. Forward secrecy protects past sessions against future compromises of keys or passwords. By generating a unique session key for every session a user initiates, the compromise of a single session key will not affect any data other than that exchanged in the specific session protected by that particular key.

Key Generation

Generally, Diffie-Hellman (DH) algorithm is used to calculate shared secret. In TLS terms, this is called as pre_master_secret which is used to calculate master_secret which is then used to derieve session_key (this is not the exact TLS term but assumed for simlicity) and other keys.

where the “pre_master_secret” is the result of some key exchange protocol. For example, when the handshake uses an RSA ciphersuite, this value is generated uniformly at random by the client, whereas for Ephemeral Diffie-Hellman (DHE) ciphersuites, it is the result of a Diffie-Hellman key agreement.

The is roughly the Diffie-Hellman key agreement:

Diffie–Hellman Example

• The Client and Server publicly agree on:

  • p = 23 -> Random Prime Number
  • g = 5 -> Co-prime Number of p-1

• Client chooses a private value:

  • a = 4 -> Not known to anybody except the Client, never sent anywhere
  • Calculates A = g^a mod p = 5^4 mod 23 = 4
  • Sends A = 4 to the Server.

• Server chooses a private value:

  • b = 3 -> Not known to anybody except the Server, never sent anywhere
  • Calculates B = g^b mod p = 5^3 mod 23 = 10
  • Sends B = 10 to the Client.

• Client calculates the shared secret:

  • s = B^a mod p = 10^4 mod 23 = 18

• Server calculates the shared secret:

  • s = A^b mod p = 4^3 mod 23 = 18

• Both now have the same shared secret:

  • s = 18

The reason they arrive at the same value is:

B^a mod p
= (g^b mod p)^a mod p
= g^(ba) mod p
= (g^a mod p)^b mod p
= A^b mod p
i.e B^a mod p (Client's Caclulation) == A^b mod p (Server's Calculation)

The private values a and b are never transmitted.

    sequenceDiagram
  participant C as Client<br/>(Browser)
  participant S as Server<br/>(example.com)

  Note over C,S: Shared Secret Generation

  C->>S: DH Parameters (`p` and `g`)

  S-->>S: Computes `A = g^a mod p`<br/>(`a` is private to Server) 
  C-->>C: Computes `B = g^a mod p`<br/>(`b` is private to Client)

  S-->>C: Hey Client! Take `A` for further calculation
  C->>S: Hey Server! Take `B` for further calculation

  S->>S: Shared Secret = `B^a mod p`
  C->>C: Shared Secret = `A^b mod p`

  Note over C,S: Both have same Shared Secret
  

As already state this shared secret (pre_master_secret) is then used to calculate master_secret by which session_key is derieved and session key is what encrypts the underlying HTTP messages. This much is enough to know about keys but here are some interesting security facts pointed by RFC 7627 — TLS 1.2 (you can skip to next if you find it boring):

The Transport Layer Security (TLS) master secret is not cryptographically bound to important session parameters such as the server certificate. Consequently, it is possible for an active attacker to set up two sessions, one with a client and another with a server, such that the master secrets on the two sessions are the same.

master_secret = PRF(pre_master_secret, "master secret",
                    ClientHello.random + ServerHello.random)
                    [0..47];

Therefore it adds:

When the extended master secret extension is negotiated in a full handshake, the “master_secret” is computed as:

master_secret = PRF(pre_master_secret, "extended master secret",
                    session_hash)
                    [0..47];

where:

session_hash = Hash(handshake_messages) where “handshake_messages” refers to all handshake messages sent or received, starting at the ClientHello up to and including the ClientKeyExchange message, including the type and length fields of the handshake messages.

4. Encypted HTTP Data Transmission

After you have TLS session_key, all HTTP Data are transmitted encrypted using that session_key.

    sequenceDiagram
  autonumber
  participant C as Client<br/>(Browser)
  participant S as Server<br/>(example.com)

  Note over C,S: Data Transmission
  C->>S: Encrypted Request
  S-->>C: Encrypted Response
  Note over C,S: Connection Established
  

Summary Diagram of Steps

    sequenceDiagram
  autonumber
  participant C as Client<br/>(Browser)
  participant S as Server<br/>(example.com)

  Note over C,S: 1. Underlying TCP<br/>Handshake
  Note over C,S: 2. Certificate &<br/>Signature Validation
  Note over C,S: 3. Key Exchange &<br/>Change Cipher Spec
  Note over C,S: 4. Encrypted HTTP<br/>Data Transmission
  

After HTTPS

After HTTPS, Privacy, Integrity and Identification all are ensured.

Privacy Ensured

    sequenceDiagram
  participant You as Your Host
  participant Net as Networks & Devices<br/>(Outside Your Control)
  participant Server as HTTPS Server
  participant Eve as Eavesdropper

  You->>Net: HTTPS Request<br/>🔒 Encrypted data
  Net->>Server: HTTPS Request<br/>🔒 Encrypted data

  Note over Net,Eve: Traffic is encrypted
  Eve-->>Net: Captures the HTTPS traffic
  Eve-->>Eve: Sees encrypted data<br/>Cannot read password
  Eve-->>Eve: Cannot use the protected data

  Server-->>Net: HTTPS Response<br/>🔒 Encrypted data
  Net-->>You: HTTPS Response<br/>🔒 Encrypted data

  Note over You,Server: Only You and the Server<br/>can read the protected contents
  

Integrity Ensured

    sequenceDiagram
  participant You as You
  participant Postman as Postman<br/>(Still in the Middle)
  participant Friend as Friend

  Friend->>You: 🔓 Sends an open padlock<br/>(public key)
  Note over You: Locks the letter<br/>with that padlock
  You->>Postman: 🔒 "Let's have dinner tonight!"<br/>(encrypted, unreadable)
  Note over Postman: Can carry it<br/>but can't read it
  Postman->>Postman: ✏ Tries to change it...<br/>lock breaks, tampering visible
  Postman->>Friend: 🔒 Locked letter (or a broken one)
  Note over Friend: Opens it with the only key<br/>(private key)
  Friend->>You: "Yes! 7pm works!" 🔒
  Note over You,Friend: Postman can still deliver the mail,<br/>but can't read or silently alter it
  

Identity Ensured

    sequenceDiagram
  participant C as Your Browser
  participant A as Attacker / Middleman
  participant S as example.com
  participant CA as Certificate Authority

  S->>CA: Proves it owns example.com
  CA->>S: Signed certificate<br/>(example.com + public key)
  Note over C: Already trusts the CA<br/>(built into browser / OS)

  C->>A: TLS ClientHello<br/>"I want example.com"
  Note over A: Intercepts the request

  alt Attacker impersonates example.com
    A-->>C: Fake certificate<br/>("I am example.com")
    Note over C: CA signature invalid<br/>or name mismatch ❌
    C--xA: Connection rejected,<br/>no data sent
  else Attacker only relays
    A->>S: Forwards request
    S->>A: Real certificate + public key
    A->>C: Delivers it as-is
    Note over C: CA signature valid,<br/>name matches ✅
    C->>S: Key exchange<br/>(session key, encrypted)
    C->>A: 🔒 Encrypted request
    A->>S: 🔒 Forwards unreadable data
    Note over A: Can carry it but<br/>can't read or modify it
  end
  

View HTTPS Handshake on Wireshark

Generate server’s private key and certificate:

openssl req -x509 -newkey rsa:2048 -keyout server.key -out server.crt -days 365 -nodes

Use this python script as server:

import socket
import ssl

ADDRESS = "127.0.0.1"
PORT = 42437

context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(
    certfile="server.crt",
    keyfile="server.key",
)

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as server:
    server.bind((ADDRESS, PORT))
    server.listen(1)

    print(f"Listening on {ADDRESS}:{PORT}")

    raw_connection, client_address = server.accept()

    with context.wrap_socket(raw_connection, server_side=True) as connection:
        print(f"Client connected: {client_address}")

        while True:
            message = connection.recv(1024)

            if not message:
                break

            connection.sendall(message)

python3 server.py

Use openssl client to connect:

openssl s_client -connect localhost:42437 -showcerts -debug

Open wireshark and filter by port 42437 and see traffic.

References

• Nice GIF Image -> https://www.linkedin.com/posts/alexxubyte_systemdesign-coding-interviewtips-activity-7355986800470827010-cnFk
• Comic Style howhttps -> https://howhttps.works/why-do-we-need-https/
• Key Exchange (DH) in plain English -> https://security.stackexchange.com/questions/45963/diffie-hellman-key-exchange-in-plain-english
• Stackoverflow Explanations -> https://stackoverflow.com/questions/188266/how-are-ssl-certificates-verified
• A Long Image -> https://imgur.com/gallery/ssl-conversation-between-computer-server-5T2fJsG