Solving The Challenge Using K8s Service

Kubernetes Service is minimal when you want to deploy your app in K8s because we have already seen the challenges of Kubernetes Deployment which were, to highlight, Pods’ IP Address changes too often (Pods are ephemeral) which impacts local development and even with ReplicaSet there is no Load Balancing which means there is no real point of having Pod replicas. Kubernetes Service intends to solve the problem by exposing the application on one virtual endpoint (IP:PORT) but there are still nuances on who or which network can access the exposed application — let me uncover all.

Rewinding The World Before K8s Service

The world before K8s Service is messy. Yes, you’ve multiple replicas of your Pod or application running, But What?

1. Who will Load Balance?
2. How all Pods are discovered?
3. ReplicaSet recreate when Pod dies but where is endpoint mapping for the new Pod?
4. How other developers or service (say frontend) access app running in another Pod?

We have already been through this mess if you are comming from the previous chapter (which is mandatory because this is continuation to that). What’s interesting here is Service cleans all these mess but the obvious question is “How?”.

Let’s understand the problem more deeply here. Deployment rolls out ReplicaSet which makes sure N Pods exist at any given time BUT THOSE PODS ARE STILL ONLY ACCESSIBLE INSIDE THE CLUSTER. What’s Cluster?

Cluster => Control Node + Worker Node
And if you are in minikube that’s actually the same Node acting as Control and Worker Node which is not the case in production however.

In simpler words, anyone have SSH access or machine level access to any Node in the Cluster can access the application but it might be highly impractical giving Cluster level access to every user even inside the organization. And if the app is meant to be public — consider the game is already over — unless you’re using K8s Service.

Three Major Ways of Exposing Application

Kubernetes mentions “Exposing Application” is the primary concern of Service and Service does that in one of the three ways:

ClusterIP

The app is ccessible from inside the Cluster which means you still need SSH access to the Cluster but this time instead of changing IP Adresss, you get ClusterIP which rarely changes or even better you get Cluster DNS Name for that ClusterIP so you no longer need to worry Pods being ephemeral. We will look into YAML later.

    ---
title: Your Application
---
flowchart
    subgraph K8s Cluster
        subgraph frontend
            direction LR
            P1["Pod3"]
            P2["Pod4"]
        end

        subgraph BackendService["Backend Service<br>(Listening on ClusterIP:port)"]
            BgSvc["Service Port: 80"]
        end

        subgraph backend
            direction LR
            P3["Pod1"]
            P4["Pod2"]
        end

        frontend -- ClusterIP / Cluster DNS --> BackendService
        BackendService -- balances load --> backend
    end
  

The IP Addresses are still there and still changes often but now fronend not longer need to worry on that. This is suitable for backend because this makes frontend acessing backend easily possible and backend is not something user access directly.

NodePort

The backend is exposed using a ClusterIP Service because it only needs to be accessible within the cluster. However, when users need to access your application, you must expose the frontend externally.

Exposing the frontend using ClusterIP alone doesn’t make much sense for direct external access because users shouldn’t need to log in to a Worker Node to use your application. Instead, you can expose the frontend using a NodePort Service.

A NodePort Service makes the application accessible through a specific port on each Node in the cluster, provided network connectivity and firewall rules allow access.

For example, suppose your cluster has four Nodes with the following IP addresses:

• 200.0.0.1
• 200.0.0.2
• 200.0.0.3
• 200.0.0.4

If you configure the NodePort as 30380, users who can reach these Nodes can access your application using any of the following addresses:

200.0.0.1:30380
200.0.0.2:30380
200.0.0.3:30380
200.0.0.4:30380

Notice that 30380 is within Kubernetes’ default NodePort range of 30000–32767. You can configure a different range if needed.

Also, accessing the application through 200.0.0.1:30380 doesn’t mean the frontend Pod must run on the Node with IP 200.0.0.1. Kubernetes networking, typically implemented through kube-proxy or an alternative data plane, routes the incoming traffic to an appropriate Pod, even if that Pod runs on another Node.

This means users can access the application through any reachable Node’s IP address and the configured NodePort without needing to know which Node hosts the frontend Pod.

Much simplified diagram can be:

    ---
title: Your Application — NodePort
---
flowchart LR
    U["Users"]
    subgraph K8s Cluster
        N1["Node 1"]
        N2["Node 2"]
        N3["Node 3"]

        subgraph FrontendService
            NP["NodePort: 30380<br/>Service Port: 80"]
        end

        subgraph frontend
            direction LR
            P1["Pod3"]
            P2["Pod4"]
        end

        subgraph BackendService
            BgSvc["Service Port: 80"]
        end

        subgraph backend
            direction LR
            P3["Pod1"]
            P4["Pod2"]
        end

        N1 -- Cluster IP / Cluster DNS --> FrontendService
        N2 -- Cluster IP / Cluster DNS --> FrontendService
        N3 -- Cluster IP / Cluster DNS --> FrontendService

        FrontendService -- balances load --> frontend
        frontend -- Cluster IP / Cluster DNS --> BackendService
        BackendService -- balances load --> backend
    end

    U -- NodeIP:30380 --> N1
    U -- NodeIP:30380 --> N2
    U -- NodeIP:30380 --> N3
  

One Question

One question may arise here: if the application is accessible through any of the four Nodes, how is traffic distributed? What happens if you use 200.0.0.1 to access the application and that Node goes down or its IP address changes?

Well, a NodePort Service allows you to access the application through any available Node, but it doesn’t provide a single, stable IP address for users. For fronend to backend talk, there is already internal ClusterIP but for users, if a Node becomes unavailable, you can access the application through another reachable Node, but users or clients must know which Node to access.

Similarly, NodePort itself doesn’t distribute traffic from a single entry point across all Nodes. It provides the same port on each Node, and Kubernetes routes incoming traffic to an appropriate backend Pod.

So, how do you provide a single entry point that remains stable and distributes incoming traffic across your application? That’s where the next approach comes in.

LoadBalancer

As we discussed earlier, NodePort allows users to access the application through any reachable Node, but it doesn’t provide a single, stable entry point. Users still need to know which Node to access.

This is where LoadBalancer comes in.

When you expose your frontend using a LoadBalancer Service, Kubernetes can provision an external load balancer through a supported cloud provider or load-balancer implementation. Users can then access the application through the load balancer’s IP address or hostname instead of connecting directly to individual Nodes.

One important distinction for our Minikube lab environment is that setting type: LoadBalancer alone does not automatically provision an external load balancer. This Service type is primarily used with cloud providers or environments that have a load-balancer implementation.

In Minikube, the Service may remain in a Pending state until you run minikube tunnel or configure another suitable load-balancer implementation.

The load balancer provides a single entry point and distributes incoming traffic across the application. If a Node becomes unavailable, traffic can be directed to healthy destinations, depending on the load balancer’s configuration and health checks.

For example, suppose your cluster has four Nodes:

                 Users
                   |
                   v
          Load Balancer
          203.0.113.10
                   |
          +--------+--------+
          |        |        |
          v        v        v
       Node 1   Node 2   Node 3 ... Node 4
          |        |        |        |
          +--------+--------+--------+
                   |
                   v
             Frontend Service
                   |
                   v
             Frontend Pods
                   |
                   v
             Backend Service
                   |
                   v
              Backend Pods

Users access the application through 203.0.113.10, for example, without needing to know the IP addresses of individual Nodes.

Under the hood, the exact traffic flow depends on the cloud provider and load-balancer implementation. A LoadBalancer Service commonly builds on NodePort-style routing, although some implementations can route traffic directly to Pods.

In short, ClusterIP provides internal access, NodePort exposes the application through Node IPs and a port, and LoadBalancer provides a stable external entry point through a load balancer.

The Real Question: How K8s Service Solves The Problem?

A Word of Caution: This is going to be very much oversimplified.

First of all, let’s talk on how internal load balancing is possible with K8s Service. Think it simply — to load balance, you need to keep track of all the Pods’ Endpoint i.e. IP:Port for all N replicas rolled out by ReplicaSet so that you can distribute or forward traffic accordingly and then you also need Virtual IP to listen to and then forward the traffic. But still, the underlying problem is the same:

Even with Endpoint tracking, Pods are still ephemeral, IP Address still changes.

Therefore, if Pods still dies, IP Address still changes then of course the Endpoints change too. So previsouly IP Addresses may be [10.244.0.91, 10.244.0.92, 10.244.0.93] and the next moment can be [10.244.0.99, 10.244.0.94, 10.244.0.97].

There is a concept of labels and selectors which ReplicaSet has already been using to know “Which Pods I need to make sure N replicas exist?”. While writing YAML specification, you basically give your Pod a label app=backend for example and you use the same label to select the Pods for ReplicaSet to manage them.

The same concept is used by Service too. ReplicaSet uses it for one purpose and Service uses for different. The real solution labels and selectors provides is:

The label of Pod doesn’t change no matter how many times it dies and gets recreated.

Now, this very property is used by Service for its own benefit — keeping track of Endpoints. Internally, directly or indirectly almost all components of K8s are talking to kube-api-server to get the necessary information. So, you can think that whenever Pod with the same label the Service is tracking gets deleted or recreated, Service knows that information already and use that information to update Endpoint list. That Endpoint list is actually an object in K8s which Kubernetes called as EndpointSlices. In fact, you can query K8s API for it:

kubectl get endpointslices

Okay! Service auto creates and updates EndpointSlices for all matching labels and Service now has list of Endpoints — But is it really enough?

No, it’s not. List of Endpoints is just a list like any other list — Traffic still has to be Routed. And when it comes to networking in Kubernetes kube-proxy always has a place in it. As a devops engineer, you need not perform extra hardwork because it is already installed on all Data Plane or Worker Nodes and kube-proxy does some magic now. It scans the Service sepecification and depending on type of Service (ClusterIP, NodePort, LoadBalancer), kube-proxy updates the IPTABLES or any kernel level packet filtering supported by OS and acts as a proxy (forwarder, relayer, whatever make sense to you).

There are K8s own words for kube-proxy regarding iptables proxy mode:

In this mode, kube-proxy configures packet forwarding rules using the iptables API of the kernel netfilter subsystem. For each endpoint, it installs iptables rules which, by default, select a backend Pod at random.

Few last words “select a backend Pod at random” is what makes load balancing possible. What exact rules are maintained is not necessary to know but you can view that using iptales-save in any worker nodes if you want to further investigate or even view K8s source code — it’s open source. But, for now, this much of explanation should already be enough. If you are interested more on how Virtual IP address or Cluster IP Address is assigned for internal load balancing, you can refer to the official documenation regarding Virtual IPs (assigned by API server).

Solving The Previous Chapter Code

The problem we were facing in prvious chapter while coding the frontend was that backend Pod IP Address keeps changing. The exact line:

# other lines remains the same
# BACKEND_HOST = "10.244.0.143" # Pod IP is emphemeral
# refer to by FQDN (Cluster DNS) because even ClusterIP might change
BACKEND_HOST = "backend.default.svc.cluster.local"
# other lines remains the same

We can refer the entire backend by backend.default.svc.cluster.local because Service has made available that:

apiVersion: v1
kind: Service
metadata:
  name: backend
spec:
  type: ClusterIP
  selector:
    app: backend
  ports:
    - port: 80
      targetPort: 80

Notice the type=ClusterIP because we want backend accessed only inside the Cluster by the frontend whereas frontend is exposed using type=NordPort (because of minikube limitation on type=LoadBalancer):

apiVersion: v1
kind: Service
metadata:
  name: frontend
spec:
  type: NodePort
  selector:
    app: frontend
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30380

See, the selector is selecting the exact Pod to watch out for app=frontend or app=backend which is mandatory. This aligns perfectly with the NodePort architecture diagram drawn above.