Skip to content

8.11 Service Networking in Kubernetes

Abstract

Service Networking explains how Kubernetes Services provide stable virtual IPs and access paths to Pods.

Pods are temporary and their IPs can change. Services solve this by giving applications a stable endpoint such as a ClusterIP or NodePort.

In production, Service networking depends heavily on kube-proxy, iptables/ipvs, non-overlapping CIDRs, and correct Pod networking.

Why Services Are Needed

Pods are not ideal direct communication targets because:

  • Pod IPs can change
  • Pods can be recreated
  • Pods can move across nodes
  • multiple replicas may exist behind one application

A Service provides a stable virtual endpoint in front of Pods.

Note

Applications should usually communicate through Services, not directly through Pod IPs.

Service Types Covered

This page focuses on two common Service types:

Service Type Purpose
ClusterIP Internal cluster access
NodePort External access through node IP and node port

Tip

ClusterIP is the default Service type.

ClusterIP Service

A ClusterIP Service exposes Pods inside the Kubernetes cluster.

Example:

Service IP: 10.99.13.178
Backend Pod: 10.244.1.2

Traffic flow:

Client Pod
   ↓
ClusterIP:Port
   ↓
kube-proxy rule
   ↓
Backend Pod IP:Port

Success

ClusterIP Services are ideal for internal applications such as databases, APIs, and internal backends.

ClusterIP Key Point

A Service is not a real process.

It does not have:

  • a container
  • a network namespace
  • a Linux interface
  • a process listening directly on the Service IP

Instead, it is a virtual IP implemented using forwarding rules.

Note

Services are cluster-wide virtual objects. They are not tied to one specific node.

NodePort Service

A NodePort Service exposes an application on a port across all nodes.

Example:

NodePort: 30080
Service IP: 10.99.13.179
Backend Pod: 10.244.2.2

Access pattern:

http://<node-ip>:30080

Traffic flow:

External Client
   ↓
NodeIP:NodePort
   ↓
kube-proxy forwarding rule
   ↓
Service / Backend Pod

Example

If NodePort is 30080, traffic to 192.168.1.11:30080, 192.168.1.12:30080, or 192.168.1.13:30080 can reach the backend Service.

Service vs Pod Networking

Pod networking gives every Pod an IP.

Service networking gives applications stable access to those Pods.

Layer Handles
Pod Networking Pod IPs, Pod-to-Pod communication
Service Networking Stable virtual IPs, load forwarding
kube-proxy Service forwarding rules
CNI Pod network setup

Tip

CNI handles Pod networking. kube-proxy handles Service networking.

kube-proxy Role

Every node runs kube-proxy.

kube-proxy watches the Kubernetes API server for Service and Endpoint changes.

When a Service is created, kube-proxy:

  1. detects the new Service
  2. reads the Service IP and port
  3. reads the backend Pod endpoints
  4. creates forwarding rules on every node
kube-apiserver
   ↓
kube-proxy on each node
   ↓
iptables / ipvs rules
   ↓
Traffic forwarded to Pods

Note

kube-proxy does not create Pods. kubelet creates Pods. kube-proxy creates Service forwarding rules.

kube-proxy Modes

kube-proxy supports multiple proxy modes:

Mode Description
userspace Older mode where kube-proxy proxies traffic in user space
iptables Uses Linux iptables NAT rules
ipvs Uses Linux IPVS load balancing

Example flag:

kube-proxy --proxy-mode [userspace | iptables | ipvs]

Success

iptables and ipvs are commonly used in production. ipvs is often preferred for large clusters due to performance and scalability.

iptables Mode

In iptables mode, kube-proxy creates NAT rules.

Example:

Service: 10.103.132.104:3306
Pod:     10.244.1.2:3306

iptables rule behavior:

Traffic to 10.103.132.104:3306
      ↓
DNAT
      ↓
Forward to 10.244.1.2:3306

Example

kube-proxy adds rules so traffic to a Service IP and port is redirected to a matching backend Pod IP and port.

View Service and Pod Details

Check Pod IP:

kubectl get pods -o wide

Example:

NAME   READY   STATUS    IP          NODE
db     1/1     Running   10.244.1.2  node-1

Check Service IP:

kubectl get service

Example:

NAME         TYPE        CLUSTER-IP      PORT(S)
db-service   ClusterIP   10.103.132.104  3306/TCP

Service Cluster IP Range

Service IPs come from the API server's Service CIDR.

Check API server flag:

ps aux | grep kube-apiserver

Look for:

--service-cluster-ip-range=10.96.0.0/12

Default may be:

10.0.0.0/24

Example range:

10.96.0.0  →  10.111.255.255

Warning

Service CIDR must not overlap with Pod CIDR, node CIDR, VPC CIDR, VPN CIDR, or on-prem network ranges.

Pod CIDR vs Service CIDR

Example:

Network CIDR
Pod network 10.244.0.0/16
Service network 10.96.0.0/12
Node network 192.168.1.0/24

These ranges must be separate.

Danger

If Pod CIDR and Service CIDR overlap, traffic can become unpredictable and Services may fail.

View kube-proxy iptables Rules

Search for Service rules:

iptables -L -t nat | grep db-service

Example meaning:

10.103.132.104:3306  →  10.244.1.2:3306

kube-proxy adds comments with the Service name, which makes searching easier.

Tip

When debugging Service traffic in iptables mode, search by Service name, ClusterIP, or target port.

kube-proxy Logs

Check kube-proxy logs to confirm proxy mode and Service rule creation.

Example log path may vary:

cat /var/log/kube-proxy.log

Look for messages like:

Using iptables Proxier
Adding new service "default/db-service:3306"

Warning

Log location depends on how Kubernetes was installed. In many clusters, kube-proxy runs as a Pod in kube-system.

kube-proxy as a Pod

In kubeadm-style clusters, kube-proxy usually runs as a DaemonSet.

Check it:

kubectl get pods -n kube-system -o wide | grep kube-proxy

View logs:

kubectl logs -n kube-system <kube-proxy-pod-name>

Note

kube-proxy should run on every node because every node needs Service forwarding rules.

Service Networking Flow

For a ClusterIP Service:

Pod sends request to Service IP
      ↓
Node iptables/ipvs rule matches Service IP:Port
      ↓
Traffic is forwarded to backend Pod IP:Port
      ↓
Response returns to client Pod

For NodePort:

External client sends request to NodeIP:NodePort
      ↓
Node rule matches NodePort
      ↓
Traffic is forwarded to Service backend
      ↓
Backend Pod responds

NodePort Port Range

NodePort Services expose ports from a configured range.

Default range:

30000-32767

Example:

NodePort: 30080

Warning

Ensure firewalls, cloud security groups, and network ACLs allow the required NodePort range if external access is needed.

Production Best Practices

Recommended

  • Use ClusterIP for internal services
  • Use Ingress or LoadBalancer instead of NodePort for production external traffic when possible
  • Keep Pod CIDR and Service CIDR non-overlapping
  • Monitor kube-proxy health
  • Use ipvs for large-scale clusters when appropriate
  • Restrict NodePort exposure with firewalls/security groups
  • Document Service CIDR and NodePort range
  • Use labels/selectors carefully so Services target the right Pods

Do's

  • Use Services instead of direct Pod IPs
  • Verify Service endpoints
  • Check kube-proxy mode
  • Check Service CIDR
  • Check iptables or ipvs rules when debugging
  • Use meaningful Service names
  • Restrict external exposure

Don'ts

  • Don't depend on Pod IPs for application communication
  • Don't allow Service CIDR to overlap with Pod or node networks
  • Don't expose databases through NodePort
  • Don't open the full NodePort range unnecessarily
  • Don't assume a Service is a real process listening on an interface
  • Don't ignore kube-proxy logs during Service issues

Danger

Misconfigured Service networking can break application communication even when Pods are healthy.

Troubleshooting Checklist

When a Service is not reachable, check:

  • Does the Service exist?
  • Does the Service have the correct selector?
  • Are Endpoints created?
  • Are backend Pods running and Ready?
  • Is kube-proxy running on all nodes?
  • Is kube-proxy using iptables or ipvs?
  • Are iptables/ipvs rules created?
  • Is the Service CIDR valid and non-overlapping?
  • Are firewall rules blocking NodePort traffic?
  • Is the target container listening on the expected port?

Useful Commands

List Services:

kubectl get svc

Describe Service:

kubectl describe svc <service-name>

Check Endpoints:

kubectl get endpoints

Check Pods with IPs:

kubectl get pods -o wide

Check kube-proxy Pods:

kubectl get pods -n kube-system -o wide | grep kube-proxy

Check kube-proxy logs:

kubectl logs -n kube-system <kube-proxy-pod-name>

Check Service CIDR:

ps aux | grep kube-apiserver

Check iptables rules:

iptables -L -t nat | grep <service-name>

Check ipvs rules, if using IPVS:

ipvsadm -Ln

Quick Comparison

Feature ClusterIP NodePort
Scope Internal cluster External via node IP
Has ClusterIP Yes Yes
Exposes node port No Yes
Default use case Internal service Basic external access
Production preference Common Use carefully

Tip

In production, prefer ClusterIP behind Ingress, Gateway API, or cloud LoadBalancer for controlled external access.

Summary

Quote

  • Services provide stable virtual access to Pods
  • ClusterIP is internal-only
  • NodePort exposes a Service on every node
  • Services do not have real interfaces or processes
  • kube-proxy watches Services and creates forwarding rules
  • kube-proxy can use userspace, iptables, or ipvs mode
  • Service CIDR and Pod CIDR must not overlap
  • Debug Service issues using endpoints, kube-proxy logs, and iptables/ipvs rules