Change IP pool block size

4 MINUTE READ

Big picture

Change the IP pool block size to efficiently manage IP pool addresses.

Value

Changing IP pool block size after installation requires ordered steps to minimize pod connectivity disruption.

Features

This how-to guide uses the following Calico features:

  • IPPool resource with blockSize field

Concepts

Expand or shrink IP pool block sizes

By default, the Calico IPAM block size for an IP pool is /26. To expand from the default size /26, lower the blockSize (for example, /24). To shrink the blockSize from the default /26, raise the number (for example, /28).

Best practice: change IP pool block size before installation

Because the blockSize field cannot be edited directly after Calico installation, it is best to change the IP pool block size before installation to minimize disruptions to pod connectivity.

Before you begin…

Required

  • Verify that you are using Calico IPAM.
    This guide is relevant only if you are using Calico IPAM.

    ssh to one of your Kubernetes nodes and view the CNI configuration.

      cat /etc/cni/net.d/10-calico.conflist

    Look for the “type” entry:

       "ipam": {
         "type": "calico-ipam"
    },

    If the type is “calico-ipam”, you are good to go. If the IPAM is set to something else, or the 10-calico.conflist file does not exist, you cannot use this feature in your cluster.

How to

Important! Follow the steps in order to minimize pod connectivity disruption. Pods may lose connectivity when they are redeployed, and may lose external connectivity while in the temporary pool. Also, when pods are deleted, applications may be temporarily unavailable (depending on the type of application). Plan your changes accordingly.

The high-level steps to follow are:

  1. Create a temporary IP pool
    Note: The temporary IP pool must not overlap with the existing one.
  2. Disable the existing IP pool
    Note: When you disable an IP pool, only new IP address allocations are prevented; networking of existing pods are not affected.
  3. Delete pods from the existing IP pool
    This includes any new pods that may have been created with the existing IP pool prior to disabling the pool. Verify that new pods get an address from the temporary IP pool.
  4. Delete the existing IP pool
  5. Create a new IP pool with the desired block size
  6. Disable the temporary IP pool
  7. Delete pods from the temporary IP pool
  8. Delete the temporary IP pool

Tutorial

In the following steps, our Kubernetes cluster has a default CIDR block size of /26. We want to shrink the block size to /28 to use the pool more efficiently.

Create a temporary IP pool

We add a new IPPool with the CIDR range, 10.0.0.0/16.

Create a temporary-pool.yaml.

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: temporary-pool
spec:
  cidr: 10.0.0.0/16
  ipipMode: Always
  natOutgoing: true

Apply the changes.

calicoctl apply -f temporary-pool.yaml

Let’s verify the temporary IP pool.

calicoctl get ippool -o wide

NAME                  CIDR             NAT    IPIPMODE   DISABLED
default-ipv4-ippool   192.168.0.0/16   true   Always     false
temporary-pool        10.0.0.0/16      true   Always     false

Disable the existing IP pool

Disable allocations in the default pool.

calicoctl patch ippool default-ipv4-ippool -p '{"spec": {"disabled": "true"}}'

Verify the changes.

calicoctl get ippool -o wide

NAME                  CIDR             NAT    IPIPMODE   DISABLED
default-ipv4-ippool   192.168.0.0/16   true   Always     true
temporary-pool        10.0.0.0/16      true   Always     false

Delete pods from the existing IP pool

In our example, coredns is our only pod; for multiple pods you would trigger a deletion for all pods in the cluster.

kubectl delete pod -n kube-system coredns-6f4fd4bdf-8q7zp

Restart all pods with just one command.

WARNING! The following command is disruptive and may take several minutes depending on the number of pods deployed.

kubectl delete pod -A --all

Delete the existing IP pool

Now that you’ve verified that pods are getting IPs from the new range, you can safely delete the existing pool.

calicoctl delete ippool default-ipv4-ippool

Create a new IP pool with the desired block size

In this step, we update the IPPool with the new block size of (/28).

apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
  name: default-ipv4-ippool
spec:
  blockSize: 28
  cidr: 192.0.0.0/16
  ipipMode: Always
  natOutgoing: true

Apply the changes.

calicoctl apply -f pool.yaml

Disable the temporary IP pool

calicoctl patch ippool temporary-pool -p '{"spec": {"disabled": "true"}}'

Delete pods from the temporary IP pool

In our example, coredns is our only pod; for multiple pods you would trigger a deletion for all pods in the cluster.

kubectl delete pod -n kube-system coredns-6f4fd4bdf-8q7zp

Restart all pods with just one command.

WARNING! The following command is disruptive and may take several minutes depending on the number of pods deployed.

kubectl delete pod -A --all

Validate your pods and block size are correct by running the following commands:

kubectl get pods --all-namespaces -o wide
calicoctl ipam show --show-blocks

Delete the temporary IP pool

Clean up the IP pools by deleting the temporary IP pool.

calicoctl delete pool temporary-pool