Toggle nav

Configure MTU to maximize network performance

Big picture

Configure the maximum transmission unit (MTU) for your Calico environment.

Value

Optimize network performance for workloads by configuring the MTU in Calico to best suit your underlying network.

Increasing the MTU can improve performance, and decreasing the MTU can resolve packet loss and fragmentation problems when it is too high.

Features

This how-to guide uses the following Calico features:

• FelixConfiguration resource

Concepts

MTU and Calico defaults

The maximum transmission unit (MTU) setting determines the largest packet size that can be transmitted through your network. MTU is configured on the veth attached to each workload, and tunnel devices (if you enable IP in IP and/or VXLAN).

In general, maximum performance is achieved by using the highest MTU value that does not cause fragmentation or drop packets on the path. Maximum bandwidth increases and CPU consumption may drop for a given traffic rate. The improvement is often more significant when pod to pod traffic is being encapsulated (IP in IP or VXLAN), and splitting and combining such traffic cannot be offloaded to your NICs.

For example, if you are using AWS, you may be able to use jumbo frames up to 9000 bytes. If you are using Calico overlay networks, you may need to adjust the MTU settings to ensure packets aren’t lost or dropped from the size is being too high.

Before you begin…

For help using IP in IP and/or VXLAN overlays, see Configure overlay networking.

How to

• Determine MTU size
• Configure MTU
• View current tunnel MTU values

Determine MTU size

The following table lists common MTU sizes for Calico environments. Because MTU is a global property of the network path between endpoints, you should set the MTU to the minimum MTU of any path that packets may take.

Common MTU sizes

Recommended MTU for overlay networking

The extra overlay header used in IP in IP, VXLAN and WireGuard protocols, reduces the minimum MTU by the size of the header. (IP in IP uses a 20-byte header, VXLAN uses a 50-byte header, and WireGuard uses a 60-byte header). Therefore, we recommend the following:

• If you use WireGuard encryption configure MTU size as “physical network MTU size minus 60”.
• If you don’t use WireGuard, but use VXLAN anywhere in your pod network, configure MTU size as “physical network MTU size minus 50”.
• If you don’t use WireGuard, but use only IP in IP, configure MTU size as “physical network MTU size minus 20”
• Set the workload endpoint MTU and the tunnel MTUs to the same value (so all paths have the same MTU)

eBPF mode

Implementation of NodePorts uses VXLAN tunnel to hand off packets from one node to another, therefore VXLAN MTU setting is used to set the MTUs of workloads (veths) and should be “physical network MTU size minus 50” (see above).

MTU for flannel networking

When using flannel for networking, the MTU for network interfaces should match the MTU of the flannel interface.

• If using flannel with WireGuard encryption, use the “Calico MTU with WireGuard” column in the table above for common sizes.
• Otherwise, if using flannel with VXLAN, use the “Calico MTU with VXLAN” column in the table above for common sizes.

Configure MTU

Note: The updated MTU used by Calico only applies to new workloads.

Select the appropriate instructions for configuring the MTU. This is broken down based on installation:

• Manifest based installation (if you are not using the quickstart guide, most non-OpenShift installs fall under this category)
• Operator

Manifest

For manifest based installations (i.e. ones that do not use the operator) edit the calico-config ConfigMap. For example:

kubectl patch configmap/calico-config -n kube-system --type merge \
  -p '{"data":{"veth_mtu": "1440"}}'

After updating the ConfigMap, perform a rolling restart of all calico/node pods. For example:

kubectl rollout restart daemonset calico-node -n kube-system

Operator

For Operator installations, edit the Calico operator Installation resource to set the mtu field in the calicoNetwork section of the spec. For example:

kubectl patch installation.operator.tigera.io default --type merge -p '{"spec":{"calicoNetwork":{"mtu":1440}}}'

Similarly, for OpenShift:

oc patch installation.operator.tigera.io default --type merge -p '{"spec":{"calicoNetwork":{"mtu":1440}}}'

View current tunnel MTU values

To view the current tunnel size, use the following command:

ip link show

The IP in IP tunnel appears as tunlx (for example, tunl0), along with the MTU size. For example: