Kubernetes 1.23 is a major release that includes a number of new features and improvements. Here are some of the most notable changes:

  1. A new container-based management system called Kubernetes-CNI (Container Network Interface) allows for easy deployment and management of Kubernetes clusters on large scale systems. This system replaces the previous CNI system used by Kubernetes 1.20 and earlier.
  2. A new user namespace feature allows users to create custom user namespaces for their Kubernetes clusters, which can be used to isolate specific parts of a cluster from other parts of the same cluster or from other systems on the network. This feature is useful for developing or testing applications that need to be run in isolation from other systems, or for running applications that need access to resources that are not available to other systems in the cluster.
  3. A number of bug fixes and improvements have been made in Kubernetes 1.23, including: improved performance when managing large numbers of containers, better support for container images with multiple versions, more reliable support for networking issues with large numbers of containers, and more accurate detection of errors in container logs.

Dual-Stack IPv4/IPv6 Networking

Dual-stack IPv4/IPv6 networking is generally available in v1.23. The feature makes it possible to assign a Pod or Service both IPv4 and IPv6 addresses. You need dual-stack network interfaces on your nodes and a supporting CNI networking plugin for this to work.

The .spec.ipFamilyPolicy field defines whether a Pod or Service receives a single- or dual-stack interface. Set it to PreferDualStack or RequireDualStack to activate dual-stack support. The default is SingleStack.

Dual-stack modes will allocate service cluster IPs from both the IPv4 and IPv6 address spaces. You can set the stack preference order with the .spec.ipFamilies field. This also permits specifying either IPv4 or IPv6 as the family for the SingleStack mode.

Ephemeral Volumes

Ephemeral volumes are another graduation to GA status. An ephemeral volume is tied to a Pod and gets deleted when the Pod terminates. It works with all existing storage drivers that support dynamic provisioning.

Ephemeral volumes are created by nesting a volumeClaimTemplate under a new ephemeral field in the volumes section of a Pod spec. The volumeClaimTemplate should resemble a regular PersistentVolumeClaim.

While an “ephemeral” volume may initially sound strange, there are several use cases for this functionality. Volumes are often used to provide a Pod’s process with first-run config values that are only accessed once. In this scenario, an ephemeral Pod is ideal as it’ll be deleted when the Pod stops, instead of being reattached to future Pods that’ll never use the data. Another possible case is processes which cache large amounts of data but don’t need it to be persisted between individual Pod terminations.

Horizontal Pod Autoscaler v2

After five years, v2 of the Horizontal Pod Autoscaler API has reached stable. Autoscaling allows Kubernetes to automatically adjust the replica count of your Deployments, ReplicaSets, and StatefulSets to respond to realtime metrics changes.

This promotion does not currently affect the original autoscaler implementation. The v1 API remains usable and is not being deprecated. The new autoscaling/v2 API takes over from the autoscaling/v2beta2 of previous Kubernetes releases.

Using the v2 API is advantageous as you can define autoscaling decisions that are based on custom metrics. The autoscaler controller will make arbitrary API queries to inform replica changes, instead of limiting you to node CPU and memory conditions.

Skipping Volume Ownership Changes On Pod Start

Using the fsGroup field on a volume to define its ownership currently causes Kubernetes to recursively execute chown() and chmod() on the volume’s contents each time it’s mounted to a Pod. This can be a significant performance issue when working with large volumes consisting mainly of small files. The Pod won’t start until the permissions have been changed.

Kubernetes supports an fsGroupChangePolicy field that lets you override this behavior. It’s now generally available via a Pod’s securityContextfield. Setting the change policy to OnRootMismatch will only call chown() if the root of the volume has incorrect permissions. This accelerates pod startup when permissions are already compatible with the fsGroup declaration.

Other Highlights

There are some noteworthy additions to the alpha and beta APIs. The beta channel now includes:

Structured logging – More components support the structured text logging format which produces JSON output. Structured logs are more readily parseable by external tools, facilitating simplified log ingest and query processes. PodSecurity API – PodSecurity replaces the older PodSecurityPolicy admission controller that lets you enforce security rules at the namespace level. PodSecurity provides a mechanism for stipulating that Pods can’t exist in a namespace if they lack certain security context protections. CSI Migration support – This feature provides a seamless way to move from an in-tree storage driver that’s part of the Kubernetes API, such as kubernetes. io/aws-ebs, to a vendored CSI driver. Users shouldn’t notice any changes to their storage after the migration completes. This functionality is now beta for AWS EBS, Azure Disk, and GCE PD. It’s labelled alpha for Ceph RBD and Portworx.

Over in the alpha channel, there are some additional capabilities making their debut:

Server side field validation – Server side field validation sends warnings from the server when a client tries to create resources that include unknown or duplicated fields. Kubernetes has historically dropped these fields, potentially causing confusing behavior. Enabling the ServerSideFeatureValidation feature gate provides a way to address this. Expression language validation for CRDs – A new inline expression language facilitates validation of custom resource definitions (CRDs). This helps to address the abstract nature of CRDs where user-defined resources can not currently be guaranteed to respect the requirements of existing controllers and client applications. OpenAPI v3 support – OpenAPI v3 has been added behind a feature gate (OpenAPIV3). When enabled you can request the OpenAPI v3. 0 specification for any of the Kubernetes object types, providing a way to programmatically discover and traverse resources via open API standards.

Conclusion

Kubernetes v1.23 stabilizes several significant features including dual-stack networking, the new customizable Horizontal Pod Autoscaler, and ephemeral volumes for working with non-critical data. There are dozens of other changes too, alongside two deprecations: the FlexVolume storage driver interface that predates CSI, and several logging flags which will be removed in the future.

A total of 11 existing features have been promoted to GA status. 17 further capabilities are now marked as beta, while another 19 are all-new capabilities landing in alpha. The Kubernetes release strategy focuses on shipping functionality early, allows the community to guide its development and help streamline API implementations. While alpha or beta features will usually push through to stable, this isn’t guaranteed. APIs can change significantly or get removed entirely during the development process.

You can read the full v1.23 release notes on GitHub. Downloads are available on the releases page. You can upgrade an existing cluster that was created using Kubeadm by following the guidance in the docs. The process will differ for users of managed cloud offerings and third-party Kubernetes distributions.