Proxmox CCM

A Kubernetes Node object starts out knowing nothing about the machine underneath it. It doesn't know which Proxmox VM it is, which physical host that VM runs on, or whether the VM still exists. The Proxmox cloud-controller-manager fills that in. It matches each Node to its VM, writes a providerID and topology labels, and deletes the Node when the VM is destroyed.

background

What a cloud-controller-manager is

Kubernetes used to ship AWS, GCE, OpenStack and other cloud code inside kube-controller-manager and the kubelet. That code was moved out into a separate binary, the cloud-controller-manager, built on the k8s.io/cloud-provider framework. A provider implements a small Go interface, and the framework runs the controllers that use it. The Proxmox CCM implements only InstancesV2, so only the two node controllers have anything to do:

	What it does	Proxmox CCM
cloud-node	Initializes new Nodes: providerID, addresses, instance-type and zone/region labels, then removes the uninitialized taint	implemented
cloud-node-lifecycle	Watches NotReady Nodes: deletes them when the instance is gone, taints them when it is shut down	implemented
route	Programs cloud routes for pod CIDRs	not implemented — your CNI (Cilium) handles pod routing
service	Creates cloud load balancers for type: LoadBalancer	not implemented — use Cilium LB-IPAM / L2, MetalLB or kube-vip

cloud-node controller

Demo 1: a VM joins the cluster

Pick a worker and play the sequence. With --cloud-provider=external the kubelet registers the Node with a NoSchedule taint, which keeps ordinary pods off it until the CCM has filled in the details. Turn the flag off to see what happens when that handshake is missing.

Node initialization, step by step

--cloud-provider=external

0/5

Proxmox cluster cluster-1= region

pve1

101 cp-1

111 worker-1

pve2

102 cp-2

112 worker-2

pve3

103 cp-3

113 worker-3

host name = zone

proxmox-cloud-controller-manager

controllers: cloud-node, cloud-node-lifecycle

kube-apiserver

kubelet POST /api/v1/nodes worker-3 (--cloud-provider=external → adds uninitialized taint)

pod web-7d9cPending: untolerated taint uninitialized

kubectl get node worker-3 -o yaml

apiVersion: v1
kind: Node
metadata:
  name: worker-3
  annotations:
    alpha.kubernetes.io/provided-node-ip: 10.0.0.23
  labels:
    kubernetes.io/hostname: worker-3
    kubernetes.io/os: linux
spec:
  taints:
  - key: node.cloudprovider.kubernetes.io/uninitialized
    value: "true"
    effect: NoSchedule
status:
  addresses:
  - type: Hostname
    address: worker-3
  nodeInfo:
    systemUUID: e1b85c47-2f6d-4d0a-8c3e-7f9a4b6d5c13

How the VM is found

On first sight the Node has no providerID, so the CCM lists VMs in every configured Proxmox cluster and keeps the one whose name starts with the node name and whose SMBIOS UUID equals status.nodeInfo.systemUUID. Once written, providerID is immutable and every later lookup goes straight to region/vmid. The instance type is synthesized from the VM's cores and memory.

cloud-node-lifecycle controller

Demo 2: the VM goes away

The lifecycle controller only checks Nodes that are no longer Ready, and asks the CCM two questions. Does the instance still exist? Is it shut down? Try a graceful shutdown, a destroy, and a whole Proxmox host going dark.

cloud-node-lifecycle: what happens when the VM goes away

pve1

111 worker-1running

pve2

112 worker-2running

pve3

113 worker-3running

waiting…

kubectl get nodes

NAME	STATUS	TAINTS
worker-1	Ready	<none>
worker-2	Ready	<none>
worker-3	Ready	<none>

Shutdown keeps the Node and adds a shutdown taint — the VM may come back, and its identity (providerID) is still valid.
Destroy makes InstanceExists false, so the Node object is deleted. This is the reason the project exists: scaling down with Terraform no longer leaves ghost nodes.
Host down is ambiguous. The CCM treats an unreachable Proxmox node as "can't tell" and keeps the Node rather than deleting it on a network blip.

region & zone

Demo 3: why topology labels are worth the trouble

The CCM maps topology.kubernetes.io/region to the Proxmox cluster (the region key in its config) and topology.kubernetes.io/zone to the Proxmox host the VM runs on. Several things read those labels: topology spread constraints and anti-affinity, so HA replicas don't share a power supply; the Proxmox CSI plugin, so a volume on a host's local LVM is only attached to VMs on that host; and anything else that reasons about failure domains.

Why the zone label matters: spread replicas across Proxmox hosts

pve1zone=pve1

worker-1a

pgpg

worker-1b

pve2zone=pve2

worker-2a

worker-2b

pve3zone=pve3

worker-3a

worker-3b

replicas up

3/3

survives any 1 host?

# no topologySpreadConstraints
# (only soft scheduler defaults: packing is allowed)

Nothing stops all replicas landing on VMs that share one physical box.

Labels are set once

The cloud-provider framework writes labels at initialization and doesn't update them afterwards. If you live-migrate a VM to another Proxmox host, its zone label is now wrong, and the project's FAQ answers "does CCM support online VM migration?" with "No". Treat VMs as fixed to a host. Drain and recreate them instead of migrating. CPU type host, which Talos recommends, also rules out live migration between different CPUs.

config

Setting it up

You need four things: a Proxmox API token with read-only VM access, a config file listing your Proxmox cluster(s), the CCM itself, and kubelets running with --cloud-provider=external.

On a Proxmox host: role, user, token (from the project's docs/install.md)

# Read-only: audit VMs, ask the guest agent, read cluster status
pveum role add CCM -privs "VM.Audit VM.GuestAgent.Audit Sys.Audit"
pveum user add kubernetes@pve
pveum aclmod / -user kubernetes@pve -role CCM
pveum user token add kubernetes@pve ccm -privsep 0
# → prints the token secret once; token_id is kubernetes@pve!ccm

proxmox-ccm.yaml (helm values)

config:
  clusters:
    - url: https://pve1.lan:8006/api2/json
      insecure: false          # true only for self-signed certs in a lab
      token_id: "kubernetes@pve!ccm"
      token_secret: "<secret>"
      region: cluster-1        # becomes topology.kubernetes.io/region
    # more Proxmox clusters → more regions, one Kubernetes cluster

# enabledControllers defaults to cloud-node + cloud-node-lifecycle

Install (OCI helm chart)

helm upgrade -i --namespace=kube-system -f proxmox-ccm.yaml \
  proxmox-cloud-controller-manager \
  oci://ghcr.io/sergelogvinov/charts/proxmox-cloud-controller-manager

# DaemonSet mode uses hostNetwork, so it can run before a CNI exists:
#   --set useDaemonSet=true  (pin it to control planes with nodeSelector)

Talos machine config patch (all nodes)

cluster:
  externalCloudProvider:
    enabled: true   # Talos then runs kubelet with --cloud-provider=external
machine:
  kubelet:
    extraArgs:
      cloud-provider: external   # redundant with the above, but explicit

Or ship it with the machine config (from the CSI docs)

cluster:
  inlineManifests:
    - name: proxmox-cloud-controller-manager
      contents: |-
        apiVersion: v1
        kind: Secret
        type: Opaque
        metadata:
          name: proxmox-cloud-controller-manager
          namespace: kube-system
        stringData:
          config.yaml: |
            clusters:
              - url: https://pve1.lan:8006/api2/json
                insecure: false
                token_id: "kubernetes@pve!ccm"
                token_secret: "<secret>"
                region: cluster-1
  externalCloudProvider:
    enabled: true
    manifests:
      - https://raw.githubusercontent.com/sergelogvinov/proxmox-cloud-controller-manager/main/docs/deploy/cloud-controller-manager.yml

Verify

kubectl get nodes -L topology.kubernetes.io/region,topology.kubernetes.io/zone,node.kubernetes.io/instance-type
kubectl get node worker-3 -o jsonpath='{.spec.providerID}'   # proxmox://cluster-1/113
kubectl describe node worker-3 | grep Taints                 # no uninitialized taint

Things that trip people up

The VM name must start with the Kubernetes node name, and the SMBIOS UUID must match. If you rename one side, the match fails.
The flag has to be present when the kubelet first registers. If you add it later, delete the Node object and restart the kubelet.
By default addresses come from the kubelet's --node-ip. Set features.network.mode: qemu or auto to read them from the QEMU guest agent instead, which needs the qemu-guest-agent Talos extension.

why it mattersThe CCM gives each Node a durable identity, proxmox://region/vmid, so Kubernetes and Proxmox agree on which VM is which.
why it matterszone = Proxmox host. That one label is what lets the scheduler keep HA replicas off the same physical box, and what lets the CSI driver keep local disks next to their VM.
why it mattersWhen you terraform destroy a VM, its Node is cleaned up automatically. A stopped VM keeps its Node with a shutdown taint, and an unreachable host isn't treated as proof that the VM is gone.