Proxmox CSI

The Proxmox CSI plugin (csi.proxmox.sinextra.dev) creates each PersistentVolume as an ordinary Proxmox virtual disk. The disk is allocated on a Proxmox storage pool and hot-plugged into the worker VM as an extra SCSI drive. No storage daemon runs inside Kubernetes. The data sits on the hypervisor, where Proxmox already manages it, and the plugin's job is to put it next to the right VM at the right time.

architecture

Two halves, two levels of trust

The controller (a Deployment, usually on control planes) holds the Proxmox API token. It runs next to the standard sidecars: csi-provisioner, csi-attacher, csi-resizer, and optionally csi-snapshotter. It creates disks and attaches them to VMs. The node plugin (a DaemonSet in every worker VM) never talks to Proxmox. It only finds the new block device, formats it, and mounts it, which needs privileged access to the host. While a disk is unattached it is owned by a placeholder VM ID, 9999 (features.controllerVMID), which is why volumes are named vm-9999-pvc-<uuid>.

CreateVolume → ControllerPublish → NodeStage → NodePublish

Demo 1: provisioning, end to end

The StorageClass uses WaitForFirstConsumer, so nothing is created until the scheduler has picked a node. The disk then has to be allocated on that node's Proxmox host, because local LVM on pve1 is useless to a VM on pve2. Watch the VM config gain a scsi1 line. The wwn the controller sets is how the node plugin finds the right /dev/disk/by-id entry. Switch to shared storage to see the PV's topology shrink to region only.

From PVC to a SCSI disk inside the VM

0/5

Proxmox cluster-1pod pg-0Pending

pve1

111 worker-1

local-lvm

pve2

112 worker-2

local-lvm

pve3

113 worker-3

local-lvm

pvc data-pg-0 Pending: waiting for first consumer to be created before binding

qm config 112 # worker-2 on pve2

boot: order=scsi0
cores: 4
memory: 8192
name: worker-2
net0: virtio=BC:24:11:3A:7E:02,bridge=vmbr0
scsi0: local-lvm:vm-112-disk-0,discard=on,size=32G,ssd=1
scsihw: virtio-scsi-pci
smbios1: uuid=9d2e7b13-08aa-4c4e-b6f1-6a3f0c8e2b42

inside the worker VM (node plugin's view)

$ ls /dev/disk/by-id/ | grep wwn
(nothing yet)
$ findmnt -S /dev/sdb

kubectl get pvc data-pg-0 -o yaml

kind: PersistentVolumeClaim
metadata:
  name: data-pg-0
spec:
  storageClassName: proxmox-data
  resources: { requests: { storage: 10Gi } }
status: { phase: Pending }

kubectl get pv -o yaml

# no PV yet — nothing is provisioned until a pod
# using the claim is scheduled

topology in practice

Demo 2: what if the pod moves?

Drain the node and send pg-0 somewhere else. Within one Proxmox host the plugin detaches and re-attaches the disk for you. Across hosts, what happens depends on whether the storage is shared.

What if the pod moves?

pve1

(other workloads)

local-lvm

pve2

112 worker-2a

pg-0scsi1

122 worker-2b

local-lvm

pvc-6f1c…

pve3

113 worker-3a

local-lvm

Reschedule pg-0 to:

waiting…

# PV nodeAffinity

topology.kubernetes.io/region In [cluster-1]

topology.kubernetes.io/zone In [pve2]

pod:Running on worker-2a

Local storage pins the pod to a Proxmox host

A disk on lvmthin, zfspool or a local dir only exists on one host. It can be moved to another VM on that host (detach, then attach), and the CSI plugin does this for you. Moving it to another host is an offline copy: pvecsictl migrate moves the disk and rewrites the PV's zone. On ZFS you can instead set replicate: "true" with replicateZones (up to two zones) to keep a ZFS replica ready on another host.

feature status (per the project README & docs)

What's supported

	Status	Notes
Dynamic provisioning + topology	yes	Needs topology.kubernetes.io/region on every node; zone is required for non-shared storage. Proxmox CCM sets both.
Volume expansion	yes	allowVolumeExpansion: true. Grows the Proxmox disk on the attached VM, then the node plugin grows the filesystem.
Storage capacity	yes	Publishes CSIStorageCapacity per region/zone so the scheduler avoids full pools (disable with options.enableCapacity: false).
Access modes	RWO only	Single-node modes only: a virtual disk is attached to one VM at a time.
Encryption	yes	LUKS via csi.storage.k8s.io/node-stage-secret-* (secret key encryption-passphrase).
IOPS / bandwidth limits	yes	diskIOPS / diskMBps, also adjustable later through a VolumeAttributesClass.
Snapshots & clones	experimental	Full copies, not deltas. Requires root@pam credentials and the snapshotter sidecar (controller.snapshotter.enabled).
Cross-host migration	offline, manual	pvecsictl migrate for local storage; automatic for shared storage.
ZFS zone replication	yes	replicate / replicateSchedule / replicateZones (ZFS replication; needs the extended role).

docs/options.md

StorageClass parameters

Everything under parameters: that the plugin understands. Unknown keys are not part of the documented surface, so don't guess.

	Values	Meaning
`storage`	Proxmox storage ID	Which pool to allocate from (e.g. local-lvm, local-zfs, ceph-rbd). Required.
`storageFormat`	raw \| qcow2	Disk format, where the storage type allows a choice.
`csi.storage.k8s.io/fstype`	ext4 (default) \| xfs	Filesystem created by NodeStage on first use.
`blockSize / inodeSize`	"4096" / "256"	mkfs tuning.
`cache`	directsync \| none \| writeback \| writethrough	QEMU cache mode of the attached drive.
`ssd`	"true" \| "false"	SSD emulation, and turns discard on.
`discard`	on \| ignore	TRIM/UNMAP pass-through, for when ssd would be inaccurate.
`diskIOPS / diskMBps`	"4000" / "1000"	Read+write limits on the QEMU drive.
`replicate, replicateSchedule, replicateZones`	"true", "*/15", "pve-1,pve-3"	ZFS replication to other zones.
`backup`	"true" \| "false"	Include the disk in VM backups. The docs call it dangerous unless you understand restore.
`csi.storage.k8s.io/node-stage-secret-*`	name / namespace	Enables LUKS encryption (also set node-expand-secret-* to allow resizing).

The SCSI controller matters

Volumes are attached as scsiN disks, so worker VMs need a VirtIO SCSI controller, and QEMU caps a VM at about 30 SCSI disks (the plugin defaults to 24 attachments per node). The CSI README suggests VirtIO SCSI single. Talos's Proxmox guide says to use plain VirtIO SCSI instead, because single can hang bootstrap, and the CSI FAQ fixes "device /dev/disk/by-id/wwn-… is not found" by switching to virtio-scsi-pci. On Talos, use virtio-scsi-pci.

config

Install on Talos

Proxmox: role + user + token (docs/install.md)

pveum role add CSI -privs "VM.Audit VM.Config.Disk Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"
pveum user add kubernetes-csi@pve
pveum aclmod / -user kubernetes-csi@pve -role CSI
pveum user token add kubernetes-csi@pve csi -privsep 1
pveum aclmod / -tokens 'kubernetes-csi@pve!csi' -role CSI
# ZFS replication needs a broader role: see docs/install.md

Namespace must allow privileged pods (Talos enforces Pod Security)

kubectl create ns csi-proxmox
kubectl label ns csi-proxmox pod-security.kubernetes.io/enforce=privileged

proxmox-csi.yaml (helm values)

config:
  clusters:
    - url: https://pve1.lan:8006/api2/json
      insecure: false
      token_id: "kubernetes-csi@pve!csi"
      token_secret: "<secret>"
      region: cluster-1          # must equal the CCM region label

storageClass:
  - name: proxmox-data
    storage: local-lvm
    reclaimPolicy: Delete
    fstype: xfs
    ssd: true
    annotations:
      storageclass.kubernetes.io/is-default-class: "true"

# controller next to the control plane
nodeSelector:
  node-role.kubernetes.io/control-plane: ""
tolerations:
  - key: node-role.kubernetes.io/control-plane
    effect: NoSchedule

Install

helm upgrade -i -n csi-proxmox -f proxmox-csi.yaml \
  proxmox-csi-plugin oci://ghcr.io/sergelogvinov/charts/proxmox-csi-plugin

kubectl get csidriver csi.proxmox.sinextra.dev
kubectl get csistoragecapacities -A

The StorageClass it renders (or write it yourself)

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: proxmox-data
provisioner: csi.proxmox.sinextra.dev
parameters:
  storage: local-lvm
  csi.storage.k8s.io/fstype: xfs
  ssd: "true"
  # cache: none
  # diskIOPS: "4000"
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Test it

apiVersion: v1
kind: PersistentVolumeClaim
metadata: { name: data-pg-0 }
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: proxmox-data
  resources: { requests: { storage: 10Gi } }
---
apiVersion: v1
kind: Pod
metadata: { name: pg-0 }
spec:
  containers:
    - name: app
      image: busybox
      command: ["sh", "-c", "df -h /data && sleep 3600"]
      volumeMounts: [{ name: data, mountPath: /data }]
  volumes:
    - name: data
      persistentVolumeClaim: { claimName: data-pg-0 }

Grow it, move it

# online expansion: Proxmox disk resize + filesystem grow
kubectl patch pvc data-pg-0 -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'

# move a local volume to another Proxmox host (offline, needs root@pam)
brew install sergelogvinov/tap/pvecsictl
pvecsictl migrate --config=cloud-config.yaml -n default data-pg-0 pve3 --force

# find orphaned vm-9999-pvc-* disks with no matching PV
pvecsictl clean --config=cloud-config.yaml local-lvm pve2

Using Terraform for the VMs?

Add lifecycle { ignore_changes = [disk] } to the VM resource. Otherwise the next terraform apply sees the CSI-attached scsiN disks as drift and detaches them.

why it mattersA PV is a Proxmox disk. Snapshots, backup and replication can stay at the hypervisor, and no storage daemon competes with your pods for RAM.
why it mattersWaitForFirstConsumer plus zone labels from the CCM put each disk on the same host as the VM that will use it. Without the labels, local storage can't work.
why it mattersLocal storage is fast but pins a pod to one host, so plan replication at the app level (DB replicas spread by zone). Shared storage moves freely within the region.