StorageClasses compared

A StorageClass is just a name for “how to make a volume”: a provisioner plus parameters. The real decision is what sits behind it — a directory on one node, a replicated block device, a Ceph cluster, a NAS, or a Proxmox virtual disk. That choice decides what happens on the worst day: when the node holding your data disappears.

simulator

Kill a node, see who keeps their data

The pod db-0 runs on talos-w1 with a 10 GiB volume. Pick a StorageClass, then pull the plug on w1.

Cluster · Longhorn

numberOfReplicas: 3 — the engine runs next to the pod on w1, replicas on all three nodes.

talos-w1

Ready

db-0

Running

▤ engine+replica

talos-w2

Ready

▤ replica

talos-w3

Ready

▤ replica

pod

Running

pod on

data

healthy

What Kubernetes does

The pod runs on talos-w1. Pull the plug and watch the control plane react. Toggle StatefulSet / Deployment — they behave differently while the node is merely unreachable.

Replication doesn't make failover instant

Even with Longhorn or Ceph, Kubernetes deliberately refuses to run a second copy of an RWO/StatefulSet pod while the old node might still be writing. Plan for the ~5 minute eviction window, and learn the node.kubernetes.io/out-of-service taint for when you know a node is gone.

chooser

Pick by requirements

Hard requirements remove options; preferences and what you already run reorder them. There is no universally best answer — mixing two classes (fast local for self-replicating databases, replicated for everything else) is normal.

Your requirements

must have

Data must survive losing a nodeReadWriteMany (shared filesystem)CSI VolumeSnapshotsS3-compatible object storage too

prefer

Latency-sensitive (databases)Small team — minimal ops

you already have

Talos runs as VMs on ProxmoxProxmox cluster already runs CephA NAS (TrueNAS / NFS server)

Ranked fit

4 candidates

#1Longhorndriver.longhorn.io
- · replicated: Synchronous replicas per volume (numberOfReplicas)
#2OpenEBS Replicated PV Mayastorio.openebs.csi-mayastor
- · replicated: Synchronous N replicas (repl: "3")
- ⚠ ops-heavy: Hugepages, dedicated cores, disk pools per node
#3Piraeus / LINSTOR (DRBD)linstor.csi.linbit.com
- · replicated: DRBD synchronous replicas (placementCount) + diskless tiebreaker
- ⚠ ops-heavy: Kernel module, satellites, quorum concepts
#4Rook-Cephrook-ceph.rbd.csi.ceph.com
- · replicated: CRUSH-placed replicas (size 3) or erasure coding
- ⚠ ops-heavy: Highest: MONs, OSDs, MGRs, PG health, upgrades

Ruled out (6)

local-path — single copy on one node
OpenEBS Local — single copy on one node
Ceph (Proxmox) — needs an existing Ceph cluster (e.g. Proxmox's)
Proxmox CSI — on local Proxmox storage the volume is pinned to one host
NFS — needs a NAS to point at
democratic-csi — needs a NAS to point at

compare

The full matrix

Performance and operational load are relative, 1–5. “Talos needs” is what you must add to an otherwise stock Talos image or machine config.

	Replication / HA	RWX	Snapshots	Perf	Ops load	Node failure	Talos needs
local-pathrancher.io/local-path	✗None — one copy on one node	✗No (RWO, effectively one node)	✗No	●●●●●Raw node disk, no network hop	●○○○○One Deployment + ConfigMap	PV has nodeAffinity → pod is stuck Pending until that node comes back	kubelet extraMounts for /var/local-path-provisioner, or point it at a user volume under /var/mnt (already visible to kubelet) — /var is the EPHEMERAL partition
OpenEBS Localopenebs.io/local	✗None (rely on app-level replication)	✗No	~LVM / ZFS flavours only	●●●●●Local disk speed	●●○○○Small; LVM/ZFS need a VG / pool prepared per node	Pinned to the node — stuck until it returns	hostpath: extraMounts for /var/openebs/local. ZFS: siderolabs/zfs extension. LVM: a VG on a spare disk
Mayastorio.openebs.csi-mayastor	✓Synchronous N replicas (repl: "3")	✗No (block / RWO)	✓Yes	●●●●●NVMe-oF, polling cores — very fast, but burns dedicated CPU	●●●●○Hugepages, dedicated cores, disk pools per node	Pod reschedules; a surviving replica keeps serving	2 MiB hugepages (vm.nr_hugepages sysctl), nvme-tcp kernel support, node label openebs.io/engine=mayastor, privileged namespace
Longhorndriver.longhorn.io	✓Synchronous replicas per volume (numberOfReplicas)	✓Yes — via an NFSv4 share-manager pod	✓Yes + built-in backups to S3/NFS	●●●○○Good; replication + iSCSI hop costs latency	●●●○○Nice UI; watch disk space & rebuild traffic	Pod reschedules and attaches a surviving replica; volume runs degraded until rebuilt	siderolabs/iscsi-tools + siderolabs/util-linux-tools extensions, kubelet extraMounts (rshared) for the data path — /var/lib/longhorn, or /var/mnt/longhorn on a UserVolume (Talos 1.10+) — privileged namespace
Rook-Cephrook-ceph.rbd.csi.ceph.com	✓CRUSH-placed replicas (size 3) or erasure coding	✓Yes — CephFS (rook-ceph.cephfs.csi.ceph.com)	✓Yes, plus clones	●●●●○Scales out well; needs fast network (10G+) and RAM	●●●●●Highest: MONs, OSDs, MGRs, PG health, upgrades	Pod reschedules anywhere; Ceph serves from the other replicas and recovers	Dedicated raw disks/partitions for OSDs (no extension needed — rbd/ceph kernel modules ship with Talos), privileged namespace, ≥3 nodes
Ceph (Proxmox)rbd.csi.ceph.com	✓Ceph replicas on the Proxmox hosts	✓Yes if CephFS (an MDS) is set up on Proxmox	✓Yes	●●●●○Same as your Proxmox Ceph; Talos VMs need a route to the Ceph public network	●●○○○Kubernetes side is just the CSI driver; Ceph is run by Proxmox	Pod reschedules anywhere; data lives outside the k8s cluster	No extension. Network reachability to the Ceph MONs, a Ceph client key/user per cluster
Proxmox CSIcsi.proxmox.sinextra.dev	~Whatever the Proxmox storage does (none for local-lvm, Ceph for RBD)	✗No (a disk attaches to one VM)	~Experimental: full-copy snapshots, needs root@pam — Proxmox backups are the safer bet	●●●●○Native virtio disk; as fast as the backing storage	●○○○○Low: a Proxmox API token + topology labels	Local storage: stuck in that host's zone. Shared storage: disk can be attached to a VM on another host	No extension. Nodes need topology.kubernetes.io/region & zone labels (Proxmox/Talos CCM)
NFSnfs.csi.k8s.io	~None in k8s — the NAS is a single point of failure unless it is HA	✓Yes — the classic RWX answer	~csi-driver-nfs: tar-archive snapshots (v4.3+); subdir: none	●●○○○Network FS; poor for databases (fsync, locking)	●○○○○Trivial if a NAS already exists	Pod reschedules anywhere; NAS outage = all volumes down	No extension — the kubelet does the NFS mount
democratic-csiorg.democratic-csi.nfs	~ZFS RAID inside the NAS; NAS itself is one box	✓Yes with the NFS driver; RWO with iSCSI/NVMe-oF	✓Yes — real ZFS snapshots	●●●○○Decent; iSCSI zvols beat NFS for databases	●●○○○One Helm release per driver mode; NAS API key	Pod reschedules anywhere; NAS outage = all volumes down	NFS mode: nothing. iSCSI mode: siderolabs/iscsi-tools extension
LINSTORlinstor.csi.linbit.com	✓DRBD synchronous replicas (placementCount) + diskless tiebreaker	✗No (block / RWO)	✓Yes with LVM-thin / ZFS pools	●●●●●Near-local: reads can be served by the local replica	●●●●○Kernel module, satellites, quorum concepts	Pod reschedules; DRBD promotes a surviving replica	siderolabs/drbd extension + load drbd (usermode_helper=disabled) and drbd_transport_tcp modules

On Proxmox, consider storage below the VMs

If the hypervisor already has good storage (Ceph, ZFS, shared LVM), the cheapest HA story is often to use it — via Proxmox CSI or ceph-csi — instead of running a second replicated storage system inside the VMs, which replicates data that Proxmox already replicates.

Extensions are part of the image

Talos has no package manager. iscsi-tools, util-linux-tools, drbd or zfs must be in the installer image (Image Factory schematic), and adding one later means a talosctl upgrade to the new image on every node.

yaml

What the manifests look like

The provisioner string must match the CSI driver's registered name exactly — it is how Kubernetes finds the driver.

StorageClass manifests

WaitForFirstConsumer is mandatory for local volumes: the scheduler must pick the node first, then the directory is created there. On Talos, /var is the EPHEMERAL partition, so data survives reboots but is wiped by a reset — consider a dedicated user volume for anything you care about.

storageclass.yaml

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-path
provisioner: rancher.io/local-path
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

talos patch

# Talos machine-config patch: give the provisioner a bind mount the kubelet can see
machine:
  kubelet:
    extraMounts:
      - destination: /var/local-path-provisioner
        type: bind
        source: /var/local-path-provisioner
        options: [bind, rshared, rw]

Set a default

PVCs without storageClassName get the class annotated as default. Have exactly one.

kubectl get storageclass
kubectl patch storageclass longhorn-3r \
  -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
kubectl get pv -o custom-columns=NAME:.metadata.name,CLASS:.spec.storageClassName,AFFINITY:.spec.nodeAffinity.required

why it mattersLocal volumes (local-path, OpenEBS Local, Proxmox CSI on local-lvm) are fastest and simplest, but the PV is pinned: when the node dies, the pod waits. Use them for apps that replicate themselves.
why it mattersReplicated storage (Longhorn, Rook-Ceph, Mayastor, LINSTOR) lets the pod move — after Kubernetes is sure the old node let go. Budget CPU, RAM, network and your own time for it.
why it mattersOn Talos every requirement is declarative: extensions in the image schematic, kubelet extraMounts and kernel modules in the machine config. Decide the storage stack before you build the image.