Kubernetes PVC Stuck in Pending on WSL2 Ubuntu: Comprehensive Troubleshooting Guide

Resolve Kubernetes Persistent Volume Claim (PVC) issues stuck in Pending status on Windows WSL2 Ubuntu. Diagnose and fix common storage class, provisioner, and PV configuration problems.

Introduction

As an experienced Systems Administrator and DevOps engineer, I've seen countless Kubernetes deployments, and one of the most common hurdles for stateful applications is persistent storage. When running Kubernetes on Windows Subsystem for Linux 2 (WSL2) with an Ubuntu distribution, a PersistentVolumeClaim (PVC) getting stuck in a Pending state is a frequent roadblock for developers and administrators alike. This issue typically means that Kubernetes cannot find or provision a suitable PersistentVolume (PV) to fulfill the storage request made by your PVC.

This guide provides a highly technical, accurate, and step-by-step approach to diagnose and resolve PVCs stuck in Pending status specifically within the WSL2 Ubuntu environment. We'll cover common root causes ranging from misconfigured StorageClass definitions and absent provisioners to fundamental WSL2 filesystem considerations.

Symptom & Error Signature

When your PersistentVolumeClaim is stuck in Pending, your stateful applications (like databases, message queues, or persistent web services) will fail to start or remain in a ContainerCreating state because their required storage cannot be provisioned.

You will typically observe this status when listing your PVCs:

kubectl get pvc

Expected Output (Symptom):

NAME                STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS      AGE
my-app-data         Pending                                      local-path        5m
another-pvc         Pending                                      nfs-storage       2m

To get more detailed information about why it's pending, you'll need to inspect the PVC's events:

kubectl describe pvc my-app-data

Expected Output (Error Signature – Look for "Events" section):

Name:          my-app-data
Namespace:     default
StorageClass:  local-path
Status:        Pending
Volume:
Labels:        <none>
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
DataSource:    <none>
Events:
  Type     Reason              Age                From                         Message
  ----     ------              ----               ----                         -------
  Warning  ProvisioningFailed  15s (x3 over 2m)   local-path-provisioner       failed to provision volume with StorageClass "local-path": persistentvolumeclaims "my-app-data" not found
  Warning  ProvisioningFailed  10s (x5 over 2m)   persistentvolume-controller  Failed to provision volume with StorageClass "local-path": storageclass.storage.k8s.io "local-path" not found
  Warning  ProvisioningFailed  5s (x10 over 2m)   persistentvolume-controller  no persistent volume available for this claim and no storage class is set

The Events section is crucial. Messages like "no persistent volume available for this claim and no storage class is set", "Failed to provision volume with StorageClass … not found", or "failed to provision volume with StorageClass …: persistentvolumeclaims … not found" directly point to the underlying issues.

Root Cause Analysis

A PVC gets stuck in Pending primarily because Kubernetes cannot fulfill its storage request. This can be due to several reasons, especially in a WSL2 environment:

1. No Suitable Persistent Volume (PV) Available:

   • Static Provisioning: There are no pre-created PersistentVolume objects that match the PVC's requirements (e.g., accessModes, storageClassName, capacity, volumeMode).
   • Dynamic Provisioning Failure: The StorageClass specified by the PVC is either:
     • Not found: The StorageClass resource itself does not exist in the cluster.
     • Misconfigured: The StorageClass exists, but its provisioner is incorrect or its parameters are malformed.
     • Provisioner not running/installed: The external controller responsible for dynamically creating PVs (e.g., local-path-provisioner, nfs-subdir-external-provisioner) is not deployed, is crashed, or has insufficient permissions.

2. StorageClass Issues:

   • The StorageClass referenced in the PVC does not exist or has a typo.
   • If the PVC omits storageClassName, there might be no default StorageClass defined in the cluster.

3. Capacity and Access Mode Mismatches:

   • The PVC requests more storage than available in any existing static PVs, or more than the dynamic provisioner is configured to handle (e.g., disk quota limits).
   • The accessModes requested by the PVC (e.g., ReadWriteMany) cannot be satisfied by available PVs or the chosen provisioner (e.g., HostPath typically only supports ReadWriteOnce).

4. Provisioner Specific Issues (WSL2 Context):

   • local-path-provisioner (Common with K3s/MicroK8s):
     • The host path configured for the provisioner (e.g., /var/lib/rancher/k3s/storage for K3s) does not exist within the WSL2 Ubuntu filesystem or has incorrect permissions.
     • The provisioner pod itself is failing due to internal errors or misconfiguration.
   • NFS Provisioner:
     • The NFS server is not running, not accessible from WSL2, or the configured export path is incorrect.
     • NFS client tools are not installed in the WSL2 Ubuntu distro, or required kernel modules are missing.
     • Network connectivity issues between the Kubernetes node (WSL2 Ubuntu) and the NFS server.
   • General WSL2 Filesystem Limitations: While HostPath can technically point to /mnt/c/ (Windows drives), many production-grade provisioners or even local-path-provisioner might have issues with permissions, symlinks, or performance when directly operating on Windows filesystems mounted into WSL2. It's generally best practice to keep persistent storage paths within the WSL2 Ubuntu filesystem.

Step-by-Step Resolution

Follow these steps meticulously to diagnose and resolve your PVC Pending issue on WSL2.

1. Initial Diagnosis: Check PVC, PV, StorageClass, and Events

The first step is always to gather all relevant information.

# Get details of the problematic PVC
kubectl describe pvc my-app-data

# Check if any Persistent Volumes (PVs) exist
kubectl get pv

# Check if StorageClasses are defined
kubectl get sc

# Get all events related to the PVC
kubectl get events --field-selector involvedObject.name=my-app-data --sort-by=".metadata.creationTimestamp"

Carefully examine the Events section from kubectl describe pvc. This often gives the most direct hint. Look for messages indicating missing StorageClass, failed provisioning, or no matching PVs.

2. Verify StorageClass Configuration

A missing or misconfigured StorageClass is a prime suspect.

Scenario A: PVC specifies a storageClassName (e.g., local-path, nfs-storage)

Ensure the StorageClass actually exists and its configuration is correct.

kubectl get sc <your-storage-class-name> -o yaml

Example (for local-path used by K3s/MicroK8s):

# kubectl get sc local-path -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.kubernetes.io/is-default-class: "true" # May or may not be default
  name: local-path
provisioner: rancher.io/local-path
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

• Action: If the StorageClass does not exist, you must create it. If it exists, verify the provisioner field matches the expected provisioner. If the provisioner is incorrect or misspelled, you'll need to correct it (often by deleting and recreating the StorageClass).

Scenario B: PVC does NOT specify a storageClassName

In this case, Kubernetes attempts to use the default StorageClass.

kubectl get sc -o json | jq -r '.items[] | select(.metadata.annotations."storageclass.kubernetes.io/is-default-class" == "true") .metadata.name'

• Action: If this command returns nothing, there is no default StorageClass. You must either:
  • Define a default StorageClass by adding the annotation storageclass.kubernetes.io/is-default-class: "true" to an existing StorageClass (and ensure only one is marked as default).
  • Modify your PVC to explicitly specify an existing storageClassName.

3. Ensure Storage Provisioner is Running (for Dynamic Provisioning)

If your StorageClass uses dynamic provisioning, the associated provisioner controller must be running and healthy.

A. For local-path-provisioner (Common with K3s/MicroK8s on WSL2)

This provisioner relies on a path within the WSL2 Ubuntu filesystem to store PVs.

1. Check the provisioner pod status:

   kubectl get pods -n kube-system -l app=local-path-provisioner
   

   Ensure the pod is Running and healthy. If not, inspect its logs:

   kubectl logs -n kube-system -l app=local-path-provisioner
   kubectl describe pod -n kube-system -l app=local-path-provisioner
   

2. Verify the host path: The local-path-provisioner typically uses /var/lib/rancher/k3s/storage (for K3s) or similar. Ensure this directory exists and has correct permissions inside your WSL2 Ubuntu distribution.

   # From inside your WSL2 Ubuntu terminal
   sudo ls -ld /var/lib/rancher/k3s/storage # Or relevant path for MicroK8s
   

   The directory should typically be owned by root:root with permissions like drwxr-xr-x. If it doesn't exist, create it:

   sudo mkdir -p /var/lib/rancher/k3s/storage
   sudo chown root:root /var/lib/rancher/k3s/storage
   sudo chmod 755 /var/lib/rancher/k3s/storage
   

   Do NOT attempt to use paths directly on your Windows host filesystem (e.g., /mnt/c/some/path) for local-path-provisioner. While technically possible with HostPath for a static PV, it's generally problematic for dynamic provisioners and can lead to permission issues and performance degradation within WSL2. Stick to paths within the WSL2 filesystem.

B. For nfs-subdir-external-provisioner (or other NFS-based storage)

If you're using an NFS server, ensure it's accessible and correctly configured.

1. Check NFS provisioner pod status:

   kubectl get pods -n <your-nfs-namespace> -l app=nfs-subdir-external-provisioner
   

   Ensure the pod is Running. Check its logs if it's not:

   kubectl logs -n <your-nfs-namespace> -l app=nfs-subdir-external-provisioner
   kubectl describe pod -n <your-nfs-namespace> -l app=nfs-subdir-external-provisioner
   

2. Verify NFS server accessibility from WSL2: From your WSL2 Ubuntu terminal, try to manually mount the NFS share:

   sudo apt update && sudo apt install -y nfs-common # Install NFS client if not present
   sudo mount -t nfs <NFS_SERVER_IP>:<EXPORT_PATH> /mnt/test_nfs
   ls /mnt/test_nfs # Check contents
   sudo umount /mnt/test_nfs
   

   If this fails, your NFS server configuration or network connectivity is the issue.

3. Check the StorageClass and provisioner deployment for correct NFS server details: The StorageClass for NFS provisioner typically points to the NFS server and path. The nfs-subdir-external-provisioner deployment itself will also contain these details.

   # Example NFS StorageClass
   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: nfs-storage
   provisioner: nfs-subdir-external-provisioner
   parameters:
     archiveOnDelete: "false" # or "true"
   

   The actual NFS server and path are usually configured in the nfs-subdir-external-provisioner deployment/statefulset via environment variables or volume mounts.

4. Create a Static Persistent Volume (If Dynamic Provisioning is Not Used/Desired)

If dynamic provisioning is not set up, or you prefer static provisioning for a specific scenario (e.g., local development with a HostPath), you need to manually create a PersistentVolume that matches your PVC's requirements.

HostPath PVs are not suitable for production environments. They bind storage directly to the Kubernetes node's filesystem, making pods non-portable and introducing single points of failure. They are acceptable for single-node development environments like K3s/MicroK8s on WSL2.

1. Define the PV YAML: Create a file, e.g., my-app-pv.yaml. Ensure capacity, accessModes, storageClassName, and path match your PVC's expectations. The path MUST exist inside your WSL2 Ubuntu distro.

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     name: my-app-pv
     labels:
       type: local
   spec:
     storageClassName: local-path # Must match your PVC's storageClassName or be omitted if PVC omits it
     capacity:
       storage: 1Gi
     accessModes:
       - ReadWriteOnce
     hostPath:
       path: "/var/data/my-app" # This path must exist inside your WSL2 Ubuntu filesystem!
       type: DirectoryOrCreate # Ensures the directory exists if not already present
     persistentVolumeReclaimPolicy: Retain # Or Delete, Recycle
   

2. Create the directory in WSL2:

   # From inside your WSL2 Ubuntu terminal
   sudo mkdir -p /var/data/my-app
   sudo chown 1000:1000 /var/data/my-app # Adjust UID:GID to match your application's user
   sudo chmod 777 /var/data/my-app # Or more restrictive permissions
   

   Carefully consider the chown and chmod permissions. Your application inside the container will run as a specific user/group ID. The directory on the host (WSL2) must have permissions that allow this user/group to write to it. Often, 777 or matching container UID/GID is required for simplicity in development.

3. Apply the PV:

   kubectl apply -f my-app-pv.yaml
   

   After creating the PV, Kubernetes should automatically bind it to your Pending PVC if all criteria (capacity, access modes, storageClassName) match. Verify with kubectl get pvc.

5. Address Capacity and Access Mode Mismatches

If PVs exist but your PVC is still Pending, check if the requested resources or access modes can be fulfilled.

1. Check PVC requirements:

   kubectl get pvc my-app-data -o yaml | grep -E "storage|accessModes" -A 1
   

2. Check available PVs:

   kubectl get pv -o yaml
   

   • Capacity: Ensure there's a PV with capacity.storage equal to or greater than your PVC's resources.requests.storage.
   • Access Modes: The PV's accessModes must be compatible with (contain) the PVC's requested accessModes. For example, a PVC requesting ReadWriteMany cannot be satisfied by a PV offering only ReadWriteOnce.

   If a static PV requires ReadWriteMany and you're using HostPath, it will only work if multiple pods on the same node require access. HostPath fundamentally doesn't support true ReadWriteMany across multiple nodes in a multi-node cluster. For WSL2 single-node K3s/MicroK8s, ReadWriteOnce is usually sufficient, or ReadWriteMany if all accessing pods are on the same (and only) node.

• Action: If there's a mismatch, either modify your PVC to request less storage or compatible access modes (requires deleting and recreating the PVC), or create/configure a PV/StorageClass that can meet the PVC's demands.

6. Examine Provisioner Logs (If Dynamic Provisioning is Failing)

If your dynamic provisioner pod is running but PVCs are still pending, dive into its logs for specific errors.

# Get the provisioner pod name (adjust namespace and label selector as needed)
PROVISIONER_POD=$(kubectl get pods -n kube-system -l app=local-path-provisioner -o jsonpath='{.items[0].metadata.name}')

# Stream its logs
kubectl logs -n kube-system $PROVISIONER_POD --follow

The logs might reveal file system permissions errors, invalid arguments, or connectivity issues that kubectl describe doesn't expose.

7. WSL2 Specific Considerations

While most Kubernetes troubleshooting applies universally, WSL2 has unique characteristics.

• Filesystem Permissions: Ensure that any paths used by HostPath PVs or local-path-provisioner within the WSL2 filesystem have the correct user/group ownership and permissions for the processes running inside your containers. Kubernetes typically uses abstract UIDs/GIDs (e.g., 1000, 65534), which must map correctly to the WSL2 host.

• Resource Allocation: WSL2 by default allocates a portion of your Windows machine's resources. Ensure your WSL2 VM has sufficient memory and CPU allocated to run Kubernetes and your workloads, especially if using a resource-intensive storage provisioner. You can configure this in your ~/.wslconfig file in Windows.

• Networking: If your storage solution involves external services (like an NFS server on your local network or another VM), ensure Windows Firewall is not blocking communication between your WSL2 Ubuntu distro and the storage server.

• Reboot WSL2: Sometimes, a fresh start for the WSL2 environment can resolve transient issues.

  # From Windows PowerShell/CMD
  wsl --shutdown
  wsl # To restart your default distro
  

  Then, recheck your Kubernetes cluster and PVCs.