Kubernetes: Can't delete PersistentVolumeClaim (pvc)

KubernetesPersistent VolumesPersistent Volume-Claims

Kubernetes Problem Overview


I created the following persistent volume by calling

kubectl create -f nameOfTheFileContainingTheFollowingContent.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-monitoring-static-content
spec:
  capacity:
    storage: 100Mi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/some/path"

---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv-monitoring-static-content-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: ""
  resources:
    requests:
      storage: 100Mi

After this I tried to delete the pvc. But this command stuck. when calling kubectl describe pvc pv-monitoring-static-content-claim I get the following result

Name:          pv-monitoring-static-content-claim
Namespace:     default
StorageClass:
Status:        Terminating (lasts 5m)
Volume:        pv-monitoring-static-content
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed=yes
               pv.kubernetes.io/bound-by-controller=yes
Finalizers:    [foregroundDeletion]
Capacity:      100Mi
Access Modes:  RWO
Events:        <none>

And for kubectl describe pv pv-monitoring-static-content

Name:            pv-monitoring-static-content
Labels:          <none>
Annotations:     pv.kubernetes.io/bound-by-controller=yes
Finalizers:      [kubernetes.io/pv-protection foregroundDeletion]
StorageClass:
Status:          Terminating (lasts 16m)
Claim:           default/pv-monitoring-static-content-claim
Reclaim Policy:  Retain
Access Modes:    RWO
Capacity:        100Mi
Node Affinity:   <none>
Message:
Source:
    Type:          HostPath (bare host directory volume)
    Path:          /some/path
    HostPathType:
Events:            <none>

There is no pod running that uses the persistent volume. Could anybody give me a hint why the pvc and the pv are not deleted?

Kubernetes Solutions


Solution 1 - Kubernetes

This happens when persistent volume is protected. You should be able to cross verify this:

Command:

kubectl describe pvc PVC_NAME | grep Finalizers

Output:

Finalizers: [kubernetes.io/pvc-protection]

You can fix this by setting finalizers to null using kubectl patch:

kubectl patch pvc PVC_NAME -p '{"metadata":{"finalizers": []}}' --type=merge

Ref; Storage Object in Use Protection

Solution 2 - Kubernetes

I'm not sure why this happened, but after deleting the finalizers of the pv and the pvc via the kubernetes dashboard, both were deleted. This happened again after repeating the steps I described in my question. Seems like a bug.

Solution 3 - Kubernetes

You can get rid of editing your pvc! Remove pvc protection.

  1. kubectl edit pvc YOUR_PVC -n NAME_SPACE
  2. Manually edit and put # before this line enter image description here
  3. All pv and pvc will be deleted

Solution 4 - Kubernetes

The PV is protected. Delete the PV before deleting the PVC. Also, delete any pods/ deployments which are claiming any of the referenced PVCs. For further information do check out Storage Object in Use Protection

Solution 5 - Kubernetes

For me pv was in retain state, hence doing the above steps did not work.

1st we need to change policy state as below :

kubectl patch pv PV_NAME -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}'

Then delete pvc as below.

kubectl get pvc

kubectl delete pvc PVC_NAME

finally, delete pv with

kubectl delete pv PV_NAME

Solution 6 - Kubernetes

Just met this issue hours ago.

I deleted deployments that used this references and the PV/PVCs are automatically terminated.

Solution 7 - Kubernetes

If PV still exists it may be because it has ReclaimPolicy set to Retain in which case it won't be deleted even if PVC is gone. From the docs:

> PersistentVolumes can have various reclaim policies, including > “Retain”, “Recycle”, and “Delete”. For dynamically provisioned > PersistentVolumes, the default reclaim policy is “Delete”. This means > that a dynamically provisioned volume is automatically deleted when a > user deletes the corresponding PersistentVolumeClaim. This automatic > behavior might be inappropriate if the volume contains precious data. > In that case, it is more appropriate to use the “Retain” policy. With > the “Retain” policy, if a user deletes a PersistentVolumeClaim, the > corresponding PersistentVolume is not be deleted. Instead, it is moved > to the Released phase, where all of its data can be manually recovered

Solution 8 - Kubernetes

In my case, as long as I delete the pod associated to both pv and pvc, the pv and pvc in terminating status are gone

Solution 9 - Kubernetes

in my case a pvc was not deleted because missing namespace (I deleted the namespace before deleting all resources/pvc) solution : create namespace with the same name as it was before and then I was able to remove the finalizers and finally pvc

Solution 10 - Kubernetes

In case you have already deleted PV and trying to delete PVC

Check if the volume is attached by this command

kubectl get volumeattachment

Deleting the PVC :-

First you have to delete pvc pne by one using this command

kubectl delete pvc --grace-period=0 --force

Or you can delete all PVC's using

kubectl delete pvc --all

Now you can see the status of PVC as terminating by using

kubectl get pvc

and then you have to apply this delete using

kubectl patch pvc {PVC_NAME} -p '{"metadata":{"finalizers":null}}'

Solution 11 - Kubernetes

kubectl get pvc pvc_name -o yaml > pvcfile.yaml

Then open pvcfile.yaml and delete the finalizers line, save and apply :

kubectl apply -f pvcfile.yaml 

Solution 12 - Kubernetes

Additionally to the other answers about the finalizer...

I could free up the resources only after deleting the Deployment. After that, the Terminating resources got released.


Delete all the resources listed by:

kubectl get all -n YOURNAMESPACE

Use kubectl delete -n YOURNAMESPACE <resource> <id> or (if you copy paste from the above output) kubectl delete -n YOURNAMESPACE <resource>/<id>, for each resource that you see listed there.

You can also do it at once

kubectl delete -n YOURNAMESPACE <resource>/<id1>  <resource>/<id2>  <resource2>/<id3>  <resource2>/<id4>  <resource3>/<id5>

Probably you tried to remove resources but they are getting recreated because of the deployment or replicaset resource, preventing the namespace from freeing up depending resources and from being cleaned up.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionYannic KlemView Question on Stackoverflow
Solution 1 - KubernetesXiakView Answer on Stackoverflow
Solution 2 - KubernetesYannic KlemView Answer on Stackoverflow
Solution 3 - KubernetesAli AtakanView Answer on Stackoverflow
Solution 4 - KubernetesgeorgekuruvillakView Answer on Stackoverflow
Solution 5 - KubernetesBibekView Answer on Stackoverflow
Solution 6 - KubernetesakaiView Answer on Stackoverflow
Solution 7 - KubernetesAnna SlastnikovaView Answer on Stackoverflow
Solution 8 - Kubernetesj3ffyangView Answer on Stackoverflow
Solution 9 - Kubernetesdansl1982View Answer on Stackoverflow
Solution 10 - KubernetesAbhishek DeshmukhView Answer on Stackoverflow
Solution 11 - Kuberneteswiem limView Answer on Stackoverflow
Solution 12 - KubernetesKamafeatherView Answer on Stackoverflow