How to restart a failed pod in kubernetes deployment
KubernetesKubernetes Problem Overview
I have 3 nodes in kubernetes cluster. I create a daemonset
and deployed it in all the 3 devices. This daemonset
created 3 pods and they were successfully running. But for some reasons, one of the pod failed.
I need to know how can we restart this pod without affecting other pods in the daemon set, also without creating any other daemon set deployment?
Thanks
Kubernetes Solutions
Solution 1 - Kubernetes
kubectl delete pod <podname>
it will delete this one pod and Deployment/StatefulSet/ReplicaSet/DaemonSet will reschedule a new one in its place
Solution 2 - Kubernetes
There are other possibilities to acheive what you want:
-
Just use
rollout
commandkubectl rollout restart deployment mydeploy
-
You can set some environment variable which will force your deployment pods to restart:
kubectl set env deployment mydeploy DEPLOY_DATE="$(date)"
-
You can scale your deployment to zero, and then back to some positive value
kubectl scale deployment mydeploy --replicas=0
kubectl scale deployment mydeploy --replicas=1
Solution 3 - Kubernetes
Just for others reading this...
A better solution (IMHO) is to implement a liveness prob that will force the pod to restart the container if it fails the probe test.
This is a great feature K8s offers out of the box. This is auto healing.
Also look into the pod lifecycle docs.
Solution 4 - Kubernetes
kubectl -n <namespace> delete pods --field-selector=status.phase=Failed
I think the above command is quite useful when you want to restart 1 or more failed pods :D
And we don't need to care about name of the failed pod.