In this lecture we will see different ways of troubleshooting worker node failures.
Again, we start by checking the status of the nodes in the cluster. Are they reported as Ready or
not ready.
If they are reported as Not Ready, check details about the nodes using the kubectl describe node
command. Each node has a set of conditions that can point us in a direction as to why a node might have
failed. Depending on the status they are either set to true or false or unknown. When the node is out
of this space
the out of disk flag is that a true. When a node is out of memory the memory pressure flag is set to
true. When the disk capacity is low, the disk pressure flag is set to true.
Similarly when there are too many processes the PID Pressure flag is set to true.
And finally if the node as a whole is health the ready flag is set to true.
When a worker node stops communicating with the master, may be due to a crash, these statuses are set to
unknown.
This can indicate a possible loss of a node.
Check the last Heart Beat time field to find out the time when the node might have crashed. In such cases
proceed to checking the status of the node itself.
If the node is online at all or is crashed if it's crashed bring it back up.
Check for possible CPU, Memory, and Disk space on the node.
Check the status of the kubelet.
Check the kubelet logs for possible issues.
Check the kubelet certificates. Ensure they are not expired and that they are part of the right group, and that
the certificates are issued by the right CA.
Well that's it for this lecture.
Head over to the practice test and practice fixing broken clusters.