Editing Renewing Kubernetes Cluster Certificates

Kubernetes is supposed to renew its own certificates before there is a problem ... this was implemented in 1.15.x (??) ... but it is possible for there to be a problem if the master node is rebooted '''on the exact one-year anniversary date''' of the cluster:  after the certs have expired, but before the renewal process is triggered.  What you get is a message in the system logs:

 Client rotation is on, will bootstrap in background
 failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory

Very informative (not).  After poking around, I found this post in [https://stackoverflow.com/questions/56320930/renew-kubernetes-pki-after-expired/56334732#56334732 Stack Overflow] that had the answer -- summarized here ... 

So the solution was to (first a backup) -- all commands as root:

 mkdir ~/k8s
 cd /etc/kubernetes/pki/
 mv {apiserver.crt,apiserver-etcd-client.key,apiserver-kubelet-client.crt,front-proxy-ca.crt,front-proxy-client.crt,front-proxy-client.key,front-proxy-ca.key,apiserver-kubelet-client.key,apiserver.key,apiserver-etcd-client.crt} ~/k8s/
 kubeadm init phase certs all --apiserver-advertise-address <IP>
 cd /etc/kubernetes/
 mv {admin.conf,controller-manager.conf,kubelet.conf,scheduler.conf} ~/k8s/
 kubeadm init phase kubeconfig all
 reboot

Then copy the new certs to the default location so we can use kubectl:

 cp -i /etc/kubernetes/admin.conf $HOME/.kube/config