Editing K3s - Kubernetes Simplified (section)

=== External LoadBalancer/Reverse Proxy ===
The one remaining item needed for a fully HA control plane is an external load balancer or reverse proxy to provide a single IP address for accessing the control API from inside or outside the cluster.  Lacking something like a BigIron F5 load balancer or something equivalent, the poor-man's approach is to use a separate reverse proxy.  This can be done with HAproxy or NGINX.  HAproxy can support a HA reverse proxy if you need that level of redundancy, or a simple NGINX configuration like this will work:

<pre>stream {
  upstream k3s-servers {
    server 10.0.0.10:6443;
    server 10.0.0.74:6443;
    server 10.0.0.64:6443;
  }

  server {
    listen 6443;
    proxy_pass k3s-servers;
  }
}
</pre>

This needs to be at the top level in the NGINX configuration -- either in the /etc/nginx/nginx.conf itself, or included at the top level.  This process is more completely described in the K3S documentation:

  https://docs.k3s.io/datastore/cluster-loadbalancer

If setting up a reverse proxy is not an option, you can set up a round robin in the DNS.  Problem is, the pfSense firewall doesn't do round robins by default, so a little arm twisting is needed.  Put this in the 'custom options' in the DNS Resolver settings page:
 server:
 rrset-roundrobin: yes
 local-zone: "kube-test.williams.localnet" transparent
 local-data: "kube-test.williams.localnet A 10.0.0.10"
 local-data: "kube-test.williams.localnet A 10.0.0.64"
 local-data: "kube-test.williams.localnet A 10.0.0.74"
Change the name and IP addresses as needed, and then load the changes.  

Using either approach somewhat complicates the deployment commands below, so pay attention ...