Kubernetes Controller - Revision history

DrEdWilliams: /* Debian 10 */

2021-04-10T22:06:42Z

Debian 10

← Older revision		Revision as of 22:06, 10 April 2021
Line 22:		Line 22:
	=== Debian 10 ===		=== Debian 10 ===
	apt-get update && apt-get install -y apt-transport-https curl		apt-get update && apt-get install -y apt-transport-https curl
	curl -s <nowiki>https://packages.cloud.google.com/apt/doc/apt-key.gpg</nowiki> ~~\| apt-key add -~~		sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg <nowiki>https://packages.cloud.google.com/apt/doc/apt-key.gpg</nowiki>
	~~cat <<EOF >~~/~~etc~~/~~apt~~/~~sources.list.d~~/kubernetes.~~list~~		echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] <nowiki>https://apt.kubernetes.io/</nowiki> kubernetes-xenial main" \| sudo tee /etc/apt/sources.list.d/kubernetes.list
	~~deb~~ <nowiki>https://apt.kubernetes.io/</nowiki> kubernetes-xenial main
	~~EOF~~
	apt-get update		apt-get update
	apt-get install -y kubelet kubeadm kubectl		apt-get install -y kubelet kubeadm kubectl

DrEdWilliams: /* Deploy the NVIDIA device controller */

2020-12-26T01:49:26Z

Deploy the NVIDIA device controller

← Older revision		Revision as of 01:49, 26 December 2020
Line 95:		Line 95:

	Complete instructions can be found at https://github.com/NVIDIA/k8s-device-plugin		Complete instructions can be found at https://github.com/NVIDIA/k8s-device-plugin

			If using '''containerd''' as the runtime, see here - https://josephb.org/blog/containerd-nvidia/

DrEdWilliams at 00:19, 2 August 2020

2020-08-02T00:19:53Z

← Older revision		Revision as of 00:19, 2 August 2020
Line 83:		Line 83:

	kubectl taint nodes --all node-role.kubernetes.io/master-		kubectl taint nodes --all node-role.kubernetes.io/master-

			== Install kubectl plugins ==

			[[Kubectl Plugins and Krew]] -- just do it ... you will thank me later

	== Deploy the NVIDIA device controller ==		== Deploy the NVIDIA device controller ==

DrEdWilliams at 06:35, 11 July 2020

2020-07-11T06:35:15Z

← Older revision		Revision as of 06:35, 11 July 2020
Line 59:		Line 59:
	Full instructions on how to install and use this plugin can be found at https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md. This plugin implements both the pod network and the proxy functions -- so the ''kube-proxy'' daemonset is no longer needed.		Full instructions on how to install and use this plugin can be found at https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md. This plugin implements both the pod network and the proxy functions -- so the ''kube-proxy'' daemonset is no longer needed.

	Grab ~~and install~~ the manifest straight from their GitHub site:		Grab the manifest straight from their GitHub site:

	~~KUBECONFIG=/$HOME/.kube/config kubectl apply -f~~ https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml		wget https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml

			Edit the manifest to remove the '--run-firewall=true' option, set the '--run-service-proxy' option to 'ture', and add another option that says:

			--kubeconfig=/var/lib/kube-router/kubeconfig/config

			Then you will need to create that directory and populate it:

			sudo mkdir -p /var/lib/kube-router/kubeconfig
			sudo cp /etc/kubernetes/admin.conf /var/lib/kube-router/kubeconfig/config

			Not sure yet if this needs to be done for all nodes, or just the master ...

	This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the '''/etc/sysconfig/kubelet''' file, adding one parameter to the end of the line:		This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the '''/etc/sysconfig/kubelet''' file, adding one parameter to the end of the line:

DrEdWilliams at 05:45, 11 July 2020

2020-07-11T05:45:09Z

← Older revision		Revision as of 05:45, 11 July 2020
Line 57:		Line 57:
	Pod networking is provided by an external plugin implementing the CNI specification; this section describes the deployment of the '''kube-router''' plugin. Instructions on how to deploy the '''flannel''' plugin can be found [[Kubernetes Flannel Network Plugin\|here]].		Pod networking is provided by an external plugin implementing the CNI specification; this section describes the deployment of the '''kube-router''' plugin. Instructions on how to deploy the '''flannel''' plugin can be found [[Kubernetes Flannel Network Plugin\|here]].

	Full instructions on how to install and use this plugin can be found at https://github.com/cloudnativelabs/kube-router. This plugin implements both the pod network and the proxy functions -- so the ''kube-proxy'' daemonset ~~will be deleted as part of the installation~~.		Full instructions on how to install and use this plugin can be found at https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md. This plugin implements both the pod network and the proxy functions -- so the ''kube-proxy'' daemonset is no longer needed.

	~~Since~~ the ~~default parameter set needs to be adjusted, download the yaml~~ manifest:		Grab and install the manifest straight from their GitHub site:

	~~wget~~ https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter~~-all-features~~.yaml		KUBECONFIG=/$HOME/.kube/config kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml

	~~Edit the file to remove the <pre>--run-firewall=true</pre> parameter, then apply it to the cluster:~~

	~~kubectl apply -f kubeadm-kuberouter-all-features.yaml~~

	~~Also, as mentioned above, we need to shut down and clean up the kube-proxy pods (only needed if you forgot to add the "--skip-phases ..." parameter on kubeadm init:~~

	~~kubectl -n kube-system delete ds kube-proxy~~
	~~docker run --privileged -v /lib/modules:/lib/modules --net=host k8s.gcr.io/kube-proxy-amd64:v1.15.1 kube-proxy --cleanup~~

	This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the '''/etc/sysconfig/kubelet''' file, adding one parameter to the end of the line:		This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the '''/etc/sysconfig/kubelet''' file, adding one parameter to the end of the line:

DrEdWilliams at 15:04, 25 January 2020

2020-01-25T15:04:31Z

← Older revision		Revision as of 15:04, 25 January 2020
Line 42:		Line 42:
	Initialize the masternode with the CIDR parameter to support pod networking (as root):		Initialize the masternode with the CIDR parameter to support pod networking (as root):

	kubeadm init --pod-network-cidr=10.244.0.0/16 --token-ttl 0 --ignore-preflight-errors Swap --node-name `hostname -s`		kubeadm init --pod-network-cidr=10.244.0.0/16 --token-ttl 0 --ignore-preflight-errors Swap --node-name `hostname -s` --skip-phases=addon/kube-proxy

	If you want the master to not use the default network interface for API communication, specify the following parameter on the kubeadm init command (per [https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/ this page]):		If you want the master to not use the default network interface for API communication, specify the following parameter on the kubeadm init command (per [https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/ this page]):
Line 67:		Line 67:
	kubectl apply -f kubeadm-kuberouter-all-features.yaml		kubectl apply -f kubeadm-kuberouter-all-features.yaml

	Also, as mentioned above, we need to shut down and clean up the kube-proxy pods:		Also, as mentioned above, we need to shut down and clean up the kube-proxy pods (only needed if you forgot to add the "--skip-phases ..." parameter on kubeadm init:

	kubectl -n kube-system delete ds kube-proxy		kubectl -n kube-system delete ds kube-proxy

DrEdWilliams at 22:39, 26 December 2019

2019-12-26T22:39:37Z

← Older revision		Revision as of 22:39, 26 December 2019
Line 38:		Line 38:
	systemctl enable kubelet		systemctl enable kubelet

	Initialize ~~master~~ with parameter to support ~~flannel network~~ (as root)		See below for one more parameter that could go on the end of this line now to save some time later.

			Initialize the masternode with the CIDR parameter to support pod networking (as root):

	kubeadm init --pod-network-cidr=10.244.0.0/16 --token-ttl 0 --ignore-preflight-errors Swap --node-name `hostname -s`		kubeadm init --pod-network-cidr=10.244.0.0/16 --token-ttl 0 --ignore-preflight-errors Swap --node-name `hostname -s`

	If you want the master to not use the default network interface for ~~inter-node~~ communication, specify the following parameter on the kubeadm init command (per [https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/ this page]):		If you want the master to not use the default network interface for API communication, specify the following parameter on the kubeadm init command (per [https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/ this page]):
	--apiserver-advertise-address A.B.C.D		--apiserver-advertise-address A.B.C.D

Line 70:		Line 72:
	docker run --privileged -v /lib/modules:/lib/modules --net=host k8s.gcr.io/kube-proxy-amd64:v1.15.1 kube-proxy --cleanup		docker run --privileged -v /lib/modules:/lib/modules --net=host k8s.gcr.io/kube-proxy-amd64:v1.15.1 kube-proxy --cleanup

	This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the ~~<pre>~~/~~var~~/~~lib~~/kubelet~~/kubeadm-flags.env</pre>~~ file, adding one parameter ~~inside~~ the ~~quoted string~~:		This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the '''/etc/sysconfig/kubelet''' file, adding one parameter to the end of the line:

	~~KUBEADM_EXTRA_ARGS~~=" ..... --node-ip=10.1.0.99"		KUBELET_EXTRA_ARGS= ..... --node-ip=10.1.0.99

	~~Then~~ restart kubelet and it will be happy, and (eventually) everything else will be happy too -- though some pods may need to be manually restarted (such as the kube-router pods that get started as soon as the node is joined to the network). Ideally we need to find a way to set this during the kubeadm join process ... stay tuned ...		This should be done before the ''kubeadm init'' command above -- in which case the node's InternalIP address is already set. If you do it after the ''kubeadm init'', then restart kubelet now and it will be happy, and (eventually) everything else will be happy too -- though some pods may need to be manually restarted (such as the kube-router pods that get started as soon as the node is joined to the network). Ideally we need to find a way to set this during the kubeadm join process ... stay tuned ...

	== Enable pods to run on master node if desired ==		== Enable pods to run on master node if desired ==

DrEdWilliams at 22:03, 26 December 2019

2019-12-26T22:03:10Z

← Older revision		Revision as of 22:03, 26 December 2019
Line 70:		Line 70:
	docker run --privileged -v /lib/modules:/lib/modules --net=host k8s.gcr.io/kube-proxy-amd64:v1.15.1 kube-proxy --cleanup		docker run --privileged -v /lib/modules:/lib/modules --net=host k8s.gcr.io/kube-proxy-amd64:v1.15.1 kube-proxy --cleanup

	This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the <pre>/var/lib/kubelet/kubeadm-flags.env, adding one parameter inside the quoted string:		This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the <pre>/var/lib/kubelet/kubeadm-flags.env</pre> file, adding one parameter inside the quoted string:

	KUBEADM_EXTRA_ARGS=" ..... --node-ip=10.1.0.99"		KUBEADM_EXTRA_ARGS=" ..... --node-ip=10.1.0.99"

DrEdWilliams: cut out (and linked to) flannel -- added kube-router as the pod networking plugin

2019-12-26T21:59:28Z

cut out (and linked to) flannel -- added kube-router as the pod networking plugin

← Older revision		Revision as of 21:59, 26 December 2019
Line 53:		Line 53:
	== Install pod network ==		== Install pod network ==

	Pod networking is provided by an external plugin; this section describes the deployment of the '''kube-router''' plugin. Instructions on how to deploy the '''flannel''' plugin can be found [[Kubernetes Flannel Network Plugin\|here]].		Pod networking is provided by an external plugin implementing the CNI specification; this section describes the deployment of the '''kube-router''' plugin. Instructions on how to deploy the '''flannel''' plugin can be found [[Kubernetes Flannel Network Plugin\|here]].

	~~The Flannel~~ pod ~~networking can~~ be ~~installed directly from~~ the ~~repository using this command:~~		Full instructions on how to install and use this plugin can be found at https://github.com/cloudnativelabs/kube-router. This plugin implements both the pod network and the proxy functions -- so the ''kube-proxy'' daemonset will be deleted as part of the installation.

	~~kubectl apply -f https~~:~~//raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml~~		Since the default parameter set needs to be adjusted, download the yaml manifest:

	~~On Kubernetes v1~~.~~16+, if there are issues (specifically, nodes don't reach the '''''Ready''''' state), try this specific version:~~		wget https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter-all-features.yaml

	~~kubectl~~ apply ~~-f https~~:~~//raw.githubusercontent.com/coreos/flannel/2140ac876ef134e0ed5af15c65e414cf26827915/Documentation/kube-flannel.yml~~		Edit the file to remove the <pre>--run-firewall=true</pre> parameter, then apply it to the cluster:

	By default, flannel uses the primary network interface (eth0 on 10.0.0.0/24); but if you want to push the traffic onto the secondary interfaces (such as for the development cluster), download the YAML manifest and add one parameter to the argument list on/about line 190 (in the spec -> template -> spec -> containers [where name == 'kube-~~flannel~~-ds-~~amd64']~~ -~~> args)~~. ~~The result should look like this:~~		kubectl apply -f kubeadm-kuberouter-all-features.yaml

	~~...~~		Also, as mentioned above, we need to shut down and clean up the kube-proxy pods:
	~~args:~~
	~~- --ip-masq~~
	~~- --~~kube-~~subnet-mgr~~
	~~- --iface=eth1~~
	~~resources~~:
	~~...~~

	~~substitute the secondary interface name for 'eth1' if it is different~~. ~~Then apply the manifest~~:		kubectl -n kube-system delete ds kube-proxy
			docker run --privileged -v /lib/modules:/lib/modules --net=host k8s.gcr.io/kube-proxy-amd64:v1.15.1 kube-proxy --cleanup

	~~kubectl apply~~ -~~f kube~~-~~flannel~~.~~yaml~~		This will implement all routing onto the network interface that supports the default route for the host systems. If, however, there is a secondary network that is targeted for cluster communications, we need to do one more thing: Change the default IP address for the node -- at least as far as the kubelet daemon is concerned. The brute force way of doing this is to modify the <pre>/var/lib/kubelet/kubeadm-flags.env, adding one parameter inside the quoted string:

	~~Other configuration options can~~ be ~~passed~~ to ~~flannel per [https://github~~.~~com/coreos/flannel/blob/master/Documentation/configuration~~.~~md this page]~~		KUBEADM_EXTRA_ARGS=" ..... --node-ip=10.1.0.99"

			Then restart kubelet and it will be happy, and (eventually) everything else will be happy too -- though some pods may need to be manually restarted (such as the kube-router pods that get started as soon as the node is joined to the network). Ideally we need to find a way to set this during the kubeadm join process ... stay tuned ...

	== Enable pods to run on master node if desired ==		== Enable pods to run on master node if desired ==

DrEdWilliams at 21:39, 26 December 2019

2019-12-26T21:39:12Z

← Older revision		Revision as of 21:39, 26 December 2019
Line 51:		Line 51:
	sudo chown $(id -u):$(id -g) $HOME/.kube/config		sudo chown $(id -u):$(id -g) $HOME/.kube/config

	== Install ~~flannel~~ pod network ==		== Install pod network ==

			Pod networking is provided by an external plugin; this section describes the deployment of the '''kube-router''' plugin. Instructions on how to deploy the '''flannel''' plugin can be found [[Kubernetes Flannel Network Plugin\|here]].

	The Flannel pod networking can be installed directly from the repository using this command:		The Flannel pod networking can be installed directly from the repository using this command: