Production Cluster Configuration - Revision history

DrEdWilliams at 23:09, 14 September 2024

2024-09-14T23:09:36Z

← Older revision		Revision as of 23:09, 14 September 2024
Line 10:		Line 10:
	\|-		\|-
	\| [[Rook Storage for Kubernetes\|Rook Storage]] \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme		\| [[Rook Storage for Kubernetes\|Rook Storage]] \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme
	~~\|-\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|~~
	\|-		\|-
	\| ~~mail~~ \|\| ~~k8s/mail~~ \|\| \|\| ~~10.0.0.114~~ \|\| ~~mail.williams.localnet~~		\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|
	\|-		\|-
	\| wordpress (dredwilliams.com) \|\| k8s/dredwilliams \|\| \|\| \|\| dredwilliams.williams-net.org		\| wordpress (dredwilliams.com) \|\| k8s/dredwilliams \|\| \|\| \|\| dredwilliams.williams-net.org

DrEdWilliams at 00:21, 18 May 2022

2022-05-18T00:21:36Z

← Older revision		Revision as of 00:21, 18 May 2022
Line 6:		Line 6:
	\|-		\|-
	! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)		! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)
	\|-
	~~\| K8Dash Dashboard \|\| k8s-admin \|\| \|\| 10.0.0.191 \|\|~~
	\|-		\|-
	\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|		\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|
	\|-		\|-
	\| [[Rook Storage for Kubernetes\|Rook Storage]] ~~(future)~~ \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme		\| [[Rook Storage for Kubernetes\|Rook Storage]] \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme
	\|-
	~~\| [[Contour Ingress Controller]] (optional)\|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|~~
	\|-\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|		\|-\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|
	\|-
	~~\| rsyslog\|\| k8s/rsyslog \|\| \|\| 10.0.0.113 \|\| rsyslog.williams.localnet~~
	\|-		\|-
	\| mail \|\| k8s/mail \|\| \|\| 10.0.0.114 \|\| mail.williams.localnet		\| mail \|\| k8s/mail \|\| \|\| 10.0.0.114 \|\| mail.williams.localnet
Line 25:		Line 19:
	\|-		\|-
	\| [[MariaDB]] \|\| mariadb \|\| \|\| 10.0.0.117 \|\| database.williams.localnet		\| [[MariaDB]] \|\| mariadb \|\| \|\| 10.0.0.117 \|\| database.williams.localnet
	\|-
	~~\| [[Rocket.Chat]] (not currently deployed) \|\| rocketchat \|\| \|\| (contour) \|\| rocket.williams-net.org~~
	\|}		\|}

Line 35:		Line 27:
	! system !! function !! storage !! size		! system !! function !! storage !! size
	\|-		\|-
	\| ~~telmar~~ \|\| master \|\| NVMe<br/>HDD \|\| 1TB<br/>1TB		\| caspian \|\| master \|\| NVMe<br/>HDD \|\| 1TB<br/>1TB
	\|-		\|-
	\| ~~compute4~~ \|\| node \|\| NVMe<br/>HDD<br/>HDD \|\| 1TB<br/>1TB<br/>1TB		\| uvilas \|\| node \|\| NVMe<br/>HDD<br/>HDD \|\| 1TB<br/>1TB<br/>1TB
	\|-		\|-
	\| ~~pro5~~ \|\| node \|\| NVMe<br/>HDD \|\| 1TB<br/>250GB		\| belisar \|\| node \|\| NVMe<br/>HDD \|\| 1TB<br/>250GB
	\|}		\|}

	~~All systems mount the '''/shared''' ceph filesystem following the directions in the installation page. The relevant line for /etc/fstab is:~~

	~~10.0.0.10:/ /shared ceph name=prodcluster,_netfs 0 0~~

	~~The client keyring must be copied from the master node (calormen) and placed in the /etc/ceph directory on the client system prior to mounting.~~

	The work filesystem can be mounted via NFS:		The work filesystem can be mounted via NFS:
Line 56:		Line 42:
	In addition to the normal backups configured in the basic OS installation steps, the databases in the production cluster must be backed up daily using the 'mysqldump' command:		In addition to the normal backups configured in the basic OS installation steps, the databases in the production cluster must be backed up daily using the 'mysqldump' command:

	~~mysqldump -u root -pmenagerie --all-databases -h 10.96.244.162 > /shared/mediawiki-all.dump~~		TBD
	~~mysqldump -u root -pmenagerie bitnami_mediawiki -h 10.96.244.162 > /shared/mediawiki.dump~~
	~~mysqldump -u root -pmenagerie --all-databases -h database.williams.localnet > /shared/database.dump~~

	These commands should be inserted into the /etc/cron.daily/backup file on one of the cluster nodes (telmar is a good choice). The first does a complete database dump of the MediaWiki database server, the second dumps just the mediawiki database itself, and the third dumps the general purpose database server. Additional dump commands should be inserted for additional significant databases, as parsing individual databases out of a system dump can be tedious.		These commands should be inserted into the /etc/cron.daily/backup file on one of the cluster nodes (telmar is a good choice). The first does a complete database dump of the MediaWiki database server, the second dumps just the mediawiki database itself, and the third dumps the general purpose database server. Additional dump commands should be inserted for additional significant databases, as parsing individual databases out of a system dump can be tedious.

	~~=== Dashboard Token ===~~
	~~Obtain the token needed to log into the dashboard with this command:~~
	~~kubectl -n kube-system describe secrets \~~
	~~`kubectl -n kube-system get secrets \| awk '/clusterrole-aggregation-controller/ {print $1}'` \~~
	~~\| awk '/token:/ {print $2}'~~
	~~The current token for the Production cluster is:~~
	eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjbHVzdGVycm9sZS1hZ2dyZWdhdGlvbi1jb250cm9sbGVyLXRva2VuLTdydDQ3Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImNsdXN0ZXJyb2xlLWFnZ3JlZ2F0aW9uLWNvbnRyb2xsZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIwYjk1NmU5Yi01MmJiLTQwMWEtYTgwOC03MWI5YWVjNDZjNGQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06Y2x1c3RlcnJvbGUtYWdncmVnYXRpb24tY29udHJvbGxlciJ9.bOL_ObIZ5vNkTlMd1Cdxsy6AHd_LRH-uf3-6g3YeKVoCtaKkGyR9C7mZlTQrpc6844l4sGMWBWW5HytCK9JTBoHpDADeJZQa0Q5S8cyQMPpNJUukatxzUtHN07FZ6iIl6j_wqLvVJq1dPcu_orD2HGUt7peb0FJ8Ut17opGjR9elLdR0AbZy91EJMoNj5tDCXn0-hdtjbNTu0mGzXfON9Mt3ZIjbXE31uJlji-5KfZjPzhqV0UI7v0R3yoEfPINZlqX7xmqeJt8lI0z-rgRdygLmepRaT6CYpP6IJvAsog06JpQpoU0mZmWKOqEYHS7K_AFGRV5z3vp7QLSPi1PKFA


	~~=== Kubernetes Node Join Command ===~~
	~~kubeadm join 10.0.0.10:6443 --token hqxg8k.bcz5utygyd2sa4yn \~~
	~~--discovery-token-ca-cert-hash sha256:ec16325aa0d701961337bc15889e8a90dd1f2d37e08f47d6211d4d7b839b4eb3 \~~
	~~--ignore-preflight-errors Swap --node-name=`hostname -s`~~

DrEdWilliams at 22:37, 15 August 2021

2021-08-15T22:37:56Z

← Older revision		Revision as of 22:37, 15 August 2021
Line 6:		Line 6:
	\|-		\|-
	! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)		! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)
			\|-
			\| K8Dash Dashboard \|\| k8s-admin \|\| \|\| 10.0.0.191 \|\|
	\|-		\|-
	\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|		\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|

DrEdWilliams: /* Storage */

2021-08-15T22:02:59Z

Storage

← Older revision		Revision as of 22:02, 15 August 2021
Line 33:		Line 33:
	! system !! function !! storage !! size		! system !! function !! storage !! size
	\|-		\|-
	\| ~~calormen~~ \|\| master \|\| ~~NVME~~ \|\| 1TB		\| telmar \|\| master \|\| NVMe<br/>HDD \|\| 1TB<br/>1TB
	\|-		\|-
	\| ~~telmar~~ \|\| node \|\| NVMe<br/>HDD \|\| 1TB<br/>1TB		\| compute4 \|\| node \|\| NVMe<br/>HDD<br/>HDD \|\| 1TB<br/>1TB<br/>1TB
	\|-		\|-
	\| ~~compute4~~ \|\| node \|\| NVMe~~<br/>HDD~~<br/>HDD \|\| 1TB<br/>~~1TB<br/>1TB~~		\| pro5 \|\| node \|\| NVMe<br/>HDD \|\| 1TB<br/>250GB
	\|}		\|}

	All systems mount the '''/shared''' ceph filesystem following the directions in the installation page. The relevant line for /etc/fstab is:		All systems mount the '''/shared''' ceph filesystem following the directions in the installation page. The relevant line for /etc/fstab is:

	10.0.0.3:/ /shared ceph name=prodcluster,_netfs 0 0		10.0.0.10:/ /shared ceph name=prodcluster,_netfs 0 0

	The client keyring must be copied from the master node (calormen) and placed in the /etc/ceph directory on the client system prior to mounting.		The client keyring must be copied from the master node (calormen) and placed in the /etc/ceph directory on the client system prior to mounting.

	The ~~workspace~~ filesystem ~~is not generally available from the production as it lives on the StorageNet VLAN, but if needed, it~~ can be mounted via NFS:		The work filesystem can be mounted via NFS:

	~~controller~~:/~~workspace~~ /~~workspace~~ nfs4 soft 0 0		10.0.0.75:/work /work nfs4 soft 0 0

	=== Backups ===		=== Backups ===

DrEdWilliams at 21:59, 15 August 2021

2021-08-15T21:59:16Z

← Older revision		Revision as of 21:59, 15 August 2021
Line 9:		Line 9:
	\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|		\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|
	\|-		\|-
	\| [[Rook Storage for Kubernetes\|Rook Storage]] future) \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme		\| [[Rook Storage for Kubernetes\|Rook Storage]] (future) \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme
	\|-		\|-
	\| [[Contour Ingress Controller]] (optional)\|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|		\| [[Contour Ingress Controller]] (optional)\|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|

DrEdWilliams at 21:58, 15 August 2021

2021-08-15T21:58:23Z

← Older revision		Revision as of 21:58, 15 August 2021
Line 9:		Line 9:
	\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|		\| [[Ceph Storage Cluster]] \|\| k8s-admin \|\| \|\| \|\|
	\|-		\|-
	\| [[Rook Storage for Kubernetes\|Rook Storage]] \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme		\| [[Rook Storage for Kubernetes\|Rook Storage]] future) \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme
	\|-		\|-
	\| [[Contour Ingress Controller]] (optional)\|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|		\| [[Contour Ingress Controller]] (optional)\|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|

DrEdWilliams at 21:57, 15 August 2021

2021-08-15T21:57:29Z

← Older revision		Revision as of 21:57, 15 August 2021
Line 24:		Line 24:
	\| [[MariaDB]] \|\| mariadb \|\| \|\| 10.0.0.117 \|\| database.williams.localnet		\| [[MariaDB]] \|\| mariadb \|\| \|\| 10.0.0.117 \|\| database.williams.localnet
	\|-		\|-
	\| [[Rocket.Chat]] \|\| rocketchat \|\| \|\| (contour) \|\| rocket.williams-net.org		\| [[Rocket.Chat]] (not currently deployed) \|\| rocketchat \|\| \|\| (contour) \|\| rocket.williams-net.org
	\|}		\|}

DrEdWilliams at 21:51, 15 August 2021

2021-08-15T21:51:11Z

← Older revision		Revision as of 21:51, 15 August 2021
Line 11:		Line 11:
	\| [[Rook Storage for Kubernetes\|Rook Storage]] \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme		\| [[Rook Storage for Kubernetes\|Rook Storage]] \|\| k8s-admin \|\| \|\| \|\| (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme
	\|-		\|-
	\| [[Contour Ingress Controller]] \|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|		\| [[Contour Ingress Controller]] (optional)\|\| k8s-admin \|\| \|\| 10.0.0.115 \|\|
	\|-		\|-\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|
	~~\| [[GitLab]] \|\| gitlab \|\| \|\| 10.0.0.112 \|\| gitlab.williams.localnet <br />gitlab.williams-net.org~~
	\|-
	\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|
	\|-		\|-
	\| rsyslog\|\| k8s/rsyslog \|\| \|\| 10.0.0.113 \|\| rsyslog.williams.localnet		\| rsyslog\|\| k8s/rsyslog \|\| \|\| 10.0.0.113 \|\| rsyslog.williams.localnet

DrEdWilliams at 00:29, 5 May 2021

2021-05-05T00:29:13Z

← Older revision		Revision as of 00:29, 5 May 2021
Line 16:		Line 16:
	\|-		\|-
	\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|		\| gitlab registry secrets \|\| \|\| gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml \|\| \|\|
	\|-
	~~\| [[Quay Container Registry]] \|\| k8s/quay \|\| \|\| \|\| quay.williams.localnet <br/> (CNAME to lamppost.williams.localnet)~~
	\|-		\|-
	\| rsyslog\|\| k8s/rsyslog \|\| \|\| 10.0.0.113 \|\| rsyslog.williams.localnet		\| rsyslog\|\| k8s/rsyslog \|\| \|\| 10.0.0.113 \|\| rsyslog.williams.localnet

DrEdWilliams at 00:22, 27 August 2020

2020-08-27T00:22:04Z

← Older revision		Revision as of 00:22, 27 August 2020
Line 25:		Line 25:
	\| wordpress (dredwilliams.com) \|\| k8s/dredwilliams \|\| \|\| \|\| dredwilliams.williams-net.org		\| wordpress (dredwilliams.com) \|\| k8s/dredwilliams \|\| \|\| \|\| dredwilliams.williams-net.org
	\|-		\|-
	\| mediawiki \|\| ~~k8s/~~mediawiki \|\| \|\| 10.0.0.116 \|\| wiki.williams.localnet <br />wiki.williams-net.org		\| mediawiki \|\| mediawiki \|\| \|\| 10.0.0.116 \|\| wiki.williams.localnet <br />wiki.williams-net.org
	\|-		\|-
	\| [[MariaDB]] \|\| ~~k8s/~~mariadb \|\| \|\| 10.0.0.117 \|\| database.williams.localnet		\| [[MariaDB]] \|\| mariadb \|\| \|\| 10.0.0.117 \|\| database.williams.localnet
			\|-
			\| [[Rocket.Chat]] \|\| rocketchat \|\| \|\| (contour) \|\| rocket.williams-net.org
	\|}		\|}