Production Cluster Configuration: Difference between revisions

From WilliamsNet Wiki
Jump to navigation Jump to search
(Created page with "These packages form the basic functionality of the production cluster. The packages need to be installed in this order to preserve the IP address assignments. If the order i...")
 
mNo edit summary
 
(36 intermediate revisions by the same user not shown)
Line 1: Line 1:
These packages form the basic functionality of the production cluster.  The packages need to be installed in this order to preserve the IP address assignments.  If the order is changed (or one is left out) you may need to review and modify IP assignments in the DNS server on the firewall.
These packages form the basic functionality of the production cluster.   


Scripts & config files are checked into gitlab under the Kubernetes group project listed.
Scripts & config files are checked into gitlab under the Kubernetes group project listed.
Line 7: Line 7:
! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)
! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)
|-
|-
| [[NGINX-ingress]] || k8s-admin || || 10.0.0.111 ||
| [[Ceph Storage Cluster]] || k8s-admin || || ||
|-
|-
| gitlab || gitlab || || 10.0.0.112 || gitlab.williams.localnet <br />gitlab.williams-net.org
| [[Rook Storage for Kubernetes|Rook Storage]] || k8s-admin || || || (StorageClass)<br/>rook-ceph-hdd<br/>rook-ceph-nvme
|-
|-
| gitlab registry secrets || || gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml || ||
| gitlab registry secrets || || gitlab-registry-kube-system.yaml <br /> gitlab-registry-secret.yaml || ||
|-
|-
| rsyslog|| rsyslog || || 10.0.0.113 || rsyslog.williams.localnet
| wordpress (dredwilliams.com) || k8s/dredwilliams || || || dredwilliams.williams-net.org
|-
|-
| mail || mail || || 10.0.0.114 || mail.williams.localnet
| mediawiki || mediawiki || || 10.0.0.116 || wiki.williams.localnet <br />wiki.williams-net.org
|-
| [[MariaDB]] || mariadb || || 10.0.0.117 || database.williams.localnet
|}
 
=== Storage ===
The production cluster depends on the '''/shared''' filesystem for its persistent storage as provided by the production Ceph cluster.  The Ceph is configured as shown here:
{| class="wikitable"
|-
|-
| wordpress (dredwilliams.com) || dredwilliams || || || dredwilliams.williams-net.org
! system !! function !! storage !! size
|-
|-
| harbor || k8s-admin || || 10.0.0.115 || harbor.williams.localnet
| caspian || master || NVMe<br/>HDD || 1TB<br/>1TB
|-
|-
| mediawiki || mediawiki || || 10.0.0.116 || wiki.williams.localnet <br />wiki.williams-net.org
| uvilas || node || NVMe<br/>HDD<br/>HDD || 1TB<br/>1TB<br/>1TB
|-
| belisar || node || NVMe<br/>HDD || 1TB<br/>250GB
|}
|}


=== Dashboard Token ===
The work filesystem can be mounted via NFS:
Obtain the token needed to log into the dashboard with this command:
 
kubectl -n kube-system describe secrets \
10.0.0.75:/work /work nfs4 soft 0 0
    `kubectl -n kube-system get secrets | awk '/clusterrole-aggregation-controller/ {print $1}'` \
 
    | awk '/token:/ {print $2}'   
=== Backups ===
The current token for the Production cluster is:
 
  eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjbHVzdGVycm9sZS1hZ2dyZWdhdGlvbi1jb250cm9sbGVyLXRva2VuLWs2MnFkIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImNsdXN0ZXJyb2xlLWFnZ3JlZ2F0aW9uLWNvbnRyb2xsZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIwNDE2ZTdlYi0wYWZlLTQ2ZmYtOTY4NS02YzEwMmU1NTcxMjIiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06Y2x1c3RlcnJvbGUtYWdncmVnYXRpb24tY29udHJvbGxlciJ9.Jv2EvhmCB3Qp_Lg8nyzDE69DhmSSk59Ewk0Xu0zROLMR0MFqOVQJCSBGQ4WV8VUi0Nq70txdO0vAFkSjo_0tcB_JA1-1615_lXLQ9RbNjIGr9AuquYjEV7GJ4yaBIhhDju7qRGTLjzB9lXlm7DrI73EduKE-PGiUq7y2oHGuSMUce8S2vh7LattV8Ud6hPbz_J9pNVqG9oXG4WVfn8-HoG19UDSgUyRj01h69mABzTcjPIWenAQnpqHYq-oMlvwHEUQ78I41YaKwI-9YBM2AP0ICSRrIcZY2GpoRmm90t9paBgLbESZYGXysJwzWSlJO3jwzQN9wpkCAnIbAV_TlzQ
In addition to the normal backups configured in the basic OS installation steps, the databases in the production cluster must be backed up daily using the 'mysqldump' command:
 
  TBD


=== Kubernetes Node Join Command ===
These commands should be inserted into the /etc/cron.daily/backup file on one of the cluster nodes (telmar is a good choice). The first does a complete database dump of the MediaWiki database server, the second dumps just the mediawiki database itself, and the third dumps the general purpose database server. Additional dump commands should be inserted for additional significant databases, as parsing individual databases out of a system dump can be tedious.
kubeadm join 10.0.0.9:6443 --token 81z3dx.aiy9j7nsdcizf3lz \
    --discovery-token-ca-cert-hash sha256:f5db5ecd247bed235c49675f01c5a1cccb793cd8a25db590f71b64a1d73fc5b6 \
    --ignore-preflight-errors Swap

Latest revision as of 23:09, 14 September 2024

These packages form the basic functionality of the production cluster.

Scripts & config files are checked into gitlab under the Kubernetes group project listed.

activity gitlab script/procedures/config IP hostname(s)
Ceph Storage Cluster k8s-admin
Rook Storage k8s-admin (StorageClass)
rook-ceph-hdd
rook-ceph-nvme
gitlab registry secrets gitlab-registry-kube-system.yaml
gitlab-registry-secret.yaml
wordpress (dredwilliams.com) k8s/dredwilliams dredwilliams.williams-net.org
mediawiki mediawiki 10.0.0.116 wiki.williams.localnet
wiki.williams-net.org
MariaDB mariadb 10.0.0.117 database.williams.localnet

Storage[edit]

The production cluster depends on the /shared filesystem for its persistent storage as provided by the production Ceph cluster. The Ceph is configured as shown here:

system function storage size
caspian master NVMe
HDD		 1TB
1TB
uvilas node NVMe
HDD
HDD		 1TB
1TB
1TB
belisar node NVMe
HDD		 1TB
250GB

The work filesystem can be mounted via NFS: 

10.0.0.75:/work /work nfs4 soft 0 0

Backups[edit]

In addition to the normal backups configured in the basic OS installation steps, the databases in the production cluster must be backed up daily using the 'mysqldump' command: 

TBD

These commands should be inserted into the /etc/cron.daily/backup file on one of the cluster nodes (telmar is a good choice). The first does a complete database dump of the MediaWiki database server, the second dumps just the mediawiki database itself, and the third dumps the general purpose database server. Additional dump commands should be inserted for additional significant databases, as parsing individual databases out of a system dump can be tedious.

Retrieved from "http://wiki.williams-net.org/index.php?title=Production_Cluster_Configuration&oldid=703"