Development Cluster Configuration: Difference between revisions

From WilliamsNet Wiki
Jump to navigation Jump to search
mNo edit summary
Tag: visualeditor
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
These packages form the basic functionality of the development cluster.  The packages need to be installed in this order to preserve the IP address assignments.  If the order is changed (or one is left out) you may need to review and modify IP assignments in the DNS server on the firewall.
The Development cluster is deployed using [[K3s - Kubernetes Simplified]].
 
These packages form the basic functionality of the development cluster.  The packages need to preserve the IP address assignments.   


Scripts & config files are checked into gitlab under the Kubernetes group project listed.
Scripts & config files are checked into gitlab under the Kubernetes group project listed.
Line 6: Line 8:
|-
|-
! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)
! activity !! gitlab !! script/procedures/config !! IP !! hostname(s)
|-
| [[BeeGFS Installation]] || install the parallel filesystem components on controller & nodes to support the /workspace filesystem || || ||
|-
|-
| NVIDIA device plugin || || https://github.com/NVIDIA/k8s-device-plugin || ||
| NVIDIA device plugin || || https://github.com/NVIDIA/k8s-device-plugin || ||
|-
|-
| K8Dash Dashboard || k8s-admin || || 10.0.0.200 ||
| GitLab - Helm deployment || [https://gitlab.dev.williams.localnet/admin/projects/k8s/gitlab Kubernetes/gitlab] || kubernetes/gitlab/helm || 10.0.0.203 || gitlab.dev.williams.localnet
|-
| [[Dynamic Provisioning]] || k8s-admin || (k8s-admin wiki) || ||
|-
| [[Harbor Registry]] || k8s-admin || || 10.0.0.201 || harbor-dev.williams.localnet
|}
|}


=== Storage ===
=== Storage ===
The production cluster depends on the '''/workspace''' filesystem for its persistent storage.  The BeeGFS components are installed as shown here:
The production cluster uses Rook/Ceph for its persistent storage.  The storage components are arranged:
{| class="wikitable"
{| class="wikitable"
|-
|-
! component !! system !! location !! storage !! size
! component !! system !! storageclass !! storage !! size
|-
| Management Server || controller || /beegfs/beegfs-mgmtd || local SSD || ~200G (shared with OS)
|-
| Metadata Server || controller || /beegfs/beegfs-meta || local SSD || ~200G (shared with OS)
|-
| Storage Server || controller || /ws_data/beegfs/beegfs-storage || mounted from Equalogic array || 7.9T
|-
|-
| Storage Server || storage1 || /ws_data_2 || local 5x4TB RAID5 array || 15T
| Storage Server || storage1 || rook-ceph || local 5x4TB drives || 20TB (ceph)
|}
|}
Systems that require access to both the development filesystem ('''/workspace''') and the production filesystem ('''/shared''') require a [[BeeGFS Installation#Mounting multiple filesystems on the same client|special client configuration]].
=== Dashboard Token ===
Obtain the current dashboard token with these commands:
kubectl create serviceaccount k8dash-sa
kubectl create clusterrolebinding k8dash-sa --clusterrole=cluster-admin --serviceaccount=default:k8dash-sa
kubectl describe secrets `kubectl get secrets | awk '/k8dash-sa/ {print $1}'` | awk '/token:/ {print $2}'
The current Development cluster dashboard token is:
    eyJhbGciOiJSUzI1NiIsImtpZCI6Ims0NGhxMmxFZnVSUTBnYVA4VXdta29KeFJVRjRUQkdSUWs3X1NKQ1RLTU0ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6Ims4ZGFzaC1zYS10b2tlbi1oZGo5cCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrOGRhc2gtc2EiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzMjk0MzdhZS1kMmJjLTQwZTAtODY3YS1hZmZmNTE0ZTNhMmYiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDprOGRhc2gtc2EifQ.BlsyyRHIeTeJWv3x7JPuxix0xGJqg5YYozxsvPcbTcHV_TXw7T6zE_oeizIuKietyr2_RnXa-523OyhLoZo448NDK0K5XLX5sRqIbGYkDFH4AnawADTkBNPVRbRZvaBr7R8BGurRAWcDLBp0sI6D4PhZSk7O_evrrtmqOTo6KRO7fApg58OC5SpDKoJq8SpFASJNOKJs1KGURP8_tFvstsLkOwJj25uErO6Qt87mNFyWQ2r2Ta-QFtS43kTexp4FeOZAShRfUlmhJO9HWPyl-ES7HrnWpVSd9H8w6AN-dXxF-P_W6xCAF8jV4GqFIYozSrimRikRitxc23mkrYnXaQ
=== Kubernetes Node Join Command ===
<pre>
kubeadm join 10.1.0.10:6443 --token k7ujoy.bsgp5bsdkl3mybhw \
    --discovery-token-ca-cert-hash sha256:76ab40add23981ca5d86396fc81c267b50084b0fbd28fc389a4f53c5d9a2cbf0 \
    --ignore-preflight-errors Swap --node-name=`hostname -s`
</pre>

Latest revision as of 11:26, 29 October 2021

The Development cluster is deployed using K3s - Kubernetes Simplified.

These packages form the basic functionality of the development cluster. The packages need to preserve the IP address assignments.

Scripts & config files are checked into gitlab under the Kubernetes group project listed.

activity gitlab script/procedures/config IP hostname(s)
NVIDIA device plugin https://github.com/NVIDIA/k8s-device-plugin
GitLab - Helm deployment Kubernetes/gitlab kubernetes/gitlab/helm 10.0.0.203 gitlab.dev.williams.localnet

Storage[edit]

The production cluster uses Rook/Ceph for its persistent storage. The storage components are arranged:

component system storageclass storage size
Storage Server storage1 rook-ceph local 5x4TB drives 20TB (ceph)
Retrieved from "http://wiki.williams-net.org/index.php?title=Development_Cluster_Configuration&oldid=592"