Editing Rook Storage for Kubernetes (section)

=== Storage Classes ===
Rook will allow us to create StorageClasses for the different APIs provided by Ceph.  The most common one is for RBD (block storage), but you can also use Ceph Filesystems.

==== RBD Storage Class ====
Next we create the StorageClass that kubernetes will use to request RBDs (block images) from the ceph cluster.  The starting point is the manifest '''''csi/rbd/storageclass.yaml''''' in the rook distribution.  Copy it to a local directory and customize it as described below.

The manifest creates two resources:  A CephPool resource that will be used to create a new pool (or use an existing pool) in the ceph cluster, and a StorageClass entry that will allow pods to request storage.  Customize this file as follows:
* change 'failureDomain' in the pool definition from 'host' to 'osd' ... this will enable replication within a host, which is necessary for a small cluster
* change the 'name' in the pool definition and the corresponding 'pool' in the storage class to something descriptive -- this will be the name of the pool that is created in the ceph cluster.
* set the namespace (if it isn't set already) to 'rook-ceph' to match whatever was used to create the cluster
When done, install the manifest:  

 kubectl create -f storageclass.yaml

At this point, the Storage Provisioner is ready.

==== Ceph Filesystem Storage Class ====
There are limitations on using Ceph Filesystems within rook ...

instructions TBD

==== Default Storage Class (Optional) ====
Not all manifests specify a storage class -- this can be especially problematic with helm charts that don't expose the opportunity to specify a storage class.  Kubernetes has the concept of a '''default''' storage class that is widely used by the cloud providers to point to their preferred storage solution.  While not required, specifying a Rook StorageClass as default can simplify 'automatic' deployments.  In reality, all that needs to be done is to set a flag on the storage class that you want to be default ... but there's some prep work ...

First, identify all the defined storage classes:

 kubectl get sc

All the storage classes will be shown -- if one of them is already defined as a 'default' class, it will have '''(default)''' after the name.  If there is a default class identified (and it isn't yours) you need to turn off the default status for that class (i.e. setting a new default does NOT reset the old default):

 kubectl patch storageclass <old-default> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

Then you can set your storageclass as the default:

 kubectl patch storageclass <new-default> -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Validate that it took by looking at the storageclasses again:

 kubectl get sc

Your class should now be marked '(default)'.

(excerpted from https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/)