nplus/samples/static

Static Volumes

Assigning PVs

For security reasons, you might want to use a storage class that does not perform automatic provisioning of PVs. In that case, you want to reference a pre-created volume in the PVC. In nplus, you can do so by setting the volumeName in the values.

Please review values.yaml as an example:

database:
  mounts:
    data:
      volumeName: "pv-{{ .component.fullName }}-data"
nstl:
  mounts:
    data:
      volumeName: "pv-{{ .component.fullName }}-data"

You can also set the environment config volume. Please refer to the environment documentation for that.

helm install \
  --values samples/environment/demo.yaml \
  --values samples/static/values.yaml
  sample-static nplus/nplus-instance

Creating PVs

https://github.com/ceph/ceph-csi/blob/devel/docs/static-pvc.md

Data Disk:

1. Create a pool on your cep cluster

   ceph osd pool create k-lab 64 64
   

2. Create a block device pool

   rbd pool init k-lab
   

3. Create an image

   rbd create -s 50G k-lab/pv-sample-static-database-data
   rbd create -s 50G k-lab/pv-sample-static-nstl-data
   rbd ls k-lab | grep pv-sample-static-
   

   Resize:

   rbd resize --size 50G k-lab/pv-no-provisioner-database-data --allow-shrink
   

File Share:

1. Create a Subvolume (FS)

   ceph fs subvolume create cephfs pv-no-provisioner-rs-file --size 53687091200
   

2. Get the path of the subvolume

   ceph fs subvolume getpath  cephfs pv-no-provisioner-rs-file 
   

Troubleshooting

kubectl describe pv/pv-no-provisioner-rs-file pvc/no-provisioner-rs-file
kubectl get volumeattachment

PV Manifests

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-no-provisioner-database-data
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 50Gi
  csi:
    driver: rook-ceph.rbd.csi.ceph.com
    fsType: ext4
    nodeStageSecretRef:
      # node stage secret name
      name: rook-csi-rbd-node
      # node stage secret namespace where above secret is created
      namespace: rook-ceph-external
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph-external
      pool: k-lab
      staticVolume: "true"
      imageFeatures: layering
      #mounter: rbd-nbd
    # volumeHandle should be same as rbd image name
    volumeHandle: pv-no-provisioner-database-data
  persistentVolumeReclaimPolicy: Retain
  # The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
  volumeMode: Filesystem
  storageClassName: ceph-rbd
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-no-provisioner-nstl-data
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 50Gi
  csi:
    driver: rook-ceph.cephfs.csi.ceph.com
    fsType: ext4
    nodeStageSecretRef:
      # node stage secret name
      name: rook-csi-rbd-node
      # node stage secret namespace where above secret is created
      namespace: rook-ceph-external
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph-external
      pool: k-lab
      staticVolume: "true"
      imageFeatures: layering
      #mounter: rbd-nbd
    # volumeHandle should be same as rbd image name
    volumeHandle: pv-no-provisioner-nstl-data
  persistentVolumeReclaimPolicy: Retain
  # The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
  volumeMode: Filesystem
  storageClassName: ceph-rbd
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-no-provisioner-rs-file
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 50Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: rook-csi-cephfs-secret
      #rook-csi-cephfs-node
      namespace: rook-ceph-external
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph-external
      fsName: cephfs
      pool: cephfs_data
      staticVolume: "true"
      # rootPath kriegt man per ceph fs subvolume getpath  cephfs pv-no-provisioner-rs-file 
      rootPath: "/volumes/_nogroup/pv-no-provisioner-rs-file/3016f512-bc19-4bfb-8eb2-5118430fbbe5"
      #mounter: rbd-nbd
    # volumeHandle should be same as rbd image name
    volumeHandle: pv-no-provisioner-rs-file
  persistentVolumeReclaimPolicy: Retain
  # The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
  volumeMode: Filesystem
  storageClassName: cephfs