nplus/samples/static/README.md

# Static Volumes

## Assigning PVs

For security reasons, you might want to use a storage class that does not perform automatic provisioning of PVs.
In that case, you want to reference a pre-created volume in the PVC.
In nplus, you can do so by setting the volumeName in the values.

Please review `values.yaml` as an example:

```yaml
database:
  mounts:
    data:
      volumeName: "pv-{{ .component.fullName }}-data"
nstl:
  mounts:
    data:
      volumeName: "pv-{{ .component.fullName }}-data"
```

You can also set the environment config volume. Please refer to the environment documentation for that.

```
helm install \
  --values samples/environment/demo.yaml \
  --values samples/static/values.yaml
  sample-static nplus/nplus-instance
```

## Creating PVs

https://github.com/ceph/ceph-csi/blob/devel/docs/static-pvc.md

### Data Disk:

1. Create a pool on your cep cluster
   ```
   ceph osd pool create k-lab 64 64
   ```
2. Create a block device pool
   ```
   rbd pool init k-lab
   ```
3. Create an image
   ```
   rbd create -s 50G k-lab/pv-sample-static-database-data
   rbd create -s 50G k-lab/pv-sample-static-nstl-data
   rbd ls k-lab | grep pv-sample-static-
   ```
   Resize:
   ```
   rbd resize --size 50G k-lab/pv-no-provisioner-database-data --allow-shrink
   ```

### File Share:

1. Create a Subvolume (FS)
   ```
   ceph fs subvolume create cephfs pv-no-provisioner-rs-file --size 53687091200
   ```
2. Get the path of the subvolume
   ```
   ceph fs subvolume getpath  cephfs pv-no-provisioner-rs-file 
   ```

### Troubleshooting

```
kubectl describe pv/pv-no-provisioner-rs-file pvc/no-provisioner-rs-file
kubectl get volumeattachment
```

### PV Manifests

```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-no-provisioner-database-data
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 50Gi
  csi:
    driver: rook-ceph.rbd.csi.ceph.com
    fsType: ext4
    nodeStageSecretRef:
      # node stage secret name
      name: rook-csi-rbd-node
      # node stage secret namespace where above secret is created
      namespace: rook-ceph-external
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph-external
      pool: k-lab
      staticVolume: "true"
      imageFeatures: layering
      #mounter: rbd-nbd
    # volumeHandle should be same as rbd image name
    volumeHandle: pv-no-provisioner-database-data
  persistentVolumeReclaimPolicy: Retain
  # The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
  volumeMode: Filesystem
  storageClassName: ceph-rbd
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-no-provisioner-nstl-data
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 50Gi
  csi:
    driver: rook-ceph.cephfs.csi.ceph.com
    fsType: ext4
    nodeStageSecretRef:
      # node stage secret name
      name: rook-csi-rbd-node
      # node stage secret namespace where above secret is created
      namespace: rook-ceph-external
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph-external
      pool: k-lab
      staticVolume: "true"
      imageFeatures: layering
      #mounter: rbd-nbd
    # volumeHandle should be same as rbd image name
    volumeHandle: pv-no-provisioner-nstl-data
  persistentVolumeReclaimPolicy: Retain
  # The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
  volumeMode: Filesystem
  storageClassName: ceph-rbd
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-no-provisioner-rs-file
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 50Gi
  csi:
    driver: cephfs.csi.ceph.com
    nodeStageSecretRef:
      name: rook-csi-cephfs-secret
      #rook-csi-cephfs-node
      namespace: rook-ceph-external
    volumeAttributes:
      # Required options from storageclass parameters need to be added in volumeAttributes
      clusterID: rook-ceph-external
      fsName: cephfs
      pool: cephfs_data
      staticVolume: "true"
      # rootPath kriegt man per ceph fs subvolume getpath  cephfs pv-no-provisioner-rs-file 
      rootPath: "/volumes/_nogroup/pv-no-provisioner-rs-file/3016f512-bc19-4bfb-8eb2-5118430fbbe5"
      #mounter: rbd-nbd
    # volumeHandle should be same as rbd image name
    volumeHandle: pv-no-provisioner-rs-file
  persistentVolumeReclaimPolicy: Retain
  # The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
  volumeMode: Filesystem
  storageClassName: cephfs
```