public/nplus
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Public Information

Browse Source
This commit is contained in:
Andreas Ahmann
2025-01-24 16:18:47 +01:00
commit 0bd2038c86
449 changed files with 108655 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

169
samples/static/README.md Normal file
View File

@@ -0,0 +1,169 @@
# Static Volumes
## Assigning PVs
For security reasons, you might want to use a storage class that does not perform automatic provisioning of PVs.
In that case, you want to reference a pre-created volume in the PVC.
In nplus, you can do so by setting the volumeName in the values.
Please review `values.yaml` as an example:
```yaml
database:
mounts:
data:
volumeName: "pv-{{ .component.fullName }}-data"
nstl:
mounts:
data:
volumeName: "pv-{{ .component.fullName }}-data"
```
You can also set the environment config volume. Please refer to the environment documentation for that.
```
helm install \
--values samples/environment/demo.yaml \
--values samples/static/values.yaml
sample-static nplus/nplus-instance
```
## Creating PVs
https://github.com/ceph/ceph-csi/blob/devel/docs/static-pvc.md
### Data Disk:
1. Create a pool on your cep cluster
```
ceph osd pool create k-lab 64 64
```
2. Create a block device pool
```
rbd pool init k-lab
```
3. Create an image
```
rbd create -s 50G k-lab/pv-sample-static-database-data
rbd create -s 50G k-lab/pv-sample-static-nstl-data
rbd ls k-lab | grep pv-sample-static-
```
Resize:
```
rbd resize --size 50G k-lab/pv-no-provisioner-database-data --allow-shrink
```
### File Share:
1. Create a Subvolume (FS)
```
ceph fs subvolume create cephfs pv-no-provisioner-rs-file --size 53687091200
```
2. Get the path of the subvolume
```
ceph fs subvolume getpath cephfs pv-no-provisioner-rs-file
```
### Troubleshooting
```
kubectl describe pv/pv-no-provisioner-rs-file pvc/no-provisioner-rs-file
kubectl get volumeattachment
```
### PV Manifests
```yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-no-provisioner-database-data
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 50Gi
csi:
driver: rook-ceph.rbd.csi.ceph.com
fsType: ext4
nodeStageSecretRef:
# node stage secret name
name: rook-csi-rbd-node
# node stage secret namespace where above secret is created
namespace: rook-ceph-external
volumeAttributes:
# Required options from storageclass parameters need to be added in volumeAttributes
clusterID: rook-ceph-external
pool: k-lab
staticVolume: "true"
imageFeatures: layering
#mounter: rbd-nbd
# volumeHandle should be same as rbd image name
volumeHandle: pv-no-provisioner-database-data
persistentVolumeReclaimPolicy: Retain
# The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
volumeMode: Filesystem
storageClassName: ceph-rbd
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-no-provisioner-nstl-data
spec:
accessModes:
- ReadWriteOnce
capacity:
storage: 50Gi
csi:
driver: rook-ceph.cephfs.csi.ceph.com
fsType: ext4
nodeStageSecretRef:
# node stage secret name
name: rook-csi-rbd-node
# node stage secret namespace where above secret is created
namespace: rook-ceph-external
volumeAttributes:
# Required options from storageclass parameters need to be added in volumeAttributes
clusterID: rook-ceph-external
pool: k-lab
staticVolume: "true"
imageFeatures: layering
#mounter: rbd-nbd
# volumeHandle should be same as rbd image name
volumeHandle: pv-no-provisioner-nstl-data
persistentVolumeReclaimPolicy: Retain
# The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
volumeMode: Filesystem
storageClassName: ceph-rbd
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-no-provisioner-rs-file
spec:
accessModes:
- ReadWriteMany
capacity:
storage: 50Gi
csi:
driver: cephfs.csi.ceph.com
nodeStageSecretRef:
name: rook-csi-cephfs-secret
#rook-csi-cephfs-node
namespace: rook-ceph-external
volumeAttributes:
# Required options from storageclass parameters need to be added in volumeAttributes
clusterID: rook-ceph-external
fsName: cephfs
pool: cephfs_data
staticVolume: "true"
# rootPath kriegt man per ceph fs subvolume getpath cephfs pv-no-provisioner-rs-file
rootPath: "/volumes/_nogroup/pv-no-provisioner-rs-file/3016f512-bc19-4bfb-8eb2-5118430fbbe5"
#mounter: rbd-nbd
# volumeHandle should be same as rbd image name
volumeHandle: pv-no-provisioner-rs-file
persistentVolumeReclaimPolicy: Retain
# The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
volumeMode: Filesystem
storageClassName: cephfs
```

48
samples/static/build.sh Executable file
View File

@@ -0,0 +1,48 @@
#!/bin/bash
#
# This sample script builds the example as described. It is also used to build the test environment in our lab,
# so it should be well tested.
#
# Make sure it fails immediately, if anything goes wrong
set -e
# -- ENVironment variables:
# CHARTS: The path to the source code
# DEST: The path to the build destination
# SAMPLE: The directory of the sample
# NAME: The name of the sample, used as the .Release.Name
# KUBE_CONTEXT: The name of the kube context, used to build this sample depending on where you run it against. You might have different Environments such as lab, dev, qa, prod, demo, local, ...
# Check, if we have the source code available
if [ ! -d "$CHARTS" ]; then
echo "ERROR Building $SAMPLE example: The Charts Sources folder is not set. Please make sure to run this script with the full Source Code available"
exit 1
fi
if [ ! -d "$DEST" ]; then
echo "ERROR Building $SAMPLE example: DEST folder not found."
exit 1
fi
if [ ! -d "$CHARTS/instance" ]; then
echo "ERROR Building $SAMPLE example: Chart Sources in $CHARTS/instance not found. Are you running this script as a subscriber?"
exit 1
fi
# Set the Variables
SAMPLE="static"
NAME="sample-$SAMPLE"
# Output what is happening
echo "Building $NAME"
# Create the manifest
mkdir -p $DEST/instance
helm template --debug \
--values $SAMPLES/application/empty.yaml \
--values $SAMPLES/environment/$KUBE_CONTEXT.yaml \
--values $SAMPLES/resources/$KUBE_CONTEXT.yaml \
--values $SAMPLES/static/values.yaml \
$NAME $CHARTS/instance > $DEST/instance/$SAMPLE.yaml
# Adding the static PV to the manifest
echo -e "\n---\n" >> $DEST/instance/$SAMPLE.yaml
cat $SAMPLES/static/pv.yaml >> $DEST/instance/$SAMPLE.yaml

74
samples/static/pv.yaml Normal file
View File

@@ -0,0 +1,74 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-sample-static-database-data
spec:
# -- set an empty string must be explicitly set otherwise default StorageClass will be set
# see https://kubernetes.io/docs/concepts/storage/persistent-volumes/
storageClassName: ""
# -- make sure, this PV may only by bound to a specific claim
claimRef:
name: sample-static-database-data
namespace: lab
accessModes:
- ReadWriteOnce
capacity:
storage: 50Gi
csi:
driver: rook-ceph.rbd.csi.ceph.com
fsType: ext4
nodeStageSecretRef:
# node stage secret name
name: rook-csi-rbd-node
# node stage secret namespace where above secret is created
namespace: rook-ceph-external
volumeAttributes:
# Required options from storageclass parameters need to be added in volumeAttributes
clusterID: rook-ceph-external
pool: k-lab
staticVolume: "true"
imageFeatures: layering
#mounter: rbd-nbd
# volumeHandle should be same as rbd image name
volumeHandle: pv-sample-static-database-data
persistentVolumeReclaimPolicy: Delete
# The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
volumeMode: Filesystem
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-sample-static-nstl-data
spec:
# -- set an empty string must be explicitly set otherwise default StorageClass will be set
# see https://kubernetes.io/docs/concepts/storage/persistent-volumes/
storageClassName: ""
# -- make sure, this PV may only by bound to a specific claim
claimRef:
name: sample-static-nstl-data
namespace: lab
accessModes:
- ReadWriteOnce
capacity:
storage: 50Gi
csi:
driver: rook-ceph.rbd.csi.ceph.com
fsType: ext4
nodeStageSecretRef:
# node stage secret name
name: rook-csi-rbd-node
# node stage secret namespace where above secret is created
namespace: rook-ceph-external
volumeAttributes:
# Required options from storageclass parameters need to be added in volumeAttributes
clusterID: rook-ceph-external
pool: k-lab
staticVolume: "true"
imageFeatures: layering
#mounter: rbd-nbd
# volumeHandle should be same as rbd image name
volumeHandle: pv-sample-static-nstl-data
persistentVolumeReclaimPolicy: Delete
# The volumeMode can be either `Filesystem` or `Block` if you are creating Filesystem PVC it should be `Filesystem`, if you are creating Block PV you need to change it to `Block`
volumeMode: Filesystem

22
samples/static/values.yaml Normal file
View File

@@ -0,0 +1,22 @@
database:
mounts:
data:
volumeName: "pv-{{ .component.fullName }}-data"
nstl:
mounts:
data:
volumeName: "pv-{{ .component.fullName }}-data"
# # mon:
# # mounts:
# # data:
# # volumeName: "pv-{{ .component.fullName }}-data"
# # pipeliner:
# # mounts:
# # data:
# # volumeName: "pv-{{ .component.fullName }}-data"
# # rs:
# # mounts:
# # file:
# # volumeName: "pv-{{ .component.fullName }}-file"