247 lines
20 KiB
Markdown
247 lines
20 KiB
Markdown
|
|
|
|
# nplus-component-nstl
|
|
|
|
nscale Server Storage Layer, virtualizing the storage to be used by the nscale Server
|
|
|
|
## Server Ids
|
|
|
|
if you use nstlHA, both server must have unique server IDs.
|
|
Since the probaly use the same license file, you can set it via
|
|
```
|
|
nstla:
|
|
serverId: 4711
|
|
nstlb:
|
|
serverId: 4712
|
|
```
|
|
to make this work, you **must** have a license without nstl ServerID in it.
|
|
|
|
## High Availability
|
|
|
|
*nscale Server Storage Layer* does not support multiple replicas because of configuration restraints. To get HA,
|
|
nplus defines nstl[a-d].
|
|
|
|
If you want multiple server, just switch the single server *nstl* off and turn on *nstla* and *nstlb* [..] and configure them individually. You will find individual config directories on the `conf` shared volume for each nstl instance.
|
|
|
|
## Cluster Service
|
|
|
|
in HA mode, the Instance Chart deploys a cluster service that balances across nstl A and B.
|
|
You **can** use it if you want, or you use the services for nstla and nstlb and load balance via nappl.
|
|
Both is ok, it just uses different load balancers.
|
|
|
|
## Remote Servers
|
|
|
|
If you want to connect to nscale Storage Layer servers outside the cluster, you can add the IP Range of those
|
|
server to open the firewall (if used):
|
|
|
|
```
|
|
nstla:
|
|
security:
|
|
cni:
|
|
nstlIpRange: 89.207.132.170/24
|
|
```
|
|
|
|
## Accounting
|
|
|
|
You can enable accouting by setting `accounting: true`. This will create the accounting csv files in *ptemp* under `<instance>/<component>/accounting`.
|
|
|
|
Additionally, a log forwarder can be implemented to output the accounting log to stdout, so that your log aggregator is able to collect that information and add
|
|
it to whatever tool you use for log monitoring / analysis.
|
|
|
|
You can *optionally* set the db path, where the logForwarder stores the information which log entry has been collected. If you do not set this, the default value is `logs.db` in the path of the log Pattern (of the `path`value).
|
|
|
|
Example: (in Instance Chart)
|
|
|
|
```
|
|
nstl:
|
|
accounting: true
|
|
logForwarder:
|
|
- name: Accounting
|
|
path: "/opt/ceyoniq/nscale-server/storage-layer/accounting/*.csv"
|
|
db: "/opt/ceyoniq/nscale-server/storage-layer/logsdb/logs.db"
|
|
```
|
|
|
|
## The DA_HID.DAT
|
|
|
|
the *nscale Server Storage Layer* offers a safety net, storing the latest ever assigned highest Document ID into a file called `DA_HID.DAT`.
|
|
This file normally resides next to the Document Administration. However, in a production environment, you want to place this file on a separate disk to build a safety net
|
|
for the case that you restore the DA to a former version so the counter starts assigning a document ID twice. So in *nappl*, you end up having documents with the same storage layer doc id.
|
|
|
|
This must not happen, so a good practise is to store the `DA_HID.DAT` on a second disk and have the nstl check it during startup.
|
|
This is not the default, as it would not make a lot of sense until you really have a second disk.
|
|
|
|
To switch this on, you can place the hid file on the secondary disk (like in the example below). The path is mounted by default. Just set it and turn it on like this:
|
|
|
|
```yaml
|
|
# -- enables checking the highest DocID when starting the server.
|
|
# this only makes sense, if you also set a separate volume for the highest ID
|
|
# This is a backup / restore feature to avoid data mangling
|
|
checkHighestDocId: "1"
|
|
# -- sets the path of the highest ID file.
|
|
dvCheckPath: "/opt/ceyoniq/nscale-server/storage-layer/hid"
|
|
```
|
|
|
|
## Updating from Releases < 9.3 to 9.3
|
|
|
|
With version 9.3, nplus introduces a new *disk* volume, to separate the *arc Buffer* and the *retrieval Buffer* from the *data* volume that holds the *DA*. This is necessary, as under heavy load it may result in performance loss because some cloud providers have iop limitations on volumes.
|
|
|
|
To make sure we stay compatible though, there is a new init container that copies the buffers from the old location on *data* over to the new location on *disk*.
|
|
|
|
Please make sure, you set the size, class and optionally the volumeName (in case of static volumes) according to your requirements.
|
|
|
|
## nplus-component-nstl Chart Configuration
|
|
|
|
You can customize / configure nplus-component-nstl by setting configuration values on the command line or in values files,
|
|
that you can pass to helm. Please see the samples directory for details.
|
|
|
|
In case there is no value set, the key will not be used in the manifest, resulting in values taken from the config files of the component.
|
|
|
|
### Template Functions
|
|
|
|
You can use template functions in the values files. If you do so, make sure you quote correctly (single quotes, if you have double quotes in the template,
|
|
or escaped quotes).
|
|
|
|
### Global Values
|
|
|
|
All values can be set per component, per instance or globally per environment.
|
|
|
|
Example: `global.ingress.domain` sets the domain on instance level. You can still set a different domain on a component, such as administrator.
|
|
In that case, simply set `ingress.domain` for the administrator chart and that setting will have priority:
|
|
|
|
- Prio 1 - Component Level: `ingress.domain`
|
|
- Prio 2 - Instance Level: `global.ingress.domain`
|
|
- Prio 3 - Environment Level: `global.environment.ingress.domain`
|
|
|
|
### Using Values in Templates
|
|
|
|
As it would be a lot of typing to write `.Values.ingress.domain | default .Values.global.ingress.domain | default .Values.global.environment.ingress.domain`in your
|
|
template code, this is automatically done by nplus. You can simply type `.this.ingress.domain` and you will get a condensed and defaulted version
|
|
of your Values.
|
|
|
|
So an example in your `values.yaml` would be:
|
|
|
|
```
|
|
administrator:
|
|
waitFor:
|
|
- '-service {{ .component.prefix }}nappljobs.{{ .Release.Namespace }}.svc.cluster.local:\{{ .this.nappl.port }} -timeout 600'
|
|
```
|
|
|
|
This example shows `.this.nappl.port` which might come from a component, instance or global setting. You do not need to care.
|
|
The `.Release.Namespace` is set by helm. You have access to all Release and Chart Metadata, just like in your chart code.
|
|
|
|
The `.component.prefix` is calculated by nplus and gives you some handy shortcuts to internal variables:
|
|
|
|
- `.component.chartName`
|
|
The name of the chart as in `.Chart.Name`, but with override by `.Values.nameOverride`
|
|
- `.component.shortChartName`
|
|
A shorter Version of the name - `nappl` instead of `nplus-component-nappl`
|
|
- `.component.prefix`
|
|
The instance Prefix used to name the resources including `-`. This prefix is dropped, if the
|
|
`.Release.Name` equals `.Release.Namespace` for those of you that only
|
|
run one nplus Instance per namespace
|
|
- `.component.name`
|
|
The name of the component, including `.Values.nameOverride` and some logic
|
|
- `.component.fullName`
|
|
The fullName inlcuding `.Values.fullnameOverride` and some logic
|
|
- `.component.chart`
|
|
Mainly the `Chart.Name` and `Chart.Version`
|
|
- `.component.storagePath`
|
|
The path where the component config is stored in the conf PVC
|
|
- `.component.handler`
|
|
The handler (either helm, argoCD or manual)
|
|
|
|
- `.instance.name`
|
|
The name of the instance, but with override by `.Values.instanceOverride`
|
|
- `.instance.group`
|
|
The group, this instance belongs to. Override by `.Values.groupOverride`
|
|
- `.instance.version`
|
|
The *nscale* version (mostly taken from Application Layer), this instance is deploying.
|
|
|
|
- `.environment.name`
|
|
The name of the environment, but with override by `.Values.environmentNameOverride`
|
|
|
|
### Keys
|
|
|
|
You can set any of the following values for this component:
|
|
|
|
| Key | Description | Default |
|
|
|-----|-------------|---------|
|
|
accounting | sets and enables / disables the accounting function. If enabled, it writes the csv files to *ptemp* (`<instance>/<component>/accounting`) The internal path is set to `/opt/ceyoniq/nscale-server/storage-layer/accounting` by `mounts.ptemp.paths` | |
|
|
checkHighestDocId | enables checking the highest DocID when starting the server. this only makes sense, if you also set a separate volume for the highest ID This is a backup / restore feature to avoid data mangling | |
|
|
dvCheckPath | sets the path of the highest ID file. | |
|
|
env | Sets additional environment variables for the configuration. | |
|
|
envMap | Sets the name of a configMap, which holds additional environment variables for the configuration. It is added as envFrom configMap to the container. | |
|
|
envSecret | Sets the name of a secret, which holds additional environment variables for the configuration. It is added as envFrom secretRef to the container. | |
|
|
fullnameOverride | This overrides the output of the internal fullname function | |
|
|
**image**​.name | the name of the image to use | `"storage-layer"` |
|
|
**image**​.pullSecrets | you can provide your own pullSecrets, in case you use a private repo. | `["nscale-cr", "nplus-cr"]` |
|
|
**image**​.repo | if you use a private repo, feel free to set it here | `"ceyoniq.azurecr.io/release/nscale"` |
|
|
**image**​.tag | the tag of the image to use | `"latest"` |
|
|
logForwarder | | |
|
|
**meta**​.language | Sets the language of the main service (in the *service* container). This is used for instance if you turn OpenTelemetry on, to know which Agent to inject into the container. | `"cpp"` |
|
|
**meta**​.ports​.tcp | A potential tcp port, this component uses (if any) <br>this is a constant value of the component and should not be changed. | **info only**, do not change<br> `3005` |
|
|
**meta**​.ports​.tcps | A potential tls / tcps port, this component uses (if any) <br>this is a constant value of the component and should not be changed. | **info only**, do not change<br> `3006` |
|
|
**meta**​.provider | sets provider (partner, reseller) information to be able to invoice per use in a cloud environment | |
|
|
**meta**​.serviceContainer | The container name of the main service for this component. This is used to define where to inject the telemetry agents, if any | `"storage-layer"` |
|
|
**meta**​.stage | A optional parameter to indicate the stage (DEV, QA, PROD, ...) this component, instance or environment runs in. This can be used in template functions to add the stage to for instance the service name of telemetry services like open telemetry. (see telemetry example) | |
|
|
**meta**​.tenant | sets tenant information to be able to invoice per use in a cloud environment | |
|
|
**meta**​.type | the type of the component. You should not change this value, except if you use a pipeliner in core mode. In core mode, it should be *core*, else *pipeliner* This type is used to create cluster communication for nappl and nstl and potentially group multiple replicaSets into one service. | `"nstl"` |
|
|
**meta**​.wave | Sets the wave in which this component should be deployed within an ArgoCD deployment if unset, it uses the default wave thus all components are installed in one wave, then relying on correct wait settings just like in a helm installation | |
|
|
minReplicaCountType | if you set minReplicaCountType, a podDesruptionBudget will be created with this value as minAvailable, using the component type as selector. This is useful for components, that are spread across multiple replicaSets, like sharepoint or storage layer | |
|
|
**mounts**​.caCerts​.configMap | Alternative 2: the name of the configMap to use. The Key has to be the File Name used in the path setting | |
|
|
**mounts**​.caCerts​.paths | Sets the path to the certs folder. <br>do not change this value | **info only**, do not change<br> `["/opt/ceyoniq/nscale-server/storage-layer/etc/CA.CER"]` |
|
|
**mounts**​.caCerts​.secret | Alternative 1: the name of the secret to use. The Key has to be the File Name used in the path setting | |
|
|
**mounts**​.componentCerts​.configMap | Alternative 2: the name of the configMap to use. The Key has to be the File Name used in the path setting | |
|
|
**mounts**​.componentCerts​.secret | Alternative 1: the name of the secret to use. The Key has to be the File Name used in the path setting | |
|
|
**mounts**​.conf​.path | Sets the path to the conf files <br>do not change this value | **info only**, do not change<br> `"/opt/ceyoniq/nscale-server/storage-layer/etc"` |
|
|
**mounts**​.data​.class | Sets the class of the data disk | |
|
|
**mounts**​.data​.paths | Sets a list of paths to the data files <br>do not change this value | **info only**, do not change<br> `["/opt/ceyoniq/nscale-server/storage-layer/da", "/opt/ceyoniq/nscale-server/storage-layer/hd", "/opt/ceyoniq/nscale-server/storage-layer/logsdb"]` |
|
|
**mounts**​.data​.size | Sets the size of the data disk | `"50Gi"` |
|
|
**mounts**​.data​.volumeName | If you do not want to have a Volume created by the provisioner, you can set the name of your volume here to attach to this pre-existing one | |
|
|
**mounts**​.disk​.class | Sets the class of the disk | |
|
|
**mounts**​.disk​.enabled | enables the use of the second data disk. If enabled, all paths defined will end up on this disk. In case of the (default) disabled, the paths will be added to the primaty data disk. | `false` |
|
|
**mounts**​.disk​.migration | Enables the migration init container. This will copy the data in paths from the primary data disk to the newly enabled secondary disk. This is done only once and only if there is legacy data at all. No files are overwritten! | `false` |
|
|
**mounts**​.disk​.paths | Sets a list of paths to the data files <br>do not change this value | **info only**, do not change<br> `["/opt/ceyoniq/nscale-server/storage-layer/arc", "/opt/ceyoniq/nscale-server/storage-layer/ret", "/opt/ceyoniq/nscale-server/storage-layer/hid"]` |
|
|
**mounts**​.disk​.size | Sets the size of the disk | `"50Gi"` |
|
|
**mounts**​.disk​.volumeName | If you do not want to have a Volume created by the provisioner, you can set the name of your volume here to attach to this pre-existing one | |
|
|
**mounts**​.file​.class | Sets the class of the shared disk | |
|
|
**mounts**​.file​.size | Sets the size of the shared disk | |
|
|
**mounts**​.file​.volumeName | If you do not want to have a Volume created by the provisioner, you can set the name of your volume here to attach to this pre-existing one | |
|
|
**mounts**​.generic | Allows to define generic mounts of pre-provisioned PVs into any container. This can be used e.g. to mount migration nfs, cifs / samba shares into a pipeliner container. | |
|
|
**mounts**​.license​.path | Sets the path to the license files <br>do not change this value | **info only**, do not change<br> `"/opt/ceyoniq/nscale-server/storage-layer/etc/license.xml"` |
|
|
**mounts**​.logs​.medium | the medium for the emptyDisk volume if you unset it, it drops it from the manifest | |
|
|
**mounts**​.logs​.size | Sets the size of the log disk (all paths) | `"5Gi"` |
|
|
**mounts**​.ptemp​.paths | Sets a list of paths for temporary files that are persisted <br>do not change this value | **info only**, do not change<br> `["/opt/ceyoniq/nscale-server/storage-layer/accounting", "/opt/ceyoniq/nscale-server/storage-layer/log", "/var/crash"]` |
|
|
**mounts**​.temp​.paths | Sets a list of paths to the temporary files <br>do not change this value | **info only**, do not change<br> `["/opt/ceyoniq/nscale-server/storage-layer/tmp"]` |
|
|
**mounts**​.temp​.size | Sets the size of the temporary disk (all paths) | `"500Mi"` |
|
|
nameOverride | This overrides the output of the internal name function | |
|
|
nodeSelector | select specific nodes for this component | |
|
|
**resources**​.limits​.cpu | The maximum allowed CPU for the container | |
|
|
**resources**​.limits​.memory | The maximum allowed RAM for the container | |
|
|
**resources**​.requests​.cpu | Set the share of guaranteed CPU to the container. | |
|
|
**resources**​.requests​.memory | Set the share of guaranteed RAM to the container | |
|
|
**security**​.cni​.nstlIpRange | You might want to access storage layer outside the cluster (proxy concept) To do so, you can add a specific IP Range here, which is set within the network policy. | |
|
|
**security**​.containerSecurityContext​.allowPrivilegeEscalation | Some functionality may need the possibility to allow privilege escalation. This should be very restrictive <br>you should not change this | **info only**, do not change<br> `false` |
|
|
**security**​.containerSecurityContext​.readOnlyRootFilesystem | sets the container root file system to read only. This should be the case in production environment <br>you should not change this | **info only**, do not change<br> `true` |
|
|
**security**​.podSecurityContext​.fsGroup | The file system group as which new files are created <br>there is normally no need to change this | **info only**, do not change<br> `1001` |
|
|
**security**​.podSecurityContext​.fsGroupChangePolicy | Under which condition should the fsGroup be changed <br>there is normally no need to change this | **info only**, do not change<br> `"OnRootMismatch"` |
|
|
**security**​.podSecurityContext​.runAsUser | The user under which the container ist run. Avoid 0 / root. The container should run in a non-root context for security <br>there is normally no need to change this | **info only**, do not change<br> `1001` |
|
|
**security**​.zeroTrust | turns on *Zero Trust* Mode, disabling *all* http communication, even the internal http probes | `false` |
|
|
**service**​.annotations | adds extra Annotations to the service | |
|
|
**service**​.enabled | enables the service to be consumed by group components and a potential ingress Disabling the service also disables the ingress. | `true` |
|
|
**service**​.selector | The selector can be `component` or `type` *component* selects only pods that are in the replicaset. *type* selects any pod that has the given type | `"component"` |
|
|
**telemetry**​.openTelemetry | turns Open Telemetry on | |
|
|
**telemetry**​.serviceName | Sets the service name for the telemetry service to more convenient identify the displayed component Example: "{{ .this.meta.type }}-{{ .instance.name }}" | |
|
|
**template**​.annotations | set additional annotations for pods | |
|
|
**template**​.labels | set additional labels for pods | |
|
|
terminationGracePeriodSeconds | Sets the terminationGracePeriodSeconds for the component If not set, it uses the Kubernetes defaults | |
|
|
timezone | set the time zone for this component to make sure log output has a specific timestamp, internal dates and times are correct (like the creationDate in nappl) etc. | `Europe/Berlin` |
|
|
tolerations | Set tolerations for this component | |
|
|
**utils**​.debug | Turn debugging *on* will give you stack trace etc. Please check out the Chart Developer Guide | `false` |
|
|
**utils**​.disableWait | in case you use the argoCD Wave feature, you might think about switching off the waitFor mechanism, that makes sure PODs are only started after pre-requisites are fulfilled. You can disable the starndard wait mechanism, but at your own risk, as this might start components even if they are not intended to run yet. | `false` |
|
|
**utils**​.disableWave | If you use argoCD, you most likely want to use the argo Wave Feature as well, making sure the components of an instance are deployed ordered. However, in DEV you might want to disable this to allow live changing components while previous waves are not finished yet. | `false` |
|
|
**utils**​.includeNamespace | By default, the namespace is rendered into the manifest. However, if you want to use `helm template` and store manifests for later applying them to multiple namespaces, you might want to turn this `false` to be able to use `kubectl apply -n <namespace> -f template.yaml` later | `true` |
|
|
**utils**​.maintenance | in Maintenance Mode, all *waitFor* actions will be skipped, the *Health Checks* are ignored and the pods will start in idle, not starting the service at all. This will allow you to gain access to the container to perform recovery and maintenance tasks while having the real container up. | `false` |
|
|
**utils**​.renderComments | You can turn Comment rendering *on* to get descriptive information inside the manifests. It will also fail on depricated functions and keys, so it is recommended to only switch it off in PROD | `true` |
|
|
|