Configure PowerScale Telemetry¶

Configure deployment of PowerScale Telemetry to collect storage performance metrics and logs from Dell PowerScale storage nodes.

Overview¶

PowerScale Telemetry collects storage performance metrics and logs. It includes the following components:

Components¶

CSM Metrics for PowerScale -- Queries the OneFS API and emits metrics to an OpenTelemetry Collector.
OpenTelemetry Collector -- Receives metrics from CSM Metrics and exposes a Prometheus endpoint for scraping.
vmagent -- Scrapes the OpenTelemetry Collector Prometheus endpoint over plain HTTP and forwards metrics to VictoriaMetrics.
VLAgent -- Receives plaintext PowerScale syslog events on TCP or UDP port 514 and forwards them to VictoriaLogs.
CSI Driver for Dell PowerScale -- Required for Omnia-orchestrated deployment mode.
cert-manager -- Required for TLS certificate management in Omnia-orchestrated mode.

Data flow¶

PowerScale nodes → CSM Metrics PowerScale → OTEL Collector → vmagent (shared) → VictoriaMetrics
PowerScale nodes → syslog → VLAgent → VictoriaLogs

Supported metrics and logs¶

Metrics:

Category	Metrics collected
Performance	Protocol-level IOPS (NFS, SMB, S3), throughput (bytes/s), and read/write latency
Capacity	Total cluster capacity, used capacity, available capacity, and per-node capacity
Health	Node online/offline status, disk health, cluster rebalance status, and protection group status
Topology	Cluster node membership, node roles, interconnect layout, and protection domain mapping

For the complete list of PowerScale Telemetry metrics, see the PowerScale Metrics reference and Dell CSM Observability PowerScale metrics.

Logs:

Category	Logs collected
System Events	Capacity warnings, disk failures, node state changes, and protocol errors
Labels	Events labeled with host or cluster, severity, and facility

Health Monitor Metrics¶

When the CSI PowerScale health monitor is enabled with controller.healthMonitor.enabled: true and node.healthMonitor.enabled: true in the CSI PowerScale values.yaml, Omnia collects the following additional health metrics.

PV Metrics:

powerscale_volume_status -- PV phase (1=Bound, 0=Other), labeled by pv_name and phase.
powerscale_volume_count -- Total PowerScale PVs by phase.
powerscale_volume_capacity_bytes -- PV capacity in bytes.
powerscale_volume_info -- PV metadata, including name, phase, storage class, reclaim policy, access modes, volume handle, PVC name, and namespace.
powerscale_volume_age_seconds -- Seconds since PV creation.

PVC Metrics:

powerscale_pvc_status_phase -- PVC phase (1=Bound, 0=Other), labeled by PVC name, namespace, and phase.
powerscale_pvc_requested_bytes -- Requested PVC storage in bytes.
powerscale_pvc_count -- Total PowerScale PVCs by phase.

Health Event Metrics:

powerscale_volume_health_abnormal -- Volume condition (1=abnormal, 0=healthy), labeled by PVC name, namespace, and PV name.
powerscale_volume_abnormal_events_total -- Total VolumeConditionAbnormal events.
powerscale_node_failure_events_total -- Total node failure events.

Node Metrics:

powerscale_node_ready -- Node Ready condition (1=True, 0=False).

Storage Class Metrics:

powerscale_storageclass_info -- StorageClass metadata, including the provisioner, reclaim policy, volume binding mode, and volume expansion flag.

Aggregate Summary:

powerscale_total_capacity_bytes -- Total capacity of all PowerScale PVs in bytes.

TLS and Authentication¶

All metric scraping between the OpenTelemetry Collector and VictoriaMetrics uses TLS encryption. Authentication uses Kubernetes service-account tokens. Mutual TLS (mTLS) is not required — the connection is encrypted but the PowerScale-side endpoint does not validate client identity via certificate exchange. TLS is enforced for all off-cluster communications.

Prerequisites¶

Complete the common Telemetry deployment prerequisites.
Deploy the PowerScale CSI driver in the isilon namespace and ensure its pods are running. The external-health-monitor-controller is required for the CSI volume exporter health metrics.
Provide a CSM Observability values YAML containing karaviMetricsPowerscale.image, otelCollector.image, and cert-manager.enabled: true. Do not enable PowerFlex, PowerStore, or PowerMax metrics in this file.
Keep images.powerscale.csm_metrics and images.powerscale.otel_collector in telemetry_packages.yml; in offline mode, values-file image versions must match the package manifest.
Provide csi_username and csi_password when prompted.

Procedure¶

Configure the PowerScale source and its required metrics target in the telemetry_config.yml input file:
```
telemetry_sources:
  powerscale:
    metrics_enabled: true
    logs_enabled: false
    collection_targets:
      - victoria_metrics

powerscale_configurations:
  otel_collector_storage_size: "5Gi"
  csm_observability_values_file_path: ""
```
Note

The default value of csm_observability_values_file_path is empty. When PowerScale metrics are enabled, replace it with the absolute path to an existing CSM Observability values.yaml file before running validation or deployment.
To prepare PowerScale log ingestion as well, keep metrics enabled, set logs_enabled: true, and add victoria_logs to collection_targets. The root workflow imports the PowerScale source only when metrics_enabled: true; a logs-only configuration is not supported.
Keep the csm_metrics_powerscale_storage and csi_volume_exporter_storage sections in telemetry_storage_config.yml.

Run the Telemetry precheck. Choose one execution method; do not run both commands for the same operation.

Using omnia.sh (recommended)Using ansible-playbook

Run on: OIM

cd src/main
./omnia.sh --run telemetry --tags precheck

Run on: OIM

source /opt/omnia/activate-omnia.sh
cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks
ansible-playbook telemetry.yml --tags precheck

Validate the Telemetry inputs and collect the required credentials:

Using omnia.sh (recommended)Using ansible-playbook

Run on: OIM

cd src/main
./omnia.sh --run telemetry --tags validate

Run on: OIM

source /opt/omnia/activate-omnia.sh
cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks
ansible-playbook telemetry.yml --tags validate

Deploy the enabled Telemetry configuration:

Using omnia.sh (recommended)Using ansible-playbook

Run on: OIM

cd src/main
./omnia.sh --run telemetry --tags deploy

Run on: OIM

source /opt/omnia/activate-omnia.sh
cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks
ansible-playbook telemetry.yml --tags deploy

To run validation and deployment in one invocation, omit the tag:
Using omnia.sh (recommended)Using ansible-playbook
Run on: OIM
cd src/main ./omnia.sh --run telemetry
Run on: OIM
source /opt/omnia/activate-omnia.sh cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks ansible-playbook telemetry.yml
The untagged flow does not run the opt-in precheck. Run step 4 separately when an environment precheck is required.

When logs are enabled, export the generated VLAgent target and PowerScale isi audit commands:

Using omnia.sh (recommended)Using ansible-playbook

Run on: OIM

cd src/main
./omnia.sh --run telemetry --tags external_victoria

Run on: OIM

source /opt/omnia/activate-omnia.sh
cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks
ansible-playbook telemetry.yml --tags external_victoria

Run the commands recorded under powerscale.isi_audit_commands in the generated connection-details file on the PowerScale system:

$OMNIA_DATA_PATH/telemetry/output/$OMNIA_PROJECT_NAME/external_victoria/external_victoria_connect_details.yml

Verification¶

Verify PowerScale Telemetry pods¶

Verify that the VictoriaMetrics pods are running:
Run on: Kubernetes control plane
```
kubectl get pods -n telemetry -o wide | grep vm
```
Verify that the VictoriaMetrics services are running:
Run on: Kubernetes control plane
```
kubectl get service -n telemetry -o wide | grep vm
```

Verify the PowerScale metrics and OpenTelemetry Collector resources:

Run on: Kubernetes control plane

kubectl get pods -n telemetry -l app.kubernetes.io/name=karavi-metrics-powerscale
kubectl get deployment otel-collector -n telemetry

View PowerScale metrics in VictoriaMetrics UI¶

Identify the external vmselect service:
Run on: Kubernetes control plane
```
kubectl get svc -n telemetry | grep vmselect
```

Access the VMUI in a web browser:

https://<external vmselect loadbalancer IP>:8481/select/0/vmui

Query PowerScale metrics in VMUI:
```
{__name__=~"powerscale_.*"}
```

View PowerScale logs in VictoriaLogs¶

Complete these steps only when PowerScale log collection is enabled.

Verify that the VictoriaLogs pods are running:
Run on: Kubernetes control plane
```
kubectl get pods -n telemetry -o wide | grep vl
```
Verify that the VictoriaLogs services are running:
Run on: Kubernetes control plane
```
kubectl get service -n telemetry -o wide | grep vl
```

Access the VictoriaLogs UI in a web browser:

https://<external vlselect loadbalancer IP>:9471/select/vmui

Use * in the VictoriaLogs query field to display all logs and confirm that PowerScale log records are present.

Confirm sources.powerscale.metrics: deployed in telemetry_status.yml. When logs are enabled, confirm sources.powerscale.logs: deployed and verify that the generated external Victoria file reports vlagent.available: true.

The log status confirms that the shared VLAgent is available; it does not configure PowerScale log forwarding or prove ingestion. Run the exported isi audit commands and query VictoriaLogs for an end-to-end check. Likewise, query VictoriaMetrics to confirm that PowerScale metrics are being ingested.

Next steps¶

Optional: To disable PowerScale Telemetry metrics and logs without removing its infrastructure, set telemetry_sources.powerscale.metrics_enabled: false and telemetry_sources.powerscale.logs_enabled: false in telemetry_config.yml. Then rerun deployment:
Using omnia.sh (recommended)Using ansible-playbook
Run on: OIM
cd <OMNIA_SOURCE_PATH>/src/main ./omnia.sh --run telemetry --tags deploy
Run on: OIM
source /opt/omnia/activate-omnia.sh cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks ansible-playbook telemetry.yml --tags deploy

The PowerScale Telemetry pods are scaled down to zero. Its services and ConfigMaps remain available for later use. - To re-enable PowerScale Telemetry, set metrics_enabled: true, set logs_enabled to the required value, and rerun the same deploy command. - To remove PowerScale Telemetry resources and source-owned persistent volumes, see Clean Up Telemetry. - Use Export Victoria Connection Details to obtain write, query, and syslog endpoints.

Troubleshooting¶

Validation cannot find the values file: Set an existing path in csm_observability_values_file_path and ensure it contains valid YAML.
The values file is rejected: Enable cert-manager, provide the required PowerScale and OTEL images, disable unsupported storage metrics, and align image versions with telemetry_packages.yml in offline mode.
PowerScale precheck warns about privileges: Verify the PowerScale account has the permissions required by the enabled metrics or log path.
CSI volume exporter is skipped: Enable the external health monitor in the PowerScale CSI driver and rerun Telemetry.