telemetry_config.yml¶

This file configures telemetry sources (iDRAC, LDMS, PowerScale, UFM, VAST, OpenManage Enterprise), telemetry bridges (Vector-LDMS, Vector-OME), and telemetry sinks (VictoriaMetrics, VictoriaLogs, Kafka). It also includes component-specific configurations for each telemetry source.

Supported Telemetry Sources, Bridges and Sinks¶

Source	Description	Sinks
iDRAC	Out-of-band hardware metrics (power, thermal, storage health) from Dell servers via Redfish	Kafka, VictoriaMetrics
LDMS	In-band OS metrics (CPU, memory, network, I/O) from Slurm compute nodes	Kafka, VictoriaMetrics (via Vector-LDMS)
PowerScale	Storage performance metrics and logs from Dell PowerScale clusters	VictoriaMetrics, VictoriaLogs
UFM	NVIDIA UFM InfiniBand Fabric Manager metrics (IB port state, transmit/receive data, error counters, fabric topology) and syslog logs	VictoriaMetrics, VictoriaLogs
VAST	Storage performance metrics and syslog events from VAST Storage appliances	VictoriaMetrics, VictoriaLogs
OpenManage Enterprise (OME)	Server inventory, health, alerts, and audit logs from Dell OME via Kafka mTLS	Kafka, VictoriaMetrics, VictoriaLogs (via Vector-OME)

Parameter Reference¶

Telemetry Configuration Parameters¶

Parameter	Details
cluster_inventory string / required	Path to the existing Ansible YAML inventory containing the Kubernetes VIP and cluster node groups.
telemetry_sources dict / required	Data collectors — each source can be independently enabled/disabled.
idrac dict / required	iDRAC hardware metrics from Dell PowerEdge servers.
metrics_enabled boolean / required	Enable or disable iDRAC metrics collection from Dell PowerEdge servers Collected metrics: temperature, power, fan speed, storage health, CPU/memory errors Data path: iDRAC Receiver -> ActiveMQ -> KafkaPump -> Kafka 'idrac' topic iDRAC Receiver -> ActiveMQ -> VictoriaPump -> vmagent -> victoria_metrics Accepted values: `true` or `false` Default value: `true` Note If iDRAC telemetry is enabled, the required database credentials are collected in the encrypted `telemetry_credentials.yml` file.
collection_targets list / elements=string / required	Collection targets define where iDRAC data is sent before Vector processing Supported values: `victoria_metrics`, `kafka` Default: [`victoria_metrics`, `kafka`]
ldms dict / required	Lightweight Distributed Metric Service for compute node metrics.
metrics_enabled boolean / required	Enable or disable LDMS metrics collection from compute nodes Collected metrics: CPU, memory, network, disk metrics Data path: LDMS samplers → LDMS aggregator → store_avro_kafka → Kafka 'ldms' topic Accepted values: `true` or `false` Default value: `true`
collection_targets list / elements=string / required	LDMS only supports Kafka collection (no direct victoria_metrics path) Vector-LDMS bridge consumes from Kafka and routes to victoria_metrics Supported values: `kafka` Default: [`kafka`]
powerscale dict / required	Dell PowerScale (OneFS) storage telemetry.
metrics_enabled boolean / required	Enable or disable PowerScale metrics collection from Dell PowerScale (OneFS) storage Collected metrics: Storage metrics from Dell PowerScale clusters Requires: CSM Observability (Karavi) values file configured Data path: CSM Metrics PowerScale → OTEL Collector → vmagent(shared) → victoria_metrics Accepted values: `true` or `false` Default value: `true`
logs_enabled boolean	Enable or disable PowerScale logs collection Accepted values: `true` or `false` Default value: `true`
collection_targets list / elements=string / required	PowerScale uses dedicated vmagent(shared) (no Kafka, no Vector) Supported values: `victoria_metrics`, `victoria_logs` Default: [`victoria_metrics`, `victoria_logs`]
ufm dict / required	NVIDIA UFM InfiniBand Fabric telemetry.
metrics_enabled boolean / required	Enable or disable UFM (NVIDIA UFM InfiniBand Fabric Manager) metrics collection Collected metrics: IB port state, transmit/receive data, error counters, fabric topology Requires: NVIDIA UFM appliance with Prometheus exporter enabled Data path: UFM Prometheus Exporter → vmagent(shared) → victoria_metrics Accepted values: `true` or `false` Default value: `false`
logs_enabled boolean	Enable or disable UFM syslog logs collection Accepted values: `true` or `false` Default value: `false`
collection_targets list / elements=string / required	UFM uses vmagent(shared) for metrics and VLAgent for logs Supported values: `victoria_metrics`, `victoria_logs` Default: [`victoria_metrics`, `victoria_logs`]
vast dict / required	VAST Data Storage telemetry.
metrics_enabled boolean / required	Enable or disable VAST (Data Storage) metrics collection Collected metrics: IB port state, transmit/receive data, error counters, fabric topology Requires: VAST appliance with Prometheus exporter enabled Data path: Prometheus Exporter → vmagent(shared) → victoria_metrics Accepted values: `true` or `false` Default value: `false`
logs_enabled boolean	Enable or disable VAST syslog logs collection Accepted values: `true` or `false` Default value: `false`
collection_targets list / elements=string / required	VAST uses vmagent(shared) for metrics and VLAgent for logs Supported values: `victoria_metrics`, `victoria_logs` Default: [`victoria_metrics`, `victoria_logs`]
ome dict / required	Dell OpenManage Enterprise telemetry. OME publishes to Kafka; Vector-OME bridge routes to Victoria sinks.
metrics_enabled boolean / required	Enable or disable Vector-OME metrics routing (Kafka-to-Victoria bridge for OME metrics) Data flow: Kafka 'ome.' topics → Vector-OME → vmagent-vector (metrics) Requires:* OME to be configured with kafka Accepted values: `true` or `false` Default value: `true`
logs_enabled boolean / required	Enable or disable OME logs collection Accepted values: `true` or `false` Default value: `true`
collection_targets list / elements=string / required	OME only supports Kafka collection. Vector-OME bridge routes to victoria_metrics/victoria_logs. Default: [`kafka`]
telemetry_bridges dict / required	Data routers — Vector pods that consume from Kafka and produce to Victoria sinks.
vector_ldms dict / required	Vector-LDMS: Kafka-to-victoria_metrics bridge for LDMS metrics.
metrics_enabled boolean / required	Enable or disable Vector-LDMS bridge (Kafka-to-victoria_metrics bridge for LDMS metrics) Purpose: Consume LDMS metrics from Kafka 'ldms' topic, transform NERSC schema to Prometheus format, and write to victoria_metrics Data flow: Kafka 'ldms' topic → Vector-LDMS → vmagent-vector → victoria_metrics Requires: telemetry_sources > ldms > metrics_enabled = true Accepted values: `true` or `false` Default value: `true`
vector_ome dict / required	Vector-OME: Kafka-to-Victoria bridge for OME metrics and logs.
metrics_enabled boolean / required	Enable or disable Vector-OME metrics routing (Kafka-to-Victoria bridge for OME metrics) Data flow: Kafka 'ome.' topics → Vector-OME → vmagent-vector (metrics) Requires:* OME to be configured with kafka Accepted values: `true` or `false` Default value: `true`
logs_enabled boolean / required	Enable or disable Vector-OME logs routing Data flow: Kafka 'ome.' topics → Vector-OME → vlagent-vector (logs) Accepted values:* `true` or `false` Default value: `true`
ome_identifier string / required	Identifier used by Vector-OME for topic identification and routing Internally used to match topics with the prefix (e.g., `^ome\\..$`) minLength:* 1 Default value: `ome` Note: Change only if your OME Kafka topics use a different prefix
telemetry_sinks dict / required	Storage backends — auto-enabled when at least one source targets them.
victoria_metrics dict / required	victoria_metrics time-series database for metrics.
persistence_size string / required	Storage per vmstorage pod PVC Important: Total VictoriaMetrics storage depends on deployment mode: Single-node mode: Total storage = `persistence_size * 1 pod` Cluster mode: Total storage = `persistence_size * 3 vmstorage pods` Example (cluster): `8Gi * 3 = 24Gi` total VictoriaMetrics storage Accepted values: Must be specified in the form of `X[Ki\|Mi\|Gi\|Ti\|Pi\|Ei]` Default value: `8Gi` (results in 24Gi total storage for cluster mode)
retention_period integer / required	Metric retention period in hours Default: 168 (7 days)
additional_metric _remote_write_endpoints list / elements=dict	Additional remote write endpoints for metrics (optional) Purpose: Send metrics to external VictoriaMetrics instances in addition to Omnia-managed VictoriaMetrics Format: List of endpoint objects with 'url' field (must start with http:// or https://) TLS: Set 'tls_insecure_skip_verify: true' to skip TLS certificate verification Default: [] (empty — only Omnia VictoriaMetrics receives metrics) Example: - url: https://external-metrics-server:8480/insert/0/prometheus/api/v1/write tls_insecure_skip_verify: false
url string / required	Metrics remote_write endpoint URL.
tls_insecure_skip_verify boolean	Skip TLS certificate verification for this endpoint. Default: `False`.
victoria_logs dict / required	victoria_logs centralized log storage.
storage_size string / required	Storage per vlstorage pod PVC Total storage = storage_size × 3 vlstorage pods Accepted values: Must be specified in the form of `X[Ki\|Mi\|Gi\|Ti\|Pi\|Ei]` Default value: `8Gi` (results in 24Gi total storage) Sizing formula: (140 MB/day × retention_days × node_count) / 3 replicas Warning Storage under-provisioning can lead to data loss before the retention period is reached. Calculate storage requirements based on expected log volume and retention needs.
retention_period integer / required	Log retention period in hours Accepted values: 24-8760 (1 day to 1 year) Default: 168 (7 days) Note Retention is global and applies to all log streams uniformly. Deletion occurs asynchronously during background merge operations. VictoriaLogs does not return an error when log entries with timestamps outside the configured retention window are submitted. Log entries will be automatically removed from VictoriaLogs after the retention period.
additional_log_write_endpoints list / elements=dict	Additional remote write endpoints for logs (optional) Purpose: Send logs to external VictoriaLogs instances in addition to Omnia-managed VictoriaLogs Format: List of endpoint objects with 'url' field (must start with http:// or https://) TLS: Set 'tls_insecure_skip_verify: true' to skip TLS certificate verification Default: [] (empty — only Omnia VictoriaLogs receives logs) Example: - url: https://external-logs-server:9481/internal/insert tls_insecure_skip_verify: false
url string / required	Logs remote_write endpoint URL.
tls_insecure_skip_verify boolean	Skip TLS certificate verification for this endpoint. Default: `False`.
kafka dict / required	Apache Kafka distributed streaming platform.
persistence_size string / required	Storage per Kafka pod PVC Total = persistence_size × 6 pods (3 brokers + 3 controllers) Accepted values: Must be specified in the form of `X[Ki\|Mi\|Gi\|Ti\|Pi\|Ei]` Default value: `8Gi` (results in 48Gi total storage) The default `8Gi` size is suitable for small clusters (typically fewer than 5 nodes). For larger clusters, you should increase the `persistence_size` and adjust `log_retention_hours` and `log_retention_bytes` based on expected data volume and cluster size. Caution Ensure that the Kafka broker settings `persistence_size`, `log_retention_hours`, and `log_retention_bytes` are configured based on your data retention requirements. If the persistent volume reaches its capacity before logs are deleted according to the log retention period configured, Kafka brokers may run out of disk space. For more details on managing Kafka log retention and cleanup policies, see Managing Kafka logs with delete and compact policies.
log_retention_hours integer / required	Log retention period in hours Default: 168 (7 days)
log_retention_bytes integer / required	Maximum size of Kafka logs (in bytes) before deletion Default: -1 (unlimited)
log_segment_bytes integer / required	Maximum size of Kafka log segments (in bytes) Default: 1073741824 (1 GB)
topic_partitions dict / required	Topic partitions per source (object format, not array) Format: {topic_name: partition_count} Required when: Source has kafka in collection_targets Allowed topics: `idrac`, `ldms` only Default partition counts: `idrac=1`, `ldms=2` Example: {idrac: 1, ldms: 2}
idrac integer	Number of partitions for the 'idrac' topic. Range: 1-100. Default: `1`.
ldms integer	Number of partitions for the 'ldms' topic. Range: 1-100. Default: `2`.
ldms_configurations dict / required	LDMS-specific configurations for aggregator, store, and sampler.
agg_port integer / required	Aggregator port on service K8s cluster Valid range: 6001-6100 Default: 6001
store_port integer / required	Store daemon port on service K8s cluster The port can be the same as LDMS aggregator port Valid range: 6001-6100 Default: 6001
sampler_port integer / required	Sampler port on compute nodes Valid range: 10001-10100 Default: 10001
sampler_plugins list / elements=dict / required	Sampler plugins define which metrics to collect from compute nodes Parameters: plugin_name: Name of the LDMS sampler plugin config_parameters: Plugin-specific configuration (as a single string) activation_parameters: Collection schedule in MICROSECONDS Format: `interval=<microseconds> offset=<microseconds>` Example: `interval=30000000` (30 seconds) Default plugins: `meminfo`: Memory usage statistics (free, used, buffers, cached) `procstat2`: Process statistics (CPU, memory, I/O per process) `vmstat`: Virtual memory statistics (paging, swapping, memory pressure) `loadavg`: System load average (1, 5, and 15 minute averages) `procnetdev2`: Network interface statistics (bytes, packets, errors, drops per interface) Default activation_parameters: `interval=30000000` (30 seconds for all plugins except procnetdev2 which includes offset=0)
plugin_name string / required	Name of the LDMS sampler plugin.
config_parameters string	Plugin-specific configuration parameters as a single string.
activation_parameters string / required	Activation parameters (e.g., 'interval=30000000 offset=0').
powerscale_configurations dict / required	PowerScale telemetry detailed configurations.
otel_collector_storage_size string / required	PVC size for OTEL Collector metric batching and buffering Accepted values: Must be specified in the form of `X[Ki\|Mi\|Gi\|Ti\|Pi\|Ei]` Default value: `5Gi`
csm_observability_values_file_path string / required	Path to the CSM Observability (Karavi Observability) values.yaml file Required when: telemetry_sources > powerscale > metrics_enabled: true Reference: https://raw.githubusercontent.com/dell/helm-charts/refs/heads/release-v1.17.1/charts/karavi-observability/values.yaml Default value: `""` Note In the values.yaml file, only set `karaviMetricsPowerscale -> enabled: true`. Set the following parameters to false: `karaviMetricsPowerflex -> enabled=false`, `karaviMetricsPowerstore -> enabled=false`, `karaviMetricsPowerscaleauthorization.-> enabled=false`, `karaviMetricsPowermax -> enabled=false`. Update the `isiAuthType` in the values.yaml file based on the current auth type setting. To check the current auth type setting, use the command `isi http settings view`. For CSI PowerScale health metrics, enable `controller > healthMonitor > enabled: true` and `node > healthMonitor > enabled: true` in the CSI PowerScale values.yaml (https://raw.githubusercontent.com/dell/helm-charts/csi-isilon-2.17.0/charts/csi-isilon/values.yaml).
ufm_configuration dict / required	UFM telemetry detailed configurations.
ufm_endpoint string	UFM appliance IP address or hostname Required when: telemetry_sources > ufm > metrics_enabled is true Example: `172.20.44.180` or `ufm.example.com` Default value: `""`
ufm_metrics_port integer	UFM Prometheus exporter port Default value: 9001 (UFM default Prometheus port)
scrape_interval string	Prometheus scrape interval for UFM metrics Accepted values: Prometheus duration format (e.g., `15s`, `30s`, `1m`) Default value: `30s`
scrape_timeout string	Prometheus scrape timeout (must be <= scrape_interval) Accepted values: Prometheus duration format (e.g., `10s`, `15s`) Default value: `15s`
tls_mode string	TLS mode for connecting to UFM Prometheus endpoint Accepted values: `self_signed`, `ca_signed` self_signed: Skip TLS verification (insecure_skip_verify=true) ca_signed: Use CA certificate for TLS verification Default value: `self_signed`
ufm_ca_cert_path string	Path to CA certificate file for UFM TLS verification Required when: tls_mode is `ca_signed` Must be a valid PEM-format certificate file Default value: `""` (empty — not used when tls_mode is `self_signed`)
auth_mode string	Authentication mode for UFM Prometheus endpoint Accepted values: `basic`, `none` basic: Use ufm_username/ufm_password from telemetry_credentials.yml none: No authentication (UFM endpoint is open) Default value: `basic`
vast_configuration dict / required	VAST telemetry detailed configurations.
vast_endpoint string	VAST appliance IP address or hostname Required when: telemetry_sources > vast > metrics_enabled is true Example: `172.20.44.180` or `vast.example.com` Default value: `""`
vast_metrics_port integer	VAST Prometheus exporter port Default value: 9001 (VAST default Prometheus port)
metrics_path string	Prometheus metrics path for VAST API. Default: `/api/prometheusmetrics/all`.
scrape_interval string	Prometheus scrape interval for VAST metrics Accepted values: Prometheus duration format (e.g., `15s`, `30s`, `1m`) Default value: `30s`
scrape_timeout string	Prometheus scrape timeout (must be <= scrape_interval) Accepted values: Prometheus duration format (e.g., `10s`, `15s`) Default value: `15s`
tls_mode string	TLS mode for connecting to VAST Prometheus endpoint Accepted values: `self_signed`, `ca_signed` self_signed: Skip TLS verification (insecure_skip_verify=true) ca_signed: Use CA certificate for TLS verification Default value: `self_signed`
vast_ca_cert_path string	Path to CA certificate file for VAST TLS verification Required when: tls_mode is `ca_signed` Must be a valid PEM-format certificate file Default value: `""` (empty — not used when tls_mode is `self_signed`)
auth_mode string	Authentication mode for VAST Prometheus endpoint Accepted values: `basic`, `none` basic: Use vast_username/vast_password from telemetry_credentials.yml none: No authentication (VAST endpoint is open) Default value: `basic`

Enable or disable Telemetry sources¶

Set a source's metrics_enabled or logs_enabled value to true or false, and then rerun deployment:

Using omnia.sh (recommended)Using ansible-playbook

Run on: OIM

cd <OMNIA_SOURCE_PATH>/src/main
./omnia.sh --run telemetry --tags deploy

Run on: OIM

source /opt/omnia/activate-omnia.sh
cd <OMNIA_SOURCE_PATH>/src/telemetry/playbooks
ansible-playbook telemetry.yml --tags deploy

When a source is disabled, its active pods are scaled down to zero while its services and ConfigMaps remain available. Set the applicable value back to true and rerun deployment to enable the source again. For LDMS and OME, keep the corresponding telemetry_bridges enable values aligned with the source channels they consume.

Disabling a source is different from cleaning it up. See Clean Up Telemetry when the source resources and source-owned persistent volumes must be removed.

iDRAC MySQL settings¶

The iDRAC source uses an internal MySQL database to store the service inventory used by the receiver. Configure it under idrac_telemetry_configurations.

Parameter	Requirement	Description
`bmc_group_data_path`	Required when iDRAC metrics are enabled	Absolute path to a CSV with the header `BMC_IP,GROUP_NAME,PARENT`. This can reference the BMC inventory generated by Orchestrator or a customer-managed file.
`mysqldb_storage`	Required when iDRAC metrics are enabled; default `1Gi`	Requested capacity for the ReadWriteOnce MySQL persistent volume.

MySQL credentials are not stored in this configuration. They are collected as conditional credentials in the encrypted project telemetry_credentials.yml file.

Usage example¶

Resolve the component paths from /etc/omnia/omnia.env and replace the angle-bracket placeholders below with absolute paths. Environment-variable expressions are not expanded inside this YAML file.

File: <OMNIA_DATA_PATH>/telemetry/input/<OMNIA_PROJECT_NAME>/telemetry_config.yml

---
cluster_inventory: "$OMNIA_DATA_PATH/orchestrator/output/<OMNIA_PROJECT_NAME>/orchestrator_inventory.yml"

telemetry_sources:

  idrac:
    metrics_enabled: true
    collection_targets:
      - "victoria_metrics"
      - "kafka"

  ldms:
    metrics_enabled: true
    collection_targets:
      - "kafka"

  powerscale:
    metrics_enabled: true
    logs_enabled: true
    collection_targets:
      - "victoria_metrics"
      - "victoria_logs"

  ufm:
    metrics_enabled: false
    logs_enabled: false
    collection_targets:
      - "victoria_metrics"
      - "victoria_logs"

  vast:
    metrics_enabled: false
    logs_enabled: false
    collection_targets:
      - "victoria_metrics"
      - "victoria_logs"

  ome:
    metrics_enabled: true
    logs_enabled: true
    collection_targets:
      - "kafka"

telemetry_bridges:

  vector_ldms:
    metrics_enabled: true

  vector_ome:
    metrics_enabled: true
    logs_enabled: true
    ome_identifier: "ome"

telemetry_sinks:

  victoria_metrics:
    persistence_size: "8Gi"
    retention_period: 168
    additional_metric_remote_write_endpoints: []

  victoria_logs:
    storage_size: "8Gi"
    retention_period: 168
    additional_log_write_endpoints: []

  kafka:
    persistence_size: "8Gi"
    log_retention_hours: 168
    log_retention_bytes: -1
    log_segment_bytes: 1073741824
    topic_partitions:
      idrac: 1
      ldms: 2

idrac_telemetry_configurations:
  bmc_group_data_path: "$OMNIA_DATA_PATH/orchestrator/output/<OMNIA_PROJECT_NAME>/bmc_group_data.csv"
  mysqldb_storage: "1Gi"

ldms_configurations:
  agg_port: 6001
  store_port: 6001
  sampler_port: 10001
  sampler_plugins:
    - plugin_name: meminfo
      config_parameters: ""
      activation_parameters: "interval=30000000"

    - plugin_name: procstat2
      config_parameters: ""
      activation_parameters: "interval=30000000"

    - plugin_name: vmstat
      config_parameters: ""
      activation_parameters: "interval=30000000"

    - plugin_name: loadavg
      config_parameters: ""
      activation_parameters: "interval=30000000"

    - plugin_name: procnetdev2
      config_parameters: ""
      activation_parameters: "interval=30000000 offset=0"

powerscale_configurations:
  otel_collector_storage_size: "5Gi"
  csm_observability_values_file_path: ""

ufm_configuration:
  ufm_endpoint: ""
  ufm_metrics_port: 9001
  scrape_interval: "30s"
  scrape_timeout: "15s"
  tls_mode: "self_signed"
  ufm_ca_cert_path: ""
  auth_mode: "basic"

vast_configuration:
  vast_endpoint: ""
  vast_metrics_port: 443
  metrics_path: "/api/prometheusmetrics/all"
  scrape_interval: "30s"
  scrape_timeout: "15s"
  tls_mode: "self_signed"
  vast_ca_cert_path: ""
  auth_mode: "basic"

Info

cluster_inventory is required and must identify an existing Ansible YAML inventory. Telemetry reads the Kubernetes VIP and target node groups from this file.
Telemetry Storage Configuration -- Resource, replica, and persistent storage settings for telemetry components.
Telemetry Packages -- Package sources, images, charts, repositories, and Python modules.
Idrac Metrics -- iDRAC metric catalog.
Ldms Metrics -- LDMS sampler metric catalog.
Ports -- Ports used by telemetry services.