Release Information¶
This page summarizes the features, enhancements, and changes introduced in each Omnia release.
Omnia v2.3.0.0-rc1¶
Omnia v2.3.0.0-rc1 is validated with RHEL 10.0 on the Omnia Infrastructure Manager (OIM) and cluster nodes. For operating-system requirements, see the Operating Systems Matrix. The source distribution also includes RHEL 10.2 cluster-node catalogs, but their presence does not declare RHEL 10.2 supported, validated, or a technology preview. No RHEL 10.2 OIM and cluster-node combination has a published product classification for this release.
| Feature | Description |
|---|---|
| Domain-Based Deployment Architecture | Repository Manager, Image Build Manager, Discovery, Orchestrator, Telemetry, BuildStreaM, and Utils operate as independent domains. omnia.sh initializes and runs each domain through its supported lifecycle tags. Domain inputs, outputs, and logs are organized by $OMNIA_DATA_PATH and $OMNIA_PROJECT_NAME. For details, see the Playbook Reference. |
| AI-Assisted Catalog Authoring Skills | Provides optional skills for catalog generation, single-catalog editing, bulk editing, impact analysis, compatibility analysis, and catalog comparison. Users invoke the required skill through an approved AI assistant. For details, see AI-Assisted Catalog Authoring. |
| Automated Build and Deployment with Cadence | Adds optional periodic reconciliation for RPM repositories referenced by the cadence catalog. When package additions or removals are detected, BuildStreaM increments the catalog version and runs repository configuration, image build, deployment, node restart, and validation in one GitLab pipeline. The independent build and deploy pipelines remain available. For details, see Automate Build and Deployment with Cadence. |
| Versioned Catalogs and Catalog-Aware Image Reuse | Adds catalog schema versioning, composite <identifier>-v<version> image-group identities, catalog-specific build-status output, and a project-level global dictionary that reuses valid functional-group images when their package and repository inputs match. build_image.force_rebuild remains the single control for bypassing reuse. For details, see Execute Build Pipeline. |
| Catalog-Driven Repository Management | Repository Manager uses catalog content to resolve and synchronize RPM repositories, container images, Python packages, files, and source artifacts for x86_64 and aarch64. It also supports catalog management, validation, and selective content cleanup. For details, see Configure Repositories. |
| Image Build Manager Enhancements | Image Build Manager supports the image-builder and image-thrillhouse engines, catalog- or configuration-based functional groups, parallel builds, and selective image cleanup. Builds can produce x86_64 and aarch64 images; a native aarch64 build host is required for aarch64 images. For details, see Build OS Images. |
| OpenCHAMI Fabrica-Based Service Support | Adds support for the Fabrica-based OpenCHAMI stack, including SMD, boot-service, metadata-service, tokensmith, and supporting services managed under openchami.target for node inventory and provisioning. For details, see Deploy OpenCHAMI. |
| Telemetry Sink and Source Framework | Provides Kafka (Strimzi), VictoriaMetrics, and VictoriaLogs as shared telemetry sinks and iDRAC, LDMS, OME, PowerScale, UFM, VAST, and SFM as pluggable telemetry sources. SFM connects externally for fabric monitoring. Supports sink-only deployments and full source deployments after provisioning. Configure sources in telemetry_config.yml and deploy them independently to the telemetry Kubernetes namespace. For details, see Deploy the Telemetry Stack. |
| OIM Domain Log Backup | Utils can archive selected Omnia domain logs from the OIM to a local directory or NFS export. Each timestamped backup includes a compressed archive, metadata, and a SHA-256 checksum, with a dedicated cleanup operation. For details, see Back Up OIM Logs. |
Omnia 2.2.0.0¶
| Feature | Description |
|---|---|
| BuildStreaM Pipeline Architecture and API Enhancements | Enhanced pipeline architecture and API capabilities with resume and retry functionality, pipeline decomposition, dynamic child pipeline generation, image group lifecycle tracking, manual cleanup operations, and PowerScale S3 backend support. For more details, see Path D: BuildStreaM Automated Deployment. |
| BMC Discovery via Dell OpenManage Enterprise | Automated BMC discovery via Dell OpenManage Enterprise (OME) with paginated API queries, automatic extraction of service tags and iDRAC details, Scalable Unit extraction, timestamped file generation, and OME group mapping. For more details, see BMC Discovery. |
| Multi-Subnet DHCP for Rack-Based Provisioning | Multi-subnet DHCP configuration for rack-based network provisioning with per-rack /24 subnet assignment, CoreDHCP multi-subnet configuration generation, and CoreDNS forward and reverse zone generation. For more details, see Configure Multi-Subnet DHCP. |
| CoreDNS-Based Hostname Resolution for Slurm and MPI | Dynamic DNS resolution powered by coresmd replacing static /etc/hosts file management with automatic hostname resolution, real-time inventory updates from OpenCHAMI SMD, cloud-init based /etc/resolv.conf configuration, and Kubernetes CoreDNS forwarding. For more details, see Configure Cluster DNS. |
| Vector Telemetry Pipeline for Data Routing | Vector high-performance data pipeline for collecting, transforming, and routing telemetry data from LDMS and OME sources to VictoriaMetrics and VictoriaLogs with dedicated write-buffer components. For more details, see Telemetry Architecture. |
| PowerScale Telemetry for Storage Monitoring | PowerScale Telemetry for comprehensive storage observability collecting storage performance metrics and logs from Dell PowerScale storage nodes with CSM Metrics, OpenTelemetry Collector, and CSI Driver integration. For more details, see Configure PowerScale Telemetry. |
| UFM Telemetry to VictoriaMetrics | UFM (Unified Fabric Manager) telemetry collection for InfiniBand fabric monitoring through vmagent scraping with secure HTTPS, TLS certificate validation, and dual-destination forwarding to local and remote VictoriaMetrics clusters. For more details, see Configure UFM (NVIDIA Unified Fabric Manager) Telemetry. |
| VAST Storage Telemetry Integration | VAST storage telemetry integration through VMagent scraping of VAST Prometheus endpoints and VLAgent syslog log collection with secure HTTPS, TLS authentication, and dual-destination forwarding. For more details, see Configure VAST (VAST Storage) Telemetry. |
| External Log Aggregation to VictoriaLogs | Centralized log collection from external sources including network devices, storage systems, and fabric managers through VLAgent with syslog (plaintext/TLS) and HTTP forwarding support, TLS certificate validation, and JSON Lines format ingestion. For more details, see Collect Logs from External Clients to VictoriaLogs. |
| Configurable Pod Replicas for vmagent and vlagent | Configurable replica counts for vmagent and vlagent pods with default value of 2 replicas each, providing improved availability and scalability for telemetry data collection and log aggregation. For more details, see Telemetry Configuration. |
| Minimal OS Functional Groups | Minimal OS functional groups (os_x86_64 and os_aarch64) providing a clean operating system baseline designed specifically for downstream platform software installation. For more details, see Create a Mapping File. |
| Custom Cloud-Init for Post-Boot Mounts and Scripts | Added support for injecting custom cloud-init directives during node provisioning to enable boot-time node customization without modifying platform-managed templates. Administrators can apply configurations globally to all nodes (common) or to specific functional groups (groups). Supported cloud-init directives include write_files for creating or modifying files and runcmd for executing setup commands during node initialization. For more information, see Configure Additional Cloud-Init. |
| NVIDIA DCGM and CUDA Toolkit Provisioning for Slurm GPU Nodes | End-to-end automated GPU readiness for Slurm clusters with NVIDIA driver installation, CUDA toolkit distribution to shared cluster storage, and DCGM setup during stateless node provisioning. For more details, see Set Up Slurm. |
| NVIDIA HPC SDK Provisioning for Slurm Clusters | Cluster-wide deployment of NVIDIA HPC SDK (nvhpc) for Slurm compiler and compute nodes with single installation on compiler node, NFS sharing, and bind mount distribution. For more details, see NVIDIA HPC SDK Setup. |
| Vast Repo and Vast Client Installation | Vast NFS client installation streamlined by building Vast repository from source, hosting RPMs on HTTP server, configuring repository, and automatic installation during provisioning when InfiniBand NIC is present. For more details, see Configure VAST Storage. |
| One-Shot Combined Log Extraction for Debugging | One-shot log collection playbook for gathering cluster logs from Kubernetes and Slurm nodes with full and curated support collection modes, log collection from all node types, and timestamped tar.gz bundle output. For more details, see Log Management. |
| ETCD on Local Disk Support for Kubernetes Service Cluster | ETCD deployment on local disk instead of NFS for Kubernetes service cluster with configurable etcd_on_local_disk setting in omnia_config.yml, automatic disk selection prioritizing BOSS cards (BOSS-N1/N2) with fallback to SSD/SATA disks, /var/lib/etcd mount point, support for pre-configured RAID 1 or RAID 10 on BOSS cards, and minimum 20 GB disk space recommendation. For more details, see Configure Kubernetes HA. |
| Unattended OS Installation via iDRAC Virtual Media | Automated bare-metal OS installation using the unified Utils install_os.yml workflow with embedded or NFS-based Kickstart, one server at a time. The same workflow supports x86_64 and aarch64. For more details, see Unattended OS Installation via iDRAC. |
Known Issues¶
| Issue | Description | Workaround |
|---|---|---|
| BuildStreaM Pipeline Retry Limitation | When a build pipeline fails partially (e.g., one architecture succeeds while another fails due to resource constraints, network issues, or configuration problems), retrying the pipeline may result in INTERNAL_ERROR for previously completed image builds. BuildStreaM currently does not skip or reuse already-successful builds during retry operations. | If you encounter INTERNAL_ERROR during pipeline retry, start a fresh pipeline instead of retrying the failed one. Ensure adequate system resources (including 200 GB free disk space on OIM / partition) before initial pipeline execution to minimize the risk of partial failures. |
Omnia 2.1.0.0¶
| Feature | Description |
|---|---|
| BuildStreaM: Catalog-Driven Build Automation | Omnia BuildStreaM provides a comprehensive automation solution for managing infrastructure build workflows. It uses a catalog-driven approach where you define your build requirements in a structured catalog file, and BuildStreaM executes automated pipelines to create and deploy images according to your specifications. For more details, see BuildStreaM Documentation. |
| Support for Installation of Additional Packages | Enables the installation of additional packages on the cluster nodes, allowing to extend cluster functionality with custom software and tools. For more details, see Configure Catalog Content. |
| Add and Remove Slurm Compute Nodes | Provides the ability to add and remove Slurm compute nodes from the cluster, allowing for dynamic scaling of the cluster. See Add Nodes and Remove Slurm Compute Nodes. |
| Support for Apptainer | Run apptainer pull to store the SIF container image on the cluster's NFS-mounted shared storage. This ensures uniform access across all compute nodes, enabling them to run jobs from the same SIF file. For more details, see Use Apptainer. |
| Telemetry Collection from OME and SFM | Enables collection of telemetry data from OpenManage Enterprise (OME) and Smart Fabric Manager (SFM), providing insights into cluster health, performance, and resource utilization. For more details, see Deploy Telemetry. |
| PowerVault Storage Integration | The PowerVault integration, with a preconfigured volume, provides persistent storage for critical Slurm controller components using iSCSI block storage with multipath support. This ensures data persistence for Slurm's StateSaveLocation and the MariaDB/MySQL database. |
| InfiniBand Networking Support | Provides comprehensive support for InfiniBand (IB) networking in HPC clusters, including automatic DOCA-OFED driver installation for NVIDIA InfiniBand cards, and static IP assignment for IB interfaces. InfiniBand Networking requires an InfiniBand subnet manager (SM) configured and running to manage the IB fabric. For more details, see Configure InfiniBand. |
Omnia 2.0.0.0¶
| Feature | Description |
|---|---|
| Support for Podman Services | Enables deployment of supporting services such as OpenCHAMI, Pulp, MinIO, the OCI registry, and BuildStreaM components as Podman containers. The Omnia deployment modules themselves run directly on the OIM through omnia.sh. |
| Repository Management | Provides a Pulp-based local repository service deployed as a Podman container, enabling secure and efficient package distribution in air-gapped HPC environments. This reduces dependency on external networks and accelerates provisioning workflows. For more details, see Create Local Repositories. |
| Authentication Service | Integrates an LDAP server within the Omnia Auth Podman container for centralized authentication and directory services. This enhances security and simplifies identity management across HPC clusters. For more details, see Deploy OpenLDAP. |
| Telemetry Collection and Monitoring | Automates the configuration of Kubernetes Service Clusters to host essential monitoring components for telemetry collection and monitoring. Supported capabilities include iDRAC Telemetry for out-of-band system metrics, LDMS Telemetry for in-band performance metrics, and air-gapped telemetry support for offline environments. For more details, see Deploy Telemetry. |
| Kubernetes Cluster High Availability | Delivers built-in high-availability (HA) failover for Service Kubernetes Cluster control plane nodes, ensuring uninterrupted cluster management and improved resilience for HPC workloads. For more details, see Configure High Availability. |
| Provisioning and Deployment Based on Functional Groups | Enables role-based provisioning for HPC clusters using mapping files. Automatically assigns functional roles and deploys customized operating system images tailored to workload-specific configurations. Supported roles: Login Node, Login Compiler Node, Slurm Node, Slurm Control Node, Service Kubernetes Node, and Service Kubernetes Control Plane. For more details, see Composable Roles. |
| Stateless Boot | Introduces stateless provisioning for RHEL 10 using OpenCHAMI, reducing deployment time and storage overhead for HPC clusters. For more details, see Provision Nodes. |
| Automatic CUDA Installation for GPU Workloads | Automatically installs CUDA during node provisioning, ensuring GPU-enabled nodes are ready for HPC workloads immediately after deployment. This reduces manual setup time and accelerates readiness for GPU-intensive applications. For more details, see Slurm with GPU. |
| Security Enhancements | Credentials are now encrypted using industry-standard algorithms (for example, AES-256), improving compliance with security best practices and reducing the risk of data exposure. For more details, see Product and Subsystem Security. |
| Platform Support | Supports x86_64 and aarch64 architectures, enabling deployment on both traditional and ARM-based HPC nodes for improved flexibility and energy efficiency. For more details, see Software Requirements. |
| Input Template and Validator | Provides predefined configuration templates and early input validation to reduce configuration errors and accelerate HPC cluster provisioning. This improves deployment reliability and overall user experience. For current task-specific procedures, see the module how-to guides. |