Prerequisites Checklist¶
Complete every item on this checklist before starting any deployment path. Skipping a prerequisite is the single most common cause of failed deployments.
Tip
Print this page and physically check off each item as you verify it in your datacenter. Hand it to your rack-and-stack technician alongside the server placement diagram.
Hardware Requirements¶
Servers¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Dell PowerEdge servers (16th or 17th generation) | Omnia 2.3.0.0 supports 16G and 17G PowerEdge servers. |
| ☐ | Intel 16G models | Supported: C6620, R660, R760, R760xa, R760xd2, R260, R360. Validated: R660, R260, C6620. |
| ☐ | AMD 16G models | Supported: R6625, R7625, R6615. Validated: R7625. |
| ☐ | AMD 17G models | Supported: R6725, R7725, R6715, R7715, R7725xd. Validated: R7725xd. |
| ☐ | NVIDIA Grace 17G models | Supported: XE8712 with GB200. Validated: XE8712 with GB200. |
| ☐ | OS installation method selected, when required | Install the OS manually or use the standalone unattended OS installation workflow, which supports both x86_64 and aarch64 through iDRAC Virtual Media. Verify the target installation disk before starting either method. |
NICs¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Supported NICs installed in every target node | At least one NIC port must be PXE-capable. |
| ☐ | Intel NICs | Supported: Ethernet 10G 4P X710/I350 rNDC, Ethernet Converged Network Adapter X710, 25GbE Ethernet Network Adapter E810, 100GbE Ethernet Network Adapter E810, I350GbE Ethernet Controller. |
| ☐ | Broadcom NICs | Supported: 10GBASE-T Ethernet, Gigabit Ethernet BCM5720, Adv Dual 10GBASE-T Ethernet, Adv Dual 25Gb Ethernet, NetXtreme Gigabit Ethernet, BCM5760x. |
Switches¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Network switches racked and cabled | One admin/data switch and one BMC management switch at minimum. InfiniBand switch required only if using IB fabric. |
| ☐ | Dell PowerSwitch models | Supported: S3048-ON, S5232F-ON, Z9264F-ON, N3248TE-ON, S4148, Z9664F, Z9432-ON, Z9864F-ON. Validated: S3048-ON, S5232F-ON, S4148, Z9432-ON. |
| ☐ | OS10 installed on Ethernet switches | Omnia requires OS10 on Ethernet switches. Switches running SONiC OS must be configured manually. |
Note
Switches that have reached EOL might not function properly. Use switch models listed in the support matrix.
Storage¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Dell PowerScale (if applicable) | Supported and validated models: H500, F600. |
| ☐ | Dell PowerVault (if applicable) | Supported and validated model: ME5084. |
OIM (Management Node) Requirements¶
The Omnia Infrastructure Manager (OIM) is the single node from which all Omnia playbooks execute. It does not participate in the compute cluster.
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Choose a server outside your intended cluster | The OIM must meet the required storage and system requirements. |
| ☐ | 64 GB RAM minimum | Verify with free -h. Local repositories, service containers, and image-building tasks are memory-intensive. |
| ☐ | RHEL 10.0 with Server with GUI Base Environment | RHEL 10.0 is the documented validated OIM baseline. Minimal installs are not supported. The GUI group pulls in required libraries used by Ansible and Podman. See supported operating systems. |
| ☐ | Podman 5.0 or later installed | Verify with podman --version. If missing, install with dnf install -y podman. |
| ☐ | Two active NIC ports | NIC 1 (public): Internet-facing, for downloading packages and container images. NIC 2 (internal/admin): Connected to the admin switch for PXE provisioning and cluster management. |
| ☐ | Internet access (direct or via proxy) | Required while the Repository Manager download workflow synchronizes OS packages, Python modules, and container images. After synchronization, air-gapped operation is possible. |
| ☐ | Git installed | dnf install git -y. Needed to clone the Omnia repository. |
| ☐ | 500 GB+ free disk on / | Local repos, container images, and node OS images consume significant space. Use df -h / to check. |
| ☐ | Required Omnia ports available | See Ports used by Omnia below for the complete list of ports and the systems on which they must be available. |
| ☐ | OIM setup completed | Complete Setup the OIM to install the shared environment and initialize the deployment modules. |
| ☐ | All target bare-metal servers reachable from OIM | Ensure network connectivity from OIM to all cluster nodes. |
Ports used by Omnia¶
Omnia uses the following ports across the OIM, service Kubernetes nodes, Slurm nodes, and other functional hosts. Ensure the applicable ports are not assigned to any other service, on the systems identified in the Functional Group/OIM column.
Container Ports
| Port | Protocol | Service | Functional Group/OIM |
|---|---|---|---|
| 2225 | TCP | Pulp Container (configurable using pulp_server_port) |
OIM |
OpenCHAMI Ports
| Port | Protocol | Service | Functional Group/OIM |
|---|---|---|---|
| 5432 | TCP | postgres | OIM |
| 27778 | TCP | OpenCHAMI compatibility port reserved by the current deployment | OIM |
| 27779 | TCP | smd | OIM |
| 8081 | TCP | boot-service (iPXE boot script) | OIM |
| 8443 | TCP | OpenCHAMI HTTPS API gateway | OIM |
| 67, 68 | UDP | CoreDHCP and PXE clients | OIM |
| 69 | UDP | TFTP | OIM |
| 53 | TCP, UDP | CoreDNS (when dns_enabled is true) |
OIM |
| 389, 636 | TCP | omnia_auth LDAP (when OpenLDAP is selected) |
OIM |
The object-store and container-registry ports are owned by Image Build Manager, not by the OpenCHAMI deployment. Internal Podman-network ports for local CA, TokenSmith, and metadata-service do not need to be exposed as OIM host ports.
Image Build Manager Ports
| Port | Protocol | Service | Functional Group/OIM |
|---|---|---|---|
| 9000 | TCP | MinIO S3 API (when local MinIO is selected) | OIM |
| 9001 | TCP | MinIO Console (when local MinIO is selected) | OIM |
| 5000 | TCP | OCI container registry | OIM |
Telemetry Ports
| Port | Protocol | Service | Functional Group/OIM |
|---|---|---|---|
| 6001–6100 | TCP | LDMS aggregator | Slurm nodes |
| 6001–6100 | TCP | LDMS store daemon | Slurm nodes |
| 10001–10100 | TCP | LDMS sampler | Slurm nodes |
| 9092 | TCP | Kafka internal (TLS) | Service Kubernetes nodes |
| 9093 | TCP | Kafka internal (mTLS) | Service Kubernetes nodes |
| 9094 | TCP | Kafka external LoadBalancer (mTLS) | Service Kubernetes nodes |
| 8480 | TCP | VictoriaMetrics vminsert LB | Service Kubernetes nodes |
| 8481 | TCP | VictoriaMetrics vmselect LB | Service Kubernetes nodes |
| 8482 | TCP | VictoriaMetrics vmstorage health | Service Kubernetes nodes |
| 8889 | TCP | OTEL Collector (Prometheus) | Service Kubernetes nodes |
| 9090 | TCP | CSI volume exporter (PowerScale metrics) | Service Kubernetes nodes |
| 514 | TCP, UDP | PowerScale Syslog receiver (VLAgent) | Service Kubernetes nodes |
| 9481 | TCP | VictoriaLogs vlinsert | Service Kubernetes nodes |
| 9471 | TCP | VictoriaLogs vlselect | Service Kubernetes nodes |
| 9491 | TCP | VictoriaLogs vlstorage (HTTP) | Service Kubernetes nodes |
| 9400 | TCP | VictoriaLogs vlstorage insert | Service Kubernetes nodes |
| 9401 | TCP | VictoriaLogs vlstorage select | Service Kubernetes nodes |
| 9429 | TCP | VictoriaLogs vlagent | Service Kubernetes nodes |
| 3306 | TCP | iDRAC telemetry MySQL | Service Kubernetes nodes |
| 33060 | TCP | iDRAC telemetry MySQL X Protocol | Service Kubernetes nodes |
| 8161 | TCP | iDRAC telemetry ActiveMQ HTTP | Service Kubernetes nodes |
| 61616 | TCP | iDRAC telemetry ActiveMQ OpenWire | Service Kubernetes nodes |
| 61613 | TCP | iDRAC telemetry ActiveMQ STOMP | Service Kubernetes nodes |
| 8082 | TCP | iDRAC telemetry Config UI | Service Kubernetes nodes |
BuildStreaM Ports
| Port | Protocol | Service | Functional Group/OIM |
|---|---|---|---|
| 8010 | TCP | BuildStreaM Manager (BSM) API (configurable using build_stream_port) |
OIM |
| 443 | TCP | GitLab HTTPS (configurable using gitlab_https_port) |
GitLab host |
| 5432 | TCP | PostgreSQL (containerized) | OIM |
Networking Prerequisites¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Admin and BMC switches configured and reachable | Ensure admin and BMC switches are properly configured. |
| ☐ | Admin network switch configured | A dedicated VLAN or flat L2 segment connecting OIM NIC 2 to all target-node admin NICs. DHCP must not already be running on this segment (Omnia provides its own). |
| ☐ | BMC network switch configured | A separate VLAN or segment connecting OIM to all target-node iDRAC BMC ports. Can share a physical switch with admin if VLANs are used. |
| ☐ | IP ranges planned for admin and BMC subnets | You will enter these CIDRs in network_spec.yml. Example: admin 10.5.0.0/16, BMC 10.3.0.0/16. |
| ☐ | No conflicting DHCP servers on admin or BMC subnets | OpenCHAMI CoreDHCP is deployed by the Orchestrator prepare phase. Ensure that no other DHCP server serves the PXE/admin-network address ranges configured in network_spec.yml. |
| ☐ | InfiniBand Subnet Manager running (if using IB fabric) | Ensure the Subnet Manager (SM) service is enabled and running on the InfiniBand switch or host. Failure to meet this prerequisite may result in InfiniBand ports remaining in the Initializing state. |
| ☐ | DNS resolution working on OIM | nslookup google.com must succeed. Configure /etc/resolv.conf or NetworkManager DNS if needed. |
NFS / Storage Prerequisites¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | NFS server reachable from OIM and all cluster nodes | Dell PowerScale (H500, F600), PowerVault (ME5084), or any NFS-capable appliance. Choose an NFS server located outside your cluster. |
| ☐ | NFS share accessible from all nodes | Ensure the external NFS share is accessible from all nodes (both diskless and diskful) and is reachable via the admin network. |
| ☐ | Export permissions: 755, no_root_squash |
no_root_squash is required so Ansible (running as root) can write to NFS-mounted paths. To enable, edit /etc/exports: /<your_exported_path> *(rw,sync,no_root_squash,no_subtree_check) |
| ☐ | PowerScale NFSv4 settings (if using NFSv4) | If PowerScale is configured with NFSv4, enable the following to prevent nobody:nobody ownership: nfsv4-no-names=true, nfsv4-no-domain=true, nfsv4-no-domain-uids=true, nfsv4-allow-numeric-ids=true. See Dell KB 000023023. |
| ☐ | Minimum storage capacity allocated | Kubernetes NFS: 200 GB | Slurm NFS: 50 GB | OIM NFS: 200 GB. Increase based on cluster size. |
| ☐ | Dedicated mount point for each NFS | Ensure there is a dedicated mount point for each NFS (K8s, Slurm, OIM). |
| ☐ | NFS exports tested from OIM | mount -t nfs <nfs_server>:/export /mnt/test && ls /mnt/test |
PowerScale NFSv4 CLI Configuration
To configure PowerScale NFSv4 settings via CLI, run:
isi nfs settings zone modify \
--nfsv4-no-names=true \
--nfsv4-no-domain=true \
--nfsv4-no-domain-uids=true \
--nfsv4-allow-numeric-ids=true \
--zone=System
Important considerations:
- Ensure UID/GID mappings are consistent across all NFS client nodes and the PowerScale cluster. Numeric ID mode bypasses name-based identity resolution, so mismatched UIDs/GIDs will result in incorrect file ownership.
- This setting degrades NFSv4 ACL support. If your environment requires NFSv4 ACLs, consider aligning the NFSv4 domain between PowerScale and all clients instead.
PowerScale S3 Storage (if applicable)¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | PowerScale cluster within admin subnet | Must be accessible from all cluster nodes. |
| ☐ | S3 and HTTP services enabled | Omnia uses HTTP access only (default port 9020). |
| ☐ | Valid S3 Access Key ID and Secret Access Key | Required for authentication. S3 keys are tightly associated with S3 buckets. |
RHEL Subscriptions and Repositories¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | RHEL subscription active on OIM | subscription-manager status must show Registered. Required for AppStream, BaseOS, and codeready-builder repos. |
| ☐ | Docker Hub credentials available | Provide Docker Hub credentials to Repository Manager when anonymous pulls would exceed registry limits or the selected catalog uses private content. |
| ☐ | OIM has access to public network | Required to download and store packages/images to the desired NFS share. |
| ☐ | Certificates stored using Ansible Vault | Ensure all required certificates are stored using Ansible Vault for confidentiality and integrity within the cluster. |
| ☐ | All repository URLs accessible | Verify that all repository URLs for software packages are accessible. If not, the download will fail for that specific package. |
| ☐ | OIM RHEL release pinned to 10.0 | By default, an active RHEL subscription may configure the repository to a later point release. Pin the OIM to the documented validated RHEL 10.0 baseline. Verify and set: subscription-manager release --show and sudo subscription-manager release --set=10.0. The cluster-node version is selected separately by the catalog and must match an approved combination in the Operating Systems Matrix. |
BIOS Settings on Target Nodes¶
Apply these settings to every target node (compute, head, login, K8s) via iDRAC or BIOS Setup (F2 at POST).
| ☑ | Setting | Value |
|---|---|---|
| ☐ | System Profile (Performance) | Set System Profile to Performance in BIOS > System Profile Settings. This maximizes CPU frequency and disables power-saving C-states. |
| ☐ | Power Cap disabled | BIOS > System Profile Settings > Power Cap Policy: Disabled. Power capping can throttle CPUs during Slurm jobs. |
| ☐ | PXE boot enabled on admin NIC | BIOS > Network Settings > NIC Configuration > enable PXE Boot on the NIC connected to the admin switch. Disable PXE on all other NICs to avoid boot-order confusion. |
| ☐ | Boot order: NIC first, then disk | BIOS > Boot Settings > Boot Sequence: move the PXE-enabled NIC above the hard drive. After initial provisioning, Omnia configures disk-first boot automatically. |
| ☐ | Virtualization Technology (VT-x/VT-d) enabled | Required for K8s nodes running containerized workloads. |
| ☐ | SR-IOV enabled (if using SR-IOV NICs) | BIOS > Integrated Devices > SR-IOV Global Enable: Enabled. |
iDRAC Settings¶
| ☑ | Setting | Value |
|---|---|---|
| ☐ | Redfish API enabled | iDRAC Settings > Network > Services: Redfish enabled. Omnia uses Redfish for out-of-band discovery and firmware inventory. |
| ☐ | iDRAC firmware updated to latest version | Download from Dell Support. |
| ☐ | Datacenter license installed (for telemetry) | The Datacenter license enables streaming telemetry via iDRAC. Enterprise license is insufficient. |
| ☐ | iDRAC IP assigned on BMC network | Can be DHCP (Omnia will assign) or static. If static, record each iDRAC IP for the mapping file. |
| ☐ | Current iDRAC credentials known | Provide them through the Orchestrator credential workflow when prompted. |
Service Kubernetes (K8s) Requirements¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Minimum 3 Kubernetes controller nodes allocated | Kubernetes HA requires an odd number of control-plane nodes (3 or 5). Each must have 64 GB RAM minimum. |
| ☐ | At least 1 kube node allocated | Kube nodes run telemetry collectors and monitoring services such as VictoriaMetrics. 64 GB RAM minimum. |
| ☐ | Dedicated IP range for K8s pod and service networks | Defaults: pod CIDR 10.244.0.0/16, service CIDR 10.96.0.0/12. These must not overlap with admin or BMC subnets. |
| ☐ | Virtual IP (VIP) reserved for K8s API HA | A single unused IP on the admin network that will float across control-plane nodes. |
Slurm Prerequisites¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Each Slurm compute node has at least 64 GB RAM | Verify with free -h. |
| ☐ | Slurm RPMs available in user repository | If not available, build and host the Slurm repository. |
| ☐ | Slurm repository configured | For each required architecture, set repositories."10.0".<architecture>.user_repos.slurm_custom.url in $OMNIA_DATA_PATH/repo_manager/input/$OMNIA_PROJECT_NAME/repo_manager_config.yml. The selected catalog must reference slurm_custom. |
| ☐ | Mixed architecture: Slurm binaries for both architectures | In environments with x86_64 control nodes and aarch64 compute nodes, ensure Slurm binaries for both architectures are compiled and available. |
| ☐ | Slurm RPM names match the catalog | Verify that the Slurm package names in the selected catalog match the RPMs hosted in the slurm_custom repository. |
CUDA and DCGM Prerequisites (for Slurm GPU Nodes)¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | NVIDIA GPU hardware present | Must be present on any Slurm node intended for GPU workloads. Nodes without GPU hardware are automatically skipped. |
| ☐ | GPU packages synchronized | Synchronize the selected RHEL 10.0 Slurm catalog through Repository Manager. It must provide the NVIDIA driver, CUDA toolkit, DCGM, and matching kernel-development packages for each target architecture. |
| ☐ | GPU repositories reachable | Confirm Repository Manager completed successfully and Slurm compute nodes can reach the repositories recorded in repo_status.yml. |
| ☐ | DCGM installation setting reviewed | DCGM installation on GPU-capable Slurm nodes is controlled by dcgm_enabled in the active project's Orchestrator orchestrator_config.yml. The shipped value is true. |
| ☐ | NFS path for HPC tools reachable | The shared NFS path for Slurm HPC tools must be reachable from all Slurm compute and login/compiler nodes. Minimum 30 GB recommended for hpc_tools/cuda. The NFS share must be exported with no_root_squash. |
Note
During provisioning, Omnia attempts to build and load nvidia-peermem
through DKMS on Slurm nodes that have an NVIDIA GPU and a working NVIDIA
driver. Ensure that matching kernel headers are available. Nodes without
an NVIDIA GPU or working driver skip this operation. The module is required
only for workloads that use GPUDirect RDMA.
HPC Benchmark Image Layer¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Slurm shared storage available | Ensure the Slurm shared storage path (/hpc_tools) is available and accessible from all Slurm compute and login/compiler nodes. |
| ☐ | Local repository content prepared | Ensure the local repository content is prepared before runtime staging. |
| ☐ | Sufficient storage capacity | Allocate adequate storage for benchmark artifacts pulled from the local Pulp mirror to /hpc_tools/ |
Note
The HPC Benchmark Image Layer is runtime script-driven. Provisioning deploys pull_benchmarks.sh and benchmark_tools.list to /hpc_tools/scripts. Runtime staging is executed via /hpc_tools/scripts/pull_benchmarks.sh. The feature is staging-only; Omnia does not compile or execute benchmark workloads.
LDAP Prerequisites¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | OpenLDAP selected in the catalog | Include the OpenLDAP group when centralized authentication is required. Orchestrator derives OpenLDAP enablement from the selected catalog. |
| ☐ | OpenLDAP deployment inputs available | Configure SYSTEM_DOMAIN_NAME, SYSTEM_ADMIN_NIC_IPV4, and ldap_connection_type, and have the OpenLDAP database credentials available when Orchestrator prompts for them. See Deploy OpenLDAP. |
Telemetry Prerequisites¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | iDRAC Datacenter license installed | The Datacenter license enables streaming telemetry via iDRAC. Enterprise license is insufficient for iDRAC telemetry. |
| ☐ | Telemetry ports available | Ensure telemetry ports are accessible on the applicable systems (see Ports used by Omnia for the complete list). |
| ☐ | S3 storage configured for telemetry data | If using PowerScale S3, ensure it is configured and accessible from the OIM. |
LDMS Prerequisites (for HPC Telemetry)¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | EPEL and AppStream repositories configured | Ensure python3-devel and python3-Cython are installed: sudo dnf install -y python3-devel python3-Cython |
| ☐ | LDMS RPM available in user repository | If not available, build and host the LDMS producer RPM repository. Verify that the selected catalog references ldms and lists the hosted LDMS packages. |
| ☐ | LDMS repository configured | For each required architecture, set repositories."10.0".<architecture>.user_repos.ldms.url in $OMNIA_DATA_PATH/repo_manager/input/$OMNIA_PROJECT_NAME/repo_manager_config.yml. |
iDRAC Telemetry Prerequisites (for Service Cluster)¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | iDRAC firmware updated to latest version | Download from Dell Support. |
| ☐ | iDRAC Redfish API enabled | iDRAC Settings > Network > Services: Redfish enabled. |
| ☐ | Telemetry configuration file prepared | Configure telemetry_config.yml with iDRAC telemetry sources enabled. |
Note
For detailed telemetry configuration, see Configure Telemetry.
BuildStreaM Prerequisites¶
| ☑ | Requirement | Details |
|---|---|---|
| ☐ | Dedicated node for BuildStreaM GitLab deployment | A separate node is required for GitLab. |
| ☐ | Minimum system resources for BuildStreaM node | 4 GB RAM, 2 CPU cores, 20 GB free disk space. GitLab requires a minimum of 2 CPU cores; more may be needed for production workloads. |
| ☐ | Network connectivity for GitLab services | Ensure the BuildStreaM node has network access. |
| ☐ | BuildStreaM containers deployed on OIM | Ensure that Omnia BuildStreaM container, PostgreSQL container, and Playbook Watcher service are deployed. See Prepare OIM. |
| ☐ | 200 GB free disk space on OIM / partition | Required before triggering any BuildStreaM build pipeline to prevent "No space left on device" errors during image builds. |
Final Pre-Flight Checks¶
Run these commands on the OIM before starting any deployment path:
Run on OIM (as root)
# Verify OS and kernel
cat /etc/redhat-release
uname -r
# Verify RAM (expect >= 64 GB)
free -h | grep Mem
# Verify disk space
df -h
# Verify Podman
podman --version
# Verify Git
git --version
# Verify NICs (expect at least 2 interfaces up)
ip -br link show | grep UP
# Verify internet connectivity
curl -s -o /dev/null -w "%{http_code}" https://github.com
# Verify RHEL subscription
subscription-manager status
# Verify RHEL version pinning
subscription-manager release --show
# Verify repos
dnf repolist
Warning
If any of the above checks fail, resolve the issue before proceeding. Deploying with unmet prerequisites will produce difficult-to-debug errors deep in the Ansible playbook execution.
You are now ready to choose your deployment path. Return to Get Started Index.