| WEKA version | Description of changes |
|---|---|
| 5.1.31 | Multi-tenancy:
|
| 5.1.30 |
|
Local WEKA Home 5.0:
| |
| |
| |
| |
| 5.1.22 |
|
| 5.1.21 |
|
| |
| 5.1.20 |
|
| |
| 5.1 |
|
The NeuralMesh architecture components
NeuralMesh software-based storage architecture
Failure domain in action (example)
Data reduction capacity saving
Data reduction process during write at a glance
Cluster CPU statistics
Data reduction fingerprints and ingest rate
| File size | Number of metadata units | Example |
|---|---|---|
| < 0.5 MB | 1 | A filesystem containing 1 billion files, each sized at 64 KB, requires 1 billion metadata units. |
| 0.5 MB - 1 MB | 2 | A filesystem containing 1 billion files, each sized at 750 KB, requires 2 billion metadata units. |
| > 1 MB | 2 for the first 1 MB plus 1 per MB for the rest MBs |
|
| Role | Description |
|---|---|
| IP-only | Handles management process and data-path IP traffic. RDMA is not used on this adapter. |
| IP and RDMA | Handles both IP traffic and RDMA operations. |
| RDMA-only | Dedicated exclusively to RDMA traffic. Does not carry IP traffic. |
MCB architecture
| CPU family/architecture | Supported on backends | Supported on clients |
|---|---|---|
| Intel Xeon E5 v3 (Haswell) through Xeon 6 (Granite Rapids / Sierra Forest) with up to 256 total CPU cores | 👍 Dual-socket | 👍 Single-socket and dual-socket |
| AMD EPYC™ processor families 2nd (Rome) through 5th (Turin) Generations with up to 256 total CPU cores | 👍 Single-socket | 👍 Single-socket and dual-socket |
| ARM (AArch64) - NVIDIA Grace single or dual-processor with up to 144 total CPU cores | 👍 NVIDIA Grace | 👍 NVIDIA Grace |
| Adapter | Protocol | Supported features | Unsupported features |
|---|---|---|---|
| Amazon ENA | Ethernet | ✅ SR-IOV VF | ❌ Single IP ❌ HA ❌ Routed network ❌ LACP ❌ Mixed networks ❌ RX interrupts ❌ RDMA ❌ IOMMU |
| NVIDIA Mellanox CX-8 | Ethernet InfiniBand | ✅ Mixed networks ✅ Single IP ✅ RX interrupts ✅ RDMA ✅ HA ✅ Routed network (ETH only) ✅ PKEY (IB only) ✅ IOMMU | ❌ LACP ❌ SR-IOV VF |
| NVIDIA Mellanox CX-7 single-port | InfiniBand | ✅ Single IP ✅ RX interrupts ✅ RDMA ✅ HA ✅ PKEY ✅ IOMMU | ❌ Mixed networks ❌ SR-IOV VF ❌ Routed network |
| NVIDIA Mellanox CX-7 dual-port | InfiniBand | ✅ Single IP ✅ RX interrupts ✅ RDMA ✅ HA ✅ PKEY ✅ IOMMU | ❌ Mixed networks ❌ SR-IOV VF ❌ Routed network |
| NVIDIA Mellanox CX-7-ETH single-port | Ethernet | ✅ Single IP ✅ RDMA ✅ HA ✅ Routed network ✅ IOMMU | ❌ LACP ❌ Mixed networks ❌ SR-IOV VF ❌ RX interrupts |
| NVIDIA Mellanox CX-7-ETH dual-port | Ethernet | ✅ LACP ✅ Single IP ✅ RDMA ✅ HA ✅ Routed network ✅ IOMMU | ❌ Mixed networks ❌ SR-IOV VF ❌ RX interrupts |
| NVIDIA Mellanox CX-6 LX | Ethernet | ✅ Single IP ✅ RDMA ✅ RX interrupts ✅ HA ✅ Routed network ✅ IOMMU | ❌ LACP ❌ Mixed networks ❌ SR-IOV VF |
| NVIDIA Mellanox CX-6 DX | Ethernet | ✅ LACP ✅ Single IP ✅ RX interrupts ✅ RDMA ✅ HA ✅ Routed network ✅ IOMMU | ❌ Mixed networks ❌ SR-IOV VF |
| NVIDIA Mellanox CX-6 | Ethernet InfiniBand | ✅ Mixed networks ✅ Single IP ✅ RX interrupts ✅ RDMA ✅ HA ✅ IOMMU | ❌ Routed network ❌ LACP ❌ SR-IOV VF |
| NVIDIA Mellanox CX-5 EX | Ethernet InfiniBand | ✅ Mixed networks ✅ RDMA ✅ HA ✅ PKEY (IB only) ✅ IOMMU | ❌ Single IP ❌ Routed network ❌ LACP ❌ SR-IOV VF ❌ RX interrupts |
| NVIDIA Mellanox CX-5 BF | Ethernet | ✅ Mixed networks ✅ RDMA ✅ HA ✅ IOMMU | ❌ Single IP ❌ Routed network ❌ LACP ❌ SR-IOV VF ❌ RX interrupts |
| NVIDIA Mellanox CX-5 | Ethernet InfiniBand | ✅ Mixed networks ✅ RX interrupts ✅ RDMA ✅ HA ✅ Routed network (ETH only) ✅ PKEY (IB only) ✅ IOMMU | ❌ Single IP ❌ LACP ❌ SR-IOV VF ❌ Routed network (IB) |
| VirtIO | Ethernet | ✅ HA ✅ Routed network | ❌ Mixed networks ❌ Single IP ❌ LACP ❌ RX interrupts ❌ SR-IOV VF ❌ IOMMU |
| Purpose | Source | Target | Target Ports | Protocol | Comments |
|---|---|---|---|---|---|
| WEKA server traffic for bare-metal deployments | All WEKA backend IPs | All WEKA backend IPs | 14000-14059 (drives) 14200-14259 (frontend) 14300-14359 (compute) | TCP and UDP TCP and UDP TCP and UDP | These ports are the default for the Resources Generator for the first three containers. You can customize the ports. |
| WEKA client traffic | Client host IPs | All WEKA backend IPs | 14000-14059 (drives) 14300-14359 (compute) | TCP and UDP TCP and UDP | These ports are the default. You can customize the ports. |
| WEKA backend to client traffic | All WEKA backend IPs | Client host IPs | 14000-14059 (frontend) | TCP and UDP | These ports are the default. You can customize the ports. |
| WEKA SSH management traffic | All WEKA backend IPs | All WEKA backend IPs | 22 | TCP | |
| WEKA server traffic for cloud deployments | All WEKA backend IPs | All WEKA backend IPs | 14000-14059 (drives) 15000-15059 (compute) 16000-16059 (frontend) | TCP and UDP TCP and UDP TCP and UDP | These ports are the default. You can customize the ports. |
| WEKA client traffic (on cloud) | Client host IPs | All WEKA backend IPs | 14000-14059 (drives) 15000-15059 (compute) | TCP and UDP TCP and UDP | These ports are the default. You can customize the ports. |
| WEKA backend to client traffic (on cloud) | All WEKA backend IPs | Client host IPs | 14000-14059 (frontend) | TCP and UDP | These ports are the default. You can customize the ports. |
| WEKA GUI access | Admin workstation IPs | All WEKA management IPs | 14000 | TCP | User web browser IP |
| NFS | NFS client IPs | WEKA NFS backend IPs | 2049 <mountd port> | TCP and UDP TCP and UDP | You can set the mountd port using the command: weka nfs global-config set --mountd-port |
| NFSv3 (used for locking) | NFS client IPs | WEKA NFS backend IPs | 46999 (status monitor) 111 (rpcbind) | TCP and UDP | |
| SMB/SMB-W | SMB client IPs | WEKA SMB backend IPs | 139 445 | TCP TCP | |
| SMB-W | All WEKA SMB-W backend IPs | All WEKA SMB-W backend IPs | 2224 | TCP | This port is required for internal clustering processes. |
| SMB/SMB-W | WEKA SMB backend IPs | All Domain Controllers for the selected Active Directory Domain | 88 389 | TCP and UDP TCP and UDP TCP and UDP TCP and UDP TCP and UDP TCP and UDP | These ports are required for SMB/SMB-W to use Active Directory as the identity source. Furthermore, every Domain Controller within the selected AD domain must be accessible from the WEKA SMB servers. |
| SMB/SMB-W | WEKA SMB backend IPs | DNS servers | 53 | TCP and UDP | |
| S3 | S3 client IPs | WEKA S3 backend IPs | 9000 | TCP | This port is the default. You can customize the port. |
| wekatester | All WEKA backend IPs | All WEKA backend IPs | 8501 9090 | TCP TCP | Port 8501 is used by wekanetperf. |
| WEKA Management Station | User web browser IP | WEKA Management Station IP | 80 <LWH> 443 <LWH> 3000 <mon> 7860 <admin UI> 8760 <deploy> 8090 <snap> 8501 <mgmt> 9091 <mon> | HTTP HTTPS TCP TCP TCP TCP TCP | |
| Cloud WEKA Home, Local WEKA Home | All WEKA backend IPs | Cloud WEKA Home or Local WEKA Home | 80 443 | HTTP HTTPS | Open according to the directions in the deployment scenario:
|
| Client telemetry and statistics | WEKA client IPs | Cloud WEKA Home or Local WEKA Home | 80 443 | HTTP HTTPS | Lack of connectivity prevents client-related statistics from being reported. |
| Troubleshooting by the Customer Success Team (CST) | All WEKA backend IPs | CST remote access | 4000 4001 | TCP TCP | |
| Traces remote viewer | All WEKA backend IPs | CST remote access | 443 | TCP | |
| KMS: Vault or OpenBao | All WEKA backend IPs | Vault/OpenBao server | 8200 8201 | TCP TCP | Default vault ports: 8200 is configurable for client requests, while 8201 (base_port+1) handles internal cluster communication. |
| KMS: KMIP | All WEKA backend IPs | KMIP server | 5696 | TCP | The default KMIP port, 5696, is configurable. Per the KMIP specification, servers must use this port when operating with the TTLV encoding format. |
| Object Store | Storage Class / Version | Supportability notes |
|---|---|---|
| Amazon S3 | S3 Standard S3 Intelligent-Tiering | Tiering and snap-to-object |
| S3 Standard-IA S3 One Zone-IA S3 Glacier Instant Retrieval | Snap-to-object supported; tiering not recommended (slow retrieval, storage period, access costs). Best for backups/DR. Use Intelligent-Tiering if unsure. | |
| Azure Blob Storage | Tiering and snap-to-object | |
| Cloudian HyperStore | 7.3 | Tiering and snap-to-object |
| CoreWeave AI Object Storage (CAIOS) | Snap-to-object; tiering not supported | |
| Google Cloud Storage (GCS) | Tiering and snap-to-object | |
| Dell EMC ECS | 3.5 | Tiering and snap-to-object |
| Dell PowerScale S3 | 9.8.0.0 | Tiering and snap-to-object (all-flash models only) |
| HCP Classic | 9.2+ (versioned buckets) | Tiering and snap-to-object |
| HCP for Cloud-Scale | 2.x | Tiering and snap-to-object |
| IBM Cloud Object Storage | 3.14.7 | Tiering and snap-to-object |
| Lenovo MagnaScale | 3 | Tiering and snap-to-object |
| Quantum ActiveScale | 5.5.1 | Tiering and snap-to-object |
| Red Hat Ceph Storage | 5 | Tiering and snap-to-object |
| Scality Artesca | 1.5.2 | Tiering and snap-to-object |
| Scality RING S3 Connector | 8.5 | Tiering and snap-to-object |
| Scality RING WEKA Connector | 9.5 | Tiering and snap-to-object |
| SwiftStack | 6.3 | Tiering and snap-to-object |
| WEKA S3 | Tiering and snap-to-object |
High-level deployment workflow
| Purpose | Per-server memory |
|---|---|
| Fixed | 2.61 GiB |
| Frontend processes | 2.05 GiB × # of Frontend processes |
| Compute processes | 3.63 GiB × # of Compute processes |
| Drive processes | 1.86 GiB × # of Drive processes |
| SSD capacity management | (Total SSD raw capacity in GiB ÷ Number of Servers ÷ 2,000) + (Number of Cores × 2.79 GiB) |
| Operating System | The maximum between 7.45 GiB and 2% of the total RAM |
| Additional protocols (NFS/SMB/S3) | 14.9 GiB |
| RDMA | 1.86 GiB |
| Metadata (pointers) | 20 Bytes × # Metadata units per server See Metadata units calculation. |
| Dedicated Data Services container | If you intend to add a Data Services container for background tasks, it requires additional memory:
|
| Purpose | Per-client memory |
|---|---|
| Base | 3 GiB |
| Frontend (FE) processes | 2.5 GiB × number of FE processes |
| HugePages (configured) | Container HugePages |
| HugePages (default) | 1.4 GiB × number of FE processes |
| OS and kernel cache | See note below |
get.weka.io dashboard
Releases download page
WEKA cluster installation using the WSA
Example: weka status with STEM mode
| RHEL and derivatives | Ubuntu |
|---|---|
| |
mst start && for MLXDEV in /dev/mst/* ; do mlxconfig -d ${MLXDEV} -y set ADVANCED_PCI_SETTINGS=1 PCI_WR_ORDERING=1; done
# --- Disable NUMA balancing (Mandatory) ---
kernel.numa_balancing = 0
# --- Configure automatic reboot on kernel panic ---
kernel.panic = 300
# --- Minimal configuration per specific IB/Eth interface ---
# Replace ib0 and ib1 with your specific interface names
net.ipv4.conf.ib0.arp_announce = 2
net.ipv4.conf.ib1.arp_announce = 2
net.ipv4.conf.ib0.arp_filter = 1
net.ipv4.conf.ib1.arp_filter = 1
net.ipv4.conf.ib0.arp_ignore = 1
net.ipv4.conf.ib1.arp_ignore = 1
# --- Alternative network ARP settings for all interfaces ---
net.ipv4.conf.all.arp_filter = 1
net.ipv4.conf.default.arp_filter = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.all.ignore_routes_with_linkdown = 1
| Parameter | Value | Notes |
|---|---|---|
| Marking method | Layer 3 (DSCP) | Required when WEKA traffic is routed across subnets. Layer 2 (TOS) marking works only when all servers share a subnet, but WEKA doesn't recommend it. |
| DSCP value | 24 | Register value 96 (DSCP x 4) on the NIC. |
| Traffic class, data | 3 | Maps to switch priority 3. |
| Traffic class, congestion notification (CNP) | 6 | Maps to switch priority 6. |
| PFC priority | 3 | Enable Priority Flow Control on this priority so RoCE traffic isn't dropped. |
| Congestion control | ECN | Enable alongside PFC. |
| Goal | Command |
|---|---|
| Full details (IP, DNS, routes) | nmcli device show |
| Brief status | nmcli device status |
| Active connections | nmcli connection show --active |
| Specific device | nmcli device show eth0 |
# Host: weka-node-01
# Apply: sudo netplan apply
network:
version: 2
renderer: networkd
ethernets:
enp2s0:
dhcp4: yes
dhcp6: no
nameservers:
addresses: [8.8.8.8, 8.8.4.4]
# Default route via DHCP — no SBR policy needed on management interface
ib1:
dhcp4: no
dhcp6: no
optional: true
ignore-carrier: true # configure even without physical link (IB/RDMA NICs may be slow to init)
addresses:
- 10.222.0.10/24
routes:
# Main table entries
- to: 10.222.0.0/24
scope: link
# ── Additional subnets reachable via ib1 (main table) ──────────────
# Add any other CIDRs that hosts/clients on this fabric need to reach.
# Without a main-table entry, locally-originated traffic (e.g. weka
# backend<->backend, S3, SMB, NFS) may egress the wrong interface.
# - to: <ADDITIONAL_SUBNET>
# via: 10.222.0.1
# Source-based routing table 101
- to: 0.0.0.0/0
via: 10.222.0.1
table: 101
- to: 10.222.0.0/24
scope: link
table: 101
routing-policy:
- from: 10.222.0.10/32
table: 101
priority: 1010
ib2:
dhcp4: no
dhcp6: no
optional: true
ignore-carrier: true # configure even without physical link (IB/RDMA NICs may be slow to init)
addresses:
- 10.222.0.20/24
routes:
# Main table entries
- to: 10.222.0.0/24
scope: link
# ── Additional subnets reachable via ib2 (main table) ──────────────
# Add any other CIDRs that hosts/clients on this fabric need to reach.
# Without a main-table entry, locally-originated traffic (e.g. weka
# backend<->backend, S3, SMB, NFS) may egress the wrong interface.
# - to: <ADDITIONAL_SUBNET>
# via: 10.222.0.1
# Source-based routing table 102
- to: 0.0.0.0/0
via: 10.222.0.1
table: 102
- to: 10.222.0.0/24
scope: link
table: 102
routing-policy:
- from: 10.222.0.20/32
table: 102
priority: 1020
Main screen
Advanced screen
Enable SR-IOV
Save and Exit
Example: wekaconfig detection results
config.sh output exampleApply the configuration
| Name | Value | Default |
|---|---|---|
compute-core-ids | Specify the CPUs to allocate for the compute processes. Format: space-separated numbers | |
compute-dedicated-cores | Specify the number of cores to dedicate for the compute processes. | The maximum available cores |
compute-memory | Specify the total memory to allocate for the compute processes. Format: value and unit without a space. Examples: 1024B, 10GiB, 5TiB. | The maximum available memory |
core-ids | Specify the CPUs to allocate for the WEKA processes. Format: space-separated numbers. | |
drive-core-ids | Specify the CPUs to allocate for the drive processes. Format: space-separated numbers. | |
drive-dedicated-cores | Specify the number of cores to dedicate for the drive processes. | 1 core per each detected drive |
drives | Specify the drives to use. This option overrides automatic detection. | All unmounted NVME devices |
frontend-core-ids | Specify the CPUs to allocate for the frontend processes. Format: space-separated numbers. | - |
frontend-dedicated-cores | Specify the number of cores to dedicate for the frontend processes. | 1 |
max-cores-per-container | Override the default maximum number of cores per container for IO processes (19). If provided, the new value must be lower. | 19 |
minimal-memory | Set each container's hugepages memory to 1.4 GiB * number of IO processes on the container. | |
net* | Specify the network devices to use. Format: space-separated strings. | |
no-rdma | Don't take RDMA support into account when computing memory requirements. | False |
num-cores | Override the auto-deduction of the number of cores. | All available cores |
path | Specify the path to write the resource files. | '.' |
spare-cores | Specify the number of cores to leave for OS and non-WEKA processes. | 1 |
spare-memory | Specify the memory to reserve for non-WEKA requirements. Argument format: a value and unit without a space. Examples: 10GiB, 1024B, 5TiB. | The maximum between 8 GiB and 2% of the total RAM |
weka-hugepages-memory | Specify the memory to allocate for compute, frontend, and drive processes. Argument format: a value and unit without a space. Examples: 10GiB, 1024B, 5TiB. | The maximum available memory |
| Parameter | Description |
|---|---|
hostnames* | (Required) Hostnames or IP addresses, separated by spaces. If port 14000 is not the default for the drives, you can specify hostnames:port or ips:port. |
host-ips | IP addresses of the management interfaces, separated by commas.
|
| Name | Value |
|---|---|
container-id* | The Identifier of the drive container to add the local SSD drives. |
device-paths* | List of block devices that identify local SSDs. It must be a valid Unix network device name. Format: Space-separated strings. Example, /dev/nvme0n1 /dev/nvme1n1 |
| Name | Value |
|---|---|
resources-path* | A valid path to the resource file. |
join-ips |
To restrict the client’s operations to only the essential APIs for mounting and unmounting operations, connect to WEKA clusters through TCP base port + 3 (for example, 14003). The |
| Name | Value |
|---|---|
resources-path* | A valid path to the resource file. |
join-ips | IP:port pairs for the management processes to join the cluster. In the absence of a specified port, the command defaults to using the standard WEKA port 14000. Set the values, only if you want to customize the port. Format: comma-separated IP addresses. Example: --join-ips 10.10.10.1,10.10.10.2,10.10.10.3:15000 |
| Parameter | Description |
|---|---|
range* | A range of IP addresses reserved for dynamic allocation across the entire cluster.. Example: |
netmask-bits* | Number of bits in the netmask that define a network ID in CIDR notation. |
gateway | The IP address assigned to the default routing gateway. It is imperative that the gateway resides within the same IP network as defined by the specified range and netmask-bits. This parameter is not applicable to InfiniBand (IB) or Layer 2 (L2) non-routable networks. |
| Parameter | Description |
|---|---|
-n, --name | The name to give the container. |
--cores | Number of CPU cores dedicated to WEKA. |
--memory | Memory dedicated to WEKA in bytes. Set to 0 to let the system decide. Use decimal or binary units, such as 1GB or 1GiB. |
--bandwidth | Bandwidth limit per second. Use unlimited or a decimal or binary value, such as 1GB or 1GiB. |
-t, --timeout | Join command timeout. Use values such as 3s, 2h, 4m, 1d, 1d5h, 1w, infinite, or unlimited. |
--base-port | First port used by the WEKA container. WEKA uses 100 ports starting at this port. |
--weka-version | WEKA version used to start the container. |
--fqdn | Fully qualified domain name used by other containers for TLS hostname verification. |
--nvidia-vf-single-ip | Configures NVIDIA virtual functions to use a single IP address. Use yes, no, true, false, on, off, y, or n. |
--dedicated-mode | Sets DPDK core dedication to full or none. none requires NIC driver support. |
--scan-rdma | Scans for unused network devices and adds them for RDMA use. Use off, ib, eth, or all. |
--color | Sets color output. Use auto, disabled, or enabled. |
--core-ids... | WEKA dedicated core IDs. Repeat the parameter or provide comma-separated values. |
--management-ips... | Management process IP addresses. Repeat the parameter or provide comma-separated values. |
--join-ips... | Management process IP:port pairs. If no port is specified, WEKA uses the default port. Repeat the parameter or provide comma-separated values. |
--net... | Network specification. Use a device name such as ib1 or eth1, <device>/<ip>/<bits>/<gateway>, <device>/rdma-only/inet4, or <device>/rdma-only/inet6. Use udp to force UDP mode. Repeat the parameter or provide comma-separated values. |
--disable | Creates the container as disabled. |
--no-start | Does not start the container after creation. |
--allow-mix-setting | Allows specified core IDs when containers with automatic core-ID allocation run on the same server. |
--restricted | Enables restricted client mode. |
CDM Web welcome
CDM Web Login
CDM Web main dashboard
CDM Web dashboard main sections
Basic configuration page
Basic configuration required fields
Instance Type
Region
WEKA Version
Terraform Module Release Version
WEKA Terraform module deployment on AWS
| EC2 instance type | Supported instances |
|---|---|
| i8ge | i8ge.2xlarge, i8ge.3xlarge, i8ge.6xlarge, i8ge.12xlarge, i8ge.18xlarge, i8ge.24xlarge, i8ge.48xlarge |
| I3en | i3en.2xlarge, i3en.3xlarge, i3en.6xlarge, i3en.12xlarge, i3en.24xlarge |
| EC2 instance type | Supported instances |
|---|---|
| M5 | m5.xlarge, m5.2xlarge, m5.4xlarge, m5.8xlarge, m5.12xlarge, m5.16xlarge, m5.24xlarge |
| M5n | m5n.xlarge, m5n.2xlarge, m5n.4xlarge, m5n.8xlarge, m5n.12xlarge, m5n.16xlarge, m5n.24xlarge, m5dn.xlarge, m5dn.2xlarge, m5dn.4xlarge, m5dn.8xlarge, m5dn.12xlarge, m5dn.16xlarge, m5dn.24xlarge |
| M6a | m6a.xlarge, m6a.2xlarge, m6a.4xlarge, m6a.8xlarge, m6a.12xlarge, m6a.16xlarge, m6a24xlarge, m6a.32xlarge, m6a.48xlarge |
| M6g | m6g.xlarge, m6g.2xlarge, m6g.4xlarge, m6g.8xlarge, m6g.12xlarge, m6g.16xlarge |
| M6gd | m6gd.xlarge, m6gd.2xlarge, m6gd.4xlarge, m6gd.8xlarge, 1m6gd.2xlarge, m6gd.16xlarge |
| M6i | m6i.xlarge, m6i.2xlarge, m6i.4xlarge, m6i.8xlarge, m6i.12xlarge, m6i.16xlarge, m6i.24xlarge, m6i.32xlarge |
| M6id | m6id.xlarge, m6id.2xlarge, m6id.4xlarge, m6id.8xlarge, m6id.12xlarge, m6id.16xlarge, m6id.24xlarge, m6id.32xlarge |
| M6idn | m6idn.xlarge, m6idn.2xlarge, m6idn.4xlarge, m6idn.8xlarge, m6idn.12xlarge, m6idn.16xlarge, m6idn.24xlarge, m6idn.32xlarge |
| M6in | m6in.xlarge , m6in.2xlarge , m6in.4xlarge , m6in.8xlarge, m6in.12xlarge , m6in.16xlarge , m6in.24xlarge |
| M7a | m7a.xlarge , m7a.2xlarge, m7a.4xlarge, m7a.8xlarge, m7a.12xlarge, m7a.16xlarge, m7a.24xlarge, m7a.32xlarge, m7a.48xlarge |
| M7i | m7i.xlarge, m7i.2xlarge, m7i.4xlarge, m7i.8xlarge, m7i.12xlarge, m7i.16xlarge, m7i.24xlarge, m7i.48xlarge |
| M7g | m7g.xlarge, m7g.2xlarge, m7g.4xlarge, m7g.8xlarge, m7g.12xlarge, m7g.16xlarge |
| M7gd | m7gd.xlarge, m7gd.2xlarge, m7gd.4xlarge, m7gd.8xlarge, m7gd.12xlarge, m7gd.16xlarge |
| EC2 instance type | Supported instances |
|---|---|
| C5 | c5.2xlarge, c5.4xlarge, c5.9xlarge, c5.12xlarge, c5.18xlarge, c5.24xlarge |
| C5a | c5a.2xlarge , c5a.4xlarge, c5a.8xlarge, c5a.12xlarge, c5a.16xlarge, c5a.24xlarge |
| C5ad | c5ad.2xlarge , c5ad.4xlarge, c5ad.8xlarge, c5ad.12xlarge, c5ad.16xlarge, c5ad.24xlarge |
| C5n | c5n.2xlarge, c5n.4xlarge, c5n.9xlarge, c5n.18xlarge |
| C6a | c6a.2xlarge, c6a.4xlarge, c6a.8xlarge, c6a.12xlarge, c6a.16xlarge, c6a.32xlarge, c6a.48xlarge |
| C6g | c6g.2xlarge, c6g.4xlarge, c6g.8xlarge, c6g.12xlarge, c6g.16xlarge |
| C6gd | c6gd.2xlarge, c6gd.4xlarge, c6gd.8xlarge, c6gd.12xlarge, c6gd.16xlarge |
| C6gn | c6gn.2xlarge, c6gn.4xlarge, c6gn.8xlarge, c6gn.12xlarge, c6gn.16xlarge |
| C6in | c6in.2xlarge, c6in.4xlarge, c6in.8xlarge, c6in.12xlarge, c6in.16xlarge, c6in.24xlarge, c6in.32xlarge |
| C7a | c7a.2xlarge, c7a.4xlarge, c7a.8xlarge, c7a.12xlarge, c7a.16xlarge, c7a.24xlarge, c7a.32xlarge, c7a.48xlarge |
| C7g | c7g.2xlarge, c7g.4xlarge, c7g.8xlarge, c7g.12xlarge, c7g.16xlarge |
| C7gd | c7gd.2xlarge, c7gd.4xlarge, c7gd.8xlarge, c7gd.12xlarge, c7gd.16xlarge |
| C7i | c7i.2xlarge, c7i.4xlarge, c7i.8xlarge, c7i.12xlarge, c7i.16xlarge, c7i.24xlarge, cC7i.48xlarge |
| EC2 instance type | Supported instances |
|---|---|
| R5 | r5.xlarge, r5.2xlarge, r5.4xlarge, r5.8xlarge, r5.12xlarge, r5.16xlarge, r5.24xlarge |
| R5n | r5n.xlarge, r5n.2xlarge, r5n.4xlarge, r5n.8xlarge, r5n.12xlarge, r5n.16xlarge, r5n.24xlarge |
| R6a | r6a.xlarge, r6a.2xlarge, r6a.4xlarge, r6a.8xlarge, r6a.12xlarge, r6a.16xlarge, r6a.32xlarge, r6a.48xlarge |
| R6i | r6i.xlarge, r6i.2xlarge, r6i.4xlarge, r6i.8xlarge, r6i.12xlarge, r6i.16xlarge, r6i.24xlarge, r6i.32xlarge |
| R6id | r6id.xlarge, r6id.2xlarge, r6id.4xlarge, r6id.8xlarge, r6id.12xlarge, r6id.16xlarge, r6id.24xlarge, r6id.32xlarge |
| R6idn | r6idn.xlarge, r6idn.2xlarge, r6idn.4xlarge, r6idn.8xlarge, r6idn.12xlarge, r6idn.16xlarge, r6idn.24xlarge, r6idn.32xlarge |
| R6in | r6in.xlarge, r6in.2xlarge, r6in.4xlarge, r6in.8xlarge, r6in.12xlarge, r6in.16xlarge, r6in.24xlarge, r6in.32xlarge |
| R6g | r6g.xlarge, r6g.2xlarge, r6g.4xlarge, r6g.8xlarge, r6g.12xlarge, r6g.16xlarge |
| R6gd | r6gd.xlarge, r6gd.2xlarge, r6gd.4xlarge, r6gd.8xlarge, r6gd.12xlarge, r6gd.16xlarge |
| R7a | r7a.xlarge, r7a.2xlarge, r7a.4xlarge, r7a.8xlarge, r7a.12xlarge, r7a.16xlarge, r7a.24xlarge, r7a.32xlarge, r7a.48xlarge |
| R7iz | r7iz.xlarge, r7iz.2xlarge, r7iz.4xlarge, r7iz.8xlarge, r7iz.12xlarge, r7iz.16xlarge, r7iz.32xlarge |
| R7g | r7g.xlarge, r7g.2xlarge, r7g.4xlarge, r7g.8xlarge, r7g.12xlarge, r7g.16xlarge |
| R7gd | r7gd.xlarge, r7gd.2xlarge, r7gd.4xlarge, r7gd.8xlarge, r7gd.12xlarge, r7gd.16xlarge |
| X1 | x1.16xlarge, x1.32xlarge |
| X1e | x1e.16xlarge, x1e.32xlarge |
| X2idn | x2idn.16xlarge, x2idn.24xlarge, x2idn.32xlarge |
| X2iedn | x2iedn.xlarge, x2iedn.2xlarge, x2iedn.4xlarge, x2iedn.8xlarge, x2iedn.16xlarge, x2iedn.24xlarge |
| Z1d | z1d.xlarge, z1d.2xlarge, z1d.3xlarge, z1d.6xlarge, z1d.12xlarge |
| EC2 instance type | Supported instances |
|---|---|
| F1 | f1.2xlarge, f1.4xlarge, f1.16xlarge |
| G3 | g3.4xlarge, g3.8xlarge, g3.16xlarge |
| G4dn | g4dn.2xlarge, g4dn.4xlarge, g4dn.8xlarge, g4dn.12xlarge, g4dn.16xlarge |
| G5 | g5.xlarge, g5.2xlarge, g5.4xlarge, g5.8xlarge, g5.12xlarge, g5.16xlarge |
| G5g | g5g.2xlarge, g5g.4xlarge, g5g.8xlarge, g5g.16xlarge |
| G6 | g6.xlarge, g6.2xlarge, g6.4xlarge, g6.8xlarge, g6.12xlarge, g6.16xlarge, g6.24xlarge, g6.48xlarge |
| GR6 | gr6.4xlarge, gr6.8xlarge |
| Inf1 | inf1.2xlarge, inf1.6xlarge, inf1.24xlarge |
| Inf2 | inf2.xlarge, inf2.8xlarge, inf2.24xlarge, inf2.48xlarge |
| P2 | p2.xlarge, p2.8xlarge, p2.16xlarge |
| P3 | p3.2xlarge, p3.8xlarge, p3.16xlarge |
| P4 | p4d.24xlarge, p4de.24xlarge |
| P5 | p5.48xlarge |
| Trn1 | trn1.2xlarge, trn1.32xlarge , trn1n.32xlarge |
| EC2 instance type | Supported instances |
|---|---|
| I3en | i3en.xlarge, i3en.2xlarge, i3en.3xlarge, i3en.6xlarge, i3en.12xlarge, i3en.24xlarge |
| EC2 instance type | Supported instances |
|---|---|
| HPc6 | hpc6.48xlarge |
| HPc6a | hpc6a.48xlarge |
| HPc7a | hpc7a.12xlarge, hpc7a.48xlarge, hpc7a.96xlarge |
VPC
Subnet in VPC
Security Group in EC2
WEKA cluster on Azure deployment
smb_protocol_gateways_number = 3
smb_protocol_gateway_instance_type = "Standard_L48s_v3"
smbw_enabled = true
smb_domain_name = "CUSTOMER_DOMAIN"
smb_share_name = "SPECIFY_SMB_SHARE_NAMING"
smb_setup_protocol = true
| VM size | vCPU | Memory (GiB) | NVMe disks | Max NICs | BW (Mbps) |
|---|---|---|---|---|---|
| Standard_L8s_v3 | 8 | 64 | 1x1.92 TB | 4 | 12500 |
| Standard_L16s_v3 | 16 | 128 | 2x1.92 TB | 8 | 12500 |
| Standard_L32s_v3 | 32 | 256 | 4x1.92 TB | 8 | 16000 |
| Standard_L48s_v3 | 48 | 384 | 6x1.92 TB | 8 | 24000 |
| Standard_L64s_v3 | 64 | 512 | 8x1.92 TB | 8 | 30000 |
| Standard_L80s_v3 | 80 | 640 | 10x1.92 TB | 8 | 32000 |
| Standard_L8as_v3 | 8 | 64 | 1x1.92 TB | 4 | 12500 |
| Standard_L16as_v3 | 16 | 128 | 2x1.92 TB | 8 | 12500 |
| Standard_L32as_v3 | 32 | 256 | 4x1.92 TB | 8 | 16000 |
| Standard_L48as_v3 | 48 | 384 | 6x1.92 TB | 8 | 24000 |
| Standard_L64as_v3 | 64 | 512 | 8x1.92 TB | 8 | 32000 |
| Standard_L80as_v3 | 80 | 640 | 10x1.92 TB | 8 | 32000 |
Mapped WEKA processes for a standard_L16s_v3
| VM size | # of compute cores | # of drive cores | # of frontend cores |
|---|---|---|---|
| Standard_L8s_v3 | 1 | 1 | 1 |
| Standard_L16s_v3 | 4 | 2 | 1 |
| Standard_L32s_v3 | 4 | 2 | 1 |
| Standard_L48s_v3 | 3 | 3 | 1 |
| Standard_L64s_v3 | 4 | 2 | 1 |
| Standard_L80s_v3 | 4 | 2 | 1 |
| Standard_L8as_v3 | 1 | 1 | 1 |
| Standard_L16as_v3 | 4 | 2 | 1 |
| Standard_L32as_v3 | 4 | 2 | 1 |
| Standard_L48as_v3 | 3 | 3 | 1 |
| Standard_L64as_v3 | 4 | 2 | 1 |
| Standard_L80as_v3 | 4 | 2 | 1 |
| VM series | VM size |
|---|---|
| Dsv3 | Standard_D4s_v3, Standard_D8s_v3, Standard_D16s_v3 |
| Dasv4 | Standard_D2as_v4, Standard_D4as_v4, Standard_D8as_v4 |
| Ddsv4 | Standard_D16ds_v4 |
| Dasv4 | Standard_D4as_v4, Standard_D16as_v4, Standard_D32as_v4, Standard_D96as_v4 |
| Dv5 | Standard_D8_v5 |
| Dsv5 | Standard_D4s_v5 ,Standard_D16s_v5, Standard_D48s_v5 , Standard_D64s_v5 |
| Dadsv5 | Standard_D4ads_v5, Standard_D16ads_v5, Standard_D48ads_v5, Standard_D96ads_v5 |
| Dcsv2 | Standard_DC4s_v2 (UDP only) |
| Dasv5 | Standard_D2as_v5, Standard_D8as_v5 |
| Dpldsv5 | Standard_D8plds_v5, Standard_D32plds_v5, Standard_D64plds_v5 |
| Dpsv5 | Standard_D4ps_v5, Standard_D8ps_v5, Standard_D16ps_v5, Standard_D32ps_v5, Standard_D48ps_v5, Standard_D64ps_v5 |
| VM series | VM size |
|---|---|
| Edsv4 | Standard_E16ds_v4, Standard_E16-8ds_v4, Standard_E32ds_v4, |
| Easv4 | Standard_E32-16as_v4 |
| Easv5 | Standard_E32-16as_v5 |
| Edsv4 | Standard_E32-16ds_v4, Standard_E48ds_v4 |
| Eadsv5 | Standard_E96ads_v5 |
| Mdmsv2 | Standard_M64dms_v2 |
| Msv2 | Standard_M208s_v2 |
| Mmsv2 | Standard_M208ms_v2, Standard_M416ms_v2 |
| Epsv5 | Standard_E4ps_v5, Standard_E8ps_v5, Standard_E16ps_v5, Standard_E20ps_v5, Standard_E32ps_v5 |
| VM series | VM size |
|---|---|
| FXmds | Standard_FX48mds |
| Fsv2 | Standard_F8s_v2, Standard_F32s_v2, Standard_F64s_v2, Standard_F72s_v2 |
| VM series | VM size |
|---|---|
| Lsv3 | Standard_L8s_v3, Standard_L16s_v3, Standard_L32s_v3, Standard_L48s_v3, Standard_L64s_v3, Standard_L80s_v3 |
| VM series | VM size |
|---|---|
| HBv4 | Standard_HB176rs_v4, Standard_HB176-24rs_v4, Standard_HB176-96rs_v4 |
| HBv3 | Standard_HB120rs_v3 |
| VM series | VM size |
|---|---|
| NGads V620 | Standard_NG8ads_V620_v1, Standard_NG16ads_V620_v1, Standard_NG32ads_V620_v1 |
| NVadsA10 | Standard_NVadsA10_v5 |
ip a command output exampleroot@jack:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 3900 qdisc mq state UP group default qlen 1000
link/ether 00:0d:3a:8e:3a:67 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.30/24 brd 10.0.0.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::20d:3aff:fe8e:3a67/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 4038 qdisc mq state UP group default qlen 1000
link/ether 00:0d:3a:8b:d9:bd brd ff:ff:ff:ff:ff:ff
inet 10.0.0.31/24 brd 10.0.0.255 scope global eth1
valid_lft forever preferred_lft forever
inet6 fe80::20d:3aff:fe8b:d9bd/64 scope link
valid_lft forever preferred_lft forever
4: enP18334s1np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 3900 qdisc mq master eth0 state UP group default qlen 1000
link/ether 00:0d:3a:8e:3a:67 brd ff:ff:ff:ff:ff:ff
5: enP39539s2np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 4038 qdisc mq master eth1 state UP group default qlen 1000
link/ether 00:0d:3a:8b:d9:bd brd ff:ff:ff:ff:ff:ff
6: dtap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 4038 qdisc multiq state UNKNOWN group default qlen 1000
link/ether 00:0d:3a:8b:d9:bd brd ff:ff:ff:ff:ff:ff
inet6 fe80::20d:3aff:fe8b:d9bd/64 scope link
valid_lft forever preferred_lft forever
Server infrastructure in GCP
WEKA project on the GCP
smb_protocol_gateways_number = 3
smb_protocol_gateway_instance_type = "c2-standard-8"
smbw_enabled = true
smb_domain_name = "CUSTOMER_DOMAIN"
smb_share_name = "SPECIFY_SMB_SHARE_NAMING"
smb_setup_protocol = true
| Machine series | Machine types |
|---|---|
| C2 | c2-standard-8, c2-standard-16 |
| N2 | n2-standard-8, n2-standard-16 |
| Z3 | z3-highmem-8-highlssd, z3-highmem-16-highlssd, z3-highmem-22-highlssd, z3-highmem-32-highlssd, z3-highmem-44-highlssd, z3-highmem-88-highlssd |
| Machine series | Machine type |
|---|---|
| A2 | a2-highgpu-1g, a2-highgpu-2g, a2-highgpu-4g, a2-highgpu-8g, a2-megagpu-16g, a2-ultragpu-1g |
| C2 | c2-standard-8, c2-standard-16 |
| C2D | c2d-standard-4, c2d-standard-8, c2d-standard-16, c2d-standard-32, c2d-standard-56, c2d-standard-112, c2d-highmem-56 |
| E2 | e2-standard-4, e2-standard-8, e2-standard-16, e2-highmem-4, e2-highcpu-8 |
| N2 | n2-standard-4, n2-standard-8, n2-standard-16, n2-standard-32, n2-standard-48, n2-standard-96, n2-standard-128, n2-highmem-32 |
| N2D | n2d-standard-32, n2d-standard-64, n2d-highmem-32, n2d-highmem-64 |
| Machine series | Machine type |
|---|---|
| A2 | a2-highgpu-1g, a2-highgpu-2g, a2-highgpu-4g, a2-highgpu-8g, a2-megagpu-16g, a2-ultragpu-1g |
| A3 | a3-highgpu-8g |
| C2 | c2-standard-8, c2-standard-16, c2-standard-30, c2-standard-60 |
| C2D | c2d-standard-4, c2d-standard-8, c2d-standard-16, c2d-standard-32, c2d-standard-56, c2d-standard-112, c2d-highmem-56 |
| C3 | c3-standard-4, c3-standard-8, c3-standard-22, c3-standard-44, c3-standard-88, c3-standard-176, c3-highcpu-4, c3-highcpu-8, c3-highcpu-22, c3-highcpu-44, c3-highcpu-88, c3-highcpu-176, c3-highmem-4, c3-highmem-8, c3-highmem-22, c3-highmem-44, c3-highmem-88, c3-highmem-176, c3-standard-4-lssd, c3-standard-8-lssd, c3-standard-22-lssd, c3-standard-44-lssd, c3-standard-88-lssd, c3-standard-176-lssd |
| C3D | c3d-standard-4, c3d-standard-8, c3d-standard-16, c3d-standard-30, c3d-standard-60, c3d-standard-90, c3d-standard-180, c3d-standard-360, c3d-highcpu-4, c3d-highcpu-8, c3d-highcpu-16, c3d-highcpu-30, c3d-highcpu-60, c3d-highcpu-90, c3d-highcpu-180, c3d-highcpu-360, c3d-highmem-4, c3d-highmem-8, c3d-highmem-16, c3d-highmem-30, c3d-highmem-60, c3d-highmem-90, c3d-highmem-180, c3d-highmem-360, c3d-standard-8-lssd, c3d-standard-16-lssd, c3d-standard-30-lssd, c3d-standard-60-lssd, c3d-standard-90-lssd, c3d-standard-180-lssd, c3d-standard-360-lssd, c3d-highmem-8-lssd, c3d-highmem-16-lssd, c3d-highmem-30-lssd, c3d-highmem-60-lssd, c3d-highmem-90-lssd, c3d-highmem-180-lssd, c3d-highmem-360-lssd |
| C4 | c4-standard-4, c4-standard-8, c4-standard-16, c4-standard-32, c4-standard-48, c4-standard-96, c4-standard-192, c4-highcpu-4, c4-highcpu-8, c4-highcpu-16, c4-highcpu-32, c4-highcpu-48, c4-highcpu-96, c4-highcpu-192, c4-highmem-4, c4-highmem-8, c4-highmem-16, c4-highmem-32, c4-highmem-48, c4-highmem-96, c4-highmem-192 |
| G2 | g2-standard-4, g2-standard-8, g2-standard-12, g2-standard-16, g2-standard-24, g2-standard-32, g2-standard-48, g2-standard-96 |
| M3 | m3-ultramem-32, m3-ultramem-64, m3-ultramem-128, m3-megamem-64, m3-megamem-128 |
| N2 | n2-standard-4, n2-standard-8, n2-standard-16, n2-standard-32, n2-standard-48, n2-standard-64, n2-standard-80, n2-standard-96, n2-standard-128, n2-highmem-4, n2-highmem-8, n2-highmem-16, n2-highmem-32, n2-highmem-48, n2-highmem-64, n2-highmem-80, n2-highmem-96, n2-highmem-128, n2-highcpu-8, n2-highcpu-16, n2-highcpu-32, n2-highcpu-48, n2-highcpu-64, n2-highcpu-80, n2-highcpu-96 |
| N2D | n2d-standard-4, n2d-standard-8, n2d-standard-16, n2d-standard-32, n2d-standard-48, n2d-standard-64, n2d-standard-80, n2d-standard-96, n2d-standard-224, n2d-highmem-4, n2d-highmem-8, n2d-highmem-16, n2d-highmem-32, n2d-highmem-48, n2d-highmem-64, n2d-highmem-80, n2d-highmem-96, n2d-highcpu-8, n2d-highcpu-16, n2d-highcpu-32, n2d-highcpu-48, n2d-highcpu-64, n2d-highcpu-80, n2d-highcpu-96, n2d-highcpu-128, n2d-highcpu-224 |
| N4 | n4-standard-4, n4-standard-8, n4-standard-16, n4-standard-32, n4-standard-48, n4-standard-64, n4-standard-80, n4-highcpu-4, n4-highcpu-8, n4-highcpu-16, n4-highcpu-32, n4-highcpu-48, n4-highcpu-64, n4-highcpu-80, n4-highmem-4, n4-highmem-8, n4-highmem-16, n4-highmem-32, n4-highmem-48, n4-highmem-64, n4-highmem-80 |
| Z3 | z3-highmem-8-highlssd, z3-highmem-16-highlssd, z3-highmem-22-highlssd, z3-highmem-32-highlssd, z3-highmem-44-highlssd, z3-highmem-88-highlssd |
GCP networking
Example: Scale-out from 7 backends to 8
WEKA cluster on OCI deployment
NeuralMesh Axon deployments with compute and storage on the same infrastructure
| Port/NodePort | Protocol | Purpose |
|---|---|---|
| 14000-14100, 15000-15100, 16000-16100 | TCP + UDP | NeuralMesh Axon Core container traffic |
| 443 | TCP | NeuralMesh Axon Core traces remote viewer |
| 22 | TCP | SSH management access |
| 123 | TCP | NTP management access |
| 8200, 8201 | TCP | Hashicorp Vault traffic |
| 5696 | TCP | KMIP traffic |
One Drive core per NVMe drive (up to 8 cores).
Two Compute cores per Drive core (up to 16 cores).
Twelve Frontend cores (regardless of number of drives).
NeuralMesh Axon integration with Slurm
The chart installs a ClusterRole and ClusterRoleBinding that grant the operator broad permissions across the cluster, because the operator manages resources in multiple namespaces. Run helm install with cluster-admin privileges.
| Setting | Value | Purpose |
|---|---|---|
| CPU Manager Policy | static | Allows containers to request dedicated CPU cores instead of sharing them. WEKA's storage processes require exclusive access to specific cores for consistent performance. |
| Reserved CPUs | 1000m plus CPUs 0 and 1 | Reserves one CPU core and two dedicated system CPUs for Kubernetes system processes, so WEKA does not compete with the operating system for CPU time. |
| Reserved Memory | 1 GiB | Reserves RAM for Kubernetes system processes, separate from the memory WEKA itself uses. |
| Hugepages | 25,000 x 2 MB (50 GB per node) | Reserves large memory regions that bypass normal operating system memory management, letting WEKA move data at high speed with minimal CPU overhead. Confirm each node has at least 50 GB of free RAM above what other workloads need. |
| Field | Requirement | Description |
|---|---|---|
| Quay Username | Required | The Quay.io account granted access to WEKA's private container images. |
| Quay Password | Required | The Quay.io password or robot token. A robot token is recommended for production environments: it can be rotated without affecting the main login, and it distinguishes a service credential from a personal one in audit logs. |
| Operator Version | Default: v1.13.0 | The version of the WEKA Kubernetes operator to install. Leave the default if unsure. The version must exist on Quay.io, entered as v1.13.0 or 1.13.0. |
| Field | Requirement | Description |
|---|---|---|
| WEKA Endpoints | Required | A comma-separated list of host:port entries for the NeuralMesh cluster servers. At least one entry is required; three or more allow the client to stay connected if one server is temporarily unavailable. Example: 10.0.1.10:14000,10.0.1.11:14000,10.0.1.12:14000 |
| WEKA Image Version | Required | The exact WEKA software version running on the cluster. The containerized client version must match the version on the NeuralMesh servers: even a minor version mismatch prevents the client from connecting. Example: 5.1.0.605 |
| Endpoint Scheme | Default: http | The protocol used to reach the NeuralMesh cluster management API. Leave as http unless the NeuralMesh cluster has TLS enabled on its management API endpoint, in which case select https. |
| Mode | Description |
|---|---|
| Auto (recommended) | WEKA automatically selects the best available networking mode for the environment. The safest choice when the available NICs or supported modes are unclear. |
| DPDK (high performance) | Uses kernel-bypass networking for the highest throughput and lowest latency. Requires DPDK-capable network cards, typically Mellanox/NVIDIA ConnectX series or Intel adapters. Selecting DPDK reveals a field for the NIC device names: enter the storage network interface names, comma-separated, for example ens3f0,ens3f1. Find NIC names by running ip link show on a worker node and identifying the interfaces connected to the storage network. |
| UDP (compatibility mode) | Uses standard UDP-based software networking. Lower throughput than DPDK, but compatible with any network card and environment. Use this when DPDK is unavailable or maximum throughput is not required. |
| Field | Requirement | Description |
|---|---|---|
| WEKA Organization | Default: Root | The WEKA organization the CSI user account belongs to. Leave as Root if the cluster does not use multi-tenancy, otherwise enter the organization name where the CSI service account was created. |
| WEKA Username | Required | The username of the WEKA CSI service account. The account must already exist in WEKA with the CSI role assigned. |
| WEKA Password | Required | The password for the WEKA CSI service account. |
| Phase | Description |
|---|---|
| Phase 1: WEKA integration | Installs the WEKA Kubernetes operator, deploys the CSI storage driver, sets up image pull secrets, launches the WEKA client DaemonSet on every worker node, and creates the storage classes Kubernetes uses to provision NeuralMesh-backed volumes. |
| Phase 2: Cluster initialization | Deploys the monitoring stack (Prometheus and Grafana), the NVIDIA NIM Operator for AI workloads, and the Envoy Gateway networking layer that routes external traffic to the App Store and other deployed applications. Phase 2 starts automatically once Phase 1 completes. |
| Storage class | Description |
|---|---|
wekafs-dir-api | The default storage class. Volumes are created as directories within a NeuralMesh filesystem. Deleting a Kubernetes PVC also deletes the data. |
wekafs-dir-api-retain | The same as above, with a Retain reclaim policy. Data in NeuralMesh persists after the Kubernetes PVC is deleted. Use this for data that should outlive the workload's lifecycle. |
wekafs-fs-api | Filesystem-backed volumes. Each volume gets a dedicated NeuralMesh filesystem instead of a directory within a shared filesystem, suited to workloads with filesystem-level isolation requirements. |
The warp-edge-gateway LoadBalancer service with an external IP assigned by the cluster's load-balancer solution
| Credential | Steps |
|---|---|
| NVIDIA NGC API Keys | Under NVIDIA NGC API Keys, select + Add. Enter a display name and the NGC API key. The App Store derives the image-pull secret and the NIM model-download key from this entry. |
| WEKA Storage API Tokens | Under WEKA Storage API Tokens, select + Add. Enter the WEKA management endpoint (host or IP address), a username, and an API token or password. This becomes the credential the AI Data Platform operator and Space Manager use to reach the cluster's REST API. |
| WEKA Quay Token | Under Quay Token, select + Add. To pull images for AIDP from the WEKA Quay repository, you will need to enter your Quay Robot username and password. Failure to do this will produce imagepullerror on deployment. |
Settings → Credential Management, with both credentials registered and showing a Ready status
The App Store catalog: AIDP listed under Browse by category
The AIDP detail page: Cluster Compatibility panel on the right, Configure form below it
| Field | Requirement | Description |
|---|---|---|
| Namespace | Required | The Kubernetes namespace to deploy into, created automatically if it does not already exist. Select an existing namespace from the dropdown or enter a new name, for example rag. |
| NGC Credential | Credential | Dropdown of registered NVIDIA NGC credentials from step 1. Pulls NIM images from nvcr.io and supplies the NGC API key to the model-serving NIMs. |
| WEKA Credential | Credential | Dropdown of registered WEKA Storage credentials from step 1. Supplies the WEKA REST API token the AI Data Platform operator and Space Manager use to manage filesystems. |
| WEKA S3 Access Key | Required | Access key for the WEKA S3-compatible bucket, used by Milvus and the NVIDIA ingestion pipeline. |
| WEKA S3 Secret Key | Required | Secret key paired with the access key above. |
| WEKA S3 Bucket Name | Default:aidp-bucket | Name of the S3 bucket Milvus and the ingestion pipeline use. This bucket must already exist on the cluster. |
| WEKA S3 Node IPs | Required | Comma or space-separated list of the cluster's S3 server IP addresses, across which the built-in load balancer distributes traffic. Get the list by running weka cluster nodes --role S3 on a WEKA server. Example: 192.168.1.1, 192.168.1.2, 192.168.1.3 |
| WEKA API Host | Required | The bare WEKA management IP address or hostname, without http:// or a port (port 14000 is assumed). Example: 172.3.4.248 |
| Keycloak FQDN | Required | External hostname for Keycloak sign-in, hostname only, without a scheme or path. Example: keycloak.example.com |
| GUI Base URL | Required | Full external URL of the AI Data Platform GUI. The Keycloak OIDC redirect URI is derived from this value. Example: https://aidp.example.com |
| Attu Hostname | Required | External hostname for the Milvus Attu admin UI, hostname only. Example: attu.example.com |
| Cluster FQDN | Required | The general cluster domain, used to derive the observability hostname (phoenix.<domain>). Example: example.com |
| Vector DB Filesystem Name | Required | The human-readable NeuralMesh name backing the vector database, exactly as shown in the WEKA console. |
| Attribute | Description |
|---|---|
uid | The user's POSIX UID as it exists on NeuralMesh, a single numeric value, for example 1001. |
gids | Every POSIX GID the user belongs to, multi-valued, each a numeric value, for example 1001, 2010. |
Core Usage widget
Display servers in 3D view
Example: Display the filesystems table
| Option | Description |
|---|---|
-J|--json | Prints the raw JSON value returned by the cluster. |
-H|--hostname | Directs the CLI to communicate with the cluster through the specified hostname or IP. |
--raw-units | Sets the units such as capacity and bytes to be printed in their raw format, as returned by the cluster. |
--UTC | Sets the timestamps to be printed in UTC timezone, instead of the local time of the server running the CLI command. |
-f|--format | Specifies the format to output the result (view, csv, markdown, or JSON). |
-o|--output | Specifies the columns to include in the output. |
-s|--sort | Specifies the order to sort the output. May include a '+' or '-' before the column name to sort by ascending or descending order. |
-F| --filter | Specifies the filter values for a member (without forcing it to be in the output). |
--no-header | Indicates that the column header should not be shown when printing the output. |
-C|--CONNECT-TIMEOUT | Modifies the default timeout used for connecting to the system via the JRPC protocol. |
-T|--TIMEOUT | Modifies the default timeout for which the commands wait for a response before giving up. |
--color | Controls the usage of color in the outputs. Possible values: Default: |