W E K A
4.1
4.44.34.24.14.03.14
4.1
4.44.34.24.14.03.14
  • WEKA v4.1 documentation
  • WEKA System Overview
    • About the WEKA system
    • SSD capacity management
    • Filesystems, object stores, and filesystem groups
    • WEKA networking
    • Data lifecycle management
    • WEKA client and mount modes
    • WEKA containers architecture overview
    • Glossary
  • Getting Started with WEKA
    • Quick installation guide
    • Manage the system using the WEKA CLI
    • Manage the system using the WEKA GUI
    • Run first IOs with WEKA filesystem
    • Getting started with WEKA REST API
  • Planning and Installation
    • Prerequisites for installation
    • WEKA installation on bare metal
      • Plan the WEKA system Installation
      • Prepare the system for WEKA software installation
        • Enable the SR-IOV
      • Obtain the WEKA software installation package
      • WEKA cluster installation
        • WEKA legacy system installation process
      • Add clients
    • WEKA installation on AWS
      • Self-service portal
      • CloudFormation template generator
      • Deployment types
      • AWS outposts deployment
      • Supported EC2 instance types
      • Add clients
      • Auto scaling group
      • Troubleshooting
    • WEKA installation on Azure
    • WEKA installation on GCP
      • WEKA project description
      • Deployment on GCP using Terraform
      • GCP Terraform package description
      • Required services and supported regions
      • Supported machine types and storage
      • Auto-scale instances in GCP
      • Add clients
      • Troubleshooting
  • Performance
    • WEKA performance tests
      • Test environment details
  • WEKA Filesystems & Object Stores
    • Manage object stores
      • Manage object stores using the GUI
      • Manage object stores using the CLI
    • Manage filesystem groups
      • Manage filesystem groups using the GUI
      • Manage filesystem groups using the CLI
    • Manage filesystems
      • Manage filesystems using the GUI
      • Manage filesystems using the CLI
    • Attach or detach object store buckets
      • Attach or detach object store bucket using the GUI
      • Attach or detach object store buckets using the CLI
    • Advanced data lifecycle management
      • Advanced time-based policies for data storage location
      • Data management in tiered filesystems
      • Transition between tiered and SSD-only filesystems
      • Manual fetch and release of data
    • Mount filesystems
    • Snapshots
      • Manage snapshots using the GUI
      • Manage snapshots using the CLI
    • Snap-To-Object
      • Manage Snap-To-Object using the GUI
      • Manage Snap-To-Object using the CLI
    • Quota management
      • Manage quotas using the GUI
      • Manage quotas using the CLI
  • Additional Protocols
    • Manage the NFS protocol
      • Supported NFS client mount options
      • Manage NFS networking using the GUI
      • Manage NFS networking using the CLI
    • Manage the SMB protocol
      • Manage SMB using the GUI
      • Manage SMB using the CLI
    • Manage the S3 protocol
      • S3 cluster management
        • Manage the S3 service using the GUI
        • Manage the S3 service using the CLI
      • S3 buckets management
        • Manage S3 buckets using the GUI
        • Manage S3 buckets using the CLI
      • S3 users and authentication
        • Manage S3 users and authentication using the CLI
        • Manage S3 service accounts using the CLI
      • S3 rules information lifecycle management (ILM)
        • Manage S3 lifecycle rules using the GUI
        • Manage S3 lifecycle rules using the CLI
      • Audit S3 APIs
        • Configure audit webhook using the GUI
        • Configure audit webhook using the CLI
        • Example: How to use Splunk to audit S3
      • S3 supported APIs and limitations
      • S3 examples using boto3
  • Operation Guide
    • Alerts
      • Manage alerts using the GUI
      • Manage alerts using the CLI
      • List of alerts and corrective actions
    • Events
      • Manage events using the GUI
      • Manage events using the CLI
      • List of events
    • Statistics
      • Manage statistics using the GUI
      • Manage statistics using the CLI
      • List of statistics
    • System congestion
    • Security management
      • Obtain authentication tokens
      • KMS management
        • Manage KMS using the GUI
        • Manage KMS using the CLI
      • TLS certificate management
        • Manage the TLS certificate using the GUI
        • Manage the TLS certificate using the CLI
      • CA certificate management
        • Manage the CA certificate using the GUI
        • Manage the CA certificate using the CLI
      • Account lockout threshold policy management
        • Manage the account lockout threshold policy using GUI
        • Manage the account lockout threshold policy using CLI
      • Manage the login banner
        • Manage the login banner using the GUI
        • Manage the login banner using the CLI
    • User management
      • Manage users using the GUI
      • Manage users using the CLI
    • Organizations management
      • Manage organizations using the GUI
      • Manage organizations using the CLI
      • Mount authentication for organization filesystems
    • Expand and shrink cluster resources
      • Add a backend server in a multiple containers architecture
      • Add a backend server in a legacy architecture
      • Expand specific resources of a container
      • Shrink a cluster
    • Background tasks
    • Upgrade WEKA versions
  • Billing & Licensing
    • License overview
    • Classic license
    • Pay-As-You-Go license
  • Support
    • Prerequisites and compatibility
    • Get support for your WEKA system
    • Diagnostics management
      • Traces management
        • Manage traces using the GUI
        • Manage traces using the CLI
      • Protocols debug level management
        • Manage protocols debug level using the GUI
        • Manage protocols debug level using the CLI
      • Diagnostics data management
    • Weka Home - The WEKA support cloud
      • Local Weka Home overview
      • Local Weka Home deployment
      • Set the Local Weka Home to send alerts or events
      • Download the Usage Report or Analytics
  • Appendix
    • WEKA CSI Plugin
    • Set up the WEKAmon external monitoring
    • Set up the SnapTool external snapshots manager
  • REST API Reference Guide
Powered by GitBook
On this page
  • CPU
  • Memory
  • Operating system
  • Types
  • Configuration
  • Kernel
  • Weka installation directory
  • Networking
  • Ethernet
  • InfiniBand
  • HA
  • SSDs
  • Object store
  • Virtual Machines
  • KMS
  1. Support

Prerequisites and compatibility

This page describes the prerequisites and compatibility for the installation of the WEKA system.

The versions specified in the prerequisites and compatibility page apply to the latest minor version of the WEKA system. See the relevant release notes in get.weka.io for more details.

CPU

  • Intel SandyBridge+ processors

  • AMD 2nd and 3rd Gen EPYC processors

Ensure the BIOS settings meet the following requirements:

  • AES must be enabled.

  • Secure Boot must be disabled.

Memory

  • Sufficient memory to support the WEKA system needs as described in memory requirements.

  • More memory support for the OS kernel or any other application.

Operating system

Types

Backends and Clients:

  • RHEL:

    • 7.9, 7.8, 7.7, 7.6, 7.5, 7.4, 7.3, 7.2

    • 8.7, 8.6, 8.5, 8.4, 8.3, 8.2, 8.1, 8.0

  • CentOS:

    • 7.9, 7.8, 7.7, 7.6, 7.5, 7.4, 7.3, 7.2

    • 8.5, 8.4, 8.3, 8.2, 8.1, 8.0

  • Rocky Linux

    • 8.7, 8.6

  • Ubuntu:

    • 18.04.6, 18.04.5, 18.04.4, 18.04.3, 18.04.2, 18.04.1, 18.04.0

    • 20.04.3, 20.04.2, 20.04.1, 20.04.0

  • Amazon Linux: 18.03, 17.09

  • Amazon Linux 2 LTS (formerly Amazon Linux 2 LTS 17.12)

Ensure using an OS with a supported kernel. Do not use an OS with kernel 5.13 for backends in MCB architecture.

Clients only:

  • SuSe:

    • 15 SP2

    • 12 SP5

For kernel 5.13 in clients, do not use applications that use 1GB hugepages.

Configuration

General

  • All WEKA servers must be synchronized in date/time (NTP recommended)

  • A watchdog driver should be installed in /dev/watchdog (hardware watchdog recommended); search the WEKA knowledge base in the WEKA support portal for more information and how-to articles.

  • If using mlocate or alike, it's advisable to exclude wekafs from updatedb filesystems lists; search the WEKA knowledge base in the WEKA support portal for more information and how-to articles.

SELinux

  • SELinux is supported in both permissive and enforcing modes.

    • The targeted policy is supported.

    • The mls policy is not supported yet.

Note: To set the SELinux security context for files, use the -o acl in the mount command, and define the wekafs to use extended attributes in the SELinux policy configuration (fs_use_xattr).

Kernel

  • 5.3-5.15

  • 4.4.0-1106 to 4.19

  • 3.10

Note: Kernel 5.15 is not supported with Amazon Linux operating systems.

Note: It is advisable to turn off auto kernel updates so it will not get upgraded to an unsupported version.

Weka installation directory

  • Directory: /opt/weka

  • Must be on an SSD or SSD-like performance, for example, M.2.

    • It cannot be shared remotely, NFS mounted, or on a RAM drive.

  • If two boot drives are available, it is recommended to dedicate one for the OS and one for the WEKA /opt/weka directory (there is no need to set software RAID, and some of its implementations are also known to have issues).

  • At least 26 GB is available for the WEKA system installation, with an additional 10 GB for each core used by WEKA.

  • Use a separate filesystem on a separate partition for /opt/weka.

Networking

Note: At least 4k MTU is advised on WEKA cluster servers, NICs, and the switches the servers are connected to.

A WEKA system can be configured without jumbo frames for Ethernet and Infiniband configurations. However, it will provide minimal performance and cannot handle high data loads. Consult with the Customer Success Team before running in this mode.

Jumbo Frames are not required for clients. However, performance might be limited.

Ethernet

NIC

  • Amazon ENA

  • Intel E810 2CQDA2

  • Intel X540

  • Intel X550-T1

  • Intel X710

  • Intel X710-DA2

  • Intel XL710

  • Intel XL710-Q2

  • Intel XXV710

  • Intel 82599ES

  • Intel 82599

  • Mellanox ConnectX-6-Lx

  • Mellanox ConnectX-6-Dx

  • Mellanox ConnectX-6

  • Mellanox ConnectX-5-Ex

  • Mellanox ConnectX-5-Bf

  • Mellanox ConnectX-5

  • Mellanox ConnectX-4-Lx

  • Mellanox ConnectX-4

Intel E810 NIC has specific requirements and certain limitations:

  • The Ice Linux Base Driver version 1.9.11 and firmware version 4.0.0.

  • Working with this NIC is only supported on RHEL 8.6 and Rocky Linux 8.6. For other operating systems, contact the Customer Success Team.

  • Only non-routed network is supported with this NIC.

  • This NIC is supported only by clients.

  • Multiple containers architecture is not yet supported with this NIC.

Note: Link aggregation (LACP, also known as bond interfaces) is currently supported between ports on a single Mellanox NIC and is not supported when using virtual functions (VFs).

NIC drivers

Supported Mellanox OFED versions:

  • 5.8-1.1.2.1 LTS

  • 5.7-1.0.2.0

  • 5.6-2.0.9.0

  • 5.6-1.0.3.3

  • 5.4-3.5.8.0 LTS

  • 5.4-3.4.0.0 LTS

  • 5.1-2.6.2.0

  • 5.1-2.5.8.0

Supported ENA drivers:

  • 1.0.2 - 2.0.2

  • A current driver from an official OS repository is recommended

Supported ixgbevf drivers:

  • 3.2.2 - 4.1.2

  • A current driver from an official OS repository is recommended

Supported Intel 40 drivers:

  • 3.0.1-k - 4.1.0

  • A current driver from an official OS repository is recommended

Supported ice drivers:

  • 1.9.11

Ethernet configuration

  • Ethernet speeds: 200 GbE / 100 GbE / 50GbE / 40 GbE / 25 GbE / 10 GbE

  • NICs bonding: Can bond dual ports on the same NIC (modes 1 or 4)

  • VLAN: Not supported

  • Connectivity between servers:

    • For multiple containers architecture, the default for the Resources Generator is 14000-14100, 14200-14300, and 14400-14500 for the first three containers (these values can be customized).

    • For single container architecture: Ports 14000-14300.

  • Mellanox NICs:

    • One Weka system IP address for management and data plane

  • Other vendors NICs

    • Weka system management IP address: One IP per server (configured before Weka installation)

    • Weka system data plane IP address: One IP address for each Weka core in each server (Weka will apply these IPs during the cluster initialization)

    • Weka system management IP: Ability to communicate with all Weka system data plane IPs

    • Virtual Functions (VFs): The maximum number of VFs supported by the device must be bigger than the number of physical cores on the server; you should set the number of VFs to the number of cores you wish to dedicate to Weka; some configurations may be required in the BIOS

    • SR-IOV: Enabled in BIOS

Note: When assigning a network device to the Weka system, no other application can create VFs on that device.

InfiniBand

NIC

  • Mellanox ConnectX-6

  • Mellanox ConnectX-5

  • Mellanox ConnectX-5-Ex

  • Mellanox ConnectX-4

  • Mellanox ConnectX-4-Lx

NIC Drivers

Supported Mellanox OFED versions:

  • 5.8-1.1.2.1 LTS

  • 5.7-1.0.2.0

  • 5.6-2.0.9.0

  • 5.6-1.0.3.3

  • 5.4-3.5.8.0 LTS

  • 5.4-3.4.0.0 LTS

  • 5.1-2.6.2.0

  • 5.1-2.5.8.0

Infiniband Configuration

  • InfiniBand speeds: FDR / EDR / HDR

  • Subnet manager: Configured to 4092

  • One Weka system IP address for management and data plane

  • PKEYs: Supported

  • Dual InfiniBand can be used for both HA and higher bandwidth

Note: If it is necessary to change PKEYs, contact the Customer Success Team.

HA

  • The network is configured as described in Weka Networking - HA.

SSDs

  • Support PLP (Power Loss Protection)

  • Dedicated for Weka system storage (partition not supported)

  • Supported drive capacity: Up to 16 TB

  • IOMMU mode for SSD drives is not supported. When IOMMU configuration is required on the Weka cluster servers (e.g., due to specific applications when running the Weka cluster in converged mode), contact the Customer Success Team.

Note: To get the best performance, ensure TRIM is supported by the device and enabled in the operating system.

Object store

  • API must be S3 compatible:

    • GET

      • Including byte-range support with expected performance gain when fetching partial objects

    • PUT

      • Supports any byte size of up to 65 MiB

    • DELETE

  • Data Consistency: AWS S3 consistency guarantee:

    • GET after a single PUT must be entirely consistent

    • Multiple PUTs should eventually be consistent

Certified object stores:

  • AWS S3

    • S3 Standard

    • S3 Intelligent-Tiering

    • S3 Standard-IA

    • S3 One Zone-IA

    • S3 Glacier Instant Retrieval

  • Azure Blob Storage

  • Google Cloud Storage (GCS)

  • Cloudian HyperStore (version 7.3 and up)

  • Dell EMC ECS v3.5 and up

  • HCP Classic V9.2 and up (with versioned buckets only)

  • HCP for Cloud-Scale V2.x

  • IBM Cloud Object Storage System (version 3.14.7 and up)

  • Quantum ActiveScale (version 5.5.1 and up)

  • Red Hat Ceph Storage (version 5.0 and up)

  • Oracle Cloud Infrastructure (OCI) S3 compatibility layer

  • Scality Ring8 (version 7.4.4.8 and up)

  • Scality Artesca (version 1.5.2 and up)

  • SwiftStack (version 6.30 and up)

  • WEKA S3

Virtual Machines

Virtual Machines (VMs) can be used as clients only. Ensure the following prerequisites are met for the relevant client type:

  • To avoid irregularities, crashes, and inability to handle application load, make sure there is no CPU starvation to the Weka process by reserving the CPU in the virtual platform and dedicating a core to the Weka client.

  • The root filesystem must handle a 3K IOPS load by the Weka client.

  • To avoid irregularities, crashes, and inability to handle application load, make sure there is no CPU starvation to the Weka process by reserving the CPU in the virtual platform and dedicating a core to the Weka client.

  • The root filesystem must handle a 3K IOPS load by the Weka client.

  • The virtual platform interoperability, such as a hypervisor, NICs, CPUs, and different versions, must support DPDK and virtual network driver.

Special note for a VMware platform

  • If using vmxnet3 devices, do not enable the SR-IOV feature (which prevents the vMotion feature). Each frontend process requires a vmxnet3 device and IP, with an additional device and IP per client VM (for the management process).

  • Using vmxnet3 is only supported with core dedication.

For additional information and how-to articles, search the Weka Knowledge Base in the Weka support portal or contact the Customer Success Team.

KMS

  • HashiCorp Vault (version 1.1.5 up to 1.9.x)

  • KMIP-compliant KMS (protocol version 1.2 and up)

    • The KMS should support encryption-as-a-service (KMIP encrypt/decrypt APIs)

    • KMIP certification has been conducted with Equinix SmartKey (powered by Fortanix KMS)

PreviousPay-As-You-Go licenseNextGet support for your WEKA system

Last updated 1 month ago