W E K A
4.2
4.2
  • WEKA v4.2 documentation
    • Documentation revision history
  • WEKA System Overview
    • Introduction
      • WEKA system functionality features
      • Converged WEKA system deployment
      • Optimize redundancy in WEKA deployments
    • SSD capacity management
    • Filesystems, object stores, and filesystem groups
    • WEKA networking
    • Data lifecycle management
    • WEKA client and mount modes
    • WEKA containers architecture overview
    • Glossary
  • Planning and Installation
    • Prerequisites and compatibility
    • WEKA cluster installation on bare metal servers
      • Plan the WEKA system hardware requirements
      • Obtain the WEKA installation packages
      • Install the WEKA cluster using the WMS with WSA
      • Install the WEKA cluster using the WSA
      • Manually install OS and WEKA on servers
      • Manually prepare the system for WEKA configuration
        • Broadcom adapter setup for WEKA system
        • Enable the SR-IOV
      • Configure the WEKA cluster using the WEKA Configurator
      • Manually configure the WEKA cluster using the resources generator
      • Perform post-configuration procedures
      • Add clients
    • WEKA installation on AWS
      • WEKA installation on AWS using Terraform
        • Terraform-AWS-WEKA module description
        • Deployment on AWS using Terraform
        • Required services and supported regions
        • Supported EC2 instance types using Terraform
        • WEKA cluster auto-scaling in AWS
        • Detailed deployment tutorial: WEKA on AWS using Terraform
      • WEKA installation on AWS using the Cloud Formation
        • Self-service portal
        • CloudFormation template generator
        • Deployment types
        • AWS Outposts deployment
        • Supported EC2 instance types using Cloud Formation
        • Add clients
        • Auto scaling group
        • Troubleshooting
    • WEKA installation on Azure
    • WEKA installation on GCP
      • WEKA project description
      • GCP-WEKA deployment Terraform package description
      • Deployment on GCP using Terraform
      • Required services and supported regions
      • Supported machine types and storage
      • Auto-scale instances in GCP
      • Add clients
      • Troubleshooting
  • Getting Started with WEKA
    • Manage the system using the WEKA GUI
    • Manage the system using the WEKA CLI
      • WEKA CLI hierarchy
      • CLI reference guide
    • Run first IOs with WEKA filesystem
    • Getting started with WEKA REST API
    • WEKA REST API and equivalent CLI commands
  • Performance
    • WEKA performance tests
      • Test environment details
  • WEKA Filesystems & Object Stores
    • Manage object stores
      • Manage object stores using the GUI
      • Manage object stores using the CLI
    • Manage filesystem groups
      • Manage filesystem groups using the GUI
      • Manage filesystem groups using the CLI
    • Manage filesystems
      • Manage filesystems using the GUI
      • Manage filesystems using the CLI
    • Attach or detach object store buckets
      • Attach or detach object store bucket using the GUI
      • Attach or detach object store buckets using the CLI
    • Advanced data lifecycle management
      • Advanced time-based policies for data storage location
      • Data management in tiered filesystems
      • Transition between tiered and SSD-only filesystems
      • Manual fetch and release of data
    • Mount filesystems
      • Mount filesystems from Single Client to Multiple Clusters (SCMC)
      • Manage authentication across multiple clusters with connection profiles
    • Snapshots
      • Manage snapshots using the GUI
      • Manage snapshots using the CLI
    • Snap-To-Object
      • Manage Snap-To-Object using the GUI
      • Manage Snap-To-Object using the CLI
    • Quota management
      • Manage quotas using the GUI
      • Manage quotas using the CLI
  • Additional Protocols
    • Additional protocol containers
    • Manage the NFS protocol
      • Supported NFS client mount parameters
      • Manage NFS networking using the GUI
      • Manage NFS networking using the CLI
    • Manage the S3 protocol
      • S3 cluster management
        • Manage the S3 service using the GUI
        • Manage the S3 service using the CLI
      • S3 buckets management
        • Manage S3 buckets using the GUI
        • Manage S3 buckets using the CLI
      • S3 users and authentication
        • Manage S3 users and authentication using the CLI
        • Manage S3 service accounts using the CLI
      • S3 rules information lifecycle management (ILM)
        • Manage S3 lifecycle rules using the GUI
        • Manage S3 lifecycle rules using the CLI
      • Audit S3 APIs
        • Configure audit webhook using the GUI
        • Configure audit webhook using the CLI
        • Example: How to use Splunk to audit S3
      • S3 supported APIs and limitations
      • S3 examples using boto3
    • Manage the SMB protocol
      • Manage SMB using the GUI
      • Manage SMB using the CLI
  • Operation Guide
    • Alerts
      • Manage alerts using the GUI
      • Manage alerts using the CLI
      • List of alerts and corrective actions
    • Events
      • Manage events using the GUI
      • Manage events using the CLI
      • List of events
    • Statistics
      • Manage statistics using the GUI
      • Manage statistics using the CLI
      • List of statistics
    • Insights
    • System congestion
    • Security management
      • Obtain authentication tokens
      • KMS management
        • Manage KMS using the GUI
        • Manage KMS using the CLI
      • TLS certificate management
        • Manage the TLS certificate using the GUI
        • Manage the TLS certificate using the CLI
      • CA certificate management
        • Manage the CA certificate using the GUI
        • Manage the CA certificate using the CLI
      • Account lockout threshold policy management
        • Manage the account lockout threshold policy using GUI
        • Manage the account lockout threshold policy using CLI
      • Manage the login banner
        • Manage the login banner using the GUI
        • Manage the login banner using the CLI
    • User management
      • Manage users using the GUI
      • Manage users using the CLI
    • Organizations management
      • Manage organizations using the GUI
      • Manage organizations using the CLI
      • Mount authentication for organization filesystems
    • Expand and shrink cluster resources
      • Add a backend server
      • Expand specific resources of a container
      • Shrink a cluster
    • Background tasks
      • Manage background tasks using the GUI
      • Manage background tasks using the CLI
    • Upgrade WEKA versions
  • Billing & Licensing
    • License overview
    • Classic license
  • Monitor the WEKA Cluster
    • Deploy monitoring tools using the WEKA Management Station (WMS)
    • WEKA Home - The WEKA support cloud
      • Local WEKA Home overview
      • Deploy Local WEKA Home v3.0 or higher
      • Deploy Local WEKA Home v2.x
      • Explore cluster insights and statistics
      • Manage alerts and integrations
      • Enforce security and compliance
      • Optimize support and data management
    • Set up the WEKAmon external monitoring
    • Set up the SnapTool external snapshots manager
  • Support
    • Get support for your WEKA system
    • Diagnostics management
      • Traces management
        • Manage traces using the GUI
        • Manage traces using the CLI
      • Protocols debug level management
        • Manage protocols debug level using the GUI
        • Manage protocols debug level using the CLI
      • Diagnostics data management
  • Best Practice Guides
    • WEKA and Slurm integration
      • Avoid conflicting CPU allocations
    • Storage expansion best practice
  • Appendices
    • WEKA CSI Plugin
      • Deployment
      • Storage class configurations
      • Tailor your storage class configuration with mount options
      • Dynamic and static provisioning
      • Launch an application using WEKA as the POD's storage
      • Add SELinux support
      • NFS transport failback
      • Upgrade legacy persistent volumes for capacity enforcement
      • Troubleshooting
    • Convert cluster to multi-container backend
    • Create a client image
Powered by GitBook
On this page
  1. WEKA v4.2 documentation

Documentation revision history

WEKA version
Description of changes

4.2.18

Maintenance release.

Removed Azure deployment sections due to a known network issue in Azure deployments that causes performance degradation in versions 4.1 to 4.4.0. This issue is resolved in version 4.4.1 and later.

4.2.17

  • DPDK and AWS Xen-on-Nitro incompatibility identified: Customers using legacy Amazon EC2 Xen-on-Nitro instances must contact AWS Support to request their account be added to the deny list.

  • Corrected client address representation in WEKA events: Resolved an issue where client addresses were displayed as loopback addresses; they now display the correct IP address.

  • Duplicate management address alert added: WEKA now generates an alert when duplicate management addresses are configured, indicating a binding failure.

  • Maximum number of processes increased: The maximum number of backend processes, drive processes, management processes, and total processes have all been increased.

Synchronous Snap: The Synchronous Snap feature, which allows incremental snapshots to be downloaded from an object store, was temporarily disabled in version 4.2.3. It has been re-enabled in version 4.3.0.

4.2.16

  • New event: DriverNotAccepting: Introduced a new event signaling that all I/O operations on a frontend are unresponsive or hanging.

  • New event: DriveImmediateShutdown: Added a new event to indicate an NVMe failure, triggering an immediate shutdown of the affected drive to prevent further system impact.

  • Network virtual function limitation: Implemented the ability to limit the quantity of network virtual functions (VFs). This feature is particularly relevant for clusters with high core counts using Intel E810 NICs.

  • Added support for GCP regions asia-southeast2 and europe-central2 in Terraform configuration.

  • Added Dell PowerScale S3 (version 9.8.0.0 and higher) to the certified object stores.

4.2.15

  • Added a verification step for LLQ and WC in the upgrade workflow. To ensure proper LLQ functionality after upgrades, verify that Write Combining (WC) is enabled in the igb_uio driver.

  • Extended support for RHEL/Rocky Linux 8.10 on backends and clients.

4.2.14

  • Extended support for operating systems on:

    • Clients: RHEL/Rocky Linux 9.4, AlmaLinux 9.4, 8.10, Debian 12.

    • Backends: RHEL/Rocky Linux 9.4.

  • Update: Object store types now included in Analytics reports on WEKA Home.

4.2.12.92

  • Extended support for Linux kernel to Ubuntu 22:

    • 5.19, 6.2, 6.5.

4.2.12

  • Extended support for operating systems on clients: RHEL/Rocky Linux 9.3, Oracle Linux 9.

  • Extended support for Las_v3 machine types for backends on Microsoft Azure.

4.2.11

  • Customers can now see originating IP address information in S3 logs via the X-Forwarded-For header.

  • NFS Floating IP failover now contains a timeout as a mitigating factor during prolonged outages.

  • Limit increases for drives (20k), processes (20k), and NUMA nodes (20).

  • Added Rocky Linux 9.3 to supported operating systems on backends.

4.2.10

  • For both files and directories, relatime now produces reliable atime updates on large clusters. Previously, some conditions caused atime values to revert.

  • Customers can now transition between custom and predefined S3 bucket policies with one command.

4.2.9

  • Path optimization for S3 requests.

  • Custom bucket policies now accept only valid JSON through REST API.

4.2.8

  • Active-active network port usage in HA configurations using RDMA.

  • Better handling for cgroups with small memory footprints.

  • More graceful client upgrades with clusters that have been scaled in, preventing communication with backends that no longer exist.

  • STS session duration option available for IAM AssumeRole use with Amazon S3.

4.2.7

  • Initial support for the namespace feature in Hashicorp Vault.

  • Enhanced performance serving S3 GET requests using byte-range fetching.

  • Faster cleanup of failed multi-part upload parts.

  • Added support for Ubuntu 22.04.3 point release with 6.2-based kernel.

  • SMB-W now supports creating local mappings for AD users and groups using the rid ID-mapping, alongside the existing RFC2307 support.

  • Added support for using AssumeRole with Amazon IAM STS tokens when using Amazon S3.

  • Added support for floating IPs in AWS when using NFS.

4.2.6

  • Added IMDSv2 to the supported Amazon EC2 instances.

  • Certified Broadcom BCM957508-P2100G Dual-Port 100 Gb/s QSFP56 as supported.

  • Added support for IOMMU on WEKA backend servers with Mellanox NICs.

  • Certified OFED 23.10-0.5.5.0.

  • Added support for RHEL/Rocky Linux 9.2 operating system.

  • Updated the default provider of SMB services to SMB-W, replacing the legacy SMB.

4.2.5

  • Alerts are in place for the use of duplicate IP addresses.

  • Reduced likelihood of receiving false alerts for full quotas.

  • Better error handling for aborted S3 MultiPartUpload requests.

  • The weka cluster failure-domain CLI command is enhanced with more component fields.

  • No more umount/mount cycling for SCMC clients during upgrades.

4.2.4

  • Certified CX7 (Infiniband) as supported.

  • Non-disruptive upgrades of clients in more configurations.

  • When using NFS, the df command now reports WEKA quotas as expected.

  • ILM Policy deploys now use less memory when scaled to a billion objects.

  • Deleting empty buckets via S3 API works as expected.

  • To upgrade to 4.2.4, the source version must be 4.1.2.

  • If the S3 protocol is configured, contact Customer Success to confirm that ETCD (internal key-value store) has been upgraded to KWAS.

4.2.3

  • Added operating systems supported on clients: RHEL/Rocky Linux 8.8 and SuSe 15 SP4.

  • Added support for Vault 1.14 (certified from WEKA release 4.2.1).

  • NFS client connections are steady during floating address migration.

  • The weka local upgrade command now supports servers without frontend containers.

  • The weka status command output newly reflects the unavailable capacity.

  • The Synchronized Snapshot feature (the ability to perform incremental snapshots downloaded from an object store) is temporarily disabled. This feature will be reinstated upon subsequent release.

4.2.2

N/A

4.2.1

  • IAM support on GCP: You can access Google Cloud Storage using a service account with the required permissions granted by the IAM role.

  • NDU improvement: The non-disruptive upgrade process is now improved by upgrading the compute containers one at a time (rolling upgrade) while the remaining containers continue serving the clients.

  • E810 NIC support on MCB: The Intel E810 NIC is now supported in the multi-container backends (MCB) architecture.

  • Added support for Rocky 9.0, 9.1, and Ubuntu 22.04 on backends.

4.2.0

  • Mount filesystems from multiple clusters on a single client: You can mount filesystems from up to seven clusters in parallel on a single WEKA client for enhanced performance and use cases.

  • Snapshots improvements:

    • Quickly download a previously taken snapshot to another cluster. Following that operation, metadata is auto-prefetched.

    • Allow IO operations continuity to the filesystem while restoring a snapshot (using a preserved snapshot name).

    • Added abort/pause snapshot download functions.

  • WEKA CSI Plugin enhancements:

    • Can control WEKA mount options through the storage class.

    • Snapshots and volume cloning.

    • Added support for k8s fsGroups.

  • Increased organizations support: Increased the maximum number of supported organizations to 256 per cluster.

  • New GUI improvements: The GUI is improved with new features and operations, such as the insights page with top processes usage, drives load, and latency.

  • Azure cloud enhancement:

    • Improved performance using DPDK networking and higher MTU.

    • Added support for auto-scaling.

  • Interoperability updates:

    • Added support for Mellanox OFED version 5.9-0.5.6.0.

    • Added support RHEL/Rocky Linux 9.1/9.0 and Ubuntu 22.04 for clients only (only use applications set with 2MB hugepages).

  • HashiCorp Vault: Added support for HashiCorp Vault up to version 1.13.

Breaking changes and deprecations:

  • Removed the auth_token mount option.

  • Single protocol type per server support: A single WEKA server can now only have one protocol server (S3, SMB, or NFS). Adding an additional protocol process/server is no longer allowed. For clusters being upgraded, distribute the various (S3, SMB, or NFS) protocols across all the backend servers before running the upgrade.

  • Only the multi-container backend (MCB) architecture is supported: The legacy Single Container Backend (SCB) architecture, where each server in the cluster includes a single container with all the processes running on it, is deprecated in 4.1. To upgrade to 4.2, the source cluster must be in MCB architecture. Contact the WEKA Customer Success team to convert a 4.1 cluster in a legacy architecture to the MCB architecture.

  • ETCD replacement for S3 protocol improvement: The ETCD component, which stores the IAM format, policies, service accounts, users, STS, and policy mappings, is replaced by a more robust mechanism. When upgrading a cluster running with ETCD from V4.1, the cluster continues to run with ETCD in V4.2, and an alert is raised to migrate to the new mechanism. Contact the WEKA Customer Success team to perform this update.

PreviousWEKA v4.2 documentationNextIntroduction

Last updated 3 months ago

For updated guidance, see the sections in the Version 4.4 documentation.

New CLI reference guide: This CLI reference guide is generated from the output of running the weka command with the help option. It provides detailed descriptions of available commands, arguments, and options.

New topic:

Manage authentication across multiple clusters with connection profiles
WEKA Installation on Azure
>>>