W E K A
4.3
4.3
  • WEKA v4.3 documentation
    • Documentation revision history
  • WEKA System Overview
    • WEKA Data Platform introduction
      • WEKA system functionality features
      • Converged WEKA system deployment
      • Optimize redundancy in WEKA deployments
    • SSD capacity management
    • Filesystems, object stores, and filesystem groups
    • WEKA networking
    • Data lifecycle management
    • WEKA client and mount modes
    • WEKA containers architecture overview
    • Glossary
  • Planning and Installation
    • Prerequisites and compatibility
    • WEKA cluster installation on bare metal servers
      • Plan the WEKA system hardware requirements
      • Obtain the WEKA installation packages
      • Install the WEKA cluster using the WMS with WSA
      • Install the WEKA cluster using the WSA
      • Manually install OS and WEKA on servers
      • Manually prepare the system for WEKA configuration
        • Broadcom adapter setup for WEKA system
        • Enable the SR-IOV
      • Configure the WEKA cluster using the WEKA Configurator
      • Manually configure the WEKA cluster using the resource generator
      • Perform post-configuration procedures
      • Add clients to an on-premises WEKA cluster
    • WEKA Cloud Deployment Manager Web (CDM Web) User Guide
    • WEKA Cloud Deployment Manager Local (CDM Local) User Guide
    • WEKA installation on AWS
      • WEKA installation on AWS using Terraform
        • Terraform-AWS-WEKA module description
        • Deployment on AWS using Terraform
        • Required services and supported regions
        • Supported EC2 instance types using Terraform
        • WEKA cluster auto-scaling in AWS
        • Detailed deployment tutorial: WEKA on AWS using Terraform
      • WEKA installation on AWS using the Cloud Formation
        • Self-service portal
        • CloudFormation template generator
        • Deployment types
        • AWS Outposts deployment
        • Supported EC2 instance types using Cloud Formation
        • Add clients to a WEKA cluster on AWS
        • Auto scaling group
        • Troubleshooting
      • Install SMB on AWS
    • WEKA installation on Azure
    • WEKA installation on GCP
      • WEKA project description
      • GCP-WEKA deployment Terraform package description
      • Deployment on GCP using Terraform
      • Required services and supported regions
      • Supported machine types and storage
      • Auto-scale instances in GCP
      • Add clients to a WEKA cluster on GCP
      • Troubleshooting
      • Detailed deployment tutorial: WEKA on GCP using Terraform
      • Google Kubernetes Engine and WEKA over POSIX deployment
  • Getting Started with WEKA
    • Manage the system using the WEKA GUI
    • Manage the system using the WEKA CLI
      • WEKA CLI hierarchy
      • CLI reference guide
    • Run first IOs with WEKA filesystem
    • Getting started with WEKA REST API
    • WEKA REST API and equivalent CLI commands
  • Performance
    • WEKA performance tests
      • Test environment details
  • WEKA Filesystems & Object Stores
    • Manage object stores
      • Manage object stores using the GUI
      • Manage object stores using the CLI
    • Manage filesystem groups
      • Manage filesystem groups using the GUI
      • Manage filesystem groups using the CLI
    • Manage filesystems
      • Manage filesystems using the GUI
      • Manage filesystems using the CLI
    • Attach or detach object store buckets
      • Attach or detach object store bucket using the GUI
      • Attach or detach object store buckets using the CLI
    • Advanced data lifecycle management
      • Advanced time-based policies for data storage location
      • Data management in tiered filesystems
      • Transition between tiered and SSD-only filesystems
      • Manual fetch and release of data
    • Mount filesystems
      • Mount filesystems from Single Client to Multiple Clusters (SCMC)
    • Snapshots
      • Manage snapshots using the GUI
      • Manage snapshots using the CLI
    • Snap-To-Object
      • Manage Snap-To-Object using the GUI
      • Manage Snap-To-Object using the CLI
    • Quota management
      • Manage quotas using the GUI
      • Manage quotas using the CLI
  • Additional Protocols
    • Additional protocol containers
    • Manage the NFS protocol
      • Supported NFS client mount parameters
      • Manage NFS networking using the GUI
      • Manage NFS networking using the CLI
    • Manage the S3 protocol
      • S3 cluster management
        • Manage the S3 service using the GUI
        • Manage the S3 service using the CLI
      • S3 buckets management
        • Manage S3 buckets using the GUI
        • Manage S3 buckets using the CLI
      • S3 users and authentication
        • Manage S3 users and authentication using the CLI
        • Manage S3 service accounts using the CLI
      • S3 rules information lifecycle management (ILM)
        • Manage S3 lifecycle rules using the GUI
        • Manage S3 lifecycle rules using the CLI
      • Audit S3 APIs
        • Configure audit webhook using the GUI
        • Configure audit webhook using the CLI
        • Example: How to use Splunk to audit S3
      • S3 supported APIs and limitations
      • S3 examples using boto3
      • Access S3 using AWS CLI
    • Manage the SMB protocol
      • Manage SMB using the GUI
      • Manage SMB using the CLI
  • Operation Guide
    • Alerts
      • Manage alerts using the GUI
      • Manage alerts using the CLI
      • List of alerts and corrective actions
    • Events
      • Manage events using the GUI
      • Manage events using the CLI
      • List of events
    • Statistics
      • Manage statistics using the GUI
      • Manage statistics using the CLI
      • List of statistics
    • Insights
    • System congestion
    • Security management
      • Obtain authentication tokens
      • KMS management
        • Manage KMS using the GUI
        • Manage KMS using the CLI
      • TLS certificate management
        • Manage the TLS certificate using the GUI
        • Manage the TLS certificate using the CLI
      • CA certificate management
        • Manage the CA certificate using the GUI
        • Manage the CA certificate using the CLI
      • Account lockout threshold policy management
        • Manage the account lockout threshold policy using GUI
        • Manage the account lockout threshold policy using CLI
      • Manage the login banner
        • Manage the login banner using the GUI
        • Manage the login banner using the CLI
      • Manage Cross-Origin Resource Sharing
    • User management
      • Manage users using the GUI
      • Manage users using the CLI
    • Organizations management
      • Manage organizations using the GUI
      • Manage organizations using the CLI
      • Mount authentication for organization filesystems
    • Expand and shrink cluster resources
      • Add a backend server
      • Expand specific resources of a container
      • Shrink a cluster
    • Background tasks
      • Set up a Data Services container for background tasks
      • Manage background tasks using the GUI
      • Manage background tasks using the CLI
    • Upgrade WEKA versions
  • Licensing
    • License overview
    • Classic license
  • Monitor the WEKA Cluster
    • Deploy monitoring tools using the WEKA Management Station (WMS)
    • WEKA Home - The WEKA support cloud
      • Local WEKA Home overview
      • Deploy Local WEKA Home v3.0 or higher
      • Deploy Local WEKA Home v2.x
      • Explore cluster insights and statistics
      • Manage alerts and integrations
      • Enforce security and compliance
      • Optimize support and data management
    • Set up the WEKAmon external monitoring
    • Set up the SnapTool external snapshots manager
  • Support
    • Get support for your WEKA system
    • Diagnostics management
      • Traces management
        • Manage traces using the GUI
        • Manage traces using the CLI
      • Protocols debug level management
        • Manage protocols debug level using the GUI
        • Manage protocols debug level using the CLI
      • Diagnostics data management
  • Best Practice Guides
    • WEKA and Slurm integration
      • Avoid conflicting CPU allocations
    • Storage expansion best practice
  • WEKApod
    • WEKApod Data Platform Appliance overview
    • WEKApod servers overview
    • Rack installation
    • WEKApod initial system setup and configuration
    • WEKApod support process
  • Appendices
    • WEKA CSI Plugin
      • Deployment
      • Storage class configurations
      • Tailor your storage class configuration with mount options
      • Dynamic and static provisioning
      • Launch an application using WEKA as the POD's storage
      • Add SELinux support
      • NFS transport failback
      • Upgrade legacy persistent volumes for capacity enforcement
      • Troubleshooting
    • Convert cluster to multi-container backend
    • Create a client image
    • Update WMS and WSA
    • BIOS tool
Powered by GitBook
On this page
  1. WEKA v4.3 documentation

Documentation revision history

PreviousWEKA v4.3 documentationNextWEKA Data Platform introduction

Last updated 2 months ago

WEKA version
Description of changes

Added new topics:

  • Access S3 using AWS CLI: Configure and use the AWS CLI with WEKA’s S3 storage.

  • Install SMB on AWS: Set up SMB in AWS for secure file sharing and access control.

Removed Azure deployment sections due to a known network issue in Azure deployments that causes performance degradation in versions 4.1 to 4.4.0. This issue is resolved in version 4.4.1 and later.

For updated guidance, see the sections in the Version 4.4 documentation.

  • Updated the note to specify that only snapshots uploaded from version 4.3 or later can be downloaded using Synchronous Snap. Previously, the note indicated version 4.0 or later.

  • Added support for GCP regions asia-southeast2 and europe-central2 in Terraform configuration.

  • CDM Local version 1.2 updates: Supports automated Terraform deployment, removes the Windows installation package, updates the launch process, and enhances information gathering options.

  • Introducing WEKA CSI Plugin version 2.5.0, which provides NFS transport support designed for non-performance-critical scenarios or environments where installing the WEKA client is not feasible.

  • Added the WEKApod Data Platform Appliance Guide, which provides comprehensive instructions for setting up and configuring the WEKApod™. The WEKApod is a turnkey data platform appliance designed as a high-performance datastore for NVIDIA DGX SuperPOD. Each appliance includes pre-configured storage servers and software, enabling simplified deployment and faster time to value. For details, see .

4.3.5

This release aims to enhance usability, focusing on WEKA SMB-W. It also contains field fixes and support for new platforms and hardware.

SMB-W enhancements

  • Case sensitivity in SMB-W shares, beyond the case insensitive limits of the Microsoft implementation.

  • Symbolic link following in SMB-W shares.

  • Named streams support in shares, sometimes called alternate data streams (ADS).

  • Optional reduction of shared state between SMB-W containers for workloads that don't rely on it.

  • Zero copy support for MacOS clients is enabled by default, but can be disabled in share options.

  • Share updates support read-only and hidden properties.

S3 enhancements

  • Updated the s3 cluster status output to align with the style of weka cluster container, showing details like Hostname, Status, Uptime, Last Error, and Active Requests. Use weka s3 cluster status-old during upgrades, as initial output may be inaccurate until all frontend containers are updated.

Additional enhancements

  • The REST API for quota listing includes the full directory path, aligning with the CLI command output.

  • Extended support for operating systems:

    • Clients: RHEL/Rocky Linux 9.4, AlmaLinux 9.4, 8.10, Debian 12, Ubuntu 24.04.

    • Backends: RHEL/Rocky Linux 9.4.

  • Extended support for OFED 24.04-07.0.0.

  • Improved NFS read operation performance.

  • Added support for Broadcom P2200.

  • Added support for colored output in the weka status and other table-returning commands, configurable by the --color option or WEKA_CLI_COLOR environment variable. See .

  • Introduced the Data Services container for running resource-intensive background tasks, starting with the Quota Coloring task, with more tasks to be supported in future releases. See .

4.3.4

  • This release contains an important enhancement for clusters under extreme memory pressure.

N/A

  • Added a new topic to WEKA and Slurm integration: .

4.3.3

  • NFS locking is fully supported for protocol versions 3 and 4, and configurable through global parameters.

  • Added support for additional QLC drives for higher density deployments while maintaining high performance and longevity standards.

  • SMB-W performance is optimized for macOS clients.

  • SMB-W share access now supports allow and deny statements based on IP address.

  • For Windows clients with many top-level entities, folder options appear without delay.

  • WEKA client support extended to Rocky/RHEL 8.9 and 9.3, and Oracle Linux 9.

4.3.2

  • Added graceful container management to ensure safe stopping or restarting of a container. See .

  • Added the , simplifying the deployment of WEKA clusters in public cloud environments (including AWS, Azure, and GCP).

  • A new algorithm for multi-part uploads is now the default, decreasing resource overhead for most customers using WEKA S3.

  • WEKA S3 response codes are now in weka stats, with types and categories listed under Operation (Envoy).

  • WEKA S3 logs now report real remote client addresses via the X-Forwarded-For header.

  • The default memory limit for SMB-W containers has been raised to 32GB to accommodate trends in customer requirements related to SMB protocol performance.

Deprecation

  • RHEL 7 and CentOS 7 support have reached EOL status.

  • Deployment on Amazon EC2 instance families M4, R3, R4, and C4 are no longer supported.

4.3.1

  • Added a topic to restrict a stateless client’s operations to only the essential APIs for mounting and unmounting operations. See .

  • Added the following instance types to the section: C7a, M7i, R7a, X2idn, X2iedn, R7iz, R7g, R7gd, R6g, R6gd, G6, GR6, F1, and Z1d.

4.3.0

  • Initial support for ARM, specifically for ARM-based EC2 instances in AWS. To reflect the instruction set architecture of hosts in a cluster, APIs, and commands like the Weka cluster server list now contain an architecture column.

  • Added .

  • Synchronized snapshots have been re-enabled.

  • A new --client-only flag in the weka version get and weka version set commands. When used, this flag ensures that only components necessary for stateless clients are considered for installation.

  • Discontinued support for Intel processor families SandyBridge (2011) and IvyBridge (2012).

  • Discontinued support for the legacy NFS stack in 4.3.0. If legacy NFS is enabled, upgrading to 4.3.0 will fail.

  • weka alerts command output update The muted column is removed from the weka alerts command output unless --muted or -v is provided.

  • weka nfs interface-group add command update The manage-gids parameter, which was always set to true, has been eliminated from the weka nfs interface-group add command and associated API.

  • weka fs tier s3 command output update The following columns have been removed from the weka fs tier s3 command output: NODES UP, NODES DOWN, and NODES UNKNOWN. Instead, more detailed columns have been added in the verbose output (using -v).

WEKApod
Set up a Data Services container for background tasks
Avoid conflicting CPU allocations
>>>
>>>
WEKA Installation on Azure
WEKA Cloud Deployment Manager (CDM) User Guide
Client EC2 instances
NFS integration with Kerberos service
WEKA CLI command output colors
Isolated port for restricted stateless client operations
Graceful container management: ensuring safe actions