W E K A
3.14
3.14
  • WEKA v3.14 Documentation
  • Weka System Overview
    • About the WEKA System
    • SSD Capacity Management
    • Filesystems, Object Stores & Filesystem Groups
    • Weka Networking
    • Data Lifecycle Management
    • Weka Client & Mount Modes
    • Glossary
  • Getting Started with Weka
    • Quick Install Guide
    • Managing the Weka System
    • CLI Overview
    • GUI Overview
    • Serving IOs with WekaFS
  • Planning & Installation
    • Prerequisites for Installation
    • Bare Metal Installation
      • Planning a Weka System Installation
      • Setting Up the Hosts
        • SR-IOV Enablement
      • Obtaining the Weka Install File
      • Weka System Installation Process Using the CLI
      • Adding Clients
    • AWS Installation
      • Self-Service Portal
      • CloudFormation Template Generator
      • Deployment Types
      • AWS Outposts Deployment
      • Supported EC2 Instance Types
      • Adding Clients
      • Auto Scaling Group
      • Troubleshooting
  • Performance
    • Testing Weka Performance
      • Test Environment Details
  • WekaFS Filesystems
    • Managing Filesystems, Object Stores & Filesystem Groups
      • Managing Object Stores
      • Managing Filesystem Groups
      • Managing Filesystems
      • Attaching/Detaching Object Stores to/from Filesystems
      • KMS Management
    • Advanced Data Lifecycle Management
      • Advanced Time-based Policies for Data Storage Location
      • Data Management in Tiered Filesystems
      • Transition Between Tiered and SSD-Only Filesystems
      • Manual fetch and release of data
    • Mounting Filesystems
    • Snapshots
    • Snap-To-Object
    • Quota Management
  • Additional Protocols
    • NFS
    • SMB
      • SMB Management Using CLIs
      • SMB Management Using the GUI
    • S3
      • S3 Cluster Management
      • S3 Buckets Management
      • S3 Users and Authentication
      • S3 Information Lifecycle Management
      • Audit S3 APIs
      • S3 Limitations
      • S3 Examples using boto3
  • Operation Guide
    • Alerts
      • List of Alerts
    • Events
      • List of Events
    • Statistics
      • List of Statistics
    • System Congestion
    • Security
      • User Management
      • Organizations
    • Expanding & Shrinking Cluster Resources
      • Expand & Shrink Overview
      • Stages in Adding a Backend Host
      • Expansion of Specific Resources
      • Shrinking a Cluster
    • Background Tasks
    • Upgrading Weka Versions
  • Billing & Licensing
    • License Overview
    • Classic License
    • Pay-As-You-Go License
  • Support
    • Prerequisites and Compatibility
    • Getting Support for Your Weka System
    • The Weka Support Cloud
    • Diagnostics CLI Command
  • Appendix
    • Weka CSI Plugin
    • External Monitoring
    • Snapshot Management
  • REST API
Powered by GitBook
On this page
  • About Expanding & Shrinking Cluster
  • Planning an Expansion or Shrink
  • Possible Expansion Options
  • Expansion Limitations
  • The Cluster Expansion Process
  1. Operation Guide
  2. Expanding & Shrinking Cluster Resources

Expand & Shrink Overview

This page presents an overview of the cluster expand and shrink process in a homogeneous Weka system configuration.

PreviousExpanding & Shrinking Cluster ResourcesNextStages in Adding a Backend Host

Last updated 3 years ago

Note: The cluster expansion process described here is only applicable to a homogeneous Weka system configuration, which is highly recommended. For non-homogeneous Weka system configurations, contact the Weka Support Team.

Note: For AWS deployments, **CloudFormation should only be used for initial deployment, and not for expanding & shrinking cluster resources.

About Expanding & Shrinking Cluster

In the Weka system, it is possible to expand and shrink a cluster as follows:

  1. Add or delete backend hosts

  2. Add or delete SSDs from an existing backend host

  3. Change the number of cores assigned to the Weka system in existing backend hosts

  4. Change the amount of memory allocated to the Weka system in existing backend hosts

  5. Change the network resources assigned to the Weka system in existing backend hosts

Note: The expansion or shrinking of networking resources is performed infrequently.

Planning an Expansion or Shrink

Note: The expansion of a Weka system offers the opportunity to increase performance, while the shrinking of a Weka system may reduce performance. Contact the Weka Support Team for more details and to receive estimates.

Note: In the following descriptions, cluster expansion also relates to cluster shrinking.

Expansion procedures are similar to the . Similar to planning a new cluster, the objectives of the expansion, in terms of space and performance, need to be translated to the actual cluster resources. This process is practically a repeat of the planning process for new clusters, with the following options and limitations:

Possible Expansion Options

  • Addition of new backend hosts.

  • Addition of new failure domains, as long the system was installed with failure domains.

  • Addition of new SSDs to existing backend hosts.

  • Assignment of additional cores to Weka in existing backend hosts.

  • Assignment of more memory to Weka in existing backend hosts.

  • Assignment of additional network resources to Weka in existing backend hosts.

  • Reconfiguration of hot spares.

Expansion Limitations

  • It is not possible to change the defined Weka system protection scheme.

  • It is not possible to define failure domains on a system that was installed without failure domains.

  • A Weka system configured with failure domains cannot be configured to be without failure domains.

  • Only the same network technology can be implemented i.e., it is not possible to mix between Ethernet and InfiniBand.

The Cluster Expansion Process

Once an expansion of more SSDs or backend hosts has been planned and executed, the Weka system starts a redistribution process. This involves the redistribution of all the existing data to be perfectly balanced between the original hosts or SSDs and newly added resources. This process can take from minutes to hours, depending on the capacity and the networking CPU resources. However, the capacity increase is instant, and therefore it is possible to define more filesystems immediately, without waiting for the completion of the redistribution process.

Note: If necessary, contact the Weka Support Team for more details on the redistribution process and its expected duration.

Once the expansion of more cores or backend hosts has been implemented, the added CPU resources are operational in less than a minute. Write performance improves almost immediately, while read performance only improves on completion of the redistribution of the data.

Note: As part of the requirements for a homogeneous Weka system configuration, when expanding memory resources, the new hosts must have the same memory as the existing hosts.

To plan the capacity of the Weka system after expansion, refer to .

Bare Metal Weka system Installation Procedure
SSD Capacity Management