W E K A
3.14
3.14
  • WEKA v3.14 Documentation
  • Weka System Overview
    • About the WEKA System
    • SSD Capacity Management
    • Filesystems, Object Stores & Filesystem Groups
    • Weka Networking
    • Data Lifecycle Management
    • Weka Client & Mount Modes
    • Glossary
  • Getting Started with Weka
    • Quick Install Guide
    • Managing the Weka System
    • CLI Overview
    • GUI Overview
    • Serving IOs with WekaFS
  • Planning & Installation
    • Prerequisites for Installation
    • Bare Metal Installation
      • Planning a Weka System Installation
      • Setting Up the Hosts
        • SR-IOV Enablement
      • Obtaining the Weka Install File
      • Weka System Installation Process Using the CLI
      • Adding Clients
    • AWS Installation
      • Self-Service Portal
      • CloudFormation Template Generator
      • Deployment Types
      • AWS Outposts Deployment
      • Supported EC2 Instance Types
      • Adding Clients
      • Auto Scaling Group
      • Troubleshooting
  • Performance
    • Testing Weka Performance
      • Test Environment Details
  • WekaFS Filesystems
    • Managing Filesystems, Object Stores & Filesystem Groups
      • Managing Object Stores
      • Managing Filesystem Groups
      • Managing Filesystems
      • Attaching/Detaching Object Stores to/from Filesystems
      • KMS Management
    • Advanced Data Lifecycle Management
      • Advanced Time-based Policies for Data Storage Location
      • Data Management in Tiered Filesystems
      • Transition Between Tiered and SSD-Only Filesystems
      • Manual fetch and release of data
    • Mounting Filesystems
    • Snapshots
    • Snap-To-Object
    • Quota Management
  • Additional Protocols
    • NFS
    • SMB
      • SMB Management Using CLIs
      • SMB Management Using the GUI
    • S3
      • S3 Cluster Management
      • S3 Buckets Management
      • S3 Users and Authentication
      • S3 Information Lifecycle Management
      • Audit S3 APIs
      • S3 Limitations
      • S3 Examples using boto3
  • Operation Guide
    • Alerts
      • List of Alerts
    • Events
      • List of Events
    • Statistics
      • List of Statistics
    • System Congestion
    • Security
      • User Management
      • Organizations
    • Expanding & Shrinking Cluster Resources
      • Expand & Shrink Overview
      • Stages in Adding a Backend Host
      • Expansion of Specific Resources
      • Shrinking a Cluster
    • Background Tasks
    • Upgrading Weka Versions
  • Billing & Licensing
    • License Overview
    • Classic License
    • Pay-As-You-Go License
  • Support
    • Prerequisites and Compatibility
    • Getting Support for Your Weka System
    • The Weka Support Cloud
    • Diagnostics CLI Command
  • Appendix
    • Weka CSI Plugin
    • External Monitoring
    • Snapshot Management
  • REST API
Powered by GitBook
On this page
  • About Filesystems
  • Thin Provisioning
  • Filesystem Limits
  • Encrypted Filesystems
  • Metadata Limitations
  • Metadata Calculations
  • About Object Stores
  • About Filesystem Groups
  1. Weka System Overview

Filesystems, Object Stores & Filesystem Groups

This page describes the three types of entities relevant to data storage in the Weka system: filesystems, object stores and filesystem groups.

PreviousSSD Capacity ManagementNextWeka Networking

Last updated 3 years ago

About Filesystems

A Weka filesystem is similar to a regular on-disk filesystem, with the key difference that it's distributed across all the hosts in the cluster. Consequently, in the Weka system, filesystems are not associated with any physical object and are therefore nothing but a root directory with space limitations.

A total of up to 1024 filesystems are supported, all of which are equally and perfectly balanced on all SSDs and CPU cores assigned to the Weka system. This means that the allocation of a new filesystem, or the resizing of a filesystem, are instant management operations that are performed instantly, and without any constraints.

Note: A has to be created before creating a filesystem, and every filesystem must belong to one filesystem group.

A filesystem must have a defined capacity limit. A filesystem that belongs to a tiered must have a total capacity limit and an SSD capacity limit. The total (base) SSD capacity of all filesystems cannot exceed the total SSD capacity as defined in the total SSD net capacity.

Thin Provisioning

While a filesystem must have a minimum allocated SSD size, on top of that, filesystems can have an additional SSD capacity cap. The total can exceed the total SSD net capacity and be dynamically allocated between the different thin-provisioned filesystems.

Thin provisioning can help in various use cases:

  • On tiered filesystems, available SSD space will be leveraged for extra performance and released to the object-store once needed by other filesystems.

  • When using auto-scaling groups, using thin provisioning can help to automatically expand and shrink the filesystem's SSD capacity for extra performance.

  • Thin provisioning can be used for the logical separation of projects to filesystems when the administrator doesn't expect all to be fully utilized at the same time. Thin provisioned filesystems can replace the using the same filesystem with different directories with quotas and provide better data separation, management, and access control.

Filesystem Limits

  • Up to 6.4 trillion files/directories

  • Up to 6.4 billion files in a single directory

  • Up to 14 EB (using and object-store) with up to 512 PB on SSD

  • UP to 4 PB file size

Encrypted Filesystems

Note: Setting data encryption (on/off) can only be done when creating a filesystem.

Metadata Limitations

By default, the metadata limit associated with a filesystem is proportional to the filesystem SSD size. It is possible to override this default by defining a filesystem-specific max-files parameter. The filesystem limit is a logical limit to control the specific filesystem usage, and can always be updated by the administrator when necessary.

The total limits of the metadata for all the filesystems can exceed the total system metadata information that can fit in RAM. For minimal impact, in such a case, the least-recently-used units will be paged to disk, as necessary.

Metadata Calculations

Each metadata unit consumes 4KB of space.

Throughout this documentation, the metadata limitation per filesystem is referred to as a parameter named max-files , which describes the number of metadata units, and not the number of files. This parameter encapsulates both the file count and the file sizes, as follows:

  • Each file requires two metadata units.

  • If a file exceeds 0.5 MB, an additional metadata unit is required.

  • For each additional 1 MB over the first MB, an additional metadata unit is required.

The definitions above apply to files residing on SSDs or object stores.

For Example:

  • For a filesystem with potentially 1,000,000,000 files of 64 KB in size, 2,000,000,000 metadata units are required.

  • For a filesystem with potentially 1,000,000 files of 128 MB in size, 130,000,000 metadata units are required.

  • For a filesystem with 1,000,000 files of 750 KB in size, 3,000,000 metadata units are required.

About Object Stores

In the Weka system, object stores represent an optional external storage media, ideal for the storage of warm data. Object stores used in tiered Weka system configurations can be cloud-based, located in the same location, or at a remote location. Weka supports object stores for tiering (tiering and local snapshots) and backup (snapshots only), and both can be used for the same filesystem.

Object stores are optimally used when a cost-effective data storage tier is required at a price point that cannot be satisfied by server-based SSDs. An object store definition contains the object store DNS name, bucket identifier, and access credentials. The bucket must be dedicated to the Weka system and must not be accessible by other applications. However, a single object store bucket may serve different filesystems and multiple Weka systems.

The Weka system supports up to three different object store buckets per filesystem. While object stores can be shared between filesystems, when possible, it is recommended to create and attach a separate object store bucket per filesystem.

About Filesystem Groups

In the Weka system, filesystems are grouped into up to 8 filesystem groups.

Both data at rest (residing on SSD and object store) and data in transit can be encrypted. This is achieved by enabling the filesystem encryption feature. A decision on whether a filesystem is to be encrypted is made when .

For proper security, a KMS (Key Management System) must be used when creating encrypted filesystems. See for more information about KMS support in the Weka system.

In addition to the capacity limitation, each filesystem also has a limitation on the amount of metadata. The system-wide metadata limit is determined by the SSD capacity allocated to the Weka system, as well as the allocated to the Weka system processes. The Weka system will keep tracking of metadata units in RAM, and if it reaches the RAM limit, it will page these metadata tracking units to the SSD (and alert). This leaves enough time for the administrator to increase system resources, as the system keeps serving IOs with a minimal performance impact.

Filesystem connectivity to object stores can be used in both the and features. It is possible to define three object-store buckets for a filesystem, two of them for tiering (out of which only one bucket can be writable) and one for backup only. In such cases, the Weka system will search for relevant data in both the SSD and the readable and writable object stores. This allows a range of use cases, including migration to different object stores, scaling of object store capacity, and increasing the total tiering capacity of filesystems.

Each filesystem group has tiering control parameters (see ). While tiered filesystems have their own object store, the tiering policy will be the same for each tiered filesystem under the same filesystem group.

For information on managing these entities, refer to .

KMS Management
data lifecycle management
Snap to Object
Managing Filesystems, Object Stores and Filesystem Groups
filesystem group
filesystem group
RAM resources
creating the filesystem
Guidelines for Data Storage in Tiered Weka System Configurations