The WEKA Data Platform redefines storage solutions with its software-only approach, compatible with standard AMD or Intel x86-based servers and NVMe SSDs. It eliminates the need for specialized hardware, allowing easy integration of technological advancements without disruptive upgrades. WEKA addresses common storage challenges by removing performance bottlenecks, making it suitable for environments requiring low latency, high performance, and cloud scalability.

Use cases span various sectors, including AI/ML, Life Sciences, Financial Trading, Engineering DevOps, EDA, Media Rendering, HPC, and GPU pipeline acceleration. Combining existing technologies and engineering innovations, WEKA delivers a powerful, unified solution that outperforms traditional storage systems, efficiently supporting various workloads.

WEKA is a fully distributed parallel filesystem (WekaFS™) leveraging NVMe Flash for file services. Integrated tiering seamlessly expands the namespace to and from HDD object storage, simplifying data management. The intuitive GUI allows easy administration of exabytes of data without specialized storage training.

WekaFS stands out with its unique architecture, overcoming legacy systems’ scaling and file-sharing limitations. Supporting POSIX, NFS, SMB, S3, and GPUDirect Storage, it offers a rich enterprise feature set, including snapshots, clones, tiering, cloud-bursting, and more.

Benefits include high performance across all IO profiles, scalable capacity, robust security, hybrid cloud support, private/public cloud backup, and cost-effective flash-disk combination. WekaFS ensures a cloud-like experience, seamlessly transitioning between on-premises and cloud environments.

WekaFS functionality running in its RTOS within the Linux container (LXC) is comprised of the following software components:

• File services (frontend client access): Manages multi-protocol connectivity.

• File system computing and clustering (backend: compute): Manages data distribution, data protection, file system metadata services, and tiering.

• SSD drive agent (backend: drive): Transforms the SSD into an efficient networked device.

By bypassing the kernel, WekaFS achieves faster, lower-latency performance, portable across bare-metal, VM, containerized, and cloud environments. Efficient resource consumption minimizes latency and optimizes CPU usage, offering flexibility in shared or dedicated environments.

WekaFS design departs from traditional NAS solutions, introducing multiple filesystems within a global namespace that share the same physical resources. Each filesystem has its unique identity, allowing customization of snapshot policies, tiering, role-based access control (RBAC), quota management, and more. Unlike other solutions, filesystem capacity adjustments are dynamic, enhancing scalability without disrupting I/O.

The WEKA system offers a robust, distributed, and highly scalable storage solution, allowing multiple application servers to access shared filesystems efficiently and with solid consistency and POSIX compliance.

Related information

The WEKA system offers a converged deployment configuration as an alternative to the standard setup. In this configuration, hundreds of application servers running user applications are equipped with WEKA clients, allowing them to access the WEKA cluster.

Unlike the standard deployment that dedicates specific servers to WEKA backends, the converged setup involves installing a WEKA client on each application server. Additionally, one or more SSDs and backend processes (WekaFS) are integrated into the existing application servers.

The WEKA backend processes function collectively as a single, distributed, and scalable filesystem, leveraging the local SSDs. This filesystem is accessible to the application servers, much like in the standard WEKA system deployment. The critical distinction is that, in this configuration, WEKA backends share the same physical infrastructure as the application servers.

This blend of storage and computing capabilities enhances overall performance and resource usage. However, unlike the standard deployment, where an application server failure does not impact other backends, the converged setup is affected if an application server is rebooted or experiences a failure. The N+2 (or N+4) scheme still protects the cluster and can tolerate two concurrent failures. As a result, converged WEKA deployments require more careful integration and detailed coordination between computational and storage management practices.

Redundancy in WEKA system deployments can vary, ranging from 3+2 to 16+4. Choosing the most suitable configuration involves several key considerations, including redundancy levels, data stripe width, hot spare capacity, and the performance required during data rebuilds.

Redundancy levels

Redundancy can be configured as N+2 or N+4, directly impacting capacity and performance. A redundancy level of 2 is typically sufficient for most configurations, while redundancy levels of 4 are reserved for larger clusters with 100 or more backends or critical data scenarios.

Welcome to the WEKA Documentation Portal, your guide to the latest WEKA version. Whether you're a newcomer or a seasoned user, explore topics from system fundamentals to advanced optimization strategies. Choose your WEKA version from the top menu for version-specific documentation.

Get answers from WEKA documentation with Sevii AI

Sevii AI quickly delivers answers from WEKA documentation. Type your question and click . For the best results, ask clear, context-rich questions.

About WEKA documentation

This portal encompasses all documentation essential for comprehending and operating the WEKA system. It covers a range of topics:

WEKA system overview: Delve into the fundamental components, principles, and entities constituting the WEKA system.

Planning and installation: Discover prerequisites, compatibility details, and installation procedures for WEKA clusters on bare metal, AWS, GCP, and Azure environments.

Getting started with WEKA: Initiate your WEKA journey by learning the basics of managing a WEKA filesystem through the GUI and CLI, executing initial IOs, and exploring the WEKA REST API.

Performance: Explore the results of FIO performance tests on the WEKA filesystem, ensuring optimal system performance.

WEKA filesystems & object stores: Understand the role and management of filesystems, object stores, filesystem groups, and key-management systems within WEKA configurations.

Additional protocols: Learn about the supported protocols—NFS, SMB, and S3—for accessing data stored in a WEKA filesystem.

Operation guide: Navigate through various WEKA system operations, including events, statistics, user management, upgrades, expansion, and more.

Licensing: Gain insights into WEKA system licensing options. How to obtain a classic WEKA license and apply it to the WEKA cluster.

Monitor the WEKA cluster: Deploy the WEKA Management Server (WMS) alongside tools like Local WEKA Home, WEKAmon, and SnapTool to effectively monitor your WEKA cluster.

WEKA support: Find guidance on obtaining support for the WEKA system and effectively managing diagnostics.

Best practice guides: Explore our carefully selected guides, starting with WEKA and Slurm integration, to discover expert-recommended strategies and insights for optimizing your WEKA system and achieving peak performance in various scenarios.

WEKApod: Explore the WEKApod Data Platform Appliance Guide for step-by-step instructions on setting up and configuring the WEKApod™. This turnkey solution, designed for NVIDIA DGX SuperPOD, features pre-configured storage and software for quick deployment and faster value.

Appendices: Explore the Appendices for various topics, including the WEKA CSI Plugin, which connects Kubernetes worker nodes to the WEKA data platform, and other tools and procedures that can enhance your work with WEKA.

Conventions

• The documentation marks the CLI mandatory parameters with an asterisk (*).

• New additions in V4.3 are marked with two asterisks (**) in the and topics.

Documentation feedback

We welcome your feedback to improve our documentation. Include the document version and topic title with your suggestions and email them to . For technical inquiries, contact our . Thank you for helping us maintain high-quality resources.

WEKA provides a ready-to-deploy Terraform package for installing the WEKA cluster on AWS Virtual Private Cloud (VPC).

The following diagram provides an overview of the various steps automated with the Terraform-driven provisioning of the WEKA cluster backend servers on AWS EC2 instances.

Workflow description

Once the Terraform modules are applied, two workflows are running every minute. One for scale-up and the other for scale-down.

WEKA provides a cloud function for scale-up or scale-down of the number of compute engine instances (cluster size). Terraform automatically creates the cluster according to the specified target value in a few minutes.

To change the cluster size (up or down), specify the link to the resize cloud function on GCP and the resize target value for the number of compute engine instances in the following command and run it:

Example:

Related topics

Scale-out the WEKA cluster backend servers

Scale-out is the process of increasing the number of EC2 instances in the system to handle higher workloads or enhance redundancy.

Scale-out is essential to ensure a system can meet growing demands, maintain performance, and distribute workloads effectively. This proactive approach helps prevent overloads, reduce response times, and maintain high availability.

Action

Deployment prerequisites

• Check that your AWS account limits allow for the deployment of your selected configuration. You can check your limits under the Limits tab in the EC2 console.

Auto-scaling is useful to easily scale the number of EC2 instances up or down at need.

After deploying the WEKA cluster via CloudFormation, it is possible to create an auto-scaling group to ease the WEKA cluster size management.

You can create an auto-scaling group for your cluster by running the .

You can control the number of instances by either changing the desired capacity of instances from the AWS auto-scaling group console or defining your custom metrics and scaling policy in AWS. Once the desired capacity has changed, WEKA takes care of safely scaling the instances.

A filesystem group in the WEKA system is used specifically to manage tiering policies for filesystems. It defines key parameters, including the drive retention period and the tiering queue time, which determine how and when data is tiered.

When you add a filesystem, it must be associated with a filesystem group to apply these tiering behaviors. The WEKA system supports up to eight filesystem groups, allowing flexibility in managing tiering policies across different filesystems.

Related topics

The transition from SSD-only filesystem to tiered filesystem

An SSD-only filesystem can be reconfigured as a tiered filesystem by attaching an object store. In such a situation, the default is to maintain the filesystem size. In order to increase the filesystem size, the total capacity field can be modified, while the existing SSD capacity remains the same.

This section explains how data lifecycle is maintained when working with a tiered WEKA system configuration, together with the options for control. The following subjects are covered:

• Advanced explanation of .

• System behavior when .

The management of filesystems is an integral part of the successful running and performance of the WEKA system and overall data lifecycle management.

Related topics

Filesystems, object stores, and filesystem groups

Manage filesystems using the GUI

Manage filesystems using the CLI

The WEKA system offers a range of powerful functionalities designed to enhance data protection, scalability, and efficiency, making it a versatile solution for various storage requirements.

Protection

The WEKA system employs N+2 or N+4 protection, ensuring data protection even in the face of concurrent drive or backend failures. This complex protection scheme is determined during cluster formation and can vary, offering configurations starting from 3+2 up to 16+2 for larger clusters.

Distributed network scheme

The WEKA system incorporates an any-to-any protection scheme that ensures the rapid recovery of data in the event of a backend failure. Unlike traditional storage architectures, where redundancy is often established across backend servers (backends), WEKA's approach leverages groups of datasets to protect one another within the entire cluster of backends.

Here's how it works:

• Data recovery process: If a backend within the cluster experiences a failure, the WEKA system initiates a rebuilding process using all the other operational backends. These healthy backends work collaboratively to recreate the data that originally resided on the failed backend. Importantly, all this occurs in parallel, with multiple backends simultaneously reading and writing data.

• Speed of rebuild: This approach results in a speedy rebuild process. In a traditional storage setup, only a small subset of backends or drives actively participate in rebuilding, often leading to slow recovery. In contrast, in the WEKA system, all but the failed backend are actively involved, ensuring swift recovery and minimal downtime.

• Scalability benefits: The advantages of this distributed network scheme become even more apparent as the cluster size grows. In larger clusters, the rebuild process is further accelerated, making the WEKA system an ideal choice for organizations that need to handle substantial data volumes without sacrificing data availability.

In summary, the WEKA system's distributed network scheme transforms data recovery by involving all available backends in the rebuild process, ensuring speedy and efficient recovery, and this efficiency scales with larger clusters, making it a robust and scalable solution for data storage and protection.

Efficient component replacement

In the WEKA system, a hot spare is configured within the cluster to provide the additional capacity needed for a full recovery after a rebuild across the entire cluster. This differs from traditional approaches, where specific physical components are designated hot spares. For instance, in a 100-backend cluster, sufficient capacity is allocated to rebuild the data and restore full redundancy even after two failures. The system can withstand two additional failures depending on the protection policy and cluster size.

This strategy for replacing failed components does not compromise system reliability. In the event of a system failure, there's no immediate need to physically replace a failed component with a functional one to recreate the data. Instead, data is promptly regenerated, while replacing the failed component with a working one is a background process.

Enhanced fault tolerance with failure domains

In the WEKA system, failure domains are groups of backends that could fail due to a single underlying issue. For instance, if all servers within a rack rely on a single power circuit or connect through a single ToR switch, that entire rack can be considered a failure domain. Imagine a scenario with ten racks, each containing five WEKA backends, resulting in a cluster of 50 backends.

To enhance fault tolerance, you can configure a protection scheme, such as 6+2 protection, during the cluster setup. This makes the WEKA system aware of these possible failure domains and creates a protection stripe across the racks. This means the 6+2 stripe is distributed across different racks, ensuring that the system remains operational even in case of a complete rack failure, preventing data loss.

It's important to note that the stripe width must be less than or equal to the count of failure domains. For instance, if there are ten racks, and one rack represents a single point of failure, having a 16+4 cluster protection is not feasible. Therefore, the level of protection and support for failure domains depends on the stripe width and the chosen protection scheme.

Prioritized data rebuild process

In the event of a failure in the WEKA system, the data recovery process begins by reading all affected data stripes, reconstructing the lost data, and restoring full protection. If multiple failures occur, the affected stripes can be categorized as follows:

• Stripes unaffected by any of the failed components, requiring no action.

• Stripes affected by one of the failed components.

• Stripes affected by multiple failed components.

Typically, the number of stripes affected by multiple failed components is significantly smaller than those affected by a single failed component. However, if any of these multi-affected stripes remain unrecovered, additional failures could lead to data loss.

To mitigate this risk, the WEKA system employs a prioritized rebuild process. The system first restores stripes affected by the greatest number of failed components, depending on the data protection level, as these are fewer in number and can be recovered quickly. Once these high-risk stripes are rebuilt, the system proceeds with restoring stripes impacted by fewer failures. This structured approach ensures that the probability of data loss remains low while maintaining system performance and availability.

Seamless distribution, scaling, and enhanced performance

In the WEKA system, every client installed on an application server directly connects to the relevant WEKA backends that store the required data. There's no intermediary backend that forwards access requests. Each WEKA client maintains a synchronized map, specifying which backend holds specific data types, creating a unified configuration shared by all clients and backends.

When a WEKA client attempts to access a particular file or offset in a file, a cryptographic hash function guides it to the appropriate backend containing the needed data. This unique mechanism enables the WEKA system to achieve linear performance growth. It synchronizes scaling size with scaling performance, providing remarkable efficiency.

For instance, when new backends are added to double the cluster's size, the system instantly redistributes part of the filesystem data between the backends, resulting in an immediate double performance increase. Complete data redistribution is unnecessary even in modest cluster growths, such as moving from 100 to 110 backends. Only a fraction (10% in this example) of the existing data is copied to the new backends, ensuring a balanced distribution and active participation of all backends in read operations.

The speed of these seamless operations depends on the capacity of the backends and network bandwidth. Importantly, ongoing operations remain unaffected, and the system's performance improves as data redistribution occurs. The finalization of the redistribution process optimizes both capacity and performance, making the WEKA system an ideal choice for scalable and high-performance storage solutions.

Efficient data reduction

WEKA offers a cluster-wide data reduction feature that can be activated for individual filesystems. This capability employs block-variable differential compression and advanced de-duplication techniques across all filesystems to significantly reduce the storage capacity required for user data, resulting in substantial cost savings for customers.

The effectiveness of the compression ratio depends on the specific workload. It is particularly efficient when applied to text-based data, large-scale unstructured datasets, log analysis, databases, code repositories, and sensor data. For more information, refer to the dedicated topic.

Many hardware vendors ship their products with the SR-IOV feature disabled. The feature must be enabled on such platforms before installing the Weka system. Enabling the SR-IOV applies to the server BIOS.

If the SR-IOV is already enabled, it is recommended to verify the current state before proceeding with the installation of the Weka system.

Before you begin

Verify that the NIC drivers are installed and loaded successfully. If it still needs to be done, perform the Install NIC drivers procedure.

Enable SR-IOV in the server BIOS

The following procedure is a vendor-specific example and is provided as a courtesy. Depending on the vendor, the same settings may appear differently or be located elsewhere. Therefore, refer to your hardware platform and NIC vendor documentation for the latest information and updates.

The WEKA project uses internal GCP resources. A basic WEKA project includes a cluster with multiple virtual private clouds (VPCs), virtual machines (VMs), a load balancer, DNS, cloud storage, a secret manager, and other components for managing cluster resizing. Peering between all virtual networks enables functions to run across them, with each VPC connected to every other VPC in a full mesh.

Resize cloud function operation

A resize cloud function in vpc-0 and a workload listener are deployed for auto-scale instances in GCP. Once a user sends a request for resizing the number of instances in the cluster, the workload listener checks the cluster state file in the cloud storage and triggers the resize cloud function if a resize is required. The cluster state file is an essential part of the resizing decision. It indicates states such as:

• Readiness of the cluster.

• The number of existing instances.

• The number of requested instances.

The secret manager retains the user name (usually admin) and the Terraform-generated password. The resize cloud function uses the user name and password to operate on the cluster instances.

GCP internet connectivity

During WEKA deployment, the instances require access to a YUM repository and the WEKA software. This can be achieved through one of the following methods:

• External IPs: Allow an external IP address in VPC0 for each instance. This enables direct connection to public internet resources such as a YUM repository and get.weka.io.

• NAT gateway: Add a NAT Gateway to VPC0 to allow the instances to connect to the public internet resources through the gateway.

• Private YUM repository: If connecting to an external YUM repository and get.weka.io is impossible, specify an internal YUM repository and a private download link for the WEKA software.

Overview

AWS Outposts is a fully managed service that extends AWS infrastructure, AWS services, APIs, and tools to virtually any data center, co-location space, or on-premises facility for a consistent hybrid experience. AWS Outposts is ideal for workloads that require low latency access to on-premises systems, local data processing, or local data storage.

Deployment of a WEKA cluster in AWS Outposts

A WEKA cluster deployment in AWS Outposts follows the guidelines specified in the section.

To deploy a WEKA cluster in AWS Outposts, use a CloudFormation template, which can be obtained as specified in the section.

This template can be customized. For further assistance, contact the .

Object stores in WEKA are optional external storage media, complementing SSD storage with a more cost-effective solution. This allows for the strategic allocation of resources, with object stores accommodating warm data (infrequently accessed) and SSDs handling hot data (frequently accessed).

In WEKA, object store buckets can be distributed across different physical object stores. However, to ensure optimal Quality of Service (QoS), a crucial mapping between the bucket and the physical object store is required.

WEKA treats object stores as physical entities, either on-premises or in the cloud, grouping multiple object store buckets. These buckets can be categorized as either local (used for tiering and snapshots) or remote (exclusively for snapshots). An object-store bucket must be added to an object store with the same type and remain inaccessible to other applications.

While a single object store bucket can potentially serve different filesystems and multiple WEKA systems, it is advisable to dedicate each bucket to a specific filesystem. For instance, if managing three-tiered file systems, assigning a dedicated local object storage bucket to each file system is recommended.

For each filesystem, users can attach up to three object store buckets:

• A local object store bucket for tiering and snapshots.

• A second local object store bucket for additional tiering and snapshots. Note that adding a second local bucket renders the first local bucket read-only.

• A remote object store bucket exclusively for snapshots.

Multiple object store buckets offer flexibility for various use cases, including:

• Migrating to different local object stores when detaching a read-only bucket from a filesystem tiered to two local object store buckets.

• Scaling object store capacity.

• Increasing total tiering capacity for filesystems.

• Backing up data in a remote site.

In cloud environments, users can employ cloud lifecycle policies to transition storage tiers or classes. For example, in AWS, users can move objects from the S3 standard storage class to the S3 intelligent tiering storage class for long-term retention using the AWS lifecycle policy.

Related topics

The GCP Console has a Logs Explorer interface in which you can view the cloud function logs related to the WEKA cluster activities, such as when scaling instances up or down. In addition, the cluster state file retained in the cloud storage provides you with the status of the operations in the WEKA project.

Typical troubleshooting flow if the resize cloud function does not resize the cluster

1. Open the cluster state file and check that the desired_size is as expected and the clusterized value is true. The cluster state file is in the cloud storage, and its name comprises the prefix and cluster_name provided in the .

2. Check the scale-up workflow (or scale-down workflow). Check the function that didn't work and its related logs in the Logs Explorer of the GCP Console.

WEKA supports client instances with at least two NICs, one for management and one for the frontend data. It is possible to add more NICs for redundancy and higher performance.

A client with the same VPC networks and subnets as the cluster can connect without additional configuration. If a client is on another VPC network, peering is required between the VPC networks.

The client instance must be in the same region as the WEKA cluster on GCP.

Mount the filesystem

Snapshots allow the saving of a filesystem state to a .snapshots directory under the root filesystem. They can be used for:

• Physical backup: The snapshots directory can be copied into a different storage system, possibly on another site, using either the WEKA system Snap-To-Object feature or third-party software.

• Logical backup: Periodic snapshots enable filesystem restoration to a previous state if logical data corruption occurs.

Overview

The WEKA system offers multiple layers where you can limit capacity usage:

• Organization level: You can monitor an organization’s usage (SSD and total) and restrict usage with quotas per organization. This feature can be used for charge-backs based on the capacity used or allocated by SSD or object store data. For more details, see .

Cluster architecture basics

In the WEKA system, servers operate as members of a cluster, with each server hosting multiple containers. These containers run software instances, referred to as processes, that collaborate and communicate within the cluster to deliver robust and efficient storage services. This architecture ensures scalability and fault tolerance by distributing storage functionality across interconnected containers.

WEKA on GCP overview

Leveraging GCP's advantages, WEKA offers a customizable terraform-gcp-weka module for deploying the WEKA cluster on GCP. In GCP, WEKA operates on instances, each capable of using up to eight partitions of drives on the connected physical server (without direct drive usage). These drives can be shared among partitions for other clients on the same server.

WEKA requires either four or seven VPC networks, depending on which instance type is being used for the WEKA backends. This configuration aligns with the four key WEKA processes: Management, Drive, Compute and optionally Frontend, with each process requiring a dedicated network interface as follows:

Using the GUI, you can:

• Attach object store bucket to a filesystem

• Detach object store bucket from a filesystem

Attach object store bucket to a filesystem

Before you begin

Verify that an object store bucket is available.

Procedure

1. From the menu, select Manage > Filesystems.

2. On the Filesystem page, select the three dots on the right of the filesystem that you want to attach to the object store bucket. Then, from the menu, select Attach Object Store Bucket.

3. On the Attach Object Store Bucket dialog, select the relevant object store bucket.

Detach object store bucket from a filesystem

Detaching a local object store bucket from a filesystem migrates the filesystem data residing in the object store bucket either to the writable object store bucket (if one exists) or to the SSD.

Procedure

1. From the menu, select Manage > Filesystems.

2. On the Filesystem page, select the filesystem from which you want to detach the object store bucket.

3. From the Detach Object Store Bucket dialog, select Detach. If the filesystem is attached to two object store buckets (one is read-only, and the other is writable), you can detach only the read-only one. The data of the detached object store bucket is migrated to the writable object store bucket.

1. If the filesystem is tiered and only one object store is attached, detaching the object store bucket opens the following message:

1. Object store buckets usually expand the filesystem capacity. Un-tiering of a filesystem requires adjustment of its total capacity. Select one of the following options:

   • Increase the SSD capacity to match the current total capacity.

   • Reduce the total filesystem capacity to match the SSD or used capacity (the decrease option depends on the used capacity).

1. Select the option that best meets your needs, and select Continue.

2. In the message that appears, select Detach to confirm the action.

The Terraform-AWS-WEKA module is an open-source repository. It contains modules to customize the WEKA cluster installation on AWS. The default protocol deployed using the module is POSIX.

The Terraform-AWS-WEKA module supports public and private cloud deployments. All deployment types require passing the get.weka.io token to Terraform to download the WEKA release from the public get.weka.io service.

Terraform-AWS-WEKA module components

The Terraform-AWS-WEKA module consists of the following components:

• Required module:

  • WEKA Root Module is located in the main Terraform module.

• Optional sub-modules:

Terraform-AWS-WEKA example

The following is a basic example in which you provide the minimum detail of your cluster, and the Terraform module completes the remaining required resources, such as VPC, subnets, security group, placement group, DNS zone, and IAM roles.

You can use this example as a reference to create the main.tf file.

Using the CLI, you can:

• Attach an object store bucket to a filesystem

• Detach an object store bucket from a filesystem

Attach an object store bucket to a filesystem

Command: weka fs tier s3 attach

To attach an object store to a filesystem, use the following command:

weka fs tier s3 attach <fs-name> <obs-name> [--mode mode]

Parameters

Detach an object store bucket from a filesystem

Command: weka fs tier s3 detach

To detach an object store from a filesystem, use the following command:

weka fs tier s3 detach <fs-name> <obs-name>

Parameters

WEKA provides a variety of tools for automating the WEKA software installation process. These include:

• WEKA Management Station (WMS)

• WEKA Software Appliance (WSA)

• WEKA Configurator

Using CloudFormation deployment saves a lot of potential errors that may occur during installation, especially in the configuration of security groups and other connectivity-related issues. However, the following errors related to the following subjects may occur during installation:

• Installation logs

• AWS account limits

Before a WEKA system can use a Broadcom adapters, the server must have the necessary drivers and firmware from Broadcom's download center.

Set up the drivers and software

Procedure:

1. Download software bundle: Access Broadcom's download center and download the software bundle onto the target server. Carefully review the instructions included in the bundle.

Create a main.tf file

The main Terraform configuration settings are included in the main.tf file. You can create it by following this procedure or using the WEKA Cloud Deployment Manager. See .

Required services

The AWS region must support the following services used in WEKA on AWS.

• AWS EC2 Instances

Attachment of a local object store bucket to a filesystem

Two local object store buckets can be attached to a filesystem, but only one of the buckets is writable. A local object store bucket is used for both tiering and snapshots. When attaching a new local object store bucket to an already tiered filesystem, the existing local object store bucket becomes read-only, and the new object store bucket is read/write. Multiple local object stores allow a range of use cases, including migration to different object stores, scaling of object store capacity, and increasing the total tiering capacity of filesystems.

When attaching a local object store bucket to a non-tiered filesystem, the filesystem becomes tiered.

Register to get.weka.io

To sign in to , you first need to create an account and fill in your details. If you already have a registered account for get.weka.io, skip this procedure.

Procedure

The WEKA client

The WEKA client is a standard POSIX-compliant filesystem driver installed on application servers, facilitating file access to WEKA filesystems. Acting as a conventional filesystem driver, it intercepts and executes all filesystem operations, providing applications with local filesystem semantics and performance—distinct from NFS mounts. This approach ensures centrally managed, shareable, and resilient storage for WEKA.

Tightly integrated with the Linux Page Cache, the WEKA client leverages this transparent caching mechanism to store portions of filesystem content in the client's RAM. The Linux operating system maintains a page cache in the unused RAM, allowing rapid access to cached pages and yielding overall performance enhancements.

The Linux Page Cache, implemented in the Linux kernel, operates transparently to applications. Utilizing unused RAM capacity, it incurs minimal performance penalties, often appearing as "free" or "available" memory.

AWS

AWS cluster

Raw capacity

Raw capacity is the total capacity of all SSDs assigned to a WEKA system cluster. For example, 10 SSDs of one terabyte each provide a total raw capacity of 10 terabytes. This represents the total capacity available for the WEKA system. This capacity automatically adjusts when more servers or SSDs are added.

Net capacity

Net capacity is the space available for user data on the SSDs in a configured WEKA system. It is derived from the raw capacity minus the WEKA filesystem overheads for redundancy protection and other requirements. This capacity automatically adjusts when more servers or SSDs are added.

Stripe width

The stripe width is the number of blocks within a common protection set, ranging from 3 to 16. The WEKA system employs distributed any-to-any protection. In a system with a stripe width of 8, many groups of 8 data units spread across various servers protect each other, rather than a fixed group of 8 servers forming a protection group.

The stripe width is set during cluster formation and cannot be changed. The choice of stripe width impacts performance and net capacity.

If not configured, the stripe width is automatically set to: #Failure Domains - Protection Level -1.

Protection level

Protection level refers to the number of extra protection blocks added to each data stripe in your storage system. These blocks help protect your data against hardware failures. The protection levels available are:

• Protection level 2: Can survive 2 concurrent disk or server failures.

• Protection level 4: Can survive 4 concurrent disk failures or 2 concurrent server failures.

A higher protection level means better data durability and availability but requires more storage space and can affect performance.

Key points:

• Durability:

  • Higher protection levels offer better data protection.

  • Level 4 is more durable than level 2.

• Availability:

Resilience to serial failures

Beyond the +2 or +4 concurrent failure protection, a WEKA cluster is also resilient to serial failures of additional failure domains. Providing that each data rebuild completes successfully, and there is sufficient free NVMe capacity in the cluster. A cluster is resilient to an additional server failure, even if the failure would reduce the number of available servers beyond what is expected by the concurrent protection level.

Example of failure resilience after rebuild completion

Consider a cluster of 20 servers with a stripe width of 18 (16+2). After rebuilding from a concurrent failure of 2 servers, the cluster still resilient to two additional concurrent server failures.

In the event of subsequent server failures, the cluster rebuilds with the remaining healthy servers to support a stripe width of 18. If further serial server failures occur, the system rebuilds its data stripes as those individual servers fail, subject to sufficient NVMe space, until the lower limit of 9 servers is reached (in this case). Failures beyond this point result in the filesystem going offline.

In the event of serial server failures and insufficient NVMe capacity, the cluster attempts to tier data that currently resides in NVMe out to its object stores if configured. In contrast to the usual age-related orderly tiering that occurs in normal usage, this backpressure mode does not consider data's age when making tiering decisions, and instead will tier data in an approximately random fashion. This is not the desired mode of operation, but it ensures data integrity in the event of continually-decreasing NVMe capacity by offloading data to an object store.

Resilience level and minimum required healthy servers

The stripe width and protection level determine the minimum number of required healthy servers. This can be represented by the following formula:

Where:

• D is the data blocks in the stripe.

• P is the protection blocks in the stripe.

• H is the minimum number of healthy servers.

The following are a few examples:

Failure domains (optional)

A failure domain is a set of WEKA servers susceptible to simultaneous failure due to a single root cause, such as a power circuit or network switch malfunction.

A cluster can be configured with either explicit or implicit failure domains:

• Explicit failure domains: In this setup, blocks that offer mutual protection are distributed across distinct failure domains.

• Implicit failure domains: Here, blocks are distributed across multiple servers, with each server considered a separate failure domain. Additional failure domains and servers can be integrated into existing or new failure domains.

Hot spare

A hot spare is reserved capacity designed to handle data rebuilds while maintaining the system’s net capacity, even in the event of failure domains being lost. It represents the number of failure domains the system can afford to lose and still perform a complete data rebuild successfully.

All failure domains actively contribute to data storage, and the hot spare capacity is evenly distributed among them. While a higher hot spare count requires additional hardware to maintain the same net capacity, it provides greater flexibility for IT maintenance and hardware replacements.

WEKA filesystem overhead

After accounting for protection and hot spare capacity, only 90% of the remaining capacity is available as net user capacity, with the other 10% reserved for the WEKA filesystems. This is a fixed formula and cannot be configured.

Provisioned capacity

Provisioned capacity is the total capacity assigned to filesystems, including both SSD and object store capacity.

Available capacity

Available capacity is the total capacity used to allocate new filesystems, calculated as net capacity minus provisioned capacity.

Deductions from raw capacity to obtain net storage capacity

The net capacity of the WEKA system is determined by making the following three deductions during configuration:

• Protection level: Storage capacity dedicated to system protection.

• Hot spare(s): Storage capacity reserved for redundancy and rebuilding following component failures.

• WEKA filesystem overhead: Storage capacity allocated to enhance overall performance.

SSD net storage capacity calculation

Examples:

The WEKA system provides a RESTful API, enabling you to automate interactions with the WEKA system and integrate them into your workflows or monitoring systems.

Access the REST API

You can access the REST API using one of the following methods:

Explore the REST API through the GUI

Obtain an access token

To use the WEKA REST API, provide an access or refresh token.

You can generate an access or refresh for the REST API usage through the CLI or the GUI. See .

You can also call the login API to obtain access or refresh tokens through the API, providing it with a username and password.

If you already obtained a refresh token, you can use the login/refresh API to refresh the access token.

The response includes the access token (valid for 5 minutes) to use in the other APIs requiring token authentication, along with the refresh token (valid for 1 year), for getting additional access tokens without using the username/password.

Call the REST API

Once you obtain an access token, you can call WEKA REST API commands with it. For example, you can query the cluster status:

Related topics

Using the CLI, you can perform the following actions:

• View filesystem groups

• Add filesystem groups

• Edit filesystem groups

View filesystem groups

Command: weka fs group

Use this command to view information on the filesystem groups in the WEKA system.

Add a filesystem group

Command: weka fs group create

Use the following command to add a filesystem group:

weka fs group create <name> [--target-ssd-retention=<target-ssd-retention>] [--start-demote=<start-demote>]

Parameters

Edit a filesystem group

Command: weka fs group update

Use the following command to edit a filesystem group:

weka fs group update <name> [--new-name=<new-name>] [--target-ssd-retention=<target-ssd-retention>] [--start-demote=<start-demote>]

Parameters

Delete a filesystem group

Command: weka fs group delete

Use the following command line to delete a filesystem group:

weka fs group delete <name>

Parameters

Related topics

To learn about the tiring policy, see:

Supported machine types for backends

The following table provides the supported machine types (VM instance) for backends (and clients) applied by the Terraform package:

Supported machine types for clients

Explore the two key technologies in network virtualization: VirtIO in DPDK mode and gVNIC in UDP mode. VirtIO in DPDK mode offers high-performance network interfaces in virtual machines, while gVNIC in UDP mode provides reliable, high-speed network connectivity.

Supported machine types for clients over VirtIO in DPDK mode

Supported machine types for clients over gVNIC in UDP mode

Related information

Directory quotas monitor a directory's filesystem capacity usage and allow restricting the amount of space used by the directory.

Using the GUI, you can:

• Set directory quota

• View directory quotas and default quota

Set directory quota

The organization admin can set a quota on a directory. This action initiates calculating the current directory usage in a background task. Once this calculation is complete, the quota is considered.

Before you begin

To set a quota on a directory, a mount point to the relevant filesystem is necessary. The quota set command mustn’t be interrupted until the quota accounting process is finished.

Procedure

1. From the menu, select Manage > Directory Quotas.

2. Select Directory Quotas.

3. Select the filesystem in which you want to set the directory quotas.

4. In the Create Quota dialog, set the following:

View directory quotas and default quota

You can view existing directory quotas and the default quota that are already set.

Procedure

1. From the menu, select Manage > Directory Quotas.

2. Select the relevant tab: Directory Quotas or Default Directories Quota.

3. Select the filesystem in which the directory quotas are already set.

Update a directory quota or default quota

You can update an existing directory quota or the default quota for directories. Updating the default quota only applies to new directories.

Procedure

1. From the menu, select Manage > Directory Quotas.

2. Select the relevant tab: Directory Quotas or Default Directories Quota.

3. Select the filesystem in which the directory quotas are set (through the CLI).

1. In the Quota Settings Update dialog, modify the following settings according to your needs:

   • Hard Quota Limit: The hard quota limit defines the maximum used capacity above the soft quota limit, which prevents writing to the directory.

   • Soft Quota Limit: The soft quota limit defines the maximum used capacity that triggers a grace period timer. Data can be written to the directory until the grace period ends or the hard quota limit is reached.

Remove a directory quota

You can remove (unset) a directory quota if it is no longer required.

Procedure

1. From the menu, select Manage > Directory Quotas.

2. Select the Directory Quotas tab.

3. Select the filesystem in which the directory quota is set.

4. Select the three dots on the right of the required default quota. From the menu, select Remove.

Remove the default quota for new directories

You can remove (unset) the default quota settings for new directories created in a specific filesystem. The quota of existing directories is not affected.

Procedure

1. From the menu, select Manage > Directory Quotas.

2. Select the Default Directories Quota tab.

3. Select the filesystem in which the default quotas are already set (through the CLI).

4. Select the three dots on the right of the required default quota. From the menu, select Remove.

Overview

Mounting filesystems from a single WEKA client to multiple clusters provides several advantages:

• Expanded cluster connectivity: A single client can connect to up to seven clusters simultaneously, increasing storage capacity and computational capabilities.

• Unified data access: Provides a consolidated view of data across multiple clusters, simplifying access and management while improving data availability, flexibility, and resource efficiency.

• Optimized workload distribution: Enables efficient workload distribution across clusters, supporting scalable applications and enhancing overall performance.

• Seamless integration: WEKA’s SCMC feature ensures smooth and efficient integration for clients accessing multiple clusters.

Bandwidth division considerations in SCMC

The bandwidth division in SCMC is a universal consideration based on the specific NIC's bandwidth. It applies across various NIC types, including those using DPDK or specific models like the X500-T1.

During SCMC mounts, each active connection can use the bandwidth available on its associated NIC port. This is true during peak usage and idle cases. In scenarios where NICs are dual-ported, each connection operates independently, leveraging its dedicated port.

When working with low-bandwidth NICs such as the X500-T1, a 10Gb/s NIC, consider bandwidth calculations. In the context of SCMC, each container (representing connectivity to a different cluster) uses half of the available bandwidth (5Gb/s) for a shared port. Note that a dual-port NIC has a dedicated port for each container, optimizing bandwidth distribution. Keep these factors in mind for an optimal SCMC setup.

When a stateless client mounts a filesystem in a cluster, it creates a client container with the same version as provided by the cluster. Because there may be situations where some of the clusters run a different WEKA version than the others, such as during an upgrade, it is required to set the same client target version on all clusters. The client target version is retained regardless of the cluster upgrade.

Procedure:

1. Connect to each cluster and run the following command to set the client target version.

Where: <version> is the designated client target version, which will be installed on the client container upon the mount command. Ensure this version is installed on the backend servers.

1. To display the existing client target version in the cluster, run the following command:

1. To reset the client target version to the cluster version, run the following command:

Mount a stateless client container on multiple clusters

Use the same commands as with a single client.

To mount a stateless client using UDP mode, add -o net=udp -o core=<core-id> to the command line. For example:

Mount persistent client containers on multiple clusters

For persistent client containers, the client-target-version parameter is not relevant. The version of the client container is determined when creating the container in the WEKA client using the weka local setup container command. Therefore, ensure that all client containers in the WEKA client have the same minor version as in the clusters.

To mount a persistent client container to a cluster, specify the container name for that mount.

Run commands from a server with multiple client containers

When running WEKA CLI commands from a server hosting multiple client containers, each connected to a different WEKA cluster, it’s required to specify the client container port or the backend IP address/name of the cluster (linked to that client) in the command.

Consider a server with two client containers:

To run a WEKA CLI command on the second cluster (associated with client2), use either of the following methods:

• By specifying the backend IP address or name linked to that client container (assuming the backend name is DataSphere2-1):

• By specifying the client container port:

This approach ensures that your WEKA CLI command targets the correct WEKA cluster associated with the specified client container.

Related topics

The WEKA® Data Platform on AWS provides a fast and scalable platform for running performance-intensive applications and hybrid cloud workflows.

WEKA provides a ready-to-deploy Terraform package that you can customize for installing the WEKA cluster on AWS. Optionally, you can install the WEKA cluster using the AWS CloudFormation.

Ensure you are familiar with the following concepts and services that are used for the WEKA installation on AWS:

WEKA provides a ready-to-deploy you can customize to install the WEKA cluster on GCP.

The Terraform package contains the following modules:

• setup_network: includes vpcs, subnets, peering, firewall, and health check.

• service_account: includes the service account used for deployment with all necessary permissions.

During the deployment of the WEKA system, the EC2 instances require access to the internet to download the WEKA software. For this reason, you need to deploy the WEKA system in one of the following deployment types in AWS:

• Public subnet: Use a public subnet within your VPC with an internet gateway, and allow public IP addresses for your instances.

• Private subnet with NAT Gateway: Create a private subnet with a route to a NAT gateway with an elastic IP in the public subnet.

Media options for data storage in the WEKA system

In the WEKA system, data can be stored on two forms of media:

• On locally-attached SSDs, which are an integral and required part of the WEKA system configuration.

Storage EC2 instances

The following EC2 instance types can operate as backend, client, or converged instances:

If you are not using the WMS or WSA automated tools for installing a WEKA cluster, manually install a supported OS and the WEKA software on the bare metal server.

Once the system is installed and you are familiar with the CLI and GUI, you can connect to one of the servers and try it out.

To perform a sanity check that the WEKA cluster is configured and IOs can be performed on it, do the following procedures:

1. .

2. .

Using the GUI, you can perform the following actions:

WSA is a package consisting of a base version of Linux (based on Rocky 8.6), network drivers and other required packages, WEKA software, and various diagnostic and configuration tools. Using the WSA facilitates the post-installation administration, security, and other KB updates controlled and distributed by WEKA, following a Long Term Support (LTS) plan.

The WSA generally works like any OS install disk (Linux/Windows).