Links

Mount filesystems from multiple clusters on a single client

Connect a single WEKA client to multiple clusters simultaneously, optimizing data access and workload distribution.

Overview

Mounting filesystems from multiple clusters on a single WEKA client, known as Single Client Multiple Clusters (SCMC), provides several benefits:
  • Expanded cluster connectivity: Connect a single client to a maximum of 7 clusters simultaneously, enhancing storage capacity and computational capabilities.
  • Streamlined data access: Gain a unified view of data across multiple clusters for simplified access and management, improving data availability, flexibility, and efficient resource usage.
  • Efficient workload distribution: Distribute workloads efficiently across multiple clusters, enabling scalable applications and optimized performance.
  • Seamless integration: WEKA ensures a seamless integration for clients leveraging the SCMC feature.
Mount filesystems from multiple clusters on a single client

Bandwidth division considerations in SCMC

The bandwidth division in SCMC is a universal consideration based on the specific NIC's bandwidth. It applies across various NIC types, including those using DPDK or specific models like the X500-T1.
During SCMC mounts, each active connection can use the bandwidth available on its associated NIC port. This is true during peak usage and idle cases. In scenarios where NICs are dual-ported, each connection operates independently, leveraging its dedicated port.
When working with low-bandwidth NICs such as the X500-T1, a 10Gb/s NIC, consider bandwidth calculations. In the context of SCMC, each container (representing connectivity to a different cluster) uses half of the available bandwidth (5Gb/s) for a shared port. Note that a dual-port NIC has a dedicated port for each container, optimizing bandwidth distribution. Keep these factors in mind for an optimal SCMC setup.

Prerequisites

Ensure the following requirements are met:
  • All clusters that run in this configuration must be at least version 4.2.
  • All client containers in the WEKA client must run the same minor version, at least version 4.2. The client version must be the same as the cluster or one version earlier at most.
  • All client containers must be of the same type, stateful or stateless clients. Mixing different client types in a single WEKA client is not allowed.
  • Each client container must run on its port. The default ports are 14000, 14101, 14201, 14301, 14401, 14501, and 14601. Ensure these ports allow egress on the client and ingress on the cluster.
  • For DPDK, each client container must have 5 GB of free RAM, and it is recommended to have a dedicated CPU core to get optimal performance.
Mounting a filesystem without these requirements may fail or overload the WEKA client.
  • Mounting filesystems from multiple clusters on a single client with Intel E810 are only supported using UDP mode.
  • Mounting a stateful client using autofs is only supported on filesystems on a single cluster.

Set the client target version in the clusters

When a stateless client mounts a filesystem in a cluster, it creates a client container with the same version as provided by the cluster. Because there may be situations where some of the clusters run a different WEKA version than the others, such as during an upgrade, it is required to set the same client target version on all clusters. The client target version is retained regardless of the cluster upgrade.
The client target version must be the same in all clusters. It can be the same as in the cluster or one major version earlier and available in the cluster for download by the client. If you need to upgrade the cluster to a higher version than the one above the client version, you must update the client-target-version first in all clusters, upgrade the clients, and only then upgrade the clusters.

Procedure:

  1. 1.
    Connect to each cluster and run the following command to set the client target version.
weka cluster client-target-version set <version-string>
Where: <version-string> is the designated client target version, which will be installed on the client container upon the mount command. Ensure this version is installed on the backend servers.
  1. 2.
    To display the existing client target version in the cluster, run the following command:
weka cluster client-target-version show
  1. 3.
    To reset the client target version to the cluster version, run the following command:
weka cluster client-target-version reset

Mount stateless client containers on multiple clusters

Use the same commands as with a single client.
mount -t wekafs <backend-name><fs-name> <mount-point>
To mount a stateless client using UDP mode, add -o net=udp -o core=<core-id> to the command line. For example:
mount -t wekafs backend-server-0/my_fs /mnt/weka -o net=udp -o core=2

Mount stateful client containers on multiple clusters

For stateful client containers, the client-target-version parameter is not relevant. The version of the client container is determined when creating the container in the WEKA client using the weka local setup container command. Therefore, ensure that all client containers in the WEKA client have the same minor version as in the clusters.
To mount a stateful client container to a cluster, specify the container name for that mount.
mount -t wekafs <fs-name> <mount-point> -o container_name=<container-name>

Mount with a specific port (not default)

If the cluster does not listen to the default port, add WEKA_PORT=<port number> before the mount command:
WEKA_PORT=<port number> mount -t wekafs <fs-name> <mount-point> -o container_name=<container-name>

Run commands from a server with multiple client containers

When running WEKA CLI commands against one of the connected WEKA clusters from a server with multiple client containers, it's necessary to specify the client container port in the command. Here's an example of a server with two client containers:
weka local ps
CONTAINER STATE DISABLED UPTIME MONITORING PERSISTENT PORT PID STATUS VERSION LAST FAILURE
client Running False 3:15:57h True False 14000 58318 Ready 4.2.6.36-663f57024276b0b6aa17036a39835d1d
client2 Running False 3:14:35h True False 14101 59529 Ready 4.2.6.36-663f57024276b0b6aa17036a39835d1d
To run a WEKA CLI command against the second cluster (client2 in this case), specify the port for that client container like this:
weka status -P 14101
This ensures that your WEKA CLI command is directed to the correct WEKA cluster associated with the specified client container.