Weka cluster installation
This page provides a detailed workflow for WEKA cluster installation with multiple containers using the CLI. These are complementary details for the quick installation guide.
Workflow
1. Install the WEKA software
Once the WEKA software is downloaded from get.weka.io, run the untar command and install.sh command on each server, according to the instructions in the Install tab.
Once completed, the WEKA software is installed on all the allocated servers and runs in stem mode (no cluster is attached).
Note: If a failure occurs during the WEKA software installation process, an error message prompts detailing the source of the failure. Review the details and try to resolve the failure. If required, contact the Customer Success Team.
2. Remove the default container
Command: weka local stop default && weka local rm -f default
Stop and remove the auto-created default container created on each server.
3. Generate the resource files
Command: resources_generator.py
To generate the resource files for the drive, compute, and frontend processes, download the resource_generator.py and run the following command on each backend server:
./resources_generator.py --net <net-devices> [options]
The resource generator allocates the number of cores, memory, and other resources according to the values specified in the parameters.
The best practice for resources allocation is as follows:
1 drive core per NVMe device (SSD).
2-3 compute cores per drive core.
1-2 frontend cores if deploying a protocol container. If there is a spare core, it is used for a frontend container.
Minimum of 1 core for the OS.
Example 1: according to the best practice
For a server with 24 cores and 6 SSDs, allocate 6 drive cores and 12 compute cores, and optionally you can use 2 cores of the remaining cores for the frontend container. The OS uses the remaining 4 cores.
Run the following command line:
./resources_generator.py --net eth1 --net eth2 --drive-dedicated-cores 6 --compute-dedicated-cores 12 --frontend-dedicated-cores 2
Example 2: a server with a limited number of cores
For a server with 14 cores and 6 SSDs, allocate 6 drive cores and 6 compute cores, and optionally you can use 1 core of the remaining cores for the frontend container. The OS uses the remaining 1 core.
Run the following command line:
./resources_generator.py --net eth1 --net eth2 --drive-dedicated-cores 6 --compute-dedicated-cores 6 --frontend-dedicated-cores 1
Note: Contact Professional Services for the recommended resource allocation settings for your system.
Parameters
Name | Type | Value | Mandatory | Default |
| Space-separated numbers | Specify the CPUs to allocate for the compute processes. | No | - |
| Number | Specify the number of cores to dedicate for the compute processes. | No | The maximum available cores |
| String | Specify the total memory to allocate for the compute processes. Argument format: value and unit without a space. Examples: 1024B, 10GiB, 5TiB. | No | The maximum available memory |
| Space-separated numbers | Specify the CPUs to allocate for the WEKA processes. | No | - |
| Space-separated numbers | Specify the CPUs to allocate for the drive processes. | No | - |
| Number | Specify the number of cores to dedicate for the drive processes. | No | 1 core per each detected drive |
| Space-separated strings | Specify the drives to use. This option overrides automatic detection. | No | All unmounted NVME devices |
| Space-separated numbers | Specify the CPUs to allocate for the frontend processes. | No | - |
| Number | Specify the number of cores to dedicate for the frontend processes. | No | 1 |
| Number | Override the default maximum number of cores per container for IO processes (19). If provided, the new value must be lower. | No | 19 |
| Flag | Set each container's hugepages memory to 1.4 GiB * number of IO processes on the container. | No | - |
| Space-separated strings | Specify the network devices to use. | Yes | - |
| Boolean | Don't take RDMA support into account when computing memory requirements. | No | False |
| Number | Override the auto-deduction of the number of cores. | No | All available cores |
| String | Specify the path to write the resource files. | No | The default is '.' |
| Number | Specify the number of cores to leave for OS and non-WEKA processes. | No | 1 |
| String | Specify the memory to reserve for non-WEKA requirements. Argument format: a value and unit without a space. Examples: 10GiB, 1024B, 5TiB. | No | The maximum between 8 GiB and 2% of the total RAM |
| String | Specify the memory to allocate for compute, frontend, and drive processes. Argument format: a value and unit without a space. Examples: 10GiB, 1024B, 5TiB. | No | The maximum available memory |
4. Create drive containers
Command: weka local setup container
For each server in the cluster, create the drive containers using the resource generator output file drives0.json.
The drives JSON file includes all the required values for creating the drive containers. Therefore, it is not required to set all the options of the weka local setup container command. Only the path to the JSON resource file is required.
Parameters
Name | Type | Value | Limitations | Mandatory | Default |
| String | The path to the resource file. | Must be a valid path | Yes |
5. Create a cluster
Command: weka cluster create
To create a cluster of the allocated containers, use the following command:
Parameters
Name | Type | Value | Limitations | Mandatory | Default |
| Space-separated strings | Hostnames or IP addresses. If port 14000 is not the default for the drives, you can specify hostnames:port or ips:port. | Need at least 6 strings, as this is the minimal cluster size | Yes | |
| Comma-separated IP addresses | IP addresses of the management interfaces. Use a list of | The same number of values as in | No | IP of the first network device of the container |
Note: It is possible to use either a hostname or an IP address; this string serves as the identifier of the container in subsequent commands.
Note: If a hostname is used, make sure that the hostname to IP resolution mechanism is reliable. A failure of this mechanism causes a loss of service in the cluster. It is recommended to add the hostnames to /etc/hosts.
Note: After the successful completion of this command, the cluster is in the initialization phase, and some commands can only run in this phase.
Note: For configuring HA, at least two cards must be defined for each container.
On successful completion of the formation of the cluster, every container receives a container-ID. To display the list of the containers and IDs, run weka cluster container.
Note: In IB installations the --containers-ips parameter must specify the IP addresses of the IPoIB interfaces.
6. Configure the SSD drives
Command: weka cluster drive add
To configure the SSD drives on each server in the cluster, or add multiple drive paths, use the following command:
Parameters
Name | Type | Value | Limitations | Mandatory | Default |
| String | The Identifier of the drive container to add the local SSD drives | Must be a valid container identifier | Yes | |
| Space-separated list of strings | List of block devices that identify local SSDs. For example, | Must be a valid Unix network device name | Yes |
7. Create compute containers
Command: weka local setup container
For each server in the cluster, create the compute containers using the resource generator output file compute0.json.
The compute JSON file includes all the required values for creating the compute containers. Therefore, it is not required to set all the options of the weka local setup container command. Only the path to the JSON resource file is required.
Parameters
Name | Type | Value | Limitations | Mandatory | Default |
| String | The path to the resource file. | Must be a valid path | yes | |
Space-separated strings | IP addresses of the cluster servers to join the container.
If the cluster does not use the default port 14000, specify the actual port. For example: | Must be valid IP addresses |
8. Name the cluster
Command: weka cluster update --cluster-name=<cluster name>
9. Name the cluster and enable event notifications to the cloud (optional)
Enable event notifications to the cloud for support purposes using one of the following options:
Enable support through Weka Home
Enable support through a private instance of Weka Home
Enable support through Weka Home
Command: weka cloud enable
This command enables cloud event notification (via Weka Home), which increases the ability of the Weka Support Team to resolve any issues that may occur.
To learn more about this and how to enable cloud event notification, refer to Weka Support Cloud.
Enable support through a private instance of Weka Home
In closed environments, such as dark sites and private VPCs, it is possible to install a private instance of Weka Home.
Command: weka cloud enable --cloud-url=http://<weka-home-ip>:<weka-home-port>
This command enables the use of a private instance of Weka Home.
For more information, refer to the Private Instance of Weka Home and contact the Customer Success Team.
10. Set the license
Command: weka cluster license set / payg
To run IOs against the cluster, a valid license must be set. Obtain a valid license, classic or PAYG, and set it to the Weka cluster. For details, see License overview.
11. Start the cluster IO service
Command: weka cluster start-io
To start the system IO and exit from the initialization state, use the following command line:
weka cluster start-io
12. Create frontend containers
Command: weka local setup container
For each server in the cluster, create the frontend containers using the resource generator output file frontend0.json.
The compute JSON file includes all the required values for creating the frontend containers. Therefore, it is not required to set all the options of the weka local setup container command. Only the path to the JSON resource file is required.
Parameters
Name | Type | Value | Limitations | Mandatory | Default |
| String | The path to the resource file. | Must be a valid path | yes | |
Space-separated strings | IP addresses of the cluster servers to join the container.
If the cluster does not use the default port 14000, specify the actual port. For example: | Must be valid IPs | Yes |
13. Check the cluster configuration
Check the cluster container
Command: weka cluster container
Use this command to display the list of containers and their details.
Check cluster container resources
Command: weka cluster container resources
Use this command to check the resources of each container in the cluster.
weka cluster container resources <container-id>
Example for a drive container resources output:
Example of a compute container resources output:
Example of a frontend container resources output:
Check cluster drives
Command: weka cluster drive
Use this command to check all drives in the cluster.
Example:
Check Weka cluster status
Command: weka status
The weka status command displays the overall status of the Weka cluster.
For details, see Cluster status.
14. Bypass the proxy server
If the WEKA cluster is deployed in an environment with a proxy server, a WEKA client trying to mount or download the client installation from the WEKA cluster may be blocked by the proxy server. You can disable the proxy for specific URLs using the shell no_proxy environment variable.
Procedure
Connect to one of the WEKA backend servers.
Open the
/etc/wekaio/service.conffile.In the
[downloads_proxy]section, add to theno_proxyparameter a comma-separated list of IP addresses or qualified domain names of your WEKA clients and cluster backend servers. Do not use wildcards (*).
Restart the agent service.
Last updated