WEKA cluster installation
This page provides a detailed workflow for WEKA cluster installation with multiple containers using the CLI. These are complementary details for the quick installation guide.
Workflow
1. Install the WEKA software
Once the WEKA software is downloaded from get.weka.io, run the untar command and install.sh command on each server, according to the instructions in the Install tab.
Once completed, the WEKA software is installed on all the allocated servers and runs in stem mode (no cluster is attached).
Note: If a failure occurs during the WEKA software installation process, an error message prompts detailing the source of the failure. Review the details and try to resolve the failure. If required, contact the Customer Success Team.
2. Remove the default container
Command: weka local stop default && weka local rm -f default
Stop and remove the auto-created default container created on each server.
3. Generate the resource files
Command: resources_generator.py
To generate the resource files for the drive, compute, and frontend processes, download the resource_generator.py and run the following command on each backend server:
./resources_generator.py --net <net-devices> [options]
The resource generator allocates the number of cores, memory, and other resources according to the values specified in the parameters.
The best practice for resources allocation is as follows:
1 drive core per NVMe device (SSD).
2-3 compute cores per drive core.
1-2 frontend cores if deploying a protocol container. If there is a spare core, it is used for a frontend container.
Minimum of 1 core for the OS.
Example 1: according to the best practice
For a server with 24 cores and 6 SSDs, allocate 6 drive cores and 12 compute cores, and optionally you can use 2 cores of the remaining cores for the frontend container. The OS uses the remaining 4 cores.
Run the following command line:
./resources_generator.py --net eth1 --net eth2 --drive-dedicated-cores 6 --compute-dedicated-cores 12 --frontend-dedicated-cores 2
Example 2: a server with a limited number of cores
For a server with 14 cores and 6 SSDs, allocate 6 drive cores and 6 compute cores, and optionally you can use 1 core of the remaining cores for the frontend container. The OS uses the remaining 1 core.
Run the following command line:
./resources_generator.py --net eth1 --net eth2 --drive-dedicated-cores 6 --compute-dedicated-cores 6 --frontend-dedicated-cores 1
Note: Contact Professional Services for the recommended resource allocation settings for your system.
Parameters
Name
Type
Value
Mandatory
Default
compute-core-ids
Space-separated numbers
Specify the CPUs to allocate for the compute processes.
No
-
compute-dedicated-cores
Number
Specify the number of cores to dedicate for the compute processes.
No
The maximum available cores
compute-memory
String
Specify the total memory to allocate for the compute processes.
Argument format: value and unit without a space.
Examples: 1024B, 10GiB, 5TiB.
No
The maximum available memory
core-ids
Space-separated numbers
Specify the CPUs to allocate for the WEKA processes.
No
-
drive-core-ids
Space-separated numbers
Specify the CPUs to allocate for the drive processes.
No
-
drive-dedicated-cores
Number
Specify the number of cores to dedicate for the drive processes.
No
1 core per each detected drive
drives
Space-separated strings
Specify the drives to use.
This option overrides automatic detection.
No
All unmounted NVME devices
frontend-core-ids
Space-separated numbers
Specify the CPUs to allocate for the frontend processes.
No
-
frontend-dedicated-cores
Number
Specify the number of cores to dedicate for the frontend processes.
No
1
max-cores-per-container
Number
Override the default maximum number of cores per container for IO processes (19). If provided, the new value must be lower.
No
19
minimal-memory
Flag
Set each container's hugepages memory to 1.4 GiB * number of IO processes on the container.
No
-
net-devices
Space-separated strings
Specify the network devices to use.
Yes
-
no-rdma
Boolean
Don't take RDMA support into account when computing memory requirements.
No
False
num-cores
Number
Override the auto-deduction of the number of cores.
No
All available cores
path
String
Specify the path to write the resource files.
No
The default is '.'
spare-cores
Number
Specify the number of cores to leave for OS and non-WEKA processes.
No
1
spare-memory
String
Specify the memory to reserve for non-WEKA requirements.
Argument format: a value and unit without a space.
Examples: 10GiB, 1024B, 5TiB.
No
The maximum between 8 GiB and 2% of the total RAM
weka-hugepages-memory
String
Specify the memory to allocate for compute, frontend, and drive processes.
Argument format: a value and unit without a space.
Examples: 10GiB, 1024B, 5TiB.
No
The maximum available memory
4. Create drive containers
Command: weka local setup container
For each server in the cluster, create the drive containers using the resource generator output file drives0.json.
The drives JSON file includes all the required values for creating the drive containers. Therefore, it is not required to set all the options of the weka local setup container command. Only the path to the JSON resource file is required.
Parameters
Name
Type
Value
Limitations
Mandatory
Default
resources-path
String
The path to the resource file.
Must be a valid path
Yes
5. Create a cluster
Command: weka cluster create
To create a cluster of the allocated containers, use the following command:
Parameters
Name
Type
Value
Limitations
Mandatory
Default
hostnames
Space-separated strings
Hostnames or IP addresses. If port 14000 is not the default for the drives, you can specify hostnames:port or ips:port.
Need at least 6 strings, as this is the minimal cluster size
Yes
host-ips
Comma-separated IP addresses
IP addresses of the management interfaces. Use a list of ip+ip addresses pairs of two cards for HA configuration. In case the cluster is connected to both IB and Ethernet, it is possible to set up to 4 management IPs for redundancy of both the IB and Ethernet networks using a list of ip+ip+ip+ip addresses.
The same number of values as in hostnames.
No
IP of the first network device of the container
Note: It is possible to use either a hostname or an IP address; this string serves as the identifier of the container in subsequent commands.
Note: If a hostname is used, make sure that the hostname to IP resolution mechanism is reliable. A failure of this mechanism causes a loss of service in the cluster. It is recommended to add the hostnames to /etc/hosts.
Note: After the successful completion of this command, the cluster is in the initialization phase, and some commands can only run in this phase.
Note: For configuring HA, at least two cards must be defined for each container.
On successful completion of the formation of the cluster, every container receives a container-ID. To display the list of the containers and IDs, run weka cluster container.
Note: In IB installations the --containers-ips parameter must specify the IP addresses of the IPoIB interfaces.
6. Configure the SSD drives
Command: weka cluster drive add
To configure the SSD drives on each server in the cluster, or add multiple drive paths, use the following command:
Parameters
Name
Type
Value
Limitations
Mandatory
Default
container-id
String
The Identifier of the drive container to add the local SSD drives
Must be a valid container identifier
Yes
device-paths
Space-separated list of strings
List of block devices that identify local SSDs. For example, /dev/nvme0n1 /dev/nvme1n1
Must be a valid Unix network device name
Yes
7. Create compute containers
Command: weka local setup container
For each server in the cluster, create the compute containers using the resource generator output file compute0.json.
The compute JSON file includes all the required values for creating the compute containers. Therefore, it is not required to set all the options of the weka local setup container command. Only the path to the JSON resource file is required.
Parameters
Name
Type
Value
Limitations
Mandatory
Default
resources-path
String
The path to the resource file.
Must be a valid path
yes
Space-separated strings
IP addresses of the cluster servers to join the container.
If the cluster does not use the default port 14000, specify the actual port. For example: --join-ips 10.10.10.23:15000
Must be valid IP addresses
8. Name the cluster
Command: weka cluster update --cluster-name=<cluster name>
9. Name the cluster and enable event notifications to the cloud (optional)
Enable event notifications to the cloud for support purposes using one of the following options:
Enable support through Weka Home
Enable support through a private instance of Weka Home
Enable support through Weka Home
Command: weka cloud enable
This command enables cloud event notification (via Weka Home), which increases the ability of the Weka Support Team to resolve any issues that may occur.
To learn more about this and how to enable cloud event notification, refer to Weka Support Cloud.
Enable support through a private instance of Weka Home
In closed environments, such as dark sites and private VPCs, it is possible to install a private instance of Weka Home.
Command: weka cloud enable --cloud-url=http://<weka-home-ip>:<weka-home-port>
This command enables the use of a private instance of Weka Home.
For more information, refer to the Private Instance of Weka Home and contact the Customer Success Team.
10. Set the license
Command: weka cluster license set / payg
To run IOs against the cluster, a valid license must be set. Obtain a valid license, classic or PAYG, and set it to the Weka cluster. For details, see License overview.
11. Start the cluster IO service
Command: weka cluster start-io
To start the system IO and exit from the initialization state, use the following command line:
weka cluster start-io
12. Create frontend containers
Command: weka local setup container
For each server in the cluster, create the frontend containers using the resource generator output file frontend0.json.
The compute JSON file includes all the required values for creating the frontend containers. Therefore, it is not required to set all the options of the weka local setup container command. Only the path to the JSON resource file is required.
Parameters
Name
Type
Value
Limitations
Mandatory
Default
resources-path
String
The path to the resource file.
Must be a valid path
yes
Space-separated strings
IP addresses of the cluster servers to join the container.
If the cluster does not use the default port 14000, specify the actual port. For example: --join-ips 10.10.10.23:15000
Must be valid IPs
Yes
13. Check the cluster configuration
Check the cluster container
Command: weka cluster container
Use this command to display the list of containers and their details.
Check cluster container resources
Command: weka cluster container resources
Use this command to check the resources of each container in the cluster.
weka cluster container resources <container-id>
Example for a drive container resources output:
Example of a compute container resources output:
Example of a frontend container resources output:
Check cluster drives
Command: weka cluster drive
Use this command to check all drives in the cluster.
Example:
Check Weka cluster status
Command: weka status
The weka status command displays the overall status of the Weka cluster.
For details, see Cluster status.
14. Bypass the proxy server
If the WEKA cluster is deployed in an environment with a proxy server, a WEKA client trying to mount or download the client installation from the WEKA cluster may be blocked by the proxy server. You can disable the proxy for specific URLs using the shell no_proxy environment variable.
Procedure
Connect to one of the WEKA backend servers.
Open the
/etc/wekaio/service.conffile.In the
[downloads_proxy]section, add to theno_proxyparameter a comma-separated list of IP addresses or qualified domain names of your WEKA clients and cluster backend servers. Do not use wildcards (*).
Restart the agent service.
Last updated