Mount filesystems
To use a filesystem via the WEKA filesystem driver, it has to be mounted on one of the cluster servers. This page describes how this is performed.
Overview
There are two methods available for mounting a filesystem in one of the cluster servers:
Using the traditional method: See below and also refer to Adding Clients (Bare Metal Installation) or Adding Clients (AWS Installation), where first a client is configured and joins a cluster, after which a mount command is executed.
Using the Stateless Clients feature: See Mounting Filesystems Using the Stateless Clients Feature below, which simplifies and improves the management of clients in the cluster and eliminates the Adding Clients process.
Mount a filesystem using the traditional method
Note: Using the mount command as explained below first requires the installation of the WEKA client, configuring the client, and joining it to a WEKA cluster.
To mount a filesystem on one of the cluster servers, let’s assume the cluster has a filesystem called demo
. To add this filesystem to a server, SSH into one of the servers and run the mount
command as the root
user, as follows:
The general structure of the mount
command for a WEKA filesystem is as follows:
Two options for mounting a filesystem on a cluster client are read cache and write cache. Refer to the descriptions in the links below to understand the differences between these modes:
Mount a filesystem using the stateless clients feature
The stateless clients feature defers the process of joining the cluster until the mount is performed. They are simplifying and improving the management of clients in the cluster. It removes tedious client management procedures, which is particularly beneficial in AWS installations where clients may join and leave at high frequency.
Furthermore, it unifies all security aspects in the mount command, eliminating the search for separate credentials at cluster join and mount.
To use the Stateless Clients feature, a WEKA agent must be installed. Once this is complete, mounts can be created and configured using the mount command and can be removed from the cluster using the unmount command.
Note: To allow only WEKA authenticated users to mount a filesystem, set the filesystem --auth-required
flag to yes
. For more information, refer to the Mount authentication for organization filesystems topic.
Assuming the WEKA cluster is using the backend IP of 1.2.3.4
, running the following command as root
on a client will install the agent:
curl http://1.2.3.4:14000/dist/v1/install | sh
On completion, the agent is installed on the client.
Run the mount command
Command: mount -t wekafs
Use one of the following command lines to invoke the mount command (note, the delimiter between the server and filesystem can be either :/
or /
):
mount -t wekafs -o <options> <backend0>[,<backend1>,...,<backendN>]/<fs> <mount-point>
mount -t wekafs -o <options> <backend0>[,<backend1>,...,<backendN>]:/<fs> <mount-point>
Parameters
Mount command options
Each mount option can be passed by an individual -o
flag to mount.
For all clients types
Remount of general options
You can remount using the mount options marked as Remount Supported
in the above table (mount -o remount)
.
When a mount option has been explicitly changed, you must set it again in the remount operation to ensure it retains its value. For example, if you mount with ro
, a remount without it changes the mount option to the default rw
. If you mount with rw
, it is not required to re-specify the mount option because this is the default.
Additional mount options using the stateless clients feature
Note: These parameters, if not stated otherwise, are only effective on the first mount command for each client.
Note: By default, the command selects the optimal core allocation for Weka. If necessary, multiple core
parameters can be used to allocate specific cores to the WekaFS client. E.g., mount -t wekafs -o core=2 -o core=4 -o net=ib0 backend-server-0/my_fs /mnt/weka
Example: On-Premise Installations
mount -t wekafs -o num_cores=1 -o net=ib0 backend-server-0/my_fs /mnt/weka
Running this command on a server installed with the Weka agent will download the appropriate Weka version from the backend-server-0
and create a Weka container that allocates a single core and a named network interface (ib0
). Then it will join the cluster that backend-server-0
is part of and mount the filesystem my_fs
on /mnt/weka.
mount -t wekafs -o num_cores=0 -o net=udp backend-server-0/my_fs /mnt/weka
Running this command will use UDP mode (usually selected when the use of DPDK is not available).
Example: AWS Installations
mount -t wekafs -o num_cores=2 backend1,backend2,backend3/my_fs /mnt/weka
Running this command on an AWS EC2 instance allocates two cores (multiple-frontends) and attach and configure two ENIs on the new client. The client will attempt to rejoin the cluster via all three backends used in the command line.
For stateless clients, the first mount
command installs the weka client software and joins the cluster). Any subsequent mount
command, can either use the same syntax or just the traditional/per-mount parameters as defined in Mounting Filesystems since it is not necessary to join a cluster.
It is now possible to access Weka filesystems via the mount-point, e.g., by cd /mnt/weka/
command.
After the execution of anumount
command, which unmounts the last Weka filesystem, the client is disconnected from the cluster and will be uninstalled by the agent. Consequently, executing a new mount
command requires the specification of the cluster, cores, and networking parameters again.
Note: When running in AWS, the instance IAM role is required to provide permissions to several AWS APIs, as described in IAM Role Created in Template.
Note: Memory allocation for a client is predefined. Contact the Weka Support Team when it is necessary to change the amount of memory allocated to a client.
Remount of stateless clients options
Mount options marked as Remount Supported
in the above table can be remounted (using mount -o remount
). When a mount option is not set in the remount operation, it will retain its current value. To set a mount option back to its default value, use the default
modifier (e.g., memory_mb=default)
.
Set mount option default values
The defaults of the mount options qos_max_throughput_mbps
and qos_preferred_throughput_mbps
have no limit.
The cluster admin can set these default values to meet the organization's requirements, reset to the initial default values (no limit), or show the existing values.
The mount option defaults are only relevant for new mounts performed and do not influence the existing ones.
Commands:
weka cluster mount-defaults set
weka cluster mount-defaults reset
weka cluster mount-defaults show
To set the mount option default values, run the following command:
weka cluster mount-defaults set [--qos-max-throughput qos-max-throughput] [--qos-preferred-throughput qos-preferred-throughput]
Parameters
Advanced network configuration by mount options
When using a stateless client, it is possible to alter and control many different networking options, such as:
Virtual functions
IPs
Gateway (in case the client is on a different subnet)
Physical network devices (for performance and HA)
UDP mode
Use -o net=<netdev>
mount option with the various modifiers as described below.
<netdev>
is either the name, MAC address, or PCI address of the physical network device (can be a bond device) to allocate for the client.
Note: When using wekafs
mounts, both clients and backends should use the same type of networking technology (either IB or Ethernet).
IP, subnet, gateway, and virtual functions
For higher performance, the usage of multiple Frontends may be required. When using a NIC other than Mellanox or Intel E810 or mounting a DPDK client on a VM, it is required to use SR-IOV to expose a VF of the physical device to the client. Once exposed, it can be configured via the mount command.
When you want to determine the VFs IP addresses or when the client resides in a different subnet and routing is needed in the data network, usenet=<netdev>/[ip]/[bits]/[gateway]
.
ip, bits, gateway
are optional. If these options are not provided, the Weka system performs one of the following depending on the environment:
Cloud environment: the Weka system deduces the values of these options.
On-premises environment: the Weka system allocates values of these options from the cluster default network (the
weka cluster default-net
must be set before running the mount command). Otherwise, the Weka cluster does not allocate the IP for the client. For more details, see Optional: Configure default data networking.
For example, the following command allocates two cores and a single physical network device (intel0). It will configure two VFs for the device and assign each one of them to one of the frontend processes. The first container will receive a 192.168.1.100 IP address, and the second will use a 192.168.1.101 IP address. Both of the IPs have 24 network mask bits and a default gateway of 192.168.1.254.
Multiple physical network devices for performance and HA
For performance or high availability, it is possible to use more than one physical network device.
Using multiple physical network devices for better performance
It's easy to saturate the bandwidth of a single network interface when using WekaFS. For higher throughput, it is possible to leverage multiple network interface cards (NICs). The -o net
notation shown in the examples above can be used to pass the names of specific NICs to the WekaFS server driver.
For example, the following command will allocate two cores and two physical network devices for increased throughput:
Using multiple physical network devices for HA configuration
Multiple NICs can also be configured to achieve redundancy (refer to Weka Networking HA section for more information) in addition to higher throughput, for a complete, highly available solution. For that, use more than one physical device as previously described and, also, specify the client management IPs using -o mgmt_ip=<ip>+<ip2>
command-line option.
For example, the following command will use two network devices for HA networking and allocate both devices to four Frontend processes on the client. Note the modifier ha
is used here, which stands for using the device on all processes.
Advanced mounting options for multiple physical network devices
With multiple Frontend processes (as expressed by -o num_cores
), it is possible to control what processes use what NICs. This can be accomplished through the use of special command line modifiers called slots. In WekaFS, slot is synonymous with a process number. Typically, the first WekaFS Frontend process will occupy slot 1, then the second - slot 2 and so on.
Examples of slot notation include s1
, s2
, s2+1
, s1-2
, slots1+3
, slot1
, slots1-4
, where -
specifies a range of devices, while +
specifies a list. For example, s1-4
implies slots 1, 2, 3 and 4, while s1+4
specifies slots 1 and 4 only.
For example, in the following command, mlnx0
is bound to the second Frontend process whilemlnx1
to the first one for improved performance.
For example, in the following HA mounting command, two cores (two Frontend processes) and two physical network devices (mlnx0
, mlnx1
) are allocated. By explicitly specifying s2+1
, s1-2
modifiers for network devices, both devices will be used by both Frontend processes. Notation s2+1
stands for the first and second processes, while s1-2
stands for the range of 1 to 2, and are effectively the same.
UDP mode
In cases where DPDK cannot be used, it is possible to use WekaFS in UDP mode through the kernel. Use net=udp
in the mount command to set the UDP networking mode, for example:
Note: A client in UDP mode cannot be configured in HA mode. However, the client can still work with a highly available cluster.
Note: Providing multiple IPs in the <mgmt-ip> in UDP mode will utilize their network interfaces for more bandwidth (can be useful in RDMA environments), rather than using only one NIC.
Mount filesystems using fstab
Note: This option works when using stateless clients and with OS that supports systemd
(e.g.: RHEL/CentOS 7.2 and up, Ubuntu 16.04 and up, Amazon Linux 2 LTS).
Edit /etc/fstab
file to include the filesystem mount entry:
A comma-separated list of backend servers, with the filesystem name
The mount point
Filesystem type -
wekafs
Mount options:
Configure
systemd
to wait for theweka-agent
service to come up, and set the filesystem as a network filesystem, e.g.:x-systemd.requires=weka-agent.service,x-systemd.mount-timeout=infinity,_netdev
Any additional
wekafs
supported mount option
Reboot the server for the systemd
unit to be created and marked correctly.
The filesystem should now be mounted at boot time.
Note: Do not configure this entry for a mounted filesystem before un-mounting it (umount
), as the systemd
needs to mark the filesystem as a network filesystem (occurs as part of the reboot
). Trying to reboot a server when there is a mounted WekaFS filesystem when setting its fstab
configuration might yield a failure to unmount the filesystem and leave the system hanged.
Mount filesystems using autofs
Procedure:
Install
autofs
on the server using one of the following commands according to your deployment:
On RedHat or Centos:
On Debian or Ubuntu:
2. To create the autofs
configuration files for Weka filesystems, do one of the following
depending on the client type:
For a stateless client, run the following commands (specify the backend names as parameters):
For a stateful client (traditional), run the following commands:
3. Restart the autofs
service:
4. The configuration is distribution-dependent. Verify that the service is configured to start
automatically after restarting the server. Run the following command:
systemctl is-enabled autofs.
If the output is enabled
the service is configured to start automatically.
Example: In Amazon Linux, you can verify that the autofs
service is configured to start automatically by running the command chkconfig
.
If the output is on
for the current runlevel (you can check with therunlevel
command), autofs
is enabled upon restart.
Once you complete this procedure, it is possible to access Weka filesystems using the command cd /mnt/weka/<fs-name>
.
Last updated