There are two methods available for mounting a filesystem in one of the cluster hosts:
Using the Stateless Clients feature: See Mounting Filesystems Using the Stateless Clients Feature below, which simplifies and improves the management of clients in the cluster and eliminates the Adding Clients process.
To mount a filesystem on one of the cluster hosts, let’s assume the cluster has a filesystem called
demo. To add this filesystem to a host, SSH into one of the hosts and run the
mountcommand as the
root user, as follows:
mkdir -p /mnt/weka/demomount -t wekafs demo /mnt/weka/demo
The general structure of a
mount command for a Weka filesystem is:
mount -t wekafs [-o option[,option]...]] <fs-name> <mount-point>
There are two options for mounting a filesystem on a cluster client: read cache and write cache. Refer to the descriptions in the links below to understand the differences between these modes:
The Stateless Clients feature defers the process of joining the cluster until a mount is performed. Simplifying and improving the management of clients in the cluster, it removes tedious client management procedures, which is particularly beneficial in AWS installations where clients may join and leave in high frequency. Furthermore, it unifies all security aspects in the mount command, eliminating the search of separate credentials at cluster join and mount.
To use the Stateless Clients feature, a Weka agent must be installed. Once this is complete, mounts can be created and configured using the mount command and can be easily removed from the cluster using the unmount command.
Assuming the Weka cluster is using the backend IP of
188.8.131.52, running the following command as
root on a client will install the agent:
curl http://184.108.40.206:14000/dist/v1/install | sh
On completion, the agent is installed on the client machine.
mount -t wekafs
Use the following command line to invoke the mount command:
mount -t wekafs -o <options> <backend0>[,<backend1>,...,<backendN>]/<fs> <mount-point>
Parameters in Command Line
See Additional Mount Options below
IP/hostname of a backend host
Must be a valid name
Must be a valid name
Path to mount on the local machine
Must be a valid path-name
Each mount option can be passed with an individual
-o flag to
Set mode to read cache
Set mode to write cache
Number in milliseconds
After the defined time period, every metadata cached entry is refreshed from the system, allowing the host to take into account metadata changes performed by other hosts.
Number in milliseconds
Each time a file or directory lookup fails, an entry specifying that the file or directory does not exist is created in the local dentry cache. This entry is refreshed after the defined time, allowing the host to use files or directories created by other hosts.
Mount filesystem as read-only
Mount filesystem as read-write
32, 64 or auto
Size of the inode in bits, which may be required for 32-bit applications.
Write debug logs to the console
Don't show any logs to console
Can be defined per mount.
Setting POSIX ACLs can change the effective group permissions (via the
See Object-store Direct Mount section
Do not update inode access times
Always update inode access times
Update inode access times only on modification or change, or if inode has been accessed and
Number in seconds
How much time (in seconds) to wait since an inode has been accessed (not modified) before updating the access time.
0 means to never update the access time on access only.
This option is relevant only if
Do not take
Do not interpret character or block special devices.
Do not allow direct execution of any binaries.
Amount of memory to be used by the client (for huge pages)
The number of frontend cores to allocate for the client.
If none are specified, the client will be configured with 1 core.
If 0 is specified then you must use
Specify explicit cores to be used by the WekaFS client. Multiple cores can be specified.
This option must be specified for on-premises installation, and must not be specified for AWS installations.
For more info refer to Advanced Network Configuration via Mount Options section.
Network bandwidth limitation for the entire container, in Mb/s.
This limitation is for all nodes running within the container, and an attempt is made to detect it automatically based on the environment e.g., when in AWS. Setting a per-node limitation can be performed in the container definition file.
The number of seconds without connectivity after which the client will be removed from the cluster. Minimum value: 60 seconds.
86,400 seconds (24 hours)
Traces capacity limit in MB.
Minimum value: 512 MB.
Controls the page allocation algorithm if to reserve only 2MB huge pages or also 1GB ones
Number in KB
Controls the readahead per mount (higher readahead better for sequential reads of large files)
Path to the mount authentication token (per mount)
For stateless clients, the first
mount command installs the weka client software and joins the cluster). Any subsequent
mount command, can either use the same syntax or just the traditional/per-mount parameters as defined in Mounting Filesystems since it is not necessary to join a cluster.
It is now possible to access Weka filesystems via the mount-point, e.g., by
cd /mnt/weka/ command.
After the execution of an
umount command which unmounts the last Weka filesystem, the client is disconnected from the cluster and will be uninstalled by the agent. Consequently, executing a new
mount command requires the specification of the cluster, cores, and networking parameters again.
When using a stateless client, it is possible to alter and control many different networking options, such as:
Gateway (in case the client is on a different subnet)
Physical network devices (for performance and HA)
-o net=<netdev> mount option with the various modifiers as described below.
<netdev> is either the name, MAC address, or PCI address of the physical network device (can be a bond device) to allocate for the client.
For higher performance, the usage of multiple Frontends may be required. When using a NIC other than Mellanox, or when mounting a DPDK client on a VM, it is required to use SR-IOV to expose a VF of the physical device to the client. Once exposed, it can be configured via the mount command.
When you want to determine the VFs IP addresses, or when the client resides in a different subnet and routing is needed in the data network, use
ip, bits, gateway are optional. In case they are not provided, the Weka system tries to deduce them when in AWS or IB environments, or allocate from the default data network otherwise. If both approaches fail, the mount command will fail.
For example, the following command will allocate two cores and a single physical network device (intel0). It will configure two VFs for the device and assign each one of them to one of the frontend nodes. The first node will receive 192.168.1.100 IP address, and the second will use 192.168.1.101 IP address. Both of the IPs have a 24 network mask bits and default gateway of 192.168.1.254.
mount -t wekafs -o num_cores=2 -o net=intel0/192.168.1.100+192.168.1.101/24/192.168.1.254 backend1/my_fs /mnt/weka
For performance or high availability, it is possible to use more than one physical network device.
It's easy to saturate the bandwidth of a single network interface when using WekaFS. For higher throughput, it is possible to leverage multiple network interface cards (NICs). The
-o net notation shown in the examples above can be used to pass the names of specific NICs to WekaFS host driver.
For example, the following command will allocate two cores and two physical network devices for increased throughput:
mount -t wekafs -o num_cores=2 -o net=mlnx0,net=mlnx1 backend1/my_fs /mnt/weka
Multiple NICs can also be configured to achieve redundancy (refer to Weka Networking HA section for more information) in addition to higher throughput, for a complete, highly available solution. For that, use more than one physical device as previously described and, also, specify the client management IPs using
-o mgmt_ip=<ip>+<ip2> command-line option.
For example, the following command will use two network devices for HA networking and allocate both devices to four Frontend processes on the client. Note the modifier
ha is used here, which stands for using the device on all processes.
mount -t wekafs -o num_cores=4 -o net:ha=mlnx0,net:ha=mlnx1 backend1/my_fs -o mgmt_ip=10.0.0.1+10.0.0.2 /mnt/weka
Advanced mounting options for multiple physical network devices
With multiple Frontend processes (as expressed by
-o num_cores), it is possible to control what processes use what NICs. This can be accomplished through the use of special command line modifiers called slots. In WekaFS, slot is synonymous with a process number. Typically, the first WekaFS Frontend process will occupy slot 1, then the second - slot 2 and so on.
Examples of slot notation include
slots1-4 , where
- specifies a range of devices, while
+ specifies a list. For example,
s1-4 implies slots 1, 2, 3 and 4, while
s1+4 specifies slots 1 and 4 only.
For example, in the following command,
mlnx0 is bound to the second Frontend process while
mlnx1 to the first one for improved performance.
mount -t wekafs -o num_cores=2 -o net:s2=mlnx0,net:s1=mlnx1 backend1/my_fs /mnt/weka
For example, in the following HA mounting command, two cores (two Frontend processes) and two physical network devices (
mlnx1) are allocated. By explicitly specifying
s1-2 modifiers for network devices, both devices will be used by both Frontend processes. Notation
s2+1 stands for the first and second processes, while
s1-2 stands for the range of 1 to 2, and are effectively the same.
mount -t wekafs -o num_cores=2 -o net:s2+1=mlnx0,net:s1-2=mlnx1 backend1/my_fs -o mgmt_ip=10.0.0.1+10.0.0.2 /mnt/weka
In cases were DPDK cannot be used, it is possible to use WekaFS in UDP mode through the kernel. Use
net=udp in the mount command to set the UDP networking mode, for example:
mount -t wekafs -o num_cores=0 -o net=udp backend-host-0/my_fs /mnt/weka
/etc/fstab file to include the filesystem mount entry:
A comma-separated list of backend hosts, with the filesystem name
The mount point
Filesystem type -
systemd to wait for the
weka-agent service to come up, and set the filesystem as a network filesystem, e.g.:
wekafs supported mount option
# create a mount pointmkdir -p /mnt/weka/my_fs# edit fstab filevi /etc/fstab# fstab with weka options (example, change with your desired settings)backend-0,backend-1,backend-3/my_fs /mnt/weka/my_fs wekafs num_cores=1,net=eth1,x-systemd.requires=weka-agent.service,x-systemd.mount-timeout=infinity,_netdev 0 0
Reboot the machine for the
systemd unit to be created and marked correctly.
The filesystem should now be mounted at boot time.
It is possible to mount a Weka filesystem using the
To get started, install
autofs on the host:
# On RedHat/Centosyum install -y autofs
# On Debian/Ubuntuapt-get install -y autofs
Then run the following commands to create the
autofs configuration files for Weka filesystems:
echo "/mnt/weka /etc/auto.wekafs -fstype=wekafs" > /etc/auto.master.d/wekafs.autofsecho "* &" > /etc/auto.wekafs
Or run the following commands for stateless clients (which require the backend names as parameters):
echo "/mnt/weka /etc/auto.wekafs -fstype=wekafs,num_cores=1,net=<netdevice>" > /etc/auto.master.d/wekafs.autofsecho "* <backend-1>,<backend-2>/&" > /etc/auto.wekafs
Finally, restart the
service autofs restart
The configuration is distribution-dependent, and it is necessary to ensure that the service is configured to start automatically after the host is rebooted. To verify that the
autofs service automatically starts after restarting the server, run the following command:
systemctl is-enabled autofs. If the output is
enabled the service is configured to start automatically.
In Amazon Linux, for example,
autofs service can be verified with
chkconfig command. If the output is
on for the current runlevel (can be checked with
autofs will be enabled upon reboot.
# chkconfig | grep autofsautofs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
It is now possible to access Weka filesystems using the
cd /mnt/weka/<fs-name> command.