Quota Management

This page describes how to manage quotas to alert or restrict usage of the WekaFS filesystem.

Overview

There are several levels on the Weka system where capacity usage can be restricted.

  1. On an organization level - setting a different organization to manage its own filesystems, where quotas for an organization can be set, as described in the organization's usage and quota management section.

  2. On a filesystem level - setting a different filesystem per department/project.

  3. On a directory level - setting a different quota per project directory (useful when users are part of several projects) or per user home directory.

Directory Quotas

The organization admin can set a quota on a directory. Setting a quota will start the process of counting the current directory usage. Until this process is done, the quota will not be considered (for empty directories, this process is instantly done).

Note: Currently, a mount point to the relevant filesystem is required to set a quota on a directory, and the quota set command should not be interrupted until the quota accounting is over.

The organization admin sets quotas to inform/restrict users from using too much of the filesystem capacity. For that, only data in the user's control is considered. Hence, the quota doesn't count the overhead of the protection bits and snapshots. It takes into account the data and metadata of files in the directory, regardless of whether they are tiered.

Working With Quotas

When working with quotas, consider the following:

  • To set a quota, the relevant filesystem must be mounted on the host where the set quota command is to be run.

  • When setting a quota, you should go through a new mount-point. Meaning if you are using a host that has mounts from Weka versions before 3.10, first unmount all relevant mount point and then mount them again.

  • Quotas can be set within nested directories (up to 4 levels of nested quotas are supported) and over-provisioned under the same directory quota tree. E.g., /home can have a quota of 1TiB, and each user directory under it can have a quota of 10GiB, while there are 200 users.

  • Moving files (or directories) between two directories with quotas, into a directory with a quota, or outside a directory with a quota is not supported. The WekaFS filesystem returns EXDEV in such a case, which is usually converted by the operating system to copy and delete but is OS-dependent.

  • Quotas and hardlinks:

    • An existing hardlink is not counted as part of the quota.

    • Once a directory has a quota, it is not allowed to create a hardlink to files residing under directories with different (or without) directory quotas.

  • Restoring a filesystem from a snapshot turns the quotas back to the configuration at the time of the snapshot.

  • Creating a new filesystem from a snap-2-obj does not preserve the original quotas.

  • When working with enforcing quotas along with a writecache mount-mode, similarly to other POSIX solutions, getting above the quota might not sync all the cache writes to the backend servers. Use sync, syncfs, or fsync to commit the cached changes to the system (or fail due to exceeding the quota).

Integration with the df utility

When a hard quota is set on a directory, running the df utility will consider the hard quota as the total capacity of the directory and provide the use% relative to the quota. This can help users understand their usage and how close they are to the hard quota.

Note: The df utility behavior with quotas is currently global to the Weka system.

To change the global behavior, contact the Weka Support Team.

Managing Quotas using the CLI

Setting directory quotas/default quotas

Command: weka fs quota set / weka fs quota set-default

Use the following commands to set a directory quota:

weka fs quota set <path> [--soft soft] [--hard hard] [--grace grace] [--owner owner]

It is also possible to set a default quota on a directory. It does not account for this directory (or existing child directories) but will automatically set the quota on new directories created directly under it. Use the following command to set a default quota of a directory:

weka fs quota set-default <path> [--soft soft] [--hard hard] [--grace grace] [--owner owner]

Parameters in Command Line

Note: To set advisory-only quotas, use a soft quota limit without setting a grace period.

Note: When both hard and soft quotas exist, setting the value of one of them to 0 will clear this quota.

Listing directory quotas/default quotas

Command: weka fs quota list / weka fs quota list-default

Use the following command to list the directory quotas (by default, only exceeding quotas are listed) :

weka fs quota list [fs-name] [--snap-name snap-name] [--path path] [--under under] [--over over] [--quick] [--all]

Parameters in Command Line

Use the following command to list the directory default quotas:

weka fs quota list-default [fs-name] [--snap-name snap-name] [--path path]

Parameters in Command Line

Unsetting directory quotas/default quotas

Command: weka fs quota unset / weka fs quota unset-default

Use the following commands to unset a directory quota:

weka fs quota unset <path>

Use the following command to unset a default quota of a directory:

weka fs quota unset-default <path>

Parameters in Command Line

Last updated