Upgrade WEKA versions

Upgrade your WEKA system with the latest version.

Upgrade overview

The WEKA upgrade process supports upgrades of a WEKA system using a non-disruptive upgrade (NDU).

When considering an upgrade, adhere to the following guidelines:

  • To upgrade to 4.3.Y, the minimum source version allowed is 4.2.0.

  • The upgrade must always be from an older version to a newer one. For instance, an upgrade from 4.2.X to 4.3.Y is permissible only if 4.3.Y was released after 4.2.X.

  • Stay within the same major version or upgrade from an older major version to its successor.

For instance, an upgrade from version 4.2.10 (released on April 12, 2024) to version 4.3.0 (released on April 23, 2024) is possible. However, for future LTS releases, like 4.2.11, it is not allowed to upgrade to 4.3.0. For information on the release dates, visit get.weka.io.

The source system must be set up in MCB architecture. If not, contact the Customer Success Team to convert the cluster architecture to MCB. See Convert the cluster architecture from a single-container backend to a multi-container backend. This workflow is only intended for professional services.

Customers running WEKA clusters on AWS with auto-scaling groups must contact the Customer Success Team before converting the cluster to MCB.

What is a non-disruptive upgrade (NDU)

In MCB architecture, each container serves a single type of process, drive, frontend, or compute function. Therefore, upgrading one container at a time (rolling upgrade) is possible while the remaining containers continue serving the clients.

Some background tasks, such as snapshot uploads or downloads, must be postponed or aborted. See the prerequisites in the upgrade workflow for details.

Internal upgrade process

Once you run the upgrade command in ndu mode, the following occurs:

  1. Downloading the version and preparing all backend servers.

  2. Rolling upgrade of the drive containers.

  3. Rolling upgrade of the compute containers.

  4. Rolling upgrade of the frontend and protocol containers and the protocol gateways.

Related topics

WEKA containers architecture overview

Upgrade workflow

Adhere to the following considerations:

  • Protocol separation: Upgrading a WEKA cluster with a server used for more than one of the following protocols, NFS, SMB, or S3, is not permitted. If such a case arises, the upgrade process does not initiate and indicates the servers that require protocol separation. Contact the Customer Success Team to ensure only one additional protocol is installed on each server.

  • Legacy NFS protocol: If a legacy NFS protocol is implemented, it’s advised to contact the Customer Success Team. In this case, the upgrade is blocked.

  • S3 Cluster Creation: If there is a plan to create an S3 cluster, it’s crucial to ensure the upgrade process is complete and all containers are up before initiating the S3 cluster creation.

1. Verify system upgrade prerequisites

Before proceeding with any system upgrade, ensuring that the environment meets the necessary prerequisites is crucial. The WEKA Upgrade Checker Tool automates these essential checks, comprehensively assessing the system’s readiness. Whether performing a single-version upgrade or a multi-hop upgrade, following this procedure is mandatory.

Summary of the WEKA Upgrade Checker Tool results:

  1. Passed checks (Green): The system meets all prerequisites for the upgrade.

  2. Warnings (Yellow): Address promptly to resolve potential issues.

  3. Failures (Red): Do not proceed; they may lead to data loss.

Sample list of the verification steps performed by the WEKA Upgrade Checker Tool

Multi-hop version upgrades:

After completing an upgrade, a background process initiates the conversion of metadata to a new format (in specific versions). This conversion may take several minutes before another upgrade can commence. To monitor the progress, use the weka status CLI command and check if a data upgrade task is RUNNING.

By diligently following this system readiness validation procedure, you can confidently proceed with system upgrades, minimizing risks and ensuring a smooth upgrade.

Demo: WEKA Upgrade Checker
  • Prioritize running the WEKA Upgrade Checker 24 hours before any scheduled upgrades. This step is critical to identify and address any potential issues proactively.

  • Ensure passwordless SSH access is set up on all backend servers. This is crucial for the seamless execution of the Python script while running the WEKA Upgrade Checker.

Procedure

  1. Log in to one of the backend servers as a root user:

    • Access the server using the appropriate credentials.

  2. Obtain the WEKA Upgrade Checker: Choose one of the following methods:

    • Method A: Direct download

      • Clone the WEKA Upgrade Checker GIT repository with the command: git clone https://github.com/weka/tools.git

    • Method B: Update from existing tools repository

      • If you have previously downloaded the tools repository, navigate to the tools directory.

      • Run git pull to update the tools repository with the latest enhancements. (The WEKA tools, including the WEKA Upgrade Checker, continuously evolve.)

  3. Run the WEKA Upgrade Checker:

    • Navigate to the weka_upgrade_checker directory. It includes a binary version and a Python script of the tool. A minimum of Python 3.8 is required if you run the Python script.

    • Run the Python script: python3.8 ./weka_upgrade_checker.py

      The tool scans the backend servers and verifies the upgrade prerequisites.

  4. Review the results:

    • Pay attention to the following indicators:

      • Green: Passed checks. Ensure the tool's version is the latest.

      • Yellow: Warnings that require attention and remedy.

      • Red: Failed checks. If any exist, do not proceed. Contact the Customer Success Team.

  5. Send the log file to the Customer Success Team:

    • The weka_upgrade_checker.log is located in the same directory you ran the tool from. Share the latest log file with the Customer Success Team for further analysis.

2. Prepare the cluster for upgrade

Download the new WEKA version to one of the backend servers using one of the following methods depending on the cluster deployment:

  • Method A: Using a distribution server

  • Method B: Direct download and install from get.weka.io

  • Method C: If the connectivity to get.weka.io is limited

For details, select the relevant tab.

Use this method if the cluster environment includes a distribution server from which the target WEKA version can be downloaded.

If the distribution server contains the target WEKA version, run the following commands from the cluster backend server:

weka version get <version>
weka version prepare <version>

Where: <version> is the target WEKA version, for example: 4.3.0.

If the distribution server does not contain the target WEKA version, add the option --from to the command, and specify the get.weka.io distribution site, along with the token.

Example:

weka version get <version> --from https://[GET.WEKA.IO-TOKEN]@get.weka.io
weka version prepare <version>

3. Prepare the backend servers for upgrade (optional)

When working with many backend servers, preparing them separately from the upgrade process in advance is possible to minimize the total upgrade time. For a small number of backend servers, this step is not required.

The preparation phase prepares all the connected backend servers for the upgrade, which includes downloading the new version and getting it ready to be applied.

Once the new version is downloaded to one of the backend servers, run the following CLI command:

weka local run --container <container-name> --in <new-version> upgrade --prepare-only

Where:

<new-version>: Specify the new version. For example,4.3.0.

<container-name>: Specify only one container name. For example: drives0.

The default upgrade mode to 4.3.x is ndu. Therefore, no need to specify it.

4. Upgrade the backend servers

Once a new software version is installed on one of the backend servers, upgrade the cluster to the new version by running the following command on the backend server.

If you already ran the preparation step, the upgrade command skips the download and preparation operations.

weka local run --container <container-name> --in <new-version> upgrade

Example:

weka local run --container drives0 --in 4.3.1 upgrade

Consider the following guidelines:

  • Before switching the cluster to the new software release, the upgrade command distributes the new release to all cluster servers. It makes the necessary preparations, such as compiling the new wekafs driver.

  • If a failure occurs during the preparation, such as a disconnection of a server or failure to build a driver, the upgrade process stops, and a summary message indicates the problematic server.

  • If cleanup issues occur during a specific upgrade phase, rerun it with the relevant option:

    --ndu-drives-phase
    --ndu-frontends-phase
    --ndu-computes-phase
  • In a successful process, the upgrade stops the cluster IO service, switches all servers to the new release, and then turns the IO service back on. This process takes about 1 minute, depending on the cluster size.

  • If the container running the upgrade process uses a port other than the default (14000), include the option --mgmt-port <existing-port> to the command.

5. Upgrade the clients

Once all backends are upgraded, the clients remain with the existing version and continue working with the upgraded backends. The client's version can only be one major version behind the version of the backends. Therefore, clients must be upgraded before the next cluster software version upgrade. The minimum source version for client upgrades is 4.3.X.

Stateless client upgrade options

  • If a stateless client is mounted on a single cluster, it is automatically upgraded to the backend version after rebooting, or a complete umount and mount is performed.

  • If a stateless client is mounted on multiple clusters, the client container version is the same as the client-target-version in the cluster (see Mount filesystems from multiple clusters on a single client).

  • Stateless clients can also be upgraded manually.

  • To ensure downloading only the essential components relevant to the stateless client operation, excluding non-relevant packages, use the --client-only flag in the weka version get command.

  • To limit the display of versions unless the complete set of components is present, use the --full flag with the weka version command This provides you with finer control over version information visibility.

  • You can manually upgrade the clients locally (one by one) or remotely (in batches), usually during a maintenance window.

Stateful client upgrade options

  • You can manually upgrade the clients locally (one by one) or remotely (in batches), usually during a maintenance window.

  • A gateway, which is a stateful client running a protocol, is upgraded with the backend servers.

Client upgrade procedures

To upgrade a stateless or stateful client locally, connect to the client and run the following command line:

  1. Run: weka version get <target-version> --from <backend name or IP>:<port>

  2. Upgrade the agent by running the following: /opt/weka/dist/cli/<target_client> agent install-agent --no-update

  3. Upgrade the client containers. Do one the following following:

    • For clients connected to a single cluster, run weka local upgrade

    • For clients connected to a multiple clusters, upgrade all containers simultaneously by running weka local upgrade --all

An alert is raised if there is a mismatch between the clients' and the cluster versions.

Add the --from <backend name or IP> option to download the client package only from the backend, thus avoiding downloading from get.weka.io. The default port is 14000.

6. Check the status after the upgrade

Once the upgrade is complete, verify that the cluster is in the new version by running the weka status command.

Example: The following is returned when the system is upgraded to version 4.3.0:

# weka status Weka v4.3.0 ...

Last updated