> For the complete documentation index, see [llms.txt](https://docs.weka.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.weka.io/6.0/weka-filesystems-and-object-stores/asynchronous-replication/manage-asynchronous-replication.md).

# Manage asynchronous replication

Configure, operate, and recover asynchronous replication between NeuralMesh clusters.

All procedures require ClusterAdmin privileges. Manage asynchronous replication through this workflow:

1. Prepare both clusters.
2. Link the clusters.
3. Create a replication pair.

After you create a pair, use the on-demand procedures to monitor, modify, pause, or remove replication, manage file hydration, fail over to the target cluster, and fail back to the original source cluster.

The procedures use the `weka replication link` and `weka replication fs-pair` command groups.

## Prepare the clusters for replication

Prepare both clusters for replication. Replication traffic passes through the S3 service of each cluster, over a dedicated replica route. Replication creates no bucket, S3 user, or object store filesystem of its own.

**Before you begin**

* Ensure the network between the clusters meets the requirements in [Network requirements for asynchronous replication](https://docs.weka.io/6.0/weka-filesystems-and-object-stores/asynchronous-replication/network-requirements-for-asynchronous-replication.md).

**Procedure**

Run steps 1 to 3 on both clusters.

1. Verify the replication license. Run `weka cluster license` and check that **Cross-Cluster Replication** under **Installed License** shows **Licensed**. To add the entitlement, contact your WEKA account team.
2. Set the cluster name. Run `weka cluster update --cluster-name <name>`. The link is named after the cluster at the other end.
3. Create an S3 cluster with at least one server serving S3. Run `weka s3 cluster add`. See [Manage the S3 cluster using the CLI](https://docs.weka.io/6.0/additional-protocols/s3/s3-cluster-management/s3-cluster-management-1.md).
4. Optional: For Selective copy - critical paths (`--copy-path` with specific directories), set up a Data Services container on the **target** cluster. Full data copy (`--copy-path full`) and Selective copy - metadata only (`--copy-path none`) do not need it. See [Set up a Data Services container for background tasks](https://docs.weka.io/6.0/operation-guide/set-up-a-data-services-container-for-background-tasks.md).

## Link the clusters

Link the two clusters so they can authenticate each other and replicate filesystems.

One `weka replication link add` command, run on either cluster, creates the link on both clusters. It authenticates to the other cluster with a cluster admin account on that cluster, pins the TLS certificate the other cluster presents, and creates the replication credentials for both clusters. There are no S3 users or keys to manage for replication. To rotate the credentials of a link, contact the Customer Success Team.

**Before you begin**

* Complete the preparation on both clusters.
* Get the management hostname or IP address of the other cluster, and the username and password of a cluster admin on it.
* If the clusters can reach each other only through public addresses, contact the Customer Success Team before you link them.

The source cluster hosts the filesystem being replicated. The target cluster receives the replicated filesystem. This procedure runs the link command on the source cluster.

**Procedure**

1. On the **target cluster**, display the fingerprint of its TLS certificate, and record the colon-separated hexadecimal value:

```bash
weka security tls status
```

2. On the **source cluster**, link to the target cluster:

<pre class="language-bash"><code class="lang-bash"><strong>weka replication link add \
</strong>  --peer-host &#x3C;target management host> \
  --username &#x3C;cluster admin on the target> \
  --fingerprint &#x3C;target fingerprint>
</code></pre>

Provide the password with `--password`, with the `WEKA_PEER_PASSWORD` environment variable, or at the prompt. On success, the command reports the ID of the new link: `Cluster link added with ID <ID>.`

If you run the command interactively without `--fingerprint`, it displays the fingerprint the other cluster presents and asks you to confirm it. Compare it with the `weka security tls status` output from the other cluster before you continue.

If a link to the other cluster already exists on either cluster, the command reports which cluster holds it and makes no change. Remove the existing link, then add it again. See [Remove a replication link](#remove-a-replication-link).

**Parameters for weka replication link add**

<table><thead><tr><th width="266.5703125">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>--peer-host</code>*</td><td>Management hostname or IP address of the cluster to link to.</td></tr><tr><td><code>--username</code>*</td><td>Cluster admin username on the other cluster.</td></tr><tr><td><code>--password</code></td><td>Password for <code>--username</code>. Alternatively, use the <code>WEKA_PEER_PASSWORD</code> environment variable or the interactive prompt.</td></tr><tr><td><code>--fingerprint</code></td><td>Expected SHA-256 fingerprint of the TLS certificate of the other cluster, as shown by <code>weka security tls status</code> on that cluster. Required in non-interactive mode unless you use <code>--auto-accept-peer-cert</code>.</td></tr><tr><td><code>--auto-accept-peer-cert</code></td><td>Accepts the TLS certificate the other cluster presents, without confirming its fingerprint. The certificate is still pinned and checked on every later connection.</td></tr><tr><td><code>--peer-port</code></td><td>Management API port of the other cluster. Default: <code>14000</code>.</td></tr></tbody></table>

3. Verify the link on either cluster:

```bash
weka replication link
```

Confirm that **Connection** shows `connected` and **Pairing** shows `mutual`. Record the link **ID**. You use it to create replication pairs.

A link is identified by its ID. Its name follows the name of the other cluster, so two links can have the same name. Every `weka replication link` command, and `weka replication fs-pair add --link-id`, takes the ID.

### Refresh a replication link

Refresh a link when the other cluster changes its name, management addresses, or S3 endpoints. Both clusters replace what they hold about each other with what the other cluster now reports. The refresh runs over the existing authenticated link and requires no password.

**Procedure**

1. Refresh the link on either cluster:

```bash
weka replication link refresh <link ID> [--peer-host <hostname>] [--peer-port <port>]
```

Use `--peer-host` and `--peer-port` to reach the other cluster for this run when its recorded addresses no longer work. The command does not store these values.

If a different cluster answers at the recorded address, the refresh stops. Remove the link and add it again.

To confirm replication after you refresh a link, write data to a source filesystem and check that **Last Replication** advances in `weka replication fs-pair`.

## Create a replication pair

Create a replication pair between a local filesystem and a filesystem on a linked cluster, and define its replication policy.

The replication policy determines the replication interval, which paths are copied proactively, when snapshots become visible on the target, and how many snapshots are retained.

**Before you begin**

* Choose the copy option and access strategy for the workload. See [Asynchronous replication](https://docs.weka.io/6.0/weka-filesystems-and-object-stores/asynchronous-replication.md). For a full data copy, ensure the target capacity is at least the source filesystem size.
* Ensure the link to the target cluster shows **Connection** `connected` or `degraded` and **Pairing** `mutual` in `weka replication link`.
* Ensure the target cluster has a filesystem group with the same name as the group of the source filesystem. The target filesystem is created in that group.

**Procedure**

1. Create the replication pair on the source cluster:

```bash
weka replication fs-pair add \
  --source-filesystem <filesystem> \
  --link-id <link ID> \
  --target-filesystem <name> \
  --interval <duration> \
  [--copy-path <paths>] \
  [--access-strategy <INSTANT_ACCESS | COPY_FIRST>] \
  [--apply-strategy <AUTOMATIC>] \
  [--snapshots-to-keep <number>] \
  [--target-total-capacity <capacity>] \
  [--now]
```

The target filesystem is created automatically on the target cluster during the first replication cycle. It must not already exist. If it does, the command fails. On success, the command returns the new pair and its ID:

```
╭────┬──────────────────────────────────────────────────────╮
│ ✅ │ Added replication pair data_fs → data_fs_dest (3).   │
╰────┴──────────────────────────────────────────────────────╯
```

**Parameters**

The following parameters apply to `weka replication fs-pair add`. They do not apply to `weka replication fs-pair`.

<table><thead><tr><th width="266.5703125">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>--source-filesystem</code></td><td>Name of the local source filesystem.</td></tr><tr><td><code>--link-id</code></td><td>ID of the replication link to the target cluster, as shown by <code>weka replication link</code>.</td></tr><tr><td><code>--target-filesystem</code></td><td>Name of the filesystem on the remote cluster. It can be the same as the source filesystem name.</td></tr><tr><td><code>--interval</code></td><td>Replication interval, for example, <code>5m</code> or <code>1h</code>. From 5 minutes to 30 days.</td></tr><tr><td><code>--copy-path</code></td><td>Specifies up to 10 paths to copy proactively. Each path can be up to 1,023 characters long, and all paths together up to 2,048 characters. Separate paths with commas or repeat the option. Use <code>full</code>, <code>all</code>, or <code>/</code> to copy all data. Use <code>none</code> or <code>null</code> to replicate metadata only. Default: metadata-only replication. The command checks the syntax of each path, not whether it exists.</td></tr><tr><td><code>--access-strategy</code></td><td>Controls when a target snapshot becomes accessible. <code>INSTANT_ACCESS</code> (default) exposes the snapshot immediately. Data not copied locally is retrieved on demand. <code>COPY_FIRST</code> exposes the snapshot only after the <code>--copy-path</code> data is local.</td></tr><tr><td><code>--apply-strategy</code></td><td>Controls when the target applies a replicated snapshot. <code>AUTOMATIC</code> applies the snapshot after the prerequisite phase completes.</td></tr><tr><td><code>--snapshots-to-keep</code></td><td>Number of snapshots to retain, from 2 to 25. Default: 3. Retaining more snapshots requires more storage. Enforced only while the pair is running; see <a href="#pause-and-resume-replication">Pause and resume replication</a>.</td></tr><tr><td><code>--target-total-capacity</code></td><td>Total capacity for the target filesystem. Default: same as the source filesystem. A smaller target is allowed for a selective copy. A full copy requires at least the source size.</td></tr><tr><td><code>--now</code></td><td>Triggers the first replication cycle immediately instead of waiting one full interval.</td></tr></tbody></table>

**Examples**

Create a full data copy pair for disaster recovery over link 1, replicating every 5 minutes:

```bash
weka replication fs-pair add --source-filesystem data_fs --link-id 1 --target-filesystem data_fs_dest --interval 5m --copy-path full --snapshots-to-keep 25
```

Replicate only selected directories:

```bash
weka replication fs-pair add --source-filesystem data_fs3 --link-id 1 --target-filesystem data_fs3_dest --interval 5m --copy-path "/dir1,/dir2" --snapshots-to-keep 10
```

2. Verify that the pair is created and running:

```bash
weka replication fs-pair
```

## Monitor replication status

Monitor the state, progress, and health of replication pairs and cluster links.

### View replication pairs

List the replication pairs and their current status:

```bash
weka replication fs-pair
```

The output shows for each pair:

* **Role**: `SOURCE` for a pair this cluster drives, or `TARGET` for a pair that another cluster replicates into this one. A `TARGET` row is read-only. To change the pair, run the command on the source cluster.
* **State**: The overall state of the pair, such as `RUNNING` or `ERROR`.
* **Link**: The name of the linked cluster.
* **Interval, Apply, Access, Copy**: The configured replication policy.
* **Last Replication**: The timestamp of the last completed replication cycle. Use it to assess the current recovery point.
* **Current Status**: The phase of the running cycle, with its progress and snapshot, or `IDLE` between cycles. When a transfer phase stops progressing, the status ends with `(stuck: <reason>)`. For the `SNAPSHOT_CREATE` and `PRUNE_*` phases, the status shows the phase name only. Check **Last Error** for these phases.

Each cluster assigns its own pair IDs, so the same pair can have different IDs on the source and the target clusters.

**Replication cycle phases**

| Phase                                | Description                                                                                                                     |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
| `SNAPSHOT_CREATE`                    | The system takes a snapshot of the source filesystem.                                                                           |
| `METADATA_SYNC`                      | The source sends the snapshot metadata to the target.                                                                           |
| `METADATA_SYNC + COPY`               | The source sends metadata and data together. Applies to Copy first pairs with a full data copy.                                 |
| `COPY`                               | The target copies file data from the source: the whole filesystem for a full data copy, or the copy paths for a selective copy. |
| `APPLY`                              | The target applies the snapshot to the target filesystem.                                                                       |
| `PRUNE_AT_TARGET`, `PRUNE_AT_SOURCE` | The system deletes snapshots beyond the `--snapshots-to-keep` limit.                                                            |
| `IDLE`                               | No cycle is running.                                                                                                            |

For the pair UID, the link ID, the number of snapshots to keep, and the last error, run:

```bash
weka replication fs-pair -v
```

To customize the output, use the `--output`, `--filter`, and `--sort` options with any of the available columns, including `last-error` and `last-error-time` for troubleshooting.

**Last Error** records the cause of the most recent failure or stall, in the form `<CODE> on <ORIGIN> during <PHASE>`, for example, `PEER_UNREACHABLE on TARGET during COPY`. **Last Error Time** shows when it was recorded.

* The system records the first cause of a stall. If the cause changes while the pair stays stuck, **Last Error** keeps the first cause.
* A successful cycle clears **Last Error**. Resuming the pair keeps it. To tell whether the value describes the current cycle, compare **Last Error Time** with the time you resumed the pair.

### Monitor replication interval compliance

The replication interval sets the target for each pair. There is no separate setting. The system times each cycle from its start and raises an alert when the cycle overruns the interval by more than one minute. Replication alerts are raised only on the source cluster and are not propagated to the target cluster.

<table><thead><tr><th width="244.77734375">Alert</th><th width="91.3828125">Severity</th><th>Raised when the current cycle has been running longer than</th></tr></thead><tbody><tr><td>Replication behind schedule</td><td>Minor</td><td>The configured interval, plus one minute</td></tr><tr><td>Replication behind schedule</td><td>Major</td><td>Three times the configured interval, plus one minute</td></tr></tbody></table>

The alert names the filesystem and the linked cluster, and reports the interval, how long the cycle has run, by how much it exceeds the target, and the time of the last completed cycle. Use the alert history as the compliance record.

No replication interval alert is raised for a pair that is paused, not running, or still in its first replication cycle.

View active alerts:

```bash
weka alerts
```

If a replication interval alert fires, inspect the cycle:

```bash
weka replication fs-pair --verbose
```

Common causes are a slow or broken connection to the linked cluster, an S3 service problem on the linked cluster, or low free capacity on the target. Repair the connection or the S3 service, or free space. If the source change rate consistently outpaces the connection, lengthen the interval or add bandwidth.

### View replication link health

Check the connection and pairing status of each replication link. The status reflects the management endpoints of the other cluster. To check its S3 service, watch the **Current Status** of the pairs that use the link, and the replication interval alerts. If a pair shows `(stuck: OBS connectivity errors)` while the link shows **Connection** `connected`, contact the Customer Success Team.

**Procedure**

1. List the replication links:

```bash
weka replication link
```

To include the link UID, data IPs, and data port, add `-v`.

2. Check the **Connection** and **Pairing** columns:

<table><thead><tr><th width="114.98828125">Column</th><th width="134.7578125">Value</th><th>Meaning</th></tr></thead><tbody><tr><td>Connection</td><td><code>connected</code></td><td>Every management endpoint of the other cluster answers.</td></tr><tr><td>Connection</td><td><code>degraded</code></td><td>Some management endpoints of the other cluster answer.</td></tr><tr><td>Connection</td><td><code>disconnected</code></td><td>No management endpoint of the other cluster answers.</td></tr><tr><td>Pairing</td><td><code>mutual</code></td><td>The other cluster holds a matching link.</td></tr><tr><td>Pairing</td><td><code>local-only</code></td><td>The other cluster holds no matching link. To repair it, remove the link and add it again.</td></tr><tr><td>Pairing</td><td><code>unknown</code></td><td>No management endpoint of the other cluster answers, so the pairing can’t be determined.</td></tr></tbody></table>

### Troubleshoot replication

Use the **Current Status** and **Last Error** of a pair to identify the cause of a stall or an error, and the action that clears it.

**Stuck reasons**

A stuck pair stays in the `RUNNING` state and retries. After you fix the cause, the cycle continues without a resume.

| Current Status ends with                   | Action                                                                                                                                                                                                              |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `(stuck: OBS connectivity errors)`         | Check whether the target filesystem of any pair on this link is full. Check the S3 service of the target cluster and the network path between the clusters. If both are healthy, contact the Customer Success Team. |
| `(stuck: Target filesystem is full)`       | Increase the size of the target filesystem.                                                                                                                                                                         |
| `(stuck: Target cluster version mismatch)` | Upgrade the clusters to compatible versions.                                                                                                                                                                        |
| `(stuck: Corrupt transmissions)`           | Check the network path between the clusters.                                                                                                                                                                        |

**Last Error codes**

For a recoverable cause, such as an unreachable cluster, the system retries for about 30 minutes before it moves the pair to the `ERROR` state. After you fix the cause, resume the pair with `weka replication fs-pair resume <pair ID>`.

| Code                                                     | Action                                                                                                                                                                                       |
| -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PEER_UNREACHABLE`                                       | Restore the connection to the other cluster.                                                                                                                                                 |
| `PEER_DIVERGING_SNAPSHOT`                                | Delete the snapshot that was created manually on the target filesystem.                                                                                                                      |
| `UPGRADE_IN_PROGRESS`                                    | Wait for the upgrade of the target cluster to complete.                                                                                                                                      |
| `VERSION_MISMATCH`                                       | Upgrade the target cluster to the version of the source cluster or later.                                                                                                                    |
| `SNAPSHOT_DELETION_IN_PROGRESS`                          | Wait for the snapshot deletions on the filesystem to finish. If the pair's own snapshot deletions delay its cycles, lengthen the interval.                                                   |
| `FILE_COPY_FAILED`                                       | A file in a copy path could not be copied. Free space on the target filesystem, and check that the copy paths exist on the source. If the error persists, contact the Customer Success Team. |
| `FILESYSTEM_GROUP_NOT_FOUND`                             | Create a filesystem group on the target cluster with the same name as the group of the source filesystem.                                                                                    |
| `TASK_STUCK during PRUNE_AT_TARGET` or `PRUNE_AT_SOURCE` | The pair stays `RUNNING`. The cycle continues after the competing snapshot deletion or task finishes. To list running tasks, run `weka cluster task`.                                        |
| `RETRIES_EXHAUSTED`                                      | Contact the Customer Success Team.                                                                                                                                                           |

## Modify the replication policy

Modify the policy of an existing replication pair without recreating it. Run the commands on the source cluster.

**Before you begin**

Identify the replication pair ID:

```bash
weka replication fs-pair
```

**Procedure**

1. Update the pair policy:

```bash
weka replication fs-pair update <pair ID> \
  [--interval <duration>] \
  [--copy-path <paths> | --add-copy-path <paths> --remove-copy-path <paths>] \
  [--access-strategy <INSTANT_ACCESS | COPY_FIRST>] \
  [--apply-strategy <AUTOMATIC>] \
  [--snapshots-to-keep <number>]
```

Use the parameter descriptions in the preceding table, with the following additions:

* `--copy-path` replaces the entire copy path set. Use `none` or `null` to clear the set. Mutually exclusive with `--add-copy-path` and `--remove-copy-path`.
* `--add-copy-path` adds paths to the copy path set. Separate multiple paths with commas or repeat the option.
* `--remove-copy-path` removes paths from the copy path set. Separate multiple paths with commas or repeat the option.

2. Verify the updated policy:

```bash
weka replication fs-pair
```

**Example**

Switch a pair to Selective copy - metadata only and lengthen its interval:

```bash
weka replication fs-pair update 3 --copy-path none --interval 6m
```

## Pause and resume replication

Pause a replication pair before maintenance or before removing it, and resume it to continue replication.

While a pair is paused, no new snapshot deltas are transferred to the target. On-demand data access on the target continues to work.

{% hint style="info" %}
Snapshot pruning runs only while a pair is running. A paused pair, or one in an error state, retains all of its snapshots until it resumes, so the `--snapshots-to-keep` limit is not enforced during that time. Expect snapshot count and capacity use to grow while a pair is left paused or unattended in error.
{% endhint %}

### Pause a replication pair

**Procedure**

1. On the source cluster, pause the pair:

```bash
weka replication fs-pair pause <pair ID>
```

### Resume a replication pair

**Procedure**

1. On the source cluster, resume the pair:

```bash
weka replication fs-pair resume <pair ID>
```

Replication continues from the last consistent state.

## Manage file hydration on the target

Fetch the data of individual files to the target cluster proactively, monitor hydration progress, and release file data back to on-demand mode.

With a selective replication policy, file data is retrieved from the source cluster when files are accessed on the target. Use the hydration commands to control this behavior per file: fetch a file before a workload needs it, or release local data to free capacity on the target.

### Fetch a file proactively

Fetch a file's data blocks from the source cluster in the background:

```bash
weka replication fs-pair fetch <path> [--filesystem <name>] [--snapshot <name>]
```

You can run up to five fetch commands concurrently. A sixth concurrent fetch command fails.

* `<path>` is a local mount path. Alternatively, specify `--filesystem` and provide a path relative to the filesystem root.
* Use `--snapshot` to resolve the path within a snapshot view instead of the live filesystem.

To confirm that the data is on the target, run `weka replication fs-pair hydration status <path>` and check that the status is `Copied`. If the status is `Partial-Copy` or `Pointer` after the fetch completes, free space on the target filesystem and run the fetch again.

### Monitor hydration progress

Fetch and release operations run in the background. Check the hydration status for a file:

```bash
weka replication fs-pair hydration status <path> [--filesystem <name>] [--snapshot <name>]
```

The command reports one of these statuses:

| Status         | Description                                                                             |
| -------------- | --------------------------------------------------------------------------------------- |
| `Copied`       | All file data is on the target cluster.                                                 |
| `In-Progress`  | A fetch is running on the filesystem, and part of the file data is still on the source. |
| `Partial-Copy` | Part of the file data is on the target. The rest is read from the source when accessed. |
| `Pointer`      | The file data is on the source only, and is read from the source when accessed.         |

### Release file data

Return a file's data blocks to on-demand mode to free capacity on the target. The release runs in the background:

```bash
weka replication fs-pair release <path> [<path> ...]
```

By default, background dehydration starts when the target filesystem uses 95% of its SSD capacity. It stops when usage falls to 90%.

Each `<path>` is a local mount path. Release several files in one command by listing more than one path. Monitor progress with `weka replication fs-pair hydration status`.

This command runs on the container that holds the mount, so it cannot be directed at another cluster with `--HOST`.

## Remove a replication pair

Remove a replication pair when you no longer need to synchronize the source and target filesystems, or as part of a failover procedure.

{% hint style="warning" %}
Removing a pair stops further snapshot replication for the filesystem pair, including the retrieval of data on demand. If the target filesystem was created with a selective replication policy, files that were never hydrated become inaccessible after removal. Before removal, run `weka replication fs-pair hydration status` on the files you need, and confirm that each shows `Copied`. Pause the pair, then run `weka replication fs-pair` and confirm that **Current Status** shows `IDLE`.
{% endhint %}

**Before you begin**

Identify the replication pair ID on the source cluster:

```bash
weka replication fs-pair
```

**Procedure**

1. On the **source** cluster, pause the replication pair. A running pair cannot be removed. A pair in the `ERROR` state can be removed without pausing it:

```bash
weka replication fs-pair pause <pair ID>
```

Wait until the pair finishes its current cycle and **Current Status** shows `IDLE`.

2. Remove the pair:

```bash
weka replication fs-pair remove <pair ID> [--force]
```

The command removes the pair from both clusters, and reports success only after both are done. It prompts for confirmation. Use `--force` to skip the confirmation prompt, for example, in scripts.

If the target cluster is unreachable, the removal fails and the pair stays on both clusters. Retry when the target is reachable again. To remove the pair from the source cluster only, add `--local-only`, and then remove it on the target cluster as well.

3. Verify that the pair is no longer listed:

```bash
weka replication fs-pair
```

The target filesystem remains write-protected after the pair is removed. To make it writable, run this command on the target cluster:

```bash
weka fs update <target filesystem> --access rw
```

Do this only when you intend to promote the target, such as during failover. See [Activate the target cluster during failover](#activate-the-target-cluster-during-failover).

### Remove a pair on the target cluster

On the target cluster, `weka replication fs-pair remove` removes an incoming pair (role `TARGET`) only with `--local-only`. Use it when the source cluster is unavailable. The pair does not need to be paused:

```bash
weka replication fs-pair remove <pair ID on the target> --local-only
```

If the source cluster is still running, its next replication cycle for the pair moves to the error state. Remove the pair on the source cluster as well.

## Remove a replication link

Remove a replication link when replication between the two clusters is no longer required. One command removes the link from both clusters.

**Before you begin**

Remove every replication pair that uses the link, in both directions. See [Remove a replication pair](#remove-a-replication-pair). Identify the link ID:

```bash
weka replication link
```

**Procedure**

1. Remove the link on either cluster:

```bash
weka replication link remove <link ID> [--force]
```

The command prompts for confirmation. Use `--force` to skip the confirmation prompt.

2. Verify on both clusters that the link is gone:

```bash
weka replication link
```

If the command reports that the link is in use, a replication pair or a task still uses it, or a filesystem on this cluster still receives a replica over it. Remove the remaining pairs, or wait for the task to finish, and run the command again.

To remove the link on this cluster only, add `--local-only`. Use it when the other cluster is unreachable or no longer exists, or when the link exists on only one of the clusters. `--local-only` also removes a link that a target filesystem still receives a replica over.

Removing a link does not restore write access to the target filesystems. They stay write-protected until you promote them, as described in [Activate the target cluster during failover](#activate-the-target-cluster-during-failover).

## Activate the target cluster during failover

Activate the target filesystem when the source cluster becomes unavailable, and redirect clients to it.

{% hint style="warning" %}
Failover is a manual procedure. Because replication is asynchronous, the target reflects the last replicated snapshot. Writes made on the source after that snapshot are lost. Except for `COPY_FIRST` pairs with a full data copy, the target can reference data that exists only on the source. Removing the pair makes that data permanently unreadable.
{% endhint %}

**Before you begin**

* Confirm the recovery point. On the source cluster, if it is still reachable, check the **Last Replication** timestamp:

```bash
weka replication fs-pair
```

* Confirm that the target contains the data you need:
  * With `COPY_FIRST` and `--copy-path full`, all data is local. No action is required.
  * Hydrate data before breaking any other pair type. This includes `INSTANT_ACCESS`, Selective copy - critical paths pairs, and smaller targets.
  * Run `weka replication fs-pair fetch <path>` for each path you need on the target, and confirm that `weka replication fs-pair hydration status <path>` shows `Copied` for each one. Reading a file on the target returns its data, and does not always store it on the target. See [Manage file hydration on the target](#manage-file-hydration-on-the-target).

{% hint style="info" %}
On the target cluster, `weka replication fs-pair` lists the pair with the role `TARGET`. Its pair ID on the target can differ from its ID on the source.
{% endhint %}

**Procedure**

The steps differ depending on whether the source cluster is still reachable.

*Planned failover, source cluster reachable:*

1. On the **source** cluster, pause the pair, wait until **Current Status** shows `IDLE`, and remove it. The removal clears the pair on both clusters:

```bash
weka replication fs-pair pause <pair ID>
weka replication fs-pair remove <pair ID>
```

*Disaster failover, source cluster unavailable:*

1. On the **target** cluster, remove the incoming pair. The source is unreachable, so the pair is removed on the target only:

```bash
weka replication fs-pair remove <pair ID on the target> --local-only
```

*Both cases, to promote the target:*

2. Remove write protection from the target filesystem. On the target cluster:

```bash
weka fs update <name> --access rw
```

3. Mount the filesystem on a client and verify the data before redirecting production traffic to it.

When the source cluster returns, its pair moves to the error state on the next cycle. Remove the pair on the source cluster with `weka replication fs-pair remove <pair ID>`.

## Fail back to the original source cluster

After a failover, return replication and clients to the original source cluster by creating a replication pair in the reverse direction, from the former target to the original source. Keep the cluster link: one link carries replication in both directions. The new pair starts with a full first replication cycle.

**Before you begin**

* Confirm that the original source cluster is running. On the former target cluster, run `weka replication link` and check that the link shows **Connection** `connected` or `degraded` and **Pairing** `mutual`.
* After a disaster failover, the original source cluster still lists the old pair, in the `ERROR` state. Remove it on that cluster with `weka replication fs-pair remove <pair ID>`. A pair in the `ERROR` state does not need to be paused. After a planned failover, the pair is already removed from both clusters.
* Choose the filesystem name on the original source cluster. The replication pair creates this filesystem in its first cycle, so the name must not exist there. Use a new name, or remove the original filesystem first.

**Procedure**

1. On the former target cluster, identify the link ID:

```bash
weka replication link
```

2. On the former target cluster, create the pair. Use a full data copy with `COPY_FIRST`, so all data is local on the original source cluster before you move clients back:

```bash
weka replication fs-pair add --source-filesystem <name> --link-id <link ID> --target-filesystem <name on the original source> --interval 5m --copy-path full --access-strategy COPY_FIRST
```

3. Wait until **Last Replication** shows a completed cycle:

```bash
weka replication fs-pair
```

4. Move clients back by following [Activate the target cluster during failover](#activate-the-target-cluster-during-failover), with the original source cluster as the target.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.weka.io/6.0/weka-filesystems-and-object-stores/asynchronous-replication/manage-asynchronous-replication.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
