System congestion
This page describes possible congestion issues in the WEKA system.
Last updated
This page describes possible congestion issues in the WEKA system.
Last updated
The WEKA system is designed for efficiency, delivering maximum performance and fully utilizing network links. However, in certain situations, the system may slow down I/O operations or even block new I/Os if specific limits are reached, until the congested resource is alleviated.
While these situations are often temporary and may resolve themselves quickly, persistent congestion can indicate an underlying issue, such as a workload overwhelming the cluster's resources. In such cases, expanding the cluster's resources, as detailed in Expand and shrink cluster resources. For further assistance, contact the Customer Success Team.
The WEKA system can issue several types of congestion events and alerts:
Type | Description | Actions |
---|---|---|
FIBERS
Extreme load of concurrent system operations on a process.
This is typically a transient condition caused by system load. If the load is persistent, consider adding more resources, such as servers or cores. See Add a backend server or for guidance.
DESTAGER
Excessive pending I/O operations waiting to be written for a specific process.
This situation is often temporary due to system load. If the condition persists, increase the number of servers in the cluster.
See Add a backend server or Expand specific resources of a container for guidance.
SSD
An excessive number of pending I/O operations to the SSD.
If there is only one SSD, it may be faulty and require replacement. If multiple SSDs are involved, the system load is too high.
To manage the load, add more SSDs to the system. See Expand specific resources of a container for guidance.
RAID_NOT_OK
I/O failures exceed the system's handling capacity, and I/Os cannot be processed.
Ensure all servers are operational. If any server is down, bring it up. If all servers are active and the issue persists, contact the Customer Success Team.
XDESTAGE
Auxiliary cluster resources are low
This is usually a temporary condition due to system load. If the problem persists, add more servers to the cluster.
See Add a backend server for guidance or contact the Customer Success Team.