Links

List of statistics

This page lists all the statistics generated by the Weka system, according to category.

Attribute Cache

Type
Description
Units
GP_GETATTR_CACHE_MISS
Number of general purpose getAttr cache misses per second
Ops/Sec
GP_GETATTR
Number of general purpose getAttr calls per second
Ops/Sec

Block Cache

Type
Description
Units
BUCKET_CACHED_METADATA_BLOCKS
Bucket number of cached metadata blocks
Blocks
BUCKET_CACHED_REGISTRY_L2_BLOCKS
Bucket number of cached registry L2 blocks
Blocks
BUCKET_CACHE_METADATA_HITS
Bucket block cache metadata hits
Queries
BUCKET_CACHE_METADATA_MISSES
Bucket block cache metadata misses
Queries
BUCKET_CACHE_REGISTRY_L2_HITS
Bucket block cache registry L2 hits
Queries
BUCKET_CACHE_REGISTRY_L2_MISSES
Bucket block cache registry L2 misses
Queries
BUCKET_REGISTRY_L2_BLOCKS_NUM
Bucket number of registry L2 blocks
Blocks

Block Writes

Type
Description
Units
BLOCK_FULL_WRITES
Full block writes
Writes
BLOCK_PARTIAL_WRITES
Partial block writes
Writes

Bucket

Type
Description
Units
BUDGET_UNDERFLOW_BLOCKS
Blocks/Sec
CHOKING_LEVEL_ALL
Throttling level applied on all types of IOs
%
CHOKING_LEVEL_NON_MUTATING
Throttling level applied on non-mutating only types of IOs
%
DESTAGED_BLOCKS_COUNT
Blocks/Sec
DESTAGE_COUNT
Destages/Sec
DIR_MOVE_TIME
Ops
EXTENT_BLOCKS_COUNT
Blocks
FREEABLE_LRU_BUFFERS
Buffers
HASH_BLOCKS_COUNT
Blocks
INODE_BLOCKS_COUNT
Blocks
INODE_REFRESHER_QUEUE_LENGTH
Items
JOURNAL_BLOCKS_COUNT
Blocks
JOURNAL_ITERATIONS
Histogram of number of batches of stripes committed in a single request
READS
Number of read operations per second
Ops/Sec
READ_BYTES
Number of bytes read per second
Bytes/Sec
READ_LATENCY
Average latency of READ operations
Microseconds
REGISTRY_L1_BLOCKS_COUNT
Blocks
REGISTRY_L2_BLOCKS_COUNT
Blocks
REGISTRY_SEARCHES_COUNT
Queries/Sec
RESIDENT_BLOCKS_COUNT
Blocks/Sec
SNAPSHOT_CREATION_TIME
Snaps
SPATIAL_SQUELCH_BLOCKS_COUNT
Blocks
SUCCESSFUL_DATA_WEDGINGS
Attempts/Sec
SUPERBLOCK_BLOCKS_COUNT
Blocks
TAKEOVERS_SUCCESSFUL
Takeover Attempts/Sec
TAKEOVER_ATTEMPTS
Takeover Attempts/Sec
TEMPORAL_SQUELCH_BLOCKS_COUNT
Blocks
UNSUCCESSFUL_DATA_WEDGINGS
Attempts/Sec
USER_DATA_BUFFERS_IN_USE
Buffers
WRITES
Number of write operations per second
Ops/Sec
WRITE_BYTES
Number of byte writes per second
Bytes/Sec
WRITE_LATENCY
Average latency of WRITE operations
Microseconds

Bucket Failovers

Type
Description
Units
BUCKET_FAILOVERS
Amount of times swapping from a remote primary node to a secondary
Failovers
INVALID_BUCKET_TERM
Number of times a remote bucket rejected a request becaue the term was invalid
Exceptions
REMOTE_BUCKET_IS_SECONDARY
Number of times a remote bucket reported it is secondary and cannot serve us
Exceptions

Bucket Rebalances

Type
Description
Units
BUCKET_INITS
Number of bucket initializations
Times
BUCKET_INIT_LATENCY_HIST
Milliseconds
BUCKET_INIT_LATENCY
Average latency of bucket initialization
Seconds

CPU

Type
Description
Units
CPU_UTILIZATION
Percentage of the CPU time utilized for handling I/Os
%

Chocking

Type
Description
Units
CHOKING_LEVEL_ALL
Throttling level applied on all types of IOs, both mutating and non-mutating
CHOKING_LEVEL_NON_MUTATING
Throttling level applied on non-mutating only types of IOs

Clients

Type
Description
Units
CLIENTS_CONNECTED
Clients connected
Clients/Sec
CLIENTS_DISCONNECTED
Clients left or were removed
Clients/Sec
CLIENTS_LEFT
Clients left
Clients/Sec
CLIENTS_RECONNECTED
Clients reconnected instead of an old instance of theirs
Clients/Sec
CLIENTS_REMOVED
Clients removed
Clients/Sec

Config

Type
Description
Units
AVERAGE_CHANGES_IN_CHANGESET
Average changes in changeset
Changes/Sec
AVERAGE_CHANGES_IN_GENERATION
Average changes in generation
Changes/Sec
BACKEND_NODE_REJOIN_TIME
Milliseconds
CHANGESET_COMMIT_LATENCY
Average latency of committing a config changeset
Microseconds
CLIENT_NODE_REJOIN_TIME
Milliseconds
GENERATION_COMMIT_LATENCY
Average latency of committing a config generation
Microseconds
HEARTBEAT_PROCESSING_TIME_OLD
Seconds
HEARTBEAT_PROCESSING_TIME
Seconds
LEADER_HEARTBEAT_PROCESSING_TIME_OLD
Seconds
LEADER_HEARTBEAT_PROCESSING_TIME
Seconds
TOTAL_CHANGESETS_COMMITTED
Total number of changesets committed
Change Sets
TOTAL_COMMITTED_CHANGES
Total number of config changes committed
Changes
TOTAL_GENERATIONS_COMMITTED
Number of generations committed per second
Generations

Filesystem OBS

Type
Description
Units
BACKPRESSURED_BUCKETS_IN_FSS
Number of backpressured buckets
Buckets
CONCURRENT_DEMOTES
How many demotes are executed concurrently
Demotes
DEMOTE_EXTENT_OBS_FETCH_BACKPRESSURE
Number of extent BACKPRESSURE fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH_IMMEDIATE_RELEASE
Number of extent IMMEDIATE_RELEASE fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH_MANHOLE
Number of extent MANHOLE fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH_MIGRATE
Number of extent MIGRATE fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH_POLICY
Number of extent POLICY fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH_RECLAMATION_REUPLOAD
Number of extent RECLAMATION_REUPLOAD fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH_STOW
Number of extent STOW fetch operations per second
Ops/Sec
DEMOTE_EXTENT_OBS_FETCH
Number of extent fetch operations per second
Ops/Sec
DEMOTE_WAITING_FOR_SLOT
Average time waiting for a demotion concurrency slot
Microseconds
DESERIALIZED_EXTENTS_WITH_INVALID_BLOBS
Number of deserialized extents with invalid blob id
Extents
DOWNLOADS
Number of promotes operations per second
Ops/Sec
DOWNLOAD_LATENCY
Latency of promote operations
Microseconds
FAILED_DOWNLOADS
Number of failed promotes operations per second
Ops/Sec
FAILED_UPLOADS
Number of failed demotes operations per second
Ops/Sec
OBS_4K_IOPS_READ
Number of object store dedicated 4K read operations per second
Ops/Sec
OBS_BACKPRESSURE_FREED
Number of bytes freed from disk due to backpressure
Bytes/Sec
OBS_BLOB_HEADER_DOWNLOAD_LATENCY
Average latency of blob header download
Microseconds
OBS_BLOB_SCAVENGE_LATENCY
Average latency of blob scavenges
Microseconds
OBS_BLOB_TIERING_DURATION
Milliseconds
OBS_COMPLETELY_ALIVE_BLOBS
Percentage of blobs with only live extents linked to them
%
OBS_COMPLETELY_DEAD_BLOBS
Percentage of blobs with no live extent linked to them
%
OBS_EXTENTS_PREFETCH
Number of pre-fetched extents
Ops/Sec
OBS_FREED
Number of bytes freed from disk because they are in the OBS
Bytes/Sec
OBS_IMMEDIATE_RELEASE_FREED
Number of bytes freed from disk due to immediate release
Bytes/Sec
OBS_INODES_PREFETCH
Number of pre-fetched inodes
Ops/Sec
OBS_INODES_RELEASE
Number of pre-fetched inodes
Ops/Sec
OBS_ONGOING_RECLAMATIONS
Number of ongoing reclamations
Ops
OBS_POLICY_FREED
Number of bytes freed from disk due to policy
Bytes/Sec
OBS_PROMOTE_EXTENT_WRITE_LATENCY
Microseconds
OBS_PROMOTE_EXTENT_WRITE
Ops/Sec
OBS_PROMOTE_WRITE
Bytes/Sec
OBS_READ
Reads that needed data from the OBS
Ops/Sec
OBS_RECLAMATION_PURGED_BYTES
Number of bytes purged per second
Bytes/Sec
OBS_RECLAMATION_SCAVENGED_BLOBS
Number of blobs scavenged per second
Ops/Sec
OBS_RECLAMATION_SCAVENGED_BYTES
Number of bytes scavenged per second
Bytes/Sec
OBS_RECLAMATION_WAIT_FOR_DESTAGE
Average time waiting for destage on space reclamation
Microseconds
OBS_RELOC_DOWNLOAD
Number of relocation blobs downloaded per second
Ops/Sec
OBS_RELOC_UPLOAD
Number of relocation blobs uploaded per second
Ops/Sec
OBS_SCAVENGED_BLOB_WASTE_LEVEL
Histogram of waste level found in blobs
OBS_SHARED_DOWNLOADS_LATENCY
Microseconds
OBS_SHARED_DOWNLOADS
Ops/Sec
OBS_TRUNCATE
Truncates that needed data from the OBS
Ops/Sec
OBS_UNEXPECTED_TAG_ON_DOWNLOAD
Unexpected tag when downloading an extent
Occurences
OBS_WRITE
Writes that needed data from the OBS
Ops/Sec
STOW_SERIALIZED_EXTENT_DATA
Number of extent descriptors uploaded that contain data
Extent descriptors
STOW_SERIALIZED_EXTENT_DESCS
Number of extent descriptors uploaded
Extent descriptors
STOW_SERIALIZED_EXTENT_REDIRECTS
Number of extent descriptors uploaded that redirect to previous snapshot
Extent descriptors
TIERED_FS_BREAKING_POLICY
Tiered Filesystem Breaking Policy Counter
Activations
TIMEOUT_DOWNLOADS
Number of timeout'ed promotes operations per second
Ops/Sec
TIMEOUT_OPERATIONS
Total timeouted operations per second
Ops/Sec
TIMEOUT_UPLOADS
Number of timeout'ed demotes operations per second
Ops/Sec
UNEXPECTED_BLOCK_VERSION_POST_UPGRADE
Unexpected block version after upgrade completed
Occurences
UNEXPECTED_HASHBLOCK_KV_VERSION_POST_UPGRADE
Unexpected hash block KV version after upgrade completed
Occurences
UPLOADS
Number of upload attempts per second
Ops/Sec
UPLOAD_CHOKING_LATENCY
Average latency of waiting for demote choking budget
Microseconds
UPLOAD_LATENCY
Latency of demote
Microseconds

Frontend

Type
Description
Units
FE_IDLE_CYCLES
Cycles/Sec
FE_IDLE_TIME
Percentage of the CPU time not utilized for handling I/Os on frontend
%

Frontend Encryption

Type
Description
Units
FE_BLOCKS_DECRYPTED
Number of blocks decrypted in the frontend
Blocks
FE_BLOCKS_ENCRYPTED
Number of blocks encrypted in the frontend
Blocks
FE_BLOCK_CRYPTO_LATENCY
Average latency of frontend block crypto
Microseconds
FE_BLOCK_DECRYPT_DURATION
Duration of decryption of blocks in the frontend
Microseconds
FE_BLOCK_ENCRYPT_DURATION
Duration of encryption of blocks in the frontend
Microseconds
FE_FILENAMES_DECRYPTED
Number of filenames decrypted in the frontend
Filenames
FE_FILENAMES_ENCRYPTED
Number of filenames encrypted in the frontend
Filenames
FE_FILENAME_CRYPTO_LATENCY
Average latency of frontend filename crypto
Microseconds
FE_FILENAME_DECRYPT_DURATION
Duration of decryption of filenames in the frontend
Microseconds
FE_FILENAME_ENCRYPT_DURATION
Duration of encryption of filenames in the frontend
Microseconds

Garbage Collection

Type
Description
Units
GC_FREE_SIZE_AFTER_SCAN
GC pool size after the scan ends
Bytes
GC_FREE_SIZE_BEFORE_SCAN
GC pool size before the scan starts
Bytes
GC_SCAN_TIME
GC scan time
Msec
GC_USED_SIZE_AFTER_SCAN
GC used size after the scan ends
Bytes
GC_USED_SIZE_BEFORE_SCAN
GC used size before the scan starts
Bytes

JRPC

Type
Description
Units
JRPC_SERVER_PROCESSING_AVG
Microseconds
JRPC_SERVER_PROCESSING_TIME

Journal

Type
Description
Units
JOURNAL_CURRENT_OPS
Operations currently in journal
Journal Entries
JOURNAL_OPS_IN
Operations added to the journal
Journal Entries/Sec
JOURNAL_OPS_OUT
Operations removed from the journal
Journal Entries/Sec

Memory

Type
Description
Units
RSS_CURRENT
MB
RSS_PEAK
MB

Network

Type
Description
Units
BAD_RECV_CSUM
Number of packets received with a bad checksum
Packets/Sec
CORRUPT_PACKETS
Number of packets received and deemed corrupted
Packets/Sec
DOUBLY_RECEIVED_PACKETS
Number of packets that were received multiple times
Packets/Sec
DROPPED_LARGE_PACKETS
Number of large packets dropped in the socket backend
Packets/Sec
DROPPED_PACKETS
Number of packets received that we dropped
Packets/Sec
ECN_ENCOUNTERED
Number of ECN Encountered packets
Packets/Sec
FAULT_RECV_DELAYED_PACKETS
Number of received packets delayed due to a fault injection
Packets/Sec
FAULT_RECV_DROPPED_PACKETS
Number of received packets dropped due to a fault injection
Packets/Sec
FAULT_SENT_DELAYED_PACKETS
Number of sent packets delayed due to a fault injection
Packets/Sec
FAULT_SENT_DROPPED_PACKETS
Number of sent packets dropped due to a fault injection
Packets/Sec
GOODPUT_RX_RATIO
Percentage of goodput RX packets out of total data packets received
%
GOODPUT_TX_RATIO
Percentage of goodput TX packets out of total data packets sent
%
GW_MAC_RESOLVE_FAILURES
Number of times we failed to ARP resolve the gateway IP
Failures
GW_MAC_RESOLVE_SUCCESSES
Number of times we succeeded to ARP resolve the gateway IP
Successes
NODE_RECONNECTED
Number of reconnections
Reconnects/Sec
PACKETS_PUMPED
Number of packets received in each call to recvPackets
PEER_RTT
RTT per peer node
Microseconds
PORT_RX_BYTES
Number of bytes received
Bytes/Sec
PORT_RX_PACKETS
Number of packets received
Packets/Sec
PORT_TX_BYTES
Number of bytes transmitted
Bytes/Sec
PORT_TX_PACKETS
Number of packets transmitted
Packets/Sec
PUMPS_TXQ_FULL
Number of times we couldn't send any new packets to the NIC queue
Pumps/Sec
PUMPS_TXQ_PARTIAL
Number of times we only sent some of our queued packets to the NIC queue
Pumps/Sec
PUMP_DURATION
Duration of each pump
PUMP_INTERVAL
Interval between pumps
RDMA_ADD_CHUNK_FAILURES
Failures/Sec
RDMA_BINDING_FAILOVERS
Fail-overs/Sec
RDMA_CANCELED_COMPLETIONS
Completions/Sec
RDMA_CLIENT_BINDING_INVALIDATIONS
Invalidations/Sec
RDMA_COMPLETIONS
Completions/Sec
RDMA_COMP_DURATION
RDMA_COMP_FAILURES
Failures/Sec
RDMA_COMP_LATENCY
Average time of RDMA requests completion
Microseconds
RDMA_NET_ERR_RETRY_EXCEEDED
Occurences/Sec
RDMA_POOL_ALLOC_FAILED
Failures/Sec
RDMA_POOL_LOW_CAPACITY
Failures/Sec
RDMA_PORT_WAITING_FIBERS
Waiting fibers
RDMA_REQUESTS
Requests/Sec
RDMA_RX_BYTES
Bytes/Sec
RDMA_SERVER_BINDING_RESTARTS
Restarts/Sec
RDMA_SUBMIT_FAILURES
Failures/Sec
RDMA_SUBMIT_TIMEOUTS
Timeouts/Sec
RDMA_TX_BYTES
Bytes/Sec
RECEIVED_CONTROL_PACKETS
Number of received control packets
Packets/Sec
RECEIVED_DATA_PACKETS
Number of received data packets
Packets/Sec
RECEIVED_PACKETS
Number of packets received
Packets/Sec
RECEIVED_PACKET_GENERATIONS
The generation ("resend count") of the first incarnation of the packet seen by the receiver (indicates packet loss)
REORDERED_PACKETS
Number of reordered packets
Packets/Sec
RESEND_BATCH_SIZE
Number of packets sent in a resend batch
RESENT_DATA_PACKETS
Number of data packets resent
Packets/Sec
SEND_QUEUE_TIMEOUTS
Number of packets cancelled due to envelope timeout and were not in the send window
Packets/Sec
SEND_WINDOW_TIMEOUTS
Number of packets cancelled due to envelope timeout while in the send window
Packets/Sec
SENT_ACKS
Number of ACK packets sent
Packets/Sec
SENT_CONTROL_PACKETS
Number of control packets sent
Packets/Sec
SENT_DATA_PACKETS
Number of data packets sent
Packets/Sec
SENT_PACKETS
Number of sent packets
Packets/Sec
SENT_REJECTS
Number of rejects sent
Packets/Sec
SHORT_CIRCUIT_SENDS
Number of packets sent to the same node
Packets/Sec
SLOW_PATH_CSUM
Number of packets that went through checksum calculation on the CPU
Packets/Sec
TIMELY_RESENDS
Number of packets resent due to timely resend
Packets/Sec
TIME_TO_ACK
Histogram of time to ack a data packet
TIME_TO_FIRST_SEND
Time from queueing to first send
UCX_SEND_CB
Packets/Sec
UCX_SEND_ERROR
Packets/Sec
UCX_SENT_PACKETS_ASYNC
Packets/Sec
UCX_SENT_PACKETS_IMMEDIATE
Packets/Sec
UCX_TXQ_FULL
Packets/Sec
UDP_SENDMSG_FAILED_EAGAIN
Number of packets that failed to be sent on the socket backend with EAGAIN
Packets/Sec
UDP_SENDMSG_FAILED_OTHER
Number of packets that failed to be sent on the socket backend with an unknown error
Packets/Sec
UDP_SENDMSG_PARTIAL_SEND
Number of packets that we failed to send but in the same pump some packets were sent
Packets/Sec
UNACKED_RESENDS