Tribblix: manual page: cmap

CMAP_KEYS(8) Corosync Cluster Engine Programmer's Manual CMAP_KEYS(8)

NAME

cmap_keys - Overview of keys stored in the Configuration Map

OVERVIEW

There are 3 main types of keys stored in CMAP:

* Mapping of values stored in the config file.

* Runtime statistics.

* Other user created values.

In this man page, wild-cards have the usual meaning.

KEYS

internal_configuration.*
Internal configuration data. All keys in this prefix are read
only. It's only useful for getting a list of loaded services.

logging.*
Values read from the configuration file. It's possible to
change them at runtime. If subsystem specific configuration
is needed, the key must be in the form
logging.logger_subsys.SERVICE.key, where SERVICE is upper case
name of the service and key is same as in the configuration
file. All values are of string type.

nodelist.*
Values read from the configuration file. Each node element in
the configuration file gets assigned it's position starting
from zero. So the first node from the config file has
nodelist.node.0. prefix. To be a valid entry, each node must
have ring0_addr key. To change the nodeid key, use a u32 data
type.

Local node position is stored in local_node_pos key (RO), so
it's easy to find out nodeid/ring addresses of the local node
directly from cmap.

runtime.blackbox.*
Trigger keys for storing fplay data. It's recommended that you
the corosync-blackbox command to change keys in this prefix.

runtime.connections.*
There is information about total number of active connections
in given moment in the active key, number of closed
connections during whole runtime of corosync in the closed key
and information about each active IPC connection. All keys in
this prefix are read-only.

runtime.connections.ID.*
Each IPC connection has a unique ID. This is in the form
[[short_name:][PID:]internal_id. On some platforms, short_name
and PID are not filled and only internal_id is used.

Typical keys in this prefix are:

client_pid containing PID of IPC connection (unavailable on
some platforms).

dispatched number of dispatched messages.

invalid_request number of requests made by IPC which are
invalid (calling non-existing call, ...).

name contains short name of the IPC connection (unavailable on
some platforms).

overload is number of requests which were not processed
because of overload.

queue_size contains the number of messages in the queue
waiting for send.

recv_retries is the total number of interrupted receives.

requests contains the number of requests made by IPC.

responses is the number of responses sent to the IPC client.

send_retries contains the total number of interrupted sends.

service_id contains the ID of service which the IPC is
connected to.

runtime.config.*
Contains the values actually in use by the totem membership
protocol. Values here are either taken from the Corosync
configuration file, defaults or computed from entries in the
config file. For information on individual keys please refer
to the man page corosync.conf(5).

runtime.services.*
Prefix with statistics for service engines. Each service has
it's own service_id key in the prefix with the name
runtime.services.SERVICE., where SERVICE is the lower case
name of the service. Inside the service prefix is the number
of messages received and sent by the corosync engine in the
format runtime.services.SERVICE.EXEC_CALL.rx and
runtime.services.SERVICE.EXEC_CALL.tx, where EXEC_CALL is the
internal id of the service call (so for example 3 in cpg
service is receive of multicast message from other nodes).

runtime.totem.pg.mrp.srp.*
Prefix containing statistics about totem. All keys here are
read only. Typical key prefixes:

commit_entered Number of times the processor entered COMMIT
state.

commit_token_lost Number of times the processor lost token in
COMMIT state.

consensus_timeouts How many times the processor timed out
forming a consensus about membership.

continuous_gather How many times the processor was not able to
reach consensus.

firewall_enabled_or_nic_failure Set to 1 when processor was
not able to reach consensus for long time. The usual reason is
a badly configured firewall or connection failure.

gather_entered Number of times the processor entered GATHER
state.

gather_token_lost Number of times the processor lost token in
GATHER state.

mcast_retx Number of retransmitted messages.

mcast_rx Number of received multicast messages.

mcast_tx Number of transmitted multicast messages.

memb_commit_token_rx Number of received commit tokens.

memb_commit_token_tx Number of transmitted commit tokens.

memb_join_rx Number of received join messages.

memb_join_tx Number of transmitted join messages.

memb_merge_detect_rx Number of received member merge messages.

memb_merge_detect_tx Number of transmitted member merge
messages.

orf_token_rx Number of received orf tokens.

orf_token_tx Number of transmitted orf tokens.

recovery_entered Number of times the processor entered
recovery.

recovery_token_lost Number of times the token was lost in
recovery state.

rx_msg_dropped Number of received messages which were dropped
because they were not expected (as example multicast message
in commit state).

token_hold_cancel_rx Number of received token hold cancel
messages.

token_hold_cancel_tx Number of transmitted token hold cancel
messages.

mtt_rx_token Mean transit time of token in milliseconds. In
other words, time between two consecutive token receives.

avg_token_workload Average time in milliseconds of holding
time of token on the current processor.

avg_backlog_calc Average number of not yet sent messages on
the current processor.

runtime.totem.pg.mrp.srp.members.*
Prefix containing members of the totem single ring protocol.
Each member keys has format
runtime.totem.pg.mrp.srp.members.NODEID.KEY, where key is one
of:

ip IP address of member. It's stored in format r(RING_ID)
ip(IP_ADDRESS).

join_count Number of times the processor joined membership
with local cluster. When processor fails and rejoins again,
this value is incremented.

status Status of the processor. Can be one of joined and left.

config_version Config version of the member node.

runtime.schedmiss.*
If corosync is not scheduled after the required period of time
it will log this event and also write an entry to cmap under
following keys:

timestamp The timestamp of the last time when corosync failed
to be scheduled for the required amount of time. The time is
milli-seconds since the epoch.

delay The amount of time (milliseconds as a float) that
corosync was delayed.

resources.process.PID.*
Prefix created by applications using SAM with CMAP
integration. It contains the following keys:

recovery Recovery policy of the process. Can be one of quit or
restart.

poll_period Value passed in sam_initialize as a time_interval.

last_updated Last time SAM received a heartbeat from the
client.

state State of the client. Can be one of failed, stopped,
running and waiting for quorum.

uidgid.*
Information about users/groups which are allowed to make IPC
connections to corosync. Entries loaded from configuration
file are stored with uidgid.config.* prefix and are pruned on
configuration file reload. Dynamic entries has uidgid.* prefix
and a configuration file reload doesn't affect them.

quorum.cancel_wait_for_all
Tells votequorum to cancel waiting for all nodes at cluster
startup. Can be used to unblock quorum if notes are known to
be down. For pcs use only.

config.reload_in_progress
This value will be set to 1 (or created) when a corosync.conf
reload is started, and set to 0 when the reload is completed.
This allows interested subsystems to do atomic reconfiguration
rather than changing each key. Note that individual
add/change/delete notifications will still be sent during a
reload.

config.totemconfig_reload_in_progress
This key is similar to config.totemconfig_reload_in_progress
but changed after the totem config trigger is processed. It is
useful (mainly) for situations when nodelist.local_node_pos
must be correctly reinstated before anything else.

DYNAMIC CHANGE USER/GROUP PERMISSION TO USE COROSYNC IPC
Is the same as in the configuration file. eg: to add UID 500 use

# corosync-cmapctl -s uidgid.uid.500 u8 1

GID is similar, so to add a GID use

# corosync-cmapctl -s uidgid.gid.500 u8 1

For removal of permissions, simply delete the key

# corosync-cmapctl -d uidgid.gid.500

DYNAMIC ADD/REMOVE OF UDPU NODE
Eg. To add the node with address 10.34.38.108 and nodeid 3. This node
is called NEW and it's not running corosync yet.

* Find a node position in the node list which is not used yet. It's
recommended that you use highest_number + 1. Let's say output of
corosync-cmapctl looks like:

nodelist.local_node_pos (u32) = 1
nodelist.node.0.nodeid (u32) = 1
nodelist.node.0.ring0_addr (str) = 10.34.38.106
nodelist.node.1.nodeid (u32) = 2
nodelist.node.1.ring0_addr (str) = 10.34.38.107

So next node position will be 2.

* Add all entries needed for the node on all running nodes, as:

# corosync-cmapctl -s nodelist.node.2.nodeid u32 3
# corosync-cmapctl -s nodelist.node.2.ring0_addr str 10.34.38.108

Always add the ring0_addr key last. The Corosync engine on all nodes
should reply with notice [TOTEM ] adding new UDPU member
{10.34.38.108} message.

* Add node information to the configuration file on all nodes so that
it will survive a restart of corosync.

* Copy and edit configuration file to the NEW node.

* Start corosync on the NEW node.

Removal of a UDPU node is a very similar, slightly reversed action,
so

* Stop corosync on the OLD node.

* Remove the relevant entries from cmap on all nodes.

* Change the configuration file on all nodes.

NAME

OVERVIEW

KEYS

SEE ALSO