JavaScript Editor Source code editor     What Is Ajax 


Main Page

15.8. Cluster Utility Programs

This section discusses the MySQL Cluster utility programs that can be found in the mysql/bin directory. Each of these — except for ndb_size.pl and ndb_error_reporter — is a standalone binary that can be used from a system shell, and that does not need to connect to a MySQL server (nor even requires that a MySQL server be connected to the cluster).

These utilities can also serve as examples for writing your own applications using the NDB API. The source code for most of these programs may be found in the ndb/tools directory of the MySQL 5.0 tree (see Section 2.4.14, “MySQL Installation Using a Source Distribution”). The NDB API is not covered in this manual; please refer to the NDB API Guide for information about this API.

All of the NDB utilities are listed here with brief descriptions:

Most of these utilities need to connect to a Cluster management server in order to function. The exceptions are ndb_size.pl (see below), and the following utilities which access a cluster data node filesystem and so need to be run on a data node host:

ndb_size.pl is a Perl script which is also intended to be used from the shell; however it is a MySQL application and must be able to connect to a MySQL server. See Section 15.8.14, “ndb_size.pl — NDBCluster Size Requirement Estimator”, for additional requirements for using this script.

ndb_error_reporter is also a Perl script. It is used to gather cluster data node and management node logs together into a tarball to submit along with a bug report. It can use ssh or scp to access the node filesystems remotely.

Additional information about each of these utilities (except for ndb_mgm and ndb_restore) can be found in the sections that follow.

Note: All of these utilities (except for ndb_size.pl and ndb_config) can use the options discussed in Section 15.5.5, “Command Options for MySQL Cluster Processes”. Additional options specific to each utility program are discussed in the individual program listings.

The order in which these options are used is generally not important. For example, all of these commands produce exactly the same output:

15.8.1. ndb_config — Extract NDB Configuration Information

This tool extracts configuration information for data nodes, SQL nodes, and API nodes from a cluster management node (and possibly its config.ini file).

Usage:

ndb_config options

The options available for this utility differ somewhat from those used with the other utilities, and so are listed in their entirety in the next section, followed by some examples.

Options:

  • --usage, --help, or -?

    Causes ndb_config to print a list of available options, and then exit.

  • --version, -V

    Causes ndb_config to print a version information string, and then exit.

  • --ndb-connectstring=connect_string

    Specifies the connectstring to use in connecting to the management server. The format for the connectstring is the same as described in Section 15.3.4.2, “The Cluster Connectstring”, and defaults to localhost:1186.

    The use of -c as a short version for this option is supported for ndb_config beginning with MySQL 5.0.29.

  • --config-file=path-to-file

    Gives the path to the management server's configuration file (config.ini). This may be a relative or absolute path. If the management node resides on a different host from the one on which ndb_config is invoked, then an absolute path must be used.

  • --query=query-options, -q query-options

    This is a comma-delimited list of query options — that is, a list of one or more node attributes to be returned. These include id (node ID), type (node type — that is, ndbd, mysqld, or ndb_mgmd), and any configuration parameters whose values are to be obtained.

    For example, --query=id,type,indexmemory,datamemory would return the node ID, node type, DataMemory, and IndexMemory for each node.

    Note: If a given parameter is not applicable to a certain type of node, than an empty string is returned for the corresponding value. See the examples later in this section for more information.

  • --host=hostname

    Specifies the hostname of the node for which configuration information is to be obtained.

  • --id=node_id, --nodeid=node_id

    Used to specify the node ID of the node for which configuration information is to be obtained.

  • --nodes

    (Tells ndb_config to print information from parameters defined in [ndbd] sections only. Currently, using this option has no affect, since these are the only values checked, but it may become possible in future to query parameters set in [tcp] and other sections of cluster configuration files.)

  • --type=node_type

    Filters results so that only configuration values applying to nodes of the specified node_type (ndbd, mysqld, or ndb_mgmd) are returned.

  • --fields=delimiter, -f delimiter

    Specifies a delimiter string used to separate the fields in the result. The default is “,” (the comma character).

    Note: If the delimiter contains spaces or escapes (such as \n for the linefeed character), then it must be quoted.

  • --rows=separator, -r separator

    Specifies a separator string used to separate the rows in the result. The default is a space character.

    Note: If the separator contains spaces or escapes (such as \n for the linefeed character), then it must be quoted.

Examples:

  1. To obtain the node ID and type of each node in the cluster:

    shell> ./ndb_config --query=id,type --fields=':' --rows='\n'
    1:ndbd
    2:ndbd
    3:ndbd
    4:ndbd
    5:ndb_mgmd
    6:mysqld
    7:mysqld
    8:mysqld
    9:mysqld

    In this example, we used the --fields options to separate the ID and type of each node with a colon character (:), and the --rows options to place the values for each node on a new line in the output.

  2. To produce a connectstring that can be used by data, SQL, and API nodes to connect to the management server:

    shell> ./ndb_config --config-file=usr/local/mysql/cluster-data/config.ini --query=hostname,portnumber --fields=: --rows=, --type=ndb_mgmd
    192.168.0.179:1186
  3. This invocation of ndb_config checks only data nodes (using the --type option), and shows the values for each node's ID and hostname, and its DataMemory, IndexMemory, and DataDir parameters:

    shell> ./ndb_config --type=ndbd --query=id,host,datamemory,indexmemory,datadir -f ' : ' -r '\n'
    1 : 192.168.0.193 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
    2 : 192.168.0.112 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
    3 : 192.168.0.176 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
    4 : 192.168.0.119 : 83886080 : 18874368 : /usr/local/mysql/cluster-data

    In this example, we used the short options -f and -r for setting the field delimiter and row separator, respectively.

  4. To exclude results from any host except one in particular, use the --host option:

    shell> ./ndb_config --host=192.168.0.176 -f : -r '\n' -q id,type
    3:ndbd
    5:ndb_mgmd

    In this example, we also used the short form -q to determine the attributes to be queried.

    Similarly, you can limit results to a node with a specific ID using the --id or --nodeid option.

15.8.2. ndb_cpcd — Automate Testing for NDB Development

This utility is found in the libexec directory. It is part of an internal automated test framework used in testing and bedugging MySQL Cluster. Because it can control processes on remote systems, it is not advisable to use ndb_cpcd in a production cluster.

Because some users may be interested in employing the Cluster testing framework for their own development or testing purposes, we intend to make details of this application's usage available in the near future as part of the MySQL Internals Manual.

The source files for ndb_cpcd may be found in the directory storage/ndb/src/cw/cpcd, in the MySQL 5.0 source tree.

15.8.3. ndb_delete_all — Delete All Rows from NDB Table

ndb_delete_all deletes all rows from the given NDB table. In some cases, this can be much faster than DELETE or even TRUNCATE.

Usage:

ndb_delete_all -c connect_string tbl_name -d db_name

This deletes all rows from the table named tbl_name in the database named db_name. It is exactly equivalent to executing TRUNCATE db_name.tbl_name in MySQL.

Additional Options:

  • --transactional, -t

    Use of this option causes the delete operation to be performed as a single transaction.

    Warning: With very large tables, this using this option may cause the number of operations available to the cluster to be exceeded.

15.8.4. ndb_desc — Describe NDB Tables

ndb_desc provides a detailed description of one or more NDB tables.

Usage:

ndb_desc -c connect_string tbl_name -d db_name

Sample Output:

MySQL table creation and population statements:

USE test;

CREATE TABLE fish (
    id INT(11) NOT NULL AUTO_INCREMENT,
    name VARCHAR(20),

    PRIMARY KEY pk (id),
    UNIQUE KEY uk (name)
) ENGINE=NDBCLUSTER;

INSERT INTO fish VALUES 
    ('','guppy'), ('','tuna'), ('','shark'), 
    ('','manta ray'), ('','grouper'), ('','puffer');

Output from ndb_desc:

shell> ./ndb_desc -c localhost fish -d test -p
-- fish --
Version: 16777221
Fragment type: 5
K Value: 6
Min load factor: 78
Max load factor: 80
Temporary table: no
Number of attributes: 2
Number of primary keys: 1
Length of frm data: 268
Row Checksum: 1
Row GCI: 1
TableStatus: Retrieved
-- Attributes --
id Int PRIMARY KEY DISTRIBUTION KEY AT=FIXED ST=MEMORY
name Varchar(20;latin1_swedish_ci) NULL AT=SHORT_VAR ST=MEMORY

-- Indexes --
PRIMARY KEY(id) - UniqueHashIndex
uk(name) - OrderedIndex
PRIMARY(id) - OrderedIndex
uk$unique(name) - UniqueHashIndex

-- Per partition info --
Partition  Row count  Commit count  Frag fixed memory  Frag varsized memory
2          2          2             65536              327680
1          2          2             65536              327680
3          2          2             65536              327680

NDBT_ProgramExit: 0 - OK

Additional Options:

  • --extra-partition-info, -p

    Prints additional information about the table's partitions.

  • Information about multiple tables can be obtained in a single invocation of ndb_desc by using their names, separated by spaces. All of the tables must be in the same database.

15.8.5. ndb_drop_index — Drop Index from NDB Table

ndb_drop_index drops the specified index from an NDB table. It is recommended that you use this utility only as an example for writing NDB API applications — see the Warning later in this section for details.

Usage:

ndb_drop_index -c connect_string table_name index -d db_name

The statement shown above drops the index named index from the table in the database.

Additional Options: None that are specific to this application.

Warning: Operations performed on Cluster table indexes using the NDB API are not visible to MySQL and make the table unusable by a MySQL server. If you use this program to drop an index, then try to access the table from an SQL node, an error results, as shown here:

shell> ./ndb_drop_index -c localhost dogs ix -d ctest1
Dropping index dogs/idx...OK

NDBT_ProgramExit: 0 - OK

shell> ./mysql -u jon -p ctest1
Enter password: *******
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7 to server version: 5.1.12-beta-20060817

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> SHOW TABLES;
+------------------+
| Tables_in_ctest1 |
+------------------+
| a                |
| bt1              |
| bt2              |
| dogs             |
| employees        |
| fish             |
+------------------+
6 rows in set (0.00 sec)

mysql> SELECT * FROM dogs;
ERROR 1296 (HY000): Got error 4243 'Index not found' from NDBCLUSTER

In such a case, your only option for making the table available to MySQL again is to drop the table and re-create it. You can use either the SQL statementDROP TABLE or the ndb_drop_table utility (see Section 15.8.6, “ndb_drop_table — Drop NDB Table”) to drop the table.

15.8.6. ndb_drop_table — Drop NDB Table

ndb_drop_table drops the specified NDB table. (If you try to use this on a table created with a storage engine other than NDB, it fails with the error 723: No such table exists.) This operation is extremely fast — in some cases, it can be an order of magnitude faster than using DROP TABLE on an NDB table from MySQL.

Usage:

ndb_drop_table -c connect_string tbl_name -d db_name

Additional Options: None.

15.8.7. ndb_error_reporter — NDB Error-Reporting Utility

ndb_error_reporter creates an archive from data node and management node log files that can be used to help diagnose bugs or other problems with a cluster. It is highly recommended that you make use of this utility when filing reports of bugs in MySQL Cluster.

Usage:

ndb_error_reporter path/to/config-file [username] [--fs]

This utility is intended for use on a management node host, and requires the path to the management host configuration file (config.ini). Optionally, you can supply the name of a user that is able to access the cluster's data nodes via SSH, in order to copy the data node log files. ndb_error_reporter then includes all of these files in archive that is created in the same directory in which it is run. The archive is named ndb_error_report_YYYYMMDDHHMMSS.tar.bz2, where YYYYMMDDHHMMSS is a datetime string.

If the --fs is used, then the data node filesystems are also copied to the management host and included in the archive that is produced by this script. As data node filesystems can be extremely large even after being compressed, we ask that you please do not send archives created using this option to MySQL AB unless you are specifically requested to do so.

15.8.8. ndb_print_backup_file — Print NDB Backup File Contents

ndb_print_backup_file obtains diagnostic information from a cluster backup file.

Usage:

ndb_print_backup_file file_name

file_name is the name of a cluster backup file. This can be any of the files (.Data, .ctl, or .log file) found in a cluster backup directory. These files are found in the data node's backup directory under the subdirectory BACKUP-#, where # is the sequence number for the backup. For more information about cluster backup files and their contents, see Section 15.7.1, “Cluster Backup Concepts”.

Like ndb_print_schema_file and ndb_print_sys_file (and unlike most of the other NDB utilities that are intended to be run on a management server host or to connect to a management server) ndb_print_backup_file must be run on a cluster data node, since it accesses the data node filesystem directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

Additional Options: None.

15.8.9. ndb_print_schema_file — Print NDB Schema File Contents

ndb_print_schema_file obtains diagnostic information from a cluster schema file.

Usage:

ndb_print_schema_file file_name

file_name is the name of a cluster schema file.

Like ndb_print_backup_file and ndb_print_sys_file (and unlike most of the other NDB utilities that are intended to be run on a management server host or to connect to a management server) ndb_schema_backup_file must be run on a cluster data node, since it accesses the data node filesystem directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

Additional Options: None.

15.8.10. ndb_print_sys_file — Print NDB System File Contents

ndb_print_sys_file obtains diagnostic information from a cluster system file.

Usage:

ndb_print_sys_file file_name

file_name is the name of a cluster system file (sysfile). Cluster system files are located in a data node's data directory (DataDir); the path under this directory to system files matches the pattern ndb_#_fs/D#/DBDIH/P#.sysfile. In each case, the # represents a number (not necessarily the same number).

Like ndb_print_backup_file and ndb_print_schema_file (and unlike most of the other NDB utilities that are intended to be run on a management server host or to connect to a management server) ndb_print_backup_file must be run on a cluster data node, since it accesses the data node filesystem directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.

Additional Options: None.

15.8.11. ndb_select_all — Print Rows from NDB Table

ndb_select_all prints all rows from an NDB table to stdout.

Usage:

ndb_select_all -c connect_string tbl_name -d db_name [> file_name]

Additional Options:

  • --lock=lock_type, -l lock_type

    Employs a lock when reading the table. Possible values for lock_type are:

    • 0: Read lock

    • 1: Read lock with hold

    • 2: Exclusive read lock

    There is no default value for this option.

  • --order=index_name, -o index_name

    Orders the output according to the index named index_name. Note that this is the name of an index, not of a column, and that the index must have been explicitly named when created.

  • --descending, -z

    Sorts the output in descending order. This option can be used only in conjunction with the -o (--order) option.

  • --header=FALSE

    Excludes column headers from the output.

  • --useHexFormat -x

    Causes all numeric values to be displayed in hexadecimal format. This does not affect the output of numerals contained in strings or datetime values.

  • --delimiter=character, -D character

    Causes the character to be used as a column delimiter. Only table data columns are separated by this delimiter.

    The default delimiter is the tab character.

  • --rowid

    Adds a ROWID column providing information about the fragments in which rows are stored.

  • --gci

    Adds a column to the output showing the global checkpoint at which each row was last updated. See Section 15.12, “MySQL Cluster Glossary”, and Section 15.6.3.2, “Log Events”, for more information about checkpoints.

  • --tupscan, -t

    Scan the table in the order of the tuples.

  • --nodata

    Causes any table data to be omitted.

Sample Output:

Output from a MySQL SELECT statement:

mysql> SELECT * FROM ctest1.fish;
+----+-----------+
| id | name      |
+----+-----------+
|  3 | shark     |
|  6 | puffer    |
|  2 | tuna      |
|  4 | manta ray |
|  5 | grouper   |
|  1 | guppy     |
+----+-----------+
6 rows in set (0.04 sec)

Output from the equivalent invocation of ndb_select_all:

shell> ./ndb_select_all -c localhost fish -d ctest1
id      name
3       [shark]
6       [puffer]
2       [tuna]
4       [manta ray]
5       [grouper]
1       [guppy]
6 rows returned

NDBT_ProgramExit: 0 - OK

Note that all string values are enclosed by square brackets (“[...]”) in the output of ndb_select_all. For a further example, consider the table created and populated as shown here:

CREATE TABLE dogs (
    id INT(11) NOT NULL AUTO_INCREMENT,
    name VARCHAR(25) NOT NULL,
    breed VARCHAR(50) NOT NULL,
    PRIMARY KEY pk (id),
    KEY ix (name)
) 
ENGINE=NDB;

INSERT INTO dogs VALUES 
    ('', 'Lassie', 'collie'),
    ('', 'Scooby-Doo', 'Great Dane'),
    ('', 'Rin-Tin-Tin', 'German Shepherd'),
    ('', 'Rosscoe', 'Mutt');

This demonstrates the use of several additional ndb_select_all options:

shell> ./ndb_select_all -d ctest1 dogs -o ix -z --gci        
GCI     id name           breed
834461  2  [Scooby-Doo]   [Great Dane]
834878  4  [Rosscoe]      [Mutt]
834463  3  [Rin-Tin-Tin]  [German Shepherd]
835657  1  [Lassie]       [Collie]
4 rows returned

NDBT_ProgramExit: 0 - OK

15.8.12. ndb_select_count — Print Row Counts for NDB Tables

ndb_select_count prints the number of rows in one or more NDB tables. With a single table, the result is equivalent to that obtained by using the MySQL statement SELECT COUNT(*) FROM tbl_name.

Usage:

ndb_select_count [-c connect_string] -ddb_name tbl_name[, tbl_name2[, ...]]

Additional Options: None that are specific to this application. However, you can obtain row counts from multiple tables in the same database by listing the table names separated by spaces when invoking this command, as shown under Sample Output.

Sample Output:

shell> ./ndb_select_count -c localhost -d ctest1 fish dogs
6 records in table fish
4 records in table dogs

NDBT_ProgramExit: 0 - OK

15.8.13. ndb_show_tables — Display List of NDB Tables

ndb_show_tables displays a list of all NDB database objects in the cluster. By default, this includes not only both user-created tables and NDB system tables, but NDB-specific indexes, and internal triggers, as well.

Usage:

ndb_show_tables [-c connect_string]

Additional Options:

  • --loops, -l

    Specifies the number of times the utility should execute. This is 1 when this option is not specified, but if you do use the option, you must supply an integer argument for it.

  • --parsable, -p

    Using this option causes the output to be in a format suitable for use with LOAD DATA INFILE.

  • --type, -t

    Can be used to restrict the output to one type of object, specified by an integer type code as shown here:

    • 1: System table

    • 2: User-created table

    • 3: Unique hash index

    Any other value causes all NDB database objects to be listed (the default).

  • --unqualified, -u

    If specified, this causes unqualified object names to be displayed.

Note: Only user-created Cluster tables may be accessed from MySQL; system tables such as SYSTAB_0 are not visible to mysqld. However, you can examine the contents of system tables using NDB API applications such as ndb_select_all (see Section 15.8.11, “ndb_select_all — Print Rows from NDB Table”).

15.8.14. ndb_size.pl — NDBCluster Size Requirement Estimator

This is a Perl script that can be used to estimate the amount of space that would be required by a MySQL database if it were converted to use the NDBCluster storage engine. Unlike the other utilities discussed in this section, it does not require access to a MySQL Cluster (in fact, there is no reason for it to do so). However, it does need to access the MySQL server on which the database to be tested resides.

Requirements:

  • A running MySQL server. The server instance does not have to provide support for MySQL Cluster.

  • A working installation of Perl.

  • The DBI and HTML::Template modules, both of which can be obtained from CPAN if they are not already part of your Perl installation. (Many Linux and other operating system distribution provide their own packages for one or both of these libraries.)

  • The ndb_size.tmpl template file, which you should be able to find in the share/mysql directory of your MySQL installation. This file should be copied or moved into the same directory as ndb_size.pl — if it is not there already — before running the script.

  • A MySQL user account having the necessary privileges. If you do not wish to use an existing account, then creating one using GRANT USAGE ON db_name.* — where db_name is the name of the database to be examined — is sufficient for this purpose.

ndb_size.pl and ndb_size.tmpl can also be found in the MySQL sources in storage/ndb/tools. If these files are not present in your MySQL installation, you can obtain them from the MySQLForge project page.

Usage:

perl ndb_size.pl db_name hostname username password > file_name.html

The command shown connects to the MySQL server at hostname using the account of the user username having the password password, analyses all of the tables in database db_name, and generates a report in HTML format which is directed to the file file_name.html. (Without the redirection, the output is sent to stdout.) This figure shows partial sample output as viewed in a Web browser:

Partial sample output from
              ndb_size.pl as viewed in a Web
              browser.

The output from this script includes:

  • Minimum values for the DataMemory, IndexMemory, MaxNoOfTables, MaxNoOfAttributes, MaxNoOfOrderedIndexes, MaxNoOfUniqueHashIndexes, and MaxNoOfTriggers configuration parameters required to accommodate the tables analysed.

  • Memory requirements for all of the tables, attributes, ordered indexes, and unique hash indexes defined in the database.

  • The IndexMemory and DataMemory required per table and table row.

15.8.15. ndb_waiter — Wait for Cluster to Reach a Given Status

ndb_waiter repeatedly (each 100 milliseconds) prints out the status of all cluster data nodes until either the cluster reaches a given status or the --timeout limit is exceeded, then exits. By default, it waits for the cluster to achieve STARTED status, in which all nodes have started and connected to the cluster. This can be overridden using the --no-contact and --not-started options (see Additional Options).

The node states reported by this utility are as follows:

  • NO_CONTACT: The node cannot be contacted.

  • UNKNOWN: The node can be contacted, but its status is not yet known. Usually, this means that the node has received a START or RESTART command from the management server, but has not yet acted on it.

  • NOT_STARTED: The node has stopped, but remains in contact with the cluster. This is seen when restarting the node using the management client's RESTART command.

  • STARTING: The node's ndbd process has started, but the node has not yet joined the cluster.

  • STARTED: The node is operational, and has joined the cluster.

  • SHUTTING_DOWN: The node is shutting down.

  • SINGLE USER MODE: This is shown for all cluster data nodes when the cluster is in single user mode.

Usage:

ndb_waiter [-c connect_string]

Additional Options:

  • --no-contact, -n

    Instead of waiting for the STARTED state, ndb_waiter continues running until the cluster reaches NO_CONTACT status before exiting.

  • --not-started

    Instead of waiting for the STARTED state, ndb_waiter continues running until the cluster reaches NOT_STARTED status before exiting.

  • --timeout=seconds, -t seconds

    Time to wait. The program exits if the desired state is not achieved within this number of seconds. The default is 120 seconds (1200 reporting cycles).

Sample Output:

Shown here is the output from ndb_waiter when run against a 4-node cluster in which two nodes have been shut down and then started again manually. Duplicate reports (indicated by “...”) are omitted.

shell> ./ndb_waiter -c localhost

Connecting to mgmsrv at (localhost)
State node 1 STARTED
State node 2 NO_CONTACT
State node 3 STARTED
State node 4 NO_CONTACT
Waiting for cluster enter state STARTED

...

State node 1 STARTED
State node 2 UNKNOWN
State node 3 STARTED
State node 4 NO_CONTACT
Waiting for cluster enter state STARTED

...

State node 1 STARTED
State node 2 STARTING
State node 3 STARTED
State node 4 NO_CONTACT
Waiting for cluster enter state STARTED

...

State node 1 STARTED
State node 2 STARTING
State node 3 STARTED
State node 4 UNKNOWN
Waiting for cluster enter state STARTED

...

State node 1 STARTED
State node 2 STARTING
State node 3 STARTED
State node 4 STARTING
Waiting for cluster enter state STARTED

...

State node 1 STARTED
State node 2 STARTED
State node 3 STARTED
State node 4 STARTING
Waiting for cluster enter state STARTED

...

State node 1 STARTED
State node 2 STARTED
State node 3 STARTED
State node 4 STARTED
Waiting for cluster enter state STARTED

NDBT_ProgramExit: 0 - OK

Note: If no connectstring is specified, then ndb_waiter tries to connect to a management on localhost, and reports Connecting to mgmsrv at (null).

JavaScript Editor Source code editor     What Is Ajax