Source code editor What Is Ajax
↑
This section discusses the MySQL Cluster utility programs that can be found in the mysql/bin
directory. Each of these — except for ndb_size.pl
and ndb_error_reporter — is a standalone binary that can be used from a system shell, and that does not need to connect to a MySQL server (nor even requires that a MySQL server be connected to the cluster).
These utilities can also serve as examples for writing your own applications using the NDB
API. The source code for most of these programs may be found in the ndb/tools
directory of the MySQL 5.0 tree (see Section 2.4.14, “MySQL Installation Using a Source Distribution”). The NDB
API is not covered in this manual; please refer to the NDB
API Guide for information about this API.
All of the NDB
utilities are listed here with brief descriptions:
ndb_config: Retrieves Cluster configuration options.
ndb_cpcd: Used in testing and debugging MySQL Cluster.
ndb_delete_all: Deletes all rows from a given table.
ndb_desc: Lists all properties of an NDB
table.
ndb_drop_index: Drops the specified index from an NDB
table.
ndb_drop_table: Drops an NDB
table.
ndb_error_reporter: Can be used to gather information useful for diagnosing problems with the cluster.
ndb_mgm: This is the MySQL Cluster management client, which is discussed in Section 15.6.2, “Commands in the MySQL Cluster Management Client”.
ndb_print_backup_file: Prints diagnostic information obtained from cluster backup files.
ndb_print_schema_file: Prints diagnostic information obtained from cluster schema files.
ndb_print_sys_file: Prints diagnostic information obtained from cluster system files.
ndb_restore: This utility is used to restore a cluster from backup. See Section 15.7.3, “ndb_restore — Restore a Cluster Backup”, for more information.
ndb_select_all: Prints all rows from an NDB
table.
ndb_select_count: Gets the number of rows in one or more NDB
tables.
ndb_show_tables: Shows all NDB
tables anywhere in the cluster.
ndb_size.pl: Examines all the tables in a given non-Cluster database and calculates the amount of storage each would require if it were converted to use the NDB
storage engine.
ndb_waiter: Reports on the status of cluster data nodes in a manner similar to that of the management client command ALL STATUS
.
Most of these utilities need to connect to a Cluster management server in order to function. The exceptions are ndb_size.pl
(see below), and the following utilities which access a cluster data node filesystem and so need to be run on a data node host:
ndb_print_backup_file
ndb_print_schema_file
ndb_print_sys_file
ndb_size.pl
is a Perl script which is also intended to be used from the shell; however it is a MySQL application and must be able to connect to a MySQL server. See Section 15.8.14, “ndb_size.pl — NDBCluster Size Requirement Estimator”, for additional requirements for using this script.
ndb_error_reporter is also a Perl script. It is used to gather cluster data node and management node logs together into a tarball to submit along with a bug report. It can use ssh or scp to access the node filesystems remotely.
Additional information about each of these utilities (except for ndb_mgm and ndb_restore) can be found in the sections that follow.
Note: All of these utilities (except for ndb_size.pl and ndb_config) can use the options discussed in Section 15.5.5, “Command Options for MySQL Cluster Processes”. Additional options specific to each utility program are discussed in the individual program listings.
The order in which these options are used is generally not important. For example, all of these commands produce exactly the same output:
ndb_desc -c localhost fish -d test
ndb_desc fish -c localhost -d test
ndb_desc -d test fish -c localhost
This tool extracts configuration information for data nodes, SQL nodes, and API nodes from a cluster management node (and possibly its config.ini
file).
Usage:
ndb_config options
The options
available for this utility differ somewhat from those used with the other utilities, and so are listed in their entirety in the next section, followed by some examples.
Options:
Causes ndb_config to print a list of available options, and then exit.
Causes ndb_config to print a version information string, and then exit.
--ndb-connectstring=
connect_string
Specifies the connectstring to use in connecting to the management server. The format for the connectstring is the same as described in Section 15.3.4.2, “The Cluster Connectstring”, and defaults to localhost:1186
.
The use of -c
as a short version for this option is supported for ndb_config beginning with MySQL 5.0.29.
Gives the path to the management server's configuration file (config.ini
). This may be a relative or absolute path. If the management node resides on a different host from the one on which ndb_config is invoked, then an absolute path must be used.
--query=
, query-options
-q
query-options
This is a comma-delimited list of query options — that is, a list of one or more node attributes to be returned. These include id
(node ID), type (node type — that is, ndbd
, mysqld
, or ndb_mgmd
), and any configuration parameters whose values are to be obtained.
For example, --query=id,type,indexmemory,datamemory
would return the node ID, node type, DataMemory
, and IndexMemory
for each node.
Note: If a given parameter is not applicable to a certain type of node, than an empty string is returned for the corresponding value. See the examples later in this section for more information.
Specifies the hostname of the node for which configuration information is to be obtained.
--id=
, node_id
--nodeid=
node_id
Used to specify the node ID of the node for which configuration information is to be obtained.
(Tells ndb_config to print information from parameters defined in [ndbd]
sections only. Currently, using this option has no affect, since these are the only values checked, but it may become possible in future to query parameters set in [tcp]
and other sections of cluster configuration files.)
Filters results so that only configuration values applying to nodes of the specified node_type
(ndbd
, mysqld
, or ndb_mgmd
) are returned.
--fields=
, delimiter
-f
delimiter
Specifies a delimiter
string used to separate the fields in the result. The default is “,
” (the comma character).
Note: If the delimiter
contains spaces or escapes (such as \n
for the linefeed character), then it must be quoted.
--rows=
, separator
-r
separator
Specifies a separator
string used to separate the rows in the result. The default is a space character.
Note: If the separator
contains spaces or escapes (such as \n
for the linefeed character), then it must be quoted.
Examples:
To obtain the node ID and type of each node in the cluster:
shell> ./ndb_config --query=id,type --fields=':' --rows='\n'
1:ndbd
2:ndbd
3:ndbd
4:ndbd
5:ndb_mgmd
6:mysqld
7:mysqld
8:mysqld
9:mysqld
In this example, we used the --fields
options to separate the ID and type of each node with a colon character (:
), and the --rows
options to place the values for each node on a new line in the output.
To produce a connectstring that can be used by data, SQL, and API nodes to connect to the management server:
shell> ./ndb_config --config-file=usr/local/mysql/cluster-data/config.ini --query=hostname,portnumber --fields=: --rows=, --type=ndb_mgmd
192.168.0.179:1186
This invocation of ndb_config checks only data nodes (using the --type
option), and shows the values for each node's ID and hostname, and its DataMemory
, IndexMemory
, and DataDir
parameters:
shell> ./ndb_config --type=ndbd --query=id,host,datamemory,indexmemory,datadir -f ' : ' -r '\n'
1 : 192.168.0.193 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
2 : 192.168.0.112 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
3 : 192.168.0.176 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
4 : 192.168.0.119 : 83886080 : 18874368 : /usr/local/mysql/cluster-data
In this example, we used the short options -f
and -r
for setting the field delimiter and row separator, respectively.
To exclude results from any host except one in particular, use the --host
option:
shell> ./ndb_config --host=192.168.0.176 -f : -r '\n' -q id,type
3:ndbd
5:ndb_mgmd
In this example, we also used the short form -q
to determine the attributes to be queried.
Similarly, you can limit results to a node with a specific ID using the --id
or --nodeid
option.
This utility is found in the libexec
directory. It is part of an internal automated test framework used in testing and bedugging MySQL Cluster. Because it can control processes on remote systems, it is not advisable to use ndb_cpcd in a production cluster.
Because some users may be interested in employing the Cluster testing framework for their own development or testing purposes, we intend to make details of this application's usage available in the near future as part of the MySQL Internals Manual.
The source files for ndb_cpcd may be found in the directory storage/ndb/src/cw/cpcd
, in the MySQL 5.0 source tree.
ndb_delete_all deletes all rows from the given NDB
table. In some cases, this can be much faster than DELETE
or even TRUNCATE
.
Usage:
ndb_delete_all -cconnect_string
tbl_name
-ddb_name
This deletes all rows from the table named tbl_name
in the database named db_name
. It is exactly equivalent to executing TRUNCATE
in MySQL. db_name
.tbl_name
Additional Options:
ndb_desc provides a detailed description of one or more NDB
tables.
Usage:
ndb_desc -cconnect_string
tbl_name
-ddb_name
Sample Output:
MySQL table creation and population statements:
USE test; CREATE TABLE fish ( id INT(11) NOT NULL AUTO_INCREMENT, name VARCHAR(20), PRIMARY KEY pk (id), UNIQUE KEY uk (name) ) ENGINE=NDBCLUSTER; INSERT INTO fish VALUES ('','guppy'), ('','tuna'), ('','shark'), ('','manta ray'), ('','grouper'), ('','puffer');
Output from ndb_desc:
shell> ./ndb_desc -c localhost fish -d test -p
-- fish --
Version: 16777221
Fragment type: 5
K Value: 6
Min load factor: 78
Max load factor: 80
Temporary table: no
Number of attributes: 2
Number of primary keys: 1
Length of frm data: 268
Row Checksum: 1
Row GCI: 1
TableStatus: Retrieved
-- Attributes --
id Int PRIMARY KEY DISTRIBUTION KEY AT=FIXED ST=MEMORY
name Varchar(20;latin1_swedish_ci) NULL AT=SHORT_VAR ST=MEMORY
-- Indexes --
PRIMARY KEY(id) - UniqueHashIndex
uk(name) - OrderedIndex
PRIMARY(id) - OrderedIndex
uk$unique(name) - UniqueHashIndex
-- Per partition info --
Partition Row count Commit count Frag fixed memory Frag varsized memory
2 2 2 65536 327680
1 2 2 65536 327680
3 2 2 65536 327680
NDBT_ProgramExit: 0 - OK
Additional Options:
ndb_drop_index drops the specified index from an NDB
table. It is recommended that you use this utility only as an example for writing NDB API applications — see the Warning later in this section for details.
Usage:
ndb_drop_index -cconnect_string
table_name
index
-ddb_name
The statement shown above drops the index named index
from the table
in the database
.
Additional Options: None that are specific to this application.
Warning: Operations performed on Cluster table indexes using the NDB API are not visible to MySQL and make the table unusable by a MySQL server. If you use this program to drop an index, then try to access the table from an SQL node, an error results, as shown here:
shell>./ndb_drop_index -c localhost dogs ix -d ctest1
Dropping index dogs/idx...OK NDBT_ProgramExit: 0 - OK shell>./mysql -u jon -p ctest1
Enter password: ******* Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 to server version: 5.1.12-beta-20060817 Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> SHOW TABLES; +------------------+ | Tables_in_ctest1 | +------------------+ | a | | bt1 | | bt2 | | dogs | | employees | | fish | +------------------+ 6 rows in set (0.00 sec) mysql> SELECT * FROM dogs; ERROR 1296 (HY000): Got error 4243 'Index not found' from NDBCLUSTER
In such a case, your only option for making the table available to MySQL again is to drop the table and re-create it. You can use either the SQL statementDROP TABLE
or the ndb_drop_table utility (see Section 15.8.6, “ndb_drop_table — Drop NDB Table”) to drop the table.
ndb_drop_table drops the specified NDB
table. (If you try to use this on a table created with a storage engine other than NDB, it fails with the error 723: No such table exists.) This operation is extremely fast — in some cases, it can be an order of magnitude faster than using DROP TABLE
on an NDB
table from MySQL.
Usage:
ndb_drop_table -cconnect_string
tbl_name
-ddb_name
Additional Options: None.
ndb_error_reporter creates an archive from data node and management node log files that can be used to help diagnose bugs or other problems with a cluster. It is highly recommended that you make use of this utility when filing reports of bugs in MySQL Cluster.
Usage:
ndb_error_reporterpath/to/config-file
[username
] [--fs]
This utility is intended for use on a management node host, and requires the path to the management host configuration file (config.ini
). Optionally, you can supply the name of a user that is able to access the cluster's data nodes via SSH, in order to copy the data node log files. ndb_error_reporter then includes all of these files in archive that is created in the same directory in which it is run. The archive is named ndb_error_report_
, where YYYYMMDDHHMMSS
.tar.bz2YYYYMMDDHHMMSS
is a datetime string.
If the --fs
is used, then the data node filesystems are also copied to the management host and included in the archive that is produced by this script. As data node filesystems can be extremely large even after being compressed, we ask that you please do not send archives created using this option to MySQL AB unless you are specifically requested to do so.
ndb_print_backup_file obtains diagnostic information from a cluster backup file.
Usage:
ndb_print_backup_file file_name
file_name
is the name of a cluster backup file. This can be any of the files (.Data
, .ctl
, or .log
file) found in a cluster backup directory. These files are found in the data node's backup directory under the subdirectory BACKUP-
, where #
#
is the sequence number for the backup. For more information about cluster backup files and their contents, see Section 15.7.1, “Cluster Backup Concepts”.
Like ndb_print_schema_file and ndb_print_sys_file (and unlike most of the other NDB
utilities that are intended to be run on a management server host or to connect to a management server) ndb_print_backup_file must be run on a cluster data node, since it accesses the data node filesystem directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.
Additional Options: None.
ndb_print_schema_file obtains diagnostic information from a cluster schema file.
Usage:
ndb_print_schema_file file_name
file_name
is the name of a cluster schema file.
Like ndb_print_backup_file and ndb_print_sys_file (and unlike most of the other NDB
utilities that are intended to be run on a management server host or to connect to a management server) ndb_schema_backup_file must be run on a cluster data node, since it accesses the data node filesystem directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.
Additional Options: None.
ndb_print_sys_file obtains diagnostic information from a cluster system file.
Usage:
ndb_print_sys_file file_name
file_name
is the name of a cluster system file (sysfile). Cluster system files are located in a data node's data directory (DataDir
); the path under this directory to system files matches the pattern ndb_
. In each case, the #
_fs/D#
/DBDIH/P#
.sysfile#
represents a number (not necessarily the same number).
Like ndb_print_backup_file and ndb_print_schema_file (and unlike most of the other NDB
utilities that are intended to be run on a management server host or to connect to a management server) ndb_print_backup_file must be run on a cluster data node, since it accesses the data node filesystem directly. Because it does not make use of the management server, this utility can be used when the management server is not running, and even when the cluster has been completely shut down.
Additional Options: None.
ndb_select_all prints all rows from an NDB
table to stdout
.
Usage:
ndb_select_all -cconnect_string
tbl_name
-ddb_name
[>file_name
]
Additional Options:
--lock=
, lock_type
-l
lock_type
Employs a lock when reading the table. Possible values for lock_type
are:
0
: Read lock
1
: Read lock with hold
2
: Exclusive read lock
There is no default value for this option.
--order=
, index_name
-o
index_name
Orders the output according to the index named index_name
. Note that this is the name of an index, not of a column, and that the index must have been explicitly named when created.
Sorts the output in descending order. This option can be used only in conjunction with the -o
(--order
) option.
Excludes column headers from the output.
Causes all numeric values to be displayed in hexadecimal format. This does not affect the output of numerals contained in strings or datetime values.
--delimiter=
, character
-D
character
Causes the character
to be used as a column delimiter. Only table data columns are separated by this delimiter.
The default delimiter is the tab character.
Adds a ROWID
column providing information about the fragments in which rows are stored.
Adds a column to the output showing the global checkpoint at which each row was last updated. See Section 15.12, “MySQL Cluster Glossary”, and Section 15.6.3.2, “Log Events”, for more information about checkpoints.
Scan the table in the order of the tuples.
Causes any table data to be omitted.
Sample Output:
Output from a MySQL SELECT
statement:
mysql> SELECT * FROM ctest1.fish;
+----+-----------+
| id | name |
+----+-----------+
| 3 | shark |
| 6 | puffer |
| 2 | tuna |
| 4 | manta ray |
| 5 | grouper |
| 1 | guppy |
+----+-----------+
6 rows in set (0.04 sec)
Output from the equivalent invocation of ndb_select_all:
shell> ./ndb_select_all -c localhost fish -d ctest1
id name
3 [shark]
6 [puffer]
2 [tuna]
4 [manta ray]
5 [grouper]
1 [guppy]
6 rows returned
NDBT_ProgramExit: 0 - OK
Note that all string values are enclosed by square brackets (“[
...]
”) in the output of ndb_select_all. For a further example, consider the table created and populated as shown here:
CREATE TABLE dogs ( id INT(11) NOT NULL AUTO_INCREMENT, name VARCHAR(25) NOT NULL, breed VARCHAR(50) NOT NULL, PRIMARY KEY pk (id), KEY ix (name) ) ENGINE=NDB; INSERT INTO dogs VALUES ('', 'Lassie', 'collie'), ('', 'Scooby-Doo', 'Great Dane'), ('', 'Rin-Tin-Tin', 'German Shepherd'), ('', 'Rosscoe', 'Mutt');
This demonstrates the use of several additional ndb_select_all options:
shell> ./ndb_select_all -d ctest1 dogs -o ix -z --gci
GCI id name breed
834461 2 [Scooby-Doo] [Great Dane]
834878 4 [Rosscoe] [Mutt]
834463 3 [Rin-Tin-Tin] [German Shepherd]
835657 1 [Lassie] [Collie]
4 rows returned
NDBT_ProgramExit: 0 - OK
ndb_select_count prints the number of rows in one or more NDB
tables. With a single table, the result is equivalent to that obtained by using the MySQL statement SELECT COUNT(*) FROM
. tbl_name
Usage:
ndb_select_count [-cconnect_string
] -ddb_name
tbl_name
[,tbl_name2
[, ...]]
Additional Options: None that are specific to this application. However, you can obtain row counts from multiple tables in the same database by listing the table names separated by spaces when invoking this command, as shown under Sample Output.
Sample Output:
shell> ./ndb_select_count -c localhost -d ctest1 fish dogs
6 records in table fish
4 records in table dogs
NDBT_ProgramExit: 0 - OK
ndb_show_tables displays a list of all NDB
database objects in the cluster. By default, this includes not only both user-created tables and NDB
system tables, but NDB
-specific indexes, and internal triggers, as well.
Usage:
ndb_show_tables [-c connect_string
]
Additional Options:
Specifies the number of times the utility should execute. This is 1 when this option is not specified, but if you do use the option, you must supply an integer argument for it.
Using this option causes the output to be in a format suitable for use with LOAD DATA INFILE
.
Can be used to restrict the output to one type of object, specified by an integer type code as shown here:
1: System table
2: User-created table
3: Unique hash index
Any other value causes all NDB
database objects to be listed (the default).
If specified, this causes unqualified object names to be displayed.
Note: Only user-created Cluster tables may be accessed from MySQL; system tables such as SYSTAB_0
are not visible to mysqld. However, you can examine the contents of system tables using NDB
API applications such as ndb_select_all (see Section 15.8.11, “ndb_select_all — Print Rows from NDB Table”).
This is a Perl script that can be used to estimate the amount of space that would be required by a MySQL database if it were converted to use the NDBCluster
storage engine. Unlike the other utilities discussed in this section, it does not require access to a MySQL Cluster (in fact, there is no reason for it to do so). However, it does need to access the MySQL server on which the database to be tested resides.
Requirements:
A running MySQL server. The server instance does not have to provide support for MySQL Cluster.
A working installation of Perl.
The DBI
and HTML::Template
modules, both of which can be obtained from CPAN if they are not already part of your Perl installation. (Many Linux and other operating system distribution provide their own packages for one or both of these libraries.)
The ndb_size.tmpl
template file, which you should be able to find in the share/mysql
directory of your MySQL installation. This file should be copied or moved into the same directory as ndb_size.pl
— if it is not there already — before running the script.
A MySQL user account having the necessary privileges. If you do not wish to use an existing account, then creating one using GRANT USAGE ON
— where db_name
.*db_name
is the name of the database to be examined — is sufficient for this purpose.
ndb_size.pl
and ndb_size.tmpl
can also be found in the MySQL sources in storage/ndb/tools
. If these files are not present in your MySQL installation, you can obtain them from the MySQLForge project page.
Usage:
perl ndb_size.pldb_name
hostname
username
password
>file_name
.html
The command shown connects to the MySQL server at hostname
using the account of the user username
having the password password
, analyses all of the tables in database db_name
, and generates a report in HTML format which is directed to the file
. (Without the redirection, the output is sent to file_name
.htmlstdout
.) This figure shows partial sample output as viewed in a Web browser:
The output from this script includes:
Minimum values for the DataMemory
, IndexMemory
, MaxNoOfTables
, MaxNoOfAttributes
, MaxNoOfOrderedIndexes
, MaxNoOfUniqueHashIndexes
, and MaxNoOfTriggers
configuration parameters required to accommodate the tables analysed.
Memory requirements for all of the tables, attributes, ordered indexes, and unique hash indexes defined in the database.
The IndexMemory
and DataMemory
required per table and table row.
ndb_waiter repeatedly (each 100 milliseconds) prints out the status of all cluster data nodes until either the cluster reaches a given status or the --timeout
limit is exceeded, then exits. By default, it waits for the cluster to achieve STARTED
status, in which all nodes have started and connected to the cluster. This can be overridden using the --no-contact
and --not-started
options (see Additional Options).
The node states reported by this utility are as follows:
NO_CONTACT
: The node cannot be contacted.
UNKNOWN
: The node can be contacted, but its status is not yet known. Usually, this means that the node has received a START
or RESTART
command from the management server, but has not yet acted on it.
NOT_STARTED
: The node has stopped, but remains in contact with the cluster. This is seen when restarting the node using the management client's RESTART
command.
STARTING
: The node's ndbd process has started, but the node has not yet joined the cluster.
STARTED
: The node is operational, and has joined the cluster.
SHUTTING_DOWN
: The node is shutting down.
SINGLE USER MODE
: This is shown for all cluster data nodes when the cluster is in single user mode.
Usage:
ndb_waiter [-c connect_string
]
Instead of waiting for the STARTED
state, ndb_waiter continues running until the cluster reaches NO_CONTACT
status before exiting.
Instead of waiting for the STARTED
state, ndb_waiter continues running until the cluster reaches NOT_STARTED
status before exiting.
Time to wait. The program exits if the desired state is not achieved within this number of seconds. The default is 120 seconds (1200 reporting cycles).
Sample Output:
Shown here is the output from ndb_waiter when run against a 4-node cluster in which two nodes have been shut down and then started again manually. Duplicate reports (indicated by “...
”) are omitted.
shell> ./ndb_waiter -c localhost
Connecting to mgmsrv at (localhost)
State node 1 STARTED
State node 2 NO_CONTACT
State node 3 STARTED
State node 4 NO_CONTACT
Waiting for cluster enter state STARTED
...
State node 1 STARTED
State node 2 UNKNOWN
State node 3 STARTED
State node 4 NO_CONTACT
Waiting for cluster enter state STARTED
...
State node 1 STARTED
State node 2 STARTING
State node 3 STARTED
State node 4 NO_CONTACT
Waiting for cluster enter state STARTED
...
State node 1 STARTED
State node 2 STARTING
State node 3 STARTED
State node 4 UNKNOWN
Waiting for cluster enter state STARTED
...
State node 1 STARTED
State node 2 STARTING
State node 3 STARTED
State node 4 STARTING
Waiting for cluster enter state STARTED
...
State node 1 STARTED
State node 2 STARTED
State node 3 STARTED
State node 4 STARTING
Waiting for cluster enter state STARTED
...
State node 1 STARTED
State node 2 STARTED
State node 3 STARTED
State node 4 STARTED
Waiting for cluster enter state STARTED
NDBT_ProgramExit: 0 - OK
Note: If no connectstring is specified, then ndb_waiter tries to connect to a management on localhost
, and reports Connecting to mgmsrv at (null)
.