22.1.4 What is New in NDB Cluster

INFORMATION_SCHEMA changes.  The following changes are made in the display of information about Disk Data files in the INFORMATION_SCHEMA.FILES table:

In addition, INFORMATION_SCHEMA tables now are populated with tablespace statistics for MySQL Cluster tables. (Bug #27167728)

Error information with ndb_perror.  Removed the deprecated --ndb option for perror. Use ndb_perror to obtain error message information from NDB error codes instead. (Bug #81704, Bug #81705, Bug #23523926, Bug #23523957)

Development in parallel with MySQL server.  Beginning with this release, MySQL NDB Cluster is being developed in parallel with the standard MySQL 8.0 server under a new unified release model with the following features:

To build the MySQL 8.0.13 (or later) source with NDB Cluster support, use the CMake option -DWITH_NDBCLUSTER.

Offline multithreaded index builds.  It is now possible to specify a set of cores to be used for I/O threads performing offline multithreaded builds of ordered indexes, as opposed to normal I/O duties such as file I/O， compression， or decompression. “Offline” in this context refers to building of ordered indexes performed when the parent table is not being written to; such building takes place when an NDB cluster performs a node or system restart, or as part of restoring a cluster from backup using ndb_restore --rebuild-indexes.

In addition, the default behaviour for offline index build work is modified to use all cores available to ndbmtd, rather limiting itself to the core reserved for the I/O thread. Doing so can improve restart and restore times and performance, availability, and the user experience.

This enhancement is implemented as follows:

In addition, NDB now distinguishes the thread types that are accessible to “ThreadConfig” by the following two criteria:

For additonal information, see the descriptions of the parameters in the Manual. (Bug #25835748, Bug #26928111)

logbuffers table backup process information.  When performing an NDB backup, the ndbinfo.logbuffers table now displays information regarding buffer usage by the backup process on each data node. This is implemented as rows reflecting two new log types in addition to REDO and DD-UNDO. One of these rows has the log type BACKUP-DATA, which shows the amount of data buffer used during backup to copy fragments to backup files. The other row has the log type BACKUP-LOG, which displays the amount of log buffer used during the backup to record changes made after the backup has started. One each of these log_type rows is shown in the logbuffers table for each data node in the cluster. Rows having these two log types are present in the table only while an NDB backup is currently in progress. (Bug #25822988)

processes table on Windows.  The process ID of the monitor process used on Windows platforms by RESTART to spawn and restart a mysqld is now shown in the ndbinfo.processes table as an angel_pid.

ODirectSyncFlag.  Added the ODirectSyncFlag configuration parameter for data nodes. When enabled, the data node treats all completed filesystem writes to the redo log as though they had been performed using fsync.

(Bug #25428560)

Data node log buffer size control.  Added the --logbuffer-size option for ndbd and ndbmtd, for use in debugging with a large number of log messages. This controls the size of the data node log buffer; the default (32K) is intended for normal operations. (Bug #89679, Bug #27550943)

String hashing improvements.  Prior to NDB 8.0, all string hashing was based on first transforming the string into a normalized form, then MD5-hashing the resulting binary image. This could give rise to some performance problems, for the following reasons:

Collations provide their own hash functions, which hash the string directly without first creating a normalized string. In addition, for Unicode 9.0 collations, the hashes are computed without padding. NDB now takes advantage of this built-in function whenever hashing a string identified as using a Unicode 9.0 collation.

Since, for other collations there are existing databases which are hash partitioned on the transformed string, NDB continues to employ the previous method for hashing strings that use these, to maintain compatibility. (Bug #89590, Bug #89604, Bug #89609, Bug #27515000, Bug #27523758, Bug #27522732)

(See also.)

On-the-fly upgrades of tables using .frm files.  A table created in NDB 7.6 and earlier contains metadata in the form of a compressed .frm file, which is no longer supported in MySQL 8.0. To facilitate online upgrades to NDB 8.0, NDB performs on-the-fly translation of this metadata and writes it into the MySQL Server's data dictionary, which enables the mysqld in NDB Cluster 8.0 to work with the table without preventing subsequent use of the table by a previous version of the NDB software.

This enhancement also makes it possible to restore an NDB backup made using an earlier version to a cluster running NDB 8.0 (or later).

Schema synchronization of tablespace objects.  When a MySQL Server connects as an SQL node to an NDB cluster, it synchronizes its data dictionary with the information found in NDB dictionary.

Previously, the only NDB objects synchronized on connection of a new SQL node were databases and tables; MySQL NDB Cluster 8.0.14 and later also implement schema synchronization of disk data objects including tablespaces and log file groups. Among other benefits, this eliminates the possibility of a mismatch between the MySQL data dictionary and the NDB dictionary following a native backup and restore, in which tablespaces and log file groups were restored to the NDB dictionary, but not to the MySQL Server's data dictionary.

Handling of NO_AUTO_CREATE_USER in mysqld options file.  An error now is written to the server log when the presence of the NO_AUTO_CREATE_USER value for the sql_mode option in the options file prevents mysqld from starting.

Handling of references to nonexistent tablespaces.  It is no longer possible to issue a CREATE TABLE statement that refers to a nonexistent tablespace. Such a statement now fails with an error.

RESET MASTER changes.  Because the MySQL Server now executes RESET MASTER with a global read lock, the behavior of this statement when used with NDB Cluster has changed in the following two respects:

NDB table extra metadata changes.  In NDB 8.0.14 and later, the extra metadata property of an NDB table is used for storing serialized metadata from the MySQL data dictionary rather than storing the binary representation of the table as in previous versions. (This was a .frm file, no longer used by the MySQL Server—see Chapter 14, MySQL Data Dictionary.) As part of the work to support this change, the available size of the table's extra metadata has been increased. This means that NDB tables created in NDB Cluster 8.0.14 and later are not compatible with previous NDB Cluster releases. Tables created in previous releases can be used with NDB 8.0.14 and later, but cannot be opened afterwards by an earlier version.

For more information, see Section 22.2.8, “Upgrading and Downgrading NDB Cluster”.

Disk data file distribution.  Beginning with NDB Cluster 8.0.14, NDB uses the MySQL data dictionary to make sure that disk data files and related constructs such as tablespaces and log file groups are correctly distributed between all connected SQL nodes.

ndb_restore options.  Beginning with NDB 8.0.16, the --nodeid and --backupid options are both required when invoking ndb_restore.

ndb_log_bin system variable.  Beginning with NDB 8.0.16, the default value of the ndb_log_bin system variable has changed from TRUE to FALSE.

NDB metadata change detection.  NDB 8.0.16 implements a new mechanism for detection of updates to metadata for data objects such as tables, tablespaces, and log file groups with the MySQL data dictionary. This is done using a thread, the NDB metadata change monitor thread, which runs in the background and checks periodically for inconsistencies between the NDB dictionary and the MySQL data dictionary.

The monitor performs metadata checks every 60 seconds by default. The polling interval can be adjusted by setting the value of the ndb_metadata_check_interval system variable; polling can be disabled altogether by setting the ndb_metadata_check system variable to OFF. A status variable, Ndb_metadata_detected_count shows the number of times since mysqld was last started that inconsistencies have been detected.

NDB 7.x distributed privileges no longer supported.  Most NDB code supporting distribution of privileges between MySQL servers connected to an NDB Cluster, as implemented in NDB 7.6 and earlier (see Distributed MySQL Privileges for NDB Cluster), has been removed. This is due to the fact that such privilege tables do not function correctly in NDB 8.0, and is the result of changes to the MySQL server's privilege system (see Section 6.2.3, “Grant Tables”).

It is safe but not necessary to retain privilege tables that use the NDB storage engine and were created in a previous release, but they are no longer used for access control in NDB Cluster 8.0. In NDB 8.0.16 and later, when a mysqld acting as an SQL node detects such tables in NDB, it writes a warning to the MySQL server log, and creates InnoDB shadow tables local to itself; such shadow tables are created on each MySQL server connected to an NDB Cluster. When performing an upgrade from NDB 7.6 or earlier, the privilege tables in NDB can be removed safely using ndb_drop_table once all MySQL servers acting as SQL nodes have been upgraded.

The ndb_restore utility's --restore-privilege-tables option continues to be honored in NDB 8.0, and can still be used to restore distributed privilege tables present in a backup taken from a previous release of NDB Cluster to a cluster running NDB 8.0. These tables are handled as described in the preceeding paragraph.

For additional information regarding upgrades from previous NDB Cluster releases to NDB 8.0, see Section 22.2.8, “Upgrading and Downgrading NDB Cluster”.

Dynamic transactional resource allocation.  Allocation of resources in the transaction corrdinator (see The DBTC Block) is now performed using dynamic memory pools. This means that resource allocation determined by data node configuration parameters such as MaxDMLOperationsPerTransaction, MaxNoOfConcurrentIndexOperations, MaxNoOfConcurrentOperations, MaxNoOfConcurrentScans, MaxNoOfConcurrentTransactions, MaxNoOfFiredTriggers, MaxNoOfLocalScans, and TransactionBufferMemory is now done in such a way that, if the load represented by each of these parameters is within the target load for all such resources, others of these resources can be limited so as not to exceed the total resources available.

As part of this work, several new data node parameters controlling transactional resources in DBTC, listed here, have been added:

Backups using multiple LDMs per data node.  NDB backups can now be performed in a parallel fashion on individual data nodes using multiple local data managers (LDMs). (Previously, backups were done in parallel across data nodes, but were always serial within data node processes.) No special syntax is required for the START BACKUP command in the ndb_mgm client to enable this feature, but all data nodes must be using multiple LDMs. This means that data nodes must be running ndbmtd (ndbd is single-threaded and thus always has only one LDM) and they must be configured to use multiple LDMs before taking the backup; you can do this by choosing an appropriate setting for one of the multi-threaded data node configuration parameters MaxNoOfExecutionThreads or ThreadConfig.

Backups using multiple LDMs create subdirectories, one per LDM, under the BACKUP/BACKUP-backup_id/ directory. ndb_restore now detects these subdirectories automatically, and if they exist, attempts to restore the backup in parallel; see Section 22.4.23.2, “Restoring from a backup taken in parallel”, for details. (Single-threaded backups are restored as in previous versions of NDB.) It is also possible to restore backups taken in parallel using an ndb_restore binary from a previous version of NDB Cluster by modifying the usual restore procedure; Section 22.4.23.2.2, “Restoring a parallel backup serially”, provides information on how to do this.

Condition pushdown enhancements.  Previously, condition pushdown was limited to predicate terms referring to column values from the same table to which the condition was being pushed. In NDB 8.0.16, this restriction is removed such that column values from tables earlier in the query plan can also be referred to from pushed conditions.

Pushing down larger parts of a condition allows more rows to be filtered out by the data nodes, thereby reducing the number of rows which mysqld must handle during join processing. Another benefit of this enhancement is that filtering can be performed in parallel in the LDM threads, rather than in a single mysqld process on an SQL node; this has the potential to improve query performance significantly.

Existing rules for type compatibility between column values being compared continue to apply (see Section 8.2.1.4, “Engine Condition Pushdown Optimization”).

ndb_mgm SHOW command and single user mode.  Beginning with NDB 8.0.17, when the cluster in single user mode, the output of the management client SHOW command indicates the API or SQL node having exclusive access while this mode is in effect.

Schema distribution enhancements.  The NDB schema distribution coordinator, which handles schema operations and tracks their progress, has been extended in NDB 8.0.17 to ensure that resources used during a schema operation are released at its conclusion. Previously, some of this work was done by the schema distribution client; this has been changed due to the fact that the client did not always have all needed state information, which could lead to resource leaks when the client decided to abandon the schema operation prior to completion and without informing the coordinator.

To help fix this issue, schema operation timeout detection has been moved from the schema distribution client to the coordinator, providing the coordinator with an opportunity to clean up any resources used during the schema operation. The coordinator now checks ongoing schema operations for timeout at regular intervals, and marks participants that have not yet completed a given schema operation as failed when detecting timeout. It also provides suitable warnings whenever a schema operation timeout occurs. (It should be noted that, after such a timeout is detected, the schema operation itself continues.) Additional reporting is done by printing a list of active schema operations at regular intervals whenever one or more of these operations is ongoing.

As an additional part of this work, a new mysqld option --ndb-schema-dist-timeout makes it possible to set the length of time to wait until a schema operation is marked as having timed out.

Database DDL synchronization enhancements.  Work done in NDB 8.0.17 insures that synchronization of databases by newly joined (or rejoined) SQL nodes with those on existing SQL nodes now makes proper use of the data dictionary so that any database-level operations (CREATE DATABASE, ALTER DATABASE, or DROP DATABASE) that may have been misssed by this SQL node are now correctly duplicated on it when it connects (or reconnects) to the cluster.

As part of the database synchronization procedure performed when starting, an SQL node now compares all databases on the cluster's data nodes with those in its own data dictionary, and if any of these is found to be missing from the SQL node's data dictionary, the SQL Node installs it locally by executing a CREATE DATABASE statement. A database thus created uses the default MySQL Server database properties (such as those as determined by character_set_database and collation_database) that are in effect in effect on this SQL node at the time the statement is executed.