Conflict detection and resolution.  It is now possible to detect and resolve conflicts that arise in multi-master replication scenarios, such as circular replication, when different masters may try to update the same row on the slave with different data. Both “greatest timestamp wins” and “same timestamp wins” scenarios are supported. For more information, see Section 17.6.11, “MySQL Cluster Replication Conflict Resolution”.

Recovery of “one master, many slaves” replication setups.  Recovery of multi-way replication setups (“one master, many slaves”) is now supported via the --ndb-log-orig server option and changes in the mysql.ndb_binlog_index table. See Section 17.6.4, “MySQL Cluster Replication Schema and Tables”, for more information.

Enhanced selection options for transaction coordinator.  New values and behaviors are introduced for --ndb_optimized_node_selection allowing for greater flexibility when an SQL node chooses a transaction coordinator. For more information, see the description of ndb_optimized_node_selection in Section 17.3.4.3, “MySQL Cluster System Variables”.

Replication heartbeats.  Replication heartbeats facilitate the task of monitoring and detecting failures in master-slave connections in real time. This feature is implemented via a new MASTER_HEARTBEAT_PERIOD = value clause for the CHANGE MASTER TO statement and the addition of two status variables Slave_heartbeat_period and Slave_received_heartbeats. For more information, see Section 12.5.2.1, “CHANGE MASTER TO Syntax”.

NDB thread locks.  It is possible to lock NDB execution threads and maintenance threads (such as file system and other operating system threads) to specific CPUs on multiprocessor data node hosts, and to leverage real-time scheduling.

Improved performance of updates using primary keys or unique keys.  The number of unnecessary reads when performing a primary key or unique key update has been greatly reduced. Since it is seldom necessary to read a record prior to an update, this can yield a considerable improvement in performance. In addition, primary key columns are no longer written to when not needed during update operations.

Batching improvements.  Support of batched DELETE and UPDATE operations has been significantly improved. Batching of UPDATE WHERE... and multiple DELETE operations is also now implemented.

Improved SQL statement performance metrics.  The Ndb_execute_count system status variable measures the number of round trips made by SQL statements to the NDB kernel, providing an improved metric for determining efficiency with which statements are excuted. For more information, see MySQL Cluster Status Variables: Ndb_execute_count.

Compressed LCPs and backups.  Compressed local checkpoints and backups can save 50% or more of the disk space used by uncompressed LCPs and backups. These can be enabled using the two new data node configuration parameters CompressedLCP and CompressedBackup, respectively. See MySQL Cluster Status Variables: CompressedBackup, and MySQL Cluster Status Variables: CompressedLCP, for more information about these parameters.

OPTIMIZE TABLE support with NDBCLUSTER tables.  OPTIMIZE TABLE is supported for dynamic columns of in-memory NDB tables. In such cases, it is no longer necessary to drop (and possibly to re-create) a table, or to perform a rolling restart, in order to recover memory from deleted rows for general re-use by Cluster. The performance of OPTIMIZE on Cluster tables can be tuned by adjusting the value of the ndb_optimization_delay system variable, which controls the number of milliseconds to wait between processing batches of rows by OPTIMIZE TABLE. In addition, OPTIMIZE TABLE on an NDBCLUSTER table can be interrupted by, for example, killing the SQL thread performing the OPTIMIZE operation.

Batching of transactions.  It is possible to cause statements occurring within the same transaction to be run as a batch by setting the session variable transaction_allow_batching to 1 or ON. To use this feature, autocommit must be set to 0 or OFF. Batch sizes can be controlled using the --ndb-batch-size option for mysqld. For additional information, see Section 17.3.4.2, “mysqld Command Options for MySQL Cluster”, and Section 17.3.4.3, “MySQL Cluster System Variables”.

Attribute promotion with ndb_restore.  It is possible using ndb_restore to restore data reliably from a column of a given type to a column that uses a “larger” type. This is sometimes referred to as attribute promotion. For example, MySQL Cluster backup data that originated in a SMALLINT column can be restored to a MEDIUMINT, INT, or BIGINT column. See Section 17.4.17, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.

Parallel data node recovery.  Recovery of multiple data nodes can now be done in parallel, rather than sequentially. In other words, several data nodes can be restored concurrently, which can often result in much faster recovery times than when they are restored one at a time.

Increased local checkpoint efficiency.  Only 2 local checkpoints are stored, rather than 3, lowering disk space requirements and the size and number of redo log files.

NDBCLUSTER table persistence control.  Persistence of NDB tables can be controlled using the session variables ndb_table_temporary and ndb_table_no_logging. ndb_table_no_logging causes NDB tables not to be checkpointed to disk; ndb_table_temporary does the same, and in addition, no schema files are created. See Section 17.3.4.1, “MySQL Cluster Server Option and Variable Reference”.

Epoll support (Linux only).  Epoll is an improved method for handling file descriptors, which is more efficient than scanning to determine whether a file descriptor has data to be read. (The term epoll is specific to Linux and equivalent functionality is known by other names on other platforms such as Solaris and FreeBSD.) Currently, MySQL Cluster supports this functionality on Linux only.

Distribution awareness (SQL nodes).  In MySQL Cluster NDB 6.3, SQL nodes can take advantage of distribution awareness. Here we provide a brief example showing how to design a table to make a given class of queries distrubtion-aware. Suppose an NDBCLUSTER table t1 has the following schema:

CREATE TABLE t1 (
    userid INT NOT NULL,
    serviceid INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    data VARCHAR(255)
)   ENGINE=NDBCLUSTER;

Suppose further that most of the queries to be used in our application test values of the userid column of this table. The form of such a query looks something like this:

SELECT columns FROM t1
    WHERE userid relation value;

In this query, relation represents some relational operator, such as =, <, >, and so on. Queries using IN and a list of values can also be used:

SELECT columns FROM t1
    WHERE userid IN value_list;

In order to make use of distribution awareness, we need to make the userid column part of the table's primary key, then explicitly partition the table with this column being used as the partitioning key. (Recall that for a partitioned table having one or more unique keys, all columns of the table's partitioning key must also be part of all of the unique keys — for more information and examples, see Section 18.5.1, “Partitioning Keys, Primary Keys, and Unique Keys”.) In other words, the table schema should be equivalent to the following CREATE TABLE statement:

CREATE TABLE t1 (
    userid INT NOT NULL,
    serviceid INT NOT NULL AUTO_INCREMENT,
    data VARCHAR(255),
    PRIMARY KEY p (userid,serviceid)
)   ENGINE=NDBCLUSTER
    PARTITION BY KEY(userid);

When the table is partitioned in this way, all rows having the same userid value are found on the same node group, and the MySQL Server can immediately select the optimal node to use as the transaction coordinator.

Realtime extensions for multiple CPUs.  When running MySQL Cluster data nodes on hosts with multiple processors, the realtime extensions make it possible to give priority to the data node process and control on which CPU cores it should operate. This can be done using the data node configuration parameters RealtimeScheduler, SchedulerExecutionTimer and SchedulerSpinTimer. Doing so properly can significantly lower response times and make them much more predictable response. For more information about using these parameters, see Defining Data Nodes: Realtime Performance Parameters

Fully automatic database discovery.  It is no longer a requirement for database autodiscovery that an SQL node already be connected to the cluster at the time that a database is created on another SQL node, or for a CREATE DATABASE or CREATE SCHEMA statement to be issued on the new SQL node after it joins the cluster.

Restoring specific databases, tables, or columns from a MySQL Cluster backup.  It is now possible to exercise more fine-grained control when restoring a MySQL Cluster from backup using ndb_restore. Beginning with MySQL Cluster NDB 6.3.22, you can choose to restore only specified tables or databases, or exclude specific tables or databases from being restored, using the new ndb_restore options --include-tables, --include-databases, --exclude-tables, and --exclude-databases. Beginning with MySQL Cluster NDB 6.3.26, it is also possible to restore to a table having fewer columns than the original using the --exclude-missing-columns option. For more information about all of these options, see Section 17.4.17, “ndb_restore — Restore a MySQL Cluster Backup”.

Improved Disk Data filesystem configuration.  As of MySQL Cluster NDB 6.3.22, you can specify default locations for MySQL Cluster Disk Data data files and undo log files using the data node configuration parameters FileSystemPathDD, FileSystemPathDataFiles, and FileSystemPathUndoFiles. This eliminates the need to use symbolic links in order to place Disk Data files separately from other files in data node filesystems to improve Disk Data performance. For more information, see Disk Data filesystem parameters.

Automatic creation of Disk Data log file groups and tablespaces.  Beginning with MySQL Cluster NDB 6.3.22, using the data node configuration parameters InitialLogFileGroup and InitialTablespace, you can cause the creation of a MySQL Cluster Disk Data log file group, tablespace, or both, when the cluster is first started. When using these parameters, no SQL statements are required to create these Disk Data objects. For more information, see Disk Data object creation parameters.

Configuration parameter data dumps.  Starting with MySQL Cluster NDB 6.3.25, the ndb_config utility supports a --configinfo option that causes it to dump a list of all configuration parameters supported by the cluster, along with brief descriptions, information about the parameters' default and allowed values, and the sections of the config.ini file in which the parameters apply. An additional --xml switch causes ndb_config to use XML rather than plaintext output. Using ndb_config --configinfo or ndb_config --configinfo --xml requires no access to a running MySQL Cluster, any other programs, or any files. For more information and examples, see Section 17.4.6, “ndb_config — Extract MySQL Cluster Configuration Information”.

Per-table reporting of free space on disk.  The INFORMATION_SCHEMA.FILES table shows information about used and free space in MySQL Cluster Disk Data data files, but this information is not applicable to individual tables. In MySQL Cluster NDB 6.3.27 and later, the ndb_desc utility provides two additional columns in its output that show the amount of space allocated on disk for a given NDB table as well the amount of space that remains available for additional storage of disk-based column data for that table. For more information, see Section 17.4.9, “ndb_desc — Describe NDB Tables”.

Improved restart times.  Optimizations in redo log handling and other filesystem operations introduced in MySQL Cluster NDB 6.3.28 have the potential to reduce considerably the time required for restarts. While actual performance benefits observed in production setups will naturally vary depending on database size, hardware, and other conditions, our own preliminary testing has shown that these improvements can yield startup times that are faster than those typical of previous MySQL Cluster NDB 6.3 releases by a factor of 50 or more.

Increased flexibility in online upgrade procedure.  Previously, when performing an upgrade of a running MySQL cluster, the order in which the types of cluster nodes had to be upgraded was very strict. However, beginning with MySQL Cluster NDB 6.3.29, MySQL Cluster supports online upgrading of API nodes (including MySQL servers running as SQL nodes) before upgrading management nodes, data nodes, or both.

New replication conflict resolution strategy.  Beginning with MySQL Cluster NDB 6.3.31, the function NDB$MAX_DELETE_WIN() is available to implement “greatest timestamp, delete wins” conflict resolution. See NDB$MAX_DELETE_WIN(column_name), for more information.

Heartbeat thread policy and priority.  Beginning with MySQL Cluster NDB 6.3.32, a new configuration parameter HeartbeatThreadPriority makes it possible to set the policy and the priority for the heartbeat thread on management and API nodes.

Improved access to partitioning information.  The ndb_desc utility now provides additional information about the partitioning of data stored in MySQL Cluster. Beginning with MySQL Cluster NDB 6.3.32, the --blob-info option causes this program to include partition information for BLOB tables in its output. Beginning with MySQL Cluster NDB 6.3.33, the --extra-node-info option causes ndb_desc to include information about data distribution (that is, which table fragments are stored on which data nodes). Each of these options also requires the use of the --extra-partition-info option.

Information about partition-to-node mappings can also be obtained using the Table::getFragmentNodes() method, also added in MySQL Cluster NDB 6.3.33.

Replication attribute promotion and demotion.  Beginning with MySQL Cluster NDB 6.3.33, MySQL Cluster Replication supports attribute promotion and demotion when replicating between columns of different but similar types on the master and the slave. For example, it is possible to promote an INT column on the master to a BIGINT column on the slave, and to demote a TEXT column to a VARCHAR column.

The implementation of type demotion distinguishes between lossy and non-lossy type conversions, and their use on the slave can be controlled by setting the slave_type_conversions global server system variable.

For more information, see Attribute promotion and demotion (MySQL Cluster).