第5章 MySQL服务器管理

目录

5.1 MySQL服务器
5.1.1 配置服务器
5.1.2 服务器默认配置
5.1.3 服务器选项和变量参考
5.1.4 服务器命令选项
5.1.5 服务器系统变量
5.1.6 使用系统变量
5.1.7 服务器状态变量
5.1.8 服务器SQL模式
5.1.9 IPv6 支持
5.1.10 服务器端帮助
5.1.11 服务器响应信号
5.1.12 服务器关闭过程
5.2 The MySQL Data Directory
5.3 The mysql System Database
5.4 MySQL 服务器日志
5.4.1 选择普通日志和慢查询日志的输出位置
5.4.2 错误日志
5.4.3 普通查询日志
5.4.4 The Binary Log
5.4.5 慢查询日志
5.4.6 DDL日志
5.4.7 服务器日志维护
5.5 MySQL服务器组件
5.5.1 错误日志组件
5.6 MySQL 服务器插件
5.6.1 安装和卸载插件
5.6.2 获取服务器插件信息
5.6.3 The Rewriter Query Rewrite Plugin
5.6.4 版本标记
5.7 在一台机器上运行多个MySQL实例
5.7.1 设置多个数据目录
5.7.2 Windows上运行多个MySQL实例
5.7.3 Unix上运行多个MySQL实例
5.7.4 多实例环境下的客户端使用

MySQL Server (mysqld) is the main program that does most of the work in a MySQL installation. This chapter provides an overview of MySQL Server and covers general server administration:

其它关于关于管理主题的信息,还可以参阅:

5.1 The MySQL Server

mysqld is the MySQL server. The following discussion covers these MySQL server configuration topics:

  • Startup options that the server supports. You can specify these options on the command line, through configuration files, or both.

  • Server system variables. These variables reflect the current state and values of the startup options, some of which can be modified while the server is running.

  • Server status variables. These variables contain counters and statistics about runtime operation.

  • How to set the server SQL mode. This setting modifies certain aspects of SQL syntax and semantics, for example for compatibility with code from other database systems, or to control the error handling for particular situations.

  • The server shutdown process. There are performance and reliability considerations depending on the type of table (transactional or nontransactional) and whether you use replication.

For listings of MySQL server variables and options that have been added, deprecated, or removed in MySQL 8.0, see Section 1.5, “Server and Status Variables and Options Added, Deprecated, or Removed in MySQL 8.0”.

Note

Not all storage engines are supported by all MySQL server binaries and configurations. To find out how to determine which storage engines your MySQL server installation supports, see Section 13.7.5.16, “SHOW ENGINES Syntax”.

5.1.1 Configuring the Server

The MySQL server, mysqld, has many command options and system variables that can be set at startup to configure its operation. To determine the command option and system variable values used by the server, execute this command:

shell> mysqld --verbose --help

The command produces a list of all mysqld options and configurable system variables. Its output includes the default option and variable values and looks something like this:

abort-slave-event-count           0
allow-suspicious-udfs             FALSE
archive                           ON
auto-increment-increment          1
auto-increment-offset             1
autocommit                        TRUE
automatic-sp-privileges           TRUE
avoid-temporal-upgrade            FALSE
back-log                          80
basedir                           /home/jon/bin/mysql-8.0/
...
tmpdir                            /tmp
transaction-alloc-block-size      8192
transaction-isolation             REPEATABLE-READ
transaction-prealloc-size         4096
transaction-read-only             FALSE
transaction-write-set-extraction  OFF
updatable-views-with-limit        YES
validate-user-plugins             TRUE
verbose                           TRUE
wait-timeout                      28800

To see the current system variable values used by the server as it runs, connect to it and execute this statement:

mysql> SHOW VARIABLES;

To see some statistical and status indicators for a running server, execute this statement:

mysql> SHOW STATUS;

System variable and status information also is available using the mysqladmin command:

shell> mysqladmin variables
shell> mysqladmin extended-status

For a full description of all command options, system variables, and status variables, see these sections:

More detailed monitoring information is available from the Performance Schema; see Chapter 25, MySQL Performance Schema.

If you specify an option on the command line for mysqld or mysqld_safe, it remains in effect only for that invocation of the server. To use the option every time the server runs, put it in an option file. See Section 4.2.6, “Using Option Files”.

5.1.2 Server Configuration Defaults

The MySQL server has many operating parameters, which you can change at server startup using command-line options or configuration files (option files). It is also possible to change many parameters at runtime. For general instructions on setting parameters at startup or runtime, see Section 5.1.4, “Server Command Options”, and Section 5.1.5, “Server System Variables”.

On Windows, MySQL Installer interacts with the user and creates a file named my.ini in the base installation directory as the default option file.

Note

On Windows, the .ini or .cnf option file extension might not be displayed.

On any platform, after completing the installation process, you can edit the default option file at any time to modify the parameters used by the server. For example, to use a parameter setting in the file that is commented with a # character at the beginning of the line, remove the #, and modify the parameter value if necessary. To disable a setting, either add a # to the beginning of the line or remove it.

For additional information about option file format and syntax, see Section 4.2.6, “Using Option Files”.

5.1.3 Server Option and Variable Reference

The following table provides a list of all the command line options, server and status variables applicable within mysqld.

The table lists command-line options (Cmd-line), options valid in configuration files (Option file), server system variables (System Var), and status variables (Status var) in one unified list, with notification of where each option/variable is valid. If a server option set on the command line or in an option file differs from the name of the corresponding server system or status variable, the variable name is noted immediately below the corresponding option. For status variables, the scope of the variable is shown (Scope) as either global, session, or both. Please see the corresponding sections for details on setting and using the options and variables. Where appropriate, a direct link to further information on the item as available.

表格 5.选项/变量汇总

数量太多,此处省略,请移步,并参阅官方所示,谢谢!


5.1.4 服务器命令选项

When you start the mysqld server, you can specify program options using any of the methods described in Section 4.2.3, “Specifying Program Options”. The most common methods are to provide options in an option file or on the command line. However, in most cases it is desirable to make sure that the server uses the same options each time it runs. The best way to ensure this is to list them in an option file. See Section 4.2.6, “Using Option Files”. That section also describes option file format and syntax.

mysqld reads options from the [mysqld] and [server] groups. mysqld_safe reads options from the [mysqld], [server], [mysqld_safe], and [safe_mysqld] groups. mysql.server reads options from the [mysqld] and [mysql.server] groups.

mysqld accepts many command options. For a brief summary, execute this command:

mysqld --help

To see the full list, use this command:

mysqld --verbose --help

Some of the items in the list are actually system variables that can be set at server startup. These can be displayed at runtime using the SHOW VARIABLES statement. Some items displayed by the preceding mysqld command do not appear in SHOW VARIABLES output; this is because they are options only and not system variables.

The following list shows some of the most common server options. Additional options are described in other sections:

Some options control the size of buffers or caches. For a given buffer, the server might need to allocate internal data structures. These structures typically are allocated from the total memory allocated to the buffer, and the amount of space required might be platform dependent. This means that when you assign a value to an option that controls a buffer size, the amount of space actually available might differ from the value assigned. In some cases, the amount might be less than the value assigned. It is also possible that the server will adjust a value upward. For example, if you assign a value of 0 to an option for which the minimal value is 1024, the server will set the value to 1024.

Values for buffer sizes, lengths, and stack sizes are given in bytes unless otherwise specified.

Some options take file name values. Unless otherwise specified, the default file location is the data directory if the value is a relative path name. To specify the location explicitly, use an absolute path name. Suppose that the data directory is /var/mysql/data. If a file-valued option is given as a relative path name, it will be located under /var/mysql/data. If the value is an absolute path name, its location is as given by the path name.

You can also set the values of server system variables at server startup by using variable names as options. To assign a value to a server system variable, use an option of the form --var_name=value. For example, --sort_buffer_size=384M sets the sort_buffer_size variable to a value of 384MB.

When you assign a value to a variable, MySQL might automatically correct the value to stay within a given range, or adjust the value to the closest permissible value if only certain values are permitted.

To restrict the maximum value to which a system variable can be set at runtime with the SET statement, specify this maximum by using an option of the form --maximum-var_name=value at server startup.

You can change the values of most system variables at runtime with the SET statement. See Section 13.7.4.1, “SET Syntax for Variable Assignment”.

Section 5.1.5, “Server System Variables”, provides a full description for all variables, and additional information for setting them at server startup and runtime. For information on changing system variables, see Section 5.1.1, “Configuring the Server”.

5.1.6 Using System Variables

The MySQL server maintains many system variables that indicate how it is configured. Section 5.1.5, “Server System Variables”, describes the meaning of these variables. Each system variable has a default value. System variables can be set at server startup using options on the command line or in an option file. Most of them can be changed dynamically while the server is running by means of the SET statement, which enables you to modify operation of the server without having to stop and restart it. You can also use system variable values in expressions.

There are two scopes in which system variables exist. Global variables affect the overall operation of the server. Session variables affect its operation for individual client connections. A given system variable can have both a global and a session value. Global and session system variables are related as follows:

  • When the server starts, it initializes each global variable to its default value. These defaults can be changed by options specified on the command line or in an option file. (See Section 4.2.3, “Specifying Program Options”.)

  • The server also maintains a set of session variables for each client that connects. The client's session variables are initialized at connect time using the current values of the corresponding global variables. For example, a client's SQL mode is controlled by the session sql_mode value, which is initialized when the client connects to the value of the global sql_mode value.

    For some system variables, the session value is not initialized from the corresponding global value; if so, that is indicated in the variable description.

System variable values can be set globally at server startup by using options on the command line or in an option file. When you use a startup option to set a variable that takes a numeric value, the value can be given with a suffix of K, M, or G (either uppercase or lowercase) to indicate a multiplier of 1024, 10242 or 10243; that is, units of kilobytes, megabytes, or gigabytes, respectively. Thus, the following command starts the server with an InnoDB log file size of 16 megabytes and a maximum packet size of one gigabyte:

mysqld --innodb_log_file_size=16M --max_allowed_packet=1G

Within an option file, those variables are set like this:

[mysqld]
innodb_log_file_size=16M
max_allowed_packet=1G

The lettercase of suffix letters does not matter; 16M and 16m are equivalent, as are 1G and 1g.

To restrict the maximum value to which a system variable can be set at runtime with the SET statement, specify this maximum by using an option of the form --maximum-var_name=value at server startup. For example, to prevent the value of innodb_log_file_size from being increased to more than 32MB at runtime, use the option --maximum-innodb_log_file_size=32M.

Many system variables are dynamic and can be changed at runtime by using the SET statement. For a list, see Section 5.1.6.2, “Dynamic System Variables”. To change a system variable with SET, refer to it by name, optionally preceded by a modifier:

  • To indicate that a variable is a global variable, precede its name by the GLOBAL keyword or the @@global. qualifier:

    SET GLOBAL max_connections = 1000;
    SET @@global.max_connections = 1000;
    

    The SYSTEM_VARIABLES_ADMIN or SUPER privilege is required to set global variables.

  • Another way to set a global variable is to precede its name by the PERSIST keyword or the @@persist. qualifier:

    SET PERSIST max_connections = 1000;
    SET @@persist.max_connections = 1000;
    

    This SET syntax enables you to make configuration changes at runtime that also persist across server restarts. Like SET GLOBAL, SET PERSIST changes the runtime variable value, but also writes the variable setting to an option file named mysqld-auto.cnf in the data directory (replacing any existing variable setting if there is one). At startup, the server processes this file after all other option files. The SYSTEM_VARIABLES_ADMIN or SUPER privilege is required to persist global variables.

    Note

    Management of the mysqld-auto.cnf file should be left to the server and not performed manually:

    • Removal of the file results in a loss of all persisted settings at the next server startup. (This is permissible if your intent is to reconfigure the server without these settings.) To remove all settings in the file without removing the file itself, use this statement:

      RESET PERSIST;
      
    • Manual changes to the file may result in a parse error at server startup. In this case, the server reports an error and exits. If this issue occurs, start the server with the persisted_globals_load system variable disabled or with the --no-defaults option. Alternatively, remove the mysqld-auto.cnf file, but, as noted previously, removing this file results in a loss of all persisted settings.

    A plugin variable can be persisted if the plugin is installed when SET PERSIST is executed. Assignment of the persisted plugin variable takes effect for subsequent server restarts if the plugin is still installed. If the plugin is no longer installed, the plugin variable will not exist when the server reads the mysqld-auto.cnf file. In this case, the server writes a warning to the error log and continues:

    currently unknown variable 'var_name'
    was read from the persisted config file
    
  • The PERSIST_ONLY keyword or @@persist_only. qualifier is similar to PERSIST:

    SET PERSIST_ONLY back_log = 1000;
    SET @@persist_only.back_log = 1000;
    

    Like PERSIST, PERSIST_ONLY writes the variable setting to mysqld-auto.cnf. However, unlike PERSIST, PERSIST_ONLY does not modify the runtime global system variable value, making it suitable for configuring read-only system variables that can be set only at server startup. The PERSIST_RO_VARIABLES_ADMIN privilege is required to use PERSIST_ONLY.

  • To indicate that a variable is a session variable, precede its name by the SESSION keyword or either the @@session. or @@ qualifier:

    SET SESSION sql_mode = 'TRADITIONAL';
    SET @@session.sql_mode = 'TRADITIONAL';
    SET @@sql_mode = 'TRADITIONAL';
    

    Setting a session variable normally requires no special privilege, although there are exceptions that require the SYSTEM_VARIABLES_ADMIN or SUPER privilege (such as sql_log_bin). A client can change its own session variables, but not those of any other client.

    Session-only system variables cannot be persisted. They cannot be set at server startup, so there is no reason to list them in mysqld-auto.cnf.

  • LOCAL and @@local. are synonyms for SESSION and @@session..

  • If no modifier is present, SET changes the session variable.

  • An error occurs under these circumstances:

    • Use of SET GLOBAL (or @@global.), SET PERSIST (or @@persist.), or SET PERSIST_ONLY (or @@persist_only.), when setting a variable that has only a session value:

      mysql> SET GLOBAL sql_log_bin = ON;
      ERROR 1228 (HY000): Variable 'sql_log_bin' is a SESSION
      variable and can't be used with SET GLOBAL
      
    • Omission of GLOBAL (or @@global.), PERSIST (or @@persist.), or PERSIST_ONLY (or @@persist_only.) when setting a variable that has only a global value:

      mysql> SET max_connections = 1000;
      ERROR 1229 (HY000): Variable 'max_connections' is a
      GLOBAL variable and should be set with SET GLOBAL
      
    • Use of SET SESSION (or @@SESSION.) when setting a variable that has only a global value:

      mysql> SET SESSION max_connections = 1000;
      ERROR 1229 (HY000): Variable 'max_connections' is a
      GLOBAL variable and should be set with SET GLOBAL
      

The preceding modifiers apply only to system variables. An error occurs for attempts to apply them to user-defined variables, stored procedure or function parameters, or stored program local variables.

A SET statement can contain multiple variable assignments, separated by commas. This statement assigns values to a user-defined variable and a system variable:

SET @x = 1, SESSION sql_mode = '';

If you set multiple system variables, the most recent GLOBAL, PERSIST, PERSIST_ONLY, or SESSION modifier in the statement is used for following assignments that have no modifier specified.

Examples of multiple-variable assignment:

SET GLOBAL sort_buffer_size = 1000000, SESSION sort_buffer_size = 1000000;
SET @@global.sort_buffer_size = 1000000, @@local.sort_buffer_size = 1000000;
SET GLOBAL max_connections = 1000, sort_buffer_size = 1000000;

If any variable assignment in a SET statement fails, the entire statement fails and no variables are changed, nor is the mysqld-auto.cnf file changed.

If you change a session system variable, the value remains in effect within your session until you change the variable to a different value or the session ends. The change has no effect on other sessions.

If you change a global system variable, the value is remembered and used for new sessions until you change the variable to a different value or the server exits. The change is visible to any client that accesses the global variable. However, the change affects the corresponding session variable only for clients that connect after the change. The global variable change does not affect the session variable for any current client sessions (not even the session within which the SET GLOBAL statement occurred).

To make a global system variable setting permanent so that it applies across server restarts, modify it with SET PERSIST or PERSIST_ONLY to record it in the mysqld-auto.cnf file. It is also possible to use SET GLOBAL and manually modify a my.cnf file, but that is more cumbersome, and an error in a manually entered setting might not be discovered until much later. SET PERSIST or PERSIST_ONLY is more convenient and avoids the possibility of malformed settings.

The Performance Schema persisted_variables table provides an SQL interface to the mysqld-auto.cnf file, enabling its contents to be inspected at runtime using SELECT statements. See Section 25.11.13.1, “Performance Schema persisted_variables Table”.

The Performance Schema variables_info table contains information showing when and by which user each system variable was most recently set. See Section 25.11.13.2, “Performance Schema variables_info Table”.

To set a GLOBAL value to the compiled-in MySQL default value or a SESSION variable to the current corresponding GLOBAL value, set the variable to the value DEFAULT. For example, the following two statements are identical in setting the session value of max_join_size to the current global value:

SET @@session.max_join_size=DEFAULT;
SET @@session.max_join_size=@@global.max_join_size;

Not all system variables can be set to DEFAULT. In such cases, assigning DEFAULT results in an error.

With SET PERSIST (or @@persist.), setting a global variable to DEFAULT or to its literal default value assigns the variable its default value and adds a setting for the variable to mysqld-auto.cnf. To remove the variable from the file, use RESET PERSIST.

An error occurs for attempts to assign DEFAULT to user-defined variables, stored procedure or function parameters, or stored program local variables.

To refer to the value of a system variable in expressions, use one of the @@-modifiers (except @@persist., which is not permitted in expressions). For example, you can retrieve values in a SELECT statement like this:

SELECT @@global.sql_mode, @@session.sql_mode, @@sql_mode;

For a reference to a system variable in an expression as @@var_name (rather than with @@global. or @@session.), MySQL returns the session value if it exists and the global value otherwise. This differs from SET @@var_name = expr, which always refers to the session value.

Suffixes for specifying a value multiplier can be used when setting a variable at server startup, but not to set the value with SET at runtime. On the other hand, with SET you can assign a variable's value using an expression, which is not true when you set a variable at server startup. For example, the first of the following lines is legal at server startup, but the second is not:

shell> mysql --max_allowed_packet=16M
shell> mysql --max_allowed_packet=16*1024*1024

Conversely, the second of the following lines is legal at runtime, but the first is not:

mysql> SET GLOBAL max_allowed_packet=16M;
mysql> SET GLOBAL max_allowed_packet=16*1024*1024;
Note

Some system variables can be enabled with the SET statement by setting them to ON or 1, or disabled by setting them to OFF or 0. However, to set such a variable on the command line or in an option file, you must set it to 1 or 0; setting it to ON or OFF will not work. For example, on the command line, --delay_key_write=1 works but --delay_key_write=ON does not.

To display system variable names and values, use the SHOW VARIABLES statement:

mysql> SHOW VARIABLES;
+---------------------------------+-----------------------------------+
| Variable_name                   | Value                             |
+---------------------------------+-----------------------------------+
| auto_increment_increment        | 1                                 |
| auto_increment_offset           | 1                                 |
| automatic_sp_privileges         | ON                                |
| back_log                        | 151                               |
| basedir                         | /home/mysql/                      |
| binlog_cache_size               | 32768                             |
| bulk_insert_buffer_size         | 8388608                           |
| character_set_client            | utf8                              |
| character_set_connection        | utf8                              |
| character_set_database          | utf8mb4                           |
| character_set_filesystem        | binary                            |
| character_set_results           | utf8                              |
| character_set_server            | utf8mb4                           |
| character_set_system            | utf8                              |
| character_sets_dir              | /home/mysql/share/mysql/charsets/ |
| collation_connection            | utf8_general_ci                   |
| collation_database              | utf8mb4_0900_ai_ci                |
| collation_server                | utf8mb4_0900_ai_ci                |
...
| innodb_autoextend_increment     | 8                                 |
| innodb_buffer_pool_size         | 8388608                           |
| innodb_commit_concurrency       | 0                                 |
| innodb_concurrency_tickets      | 500                               |
| innodb_data_file_path           | ibdata1:10M:autoextend            |
| innodb_data_home_dir            |                                   |
...
| version                         | 8.0.1-dmr-log                     |
| version_comment                 | Source distribution               |
| version_compile_machine         | i686                              |
| version_compile_os              | suse-linux                        |
| wait_timeout                    | 28800                             |
+---------------------------------+-----------------------------------+

With a LIKE clause, the statement displays only those variables that match the pattern. To obtain a specific variable name, use a LIKE clause as shown:

SHOW VARIABLES LIKE 'max_join_size';
SHOW SESSION VARIABLES LIKE 'max_join_size';

To get a list of variables whose name match a pattern, use the % wildcard character in a LIKE clause:

SHOW VARIABLES LIKE '%size%';
SHOW GLOBAL VARIABLES LIKE '%size%';

Wildcard characters can be used in any position within the pattern to be matched. Strictly speaking, because _ is a wildcard that matches any single character, you should escape it as \_ to match it literally. In practice, this is rarely necessary.

For SHOW VARIABLES, if you specify neither GLOBAL nor SESSION, MySQL returns SESSION values.

The reason for requiring the GLOBAL keyword when setting GLOBAL-only variables but not when retrieving them is to prevent problems in the future:

  • Were we to remove a SESSION variable that has the same name as a GLOBAL variable, a client with the SYSTEM_VARIABLES_ADMIN or SUPER privilege might accidentally change the GLOBAL variable rather than just the SESSION variable for its own connection.

  • Were we to add a SESSION variable with the same name as a GLOBAL variable, a client that intends to change the GLOBAL variable might find only its own SESSION variable changed.

5.1.6.1 Structured System Variables

A structured variable differs from a regular system variable in two respects:

  • Its value is a structure with components that specify server parameters considered to be closely related.

  • There might be several instances of a given type of structured variable. Each one has a different name and refers to a different resource maintained by the server.

MySQL supports one structured variable type, which specifies parameters governing the operation of key caches. A key cache structured variable has these components:

This section describes the syntax for referring to structured variables. Key cache variables are used for syntax examples, but specific details about how key caches operate are found elsewhere, in Section 8.10.2, “The MyISAM Key Cache”.

To refer to a component of a structured variable instance, you can use a compound name in instance_name.component_name format. Examples:

hot_cache.key_buffer_size
hot_cache.key_cache_block_size
cold_cache.key_cache_block_size

For each structured system variable, an instance with the name of default is always predefined. If you refer to a component of a structured variable without any instance name, the default instance is used. Thus, default.key_buffer_size and key_buffer_size both refer to the same system variable.

Structured variable instances and components follow these naming rules:

  • For a given type of structured variable, each instance must have a name that is unique within variables of that type. However, instance names need not be unique across structured variable types. For example, each structured variable has an instance named default, so default is not unique across variable types.

  • The names of the components of each structured variable type must be unique across all system variable names. If this were not true (that is, if two different types of structured variables could share component member names), it would not be clear which default structured variable to use for references to member names that are not qualified by an instance name.

  • If a structured variable instance name is not legal as an unquoted identifier, refer to it as a quoted identifier using backticks. For example, hot-cache is not legal, but `hot-cache` is.

  • global, session, and local are not legal instance names. This avoids a conflict with notation such as @@global.var_name for referring to nonstructured system variables.

Currently, the first two rules have no possibility of being violated because the only structured variable type is the one for key caches. These rules will assume greater significance if some other type of structured variable is created in the future.

With one exception, you can refer to structured variable components using compound names in any context where simple variable names can occur. For example, you can assign a value to a structured variable using a command-line option:

shell> mysqld --hot_cache.key_buffer_size=64K

In an option file, use this syntax:

[mysqld]
hot_cache.key_buffer_size=64K

If you start the server with this option, it creates a key cache named hot_cache with a size of 64KB in addition to the default key cache that has a default size of 8MB.

Suppose that you start the server as follows:

shell> mysqld --key_buffer_size=256K \
         --extra_cache.key_buffer_size=128K \
         --extra_cache.key_cache_block_size=2048

In this case, the server sets the size of the default key cache to 256KB. (You could also have written --default.key_buffer_size=256K.) In addition, the server creates a second key cache named extra_cache that has a size of 128KB, with the size of block buffers for caching table index blocks set to 2048 bytes.

The following example starts the server with three different key caches having sizes in a 3:1:1 ratio:

shell> mysqld --key_buffer_size=6M \
         --hot_cache.key_buffer_size=2M \
         --cold_cache.key_buffer_size=2M

Structured variable values may be set and retrieved at runtime as well. For example, to set a key cache named hot_cache to a size of 10MB, use either of these statements:

mysql> SET GLOBAL hot_cache.key_buffer_size = 10*1024*1024;
mysql> SET @@global.hot_cache.key_buffer_size = 10*1024*1024;

To retrieve the cache size, do this:

mysql> SELECT @@global.hot_cache.key_buffer_size;

However, the following statement does not work. The variable is not interpreted as a compound name, but as a simple string for a LIKE pattern-matching operation:

mysql> SHOW GLOBAL VARIABLES LIKE 'hot_cache.key_buffer_size';

This is the exception to being able to use structured variable names anywhere a simple variable name may occur.

5.1.6.2 Dynamic System Variables

Many server system variables are dynamic and can be set at runtime using SET GLOBAL or SET SESSION. You can also obtain their values using SELECT. See Section 5.1.6, “Using System Variables”.

The following table shows the full list of all dynamic system variables. The last column indicates for each variable whether GLOBAL or SESSION (or both) apply. The table also lists session options that can be set with the SET statement. Section 5.1.5, “Server System Variables”, discusses these options.

Variables that have a type of string take a string value. Variables that have a type of numeric take a numeric value. Variables that have a type of boolean can be set to 0, 1, ON or OFF. Variables that are marked as enumeration normally should be set to one of the available values for the variable, but can also be set to the number that corresponds to the desired enumeration value. For enumerated system variables, the first enumeration value corresponds to 0. This differs from ENUM columns, for which the first enumeration value corresponds to 1.

表格5.3 Dynamic Variable Summary


5.1.7 Server Status Variables

The MySQL server maintains many status variables that provide information about its operation. You can view these variables and their values by using the SHOW [GLOBAL | SESSION] STATUS statement (see Section 13.7.5.35, “SHOW STATUS Syntax”). The optional GLOBAL keyword aggregates the values over all connections, and SESSION shows the values for the current connection.

mysql> SHOW GLOBAL STATUS;
+-----------------------------------+------------+
| Variable_name                     | Value      |
+-----------------------------------+------------+
| Aborted_clients                   | 0          |
| Aborted_connects                  | 0          |
| Bytes_received                    | 155372598  |
| Bytes_sent                        | 1176560426 |
...
| Connections                       | 30023      |
| Created_tmp_disk_tables           | 0          |
| Created_tmp_files                 | 3          |
| Created_tmp_tables                | 2          |
...
| Threads_created                   | 217        |
| Threads_running                   | 88         |
| Uptime                            | 1389872    |
+-----------------------------------+------------+

Several status variables provide statement counts. To determine the number of statements executed, use these relationships:

  SUM(Com_xxx)
= Questions + statements executed within stored programs
= Queries

Many status variables are reset to 0 by the FLUSH STATUS statement.

The following table lists all available server status variables:

表格 5.4 Status Variable Summary


The status variables have the following meanings.

5.1.8 Server SQL Modes

The MySQL server can operate in different SQL modes, and can apply these modes differently for different clients, depending on the value of the sql_mode system variable. DBAs can set the global SQL mode to match site server operating requirements, and each application can set its session SQL mode to its own requirements.

Modes affect the SQL syntax MySQL supports and the data validation checks it performs. This makes it easier to use MySQL in different environments and to use MySQL together with other database servers.

For answers to questions often asked about server SQL modes in MySQL, see Section A.3, “MySQL 8.0 FAQ: Server SQL Mode”.

When working with InnoDB tables, consider also the innodb_strict_mode system variable. It enables additional error checks for InnoDB tables.

Setting the SQL Mode

The default SQL mode in MySQL 8.0 includes these modes: ONLY_FULL_GROUP_BY, STRICT_TRANS_TABLES, NO_ZERO_IN_DATE, NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER, and NO_ENGINE_SUBSTITUTION.

The ONLY_FULL_GROUP_BY and STRICT_TRANS_TABLES modes were added in MySQL 5.7.5. The NO_AUTO_CREATE_USER mode was added in MySQL 5.7.7. The ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE modes were added in MySQL 5.7.8. For additional discussion regarding these changes to the default SQL mode value, see SQL Mode Changes in MySQL 5.7.

To set the SQL mode at server startup, use the --sql-mode="modes" option on the command line, or sql-mode="modes" in an option file such as my.cnf (Unix operating systems) or my.ini (Windows). modes is a list of different modes separated by commas. To clear the SQL mode explicitly, set it to an empty string using --sql-mode="" on the command line, or sql-mode="" in an option file.

Note

MySQL installation programs may configure the SQL mode during the installation process.

If the SQL mode differs from the default or from what you expect, check for a setting in an option file that the server reads at startup.

To change the SQL mode at runtime, set the global or session sql_mode system variable using a SET statement:

SET GLOBAL sql_mode = 'modes';
SET SESSION sql_mode = 'modes';

Setting the GLOBAL variable requires the SYSTEM_VARIABLES_ADMIN or SUPER privilege and affects the operation of all clients that connect from that time on. Setting the SESSION variable affects only the current client. Each client can change its session sql_mode value at any time.

To determine the current global or session sql_mode value, use the following statements:

SELECT @@GLOBAL.sql_mode;
SELECT @@SESSION.sql_mode;
Important

SQL mode and user-defined partitioning.  Changing the server SQL mode after creating and inserting data into partitioned tables can cause major changes in the behavior of such tables, and could lead to loss or corruption of data. It is strongly recommended that you never change the SQL mode once you have created tables employing user-defined partitioning.

When replicating partitioned tables, differing SQL modes on master and slave can also lead to problems. For best results, you should always use the same server SQL mode on the master and on the slave.

See Section 22.6, “Restrictions and Limitations on Partitioning”, for more information.

The Most Important SQL Modes

The most important sql_mode values are probably these:

  • ANSI

    This mode changes syntax and behavior to conform more closely to standard SQL. It is one of the special combination modes listed at the end of this section.

  • STRICT_TRANS_TABLES

    If a value could not be inserted as given into a transactional table, abort the statement. For a nontransactional table, abort the statement if the value occurs in a single-row statement or the first row of a multiple-row statement. More details are given later in this section.

  • TRADITIONAL

    Make MySQL behave like a traditional SQL database system. A simple description of this mode is give an error instead of a warning when inserting an incorrect value into a column. It is one of the special combination modes listed at the end of this section.

    Note

    The INSERT or UPDATE aborts as soon as the error is noticed. This may not be what you want if you are using a nontransactional storage engine, because data changes made prior to the error may not be rolled back, resulting in a partially done update.

When this manual refers to strict mode, it means a mode with either or both STRICT_TRANS_TABLES or STRICT_ALL_TABLES enabled.

Full List of SQL Modes

The following list describes all supported SQL modes:

  • ALLOW_INVALID_DATES

    Do not perform full checking of dates. Check only that the month is in the range from 1 to 12 and the day is in the range from 1 to 31. This is very convenient for Web applications where you obtain year, month, and day in three different fields and you want to store exactly what the user inserted (without date validation). This mode applies to DATE and DATETIME columns. It does not apply TIMESTAMP columns, which always require a valid date.

    The server requires that month and day values be legal, and not merely in the range 1 to 12 and 1 to 31, respectively. With strict mode disabled, invalid dates such as '2004-04-31' are converted to '0000-00-00' and a warning is generated. With strict mode enabled, invalid dates generate an error. To permit such dates, enable ALLOW_INVALID_DATES.

  • ANSI_QUOTES

    Treat " as an identifier quote character (like the ` quote character) and not as a string quote character. You can still use ` to quote identifiers with this mode enabled. With ANSI_QUOTES enabled, you cannot use double quotation marks to quote literal strings, because it is interpreted as an identifier.

  • ERROR_FOR_DIVISION_BY_ZERO

    The ERROR_FOR_DIVISION_BY_ZERO mode affects handling of division by zero, which includes MOD(N,0). For data-change operations (INSERT, UPDATE), its effect also depends on whether strict SQL mode is enabled.

    • If this mode is not enabled, division by zero inserts NULL and produces no warning.

    • If this mode is enabled, division by zero inserts NULL and produces a warning.

    • If this mode and strict mode are enabled, division by zero produces an error, unless IGNORE is given as well. For INSERT IGNORE and UPDATE IGNORE, division by zero inserts NULL and produces a warning.

    For SELECT, division by zero returns NULL. Enabling ERROR_FOR_DIVISION_BY_ZERO causes a warning to be produced as well, regardless of whether strict mode is enabled.

    ERROR_FOR_DIVISION_BY_ZERO has an effect when named explicitly and is not part of strict mode. However, it should be used in conjunction with strict mode and is enabled by default. A warning occurs if ERROR_FOR_DIVISION_BY_ZERO is enabled without also enabling strict mode or vice versa. For additional discussion, see SQL Mode Changes in MySQL 5.7.

    Because ERROR_FOR_DIVISION_BY_ZERO is deprecated, it will be removed in a future MySQL release as a separate mode name and its effect included in the effects of strict SQL mode.

  • HIGH_NOT_PRECEDENCE

    The precedence of the NOT operator is such that expressions such as NOT a BETWEEN b AND c are parsed as NOT (a BETWEEN b AND c). In some older versions of MySQL, the expression was parsed as (NOT a) BETWEEN b AND c. The old higher-precedence behavior can be obtained by enabling the HIGH_NOT_PRECEDENCE SQL mode.

    mysql> SET sql_mode = '';
    mysql> SELECT NOT 1 BETWEEN -5 AND 5;
            -> 0
    mysql> SET sql_mode = 'HIGH_NOT_PRECEDENCE';
    mysql> SELECT NOT 1 BETWEEN -5 AND 5;
            -> 1
    
  • IGNORE_SPACE

    Permit spaces between a function name and the ( character. This causes built-in function names to be treated as reserved words. As a result, identifiers that are the same as function names must be quoted as described in Section 9.2, “Schema Object Names”. For example, because there is a COUNT() function, the use of count as a table name in the following statement causes an error:

    mysql> CREATE TABLE count (i INT);
    ERROR 1064 (42000): You have an error in your SQL syntax
    

    The table name should be quoted:

    mysql> CREATE TABLE `count` (i INT);
    Query OK, 0 rows affected (0.00 sec)
    

    The IGNORE_SPACE SQL mode applies to built-in functions, not to user-defined functions or stored functions. It is always permissible to have spaces after a UDF or stored function name, regardless of whether IGNORE_SPACE is enabled.

    For further discussion of IGNORE_SPACE, see Section 9.2.4, “Function Name Parsing and Resolution”.

  • NO_AUTO_CREATE_USER

    Prevent the GRANT statement from automatically creating new user accounts if it would otherwise do so, unless authentication information is specified. The statement must specify a nonempty password using IDENTIFIED BY or an authentication plugin using IDENTIFIED WITH.

    It is preferable to create MySQL accounts with CREATE USER rather than GRANT. As of MySQL 5.7.6, NO_AUTO_CREATE_USER is deprecated. As of 5.7.7 the default SQL mode includes NO_AUTO_CREATE_USER and assignments to sql_mode that change the NO_AUTO_CREATE_USER mode state produce a warning, except assignments that set sql_mode to DEFAULT. NO_AUTO_CREATE_USER will be removed in a future MySQL release, at which point its effect will be enabled at all times (GRANT will not create accounts).

  • NO_AUTO_VALUE_ON_ZERO

    NO_AUTO_VALUE_ON_ZERO affects handling of AUTO_INCREMENT columns. Normally, you generate the next sequence number for the column by inserting either NULL or 0 into it. NO_AUTO_VALUE_ON_ZERO suppresses this behavior for 0 so that only NULL generates the next sequence number.

    This mode can be useful if 0 has been stored in a table's AUTO_INCREMENT column. (Storing 0 is not a recommended practice, by the way.) For example, if you dump the table with mysqldump and then reload it, MySQL normally generates new sequence numbers when it encounters the 0 values, resulting in a table with contents different from the one that was dumped. Enabling NO_AUTO_VALUE_ON_ZERO before reloading the dump file solves this problem. mysqldump now automatically includes in its output a statement that enables NO_AUTO_VALUE_ON_ZERO, to avoid this problem.

  • NO_BACKSLASH_ESCAPES

    Disable the use of the backslash character (\) as an escape character within strings. With this mode enabled, backslash becomes an ordinary character like any other.

  • NO_DIR_IN_CREATE

    When creating a table, ignore all INDEX DIRECTORY and DATA DIRECTORY directives. This option is useful on slave replication servers.

  • NO_ENGINE_SUBSTITUTION

    Control automatic substitution of the default storage engine when a statement such as CREATE TABLE or ALTER TABLE specifies a storage engine that is disabled or not compiled in.

    The default SQL mode includes NO_ENGINE_SUBSTITUTION.

    Because storage engines can be pluggable at runtime, unavailable engines are treated the same way:

    With NO_ENGINE_SUBSTITUTION disabled, for CREATE TABLE the default engine is used and a warning occurs if the desired engine is unavailable. For ALTER TABLE, a warning occurs and the table is not altered.

    With NO_ENGINE_SUBSTITUTION enabled, an error occurs and the table is not created or altered if the desired engine is unavailable.

  • NO_FIELD_OPTIONS

    Do not print MySQL-specific column options in the output of SHOW CREATE TABLE. This mode is used by mysqldump in portability mode.

  • NO_KEY_OPTIONS

    Do not print MySQL-specific index options in the output of SHOW CREATE TABLE. This mode is used by mysqldump in portability mode.

  • NO_TABLE_OPTIONS

    Do not print MySQL-specific table options (such as ENGINE) in the output of SHOW CREATE TABLE. This mode is used by mysqldump in portability mode.

  • NO_UNSIGNED_SUBTRACTION

    Subtraction between integer values, where one is of type UNSIGNED, produces an unsigned result by default. If the result would otherwise have been negative, an error results:

    mysql> SET sql_mode = '';
    Query OK, 0 rows affected (0.00 sec)
    
    mysql> SELECT CAST(0 AS UNSIGNED) - 1;
    ERROR 1690 (22003): BIGINT UNSIGNED value is out of range in '(cast(0 as unsigned) - 1)'
    

    If the NO_UNSIGNED_SUBTRACTION SQL mode is enabled, the result is negative:

    mysql> SET sql_mode = 'NO_UNSIGNED_SUBTRACTION';
    mysql> SELECT CAST(0 AS UNSIGNED) - 1;
    +-------------------------+
    | CAST(0 AS UNSIGNED) - 1 |
    +-------------------------+
    |                      -1 |
    +-------------------------+
    

    If the result of such an operation is used to update an UNSIGNED integer column, the result is clipped to the maximum value for the column type, or clipped to 0 if NO_UNSIGNED_SUBTRACTION is enabled. If strict SQL mode is enabled, an error occurs and the column remains unchanged.

    When NO_UNSIGNED_SUBTRACTION is enabled, the subtraction result is signed, even if any operand is unsigned. For example, compare the type of column c2 in table t1 with that of column c2 in table t2:

    
    mysql> SET sql_mode='';
    mysql> CREATE TABLE test (c1 BIGINT UNSIGNED NOT NULL);
    mysql> CREATE TABLE t1 SELECT c1 - 1 AS c2 FROM test;
    mysql> DESCRIBE t1;
    +-------+---------------------+------+-----+---------+-------+
    | Field | Type                | Null | Key | Default | Extra |
    +-------+---------------------+------+-----+---------+-------+
    | c2    | bigint(21) unsigned | NO   |     | 0       |       |
    +-------+---------------------+------+-----+---------+-------+
    
    mysql> SET sql_mode='NO_UNSIGNED_SUBTRACTION';
    mysql> CREATE TABLE t2 SELECT c1 - 1 AS c2 FROM test;
    mysql> DESCRIBE t2;
    +-------+------------+------+-----+---------+-------+
    | Field | Type       | Null | Key | Default | Extra |
    +-------+------------+------+-----+---------+-------+
    | c2    | bigint(21) | NO   |     | 0       |       |
    +-------+------------+------+-----+---------+-------+
    

    This means that BIGINT UNSIGNED is not 100% usable in all contexts. See Section 12.10, “Cast Functions and Operators”.

  • NO_ZERO_DATE

    The NO_ZERO_DATE mode affects whether the server permits '0000-00-00' as a valid date. Its effect also depends on whether strict SQL mode is enabled.

    • If this mode is not enabled, '0000-00-00' is permitted and inserts produce no warning.

    • If this mode is enabled, '0000-00-00' is permitted and inserts produce a warning.

    • If this mode and strict mode are enabled, '0000-00-00' is not permitted and inserts produce an error, unless IGNORE is given as well. For INSERT IGNORE and UPDATE IGNORE, '0000-00-00' is permitted and inserts produce a warning.

    As of MySQL 5.7.4, NO_ZERO_DATE is deprecated. In MySQL 5.7.4 through 5.7.7, NO_ZERO_DATE does nothing when named explicitly. Instead, its effect is included in the effects of strict SQL mode. In MySQL 5.7.8 and later, NO_ZERO_DATE does have an effect when named explicitly and is not part of strict mode, as before MySQL 5.7.4. However, it should be used in conjunction with strict mode and is enabled by default. A warning occurs if NO_ZERO_DATE is enabled without also enabling strict mode or vice versa. For additional discussion, see SQL Mode Changes in MySQL 5.7.

    Because NO_ZERO_DATE is deprecated, it will be removed in a future MySQL release as a separate mode name and its effect included in the effects of strict SQL mode.

  • NO_ZERO_IN_DATE

    The NO_ZERO_IN_DATE mode affects whether the server permits dates in which the year part is nonzero but the month or day part is 0. (This mode affects dates such as '2010-00-01' or '2010-01-00', but not '0000-00-00'. To control whether the server permits '0000-00-00', use the NO_ZERO_DATE mode.) The effect of NO_ZERO_IN_DATE also depends on whether strict SQL mode is enabled.

    • If this mode is not enabled, dates with zero parts are permitted and inserts produce no warning.

    • If this mode is enabled, dates with zero parts are inserted as '0000-00-00' and produce a warning.

    • If this mode and strict mode are enabled, dates with zero parts are not permitted and inserts produce an error, unless IGNORE is given as well. For INSERT IGNORE and UPDATE IGNORE, dates with zero parts are inserted as '0000-00-00' and produce a warning.

    As of MySQL 5.7.4, NO_ZERO_IN_DATE is deprecated. In MySQL 5.7.4 through 5.7.7, NO_ZERO_IN_DATE does nothing when named explicitly. Instead, its effect is included in the effects of strict SQL mode. In MySQL 5.7.8 and later, NO_ZERO_IN_DATE does have an effect when named explicitly and is not part of strict mode, as before MySQL 5.7.4. However, it should be used in conjunction with strict mode and is enabled by default. A warning occurs if NO_ZERO_IN_DATE is enabled without also enabling strict mode or vice versa. For additional discussion, see SQL Mode Changes in MySQL 5.7.

    Because NO_ZERO_IN_DATE is deprecated, it will be removed in a future MySQL release as a separate mode name and its effect included in the effects of strict SQL mode.

  • ONLY_FULL_GROUP_BY

    Reject queries for which the select list, HAVING condition, or ORDER BY list refer to nonaggregated columns that are neither named in the GROUP BY clause nor are functionally dependent on (uniquely determined by) GROUP BY columns.

    A MySQL extension to standard SQL permits references in the HAVING clause to aliased expressions in the select list. The HAVING clause can refer to aliases regardless of whether ONLY_FULL_GROUP_BY is enabled.

    For additional discussion and examples, see Section 12.18.3, “MySQL Handling of GROUP BY”.

  • PAD_CHAR_TO_FULL_LENGTH

    By default, trailing spaces are trimmed from CHAR column values on retrieval. If PAD_CHAR_TO_FULL_LENGTH is enabled, trimming does not occur and retrieved CHAR values are padded to their full length. This mode does not apply to VARCHAR columns, for which trailing spaces are retained on retrieval.

    
    mysql> CREATE TABLE t1 (c1 CHAR(10));
    Query OK, 0 rows affected (0.37 sec)
    
    mysql> INSERT INTO t1 (c1) VALUES('xy');
    Query OK, 1 row affected (0.01 sec)
    
    mysql> SET sql_mode = '';
    Query OK, 0 rows affected (0.00 sec)
    
    mysql> SELECT c1, CHAR_LENGTH(c1) FROM t1;
    +------+-----------------+
    | c1   | CHAR_LENGTH(c1) |
    +------+-----------------+
    | xy   |               2 |
    +------+-----------------+
    1 row in set (0.00 sec)
    
    mysql> SET sql_mode = 'PAD_CHAR_TO_FULL_LENGTH';
    Query OK, 0 rows affected (0.00 sec)
    
    mysql> SELECT c1, CHAR_LENGTH(c1) FROM t1;
    +------------+-----------------+
    | c1         | CHAR_LENGTH(c1) |
    +------------+-----------------+
    | xy         |              10 |
    +------------+-----------------+
    1 row in set (0.00 sec)
    
  • PIPES_AS_CONCAT

    Treat || as a string concatenation operator (same as CONCAT()) rather than as a synonym for OR.

  • REAL_AS_FLOAT

    Treat REAL as a synonym for FLOAT. By default, MySQL treats REAL as a synonym for DOUBLE.

  • STRICT_ALL_TABLES

    Enable strict SQL mode for all storage engines. Invalid data values are rejected. For details, see Strict SQL Mode.

  • STRICT_TRANS_TABLES

    Enable strict SQL mode for transactional storage engines, and when possible for nontransactional storage engines. For details, see Strict SQL Mode.

  • TIME_TRUNCATE_FRACTIONAL

    Control whether rounding or truncation occurs when inserting a TIME, DATE, or TIMESTAMP value with a fractional seconds part into a column having the same type but fewer fractional digits. The behavior is to use rounding. If this mode is enabled, truncation occurs instead. The following sequence of statements illustrates the difference:

    CREATE TABLE t (tval TIME(1));
    SET sql_mode='';
    INSERT INTO t (tval) VALUES(1.55);
    SET sql_mode='TIME_TRUNCATE_FRACTIONAL';
    INSERT INTO t (tval) VALUES(2.55);
    

    The resulting table contents look like this, where the first value has been subject to rounding and the second to truncation:

    mysql> SELECT tval FROM t ORDER BY tval;
    +------------+
    | tval       |
    +------------+
    | 00:00:01.6 |
    | 00:00:02.5 |
    +------------+
    

Combination SQL Modes

The following special modes are provided as shorthand for combinations of mode values from the preceding list.

Strict SQL Mode

Strict mode controls how MySQL handles invalid or missing values in data-change statements such as INSERT or UPDATE. A value can be invalid for several reasons. For example, it might have the wrong data type for the column, or it might be out of range. A value is missing when a new row to be inserted does not contain a value for a non-NULL column that has no explicit DEFAULT clause in its definition. (For a NULL column, NULL is inserted if the value is missing.) Strict mode also affects DDL statements such as CREATE TABLE.

If strict mode is not in effect, MySQL inserts adjusted values for invalid or missing values and produces warnings (see Section 13.7.5.40, “SHOW WARNINGS Syntax”). In strict mode, you can produce this behavior by using INSERT IGNORE or UPDATE IGNORE.

For statements such as SELECT that do not change data, invalid values generate a warning in strict mode, not an error.

Strict mode produces an error for attempts to create a key that exceeds the maximum key length. When strict mode is not enabled, this results in a warning and truncation of the key to the maximum key length.

Strict mode does not affect whether foreign key constraints are checked. foreign_key_checks can be used for that. (See Section 5.1.5, “Server System Variables”.)

Strict SQL mode is in effect if either STRICT_ALL_TABLES or STRICT_TRANS_TABLES is enabled, although the effects of these modes differ somewhat:

  • For transactional tables, an error occurs for invalid or missing values in a data-change statement when either STRICT_ALL_TABLES or STRICT_TRANS_TABLES is enabled. The statement is aborted and rolled back.

  • For nontransactional tables, the behavior is the same for either mode if the bad value occurs in the first row to be inserted or updated: The statement is aborted and the table remains unchanged. If the statement inserts or modifies multiple rows and the bad value occurs in the second or later row, the result depends on which strict mode is enabled:

    • For STRICT_ALL_TABLES, MySQL returns an error and ignores the rest of the rows. However, because the earlier rows have been inserted or updated, the result is a partial update. To avoid this, use single-row statements, which can be aborted without changing the table.

    • For STRICT_TRANS_TABLES, MySQL converts an invalid value to the closest valid value for the column and inserts the adjusted value. If a value is missing, MySQL inserts the implicit default value for the column data type. In either case, MySQL generates a warning rather than an error and continues processing the statement. Implicit defaults are described in Section 11.7, “Data Type Default Values”.

Strict mode affects handling of division by zero, zero dates, and zeros in dates as follows:

  • Strict mode affects handling of division by zero, which includes MOD(N,0):

    For data-change operations (INSERT, UPDATE):

    • If strict mode is not enabled, division by zero inserts NULL and produces no warning.

    • If strict mode is enabled, division by zero produces an error, unless IGNORE is given as well. For INSERT IGNORE and UPDATE IGNORE, division by zero inserts NULL and produces a warning.

    For SELECT, division by zero returns NULL. Enabling strict mode causes a warning to be produced as well.

  • Strict mode affects whether the server permits '0000-00-00' as a valid date:

    • If strict mode is not enabled, '0000-00-00' is permitted and inserts produce no warning.

    • If strict mode is enabled, '0000-00-00' is not permitted and inserts produce an error, unless IGNORE is given as well. For INSERT IGNORE and UPDATE IGNORE, '0000-00-00' is permitted and inserts produce a warning.

  • Strict mode affects whether the server permits dates in which the year part is nonzero but the month or day part is 0 (dates such as '2010-00-01' or '2010-01-00'):

    • If strict mode is not enabled, dates with zero parts are permitted and inserts produce no warning.

    • If strict mode is enabled, dates with zero parts are not permitted and inserts produce an error, unless IGNORE is given as well. For INSERT IGNORE and UPDATE IGNORE, dates with zero parts are inserted as '0000-00-00' (which is considered valid with IGNORE) and produce a warning.

For more information about strict mode with respect to IGNORE, see Comparison of the IGNORE Keyword and Strict SQL Mode.

Strict mode affects handling of division by zero, zero dates, and zeros in dates in conjunction with the ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE modes.

Comparison of the IGNORE Keyword and Strict SQL Mode

This section compares the effect on statement execution of the IGNORE keyword (which downgrades errors to warnings) and strict SQL mode (which upgrades warnings to errors). It describes which statements they affect, and which errors they apply to.

The following table presents a summary comparison of statement behavior when the default is to produce an error versus a warning. An example of when the default is to produce an error is inserting a NULL into a NOT NULL column. An example of when the default is to produce a warning is inserting a value of the wrong data type into a column (such as inserting the string 'abc' into an integer column).

Operational ModeWhen Statement Default is ErrorWhen Statement Default is Warning
Without IGNORE or strict SQL modeErrorWarning
With IGNOREWarningWarning (same as without IGNORE or strict SQL mode)
With strict SQL modeError (same as without IGNORE or strict SQL mode)Error
With IGNORE and strict SQL modeWarningWarning

One conclusion to draw from the table is that when the IGNORE keyword and strict SQL mode are both in effect, IGNORE takes precedence. This means that, although IGNORE and strict SQL mode can be considered to have opposite effects on error handling, they do not cancel when used together.

The Effect of IGNORE on Statement Execution

Several statements in MySQL support an optional IGNORE keyword. This keyword causes the server to downgrade certain types of errors and generate warnings instead. For a multiple-row statement, IGNORE causes the statement to skip to the next row instead of aborting.

For example, if the table t has a primary key column i, attempting to insert the same value of i into multiple rows normally produces a duplicate-key error:

mysql> INSERT INTO t (i) VALUES(1),(1);
ERROR 1062 (23000): Duplicate entry '1' for key 'PRIMARY'

With IGNORE, the row containing the duplicate key still is not inserted, but a warning occurs instead of an error:


mysql> INSERT IGNORE INTO t (i) VALUES(1),(1);
Query OK, 1 row affected, 1 warning (0.01 sec)
Records: 2  Duplicates: 1  Warnings: 1

mysql> SHOW WARNINGS;
+---------+------+---------------------------------------+
| Level   | Code | Message                               |
+---------+------+---------------------------------------+
| Warning | 1062 | Duplicate entry '1' for key 'PRIMARY' |
+---------+------+---------------------------------------+
1 row in set (0.00 sec)

These statements support the IGNORE keyword:

  • CREATE TABLE ... SELECT: IGNORE does not apply to the CREATE TABLE or SELECT parts of the statement but to inserts into the table of rows produced by the SELECT. Rows that duplicate an existing row on a unique key value are discarded.

  • DELETE: IGNORE causes MySQL to ignore errors during the process of deleting rows.

  • INSERT: With IGNORE, rows that duplicate an existing row on a unique key value are discarded. Rows set to values that would cause data conversion errors are set to the closest valid values instead.

    For partitioned tables where no partition matching a given value is found, IGNORE causes the insert operation to fail silently for rows containing the unmatched value.

  • LOAD DATA, LOAD XML: With IGNORE, rows that duplicate an existing row on a unique key value are discarded.

  • UPDATE: With IGNORE, rows for which duplicate-key conflicts occur on a unique key value are not updated. Rows updated to values that would cause data conversion errors are updated to the closest valid values instead.

The IGNORE keyword applies to the following errors:

ER_BAD_NULL_ERROR
ER_DUP_ENTRY
ER_DUP_ENTRY_WITH_KEY_NAME
ER_DUP_KEY
ER_NO_PARTITION_FOR_GIVEN_VALUE
ER_NO_PARTITION_FOR_GIVEN_VALUE_SILENT
ER_NO_REFERENCED_ROW_2
ER_ROW_DOES_NOT_MATCH_GIVEN_PARTITION_SET
ER_ROW_IS_REFERENCED_2
ER_SUBQUERY_NO_1_ROW
ER_VIEW_CHECK_FAILED
The Effect of Strict SQL Mode on Statement Execution

The MySQL server can operate in different SQL modes, and can apply these modes differently for different clients, depending on the value of the sql_mode system variable. In strict SQL mode, the server upgrades certain warnings to errors.

For example, in non-strict SQL mode, inserting the string 'abc' into an integer column results in conversion of the value to 0 and a warning:

mysql> SET sql_mode = '';
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO t (i) VALUES('abc');
Query OK, 1 row affected, 1 warning (0.01 sec)

mysql> SHOW WARNINGS;
+---------+------+--------------------------------------------------------+
| Level   | Code | Message                                                |
+---------+------+--------------------------------------------------------+
| Warning | 1366 | Incorrect integer value: 'abc' for column 'i' at row 1 |
+---------+------+--------------------------------------------------------+
1 row in set (0.00 sec)

In strict SQL mode, the invalid value is rejected with an error:

mysql> SET sql_mode = 'STRICT_ALL_TABLES';
Query OK, 0 rows affected (0.00 sec)

mysql> INSERT INTO t (i) VALUES('abc');
ERROR 1366 (HY000): Incorrect integer value: 'abc' for column 'i' at row 1

For more information about possible settings of the sql_mode system variable, see Section 5.1.8, “Server SQL Modes”.

Strict SQL mode applies to the following statements under conditions for which some value might be out of range or an invalid row is inserted into or deleted from a table:

Within stored programs, individual statements of the types just listed execute in strict SQL mode if the program was defined while strict mode was in effect.

Strict SQL mode applies to the following errors, represent a class of errors in which an input value is either invalid or missing. A value is invalid if it has the wrong data type for the column or might be out of range. A value is missing if a new row to be inserted does not contain a value for a NOT NULL column that has no explicit DEFAULT clause in its definition.

ER_BAD_NULL_ERROR
ER_CUT_VALUE_GROUP_CONCAT
ER_DATA_TOO_LONG
ER_DATETIME_FUNCTION_OVERFLOW
ER_DIVISION_BY_ZERO
ER_INVALID_ARGUMENT_FOR_LOGARITHM
ER_NO_DEFAULT_FOR_FIELD
ER_NO_DEFAULT_FOR_VIEW_FIELD
ER_TOO_LONG_KEY
ER_TRUNCATED_WRONG_VALUE
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD
ER_WARN_DATA_OUT_OF_RANGE
ER_WARN_NULL_TO_NOTNULL
ER_WARN_TOO_FEW_RECORDS
ER_WRONG_ARGUMENTS
ER_WRONG_VALUE_FOR_TYPE
WARN_DATA_TRUNCATED

SQL Mode Changes in MySQL 5.7

In MySQL 5.7.5, the ONLY_FULL_GROUP_BY SQL mode is enabled by default because GROUP BY processing has become more sophisticated to include detection of functional dependencies. However, if you find that having ONLY_FULL_GROUP_BY enabled causes queries for existing applications to be rejected, either of these actions should restore operation:

  • If it is possible to modify an offending query, do so, either so that nonaggregated columns are functionally dependent on GROUP BY columns, or by referring to nonaggregated columns using ANY_VALUE().

  • If it is not possible to modify an offending query (for example, if it is generated by a third-party application), set the sql_mode system variable at server startup to not enable ONLY_FULL_GROUP_BY.

As of MySQL 5.7.4, the ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE SQL modes are deprecated. From MySQL 5.7.4 though 5.7.7, these modes do nothing when named explicitly. Instead, their effects are included in the effects of strict SQL mode (STRICT_ALL_TABLES or STRICT_TRANS_TABLES). In other words, strict mode means the same thing in those versions as the pre-5.7.4 meaning of strict mode plus ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE.

The MySQL 5.7.4 change to make strict mode more strict by including ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE caused some problems. For example, in MySQL 5.6 with strict mode but not NO_ZERO_DATE enabled, TIMESTAMP columns can be defined with DEFAULT '0000-00-00 00:00:00'. In MySQL 5.7.4 with the same mode settings, strict mode includes the effect of NO_ZERO_DATE and TIMESTAMP columns cannot be defined with DEFAULT '0000-00-00 00:00:00'. This causes replication of CREATE TABLE statements from 5.6 to 5.7.4 to fail if they contain such TIMESTAMP columns.

The long term plan is still to have the three affected modes be included in strict SQL mode and to remove them as explicit modes in a future MySQL release. But to restore compatibility in MySQL 5.7 with MySQL 5.6 strict mode and to provide additional time for affected applications to be modified, the following changes were made in MySQL 5.7.8:

With the preceding changes, stricter data checking is still enabled by default, but the individual modes can be disabled in environments where it is currently desirable or necessary to do so.

Although in MySQL 5.7.8 and later ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE can be used separately from strict mode, it is intended that they be used together. As a reminder, a warning occurs if they are enabled without also enabling strict mode or vice versa.

Important

The following discussion applies only for MySQL versions 5.7.4 through 5.7.7. For upgrades from a version older than MySQL 5.7.4, we recommend upgrading to MySQL 5.7.8 or later, which renders this discussion unnecessary.

The remainder of this section describes the SQL mode settings to use in MySQL 5.7.4 through 5.7.7 to achieve the same statement execution as before 5.7.4, including the cases for INSERT and UPDATE in which IGNORE is given. It also provides guidelines for determining whether applications need modification to behave the same before and after the SQL mode changes.

The following table shows how to control handling of division by zero for versions other than MySQL 5.7.4 through 5.7.7 and for MySQL 5.7.4 through 5.7.7.

Desired BehaviorMySQL 5.7.x Versions Except 5.7.4 Through 5.7.7MySQL 5.7.4 Through 5.7.7
insert NULL, produce no warningERROR_FOR_DIVISION_BY_ZERO not enabledstrict mode not enabled
insert NULL, produce warningERROR_FOR_DIVISION_BY_ZERO, or ERROR_FOR_DIVISION_BY_ZERO + strict mode + IGNOREstrict mode + IGNORE
errorERROR_FOR_DIVISION_BY_ZERO + strict modestrict mode

The following table shows how to control whether the server permits '0000-00-00' as a valid date for versions other than MySQL 5.7.4 through 5.7.7 and for MySQL 5.7.4 through 5.7.7.

Desired BehaviorMySQL 5.7.x Versions Except 5.7.4 Through 5.7.7MySQL 5.7.4 Through 5.7.7
insert '0000-00-00', produce no warningNO_ZERO_DATE not enabledstrict mode not enabled
insert '0000-00-00', produce warningNO_ZERO_DATE, or NO_ZERO_DATE + strict mode + IGNOREstrict mode + IGNORE
errorNO_ZERO_DATE + strict modestrict mode

The following table shows how to control whether the server permits dates with zero parts for versions other than MySQL 5.7.4 through 5.7.7 and for MySQL 5.7.4 through 5.7.7.

Desired BehaviorMySQL 5.7.x Versions Except 5.7.4 Through 5.7.7MySQL 5.7.4 Through 5.7.7
insert date, produce no warningNO_ZERO_IN_DATE not enabledstrict mode not enabled
insert '0000-00-00', produce warningNO_ZERO_IN_DATE, or NO_ZERO_IN_DATE + strict mode + IGNOREstrict mode + IGNORE
errorNO_ZERO_IN_DATE + strict modestrict mode

The following discussion describes the conditions under which a given statement produces the same or different result under the SQL mode changes in MySQL 5.7.4 through 5.7.7. It considers only strict mode (STRICT_ALL_TABLES or STRICT_TRANS_TABLES) and the three deprecated modes (ERROR_FOR_DIVISION_BY_ZERO, NO_ZERO_DATE, and NO_ZERO_IN_DATE). Other SQL modes such as ANSI_QUOTES or ONLY_FULL_GROUP_BY are assumed to be held constant before and after an upgrade.

This discussion also describes how to prepare for an upgrade to 5.7.4 through 5.7.7 from a version older than 5.7.4. Any modifications should be made before upgrading.

There is no change in behavior between MySQL 5.6 and 5.7 for the following SQL mode settings. A statement that executes under one of these settings needs no modification to produce the same result in 5.6 and 5.7:

  • Strict mode and the three deprecated modes are all not enabled.

  • Strict mode and the three deprecated modes are all enabled.

A change from warnings in MySQL 5.6 to no warnings in MySQL 5.7 occurs for the following SQL mode settings. The result of statement execution is the same in 5.6 and 5.7, so statements need no modification unless warnings are considered significant:

A behavior change occurs under the following SQL mode settings. A statement that executes under one of these settings must be modified to produce the same result in 5.6 and 5.7:

  • Strict mode is not enabled, NO_ZERO_IN_DATE is enabled. For this mode setting, expect these differences in statement execution:

    • In 5.6, the server inserts dates with zero parts as '0000-00-00' and produces a warning.

    • In 5.7, the server inserts dates with zero parts as is and produces no warning.

  • Strict mode is enabled, with some but not all of the three deprecated modes enabled. For this mode setting, expect these differences in statement execution:

    Statements that would be affected by enabling the not-enabled deprecated modes produce errors in 5.7 but not in 5.6. Suppose that strict mode, NO_ZERO_DATE, and NO_ZERO_IN_DATE are enabled, and a data-change statement performs division by zero:

To prepare for an upgrade to MySQL 5.7.4 through 5.7.7, the main principle is to make sure that your applications will operate the same way in MySQL 5.6 and 5.7. For example, you can adopt either of these approaches to application compatibility:

  • Modify the application to set the SQL mode on a version-specific basis. If we assume that an application will not be used with development versions of MySQL 5.7 prior to 5.7.4, it is possible to set the sql_mode value for the application based on the current server version as follows:

    SET sql_mode = IF(LEFT(VERSION(),3)<'5.7',5.6 mode,5.7 mode);
    

    The tables shown earlier in this section serve as a guide to the appropriate equivalent modes for MySQL 5.6 and 5.7.

  • Modify the application to execute under an SQL mode for which statements produce the same result in MySQL 5.6 and 5.7.

    Tip

    TRADITIONAL SQL mode in MySQL 5.6 includes strict mode and the three deprecated modes. If you write applications to operate in TRADITIONAL mode in MySQL 5.6, there is no change to make for MySQL 5.7.

When assessing SQL mode compatibility between MySQL 5.6 and 5.7, consider particularly these statement execution contexts:

  • Replication. You will encounter replication incompatibility related to the SQL mode changes under the following conditions:

    • MySQL 5.6 master and 5.7 slave

    • Statement-based replication

    • An SQL mode setting for which statements produce different results in MySQL 5.6 and 5.7, as described earlier

    To handle this incompatibility, use one of these workarounds:

    • Use row-based replication

    • Use IGNORE

    • Use an SQL mode for which statements do not produce different results in MySQL 5.6 and 5.7

  • Stored programs (stored procedures and functions, triggers, and events). Each stored program executes using the SQL mode in effect at the time it was created. To identify stored programs that may be affected by differences between MySQL 5.6 and 5.7 in SQL mode handling, use these queries:

    SELECT ROUTINE_SCHEMA, ROUTINE_NAME, ROUTINE_TYPE, SQL_MODE
    FROM INFORMATION_SCHEMA.ROUTINES
    WHERE SQL_MODE LIKE '%STRICT%'
    OR SQL_MODE LIKE '%DIVISION%'
    OR SQL_MODE LIKE '%NO_ZERO%';
    
    SELECT TRIGGER_SCHEMA, TRIGGER_NAME, SQL_MODE
    FROM INFORMATION_SCHEMA.TRIGGERS
    WHERE SQL_MODE LIKE '%STRICT%'
    OR SQL_MODE LIKE '%DIVISION%'
    OR SQL_MODE LIKE '%NO_ZERO%';
    
    SELECT EVENT_SCHEMA, EVENT_NAME, SQL_MODE
    FROM INFORMATION_SCHEMA.EVENTS
    WHERE SQL_MODE LIKE '%STRICT%'
    OR SQL_MODE LIKE '%DIVISION%'
    OR SQL_MODE LIKE '%NO_ZERO%';
    

5.1.9 IPv6 Support

Support for IPv6 in MySQL includes these capabilities:

The following sections describe how to set up MySQL so that clients can connect to the server over IPv6.

5.1.9.1 Verifying System Support for IPv6

Before MySQL Server can accept IPv6 connections, the operating system on your server host must support IPv6. As a simple test to determine whether that is true, try this command:

shell> ping6 ::1
16 bytes from ::1, icmp_seq=0 hlim=64 time=0.171 ms
16 bytes from ::1, icmp_seq=1 hlim=64 time=0.077 ms
...

To produce a description of your system's network interfaces, invoke ifconfig -a and look for IPv6 addresses in the output.

If your host does not support IPv6, consult your system documentation for instructions on enabling it. It might be that you need only reconfigure an existing network interface to add an IPv6 address. Or a more extensive change might be needed, such as rebuilding the kernel with IPv6 options enabled.

These links may be helpful in setting up IPv6 on various platforms:

5.1.9.2 Configuring the MySQL Server to Permit IPv6 Connections

The MySQL server listens on a single network socket for TCP/IP connections. This socket is bound to a single address, but it is possible for an address to map onto multiple network interfaces. To specify an address, use the --bind-address=addr option at server startup, where addr is an IPv4 or IPv6 address or a host name. If addr is a host name, the server resolves the name to an IP address and binds to that address.

The server treats different types of addresses as follows:

  • If the address is *, the server accepts TCP/IP connections on all server host IPv6 and IPv4 interfaces if the server host supports IPv6, or accepts TCP/IP connections on all IPv4 addresses otherwise. Use this address to permit both IPv4 and IPv6 connections on all server interfaces. This value is the default.

  • If the address is 0.0.0.0, the server accepts TCP/IP connections on all server host IPv4 interfaces.

  • If the address is ::, the server accepts TCP/IP connections on all server host IPv4 and IPv6 interfaces. Use this address to permit both IPv4 and IPv6 connections on all server interfaces.

  • If the address is an IPv4-mapped address, the server accepts TCP/IP connections for that address, in either IPv4 or IPv6 format. For example, if the server is bound to ::ffff:127.0.0.1, clients can connect using --host=127.0.0.1 or --host=::ffff:127.0.0.1.

  • If the address is a regular IPv4 or IPv6 address (such as 127.0.0.1 or ::1), the server accepts TCP/IP connections only for that IPv4 or IPv6 address.

If you intend to bind the server to a specific address, be sure that the mysql.user grant table contains an account with administrative privileges that you can use to connect to that address. Otherwise, you will not be able to shut down the server. For example, if you bind the server to *, you can connect to it using all existing accounts. But if you bind the server to ::1, it accepts connections only on that address. In that case, first make sure that the 'root'@'::1' account is present in the mysql.user table so you can still connect to the server to shut it down.

5.1.9.3 Connecting Using the IPv6 Local Host Address

The following procedure shows how to configure MySQL to permit IPv6 connections by clients that connect to the local server using the ::1 local host address. The instructions given here assume that your system supports IPv6.

  1. Start the MySQL server with an appropriate --bind-address option to permit it to accept IPv6 connections. For example, put the following lines in your server option file and restart the server:

    [mysqld]
    bind-address = *
    

    Alternatively, you can bind the server to ::1, but that makes the server more restrictive for TCP/IP connections. It accepts only IPv6 connections for that single address and rejects IPv4 connections. For more information, see Section 5.1.9.2, “Configuring the MySQL Server to Permit IPv6 Connections”.

  2. As an administrator, connect to the server and create an account for a local user who will connect from the ::1 local IPv6 host address:

    mysql> CREATE USER 'ipv6user'@'::1' IDENTIFIED BY 'ipv6pass';
    

    For the permitted syntax of IPv6 addresses in account names, see Section 6.2.4, “Specifying Account Names”. In addition to the CREATE USER statement, you can issue GRANT statements that give specific privileges to the account, although that is not necessary for the remaining steps in this procedure.

  3. Invoke the mysql client to connect to the server using the new account:

    shell> mysql -h ::1 -u ipv6user -pipv6pass
    
  4. Try some simple statements that show connection information:

    mysql> STATUS
    ...
    Connection:   ::1 via TCP/IP
    ...
    
    mysql> SELECT CURRENT_USER(), @@bind_address;
    +----------------+----------------+
    | CURRENT_USER() | @@bind_address |
    +----------------+----------------+
    | ipv6user@::1   | ::             |
    +----------------+----------------+
    

5.1.9.4 Connecting Using IPv6 Nonlocal Host Addresses

The following procedure shows how to configure MySQL to permit IPv6 connections by remote clients. It is similar to the preceding procedure for local clients, but the server and client hosts are distinct and each has its own nonlocal IPv6 address. The example uses these addresses:

Server host: 2001:db8:0:f101::1
Client host: 2001:db8:0:f101::2

These addresses are chosen from the nonroutable address range recommended by IANA for documentation purposes and suffice for testing on your local network. To accept IPv6 connections from clients outside the local network, the server host must have a public address. If your network provider assigns you an IPv6 address, you can use that. Otherwise, another way to obtain an address is to use an IPv6 broker; see Section 5.1.9.5, “Obtaining an IPv6 Address from a Broker”.

  1. Start the MySQL server with an appropriate --bind-address option to permit it to accept IPv6 connections. For example, put the following lines in your server option file and restart the server:

    [mysqld]
    bind-address = *
    

    Alternatively, you can bind the server to 2001:db8:0:f101::1, but that makes the server more restrictive for TCP/IP connections. It accepts only IPv6 connections for that single address and rejects IPv4 connections. For more information, see Section 5.1.9.2, “Configuring the MySQL Server to Permit IPv6 Connections”.

  2. On the server host (2001:db8:0:f101::1), create an account for a user who will connect from the client host (2001:db8:0:f101::2):

    mysql> CREATE USER 'remoteipv6user'@'2001:db8:0:f101::2' IDENTIFIED BY 'remoteipv6pass';
    
  3. On the client host (2001:db8:0:f101::2), invoke the mysql client to connect to the server using the new account:

    shell> mysql -h 2001:db8:0:f101::1 -u remoteipv6user -premoteipv6pass
    
  4. Try some simple statements that show connection information:

    mysql> STATUS
    ...
    Connection:   2001:db8:0:f101::1 via TCP/IP
    ...
    
    mysql> SELECT CURRENT_USER(), @@bind_address;
    +-----------------------------------+----------------+
    | CURRENT_USER()                    | @@bind_address |
    +-----------------------------------+----------------+
    | remoteipv6user@2001:db8:0:f101::2 | ::             |
    +-----------------------------------+----------------+
    

5.1.9.5 Obtaining an IPv6 Address from a Broker

If you do not have a public IPv6 address that enables your system to communicate over IPv6 outside your local network, you can obtain one from an IPv6 broker. The Wikipedia IPv6 Tunnel Broker page lists several brokers and their features, such as whether they provide static addresses and the supported routing protocols.

After configuring your server host to use a broker-supplied IPv6 address, start the MySQL server with an appropriate --bind-address option to permit the server to accept IPv6 connections. For example, put the following lines in the server option file and restart the server:

[mysqld]
bind-address = *

Alternatively, you can bind the server to the specific IPv6 address provided by the broker, but that makes the server more restrictive for TCP/IP connections. It accepts only IPv6 connections for that single address and rejects IPv4 connections. For more information, see Section 5.1.9.2, “Configuring the MySQL Server to Permit IPv6 Connections”. In addition, if the broker allocates dynamic addresses, the address provided for your system might change the next time you connect to the broker. If so, any accounts you create that name the original address become invalid. To bind to a specific address but avoid this change-of-address problem, you may be able to arrange with the broker for a static IPv6 address.

The following example shows how to use Freenet6 as the broker and the gogoc IPv6 client package on Gentoo Linux.

  1. Create an account at Freenet6 by visiting this URL and signing up:

    http://gogonet.gogo6.com
    
  2. After creating the account, go to this URL, sign in, and create a user ID and password for the IPv6 broker:

    http://gogonet.gogo6.com/page/freenet6-registration
    

  3. As root, install gogoc:

    shell> emerge gogoc
    
  4. Edit /etc/gogoc/gogoc.conf to set the userid and password values. For example:

    userid=gogouser
    passwd=gogopass
    
  5. Start gogoc:

    shell> /etc/init.d/gogoc start
    

    To start gogoc each time your system boots, execute this command:

    shell> rc-update add gogoc default
    
  6. Use ping6 to try to ping a host:

    shell> ping6 ipv6.google.com
    
  7. To see your IPv6 address:

    shell> ifconfig tun
    

5.1.10 Server-Side Help

MySQL Server supports a HELP statement that returns information from the MySQL Reference manual (see Section 13.8.3, “HELP Syntax”). Several tables in the mysql system database contain the information needed to support this statement (see Section 5.3, “The mysql System Database”). The proper operation of this statement requires that these help tables be initialized, which is done by processing the contents of the fill_help_tables.sql script.

If you install MySQL using a binary or source distribution on Unix, help table content initialization occurs when you initialize the data directory (see Section 2.9.1, “Initializing the Data Directory”). For an RPM distribution on Linux or binary distribution on Windows, content initialization occurs as part of the MySQL installation process.

If you upgrade MySQL using a binary distribution, help table content is not upgraded automatically, but you can upgrade it manually. Locate the fill_help_tables.sql file in the share or share/mysql directory. Change location into that directory and process the file with the mysql client as follows:

shell> mysql -u root mysql < fill_help_tables.sql

You can also obtain the latest fill_help_tables.sql at any time to upgrade your help tables. Download the proper file for your version of MySQL from http://dev.mysql.com/doc/index-other.html. After downloading and uncompressing the file, process it with mysql as described previously.

If you are working with Git and a MySQL development source tree, you must use a downloaded copy of the fill_help_tables.sql file because the source tree contains only a stub version.

Note

For a server that participates in replication, the help table content upgrade process involves multiple servers. For details, see Section 18.4.1.29, “Replication of Server-Side Help Tables”.

5.1.11 Server Response to Signals

On Unix, signals can be sent to processes. mysqld responds to signals sent to it as follows:

  • SIGTERM causes the server to shut down.

  • SIGHUP causes the server to reload the grant tables and to flush tables, logs, the thread cache, and the host cache. These actions are like various forms of the FLUSH statement. The server also writes a status report to the error log that has this format:

    Status information:
    
    Current dir: /var/mysql/data/
    Running threads: 0  Stack size: 196608
    Current locks:
    
    Key caches:
    default
    Buffer_size:       8388600
    Block_size:           1024
    Division_limit:        100
    Age_limit:             300
    blocks used:             0
    not flushed:             0
    w_requests:              0
    writes:                  0
    r_requests:              0
    reads:                   0
    
    handler status:
    read_key:            0
    read_next:           0
    read_rnd             0
    read_first:          1
    write:               0
    delete               0
    update:              0
    
    Table status:
    Opened tables:          5
    Open tables:            0
    Open files:             7
    Open streams:           0
    
    Alarm status:
    Active alarms:   1
    Max used alarms: 2
    Next alarm time: 67
    

5.1.12 服务器关闭过程

服务器关闭过程如下:

  1. 开始服务器关闭过程。

    This can occur initiated several ways. For example, a user with the SHUTDOWN privilege can execute a mysqladmin shutdown command. mysqladmin can be used on any platform supported by MySQL. Other operating system-specific shutdown initiation methods are possible as well: The server shuts down on Unix when it receives a SIGTERM signal. A server running as a service on Windows shuts down when the services manager tells it to.

  2. 如果有需要,服务器会创建一个关闭线程。

    Depending on how shutdown was initiated, the server might create a thread to handle the shutdown process. If shutdown was requested by a client, a shutdown thread is created. If shutdown is the result of receiving a SIGTERM signal, the signal thread might handle shutdown itself, or it might create a separate thread to do so. If the server tries to create a shutdown thread and cannot (for example, if memory is exhausted), it issues a diagnostic message that appears in the error log:

    Error: Can't create thread to kill server
    
  3. 服务器停止接受新的连接。

    To prevent new activity from being initiated during shutdown, the server stops accepting new client connections by closing the handlers for the network interfaces to which it normally listens for connections: the TCP/IP port, the Unix socket file, the Windows named pipe, and shared memory on Windows.

  4. The server terminates current activity.

    For each thread associated with a client connection, the server breaks the connection to the client and marks the thread as killed. Threads die when they notice that they are so marked. Threads for idle connections die quickly. Threads that currently are processing statements check their state periodically and take longer to die. For additional information about thread termination, see Section 13.7.6.4, “KILL Syntax”, in particular for the instructions about killed REPAIR TABLE or OPTIMIZE TABLE operations on MyISAM tables.

    For threads that have an open transaction, the transaction is rolled back. If a thread is updating a nontransactional table, an operation such as a multiple-row UPDATE or INSERT may leave the table partially updated because the operation can terminate before completion.

    If the server is a master replication server, it treats threads associated with currently connected slaves like other client threads. That is, each one is marked as killed and exits when it next checks its state.

    If the server is a slave replication server, it stops the I/O and SQL threads, if they are active, before marking client threads as killed. The SQL thread is permitted to finish its current statement (to avoid causing replication problems), and then stops. If the SQL thread is in the middle of a transaction at this point, the server waits until the current replication event group (if any) has finished executing, or until the user issues a KILL QUERY or KILL CONNECTION statement. See also Section 13.4.2.7, “STOP SLAVE Syntax”. Since nontransactional statements cannot be rolled back, in order to guarantee crash-safe replication, only transactional tables should be used.

    注意

    To guarantee crash safety on the slave,您必须在运行salve时启用 --relay-log-recovery来。

    还可以参阅,18.2.4 节, “复制的中继和状态日志”)。

  5. 服务器关闭,或关闭存储引擎。

    到这一步,服务器刷新表的缓存,并关闭所有的表。

    Each storage engine performs any actions necessary for tables that it manages. InnoDB flushes its buffer pool to disk (unless innodb_fast_shutdown is 2), writes the current LSN to the tablespace, and terminates its own internal threads. MyISAM flushes any pending index writes for a table.

  6. 服务器退出。

服务器在退出时,会返回下面列举之一的代码,以给管理的过程提供相关信息。在使用系统命令管理服务器的平台,括号中的短语表示系统为响应代码所采取的操作。

  • 0 = 成功的终端 (没有重启)

  • 1 = 未成功终端 (没有完成重启)

  • 2 = 未成功终端 (重启完成)

5.2 MySQL 数据目录

MySQL服务器管理的信息存储在一个名为data的目录下。下面的列表简要地描述了在数据目录中找到的项目,并给出了相关应用信息:

  • 数据目录下的子目录。数据目录下的的每个子目录都是一个库目录,且对应于被服务器管理的库。安装的每个MySQL库,都是特定的标准库:

    • mysql目录对应于 mysql 系统库,它包含了MySQL服务器在运行时所需要的信息。这个库包含了数据字典表和系统表。参阅 5.3 节, “mysql 系统库”

    • performance_schema 目录对应于 Performance Schema,它提供了用于在服务器运行时,检查服务器内部执行的信息。 第25章, MySQL Performance Schema

    • The sys 目录对应于 sys schema,它提供一组对象,以帮助更容易地解释性能模式信息。 参阅第26章, MySQL sys Schema.

    数据库的其它子目录都是对应于用户和应用创建的。

    注意

    INFORMATION_SCHEMA 是一个标准库,但是它的实现,没有使用相应对的库目录。

  • 服务的日志文件,参阅 5.4 节, “MySQL 服务器日志”

  • InnoDB 表空间和日志文件,参阅 第15章, InnoDB 存储引擎

  • 默认/自动产生的 SSL 和 RSA 认证文件和密钥文件。 See 6.4.3 节, “创建SSL和RSA认证及密钥”

  • 服务器进程ID文件(当服务器正在运行时)。

  • mysqld-auto.cnf文件是用来保存持久化全局系统变量的设置,参阅 5.1.6 节, “使用系统变量”

前面列表中的一些项可以通过重新配置服务器来转移到其他地方。此外, --datadir 选项,是用来更改数据目录本身的位置。对于一个已安装的MySQL, 可以通过检查服务器配置,以确定条目是否被移动过。

5.3 mysql系统库

mysql库是系统级别的库,它包含了存储MySQL运行所需要信息的表。 宽泛点来说,mysql库,含有存储数据库对象元数据的数据字典表,用于其它操作目的的系统表。下面更细致的探讨这些系统表。

下面分别列举每个类别下的表,没有额外说明,那么数据字典表和系统表都是使用InnoDB存储引擎。

mysql系统表和数据字典表是驻留在一个名为mysql.ibd的单独 在一个单独的InnoDB 表空间文件中。以前,这些表是 在mysql 库目录下,且被创建为单独的表空间文件。

数据字典表

这些表包含了含有数据库对象元数据的数据字典表,更多信息,参阅, 第14章, MySQL 数据字典.

重要

The data dictionary is new in MySQL 8.0. A data dictionary-enabled server entails some general operational differences compared to previous MySQL releases. For details, see Section 14.6, “Data Dictionary Usage Differences”. Also, for upgrades to MySQL 8.0 from MySQL 5.7, the upgrade procedure differs somewhat from previous MySQL releases and requires that you verify the upgrade readiness of your installation by checking specific prerequisites. For more information, see Section 2.10.1, “Upgrading MySQL”, particularly Verifying Upgrade Prerequisites for Your MySQL 5.7 Installation.

  • catalogs: 目录信息。

  • character_sets: 可用字符集的信息。

  • collations: 关于字符集校对的信息。

  • column_statistics:列值的直方图统计。参阅 8.9.6, “优化器统计信息”

  • column_type_elements: 列使用的类型的信息

  • columns: 列所在的表的信息。

  • dd_properties: 一个标识数据字典内容的表。如,其版本。服务器使用它来决定数据字典是否必须升级到更新的版本。

  • events: Information about Event Scheduler events. See  23.4 节, “Using the Event Scheduler”. The server loads events listed in this table during its startup sequence, unless started with the --skip-grant-tables option.

  • foreign_keys, foreign_key_column_usage: Information about foreign keys.

  • index_column_usage: Information about columns used by indexes.

  • index_partitions: Information about partitions used by indexes.

  • index_stats: Used to store dynamic index statistics generated when ANALYZE TABLE is executed.

  • indexes: Information about table indexes.

  • parameter_type_elements: Information about stored procedure and function parameters, and about return values for stored functions.

  • parameters: Information about stored procedures and functions.

  • routines: Information about stored procedures and functions.

  • schemata: Information about schemata. In MySQL, a schema is a database, so this table provides information about databases.

  • st_spatial_reference_systems: Information about available spatial reference systems for spatial data.

  • table_partition_values: Information about values used by table partitions.

  • table_partitions: Information about partitions used by tables.

  • table_stats: Information about dynamic table statistics generated when ANALYZE TABLE is executed.

  • tables: Information about tables in databases.

  • tablespace_files: Information about files used by tablespaces.

  • tablespaces: Information about active tablespaces.

  • triggers: Information about triggers.

  • view_routine_usage: Information about dependencies between views and stored functions used by them.

  • view_table_usage: Used to track dependencies between views and their underlying tables.

Data dictionary tables are invisible. They cannot be read with SELECT, do not appear in the output of SHOW TABLES, are not listed in the INFORMATION_SCHEMA.TABLES table, and so forth. However, in most cases there are corresponding INFORMATION_SCHEMA tables that can be queried. Conceptually, the INFORMATION_SCHEMA provides a view through which MySQL exposes data dictionary metadata. For example, you cannot select from the mysql.schemata table directly:

mysql> SELECT * FROM mysql.schemata;
ERROR 3554 (HY000): Access to data dictionary table 'mysql.schemata' is rejected.

Instead, select that information from the corresponding INFORMATION_SCHEMA table:

mysql> SELECT * FROM INFORMATION_SCHEMA.SCHEMATA\G
*************************** 1. row ***************************
              CATALOG_NAME: def
               SCHEMA_NAME: mysql
DEFAULT_CHARACTER_SET_NAME: utf8mb4
    DEFAULT_COLLATION_NAME: utf8mb4_0900_ai_ci
                  SQL_PATH: NULL
*************************** 2. row ***************************
              CATALOG_NAME: def
               SCHEMA_NAME: information_schema
DEFAULT_CHARACTER_SET_NAME: utf8
    DEFAULT_COLLATION_NAME: utf8_general_ci
                  SQL_PATH: NULL
...

There is no INFORMATION_SCHEMA table that corresponds exactly to mysql.indexes, but INFORMATION_SCHEMA.STATISTICS contains much of the same information.

As of yet, there are no INFORMATION_SCHEMA tables that correspond exactly to mysql.foreign_keys, mysql.foreign_key_column_usage. The standard SQL way to obtain foreign key information is by using the INFORMATION_SCHEMA REFERENTIAL_CONSTRAINTS and KEY_COLUMN_USAGE tables; these tables are now implemented as views on the foreign_keys, foreign_key_column_usage, and other data dictionary tables.

Some system tables from before MySQL 8.0 have been replaced by data dictionary tables and are no longer present in the mysql system database:

  • The events data dictionary table supersedes the event table from before MySQL 8.0.

  • The parameters and routines data dictionary tables together supersede the proc table from before MySQL 8.0.

Grant System Tables

These system tables contain grant information about user accounts and the privileges held by them. For additional information about the structure, contents, and purpose of the these tables, see Section 6.2.3, “Grant Tables”.

As of MySQL 8.0, the grant tables are InnoDB (transactional) tables. Previously, these were MyISAM (nontransactional) tables. The change of grant-table storage engine underlies an accompanying change in MySQL 8.0 to the behavior of account-management statements such as CREATE USER and GRANT. Previously, an account-management statement that named multiple users could succeed for some users and fail for others. The statements are now transactional and either succeed for all named users or roll back and have no effect if any error occurs.

Note

If MySQL is upgraded from an older version but the grant tables have not been upgraded from MyISAM to InnoDB, the server considers them read only and account-management statements produce an error. For upgrade instructions, see Section 2.10.1, “Upgrading MySQL”.

  • user: User accounts, global privileges, and other non-privilege columns.

  • global_grants: Assignments of dynamic global privileges to users; see Section 6.2.2, “Static Versus Dynamic Privileges”.

  • db: Database-level privileges.

  • tables_priv: Table-level privileges.

  • columns_priv: Column-level privileges.

  • procs_priv: Stored procedure and function privileges.

  • proxies_priv: Proxy-user privileges.

  • default_roles: This table lists default roles to be activated after a user connects and authenticates, or executes SET ROLE DEFAULT.

  • role_edges: This table lists edges for role subgraphs.

    A given user table row might refer to a user account or a role. The server can distinquish whether a row represents a user account, a role, or both by consulting the role_edges table for information about relations between authentication IDs.

对象信息表

These system tables contain information about stored programs, components, user-defined functions, and server-side plugins:

日志表

服务器用这些表来记录日志:

  • general_log: 普通日志表。

  • slow_log: 慢查询日志表。

日志表使用的是CSV 存储引擎。

更多信息,参阅5.4 节, “MySQL 服务器日志”

服务器端帮助表

这些系统表包含服务器端帮助信息:

  • help_category: 关于帮助类别的信息。

  • help_keyword:帮助主题的相关关键字。

  • help_relation: 帮助关键字和主题之间的映射。

  • help_topic: 帮助主题内容。

更多信息,参阅 5.1.10 节, “服务器端帮助”.

时区表

这些系统表含有时区信息:

  • time_zone: 时区ID及是否使用闰秒。

  • time_zone_leap_second: 闰秒发生时。

  • time_zone_name: 绘制时区ID和名称。

  • time_zone_transition, time_zone_transition_type: 时区描述。

更多信息,请参阅10.6 节, “MySQL 服务器时区支持”

复制表

服务器用这些系统表来支持复制:

优化器表

这些系统表是优化器使用的:

其它表

其它不符合前面类别的表:

  • servers: FEDERATED 存储引擎使用的。参阅 16.8.2.2, “使用CREATE SERVER创建一个FEDERATED 表”

  • innodb_dynamic_metadata: 被 InnoDB 存储引擎用来存储快速变化表的元数据。 如,自动增长器的值或索引树失效标识。替代了驻留在InnoDB系统表空间中的数据字典缓冲表。

5.4 MySQL服务器日志

MySQL 服务器有多个日志,可以用来帮助你超出问题的所在。

日志类型写入日志的信息
错误日志启动、运行和关闭 mysqld时遇到的问题
普通查询日志建立客户端连接和从客户端接收的语句
二进制日志变更数据的语句(也在复制中使用 )
中继日志接收来自复制中master服务器的数据变化
慢查询日志记录执行查询时间超过 long_query_time的。
DDL 日志(元数据日志)由DDL语句执行的元数据操作

默认情况,是没有启用日志,除了Windows上启用了错误日志。 (The DDL log is always created when required, and has no user-configurable options; see Section 5.4.6, “The DDL Log”.) The following log-specific sections provide information about the server options that enable logging.

By default, the server writes files for all enabled logs in the data directory. You can force the server to close and reopen the log files (or in some cases switch to a new log file) by flushing the logs. Log flushing occurs when you issue a FLUSH LOGS statement; execute mysqladmin with a flush-logs or refresh argument; or execute mysqldump with a --flush-logs or --master-data option. See Section 13.7.6.3, “FLUSH Syntax”, Section 4.5.2, “mysqladmin — Client for Administering a MySQL Server”, and Section 4.5.4, “mysqldump — A Database Backup Program”. In addition, the binary log is flushed when its size reaches the value of the max_binlog_size system variable.

You can control the general query and slow query logs during runtime. You can enable or disable logging, or change the log file name. You can tell the server to write general query and slow query entries to log tables, log files, or both. For details, see Section 5.4.1, “Selecting General Query and Slow Query Log Output Destinations”, Section 5.4.3, “The General Query Log”, and Section 5.4.5, “The Slow Query Log”.

The relay log is used only on slave replication servers, to hold data changes from the master server that must also be made on the slave. For discussion of relay log contents and configuration, see Section 18.2.4.1, “The Slave Relay Log”.

For information about log maintenance operations such as expiration of old log files, see Section 5.4.7, “Server Log Maintenance”.

For information about keeping logs secure, see Section 6.1.2.3, “Passwords and Logging”.

5.4.1 Selecting General Query and Slow Query Log Output Destinations

MySQL Server provides flexible control over the destination of output to the general query log and the slow query log, if those logs are enabled. Possible destinations for log entries are log files or the general_log and slow_log tables in the mysql database. Either or both destinations can be selected.

Log control at server startup. The --log-output option specifies the destination for log output. This option does not in itself enable the logs. Its syntax is --log-output[=value,...]:

  • If --log-output is given with a value, the value should be a comma-separated list of one or more of the words TABLE (log to tables), FILE (log to files), or NONE (do not log to tables or files). NONE, if present, takes precedence over any other specifiers.

  • If --log-output is omitted, the default logging destination is FILE.

The general_log system variable controls logging to the general query log for the selected log destinations. If specified at server startup, general_log takes an optional argument of 1 or 0 to enable or disable the log. To specify a file name other than the default for file logging, set the general_log_file variable. Similarly, the slow_query_log variable controls logging to the slow query log for the selected destinations and setting slow_query_log_file specifies a file name for file logging. If either log is enabled, the server opens the corresponding log file and writes startup messages to it. However, further logging of queries to the file does not occur unless the FILE log destination is selected.

Examples:

  • To write general query log entries to the log table and the log file, use --log-output=TABLE,FILE to select both log destinations and --general_log to enable the general query log.

  • To write general and slow query log entries only to the log tables, use --log-output=TABLE to select tables as the log destination and --general_log and --slow_query_log to enable both logs.

  • To write slow query log entries only to the log file, use --log-output=FILE to select files as the log destination and --slow_query_log to enable the slow query log. (In this case, because the default log destination is FILE, you could omit the --log-output option.)

Log control at runtime. The system variables associated with log tables and files enable runtime control over logging:

  • The global log_output system variable indicates the current logging destination. It can be modified at runtime to change the destination.

  • The global general_log and slow_query_log variables indicate whether the general query log and slow query log are enabled (ON) or disabled (OFF). You can set these variables at runtime to control whether the logs are enabled.

  • The global general_log_file and slow_query_log_file variables indicate the names of the general query log and slow query log files. You can set these variables at server startup or at runtime to change the names of the log files.

  • To disable or enable general query logging for the current connection, set the session sql_log_off variable to ON or OFF.

The use of tables for log output offers the following benefits:

  • Log entries have a standard format. To display the current structure of the log tables, use these statements:

    SHOW CREATE TABLE mysql.general_log;
    SHOW CREATE TABLE mysql.slow_log;
    
  • Log contents are accessible through SQL statements. This enables the use of queries that select only those log entries that satisfy specific criteria. For example, to select log contents associated with a particular client (which can be useful for identifying problematic queries from that client), it is easier to do this using a log table than a log file.

  • Logs are accessible remotely through any client that can connect to the server and issue queries (if the client has the appropriate log table privileges). It is not necessary to log in to the server host and directly access the file system.

The log table implementation has the following characteristics:

  • In general, the primary purpose of log tables is to provide an interface for users to observe the runtime execution of the server, not to interfere with its runtime execution.

  • CREATE TABLE, ALTER TABLE, and DROP TABLE are valid operations on a log table. For ALTER TABLE and DROP TABLE, the log table cannot be in use and must be disabled, as described later.

  • By default, the log tables use the CSV storage engine that writes data in comma-separated values format. For users who have access to the .CSV files that contain log table data, the files are easy to import into other programs such as spreadsheets that can process CSV input.

    The log tables can be altered to use the MyISAM storage engine. You cannot use ALTER TABLE to alter a log table that is in use. The log must be disabled first. No engines other than CSV or MyISAM are legal for the log tables.

  • To disable logging so that you can alter (or drop) a log table, you can use the following strategy. The example uses the general query log; the procedure for the slow query log is similar but uses the slow_log table and slow_query_log system variable.

    SET @old_log_state = @@global.general_log;
    SET GLOBAL general_log = 'OFF';
    ALTER TABLE mysql.general_log ENGINE = MyISAM;
    SET GLOBAL general_log = @old_log_state;
    
  • TRUNCATE TABLE is a valid operation on a log table. It can be used to expire log entries.

  • RENAME TABLE is a valid operation on a log table. You can atomically rename a log table (to perform log rotation, for example) using the following strategy:

    USE mysql;
    DROP TABLE IF EXISTS general_log2;
    CREATE TABLE general_log2 LIKE general_log;
    RENAME TABLE general_log TO general_log_backup, general_log2 TO general_log;
    
  • CHECK TABLE is a valid operation on a log table.

  • LOCK TABLES cannot be used on a log table.

  • INSERT, DELETE, and UPDATE cannot be used on a log table. These operations are permitted only internally to the server itself.

  • FLUSH TABLES WITH READ LOCK and the state of the read_only system variable have no effect on log tables. The server can always write to the log tables.

  • Entries written to the log tables are not written to the binary log and thus are not replicated to slave servers.

  • To flush the log tables or log files, use FLUSH TABLES or FLUSH LOGS, respectively.

  • Partitioning of log tables is not permitted.

  • A mysqldump dump includes statements to recreate those tables so that they are not missing after reloading the dump file. Log table contents are not dumped.

5.4.2 The Error Log

This section discusses how to configure the MySQL server for logging of diagnostic messages to the error log. For information about selecting the error message character set or language, see Section 10.1.6, “Error Message Character Set”, or Section 10.2, “Setting the Error Message Language”.

The error log contains a record of mysqld startup and shutdown times. It also contains diagnostic messages such as errors, warnings, and notes that occur during server startup and shutdown, and while the server is running. For example, if mysqld notices that a table needs to be automatically checked or repaired, it writes a message to the error log.

On some operating systems, the error log contains a stack trace if mysqld exits abnormally. The trace can be used to determine where mysqld exited. See Section 28.5, “Debugging and Porting MySQL”.

If used to start mysqld, mysqld_safe may write messages to the error log. For example, when mysqld_safe notices abnormal mysqld exits, it restarts mysqld and writes a mysqld restarted message to the error log.

The following sections discuss aspects of configuring error logging:

Error Log Component Configuration

In MySQL 8.0, error logging uses the MySQL component architecture described at Section 5.5, “MySQL Server Components”. The error log subsystem consists of components that perform log event filtering and writing, as well as a system variable that configures which components to enable to achieve the desired logging result.

This section discusses how to select components for error logging. For instructions specific to the system log and JSON log writers, see Error Logging to the System Log, and Error Logging in JSON Format. For additional details about all available log components, see Section 5.5.1, “Error Log Components”.

Component-based error logging offers these features:

  • Log events can be filtered by filter components to affect the information available for writing.

  • Log events are output by sink (writer) components. Multiple sink components can be enabled, to write error log output to multiple destinations.

  • Built-in filter and writer components combine to implement the default error logging format.

  • A loadable writer enables logging to the system log.

  • A loadable writer enables logging in JSON format.

  • A system variable controls which log components to enable.

The log_error_services system variable controls which log components to enable for error logging. Its value is a list of semicolon-separated components. Spaces are not significant. Component order is significant because the server executes components in the order listed.

By default, log_error_services has this value:

mysql> SELECT @@global.log_error_services;
+----------------------------------------+
| @@global.log_error_services            |
+----------------------------------------+
| log_filter_internal; log_sink_internal |
+----------------------------------------+

That value indicates that log events first pass through the built-in filter component, log_filter_internal, then through the built-in log writer component, log_sink_internal. A filter modifies log events seen by components named later in the log_error_services value. A sink is a destination for log events. Typically, a sink processes log events into log messages that have a particular format and writes these messages to its associated output, such as a file or the system log.

Note

If log_error_services is assigned a value that contains no writer components, no log output is written from that point.

The combination of log_filter_internal and log_sink_internal implements the default error log filtering and output behavior. The action of these components is affected by other server options and system variables:

To change the set of log components used for error logging, load components as necessary and modify the log_error_services value. Adding or removing log components is subject to these constraints:

For example, to use the system log writer (log_sink_syseventlog) instead of the default writer (log_sink_internal), first load the writer component, then modify the log_error_services value:

INSTALL COMPONENT 'file://component_log_sink_syseventlog';
SET GLOBAL log_error_services = 'log_filter_internal; log_sink_syseventlog';
Note

The URN to use for loading a log component with INSTALL COMPONENT is the component name prefixed with file://component_. For example, for the log_sink_syseventlog component, the corresponding URN is file://component_log_sink_syseventlog.

It is possible to configure multiple log writers to send output to multiple destinations. To use the system log writer in addition to (rather than instead of) the default writer, set the log_error_services value like this:

SET GLOBAL log_error_services = 'log_filter_internal; log_sink_internal; log_sink_syseventlog';

To revert to using only the default writer and unload the system log writer, execute these statements:

SET GLOBAL log_error_services = 'log_filter_internal; log_sink_internal;
UNINSTALL COMPONENT 'file://component_log_sink_syseventlog';

To configure a log component to be enabled at each server startup, use this procedure:

  1. If the component is loadable, load it using INSTALL COMPONENT. Loading the component registers it in the mysql.component system table so that the server loads it automatically for subsequent startups.

  2. Set the log_error_services value at startup to include the component name. Set the value either in the server my.cnf file, or use SET PERSIST, which sets the value for the running MySQL instance and also saves the value to be used for subsequent server restarts; see Section 13.7.4.1, “SET Syntax for Variable Assignment”. A value set in my.cnf takes effect at the next restart. A value set using SET PERSIST takes effect immediately, and for subsequent restarts.

Suppose that you want to configure, for every server startup, use of the JSON log writer (log_sink_json) in addition to the built-in log filter and writer (log_filter_internal, log_sink_internal). First load the JSON writer if it is not loaded:

INSTALL COMPONENT 'file://component_log_sink_json';

Then set log_error_services to take effect at server startup. You can set it in my.cnf:

[mysqld]
log_error_services='log_filter_internal; log_sink_internal; log_sink_json'

Or you can set it using SET PERSIST:

SET PERSIST log_error_services = 'log_filter_internal; log_sink_internal; log_sink_json';

The order of components named in log_error_services is significant, particularly with respect to the relative order of filters and writers. Consider this log_error_services value:

log_filter_internal; log_sink_1; log_sink_2

In this case, log events pass to the built-in filter, then to the first writer, then to the second writer. Both writers receive filtered log events.

Compare that to this log_error_services value:

log_sink_1; log_filter_internal; log_sink_2

In this case, log events pass to the first writer, then to the built-in filter, then to the second writer. The first writer receives unfiltered events. The second writer receives filtered events. You might configure error logging this way if you want one log that contains messages for all log events, and another containing messages only for a subset of log events.

Default Error Log Destination Configuration

This section discusses which server options configure the default error log destination, which can be the console or a named file. It also indicates which log writer components base their own output destination on the default destination.

In this discussion, console means stderr, the standard error output. This is your terminal or console window unless the standard error output has been redirected to a different destination.

The server interprets options that determine the default error log destination somewhat differently for Windows and Unix systems. Be sure to configure the destination using the information appropriate to your platform:

After the server interprets the default error log destination options, it sets the log_error system variable to indicate the default destination, which affects where several log writer components write error messages. See How the Default Error Log Destination Affects Log Writers.

Default Error Log Destination on Windows

On Windows, mysqld uses the --log-error, --pid-file, and --console options to determine whether the default error log destination is the console or a file, and, if a file, the file name:

  • If --console is given, the default destination is the console. (--console takes precedence over --log-error if both are given, and the following items regarding --log-error do not apply.)

  • If --log-error is not given, or is given without naming a file, the default destination is a file named host_name.err in the data directory, unless the --pid-file option is specified. In that case, the file name is the PID file base name with a suffix of .err in the data directory.

  • If --log-error is given to name a file, the default destination is that file (with an .err suffix added if the name has no suffix), located under the data directory unless an absolute path name is given to specify a different location.

If the default error log destination is the console, the server sets the log_error system variable to stderr. Otherwise, the default destination is a file and the server sets log_error to the file name.

Default Error Log Destination on Unix and Unix-Like Systems

On Unix and Unix-like systems, mysqld uses the --log-error option to determine whether the default error log destination is the console or a file, and, if a file, the file name:

  • If --log-error is not given, the default destination is the console.

  • If --log-error is given without naming a file, the default destination is a file named host_name.err in the data directory.

  • If --log-error is given to name a file, the default destination is that file (with an .err suffix added if the name has no suffix), located under the data directory unless an absolute path name is given to specify a different location.

  • If --log-error is given in an option file in a [mysqld], [server], or [mysqld_safe] section, mysqld_safe finds and uses the option, and passes it to mysqld.

Note

It is common for Yum or APT package installations to configure an error log file location under /var/log with an option like log-error=/var/log/mysqld.log in a server configuration file. Removing the file name from the option causes the host_name.err file in the data directory to be used.

If the default error log destination is the console, the server sets the log_error system variable to stderr. Otherwise, the default destination is a file and the server sets log_error to the file name.

How the Default Error Log Destination Affects Log Writers

After the server interprets the error log destination configuration options, it sets the log_error system variable to indicate the default error log destination. Log writer components may base their own output destination on the log_error value, or determine their destination independently of log_error

If log_error is stderr, the default error log destination is the console, and log writers that base their output destination on the default destination also write to the console:

  • log_sink_internal, log_sink_json, log_sink_test: These writers write to the console. This is true even for writers such as log_sink_json that can be enabled multiple times; all instances write to the console.

  • log_sink_syseventlog: This writer writes to the system log, regardless of the log_error value.

If log_error is not stderr, the default error log destination is a file and log_error indicates the file name. Log writers that base their output destination on the default destination base output file naming on that file name. (A writer might use exactly that name, or it might use some variant thereof.) Suppose that the log_error value file_name. Then log writers use the name like this:

  • log_sink_internal, log_sink_test: These writers write to file_name.

  • log_sink_json: Successive instances of this writer named in the log_error_services value write to files named file_name plus a numbered .NN.json suffix: file_name.00.json, file_name.01.json, and so forth.

  • log_sink_syseventlog: This writer writes to the system log, regardless of the log_error value.

Error Logging to the System Log

It is possible to have mysqld write the error log to the system log. This is the Event Log on Windows, and syslog on Unix and Unix-like systems.

This section describes how to configure error logging using the built-in filter, log_filter_internal, and the system log writer, log_sink_syseventlog, to take effect immediately and for subsequent server startups. For general configuration about configuring error logging, see Error Log Component Configuration.

Note

In MySQL 8.0, you must configure system log error logging explicitly. This differs from MySQL 5.7 and earlier, for which logging to the system log is enabled by default on Windows, and on all platforms requires no component loading.

To use the system log writer, first load the writer component, then modify the log_error_services value:

INSTALL COMPONENT 'file://component_log_sink_syseventlog';
SET PERSIST log_error_services = 'log_filter_internal; log_sink_syseventlog';
Note

Logging to the system log may require additional system configuration. Consult the system log documentation for your platform.

On Windows, error messages written to the Event Log within the Application log have these characteristics:

  • Entries marked as Error, Warning, and Note are written to the Event Log, but not messages such as information statements from individual storage engines.

  • Event Log entries have a source of MySQL.

On Unix and Unix-like systems, logging to the system log uses syslog. The following system variables affect syslog messages:

  • log_syslog_facility: The default facility for syslog messages is daemon. Set this variable to specify a different facility.

  • log_syslog_include_pid: Whether to include the server process ID in each line of syslog output.

  • log_syslog_tag: This variable defines a tag to add to the server identifier (mysqld) in syslog messages. If defined, the tag is appended to the identifier with a leading hyphen.

Error Logging in JSON Format

This section describes how to configure error logging using the built-in filter, log_filter_internal, and the JSON writer, log_sink_json, to take effect immediately and for subsequent server startups. For general configuration about configuring error logging, see Error Log Component Configuration.

To use the JSON writer, first load the writer component, then modify the log_error_services value:

INSTALL COMPONENT 'file://component_log_sink_json';
SET PERSIST log_error_services = 'log_filter_internal; log_sink_json';

It is permitted to name log_sink_json multiple times in the log_error_services value. For example, to write unfiltered events with one instance and filtered events with another instance, you could set log_error_services like this:

SET PERSIST log_error_services = 'log_sink_json; log_filter_internal; log_sink_json';

The JSON log writer determines its output destination based on the default error log destination, which is given by the log_error system variable. If log_error names a file, the JSON writer bases output file naming on that file name, plus a numbered .NN.json suffix, with NN starting at 00. For example, if log_error is file_name, successive instances of log_sink_json named in the log_error_services value write to file_name.00.json, file_name.01.json, and so forth.

If log_error is stderr, the JSON writer writes to the console. If log_json_writer is named multiple times in the log_error_services value, they all write to the console, which is likely not useful.

Error Log Verbosity

The log_error_verbosity system variable controls server verbosity for writing error, warning, and note messages to the error log. It affects which types of log events the log_filter_internal filter component permits or suppresses, and thus has no effect if that component is not enabled.

Permitted log_error_verbosity values are 1 (errors only), 2 (errors and warnings), 3 (errors, warnings, and notes), with a default of 3.

Selected important messages about non-error situations, such as startup and shutdown messages, are force-printed to the error log when the variable value is 1. In the MySQL error log, messages that have been force-printed are labeled as WARNING or NOTE in upper case, rather than lower case. Other log writers might or might not follow the same convention. If you apply any additional filtering or redirection based on the labeling of messages, force-printed messages do not override your filter, but are handled by it in the same way as other messages.

If the variable value is greater than 2 or greater, the server logs messages about statements that are unsafe for statement-based logging. If the value is 3, the server logs aborted connections and access-denied errors for new connection attempts. See Section B.5.2.10, “Communication Errors and Aborted Connections”.

If you use replication, setting this variable to 2 or greater is recommended, to get more information about what is happening, such as messages about network failures and reconnections.

If a slave server is started with this variable set to 2 or greater, the slave prints messages to the error log to provide information about its status, such as the binary log and relay log coordinates where it starts its job, when it is switching to another relay log, when it reconnects after a disconnect, and so forth.

Error Log Message Format

Each error log sink (writer) component has a characteristic output format it uses to write messages to its destination, but other factors may influence the content of the messages:

  • The information available to the log writer. If a log filter component executed prior to execution of the writer component removes a log event attribute, that attribute is not available for writing.

  • System variables may affect log writers.

For all log writers, the ID included in error log messages is that of the thread within mysqld responsible for writing the message. This indicates which part of the server produced the message, and is consistent with general query log and slow query log messages, which include the connection thread ID.

Output Format for log_sink_internal

This log writer produces the traditional error log output. It writes messages using this format:

timestamp thread_id [severity] [error_id] message

Examples:

2017-08-26T01:37:14.136034Z 0 [Warning] [003691] CA certificate /var/mysql/ssl/ca.pem is self signed.
2017-08-30T16:21:52.167838Z 0 [Note] [003876] IPv6 is available.

Prior to MySQL 8.0.3, the error ID field is not present. Log parsers can treat the error ID field as a part of the message text that will be present only for logs written by MySQL 8.0.3 and higher. Parsers must treat the error_id part of [error_id] indicators as a string value. Many ID values consist only of digits, but that is not guaranteed to be true.

For messages with no error ID, the error ID indicator is [000000], where the number of 0 digits depends on the current width padding.

Output Format for log_sink_json

The JSON-format log writer produces messages as JSON objects that contain key/value pairs. For example:

{ "prio" : 2, "err_code" : 3674, "subsystem" : "event",
"source_line" : 585, "source_file" : "event_scheduler.cc", "function" : "run",
"msg" : "Event Scheduler: scheduler thread started with id 4",
"time" : "2017-08-31T18:33:25.483190Z", "thread" : 4,
"err_symbol" : "ER_SCHEDULER_STARTED", "SQL_state" : "HY000",
"label" : "Note" }
Output Format for log_sink_syseventlog

The system log writer produces output that conforms to the system log format used on the local platform.

System Variables That Affect Error Log Format

The log_timestamps system variable controls the time zone of timestamps in messages written to the error log (as well as to general query log and slow query log files). Permitted values are UTC (the default) and SYSTEM (local system time zone).

Error Log File Flushing and Renaming

If you flush the error log using FLUSH ERROR LOGS, FLUSH LOGS, or mysqladmin flush-logs, the server closes and reopens any error log file to which it is writing. To rename an error log file, do so manually before flushing. Flushing the logs then opens a new file with the original file name. For example, assuming a log file name of host_name.err, to rename the file and create a new one, use the following commands:

mv host_name.err host_name.err-old
mysqladmin flush-logs
mv host_name.err-old backup-directory

On Windows, use rename rather than mv.

If the location of an error log file is not writable by the server, the log-flushing operation fails to create a new log file. For example, on Linux, the server might write the error log to the /var/log/mysqld.log file, where the /var/log directory is owned by root and is not writable by mysqld. For information about handling this case, see Section 5.4.7, “Server Log Maintenance”.

If the server is not writing to a named error log file, no error log file renaming occurs when the error log is flushed.

5.4.3 The General Query Log

The general query log is a general record of what mysqld is doing. The server writes information to this log when clients connect or disconnect, and it logs each SQL statement received from clients. The general query log can be very useful when you suspect an error in a client and want to know exactly what the client sent to mysqld.

Each line that shows when a client connects also includes using connection_type to indicate the protocol used to establish the connection. connection_type is one of TCP/IP (TCP/IP connection established without SSL), SSL/TLS (TCP/IP connection established with SSL), Socket (Unix socket file connection), Named Pipe (Windows named pipe connection), or Shared Memory (Windows shared memory connection).

mysqld writes statements to the query log in the order that it receives them, which might differ from the order in which they are executed. This logging order is in contrast with that of the binary log, for which statements are written after they are executed but before any locks are released. In addition, the query log may contain statements that only select data while such statements are never written to the binary log.

When using statement-based binary logging on a replication master server, statements received by its slaves are written to the query log of each slave. Statements are written to the query log of the master server if a client reads events with the mysqlbinlog utility and passes them to the server.

However, when using row-based binary logging, updates are sent as row changes rather than SQL statements, and thus these statements are never written to the query log when binlog_format is ROW. A given update also might not be written to the query log when this variable is set to MIXED, depending on the statement used. See Section 18.2.1.1, “Advantages and Disadvantages of Statement-Based and Row-Based Replication”, for more information.

By default, the general query log is disabled. To specify the initial general query log state explicitly, use --general_log[={0|1}]. With no argument or an argument of 1, --general_log enables the log. With an argument of 0, this option disables the log. To specify a log file name, use --general_log_file=file_name. To specify the log destination, use --log-output (as described in Section 5.4.1, “Selecting General Query and Slow Query Log Output Destinations”).

If you specify no name for the general query log file, the default name is host_name.log. The server creates the file in the data directory unless an absolute path name is given to specify a different directory.

To disable or enable the general query log or change the log file name at runtime, use the global general_log and general_log_file system variables. Set general_log to 0 (or OFF) to disable the log or to 1 (or ON) to enable it. Set general_log_file to specify the name of the log file. If a log file already is open, it is closed and the new file is opened.

When the general query log is enabled, the server writes output to any destinations specified by the --log-output option or log_output system variable. If you enable the log, the server opens the log file and writes startup messages to it. However, further logging of queries to the file does not occur unless the FILE log destination is selected. If the destination is NONE, the server writes no queries even if the general log is enabled. Setting the log file name has no effect on logging if the log destination value does not contain FILE.

Server restarts and log flushing do not cause a new general query log file to be generated (although flushing closes and reopens it). To rename the file and create a new one, use the following commands:

shell> mv host_name.log host_name-old.log
shell> mysqladmin flush-logs
shell> mv host_name-old.log backup-directory

On Windows, use rename rather than mv.

You can also rename the general query log file at runtime by disabling the log:

SET GLOBAL general_log = 'OFF';

With the log disabled, rename the log file externally; for example, from the command line. Then enable the log again:

SET GLOBAL general_log = 'ON';

This method works on any platform and does not require a server restart.

The session sql_log_off variable can be set to ON or OFF to disable or enable general query logging for the current connection.

Passwords in statements written to the general query log are rewritten by the server not to occur literally in plain text. Password rewriting can be suppressed for the general query log by starting the server with the --log-raw option. This option may be useful for diagnostic purposes, to see the exact text of statements as received by the server, but for security reasons is not recommended for production use. See also Section 6.1.2.3, “Passwords and Logging”.

An implication of password rewriting is that statements that cannot be parsed (due, for example, to syntax errors) are not written to the general query log because they cannot be known to be password free. Use cases that require logging of all statements including those with errors should use the --log-raw option, bearing in mind that this also bypasses password rewriting.

Password rewriting occurs only when plain text passwords are expected. For statements with syntax that expect a password hash value, no rewriting occurs. If a plain text password is supplied erroneously for such syntax, the password is logged as given, without rewriting. For example, the following statement is logged as shown because a password hash value is expected:

CREATE USER 'user1'@'localhost' IDENTIFIED BY PASSWORD 'not-so-secret';

The log_timestamps system variable controls the time zone of timestamps in messages written to the general query log file (as well as to the slow query log file and the error log). It does not affect the time zone of general query log and slow query log messages written to log tables, but rows retrieved from those tables can be converted from the local system time zone to any desired time zone with CONVERT_TZ() or by setting the session time_zone system variable.

5.4.4 The Binary Log

The binary log contains events that describe database changes such as table creation operations or changes to table data. It also contains events for statements that potentially could have made changes (for example, a DELETE which matched no rows), unless row-based logging is used. The binary log also contains information about how long each statement took that updated data. The binary log has two important purposes:

  • For replication, the binary log on a master replication server provides a record of the data changes to be sent to slave servers. The master server sends the events contained in its binary log to its slaves, which execute those events to make the same data changes that were made on the master. See Section 18.2, “Replication Implementation”.

  • Certain data recovery operations require use of the binary log. After a backup has been restored, the events in the binary log that were recorded after the backup was made are re-executed. These events bring databases up to date from the point of the backup. See Section 7.5, “Point-in-Time (Incremental) Recovery Using the Binary Log”.

The binary log is not used for statements such as SELECT or SHOW that do not modify data. To log all statements (for example, to identify a problem query), use the general query log. See Section 5.4.3, “The General Query Log”.

Running a server with binary logging enabled makes performance slightly slower. However, the benefits of the binary log in enabling you to set up replication and for restore operations generally outweigh this minor performance decrement.

The binary log is crash-safe. Only complete events or transactions are logged or read back.

Passwords in statements written to the binary log are rewritten by the server not to occur literally in plain text. See also Section 6.1.2.3, “Passwords and Logging”.

The following discussion describes some of the server options and variables that affect the operation of binary logging. For a complete list, see Section 18.1.6.4, “Binary Logging Options and Variables”.

To enable the binary log, start the server with the --log-bin[=base_name] option. If no base_name value is given, the default name is the value of the pid-file option (which by default is the name of host machine) followed by -bin. If the base name is given, the server writes the file in the data directory unless the base name is given with a leading absolute path name to specify a different directory. It is recommended that you specify a base name explicitly rather than using the default of the host name; see Section B.5.7, “Known Issues in MySQL”, for the reason.

If you supply an extension in the log name (for example, --log-bin=base_name.extension), the extension is silently removed and ignored.

mysqld appends a numeric extension to the binary log base name to generate binary log file names. The number increases each time the server creates a new log file, thus creating an ordered series of files. The server creates a new file in the series each time it starts or flushes the logs. The server also creates a new binary log file automatically after the current log's size reaches max_binlog_size. A binary log file may become larger than max_binlog_size if you are using large transactions because a transaction is written to the file in one piece, never split between files.

To keep track of which binary log files have been used, mysqld also creates a binary log index file that contains the names of all used binary log files. By default, this has the same base name as the binary log file, with the extension '.index'. You can change the name of the binary log index file with the --log-bin-index[=file_name] option. You should not manually edit this file while mysqld is running; doing so would confuse mysqld.

The term binary log file generally denotes an individual numbered file containing database events. The term binary log collectively denotes the set of numbered binary log files plus the index file.

A client that has the SYSTEM_VARIABLES_ADMIN or SUPER privilege can disable binary logging of its own statements by using a SET sql_log_bin=0 statement. See Section 5.1.5, “Server System Variables”.

By default, the server logs the length of the event as well as the event itself and uses this to verify that the event was written correctly. You can also cause the server to write checksums for the events by setting the binlog_checksum system variable. When reading back from the binary log, the master uses the event length by default, but can be made to use checksums if available by enabling the master_verify_checksum system variable. The slave I/O thread also verifies events received from the master. You can cause the slave SQL thread to use checksums if available when reading from the relay log by enabling the slave_sql_verify_checksum system variable.

The format of the events recorded in the binary log is dependent on the binary logging format. Three format types are supported, row-based logging, statement-based logging and mixed-base logging. The binary logging format used depends on the MySQL version. For general descriptions of the logging formats, see Section 5.4.4.1, “Binary Logging Formats”. For detailed information about the format of the binary log, see MySQL Internals: The Binary Log.

The server evaluates the --binlog-do-db and --binlog-ignore-db options in the same way as it does the --replicate-do-db and --replicate-ignore-db options. For information about how this is done, see Section 18.2.5.1, “Evaluation of Database-Level Replication and Binary Logging Options”.

From MySQL 8.0.3, a replication slave server is started with the --log-slave-updates setting enabled by default, meaning that the slave writes to its own binary log any data modifications that are received from the replication master. The binary log must be enabled for this setting to work (see Section 18.1.6.3, “Replication Slave Options and Variables”). This setting enables the slave to act as a master to other slaves in chained replication.

You can delete all binary log files with the RESET MASTER statement, or a subset of them with PURGE BINARY LOGS. See Section 13.7.6.6, “RESET Syntax”, and Section 13.4.1.1, “PURGE BINARY LOGS Syntax”.

If you are using replication, you should not delete old binary log files on the master until you are sure that no slave still needs to use them. For example, if your slaves never run more than three days behind, once a day you can execute mysqladmin flush-logs on the master and then remove any logs that are more than three days old. You can remove the files manually, but it is preferable to use PURGE BINARY LOGS, which also safely updates the binary log index file for you (and which can take a date argument). See Section 13.4.1.1, “PURGE BINARY LOGS Syntax”.

You can display the contents of binary log files with the mysqlbinlog utility. This can be useful when you want to reprocess statements in the log for a recovery operation. For example, you can update a MySQL server from the binary log as follows:

shell> mysqlbinlog log_file | mysql -h server_name

mysqlbinlog also can be used to display replication slave relay log file contents because they are written using the same format as binary log files. For more information on the mysqlbinlog utility and how to use it, see Section 4.6.8, “mysqlbinlog — Utility for Processing Binary Log Files”. For more information about the binary log and recovery operations, see Section 7.5, “Point-in-Time (Incremental) Recovery Using the Binary Log”.

Binary logging is done immediately after a statement or transaction completes but before any locks are released or any commit is done. This ensures that the log is logged in commit order.

Updates to nontransactional tables are stored in the binary log immediately after execution.

Within an uncommitted transaction, all updates (UPDATE, DELETE, or INSERT) that change transactional tables such as InnoDB tables are cached until a COMMIT statement is received by the server. At that point, mysqld writes the entire transaction to the binary log before the COMMIT is executed.

Modifications to nontransactional tables cannot be rolled back. If a transaction that is rolled back includes modifications to nontransactional tables, the entire transaction is logged with a ROLLBACK statement at the end to ensure that the modifications to those tables are replicated.

When a thread that handles the transaction starts, it allocates a buffer of binlog_cache_size to buffer statements. If a statement is bigger than this, the thread opens a temporary file to store the transaction. The temporary file is deleted when the thread ends.

The Binlog_cache_use status variable shows the number of transactions that used this buffer (and possibly a temporary file) for storing statements. The Binlog_cache_disk_use status variable shows how many of those transactions actually had to use a temporary file. These two variables can be used for tuning binlog_cache_size to a large enough value that avoids the use of temporary files.

The max_binlog_cache_size system variable (default 4GB, which is also the maximum) can be used to restrict the total size used to cache a multiple-statement transaction. If a transaction is larger than this many bytes, it fails and rolls back. The minimum value is 4096.

If you are using the binary log and row based logging, concurrent inserts are converted to normal inserts for CREATE ... SELECT or INSERT ... SELECT statements. This is done to ensure that you can re-create an exact copy of your tables by applying the log during a backup operation. If you are using statement-based logging, the original statement is written to the log.

The binary log format has some known limitations that can affect recovery from backups. See Section 18.4.1, “Replication Features and Issues”.

Binary logging for stored programs is done as described in Section 23.7, “Binary Logging of Stored Programs”.

Note that the binary log format differs in MySQL 8.0 from previous versions of MySQL, due to enhancements in replication. See Section 18.4.2, “Replication Compatibility Between MySQL Versions”.

Writes to the binary log file and binary log index file are handled in the same way as writes to MyISAM tables. See Section B.5.3.4, “How MySQL Handles a Full Disk”.

By default, the binary log is synchronized to disk at each write (sync_binlog=1). If sync_binlog was not enabled, and the operating system or machine (not only the MySQL server) crashed, there is a chance that the last statements of the binary log could be lost. To prevent this, enable the sync_binlog system variable to synchronize the binary log to disk after every N commit groups. See Section 5.1.5, “Server System Variables”. The safest value for sync_binlog is 1 (the default), but this is also the slowest.

In earlier MySQL releases, there was a chance of inconsistency between the table content and binary log content if a crash occurred, even with sync_binlog set to 1. For example, if you are using InnoDB tables and the MySQL server processes a COMMIT statement, it writes many prepared transactions to the binary log in sequence, synchronizes the binary log, and then commits the transaction into InnoDB. If the server crashed between those two operations, the transaction would be rolled back by InnoDB at restart but still exist in the binary log. Such an issue was resolved in previous releases by enabling InnoDB support for two-phase commit in XA transactions. In 5.8.0 and higher, the InnoDB support for two-phase commit in XA transactions is always enabled.

InnoDB support for two-phase commit in XA transactions ensures that the binary log and InnoDB data files are synchronized. However, the MySQL server should also be configured to synchronize the binary log and the InnoDB logs to disk before committing the transaction. The InnoDB logs are synchronized by default, and sync_binlog=1 ensures the binary log is synchronized. The effect of implicit InnoDB support for two-phase commit in XA transactions and sync_binlog=1 is that at restart after a crash, after doing a rollback of transactions, the MySQL server scans the latest binary log file to collect transaction xid values and calculate the last valid position in the binary log file. The MySQL server then tells InnoDB to complete any prepared transactions that were successfully written to the to the binary log, and truncates the binary log to the last valid position. This ensures that the binary log reflects the exact data of InnoDB tables, and therefore the slave remains in synchrony with the master because it does not receive a statement which has been rolled back.

If the MySQL server discovers at crash recovery that the binary log is shorter than it should have been, it lacks at least one successfully committed InnoDB transaction. This should not happen if sync_binlog=1 and the disk/file system do an actual sync when they are requested to (some do not), so the server prints an error message The binary log file_name is shorter than its expected size. In this case, this binary log is not correct and replication should be restarted from a fresh snapshot of the master's data.

The session values of the following system variables are written to the binary log and honored by the replication slave when parsing the binary log:

5.4.4.1 Binary Logging Formats

The server uses several logging formats to record information in the binary log:

  • Replication capabilities in MySQL originally were based on propagation of SQL statements from master to slave. This is called statement-based logging. You can cause this format to be used by starting the server with --binlog-format=STATEMENT.

  • In row-based logging (the default), the master writes events to the binary log that indicate how individual table rows are affected. You can cause the server to use row-based logging by starting it with --binlog-format=ROW.

  • A third option is also available: mixed logging. With mixed logging, statement-based logging is used by default, but the logging mode switches automatically to row-based in certain cases as described below. You can cause MySQL to use mixed logging explicitly by starting mysqld with the option --binlog-format=MIXED.

The logging format can also be set or limited by the storage engine being used. This helps to eliminate issues when replicating certain statements between a master and slave which are using different storage engines.

With statement-based replication, there may be issues with replicating nondeterministic statements. In deciding whether or not a given statement is safe for statement-based replication, MySQL determines whether it can guarantee that the statement can be replicated using statement-based logging. If MySQL cannot make this guarantee, it marks the statement as potentially unreliable and issues the warning, Statement may not be safe to log in statement format.

You can avoid these issues by using MySQL's row-based replication instead.

5.4.4.2 Setting The Binary Log Format

You can select the binary logging format explicitly by starting the MySQL server with --binlog-format=type. The supported values for type are:

  • STATEMENT causes logging to be statement based.

  • ROW causes logging to be row based. This is the default.

  • MIXED causes logging to use mixed format.

The logging format also can be switched at runtime. To specify the format globally for all clients, set the global value of the binlog_format system variable:

mysql> SET GLOBAL binlog_format = 'STATEMENT';
mysql> SET GLOBAL binlog_format = 'ROW';
mysql> SET GLOBAL binlog_format = 'MIXED';

An individual client can control the logging format for its own statements by setting the session value of binlog_format:

mysql> SET SESSION binlog_format = 'STATEMENT';
mysql> SET SESSION binlog_format = 'ROW';
mysql> SET SESSION binlog_format = 'MIXED';
Note

Each MySQL Server can set its own and only its own binary logging format (true whether binlog_format is set with global or session scope). This means that changing the logging format on a replication master does not cause a slave to change its logging format to match. (When using STATEMENT mode, the binlog_format system variable is not replicated; when using MIXED or ROW logging mode, it is replicated but is ignored by the slave.) Changing the binary logging format on the master while replication is ongoing, or without also changing it on the slave can cause replication to fail with errors such as Error executing row event: 'Cannot execute statement: impossible to write to binary log since statement is in row format and BINLOG_FORMAT = STATEMENT.'

To change the global or session binlog_format value, you must have the SYSTEM_VARIABLES_ADMIN or SUPER privilege.

There are several reasons why a client might want to set binary logging on a per-session basis:

  • A session that makes many small changes to the database might want to use row-based logging.

  • A session that performs updates that match many rows in the WHERE clause might want to use statement-based logging because it will be more efficient to log a few statements than many rows.

  • Some statements require a lot of execution time on the master, but result in just a few rows being modified. It might therefore be beneficial to replicate them using row-based logging.

There are exceptions when you cannot switch the replication format at runtime:

  • From within a stored function or a trigger

  • If the session is currently in row-based replication mode and has open temporary tables

Trying to switch the format in any of these cases results in an error.

If you are using InnoDB tables and the transaction isolation level is READ COMMITTED or READ UNCOMMITTED, only row-based logging can be used. It is possible to change the logging format to STATEMENT, but doing so at runtime leads very rapidly to errors because InnoDB can no longer perform inserts.

Switching the replication format at runtime is not recommended when any temporary tables exist, because temporary tables are logged only when using statement-based replication, whereas with row-based replication they are not logged. With mixed replication, temporary tables are usually logged; exceptions happen with user-defined functions (UDFs) and with the UUID() function.

With the binary log format set to ROW, many changes are written to the binary log using the row-based format. Some changes, however, still use the statement-based format. Examples include all DDL (data definition language) statements such as CREATE TABLE, ALTER TABLE, or DROP TABLE.

The --binlog-row-event-max-size option is available for servers that are capable of row-based replication. Rows are stored into the binary log in chunks having a size in bytes not exceeding the value of this option. The value must be a multiple of 256. The default value is 8192.

Warning

When using statement-based logging for replication, it is possible for the data on the master and slave to become different if a statement is designed in such a way that the data modification is nondeterministic; that is, it is left to the will of the query optimizer. In general, this is not a good practice even outside of replication. For a detailed explanation of this issue, see Section B.5.7, “Known Issues in MySQL”.

For information about logs kept by replication slaves, see Section 18.2.4, “Replication Relay and Status Logs”.

5.4.4.3 Mixed Binary Logging Format

When running in MIXED logging format, the server automatically switches from statement-based to row-based logging under the following conditions:

Note

A warning is generated if you try to execute a statement using statement-based logging that should be written using row-based logging. The warning is shown both in the client (in the output of SHOW WARNINGS) and through the mysqld error log. A warning is added to the SHOW WARNINGS table each time such a statement is executed. However, only the first statement that generated the warning for each client session is written to the error log to prevent flooding the log.

In addition to the decisions above, individual engines can also determine the logging format used when information in a table is updated. The logging capabilities of an individual engine can be defined as follows:

  • If an engine supports row-based logging, the engine is said to be row-logging capable.

  • If an engine supports statement-based logging, the engine is said to be statement-logging capable.

A given storage engine can support either or both logging formats. The following table lists the formats supported by each engine.

Storage EngineRow Logging SupportedStatement Logging Supported
ARCHIVEYesYes
BLACKHOLEYesYes
CSVYesYes
EXAMPLEYesNo
FEDERATEDYesYes
HEAPYesYes
InnoDBYesYes when the transaction isolation level is REPEATABLE READ or SERIALIZABLE; No otherwise.
MyISAMYesYes
MERGEYesYes
NDBYesNo

Whether a statement is to be logged and the logging mode to be used is determined according to the type of statement (safe, unsafe, or binary injected), the binary logging format (STATEMENT, ROW, or MIXED), and the logging capabilities of the storage engine (statement capable, row capable, both, or neither). (Binary injection refers to logging a change that must be logged using ROW format.)

Statements may be logged with or without a warning; failed statements are not logged, but generate errors in the log. This is shown in the following decision table, where SLC stands for statement-logging capable and RLC stands for row-logging capable.

ConditionAction
Typebinlog_formatSLCRLCError / WarningLogged as
**NoNoError: Cannot execute statement: Binary logging is impossible since at least one engine is involved that is both row-incapable and statement-incapable.-
SafeSTATEMENTYesNo-STATEMENT
SafeMIXEDYesNo-STATEMENT
SafeROWYesNoError: Cannot execute statement: Binary logging is impossible since BINLOG_FORMAT = ROW and at least one table uses a storage engine that is not capable of row-based logging.-
UnsafeSTATEMENTYesNoWarning: Unsafe statement binlogged in statement format, since BINLOG_FORMAT = STATEMENTSTATEMENT
UnsafeMIXEDYesNoError: Cannot execute statement: Binary logging of an unsafe statement is impossible when the storage engine is limited to statement-based logging, even if BINLOG_FORMAT = MIXED.-
UnsafeROWYesNoError: Cannot execute statement: Binary logging is impossible since BINLOG_FORMAT = ROW and at least one table uses a storage engine that is not capable of row-based logging.-
Row InjectionSTATEMENTYesNoError: Cannot execute row injection: Binary logging is not possible since at least one table uses a storage engine that is not capable of row-based logging.-
Row InjectionMIXEDYesNoError: Cannot execute row injection: Binary logging is not possible since at least one table uses a storage engine that is not capable of row-based logging.-
Row InjectionROWYesNoError: Cannot execute row injection: Binary logging is not possible since at least one table uses a storage engine that is not capable of row-based logging.-
SafeSTATEMENTNoYesError: Cannot execute statement: Binary logging is impossible since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine that is not capable of statement-based logging.-
SafeMIXEDNoYes-ROW
SafeROWNoYes-ROW
UnsafeSTATEMENTNoYesError: Cannot execute statement: Binary logging is impossible since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine that is not capable of statement-based logging.-
UnsafeMIXEDNoYes-ROW
UnsafeROWNoYes-ROW
Row InjectionSTATEMENTNoYesError: Cannot execute row injection: Binary logging is not possible since BINLOG_FORMAT = STATEMENT.-
Row InjectionMIXEDNoYes-ROW
Row InjectionROWNoYes-ROW
SafeSTATEMENTYesYes-STATEMENT
SafeMIXEDYesYes-STATEMENT
SafeROWYesYes-ROW
UnsafeSTATEMENTYesYesWarning: Unsafe statement binlogged in statement format since BINLOG_FORMAT = STATEMENT.STATEMENT
UnsafeMIXEDYesYes-ROW
UnsafeROWYesYes-ROW
Row InjectionSTATEMENTYesYesError: Cannot execute row injection: Binary logging is not possible because BINLOG_FORMAT = STATEMENT.-
Row InjectionMIXEDYesYes-ROW
Row InjectionROWYesYes-ROW

When a warning is produced by the determination, a standard MySQL warning is produced (and is available using SHOW WARNINGS). The information is also written to the mysqld error log. Only one error for each error instance per client connection is logged to prevent flooding the log. The log message includes the SQL statement that was attempted.

If a slave server was started with log_error_verbosity set to display warnings, the slave prints messages to the error log to provide information about its status, such as the binary log and relay log coordinates where it starts its job, when it is switching to another relay log, when it reconnects after a disconnect, statements that are unsafe for statement-based logging, and so forth.

5.4.4.4 Logging Format for Changes to mysql Database Tables

The contents of the grant tables in the mysql database can be modified directly (for example, with INSERT or DELETE) or indirectly (for example, with GRANT or CREATE USER). Statements that affect mysql database tables are written to the binary log using the following rules:

CREATE TABLE ... SELECT is a combination of data definition and data manipulation. The CREATE TABLE part is logged using statement format and the SELECT part is logged according to the value of binlog_format.

5.4.5 The Slow Query Log

The slow query log consists of SQL statements that took more than long_query_time seconds to execute and required at least min_examined_row_limit rows to be examined. The minimum and default values of long_query_time are 0 and 10, respectively. The value can be specified to a resolution of microseconds. For logging to a file, times are written including the microseconds part. For logging to tables, only integer times are written; the microseconds part is ignored.

By default, administrative statements are not logged, nor are queries that do not use indexes for lookups. This behavior can be changed using log_slow_admin_statements and log_queries_not_using_indexes, as described later.

The time to acquire the initial locks is not counted as execution time. mysqld writes a statement to the slow query log after it has been executed and after all locks have been released, so log order might differ from execution order.

By default, the slow query log is disabled. To specify the initial slow query log state explicitly, use --slow_query_log[={0|1}]. With no argument or an argument of 1, --slow_query_log enables the log. With an argument of 0, this option disables the log. To specify a log file name, use --slow_query_log_file=file_name. To specify the log destination, use --log-output (as described in Section 5.4.1, “Selecting General Query and Slow Query Log Output Destinations”).

If you specify no name for the slow query log file, the default name is host_name-slow.log. The server creates the file in the data directory unless an absolute path name is given to specify a different directory.

To disable or enable the slow query log or change the log file name at runtime, use the global slow_query_log and slow_query_log_file system variables. Set slow_query_log to 0 (or OFF) to disable the log or to 1 (or ON) to enable it. Set slow_query_log_file to specify the name of the log file. If a log file already is open, it is closed and the new file is opened.

When the slow query log is enabled, the server writes output to any destinations specified by the --log-output option or log_output system variable. If you enable the log, the server opens the log file and writes startup messages to it. However, further logging of queries to the file does not occur unless the FILE log destination is selected. If the destination is NONE, the server writes no queries even if the slow query log is enabled. Setting the log file name has no effect on logging if the log destination value does not contain FILE.

The server writes less information to the slow query log if you use the --log-short-format option.

To include slow administrative statements in the statements written to the slow query log, use the log_slow_admin_statements system variable. Administrative statements include ALTER TABLE, ANALYZE TABLE, CHECK TABLE, CREATE INDEX, DROP INDEX, OPTIMIZE TABLE, and REPAIR TABLE.

To include queries that do not use indexes for row lookups in the statements written to the slow query log, enable the log_queries_not_using_indexes system variable. When such queries are logged, the slow query log may grow quickly. It is possible to put a rate limit on these queries by setting the log_throttle_queries_not_using_indexes system variable. By default, this variable is 0, which means there is no limit. Positive values impose a per-minute limit on logging of queries that do not use indexes. The first such query opens a 60-second window within which the server logs queries up to the given limit, then suppresses additional queries. If there are suppressed queries when the window ends, the server logs a summary that indicates how many there were and the aggregate time spent in them. The next 60-second window begins when the server logs the next query that does not use indexes.

The server uses the controlling parameters in the following order to determine whether to write a query to the slow query log:

  1. The query must either not be an administrative statement, or log_slow_admin_statements must be enabled.

  2. The query must have taken at least long_query_time seconds, or log_queries_not_using_indexes must be enabled and the query used no indexes for row lookups.

  3. The query must have examined at least min_examined_row_limit rows.

  4. The query must not be suppressed according to the log_throttle_queries_not_using_indexes setting.

The log_timestamps system variable controls the time zone of timestamps in messages written to the slow query log file (as well as to the general query log file and the error log). It does not affect the time zone of general query log and slow query log messages written to log tables, but rows retrieved from those tables can be converted from the local system time zone to any desired time zone with CONVERT_TZ() or by setting the session time_zone system variable.

All log lines contain a timestamp.

The server does not write to the slow query log queries that would not benefit from the presence of an index because the table has zero rows or one row.

By default, a replication slave does not write replicated queries to the slow query log. To change this, use the log_slow_slave_statements system variable.

Passwords in statements written to the slow query log are rewritten by the server not to occur literally in plain text. See also Section 6.1.2.3, “Passwords and Logging”.

The slow query log can be used to find queries that take a long time to execute and are therefore candidates for optimization. However, examining a long slow query log can become a difficult task. To make this easier, you can process a slow query log file using the mysqldumpslow command to summarize the queries that appear in the log. See Section 4.6.9, “mysqldumpslow — Summarize Slow Query Log Files”.

5.4.6 The DDL Log

The DDL log, or metadata log, records metadata operations generated by data definition statements such as DROP TABLE and ALTER TABLE. MySQL uses this log to recover from crashes occurring in the middle of a metadata operation. When executing the statement DROP TABLE t1, t2, we need to ensure that both t1 and t2 are dropped, and that each table drop is complete. Another example of this type of SQL statement is ALTER TABLE t3 DROP PARTITION p2, where we must make certain that the partition is completely dropped and that its definition is removed from the list of partitions for table t3.

A record of metadata operations such as those just described are written to the file ddl_log.log, in the MySQL data directory. This is a binary file; it is not intended to be human-readable, and you should not attempt to modify its contents in any way.

ddl_log.log is not created until it is actually needed for recording metadata statements, and is removed following a successful start of mysqld. Thus, it is possible for this file not to be present on a MySQL server that is functioning in a completely normal manner.

Currently, ddl_log.log can hold up to 1048573 entries, equivalent 4 GB in size. Once this limit is exceeded, you must rename or remove the file before it is possible to execute any additional DDL statements. This is a known issue which we are working to resolve (Bug #83708).

There are no user-configurable server options or variables associated with this file.

5.4.7 Server Log Maintenance

As described in Section 5.4, “MySQL Server Logs”, MySQL Server can create several different log files to help you see what activity is taking place. However, you must clean up these files regularly to ensure that the logs do not take up too much disk space.

When using MySQL with logging enabled, you may want to back up and remove old log files from time to time and tell MySQL to start logging to new files. See Section 7.2, “Database Backup Methods”.

On a Linux (Red Hat) installation, you can use the mysql-log-rotate script for this. If you installed MySQL from an RPM distribution, this script should have been installed automatically. Be careful with this script if you are using the binary log for replication. You should not remove binary logs until you are certain that their contents have been processed by all slaves.

On other systems, you must install a short script yourself that you start from cron (or its equivalent) for handling log files.

For the binary log, you can set the expire_logs_days system variable to expire binary log files automatically after a given number of days (see Section 5.1.5, “Server System Variables”). If you are using replication, you should set the variable no lower than the maximum number of days your slaves might lag behind the master. To remove binary logs on demand, use the PURGE BINARY LOGS statement (see Section 13.4.1.1, “PURGE BINARY LOGS Syntax”).

You can force MySQL to start using new log files by flushing the logs. Log flushing occurs when you issue a FLUSH LOGS statement or execute a mysqladmin flush-logs, mysqladmin refresh, mysqldump --flush-logs, or mysqldump --master-data command. See Section 13.7.6.3, “FLUSH Syntax”, Section 4.5.2, “mysqladmin — Client for Administering a MySQL Server”, and Section 4.5.4, “mysqldump — A Database Backup Program”. In addition, the binary log is flushed when its size reaches the value of the max_binlog_size system variable.

FLUSH LOGS supports optional modifiers to enable selective flushing of individual logs (for example, FLUSH BINARY LOGS).

A log-flushing operation does the following:

  • If general query logging or slow query logging to a log file is enabled, the server closes and reopens the general query log file or slow query log file.

  • If binary logging is enabled, the server closes the current binary log file and opens a new log file with the next sequence number.

  • If the server was started with the --log-error option to cause the error log to be written to a file, the server closes and reopens the log file.

The server creates a new binary log file when you flush the logs. However, it just closes and reopens the general and slow query log files. To cause new files to be created on Unix, rename the current log files before flushing them. At flush time, the server opens new log files with the original names. For example, if the general and slow query log files are named mysql.log and mysql-slow.log, you can use a series of commands like this:

shell> cd mysql-data-directory
shell> mv mysql.log mysql.old
shell> mv mysql-slow.log mysql-slow.old
shell> mysqladmin flush-logs

On Windows, use rename rather than mv.

At this point, you can make a backup of mysql.old and mysql-slow.old and then remove them from disk.

A similar strategy can be used to back up the error log file, if there is one.

You can rename the general query log or slow query log at runtime by disabling the log:

SET GLOBAL general_log = 'OFF';
SET GLOBAL slow_query_log = 'OFF';

With the logs disabled, rename the log files externally; for example, from the command line. Then enable the logs again:

SET GLOBAL general_log = 'ON';
SET GLOBAL slow_query_log = 'ON';

This method works on any platform and does not require a server restart.

Note

For the server to recreate a given log file after you have renamed the file externally, the file location must be writable by the server. This may not always be the case. For example, on Linux, the server might write the error log as /var/log/mysqld.log, where /var/log is owned by root and not writable by mysqld. In this case, the log-flushing operation will fail to create a new log file.

To handle this situation, you must manually create the new log file with the proper ownershiop after renaming the original log file. For example, execute these commands as root:

shell> mv /var/log/mysqld.log /var/log/mysqld.log.old
shell> install -omysql -gmysql -m0644 /dev/null /var/log/mysqld.log

5.5 MySQL Server Components

MySQL Server includes a component-based infrastructure for extending server capabilities:

  • A component provides services that are available to the server and other components. (With respect to service use, the server is a component, equal to other components.) Components interact with each other only through the services they provide.

  • The INSTALL COMPONENT and UNINSTALL COMPONENT SQL statements enable component loading and unloading.

  • A loader service handles component loading and unloading, and also lists loaded components in the mysql.component system table that serves as a registry.

The SQL statements for component manipulation affect server operation and the mysql.component system table as follows:

  • INSTALL COMPONENT loads components into the server. The components become active immediately. The loader service also registers loaded components in the mysql.component system table. For subsequent server restarts, any components listed in mysql.component are loaded by the loader service during the startup sequence. This occurs even if the server is started with the --skip-grant-tables option.

  • UNINSTALL COMPONENT deactivates components and unloads them from the server. The loader service also unregisters the components from the mysql.component system table so that they are no longer loaded during the startup sequence for subsequent server restarts.

To see which components are installed, use this statement:

SELECT * FROM mysql.component;

MySQL distributions include several components that implement server extensions:

For information about the internal implementation of components, see http://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_EXTENDING.html. For example, if you intend to write your own components, this information is important for understanding how components work.

5.5.1 Error Log Components

This section describes the characteristics of individual error log components. For general information about configuring error logging, see Section 5.4.2, “The Error Log”.

A log component can be a filter or a sink:

  • A filter processes log events, to add, remove, or modify event fields, or to delete events entirely. The resulting events pass to the next log component named in the log_error_services system variable value.

  • A sink is a destination (writer) for log events. Typically, a sink processes log events into log messages that have a particular format and writes these messages to its associated output, such as a file or the system log.

The server executes filters and sinks in the log_error_services value in the order they are named.

The following sections describe individual log components, grouped by component type:

Component descriptions include these types of information:

  • The component name and intended purpose.

  • Whether the component is built in or must be loaded. For a loadable component, the description specifies the URN to use to load and unload the component with the INSTALL COMPONENT and UNINSTALL COMPONENT statements.

  • Whether the component can be listed multiple times in the log_error_services value.

  • For a sink component, the destination to which the component writes output.

Error Log Filter Components

Error log filter components implement filtering of error log events:

  • If no filter component is enabled, no filtering occurs.

  • Any enabled filter component affects log events only for components listed later in the log_error_services value. In particular, for any log sink component listed in log_error_services earlier than any filter component, no log event filtering occurs.

The log_filter_internal Component

Because log_error_verbosity affects the log_filter_internal component, log_error_verbosity has no effect on logging if log_filter_internal is not enabled.

Error Log Sink Components

Error log sink components are writers that implement error log output. If no sink component is enabled, no log output occurs.

Some sink component descriptions refer to the default error log destination. This is the console or a file and is indicated by the fault of the log_error system variable, determined as described in Default Error Log Destination Configuration.

The log_sink_internal Component
  • Purpose: Implements traditional error log message output format.

  • URN: This component is built in and need not be loaded with INSTALL COMPONENT before use.

  • Multiple uses permitted: No.

  • Output destination: Writes to the default error log destination.

The log_sink_json Component
  • Purpose: Implements JSON-format error logging.

  • URN: file://component_log_sink_json

  • Multiple uses permitted: Yes.

  • Output destination: The JSON log writer determines its output destination based on the default error log destination, which is given by the log_error system variable:

    • If log_error names a file, the JSON writer bases output file naming on that file name, plus a numbered .NN.json suffix, with NN starting at 00. For example, if log_error is file_name, successive instances of log_sink_json named in the log_error_services value write to file_name.00.json, file_name.01.json, and so forth.

    • If log_error is stderr, the JSON writer writes to the console. If log_json_writer is named multiple times in the log_error_services value, they all write to the console, which is likely not useful.

The log_sink_syseventlog Component
  • Purpose: Implements error logging to the system log. This is the Event Log on Windows, and syslog on Unix and Unix-like systems.

  • URN: file://component_log_sink_syseventlog

  • Multiple uses permitted: Yes.

  • Output destination: Writes to the system log. Does not use the default error log destination.

The log_sink_test Component
  • Purpose: Intended for internal use in writing test cases. Not intended for production use.

  • URN: file://component_log_sink_test

  • Multiple uses permitted: Yes.

  • Output destination: Writes to the default error log destination.

5.6 MySQL Server Plugins

MySQL supports a plugin API that enables creation of server components. Plugins can be loaded at server startup, or loaded and unloaded at runtime without restarting the server. The components supported by this interface include, but are not limited to, storage engines, INFORMATION_SCHEMA tables, full-text parser plugins, and server extensions.

MySQL distributions include several plugins that implement server extensions:

The following sections describe how to install and uninstall plugins, and how to determine at runtime which plugins are installed and obtain information about them. For information about writing plugins, see Section 28.2, “The MySQL Plugin API”.

5.6.1 Installing and Uninstalling Plugins

Server plugins must be loaded into the server before they can be used. MySQL supports plugin loading at server startup and runtime. It is also possible to control the activation state of loaded plugins at startup, and to unload them at runtime.

While a plugin is loaded, information about it is available at runtime from the INFORMATION_SCHEMA.PLUGINS table and the SHOW PLUGINS statement. See Section 5.6.2, “Obtaining Server Plugin Information”.

Installing Plugins

Before a server plugin can be used, it must be installed using one of the following methods. In the descriptions, plugin_name stands for a plugin name such as innodb, csv, or validate_password.

Built-in plugins:

A built-in plugin is known by the server automatically. Normally, the server enables the plugin at startup. Some built-in plugins permit this to be changed with the --plugin_name[=activation_state] option.

Plugins registered in the mysql.plugin system table:

The mysql.plugin table serves as a registry of plugins (other than built-in plugins, which need not be registered). At startup, the server loads each plugin listed in the table. Normally, for a plugin loaded from the mysql.plugin table, the server also enables the plugin. This can be changed with the --plugin_name[=activation_state] option.

If the server is started with the --skip-grant-tables option, it does not consult the mysql.plugin table and does not load the plugins listed there.

Plugins named with command-line options:

A plugin located in a plugin library file can be loaded at server startup with the --plugin-load, --plugin-load-add, or --early-plugin-load option. Normally, for a plugin loaded at startup, the server also enables the plugin. This can be changed with the --plugin_name[=activation_state] option.

The --plugin-load and --plugin-load-add options load plugins after built-in plugins and storage engines have initialized during the server startup sequence. The --early-plugin-load option is used to load plugins that must be available prior to initialization of built-in plugins and storage engines.

The value of each plugin-loading option is a semicolon-separated list of name=plugin_library and plugin_library values. Each name is the name of a plugin to load, and plugin_library is the name of the library file that contains the plugin code. If a plugin library is named without any preceding plugin name, the server loads all plugins in the library. The server looks for plugin library files in the directory named by the plugin_dir system variable.

Plugin-loading options do not register any plugin in the mysql.plugin table. For subsequent restarts, the server loads the plugin again only if --plugin-load, --plugin-load-add, or --early-plugin-load is given again. That is, the option produces a one-time plugin-installation operation that persists for a single server invocation.

--plugin-load, --plugin-load-add, and --early-plugin-load enable plugins to be loaded even when --skip-grant-tables is given (which causes the server to ignore the mysql.plugin table). --plugin-load, --plugin-load-add, and --early-plugin-load also enable plugins to be loaded at startup that cannot be loaded at runtime.

The --plugin-load-add option complements the --plugin-load option:

For example, these options:

--plugin-load=x --plugin-load-add=y

are equivalent to this option:

--plugin-load="x;y"

But these options:

--plugin-load-add=y --plugin-load=x

are equivalent to this option:

--plugin-load=x

Plugins installed with the INSTALL PLUGIN statement:

A plugin located in a plugin library file can be loaded at runtime with the INSTALL PLUGIN statement. The statement also registers the plugin in the mysql.plugin table to cause the server to load it on subsequent restarts. For this reason, INSTALL PLUGIN requires the INSERT privilege for the mysql.plugin table.

The plugin library file base name depends on your platform. Common suffixes are .so for Unix and Unix-like systems, .dll for Windows.

Example: The --plugin-load option installs a plugin at server startup. To install a plugin named myplugin from a plugin library file named somepluglib.so, use these lines in a my.cnf file:

[mysqld]
plugin-load=myplugin=somepluglib.so

In this case, the plugin is not registered in mysql.plugin. Restarting the server without the --plugin-load option causes the plugin not to be loaded at startup.

Alternatively, the INSTALL PLUGIN statement causes the server to load the plugin code from the library file at runtime:

INSTALL PLUGIN myplugin SONAME 'somepluglib.so';

INSTALL PLUGIN also causes permanent plugin registration: The plugin is listed in the mysql.plugin table to ensure that the server loads it on subsequent restarts.

Many plugins can be loaded either at server startup or at runtime. However, if a plugin is designed such that it must be loaded and initialized during server startup, attempts to load it at runtime using INSTALL PLUGIN produce an error:

mysql> INSTALL PLUGIN myplugin SONAME 'somepluglib.so';
ERROR 1721 (HY000): Plugin 'myplugin' is marked as not dynamically
installable. You have to stop the server to install it.

In this case, you must use --plugin-load, --plugin-load-add, or --early-plugin-load.

If a plugin is named both using a --plugin-load, --plugin-load-add, or --early-plugin-load option and (as a result of an earlier INSTALL PLUGIN statement) in the mysql.plugin table, the server starts but writes these messages to the error log:

[ERROR] Function 'plugin_name' already exists
[Warning] Couldn't load plugin named 'plugin_name'
with soname 'plugin_object_file'.

Controlling Plugin Activation State

If the server knows about a plugin when it starts (for example, because the plugin is named using a --plugin-load option or is registered in the mysql.plugin table), the server loads and enables the plugin by default. It is possible to control activation state for such a plugin using a --plugin_name[=activation_state] startup option, where plugin_name is the name of the plugin to affect, such as innodb, csv, or validate_password. As with other options, dashes and underscores are interchangeable in option names. Also, activation state values are not case sensitive. For example, --my_plugin=ON and --my-plugin=on are equivalent.

  • --plugin_name=OFF

    Tells the server to disable the plugin. This may not be possible for certain built-in plugins, such as mysql_native_password.

  • --plugin_name[=ON]

    Tells the server to enable the plugin. (Specifying the option as --plugin_name without a value has the same effect.) If the plugin fails to initialize, the server runs with the plugin disabled.

  • --plugin_name=FORCE

    Tells the server to enable the plugin, but if plugin initialization fails, the server does not start. In other words, this option forces the server to run with the plugin enabled or not at all.

  • --plugin_name=FORCE_PLUS_PERMANENT

    Like FORCE, but in addition prevents the plugin from being unloaded at runtime. If a user attempts to do so with UNINSTALL PLUGIN, an error occurs.

Plugin activation states are visible in the LOAD_OPTION column of the INFORMATION_SCHEMA.PLUGINS table.

Suppose that CSV, BLACKHOLE, and ARCHIVE are built-in pluggable storage engines and that you want the server to load them at startup, subject to these conditions: The server is permitted to run if CSV initialization fails, must require that BLACKHOLE initialization succeeds, and should disable ARCHIVE. To accomplish that, use these lines in an option file:

[mysqld]
csv=ON
blackhole=FORCE
archive=OFF

The --enable-plugin_name option format is a synonym for --plugin_name=ON. The --disable-plugin_name and --skip-plugin_name option formats are synonyms for --plugin_name=OFF.

If a plugin is disabled, either explicitly with OFF or implicitly because it was enabled with ON but failed to initialize, aspects of server operation that require the plugin will change. For example, if the plugin implements a storage engine, existing tables for the storage engine become inaccessible, and attempts to create new tables for the storage engine result in tables that use the default storage engine unless the NO_ENGINE_SUBSTITUTION SQL mode is enabled to cause an error to occur instead.

Disabling a plugin may require adjustment to other options. For example, if you start the server using --skip-innodb to disable InnoDB, other innodb_xxx options likely will need to be omitted at startup. In addition, because InnoDB is the default storage engine, it will not start unless you specify another available storage engine with --default_storage_engine. You must also set --default_tmp_storage_engine.

Uninstalling Plugins

At runtime, the UNINSTALL PLUGIN statement disables and uninstalls a plugin known to the server. The statement unloads the plugin and removes it from the mysql.plugin table, if it is registered there. For this reason, UNINSTALL PLUGIN statement requires the DELETE privilege for the mysql.plugin table. With the plugin no longer registered in the table, the server will not load the plugin automatically for subsequent restarts.

UNINSTALL PLUGIN can unload a plugin regardless of whether it was loaded at runtime with INSTALL PLUGIN or at startup with a plugin-loading option, subject to these conditions:

  • It cannot unload plugins that are built in to the server. These can be identified as those that have a library name of NULL in the output from INFORMATION_SCHEMA.PLUGINS or SHOW PLUGINS.

  • It cannot unload plugins for which the server was started with --plugin_name=FORCE_PLUS_PERMANENT, which prevents plugin unloading at runtime. These can be identified from the LOAD_OPTION column of the INFORMATION_SCHEMA.PLUGINS table.

To uninstall a plugin that currently is loaded at server startup with a plugin-loading option, use this procedure.

  1. Remove any options related to the plugin from the my.cnf file.

  2. Restart the server.

  3. Plugins normally are installed using either a plugin-loading option at startup or with INSTALL PLUGIN at runtime, but not both. However, removing options for a plugin from the my.cnf file may not be sufficient to uninstall it if at some point INSTALL PLUGIN has also been used. If the plugin still appears in the output from INFORMATION_SCHEMA.PLUGINS or SHOW PLUGINS, use UNINSTALL PLUGIN to remove it from the mysql.plugin table. Then restart the server again.

5.6.2 Obtaining Server Plugin Information

There are several ways to determine which plugins are installed in the server:

  • The INFORMATION_SCHEMA.PLUGINS table contains a row for each loaded plugin. Any that have a PLUGIN_LIBRARY value of NULL are built in and cannot be unloaded.

    mysql> SELECT * FROM information_schema.PLUGINS\G
    *************************** 1. row ***************************
               PLUGIN_NAME: binlog
            PLUGIN_VERSION: 1.0
             PLUGIN_STATUS: ACTIVE
               PLUGIN_TYPE: STORAGE ENGINE
       PLUGIN_TYPE_VERSION: 50158.0
            PLUGIN_LIBRARY: NULL
    PLUGIN_LIBRARY_VERSION: NULL
             PLUGIN_AUTHOR: MySQL AB
        PLUGIN_DESCRIPTION: This is a pseudo storage engine to represent the binlog in a transaction
            PLUGIN_LICENSE: GPL
               LOAD_OPTION: FORCE
    ...
    *************************** 10. row ***************************
               PLUGIN_NAME: InnoDB
            PLUGIN_VERSION: 1.0
             PLUGIN_STATUS: ACTIVE
               PLUGIN_TYPE: STORAGE ENGINE
       PLUGIN_TYPE_VERSION: 50158.0
            PLUGIN_LIBRARY: ha_innodb_plugin.so
    PLUGIN_LIBRARY_VERSION: 1.0
             PLUGIN_AUTHOR: Innobase Oy
        PLUGIN_DESCRIPTION: Supports transactions, row-level locking,
                            and foreign keys
            PLUGIN_LICENSE: GPL
               LOAD_OPTION: ON
    ...
    
  • The SHOW PLUGINS statement displays a row for each loaded plugin. Any that have a Library value of NULL are built in and cannot be unloaded.

    mysql> SHOW PLUGINS\G
    *************************** 1. row ***************************
       Name: binlog
     Status: ACTIVE
       Type: STORAGE ENGINE
    Library: NULL
    License: GPL
    ...
    *************************** 10. row ***************************
       Name: InnoDB
     Status: ACTIVE
       Type: STORAGE ENGINE
    Library: ha_innodb_plugin.so
    License: GPL
    ...
    
  • The mysql.plugin table shows which plugins have been registered with INSTALL PLUGIN. The table contains only plugin names and library file names, so it does not provide as much information as the PLUGINS table or the SHOW PLUGINS statement.

5.6.3 The Rewriter Query Rewrite Plugin

MySQL Server supports query rewrite plugins that can examine and possibly modify statements received by the server before the server executes them. See Query Rewrite Plugins.

MySQL distributions include a postparse query rewrite plugin named Rewriter and scripts for installing the plugin and its associated components. These components work together to provide SELECT rewriting capability:

  • A server-side plugin named Rewriter examines SELECT statements and may rewrite them, based on its in-memory cache of rewrite rules. Standalone SELECT statements and SELECT statements in prepared statements are subject to rewriting. SELECT statements occurring within view definitions or stored programs are not subject to rewriting.

  • The Rewriter plugin uses a database named query_rewrite containing a table named rewrite_rules. The table provides persistent storage for the rules that the plugin uses to decide whether to rewrite statements. Users communicate with the plugin by modifying the set of rules stored in this table. The plugin communicates with users by setting the message column of table rows.

  • The query_rewrite database contains a stored procedure named flush_rewrite_rules() that loads the contents of the rules table into the plugin.

  • A user-defined function named load_rewrite_rules() is used by the flush_rewrite_rules() stored procedure.

  • The Rewriter plugin exposes system variables that enable plugin configuration and status variables that provide runtime operational information.

The following sections describe how to install and use the Rewriter plugin, and provide reference information for its associated components.

5.6.3.1 Installing or uninstalling the Rewriter Query Rewrite Plugin

Note

If installed, the Rewriter plugin involves some overhead even when disabled. To avoid this overhead, do not install the plugin unless you plan to use it.

To install or uninstall the Rewriter query rewrite plugin, choose the approropriate script located in the share directory of your MySQL installation:

  • install_rewriter.sql: Choose this script to install the Rewriter plugin and its associated components.

    Note

    The pattern_digest and normalized_columns columns are also created. (For details about the table columns, see Section 5.6.3.3.1, “Rewriter Query Rewrite Plugin Rules Table”.)

  • uninstall_rewriter.sql: Choose this script to uninstall the Rewriter plugin and its associated components.

Run the chosen script as follows:

shell> mysql -u root -p < install_rewriter.sql
Enter password: (enter root password here)

The example here uses the install_rewriter.sql installation script. Make the appropriate substitution if you choose a different script.

Running an installation script should install and enable the plugin. To verify that, connect to the server and execute this statement:

mysql> SHOW GLOBAL VARIABLES LIKE 'rewriter_enabled';
+------------------+-------+
| Variable_name    | Value |
+------------------+-------+
| rewriter_enabled | ON    |
+------------------+-------+

For usage instructions, see Section 5.6.3.2, “Using the Rewriter Query Rewrite Plugin”. For reference information, see Section 5.6.3.3, “Rewriter Query Rewrite Plugin Reference”.

5.6.3.2 Using the Rewriter Query Rewrite Plugin

To enable or disable the plugin, enable or disable the rewriter_enabled system variable. By default, the Rewriter plugin is enabled when you install it (see Section 5.6.3.1, “Installing or uninstalling the Rewriter Query Rewrite Plugin”). To set the initial plugin state explicitly, you can set the variable at server startup. For example, to enable the plugin in an option file, use these lines:

[mysqld]
rewriter_enabled=ON

It is also possible to enable or disable the plugin at runtime:

mosql> SET GLOBAL rewriter_enabled = ON;
mysql> SET GLOBAL rewriter_enabled = OFF;

Assuming that the Rewriter plugin is enabled, it examines and possibly modifies each SELECT statement received by the server. The plugin determines whether to rewrite statements based on its in-memory cache of rewriting rules, which are loaded from the rewrite_rules table in the query_rewrite database.

Adding Rewrite Rules

To add rules for the Rewriter plugin, add rows to the rewrite_rules table, then invoke the flush_rewrite_rules() stored procedure to load the rules from the table into the plugin. The following example creates a simple rule to match statements that select a single literal value:

mysql> INSERT INTO query_rewrite.rewrite_rules (pattern, replacement)
    -> VALUES('SELECT ?', 'SELECT ? + 1');

The resulting table contents look like this:

mysql> SELECT * FROM query_rewrite.rewrite_rules\G
*************************** 1. row ***************************
                id: 1
           pattern: SELECT ?
  pattern_database: NULL
       replacement: SELECT ? + 1
           enabled: YES
           message: NULL
    pattern_digest: NULL
normalized_pattern: NULL

The rule specifies a pattern template indicating which SELECT statements to match, and a replacement template indicating how to rewrite matching statements. However, adding the rule to the rewrite_rules table is not sufficient to cause the Rewriter plugin to use the rule. You must invoke flush_rewrite_rules() to load the table contents into the plugin in-memory cache:

mysql> CALL query_rewrite.flush_rewrite_rules();
Tip

If your rewrite rules seem not to be working properly, make sure that you have reloaded the rules table by calling flush_rewrite_rules().

When the plugin reads each rule from the rules table, it computes a normalized (statement digest) form from the pattern and a digest hash value, and uses them to update the normalized_pattern and pattern_digest columns:

mysql> SELECT * FROM query_rewrite.rewrite_rules\G
*************************** 1. row ***************************
                id: 1
           pattern: SELECT ?
  pattern_database: NULL
       replacement: SELECT ? + 1
           enabled: YES
           message: NULL
    pattern_digest: 46b876e64cd5c41009d91c754921f1d4
normalized_pattern: select ?

For information about statement digesting, normalized statements, and digest hash values, see Section 25.9, “Performance Schema Statement Digests and Sampling”.

Patterns use the same syntax as prepared statements (see Section 13.5.1, “PREPARE Syntax”). Within a pattern template, ? characters act as parameter markers that match data values. Parameter markers can be used only where data values should appear, not for SQL keywords, identifiers, and so forth. The ? characters should not be enclosed within quotation marks.

Like the pattern, the replacement can contain ? characters. For a statement that matches a pattern template, the plugin rewrites it, replacing ? parameter markers in the replacement using data values matched by the corresponding markers in the pattern. The result is a complete statement string. The plugin asks the server to parse it, and returns the result to the server as the representation of the rewritten statement.

After adding and loading the rule, check whether rewriting occurs according to whether statements match the rule pattern:

mysql> SELECT PI();
+----------+
| PI()     |
+----------+
| 3.141593 |
+----------+
1 row in set (0.01 sec)

mysql> SELECT 10;
+--------+
| 10 + 1 |
+--------+
|     11 |
+--------+
1 row in set, 1 warning (0.00 sec)

No rewriting occurs for the first SELECT statement, but does for the second. The second statement illustrates that when the Rewriter plugin rewrites a statement, it produces a warning message. To view the message, use SHOW WARNINGS:

mysql> SHOW WARNINGS\G
*************************** 1. row ***************************
  Level: Note
   Code: 1105
Message: Query 'SELECT 10' rewritten to 'SELECT 10 + 1' by a query rewrite plugin

To enable or disable an existing rule, modify its enabled column and reload the table into the plugin. To disable rule 1:

mysql> UPDATE query_rewrite.rewrite_rules SET enabled = 'NO' WHERE id = 1;
mysql> CALL query_rewrite.flush_rewrite_rules();

This enables you to deactivate a rule without removing it from the table.

To re-enable rule 1:

mysql> UPDATE query_rewrite.rewrite_rules SET enabled = 'YES' WHERE id = 1;
mysql> CALL query_rewrite.flush_rewrite_rules();

The rewrite_rules table contains a pattern_database column that Rewriter uses for matching table names that are not qualified with a database name:

  • Qualified table names in statements match qualified names in the pattern if corresponding database and table names are identical.

  • Unqualified table names in statements match unqualified names in the pattern only if the default database is the same as pattern_database and the table names are identical.

Suppose that a table named appdb.users has a column named id and that applications are expected to select rows from the table using a query of one of these forms, where the second can be used only if appdb is the default database:

SELECT * FROM users WHERE appdb.id = id_value;
SELECT * FROM users WHERE id = id_value;

Suppose also that the id column is renamed to user_id (perhaps the table must be modified to add another type of ID and it is necessary to indicate more specifically what type of ID the id column represents).

The change means that applications must refer to user_id rather than id in the WHERE clause. But if there are old applications that cannot be written to change the SELECT queries they generate, they will no longer work properly. The Rewriter plugin can solve this problem. To match and rewrite statements whether or not they qualify the table name, add the following two rules and reload the rules table:

mysql> INSERT INTO query_rewrite.rewrite_rules
    -> (pattern, replacement) VALUES(
    -> 'SELECT * FROM appdb.users WHERE id = ?',
    -> 'SELECT * FROM appdb.users WHERE user_id = ?'
    -> );
mysql> INSERT INTO query_rewrite.rewrite_rules
    -> (pattern, replacement, pattern_database) VALUES(
    -> 'SELECT * FROM users WHERE id = ?',
    -> 'SELECT * FROM users WHERE user_id = ?',
    -> 'appdb'
    -> );
mysql> CALL query_rewrite.flush_rewrite_rules();

Rewriter uses the first rule to match statements that use the qualified table name. It uses the second to match statements that used the unqualified name, but only if the default database is appdb (the value in pattern_database).

How Statement Matching Works

The Rewriter plugin uses statement digests and digest hash values to match incoming statements against rewrite rules in stages. The max_digest_length system variable determines the size of the buffer used for computing statement digests. Larger values enable computation of digests that distinguish longer statements. Smaller values use less memory but increase the likelihood of longer statements colliding with the same digest value.

The plugin matches each statement to the rewrite rules as follows:

  1. Compute the statement digest hash value and compare it to the rule digest hash values. This is subject to false positives, but serves as a quick rejection test.

  2. If the statement digest hash value matches any pattern digest hash values, match the normalized (statement digest) form of the statement to the normalized form of the matching rule patterns.

  3. If the normalized statement matches a rule, compare the literal values in the statement and the pattern. A ? in the pattern matches any literal value in the statement. If the statement prepares a SELECT statement, ? in the pattern also matches ? in the statement. Otherwise, corresponding literals must be the same.

If multiple rules match a statement, it is indeterminate which one the plugin uses to rewrite the statement.

If a pattern contains more markers than the replacement, the plugin discards excess data values. If a pattern contains fewer markers than the replacement, it is an error. The plugin notices this when the rules table is loaded, writes an error message to the message column of the rule row to communicate the problem, and sets the Rewriter_reload_error status variable to ON.

Rewriting Prepared Statements

Prepared statements are rewritten at parse time (that is, when they are prepared), not when they are executed later.

Prepared statements differ from nonprepared statements in that they may contain ? characters as parameter markers. To match a ? in a prepared statement, a Rewriter pattern must contain ? in the same location. Suppose that a rewrite rule has this pattern:

SELECT ?, 3

The following table shows several prepared SELECT statements and whether the rule pattern matches them.

Prepared StatementWhether Pattern Matches Statement
PREPARE s AS 'SELECT 3, 3'Yes
PREPARE s AS 'SELECT ?, 3'Yes
PREPARE s AS 'SELECT 3, ?'No
PREPARE s AS 'SELECT ?, ?'No
Rewriter Plugin Operational Information

The Rewriter plugin makes information available about its operation by means of several status variables:

mysql> SHOW GLOBAL STATUS LIKE 'Rewriter%';
+-----------------------------------+-------+
| Variable_name                     | Value |
+-----------------------------------+-------+
| Rewriter_number_loaded_rules      | 1     |
| Rewriter_number_reloads           | 5     |
| Rewriter_number_rewritten_queries | 1     |
| Rewriter_reload_error             | ON    |
+-----------------------------------+-------+

For descriptions of these variables, see Section 5.6.3.3.4, “Rewriter Query Rewrite Plugin Status Variables”.

When you load the rules table by calling the flush_rewrite_rules() stored procedure, if an error occurs for some rule, the CALL statement produces an error, and the plugin sets the Rewriter_reload_error status variable to ON:

mysql> CALL query_rewrite.flush_rewrite_rules();
ERROR 1644 (45000): Loading of some rule(s) failed.

mysql> SHOW GLOBAL STATUS LIKE 'Rewriter_reload_error';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| Rewriter_reload_error | ON    |
+-----------------------+-------+

In this case, check the message column of rewrite_rules table rows for non-NULL values to see what the problem was.

Rewriter Plugin Use of Character Sets

When the rewrite_rules table is loaded into the Rewriter plugin, the plugin interprets statements using the current global value of the character_set_client system variable. If the global character_set_client value is changed subsequently, the rules table must be reloaded.

A client must have a session character_set_client value identical to what the global value was when the rules table was loaded or rule matching will not work for that client.

5.6.3.3 Rewriter Query Rewrite Plugin Reference

The following discussion serves as a reference to these components associated with the Rewriter query rewrite plugin:

  • The Rewriter rules table in the query_rewrite database

  • Rewriter procedures and functions

  • Rewriter system and status variables

5.6.3.3.1 Rewriter Query Rewrite Plugin Rules Table

The rewrite_rules table in the query_rewrite database provides persistent storage for the rules that the Rewriter plugin uses to decide whether to rewrite statements.

Users communicate with the plugin by modifying the set of rules stored in this table. The plugin communicates with users by setting the table's message column.

Note

The rules table is loaded into the plugin by the flush_rewrite_rules stored procedure. Unless that procedure has been called following the most recent table modification, the table contents do not necessarily correspond to the set of rules the plugin is using.

The rewrite_rules table has these columns:

  • id

    The rule ID. This column is the table primary key. You can use the ID to uniquely identify any rule.

  • pattern

    The template that indicates the pattern for statements that the rule matches. Use ? to represent parameter markers that match data values.

  • pattern_database

    The database used to match unqualified table names in statements. Qualified table names in statements match qualified names in the pattern if corresponding database and table names are identical. Unqualified table names in statements match unqualified names in the pattern only if the default database is the same as pattern_database and the table names are identical.

  • replacement

    The template that indicates how to rewrite statements matching the pattern column value. Use ? to represent parameter markers that match data values. In rewritten statements, the plugin replaces ? parameter markers in replacement using data values matched by the corresponding markers in pattern.

  • enabled

    Whether the rule is enabled. Load operations (performed by invoking the flush_rewrite_rules() stored procedure) load the rule from the table into the Rewriter in-memory cache only if this column is YES.

    This column makes it possible to deactivate a rule without removing it: Set the column to a value other than YES and reload the table into the plugin.

  • message

    The plugin uses this column for communicating with users. If no error occurs when the rules table is loaded into memory, the plugin sets the message column to NULL. A non-NULL value indicates an error and the column contents are the error message. Errors can occur under these circumstances:

    • Either the pattern or the replacement is an incorrect SQL statement that produces syntax errors.

    • The replacement contains more ? parameter markers than the pattern.

    If a load error occurs, the plugin also sets the Rewriter_reload_error status variable to ON.

  • pattern_digest

    This column is used for debugging and diagnostics. If the column exists when the rules table is loaded into memory, the plugin updates it with the pattern digest. This column may be useful if you are trying to determine why some statement fails to be rewritten.

  • normalized_pattern

    This column is used for debugging and diagnostics. If the column exists when the rules table is loaded into memory, the plugin updates it with the normalized form of the pattern. This column may be useful if you are trying to determine why some statement fails to be rewritten.

5.6.3.3.2 Rewriter Query Rewrite Plugin Procedures and Functions

Rewriter plugin operation uses a stored procedure that loads the rules table into its in-memory cache, and a helper user-defined function (UDF). Under normal operation, users invoke only the stored procedure. The UDF is intended to be invoked by the stored procedure, not directly by users.

  • flush_rewrite_rules()

    This stored procedure uses the load_rewrite_rules() UDF to load the contents of the rewrite_rules table into the Rewriter in-memory cache.

    Calling flush_rewrite_rules() implies COMMIT.

    Invoke this procedure after you modify the rules table to cause the plugin to update its cache from the new table contents. If any errors occur, the plugin sets the message column for the appropriate rule rows in the table and sets the Rewriter_reload_error status variable to ON.

  • load_rewrite_rules()

    This UDF is a helper routine used by the flush_rewrite_rules() stored procedure.

5.6.3.3.3 Rewriter Query Rewrite Plugin System Variables

The Rewriter query rewrite plugin supports the following system variables. These variables are available only if the plugin is installed (see Section 5.6.3.1, “Installing or uninstalling the Rewriter Query Rewrite Plugin”).

5.6.3.3.4 Rewriter Query Rewrite Plugin Status Variables

The Rewriter query rewrite plugin supports the following status variables. These variables are available only if the plugin is installed (see Section 5.6.3.1, “Installing or uninstalling the Rewriter Query Rewrite Plugin”).

  • Rewriter_number_loaded_rules

    The number of rewrite plugin rewrite rules successfully loaded from the rewrite_rules table into memory for use by the Rewriter plugin.

  • Rewriter_number_reloads

    The number of times the rewrite_rules table has been loaded into the in-memory cache used by the Rewriter plugin.

  • Rewriter_number_rewritten_queries

    The number of queries rewritten by the Rewriter query rewrite plugin since it was loaded.

  • Rewriter_reload_error

    Whether an error occurred the most recent time that the rewrite_rules table was loaded into the in-memory cache used by the Rewriter plugin. If the value is OFF, no error occurred. If the value is ON, an error occurred; check the message column of the rewriter_rules table for error messages.

5.6.4 Version Tokens

MySQL includes Version Tokens, a feature that enables creation of and synchronization around server tokens that applications can use to prevent accessing incorrect or out-of-date data.

The Version Tokens interface has these characteristics:

  • Version tokens are pairs consisting of a name that serves as a key or identifier, plus a value.

  • Version tokens can be locked. An application can use token locks to indicate to other cooperating applications that tokens are in use and should not be modified.

  • Version token lists are established per server; for example, to specify the server assignment or operational state. In addition, an application that communicates with a server can register its own list of tokens that indicate the state it requires the server to be in. An SQL statement sent by the application to a server not in the required state produces an error. This is a signal to the application that it should seek a different server in the required state to receive the SQL statement.

The following sections describe the components of Version Tokens, discuss how to install and use it, and provide reference information for its components.

5.6.4.1 Version Tokens Components

Version Tokens is based on a plugin library that implements these components:

  • A server-side plugin named version_tokens holds the list of version tokens associated with the server and subscribes to notifications for statement execution events. The version_tokens plugin uses the audit plugin API to monitor incoming statements from clients and matches each client's session-specific version token list against the server version token list. If there is a match, the plugin lets the statement through and the server continues to process it. Otherwise, the plugin returns an error to the client and the statement fails.

  • A set of user-defined functions (UDFs) provides an SQL-level API for manipulating and inspecting the list of server version tokens maintained by the plugin. The VERSION_TOKEN_ADMIN or SUPER privilege is required to call any of the Version Token UDFs.

  • When the version_tokens plugin loads, it defines the VERSION_TOKEN_ADMIN dynamic privilege. This privilege can be granted to users of the UDFs.

  • A system variable enables clients to specify the list of version tokens that register the required server state. If the server has a different state when a client sends a statement, the client receives an error.

5.6.4.2 Installing or Uninstalling Version Tokens

Note

If installed, Version Tokens involves some overhead. To avoid this overhead, do not install it unless you plan to use it.

This section describes how to install or uninstall Version Tokens, which is implemented in a plugin library file containing a plugin and user-defined functions. For general information about installing or uninstalling plugins and UDFs, see Section 5.6.1, “Installing and Uninstalling Plugins”, and Section 28.4.2.5, “UDF Compiling and Installing”.

To be usable by the server, the plugin library file must be located in the MySQL plugin directory (the directory named by the plugin_dir system variable). If necessary, set the value of plugin_dir at server startup to tell the server the plugin directory location.

The plugin library file base name is version_tokens. The file name suffix differs per platform (for example, .so for Unix and Unix-like systems, .dll for Windows).

To install the Version Tokens plugin and UDFs, use the INSTALL PLUGIN and CREATE FUNCTION statements (adjust the .so suffix for your platform as necessary):

INSTALL PLUGIN version_tokens SONAME 'version_token.so';
CREATE FUNCTION version_tokens_set RETURNS STRING SONAME 'version_token.so';
CREATE FUNCTION version_tokens_show RETURNS STRING SONAME 'version_token.so';
CREATE FUNCTION version_tokens_edit RETURNS STRING SONAME 'version_token.so';
CREATE FUNCTION version_tokens_delete RETURNS STRING SONAME 'version_token.so';
CREATE FUNCTION version_tokens_lock_shared RETURNS INT SONAME 'version_token.so';
CREATE FUNCTION version_tokens_lock_exclusive RETURNS INT SONAME 'version_token.so';
CREATE FUNCTION version_tokens_unlock RETURNS INT SONAME 'version_token.so';

You must install the UDFs to manage the server's version token list, but you must also install the plugin because the UDFs will not work correctly without it.

If the plugin and UDFs are used on a master replication server, install them on all slave servers as well to avoid replication problems.

Once installed as just described, the Version Tokens plugin and UDFs remain installed until uninstalled. To remove them, use the UNINSTALL PLUGIN and DROP FUNCTION statements:

UNINSTALL PLUGIN version_tokens;
DROP FUNCTION version_tokens_set;
DROP FUNCTION version_tokens_show;
DROP FUNCTION version_tokens_edit;
DROP FUNCTION version_tokens_delete;
DROP FUNCTION version_tokens_lock_shared;
DROP FUNCTION version_tokens_lock_exclusive;
DROP FUNCTION version_tokens_unlock;

5.6.4.3 Using Version Tokens

Before using Version Tokens, install it according to the instructions provided at Section 5.6.4.2, “Installing or Uninstalling Version Tokens”.

A scenario in which Version Tokens can be useful is a system that accesses a collection of MySQL servers but needs to manage them for load balancing purposes by monitoring them and adjusting server assignments according to load changes. Such a system comprises these components:

  • The collection of MySQL servers to be managed.

  • An administrative or management application that communicates with the servers and organizes them into high-availability groups. Groups serve different purposes, and servers within each group may have different assignments. Assignment of a server within a certain group can change at any time.

  • Client applications that access the servers to retrieve and update data, choosing servers according to the purposes assigned them. For example, a client should not send an update to a read-only server.

Version Tokens permit server access to be managed according to assignment without requiring clients to repeatedly query the servers about their assignments:

  • The management application performs server assignments and establishes version tokens on each server to reflect its assignment. The application caches this information to provide a central access point to it.

    If at some point the management application needs to change a server assignment (for example, to change it from permitting writes to read only), it changes the server's version token list and updates its cache.

  • To improve performance, client applications obtain cache information from the management application, enabling them to avoid having to retrieve information about server assignments for each statement. Based on the type of statements it will issue (for example, reads versus writes), a client selects an appropriate server and connects to it.

  • In addition, the client sends to the server its own client-specific version tokens to register the assignment it requires of the server. For each statement sent by the client to the server, the server compares its own token list with the client token list. If the server token list contains all tokens present in the client token list with the same values, there is a match and the server executes the statement.

    On the other hand, perhaps the management application has changed the server assignment and its version token list. In this case, the new server assignment may now be incompatible with the client requirements. A token mismatch between the server and client token lists occurs and the server returns an error in reply to the statement. This is an indication to the client to refresh its version token information from the management application cache, and to select a new server to communicate with.

The client-side logic for detecting version token errors and selecting a new server can be implemented different ways:

  • The client can handle all version token registration, mismatch detection, and connection switching itself.

  • The logic for those actions can be implemented in a connector that manages connections between clients and MySQL servers. Such a connector might handle mismatch error detection and statement resending itself, or it might pass the error to the application and leave it to the application to resend the statement.

The following example illustrates the preceding discussion in more concrete form.

When Version Tokens initializes on a given server, the server's version token list is empty. Token list maintenance is performed by calling user-defined functions (UDFs). The VERSION_TOKEN_ADMIN or SUPER privilege is required to call any of the Version Token UDFs, so token list modification is expected to be done by a management or administrative application that has that privilege.

Suppose that a management application communicates with a set of servers that are queried by clients to access employee and product databases (named emp and prod, respectively). All servers are permitted to process data retrieval statements, but only some of them are permitted to make database updates. To handle this on a database-specific basis, the management application establishes a list of version tokens on each server. In the token list for a given server, token names represent database names and token values are read or write depending on whether the database must be used in read-only fashion or whether it can take reads and writes.

Client applications register a list of version tokens they require the server to match by setting a system variable. Variable setting occurs on a client-specific basis, so different clients can register different requirements. By default, the client token list is empty, which matches any server token list. When a client sets its token list to a nonempty value, matching may succeed or fail, depending on the server version token list.

To define the version token list for a server, the management application calls the version_token_set() UDF. (There are also UDFs for modifying and displaying the token list, described later.) For example, the application might send these statements to a group of three servers:

Server 1:

mysql> SELECT version_tokens_set('emp=read;prod=read');
+------------------------------------------+
| version_tokens_set('emp=read;prod=read') |
+------------------------------------------+
| 2 version tokens set.                    |
+------------------------------------------+

Server 2:

mysql> SELECT version_tokens_set('emp=write;prod=read');
+-------------------------------------------+
| version_tokens_set('emp=write;prod=read') |
+-------------------------------------------+
| 2 version tokens set.                     |
+-------------------------------------------+

Server 3:

mysql> SELECT version_tokens_set('emp=read;prod=write');
+-------------------------------------------+
| version_tokens_set('emp=read;prod=write') |
+-------------------------------------------+
| 2 version tokens set.                     |
+-------------------------------------------+

The token list in each case is specified as a semicolon-separated list of name=value pairs. The resulting token list values result in these server assingments:

  • Any server accepts reads for either database.

  • Only server 2 accepts updates for the emp database.

  • Only server 3 accepts updates for the prod database.

In addition to assigning each server a version token list, the management application also maintains a cache that reflects the server assignments.

Before communicating with the servers, a client application contacts the management application and retrieves information about server assignments. Then the client selects a server based on those assignments. Suppose that a client wants to perform both reads and writes on the emp database. Based on the preceding assignments, only server 2 qualifies. The client connects to server 2 and registers its server requirements there by setting its version_tokens_session system variable:

mysql> SET @@session.version_tokens_session = 'emp=write';

For subsequent statements sent by the client to server 2, the server compares its own version token list to the client list to check whether they match. If so, statements execute normally:

mysql> UPDATE emp.employee SET salary = salary * 1.1 WHERE id = 4981;
Query OK, 1 row affected (0.07 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> SELECT last_name, first_name FROM emp.employee WHERE id = 4981;
+-----------+------------+
| last_name | first_name |
+-----------+------------+
| Smith     | Abe        |
+-----------+------------+
1 row in set (0.01 sec)

Discrepancies between the server and client version token lists can occur two ways:

As long as the assignment of server 2 does not change, the client continues to use it for reads and writes. But suppose that the management application wants to change server assignments so that writes for the emp database must be sent to server 1 instead of server 2. To do this, it uses version_tokens_edit() to modify the emp token value on the two servers (and updates its cache of server assignments):

Server 1:

mysql> SELECT version_tokens_edit('emp=write');
+----------------------------------+
| version_tokens_edit('emp=write') |
+----------------------------------+
| 1 version tokens updated.        |
+----------------------------------+

Server 2:

mysql> SELECT version_tokens_edit('emp=read');
+---------------------------------+
| version_tokens_edit('emp=read') |
+---------------------------------+
| 1 version tokens updated.       |
+---------------------------------+

version_tokens_edit() modifies the named tokens in the server token list and leaves other tokens unchanged.

The next time the client sends a statement to server 2, its own token list no longer matches the server token list and an error occurs:

mysql> UPDATE emp.employee SET salary = salary * 1.1 WHERE id = 4982;
ERROR 3136 (42000): Version token mismatch for emp. Correct value read

In this case, the client should contact the management application to obtain updated information about server assignments, select a new server, and send the failed statement to the new server.

Note

Each client must cooperate with Version Tokens by sending only statements in accordance with the token list that it registers with a given server. For example, if a client registers a token list of 'emp=read', there is nothing in Version Tokens to prevent the client from sending updates for the emp database. The client itself must refrain from doing so.

For each statement received from a client, the server implicitly uses locking, as follows:

  • Take a shared lock for each token named in the client token list (that is, in the version_tokens_session value)

  • Perform the comparison between the server and client token lists

  • Execute the statement or produce an error depending on the comparison result

  • Release the locks

The server uses shared locks so that comparisons for multiple sessions can occur without blocking, while preventing changes to the tokens for any session that attempts to acquire an exclusive lock before it manipulates tokens of the same names in the server token list.

The preceding example uses only a few of the user-defined included in the Version Tokens plugin library, but there are others. One set of UDFs permits the server's list of version tokens to be manipulated and inspected. Another set of UDFs permits version tokens to be locked and unlocked.

These UDFs permit the server's list of version tokens to be created, changed, removed, and inspected:

  • version_tokens_set() completely replaces the current list and assigns a new list. The argument is a semicolon-separated list of name=value pairs.

  • version_tokens_edit() enables partial modifications to the current list. It can add new tokens or change the values of existing tokens. The argument is a semicolon-separated list of name=value pairs.

  • version_tokens_delete() deletes tokens from the current list. The argument is a semicolon-separated list of token names.

  • version_tokens_show() displays the current token list. It takes no argument.

Each of those functions, if successful, returns a binary string indicating what action occurred. The following example establishes the server token list, modifies it by adding a new token, deletes some tokens, and displays the resulting token list:

mysql> SELECT version_tokens_set('tok1=a;tok2=b');
+-------------------------------------+
| version_tokens_set('tok1=a;tok2=b') |
+-------------------------------------+
| 2 version tokens set.               |
+-------------------------------------+
mysql> SELECT version_tokens_edit('tok3=c');
+-------------------------------+
| version_tokens_edit('tok3=c') |
+-------------------------------+
| 1 version tokens updated.     |
+-------------------------------+
mysql> SELECT version_tokens_delete('tok2;tok1');
+------------------------------------+
| version_tokens_delete('tok2;tok1') |
+------------------------------------+
| 2 version tokens deleted.          |
+------------------------------------+
mysql> SELECT version_tokens_show();
+-----------------------+
| version_tokens_show() |
+-----------------------+
| tok3=c;               |
+-----------------------+

Warnings occur if a token list is malformed:

mysql> SELECT version_tokens_set('tok1=a; =c');
+----------------------------------+
| version_tokens_set('tok1=a; =c') |
+----------------------------------+
| 1 version tokens set.            |
+----------------------------------+
1 row in set, 1 warning (0.00 sec)

mysql> SHOW WARNINGS\G
*************************** 1. row ***************************
  Level: Warning
   Code: 42000
Message: Invalid version token pair encountered. The list provided
         is only partially updated.
1 row in set (0.00 sec)

As mentioned previously, version tokens are defined using a semicolon-separated list of name=value pairs. Consider this invocation of version_tokens_set():

mysql> SELECT version_tokens_set('tok1=b;;; tok2= a = b ; tok1 = 1\'2 3"4')
+---------------------------------------------------------------+
| version_tokens_set('tok1=b;;; tok2= a = b ; tok1 = 1\'2 3"4') |
+---------------------------------------------------------------+
| 3 version tokens set.                                         |
+---------------------------------------------------------------+

Version Tokens interprets the argument as follows:

  • Whitespace around names and values is ignored. Whitespace within names and values is permitted. (For version_tokens_delete(), which takes a list of names without values, whitespace around names is ignored.)

  • There is no quoting mechanism.

  • Order of tokens is not significant except that if a token list contains multiple instances of a given token name, the last value takes precedence over earlier values.

Given those rules, the preceding version_tokens_set() call results in a token list with two tokens: tok1 has the value 1'2 3"4, and tok2 has the value a = b. To verify this, call version_tokens_show():

mysql> SELECT version_tokens_show();
+--------------------------+
| version_tokens_show()    |
+--------------------------+
| tok2=a = b;tok1=1'2 3"4; |
+--------------------------+

If the token list contains two tokens, why did version_tokens_set() return the value 3 version tokens set? That occurred because the original token list contained two definitions for tok1, and the second definition replaced the first.

The Version Tokens token-manipulation UDFs place these constraints on token names and values:

  • Token names cannot contain = or ; characters and have a maximum length of 64 characters.

  • Token values cannot contain ; characters. Length of values is constrained by the value of the max_allowed_packet system variable.

  • Version Tokens treats token names and values as binary strings, so comparisons are case sensitive.

Version Tokens also includes a set of UDFs enabling tokens to be locked and unlocked:

  • version_tokens_lock_exclusive() acquires exclusive version token locks. It takes a list of one or more lock names and a timeout value.

  • version_tokens_lock_shared() acquires shared version token locks. It takes a list of one or more lock names and a timeout value.

  • version_tokens_unlock() releases version token locks (exclusive and shared). It takes no argument.

Each locking function returns nonzero for success. Otherwise, an error occurs:

mysql> SELECT version_tokens_lock_shared('lock1', 'lock2', 0);
+-------------------------------------------------+
| version_tokens_lock_shared('lock1', 'lock2', 0) |
+-------------------------------------------------+
|                                               1 |
+-------------------------------------------------+

mysql> SELECT version_tokens_lock_shared(NULL, 0);
ERROR 3131 (42000): Incorrect locking service lock name '(null)'.

Locking using Version Tokens locking functions is advisory; applications must agree to cooperate.

It is possible to lock nonexisting token names. This does not create the tokens.

Note

Version Tokens locking functions are based on the locking service described at Section 28.3.1, “The Locking Service”, and thus have the same semantics for shared and exclusive locks. (Version Tokens uses the locking service routines built into the server, not the locking service UDF interface, so those UDFs need not be installed to use Version Tokens.) Locks acquired by Version Tokens use a locking service namespace of version_token_locks. Locking service locks can be monitored using the Performance Schema, so this is also true for Version Tokens locks. For details, see Section 28.3.1.2.3, “Locking Service Monitoring”.

For the Version Tokens locking functions, token name arguments are used exactly as specified. Surrounding whitespace is not ignored and = and ; characters are permitted. This is because Version Tokens simply passes the token names to be locked as is to the locking service.

5.6.4.4 Version Tokens Reference

The following discussion serves as a reference to these Version Tokens components:

Version Tokens Functions

The Version Tokens plugin library includes several user-defined functions. One set of UDFs permits the server's list of version tokens to be manipulated and inspected. Another set of UDFs permits version tokens to be locked and unlocked. The VERSION_TOKEN_ADMIN or SUPER privilege is required to invoke any Version Tokens UDF.

The following UDFs permit the server's list of version tokens to be created, changed, removed, and inspected. Interpretation of name_list and token_list arguments (including whitespace handling) occurs as described in Section 5.6.4.3, “Using Version Tokens”, which provides details about the syntax for specifying tokens, as well as additional examples.

  • version_tokens_delete(name_list)

    Deletes tokens from the server's list of version tokens using the name_list argument and returns a binary string that indicates the outcome of the operation. name_list is a semicolon-separated list of version token names to delete.

    mysql> SELECT version_tokens_delete('tok1;tok3');
    +------------------------------------+
    | version_tokens_delete('tok1;tok3') |
    +------------------------------------+
    | 2 version tokens deleted.          |
    +------------------------------------+
    

    An argument of NULL is treated as an empty string, which has no effect on the token list.

    version_tokens_delete() deletes the tokens named in its argument, if they exist. (It is not an error to delete nonexisting tokens.) To clear the token list entirely without knowing which tokens are in the list, pass NULL or a string containing no tokens to version_tokens_set():

    mysql> SELECT version_tokens_set(NULL);
    +------------------------------+
    | version_tokens_set(NULL)     |
    +------------------------------+
    | Version tokens list cleared. |
    +------------------------------+
    mysql> SELECT version_tokens_set('');
    +------------------------------+
    | version_tokens_set('')       |
    +------------------------------+
    | Version tokens list cleared. |
    +------------------------------+
    
  • version_tokens_edit(token_list)

    Modifies the server's list of version tokens using the token_list argument and returns a binary string that indicates the outcome of the operation. token_list is a semicolon-separated list of name=value pairs specifying the name of each token to be defined and its value. If a token exists, its value is updated with the given value. If a token does not exist, it is created with the given value. If the argument is NULL or a string containing no tokens, the token list remains unchanged.

    mysql> SELECT version_tokens_set('tok1=value1;tok2=value2');
    +-----------------------------------------------+
    | version_tokens_set('tok1=value1;tok2=value2') |
    +-----------------------------------------------+
    | 2 version tokens set.                         |
    +-----------------------------------------------+
    mysql> SELECT version_tokens_edit('tok2=new_value2;tok3=new_value3');
    +--------------------------------------------------------+
    | version_tokens_edit('tok2=new_value2;tok3=new_value3') |
    +--------------------------------------------------------+
    | 2 version tokens updated.                              |
    +--------------------------------------------------------+
    
  • version_tokens_set(token_list)

    Replaces the server's list of version tokens with the tokens defined in the token_list argument and returns a binary string that indicates the outcome of the operation. token_list is a semicolon-separated list of name=value pairs specifying the name of each token to be defined and its value. If the argument is NULL or a string containing no tokens, the token list is cleared.

    mysql> SELECT version_tokens_set('tok1=value1;tok2=value2');
    +-----------------------------------------------+
    | version_tokens_set('tok1=value1;tok2=value2') |
    +-----------------------------------------------+
    | 2 version tokens set.                         |
    +-----------------------------------------------+
    
  • version_tokens_show()

    Returns the server's list of version tokens as a binary string containing a semicolon-separated list of name=value pairs.

    mysql> SELECT version_tokens_show();
    +--------------------------+
    | version_tokens_show()    |
    +--------------------------+
    | tok2=value2;tok1=value1; |
    +--------------------------+
    

The following UDFs permit version tokens to be locked and unlocked:

  • version_tokens_lock_exclusive(token_name[, token_name] ..., timeout)

    Acquires exclusive locks on one or more version tokens, specified by name as strings, timing out with an error if the locks are not acquired within the given timeout value.

    mysql> SELECT version_tokens_lock_exclusive('lock1', 'lock2', 10);
    +-----------------------------------------------------+
    | version_tokens_lock_exclusive('lock1', 'lock2', 10) |
    +-----------------------------------------------------+
    |                                                   1 |
    +-----------------------------------------------------+
    
  • version_tokens_lock_shared(token_name[, token_name] ..., timeout)

    Acquires shared locks on one or more version tokens, specified by name as strings, timing out with an error if the locks are not acquired within the given timeout value.

    mysql> SELECT version_tokens_lock_shared('lock1', 'lock2', 10);
    +--------------------------------------------------+
    | version_tokens_lock_shared('lock1', 'lock2', 10) |
    +--------------------------------------------------+
    |                                                1 |
    +--------------------------------------------------+
    
  • version_tokens_unlock()

    Releases all locks that were acquired within the current session using version_tokens_lock_exclusive() and version_tokens_lock_shared().

    mysql> SELECT version_tokens_unlock();
    +-------------------------+
    | version_tokens_unlock() |
    +-------------------------+
    |                       1 |
    +-------------------------+
    

The locking functions share these characteristics:

  • The return value is nonzero for success. Otherwise, an error occurs.

  • Token names are strings.

  • In contrast to argument handling for the UDFs that manipulate the server token list, whitespace surrounding token name arguments is not ignored and = and ; characters are permitted.

  • It is possible to lock nonexisting token names. This does not create the tokens.

  • Timeout values are nonnegative integers representing the time in seconds to wait to acquire locks before timing out with an error. If the timeout is 0, there is no waiting and the function produces an error if locks cannot be acquired immediately.

  • Version Tokens locking functions are based on the locking service described at Section 28.3.1, “The Locking Service”.

Version Tokens System Variables

Version Tokens supports the following system variables. These variables are unavailable unless the Version Tokens plugin is installed (see Section 5.6.4.2, “Installing or Uninstalling Version Tokens”).

System variables:

  • version_tokens_session

    命令行格式--version-tokens-session=value
    系统变量名称version_tokens_session
    变量作用域全局, 会话
    动态变量Yes
    允许的值类型字符
    默认NULL

    The session value of this variable specifies the client version token list and indicates the tokens that the client session requires the server version token list to have.

    If the version_tokens_session variable is NULL (the default) or has an empty value, any server version token list matches. (In effect, an empty value disables matching requirements.)

    If the version_tokens_session variable has a nonempty value, any mismatch between its value and the server version token list results in an error for any statement the session sends to the server. A mismatch occurs under these conditions:

    It is not a mismatch for the server version token list to include a token not named in the version_tokens_session value.

    Suppose that a management application has set the server token list as follows:

    mysql> SELECT version_tokens_set('tok1=a;tok2=b;tok3=c');
    +--------------------------------------------+
    | version_tokens_set('tok1=a;tok2=b;tok3=c') |
    +--------------------------------------------+
    | 3 version tokens set.                      |
    +--------------------------------------------+
    

    A client registers the tokens it requires the server to match by setting its version_tokens_session value. Then, for each subsequent statement sent by the client, the server checks its token list against the client version_tokens_session value and produces an error if there is a mismatch:

    mysql> SET @@session.version_tokens_session = 'tok1=a;tok2=b';
    mysql> SELECT 1;
    +---+
    | 1 |
    +---+
    | 1 |
    +---+
    
    mysql> SET @@session.version_tokens_session = 'tok1=b';
    mysql> SELECT 1;
    ERROR 3136 (42000): Version token mismatch for tok1. Correct value a
    

    The first SELECT succeeds because the client tokens tok1 and tok2 are present in the server token list and each token has the same value in the server list. The second SELECT fails because, although tok1 is present in the server token list, it has a different value than specified by the client.

    At this point, any statement sent by the client fails, unless the server token list changes such that it matches again. Suppose that the management application changes the server token list as follows:

    mysql> SELECT version_tokens_edit('tok1=b');
    +-------------------------------+
    | version_tokens_edit('tok1=b') |
    +-------------------------------+
    | 1 version tokens updated.     |
    +-------------------------------+
    mysql> SELECT version_tokens_show();
    +-----------------------+
    | version_tokens_show() |
    +-----------------------+
    | tok3=c;tok1=b;tok2=b; |
    +-----------------------+
    

    Now the client version_tokens_session value matches the server token list and the client can once again successfully execute statements:

    mysql> SELECT 1;
    +---+
    | 1 |
    +---+
    | 1 |
    +---+
    
  • version_tokens_session_number

    命令行格式--version-tokens-session-number=N
    系统变量名称version_tokens_session_number
    变量作用域全局, 会话
    动态变量
    允许的值类型整数
    默认0

    This variable is for internal use.

5.7 Running Multiple MySQL Instances on One Machine

In some cases, you might want to run multiple instances of MySQL on a single machine. You might want to test a new MySQL release while leaving an existing production setup undisturbed. Or you might want to give different users access to different mysqld servers that they manage themselves. (For example, you might be an Internet Service Provider that wants to provide independent MySQL installations for different customers.)

It is possible to use a different MySQL server binary per instance, or use the same binary for multiple instances, or any combination of the two approaches. For example, you might run a server from MySQL 5.7 and one from MySQL 8.0, to see how different versions handle a given workload. Or you might run multiple instances of the current production version, each managing a different set of databases.

Whether or not you use distinct server binaries, each instance that you run must be configured with unique values for several operating parameters. This eliminates the potential for conflict between instances. Parameters can be set on the command line, in option files, or by setting environment variables. See Section 4.2.3, “Specifying Program Options”. To see the values used by a given instance, connect to it and execute a SHOW VARIABLES statement.

The primary resource managed by a MySQL instance is the data directory. Each instance should use a different data directory, the location of which is specified using the --datadir=dir_name option. For methods of configuring each instance with its own data directory, and warnings about the dangers of failing to do so, see Section 5.7.1, “Setting Up Multiple Data Directories”.

In addition to using different data directories, several other options must have different values for each server instance:

  • --port=port_num

    --port controls the port number for TCP/IP connections. Alternatively, if the host has multiple network addresses, you can use --bind-address to cause each server to listen to a different address.

  • --socket={file_name|pipe_name}

    --socket controls the Unix socket file path on Unix or the named pipe name on Windows. On Windows, it is necessary to specify distinct pipe names only for those servers configured to permit named-pipe connections.

  • --shared-memory-base-name=name

    This option is used only on Windows. It designates the shared-memory name used by a Windows server to permit clients to connect using shared memory. It is necessary to specify distinct shared-memory names only for those servers configured to permit shared-memory connections.

  • --pid-file=file_name

    This option indicates the path name of the file in which the server writes its process ID.

If you use the following log file options, their values must differ for each server:

For further discussion of log file options, see Section 5.4, “MySQL Server Logs”.

To achieve better performance, you can specify the following option differently for each server, to spread the load between several physical disks:

Having different temporary directories also makes it easier to determine which MySQL server created any given temporary file.

If you have multiple MySQL installations in different locations, you can specify the base directory for each installation with the --basedir=dir_name option. This causes each instance to automatically use a different data directory, log files, and PID file because the default for each of those parameters is relative to the base directory. In that case, the only other options you need to specify are the --socket and --port options. Suppose that you install different versions of MySQL using tar file binary distributions. These install in different locations, so you can start the server for each installation using the command bin/mysqld_safe under its corresponding base directory. mysqld_safe determines the proper --basedir option to pass to mysqld, and you need specify only the --socket and --port options to mysqld_safe.

As discussed in the following sections, it is possible to start additional servers by specifying appropriate command options or by setting environment variables. However, if you need to run multiple servers on a more permanent basis, it is more convenient to use option files to specify for each server those option values that must be unique to it. The --defaults-file option is useful for this purpose.

5.7.1 Setting Up Multiple Data Directories

Each MySQL Instance on a machine should have its own data directory. The location is specified using the --datadir=dir_name option.

There are different methods of setting up a data directory for a new instance:

  • Create a new data directory.

  • Copy an existing data directory.

The following discussion provides more detail about each method.

Warning

Normally, you should never have two servers that update data in the same databases. This may lead to unpleasant surprises if your operating system does not support fault-free system locking. If (despite this warning) you run multiple servers using the same data directory and they have logging enabled, you must use the appropriate options to specify log file names that are unique to each server. Otherwise, the servers try to log to the same files.

Even when the preceding precautions are observed, this kind of setup works only with MyISAM and MERGE tables, and not with any of the other storage engines. Also, this warning against sharing a data directory among servers always applies in an NFS environment. Permitting multiple MySQL servers to access a common data directory over NFS is a very bad idea. The primary problem is that NFS is the speed bottleneck. It is not meant for such use. Another risk with NFS is that you must devise a way to ensure that two or more servers do not interfere with each other. Usually NFS file locking is handled by the lockd daemon, but at the moment there is no platform that performs locking 100% reliably in every situation.

Create a New Data Directory

With this method, the data directory will be in the same state as when you first install MySQL. It will have the default set of MySQL accounts and no user data.

On Unix, initialize the data directory. See Section 2.9, “Postinstallation Setup and Testing”.

On Windows, the data directory is included in the MySQL distribution:

  • MySQL Zip archive distributions for Windows contain an unmodified data directory. You can unpack such a distribution into a temporary location, then copy it data directory to where you are setting up the new instance.

  • Windows MSI package installers create and set up the data directory that the installed server will use, but also create a pristine template data directory named data under the installation directory. After an installation has been performed using an MSI package, the template data directory can be copied to set up additional MySQL instances.

Copy an Existing Data Directory

With this method, any MySQL accounts or user data present in the data directory are carried over to the new data directory.

  1. Stop the existing MySQL instance using the data directory. This must be a clean shutdown so that the instance flushes any pending changes to disk.

  2. Copy the data directory to the location where the new data directory should be.

  3. Copy the my.cnf or my.ini option file used by the existing instance. This serves as a basis for the new instance.

  4. Modify the new option file so that any pathnames referring to the original data directory refer to the new data directory. Also, modify any other options that must be unique per instance, such as the TCP/IP port number and the log files. For a list of parameters that must be unique per instance, see Section 5.7, “Running Multiple MySQL Instances on One Machine”.

  5. Start the new instance, telling it to use the new option file.

5.7.2 Running Multiple MySQL Instances on Windows

You can run multiple servers on Windows by starting them manually from the command line, each with appropriate operating parameters, or by installing several servers as Windows services and running them that way. General instructions for running MySQL from the command line or as a service are given in Section 2.3, “Installing MySQL on Microsoft Windows”. The following sections describe how to start each server with different values for those options that must be unique per server, such as the data directory. These options are listed in Section 5.7, “Running Multiple MySQL Instances on One Machine”.

5.7.2.1 Starting Multiple MySQL Instances at the Windows Command Line

The procedure for starting a single MySQL server manually from the command line is described in Section 2.3.5.6, “Starting MySQL from the Windows Command Line”. To start multiple servers this way, you can specify the appropriate options on the command line or in an option file. It is more convenient to place the options in an option file, but it is necessary to make sure that each server gets its own set of options. To do this, create an option file for each server and tell the server the file name with a --defaults-file option when you run it.

Suppose that you want to run one instance of mysqld on port 3307 with a data directory of C:\mydata1, and another instance on port 3308 with a data directory of C:\mydata2. Use this procedure:

  1. Make sure that each data directory exists, including its own copy of the mysql database that contains the grant tables.

  2. Create two option files. For example, create one file named C:\my-opts1.cnf that looks like this:

    [mysqld]
    datadir = C:/mydata1
    port = 3307
    

    Create a second file named C:\my-opts2.cnf that looks like this:

    [mysqld]
    datadir = C:/mydata2
    port = 3308
    
  3. Use the --defaults-file option to start each server with its own option file:

    C:\> C:\mysql\bin\mysqld --defaults-file=C:\my-opts1.cnf
    C:\> C:\mysql\bin\mysqld --defaults-file=C:\my-opts2.cnf
    

    Each server starts in the foreground (no new prompt appears until the server exits later), so you will need to issue those two commands in separate console windows.

To shut down the servers, connect to each using the appropriate port number:

C:\> C:\mysql\bin\mysqladmin --port=3307 --host=127.0.0.1 --user=root --password shutdown
C:\> C:\mysql\bin\mysqladmin --port=3308 --host=127.0.0.1 --user=root --password shutdown

Servers configured as just described permit clients to connect over TCP/IP. If your version of Windows supports named pipes and you also want to permit named-pipe connections, specify options that enable the named pipe and specify its name. Each server that supports named-pipe connections must use a unique pipe name. For example, the C:\my-opts1.cnf file might be written like this:

[mysqld]
datadir = C:/mydata1
port = 3307
enable-named-pipe
socket = mypipe1

Modify C:\my-opts2.cnf similarly for use by the second server. Then start the servers as described previously.

A similar procedure applies for servers that you want to permit shared-memory connections. Enable such connections with the --shared-memory option and specify a unique shared-memory name for each server with the --shared-memory-base-name option.

5.7.2.2 Starting Multiple MySQL Instances as Windows Services

On Windows, a MySQL server can run as a Windows service. The procedures for installing, controlling, and removing a single MySQL service are described in Section 2.3.5.8, “Starting MySQL as a Windows Service”.

To set up multiple MySQL services, you must make sure that each instance uses a different service name in addition to the other parameters that must be unique per instance.

For the following instructions, suppose that you want to run the mysqld server from two different versions of MySQL that are installed at C:\mysql-5.5.9 and C:\mysql-8.0.4, respectively. (This might be the case if you are running 5.5.9 as your production server, but also want to conduct tests using 8.0.4.)

To install MySQL as a Windows service, use the --install or --install-manual option. For information about these options, see Section 2.3.5.8, “Starting MySQL as a Windows Service”.

Based on the preceding information, you have several ways to set up multiple services. The following instructions describe some examples. Before trying any of them, shut down and remove any existing MySQL services.

  • Approach 1: Specify the options for all services in one of the standard option files. To do this, use a different service name for each server. Suppose that you want to run the 5.5.9 mysqld using the service name of mysqld1 and the 8.0.4 mysqld using the service name mysqld2. In this case, you can use the [mysqld1] group for 5.5.9 and the [mysqld2] group for 8.0.4. For example, you can set up C:\my.cnf like this:

    # options for mysqld1 service
    [mysqld1]
    basedir = C:/mysql-5.5.9
    port = 3307
    enable-named-pipe
    socket = mypipe1
    
    # options for mysqld2 service
    [mysqld2]
    basedir = C:/mysql-8.0.4
    port = 3308
    enable-named-pipe
    socket = mypipe2
    

    Install the services as follows, using the full server path names to ensure that Windows registers the correct executable program for each service:

    C:\> C:\mysql-5.5.9\bin\mysqld --install mysqld1
    C:\> C:\mysql-8.0.4\bin\mysqld --install mysqld2
    

    To start the services, use the services manager, or use NET START with the appropriate service names:

    C:\> NET START mysqld1
    C:\> NET START mysqld2
    

    To stop the services, use the services manager, or use NET STOP with the appropriate service names:

    C:\> NET STOP mysqld1
    C:\> NET STOP mysqld2
    
  • Approach 2: Specify options for each server in separate files and use --defaults-file when you install the services to tell each server what file to use. In this case, each file should list options using a [mysqld] group.

    With this approach, to specify options for the 5.5.9 mysqld, create a file C:\my-opts1.cnf that looks like this:

    [mysqld]
    basedir = C:/mysql-5.5.9
    port = 3307
    enable-named-pipe
    socket = mypipe1
    

    For the 8.0.4 mysqld, create a file C:\my-opts2.cnf that looks like this:

    [mysqld]
    basedir = C:/mysql-8.0.4
    port = 3308
    enable-named-pipe
    socket = mypipe2
    

    Install the services as follows (enter each command on a single line):

    C:\> C:\mysql-5.5.9\bin\mysqld --install mysqld1
               --defaults-file=C:\my-opts1.cnf
    C:\> C:\mysql-8.0.4\bin\mysqld --install mysqld2
               --defaults-file=C:\my-opts2.cnf
    

    When you install a MySQL server as a service and use a --defaults-file option, the service name must precede the option.

    After installing the services, start and stop them the same way as in the preceding example.

To remove multiple services, use mysqld --remove for each one, specifying a service name following the --remove option. If the service name is the default (MySQL), you can omit it.

5.7.3 Running Multiple MySQL Instances on Unix

Note

The discussion here uses mysqld_safe to launch multiple instances of MySQL. For MySQL installation using an RPM distribution, server startup and shutdown is managed by systemd on several Linux platforms. On these platforms, mysqld_safe is not installed because it is unnecessary. For information about using systemd to handle multiple MySQL instances, see Section 2.5.9, “Managing MySQL Server with systemd”.

One way is to run multiple MySQL instances on Unix is to compile different servers with different default TCP/IP ports and Unix socket files so that each one listens on different network interfaces. Compiling in different base directories for each installation also results automatically in a separate, compiled-in data directory, log file, and PID file location for each server.

Assume that an existing 5.7 server is configured for the default TCP/IP port number (3306) and Unix socket file (/tmp/mysql.sock). To configure a new 8.0.4 server to have different operating parameters, use a CMake command something like this:

shell> cmake . -DMYSQL_TCP_PORT=port_number \
             -DMYSQL_UNIX_ADDR=file_name \
             -DCMAKE_INSTALL_PREFIX=/usr/local/mysql-8.0.4

Here, port_number and file_name must be different from the default TCP/IP port number and Unix socket file path name, and the CMAKE_INSTALL_PREFIX value should specify an installation directory different from the one under which the existing MySQL installation is located.

If you have a MySQL server listening on a given port number, you can use the following command to find out what operating parameters it is using for several important configurable variables, including the base directory and Unix socket file name:

shell> mysqladmin --host=host_name --port=port_number variables

With the information displayed by that command, you can tell what option values not to use when configuring an additional server.

If you specify localhost as the host name, mysqladmin defaults to using a Unix socket file connection rather than TCP/IP. To explicitly specify the connection protocol, use the --protocol={TCP|SOCKET|PIPE|MEMORY} option.

You need not compile a new MySQL server just to start with a different Unix socket file and TCP/IP port number. It is also possible to use the same server binary and start each invocation of it with different parameter values at runtime. One way to do so is by using command-line options:

shell> mysqld_safe --socket=file_name --port=port_number

To start a second server, provide different --socket and --port option values, and pass a --datadir=dir_name option to mysqld_safe so that the server uses a different data directory.

Alternatively, put the options for each server in a different option file, then start each server using a --defaults-file option that specifies the path to the appropriate option file. For example, if the option files for two server instances are named /usr/local/mysql/my.cnf and /usr/local/mysql/my.cnf2, start the servers like this: command:

shell> mysqld_safe --defaults-file=/usr/local/mysql/my.cnf
shell> mysqld_safe --defaults-file=/usr/local/mysql/my.cnf2

Another way to achieve a similar effect is to use environment variables to set the Unix socket file name and TCP/IP port number:

shell> MYSQL_UNIX_PORT=/tmp/mysqld-new.sock
shell> MYSQL_TCP_PORT=3307
shell> export MYSQL_UNIX_PORT MYSQL_TCP_PORT
shell> bin/mysqld --initialize --user=mysql
shell> mysqld_safe --datadir=/path/to/datadir &

This is a quick way of starting a second server to use for testing. The nice thing about this method is that the environment variable settings apply to any client programs that you invoke from the same shell. Thus, connections for those clients are automatically directed to the second server.

Section 4.9, “MySQL Program Environment Variables”, includes a list of other environment variables you can use to affect MySQL programs.

On Unix, the mysqld_multi script provides another way to start multiple servers. See Section 4.3.4, “mysqld_multi — Manage Multiple MySQL Servers”.

5.7.4 Using Client Programs in a Multiple-Server Environment

To connect with a client program to a MySQL server that is listening to different network interfaces from those compiled into your client, you can use one of the following methods:

  • Start the client with --host=host_name --port=port_number to connect using TCP/IP to a remote server, with --host=127.0.0.1 --port=port_number to connect using TCP/IP to a local server, or with --host=localhost --socket=file_name to connect to a local server using a Unix socket file or a Windows named pipe.

  • Start the client with --protocol=TCP to connect using TCP/IP, --protocol=SOCKET to connect using a Unix socket file, --protocol=PIPE to connect using a named pipe, or --protocol=MEMORY to connect using shared memory. For TCP/IP connections, you may also need to specify --host and --port options. For the other types of connections, you may need to specify a --socket option to specify a Unix socket file or Windows named-pipe name, or a --shared-memory-base-name option to specify the shared-memory name. Shared-memory connections are supported only on Windows.

  • On Unix, set the MYSQL_UNIX_PORT and MYSQL_TCP_PORT environment variables to point to the Unix socket file and TCP/IP port number before you start your clients. If you normally use a specific socket file or port number, you can place commands to set these environment variables in your .login file so that they apply each time you log in. See Section 4.9, “MySQL Program Environment Variables”.

  • Specify the default Unix socket file and TCP/IP port number in the [client] group of an option file. For example, you can use C:\my.cnf on Windows, or the .my.cnf file in your home directory on Unix. See Section 4.2.6, “Using Option Files”.

  • In a C program, you can specify the socket file or port number arguments in the mysql_real_connect() call. You can also have the program read option files by calling mysql_options(). See Section 27.7.7, “C API Function Descriptions”.

  • If you are using the Perl DBD::mysql module, you can read options from MySQL option files. For example:

    $dsn = "DBI:mysql:test;mysql_read_default_group=client;"
            . "mysql_read_default_file=/usr/local/mysql/data/my.cnf";
    $dbh = DBI->connect($dsn, $user, $password);
    

    See Section 27.9, “MySQL Perl API”.

    Other programming interfaces may provide similar capabilities for reading option files.