今天在线运行的一个mysql崩溃了。
查看错误日志，如下：
-----------------------------------------
161108 11:36:45 mysqld_safe Starting mysqld daemon with databases from /usr/local/mysql/var
2016-11-08 11:36:46 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2016-11-08 11:36:46 5497 [Note] Plugin 'FEDERATED' is disabled.
2016-11-08 11:36:46 7f11c48e1720 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
2016-11-08 11:36:46 5497 [Note] InnoDB: Using atomics to ref count buffer pool pages
2016-11-08 11:36:46 5497 [Note] InnoDB: The InnoDB memory heap is disabled
2016-11-08 11:36:46 5497 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2016-11-08 11:36:46 5497 [Note] InnoDB: Memory barrier is not used
2016-11-08 11:36:46 5497 [Note] InnoDB: Compressed tables use zlib 1.2.3
2016-11-08 11:36:46 5497 [Note] InnoDB: Using CPU crc32 instructions
2016-11-08 11:36:46 5497 [Note] InnoDB: Initializing buffer pool, size = 16.0M
2016-11-08 11:36:46 5497 [Note] InnoDB: Completed initialization of buffer pool
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 5.
InnoDB: You may have to recover from a backup.
2016-11-08 11:36:46 7f11c48e1720 InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex 7478d078000000050000000000000000000000000f271f4d000700000000000000000000000000000000001b4000000000000000000200f20000000000000006000000000000002d000000000000002e000000000000002f0000000000000030000000000（省略很多类似代码）
InnoDB: End of page dump
2016-11-08 11:36:46 7f11c48e1720 InnoDB: uncompressed page, stored checksum in field1 1954074744, calculated checksums for field1: crc32 993334256, innodb 2046145943, none 3735928559, stored checksum in field2 1139795846, calculated checksums for field2: crc32 993334256, innodb 1606613742, none 3735928559, page LSN 0 254222157, low 4 bytes of LSN at page end 254221236, page number (if stored to page already) 5, space id (if created with >= MySQL-4.1.1 and stored already) 0
InnoDB: Page may be a transaction system page
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 5.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
InnoDB: Ending processing because of a corrupt database page.
2016-11-08 11:36:46 7f11c48e1720 InnoDB: Assertion failure in thread 139714288817952 in file buf0buf.cc line 4201
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
03:36:46 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=1000
thread_count=0
connection_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 798063 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
/usr/local/mysql/bin/mysqld(my_print_stacktrace+0x35)[0x8e64b5]
/usr/local/mysql/bin/mysqld(handle_fatal_signal+0x41b)[0x652fbb]
/lib64/libpthread.so.0(+0xf7e0)[0x7f11c44c77e0]
/lib64/libc.so.6(gsignal+0x35)[0x7f11c315d625]
/lib64/libc.so.6(abort+0x175)[0x7f11c315ee05]
/usr/local/mysql/bin/mysqld[0xa585c5]
/usr/local/mysql/bin/mysqld[0xa6c7b4]
/usr/local/mysql/bin/mysqld[0xa6cbc7]
/usr/local/mysql/bin/mysqld[0xa5bce2]
/usr/local/mysql/bin/mysqld[0xa1e2ba]
/usr/local/mysql/bin/mysqld[0xa0bf60]
/usr/local/mysql/bin/mysqld[0x95a427]
/usr/local/mysql/bin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x48)[0x58f788]
/usr/local/mysql/bin/mysqld[0x6e4a36]
/usr/local/mysql/bin/mysqld(_Z11plugin_initPiPPci+0xb3e)[0x6e826e]
/usr/local/mysql/bin/mysqld[0x582d85]
/usr/local/mysql/bin/mysqld(_Z11mysqld_mainiPPc+0x4d8)[0x587d18]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7f11c3149d5d]
/usr/local/mysql/bin/mysqld[0x57a019]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
161108 11:36:46 mysqld_safe mysqld from pid file /usr/local/mysql/var/VM_241_49_centos.pid ended
------------------------------------------------------------------------------
从日志中可以看出是innodb引擎出了问题。日志里提示到 http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html 查看强制恢复的方法。在mysql的配置文件my.cnf里找到 [mysqld]字段下，添加 innodb_force_recovery=1：
[mysqld] innodb_force_recovery = 1
然后重启mysql，重启成功。然后使用mysqldump或 pma 导出数据，执行修复操作等。修复完成后，把该参数注释掉，还原默认值0。
配置文件的参数：innodb_force_recovery
innodb_force_recovery影响整个InnoDB存储引擎的恢复状况。默认为0，表示当需要恢复时执行所有的恢复操作（即校验数据页/purge undo/insert buffer merge/rolling back&forward），当不能进行有效的恢复操作时，mysql有可能无法启动，并记录错误日志； innodb_force_recovery可以设置为1-6,大的数字包含前面所有数字的影响。当设置参数值大于0后，可以对表进行select,create,drop操作,但insert,update或者delete这类操作是不允许的。 1(SRV_FORCE_IGNORE_CORRUPT):忽略检查到的corrupt页。 2(SRV_FORCE_NO_BACKGROUND):阻止主线程的运行，如主线程需要执行full purge操作，会导致crash。 3(SRV_FORCE_NO_TRX_UNDO):不执行事务回滚操作。 4(SRV_FORCE_NO_IBUF_MERGE):不执行插入缓冲的合并操作。 5(SRV_FORCE_NO_UNDO_LOG_SCAN):不查看重做日志，InnoDB存储引擎会将未提交的事务视为已提交。 6(SRV_FORCE_NO_LOG_REDO):不执行前滚的操作。
一般修复方法参考
第一种方法：
建立一张新表： create table email_bak #和原表结构一样，只是把INNODB改成了MYISAM。 把数据导进去 insert into email_bak select * from email; 删除掉原表： drop table email; 注释掉 innodb_force_recovery 之后，重启。 重命名： rename table email_bak to email; 最后改回存储引擎： alter table email engine = innodb
另一个方法是使用mysqldump将表格导出，然后再导回到InnoDB表中。这两种方法的结果是相同的。
备份导出(包括结构和数据)： mysqldump -uroot -p123 test > test.sql 还原方法1： use test; source test.sql 还原方法2（系统命令行）： mysql -uroot -p123 test < test.sql;
注意，我应该提醒的是CHECK TABLE命令在InnoDB数据库中基本上是没有用的。
[参考]：
1、恢复损坏的InnoDB表格 http://cgaolei.iteye.com/blog/412157
2、MySQL数据库INNODB表损坏修复处理过程分享 http://www.jb51.net/article/40289.htm