Fix innodb doublewrite test to not corrupt the first page of a table.

Summary:
innodb_corrupt_doublewrite_pages.py intentionally corrupts pages
on the .ibd files to make sure that the recovery works even when
there may be torn pages. There is a bug in innodb where doublewrite
buffer is not used properly during recovery if the first page of the
table is torn. (bugs.mysql.com/70087). In order to work around this
bug the python script has to corrupt a page other than the first.

Test Plan: Run on dev server. Read code.

Reviewers: steaphan

Reviewed By: steaphan
Suppress syscall close() in mtr tests

Summary:
Add syscall close() to the global suppression test.
The syscall close() errors are from a client process which have
nothing to do with the server. This mostly is related to the bug
http://bugs.mysql.com/bug.php?id=50414. Adding this to
global suppression list makes sense to avoid unnecessary test
failures.

Test Plan: arc unit --everything

Reviewers: steaphan

Reviewed By: steaphan
WITH_PERFSCHEMA_STORAGE_ENGINE=0 is Dumb

Summary:
In the disabling of PerfSchema, I changed the cmakedefine of
WITH_PERFSCHEMA_STORAGE_ENGINE from 1 to 0.  This define is only
used with #ifdef... so that was pointless.

It still works fine either way, but I am undoing this change here.

Test Plan:
Instrumented the code and cmake configs to confirm PS is still
completely disabled by default, and completely enabled when
cmake is passed -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 - all works.

Reviewers: mcallaghan

Reviewed By: mcallaghan
fix build breaking with perf schema

Summary:
Crash safe slave broke building with PerfSchema.

Rpl_info class need extra options if building with performace schema.
Gtid_info which is derived class of Rpl_info but is not using the extra
options needed for mutex instrumentation. we don't need this for Gtid
info now.

Test Plan:
mysqlbuild.sh with -DWITH_PERFSCHEMA_STORAGE_ENGINE=1 change

Reviewers: steaphan

Reviewed By: steaphan
Fix perfschema build

Summary:
Disabled a perfschema unittest binary. Looks like it was doing special things
to link with libraries elsewhere. I don't think we need to spend time doing
even more special things to get jemalloc linked.

Test Plan:
build with -DWITH_PERFSCHEMA_STORAGE_ENGINE=1

Reviewers: steaphan

Reviewed By: steaphan
Add public phabricator config.

Summary: This will let us handle reviews with external groups directly.

Test Plan: This diff was reviewed via public phabricator.  :)

Reviewers: jtolmer

Reviewed By: jtolmer
Do not run FLUSH TABLE WITH READ LOCK + SHOW SLAVE STATUS

Summary:
When copying from a slave, innobackupex issues a flush table with read
lock. Then it calls show slave status to get binlog file and position.
Since that call causes a deadlock, get the info directly from
slave_relay_log_info table.

Test Plan:
Ran a couple of copies from slaves.
mysql-test-run

Reviewers: santoshb, yoshinori

Reviewed By: santoshb
use rli->is_fatal_error instead of thd->is_fatal_error

Summary:
While sql_thread is executing transactions, sometimes
thd->is_fatal_error may get set even though the error is
actually lock wait timeout. Avoid this behaviour by using
ignoring thd->is_fatal_error in slave transaction
retry logic. The fatal_error flag is getting set in
  function mysql_update() {
  ......
  if (table->file->is_fatal_error(error, HA_CHECK_DUP_KEY))
    flags|= ME_FATALERROR; /* Other handler errors are fatal */
  .....
  }

  virtual bool is_fatal_error(int error, uint flags)
  {
    if (!handler::is_fatal_error(error, flags) ||
        error == HA_ERR_NO_PARTITION_FOUND ||
        error == HA_ERR_NOT_IN_LOCK_PARTITIONS)
      return FALSE;
    return TRUE;
  }

  virtual bool is_fatal_error(int error, uint flags)
  {
    if (!error ||
       ((flags & HA_CHECK_DUP_KEY) &&
       (error == HA_ERR_FOUND_DUPP_KEY ||
       error == HA_ERR_FOUND_DUPP_UNIQUE)))
       return FALSE;
    return TRUE;
  }

The thd->is_fatal_error is not checked in mysql 5.1 which was working fine.
This diff uses a new flag rli->is_fatal_error instead of global
thd->is_fatal_error to set correct sql_thread error.

Link for public bug report http://bugs.mysql.com/bug.php?id=70318

Test Plan: Tested the rpm on a admarketDb machine

Reviewers: steaphan, yoshinori

Reviewed By: steaphan
Remove FLUSH TABLES WITH READ LOCK call from innobackupex

Summary:
Now that we have transactionally stored data in slave_relay_log_info
table, we no longer need to acquire the read lock. Doing so can cause
deadlocks.

Test Plan:
mysql-test-run will check basic innobackupex functionality.
Manually checked that a newly created replica contains needed info in
slave_relay_log_info table.

Reviewers: steaphan, santoshb

Reviewed By: steaphan
locking forms of flush tables are blocked

Summary:
This diff adds a mechanism to block running
"FLUSH TABLES WITH READ LOCK" and "FLUSH TABLES .. FOR EXPORT".
The defalt value of the config option block_ftwrl is FALSE.

Test Plan: mtr tests

Reviewers: steaphan

Reviewed By: steaphan
use oracle's fix for slave sql_thread crash

Summary:
Oracle's patch for http://bugs.mysql.com/bug.php?id=69369
is ported in this diff.

Test Plan: mtr --parallel=16

Reviewers: jtolmer, steaphan

Reviewed By: jtolmer
flush result file after my_fwrite()

Summary:
mysqlbinlog writes to the file using my_fwrite() which
calls ftell() and then calls fwrite(). ftell() causes the output
buffer to get written to the file. But after doing fwrite() the
recent event is not flushed to the file until another event is received
by mysqlbinlog or when mysqlbinlog process is terminated gracefully.

This causes bug mentioned in http://bugs.mysql.com/bug.php?id=69568.
Fix this by calling fflush() after my_fwrite()in raw_mode when the current
event is XID_EVENT or when mysqlbinlog haven't flushed the result file
even after receving --flush-result-file events from the
server.

Test Plan:
run mysqlbinlog --raw and check the most recent xid event
is actually written to the result file.
run mysqlbinlog --flush-result-file=3 to check the log is flushed
after receving 3 events (none of them are xid events) from the server

Reviewers: steaphan, jtolmer, yoshinori

Reviewed By: steaphan
query to get gtid_executed upto given binlog coordiantes

Summary:
This diff adds a new command
  SHOW GTID_EXECUTED IN 'log_file_name' FROM 'log_file_pos'

This outputs the gtid_executed executed by the server up until
log_file_pos offset in binlog file log_file_name. This is used
in nrx2.php to set gtid_purged on the newly created slave.

Test Plan:
mtr_test to test the new query. Also ran on
a shadow to test gtid_executed from the new command is
same as gtid_executed from show master status.

Reviewers: rudradevbasak, jtolmer

Reviewed By: rudradevbasak
Use read_only instead of gtid_deployment_step

Summary:
Slaves should not generate GTIDs on it's own in the
slave binlog. This can break replication with GTID protocol
when the slave which generated it's own GTIDs promoted as the master.
All slaves in the replication tier can fail with SQL_thread error
1236 after promotion with message "The slave is connecting using CHANGE MASTER
TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs
containing GTIDs that the slave requires".

To avoid this slaves should not generate it's own GTIDs but should
use GTIDs received from master. This requires turning on
gtid_deployment_step on slave always. This functionality is provided by
read_only which is 1 on slave and 0 on master. So, changing the logic to
just use read_only is better than changing the static variable
gtid_deployment_step.

Test Plan: test case id modifed to use read_only.

Reviewers: jtolmer

Reviewed By: jtolmer
Fix semi-sync master assertions

Summary:
semi-sync master asserts in the following scenarios

1. setup master and slave with semi-sync

2. start slave; // on slave

3. SET global rpl_semi_sync_master_wait_no_slave = OFF; // on master

4. SET global rpl_semi_sync_master_enabled = OFF; // on master

5. stop slave; // on slave

Mysqld on master gets assertion failure

mysqld: mysql-5.6.13/plugin/semisync/semisync_master.cc:812:
int ReplSemiSyncMaster::switch_off(): Assertion `active_tranxs_ != __null' failed.
02:58:16 UTC - mysqld got signal 6 ;

Note that step 3 ensures that when a slave is stopped in step 5, master
executes switchOff() causing the assert. active_tranxs_ is deleted inside
disableMaster() when step 4 is executed.

Fix this by checking active_tranxs_ for NULL before clear_active_tranx_nodes(). The
public bug report is available at http://bugs.mysql.com/bug.php?id=70349

Fix the bug reported in http://bugs.mysql.com/bug.php?id=70327. Port the
fix for this bug from google-mysql git repository, including these patches:
* d33253210f1abaec6a0e
* 730b154d3e0257578c03

Modification to the patch include not taking LOCK_log when commit_file_name_
and commit_file_pos_ are set in after_flush hook. Since commit_file_name_
and commit_file_pos_ are set while holding LOCK_log and dump thread is not
allowed to read anything after commit_file_name_ and commit_pos_, it is
not possible for any slave to receive events after commit_file_name_ and commit_pos_.
So any slave reply packet having coordinates greater than commit_file_name_ and
commit_pos_ can be safely assumed as corrupted.

Test Plan:
Follow the above steps and see that master is not asserting. Yoshi
tested the patch doesn't assert anywhere for the second bug.

arc unit --everything

Reviewers: jtolmer, yoshinori, rudradevbasak

Reviewed By: jtolmer, rudradevbasak
Block uninstalling semi-sync plugins

Summary:
Block uninstalling semi-sync plugins.
A new cnf variable block_uninstall_semisync is added in
this diff. The default value of block_uninstall_semisync
is FALSE.

Test Plan: mtr test

Reviewers: rudradevbasak, steaphan

Reviewed By: steaphan
Limit async IO slots per array to 256 as done by upstream.

Summary:
In 5.6 there are a few configuration variables that work together
in interesting ways:
* innodb_io_capacity - max number of dirty page writes per second to do
                      (also limits other IO operations)
* innodb_write_io_threads - number of background write IO threads

This undoes a previous change. The previous change has a bad side effect
and I am not sure it is needed in 5.6. The bad side effect is that Linux
AIO initialization might fail because the number of slots in a
background IO array is much larger than upstream for the same my.cnf
values.

The change was probably useful in 5.1, but the code has change enough
now that I doubt it is needed.

In upstream MySQL each background IO thread has 256 slots so the total
number of slots is 256 * innodb_write_io_threads. When the slots are
full then a background write request blocks. Given the hashing of
requests to a thread then it is likely that one thread's request segment
will get full and block before others.

In the FB patch we used max(256, innodb_io_capacity) as the number of
slots per array. So with our change the total number of slots is much
larger than upstream. This is a problem when innodb_io_capacity is large
and even when innodb_io_capacity X innodb_write_io_threads is large.
That can make the Linux AIO init code fail and disable AIO.

With flash I think we want innodb_io_capacity to be much greater than
the number of slots that Linux AIO supports.  There is a dedicated
thread does page write requests, so it isn't horrible to make it block
when:
 256 * innodb_write_io_threads << innodb_capacity
In 5.1 that work was done by srv_master_thread which had to do many
other things so blocking was much more of a problem.

With this change, to get more background IO slots increase
innodb_write_io_threads.

Test Plan:
mtr, linkbench

Reviewers: steaphan

Reviewed By: steaphan