Fix some indentation errors.

Summary:
This diff fixes some indentation error found by our linter.

Test Plan:
'git show -w' shows no changes.

Reviewers: steaphan

Reviewed By: steaphan
Fix the bugs introduced by port of "Improve page_zip_validate()"

Summary:

This ported some code to improve the functionality of page_zip_validate(),
but similar improvements were already made to page_zip_validate() in 5.6,
and some of these conflicted.  This did not merge them properly.

Another bug that existed prior to this was that a heap object was not
always freed after being used in page_zip_validate_low().
This diff fixes this by always freeing the heap object.

Test Plan:
Build with UNIV_ZIP_DEBUG enabled and run innodb_stress.

Reviewers: rahulgulati

Reviewed By: rahulgulati
Remove some unused user stats.

Summary:
This diff removes these three unused stats:
	queries_running
	queries_waiting
	tx_slots_inuse

These were originally part of admission control, and transaction control,
both of which we did not port.

Test Plan:
Jenkins valgrind tests discovered these were not even initialized.
They will now confirm these are cleanly removed.

Reviewers: jtolmer

Reviewed By: jtolmer
Fixes for latency histograms

Summary:
This includes two small fixes for the previous latency histogram commits.

A small fix which prevents writing of a histogram value outside the
maximum bucket range.  This involves subtracting a 1 in the formula for
searching a value in the histogram.  This was causing assertion failures
for me when I was adding Histograms for per Statement Latencies per
User.  Santosh helped me debug this.

A fix to a bug which I encountered while testing of the values of
histogram buckets, and if they actually sum up to the number of requests
of that corresponding statistic.  The bug was that there is an increment
to histogram_async_read when actually there should be an increment to
histogram_async_write.  Fixed this here.

Test Plan: Jenkins

Reviewers: rongrong, steaphan

Reviewed By: rongrong
Added latency histograms to track fsync and file flush.

Summary:
Added Histograms to track Innodb Fsync Latency and Innodb File Flush Time.

This diff adds two histograms, as the following:
histogram_fsync
histgoram_file_flush_time

These are used to track the Innodb Fsync Latency and Innodb File Flush
time respectively.

The File Flush Time includes the total time taken by Innodb to flush the
contents of the file to disk device.This may include multiple calls to
fsync() as locks may not be available all the time.

This diff also adds two new system server variables , which are
configurable step sizes for the above added histograms:

innobase_histogram_step_size_file_flush_time
innobase_histogram_step_size_fsync

Test Plan:
innobase_histogram_step_size_fsync_basic
innobase_histogram_step_size_file_flush_time_basic
show_variables_histogram
jenkins

Reviewers: steaphan, rongrong

Reviewed By: steaphan
Display Histograms in SHOW GLOBAL STATUS

Summary:
This diff prints all the histogram data to the output of SHOW GLOBAL STATUS.
The value corrersponding to each bucket (of all the histograms) is displayed, and the bucket range (that is to be displayed in microseconds/milliseconds/seconds) is calculated on the fly.

Following is the format in which Histogram data is displayed in the output of SHOW GLOBAL STATUS:
<HistogramName>_<BucketLowerValue>-<BucketUpperValue><Unit>

Presently all the numeric values in the test output are replaced by constants, because after each run the output will not match with the previous.
In future a rigorous test can be written which validates the histogram values.

Test Plan:
innodb_histogram_show_global_status
jenkins

Reviewers: rongrong, steaphan

Reviewed By: rongrong
Display Histograms in SHOW ENGINE INNODB STATUS

Summary:
This diff displays the Histogram data in the output of SHOW ENGINE INNODB STATUS
Following is the format in which the data is displayed:
Histograms

Async Reads
Bucket xxx to xxx units 	Value
Bucket xxx to xxx units		Value
...............

Async Writes
.....................

and so on..

In the test output, I replaced all numeric values string constant, since these cannot be the same on each run(same is the case with the histogram bin range, because of them being of double data type).
However under the section "File I/O", the state of the Innodb threads is changing frequently on each run. While some are waiting, the others have completed and so on. So I cannot compare the output of SHOW ENGINE INNODB STATUS with the output of a previous run in any case.

A more complicated test cam be written in the future to extract certain output using Perl and then perform validation.

I checked by manually running the command and the histogram is getting printed in the correct format.It can be easily reproduced on any machine

Test Plan:
innodb_histogram_show_innodb_status
jenkins

Reviewers: rongrong, steaphan

Reviewed By: rongrong
Adding Histogram to track Binlog Fsyncs

Summary:
This diff adds a histogram (histogram_binlog_fsync) to track the latencies caused by binlog fsyncs. Along with this addition, this diff also adds a system variable which is responsible for configuring the step size of this histogram, similar to previous histograms.
The system variable added is histogram_step_size_binlog_fsync
It is a string of the form "<Value><us/ms/s>". The <Value> is of double data type.

Test Plan:
histogram_step_size_binlog_fsync_basic
show_variables_histogram
jenkins

Reviewers: rongrong, steaphan

Reviewed By: rongrong
Display Histogram for Binlog Fsyncs in SHOW GLOBAL STATUS

Summary:
This diff displays the histograms that track the binlog fsync latencies, in the output of SHOW GLOBAL STATUS
The output format is similar to how other histograms are shown:

Latency_histogram_binlog_fsync_XXX-XXXunits	Value

Test Plan:
innodb_histogram_show_global_status
jenkins

Reviewers: rongrong, steaphan

Reviewed By: rongrong
Adding Histogram for Connection Create Latency per User

Summary:
This diff adds a new histogram (histogram_connection_create) in order to track the connection create latency per user. It also adds a configurable system variable to set the step size of this histogram. This variable is named histogram_step_size_connection_create.
(The configurable step size variable is common for all users)
I have assumed that the time taken for a connection to be created is a sum of the following times:
1. Time taken to launch the new thread (which will handle the connection)
2. Time taken to login the connection and authenticate the user using Access Control Lists etc
3. Time taken to prepare connection state

This way we will be able to track the total time taken starting from when a new thread is spawned upto the time the connection is ready to handle queries.

Test Plan:
show_variables_histogram
histogram_step_size_connection_create_basic
jenkins

Reviewers: steaphan, rongrong

Reviewed By: steaphan
Add Histograms for per User Latencies

Summary:
This diff adds histograms to track the latencies corresponding to each statement type in MySQL, per user. The histograms are:
histogram_update_command
histogram_delete_command
histogram_insert_command
histogram_select_command
histogram_ddl_command
histogram_transaction_command
histogram_handler_command
histogram_other_command

This diff also adds histogram to track connection create latency per user:
histogram_connection_create

The names of these histograms themselves depict which statement type latency they track.

Along with the histograms, this diff also adds configurable step sizes as system variables, one each for each histogram. Each statement type has latency lying in different ranges, due to which also I have given them different default values (I checked for some by debugging it).The system variables added are:
histogram_step_size_update_command
histogram_step_size_delete_command
histogram_step_size_insert_command
histogram_step_size_select_command
histogram_step_size_ddl_command
histogram_step_size_transaction_command
histogram_step_size_handler_command
histogram_step_size_other_command
histogram_step_size_connection_create

Test Plan:
show_variables_histogram
histogram_step_size_update_command_basic
histogram_step_size_delete_command_basic
histogram_step_size_insert_command_basic
histogram_step_size_select_command_basic
histogram_step_size_ddl_command_basic
histogram_step_size_transaction_command_basic
histogram_step_size_handler_command_basic
histogram_step_size_other_command_basic
histogram_step_size_connection_create_basic
jenkins

Reviewers: rongrong, steaphan

Reviewed By: steaphan
Add a new Information Schema table for per User Latency Histograms

Summary:
This diff adds a new information schema table called USER_LATENCY_HISTOGRAMS which is used to display the values of histogram bins corresponding to 8 statement types per user,as well as for connection create latency. The statement types are update,delete,insert,select,transaction,handler,ddl and other.
Table columns:
USER_NAME
TYPE (This has the values SELECT/INSERT/UPDATE/DELETE/TRANSACTION/HANDLER/DDL/OTHER/CONNECTION_CREATE)
STEP_SIZE (This has the value same as the system variable configurable step size of the corresponding histogram)
VALUE1
VALUE2
VALUE3
...
...
...
VALUE15

The last 15 columns store the value corresponding to the histogram bucket.

Test Plan:
information_schema-big
information_schema
information_schema_db
mysqlshow
is_columns_is
is_tables_is
user_latency_histograms
jenkins

Reviewers: rongrong, steaphan

Reviewed By: steaphan
Tests for validating Latency Histogram values

Summary:

This adds tests to confirm the output of latency histogram values in
Information_Schema.User_Latency_Histograms table, and the values in
SHOW ENGINE INNODB STATUS, and also tests that setting the bucket size
to zero works as expected.

This diff adds a test which is used to validate the histogram values
corresponding to the per Statement type latencies. It works as follows:

The perl script sums up all the values of histogram buckets for each
statement type, and then compares it to the commands_xxx value in the
user_statistics table, corresponding to that statement.

For example: if the statement type is INSERT,

the script retrieves the row with type='INSERT' from
information_schema.user_latency_histograms and sums up all the columns
values. It then compares this sum to the 'commands_insert' value in
information_schema.user_latency_histograms

If the values match, it prints out a message.

This diff also adds rigorous testing for checking if the histogram
bucket values, that are replaced with a constant because they change on
each run, are actually valid.

The testing is done as follows:

In the output of SHOW ENGINE INNODB STATUS, there are statistics printed
corresponding to Async read|writes, Sync reads|writes and so on.

The test processes the output of SHOW ENGINE INNODB STATUS to extract
the number of requests (for example, X) in :
Async reads: X requests, ....

It then extracts the bucket values of that histogram(for example here,
histogram_async_read), before replacing them with a string constant, and
sums all of them up and checks whether they are equal to X.

Note: SHOW ENGINE INNODB STATUS prints the number of fsyncs in front of
the statistic 'File Flushes:'. (I checked this in the file os0file.cc).
I added two separate histograms for file flush time and fsync time,
where the former might contain multiple calls to fsync if there is no
lock available. So I could not yet figure out how to test the values of
histogram_file_flush_time.

This diff also adds tests and option files which sets the configurable
server variables: Histogram step sizes to be ZERO.

The code has been written such that, when the step size of a histogram
is zero, it stops tracking the statistic it is meant for.

This diff also makes sure that "0" is a valid value for histogram step
size, because it is unlikely that somebody will enter the units with
"0".

Another point to note is that the "check" function for a configurable
server system variable is called only when a "SET" command is issued. It
is not called if the system variables are set from the command line as
mysqld options or set through the -master.opt file.

Hence the check for "0" is inserted by me both in the "check" function
of the respectie server variable and also the latency_histogram_init()
function.

Test Plan:
innodb_histogram_show_innodb_status
user_latency_histograms
innodb_histogram_check_step_size_zero
user_histograms_check_step_size_zero
innodb_histogram_step_size_async_read_basic
innodb_histogram_step_size_async_write_basic
innodb_histogram_step_size_sync_read_basic
innodb_histogram_step_size_sync_write_basic
innodb_histogram_step_size_log_write_basic
innodb_histogram_step_size_double_write_basic
innodb_histogram_step_size_fsync_basic
innodb_histogram_step_size_file_flush_time_basic
innodb_histogram_step_size_binlog_fsync_basic
innodb_histogram_step_size_connection_create_basic
innodb_histogram_step_size_update_command_basic
innodb_histogram_step_size_insert_command_basic
innodb_histogram_step_size_select_command_basic
innodb_histogram_step_size_delete_command_basic
innodb_histogram_step_size_ddl_command_basic
innodb_histogram_step_size_other_command_basic
innodb_histogram_step_size_transaction_command_basic
innodb_histogram_step_size_handler_command_basic
jenkins

Reviewers: rongrong, steaphan

Reviewed By: steaphan, rongrong
Remove mysql client warning about passwords

Summary:
These warnings were added in 5.6. We get rid of these warnings to make
the functionality as close as to that of 5.1. We'll revert this diff,
and these warnings back again later.

Test Plan:
arc unit (Jenkins)

Do a 'git grep -n "Warning: Using a password on the command line
interface can be insecure."' and re-run all the tests with --record.

Reviewers: jtolmer, chip, steaphan

Reviewed By: steaphan
alter table xx defragment command

Summary:
Add parser support for defragment command.
This only includes parser support for "alter table defragment", actual
functionality is not included in this diff.
Only works for innodb tables, doesn't work for partitioned table or
other storage engines.
The command is not replicated.

Test Plan:
Jenkins
Added test doesn't do much other than confirm the new command is valid.

Reviewers: steaphan, santoshb, nizamordulu

Reviewed By: steaphan
Port v5.1: Output row counts in mysqldump

Summary:
Output the number of rows dumped by mysqldump in comment after each table.

Test Plan:
New mtr test included.  All mtr tests pass in Jenkins.

Reviewers: jtolmer, nizamordulu

Reviewed By: nizamordulu
Port v5.1 Compression: Factor out record serialization.

Summary:
Factor out serialization of a record from page_zip_write_rec().

The main issue with serializing the records is that normally
the offsets are stored backwards and this causes problems during
deserialization. The code that writes to the modification log already
reverses the direction of the offsets to be able to apply the
modification log when the page is decompressed. We use the same code to
create a function to serialize a record to a data stream. This function
will be reused for serializing the records before compressing the page.

Test Plan: Jenkins (arc unit)

Reviewers: rongrong, nizamordulu

Reviewed By: nizamordulu
Port v5.1 "Factor out deserialization of a record" to 5.6

Summary:
Port v5.1 "Factor out deserialization of a record from
page_zip_apply_log()" to 5.6

When the modification log is applied to the uncompressed version of the
compressed page image, the records are read from the modification log.
We factor out this piece of code into its own function so that we can
use this deserialization function when we need to deserialize the
serialization of records on a page.

Test Plan: arc unit

Reviewers: nizamordulu

Reviewed By: nizamordulu
Add GTID support in xtrabackup

Summary:
This diff adds support for GTIDs when --slave-info is passed to
innobackupex. The gtid set is output both in xtrabackup_binlog_info,
and in the copy log.

Test Plan: MTR test added.

Reviewers: santoshb

Reviewed By: santoshb
Don't use O_DIRECT for xtrabackup_logfile

Summary:
LOG_FILE_HDR_SIZE is 2048, while OS block size can be as high as 4096.
Thus leading to an unaligned buffer, and OS error EINVAL.

We already skip using O_DIRECT on xtrabackup_logfile when taking backup.
Do the same when restoring.

Test Plan: Ran a few restores.

Reviewers: steaphan, santoshb

Reviewed By: santoshb