Issue 1:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linus centos 7

TensorFlow installed from (source or binary): pip

TensorFlow version (use command below): 1.4

Python version: 2.7.5

Bazel version (if compiling from source): 0

GCC/Compiler version (if compiling from source):0

CUDA/cuDNN version:0

GPU model and memory:0

Exact command to reproduce:

In the following situation, TF doc is not correct.

Input tensor shape : [1, 5, 2, 1]

Kernel shape: [1, 3, 1, 1]

Stride : [1, 5, 5, 1]

Padding = "SAME"

According to the formula we can compute :

out_h = 1

out_w = 1

if (in_height % strides[1] == 0):
pad_along_height = max(filter_height - strides[1], 0)
else:
pad_along_height = max(filter_height - (in_height % strides[1]), 0)
if (in_width % strides[2] == 0):
pad_along_width = max(filter_width - strides[2], 0)
else:
pad_along_width = max(filter_width - (in_width % strides[2]), 0)

gives :

pad_along_height = 0

pad_along_width = 1

then

pad_top = pad_along_height // 2
pad_bottom = pad_along_height - pad_top
pad_left = pad_along_width // 2
pad_right = pad_along_width - pad_left

gives:

pad_top = 0

pad_bottom = 0

pad_left = 0

pad_right = 1

How tensorflow do a convolution with a kernel of height 1 on a image of height 5 and which gives output of height 1 (stride = 5) ??? How TF do this ? The doc can't explain the method used ...

Doing retro engineering, I saw that TF apply the filter on the middle of the input tensor (pad_top = -2 and pad_bottom=-2).

I agree with this method, but the formulas of the Convolution doc is doing max(.., 0) so padding could never be negative (according to the doc).

Could someone explain me clearly what is the formula used in tensorflow ?

Could someone update the doc ?

Issue 2:
System information

Have I written custom code -- YES, only to demonstrate the problem (source code is below):

OS Platform and Distribution (Linux Ubuntu 16.04):

TensorFlow installed from (binary (PIP)):

TensorFlow version (1.4.0):

Python version (2.7.12):

Bazel version (N/A):

GCC/Compiler version (N/A):

CUDA/cuDNN version (N/A):

GPU model and memory (N/A -- CPU only):

Exact command to reproduce (See Source Code Below):

Describe the problem

There is an inconsistency between the convolution documentation on padding with 'SAME' located here and the behavior of the tf.nn.conv2d operator. In the example below I create a 3x1 input with values [[1.0][1.1][1.2]] and a 1x1 filter of value [1.0]. I specify the stride to be 1x3x1x1 which should result in only a single element be output and the padding to be 'SAME'. From the padding calculation in the above link:

pad_along_height:

in_height ( = 3) % strides[1]( = 3) == 0 so
pad_along_height = max(filter_height ( = 1) - strides[1] ( = 3), 0)
pad_along_height = max(-2, 0) = 0

pad along_width:

in_width ( = 1) % strides[2] ( = 1) == 0 so
pad_along_width = max(filter_width( = 1) - strides[2] ( = 1), 0
pad_along_width = max(0,0) = 0

My hypothesis is that pad_along_* is not using the max(x,0) and as a result, pad_along_height = -2. Therefore pad_top = -1 and pad_bottom = -1. If that was the case, then our input is reduced to only the middle element [1.1] which explains why the TF result of the code below is 1.1 rather than the expected 1.0 (value of first input).

If I change the padding to be VALID (no padding) then this code below gives the result of 1.0 or if i instead change the stride to 1,2,1,1 i get the expected value of 1.0 (although in this case my hypothesis proposes that pad_bottom is still -1).

Source code / logs

import tensorflow as tf
import numpy as np

i = tf.constant((np.ones(3) + np.arange(3) * 0.1).reshape(1,3,1,1), dtype=tf.float32, name='input')
f = tf.constant(np.ones(1).reshape(1,1,1,1), dtype=tf.float32, name='filter')

conv = tf.nn.conv2d(input=i, filter=f, strides=(1,3,1,1), padding='SAME')

with tf.Session() as sess:
out = sess.run(conv)
print out

Output:

[[[[ 1.10000002]]]]