Repository: intel/ipex-llm
Pull request: Add some more input/output docs to convolutions, rnns, recursive nns
## What changes were proposed in this pull request?
Add some more input/output docs to convolutions, rnns, recursive nns
## Related links or issues (optional)
fixed https://github.com/intel-analytics/BigDL/issues/1478



Problem statement:
add input/output specifics for layers in mddocs
Some user complains layer input/output is not specified clearly in docs and difficult to use. We need to specify them in our md docs. Below is an example input/output spec for `convlstm`.
> The input tensor in forward(input) is expected to be a 5D tensor (batch x time x nInputPlane x height x width). output of forward(input) is also expected to be a 5D tensor (batch x time x outputPlane x height x width).

We should add such specs for: 
- All Convolution layers. 
- All Recurrent Layers. 
- All Recursive Layers.

Hints from issue discussion:
please merge the modifications in master.

Commit message: add some more input/output docs to convolutions, rnns, recursive nns

Patch to review:
diff --git a/docs/docs/APIGuide/Layers/Convolution-Layers.md b/docs/docs/APIGuide/Layers/Convolution-Layers.md
index 54590cb1b68..6bd1c77884b 100644
--- a/docs/docs/APIGuide/Layers/Convolution-Layers.md
+++ b/docs/docs/APIGuide/Layers/Convolution-Layers.md
@@ -13,6 +13,8 @@ SpatialConvolution is a module that applies a 2D convolution over an input image
 
 The input tensor in `forward(input)` is expected to be
 either a 4D tensor (`batch x nInputPlane x height x width`) or a 3D tensor (` nInputPlane x height x width`). The convolution is performed on the last two dimensions.
+output of `forward(input)` is also expected to be a 4D tensor (`batch x outputPlane x height x width`)
+or a 3D tensor (`outputPlane x height x width`)..
 
 As for padding, when padW and padH are both -1, we use a padding algorithm similar to the "SAME" padding of tensorflow. That is
 ```scala
@@ -194,7 +196,10 @@ module = VolumetricConvolution(n_input_plane, n_output_plane, k_t, k_w, k_h,
 ```
 
 Applies a 3D convolution over an input image composed of several input planes. The input tensor
-in forward(input) is expected to be a 4D tensor (nInputPlane x time x height x width).
+in forward(input) is expected to be a 5D tensor (`batch x nInputPlane x time x height x width`) or
+a 4D tensor (`nInputPlane x depth x height x width`).
+Output of forward(input) is also expected to be a 5D tensor (`batch x time x outputPlane x height x width`) or
+a 4D tensor (`outputPlane x depth x height x width`).
 
 * `nInputPlane` The number of expected input planes in the image given into forward()
 * `nOutputPlane` The number of output planes the convolution layer will produce.
@@ -317,7 +322,10 @@ layer = SpatialDilatedConvolution(
 
 Apply a 2D dilated convolution over an input image.
 
-The input tensor is expected to be a 3D or 4D(with batch) tensor.
+The input tensor in `forward(input)` is expected to be
+either a 4D tensor (`batch x nInputPlane x height x width`) or a 3D tensor (` nInputPlane x height x width`).
+output of `forward(input)` is also expected to be a 4D tensor (`batch x outputPlane x height x width`)
+or a 3D tensor (`outputPlane x height x width`).
 
 For a normal SpatialConvolution, the kernel will multiply with input
 image element-by-element contiguous. In dilated convolution, it’s possible
@@ -437,8 +445,10 @@ layer = SpatialShareConvolution(nInputPlane, nOutputPlane, kW, kH, dW, dH, padW,
 ```
 
  Applies a 2D convolution over an input image composed of several input planes.
- The input tensor in forward(input) is expected to be
- a 3D tensor (nInputPlane x height x width).
+ The input tensor in `forward(input)` is expected to be
+ either a 4D tensor (`batch x nInputPlane x height x width`) or a 3D tensor (` nInputPlane x height x width`).
+ output of `forward(input)` is also expected to be a 4D tensor (`batch x outputPlane x height x width`)
+ or a 3D tensor (`outputPlane x height x width`).
 
  This layer has been optimized to save memory. If using this layer to construct multiple convolution
  layers, please add sharing script for the fInput and fGradInput. Please refer to the ResNet example.
@@ -554,7 +564,10 @@ m = SpatialFullConvolution(n_input_plane,n_output_plane,kw,kh,dw=1,dh=1,pad_w=0,
 SpatialFullConvolution is a module that applies a 2D full convolution over an input image. 
 
 The input tensor in `forward(input)` is expected to be
-either a 4D tensor (`batch x nInputPlane x height x width`) or a 3D tensor (`nInputPlane x height x width`). The convolution is performed on the last two dimensions. `adjW` and `adjH` are used to adjust the size of the output image. The size of output tensor of `forward` will be :
+either a 4D tensor (`batch x nInputPlane x height x width`) or a 3D tensor (` nInputPlane x height x width`).
+output of `forward(input)` is also expected to be a 4D tensor (`batch x outputPlane x height x width`)
+or a 3D tensor (`outputPlane x height x width`).
+The convolution is performed on the last two dimensions. `adjW` and `adjH` are used to adjust the size of the output image. The size of output tensor of `forward` will be :
 ```
   output width  = (width  - 1) * dW - 2*padW + kW + adjW
   output height = (height - 1) * dH - 2*padH + kH + adjH
@@ -925,10 +938,13 @@ module = TemporalConvolution(
   )
 ```
 
-Applies a 1D convolution over an input sequence composed of nInputFrame frames.
-The input tensor in `forward(input)` is expected to be a 2D tensor
-(`nInputFrame` x `inputFrameSize`) or a 3D tensor
-(`nBatchFrame` x `nInputFrame` x `inputFrameSize`).
+ Applies a 1D convolution over an input sequence composed of nInputFrame frames.
+ The input tensor in `forward(input)` is expected to be a 3D tensor
+ (`nBatchFrame` x `nInputFrame` x `inputFrameSize`) or a 2D tensor
+ (`nInputFrame` x `inputFrameSize`).
+ Output of `forward(input)` is expected to be a 3D tensor
+ (`nBatchFrame` x `nOutputFrame` x `outputFrameSize`) or a 2D tensor
+ (`nOutputFrame` x `outputFrameSize`).
 
  * `inputFrameSize` The input frame size expected in sequences given into `forward()`.
  * `outputFrameSize` The output frame size the convolution layer will produce.
@@ -1049,7 +1065,12 @@ and the size of sizeTensor is used to set the size of the output (will ignore th
 adjH values used to construct the module). This module can be used without a bias by setting
 parameter noBias = true while constructing the module.
 
-If input is a 4D tensor nInputPlane x depth x height x width,
+Applies a 3D convolution over an input image composed of several input planes. The input tensor
+in forward(input) is expected to be a 5D tensor (`batch x nInputPlane x time x height x width`) or
+a 4D tensor (`nInputPlane x depth x height x width`).
+Output of forward(input) is also expected to be a 5D tensor (`batch x time x outputPlane x height x width`) or
+a 4D tensor (`outputPlane x depth x height x width`).
+
 ```
 odepth  = (depth  - 1) * dT - 2*padT + kT + adjT
 owidth  = (width  - 1) * dW - 2*padW + kW + adjW
diff --git a/docs/docs/APIGuide/Layers/Recurrent-Layers.md b/docs/docs/APIGuide/Layers/Recurrent-Layers.md
index 61886de145b..223540e4850 100644
--- a/docs/docs/APIGuide/Layers/Recurrent-Layers.md
+++ b/docs/docs/APIGuide/Layers/Recurrent-Layers.md
@@ -161,14 +161,17 @@ rnnCell = RnnCell(input_size, hidden_size, Tanh(), w_regularizer, u_regularizer,
 
 Implementation of vanilla recurrent neural network cell
 
-* `i2h` weight matrix of input to hidden units
-* `h2h` weight matrix of hidden units to themselves through time
+The input tensor in `forward(input)` is expected to be a 3D tensor (`batch x time x inputSize`). output of
+`forward(input)` is also expected to be a 3D tensor (`batch x time x hiddenSize`).
 
 The updating is defined as:
 
 ```
 h_t = f(i2h * x_t + h2h * h_{t-1})
 ```
+where
+* `i2h` weight matrix of input to hidden units
+* `h2h` weight matrix of hidden units to themselves through time
 
 Parameters:
 
@@ -268,6 +271,8 @@ lstm = LSTM(input_size, hidden_size)
 ```
 
 Long Short Term Memory architecture.
+The input tensor in `forward(input)` is expected to be a 3D tensor (`batch x time x inputSize`). output of
+`forward(input)` is also expected to be a 3D tensor (`batch x time x hiddenSize`).
 
 Ref:
 
@@ -400,6 +405,8 @@ model = LSTMPeephole(
 
 Long Short Term Memory architecture with peephole.
 Ref.
+The input tensor in `forward(input)` is expected to be a 3D tensor (`batch x time x inputSize`). output of
+`forward(input)` is also expected to be a 3D tensor (`batch x time x hiddenSize`).
 
 1. http://arxiv.org/pdf/1303.5778v1 (blueprint for this module)
 2. http://web.eecs.utk.edu/~itamar/courses/ECE-692/Bobby_paper1.pdf
@@ -509,6 +516,8 @@ gru = GRU(inputSize, outputSize, p, w_regularizer, u_regularizer, b_regularizer)
 ```
 
 Gated Recurrent Units architecture. The first input in sequence uses zero value for cell and hidden state.
+The input tensor in `forward(input)` is expected to be a 3D tensor (`batch x time x inputSize`). output of
+`forward(input)` is also expected to be a 3D tensor (`batch x time x outputSize`).
 
 Ref.
 
diff --git a/docs/docs/APIGuide/Layers/Recursive-Layers.md b/docs/docs/APIGuide/Layers/Recursive-Layers.md
index 9fb5f78d273..8b2c617682d 100644
--- a/docs/docs/APIGuide/Layers/Recursive-Layers.md
+++ b/docs/docs/APIGuide/Layers/Recursive-Layers.md
@@ -70,6 +70,10 @@ Tree-LSTM is a kind of recursive neural networks, as described in the paper
 [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://arxiv.org/abs/1503.00075)
  by Kai Sheng Tai, Richard Socher, and Christopher Manning.
 
+The input tensor in `forward(input)` is expected to be a table, in which the first element is the 3D embedding tensor tree
+(`batch x tree node number x (number of branches + 1)`],
+the second elment is the a 3D tensor (`batch x leaf number x inputSize`). output of
+`forward(input)` is expected to be a 3D tensor (`batch x tree node number x hiddenSize`).
 
 Parameters:
 * `inputSize` the size of each input vector


Code review comments:
diff_hunk: @@ -1049,7 +1065,12 @@ and the size of sizeTensor is used to set the size of the output (will ignore th
 adjH values used to construct the module). This module can be used without a bias by setting
 parameter noBias = true while constructing the module.
 
-If input is a 4D tensor nInputPlane x depth x height x width,
+Applies a 3D convolution over an input image composed of several input planes. The input tensor
+in forward(input) is expected to be a 5D tensor (`batch x nInputPlane x time x height x width`) or
path: docs/docs/APIGuide/Layers/Convolution-Layers.md
text: @user1:
Similar to VolumetricConvolution, 3rd dimension is not required to be time, input can also be 3D spatial images.

diff_hunk: @@ -194,7 +196,10 @@ module = VolumetricConvolution(n_input_plane, n_output_plane, k_t, k_w, k_h,
 ```
 
 Applies a 3D convolution over an input image composed of several input planes. The input tensor
-in forward(input) is expected to be a 4D tensor (nInputPlane x time x height x width).
+in forward(input) is expected to be a 5D tensor (`batch x nInputPlane x time x height x width`) or
+a 4D tensor (`nInputPlane x depth x height x width`).
+Output of forward(input) is also expected to be a 5D tensor (`batch x time x outputPlane x height x width`) or
path: docs/docs/APIGuide/Layers/Convolution-Layers.md
text: @user1:
Output with 5D tensor should be batch x outputPlane x depth(or time) x height x width ?

diff_hunk: @@ -194,7 +196,10 @@ module = VolumetricConvolution(n_input_plane, n_output_plane, k_t, k_w, k_h,
 ```
 
 Applies a 3D convolution over an input image composed of several input planes. The input tensor
-in forward(input) is expected to be a 4D tensor (nInputPlane x time x height x width).
+in forward(input) is expected to be a 5D tensor (`batch x nInputPlane x time x height x width`) or
path: docs/docs/APIGuide/Layers/Convolution-Layers.md
text: @user1:
I think for 5D tensor, the input is like batch x nInputPlane x depth(time) x height x width, the 3rd dimension is not required to be time.