Repository: ludwig-ai/ludwig

Issue:
Shape mismatch when introducing multiple levels of dependencies
**Describe the bug**

When introducing multiple levels of dependencies, the shape of the _concatenated hidden states_ does not match the _input size for the dense layer of the output feature_.

In my case, the text output feature `qty_frac` depends on text output feature `summary`, and numerical output feature `qty` in turn depends on `qty_frac`.

I get the following error when running `ludwig train`:
```python-traceback
RuntimeError: mat1 and mat2 shapes cannot be multiplied (6x768 and 512x1)
```



**To Reproduce**

Minimal, reproducible example using bash and docker as only dependencies:
```bash
#!/usr/bin/env bash
FEATURE_LIST=$(
    docker run -i mikefarah/yq -o json -I 0 e '.' - <<EOF
- name: document
  type: text
- name: summary
  type: text
- name: qty_frac
  type: text
- name: qty
  type: numerical
EOF
)

mkdir /tmp/ludwig-debug
docker run \
    -it \
    -v /tmp/ludwig-debug/:/workdir \
    ludwigai/ludwig:nightly \
    synthesize_dataset \
    --features $FEATURE_LIST \
    --dataset_size 10 \
    --output_path /workdir/synthetic_data.csv

cat <<EOF >/tmp/ludwig-debug/config.yml
input_features:
  - name: document
    type: text
    level: word
output_features:
  - name: summary
    type: text
    level: word
    decoder: generator
  - name: qty_frac
    type: text
    level: word
    decoder: generator
    dependencies:
      - summary
  - name: qty
    type: numerical
    dependencies:
      - qty_frac
EOF

docker run \
    -it \
    -v /tmp/ludwig-debug/:/workdir \
    ludwigai/ludwig:nightly \
    train \
    --dataset /workdir/synthetic_data.csv \
    --config_file /workdir/config.yml \
    --output_directory /workdir/results
```

**Expected behavior**

Training starts without error.

**Screenshots**

Excerpt from the traceback:
```python-traceback
  File "/usr/local/lib/python3.7/site-packages/ludwig/features/numerical_feature.py", line 269, in logits
    return self.decoder_obj(hidden)
  File "/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/ludwig/decoders/generic_decoders.py", line 58, in forward
    return self.dense(inputs)
  File "/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/ludwig/utils/torch_utils.py", line 212, in forward
    output = torch.squeeze(self.dense(input), dim=-1)
  File "/usr/local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 103, in forward
    return F.linear(input, self.weight, self.bias)
  File "/usr/local/lib/python3.7/site-packages/torch/nn/functional.py", line 1848, in linear
    return torch._C._nn.linear(input, weight, bias)
RuntimeError: mat1 and mat2 shapes cannot be multiplied (6x768 and 512x1)
```

**Environment:**

See reproducible example, run in environment with:
- bash: `GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)`
- docker: `Docker version 20.10.11+azure-3, build dea9396e184290f638ea873c76db7c80efd5a1d2`

The `ludwigai/ludwig:nightly` Docker image was built from main at 89d18365c41c4ded68edd2095349ce4a6caf5d18.



Discussion:
For reference, here's the [related conversation on Slack](https://ludwig-ai.slack.com/archives/C01PN6M2RSM/p1642806615331569).
After debugging, I found out that the `input_size` field of `TextOutputFeature` "qty_frac" is being overwritten after being correctly computed. 

#### Defined multiple inheritance

`SequenceOutputFeature` inherits from `OutputFeature`:
https://github.com/ludwig-ai/ludwig/blob/65b3b1a7cdfc7149a2a8559eef1edbe4d38a3080/ludwig/features/sequence_feature.py#L215

`TextOutputFeature` in turn inherits from `SequenceOutputFeature`:
https://github.com/ludwig-ai/ludwig/blob/65b3b1a7cdfc7149a2a8559eef1edbe4d38a3080/ludwig/features/text_feature.py#L311

#### Modifications to `input_size`

1. In the constructor of `OutputFeature`, the input size is computed correctly based on dependencies:
https://github.com/ludwig-ai/ludwig/blob/65b3b1a7cdfc7149a2a8559eef1edbe4d38a3080/ludwig/features/base_feature.py#L219

2. In the constructor of `SequenceOutputFeature`, it gets overwritten with the default. The computed dependency information is lost.
https://github.com/ludwig-ai/ludwig/blob/65b3b1a7cdfc7149a2a8559eef1edbe4d38a3080/ludwig/features/sequence_feature.py#L230-L232

Resolving patch:
diff --git a/ludwig/features/base_feature.py b/ludwig/features/base_feature.py
--- a/ludwig/features/base_feature.py
+++ b/ludwig/features/base_feature.py
@@ -217,6 +217,7 @@ def __init__(self, feature: Dict[str, Any], other_output_features: Dict[str, "Ou
         logger.debug("  FCStack")
 
         self.input_size = get_input_size_with_dependencies(self.input_size, self.dependencies, other_output_features)
+        feature["input_size"] = self.input_size  # needed for future overrides
 
         self.fc_stack = FCStack(
             first_layer_input_size=self.input_size,


Test patch: