[1.4.x] Fixed #13794 -- Fixed to_field usage in BaseInlineFormSet.

Thanks sebastien at clarisys.fr for the report and gautier
for the patch.

Backport of 5e2c4a4bd1 from master
[1.4.x] Revert "Fixed #13794 -- Fixed to_field usage in BaseInlineFormSet."

This reverts commit b44519072e8a0ef56a0ae9e6e4a1fb04273eb0eb.

stable/1.4.x branch is in security-fixes-only mode.