torch/torch7 — pull request #42 by nicholas-leonard (2014-06-10 22:43:39+00:00)
Title: torch.renorm

Hi,

This PR includes a `torch.renorm` function with unit tests and documentation. It is particularly useful for as a hard constraint on the norm of sub-tensors. In my lab, we use it as a regularizer instead of weight decay. It stems from the original dropout paper, but can be used for any neural network.

I used to implement with a combination of `torch.norm`, `torch.cdiv`, `torch.gt` and `torch.cmul` (see unit test), but its not as efficient as `torch.renorm`. Also the `cutorch` version of this is underway (its my main reason for this PR).

--Nick