launch/ampere huggingface dataset:
In the context of multitask reinforcement learning, this paper considers the problem of learning behaviours when given specifications of subtasks and the relationship between them, in the form of a task graph. The paper presents a neural task graph solver (NTS), which encodes this as a recursive-reverse-recursive neural network. A method for learning this is presented, and fine tuned with an actor-critic method. The approach is evaluated in a multitask grid world domain. This paper addresses an important issue in scaling up reinforcement learning to large domains with complex interdependencies in subtasks. The method is novel, and the paper is generally well written. I unfortunately have several issues with the paper in its current form, most importantly around the experimental comparisons. The paper is severely weakened by not comparing experimentally to other learning (hierarchical) schemes, such as options or HAMs. None of the comparisons in the paper feature any learning. Ideally, one should see the effect of learning with options (and not primitive actions) to fairly compare against the proposed framework. At some level, I question whether the proposed framework is doing any more than just value function propagation at a task level, and these experiments would help resolve this. Additionally, the example domain makes no sense. Rather use something more standard, with well-known baselines, such as the taxi domain. I would have liked to see a discussion in the related work comparing the proposed approach to the long history of reasoning with subtasks from the classical planning literature, notably HTNs. I found the description of the training of the method to be rather superficial, and I don't think it could be replicated from the paper in its current level of detail. The approach raises the natural questions of where the tasks and the task graphs come from. Some acknowledgement and discussion of this would be useful. The legend in the middle of Fig 4 obscures the plot (admittedly not substantially). There are also a number of grammatical errors in the paper, including the following non-exhaustive list: 2: as well as how to do -> as well as how to do it Fig 2 caption: through bottom-up -> through a bottom-up 3: Let S be a set of state -> Let S be a set of states 3: form of task graph -> form of a task graph 3: In addtion -> In addition 4: which is propagates -> which propagates 5: investigated following -> investigated the following
This paper considers the following model of a signal x = W^T h + b, where h is an m-dimensional random sparse vector, W is an m by n matrix, b is an n dimensional fixed bias vector.  The random vector h follows an iid sparse signal model,  each coordinate independently have some probability of being zero,  and the remaining probability is distributed  among nonzero values according to some reasonable pdf/pmf.  The task is to recover h, from the observation x via the activation functions like Sigmoid or ReLU.  For example, \hat{h} = Sigmoid(W^T h + b). The authors then show that, under the random sparsity model of h, it is possible to upper bound the probability P(||h-\hat{h}|| > \delta. m) in terms of the parameters of the distribution of h and W and b.  In some cases noise can also be tolerated.  In particular, if W is incoherent (columns being near-orthonormal), then the guarantee is stronger.  As far as I understood, the proofs make sense  - they basically use Chernoff-bound type argument.  It is my impression that a lot of conditions have to be satisfied for the recovery guarantee to be meaningful.  I am unsure if real datasets will satisfied so many conditions.  Also, the usual objective of autoencoders is to denoise  - i.e. recover x, without any access to W.  The authors approach in this vein seem to be only empirical.  Some recent works on associative memory also assume the sparse recovery model  - connections to this literature would have been of interest.  It is also not clear why compressed sensing-type recovery using a single ReLU or Sigmoid would be of interest:  are their complexity benefits?