text: 316,92,1202178956063064064,806058672619212800,Guillaume Lample,"Our new paper, Deep Learning for Symbolic Mathematics, is now on arXiv <LINK> We added *a lot* of new results compared to the original submission. With @f_charton (1/7) <LINK> Although neural networks struggle on simple arithmetic tasks such as addition and multiplication, we show that transformers perform surprisingly well on difficult mathematical problems such as function integration and differential equations. (2/7) We define a general framework to adapt seq2seq models to various mathematical problems, and present different techniques to generate arbitrarily large datasets of functions with their integrals, and differential equations with their solutions. (3/7) On samples of randomly generated functions, we show that transformers achieve state-of-the-art performance and outperform computer algebra systems such as Mathematica. (4/7) We show that beam search can generate alternative solutions for a differential equation, all equivalent, but written in very different ways. The model was never trained to do this, but managed to figure out that different expressions correspond to the same mathematical object 5/7 We also observe that a transformer trained on functions that SymPy can integrate, is able at test time to integrate functions that SymPy is not able to integrate, i.e. the model was able to generalize beyond the set of functions integrable by SymPy. (6/7) A purely neural approach is not sufficient, since it still requires a symbolic framework to check generated hypotheses. Yet, our models perform best on very long inputs, where computer algebra systems struggle. Symbolic computation may benefit from hybrid approaches. (7/7) @AndrewTouchet @f_charton Yes, we will open source our datasets and models soon! @ogrisel We used <LINK> to visualize what is happening in the network, but we only tried very quickly and we did not observe anything concrete. Some papers like <LINK> may be useful to get insights about the hidden layer activations / attention weights. @leloykun @f_charton Not much. We ran experiments on 8 GPUs because it's faster, but even with 1 GPU you get most of the performance in a few hours.",https://arxiv.org/abs/1912.01412,"Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be surprisingly good at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica. ",Deep Learning for Symbolic Mathematics,10,"['Our new paper, Deep Learning for Symbolic Mathematics, is now on arXiv <LINK>\nWe added *a lot* of new results compared to the original submission. With @f_charton (1/7) <LINK>', 'Although neural networks struggle on simple arithmetic tasks such as addition and multiplication, we show that transformers perform surprisingly well on difficult mathematical problems such as function integration and differential equations. (2/7)', 'We define a general framework to adapt seq2seq models to various mathematical problems, and present different techniques to generate arbitrarily large datasets of functions with their integrals, and differential equations with their solutions. (3/7)', 'On samples of randomly generated functions, we show that transformers achieve state-of-the-art performance and outperform computer algebra systems such as Mathematica. (4/7)', 'We show that beam search can generate alternative solutions for a differential equation, all equivalent, but written in very different ways. The model was never trained to do this, but managed to figure out that different expressions correspond to the same mathematical object 5/7', 'We also observe that a transformer trained on functions that SymPy can integrate, is able at test time to integrate functions that SymPy is not able to integrate, i.e. the model was able to generalize beyond the set of functions integrable by SymPy. (6/7)', 'A purely neural approach is not sufficient, since it still requires a symbolic framework to check generated hypotheses. Yet, our models perform best on very long inputs, where computer algebra systems struggle. Symbolic computation may benefit from hybrid approaches. (7/7)', '@AndrewTouchet @f_charton Yes, we will open source our datasets and models soon!', '@ogrisel We used https://t.co/XvZQbwOpQj to visualize what is happening in the network, but we only tried very quickly and we did not observe anything concrete. Some papers like https://t.co/iGqnO5ArJO may be useful to get insights about the hidden layer activations / attention weights.', ""@leloykun @f_charton Not much. We ran experiments on 8 GPUs because it's faster, but even with 1 GPU you get most of the performance in a few hours.""]",19,12,2142