hi there today we'll look at symbolic knowledge distillation from general language models to common sense models by peter west and others of the university of washington and the allen institute for artificial intelligence on a high level this paper takes a new approach to symbolic knowledge generation so to automatically coming up with knowledge graphs with symbolic knowledge graphs and rather than trying to mine this symbolic knowledge automatically from raw text or from existing knowledge bases they mine it from gpt-NUMBER_TOKEN so they use the gptNUMBER_TOKEN large language model in order to first come up with a corpus that gives them a corpus of symbolic knowledge and then they use that corpus in order to train a model that they call a common sense model but essentially a knowledge graph completion model so this is a new paradigm where you go what they say from machine to corpus to machine and it is their the paradigm they advertise here in contrast to what people did before the from human to corpus to machine which is where humans generate a corpus and then you train the machine on that corpus so we're going to look in into how they do it it's pretty surprising what they find in that for example the distilled model the models they come up with at the end they tend to be better not only than the humans or the human fed models they even tend to be better than the original teacher the gptNUMBER_TOKEN teacher and this is a result of how they combine the different elements here of the system and they strategically they strategically bring in outside help in the form of human knowledge so this could be a recipe for much more broad applications not only knowledge knowledge graph generation but various natural language tasks they combine cleverly prompting training small models and as i said bringing in small amounts of human annotated data strategically so as i said we'll go through it we'll look at the different stages and yeah tell me what you think in the comments subscribe if you haven't and let's dive in but first a quick word from our sponsor weights and biases your one-stop shop if you're a machine learning researcher practitioner a hobbyist a power user it does not matter weights and biases is with you from the inception of your idea tracking your experiments to really getting the fine details right optimizing your hyper parameters up until you deploy your model and track all of your metrics not only does it do that it also organizes your data sets your models and you can generate super cool reports from all of that in addition to that it lets you have great insight into what you research and what you produce and all of this runs in the cloud really effortless with a single line of code though today i want to talk to you about a yet not so well known feature of weights and biases and that is the weights and vices community so i believe they recently migrated this from like a giant slack onto this new sleek community website it's a discourse based forum essentially where you can get help not only for weights and biases stuff but also machine learning in general but not only is it a help page it's a discussion forum about all things machine learning also they organize regular events book reading groups and paper discussions and so on so if you're interested don't hesitate and hop over to the introduce yourself thread and take part in the discussion as i said this is still a pretty young place but it's bound to grow over the near future and of course if you want any advice on weights
START_SPONSOR_TOKEN but first a quick word from our sponsor weights and biases your one-stop shop if you're a machine learning researcher practitioner a hobbyist a power user it does not matter weights and biases is with you from the inception of your idea tracking your experiments to really getting the fine details right optimizing your hyper parameters up until you deploy your model and track all of your metrics not only does it do that it also organizes your data sets your models and you can generate super cool reports from all of that in addition to that it lets you have great insight into what you research and what you produce and all of this runs in the cloud really effortless with a single line of code though today i want to talk to you about a yet not so well known feature of weights and biases and that is the weights and vices community so i believe they recently migrated this from like a giant slack onto this new sleek community website it's a discourse based forum essentially where you can get help not only for weights and biases stuff but also machine learning in general but not only is it a help page it's a discussion forum about all things machine learning also they organize regular events book reading groups and paper discussions and so on so if you're interested don't hesitate and hop over to the introduce yourself thread and take part in the discussion as i said this is still a pretty young place but it's bound to grow over the near future and of course if you want any advice on weights END_SPONSOR_TOKEN