World Models & Digital Networks - Rob Piechocki
Channel: NG-CDI
Date: 2021-03-18

Recent achievements in machine learning underpinned by learning representations in an un(self)supervised paradigm. Such techniques are at the heart of the latest and best performing language models (BERT, GPT-3), contrastive learning computer vision or protein folding predictors (AlphaFold2). The common feature of such techniques is an attempt to build a fundamental understanding of the model i.e. its “world model”, before a subsequent attempt is made to solve the given task. This is in stark contrast to the state-of-the-art techniques (including ML/AI) currently used in digital networks, where the algorithms are specifically crafted to solve the given task(s) and trained from the outset to achieve this. Can digital networks perform better by initially learning their own digital world models?