Teaching to Teach by Structured Dark Knowledge

1 INTRODUCTION . From an infant to a fully functional adult , human being requires years of highly advanced education . It purposively uses pedagogical instruments , demonstrates suitable examples and organizes targeted examinations , so as to reduce human being ’ s time to equip with knowledge and skills . Drawing lessons from such social evolutionism , learning scientists proposed teaching Anderson et al . ( 1985 ) ; Goldman & Kearns ( 1995 ) , a coined terminology that broadly refers to the frameworks and algorithms guiding better training qualities for complicated machine learners , e.g. , networks and agents . One way to teach them is to demonstrate examples through following the leitmotiv “ from-easy-to-hard ” , famous as curriculum learning ( CL ) Bengio et al . ( 2009 ) . Specifically , Classical CLs ( CCLs ) Spitkovsky et al . ( 2010 ) ; Zaremba & Sutskever ( 2014 ) manage a syllabus ( a dynamical training criteria ) by ranking examples/tasks via increasing their difficulties from the perspective of human understanding 1 . Then the difficulty threshold continuously updates to tolerate harder examples and tasks ( Fig.1 . a.upper ) . Sometimes , CCLs inevitablely entail human priors causing extra annotation and heuristically tuning , thus , are limited due to the algorithmic transferability . By contrast , Interactive CLs ( ICLs ) including SPL Kumar et al . ( 2010 ) ; SPCL Jiang et al . ( 2015 ) ; ACL Graves et al . ( 2017 ) ; TSCL Matiisen et al . ( 2017 ) ; MentorNet Jiang et al . ( 2017 ) ; sampling-based T2L Fan et al . ( 2018 ) and more , receive online training feedbacks as the difficulty signals to update their syllabuses . Their learner-teacher communication protocols reap the curriculum advantage without labor involvement ( Fig.1 . a.lower ) . So despite inconsistent performances across different learning scenarios Sachan & Xing ( 2016 ) ; Fan et al . ( 2018 ) , ICLs maintain prevalent in the frontier of CL researches . CCLs pay attention to human-explainable teaching while ICLs believe that only black-box learners can bring up a machine-suited curriculum . These cutting-edge studies indulging the argument about these pedagogical ordering styles , rarely realize that , most knowledge emerges as a natural macrocosm rather than a pile of isolated or artificially ordered pieces . For instance , a biological taxonomy consists of levels of creature species in a hierarchy ; sentences and paragraphs imply logic rules of writing ; a knowledge base bridges concepts embedded in undirected graphs . These structures integrate sporadic 1 Example/task with less difficulty owns more priority in training , which presents as sampling with more frequency , or larger loss coefficient . pieces of knowledge , e.g. , demonstrated examples , concepts ( attributes and categories ) , multiple tasks , to reflect cognitive association characteristics , e.g. , ambiguity , diversity , complementarity , causality , etc . It promises across-the-aboard education quality and has already grabbed a great amount of attentions in the field of pedagogical psychology Kirkpatrick & Epstein ( 1992 ) . Regretfully , retrospect to massive literatures related to CLs in details and we find that , seldom were proposed under this consideration . A few of studies directly treated the case as an evolving training subset selection with diversity , based upon self-paced learning Jiang et al . ( 2014 ) Zhou & Bilmes ( 2018 ) or some tricks for a concrete problem Sachan & Xing ( 2016 ) . They are situated under the difficulty-specific background and absent for generalization . Besides , their selected sets are directly used to optimize the learner , which implies the deterministic binary weights of training examples . It performs inferior compared with the other CL strategies promoting soft weighting or stochastic sampling technique . Just as American novelist Ralph Ellison said “ Education is all a matter of building bridges ” , to bridge pieces of knowledge therefore rectify curriculum-based machine education , is what our work chases after . Note that , rather than crafting a specific CL algorithm or framework , we prefer teaching to teach ( T2T ) , namely , distilling the substructure to dig out the possible coherence of training instances , e.g. , examples , categories and tasks , then teach curriculum-routed teachers from existing CLs . As illustrated by the comic in Fig.1 .b , our methodology extends CCLs and ICLs by a teacher of teacher mastering structured knowledge , which is transferred to curriculum-routed teachers within a virtuous cycle : On one hand , this teacher of teacher “ comprehends ” the previous CCL/ICL teaching decisions , thus adaptively selects a knowledge-based substructure to “ instruct ” the curriculum-routed teachers . On the other hand , the curriculum-routed teachers merge the curricula with their updated structured knowledge to iteratively polish their teaching strategies toward learners . More specifically , let ’ s go through the technical discussion roadmap of this paper . In Section.3.1 , we first revisit CCL and ICL approaches and frame them into the background of progressive reweighting learning . Then we observe that , the weight of each training instance inferred by CCL/ICL is valued in [ 0 , 1 ] . In a stochastic sampling circumstance , they also represent the instance-selection probabilities , thus , the inference can be regulated by KL divergence , similar to transferring dark knowledge Hinton et al . ( 2015 ) by matching their activation outputs . But how to construct the structured knowledge to properly regulate the probabilistic weight inferences ? In Section.3.2 , we introduce a set function as our structured knowledge prior , namely , the teacher of teacher shown in Fig.1 .b . The set function could be submodular Fujishige ( 2005 ) or just preserves submodular-like properties Das & Kempe ( 2011 ) Zhou & Spanos ( 2016b ) ( More specification refers to our Appendix.B ) . Our set function cooperates with the previous CL teaching decisions by matching their outputs , thus selects a subset of training instances to adaptively extract the structured knowledge . When curriculum-routed teaching strategy updates , this activation matching plays a role to transfer the structured knowledge to infer the curriculum teaching strategy . This cycle can be flexibly deployed in most existing CL strategies then incorporate structured knowledge to enhance their teaching performances . Compared with previous work about CLs , our methodology embodies three apparent virtues : • Generality for structured knowledge . Our methodology employs a generic set function as the structured knowledge prior to perform a constrained subset selection , where the structured dark knowledge only performs as a modular function . It is in harmony with diverse forms of subset selection , thus , refers to various structured knowledge among data . • Flexibility for curricula . Our methodology could be flexibly deployed on all curriculum- based teaching strategies driven by , e.g. , models , algorithms , preset rules , even choices from human beings , as long as they obey the generalized CCL and ICL formulas in SubSection.3.1 . • Simplicity for implementation . Our methodology connects teaching and teaching to teach with only a structured dark knowledge term conceptually simple for implementation . Finally , we verify our methodology in three scenarios : classification , domain adaptation and sequence learning . We import the structured knowledge priors about diversity , complementarity and causality to their experimental setups . Empirical studies across diverse learners and teachers show that structured dark knowledge can be acquired by the teachers and helpful for them to educate the learners better . 2 RELATED LITERATURES . Teaching has gradually become an attractive AI research direction Khan et al . ( 2011 ) ; Zhu ( 2015 ) ; Zhu ( 2013 ) . Our work keeps cohesive with two thriving trends about how to teach . 2.1 TEACHING TO LEARN WITH CURRICULUM . Plenty of researches concerned the learning principle starting small Elman ( 1993 ) . Bengio et al . ( 2009 ) suggested that a series of training criterion by increasing the sample-based learning difficulties are able to accelerate the trainings or improve the performances of networks . This cognition-steered ideology was interpreted as curriculum learning ( CL ) . Though born as a heuristic for practitioners , CL attracts increasing interests of theoretician in explaining , e.g. , extreme strategy in teaching dimension Khan et al . ( 2011 ) , relationship with importance sampling Katharopoulos & Fleuret ( 2018 ) . Under some circumstance , the CLs using ideal difficulty score Weinshall & Cohen ( 2018 ) are proved to boost linear regression learner ’ s convergence rate . Doubtlessly , CLs inspired a line of subsequent deep-model-based AI investigations about how to imitate human behaviors , e.g. , BCD number calculation Zaremba & Sutskever ( 2014 ) , game shooting from the first-person aspect Wu & Tian ( 2016 ) , etc , and especially suit robotic control Sanger ( 1994 ) Florensa et al . ( 2017 ) . Since these CLs are human-designed , we call them classical CLs ( CCLs ) in this paper . Syllabuses in CCLs proceed in accordance with predefined schemes . For instance , a robotic arm is supposed to acquire the grasping motions from elementary to complex . By contrast , interactive curriculum learnings ( ICLs ) Graves et al . ( 2017 ) ; Kumar et al . ( 2010 ) prefer the syllabuses in adaptive dynamics , namely , self-refine the difficulties to keep consistent with the learner ’ s training feedbacks . This learner-oriented manner appeals to the co-evolution of the learner and the teacher to execute curriculum . The pioneering researches track back to Self-paced learning ( SPL ) Kumar et al . ( 2010 ) and its variants Jiang et al . ( 2015 ) ; Jiang et al . ( 2014 ) ; Zhou & Bilmes ( 2018 ) . They directly take current-step training losses as the difficulty feedbacks , then , are programmed to teach by reweighting losses in the constraints of scheme functions . Recently , using various kinds of learning progress signals Houthooft et al . ( 2016 ) , ACLGraves et al . ( 2017 ) , STCLMatiisen et al . ( 2017 ) and OACL Doan et al . ( 2018 ) apply bandit-based sampling Auer et al . ( 2002 ) to teach deep learners and GANs . Attractive progresses of ICL even employ a network to guide network training . For example , MentorNet Jiang et al . ( 2017 ) , ScreenNet Kim & Choi ( 2018 ) focus on learnable reweighting schemes ; Ho et al . ( 2016 ) ; Milli et al . ( 2017 ) Fan et al . ( 2018 ) employ agents to select training samples . The latters are known as sampling-based teaching , while also belong to ICL in a broad sense . 2.2 TEACHING TO LEARN WITH DARK KNOWLEDGE . The development of dark knowledge originates from the mockup experiments about hyper energy physics , where the deep networks were leveraged to search for dark matter from the particle collider synthesis Sadowski et al . ( 2015 ) . Hinton et al . ( 2015 ) elaborated the method as knowledge distillation ( KD ) , namely , seeking to transfer dark knowledge , the soft predictions containing not only correct but also meaty ambiguous wrong informations , from a sophisticated teacher to fresh student networks by matching their output activations . The primitive goal of dark knowledge is to compress a complicated model into a lightweight one while retaining comparative performance . There could be multiple teachers for one student Rusu et al . ( 2015 ) ; Ruder et al . ( 2016 ) , where the student is supposed to inherit all the teachers ’ capabilities . Recent researches discovered its potential to boost student network ’ s performance Furlanello et al . ( 2018 ) . It explains the successes of its wide applications in the other areas Shin et al . ( 2017 ) ; Papernot et al . ( 2016 ) ; Lopez-Paz et al . ( 2015 ) . Dark knowledge is broadly deemed as the gift from deep learning , whereas also surprisingly successful by hiring a teacher from the other ML areas . Hu et al . ( 2016 ) designed a rule-based teaching with soft first-order logic , then the student nets are iteratively updated to obey these principles . In Bayesian dark knowledge Korattikara et al . ( 2015 ) , a Monte Carlo posterior predictive density ( e.g. , stochastic gradient Lagevin dynamic ) educates a student to endow a probabilistic model reaping the benefit of Bayesian tools yet only costing the same run time as those plug-in methods .

Summary: Summarization: This paper studies how to inject structured prior knowledge into the teaching model for machine learning. The authors propose a very general framework called `teach to teach’, in which 1) the knowledge is distilled via subset selection that matches the teacher decision distribution 2) the distilled knowledge is then transferred into the teacher model by reweighting the contributions of teacher objective and coherence constraints. Extensive experiments are conducted on image classification, unsupervised domain adaptation and sequence learning.