title: Sampling permutations for Shapley value estimation

authors: Mitchell, Rory

subjects: interpretability

abstract: Game-theoretic attribution techniques based on Shapley values are used to interpret blackbox machine learning models, but their exact calculation is generally NP-hard, requiring approximation methods for non-trivial models. As the computation of Shapley values can be expressed as a summation over a set of permutations, a common approach is to sample a subset of these permutations for approximation. Unfortunately, standard Monte Carlo sampling methods can exhibit slow convergence, and more sophisticated quasi-Monte Carlo methods have not yet been applied to the space of permutations. To address this, we investigate new approaches based on two classes of approximation methods and compare them empirically. First, we demonstrate quadrature techniques in a RKHS containing functions of permutations, using the Mallows kernel in combination with kernel herding and sequential Bayesian quadrature. The RKHS perspective also leads to quasi-Monte Carlo type error bounds, with a tractable discrepancy measure defined on permutations. Second, we exploit connections between the hypersphere S d−2 and permutations to create practical algorithms for generating permutation samples with good properties. Experiments show the above techniques provide significant improvements for Shapley value estimates over existing methods, converging to a smaller RMSE in the same number of model evaluations

text: Sampling permutations for Shapley value estimation Game-theoretic attribution techniques based on Shapley values are used to interpret blackbox machine learning models, but their exact calculation is generally NP-hard, requiring approximation methods for non-trivial models. As the computation of Shapley values can be expressed as a summation over a set of permutations, a common approach is to sample a subset of these permutations for approximation. Unfortunately, standard Monte Carlo sampling methods can exhibit slow convergence, and more sophisticated quasi-Monte Carlo methods have not yet been applied to the space of permutations. To address this, we investigate new approaches based on two classes of approximation methods and compare them empirically. First, we demonstrate quadrature techniques in a RKHS containing functions of permutations, using the Mallows kernel in combination with kernel herding and sequential Bayesian quadrature. The RKHS perspective also leads to quasi-Monte Carlo type error bounds, with a tractable discrepancy measure defined on permutations. Second, we exploit connections between the hypersphere S d−2 and permutations to create practical algorithms for generating permutation samples with good properties. Experiments show the above techniques provide significant improvements for Shapley value estimates over existing methods, converging to a smaller RMSE in the same number of model evaluations

year: 2022

title: A method for assessing and developing features of a learning organization

authors: Sun, Peter Yih-Tong

subjects: learning organization

abstract: The primary objective of this thesis is to evolve  a method for assessing and developing features of a learning organization .  To fulfill this, I approached the thesis by examining several research questions and using multiple research methodologies.  The research questions were not all established at the outset.  Rather, they evolved as features of a journey down a road less traveled.  With this journey came the decision to write the thesis in the first person.

The first research question was  Q1:  What will bridge the divide between organizational learning and the learning organization?   By reviewing the extant literature on organizational learning and the learning organization, I developed a theoretical framework that linked these two streams.  The framework suggests that the extent of divide between the two streams is determined by the extent of learning transfer.  The learning transfer is affected by the learning barriers operating at the levels of learning (i.e., individuals, groups, and organizational).  This led me to my second research question Q2:  What are these barriers to learning transfer and how do they impact the levels of learning in the organization?   I cumulated the dispersed literature on learning barriers, and synthesized the learning barriers into five key dimensions: Intrapersonal, relational, cultural, structural, and societal.  I then used the Delphi technique on 17 individuals to investigate the impact of the learning barriers on the levels of learning.  This generated two additional research questions.  The third research question was Q3:  How do individuals initiate a double-loop change?   This deals with the little researched area of initiation of double-loop change whilst engaging with the interfaces at the levels of learning.  I used multiple case studies to examine this question and found that individuals transit through four distinct stages when initiating double-loop change: 'embedded', 'embedded discomfited', 'scripted', and 'unscripted'.  Once double-loop learning has been initiated at the individual level, it is important that it is transferred across the organization.  Therefore, my fourth research question was Q4:  How does a new shared understanding for a double-loop change develop across the organization?   I did an in-depth, single case based investigation of an organization.  Using Identity and Complexity theory perspectives, I tracked the evolving new shared understanding through four phases: de-identification phase, situated re-identification phase, transition phase, and identification with core ideology phase.

The key insights from examining these research questions, particularly insights from examining Q3 and Q4, enabled me to suggest nine key organizational interventions necessary to overcome the learning barriers and develop a learning organization:  Identifying, developing, and dispersing double-loop mastery; Enabling constructive contradictions; Creating a superordinate organizational identity; Building emotional intelligence (in individuals and groups); Ambidextrous leadership; Strategic support for experimentation; Promoting 'systems doing'; Accessibility of valid information; Institutionalizing scanning across industry boundaries.  When these nine organizational interventions are implemented, they produce five new learning organization orientations: genetic diversity, organizational ideology, organizational dualism, organizational coupling, and strategic play.  These five new learning organizational orientations provide the archetypes of the learning organization.  I then developed an instrument to assess these five new orientations, and did a preliminary testing of the instrument.

While aspects of my work overlaid with previous knowledge, new advances in knowledge were established by:

  Postulating a link between the streams of organizational learning and learning organization
  Synthesizing learning barriers into the five key dimensions, and investigating their impact on the levels of learning
  Understanding the stages of double-loop learning initiation by an individual, whilst engaging with the interfaces at the levels of learning
  Understanding the process of a new shared understanding evolving
  Postulating five new orientations of the learning organization

text: A method for assessing and developing features of a learning organization The primary objective of this thesis is to evolve  a method for assessing and developing features of a learning organization .  To fulfill this, I approached the thesis by examining several research questions and using multiple research methodologies.  The research questions were not all established at the outset.  Rather, they evolved as features of a journey down a road less traveled.  With this journey came the decision to write the thesis in the first person.

The first research question was  Q1:  What will bridge the divide between organizational learning and the learning organization?   By reviewing the extant literature on organizational learning and the learning organization, I developed a theoretical framework that linked these two streams.  The framework suggests that the extent of divide between the two streams is determined by the extent of learning transfer.  The learning transfer is affected by the learning barriers operating at the levels of learning (i.e., individuals, groups, and organizational).  This led me to my second research question Q2:  What are these barriers to learning transfer and how do they impact the levels of learning in the organization?   I cumulated the dispersed literature on learning barriers, and synthesized the learning barriers into five key dimensions: Intrapersonal, relational, cultural, structural, and societal.  I then used the Delphi technique on 17 individuals to investigate the impact of the learning barriers on the levels of learning.  This generated two additional research questions.  The third research question was Q3:  How do individuals initiate a double-loop change?   This deals with the little researched area of initiation of double-loop change whilst engaging with the interfaces at the levels of learning.  I used multiple case studies to examine this question and found that individuals transit through four distinct stages when initiating double-loop change: 'embedded', 'embedded discomfited', 'scripted', and 'unscripted'.  Once double-loop learning has been initiated at the individual level, it is important that it is transferred across the organization.  Therefore, my fourth research question was Q4:  How does a new shared understanding for a double-loop change develop across the organization?   I did an in-depth, single case based investigation of an organization.  Using Identity and Complexity theory perspectives, I tracked the evolving new shared understanding through four phases: de-identification phase, situated re-identification phase, transition phase, and identification with core ideology phase.

The key insights from examining these research questions, particularly insights from examining Q3 and Q4, enabled me to suggest nine key organizational interventions necessary to overcome the learning barriers and develop a learning organization:  Identifying, developing, and dispersing double-loop mastery; Enabling constructive contradictions; Creating a superordinate organizational identity; Building emotional intelligence (in individuals and groups); Ambidextrous leadership; Strategic support for experimentation; Promoting 'systems doing'; Accessibility of valid information; Institutionalizing scanning across industry boundaries.  When these nine organizational interventions are implemented, they produce five new learning organization orientations: genetic diversity, organizational ideology, organizational dualism, organizational coupling, and strategic play.  These five new learning organizational orientations provide the archetypes of the learning organization.  I then developed an instrument to assess these five new orientations, and did a preliminary testing of the instrument.

While aspects of my work overlaid with previous knowledge, new advances in knowledge were established by:

  Postulating a link between the streams of organizational learning and learning organization
  Synthesizing learning barriers into the five key dimensions, and investigating their impact on the levels of learning
  Understanding the stages of double-loop learning initiation by an individual, whilst engaging with the interfaces at the levels of learning
  Understanding the process of a new shared understanding evolving
  Postulating five new orientations of the learning organization

year: 2006

title: Challenges to legal education: The Waikato Law School experience

authors: Wilson, Margaret

subjects: legal education

abstract: The experiences of the Waikato Law School especially from the viewpoint of challenges to legal education are discussed. The impact of the performance-based model of funding on the delivery of legal education at the Waikato Law School is highlighted.

text: Challenges to legal education: The Waikato Law School experience The experiences of the Waikato Law School especially from the viewpoint of challenges to legal education are discussed. The impact of the performance-based model of funding on the delivery of legal education at the Waikato Law School is highlighted.

year: 2010