{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "Diversity and Inclusion Metrics", "provenance": [], "collapsed_sections": [ "JndnmDMp66FL", "quH3R39RRPqZ", "y6k4Nya6cvti" ] }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "JndnmDMp66FL" }, "source": [ "##### Copyright 2021 Google LLC.\n", "\n", "Licensed under the Apache License, Version 2.0 (the \"License\");" ] }, { "cell_type": "code", "metadata": { "id": "hMqWDc_m6rUC", "cellView": "form" }, "source": [ "#@title License information\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "47oyEnHoFCTp" }, "source": [ "\n", "\n", "# _Diversity and Inclusion Metrics in Subset Selection_ Demo\n", "\n", "\n", "\n", "The following notebook is an example implementation of a very simple way one might define and implement diversity and inclusion scoring for images of people.\n", "\n", "This is based on the paper [_Diversity and Inclusion Metrics in Subset Selection_](https://arxiv.org/pdf/2002.03256.pdf) and has a corresponding [PAIR explorable](https://pair.withgoogle.com/explorables/measuring-diversity/).\n", "\n", "Written by Dylan Baker." ] }, { "cell_type": "code", "metadata": { "id": "LZ9A6Ia3rZGJ", "cellView": "form" }, "source": [ "#@title Imports\n", "\n", "import numpy as np\n", "from collections import defaultdict\n", "import functools\n", "from PIL import Image, ImageDraw, ImageFont\n", "import matplotlib.pyplot as plt\n", "import attr\n", "import random\n", "from IPython.display import Image as ipython_image\n", "from IPython.display import HTML\n", "from six.moves import reload_module" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "vxFdq1AsH4ds", "cellView": "form" }, "source": [ "#@title Utility functions\n", "\n", "\"\"\"Utils for Diversity and Inclusion metrics colab notebook.\"\"\"\n", "\n", "import abc\n", "import six\n", "\n", "\n", "class Entity(object):\n", " \"\"\"An entity for scoring using diversity and inclusion metrics.\n", "\n", " Entities have properties, which are the axes along which diversity and\n", " inclusion are scored.\n", " \"\"\"\n", "\n", " def __init__(self, properties, title=''):\n", " self.properties = properties\n", " self._title = title\n", "\n", " def __repr__(self):\n", " return self._title + ' '.join(\n", " ['%s: %s' % (key, value) for key, value in self.properties.items()])\n", "\n", "\n", "class EntityImage(object):\n", " \"\"\"An image of an entity.\n", "\n", " An image can have properties that are separate from the properties of the\n", " entities.\n", " \"\"\"\n", "\n", " def __init__(self, entities, image_fetcher, properties=None):\n", " self.entities = entities\n", " self.properties = properties\n", " self._image_fetcher = image_fetcher\n", "\n", " def show(self):\n", " self._image_fetcher.fetch()\n", "\n", "\n", "@six.add_metaclass(abc.ABCMeta)\n", "class ImageFetcher(object):\n", " \"\"\"Fetches an image of an entity for display.\"\"\"\n", "\n", " @abc.abstractmethod\n", " def __init__(self):\n", " \"\"\"Initialize ImageFetcher with input parameters.\"\"\"\n", " pass\n", "\n", " @abc.abstractmethod\n", " def fetch(self):\n", " \"\"\"Fetch and display the image itself.\"\"\"\n", " pass\n" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "QEwmZUjQHtfW", "cellView": "form" }, "source": [ "#@title Enums\n", "\n", "# Lint as: python3\n", "\"\"\"Enums to be used in a demo colab notebook implementing D&I metrics.\"\"\"\n", "import enum\n", "\n", "\n", "def get_enum_values(enum_class):\n", " \"\"\"Helper function to get all possible values of an Enum.\n", "\n", " Args:\n", " enum_class: A class that inherits from enum.Enum\n", "\n", " Returns:\n", " Set containing all enum values in input class.\n", " \"\"\"\n", " return set(\n", " [enum_value for _, enum_value in list(enum_class.__members__.items())])\n", "\n", "\n", "class SkinType(enum.Enum):\n", " UNKNOWN = 0\n", " TYPE_1 = 1\n", " TYPE_2 = 2\n", " TYPE_3 = 3\n", " TYPE_4 = 4\n", " TYPE_5 = 5\n", " TYPE_6 = 6\n", "\n", "\n", "# Choosing presentation over gender to better line up with infering\n", "# this from photos.\n", "class GenderPresentation(enum.Enum):\n", " UNKNOWN = 0\n", " FEMININE = 1\n", " MASCULINE = 2\n", " # Intended to be a very broad umbrella.\n", " ANDROGYNOUS = 3\n", "\n", "\n", "# Using wide buckets because it's very hard to infer from photos.\n", "class Age(enum.Enum):\n", " UNKNOWN = 0\n", " RANGE_0_17 = 1\n", " RANGE_18_24 = 2\n", " RANGE_25_44 = 3\n", " RANGE_45_64 = 4\n", " RANGE_65_OVER = 5\n", "\n", "\n", "class Color(enum.Enum):\n", " RED = 1\n", " BLUE = 2\n", " GREEN = 3\n", "\n", "\n", "class Size(enum.Enum):\n", " SMALL = 1\n", " MEDIUM = 2\n", " LARGE = 3\n", "\n", "\n", "class ShapeType(enum.Enum):\n", " TRIANGLE = 1\n", " SQUARE = 2\n", " CIRCLE = 3\n", "\n", "\n", "def age_to_enum(age):\n", " \"\"\"Parses age integer into Age enum class.\"\"\"\n", " if age <= 17:\n", " return Age.RANGE_0_17\n", " elif age <= 24:\n", " return Age.RANGE_18_24\n", " elif age <= 44:\n", " return Age.RANGE_25_44\n", " elif age <= 64:\n", " return Age.RANGE_45_64\n", " elif age >= 65:\n", " return Age.RANGE_65_OVER\n", " else:\n", " raise ValueError('Unable to parse age: ', age)\n", "\n", "\n", "def gender_to_enum(gender):\n", " \"\"\"Parses string to GenderPresentation enum.\"\"\"\n", " if gender.lower() == 'masculine':\n", " return GenderPresentation.MASCULINE\n", " elif gender.lower() == 'feminine':\n", " return GenderPresentation.FEMININE\n", " elif gender.lower() == 'androgynous':\n", " return GenderPresentation.ANDROGYNOUS\n", " else:\n", " raise ValueError('Unable to parse gender: ', gender)\n", "\n", "\n", "def skin_type_to_enum(skin_type):\n", " \"\"\"Parses string to SkinType enum.\"\"\"\n", " return SkinType(skin_type)" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "MgF0jpWsHxke", "cellView": "form" }, "source": [ "#@title Display\n", "\n", "\"\"\"Display functions for Diversity & Inclusion Metrics Explorable Colab.\"\"\"\n", "\n", "import random\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "from PIL import Image\n", "from PIL import ImageDraw\n", "from PIL import ImageFont\n", "\n", "# Default canvas size for drawing shape images.\n", "_DEFAULT_HEIGHT, _DEFAULT_WIDTH = 150, 250\n", "\n", "# Map Sizes to the edge size of a tight square bounding box around a shape.\n", "_SIZE_TO_WIDTH = {\n", " Size.SMALL: _DEFAULT_WIDTH / 10,\n", " Size.MEDIUM: _DEFAULT_WIDTH / 5,\n", " Size.LARGE: _DEFAULT_WIDTH / 3\n", "}\n", "\n", "# Padding around canvas edge to avoid too much spillover off of the canvas.\n", "_BORDER = _DEFAULT_WIDTH / 10\n", "\n", "# Map the color enums to some nicer matplotlib colors.\n", "_COLOR_TO_MATPLOTLIB = {\n", " Color.RED: 'firebrick',\n", " Color.BLUE: 'skyblue',\n", " Color.GREEN: 'forestgreen',\n", "}\n", "\n", "\n", "def _get_bbox(center, size):\n", " \"\"\"Get a bounding box for a shape given a center point and a Size.\"\"\"\n", " center_x, center_y = center\n", " shape_edge_size = _SIZE_TO_WIDTH[size]\n", " left = int(center_x - 0.5 * shape_edge_size)\n", " right = int(center_x + 0.5 * shape_edge_size)\n", " top = int(center_y - 0.5 * shape_edge_size)\n", " bottom = int(center_y + 0.5 * shape_edge_size)\n", " return left, right, bottom, top\n", "\n", "\n", "def _get_random_center():\n", " return (random.randint(_BORDER, _DEFAULT_WIDTH - _BORDER),\n", " random.randint(_BORDER, _DEFAULT_HEIGHT - _BORDER))\n", "\n", "\n", "def _draw_triangle(draw,\n", " size=Size.SMALL,\n", " color=Color.BLUE,\n", " center=None):\n", " if not center:\n", " center = _get_random_center()\n", " left, right, bottom, top = _get_bbox(center, size)\n", " center_x, unused_center_y = center\n", " draw.polygon([(left, bottom), (center_x, top), (right, bottom)],\n", " fill=color,\n", " outline='black')\n", "\n", "\n", "def _draw_square(draw,\n", " size=Size.SMALL,\n", " color=Color.BLUE,\n", " center=None):\n", " if not center:\n", " center = _get_random_center()\n", " left, right, bottom, top = _get_bbox(center, size)\n", " draw.rectangle([(left, top), (right, bottom)],\n", " fill=color,\n", " outline='black',\n", " width=1)\n", "\n", "\n", "def _draw_circle(draw,\n", " size=Size.SMALL,\n", " color=Color.BLUE,\n", " center=None):\n", " if not center:\n", " center = _get_random_center()\n", " left, right, bottom, top = _get_bbox(center, size)\n", " draw.ellipse([(left, top), (right, bottom)],\n", " fill=color,\n", " outline='black',\n", " width=1)\n", "\n", "\n", "def _draw_one_shape(draw, shape, center):\n", " \"\"\"Draws the given shape with a given center point.\n", "\n", " Args:\n", " draw: ImageDraw.Draw object.\n", " shape: The input shape entity.\n", " center: (x,y) center coordinates.\n", " \"\"\"\n", " if shape.properties[ShapeType] == ShapeType.TRIANGLE:\n", " _draw_triangle(\n", " draw,\n", " shape.properties[Size],\n", " _COLOR_TO_MATPLOTLIB[shape.properties[Color]],\n", " center=center)\n", " if shape.properties[ShapeType] == ShapeType.SQUARE:\n", " _draw_square(\n", " draw,\n", " shape.properties[Size],\n", " _COLOR_TO_MATPLOTLIB[shape.properties[Color]],\n", " center=center)\n", " if shape.properties[ShapeType] == ShapeType.CIRCLE:\n", " _draw_circle(\n", " draw,\n", " shape.properties[Size],\n", " _COLOR_TO_MATPLOTLIB[shape.properties[Color]],\n", " center=center)\n", "\n", "\n", "def _show_image_html_from_url(img_url, max_width=200):\n", " \"\"\"Show an image inline from a URL using colabtools HTML.\"\"\"\n", " img_html = ''.format(img_url, max_width)\n", " display(HTML(img_html))\n", "\n", "\n", "\n", "class PersonFetcher(ImageFetcher):\n", " \"\"\"A Fetcher for displaying an image loaded from a URL.\"\"\"\n", "\n", " def __init__(self, image_url):\n", " self._image_url = image_url\n", "\n", " def fetch(self):\n", " _show_image_html_from_url(self._image_url)\n", "\n", "\n", "class ShapeFetcher(ImageFetcher):\n", " \"\"\"A Fetcher for displaying shapes drawn with matplotlib.\"\"\"\n", "\n", " def __init__(self, shapes, title=None, shuffle=True):\n", " self._shapes = shapes\n", " self._title = title\n", " self._shuffle_shapes_when_displaying = shuffle\n", "\n", " def fetch(self):\n", " draw_shapes(\n", " self._shapes,\n", " title=self._title,\n", " shuffle=self._shuffle_shapes_when_displaying)\n", "\n", "\n", "def create_image_from_shapes(shapes, title=''):\n", " \"\"\"Returns an EntityImage from a list of input shapes.\n", "\n", " This is initialized with a ShapeFetcher, which is used to draw the image.\n", "\n", " Args:\n", " shapes: List of shape Entities.\n", " title: An optional title for the image, for display.\n", "\n", " Returns:\n", " Initialized EntityImage.\n", " \"\"\"\n", " return EntityImage(shapes, ShapeFetcher(shapes, title))\n", "\n", "\n", "def draw_shapes(shapes, title=None, shuffle=True):\n", " \"\"\"Draws shape entities on a new matplotlib plot.\n", "\n", " Example usage:\n", "\n", " my_shapes = [\n", " Entity(properties = {\n", " Color: Color.BLUE,\n", " Size: Size.LARGE,\n", " ShapeType: ShapeType.CIRCLE,\n", " })\n", " ]\n", "\n", " draw_shapes(my_shapes, 'A large blue circle')\n", "\n", " Args:\n", " shapes: List of input shape Entities. Each entity must have Color, Size, and\n", " ShapeType specified as properties.\n", " title: A title, to be shown at the top of the image.\n", " shuffle: Whether or not to shuffle the order of the shapes when drawing.\n", " \"\"\"\n", " canvas = Image.fromarray(np.ones([_DEFAULT_HEIGHT, _DEFAULT_WIDTH]) *\n", " 240).convert('RGBA')\n", " draw = ImageDraw.Draw(canvas)\n", " center_x_coords = np.linspace(_BORDER, _DEFAULT_WIDTH - _BORDER,\n", " len(shapes) + 2)[1:-1]\n", "\n", " # Alternate upper and lower coordinates; this spaces out the shapes more\n", " # nicely.\n", " center_y_values = np.linspace(_BORDER, _DEFAULT_HEIGHT - _BORDER, 4)[1:-1]\n", " center_y_coords = [center_y_values[i % 2] for i in range(len(shapes))]\n", " centers = zip(center_x_coords, center_y_coords)\n", " if shuffle:\n", " random.shuffle(shapes)\n", " for shape, center in zip(shapes, centers):\n", " _draw_one_shape(draw, shape, center)\n", "\n", " if title:\n", " draw.text((10, 10), title, (0, 0, 0))\n", "\n", " plt.imshow(canvas, aspect='equal')\n", " plt.axis('off')\n", " plt.grid(False)\n", " plt.show()" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "UJpQ8y71H1-r", "cellView": "form" }, "source": [ "#@title Scoring\n", "\n", "\"\"\"Scoring functions for diversity and inclusion demo.\"\"\"\n", "\n", "import numpy as np\n", "import collections\n", "\n", "# Using strings here so we're able to easily check for membership in each set;\n", "# EnumMeta doesn't support comparison.\n", "# These are properties where closer enum values correspond to actual similarity.\n", "_CLOSENESS_PROPERTIES = frozenset(\n", " [str(Age), str(SkinType),\n", " str(Size)])\n", "# These are properties where enum values are independent.\n", "_MATCH_PROPERTIES = frozenset(\n", " [str(GenderPresentation),\n", " str(Color),\n", " str(ShapeType)])\n", "\n", "\n", "def _to_aggregator_fn(aggregator):\n", " \"\"\"Maps from name to aggregator function.\n", "\n", " Args:\n", " aggregator: Either a name (e.g. 'average'), or for the weighted average\n", " case, a dictionary mapping category names (e.g. Color) to weights\n", " (e.g. 0.4).\n", "\n", " Returns:\n", " A function that takes a {category name, value} dictionary and returns a\n", " float.\n", " \"\"\"\n", "\n", " # If this is a dict, we're using it to compute weighted averages.\n", " def compute_weighted_average(weights_dict):\n", " return sum([value * aggregator[key] for key, value in weights_dict.items()])\n", "\n", " if isinstance(aggregator, dict):\n", " return compute_weighted_average\n", "\n", " score_fn = None\n", " if aggregator == 'average':\n", " score_fn = np.average\n", " elif aggregator == 'min':\n", " score_fn = min\n", " elif aggregator == 'max':\n", " score_fn = max\n", " elif aggregator == 'nash':\n", " score_fn = nash_score\n", " elif aggregator == 'utilitarian':\n", " score_fn = utilitarian_score\n", " elif aggregator == 'egalitarian':\n", " score_fn = egalitarian_score\n", " else:\n", " raise NotImplementedError('Aggregation method %s not implemented' %\n", " aggregator)\n", "\n", " # Just return a function that scores each value in the input score dict.\n", " return (lambda score_dict: score_fn(\n", " [value for unused_key, value in score_dict.items()]))\n", "\n", "\n", "def nash_score(subscores):\n", " \"\"\"Compute Nash inclusivity score.\"\"\"\n", " root = len(subscores)\n", " return np.prod(subscores)**(1 / root)\n", "\n", "\n", "def utilitarian_score(subscores):\n", " \"\"\"Compute Utilitarian inclusivity score.\"\"\"\n", " return sum(subscores) / len(subscores)\n", "\n", "\n", "def egalitarian_score(subscores):\n", " \"\"\"Compute Egalitarian (maximin) inclusivity score.\"\"\"\n", " return min(subscores)\n", "\n", "\n", "# We have a separate comparison function here because we need\n", "# all of the subscores for tiebreaking.\n", "def egalitarian_compare(subscores_a, subscores_b):\n", " \"\"\"Returns 1 if subscores_a > subscores_b, 0 if they are equal, -1 otherwise.\"\"\"\n", " if len(subscores_a) != len(subscores_b):\n", " raise ValueError('Subscores must have the same length.')\n", " print(subscores_a, subscores_b)\n", " subscores_a.sort()\n", " subscores_b.sort()\n", " print(subscores_a, subscores_b)\n", " for first_a, first_b in zip(subscores_a, subscores_b):\n", " if first_a == first_b:\n", " continue\n", " else:\n", " return 1 if first_a > first_b else -1\n", " # Return 0 if they're the same.\n", " return 0\n", "\n", "\n", "class Scorer(object):\n", " \"\"\"An object that scores and ranks images or sets of images.\n", "\n", " Scoring is dependent on the person making a query and the query they made.\n", " \"\"\"\n", "\n", " def __init__(self,\n", " viewer,\n", " diversity_fn=None,\n", " inclusion_fn=None,\n", " combiner=None):\n", " \"\"\"Initializes a scorer.\n", "\n", " Args:\n", " viewer: A Viewer to serve as a reference point when scoring results.\n", " diversity_fn: A DiversityFn, which calculates per-image diversity scores.\n", " inclusion_fn: An InclusionFn, which calculates per-image inclusion scores.\n", " combiner: A Combiner, which combines inclusion and diversity scores.\n", " \"\"\"\n", " self.viewer = viewer\n", " self.diversity_fn = diversity_fn\n", " self.inclusion_fn = inclusion_fn\n", " self.combiner = combiner\n", "\n", " def get_image_subscores(self, image, query='', verbose_output=False):\n", " \"\"\"Computes diversity and inclusion subscores for a single image.\n", "\n", " Args:\n", " image: An EntityImage\n", " query: The query, used for computing relevance inclusion subscore.\n", " verbose_output: Whether or not to print out all computation steps.\n", "\n", " Returns:\n", " Dictionaries mapping diversity and inclusion subscore names to values.\n", " \"\"\"\n", " if not self.diversity_fn or not self.inclusion_fn or not self.combiner:\n", " raise ValueError('Diversity, inclusion, and combination functions '\n", " 'must all be specified. (%s, %s, %s)' %\n", " (self.diversity_fn, self.inclusion_fn, self.combiner))\n", " if verbose_output:\n", " print('\\n===================')\n", " print('===== Scoring =====')\n", " print('===================\\n')\n", " image.show()\n", " print(self.viewer)\n", " if query:\n", " print('Query:', query)\n", " diversity_scores = self.diversity_fn.score_image(\n", " image, verbose_output=verbose_output)\n", " inclusion_scores = self.inclusion_fn.score_image(\n", " viewer=self.viewer,\n", " image=image,\n", " query=query,\n", " verbose_output=verbose_output)\n", " return diversity_scores, inclusion_scores\n", "\n", " def score_image(self, image, query='', verbose_output=False):\n", " \"\"\"Computes diversity and inclusion scores for a single image.\n", "\n", " Args:\n", " image: An EntityImage\n", " query: The query, used for computing relevance inclusion subscore.\n", " verbose_output: Whether or not to print out all computation steps.\n", "\n", " Returns:\n", " A single net score for the image, float.\n", " \"\"\"\n", " diversity_scores, inclusion_scores = self.get_image_subscores(\n", " image, query=query, verbose_output=verbose_output)\n", "\n", " net_score = self.combiner.combine(\n", " diversity_scores=diversity_scores,\n", " inclusion_scores=inclusion_scores,\n", " aggregation_level='image',\n", " verbose_output=verbose_output)\n", "\n", " return net_score\n", "\n", " def rank_images(self, images, query='', verbose_output=False):\n", " \"\"\"Rank a list of images using their net diversity and inclusion scores.\n", "\n", " Args:\n", " images: A list of EntityImages.\n", " query: The query, used for computing relevance inclusion subscore.\n", " verbose_output: Whether or not to print out final ranking.\n", "\n", " Returns:\n", " A sorted list of (image, score) tuples.\n", " \"\"\"\n", " # In this case, verbose_output refers to whether or not you will print out\n", " # the ranking itself.\n", " if self.combiner.diversity_aggregate_name == 'egalitarian':\n", " # True egalitarian ranking would give us separate diversity & inclusion\n", " # rankings.\n", " logging.warn('Warning: egalitarian tiebreaking not implemented.')\n", "\n", " # Don't use verbose_output for the scoring itself, only for the ranking.\n", " images_with_scores = [(image,\n", " self.score_image(image, query, verbose_output=False))\n", " for image in images]\n", " images_with_scores.sort(key=lambda x: x[1], reverse=True)\n", " if verbose_output:\n", " print('===================')\n", " print('===== Ranking =====')\n", " print('===================')\n", " print(self.viewer)\n", " print('Query: %s' % query)\n", " print('\\n')\n", " for index, (image, score) in enumerate(images_with_scores):\n", " print('==== Ranking %d [score: %s] ====' %\n", " (index + 1, np.around(score, 3)))\n", " image.show()\n", " print('\\n')\n", " return images_with_scores\n", "\n", " def get_set_subscores(self, images, query='', verbose_output=False):\n", " \"\"\"Score a set of images.\"\"\"\n", "\n", " # Dictionaries mapping image names to their diversity/inclusion scores.\n", " diversity_per_image_subscores, inclusion_per_image_subscores = {}, {}\n", "\n", " for index, image in enumerate(images):\n", " image_name = 'image %d' % (index + 1)\n", " if verbose_output:\n", " print('\\nScoring %s\\n' % image_name)\n", " diversity_subscore, inclusion_subscore = self.get_image_subscores(\n", " image, query=query, verbose_output=verbose_output)\n", "\n", " # Aggregate across entities in each image to get the net\n", " # diversity/inclusion score for that image.\n", " diversity_per_image_subscores[\n", " image_name] = self.combiner.aggregate_diversity_scores(\n", " diversity_subscore)\n", " inclusion_per_image_subscores[\n", " image_name] = self.combiner.aggregate_inclusion_scores(\n", " inclusion_subscore)\n", "\n", " return diversity_per_image_subscores, inclusion_per_image_subscores\n", "\n", " def score_set(self, images, query='', verbose_output=False):\n", " \"\"\"Score a set of images.\n", "\n", " Args:\n", " images: A list of EntityImages.\n", " query: The query, used for computing relevance inclusion subscore.\n", " verbose_output: Whether or not to print out all computation steps.\n", "\n", " Returns:\n", " A single net score for the image set, float.\n", " \"\"\"\n", " diversity_per_image_subscores, inclusion_per_image_subscores = self.get_set_subscores(\n", " images=images, query=query, verbose_output=verbose_output)\n", " net_score = self.combiner.combine(\n", " diversity_scores=diversity_per_image_subscores,\n", " inclusion_scores=inclusion_per_image_subscores,\n", " aggregation_level='set',\n", " verbose_output=verbose_output)\n", "\n", " return net_score\n", "\n", "\n", "class Combiner(object):\n", " \"\"\"An object that combines diversity and inclusion scores.\n", "\n", " This applies to both individual images and sets of images.\n", " \"\"\"\n", "\n", " def __init__(self,\n", " diversity_weight=0.5,\n", " inclusion_weight=None,\n", " diversity_aggregate='average',\n", " inclusion_aggregate='average',\n", " set_diversity_aggregate='average',\n", " set_inclusion_aggregate='average'):\n", " \"\"\"Initializes Combiner.\n", "\n", " A Combiner combines all of the diversity and inclusion subscores using\n", " aggregator functions specified in the constructor.\n", "\n", " Args:\n", " diversity_weight: In computing a final score, amount to weight the net\n", " diversity score vs. inclusion score.\n", " inclusion_weight: In computing a final score, amount to weight the net\n", " inclusion score vs. diversity score.\n", " diversity_aggregate: How to aggregate diversity subscores across entities\n", " within one image. Can either be a string indicating a type to apply for\n", " all subscores (e.g. 'average') or a function that computes the average.\n", " inclusion_aggregate: How to aggregate inclusion subscores across entities\n", " within one image. Can either be a string indicating a type to apply for\n", " all subscores (e.g. 'average') or a function that computes the average.\n", " set_diversity_aggregate: How to aggregate diversity subscores across\n", " images in a set.\n", " set_inclusion_aggregate: How to aggregate inclusion subscores across\n", " images in a set.\n", " \"\"\"\n", " self.diversity_weight = diversity_weight\n", " self.inclusion_weight = inclusion_weight\n", "\n", " # Normalize diversity_weight and inclusion_weight to sum to 1.\n", " self._normalize_weights()\n", "\n", " self.diversity_aggregate_name = (\n", " diversity_aggregate\n", " if isinstance(diversity_aggregate, str) else 'weighed_average')\n", " self.inclusion_aggregate_name = (\n", " inclusion_aggregate\n", " if isinstance(inclusion_aggregate, str) else 'weighed_average')\n", " self.set_diversity_aggregate_name = (\n", " set_diversity_aggregate\n", " if isinstance(set_diversity_aggregate, str) else 'weighed_average')\n", " self.set_inclusion_aggregate_name = (\n", " set_inclusion_aggregate\n", " if isinstance(set_inclusion_aggregate, str) else 'weighed_average')\n", "\n", " self._diversity_agg_fn = _to_aggregator_fn(diversity_aggregate)\n", " self._inclusion_agg_fn = _to_aggregator_fn(inclusion_aggregate)\n", " self._set_diversity_agg_fn = _to_aggregator_fn(set_diversity_aggregate)\n", " self._set_inclusion_agg_fn = _to_aggregator_fn(set_inclusion_aggregate)\n", "\n", " def combine(self, diversity_scores, inclusion_scores, aggregation_level,\n", " verbose_output):\n", " \"\"\"Combines raw diversity and inclusion scores for an image or image set.\n", "\n", " More specifically, this aggregates diversity and inclusion scores from an\n", " input image and computes a weighted average of those aggregates.\n", "\n", " The method for computing these aggregates, as well as the relative weights\n", " for diversity and inclusion scores, are set when we construct a Combiner.\n", "\n", " Args:\n", " diversity_scores: Dictionary mapping diversity subscore names to values.\n", " inclusion_scores: Dictionary mapping inclusion subscore names to values.\n", " aggregation_level: What we are combining across, either 'image' or 'set'.\n", " verbose_output: Whether or not to log all of the computation steps.\n", "\n", " Returns:\n", " A single aggregated score for the image or image set, float.\n", " \"\"\"\n", "\n", " # Normalize weights so they sum to 1.\n", " self._normalize_weights()\n", "\n", " # Aggregate per-attribute diversity subscores into per-image scores.\n", " image_diversity_score = self.aggregate_scores(\n", " diversity_scores,\n", " diversity_or_inclusion='diversity',\n", " aggregation_level=aggregation_level,\n", " verbose_output=verbose_output)\n", " # Aggregate per-attribute inclusion subscores into per-image scores.\n", " image_inclusion_score = self.aggregate_scores(\n", " inclusion_scores,\n", " diversity_or_inclusion='inclusion',\n", " aggregation_level=aggregation_level,\n", " verbose_output=verbose_output)\n", " # Calculate the actual weighted average.\n", " score = ((self.diversity_weight * image_diversity_score) +\n", " (self.inclusion_weight * image_inclusion_score))\n", "\n", " if verbose_output:\n", " print()\n", " print('Combining diversity and inclusion scores at the %s level:' %\n", " aggregation_level)\n", " print(' %s * %s +' %\n", " (self.diversity_weight, np.around(image_diversity_score, 3)))\n", " print(' %s * %s =\\n' %\n", " (self.inclusion_weight, np.around(image_inclusion_score, 3)))\n", "\n", " print(' -------------------')\n", " print(' | Final score: %s |' % np.around(score, 3))\n", " print(' -------------------')\n", "\n", " return score\n", "\n", " def _get_aggregator_and_name(self, diversity_or_inclusion, aggregation_level):\n", " \"\"\"Returns specified aggregation function function and name.\"\"\"\n", "\n", " if diversity_or_inclusion == 'diversity':\n", " if aggregation_level == 'set':\n", " return self.set_diversity_aggregate_name, self._set_diversity_agg_fn\n", " elif aggregation_level == 'image':\n", " return self.diversity_aggregate_name, self._diversity_agg_fn\n", "\n", " elif diversity_or_inclusion == 'inclusion':\n", " if aggregation_level == 'set':\n", " return self.set_inclusion_aggregate_name, self._set_inclusion_agg_fn\n", " elif aggregation_level == 'image':\n", " return self.inclusion_aggregate_name, self._inclusion_agg_fn\n", "\n", " raise ValueError('No aggregator specified for %s, %s' %\n", " (diversity_or_inclusion, aggregation_level))\n", "\n", " def aggregate_scores(self,\n", " scores,\n", " diversity_or_inclusion=None,\n", " aggregation_level=None,\n", " verbose_output=False):\n", " \"\"\"Aggregate all scores using aggretation fn.\n", "\n", " Args:\n", " scores: Dictionary mapping subscore name to subscore value.\n", " diversity_or_inclusion: Which score you're aggregating (e.g. 'diversity').\n", " aggregation_level: Level at which you're aggregating (e.g. 'set', 'image')\n", " verbose_output: Whether or not to log all of the computation steps.\n", "\n", " Returns:\n", " Aggregated score, float.\n", " \"\"\"\n", "\n", " aggregation_fn_name, aggregation_fn = self._get_aggregator_and_name(\n", " diversity_or_inclusion, aggregation_level)\n", "\n", " total = aggregation_fn(scores)\n", " if verbose_output:\n", " print()\n", " print('Aggregating %s-level %s scores with method: %s' %\n", " (aggregation_level, diversity_or_inclusion, aggregation_fn_name))\n", " print('--> Net %s-level %s score: %s' %\n", " (aggregation_level, diversity_or_inclusion, np.around(total, 3)))\n", " return total\n", "\n", " def aggregate_diversity_scores(self, diversity_scores, verbose_output=False):\n", " return self.aggregate_scores(\n", " diversity_scores,\n", " diversity_or_inclusion='diversity',\n", " aggregation_level='image',\n", " verbose_output=verbose_output)\n", "\n", " def aggregate_inclusion_scores(self, inclusion_scores, verbose_output=False):\n", " return self.aggregate_scores(\n", " inclusion_scores,\n", " diversity_or_inclusion='inclusion',\n", " aggregation_level='image',\n", " verbose_output=verbose_output)\n", "\n", " def _normalize_weights(self):\n", " if not self.inclusion_weight:\n", " if (self.diversity_weight < 0.0 or self.diversity_weight > 1.0):\n", " raise ValueError(\n", " 'Weights should be between 0 and 1 if only one is specified')\n", " self.inclusion_weight = np.around(1.0 - self.diversity_weight, 3)\n", " else:\n", " total = sum([self.diversity_weight, self.inclusion_weight])\n", " self.diversity_weight = self.diversity_weight / total\n", " self.inclusion_weight = self.inclusion_weight / total\n", "\n", "\n", "def _to_integer_boolean(input_dict):\n", " \"\"\"Replaces each value in a dict with a 1 if it is truthy, 0 otherwise.\n", "\n", " This is useful for converting counts to simple presence/absence of a property.\n", "\n", " Args:\n", " input_dict: An arbitrary dictionary.\n", "\n", " Returns:\n", " A dictionary mapping each input key to either 1 or 0.\n", " \"\"\"\n", " return {key: int(bool(value)) for key, value in input_dict.items()}\n", "\n", "\n", "class DiversityFn(object):\n", " \"\"\"A function that computes diversity subscores for an EntityImage.\"\"\"\n", "\n", " def __init__(self,\n", " aggregate_method='average',\n", " count_or_binary='binary',\n", " verbose_output=False):\n", " self.aggregate_method = aggregate_method\n", "\n", " # Whether or not to take into account the number of instances of a property\n", " # or simply take into account whether or not the property was present at\n", " # all.\n", " self.count_or_binary = count_or_binary\n", " self._agg_fn = _to_aggregator_fn(aggregate_method)\n", " self._verbose_output = verbose_output\n", "\n", " def score_image(self, image, verbose_output=None):\n", " \"\"\"Computes all diversity subscores for an image.\n", "\n", " Args:\n", " image: An EntityImage.\n", " verbose_output: Whether or not to print out computation steps.\n", "\n", " Returns:\n", " Dictionary mapping subscore names to values.\n", " \"\"\"\n", " # Allow overriding default verbosity\n", " if verbose_output is not None:\n", " self._verbose_output = verbose_output\n", "\n", " if self._verbose_output:\n", " print('\\n===================================')\n", " print('Diversity subscores (single image):')\n", " print('===================================\\n')\n", " print(' - Aggregating scores across entities in image using method: %s' %\n", " self.aggregate_method)\n", " print(' - Using counts or simple binary presence/absence? %s' %\n", " self.count_or_binary)\n", "\n", " scores = self._get_scores(image)\n", "\n", " if self._verbose_output:\n", " if len(image.entities) == 1:\n", " print('Note: this image contains a single entity, so diversity is '\n", " 'low by definition.')\n", " return scores\n", "\n", " def _get_scores(self, image):\n", " \"\"\"Gets property-level scores for a single image.\n", "\n", " Args:\n", " image: An EntityImage containing one or more Entities\n", "\n", " Returns:\n", " Dictionary mapping subscore names to values.\n", " \"\"\"\n", " output_string = ''\n", "\n", " # Compute presence vectors as well as a dictionary mapping the property\n", " # to a debug string, constructed when vector was being computed. This stores\n", " # the specific enum value counts (e.g. Color.BLUE : 2), which are not\n", " # preserved in the presence vectors (e.g. [0, 2, 0]) themselves.\n", " presence_vectors, presence_string_dict = self._get_presence_vectors(image)\n", "\n", " # Filter into a dictionary with 1s if property was present and 0s otherwise.\n", " if self.count_or_binary == 'binary':\n", " presence_vectors = {\n", " key: _to_integer_boolean(value)\n", " for key, value in presence_vectors.items()\n", " }\n", "\n", " # Aggregate the per-enum-category vectors.\n", " aggregated = {\n", " key: self._agg_fn(presence_vector)\n", " for key, presence_vector in presence_vectors.items()\n", " }\n", "\n", " # Generate output string for verbose output.\n", " for key, agg_score in aggregated.items():\n", " # String generated when we were calculating presence/absences scores\n", " presence_string = presence_string_dict[key]\n", " output_string += '\\n Which of the possible %ss are present?' % key\n", " output_string += '\\n --> %s subscore: %s' % (key, np.round(\n", " agg_score, 3))\n", " output_string += presence_string\n", " if self.count_or_binary == 'binary':\n", " if self.aggregate_method == 'average':\n", " output_string += ('\\n %s %ss represented / %s total '\n", " 'categories') % (\n", " str(sum(presence_vectors[key].values())), key,\n", " str(len(presence_vectors[key].values())))\n", " output_string += '\\n %s (binary vector): %s\\n' % (\n", " self.aggregate_method, np.round(agg_score, 3))\n", " else:\n", " output_string += '\\n %s: %s\\n' % (self.aggregate_method,\n", " np.round(agg_score, 3))\n", "\n", " if self._verbose_output:\n", " print(output_string)\n", "\n", " return aggregated\n", "\n", " def _get_presence_vectors(self, image):\n", " \"\"\"Gets vectors indicating presence/absence of properties in an image.\n", "\n", " Args:\n", " image: An EntityImage.\n", "\n", " Returns:\n", " A nested dictionary, where they keys are property names\n", " (e.g. age) and the values are dictionaries mapping all values of that\n", " property (e.g. Age.RANGE_18_24) to the number of entities with that\n", " property in each image.\n", " \"\"\"\n", "\n", " output_string_dict = dict()\n", "\n", " # Establish a set of all properties _any_ entities has, as well\n", " # as all possible values for that enum. E.g. if any entity has property\n", " # 'age', all_properties should contain the mapping:\n", " # { 'age' : {, ...} }\n", " all_properties = dict()\n", "\n", " # Dict mapping property name to all properties present in the image.\n", " present_properties = collections.defaultdict(list)\n", "\n", " for person in image.entities:\n", " for name, value in person.properties.items():\n", " # Overwrite each time, these are small and it's convenient.\n", " all_properties[name] = get_enum_values(type(value))\n", " present_properties[name].append(value)\n", "\n", " # Count occurrences of each value for each property.\n", " to_return = {}\n", " for property_name, all_enum_values in all_properties.items():\n", " output_string_dict[property_name] = ''\n", " presences = {}\n", " for enum_value in all_enum_values:\n", " output_string_dict[property_name] += '\\n %s: %s' % (\n", " enum_value, present_properties[property_name].count(enum_value))\n", " presences[enum_value] = present_properties[property_name].count(\n", " enum_value)\n", " to_return[property_name] = presences\n", "\n", " return to_return, output_string_dict\n", "\n", "\n", "class InclusionFn(object):\n", " \"\"\"A function that computes inclusion subscores for an EntityImage.\"\"\"\n", "\n", " def __init__(self, aggregate_method='max', verbose_output=False):\n", " \"\"\"Initializes InclusionFn.\n", "\n", " Args:\n", " aggregate_method: Method of combining scores of entities within an image.\n", " verbose_output: Whether or not to log all of the computation steps.\n", "\n", " Returns:\n", " Initialized InclusionFn.\n", " \"\"\"\n", " self.aggregate_method = aggregate_method\n", " self._agg_fn = _to_aggregator_fn(aggregate_method)\n", " self._verbose_output = verbose_output\n", " assert _CLOSENESS_PROPERTIES is not None\n", " assert _MATCH_PROPERTIES is not None\n", "\n", " def score_image(self, viewer, image, query='', verbose_output=None):\n", " \"\"\"Scores an input image.\n", "\n", " Args:\n", " viewer: An Entity representing the entity making a query.\n", " image: An EntityImage.\n", " query: A viewer's query related to the EntityImage.\n", " verbose_output: Whether or not to print out computation steps.\n", "\n", " Returns:\n", " A dictionary mapping inclusion subscore names to values.\n", " \"\"\"\n", "\n", " # TODO(dylanbaker): Implement query relevance scoring. At minimum, this\n", " # requires EntityImages or Entities having some way of denoting their\n", " # subject.\n", " # logging.warning('Query %s is unused', query)\n", "\n", " # Allow verbose_output to be overridden here; otherwise, defualt\n", " # to initial state.\n", " if verbose_output is None:\n", " verbose_output = self._verbose_output\n", " if verbose_output:\n", " print('\\n===================================')\n", " print('Inclusion subscores (single image)')\n", " print('===================================\\n')\n", " print(' - Using aggregation method: %s\\n' % self.aggregate_method)\n", "\n", " subscores = {}\n", " if not image.entities:\n", " logging.warning('No entities to score.')\n", " return {}\n", "\n", " all_property_names = [\n", " type(value) for value in list(image.entities[0].properties.values())\n", " ]\n", "\n", " for property_name in all_property_names:\n", " if str(property_name) in _CLOSENESS_PROPERTIES:\n", " subscores[property_name] = self._compute_closeness(\n", " viewer, image, property_name, verbose_output)\n", " elif str(property_name) in _MATCH_PROPERTIES:\n", " subscores[property_name] = self._compute_match(viewer, image,\n", " property_name,\n", " verbose_output)\n", " else:\n", " raise ValueError(\n", " 'Unsure how to calculate inclusion score for property:',\n", " property_name)\n", " return subscores\n", "\n", " def _compute_closeness(self, viewer, image, property_name, verbose_output):\n", " \"\"\"Compute the closeness score between bucketed properties.\n", "\n", " This assumes enum values that are close to one another are more similar\n", " (e.g. age buckets, skin type buckets)\n", "\n", " Returns a score in [0, 1], with 1 indicating the same property and 0\n", " being as far apart as possible.\n", "\n", " Args:\n", " viewer: An Entity representing the entity making a query.\n", " image: An EntityImage.\n", " property_name: Which property we're computing the closeness of.\n", " verbose_output: Whether or not to print out computation steps.\n", "\n", " Returns:\n", " A closeness score, float.\n", " \"\"\"\n", " output_string = ' Does anyone have a similar %s to mine? (%s)\\n' % (\n", " property_name, viewer.properties[property_name].name)\n", " output_string += ' --> bucketed {label} closeness subscore: %s'.format(\n", " label=property_name)\n", " per_entity_scores = {}\n", " for index, entity in enumerate(image.entities):\n", " n_buckets = len(\n", " # get_enum_values(type(entity.properties[property_name]))\n", " get_enum_values(property_name))\n", "\n", " per_entity_scores['entity_%d' %\n", " index] = (1.0 -\n", " (abs(viewer.properties[property_name].value -\n", " entity.properties[property_name].value) /\n", " (n_buckets - 1)))\n", " output_string += '\\n per-entity scores: %s' % [\n", " '%s = %s' % (name, np.around(value, 3))\n", " for name, value in per_entity_scores.items()\n", " ]\n", "\n", " aggregate_score = self._agg_fn(per_entity_scores)\n", " output_string += '\\n %s: %s\\n' % (self.aggregate_method,\n", " np.around(aggregate_score, 3))\n", " if verbose_output:\n", " print(output_string % np.around(aggregate_score, 3))\n", "\n", " return aggregate_score\n", "\n", " def _compute_match(self, viewer, image, property_name, verbose_output):\n", " \"\"\"Computes a score based on whether or not two properties match.\n", "\n", " This assumes enum values have no relation to each other\n", " (e.g. geometric shapes).\n", "\n", " Args:\n", " viewer: An Entity representing the entity making a query.\n", " image: An EntityImage.\n", " property_name: Which property we're computing the matched-ness of.\n", " verbose_output: Whether or not to print out computation steps.\n", "\n", " Returns:\n", " A match score, float.\n", " \"\"\"\n", " output_string = ' Does anyone have the same %s as me? (%s)\\n' % (\n", " property_name, viewer.properties[property_name].name)\n", " output_string += ' --> {label} match subscore: %s'.format(\n", " label=property_name)\n", "\n", " per_entity_scores = {}\n", "\n", " for index, entity in enumerate(image.entities):\n", " per_entity_scores['entity_%d' %\n", " index] = (1.0 if viewer.properties[property_name] ==\n", " entity.properties[property_name] else 0.0)\n", " output_string += '\\n per-entity scores: %s' % [\n", " '%s = %s' % (name, np.around(value, 3))\n", " for name, value in per_entity_scores.items()\n", " ]\n", "\n", " aggregate_score = self._agg_fn(per_entity_scores)\n", " output_string += '\\n %s: %s\\n' % (self.aggregate_method,\n", " np.around(aggregate_score, 3))\n", " if verbose_output:\n", " print(output_string % np.around(aggregate_score, 3))\n", "\n", " return aggregate_score" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "K4HeYAqmuuVx" }, "source": [ "# Diversity and Inclusion Metrics in Subset Selection\n", "\n", "When we hear the terms \"diversity\" and \"inclusion\", we often hear them grouped together and used in a very general way; maybe you've heard phrases like \"diversity and inclusion in the workplace\". When putting these terms into practice, however, it can be hard to pinpoint exactly what we _mean_ when we say \"diversity and inclusion\". Diverse with respect to what? Including whom?\n", " \n", "Often, the phrase \"diversity and inclusion\" is used to talk about improving the situations of excluded or marginalized people. For example, a prestigious university with very high tuition might find itself with a homogenous student population; only a narrow range of social groups have access to good high schools and enough money to pay tuition. The school might aim to “improve diversity” by offering scholarships to students outside of those social groups. In this case, the “diversity” that the scholarship is addressing is referring to “diversity with respect to wealth and resources”. Similarly, when thinking about “improving inclusion” for students at the school, it’s implicit that some students are disproportionately excluded in the first place.\n", " \n", "That is to say, “diversity” and “inclusion”, when acted out in the world, often have implications about social power and access.\n", " \n", "But what does that mean if you want to measure or compare the relative diversity and inclusion of things?\n", " \n", "We’re going to explore one way of answering that question using the example of **image subset selection**: that is, selecting some images from a larger set of images. We'll look at some different ways of **measuring which sets of images are the most diverse or inclusive,** taking into account some of these social components.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "E8wXo1ClO09p" }, "source": [ "# 1. Defining diversity and inclusion\n", "\n", "First, if we want to measure diversity or inclusion, we'll have to define them explicitly.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "EnSqK5z7zc8r" }, "source": [ "### Defining diversity\n", "\n", "Suppose we consider every entity in an image to be comprised of different attributes. We can score diversity by simply looking at how many different attributes are present in an image. We can also weigh attributes' relative importance to one another to take into account the fact that some attributes are more important than others in different contexts.\n", "\n", "In practice, it's critically important to consider _which_ attributes are chosen, by whom, and for what purpose. \n", "\n", "In this notebook, we'll start with with simple attributes: shapes and colors.\n", "\n", "(For a more rigorous definition of diversity, you may unfold the tab below)" ] }, { "cell_type": "markdown", "metadata": { "id": "quH3R39RRPqZ" }, "source": [ "### Defining diversity (very specifically)" ] }, { "cell_type": "markdown", "metadata": { "id": "SUSUG-h6WT32" }, "source": [ "**Presence Score**: An instance $x_q$ (e.g., a recommended\n", "movie in a set of movie recommendations) is composed of one or\n", "more items (e.g., actors, objects, and settings in the movie). Each\n", "item reflects or indexes different attributes.\n", "\n", "We define the presence score of an attribute a as a function\n", "quantifying how close the presence $a(x_q)$ is to the target and upper\n", "and lower bounds on the attribute’s presence:\n", "\n", "$$\\text{Presence}_a(x_q) = f(a(x_q),l_a,u_a)$$\n", "\n", "with higher values meaning a is more present in $x_q$.\n", "\n", "**Overall Diversity Score**: With the presence score defined, we can now\n", "define the diversity of an instance $x_q$ as an aggregate statistic of\n", "the attributes in the instance:\n", "\n", "$$\\text{Diversity}_a(x_q) = g(\\text{Presence}_a(x_q)), \\text{across } a \\in A$$\n", "where $g(·)$ can return the minimum, maximum, or average presence value\n", "of the attributes\n", "\n", "The Diversity family of metrics can highlight or prioritize diversity with respect to relevant social groups. For example:\n", "\n", "- Racial Diversity: many race groups $a \\in A$ present.\n", "- Gender Diversity: many gender groups $a \\in A$ present.\n", "- Age Diversity: many age groups $a \\in A$ present.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "fBHd6Vgig6QK" }, "source": [ "#### Set Diversity\n", "\n", " The formulation for an instance giving rise to a\n", "diversity score naturally extends to a set of instances giving rise to\n", "a diversity score. [...] We define the cumulative diversity score of a\n", "set $X_q$ as a function of $\\text{Diversity}_A(x_q))$ across $x_q \\in X_q$." ] }, { "cell_type": "markdown", "metadata": { "id": "Ea14ACVGzi5A" }, "source": [ "### Defining Inclusivity\n", "\n", "When defining inclusivity, we also care about _who_ is being included and _how_ they are being included— we'll need some more context than just the content of the image itself when scoring its inclusivity. \n", "\n", "In this case, let's consider the case where somebody is searching for images containing people, such as \"doctor\" or \"engineers smiling\". Ideally, a person is included if their results are **useful to them** and the results **reflect conditions under which they feel included**. \n", "\n", "As above, suppose we consider every entity in an image to be comprised of different attributes. We can score the inclusivity of a result by looking at \n", "1. how relevant the image is to that query, and\n", "2. whether or not the individual making the query is represented in the image results.\n", "\n", "(For a more rigorous definition of inclusivity, you may unfold the tab below)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "y6k4Nya6cvti" }, "source": [ "### Defining inclusivity (very specifically)" ] }, { "cell_type": "markdown", "metadata": { "id": "amngobc2G-PL" }, "source": [ "**Instance Inclusion**. As above, we assume an instance $x_q$ (e.g., an\n", "image) is composed of one or more items (e.g., different components\n", "of the image). Each item has some relevance to a query $q$ and may\n", "be a better or a worse fit for an individual $p$ along some attribute\n", "$a$. The inclusion of an instance $x_q$ aggregates the relevance and fit\n", "of all items in $x_q$ and produces a single measure of that instance’s\n", "ability to reflect $p$ or to meet $p$’s goals." ] }, { "cell_type": "markdown", "metadata": { "id": "lOft1BoMzlO_" }, "source": [ "#### Set inclusivity\n", "\n", "(Or, how to aggregate across inclusion scores to get set-level inclusion.)" ] }, { "cell_type": "markdown", "metadata": { "id": "ltNOmROfRr7F" }, "source": [ "**Nash inclusivity**: This corresponds to the geometric mean over the inclusion metric scores for all items in the set. Set $\\text{X}_1$ is more inclusive than $\\text{X}_2$ if the product of its inclusion metric scores is greater, i.e. \n", "\n", "$$\\sqrt[\\leftroot{-2}\\uproot{2}n]{\\prod\\limits_{i}{\\text{X}_{2i}}} < \\sqrt[\\leftroot{-2}\\uproot{2}n]{\\prod\\limits_{i}{\\text{X}_{1i}}}$$\n", "\n", "Nash inclusivity can be seen as a mix of utilitarian and egalitarian inclusivity, as it monotonically increases with both of these measures" ] }, { "cell_type": "code", "metadata": { "id": "cdSKTpRQRsD5" }, "source": [ "def nash_score(subscores):\n", " root = len(subscores)\n", " return np.prod(subscores)**(1/root)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "DLLyhCDqUPf2" }, "source": [ "**Utilitarian inclusivity**: This corresponds to an arithmetic average over the inclusion scores for all items in the set, where a set $\\text{X}_1$ is\n", "more inclusive than $\\text{X}_2$ if the average of its inclusion metric scores is greater.\n", "\n", "$$\\frac{1}{n}\\sum\\limits_{i}{\\text{X}_{2i}} < \\frac{1}{n}\\sum\\limits_{i}{\\text{X}_{1i}}$$" ] }, { "cell_type": "code", "metadata": { "id": "K5HgU1McUPmS" }, "source": [ "def utilitarian_score(subscores):\n", " return sum(subscores)/len(subscores)" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "f6thRakoU8ot" }, "source": [ "**Egalitarian (maximin) inclusivity**. Set $\\text{X}_1$ may be said to be more inclusive than set $\\text{X}_2$ if the lowest inclusion score in $\\text{X}_1$ is higher than the lowest inclusion score in $\\text{X}_2$, i.e., $\\min_i(\\text{X}_{1i})>\\min_i(\\text{X}_{2i})$. If $\\min_i(\\text{X}_{1i}) = \\min_i(\\text{X}_{2i})$, then repeat for the second lowest scores, third, and so on. If the two mechanisms are equal, we are indifferent between $\\text{X}_1$ and $\\text{X}_2$. " ] }, { "cell_type": "code", "metadata": { "id": "PKj01VmQU8vR" }, "source": [ "def egalitarian_score(subscores):\n", " return min(subscores)\n", "\n", "# We have a separate comparison function here because we need\n", "# all of the subscores for tiebreaking.\n", "def egalitarian_compare(subscores_a, subscores_b):\n", " \"\"\"Returns 1 if subscores_a > subscores_b, 0 if they are equal, -1 otherwise.\"\"\"\n", " if len(subscores_a) != len(subscores_b):\n", " raise ValueError('Subscores must have the same length.')\n", " print(subscores_a, subscores_b)\n", " subscores_a.sort()\n", " subscores_b.sort()\n", " print(subscores_a, subscores_b)\n", " for first_a, first_b in zip(subscores_a, subscores_b):\n", " if first_a == first_b:\n", " continue\n", " else:\n", " return 1 if first_a > first_b else -1\n", " # Return 0 if they're the same.\n", " return 0\n" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "TQP_KvzzReoL" }, "source": [ "# 2. Shapes\n", "\n", "To test out these metrics, let's try them out on shapes first, using the example of making a query and getting some images.\n", "\n", " First, we'll want to define some **viewer**. This is the entity we want to be **included**.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "ujx0E3cIRiFo" }, "source": [ "### Define a viewer" ] }, { "cell_type": "code", "metadata": { "id": "YPVOYKxiR884", "colab": { "base_uri": "https://localhost:8080/", "height": 232 }, "cellView": "form", "outputId": "3e97fe8b-287c-4332-fa5d-09572ce327ce" }, "source": [ "#@title { run: \"auto\" }\n", "my_color = Color.BLUE #@param [\"Color.BLUE\", \"Color.RED\", \"Color.GREEN\"] {type:\"raw\"}\n", "my_shape = ShapeType.CIRCLE #@param [\"ShapeType.TRIANGLE\", \"ShapeType.SQUARE\", \"ShapeType.CIRCLE\"] {type:\"raw\"}\n", "my_size = Size.LARGE #@param [\"Size.SMALL\", \"Size.MEDIUM\", \"Size.LARGE\"] {type:\"raw\"}\n", "\n", "my_viewer = Entity(properties = {\n", " Color:my_color,\n", " Size:my_size,\n", " ShapeType:my_shape,\n", "})\n", "\n", "# Display with a ShapeFetcher.\n", "ShapeFetcher([my_viewer], 'Viewer').fetch()" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWYElEQVR4nO3deViU9d7H8ffMAMMOsghSaCCSSoq7aYiZYae0TcsW0+rSTlaerNNmZl1XpnXytD4d8+RRMzVzOVqdFgvT0tKnxTRFxDVNfcoNEBdAZOZ+/ijnaIGlwu9G5vO6Lv5gbmbmc80MH358577ndliWhYiImOG0O4CIiD9R6YqIGKTSFRExSKUrImKQSldExCCVroiIQQEn21hSUqL9yURETlFUVJSjum1a6YqIGKTSFRExSKUrImKQSldExCCVroiIQSpdERGDVLoiIgapdEVEDFLpiogYpNIVETFIpSsiYpBKV0TEIJWuiIhBKl0REYNO+tGOtcmyLLxer+97l8tlVxQREWNsW+lalkW3bt1IT0/nqquusiuGiIhRNb7SnTlzJl9//TVOp5PRo0fz4Ycfsnz5ct/2vn37kp2djcPhYPjw4ZSWlhITE1PTMURE6qQaL90vvviCmTNn4nQ6GTlyJMuXL2fq1KkAZGZmkpKSQkhICACpqakAhIaGnnAbZWVlrF271vd9cHAwrVq1Ii8vj/LyctxuN61atSI/P5+oqCgSEhJYs2YNlmWRkJBAYmKi73uAwMBAMjMzcTgc7N27l23btgHQvHlzjhw5wtatW333FRMTQ9OmTWv6YRERAWqhdB0OBw7Hb89U4XK5mDNnDi+88AK9evUC8JViRkYGy5Yt8122bds2cnJycDgcWJZFamoqK1asYPDgwWzcuJEmTZrw3Xffcccdd3DppZdy5513cumll+JwOBg6dCjDhg0jJyfHd9+xsbEUFBQQEBDAggULGD58OJZl8d5777Fjxw7uvvtu33317duXKVOm1PTDIiIC1MJM9+mnn2bGjBknXJaZmcm6deuIi4vjscceY/369RQUFNCwYcPfXP/555+nT58+AOTm5nLPPffwww8/0Lx5c8aPH899991X7X2//fbbJCUl0b17dyzLYu7cuTzxxBMUFhaSkZHBqlWruP766/nyyy9xuVwMGjSIESNGEBERQV5eHl27dq3ZB0NE5FdqfKUbFRVFgwYNsCyL0aNH88033xAWFkZCQgIAkZGRREZG4vV6q9xj4dChQxQWFgIwefJkNm7ciMfjYc+ePURFRREeHk5RURGPPPIIu3btYsmSJb6fj4mJweVysW/fPgDefPNN9uzZg2VZ7Nmzh6NHjxISEkJ8fDwAxcXFtGvXjoEDB9KoUSPuuece3G53TT8kcgosy2Ls2LEcOHDgjG6ne/fu9O7du4ZSidScWttlzLIspk2bBkCHDh1O6zZWr16N2+2mTZs2wM+z3YSEBJKTk5k4cSLwc3GuWbOmyuvn5+cTGhrqu35YWNhvfqZZs2bcfvvtAPolNWjHjh2+P47HsyyLqVOnctQVRHhM/Gnd9u7vN1BSUkJSUlKV29PT06t8LYiYUCule2yue2xmezzLsnxfxzt+n91jZsyYQUpKygm3O3DgQLp16+Yr0l/f5/Hz5AkTJtC2bdsTfqa6+65uFi0149eP+/PPP//LG6xVP+a9//okWbfcdVr39VL/bGbPnsPs2XN+nQKARYsW/eZ1oedeTHFUVYzHlJSUVL/xJI4cOcKePXvIzs5m+PDh3HLLLcTFxQEwZswY3nzzTQB2796N1+slICDA9y//kCFD6NSpE1deeSXx8fEEBPz378IHH3xAamoq27Zto02bNrz77rt8/vnnzJgxg0WLFtGwYUOOHDnC5s2b6dGjB7GxsQQGBvquP3PmTLZs2cJjjz3G7t27AQgJCSE5OZlly5ad8LNSs0aOHMnbb7/t+76kpITo5KYMevnNKn8+ODwCd2j4ad3XocK9eDyVVVy+h3/ckkNcFa+L40tY5ExFRUVV+1e8Vla6brebc845h1GjRnHhhRf6ChcgOzubBg0aVHvdCy+8kJSUFMaOHfubbceuFxMTw9ixY7nggguIjIwkOTmZpKQkHA4HgYGBNGvWjDFjxvzm+omJibjdbu69994TLg8KCsLp1BHRNW3Lli1MnjwZgMWLF1PuDKLLDYN92yPiEohq2KjG7zc8tuqxRHB4JL3vH431y4q3ovQwn7w2jldeeYXExERCQ0N59NFHdXSk1KpaWemK//J6vaxfvx6v10tewXqeHPeCb1tS89Zc/+QrNqY7UWlJMZOG9sX6ZbQVER7OpJf+jsvlIjo6mnPPPdfmhHK2OtlKV6UrNeLY66iiooK0tDQOHjxI047dGPLafJuT/XGH9xfxdE5LvB4PN9x4I/+cMMG3TTNfORXGxwvifx588EFyc3NxugIY8voCAoNDCTjLdr8LiYziwf98AxZs/epTWrduDcDEiRPp0qWLzemkvtBKV07bhg0beOuttwD4P28wB6wAnE4XXW+6g4Cgs6twf23X5gLWf54LQGNnGWEOD263m4ceeuiEN3dFqqKVrtS4nTt38tW3K5n57ocA9H7gKdp1vcTmVDUnMa0FiWktAJg3+j62r/6GkJBg+vTpQ0pKCuHhp7dnhYhWuvKHHf9aGTBgAN8f8nDrSzNOco365cjhQzzV83xmTp/u+/wQzXqlKnojTWrEokWLePDBBwHoMvghzuuUTURcgs2pzLG8Xop+3M6y155h17pVBAcH8+mnnxIcHGx3NKljVLpyxmbPns2367fwfdnP37fudS3x56XZG8om3y2YR+GO73ECrSOh/3XXkZ6ebncsqUM005XT5vV62bp1K+98tJDiwEiuHvGs3ZFs1+byfgAcLS9j/KDLaHJeKhERETRqVPMHekj9o5WunNTBgwdJS0vjxnGvc37WpZph/oplWUz/6yCahMCsWbPsjiN1xMlWujr2VaqVm5tLr8uvYOj0XFLad1XhVsHhcHD1iGeJybqSrKwsSktL7Y4kdZxKV6o0d+5cFn+1knOyriD+vHTcofooxOpEJSSR0KItCRfm8Mr48WzYsMHuSFKHaaYrJ/B6vezYsYM577zHoYiGXPXwM3ZHOis0TGnGxYMf4JUBPUlslER0dLTvg/tFjqeZrpygpKSEtLQ0Br48k7TO3TVSOAXHfpdmPjKYBKuUefPm2ZxI7KKZrvwhCxYsoM81fblr2sc0bt1RhXuKjn0Yeu+/jiax53VkZ2dz6NAhu2NJHaPSFQDmzZvH0pVriO9wMQlpLTXDPQPRieeS0LwNDdp2Y+KkyZrxygk00/VzHo+HXbt2MWHCBDwJqfR/arzdkeqF+PPSuPSukYzp2ZywkGBiYmJ8Z0cR/6aVrp8rKSkhMzOTFStW2B2l3nrkkUcYPHjw7/+g+AWVrh97//336dOnD5WVlVz7+Avk3DXC7kj1iiswgDsnv0f6RT359ttv6dGjxxmfWl7Ofhov+LGioiI2bt5C15vuoGmHLBokJdsdqV5xOJw0Ss+gzZ/6EugOIW/pR3g8Hrtjic1Uun6qsLCQkpISgkJCueL+0bj0wdy1pm3v/kTGN2L9kgXs3r0bt9tNaGio3bHEJhov+Kmbb76Zxx9/3O4YfsXj8dClS5cTTkUv/kel62eKi4vp2bMna9eupUX3PzF4wr9x6pTjte6clm24a+qHBIWG8eyzz3LvvffaHUlsotL1M0ePHmXVqlWc1+liWuVcTVLz1joIwoDg8AjOzWhHp76DKHMGsWnTJrsjiU00yPMjZWVlFBYWAtBt0N00yexkcyL/4nS5uOL+JyndX8TRnzazd+9eYmNjcTq19vEnerb9yMyZM7nooovwer12R/F7K1asoGXLlhQVFdkdRQxT6foRy7Jwh0Vw5+T3fGe6FfMuHnw/Vz86TruP+SmVrp/44IMP+Oqrr3AGBtI4syPusAi7I/mtuMapJKS1wLIs5syZw8aNG+2OJAZppusnnnvuOfLy1xGtAyDqBJcrgODIaEY+9hghISE6saUf0UrXj7S/6kbun/s5Doeedrslt2rPyI/zCI1qYHcUMUwrXX/icOJ06SmvCxwOxy/PhXbX8zda8tRzZWVlzJo1i/iW7Ui+oJ3dceR4Dgete13N1j1FLFy40O40YohO11PP/fTTT7Rs2ZK7p33MuRlt7Y4jVXh7zAN4t6/jk08+sTuK1BCdrkdEpI5Q6YqIGKTSrcc2bdrEwkWLadXrGkIi9S55XXVuRhtimrVi/vz5lJeX2x1Hapneyq7HFi5cyDMvvsyjC9bg0PH9dVbHaweyPW8Fg2+7gvXr1xMcHGx3JKlF+k0UETFIpSsiYpBKt55asmQJ+fn5dseQU5Sbm8vWrVvtjiG1SKVbT40aNYq3Zs0m0B1idxT5AxxOFwHuYP7yl7+Qm5trdxypRSrdeqzzdbdy39zPQWeGqPPOaZHJqEUFhMfE2x1Fapn2XqjHnK4AAt16J/xs4HQ6f/6vRH8g6z2tdEVEDFLpiogYpNIVETFIpSsiYpBKV0TEIJWuiIhBKl0REYNUuiIiBql0RUQMUumKiBik0hURMUilKyJikEpXRMQgla6IiEEqXRERg1S6IiIGqXRFRAxS6YqIGKTSFRExSKUrImKQSrceO1S0l5825mNZlt1R5HdUlB5mZ8FqvJVH7Y4itUylW4+t/mg+r//lBlDp1nm7thTw6sBelJYU49AZges1lW49NX/+fEaMGGF3DDkFDoeDpUuXMmDAALujSC1S6dZT8fHxREZG2h1DTlFCQgJhYWF2x5BapNIVETFIpVvPOfh5Pqg30+quY8+NZrn+QaVbjw0cOJB358zi71d1YNemdXbHkWrk/mMsq6c+z+rVq4mLi7M7jtSyALsDSO2JiIggMTGB/T/txHO0wu44Uo3SkmK8B/eTnJxsdxQxQCtdERGDVLp+RHPdukfPif9R6dZz8fHxrFy5kpVTxvHJa+PsjiPH8Xo8vDqwF1d2bMm0adPsjiOGaKZbzwUEBJCSkkL5/kK8xYV2x5FfKfpxB+HBbpKSkuyOIoZopetHyg8eoHDnNv1LWwccLS9j3/bvsbweu6OIYSpdP7L6o3lMuO1yLMtrdxS/t7NgNS/260rZgf12RxHDVLp+4o033uDhhx+2O4Ycx+l0kpubS9++fe2OIgapdP1E48aNadiwIUfLSvls8kuU7P7R7kh+q2Dpx6x6fzYAqampREVF2ZxITFLp+pGIiAgaxsawcMLf2L9rp91x/Fb+ovdZ8+FcmjRpgsvlsjuOGKbS9SP9+/dn8eLFOJ162u3Wtm1bVq5cSXR0tN1RxDD99vmR4z9QZc6ou1n4z2dtTON/PJWVTLj9CvI/W4DD4fB9iX9R6fqZ0NBQhg0bRpDnCBuXLWL5rElYXu3NUNv2/7STz6e/yo/r88jq3JF+/frZHUls4jjZPpslJSXaobOeGjBgAMuWLaPCcjBy4TpcATpOpjZt+fpzJg3tS6NGjXj66ae59tpr7Y4ktSgqKqraf2G00vVTM2bM4Mknn7Q7hl8JCAhg2bJlXHPNNXZHERupdP3UsXli2cEDjB+Yw85139kdqd76+B9jeXvsAwCa44pK15+1aNGC224dxK6N+Xz7n1lsXL7Y7kj1itfj4cs5U9j4v4sJ9lZw55134na77Y4lNtNM188VFxeTlZXF3r17yci5hv5Pjbc7Ur1RWVHBmJ7NCQ5w0qNHD6ZOnWp3JDFEM12pVnR0NHl5eXTs2NHuKPXWuHHjeP311+2OIXWEStfPORwOnE4nL774Itd268C//nwtnsqjdsc66/2w5hsm//lq5s+dTU5Ojua44qP9hASA9PR0Dhw4wLbNm/hm/jSaZ/+J6MRz7I51Vtq4fDEHv8/nsi7tad+uHcHBwXZHkjpEpSs+HTp0ID09nc6dOxPdMIng8AiCwyPtjnXWsCyLw8WF5H/0b84N8vDslCl2R5I6SOMFOUFERARr165l+3uv887Yh+yOc1aprDjCc1d35K5+VzBp0iS740gdpdKVEzgcDlwuF8888wzXX9KVSUP7UanTt/+ubau+5I1h/Zk76y2ys7P1oUJSLY0XpErNmzfn8OHDbFi3llXvzaJZ156a8VZj05efcXBLPt0zW9C5UyftiysnpdKVarVv357zzz+fDh06EBYdQ0hEFO6wcLtj1RmWZVF2YD/fvTuDppFBvPCvf9kdSc4COjhCTsqyLDweDzcPGMCPniAGjNObQ8dUlJUyNqclUya+xuWXX64PJBefkx0codKVPyQ/P5/Pln/JpFnzAOh1z0jOa9PZ5lT2+Oh/RrM971uCAgP52yP307p1K2JjY+2OJXXIyUpX4wX5QzIyMqioqGDNiq8BKC5YiedoBU07drM5mTmeyqOsyX2XxIBKEtOScbvdXHRRV4KCguyOJmcRrXTltAwZMoTN+49w1eMvAhAUHIorMNDmVLXjSOlhvJ5KKspKmXjLJUyaOJFLLrnE7lhSh2m8IDWusrKSjz7+mEG33gbATc+8xgU9r7Q3VC15fdiNbP56KeHhYaxbu5bQ0FDtEiYnpdKVWrFv3z7y8vIAmP7OB6xZvwlnQAADxk0hKCTU5nRnZtOXS1j6xisA3HFDX9JTGuNyucjKylLhyu/STFdqRVxcHD169ACgoKAADhaBw0nBpx+AK4DwmHiadsyyOeUfV1lRwbrPPsSyLLz7/o/W5/z85lhW5w5kZGTYnE7qC610pUZVVFTQunVrDh06ROPMTvT/22TfNofLSVBw3VkBW5aXitJS3/dlB/bz6k0XY3k99OvXj5dfftnGdHI200pXjAkMDGT16tVYlsWy5cvp3/N837bzMjsz5LX5NqY70eHiQsb1bo/X8gDQILoB6wvW4XK5tM+t1BqtdKXWFBYWsmLFCgAmTJjAVyu/o3HrDr7t8U3S6P3AU8bylJYUM+fxe4CfX9aVFRVs+XopTz31FOnp6QQFBdG9e3fNbOWMaaUrtoiNjeWyyy4D4IcffiA8/L+HEK9Zs4aC7d/TpG3VB1gkprUkrnHqad3v5q+WUH744G8uLyvZz4YvFpKVlUV0dDQQTIs+fbjssstIT08/rfsSOVVa6YotHn74YaZPn17ltvLycnoNG0XXG4ec1m2PH9iLou1bCKxmv+EFCxbQpk2b07ptkT9Cu4xJnVNRUYHH4/nN5ZZl0a5dO/bs24fTdXr/iFVWHOGuoUN54oknqtzudrs1QpBapfGC1DnVHTprWRbjx4+nvLz8jG4/NTWVkJCQM7oNkdqgla6ISA3TKdhFROoIla6IiEEqXRERg1S6IiIGqXRFRAxS6YqIGKTSFRExSKUrImKQSldExCCVroiIQSpdERGDVLoiIgapdEVEDFLpiogYpNIVETFIpSsiYpBKV0TEIJWuiIhBKl0REYNUuiIiBql0RUQMUumKiBik0hURMUilKyJikEpXRMQgla6IiEEqXRERg1S6IiIGqXRFRAxS6YqIGKTSFRExSKUrImKQSldExCCVroiIQSpdERGDVLoiIgapdEVEDFLpiogYpNIVETFIpSsiYpBKV0TEIJWuiIhBKl0REYNUuiIiBql0RUQMUumKiBik0hURMUilKyJikEpXRMQgla6IiEEqXRERg1S6IiIGqXRFRAxS6YqIGKTSFRExSKUrImKQSldExCCVroiIQSpdERGDVLoiIgapdEVEDFLpiogYpNIVETFIpSsiYpDDsiy7M4iI+A2tdEVEDFLpiogYpNIVETFIpSsiYpBKV0TEIJWuiIhB/w8webEy8osbwgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "NvC_CDXYekB8" }, "source": [ "Now, let's say we're looking at some images of other shapes." ] }, { "cell_type": "code", "metadata": { "id": "GVrGYPsITVfD", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "cellView": "form", "outputId": "d289f401-429a-4d92-b67c-399d15f8ca5c" }, "source": [ "#@title Some other shapes:\n", "\n", "\n", "circles = create_image_from_shapes([\n", " Entity({\n", " Color: Color.GREEN,\n", " Size: Size.SMALL,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Color: Color.BLUE,\n", " Size: Size.SMALL,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Color: Color.RED,\n", " Size: Size.SMALL,\n", " ShapeType: ShapeType.CIRCLE\n", " })\n", "], title='Small circles')\n", "\n", "greens = create_image_from_shapes([\n", " Entity({\n", " Color: Color.GREEN,\n", " Size: Size.SMALL,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Color: Color.GREEN,\n", " Size: Size.MEDIUM,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Color: Color.GREEN,\n", " Size: Size.LARGE,\n", " ShapeType: ShapeType.TRIANGLE\n", " })\n", "], title='Green shapes')\n", "\n", "bigs = create_image_from_shapes([\n", " Entity({\n", " Size: Size.LARGE,\n", " Color: Color.RED,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Size: Size.LARGE,\n", " Color: Color.RED,\n", " ShapeType: ShapeType.SQUARE\n", " }),\n", " Entity({\n", " Size: Size.LARGE,\n", " Color: Color.BLUE,\n", " ShapeType: ShapeType.TRIANGLE\n", " })\n", "], title='Big shapes')\n", "\n", "just_one = create_image_from_shapes([\n", " Entity({\n", " Size: Size.SMALL,\n", " Color: Color.RED,\n", " ShapeType: ShapeType.SQUARE\n", " }),\n", " \n", "], title='Just a small red square')\n", "\n", "diverse = create_image_from_shapes([\n", " Entity({\n", " Size: Size.LARGE,\n", " Color: Color.RED,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Size: Size.MEDIUM,\n", " Color: Color.BLUE,\n", " ShapeType: ShapeType.SQUARE\n", " }),\n", " Entity({\n", " Size: Size.MEDIUM,\n", " Color: Color.BLUE,\n", " ShapeType: ShapeType.CIRCLE\n", " }),\n", " Entity({\n", " Size: Size.SMALL,\n", " Color: Color.RED,\n", " ShapeType: ShapeType.TRIANGLE\n", " }),\n", " Entity({\n", " Size: Size.SMALL,\n", " Color: Color.GREEN,\n", " ShapeType: ShapeType.TRIANGLE\n", " }),\n", " Entity({\n", " Size: Size.SMALL,\n", " Color: Color.GREEN,\n", " ShapeType: ShapeType.SQUARE\n", " })], title='Many shapes')\n", "\n", "diverse.show()\n", "just_one.show()\n", "circles.show()\n", "greens.show()\n", "bigs.show()" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwTdf4/8NekTZP0vkvLUY5CSwu0IFc5ZDlEUXaxIrh4gogsCHIoKrDKrj9hvVAEWRHQLwsFFg8WwQW5y2HlkEsqQilHoQe97zRJk5nfH5Us5ShtSWaS9vV8PHw8dibJzLvd5sUn73xmPoIkSSAiInmolC6AiKgpYegSEcmIoUtEJCOGLhGRjBi6REQyYugSEcnItbYHS0pKOJ+MiKiefHx8hDs9xpEuEZGMGLpERDJi6BIRyYihS0QkI4YuEZGMGLpERDJi6BIRyYihS0QkI4YuEZGMGLpERDJi6BIRyYihS0QkI4YuEZGMGLpERDKq9daOTY3FYqmxrVKpIAh3vEMbEVG9caR7g3HjxqFDhw7W/5KSkpQuiYgaGZuH7rp167B48eIa+y5fvowZM2agtLTU1qezqaeeegqPPPIICgoKUFBQAJPJpHRJRNTI2Ly9cPDgQfzyyy/o06cPYmNjkZmZiQMHDmDVqlV444034O3tjYqKCpw5c8b6Gnd3d0RHR+P06dMwGo3Q6XSIiYlBSkoKDAYDtFotOnfujNOnT8NgMFhfFxMTg9LSUly9etW6LyQkBK1ataq1RrPZjJMnT0KSai6M0bt3b5SXl2P16tUAgLS0NPj7+0OlUqFr165Qqar/jRJFESdPnrS2I1QqFeLi4pCeno6CgoJbzicIAmJjY6FWqwEAkiTh1KlTqKqqsj7epUsXuLm5WR+//ru4UfPmzREWFlbrz0ZEjk24OXhu1JDleiZPnox169ZBpVIhNTUV77zzDlatWgVBEHD27FkEBQXh5MmTGDRoEARBgCRJiImJwf79+9G1a1dcuXIFkZGR+Omnn9C9e3dcvHgRbdu2xc8//4zevXsjNTXVeq7k5GTs2rULb731VvUPIwiYNGkSFixYcMf6JElCQUEBIiMjraF5vW+7ceNGFBYWYvz48TX2a7VapKWlwd3dHQBQWVmJiIgIVFZWAgDUajVSU1Mxa9YsfPPNN9Ywv/76G392oLp3HB0djby8PGtdKSkp1kAVRRGxsbHIyMiocZzZs2fjtddeq+//JUQkM4darmf+/Pl4/PHHAQD79u3D2LFjce7cOURHR2PNmjWYMGEC0tLSEBUVhZUrV+Kll15Ceno6oqKisHTpUkyfPv2Ox/7Pf/6D2bNn13r+b775BvHx8dbAHTFiBM6ePYuzZ8+iT58+NZ67bNkyLF68GJWVlYiLi8Pu3buxd+9exMbGorKyEosWLcKKFStgMpnQo0cPDBgwAKtWrbK+fsGCBUhMTIQoiujTpw82bNiAU6dOITo6Gvn5+Zg7d641pK+/9ty5c+jYsSOysrIAAO3atbPWN3ny5Ib8yonIgdgtdCVJwttvv43Dhw/X2D9o0CA89dRTAIDPP/8cx44dg9lsRk5ODvz8/ODh4QGLxYLc3Fz4+vrW2Pbx8YGnp+cdz+nv7w8vL69a6+rcuTMmT55sbRWcOXMGCxcuxMKFC5Genl7jub6+vvD19QUA5OXlwWg0om3btpg2bRrUajW2bNmCb7/91vq4m5ub9fkA4O3tDT8/PwBAfn4+KisrUVVVhdzcXIiiiD179iAxMdH6uF6vR1BQEF555RV4e3sDAAoKCqz1HTp0qPZfOhE5PLtMGfPw8EBERATWrl2L0NBQtG/fHmlpaQCA8PBwxMTEAABOnjyJ3Nzceh07JCQE0dHRNXrCarUaMTEx1o//tYmKikJoaCi+//57WCwWFBYWYvny5QCAYcOG3fX1fn5+6NGjB1QqFdLS0qzh3RDp6enQ6/WIi4sDAAQFBSEgIAAvvvgi9uzZg5ycHBgMBmt9JpMJYWFhiI6ObvA5iUhZdgndLl264Pvvv0eHDh3wyiuvoFOnThg6dCgAYPHixVi+fDkEQcBXX32FpUuX4p///Gedj/3MM8+gf//+iIuLgyRJkCQJwcHB2Lt3b53m1EqSBC8vL+zcuRMAsGHDBkyaNKnO509OTsaYMWMAAPPmzYOnpydGjRp1x3Ndd2N/93ove+rUqXjxxRdrPOf6z7R+/XoAwLlz5xAfHw8AWLVqFQ4ePIijR49y/jCRk1Ls4ghJkjBw4EBUVFQ0+BgjRoyA0Wi0fhSvi2+//RZvvvmmdfv6l2ENMWPGjFpHunPnzoWrqytUKhUOHDiA1q1bQ61WIyUlBQMGDMCCBQvwySefWJ//+uuvo3fv3khISLDuM5vN1v89ZcoUvPzyywxcIidm89AdOXIkSkpKIAgC5syZg549eyIgIADz58+Hp6cnhg8fjvDw8Nu+1s/PDw8++KD1W/6AgAAMHjwYPj4+AGDdf11+fj769et3x5Hm7URHR2PKlCm3fSwiIgIGgwHz588HAOsMh+vb1z/WX9++WVxcHLKzs63bgwcPxn333QdBENC2bVvodDoAQFhYGObOnQu9Xn/L6wMDA+9YX8+ePREcHFznn5WIHI/Np4zZU2pqKkwmE7KysjB69GhERERg3LhxeOmll5QuDQBw9epV7NmzB9OmTQNQPcVr1KhRaNu2rcKVEZGcapsy5lSh26NHD5w/f966nZyc7FBfKk2YMAFff/11jX3x8fHYtm2bQhURkRIaTehmZ2dbr+ICgGbNmlmv4nIEBQUFt/SoNRoNQkJCFKqIiJTQaEKXiMgZONQVaURETRlDl4hIRgxdIiIZMXSJiGTE0CUikhFDl4hIRgxdIiIZMXSJiGTE0CUikhFDl4hIRgxdIiIZMXSJiGTE0CUikhFDl4hIRoqtkUaNz+HDhzFr1qwGvXbatGkYOXKkjSsicjwMXWqwy5cvY+vWrdbt1NRUnE5JQc+Rz8FVXbeby0uShKP/WYNNmzYhJyfHur9v376IjY21ec1ESmPoUp0ZDAYUFRVZt4/8fAz/+PiTGs/xCWmOYS+/BY2HZ52OKYkiLh37EUmHjiDp0BHr/ukmc41FOL28vODpWbdjEjkyrhxBdbZlyxY8++yz1u2o/kPxzEerb32iINR5mXhJkoDb/A1u/Xgeflz3uXX7rbfewowZM+pfNJECuFwP3bPZs2fjhx9+wNXMLIz/7Bu4ummg9fJBYCv7rHRcfC0D5QV5kCBh9fRn4K1xRc+ePbFq1Sq7nI/IlmoLXbYX6I4uXLiAvXv3AgCyza4I7v4HhMWr0bLTfXBRq+16bt9mLeDbrAUkSUKPhKehLylChU6DlStXAgC6deuGbt262bUGIntg6NJtFRcXY//+/Xj11Vnw8AvAnxcsQ69eA2SvQxAEDJ08GwCQl56GeeP/CH1xIaZOmYLWrVvD399f9pqI7gXbC3Rbjz32GPbu3Qu1zh1zd52BWqOrc5/WXiRJgiSKWJjQG0WZ6YiIiMCRI0cUr4voZuzpUp3s2rUL77//PgAgYvgz8G7ZFoLKBS1jukJQOcZ1NJIkIevsLzCbjDAWF+DX9Z8CAMaMGYNx48YpXB1RNfZ06a62b9+OLVu24NjxE4h7+HEER3dFSNtIpcu6hSAIaN6xev5uWUEeToe1R8quLfDdtg0+Pj5ISEjgyJccGke6TZwoiiguLsaYMWPw8/ET8A5uhlc2HrL7F2W29M9nH8K182cQEhSAffv2wcfHB66uHE+Qcmob6TrGZ0ZSTHFxMTp27IjDhw+j8wN/wqubDkPlZIH1l//7LwY8/zIyMjLQoUMHnD9/XumSiO7Iud5dZBdmsxnDX30H7eMHQeXifH8SKhcXdBv+BAJbtsW/50xEbZ/eiJTmfO8wspkLFy7gwI/JiBv2ONrHD0Jwm/ZKl9RgfqEtodbq0PWR0Ug6mAyTyYS4uDilyyK6BUO3iSovL8euXbvw9wXvYs7OM3BxspbC7Xj6BWL0/1uKRaP643JaKtq2bQsvLy9+sUYOhT3dJmr06NGYM2eO0mXYzcqVK9G/f3+lyyC6hfMPbxRgMpnw7LPPYurUqejbt6/S5TSI2WxGu15/wICxU6FycVG6HJtKePMjHPtuPbKO7FGshuPHj+Ptt99W7Pw3evTRRzF27Fily6DfMXQbwGKxYP/+/WjTpg0kSUK/fv2ULqnODAYDtm7dCr+ITghrE4223Z3zH43ahHfpAX1xIdSSGRs3bsTgwYPh6+sraw2FhYVI2pcEj04ein6erDxXic6dOytXAN2CoVtPZrMZFRUVEDQCPvv8M6SmpjpN6JpMJmRnZ+OFF17Ai19sQXhsT6VLspuO9z+I0A6dMP7hOGzfvh1xcXHQaDTyFqECgkYHQaVVLnWvLryq2Lnp9tjTrafdu3cjJjYGgdMD4R7trnQ59bJ27Vr06tULoigqXYqshg8fjnfeeUfpMogAMHTrTRRFmIwmCC4C/Ab7IT0kHU8++ST0er3Spd2VxWKByk2LZxclIrhNB6XLsTsPvwA8t3gdPIPDYDablS6HCABDt16OHDmC42ePwz3GHVABmhYalOuqp145y5vaxc0Nkf2GQOctb49TCWqNFlH9HoDWw0vpUois2NOth0WLFiHpchKajWv2v50qQFALqKyshLu7u8Ne828wGGARRag1OqVLsRtJFFFlNNy6XxJhNptt9mlEo9HApZHN+CD5OGZCOBFdhA7a17WI6xmHTz/61GGXEX/00UehCo/GzI3JEITG+QGn+FoGPhp562wMs8mILy+ew5o1a2xynsTERAwZMsQmx6Kmh6FbB0ajERMnTsQFnwvwHVzzY7mgEgD17yNJi0WhCu/OaDTCTZSg1miVLsVmSvOuYdOCWdbtKkMlzEYDJgQEINgOnziMkoSPc3Px7rvv4ssvv7TunzlzJrp3727z81HjxNCtA4vFgh9++AE+f/aBRyuPW58gAO5R7vj18q84cuQIevZsvFOxlGSsKMfFnw9at00FufA8ur/Gc3q4u6OXuzuC7XBrSqMooqe7O8xnziDvzBnr/kOdOiEvL8+6zXs+UG0YundhsVhgMBgguArAHS7hF1wEhDwdgi/WfYETB09g8+bN8hZZC0mSYDQanfLOW5Iowlxlsm4XXr2E9TP/twR8mKsrFrdoIdu9FTQqFWY3a3bL/veWLsWRG/rFy5Yvh87jNv84E4Ghe1fbt2/HhEkTEPxqMFQ65+uF5uTkoHv37tDr9ejRvqvS5dRLzsWz+Oy5Ydbt5q6uSAwPt26rAIe4mc2MoCDc2Fj65+uv41BFhWL1kGNj6N6FxWKBXq+H4CZU929r4d3PG9n52Xj++eexZMkSeDjAaEeSJFRUVGDolLloHz9Q6XLq7NiWf+Pk1m9gqtTjLwEB8HZxgbtKBZ2DrNV2I7ebanpEq0WwKOLrMqNCFZEjc7y/YAdy8uRJ/HrxV+g61G0lXG0rLcq9y7F582aYTKa7Pl9OET3vR/OoLkqXUStDeSlSk/cgNXkPSpN3w+uXI+im06GPpyf6eXqim7tzXAEYpdUiRtt4vrAk2+JItxYLFizAwayDaDb21j5ebQQXAVVVVbBYLJzPeReSJMFirgIkIPfiOayZ8gQAYGxAAKaEhipcHZHtMXRtTNtGi7A5Yejetzs+nP8hRo8erXRJjk2SsOjx/igvzEMbV1cktm4NAFA7QK+WyB4YurdhMBgwY8YMXPG/Ap92PvV6reAiQKVRoby8HFVVVXaqsHEouHoJOz97FyU5mejvpsb9Og3cHbBnS2RL/Au/jaqqKmzcuBHF7sXQtW3AZbNC9Yg3LTsNJ0+etH2BTkySJFw6fggXjhxAxqEkSElbEa0S0NvDA/c5Sc+W6F5wpHsTUay+Tr96PlLDjiGoBDQb1wwrNqzAod2HsG3bNpvW6KwkSYTZZELiq8+hsrgQURoN3mveXOmyiGTFke5N/vvf/6Jbr25oNqsZdO0b781hlHD19DH8Y2gn6EuK8Ly/P+bxizJqghi6NzGZTCguKYZKq4Lgcm9f5nj39EZ+q3xMmTIFZWVlNqrQOR3dtBa7VyyEoawEE/z9cZ+7O/u31CTxr/4GZ86cwYWsC9CGaxvcWriRto0W5f7lWLduHYzGpjlRXpJEXPnlKFJ2b0FG8h5Ea7UY5OmJFm5uSpdGpAj2dG8wZ84cHC06itAJtvvYKwjVV7JZRAtEUYSqCY3uJEmC2WTCqqljYCgrQbRWi3+EhSldFpGimk4CKETTSoMWc1qg76C+2LBhg9LlyOryicN4d1gsKstL8UJAAP4aEqJ0SUSKY+gCqKysxCuvvIJrwdfg3cvbpscWXASoPFQoLCqEwXDrqgZySfpyEU5u+1a28x3dtBb7/7UElcWFGO/nh646HTx4dR4RQxeo/vJs9erVKPYpttuMBU1zDTIKMvDbb7/Z5fh3olar0b17d1w6nITLJw/b/XySKCLj1xM4vfM7XDm4E1EaDR7w9mYPl+h3TT50JUmqXpLcjledCioBoRNCsXznckydOtV+J7qNwMBA7Ny5E5GRkXY/lyRJMFeZsPIvI3H+p71o//s8XM5SIPqfJv9u+O6779D7/t4InRXasKvPyOrizz/i/eHdYNSXY2JAAF5jD5foFk1+9oLBYEBeXh7CvcLveV7u3XjGeaKkoASvv/465s6dC29v2/aPlWapMkFfkIex/v6I0+ngxR6uTUmShKIdRRArxTq/xlJqwb59+/Daa6/ZsTLlDBs2DAMHOs99ooEmHrrnz5/H1byrcAuTp9+oi9ChQl2B5UuXY+bMmbKGbmRkJEq0OmSnpqBZ+xibr7hQkHEZ+oxLaK/R4GFvb2jZUrCL8p/L4e4WBK+A4Do93yscKKgC/nvA/v18uV1LO4PQ0FCGrjOZPn06TlWeQtgkmeeOCtWjFkmSZFtuZtmyZVi5ciXmTR6FOTtSIAi2GYVWr70m4YdP3oY6eRc+4L0U7C7+ifH4w7hpSpehKEmS8P7wbkqX0SAcjshM01yDlrNbYuDDA7F27Vqly7lnZpMRCxPicfbgTqVLIXIKTTJ09Xo9/vrXv6IgtACeXT1lPbfgKsDF2wW5ebmoUGDxQpO+Als/mofCzCs2O2ZZ3jXcr3bFQ42sR01kD02yvWAwGPDZZ5/B7zE/aLw1MGbJfF8ECZAsErKzs5GWloaIiAhZTuvv7492rcPx47rP0fmBP8G/eat7Op6hvBT5l9PQytUVAzzc0ZX3w61JAkzXTBDcbNdCkiySzY5FymiSoQtU3xMh/+t8RWtYtGgRkpKSkJSUJMv5HnvsMfTr1w9RUVHVwX+PPeVLx5KxfuazSAwPd8hVehUnAllLs5SughxMkwxdHx8fnDhx4vcvgZTlJvOVWgEBATh16hQmTZ+JM/uiMGzaW7Kev6mI0WqxsmVLmx/3tSyGuLNrkqHr4uKClnZ4QziD6z+7qawEV5L3wE3njoEvzKz33c+Of78BZcl78LSfHxeRvA2NSoVgG4/+JUmCC3/XTo+fCZuo8PBwoKwAyeuXQxLrPtleEkXkXkrFsS3/RsH+H5Dg6wtXBgFRnTF0m6gVK1Zgzpw51u26tlqqjAYseXIwLh49aK/SiBo1hm4TlpCQgK2bN+GTUf2QkXK8Xq+dHBiIlwID7VQZUePF0G3CvLy80KplSxRmXMahr/8PKbu/r/Nr/Vxc4O/aJL8SILonfNc0cSqVCu3atcOZXd/BUF6K0A4x8G/R+rZTyYz6chRlpCPU1YX3ViBqIL5zmjhfX18cPnwY9913H84kbcOnTw2BaK667XPPH0rCsjED8VFQEGJ1vA0mUUMwdAmCIGDZsmX429/+BmNFORaNvh/pp44oXRZRo8T2AgEAWrZsiUGDBqGsrAwAcOn4AVw4ehCuajf0e/ovULnwT4XIFvhOamTMZjMyMzMRGhpa76vdunTpgi5dugAAJk6ciBO7d6OkQo+OAx6Ei9oN5qICBLu62nNlI6JGj6HbyFy7dg2xsbE4cOAAOnfu3ODjLFu2DBs3bsT48ePx0WN9AADx7h74tEULW5VK1CSxp9uIrF27Fo8++iiAul/scCeCIGDIkCFITk5GcnIy4uPjIaH6Bjly3XidqDHiSLcRKSgoQOaFC0jw8cG2r75CcXEx7r///gYfz8fHBz4+PgCq5/SW2qpQoiaMI91GIicnB6ayMrR0c8NYf3/8e/lybNu27Z6PK4oiMjMzYTAYbFAlEXGk20gkJCQgMjMTH4SF2fTjf0VFBbp16waj0YjevEk50T3jSLeRuN7DvR64s4KDUbVjBxISEiDW4y5idzIhIADPBwTc83GImjqOdJ1caWkpEhMT0cVkQoRWa93fws0Nx4qL8ZuN1mELVavRTK22ybGImjKOdJ1cYWEh5syZg35GI3p5eNR4TCsI8BYEXLt2DSaTSaEKiehGDN1GbKiXF2ar1egUE4OUlBSlyyEisL3g1BITE7Fh+XIsDAtDs9vcZvF6f1d0gLXgiKgaR7pO7Nq1a0g9cwYRGg00d7jVolYQ8LC3N5K+/x4//vijzBUS0c0Yuk6qsLAQFr0eXne5r62Hiwv+EhiIdUuXYuPGjTJVR0R3wvaCkxo2bBhic3PxYfPmvCyXyIlwpOukRFGEJElQ1TFwpwYGwnX/fowePdom83aJqGE40nUypaWl2LRpEzqZzYjQaOr8ujYaDX4rLMSJ4uJ7vhkOETUcR7pOJjc3Fy+//DIeqKpC/E3zcu9GLQhwB1BUVISqqtsvyUNE9sXQbUIGe3nh7zodOkZG4sSJE0qXQ9Qksb3gRBITE7F59WrMDw1FUAOWP1cJAlSCALPFwhYDkUI40nUiV65cQcrx4+ik1d5xXu7duAkCBnp64ujevThyhItPEsmNoeskysrKIJpM0DUwbK9zV6kwPTgYiR9/jDVr1tioOiKqK7YXnIAkSRg0aBB6FBbiA87LJXJqHOk6CbPZDIgiXG0UuC8EBMDj6FE888wzsFgsNjkmEd0dR7oOrrS0FDt37kRHUUQrG97PtoNWi8v5+TiYk8Mv1YhkxJGug8vKysL48eMxQhTRx9PTpsd2FQRoBQHl5eXVI2kisjuGbhM2wNMT//DyQsfISM5kIJIJ2wsObO3atdixYQPeatYMfi4uNj++iyDAFYDRaOT9GIhkwpGuAzt//jyO//QTuru7Q3uPU8XuRC0I6O3hgZTDh3mVGpEMGLoOSJIkVFZWQjKb4Wbn6WE6lQpvhIQg8YMPsGzZsjs+zyRJqOIXbkT3jO0FB9W/f3/0LSnBe2FhSpcCAPgwJwfxHh6YFRKidClETo0jXQdlNBohWSxws1Nb4WZP+/vD//RpvPDCCzVmMuh0OvzrX/9Cp7g4jnSJbICh62BKS0vxw7Zt6AAgtAE3tWmoaK0Wmrw87Nixo8aXaq6urnjooYcQHBwsWy1EjRlD18FcuXIFY558EmMA9LXxvNy7UaH6izWjwcCr1IjshKFLVv09PbHQ1xcxHTty5WAiO+EXaQ5k3bp1+PrrrwEAn+Xn222aWG3MkoQyvf6Wke6MGTNwKDoaC1euxMtBQVDzpjtEDcLQdSAmkwkqlQqDBw9WtI5BAPz9/Wvsi4+PR35+Pt795BNMCQwEGLpEDcLQdSBjx47F2LFjlS6DiOyIPV0iIhkxdKnOunbtikWfforPSkqQZjQqXQ6RU2LoUp21aNECIxISkFxZiQLeCpKoQRi61CAWABZeoUZUbwxdapBFubn4ODdX6TKInA5Dl+rFzc0NixYtQofOnVHJkS5RvXHKGNWLq6sr/vznP+PcuXO4JAj4LS8PkRoNVJy3S1QnHOlSg8ybNw9PvPEG/pqdDSNHvER1xtAlIpIRQ5caLCYmBu++/z7+VVbGebtEdcTQpQZr1aoVnnz6aSRVVuKEXo8Mk0npkogcHkOX7okgCFCpVFhTVIQVBQVKl0Pk8Bi6dE80Gg2OHz+Ohx56SOlSiJwCp4zRPREEAcHBwRg/fjyOhYdj5fr1eNbfX7a13Zqi3/ZtR2nuNaXLUJy+pFjpEhqEoUs28cADD0AQBHzw+efo7eGBcDc3eLm4KF1Wo2EURVypqoJZklCVl4FKVZXSJSkuukMEQpxwdWqGLtmUKAiYk52NmUFBGODpCYEXTdwzSZKQWVWFVzIzAQDzXnwRM2bMULgqaihBqmVie0lJCWe9U50ZDAbk5OSgb9++gF6POJ0OrznhSMTRrC8qwtaSEpSIInbt2oWoqCh4yrxoKdWPj4/PHUcbbLyRzWi1WrRo0QLz5s1Du86dcdZgwJcFBTDesKQ71Z0kSVhfVIQjFRXQBQdj/vz5iIyMZOA6ObYXyKZcXFwwYcIEZGVl4T8lJdh8+TJ6ubujFXu89WIURWRWVWFrSQl0wcHo1asXXnrpJaXLIhtge4HsZt++fRgxYgQA4OWgIAxmj7dOJElCmslk7eHOnz+fgetkamsvMHTJbgwGA7KzszFgwABI5eXorNPhDfZ472pdYSG2l5WhyGLBrl27EBkZCS8vL6XLonpgT5cUodVq0apVK8yePRvtunTBOYMBqwsLYWCP97YkScJXRUU4qtdDGxyMefPmISoqioHbyLCnS3bl4uKCSZMmIT8/H18XFuLbK1fQ3d0drdRqeLLHa2UURVwzm7G5pATawEDE9+zJaWGNFNsLJJvk5GQ8/PDDAIDJgYF40MuLPV5Uj3BTjUbMysoCALz33nuYOHGiwlXRvWBPlxyCwWBARkYGhgwZArG0FDE6HWazx4vEwkLsLitDoShix44d6NChA3x8fJQui+4BQ5cchiiKWLp0KfR6PYw5OcjcuBEA0N3dHZFarcLVySe3qgo7y8oAAIEDBsA7OhqCIGDy5Mmch9sI1Ba67OmSrFQqFaZOnQoAOH36NJ5LSkJ6ejoAwN/VFUGujf9Psshsxm9GIzYUF6N169Z4809/wsiRI5Uui2TCkS4pShRFREVFITc3F+3c3PBxixZKl2R3n+TmYnd5OVQqFVJTUxEYGKh0SWRjbC+Qw5IkCZcuXYLZbMaFs2fx6vjxAIDHfX0xuBFNlTpnMGBRXh4AYOrcuRj08MMQBAFt2ul0SXgAAAIuSURBVLSBaxMY3Tc1DF1yCllZWfjiiy8AAGVHjkCfkgIAeMTHBx5OeH9eiyRhc0kJqiQJbqGh8Bs2DADwxz/+EXFxcQpXR/bE0CWn8+GHH+KLL75AdnY23g0Lg7+LC1wFAYFOMCostlhgEEVYJAkzMjPhGRCAgQMHYsWKFUqXRjJh6JLTkSQJKSkp6N+/v3VfS7UaS1u2VLCqunkvJwc/VlRYt5cvX45Ro0ZxTnITwtAlp2QwGHDx4kXrdsbly3j1uees28FqNd4MCVE8zJbn5+OXykrr9vS//x29//AH63bz5s0577aJYehSo5Cbm4slS5ZYt8WyMlRs3nzL8wZ7ecHHDpcYV0kStpaUwHLTft2AAXC9YdbFE088gU6dOtn8/OQ8GLrUKF2+fBmPPPKIddtsNiMnJwdvN2uG5m5uNj+fQRQxLSMDPgEB0N5wIceSJUswaNAgm5+PnBdDlxqlm/9209PTZZkVsGHDBgwdOrTGPqVbHORYeEUaNUo3B11YWBj27dtn9/O2adOGIUsNxpEuEZGN8SbmREQOgqFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJqNbVgImIyLY40iUikhFDl4hIRgxdIiIZMXSJiGTE0CUikhFDl4hIRv8fscKh2JTSSxsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAQGElEQVR4nO3dfWyN9//H8dd1aDtKizo9buZ+6qba2hiZuVm32JZFZoL9sTBzGzdDyBILgoWNjX5ZJCwWMkLipt9l2ZTMphMLkbqfu063YcT0jmrRGz29fn/01+uraLHx7jbPRyKcc51zrs91qef5nE8vOK7rCgBgw1fTAwCAxwnRBQBDRBcADBFdADBEdAHAENEFAEO1q9t49epVricDgAcUGRnpVLWNmS4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCo2n/a8e/KdV2VlZVJkhzHkc/373nvuPXYbuXz+eQ4zj23W7rbWGrVqmU6BuCf5h9ZqwsXLigmJkYxMTH65JNPano4D1V6erp3bLf++P333yVJZ8+evev206dPm481NTW10hjGjBljPgbgn+aRzHR//fVXLV++vNJ9juPogw8+UERExF9+/WAwqNzcXEnS9evX//Lr/Z0EAgHNnj1bkrR161bt3LlTUvkxS1KjRo287d999522bdtWabul9u3ba8aMGZo9e7ZKSkqUn59vPgbgn+aRRPfGjRs6duyYjh49qtLSUoWHh6tz5846ePCg6tWrJ6k8wvHx8crMzNSlS5ckSfHx8QoLC1NhYaGOHz9+x+t26dJFV65c0bFjx7z7Ll26pP3790uSYmNjVbdu3XuOr7S0VEeOHJHrlv/HGLVr11ZCQsJ9L1OUlZXpyJEj9x06x3GUkJCgkJAQSeUfy48ePaqbN2962+Pj4xUaGqpGjRpp5MiRkqRz58550a0QGRnpbc/NzfWie79u3rypo0ePynVd+f1+Pfnkk3c9FxkZGZUi2r59ezVo0MC7nZGRoby8PMXGxv6p5Z2zZ88qOzu70n0VXyeSdP78+UpfF9nZ2frjjz8qPf6JJ55QXFycjh07pqKiIoWFhSkuLk6O4+jSpUs6f/6899hAIKCWLVtKkk6ePKnr168rJCRECQkJSk9P17Vr17zHduzYUcXFxTpz5ox3X6NGjdSuXbsHPk7gdo8kunFxcdqxY4diYmKUm5urTp066dtvv1Xnzp29P0iO4+jEiRNauXKlVqxYIUn66aef1KJFC509e1b9+/f3Xq9irTItLU1ff/21FixY4G3bsmWLkpOTJUm7d+9WXFxctWNzXVd5eXl65ZVXvPXI+vXrKyMjQ6GhofdcF3VdV0VFRRowYIBu3LhRaXy3PuZWPp9P6enp8vv9kspnpW+++Wal6Bw/flzNmjV75OuyeXl5evXVV1VaWqrhw4dr7ty5dz0XM2bM0K5du7x15I0bN+rll1/2bs+ZM0fbt2/3bt8v13Xluq6WLl2qtWvXSvrf+evatatSU1Pluq5WrFihlStXyufz6fDhw1qzZo0+/fTTSvtr166dDhw4oFGjRikjI0OtWrXSkSNHVFZWpk2bNmnu3Lne48ePH6+PPvpIjuNo8uTJOnTokKKionTq1ClNnTpVaWlp3mO/+eYbnT9/XhMnTvTuGzRokFavXi3HcczXzvHvYram6ziOfvzxRw0bNqzaxyUlJWnAgAHe7cmTJys9PV3p6elq06aNxo8fr++//97bPnr0aG97x44d7zmO5ORkPffccwoGg1q3bp0WLVqkgoICxcXFae/evfd8/g8//KCEhAQVFhZKknr37u3tPz09XV988cUdzykrK1OvXr20adMmHT16VJ07d1ZOTo5mzZql5ORkua6rfv363fW5D1tUVJROnDihp59+Wlu2bKn2XLz22ms6ePCgQkNDNX78eE2ZMkVFRUVeHBMTE3XkyBHVqVPnvvdfcS42b94s6X9fF+np6d6bZ79+/bR27Vp16NBBJ0+e1PDhw/X555+rXbt2Sk9PV4sWLardx+uvv67FixerSZMmOnXqlDp27Ki1a9eqb9++cl1Xmzdv1pw5c5Sbm6vY2FgdPnxYQ4YM0b59+1SrVi29/fbbev/991W/fn0dO3ZMvXr10rZt29StWzeVlJT8+ZMP6BFEd/PmzVq5cuUd9zuOI7/ff8+P/3369NE777zj3d67d6+SkpKUlJSky5cvKzw8XFFRUd72unXrKhAIKBAIeB/fqxMXF6eJEyfK5/Np06ZN2r59u1zXVVZW1n39gWrbtq2mTp3q7evs2bPe+C5evKgGDRrI5/Np3rx56t69u1q3bq2FCxeqsLBQhYWFatq0qaZPn67w8HClpqZq/fr1kqScnBxv5vwo+Xw+71wVFRXp+vXrWrhwobp166bExETNmzdPOTk5Kikp0c8//6zly5ertLRUeXl5ysvL885VcXGxwsLCFB0d/UAzP8dxNGHCBMXHx0sqn/l+9tlnSkpK0oYNG+S6rncuateurUAgoCtXrni3o6OjVbt29R/QRowYob59+6qgoED/+c9/lJ2drcLCQuXk5Egqf+OpX7++dyxjx47V8OHD1apVKy1atEgNGzZUfn6+iouLtWzZMp07d07FxcXKzs5+oFk9cDcPfXlh165dOnPmjMaNG/ennt+zZ08FAgFvLfPChQs6ePCgJOn555/Xs88+e8dzSkpKdOLECUny1imr4vf79cwzz0gqv1LgQWcurVu31tixY5WSkqLCwkLl5+dr1apVkqTu3bsrEAjI5/NpxIgROn36tEJCQjRu3DgtWbJEklSvXj316NFDISEhOnfunBeCmhIWFqaxY8d6IQsEAlqwYIHKysqUm5urw4cPe4Fs06bNX95fxbnJyclRcXGxJGnjxo0qLS1VTEyMevfu7a11/1ldunTRgQMHVFJSorS0tHu+mQ0cOFA9e/aUJI0dO1ZfffWVfvvtNwWDQe3fv19+v9+bMPybLk9EzXgka7oV63Z3c+us6PbHVFz32bJlS6WmpkqSFi5cqMWLF0sqn8G89957Gj58eKW1vczMTCUmJkqSJkyYoIULF1Y5tpSUFE2ZMkWStHTpUl24cEETJkyoNO7qZm6u6yokJEQpKSmSpJ07d2ro0KFVn4zbnDx50luvXrRokXr06KEXX3yx0vFXtd/bt916/iq232vN8W6/N7cfd8XPiYmJWr16tfc4x3FUWFhY7e/hvcZQsa9p06Zp2rRpcl1XnTp1UlZWlk6fPq2XXnqpymO8dRx3O5aK26NGjdKpU6fUvHlzpaamqm/fvt43ZivO092ed/vxh4eHa8eOHZU+QbGei7/Kqe7j0tWrVx/4s9TEiROVnJysqKgoZWZmas6cORo2bJgaN25c8ZpKS0vT0KFDFR0drevXr3uXfUVHR99xcX1BQYH3neWUlBTFxcWpbt26ysrK0sCBA3Xx4kWFh4crKytLX375pbp166bIyMgqx7du3TovulFRUQoGg8rLy5NU/h3qQYMGKSkpqcrnp6amatKkSd7t4uJiXb58WZK0atUqBQIBDR48WBkZGZo1a5bOnDmjrVu3KiYmRrNnz1aXLl286EZERCg0NNSb7UZERCg8PNx77fz8fO/c+P3+Oz5WX7t2TQUFBZKkxo0bKyQkRE2aNFFqamqVccjNzVXfvn2VlZWlmzdvynEcNWnSRGvXrlWPHj1UVlamzMxMjRo1SocPH1bDhg295/bv31/Lli1TZmamJk2apD179qhhw4bKzMyU67oKCwtTIBDQnj17VL9+/bvuPxgMqk+fPrpy5Yp3X2ZmpsrKyhQbG6stW7ZIkj7++GNt2LBBfr9fWVlZCgaD6tChg/bt26fs7GwtXbpUq1atUnR0tLe9Vq1aio6OVnZ2tkpLS72llNtvS+WXGlZcnREVFaW33npL8+fPl1S+1LNlyxbNnDlTgUCgUoT37NmjsLCwKr46gHKRkZFVvjs/9Jnu4MGDFRsb691+4YUXvOD+/2CUkJCgDz/88IFfu3Pnzt51vs2aNdP06dO94Enl67XVBVcqXwKobt9PPfVUtc9v06aN3n333btu69q1q+rWrav58+erTp06Gjx4sK5evSrHcTRz5kz16NFDUVFRf+rY71fFJXlVqVOnjiZNmnTHDLJ58+aSyj/+N23aVBMmTKh0yZVUvp5dsX3MmDF3zEornl9dlBzH0ejRo1VUVHTHNr/fr2bNmkmShg4dqpiYGLmuqyVLlnhvjI7jKDo6Wm+88Ua1y0gPqlOnTt6vGzdurMTExEpXyUhSSEgIf+MOf9lDn+kCD0thYaF++eUXDRkyRJmZmWrdurXWr1+v9u3bM9vE31p1M12ii7+t48ePq3fv3nfcf+jQIbVt27YGRgTcH6KLf6SSkhLvL9PcqmnTpvd1eSBQU4guABiqLrpcdAgAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGKpd0wPA4yUYDGrkyJHKz8+v6aGoQYMGWrNmjXw+5h6wQ3Rhbs+ePWpYUKCmISE1NoaLN2/qRGSkXNetsTHg8UR0USNejojQqxERNbb/bVev6r81tnc8zvhcBQCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYql3TA8DjKae0VGeKi2ts/7nBoORjzgF7RBc1YnNenjbn5dXoGPx+f43uH48nogtTPp9Pu3fvVjAYrOmhqFatWvIx24UxogtTjuOoefPmNT0MoMbwNg8AhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIcd13ZoeAwA8NpjpAoAhogsAhoguABgiugBgiOgCgCGiCwCG/g/OG3+9TjTROgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAVzElEQVR4nO3de3BUZZ7G8e/pJN25B0hIk4S7oGAggQEZGEYYUAeQMC6iMGhKBWcAF9CSctYZ4rgglxkKcfE2DHjDy1ruQC2rhQiuCRIVNFwWBCRoGIYBAgmQG4Fcu8/+kaE3CIREut+O6/OpSlWSk/P+3g6/PH3Oy6lzLNu2ERERMxzBnoCIyA+JQldExCCFroiIQQpdERGDFLoiIgYpdEVEDAptamN5ebmuJxMRaaG4uDjrStt0pCsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxq8taOrZHX6+VyTzB2OBxYlnXV7YGai7/Gt20br9d7yfcDMX8RMe97F7orVqzgmWeeueh7SUlJfPLJJwC8/vrrLFy48KLt8fHxbNu2jZCQEL/O5eGHH+aDDz4gLCyM7du3ExMTc81j5ufnk5GRccn3c3Jy6NKlyzWPLyLBFZDQPXToEM8///xF37Msi/nz5xMbG3tNY//0pz8lMjISgCVLlnDy5EnCw8N92wcNGsQTTzwBwDPPPMPRo0dxOAKzijJhwgQGDBiAw+HA5XL5ZUy32+2b//r168nOzgbA4/H4ZXwRCS6/h+7hw4fJzc1l9erVpKenExYWRmVlJQcPHmTMmDG0bdv2kn0iIyNJTU31fV1VVcW+fft8X4eHh9O3b18A0tPTSU9PB2DVqlWcPHnyorFSU1N9Y73xxhscPXq0xa/B4/Gwe/du32m+w+Ggf//+OBwOysrK+OabbwCIjo6mT58+OBwO31F0RUUFBw8eBKBbt264XC7y8/N9Y0dHR9OrVy/27NlDXV0d0PCGlJaWhtPppF27dkyZMgWAI0eO+EL32/bu3Ut1dfVFr/vCmxHA/v37OX/+/EX7JCYm6mhZJMj8HrpLly7l7bffxuFwsGbNGhISEtixYwejRo1i0qRJl11vTU1N5ZNPPvGtWf7tb3/jtttuw7IsbNume/fu7NixA8uyAr6uads2586dY8yYMb5QDA8Pp6CggMjISLZt28Y999zj+1kAp9NJQUEBsbGx7N69mzvuuAPbtnnhhRfo1q0bY8eO9b2WAQMGsGnTJiZOnMipU6d8dfft20dycvJVX59t29i2zZQpUygoKPCN++mnn/rebGzbZvr06b43rgtjPvjggzz99NP+/YWJSIsE7OoFr9fLT37yE3r16sXixYvJz88nPz+fzMzMS3724MGD9O7dm8LCQpYtW+Zb0/zwww+ZOXMmR44coVevXhw+fDhQ0/XZsGEDAwYMoLa2lj//+c8899xzVFVV0a9fP7Kzsxk5cqTvtYwePfqS/QcPHsy+ffuIjY1l7ty5ZGZm4nA42Lp1K5MmTWLPnj3ceOONnD59mqysLNauXYtt2wwfPpzVq1dfdX6NfxczZszgo48+AuAXv/gFS5cu5eTJk/Tu3dt3dJ2YmMiBAwfIz8/nySef9OvvSkRazu9HuhMnTiQ9PR2v18vChQs5d+4c0LC+mpWVRWRkJCkpKcyePZvFixczbNgwevTowfLly/F4PNx8881UVVWxbNkyXnnlFb7++ms8Hg/FxcXU19f7e7qXqK6u9h2BtmnThuuvv54lS5YA0KNHD1wuF263G+Cy67hOp5PExETfUkTXrl353e9+R9euXbn33ntJSEjgxRdfBCAmJoa0tDTf+AMGDLjq/Orr6ykuLgbgiy++oLy8HICSkhLOnj1LTEwMc+bMYfny5Zw4cYLKykqWLVuGZVkMGTKE8ePHX+NvSESuhd9Dt1u3bqSkpNC9e3c2btzIvn37KC4uZtWqVcyZMweA9u3bM23aNJ599lmGDBnC8OHDWb58OQCdOnXyrd/u2bOHkpISf0+xRdq2bcugQYN8n7dUUlIS06ZNA2DYsGFERET4QhcgKirKN36HDh1aNHZhYSG2bdOvXz8AUlJSiI6OZvr06Wzfvp2CggLq6up46aWXACgqKqJHjx706dNHl5+JBInflxeWLFnC7NmzsSyLdevWMW7cOIBm/5GvWrWKBx54AIC33nqL++67z7ftwjWsFz4a+/a2xtsvrINe6Rrexr69bpybm8uIESMYOXIkn3322UV1rlS/cY1v1208vm3b5OfnM2LECEaMGMGaNWuuOv6FMQAyMzPJycnxfUyfPt33c6tWrSInJ4c1a9b4fv7dd99lwoQJl70OWETMCMglY7t27fL9p055eTlpaWm88847JCQktGic0aNHU1VV5fs6IyOD0ND/m/KFZYATJ05cdPXDBadPnwYaTr0vHD3fddddLFiw4Io1x4wZQ15eHkOHDmXGjBlAw3+kbd26leTkZDZv3szMmTMBKC0tBaC2tpZBgwb5Lk2zbdt32r9r1y7S0tLIzc2lXbt2pKWlsW/fPoYPH87ixYtxOp0AbN68mZ49e3Lw4EHfEkBFRcVFv4vQ0FA6duzI/v37GTt2LCtXruTtt9/2/czUqVPJzMzklltu8YW8x+Pxff7LX/6SefPmBewSOhG5Or+H7oQJEy4JQLfbTXJyMtAQnAMHDsSyLB5//HH69etHhw4dWLRoEW3btmXUqFG0b9/e39Py6d27d5PbIyIi6NKlC0899ZTv2tjQ0FA6d+5MaGgo3bp1Y9asWS2qaVkWERERQMOab3JyMllZWRdd0tWzZ0+io6NJSEhocvy4uDiSk5N57LHHKCsru2hb//79iY6O9r0pfFufPn1avIQhIv5lNXW6XV5e3vS5uIiIXCIuLu6K66k6zxQRMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExKDTYE5D/8+qrr/Lee+99p32fffZZunTp4ucZSWugvvj/RaEbZKWlpeTk5ACwceNGcj/PJeKGiGbvb9fanD9wnnfffZeUlBQiIiIYM2YMlmUFaspiwLf7YntuLj+KaH5fVNs228+rL1ojy7btK24sLy+/8ka5ZjU1NezYsYOxY8diOS2wwNnBScqslGaPUVdSx7FnjgFg19skJSaRl5dHZGQkISEhgZq6BFDjvgi3LCygs9PJ0pTm90VxXR2zjzX0RZ1t0z5JfWFSXFzcFd/dFLpBNHfuXF566SXq6uro9NtOhMaGggVWaPOPRmzbhvqGz8s+KaP0g1JcLhfZ2dn06dMnQDOXQGrcF6s6daJdaCgWENaCo1Tbtqn7x+fvlpXxZqn6wqSmQjdk3rx5V9yxpqbmyhvlO/N6vUyfPp3s7GzOOs6SODkRZ5ITR5gDy9Gy0z/LsrBCGj5CY0Nxpbio2FPBV199RX19Pf369QvQqxB/a9wX4WfPMicxka5OJ06Hg5AWLgtYlkXIPz7iQ0Pp7nLxWYX6wpTw8PD5V9qmNV3DSktLyc3NZdOmTVRFVhHVJ4rI3pF+GTssPgxHuIPIPpHk7c4jLi6OlJQUbr31Vq3ltXKN+6JtVRUDo6IYGOmfvugQFkakw8GQyEj25Kkvgk3LCwbV1taSl5dHRkYGVphF/Lh4YofEBqTW8eePU3O8BneCm127dhEeHq61vFaqcV84LYsH4+MZExuYvvjN8eMcqqkh3q2+CKSmlhd0na5BWVlZ3HnnnQB0fKwjMYNiAlYreUYy7Ua3o6ioiOuuu469e/cGrJZcmwt9YQEvduzIz2MC1xeLkpPJbKe+CCYtLxhUV1eHHWvjznATGhOKFRK4UzsrzCKqbxQhUSGc+sspvF5vwGrJtamrqyPBtpnidtM2NLTF67ct4bQshkRFERsSwnOn1BfBoNA1wLZtcnJyOHr0KI5IB1GpUUbqhsWH+a6E2Lp1Ky6Xi9TUVCO15eoa90W0w8GPo8z0RVJYGE5LfREsWl4wwOv1Mm3aNLI3Zwf06PayrIaj3ieeeIJXX33VbG1p0oW++Dg7u0WXg/mDRcNR7+/VF8YpdA2KHxdPh193MFozJCaELv/aBWeK02hdab6p8fHM62C2L9qGhPBmly50d6ovTNPygkFWqIUjzOz7nGVZWC5Llwa1YmGWhcthvi8iLPVFMOhIV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha5JNthes7cotm0b22tjo1sjt1Y24G3ivtYBqWnbeGwbDNcVha5RZ9af4eQrJ43W9Jz1cOSpI9QW1hqtK8332pkzzD9pti/KPB4eOHKEv9aqL0xT6BpgWRYLFy5k8IDBeKsN37/UBu85L48+8ih33XWX2drSpAt90X/wYM4Zvq+tFyj3ennkUfWFaQpdAxwOB5MnT+aGG27AW+Ol+u/V2J7An9bVV9RTc7wGgHHjxjFkyJCA15Tma9wX1V4vB6urqTdwul9aX89fa9QXwaLQNciyLOqL6yl8oRBPpYemnk93rWzbpnJ3JUWvFelOUq2cZVkcra/nXwoLKfcEti+8tk1uZSULitQXwaIHUxpUUVHB559/zsSJE3FEOWh3eztiBwXmAYSFKwupOVpD+7j25ObmEh8fT1hYWEBqybVp3BdxDgf3tWvHbQF6MOXvCwv5uqaGmPbqi0DSgylbidjYWNLT0/njH/9IlCOKyp2VlH1c5tcanvMeTv/XaWpP1DL0pqE8+eSTuN1u/WG1Yo37whsVRU5lJf9Z5t++qPR4WHX6NIdraxkwVH0RTApdw9xuN9OmTeOmm24iuiKas9vPUnOsxi9rvPVn66k5UkPFZxX07NiT0aNHk5mZqdPI74HGfVEUHc1HZ89SUFPjlzXe0vp6DtTUsL6igqSe6otg0/JCED3++OOsXLkSgM5ZnQmJCwFo0R9D43+/8i3llLxfgmVZ5Obm0rdvX/9OWIxo3Bevdu5MfMi19cW68nJWl6gvTGpqeUGhG0Tl5eXk5eVx9913ExITAhY43U6SpiU1e4z60nqOv3AcAG+NF3dbN9nZ2SQmJurU8XuqcV+0CQnBAXRyOlmQ1Py+OFVfz2+ON/RFtddLrFt9YVJToatnpAVRXFwc6enpLFq0CIANGzawbfc2zrx3ptljeKu9eCo8PPzww7jdbqKjo0lOTtap4/fY5fpi77ZtvHKm+X1xzuulxKO+aI0UukGWmJjIzJkzgYZHcpeVlUFJCwdJhfvvv5/rrrvO/xOUoLhcXxS0cIxU1BetkZYXRET8TJeMiYi0EgpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYN07wVplUpKSqioqPhO+yYnJ+N0Ov08IxH/UOhKq/T000/zpz/96Tvtu3XrVm688UY/z0jEP3TDG2k1Dhw4wNSpUwEoKirCEdOWe5e+1uz9K4pP8NqsSXTv3h2Xy4Xb7WbdunW6naEYp/vpSqu3ZcsWNmzYwIEDB/jxXQ+QcFMUsQluOvTo3ewxYtt34Ob7Gm6HeGz//7Br1y6ee+457rnnHtq3bx+oqYu0iI50Jahs2+bkyZMsWLCA//jLGqITEpn5xiZiEtzXNO4Xa1eTveppzp4uYu3atQwcOJA2bdr4adYiTdPjeqTV8nq99O7dm6KiIlJ6pzPzrf8GWvY8sMuxbZvyouMsub0/AA899BB/+MMfrnm+Is2h++lKq7R//36GDx/OmTNnGHb/LCYtWoFlWX5Zg7Usi5j4RGa/nU1Cl+tYs2YN48ePx+v1+mHmIt+dQleCYsuWLbz11lvs3buX/uMm0+vmn9O+a0+/1ggJc5LcK42bxmcS07knO3bsYMWKFRQXF/u1jkhLKHQlKNatW8fKl14mNjGJn8+cS7cfDQlYrWH3zaLf7Xdjh4WTlZXFsWPHAlZL5GoUuhI0Kb368tsNu4lqGx/wWjeNz2TWv38EunxMgkyhK0Z5PB7Gjx/P+vXrAQvL4TByHW3DWnFDuz/00EMsX7484DVFLkehK8Z9+eWXxHS5nj63ZBitGxYRweC7p3Cs+AxHjx41WlvkAoWuBEX66DsZdv8sozUjYuK447dLaJPU0WhdkcYUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6EpQHNu/m28+/9hozbrqKvZsWkdVRanRuiKN6cGUYlxUVBS7179DUcEBeg7+mbG6VRXlvDN3OhHh4bhcLmN1RRrTka4Y5XA42L59O5MnTw7aHNatW8f8+fODVl9+2BS6YpRlWbhcLkJCQjh15Btef+QezleUBbzuV5s38Jff/zPYNk6nk7CwsIDXFLkcha4ERVpaGgPT08j/9CO+ynmfM0cPB6xWQV4uX23ZyLEv88jIyNCj2CWo9Ah2CZqdO3eSkZFBVVUVtz86n8ETpxLmCvfb+Lbtpa66mhUPjOH04a/p1KkTO3fuxOHQsYYEVlOPYFfoStB4vV6qqqro168fZ0pL6ZT6I2a89r7fxi8vKmTZPw2mrraaX//qVyxYsACXy2Xk8UDyw9ZU6OotX4LG4XAQGRnJihUruHXkSIoO5fPmnPs4X1ZyzWPvz3mftfMfoa6misWLFjFlyhTCw8MVuBJ0umRMgsqyLG699VaOHTtGZWUlWz/+gBt/NobIuLa4omLoPnBos8eqrTrPobxcAA5s2cix3Z8zevRobr/9drp27RqgVyDSMlpekFZjz549jBo1CoC6ujoSul3PzDc/bPb+pYV/598mDMXpdOJwOEhOTmbnzp06uhXjtKYr3wsej4fq6moA5s2bx8svv0xYeESz97dtm7rqKrKzs+nVqxeWZREREaHQFeMUuvK9s3fvXg4dOvSd9h05ciSxsbF+npFI8yl0RUQM0tULIiKthEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDLNu2gz0HEZEfDB3piogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYP+F8WXeVppM/ctAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfXhT5d0H8O85SZo0bdNXaGkLVUAKlFoQYSICGyhDiyLOF1ARmYo8wBBEp46Jgw3UTRzqNicOVp4x9+CcOgoUQcAhiGB5b0stMCgtTV9p05c0r+c8f/QiswNKX5Jz0uT7uS6vy5M0ya9p+ObOnd+5b0GWZRARkTJEtQsgIgomDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgVp27rSYrGwn4yIqIMiIyOFq13HkS4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmpzaUdqze12tzoWRRGCcNUV3IiILsORbgfMnTsXAwYM8PyXk5OjdklE1M34ZKR75swZvPPOO57jlJQULFq0yBcPpagHH3wQ4eHhWLt2LQDA4XCoXBERdTdeD92zZ89iz549yMrKQkZGBi5cuIC4uDiMGTMGGRkZMJvNqKiogCAIuPHGG1FRUYHy8nLP7ZOTkxEVFYW8vDzPZQaDAenp6Z5ju92O48ePe45DQkJw4403oqCgAFar1XN88uRJNDU1QafTISMjo91TAW63G0ePHoUkSa0uv+mmm+BwODyhe+bMGXzzzTcQRRFDhw6FRqMBAMiyjKNHj8LlcgEABEHwPBdVVVWXPd6l5yIkJMRz+xMnTsBut3t+Jj09HQaDwXN84sQJ2Gy2VveTkJCA3r17t+t3JCJ1CLJ89R15OrNdz9y5c/HBBx9AFEUUFRVhxYoVyMrKgiiKKCgowJtvvok1a9YAAPLy8vC73/0O7777LgRBgCzLWLp0Ke68806MGjXKc1nfvn2Rm5vrCc1z585h2LBhnuuTkpJw/Phx/OAHP8Dx48eRkJCAvLw8TJw4EUeOHEFsbCxOnjwJrVZ7zeCVZRkNDQ3o37+/ZyR76TZ//etfIQgCpk+f3upynU6HoqIiREZGAmgZAaempsJisXh+rrCwECtXrsT69etx6Tn/bi15eXlITEwEAEiShIyMDJSWlnp+x0OHDqFv376eGkeOHInTp0+3up+f/OQnWL58eUf/ZETkZapu17Ns2TIUFhaioKAAcXFxWLJkCT766CPIsoxx48Zh/fr1SE1NxcmTJ5GYmIg333wTkydPBgBs374d8+bNQ3FxMQYOHIizZ8/ij3/8I26//XYAwKZNm/D888/DbDZj0KBBWLlyJV566SVUVlZi8ODBWLp0KZYuXYqamhqkpaXhyJEj16x369atGD58uCdw77jjDhQWFqKwsBDjx49v9bOrV6/G+++/D4fDgREjRiA7Oxtff/010tPTYbFYsHLlSmzYsAGSJOHWW2/F4MGDsXHjRs/t//u5yMrKwrfffotBgwahrKwMixcvxubNmwEAEydOxO9///tWzwXQ8sngUn3PP/981/9gRORTPutekGUZy5cv93wkFgQBS5YsgclkQkxMDACguroakydPxr333ov4+Hj8/Oc/x4cffogvvvgCALB27VoUFRXB7XajsrISLpcLTU1NqKmpAQCsX78eJSUlkCQJlZWVMJlMiIiIuOxYlmVUVlbC6XRes+7U1FTPiNHlcuHUqVNYtWoVAGDmzJmtfjYyMhLh4eEAgKqqKthsNvTp0weLFi3C8uXLsW3bNs/ot7q6GjqdDtHR0Z7bR0REtHourFYrXC4XKisrAQBffvklzp8/DwCoqanBJ598ArPZjMWLF+ONN95AVVUVLBaLp75x48YhMzOzg38pIlKST0P38OHDMJvNqKmpgSAIePbZZ2EymVr93KhRo3D//fcDAB5++GEUFRV5QvfYsWPQ6/UYOnQoALSa0wRaPpIbDAbP9Uajsct19+/fH08++SQ2b94Mh8OBuro6z3TIuHHjIIptfzgwmUwYMWIEtFotzp4965mn7YySkhI4HA7P71daWopt27YhNzcX+/btw/nz5+FwODz1WSwWpKSkIC0tja1sRH7K66F76R+7KIr45JNPPHO6l8iyjO/OI186vnS774bFhg0bcP31119235esXbsWgwYNuur1nSHLMgwGA7Zt2wYA2Lx5Mx577LF23/7w4cOYMmUKAODVV19F3759cdddd131sa40p35pHvepp57CwoULPZe/9tpr+Mc//gFZlj3PaXFxMYYNGwYA2LhxI/bt24cTJ060u14iUpbXQ3fFihWYNGkSZsyYgdtuuw0NDQ2XXf/dEP71r3+Nzz//HJ9++ikAYNGiRRg/fjzuvvtuTJo0CVrtf0rcsmULnn76aYwdOxYTJ07ElClToNPpPNd//PHHXa4/JycHixcv9hz/d4dARyxZsqRV/f9t5cqVnpHw7t27ccMNN0Cv1yM/Px+33347fvvb3+L999/3/Lx4kwjxLhFpaWmey757wsaPf/xjvPDCC52ul4h8z+uhGxUVheHDh2PFihWtLhcEAeHh4Rg7dmyreU0AiI2N9fy/yWRCWlraZbcHgOjoaERERGDgwIFXvD4uLg6jR4/2XJeUlASdTuc57tOnzzXrv+GGGzB//vwrXjd48GAA8NzfkCFDEBIS4jkeOnQojEbjFWsDgJEjR8JqtXqOx44di1GjRnke99L8cK9evfDiiy+2esNatWoVZKeM8NBwmM1mLFy4ED169Gh1/8OGDUN8fPw1f0ciUo/XW8bo6i5cuIC9e/fi6aefBgAsXLgQjz76KPr373/V29hsNpw+fRqPzn8UjX0bETowFBV/rkDW6iwMHzqcIUvkh9pqGWPoKmjhwoWtplaAltHxpS8Or6SwsBC33HILer/QG7q4/0yllPy6BAseW4CXX37ZR9USUWe1Fbpc8EZBS5cuxbPPPtvqsq50NxBR98ORrh/7/PPPsWXXFvzz7D8RMSoCmlCN57r6r+uRFpqG8TeMx4IFC9giRuRHVD0jjTpv79692PDRBkSNj2oVuABgusWE49XHsW7dOpWqI6LOYOj6qbY+gXjj54lIHZxe8EOyLCMzMxMXelyAnCZDG33lqXe31Q2pSoJ2ixZ///vf2+yCICLlcHqhGyopKUGDo+GqgQsAGqMGCG9ZTpNr+xJ1DwxdP2O323HmzBkgAhCN1/7zCBoBujgdSstKUV1drUCFRNQVDF0/c+rUKdx8883AvS1fll2LNkqL5J8m45H5j+Ctt95SoEIi6gqGbgBguxhR98HQ9SO7d+/Ghn9sQNSEKIiGjv1pTN8zIa8uD++99x47GYj8GEPXj+zYsQPr/m8dYibFXNaXey2Rt0Xi2MVjePvtt31UHRF5A0OXiEhBDF0/IMsy7rnnHmwr34b4xzq/aljEzREQp4i49dZbUVRU5MUKichbGLp+QJZlnDp1CnXOulYriXWUJlwDIVrAyZMnu7T4OhH5DkNXZQ6HAxcuXADCAFHvhT+HCGiiNKiqqUJdXV3X74+IvIqhq7L8/Hyk35gO3UM6RIyM6PL9aSI06POzPpg+bzpef/11L1RIRN7E9XT9gQxA8E6/LXt2ifwbR7oq+vLLL/HJtk8QOSYSQoh3wzJ8WDiKGouwfv169u0S+RGuMqain/70p8jamoXkZ5J9cv+WLy3QHdShoKAAosj3VyKlcJUxIiI/wdBVgSRJ+NGPfoRdF3eh57SePnucsKFh0N2rw4TbJ6CwsNBnj0NE7cfQVcmJEydQ66xFSLzvNqbURmghxAk4cvgIrFarzx6HiNqPoaswp9OJqqoqCKECBJ0CnQZiy0kTdfV1aGho8P3jEVGbGLoKO3bsGAYNHgT9DL1X+nKvRROuQZ+X++DhBQ9j2bJlPn88Imob+3QVJssyJEnyWl/utQiCAAiALMlsHSPyAxzpKujrr7/G9i+3I2JEBESdsk+9Mc2IYnsxPvzwQ4YvkYrYp6ughQsX4m+7/oaknySp8viWrywQ94ooLCyERtOx9XqJqP3Yp0tE5CcYugpwu914+OGHsd+6H3FT41SrIywtDPopetwz5R4UFBSoVgdRMGPoKuTAgQOocddAn6xXrQZtpBaaRA327dvH9jEilTB0fczlcsFisUDQCxA0frACmACIBhFN1iaeMEGkAoauj+Xm5iJ1YCrCnghD+PBwtcuBGCYi5ZUUzFg8Az/72c/ULoco6LBP18ckSYLT6QREQBDVH+kKggBoAJfbBbfbrXY51A7r1q3Dpk2bOnXbt956CykpKV6uiLqCoetDubm52HtoL8IywiBo1Q/c7wq9IRRmtxnZ2dnIzMzk0o9+pra2Frt27QIAbNu2DXu+3oPQ1NB23152yLCetOKf//wnkpKSEBoaijvvvJOL3PsB9un60Ny5c/Hx/o+RODdR7VKuqP5APdw73Th16hS0Wr7/+gu73Y7c3FxkZma2LG4vACEJIUia3/7+budFJ0rfLAUAyC4ZvXr2wsGDB2E0GtmjrYC2+nT5L43Izyxbtgzvv/8+ACD52WRoTVqggwNUbbQW171yHQCg7ss6mHPM6N+/P3bu3IkhQ4Z4uWLqCH6m9AGXy4VZs2bhiPsIYjJj1C7nqoypRhinGDH94enIz89Xu5ygJ0kSZs+ejezsbMgmGQk/ToDGpIGgEzo8PSUILavYCToB4Rnh6PFQD9jtdjz33HPIysryzS9A7cLQ9QFZlrF7925Uy9UwpBjULueqtFFaaK/TYsfnO1BbW6t2OUGttrYW2dnZ+Oyzz1DhqkDYkDAYBxm9skaHLlYH4yAjjEOMOHj0IHJycrBjxw6uwaEShq6Xud1uNDc3t0zcdIdnVwAEnQC7ww673a52NUHJ4XAgPz8fM2fORL21HqZRJsROjvXqY2jCNEiYmQBdTx0++/wzzJ8/H1arlR0sKugOsdCt7N+/H6kDUxH5P5EIH6Z+X+61iKEtfbuzXpqF5557Tu1ygtKSJUtw3333AQCSn0v26TrLiXMSETMpBhUVFejXrx9OnDjhs8eiK+MXaV7mdrvRbGsZ6fpDX+61XJr7czgdcDgcapcTlJxOJ2STjPjJ8S1bLPnwzEVBJyAsPQyaMA2qPqxqWduZFMWRrhcdO3YMh/IPwTjI6B+n/HaA4ToDqoVq7Nixg/8QFSLLMnbu3ImSkhKIRhFhaWGKbOGki9UhdEBLz+9XX33FL1EVxj5dL3rqqaeQfTgbiXP8sy/3WhoONcC+1Y7Tp08jJMR3G2ZSC7fbjQEDBqCmtgaGFIOi/dyuehdKXiuB7JTxxBNPYNWqVYo9djDgerpEfiz27lgkPJWg6GNqIjRIeSUFIUl8c1UaQ9cLnE4n5syZg3xNPqLviFa7nE4z9DUg/J5wPPnUk8jLy1O7nKAhaAXFt28SBAGiXuRpwSpg6HqBJEnYsmULqoQqhPZr//nx/kYXrYNugA6bsjehqqpK7XKIAhJDt4s8q4hplNndVwmCVoDL5YLL5VK7FKKAw9Dtoj179mDQkEGImR+DsBvD1C6ny0SDiJSXU/D08qfxzDPPqF0OUcBh6HaRy+VCQ0OD/+wM0UWCIEA0iGi2NbecWUdEXsXQ7YKCggIU/LugZR43wJ7JkMQQ1Il12LdvH/t2ibyIZ6R1weuvv47thdvR66leapfidbGTY5F3NA8PPfQQTp8+DYPBfxfuIepOAmx8RkTk3xi6neB0OrFo0SKcMZxB5NhItcshom6EodsJLpcLGzduRKWuEsYBRrXL8RlNmAbaZC0OHDzA9XaJvISh20GyLEOSpJYVxLp/s0KbQm8IRdQjUZhy3xQcPHhQ7XIClwzIkrLLnMiyDFmSIYPLqyiNodtBu3btwo033YjYBbEIG9z9+3JJfTWba1C+tlzRx3Q3uFG8vBiOMi7nqTSGbgc5HA5crLkI0Sj63bbqviDoBMTdG4f12euxbt06tcsJKIIg4Fe/+hVuGX4LJJvCbXkyIDVJWPTMItx///3KPnaQY+h2wKlTp/DvC/+Gvrc+aJ45QSPA9D0Tdufuxq5du9QuJ6CIoojp06cjNTUVkl2C7bwNstv3H/dd9S7YL7RszXT33Xdj1KhRPn9M+g/26XbAK6+8gi/OfaHouqf+4tK6EpIktew2ESDrTPgDQRDgqnSh7Hdl6LOkT8sOwD56fmVZRuPRRlzMvsi/oUqCZLxGXdVjWg8cjTqKYcOGcQNLL1u2bBk2btwIyEDpb0vR8E2Dzx7LvMaM2u216NmzJ06ePIn09HSfPRZdGUe67eBwOLBs2TKUhJcg4nu+2zTQn2mMGthEG2oqaxTburusrAyrV6/u1G3HjRuHzMxML1fkGyaTCRkZGXjttdewYsUKNB5qhGSVEPX9KK89htvqRu32WjjMDoweMRrTpk1DfHw8R7sqYOi2g8vlwtq1axE5LTKoOxbEUBG6XjocO34MgwYOQmSkd08MsVqt+Pbbbz3HZ86cwZo1axCSENKhV6qz3AmLxYLExP9MA/Xo0QPJycneLNer4uPjMXv2bGzfvh3Hjx+H5RsLQvuHIqRXSJcXUnI1uOAodaB+Xz1SU1MxadIkPProo16qnDqKe6RdgyzLsFqt6Ne/HyIfikRYWvCGLgBIDgnFrxRjw/oNXR5JyrLcatR89OhRjB8//rKf6/2z3tBF69p9v6WrSuEob90KNed/5mDlipWeY3+el37hhRfw3nvvAUDLHG+kBkDH1mv+7vNq+ZcFF7e0zOHu2bOHUwoKaGuPNIbuNezYsQMLnl0A3SM6iCZR8W1V/I03Q/fYsWOYNm2a51iIE6CdcvmQVhOh6dB29u4G92UnG0h5EtwH3J7j6dOnY+nSpZ2o2vcsFgsOHjyIBx54AJoIDSAAIfEh6DW7/QsruWpduPC7CwAAyS4hPjoeO3fuRM+ePaHTtf8NjDqnrdDl9MI12Gw2lJeXI8WUEvSBC7S0kMVkxuDDzz/ExYsXMWPGjE7dz6effors7GyYzWZE3xENwSBAE66BIbLrq5lpIjSXXWZPscPubvkC0LLHgu3bt0Or1eLFF1+EKPrX3zUyMhIZGRlYsWIFAGDr1q3Yf3Q/ajbVtPs+JJsEd70bCxYsQHx8PMLDw5GYmOi3o/tgwtBtw7lz51BaWdoyrxbo5/y2k6AREHlbJHb8eQfsZnuHQrepqQlnz54FAGzatQmbv9qMkMQQmEaboAm7PCi9SZ+shz5ZDwCwl9hRVFGEyk2VmDx5MkRRRHR0NJKSknxaQ0f07NkT8+bNA9DSpldXVwdc7OCdpAEzZ85Ev379vF8gdRqnF9rw4IMPYm/ZXiTMVHZ77O6gPKscY3uPxd/+9jcA155vlGUZ+/fvx1133QUAiPtRHEy3mHxeZ1vcDW4U/7IYkIEZM2bg7bff5kiQvKKt6QX/+lxF3UaP+3sgv0c+RowY0a5tfV5++WU8/vjjgAAkLUxC+LBw3xd5DWKYiN4v9UZIUgg+/vhjjBkzhptxks9xeuEK7HY7fvOb38AcaUZEQnD25V6LJlwDm8aGqtKqq/btnj17FuvXrwcAHKg+ANdwF2KEGOh66CCGqP9+L4gCdNE6RI6JhNviRo2mBr/81S8hQEBGRgamTp2qdokUgBi6V+BwOPDOO+8g+pFohA0K7haxtoh6Edo4LYpOFaFf334wmf4zXVBWVoZ9+/Zh9erV0PXQIeauGESN916zvzdFDG95Y3U3u/HuH96Fs8aJSbdPwpAhQ9C/f39OOZBXqT/c8DNKnW0VCEL7hyJubhzGTxzfajEcWZbxyiuvYP78+YAIJM5PRNgQ/3/z0oRqkLw4GfreeuTk5GDChAlwOp18TZBXMXT/y2effYYx48eg54KeMPTjZowd5XA4MGbMGGzduhX6FD16/7Q3REP3epn1fLgnYjJj0NjYiJEjR+LAgQNql0QBhNML/6WpqQnF54txXfR1EHT8WHktgiggakIUNn+1GTabDVOnTkVxcTHQFzClmaCL7X6N+NpILUIHhEKySjj3xbl2fVFI1F4M3e8oKytDVV1VS1Awb9tF0AiIHh+NnL/k4MKJC8gYmgFNtAbG4cZufcq0PlEPXYwOTflNMFebUV5ejoQEtg5S17FP9zvuvfdefFP7DXo+0pNfnnRQxV8q0HS8CdAA1/3ium43pXA1siyj/P1y3Jl+J3fOoHZjn24HMXDpkkuvhe3bt2PcuHFwOLinGHUNpxfQsr7CH/7wB9TE1CAsqft+JFabNkaL8JvCu7wUob8JHxaOppAmnDlzBpKk8F5mFHA40kXLyRCvvvoqqmKrED5U/TOluhNZluG86IQQIsBwvQExP4wJuC8gI0ZEIGJkBMRIEedLzsNqtapdEnVjDF3qEtklo3RVKYwDjejxUA+1y/EZ4yAjYubE4Hujvoe9e/eqXQ51Y0Efujk5Ofjh3T9E/Px4GFLYl9sRzWeaceGtC5CdLd+3BvJc+KXfTYaMxYsX46WXXlK5Iuqugj50LRYLioqKoIvXQdQH/dPRIZJNgrPKCdNoE3Q9ul8/bkcJWgFRY6JgbjKjpKRE7XKomwrqL9KqqqpQ11jn2Q6FOk4QBcRMigmKNyxBKyDmrhjYzdwNmTovqEN31qxZONp8FMmL/XfDQiIKLIE/PGmDLMuA7N+bFPqrmk01aMprQuJPEgOuW+Fa4u6NQ35kPiZOnAi7naNe6pigHOnabDZkZWXBEmdBqDFU7XK6JWeNE0DL6bLBRherQ1NZE06cOMG+XeqwoBzpWq1WLFmyBDW9ahBxExcpJyLlBGXoEhGpJehCd8uWLbhv2n1ImJsAfVLwfTTuKskpoez3ZbD926Z2Kaqz2+2YNGkS9u3bp3Yp1I0EXejW1NTgeN5xhCSFBMxKWIqSAXupHfoUPYwDjWpXoxptjBZhI8Jw7MSxlu3RidopqFKntrYWjbZGaMI0XC+3i0yjTDCNUncLdTXpk/SIvScWgsgXEnVMUHUvTJ8+HQVyAZKfS2boEpEqgip0JUmCDJmjEyJSTVCEbnNzMz766CM09WiCIZyL2hCReoJiTrehoQHPPPMMaq+rRcTN7MslIvUERegSEfmLgA/dLVu24LGnHkP87HiEJISoXQ4RBbmAD12z2YyDhw7CcL2BfblEpLqATqHGxkbYnDaGLRH5jYDuXpg6dSpOhZxC7+d6sy+XiPxCQIeuy+WCpJUCbktwIuq+AjJ0m5ubkZOTA1sPG0Ii+eWZL9jO2iCGigjtG5zrEbvqXGg+3dyyED5RBwTkZGddXR2eeOIJWAZaYBoZvOsD+IqgF2D50gLLvyxql6Iae4kdVRurEGYIg1YbkGMX8hG+WqhDBJ2AlJ+noOJ/K9QuRXWhoaHIz89HRARPuKH2C7jQ3bJlC/70lz8hflY8dHGBvy240gRBaHnVcJocAKDX66HRcDdpar+Am14oLi7Gnq/2IHRgKDSh/MdARP4loEa6NpsNTqcTkAHZIUMW+CWHz8gtuylLDgmCLrh2U5ZdMkRZhNEYvIu4U+cJbX37arFYulVqTZgwAceOHYPL5YKgDZ4QUIPsbnlpiAYRfZb0gagPuA9NV1W+vhzfv/77WLtmLQwGQ1C94VD7REZGXvVFEVAj3V/84heor69Xu4ygcfjwYbz51ptql6G4SyPd0NDgbJejrgmo0B0zZozaJQQVURQhyzKshVYYrjdAawqol9NlZLcMa5EV7ka32qVQNxY8nwnJ60RRhF6rR+WGStjOBv7uwLJTRsX/VkCukBESwpNuqHMCe2hCPjVhwgQUFBQgPT1d7VIUtXbtWkycOFHtMqib4kiXOk2r1SIqKgpr1qzBgOYBqN1Zq3ZJPmM7Z0P9p/X403t/wsiRI2EwcNsn6hyGLnWJRqPB5MmTEeuKRVNeE6zfWj2dDYHCXmKHtcAKx7cOTM6cjISEBLVLom6MoUteodVqIZVLKP9zOWRnYIXuxZyLqP9XPXQ6HdvDqMsYuuQV77zzDtatW6d2GT6TmZmJ48eP8ws06jKGLnmF0WjEyJEj8e7v30Xj5kbYznX/bgbJIaHy/yoxb/o8zJs3DxERERzpUpcxdMlrEhIScN/U+2AvsMN60gr7BbvaJXWaq96F5lPNaDzSiNHDRuOWW25RuyQKEAxd8ipBEKDVamHZbUFNdg1kt9ztFvqWJRnN3zajIqsCWlHL0S15FUOXvEqn0+HIkSO45557YDtrQ/EviyHZJLXL6pCKrApUf1KNiIgIFBQUYPTo0WqXRAGEoUteJQgCoqKiMHv2bCyYvwBSk4TqT6q7xRlrkl1C1YdVsJfaMWLYCKxatQpxcXHQ6bguM3kPz0gjnxg9ejTCwsLwzTffAACqa6rRKDdCEATo++j9arNQe5kdkk2Cxq1BmjYNwhABEyZMwIMPPqh2aRSAAmppR/Jf8+fPxwcffABJltDn5T7QhGtUnyuVZRmQgbI/lMFx3oHY2FgUFhZyzzPqsraWduT0Aini1VdfxQcffADIQOkbpWg81Kh2SZCaJJz/5XnYS+x44IEHcPDgQW69Qz7HkS4ppqKiAlu3bgUAbMrdhNzTuYAAxGbGQjQo9/5fu6sWrosuhBvC8fyU5yEIAgYMGMAvzMhrgmYRc/Jv8fHxmDVrFgCgrq4O5kNmFBYWImxwGESjCDFEREgv35zx5ap3wVXrAgA0HmpElByFtIw0PP744xBFfuAj5XCkS6rJzc3FHXfc4TnWJ+vRa36vy39QQIfmfy/N1X5X/b56XMy+6Dl+4okn8MYbb3S4ZqL24EiX/FJGRgYKCws9x3kFeXjwscs7BpIWJEEb1f6XqvmPZjirnK0ue2z6Y3ix8EXPMbfaIbUwdEk1Op0O8fHxnmNRFLHy5ys9x2azGatXr8bFzy52aONLR7kDt918GzIzMz2Xpaent3osIrVweoH81unTp/Hkk0926rbTpk3DnDlzvFwRUfu0Nb3A0CUi8jL26RIR+QmGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpNj1WcMAAADNSURBVCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKYugSESmIoUtEpCCGLhGRghi6REQKEmRZVrsGIqKgwZEuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERAr6f+Db/fBmHuzBAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXwU5f0H8M/Mnjk32YSEhJBIgADhCGcVCEFJBa0oKLUqWlsLXigqCiimKB4RscLPEy2KArURELQthkuOQkHRopDKEQgUA0IIIcdmk+w98/sDsk0g4QjZmc3u5/168XqRnezON3t89plnnnkeQZZlEBGRMkS1CyAiCiYMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBWkvtNFisXA8GRHRZTKZTEJz29jSJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBF5za0ZdkWYYkSd6fNRqNz/fp8Xga/SyKIgSh2RnYiIhanWotXVmWMWzYMKSlpeGWW25RZJ933nkn0tLSvP++++47RfZLRFSv1Vu6eXl554VZSkoKpkyZ0ug2QRDw+OOPo66uDmazubXLaNLvf/97mM1mLFu2DADgcrkU2S8RUb1WD91t27YhLy8PAJCRkYHjx48jNjYWw4YNQ0ZGBkpKSlBaWgoASE1NBQCEhoY2egybzYY9e/YAADp06ACz2Ywff/wRAJCQkICkpKQL1uB0OvGf//wHstx44Yvhw4ejvLzcG7oHDhyAwWCAVqtFRkYGRPFMw9/j8WD37t3e7g9RFNGvXz8cPnwYVVVV5+1PFEX07dvX20UiyzJ2794Nt9sN4MwXTEZGBnQ6nXd7QUHBeaHfsWNHtG/f/oJ/GxG1bcK5wdRQS5brmTRpEvLy8iCKIgoLC5Gbm4vFixdDo9Fg3759mDdvHhYsWAAA3lDs2bMntm/f7r2tsLAQgwcPhiAIeOaZZ3Dbbbdh0KBBEAQBTz31FP74xz82u39ZlnHixAn06tXL+/j1/bZr167FgQMH8NhjjzW6PSIiAkVFRdDr9QAAq9WKLl26eEPRaDTi0KFDuP/++7FmzZrzHlen0+HgwYMwmUwAzoR+t27dYLFYvL9XWFiIdu3aATgT6unp6Th16lSjx8nNzcWkSZMu9yknIj+jynI9kiRhyJAhWL58OTIyMrBv3z7ExsYiJycHhYWF2L9/P+Li4s6739y5czF69GgAwPr16+HxeHDDDTcAAFavXo0nnnjigvtdvHgxhg8f7g3G8ePHo7CwEIWFhejXr1+j312yZAleffVVWK1W9O7dG19//TVWr16NAQMGwOl04v3338dbb70Fm82Gvn37Yty4cZg/f773/m+88QY++OADOJ1ODBo0CKtWrcKOHTvQu3dvWCwWvPLKK/jkk0+8z8WyZctQUFCA9PR0nD59GsCZL5z6+n73u9+1/AknojbBZ6MXBEHApEmTsG7dOhw5cgTz5s1DTk4OIiMjERkZCUmSmhyxUFNTg/LycgBATEwMADT6OTw8/IL7HTBgAB588EHk5uYCAHbv3o25c+cCwHmtyOjoaNTW1kKWZZw6dcrbQp08eTJefPFFfPHFF94RD2VlZTAajY26Qkwmk7eesrIy2O12OJ1Obwt27dq13tbv6dOnYbPZkJCQgCeffBK5ubmwWq0oLS311jd69GhkZWU1+7etX78eu3fvxrRp0zjqIgjIsozc3FxUV1df0eMMHz4cN910UytVRVfKp6F777334ujRo/j222+xYMECPPnkk4iMjPTVLgEAvXv3RkJCAlavXg1ZlnH69Glvd8a4ceMuev+YmBgMGDAAgiDg4MGDjYa1Xa4jR47AbDajb9++3seOj4/HAw88gPXr16OiogJ1dXXe+rRaLeLi4tC9e/cmH2/nzp1YuXIlpk2b1uKayP8cO3bMe+TTkCzLWLRoEeTKSkS1cEjlMZcLFosFiYmJTW5PS0tDWFhYix6bWsan43Qb9hfXt8xkWfb+a6ipcGvq9y5ln2azGRs3bgQALFy48LJCatOmTZgwYQIAYPbs2XA4HLj33nub3VfD/t2G/2RZxvTp0zF+/Hjv79ffDgArVqwAcKYlPmLECADA/PnzsXXrVmzbtq3JfZ37f7Z226Zz39dz587FokWLmv39CWYzxkRFtWhfk3/+GcuWLfOePD7Xxo0bG3W71b9/yXd8FrqSJCEzMxNWqxV9+vTB0qVLERsbi9zcXPz1r38FAO8ohgMHDqBnz54AgIkTJ2LVqlW4+eabccMNN8Bms13Wfuv7aevV1ta2+G946KGHLrh9ypQpEEUROp0O27dvR1JSEjQaDQoKCpCZmYmcnBxvNwcAzJo1C927d8edd97pva3hCIann34aEydOPG8/sixj1KhRiBlwLUbNfAc9e/ZEfn6+d/QHtS05OTn44osvvD9bLBak6vWY2czIlVCx5adeXkpIgKeJhkulx4Mnjx/HHXfc4R1VA5wZ8nnuuQ9qXa0euuPGjfMGaL34+Hjv4U1WVhaio6Obvf8111yDTp06ecNqw4YN2Lx58yXvv2/fvnj00Ueb3JacnIyIiAjvY3fq1Mn7RQAAXbp0gd1ubxSUDaWnp6OwsND784033ohevXpBFEVcddVV3tEPSUlJeO655+BwOBrdv1evXoiJiWm2vqFDh3pHOJyrrKwMkTIQYopGSUmJdzgatQ2HDx/GwoULAZw5mpLKynBTfVeb0QizVosYbeu3gZrrlggVRUwwmyFLEuBwwC7LyKusxNtvv4327dsjNDQUM2bMUORK0WDT6q9ydnY2srOzm92elZV1wZNFJ0+exIkTJzB8+HAA8A7l6tq1KwwGw0X3n5GRgYyMjGa3JyQkID09vdFt5/ahdu3atcn7FhcXo7i42Ptzeno6brrpJlx11VWNfk8UxSZbrPUeeeSRZredy26349ChQzAlpiDcHAeNXo/2XdNxpPgoTCYT4uPjL/mxSFmSJKGwsBCSJGHvDz9g9Ycferf1MhoxtoVdBq0hRBQbdVlYPR58U1uLH778EgAQHhmJm2++GRqNBlFRURcdG0+XrtXH6V6pWbNm4Y033mh0W+fOnbFz507V+5rGjx+P1atXN7otOzsbK1eu9Nk+CwsLcc0112Dq379DTMdO3tvn3joYv/v1GMycOdNn+6aWqf9MOZ1OdOnS5UwXm9GIl5s5meWPqj0e3FtcDAnAnXfcgffef9+7Te3PYVtwoXG6qk1405wnnngCf/jDHxrdptPp/OKFfvPNNxv1FwNASEiIStWQv5o6dSrWr18PDYBXo6JgiIqCzg/ev5cjXBSxoGNHAEDB11+jT58+AIAFCxZg8ODBapbW5vldS5f+Z8OGDdi47Wv81xOKq8f9DiGR/zsc/O7zJQitLkVaTBgee+wxv/hSCmYHDhzAp59+CgCo2bIFzuJiiABuNpnaXOCeq9jpxL/r6gAAEddfD11CAgwGA6ZNmwatD/qhA8GFWroMXT82a9Ys5P19Naas3N7k9k0fzsPB/Dzs3r2boauin3/+Gf9ctw5znnkGADAhJgb9z5lPJFC8XVaGQrsdxtBQfLRqFTqlpl70gqVg1Ka6F+iMSxmf3PBVlWWZwaughq/P9OnTUfbPf+Lds4fjgWzy2dE1dZKEEcOH45OlSzFy5EgA7Ou9VGzp+iFZlnHTTTchZsB1SL/x14hq3/SZY1t1Fcr+ewBrX5qMzz77DF26dFG40uC1ceNGTJ06FQAwzuNBLwDmIDrUlmQZp9xuLBdFHBZFGI1GbN68GUajUe3S/IIqE97QlTl27BicEJoNXAAIiYxCeLv2OHLkCJxOp4LVBbdly5bhn8uWYXBVFQZXVSFVkoIqcAFAFAS01+mQYbNhcFUV+p8+jbmvvYaDBw+qXZrfC653ShvgcDhw7NgxRMQlNjpx1hyNVoeY5FScOFmKuLg4xMbGKlBlcJIkCUeOHEF+Xh7cu3bhIT7XGH62P9chSZj6zju4qmNHREREICEhQeXK/Be7F/zMnj17kJmZielf/oDoxEvrI5RlGW/+Jgu/+dX1eOmll3xcYfCqn2f56ehoDAgJYR/mOWRZRm5pKcxZWVi6dKna5aiK3QsBjh9+31u/fj1GXXcdXm/XDj2NRj7nTRAEAQ/FxqLX/v3IzMxE3dlhZtSYZtasWc1udDgczW+kVrd582as3bQFId0HovMvhkFnuPSTEpLHA9njwbGD+7xTU1Lr+Oyzz/B9fj4i9+zB4LAw6K9gAppAFyqK0LjdqKuqwiGnE7FB2uVlNBpfaG4b3z1+5KuvvsKnn/8NIx95FiERpsu675C77ke5EIK33nrLR9UFH0mSUFxcjM8XLcLhNWtwZ3Q0NPwyu6gkvR63R0Tgk3nz8N22bd7ZBOkMhi5RM6xWKwYOHIhrDh/G/WdXMaFLowPwdlISlj73HNf9OwdD1w/IsoxbbrkFJwyxuPtPH7f4cQbcchdGz3oXQ4YM5dCdK7RmzRqMGTUKf4qPR3f24V62+snQ/xATg/6HDiErKws1NTVql+UXGLp+QJZlFBUVwSloEZvc8onJw6JjENXhKuzfvw92u70VKwwuK1euxLd//zu6nTyJFJ0OIezDbbF2Wi06eTzo8vPP+OC993DgwAG1S1Idx+mqzOl0orS0FGHmdjCEXvk17KJGA1N8B5RXVqKqqgpRKs7Z2tZ4PB6cPHkS7733HiIKCzGlidWq6fIl6fW4W6fDPa+8gtDISJjN5mYn6w8GHKersl27dmHEiBF4evVuRMYlXPFhbP3r+fb4ERg7Yhhmz57dGmUGhYqKCqSlpcHtduO68HCGbityyTLu+ekn2GQZWVlZ+Mc//qF2ST7Fcbp+TpYBtNKCgN6FBfl1eVm+/PJLjB49Gm63G4/GxuLuCywpRZdPC+DVxEQMCAnB999/j+uuu+6Kl5Zvqxi6KvrXv/6FtZu2YOj4B6APad1lsPv+ahys2jAsXrz4sldUDkYVFRUo2r8foyMj0TskBHENFmukKycIAjoZDMgKD0cfAD/++CM8Ho/aZamCoauiVatW4ZMVX2D01JcREhHZqo+dde+jqNSbMHv2bIbuRZSXl8NiscAoCJgQE4MEBq7PXBcRgV9FRkKWZZSWlgblVWsMXQp648eP51pzCvN4PBg8eHCjpeiDBUNXBZIkYdy4cTgdnojbX3zXZ/vJuOE23Pz8O/jlL3/ZaOl4OqOyshLZ2dnYs2cPrg4NxYsJCfxAKKCrwYDXEhNhBDBnzhw89thjapekKL7HVPLjjz/CpdUjvnM3n+0jIiYOsZ264Yddu4LyMO5iXC4Xdu3ahT6yjKFhYehsMPAiCAWEiiLSDAaMioyEp6QERUVFapekKIauwlwuF8rKyhBiiobO4PuVhAVRRLg5FtVWK6xWq8/311bYbDaUl5cDAMZGReHaiAiVKwou4tmr1bobDN7PhCRJapelCIauwgoKCpCe3hPj3/4MA8fe7fP9hce0w4x1P2Jyzgt44YVmJz4KOnl5eRg6dGjQfND92c6dO5Geno6Kigq1S1EEQ1dhsizDI3kgiKIih7KCIEAUNZAkiaMYGpBlGaE4M3Y0Ra9Xu5ygdXt0NB6KjQ2q4WMMXQXt2LEDW3f8G4PG3g2d0fddCw2lX3sj7EYTli9fHvThm5+fj2+//RYaAN0NBoRybgXVJOp0SNHpIMsyli9fHhQTNfHdpqClS5di0bKVuG3m/8EY3rrjci9mxMQnURMRj5ycnKA/pH799dfxtxUrEK7RqF0KAdAIAsIFATnPPovt27erXY7PMXQpKGWHh+OdpCRwrIL6uhkMWJSSgoggOeIIjr9SZR6PB+PHj0dNbCeMmfGaanWkX3sjRs98E7eMGYN9+/apVoc/EAQBmlaa74KujCAIQRVEwfS3qurbb7+F2xCGDj36qFZDZFwCEnv0xdfbvw7K4WM2mw1Lly5Fqt2ONINB7XKoAQFAZng4Tuzeja+++krtcnyKoetjbrcbFosF+tBwaHV+cJZcFGCMiESdzR50F0xUVVXh4YcfxhCLBdkcl+tXxLMrCf/n888xZ84ctcvxKU5i7mM7d+7ELWPG4Kl/7ER4jPoTN4dFxSDnq7146qFxyMrojjfeeEPtkoiCClu6PiZJElwuFzRaLURR/bPlgiBAo9PD7fEE1dhIIn/B0PWhnTt34vv/7EWfkWOh0ftXH2KXq7PgDovGqlWrgmIIWVFRETasWYPMsDAOFfNjXQ0GJNts+PzzzwN2nT8u1+NDkyZNwtf7DuPBj75Uu5Qm/fuLT7D5nRdRVFQErTawe5rmz5+P12fOxMfJyRyx4OcO2O2YXlKCwsJCxMfHq11Oi3C5HiIiP8HQ9QG324377rsPrg7dcMMTz6tdTrO6Dr4Oo3PmYfzdd2Pv3r1ql0MUFBi6PiDLMjZv3gwpLBopfQapXU6zotp3QEr/IdiwYQMqKyvVLsdntmzZwi+VNmj9+vU4cuSI2mW0usDuyFOBx+OBzWaD1mCEqOIJG5fDDvkSTpC57DbIMuBwOAJ23O6zzz6Lwr170S7A+60DhQBAD2Dy5MmYM2cOHnzwQbVLalV8F7ayb775Br+58048vvIbhJtjVavjg/vHoKToEi71lWXIkgd33XUXxAC99t3hcOBXkZG4z2xWuxS6BF0MBvwlJQUPHDumdik+wdBtZR6PB3abHVq9EaJGvafX7XSin0bESF55hQ/Ly6EBoA/QL5VAIwoC/ODaTZ9h6LaigoIC7C06jO5ZI6HRqf/UJmi1uDosTO0yVJcXwP3V1PaonwwB5J133sH3P5Xg/j8H37LSRHRpeLxFRKQgtnRbgcvlwuTJkyFc1QvZI+9Ruxwi8mNs6bYCSZKQn58PwRSH1AFD1C6HiPwYQ/cKeWcR0+kg8Ow4EV0EU+IKbd26Fb369MHDef9Er+yb1S6HiPwcQ/cKud1uWK1W6EPDoNHp1C6HiPwcQ/cK7Nu3D4ePHkfqwExVL4QgoraDSXEF5syZg32nqvGHd5erXQoRtRFs6RIRKYgt3RZwuVyYPn06DF36YtjIXmqXQ0RtCFu6LeB2u7Fs2TJo45LRdfC1apdDRG0IQ/cyybIMSZLOzJXLtbaI6DIxdC/Tpk2b0G/gIDyydCt6ZI1UuxwiamMYupfJ6XSiorwcIaZoaP1sWXUi8n8M3ctQVFSEn0vLkNSrf8CuskBEvsXRC5fh+eefx39rPHjww3+oXQoRtVFsrhERKYgt3UvgdDrxwgsvIKxbf/wipYfa5RBRG8aW7iVwu91YuHAhjB3TOGKhjaryeHDE4YAsy2qXQhdhlyQccjjgCdDXiqF7EbIsQ5ZlCByT22YJALbW1uKFkyfVLoUuQbHTiaknTsAqSQH5uWP3wkVs2LABT06dhkeXbUVEu/Zql0MtMCshAWuqq7GuulrtUugSCYKALVu2oHPnzmqX0urY0r0Iu92Ok6WliIiNh85gVLscaoEojQahHOLX5sTHxyMsLEztMlod34kX8NNPP+FURRXad+kBQeBTRURXjt0LFzB9+nQcd+kwack6tUuhK1TfMygDAPvo/Vb9ic5Afn3YfKOg8MuICPyxfXs8cOwYfnI61S6HmvGXykqsio9HQUEBYmNj1S7HJ9jSbYLD4cCf/vQnRHQfgL7JaWqXQ60gVBQRrdHglNsNd4AORQoEVo8HdRoNOnbsqHYpPsOWbhOcTifefvtthHfujV4jRqtdDhEFEIbuOTh4PrB5+3b5OvudYHlNGLrnWLduHUZcPxKT8jYhdeBQtcuhVhSl0eD9jh2RV1mJTysr1S6HGpBkGU+dOIGsJ57AkiVL1C7Hp9ine47a2loUHz0Kc2IKtAbOlxtINIKABJ0OFkmCRZLULocakAGccrkQajYjMTFR7XJ8ii3dBk6cOIEqay1iOnbiUjwBrlaScNLlCppDWn/mkCSUuFwIlq9BtnQbmDRpEir0UZictymgxwkSsLWmBgV1dVickgK+0uo65HBgRkmJ2mUohqHbBAZuYHsmLg4brFas4VwMfkMURaxduxbdunVTuxSfY+jizPwK8+fPR3T6QCR05LjcQBen0yFKo4FDlvFZVRWyIyIQq+VHQQ3f1dbiu7o6AEBqaipMJpPKFfke+3Rx5mKI2bNnI7rX1egz6la1yyEFhIgiIjQa/LWyEmVut9rlBK1vamuxxelESkoKNBqN2uUogqFLQena8HDMTUxkf64f6NevH3744QdERUWpXYoigj5016xZg9Fjb8PDS9Yhuc8gtcshhQiC4B2hMu/UKeRVVKhcUXDxyDKmHz+OHXV1EATB+y8YBH3oWiwWFBUVIS61GwyhgTd3JzXPIAi41WSCS5bxvc2GLy0WSBxC5nOnXC58YbHgv04nBo8YgXHjxqldkqKCOnTLyspQY7MjMi6B43KDUIgo4vcxMUgzGFDicuHTykowcn2vxO3GkooKxLRvj7vvvhsTJ05UuyRFBfUp2/vuuw+1pkQ88dm/1C6FVDQjPh5fWa1YzC4GxWi1Wmzfvh3R0dFql6K4oG7pNpwwOVj6k+h89a99rSThyePHccjhULmiwPWXigrMLysDELyfu6Bs6drtdixatAjm9AFISumhdjnkB5L1eoyMiMA6qxUbrVZUezzoHxqqdlkBQ5JlrK2uxg91dRDj4/Hg2LEwBOncJkHZ0q2rq0NOTg7aDxqBvr/6tdrlkB/objTiXrMZMRoN1lVXY0tNjdolBRQPgMUVFTgdEoJBgwYhNzcXoUH6pRaUoUvUlDBRxMLkZHQzctVnX3nttdfw8ccfq12GqoIudPPz83HHPb/Fgx/lo0OPPmqXQ36kvo/xkdhYdDUYkHPiBJf2aQWFdjtyTp3CilWrcP311wdlP25DQRe65eXl2Lt3HxJ7ZMAQFqF2OeSHkvR6dDUYkKzXY111NU7zMuEW+6GuDoeiojD8nnswYOBAmM1mtUtSXVCdSKusrITd6UJodAyH5dIFdTMakaTX49FjxxCj1SJEFBEmBl0bpcVkWUa1JOFfAEzdu+OjOXPULslvBFXo3nXXXfAkdMGUFdsgCPwA0YWFCgIWJidj1smT2FZTg6nx8WqX1Ga4ZBkPHD2K+QsXYszYsWqX41eCKnkkSYIsyxBFTdD3K9HFCYIAURAwMSYG6UYjZpaUwMU+3ovaZ7fjhfJyLPvb35A1fDhEHiE0EhQtXZvNhhUrViCmR3+EpfZUuxxqY5L1etglCUecTmyyWtE/NBTtOP9uk3bV1aHEbMagG27A1ddcE7RjcS8kKN45VqsVjz/+OB5YuAopGb9Quxxqg9KMRnTU6/HwsWOI0GgQLooIYQvOS5Zl1EgSNkkS4nr2xAfz5qldkt/iu4boEhnP9vFutFrx5tlLWekMhyxjwtGjuH/uXLz//vtql+PXAr6lm5+fjwUfLcKE91YivnN3tcuhNkwQBGgA/NZsxn67Hc+eOAEAuMdsRnqQXlCxuLwcBxwO6AwGfLpyJfpkZATNChAtFfChW1JSgp27dmHknMEQRb4Z6MpdpdfDJcsotNsBAAftdrhlGX1CQlSuTDluWca2mhqEDxqEnjExMBgMGJqZCb1er3Zpfi+gQ7empgYujwdGXgRBrayrwYApcXEAgNdLS1HocCD1bOAYRRHaAB0dY5MkeGQZDgCLnU58MGkSRowYoXZZbUpAh+6tt94KbafemLJiO8flks9MiYvDv+vq8NviYgDAtLg4DAkPV7kq35hTWooCmw3h4eHYu38/QsO42srlCujQdbvdEGUZGp1O7VIogGkEAT2MRjzXvj0A4Lu6Oqy1WqEB8HR8PIxtfJTD7ro6fG6xAADumTEDM3v3hkajQWhYGMfgtkBAhq7NZsOaNWsQ2y0DkWkZapdDQcCk0aDf2akKj7pcqPZ4IODMEuMaQUCURtOm+nxdsowdtbWQAdTExSF1yBAAwODsbPTsybHuVyIgQ7eqqgoTJkzApCXrkNSzn9rlUJAZYzIBZxe8vP/oUdgkCT2MRnRpcKGABoDBj1qJsizDLsveNeJqJQl/rquDBGDcgAH46M031SwvoARk6BL5Ay2AD5KTIQPYa7Phtz/95N3Ww2jEy4mJapV2HoskYeLRo94lrKLNZuw/cAAajYZDwFpZwIVufn4+/rJ0Oe594xPEJKeqXQ4FMUEQUH82obPBgBln+3z/brHgsMOBF0pKvL+bpNdjQkyMYrVZPR78X1mZN2TdAJyyjJdeeglpaWnQ6/UICQlhn60PBFzoFhcXY/uOHZjx/AIIfMOQn4jUaDDwbJ/vSZcLIQ2GlP3X6USJ293sBRYpej0SW3gyeLfNBpsknXe71ePBzro6ZGZmIioqCgDQEcCoUaOQlpbWon3RpQmo0LXb7XC5XIAMOO22oJ5JTJbP/6CRfxhtMmG0yeT9+c+nT2OD1Yp5p04BOHPhgSQIMBqNsNvt+G10NEZHRrZoXx+cPo0SWYauidAOCQnByy+/jL59+7bsD6EWEeQLTFVnsVja1Dx22dnZKCgogNvthtYQnJdl1nM7HbglIgITY2PVLoUuwi3L8DT4HH5UXo7jnTph7dq16N+/P8pOnoSmhQ0IlyzjoYcfxnPPPdfkdoPBwC4EHzCZTM2+YAHV0p01axaqq6vVLsMvPPvss0Blpdpl0CXQCkKjK9hEQYAoiggJCcG7774L+9nLjVsqNTUVIW1ouFqgC6jQHTZsmNol+I1XX32VodvGCYKA7OxstcugVsbjCiIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBWnVLoB8Z0tNDfbY7WqXQZfptNuNVLWLIJ9h6Aao22K4whYAAAC9SURBVG+/HSWZmWqXQS3UoUMHtUsgHxFkWW52o8ViaX4jERE1yWQyCc1tY58uEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkoAsuTElERK2LLV0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFPT/fTaaZjFnkwIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } } ] }, { "cell_type": "markdown", "metadata": { "id": "bykihXc8UhND" }, "source": [ "Just looking at it, it seems like some of these groups are more **diverse** than others. Some contain all different sizes, shapes, and colors; some look mostly the same.\n", "\n", "For example, the `Green shapes` image has a pretty low color diversity (they're all green), a medium amount of shape diversity (2 of the 3 shapes are represented), and lots of size diversity (all three sizes—small, medium, and large—are represented).\n", "\n", "What happens if we try and quantify that?\n", "\n", "Run the cell below to see one way of doing it:\n", "\n", "\n", "\n" ] }, { "cell_type": "code", "metadata": { "id": "6kZ0Ez1aUvoT", "colab": { "base_uri": "https://localhost:8080/", "height": 793 }, "cellView": "form", "outputId": "89b2ec96-d43c-42cb-e4f2-16a7d19724b5" }, "source": [ "#@title Try out a simple diversity scoring function:\n", "simple_diversity = DiversityFn(\n", " aggregate_method='average', \n", " count_or_binary='binary', \n", " verbose_output=True)\n", "greens.show()\n", "subscores = simple_diversity.score_image(greens)\n", "print()" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXQUVd4+8Keq13S601kgKyGsAkGSiC/8BFkcDBlAEGfUURwVD6i4AIoM6ojiwJHlOC64MkdE4T3oCKIiBAFHQHYMgbCHVQhJCEnI0tl7q/r9kZceI1uW7qpO+vmcwzlUL3W/6U4/uX3rVl1BlmUQEZEyRLULICIKJAxdIiIFMXSJiBTE0CUiUhBDl4hIQQxdIiIFaa93p81m43wyIqImslqtwrXuY0+XiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgVd99KO1JDb7W6wLYoiBOGaV3AjIroCe7pN8Mwzz+Cmm27y/Fu/fr3aJRFRK+OTnu6ZM2fwwQcfeLYTEhIwbdo0XzSlqL/85S8wm81YsmQJAMDhcKhcERG1Nl4P3bNnz2Lbtm1YunQpkpOTkZ+fj3bt2mHw4MFITk5GQUEBCgsLIQgCkpKSUFhYiIsXL3qe36FDB4SGhuLIkSOe24xGI/r06ePZttvtOHTokGdbr9cjKSkJx44dQ01NjWc7Ozsb1dXV0Ol0SE5ObvRQgNvtxoEDByBJUoPb+/btC4fD4QndM2fOYO/evRBFESkpKdBoNAAAWZZx4MABuFwuAIAgCJ7Xori4+Ir2Lr8Wer3e8/zDhw/Dbrd7HtOnTx8YjUbP9uHDh1FXV9dgP9HR0YiPj2/Uz0hE6hBk+dor8jRnuZ5nnnkGX375JURRxMmTJzF37lwsXboUoiji2LFjeOedd/DJJ58AAI4cOYIPP/wQixYtgiAIkGUZs2bNwsiRIzFgwADPbV26dEFmZqYnNM+dO4dbbrnFc39cXBwOHTqEP/zhDzh06BCio6Nx5MgRpKWlISsrCxEREcjOzoZWq71h8MqyjMrKSnTr1s3Tk738nC+++AKCIGDcuHENbtfpdDh58iSsViuA+h5wjx49YLPZPI87fvw45s2bh2XLluHya/7bWo4cOYLY2FgAgCRJSE5ORl5enudn3LdvH7p06eKpsX///jh9+nSD/UyZMgVz5sxp6ltGRF6m6nI9s2fPxvHjx3Hs2DG0a9cOM2fOxKpVqyDLMoYOHYply5ahR48eyM7ORmxsLN555x2MHj0aAPDjjz/i2WefRU5ODnr27ImzZ8/iX//6F1JTUwEAa9aswYwZM1BQUIBevXph3rx5+Pvf/46ioiIkJiZi1qxZmDVrFkpKStC7d29kZWXdsN4ffvgBt956qydwhw8fjuPHj+P48eMYNmxYg8cuXLgQixcvhsPhQL9+/bB27Vrs2bMHffr0gc1mw7x587B8+XJIkoSBAwciMTERK1as8Dz/96/F0qVLceLECfTq1QsXLlzA9OnTkZ6eDgBIS0vDRx991OC1AOq/GVyub8aMGS1/w4jIp3w2e0GWZcyZM8fzlVgQBMycORMhISEIDw8HAFy6dAmjR4/GPffcg6ioKLz66qtYuXIlfv75ZwDAkiVLcPLkSbjdbhQVFcHlcqG6uholJSUAgGXLliE3NxeSJKGoqAghISGwWCxXbMuyjKKiIjidzhvW3aNHD0+P0eVy4dSpU3j77bcBAOPHj2/wWKvVCrPZDAAoLi5GXV0dOnbsiGnTpmHOnDnYsGGDp/d76dIl6HQ6hIWFeZ5vsVgavBY1NTVwuVwoKioCAGzfvh3nz58HAJSUlKC6uhphYWGYPn063nrrLRQXF8Nms3nqGzp0KO66664mvlNEpCSfhu7+/ftRUFCAkpISCIKAF154ASEhIQ0eN2DAANx3330AgIceeggnT570hO7BgwdhMBiQkpICAA3GNIH6r+RGo9Fzv8lkanHd3bp1w+OPP4709HQ4HA6Ul5d7hkOGDh0KUbz+l4OQkBD069cPWq0WZ8+e9YzTNkdubi4cDofn54uOjobVasWkSZOwc+dOnD9/Hg6Hw1OfzWZDQkICevfuzalsRH7K66F7+cMuiiK+++47z5juZbIs47fjyJe3Lz/vt2GxfPlydO7c+Yp9X7ZkyRL06tXrmvc3hyzLMBqN2LBhAwAgPT0djz76aKOfv3//fowdOxYAMH/+fHTp0gWjRo26ZltXG1O/PI77xBNP4Pnnn7/idlmWPa9pTk4ObrnlFgDAihUrsHPnThw+fLjR9RKRsrweunPnzsWIESPwyCOPYNCgQaisrLzi/t+G8JtvvomffvoJq1evBgBMmzYNw4YNw5gxYzBixAhotf8tcd26dZg0aRKGDBmCtLQ0jB07FjqdznP/t99+2+L6169fj+nTp3u2fz9DoClmzpzZoP7fmzdvnqcnvGXLFnTv3h0GgwFHjx5Famoq3n33XSxevNjz+ClTpmDUqFEYOXKk57bfnrAxYcIEvPTSS82ul4h8z+uhGxoailtvvRVz585tcLsgCDCbzRgyZEiDcU0AiIiI8Pw/JCQEvXv3vuL5ABAWFgaLxYKePXte9f527drh9ttv99wXFxcHnU7n2e7YseMN6+/evTsmT5581fsSExMBwLO/m2++GXq93rOdkpICk8l01doAoH///qipqfFsDxkyBAMGDPC0e3l8OCYmBi+//PIVf7D69euH0NDQa9Z3yy23ICoq6oY/IxGpx+tTxuja8vPzsWPHDkyaNAkA8Pzzz+Phhx9Gt27dVK6MiLzpelPGGLoKev755xsMrQD1vePLBw6JqG1g6PqJ0tJSVFVVNbhNr9cjOjpapYqIyBcYukREClL1jDQiIvovhi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYJ8thowNc/p06cxYcKEZj33wQcfxDPPPOPliojImxi6KisqKsKqVas82xcvXsShQ4dg6WeBYGj86sZV+6uwfv36BrclJSVh0KBBXquViFqOoaswh8OBkpISz/bR7KN4bcFrDR6jCdEgLC0M2tDGvz2OAgd2HdyFXQd3eW575MFH0LVrV8+2yWSC1WptQfVE1FJcOUJhmZmZGD58uGdbH6dH7NTYqz5WEBrf073a+1ixowKla0s92xMnTsRbb73VhGqJqDm4XI+feOedd/DVV1/h5MmTiJ4YDTFYhKgXoY/S+6Q9V6ULrnIXAKD4q2JYJSuSkpKwatUqiCKPoRL5yvVCl8MLPnbx4kWkp6cDALad3oaLERcRMjAExk5GiEbfBp/WooXWUv8WW/7HAkeZA9lSNpYsWQJBENCjRw8MHjzYpzUQUUPs6fpQRUUFduzYgYceeghisIiI0RGw/I9F1ZrcVW7kvZMHd7Ub9997P+bPn4+IiIgmDWUQ0fVxeEElkydPxhdffAEZMjq+1hEas0b1cJNlGZCBCx9fgD3HjoiICJw4cQJaLb/0EHkLQ1dBWVlZeOmllwAApZ1LUdOuBgBgiDdA0PhPb9JR4IBklyC6RMQeiAVkIDU1FS+++KLapRG1ehzTVcj27duxceNGZGRkwNzXjJCI+rFbf6SPqT94J9klZB/IRk12DWRZRseOHXH//fdDo9GoXCFR28SerhfIsozy8nI899xzWJO+BhqTBh1e7ABNUOsJroufX0Tt6VqYDWZkZmYiPDwcOp1O7bKIWiUOL/iYw+HATTfdBJvNBkMXA2KeiAHEps2zVZssyajaV4XilcXQaDT45ptvcMcdd6hdFlGrdL3Q5WRNL5BlGW63G9ZhVkTcHQFBI7SqwAUAQRQQ1DMIUROi4JbdVz3ZgohajmO6LVRQUIBNmzfB0NsAfU89DLEGtUtqNq1FC7GrCHNfM7bv2w6DwYCBAweqXRZRm8KebgvU1NQgIyMDk6dOhvkus98eNGsKUS8i8oFILFqxCIsWLYLNZmOvl8iLGLotMGXKlGZfhrE1SE9PR0pKChwOh9qlELUZHF5oAZfLBU2MBpGjIiHoWtcY7o2EjwpH9eFqODOc7Omq7LPPPsOaNWua9dz33nsPCQkJXq6IWoKh2wxutxvp6eko0ZUguE8wgroHqV2S1xk6GCC7ZNgr7fh+zfcYMngIYmJi1C4rYJSVlWHz5s0AgA0bNmDbnm0I6tH43zPZIaMmuwbff/894uLiEBQUhJEjR7a6A7xtEaeMNZHL5UJ5eTmSkpJg/rMZ5mSz2iX5lFQn4dzsc1i2ZBnS0tIQFNT2/sD4G7vdjszMTNx1110Q9AIgAPpoPeImxzV6H85SJ/LeyQMAyC4ZMZExyMjIgMlk4okvCuCUMS/atGkTEhMTUVNTo3Ypipo4cSKXAlLI7Nmzcc899wAAOrzQAZ1e74TYp65+zeVr0YZp0en1Tuj0eieEpYWhoKAA3bp1Q3Z2ti9KpiZg6DaRJElwuB2IfCQSxs6tf7bCjQg6AdHjo6GJ1sDpdKpdTpsmSRKefPJJrF27FnKIjOgJ0dCEaCDoBAjapg0LCIJQ/zydAHOyGe0faA+73Y6//e1vWLp0qW9+AGoUjuk2gyAIMPUwQTS0/b9ZgkaAqacJtp02tUtp08rKyrBt2zZs3LgRtaZaBN8cDFMvk1f2rYvQQTSKMN1sQsaBDFitVsTFxSE1NZVjvCpo+6nhRXa7HXanvX6cLcAIWgGSIKGmpsYvZjM4nU7U1taqXYZXOBwOHD16FOPHj0dFTQVCBoQgYnSEV9vQBGsQPT4aukgdNv60EZMnT0ZNTQ3cbrdX26Eb44G0Jnjsscew5dctCHswDIK29Z3q2xKyS0bdsTrUpdfh2LFjqh9Qe/PNN7F69Wrs3Lmz1b8PM2bMwLJly+BwOBD/93horVqfXQZUdsqw7bShdF0pjEYjNmzYgJSUFJ+0Fch4aUcvcTgccLqcEHWB9wXhck+3rq5O1TpkWca0adOwY8cOv+hxe4PT6YQcIiNqdBS0Ft8FLlA/Rh/cJxiaYA2KVxZDkiSftUVXF3jpQa3e1q1bkWPLgRQjYcOGDaiqqlK7pGaRZRmbNm1Cbm4uRJOI4N7Bipxko4vQIeim+m8qu3btwtGjR33eJv0XQ7cRZFlGXV0dx7/+j1qvhSTV97RlUYb5VjNcA10YN24cLly4oHgt3nB5tsKmLZuUX1VEqO/1vvrqq/jss8+UbTvAMXQboba2Fr1798ZPP/2kdimqU/O1OH36NLp27Qr3PW6EDg1VvH1fiRgTgegnohVtU2PRIOH1BOjj9Iq2SwzdRquuroZ5sBmhd7SdD3tTGToa0H5ce9TYa+ByuRRvX5LqZ09AWz/GrAnRIPLhSLw6/1WsWrVK8Xq8RdAKih8nEAQBokFs9QchWyOGbhMYOxkD4oSIa9FatQi+OViVD+rp06fxy/5fENQjyDNlTzSIMCebsWXHFp5pRa0GZy9Qq7B8+XJ89OVHiJ8Rf8V9gkaAJEtwOBzQ6/l1mfwbe7rU6sVOjsWXx7/EHXfc0WamkVHbxZ4u+TVZlvHyyy8jszwT4XeFX/UxokGEQ3Kgurpa4eqImo49XfJ7P/zwA85WnEVwYvA1H6MN00KOkrFt2zaGL/k1hi75LVmW62dJCKj/dx2WvhbIw2SMHTsWOTk5itRH1BwMXfJbx48fR/fu3YH7gZBBIWqXQ+QVDF3yW263G+Xl5YABjZrHqjFr0P4v7bHgwwX47rvvFKiQqOkYuuSXzp07h0PZh2DsbGz0BbxFgwhLPwvWb16P/fv3+7hCoubh7AXyS5988gk+/e5TdHihQ5OfKwgCZMhwu90QxVZw1pUMyJIMQVSuTlmW69sFp9gpjT1danNinorBNznfYOjQoWqX0igl6SW4uOSiom26K93ImZMDxwWHou0SQ7dJKnZVwLYrcJetsefbUbK6BLLku96RLMt4/fXXkVmWidDU5l3nQmPSwO62o6yszMvVeZcgCHjjjTdw2623QapT+Lq2MiBVS5j23DTcd999yrYd4Bi6jSCKIvr27QtDkQG1J9rGEjHN4Sp1oXpfNfqm9EVYWJhP2pBlGStXrsTpqtMwJzV/eXtNiAZoD2RmZvrtys2iKGLcuHHo0aMHJLuEuvN1kN2+/7rvqnDBnm8HAIwZMwYDBgzweZv0XwzdRjAajfjhhx8wcOBAtUtRncFgwLp163zyWsiyXD/W6IWhTUtfCzR3aTB8+HCcPn265Tv0IUEQ4Cpy4cKHF+Cucvv0VGZZllF1oAqFnxf6/1h3G8XQJb9x9OhR9ErsBe2DWoTcFjjzcmfPno0VK1YAMpD3bh4q91b6rK2CTwpQ9mMZIiMjkZ2djT59+visLbo6hi75DafTiaLCIggmAaK+5b+aYpCIiLER+HDph1izZo0XKvSNkJAQJCcnY8GCBQgWg1G1rwrlP5d7tQ13jRuXVl+Co8CB2/vdjlmzZiEqKgo6nc6r7dCNMXSboHPnzugU0wn2PLtPDyb5I+clJ0wuE5KTkyGK3v+1ycvLw6mzp6DvoPfa0jWiQYR1kBWrN6zG7t27vbJPX4mKisKTTz6Jfv36wVxhRuXeyvrfMy+M8boqXbDn2FGxswLdO3THiBEj8PDDD3N4QSUM3SaYO3cuXnv8NVz46AJkZ2CF7qXvL6F3RW9s2LABBoPB6/t///33MXnWZHR4rgM0Zo3X9y9D9vvLPoqiiG+//Rb33nsvnEVO5L+XD3el+79j3U1w+TmyLKNqXxUufnYRgiDg008/xeTJk330E1Bj8OQIavNinohB+sF07L5jN7Zs2eKTnro3vfLKK0hNTcX999+P/PfzAQHQR+kR82RMo/fhLncj/8N8AIBklxAdHY1NmzYhMjLSV2VTIzF0m0GWZJSuL4WlvwWGWO/3+vyJ7JJRurEUziIn0M0H+5dlLFiwAIcqD8E62Or9BlB/TYYqqQqFhYU+2b+3Wa1WJCcnY+7cuQDqL225+8BulKwpafQ+pDoJ7go3pk6diqioKJjNZsTGxnJIwQ8wdJvIYrEgsWcisndlw9jZGBCha9thQ0JcAjp27Oj9/csyPv/8c7gHuGEd5JvQBQAxWIQQLuDo0aPo2rUrTCaTz9ryhsjISDz77LMA6hfkLC8vB0qbuJPewPjx49G1a1fvF0jNJlxvrMhms/n3IJhKamtr0a1bNwT/KRjBSeos1KgEWZYh22XkzM7Bii9XIC0tzev7lyQJvXr18nnoAoDL5sL5N85j8+bN6Nu3r0/bosBmtVqvGQr+Pbjlp4xGI3755Rf0re6L4hXFapfjMzXZNShdVIq9v+zF4MGDvb7/w4cPIyUlBYa/GmD5H4vX90/kjxi6zSAIAjp06ACDZEDd2TqUbiiF5FT43Hkfq8ioQOUvlZAqJHSI64CgoCCvt2G325GblwvBLEA0+v5XUTSKCB8Vjs9Xfo5169b5vD2iq2HotkBsbCyiTdH1E9ndalfjXdUHqqHJ1aB79+4+Odp/8eJF5F7IhS5Sp9glDUWDiNA/hGJl+kps3rxZkTaJfo+h2wLz58/HwoULPdv+Pg+0sS7/HMOHD8eWLVug1+u93saCBQvw9GtPI/5v8T6Zl9sYbeX9otaFodtCAwcOxC+7f0H5J+WoOeafV7NqCqlOQt4/8/DuS+9iwYIFapfjE1Hjo/Cf8v8gNTUVktS2hoXI/3HKWAuZTCZ0SugEd7kblZmVkGokWPq1zoNC9gt2VGVVwXnJiaiIKJ9MpJckCe+//z5O2E/A8v/UeZ20oVrYJBtqz9eyt0uKY0/XCwRBQOfOnSGeE1GxpwLOS85Wd20GV7kLtSdrUbmtEl06d/HJgTOg/iv9Bx98gFOOUwjpr96VxESjCMEq4MyvZ1BXV6daHRR4GLpeoNfrsXXrVowcORL283bk/jNX+ZUAWqjo30UoXVcKs9mMPXv24LbbblO7JJ+y9LUg6KEg9O/fHwcPHlS7HAogDF0vEQQBc+bMwccffwxIQP4H+ag+XK12WTfkrnUj961c2HPtGDVqFLZs2eKzy/1lZWXhtgG3wfRXE4KTg33SBpG/45iuF8XExGDAgAGYPn06AGB7/nac2nQKAGAdbPXKNWK9pTKzEi6bCwaNAVP+OgWCICApKcmnp4zW1tbi1KlTSBiXAE2QOjMWfkvQCwgdFoqVP6xEeXk5/vjHP6pdEgUAhq6Xde7cGa+99hoAYNasWcjZkoPCokKYepiga6dT5CSA65ElGa5yF2w7bNDb9IjvEo9Xlr0Crda3vwqXLl1CUUkRtOFav/l+JRpEhI8Ix/IPl8NR6mDokiL85Ne/bZo9ezY+//xzQAby38tH1YEqtUuCVC0hd0EuHPkO/OlPf8LWrVt9HrhA/R+gp2Y9hfiX4iEG8deOAhd7uj4kCAJSUlKwa9cuAMDb/3oba95eAwECYp6KgSZYua/YhcsL4Sh0IMwahh3bd0AURYSGhip+sR5/vDhQ5LhI7Ph1B0aMGIF169ZBo1F/6IPaLoauj5lMJiQmJgIAxqaOhVgr4uuvv4Zthw2iQYTGooHlVt/MV7Xn2VF7un7J+LpzdUjsmIjRo0cjMTFRsQt5S5KExYsX45x0DsEp/nnwTBehQ8XpCpSfKOe8XfI5hq6C7r77biQkJNT3fLP/78YIwNXVdcVjNSGaJl2TwFXhAn43S004LyDoQP182yBzENLS0vDSSy81t/xmcbvdmD9/PnRpOlgH+vbSjS0h6kXIFhn5+fmIjo72yZJERABDV3FJSUk4cuSIZzsrKwvDhg274nHxr8RDF9b4qVsFiwvgvOhscNvTTz+NeUfmNb/YABKcEgzpJgkpt6Rg7Zq1GDRokNolURvF0FXY78c0e/bsia1bt3q2z5w5gwkTJuDikotNWhXXdcmFBx98EE8//bTntvbt26s6hrpv3z48/8LzMP/VDE17/x4nvfw6yeDwAvkWQ1dlJlP9suaXRUZGNgjOphg8eHCDfamtqqoKhw8fRsKfE6Ax+XfoAoCgE2AdZEX6z+mw2+2488471S6J2iCGrp+JiYnB/Pnz1S6jxcrLy1FWUQZtiBbwvwkLVyXqRUSMicDSRUtRllvG0CWfYOiST7z44otYu38t4l+JV7sUIr/CWerkE7IsQ5ZlCILgl3Nzr6fdve2QKWRizJgxcDqdN34CUROwp0te5Xa78cUXXyBfzEdwb/+cl3sj+kg9KnIrUHywmPN2yevY0yWvcrlcmDlzJk6Jp2Ad7L/zcm9E0AoQg0VcunQJDodD7XKoDWHoEl1FcFIwrE9ZcXPyzdi5c6fa5VAbwtAlr9m7dy9G3z0aIX8Ngekmk9rltIggCIAASO7WdTF68n8MXfIam82GvZl7oYvTqbbCrzcJGgGW/hZsydiCbdu2qV0OtREMXfKKqqoqVNVUtYqTIBpL1Itof297LF61GEuXLlW7HGojOHuBvGLKlCn48fiP6DizI/+UE10HPx7kFZIkQZIkCJrWNy/3RsJHh+Ow4TDuu+8+zmSgFmPoUou43W6sWrUKxbpiBHXzzbLtajPEGmATbdizZw8kiQfWqGUYutQiDocDU6dOxWnjaYTeEap2OT4jaAQIBgEVFRU8S41ahKFL1Aim3iaETw1H7+Te+Pnnn9Uuh1oxhi41W0ZGBh586EGEjguFsYtR7XJ8ShAFCKIAp9PJIQZqEYYuNVtJSQm2bd8GfWc9tJYAmAijAcxJZmQcycDu3bvVroZaqQD4pJAv1NbWos5eB8EgtJrr5baUqBMROS4SHy/5GCeyTmDAgAFql0StEEOXmmXSpEn4+ezPSHg1AWg750MQ+RyHF6hZXC4XXG4XBG3bm5d7I2FpYThpOYmHH34Ydrtd7XKolWFPl5rE5XJh48aNKDOUwZAQmMuUG+ONsJXbsGnTJh5UoyZjT5eaxOFwYOLEifjV+ivChoWpXY56xPqFLGtra+FyudSuhloRhi5RM5h6mdD+hfbondwbP/30k9rlUCvC0KVGy8jIwITHJyDsgTAY4gNzaOEyQRQg6ATU1dbB7XarXQ61IgxdarTCwkJs/M9GGHoY6pdWD3CCKMDUy4SDpw5i3759apdDrQRDlxrF4XDA6XJC0AXWTIXrEbQCoh6NwocrP8Q///lPtcuhVoLdFWqUiRMnYkfeDnR8pSMELYOXqLnY06VGsdvtsDvtEA1iwM3LvZHQP4TibOhZPP7446irq1O7HPJzDF26LpfLhc2bN6PSWAlDXGAfPLsWYycjyo3lSE9P50E1uiGGLl1XbW0txo0bh7PtziIsNYDn5d6IAEBTP/bN4KXrYegSeYGphwlRf4tC8q3JWL9+vdrlkB/jgTS6poyMDCxavAihfw6FPlavdjl+7fLKEpVVlTxDja6LPV26ptzcXKxesxqmPiZorfz7fCOCIMDYxYgT50/g8OHDapdDfoqhS1flcrnqV/cVA+d6uS0laAXETIzB+6vexz/+8Q+1yyE/xe4LXdVjjz2GX4p/Qfzf43m9XCIvYk+Xrqqqqgo1dTXQBGk4L7eJQgaEIL9dPqZOnYra2lq1yyE/w9ClBlwuF/bs2YPaoFroInVql9MqBXULQrm5HF999RWXa6crMHSpgerqaowZMwY5HXIQnhaudjmtl1D/z+1280Ln1ABDl8gHgroGIfrFaPQb2A9r165VuxzyIzyQRh579+7F0uVLEXp3KPSRnJfbEoJWgMakQUlpCddRowbY0yWPM2fO4N8r/g1zPzO0ofx73GICYIg3IKcwBydOnFC7GvITDF0CAEiSBEmWOCfXiwSNgNhJsXjv2/cwY8YMtcshP8HuDAEAHn30Uewr34cOMzrwTzGRD/HjRQCAsrIyVNZUQmvRcl6ul5lvNaMoqggvv/wyampq1C6HVMbQDXAulwsHDhyAPcgObTi/+PiCqYcJtnAbPv30Ux5UI4ZuoKuoqMCdd96J/G75CP8j5+X6mizLkGVZ7TJIRQxdIgUYOxkR+2Isbh92O7777ju1yyEV8ftkANu7dy9WfLMCYaPCoIvgKb++JGgFaEI0yCvM4+6BnBQAAALnSURBVPUYAhx7ugEsOzsbn/3vZ7DcbuG8XIXoo/UoKC3Ar7/+qnYppBKGboDiuKLyBI2A2GdisfD7hXj22WfVLodUwu5NgHr00UdxoOoAOkzjvFwiJfHjFqAKCwtRXl0ObSjn5SrNnGRGWWwZ5syZg+rqarXLIYUxdAOMy+XC8ePH4QxyQhPCJSHUYOplgi3ShoULF/KgWgBi6AaY0tJSDBw4EIU3F3Jerp/g+HpgYegSqcAQb0Dc3+KQOjoV33zzjdrlkIJ4IC2A7Nu3D6vXrUZoaiiniKlM1IkQIgTk5OagsrJS7XJIQezpBpCsrCx89MlHsN5pZej6CV24DiWVJcjLy1O7FFIIQ5dIJYIoIHZKLN5b9x4ee+wxtcshhbC7EyDGjx+PQ7WHEPdsHC9U7kc4XS/wsKcbIM6fP4+SyhLo2un4Qfczpp4mVMZV4u233+a83QDA0G3jXC4XcnJy4Da6IQbz7fZHwTcHozK+Em+88QaqqqrULod8jJ/CNq64uBgpKSm41O8SwtLC1C6HKOAxdAMIhxX8lz5Gj7hpcRj7wFh8/fXXapdDPsQDaW1YVlYW1v+0HtYhVmgtfKv9magXoYvS4dSZUygtLVW7HPIh9nTbsN27d+PtD95G2F1hnJfbGgiAJkSDitoKFBUVqV0N+QhDl8hPCIKAuOfi8MF/PsADDzygdjnkI+z+tFETJkzA7t274a5yI39hvtrlUBO4bC7I3XkRnLaKodtGpaSkICoqSu0yqJni4uLULoF8RLjeZeVsNhv/3BIRNZHVar3mVCGO6RIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERAq67sKURETkXezpEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKSg/w8cHN7h/mwFiQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.333\n", " Color.RED: 0\n", " Color.BLUE: 0\n", " Color.GREEN: 3\n", " 1 s represented / 3 total categories\n", " average (binary vector): 0.333\n", "\n", " Which of the possible s are present?\n", " --> subscore: 1.0\n", " Size.MEDIUM: 1\n", " Size.SMALL: 1\n", " Size.LARGE: 1\n", " 3 s represented / 3 total categories\n", " average (binary vector): 1.0\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.667\n", " ShapeType.TRIANGLE: 1\n", " ShapeType.CIRCLE: 2\n", " ShapeType.SQUARE: 0\n", " 2 s represented / 3 total categories\n", " average (binary vector): 0.667\n", "\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "S3tNEjB5Y3rt" }, "source": [ "We'll play around with those parameters more later.\n", "\n", "Now, let's take the **viewer** into account. Looking at all of the groups of shapes, what would it mean for a group to be **inclusive** of shapes like you?\n", "\n", "One way of thinking of it might be saying, \"that group is inclusive because it has shapes that **look kind of like me**\". \n", "\n", "In this framework, you might find a group inclusive even if there's no shape in the group that looks _exactly_ like you, as long as it contains shapes that have your same color, size, or shape.\n", "\n", "Just like diversity, we can think of being inclusive with respect to different **attributes**.\n", "\n", "Let's see one way of quantifying that:\n" ] }, { "cell_type": "code", "metadata": { "id": "3JaznaLwdhlj", "colab": { "base_uri": "https://localhost:8080/", "height": 623 }, "cellView": "form", "outputId": "21bec593-6871-4dbf-b55d-e0fa29da6dda" }, "source": [ "#@title Example inclusion score:\n", "simple_inclusion = inclusion_fn = InclusionFn(\n", " # How to aggregate across *entities* in an image\n", " aggregate_method='max', \n", " verbose_output=True)\n", "\n", "greens.show()\n", "simple_inclusion.score_image(my_viewer, greens)\n", "print()" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU1d0H8O+9s2Yyk32BJIRFEEhK2KlQ0BYlRaCgtqhQsRb0VVJWqdhKxYoFl6q1btj64ktatCJYBBLBsimE1bAlIcEEyhIgIRuZLJPZ7/tHZDQiZJu5dyb5fp7H5/HO+sudmS/nnnvuOYIkSSAiInmIShdARNSZMHSJiGTE0CUikhFDl4hIRgxdIiIZMXSJiGSkvtGdZrOZ48mIiFopNDRUuN59bOkSEcmIoUtEJCOGLhGRjBi6REQyYugSEcmIoUtEJCOGLhGRjBi6REQyYugSEcmIoUtEJCOGLhGRjBi6REQyYugSEcmIoUtEJKMbTu1ITblcribboihCEK47gxsR0TXY0m2FtLQ03HzzzZ7/tmzZonRJRBRgfNLSPX36NN544w3Pdvfu3bFw4UJfvJWs7r33XhiNRqxatQoAYLfbFa6IiAKN10P3zJkz2L17N1avXo2BAwfi4sWLiIqKwpgxYzBw4ECUlJTg8uXLEAQBKSkpuHz5MkpLSz3PT0hIQFhYGPLy8jy36fV6DBgwwLNts9mQk5Pj2dZqtUhJSUF+fj4sFotnu6CgAPX19dBoNBg4cGCLuwJcLheOHTsGt9vd5PYhQ4bAbrd7Qvf06dP48ssvIYoiBg0aBJVKBQCQJAnHjh2D0+kEAAiC4NkX5eXl17zf1X2h1Wo9z8/NzYXNZvM8ZsCAAdDr9Z7t3NxcWK3WJq/TpUsXdOvWrUV/IxEpQ5Ck66/I05bletLS0vDBBx9AFEUUFhZi+fLlWL16NURRRH5+Pl599VX8/e9/BwDk5eXhzTffxMqVKyEIAiRJwtKlS3HnnXdi5MiRntt69eqF7OxsT2iePXsWgwcP9twfHx+PnJwc/OQnP0FOTg66dOmCvLw8pKam4ujRo4iMjERBQQHUanWzwStJEmpra9G7d29PS/bqc95//30IgoBp06Y1uV2j0aCwsBChoaEAGlvAffv2hdls9jzu5MmTWLFiBdLT03F1n3+7lry8PMTFxQEA3G43Bg4ciAsXLnj+xsOHD6NXr16eGkeMGIFTp041eZ25c+di2bJlrf3IiMjLFF2u59lnn8XJkyeRn5+PqKgoLFmyBOvXr4ckSbjtttuQnp6Ovn37oqCgAHFxcXj11VcxadIkAMB//vMf/OY3v8G5c+fQr18/nDlzBu+88w7uuOMOAMCmTZvwxBNPoKSkBP3798eKFSvw+9//HmVlZUhKSsLSpUuxdOlSVFZWIjk5GUePHm223k8//RRDhw71BO64ceNw8uRJnDx5EmPHjm3y2Ndeew3vvvsu7HY7hg8fjs2bN+PAgQMYMGAAzGYzVqxYgTVr1sDtdmPUqFFISkrC2rVrPc//7r5YvXo1vvrqK/Tv3x+XLl3CokWLkJGRAQBITU3FW2+91WRfAI1HBlfre+KJJ9r/gRGRT/ls9IIkSVi2bJnnkFgQBCxZsgQhISGIiIgAAFRUVGDSpEm46667EBsbiz/84Q/46KOP8PnnnwMAVq1ahcLCQrhcLpSVlcHpdKK+vh6VlZUAgPT0dBQXF8PtdqOsrAwhISEwmUzXbEuShLKyMjgcjmbr7tu3r6fF6HQ6UVRUhFdeeQUA8Ktf/arJY0NDQ2E0GgEA5eXlsFqtSExMxMKFC7Fs2TJs3brV0/qtqKiARqNBeHi45/kmk6nJvrBYLHA6nSgrKwMA7NmzB+fPnwcAVFZWYsOGDSgpKcGiRYvw8ssvo7y8HGaz2VPfbbfdhokTJ7bykyIiOfk0dI8cOYKSkhJUVlZCEAQ8/vjjCAkJafK4kSNH4he/+AUAYPr06SgsLPSE7vHjx6HT6TBo0CAAaNKnCTQekuv1es/9BoOh3XX37t0bDz/8MDIyMmC321FdXe3pDrntttsgijc+OAgJCcHw4cOhVqtx5swZTz9tWxQXF8Nut3v+vgsXLmDr1q3Izs7G3r17cf78edjtdk99ZrMZ3bt3R3JyMoeyEfkpr4fu1R+7KIrYsGGDp0/3KkmS8O1+5KvbV5/37bBYs2YNevbsec1rX7Vq1Sr079//uve3hSRJ0Ov12Lp1KwAgIyMDDz74YIuff+TIEUyZMgUA8Pzzz6NXr16YMGHCdd/r+/rUr/bjPvLII1iwYIHn9hdeeAEff/wxJEny7NNz585h8ODBAIC1a9di7969yM3NbXG9RCQvr4fu8uXLMX78eMyYMQOjR49GbW3tNfd/O4RfeuklbN++HZ988gkAYOHChRg7dix+9rOfYfz48VCrvykxMzMTjz76KG699VakpqZiypQp0Gg0nvv//e9/t7v+LVu2YNGiRZ7t744QaI0lS5Y0qf+7VqxY4WkJ79q1C3369IFOp8OJEydwxx134C9/+Qveffddz+PFISLECSKSk5M9t337go2ZM2fiySefbHO9ROR7Xg/dsLAwDB06FMuXL29yuyAIMBqNuPXWW5v0awJAZGSk5/9DQkKQnJx8zfMBIDw8HCaTCf369fve+6OiovCjH/3Ic198fDw0Go1nOzExsdn6+/Tpgzlz5nzvfUlJSQDgeb0f/OAH0Gq1nu1BgwbBYDB8b20AMGLECFgsFs/2rbfeipEjR3re92r/cNeuXfG73/2uyT9Yr7zyCiSHBGOQESUlJViwYAGio6ObvP7gwYMRGxvb7N9IRMrx+pAxur6LFy8iKysLjz76KABgwYIFeOCBB9C7d+/rPsdqteLUqVN4YM4DqOtVh6B+Qbj8f5ex+rXVGDpoKEOWyA/daMgYQ1dGCxYsaNK1AjS2jq+eOPw+J0+exC233IJuT3aDJuqbrpTil4ox78F5ePrpp31ULRG11Y1ClxPeyGjp0qV4/PHHm9zWntENRBR42NL1Y9u3b0fmzkxsPLMRppEmqIJUnvtqDtQgOSgZY/uMxbx58zhEjMiPKHpFGrVdVlYW1qxfg7CxYU0CFwBCbglBTkUO3nvvPYWqI6K2YOj6qRsdgXjj8USkDHYv+CFJkjBx4kRcjL4IKVmCOvz7u95dFhfc5W6oM9VYt27dDUdBEJF82L0QgIqLi1Frr71u4AKAyqACjI3TaXJuX6LAwND1MzabDadPnwZMgGho/uMRVAI0URpcuHQBFRUVMlRIRO3B0PUzRUVFGDZsGHBX48my5qjD1EhYnIBfzvkl/vrXv8pQIRG1B0O3A+BwMaLAwdD1I7t27cKaj9cg7PYwiPrWfTQhPwxBXnUe/va3v3EkA5EfY+j6kW3btuG9D99DxPiIa8blNid0dCiOVx3H66+/7qPqiMgbGLpERDJi6PoBSZIwefJkbC3ditgH2z5rmGmYCeIUEaNGjUJhYaEXKyQib2Ho+gFJklBUVIRqR3WTmcRaS2VUQQgXUFBQ0K7J14nIdxi6CrPb7bh48SIQDIg6L3wcIqAKU6G8shzV1dXtfz0i8iqGrsJOnDiBASkDoLlPA9MIU7tfT2VSIfGpREz7zTS8+OKLXqiQiLyJ8+n6AwmA4J3xthyz2zGdOnUKM2fObNNz77//fqSlpXm5Imorhq6C9uzZg21Z2xA6JhSC1rthaRxsRGFdIdLT0/Hggw8yjANMWVkZ1q9f79kuLS1FTk4OTMNNEHQt/yzrjtRhy5YtTW5LSUnB6NGjvVYrtQ5nGVPQ4sWLsfrT1UiYn+CT1zfvMUNzSIP8/HyIInuS/JndbkdlZaVn+0TBCdz30H3XPC5+bjzUYS1vK1165xIc5Y4mt824fwaeXPjNqtEGgwGhoaFtqJquh8v1EPm5nJwcjBs3zrOtjdci8Q/Nr17dnK6Pdr3mts1Zm5GelO7ZnjVrFl5++eV2vxe1DFu6CnC73Zg6dSrOhZ+DtYcV2ljfrJPmrHVCKpMQdSgKK99eiX79+vnkfah9Xn31VXz44YcoLCxEl1ldIAaLELWiT78XzmonAKD8w3KEukORkpKC9evX84jIS9jS9UO5ublwjXQhNNZ3h3VqkxpOtxNHjxyFxWLx2ftQ65WWliIjIwMAsPvUbpRGliJkVAj0PfStnnejtdQmNdSmxp++aZgJ9it2FLgLsGrVKgiCgL59+2LMmDE+raEzY+jKzOFwoKqqCkKQAEEjw8ktsfGiieqaatTW1sJkav+wNGqfmpoaHDlyBL/97W8hBouInBSJqHuiFKkl7CdhAABXnQu/W/Y7uOpdmPrzqejfvz8iIyN5AtYH2L0gs+zsbKT+NBUJv0+AKkTl8y+1JEmABJS8VYIHUh9g350fmDNnDt5//31IkJD4dGLjlYQKh9vV78mlty/Bds6GyMhIfPXVV1Cr2S5rC3Yv+BFJkuB2u702Lrc5giAAAiC5JU75qKCjR4/iyScbRwxU9axC17TGE1wqg/KBC3zzPYn+eTTcNjdEp4gJEycAEnDHHXdg8eLFSpfYYTB0ZXTgwAHs2LcDpuEmiBp5T1gYkg04ZzuHjz76CFOnTvWLH3pnsWfPHnz22Wc4dOgQjEOMCIls7Lv1R9qujSfv3DY3Co4VwFJggSRJSExMxNSpU6FStW7KUboWuxdktGDBAvxr578QPzdekfc37zNDzBJx8uRJ/nhkIEkSqqurMX/+fGzK2ASVQYWExQmtnitZSaX/V4qGUw0w6ozIzs5GREQENJq2T8rUWXA1YCIFOBwODB48GJs3b4a+px6Jf0j0+cgEb4v9VSyi7opCbW0tkpKSsHfvXqVLCniB9Q0IUC6XC9OnT8d+y35E3a3MWWoACE4Ohm6KDpOnTEZ+fr5idXQWkiTB5XIhdGwoIidHQlAJAdetI4gCgvoFIXZmLFySi+cFvIChK5ODBw+i0lUJXYJOsRrUoWqo4lTYu3cvamtrFaujMygpKcG69eugS9bB0M8AXZxyn3t7qU1qBN0UBOMQI/Yc3oN9+/YpXVJAY+j6mNPphNlshqATIKj8oJUjAKJeRL2lnhdM+IjFYsGhQ4cwZ94cGCca/fakWWuIWhEx98Vg5dqVWLlyJcxmM1u9bcTQ9bHs7Gz07dcXwbOCYRxqVLociMEiuj/THTMWzcBTTz2ldDkd0ty5c9s8DWMgyMjIwKBBg2C325UuJSBxyJiPud1uOBwOQGzsH1OaIAiACnC6nHC5XEqX0yE5nU6ouqoQMyFGnqsOZRQxIQL1ufVwHHKwpdtGbOn6UHZ2NrIOZyF4YDAEtX/9+IL6BKHEVYLNmzc3XqxB7eZyubBx40ZUaioRPCAYQX2C/KNLyYt0CToY+hug66/Dxk0bUVJSonRJAYfjdH0oLS0N/97/b8SlxSldyveqOVgD1w4XioqKeLlnOzmdTlRXVyMlJQXGe4wwDlS+K8mX3FY3zj57Fumr0pGamoqgoCClS/IrHKdL5GM7duxAUlJSpzs5OWvWLC4F1EoMXR9wOp349a9/jaOuo4iYGKF0Oddl6GuAYYoB06ZPw4kTJ5QuJ6C53W7YXXbEzIiBvmfgj1ZojqAR0OVXXaDqomo8Z0EtxtD1AUmSsGvXLlRIFdB3998foDpMDXUPNbZt34YrV64oXU7AEwQBhr4GqEM6fleNoBJg6GeAaGSEtBb3mJe5XC40NDQ0jgsJhL0rNLZabHYbbDab0tUEJJvNBpvD5vXFRQOBoBbgFtywWCwczdBCPJHmZVlZWZh671TELIqBaBT9YpjYjUiSBMkpoSq9ClNGTMEbb7yhdEkB56GHHsKu/+5C+P3hENSBd6lve0hOCdZ8K6wZVuTn5/OE2tc4n66MXC4XGqyNLV1/D1yg8ZBY0AiwO+wc7N5GdrsdDqdD9uk6/cHVlq7ValW6lIDR+b4lPnT8+HEcPnEYhv6GgBufqe+hR4VQgW3btnHcLpEPMXS96M0338RLq15Cl4e6QNQF1q6NuDMCOcjBrFmz4HQ6lS4nIEiSBKvVyiv7vsZ90TKBlQxEfqShoQHJycnYvn270qUojvui5Ri6XuBwOPDYY4/hhOoEwseFK11Om+l76WGcbMTDjzyMvLw8pcsJCPX19TCOMSLsx2FKl6IYXaIO0dOiYbFZeJTUAgxdL3C73cjMzES5UI6gmwL37K0mXAPNzRps2rwJ5eXlSpcTMPQ99J3igojrUYeqEfyD4E41aqM9GLrt5JlFTCXP6r5yENQCnE4nWy1EPsDQbafdu3ej/w/6I2JOBIJTgpUup91EvYjuT3fHo8sexfz585Uuh6jDYei2k9PpRG1trf+sDNFOgiBA1ItosDY0XllHRF7F0G2H/Px85P83v7Eft4PtSW2cFtViNfbu3ctxu0RexCvS2uHFF1/Ef07+B10f6ap0KV4XOSkSecfycN999+HUqVPQ6zvviSIib+pg7TMiIv/G0G0Dh8OBhQsX4rT+NEJvDVW6HCIKIAzdNnA6nVi7di3KNGUw3GxQuhyfUQWroE5Q4+Chg5xvl8hLGLqtJEkS3G534wxigT9Y4YaC+gQh7JdhmHLPFBw6dEjpcog6BIZuK+3cuRMpQ1IQOS8SwUmBPy6XiOTF0G0lu92OqsoqiAbR75ZV9wVBIyDqriikb07He++9p3Q5fqlmXw3M+8xKl6EY20UbKj+phOTmmgctwdBthaKiIvz34n+h66brNHtOUAkI+WEIdmXvws6dO5Uux6+IooghQ4ZAV6ZDw1ed90ISZ5UT9YfrMWTQEISHB+6ET3LhON1WeOaZZ/D52c8RlxandCmyuzqvhNvtblxtooPMM9Eeer0en376KaZPn47Pz36udDmK0ul0yMzM5HI9LdBJ2mvUXtH3R+NY2DEMHjyYC1gStQNDtwXsdjuWLFmCYmMxTD80KV2OIlQGFRyiA2VlZVz1lagdGLot4HQ6sWrVKpQZyjr1iAUxSISmqwbHc47DbO68J46+q2fPnujRtQdsF2yd7mSSo8IBg9OAgQMHQhQZJy3BvdQMSZIaW3bswkRQ7yBEzIrAnRPvRFZWltLl+I3ly5fj6YefxqW3LkFydK7QrdhYgeSaZGzduhU6nU7pcgICQ7cZ27dvx7BbhiFmfgyCbuZJAiJqH4ZuM6xWK0pLSyGGiBA13F2CSkDExAh8tP0j/POf/1S6HL8iuSVUbamC7VLHP9EoOSVUZlbCUeZQupSAwxS5gbNnz+JC2QVou2ohsH8BQGPoho4OxbaD25CZmal0OX7DZDIhqV8SavfVwlHe8YNIckowZ5kRb4pHYmKi0uUEFI7TvYHFixcj61IW4ufGK12K//n636CrIxk6+7jd0aNHY8eOHejduzeAxv3SUfeJ5zOHgD//+c9ITU1VuKLAwpYutUn0L6JxIvoEhg8fzmV9vqbX63Hw4EEMqR+C8rUddzVlS4EFVSur8OXBLzFmzBilywk4DN3vYbPZ8Kc//QkloSUwDemc43KbozKqYFfZceHCBY7b/ZogCEhISIDOrYP1jBVVW6vgdnSspY5qDtWg9mAt3DVuJMQn8Aq0NmDofg+73Y433ngD5aZyBA/ovONymyPqRKij1CgsKkRNTY3S5fiNuLg4dDF0QfXn1YBL6Wq8q/5YPVTFKvTp04fjctuIe+072GpruaDeQYhKi8LY1LGcDOdbnn/+ebz22mue7Y7ynbr6d4wbNw67du2CVqtVuKLAxND9js8++wxjxo5BzLwY6G/iYozUNqNGjcLB/QdR/fdqWPItSpfTbm6rGxf+fAF/efIveOGFF5QuJ6AxdL+jvr4e586fgzpcDVHL3dMcQRQQdnsYMvZl4MMPP1S6HL9hMBjQo3sPuKpdqM2uRe2XtUqX1Ga2SzZc2XEFjgoHYiNjERMTo3RJAY2p8i2XLl1CeXU5NJEaXvbbQoJKQPjYcGw5sAXr1q1Tuhy/IggCevbsCfGsiJoDNXBUOAJubgZntRMNhQ2o3V2LXj178cSZFzB0vyUtLQ3P/eM5xC+I7xSrQpBvabVafPHFF7jzzjthO29D8Z+L4bYG1miGsn+VoSqzCkajEQcOHMAtt9yidEkBj6H7PTrqoHZfipwciVPxpzBq1CjU1dUpXY7fEAQBy5Ytw9tvvw24gYtvXER9br3SZTXL1eBC8cvFsBXbMGHCBOzatQsajUbpsjoEXpGGxvkV3n77bVRGVCI4nkPE2kIdqoZFY0HJqRK43YHVmvO1rl27YuTIkVi0aBEAYM/FPSjaUQQACB0T6lfnDmqza+E0O6FT6TD3l3MhCAJSUlJw0003KV1ahyHcaDiL2WwOrA6oNjKbzejduzciH4qE4WaD0uUErIb/NqD642ps/2Q7uid2h9FobPNrVVVVtXnsb1xcnF8PZ1q6dCnWrl2Ly2WXET8vHpooDUS9ssEruSU4q524/I/L0Jq16NWrF3bu3Am1mu2ytggNDb3u4TJDFwxdb5EkCXAB5549h3defwf33ntvm1/rqaeeajwkb4N9+/YhKSmpze/ta5IkYf/+/ZgwYQIAIOrnUQi5JUTRmly1Lpx77hwgATNmzMDrr7/ObrZ2uFHodvp/xrZs2YJnn38WsXNioY7s9LujXQRBgASpzRcDFBQUYObMmQCAy5cvQxOtQeyM2BY/31njROn/luKBBx6ATqdDbGwsNmzY4HfhIQgCBg0ahH379gEAXnnnFWx6ZRMECOj6WFeoglWy1XJ5zWXYL9sRHhqOrD1ZEEURYWFhfrfPOpJOnzJmsxmFhYXoPr07BBW/aO0mAqGjQ7Hr2C6o1Wrcc889LXraF198gU8//RQFBQUw3WKCGCPCFGKCtmvLuwlUoSqE3haKSlTCVmzDxSMX8frrr2P69OmIjo5u61/kEwaDwdMan3LHFIgNItatWwdzlhmiToTKpIJpqG/m/bBdsKHhVOMkRdazViQlJmHSpElISkripb0y6NShW15ejuq6aqhC5WtZdHSCKCBifAQ2f7AZF/MuNhu6kiShtLQUa9euxQdrP4AqTIXwceFQh7T+q6kyqBA5KRIAULO/Ble2XcEzzzyD5ORkDBs2DGFhYW36m3xt8uTJ6N69e2PLt+DrGyMB503Oax6rClFBEFveOHDWOIHvnNcUzgsIOtY43jbIGITU1FQ8+eSTbS2fWqlT9+lOmjQJxxqOIfq+xlYQD6m8p+yDMgwxDcGmTZtu+Di3243+/fvj8uXL0CZoET+vce7i9n4WkiTBVe3C+RXnAQCzZ8/G888/367X9KXv/g6PHj2KsWPHXvO4bk91gya85UO3il8phqO06aTqs2fPxooVK5rcxu++d/FE2nVMnDgRxxuOI2YaL2v0NkeVA6pLKoTnhGPjxo0ICbn2RNGJEyfw2GOPoaCgAMGjg2EaboI2xnujDiSnBPtlO8reL0OoFIoBAwbg448/DohDaIvFgqKiIs/26dOnMXPmTGhiNa3qBnOUOXDfL+7D7NmzPbdFR0cjLi7Oq/VSUzyR9h1WqxWrV6+GOcqMIAMva/QFTYQGDVUNyM3Nhct17fyGX3zxBbZu3Yrc3FyYfmiCob/Bq4ELAIJagC5eB9MIE+oK6pCdnY2VK1di6tSpfj9/gMHQuKz5VTExMU2CszXGjBnT5LVIWZ2ypVtVVYU+ffog5pEYBN3E0PUV6xkrKv9Vid1bd6Nbt24wGL4ZjrdgwQKs/udqqIwqJCxIgMro2371moM1uLL1Clx1LuzcuRNDhgzx6ftR53ajlq7/H2dRwNL10CFucRxG/WQUNmzYcO398TokLkmEGOz7r6FphAnxC7jWHSmv04VuZmYm7rn/HnRJ6wJdvE7pcjo0QRAAoelJIpfLhbvvvhsZGRmex8hxEufb7zF79uwmk4wTyanThW5lZSVy8nKgjdcqfullpyA0tjL3n9zvCdqcnBzUhdTJvhSSoBUQMioERReKUFxcLOt7E13VqVLnypUrqLPWNV7xwxEyshBEAZGTIvHJ/k/w1ltveW43DjYi7MfyjptVBakQdXdUq4ZcEXlbpxq9MG3aNORL+Uj4bQJDl4gU0alaum63G5IkQRDl6Uekb4TfHo7SPqUYP348zGaz0uUQKaZTtHQbGhqwfv161EfXQ2/kYpNK0ERp0FDTgC+//FLpUogU1SlaurW1tZg/fz6u9LgC0zDfTCJCzRNUAkRDp/jKEV0XfwEkG12iDt2f7i7LuFwif9Xhv/2ZmZl48JEHEfs/sdB28d/VBDoDQRA6wTeO6MY6/E+gpKQEhw4fgr6nnuNyiUhxHTqF6urqYHVYGbZE5Dc69OiFu+++G0XaInT7bTeOyyUiv9ChQ9fpdMKtdnMZHiLyGx0ydBsaGrBlyxZYo63QhvLkmT+yFdtgibDIuvqy2+GG5YQFLsu18/sSyaVDdnZWV1dj1qxZMPczI2SEsktb07VErYja7Fpc+eyKrO/rtrhR9n4ZNA0a6HScYY6U0SFbuuTHBKDb4m6o2FABe6ldkRI2bNiAoUOHKvLeRB2upZuZmYm0hWmI/XUsNFGcTcrfCIIAQd04z66jzIHS90plOdyvz6tH2YdlAACtVguNht8NUkaHC91z585h977dCOoXBFUQl1b3V7p4HbRxWlgKLKjPq4ej0tH8k9qooagB9fn1kIolTJo0yW+XYqfOoUN1L1itVjgcDkACJLsESeiQS7x1CMahRmhiNCh5pwQV6yoQMSkCIaNCIGq81w6QJAmSQ0Ll5kq4y93o1q0b/vGPfwTEasDUcXWohSlvv/12HD9+HE6ns/EQlvyaJEnA1Z4FsXFuhvjfeG8dM2e1E8UvFUNySHjkkUfw3HPPQafTcVpP8rkbLUzZoUJ3z549qKmpUboMaoP09HRs370d+pv0iJ4a3bi6RzvU59WjZn8NGgobsGLFCvz4xz9GUlKSl6olurEbhW6H6l4YM2aM0iVQG5WXl6Ourg779u2DJdkC0SBC1IsIuimoxa/htrvRUNQAAKg/UQ/pvITx48djwoQJ6NGjh48qJ2qdDtXSpcB2/Phx/PSnPwUAOBwOqKJViJsX1+LnO6ucuPDyBWi1WoiiiLi4OBw+fJjdCSS7TtO9QIHN5Z3dua8AAAGzSURBVHLBarUCAP74xz/i3f99F4KmFYEpAZJDwo4dO9CvXz8IgoCgoCCGLsmOoUsBJzc3F6dPn27Tc8eOHYuQEF6JSMph6BIRyehGocsBi0REMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwYukREMmLoEhHJiKFLRCQjhi4RkYwESZKUroGIqNNgS5eISEYMXSIiGTF0iYhkxNAlIpIRQ5eISEYMXSIiGf0/RVuAGXD6tOwAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have the same as me? (BLUE)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0', 'entity_1 = 0.0', 'entity_2 = 0.0']\n", " max: 0.0\n", "\n", " Does anyone have a similar to mine? (LARGE)\n", " --> bucketed closeness subscore: 1.0\n", " per-entity scores: ['entity_0 = 1.0', 'entity_1 = 0.0', 'entity_2 = 0.5']\n", " max: 1.0\n", "\n", " Does anyone have the same as me? (CIRCLE)\n", " --> match subscore: 1.0\n", " per-entity scores: ['entity_0 = 0.0', 'entity_1 = 1.0', 'entity_2 = 1.0']\n", " max: 1.0\n", "\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "39CDUCE0JuKg" }, "source": [ "### Single image score" ] }, { "cell_type": "markdown", "metadata": { "id": "Y3Yxaz6jeQbB" }, "source": [ "Suppose now you want to find a the group of shapes that's most **diverse** and **inclusive of you**. To compare all the different groups, you'll want to boil everything down to **one score** that summarizes everything. (So, a higher score will indicate the more diverse & inclusive group, and a lower score will indicate a less diverse & inclusive group.)\n", "\n", " To get just one score, you'll have to decide on a few things:\n", " \n", "1. How do you want to combine all of the different diversity and inclusion subscores for each group? For example, how important is shape diversity vs. color diversity vs. size diversity?\n", "2. How do you want to weigh diversity vs. inclusion overall?\n", "\n", "There are lots of different options here, and they can all communicate different things. For example, if you chose to combine all of the diversity subscores by taking their minimum, it communicates that a group is only as diverse as its least-diverse quality.\n", "\n", "Try some out!" ] }, { "cell_type": "code", "metadata": { "id": "39aVm9jSmTpv", "cellView": "form" }, "source": [ "#@title How should diversity subscores be computed? { run: \"auto\" }\n", "diversity_subscore_type = \"weighted\" #@param [\"weighted\", \"egalitarian\", \"utilitarian\", \"nash\", \"min\", \"max\", \"average\"]" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "SvESFpx28ZfR", "colab": { "base_uri": "https://localhost:8080/" }, "cellView": "form", "outputId": "aae508b2-179b-4c4e-caae-2ca604bc1d52" }, "source": [ "#@title If weighted, how much should we weight each subscore?\n", "\n", "shape_weight = \"some\" #@param [\"a lot\", \"some\", \"a little\", \"none\"]\n", "color_weight = \"a little\" #@param [\"a lot\", \"some\", \"a little\", \"none\"]\n", "size_weight = \"a lot\" #@param [\"a lot\", \"some\", \"a little\", \"none\"]\n", "\n", "\n", "_to_weight = {\n", " 'a lot' : 1.0,\n", " 'some' : 0.5,\n", " 'a little' : 0.25,\n", " 'none' : 0.0\n", "}\n", "\n", "def parse_normalize_weights(weight_str_list):\n", " total_value = 0\n", " output_weights = []\n", " for weight_str in weight_str_list:\n", " output_weights.append(_to_weight[weight_str])\n", " \n", " return [np.around(weight/sum(output_weights), 3) for weight in output_weights]\n", "\n", "(shape_weight_value, \n", " color_weight_value, \n", " size_weight_value) = parse_normalize_weights([shape_weight, color_weight, size_weight])\n", " \n", "print('Normalized weights:\\n shape: %s\\n'\n", "' color: %s\\n'\n", "' size: %s\\n' % (shape_weight_value, \n", " color_weight_value, \n", " size_weight_value))\n", "\n", "\n", "print('Defined diversity scoring function')\n", "\n", "# If we're weighting different components differently, aggregate_method\n", "# expects a dictionary mapping attributes to weights. Otherwise, it will look up\n", "# the appropriate aggregation method using its name (e.g. 'nash').\n", "diversity_subscorer = None\n", "if diversity_subscore_type == 'weighted':\n", " diversity_subscorer = {\n", " ShapeType : shape_weight_value,\n", " Color : color_weight_value,\n", " Size : size_weight_value\n", " }\n", "else:\n", " diversity_subscorer = diversity_subscore_type\n", "\n", "diversity_function = DiversityFn(\n", " aggregate_method=diversity_subscorer, \n", " verbose_output=True)\n" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Normalized weights:\n", " shape: 0.286\n", " color: 0.143\n", " size: 0.571\n", "\n", "Defined diversity scoring function\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "9X2XOKETienZ", "colab": { "base_uri": "https://localhost:8080/" }, "cellView": "form", "outputId": "590ee4d3-d644-4ea3-d2c0-2ddbb2f7e79a" }, "source": [ "#@title Set some scoring parameters! { run: \"auto\" }\n", "DIVERSITY_WEIGHT = 0.5 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "\n", "DIVERSITY_AGGREGATE_ACROSS_ENTITIES = \"average\"\n", "INCLUSION_AGGREGATE_ACROSS_ENTITIES = \"max\"\n", "DIVERSITY_AGGREGATE_SUBSCORES = \"min\" #@param [\"egalitarian\", \"utilitarian\", \"nash\", \"min\", \"max\", \"average\"]\n", "INCLUSION_AGGREGATE_SUBSCORES = \"nash\" #@param [\"egalitarian\", \"utilitarian\", \"nash\", \"min\", \"max\", \"average\"]\n", "\n", "shape_scorer = Scorer(my_viewer)\n", "\n", "shape_scorer.diversity_fn = DiversityFn(\n", " # How to aggregate across entities in an image\n", " aggregate_method=DIVERSITY_AGGREGATE_ACROSS_ENTITIES, \n", " verbose_output=True)\n", "shape_scorer.inclusion_fn = InclusionFn(\n", " # How to aggregate across entities in an image\n", " aggregate_method=INCLUSION_AGGREGATE_ACROSS_ENTITIES, \n", " verbose_output=True)\n", "\n", "combiner = Combiner(diversity_weight=DIVERSITY_WEIGHT, \n", " diversity_aggregate=DIVERSITY_AGGREGATE_SUBSCORES, \n", " inclusion_aggregate=INCLUSION_AGGREGATE_SUBSCORES)\n", "\n", "\n", "\n", "\n", "shape_scorer.combiner = combiner \n", "\n", "print('Weighting diversity score at %s, inclusion score at %s\\n' % (\n", " combiner.diversity_weight, combiner.inclusion_weight))\n", "print('Using %s to aggregate inclusion subscores across entities in an image.' % combiner.inclusion_aggregate_name)\n", "print('Using %s to aggregate diversity subscores across entities in an image.' % combiner.diversity_aggregate_name)\n", "print('Using %s to aggregate different types of diversity scores for each entity.' % shape_scorer.diversity_fn.aggregate_method)\n", "print('Using %s to aggregate different types of inclusion scores for each entity.' % shape_scorer.inclusion_fn.aggregate_method)\n", "\n" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Weighting diversity score at 0.5, inclusion score at 0.5\n", "\n", "Using nash to aggregate inclusion subscores across entities in an image.\n", "Using min to aggregate diversity subscores across entities in an image.\n", "Using average to aggregate different types of diversity scores for each entity.\n", "Using max to aggregate different types of inclusion scores for each entity.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "nkuNTasPkC7W" }, "source": [ "Let's try out this metric on some images!" ] }, { "cell_type": "code", "metadata": { "id": "IGagt09bjK6w", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "outputId": "ad9af7ad-31d8-4844-9a20-26ff64472c64" }, "source": [ "shape_scorer.score_image(greens, verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXQUVd4+8Keq13S601kgKyGsAkGSiC/8BFkcDBlAEGfUURwVD6i4AIoM6ojiwJHlOC64MkdE4T3oCKIiBAFHQHYMgbCHVQhJCEnI0tl7q/r9kZceI1uW7qpO+vmcwzlUL3W/6U4/uX3rVl1BlmUQEZEyRLULICIKJAxdIiIFMXSJiBTE0CUiUhBDl4hIQQxdIiIFaa93p81m43wyIqImslqtwrXuY0+XiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgVd99KO1JDb7W6wLYoiBOGaV3AjIroCe7pN8Mwzz+Cmm27y/Fu/fr3aJRFRK+OTnu6ZM2fwwQcfeLYTEhIwbdo0XzSlqL/85S8wm81YsmQJAMDhcKhcERG1Nl4P3bNnz2Lbtm1YunQpkpOTkZ+fj3bt2mHw4MFITk5GQUEBCgsLIQgCkpKSUFhYiIsXL3qe36FDB4SGhuLIkSOe24xGI/r06ePZttvtOHTokGdbr9cjKSkJx44dQ01NjWc7Ozsb1dXV0Ol0SE5ObvRQgNvtxoEDByBJUoPb+/btC4fD4QndM2fOYO/evRBFESkpKdBoNAAAWZZx4MABuFwuAIAgCJ7Xori4+Ir2Lr8Wer3e8/zDhw/Dbrd7HtOnTx8YjUbP9uHDh1FXV9dgP9HR0YiPj2/Uz0hE6hBk+dor8jRnuZ5nnnkGX375JURRxMmTJzF37lwsXboUoiji2LFjeOedd/DJJ58AAI4cOYIPP/wQixYtgiAIkGUZs2bNwsiRIzFgwADPbV26dEFmZqYnNM+dO4dbbrnFc39cXBwOHTqEP/zhDzh06BCio6Nx5MgRpKWlISsrCxEREcjOzoZWq71h8MqyjMrKSnTr1s3Tk738nC+++AKCIGDcuHENbtfpdDh58iSsViuA+h5wjx49YLPZPI87fvw45s2bh2XLluHya/7bWo4cOYLY2FgAgCRJSE5ORl5enudn3LdvH7p06eKpsX///jh9+nSD/UyZMgVz5sxp6ltGRF6m6nI9s2fPxvHjx3Hs2DG0a9cOM2fOxKpVqyDLMoYOHYply5ahR48eyM7ORmxsLN555x2MHj0aAPDjjz/i2WefRU5ODnr27ImzZ8/iX//6F1JTUwEAa9aswYwZM1BQUIBevXph3rx5+Pvf/46ioiIkJiZi1qxZmDVrFkpKStC7d29kZWXdsN4ffvgBt956qydwhw8fjuPHj+P48eMYNmxYg8cuXLgQixcvhsPhQL9+/bB27Vrs2bMHffr0gc1mw7x587B8+XJIkoSBAwciMTERK1as8Dz/96/F0qVLceLECfTq1QsXLlzA9OnTkZ6eDgBIS0vDRx991OC1AOq/GVyub8aMGS1/w4jIp3w2e0GWZcyZM8fzlVgQBMycORMhISEIDw8HAFy6dAmjR4/GPffcg6ioKLz66qtYuXIlfv75ZwDAkiVLcPLkSbjdbhQVFcHlcqG6uholJSUAgGXLliE3NxeSJKGoqAghISGwWCxXbMuyjKKiIjidzhvW3aNHD0+P0eVy4dSpU3j77bcBAOPHj2/wWKvVCrPZDAAoLi5GXV0dOnbsiGnTpmHOnDnYsGGDp/d76dIl6HQ6hIWFeZ5vsVgavBY1NTVwuVwoKioCAGzfvh3nz58HAJSUlKC6uhphYWGYPn063nrrLRQXF8Nms3nqGzp0KO66664mvlNEpCSfhu7+/ftRUFCAkpISCIKAF154ASEhIQ0eN2DAANx3330AgIceeggnT570hO7BgwdhMBiQkpICAA3GNIH6r+RGo9Fzv8lkanHd3bp1w+OPP4709HQ4HA6Ul5d7hkOGDh0KUbz+l4OQkBD069cPWq0WZ8+e9YzTNkdubi4cDofn54uOjobVasWkSZOwc+dOnD9/Hg6Hw1OfzWZDQkICevfuzalsRH7K66F7+cMuiiK+++47z5juZbIs47fjyJe3Lz/vt2GxfPlydO7c+Yp9X7ZkyRL06tXrmvc3hyzLMBqN2LBhAwAgPT0djz76aKOfv3//fowdOxYAMH/+fHTp0gWjRo26ZltXG1O/PI77xBNP4Pnnn7/idlmWPa9pTk4ObrnlFgDAihUrsHPnThw+fLjR9RKRsrweunPnzsWIESPwyCOPYNCgQaisrLzi/t+G8JtvvomffvoJq1evBgBMmzYNw4YNw5gxYzBixAhotf8tcd26dZg0aRKGDBmCtLQ0jB07FjqdznP/t99+2+L6169fj+nTp3u2fz9DoClmzpzZoP7fmzdvnqcnvGXLFnTv3h0GgwFHjx5Famoq3n33XSxevNjz+ClTpmDUqFEYOXKk57bfnrAxYcIEvPTSS82ul4h8z+uhGxoailtvvRVz585tcLsgCDCbzRgyZEiDcU0AiIiI8Pw/JCQEvXv3vuL5ABAWFgaLxYKePXte9f527drh9ttv99wXFxcHnU7n2e7YseMN6+/evTsmT5581fsSExMBwLO/m2++GXq93rOdkpICk8l01doAoH///qipqfFsDxkyBAMGDPC0e3l8OCYmBi+//PIVf7D69euH0NDQa9Z3yy23ICoq6oY/IxGpx+tTxuja8vPzsWPHDkyaNAkA8Pzzz+Phhx9Gt27dVK6MiLzpelPGGLoKev755xsMrQD1vePLBw6JqG1g6PqJ0tJSVFVVNbhNr9cjOjpapYqIyBcYukREClL1jDQiIvovhi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKQghi4RkYJ8thowNc/p06cxYcKEZj33wQcfxDPPPOPliojImxi6KisqKsKqVas82xcvXsShQ4dg6WeBYGj86sZV+6uwfv36BrclJSVh0KBBXquViFqOoaswh8OBkpISz/bR7KN4bcFrDR6jCdEgLC0M2tDGvz2OAgd2HdyFXQd3eW575MFH0LVrV8+2yWSC1WptQfVE1FJcOUJhmZmZGD58uGdbH6dH7NTYqz5WEBrf073a+1ixowKla0s92xMnTsRbb73VhGqJqDm4XI+feOedd/DVV1/h5MmTiJ4YDTFYhKgXoY/S+6Q9V6ULrnIXAKD4q2JYJSuSkpKwatUqiCKPoRL5yvVCl8MLPnbx4kWkp6cDALad3oaLERcRMjAExk5GiEbfBp/WooXWUv8WW/7HAkeZA9lSNpYsWQJBENCjRw8MHjzYpzUQUUPs6fpQRUUFduzYgYceeghisIiI0RGw/I9F1ZrcVW7kvZMHd7Ub9997P+bPn4+IiIgmDWUQ0fVxeEElkydPxhdffAEZMjq+1hEas0b1cJNlGZCBCx9fgD3HjoiICJw4cQJaLb/0EHkLQ1dBWVlZeOmllwAApZ1LUdOuBgBgiDdA0PhPb9JR4IBklyC6RMQeiAVkIDU1FS+++KLapRG1ehzTVcj27duxceNGZGRkwNzXjJCI+rFbf6SPqT94J9klZB/IRk12DWRZRseOHXH//fdDo9GoXCFR28SerhfIsozy8nI899xzWJO+BhqTBh1e7ABNUOsJroufX0Tt6VqYDWZkZmYiPDwcOp1O7bKIWiUOL/iYw+HATTfdBJvNBkMXA2KeiAHEps2zVZssyajaV4XilcXQaDT45ptvcMcdd6hdFlGrdL3Q5WRNL5BlGW63G9ZhVkTcHQFBI7SqwAUAQRQQ1DMIUROi4JbdVz3ZgohajmO6LVRQUIBNmzfB0NsAfU89DLEGtUtqNq1FC7GrCHNfM7bv2w6DwYCBAweqXRZRm8KebgvU1NQgIyMDk6dOhvkus98eNGsKUS8i8oFILFqxCIsWLYLNZmOvl8iLGLotMGXKlGZfhrE1SE9PR0pKChwOh9qlELUZHF5oAZfLBU2MBpGjIiHoWtcY7o2EjwpH9eFqODOc7Omq7LPPPsOaNWua9dz33nsPCQkJXq6IWoKh2wxutxvp6eko0ZUguE8wgroHqV2S1xk6GCC7ZNgr7fh+zfcYMngIYmJi1C4rYJSVlWHz5s0AgA0bNmDbnm0I6tH43zPZIaMmuwbff/894uLiEBQUhJEjR7a6A7xtEaeMNZHL5UJ5eTmSkpJg/rMZ5mSz2iX5lFQn4dzsc1i2ZBnS0tIQFNT2/sD4G7vdjszMTNx1110Q9AIgAPpoPeImxzV6H85SJ/LeyQMAyC4ZMZExyMjIgMlk4okvCuCUMS/atGkTEhMTUVNTo3Ypipo4cSKXAlLI7Nmzcc899wAAOrzQAZ1e74TYp65+zeVr0YZp0en1Tuj0eieEpYWhoKAA3bp1Q3Z2ti9KpiZg6DaRJElwuB2IfCQSxs6tf7bCjQg6AdHjo6GJ1sDpdKpdTpsmSRKefPJJrF27FnKIjOgJ0dCEaCDoBAjapg0LCIJQ/zydAHOyGe0faA+73Y6//e1vWLp0qW9+AGoUjuk2gyAIMPUwQTS0/b9ZgkaAqacJtp02tUtp08rKyrBt2zZs3LgRtaZaBN8cDFMvk1f2rYvQQTSKMN1sQsaBDFitVsTFxSE1NZVjvCpo+6nhRXa7HXanvX6cLcAIWgGSIKGmpsYvZjM4nU7U1taqXYZXOBwOHD16FOPHj0dFTQVCBoQgYnSEV9vQBGsQPT4aukgdNv60EZMnT0ZNTQ3cbrdX26Eb44G0Jnjsscew5dctCHswDIK29Z3q2xKyS0bdsTrUpdfh2LFjqh9Qe/PNN7F69Wrs3Lmz1b8PM2bMwLJly+BwOBD/93horVqfXQZUdsqw7bShdF0pjEYjNmzYgJSUFJ+0Fch4aUcvcTgccLqcEHWB9wXhck+3rq5O1TpkWca0adOwY8cOv+hxe4PT6YQcIiNqdBS0Ft8FLlA/Rh/cJxiaYA2KVxZDkiSftUVXF3jpQa3e1q1bkWPLgRQjYcOGDaiqqlK7pGaRZRmbNm1Cbm4uRJOI4N7Bipxko4vQIeim+m8qu3btwtGjR33eJv0XQ7cRZFlGXV0dx7/+j1qvhSTV97RlUYb5VjNcA10YN24cLly4oHgt3nB5tsKmLZuUX1VEqO/1vvrqq/jss8+UbTvAMXQboba2Fr1798ZPP/2kdimqU/O1OH36NLp27Qr3PW6EDg1VvH1fiRgTgegnohVtU2PRIOH1BOjj9Iq2SwzdRquuroZ5sBmhd7SdD3tTGToa0H5ce9TYa+ByuRRvX5LqZ09AWz/GrAnRIPLhSLw6/1WsWrVK8Xq8RdAKih8nEAQBokFs9QchWyOGbhMYOxkD4oSIa9FatQi+OViVD+rp06fxy/5fENQjyDNlTzSIMCebsWXHFp5pRa0GZy9Qq7B8+XJ89OVHiJ8Rf8V9gkaAJEtwOBzQ6/l1mfwbe7rU6sVOjsWXx7/EHXfc0WamkVHbxZ4u+TVZlvHyyy8jszwT4XeFX/UxokGEQ3Kgurpa4eqImo49XfJ7P/zwA85WnEVwYvA1H6MN00KOkrFt2zaGL/k1hi75LVmW62dJCKj/dx2WvhbIw2SMHTsWOTk5itRH1BwMXfJbx48fR/fu3YH7gZBBIWqXQ+QVDF3yW263G+Xl5YABjZrHqjFr0P4v7bHgwwX47rvvFKiQqOkYuuSXzp07h0PZh2DsbGz0BbxFgwhLPwvWb16P/fv3+7hCoubh7AXyS5988gk+/e5TdHihQ5OfKwgCZMhwu90QxVZw1pUMyJIMQVSuTlmW69sFp9gpjT1danNinorBNznfYOjQoWqX0igl6SW4uOSiom26K93ImZMDxwWHou0SQ7dJKnZVwLYrcJetsefbUbK6BLLku96RLMt4/fXXkVmWidDU5l3nQmPSwO62o6yszMvVeZcgCHjjjTdw2623QapT+Lq2MiBVS5j23DTcd999yrYd4Bi6jSCKIvr27QtDkQG1J9rGEjHN4Sp1oXpfNfqm9EVYWJhP2pBlGStXrsTpqtMwJzV/eXtNiAZoD2RmZvrtys2iKGLcuHHo0aMHJLuEuvN1kN2+/7rvqnDBnm8HAIwZMwYDBgzweZv0XwzdRjAajfjhhx8wcOBAtUtRncFgwLp163zyWsiyXD/W6IWhTUtfCzR3aTB8+HCcPn265Tv0IUEQ4Cpy4cKHF+Cucvv0VGZZllF1oAqFnxf6/1h3G8XQJb9x9OhR9ErsBe2DWoTcFjjzcmfPno0VK1YAMpD3bh4q91b6rK2CTwpQ9mMZIiMjkZ2djT59+visLbo6hi75DafTiaLCIggmAaK+5b+aYpCIiLER+HDph1izZo0XKvSNkJAQJCcnY8GCBQgWg1G1rwrlP5d7tQ13jRuXVl+Co8CB2/vdjlmzZiEqKgo6nc6r7dCNMXSboHPnzugU0wn2PLtPDyb5I+clJ0wuE5KTkyGK3v+1ycvLw6mzp6DvoPfa0jWiQYR1kBWrN6zG7t27vbJPX4mKisKTTz6Jfv36wVxhRuXeyvrfMy+M8boqXbDn2FGxswLdO3THiBEj8PDDD3N4QSUM3SaYO3cuXnv8NVz46AJkZ2CF7qXvL6F3RW9s2LABBoPB6/t///33MXnWZHR4rgM0Zo3X9y9D9vvLPoqiiG+//Rb33nsvnEVO5L+XD3el+79j3U1w+TmyLKNqXxUufnYRgiDg008/xeTJk330E1Bj8OQIavNinohB+sF07L5jN7Zs2eKTnro3vfLKK0hNTcX999+P/PfzAQHQR+kR82RMo/fhLncj/8N8AIBklxAdHY1NmzYhMjLSV2VTIzF0m0GWZJSuL4WlvwWGWO/3+vyJ7JJRurEUziIn0M0H+5dlLFiwAIcqD8E62Or9BlB/TYYqqQqFhYU+2b+3Wa1WJCcnY+7cuQDqL225+8BulKwpafQ+pDoJ7go3pk6diqioKJjNZsTGxnJIwQ8wdJvIYrEgsWcisndlw9jZGBCha9thQ0JcAjp27Oj9/csyPv/8c7gHuGEd5JvQBQAxWIQQLuDo0aPo2rUrTCaTz9ryhsjISDz77LMA6hfkLC8vB0qbuJPewPjx49G1a1fvF0jNJlxvrMhms/n3IJhKamtr0a1bNwT/KRjBSeos1KgEWZYh22XkzM7Bii9XIC0tzev7lyQJvXr18nnoAoDL5sL5N85j8+bN6Nu3r0/bosBmtVqvGQr+Pbjlp4xGI3755Rf0re6L4hXFapfjMzXZNShdVIq9v+zF4MGDvb7/w4cPIyUlBYa/GmD5H4vX90/kjxi6zSAIAjp06ACDZEDd2TqUbiiF5FT43Hkfq8ioQOUvlZAqJHSI64CgoCCvt2G325GblwvBLEA0+v5XUTSKCB8Vjs9Xfo5169b5vD2iq2HotkBsbCyiTdH1E9ndalfjXdUHqqHJ1aB79+4+Odp/8eJF5F7IhS5Sp9glDUWDiNA/hGJl+kps3rxZkTaJfo+h2wLz58/HwoULPdv+Pg+0sS7/HMOHD8eWLVug1+u93saCBQvw9GtPI/5v8T6Zl9sYbeX9otaFodtCAwcOxC+7f0H5J+WoOeafV7NqCqlOQt4/8/DuS+9iwYIFapfjE1Hjo/Cf8v8gNTUVktS2hoXI/3HKWAuZTCZ0SugEd7kblZmVkGokWPq1zoNC9gt2VGVVwXnJiaiIKJ9MpJckCe+//z5O2E/A8v/UeZ20oVrYJBtqz9eyt0uKY0/XCwRBQOfOnSGeE1GxpwLOS85Wd20GV7kLtSdrUbmtEl06d/HJgTOg/iv9Bx98gFOOUwjpr96VxESjCMEq4MyvZ1BXV6daHRR4GLpeoNfrsXXrVowcORL283bk/jNX+ZUAWqjo30UoXVcKs9mMPXv24LbbblO7JJ+y9LUg6KEg9O/fHwcPHlS7HAogDF0vEQQBc+bMwccffwxIQP4H+ag+XK12WTfkrnUj961c2HPtGDVqFLZs2eKzy/1lZWXhtgG3wfRXE4KTg33SBpG/45iuF8XExGDAgAGYPn06AGB7/nac2nQKAGAdbPXKNWK9pTKzEi6bCwaNAVP+OgWCICApKcmnp4zW1tbi1KlTSBiXAE2QOjMWfkvQCwgdFoqVP6xEeXk5/vjHP6pdEgUAhq6Xde7cGa+99hoAYNasWcjZkoPCokKYepiga6dT5CSA65ElGa5yF2w7bNDb9IjvEo9Xlr0Crda3vwqXLl1CUUkRtOFav/l+JRpEhI8Ix/IPl8NR6mDokiL85Ne/bZo9ezY+//xzQAby38tH1YEqtUuCVC0hd0EuHPkO/OlPf8LWrVt9HrhA/R+gp2Y9hfiX4iEG8deOAhd7uj4kCAJSUlKwa9cuAMDb/3oba95eAwECYp6KgSZYua/YhcsL4Sh0IMwahh3bd0AURYSGhip+sR5/vDhQ5LhI7Ph1B0aMGIF169ZBo1F/6IPaLoauj5lMJiQmJgIAxqaOhVgr4uuvv4Zthw2iQYTGooHlVt/MV7Xn2VF7un7J+LpzdUjsmIjRo0cjMTFRsQt5S5KExYsX45x0DsEp/nnwTBehQ8XpCpSfKOe8XfI5hq6C7r77biQkJNT3fLP/78YIwNXVdcVjNSGaJl2TwFXhAn43S004LyDoQP182yBzENLS0vDSSy81t/xmcbvdmD9/PnRpOlgH+vbSjS0h6kXIFhn5+fmIjo72yZJERABDV3FJSUk4cuSIZzsrKwvDhg274nHxr8RDF9b4qVsFiwvgvOhscNvTTz+NeUfmNb/YABKcEgzpJgkpt6Rg7Zq1GDRokNolURvF0FXY78c0e/bsia1bt3q2z5w5gwkTJuDikotNWhXXdcmFBx98EE8//bTntvbt26s6hrpv3z48/8LzMP/VDE17/x4nvfw6yeDwAvkWQ1dlJlP9suaXRUZGNgjOphg8eHCDfamtqqoKhw8fRsKfE6Ax+XfoAoCgE2AdZEX6z+mw2+2488471S6J2iCGrp+JiYnB/Pnz1S6jxcrLy1FWUQZtiBbwvwkLVyXqRUSMicDSRUtRllvG0CWfYOiST7z44otYu38t4l+JV7sUIr/CWerkE7IsQ5ZlCILgl3Nzr6fdve2QKWRizJgxcDqdN34CUROwp0te5Xa78cUXXyBfzEdwb/+cl3sj+kg9KnIrUHywmPN2yevY0yWvcrlcmDlzJk6Jp2Ad7L/zcm9E0AoQg0VcunQJDodD7XKoDWHoEl1FcFIwrE9ZcXPyzdi5c6fa5VAbwtAlr9m7dy9G3z0aIX8Ngekmk9rltIggCIAASO7WdTF68n8MXfIam82GvZl7oYvTqbbCrzcJGgGW/hZsydiCbdu2qV0OtREMXfKKqqoqVNVUtYqTIBpL1Itof297LF61GEuXLlW7HGojOHuBvGLKlCn48fiP6DizI/+UE10HPx7kFZIkQZIkCJrWNy/3RsJHh+Ow4TDuu+8+zmSgFmPoUou43W6sWrUKxbpiBHXzzbLtajPEGmATbdizZw8kiQfWqGUYutQiDocDU6dOxWnjaYTeEap2OT4jaAQIBgEVFRU8S41ahKFL1Aim3iaETw1H7+Te+Pnnn9Uuh1oxhi41W0ZGBh586EGEjguFsYtR7XJ8ShAFCKIAp9PJIQZqEYYuNVtJSQm2bd8GfWc9tJYAmAijAcxJZmQcycDu3bvVroZaqQD4pJAv1NbWos5eB8EgtJrr5baUqBMROS4SHy/5GCeyTmDAgAFql0StEEOXmmXSpEn4+ezPSHg1AWg750MQ+RyHF6hZXC4XXG4XBG3bm5d7I2FpYThpOYmHH34Ydrtd7XKolWFPl5rE5XJh48aNKDOUwZAQmMuUG+ONsJXbsGnTJh5UoyZjT5eaxOFwYOLEifjV+ivChoWpXY56xPqFLGtra+FyudSuhloRhi5RM5h6mdD+hfbondwbP/30k9rlUCvC0KVGy8jIwITHJyDsgTAY4gNzaOEyQRQg6ATU1dbB7XarXQ61IgxdarTCwkJs/M9GGHoY6pdWD3CCKMDUy4SDpw5i3759apdDrQRDlxrF4XDA6XJC0AXWTIXrEbQCoh6NwocrP8Q///lPtcuhVoLdFWqUiRMnYkfeDnR8pSMELYOXqLnY06VGsdvtsDvtEA1iwM3LvZHQP4TibOhZPP7446irq1O7HPJzDF26LpfLhc2bN6PSWAlDXGAfPLsWYycjyo3lSE9P50E1uiGGLl1XbW0txo0bh7PtziIsNYDn5d6IAEBTP/bN4KXrYegSeYGphwlRf4tC8q3JWL9+vdrlkB/jgTS6poyMDCxavAihfw6FPlavdjl+7fLKEpVVlTxDja6LPV26ptzcXKxesxqmPiZorfz7fCOCIMDYxYgT50/g8OHDapdDfoqhS1flcrnqV/cVA+d6uS0laAXETIzB+6vexz/+8Q+1yyE/xe4LXdVjjz2GX4p/Qfzf43m9XCIvYk+Xrqqqqgo1dTXQBGk4L7eJQgaEIL9dPqZOnYra2lq1yyE/w9ClBlwuF/bs2YPaoFroInVql9MqBXULQrm5HF999RWXa6crMHSpgerqaowZMwY5HXIQnhaudjmtl1D/z+1280Ln1ABDl8gHgroGIfrFaPQb2A9r165VuxzyIzyQRh579+7F0uVLEXp3KPSRnJfbEoJWgMakQUlpCddRowbY0yWPM2fO4N8r/g1zPzO0ofx73GICYIg3IKcwBydOnFC7GvITDF0CAEiSBEmWOCfXiwSNgNhJsXjv2/cwY8YMtcshP8HuDAEAHn30Uewr34cOMzrwTzGRD/HjRQCAsrIyVNZUQmvRcl6ul5lvNaMoqggvv/wyampq1C6HVMbQDXAulwsHDhyAPcgObTi/+PiCqYcJtnAbPv30Ux5UI4ZuoKuoqMCdd96J/G75CP8j5+X6mizLkGVZ7TJIRQxdIgUYOxkR+2Isbh92O7777ju1yyEV8ftkANu7dy9WfLMCYaPCoIvgKb++JGgFaEI0yCvM4+6BnBQAAALnSURBVPUYAhx7ugEsOzsbn/3vZ7DcbuG8XIXoo/UoKC3Ar7/+qnYppBKGboDiuKLyBI2A2GdisfD7hXj22WfVLodUwu5NgHr00UdxoOoAOkzjvFwiJfHjFqAKCwtRXl0ObSjn5SrNnGRGWWwZ5syZg+rqarXLIYUxdAOMy+XC8ePH4QxyQhPCJSHUYOplgi3ShoULF/KgWgBi6AaY0tJSDBw4EIU3F3Jerp/g+HpgYegSqcAQb0Dc3+KQOjoV33zzjdrlkIJ4IC2A7Nu3D6vXrUZoaiiniKlM1IkQIgTk5OagsrJS7XJIQezpBpCsrCx89MlHsN5pZej6CV24DiWVJcjLy1O7FFIIQ5dIJYIoIHZKLN5b9x4ee+wxtcshhbC7EyDGjx+PQ7WHEPdsHC9U7kc4XS/wsKcbIM6fP4+SyhLo2un4Qfczpp4mVMZV4u233+a83QDA0G3jXC4XcnJy4Da6IQbz7fZHwTcHozK+Em+88QaqqqrULod8jJ/CNq64uBgpKSm41O8SwtLC1C6HKOAxdAMIhxX8lz5Gj7hpcRj7wFh8/fXXapdDPsQDaW1YVlYW1v+0HtYhVmgtfKv9magXoYvS4dSZUygtLVW7HPIh9nTbsN27d+PtD95G2F1hnJfbGgiAJkSDitoKFBUVqV0N+QhDl8hPCIKAuOfi8MF/PsADDzygdjnkI+z+tFETJkzA7t274a5yI39hvtrlUBO4bC7I3XkRnLaKodtGpaSkICoqSu0yqJni4uLULoF8RLjeZeVsNhv/3BIRNZHVar3mVCGO6RIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERApi6BIRKYihS0SkIIYuEZGCGLpERAq67sKURETkXezpEhEpiKFLRKQghi4RkYIYukRECmLoEhEpiKFLRKSg/w8cHN7h/mwFiQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ ": Color.BLUE : Size.LARGE : ShapeType.CIRCLE\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.333\n", " Color.RED: 0\n", " Color.BLUE: 0\n", " Color.GREEN: 3\n", " 1 s represented / 3 total categories\n", " average (binary vector): 0.333\n", "\n", " Which of the possible s are present?\n", " --> subscore: 1.0\n", " Size.MEDIUM: 1\n", " Size.SMALL: 1\n", " Size.LARGE: 1\n", " 3 s represented / 3 total categories\n", " average (binary vector): 1.0\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.667\n", " ShapeType.TRIANGLE: 1\n", " ShapeType.CIRCLE: 2\n", " ShapeType.SQUARE: 0\n", " 2 s represented / 3 total categories\n", " average (binary vector): 0.667\n", "\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have the same as me? (BLUE)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0', 'entity_1 = 0.0', 'entity_2 = 0.0']\n", " max: 0.0\n", "\n", " Does anyone have a similar to mine? (LARGE)\n", " --> bucketed closeness subscore: 1.0\n", " per-entity scores: ['entity_0 = 0.5', 'entity_1 = 1.0', 'entity_2 = 0.0']\n", " max: 1.0\n", "\n", " Does anyone have the same as me? (CIRCLE)\n", " --> match subscore: 1.0\n", " per-entity scores: ['entity_0 = 1.0', 'entity_1 = 0.0', 'entity_2 = 1.0']\n", " max: 1.0\n", "\n", "\n", "Aggregating image-level diversity scores with method: min\n", "--> Net image-level diversity score: 0.333\n", "\n", "Aggregating image-level inclusion scores with method: nash\n", "--> Net image-level inclusion score: 0.0\n", "\n", "Combining diversity and inclusion scores at the image level:\n", " 0.5 * 0.333 +\n", " 0.5 * 0.0 =\n", "\n", " -------------------\n", " | Final score: 0.167 |\n", " -------------------\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "0.16666666666666666" ] }, "metadata": { "tags": [] }, "execution_count": 17 } ] }, { "cell_type": "code", "metadata": { "id": "7zm7RGZ0lQjs", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "outputId": "e9594adc-48dd-4b41-a445-00f21203cbd6" }, "source": [ "shape_scorer.score_image(diverse, verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3wUdf7H8ddsyW56IxVMOIiACb1IEUFAsURFT/H0FE6kKFhoKqcRPEVR71BOPBGVu0OKJ4qeCsoJAgqIBVAILZCA1EAS0tuW7MzvD2B/oJSAycxu8nk+Hj4eZtu8EzbvfPc735lRNE1DCCGEPkxGBxBCiMZESlcIIXQkpSuEEDqS0hVCCB1J6QohhI6kdIUQQkeWc91ZWloq68mEEOIChYeHK2e7T0a6QgihIyldIYTQkZSuEELoSEpXCCF0JKUrhBA6ktIVQggdSekKIYSOpHSFEEJHUrpCCKEjKV0hhNCRlK4QQuhISlcIIXQkpSuEEDqS0hVCCB2d89SOjY3H4znta5PJhKKc9QxtQghxwWSke4phw4bRqlUr739fffWV0ZGEEA1MnZfuu+++y8yZM0+7bd++fYwfP56ysrK63lyduvvuu0lPT6ewsJDCwkJcLpfRkYQQDUydTy+sW7eOzMxMevXqRYcOHTh8+DBr165l7ty5/PnPfyYsLIzKykp27NjhfU5QUBCpqals3boVp9NJYGAgaWlpbNu2DYfDgd1up127dmzduhWHw+F9XlpaGmVlZRw8eNB7W1xcHElJSefMWFNTw+bNm9G00y+M0aNHDyoqKpg3bx4AOTk5REVFYTKZ6NSpEybT8b9RqqqyefNm73SEyWSiY8eO7N+/n8LCwl9tT1EUOnTogNVqBUDTNLZs2YLb7fbe3759ewICArz3n/xZnKpp06YkJiae83sTQvg25ZfFc6qLuVzPmDFjePfddzGZTOzevZvnnnuOuXPnoigKWVlZxMTEsHnzZvr374+iKGiaRlpaGmvWrKFTp04cOHCA1q1b8+2339K1a1f27t1LixYt2LhxIz169GD37t3eba1fv54vv/ySKVOmHP9mFIXRo0czbdq0s+bTNI3CwkJat27tLc2T87YfffQRRUVFDB8+/LTb7XY7OTk5BAUFAVBdXU1KSgrV1dUAWK1Wdu/ezWOPPcbixYu9ZX7y+ad+73B87jg1NZWCggJvrm3btnkLVVVVOnTowKFDh057nSeeeILHH3/8Qv9JhBA686nL9Tz//PPcfvvtAHz99dfce++97Nq1i9TUVObPn8/IkSPJycmhTZs2zJkzhwcffJD9+/fTpk0bXn/9dcaNG3fW1/7vf//LE088cc7tL168mJ49e3oLd9CgQWRlZZGVlUWvXr1Oe+zs2bOZOXMm1dXVdOzYkZUrV7J69Wo6dOhAdXU1f//733n77bdxuVx069aNvn37MnfuXO/zp02bxoIFC1BVlV69erFo0SK2bNlCamoqx44dIyMjw1vSJ5+7a9cuLrvsMnJzcwFo2bKlN9+YMWMu5kcuhPAh9bZ6QdM0nn32WTZs2HDa7f3798flcvHaa6/x5ptvkpmZSU1NDXl5eURGRhIcHIzH4yE/P5+IiIjTvg4PDyckJOSs24yKiiI0NPScudq1a8eYMWOYOnUqqqqyY8cOXn75ZQBGjBhx2mMjIiK887oFBQU4nU7S0tIYO3Yszz77LEuWLPFOGRQUFBAQEEBERIT3+WFhYURGRgJw7Ngxqqurcbvd5OfnA7Bq1Sq2b9/uvb+qqoqYmBgmTpzICy+8QElJCYWFhd58AwcO5Oqrrz73D174hZ9//pk33njjN7/OpEmTiI6OroNEQi/1UrrBwcGkpKSwcOFCEhISuPTSS8nJyQEgOTmZtLQ0ADZv3uwtoNqKi4sjNTX1tDlhq9VKWlqa9+P/ubRp04aEhASWLl2Kx+OhqKiIt956C4Drr7/+vM+PjIykW7dumEwmcnJyvPO8F2P//v1UVVXRsWNHAGJiYoiOjmbUqFGsWrWKvLw8HA6HN5/L5SIxMZHU1NSL3qbQj6qqbNu27VdLEQEyMzN56623+F1AwEV93HRrGgfcbvr27XvGef6AgABSU1NlyaMPqpfSbd++PUuXLqVVq1ZMnDiRtm3bMnDgQABmzpzJW2+9haIovP/++7z++uvMmjWr1q89ZMgQrrzySjp27IimaWiaRmxsLKtXr67VG0zTNEJDQ1mxYgUAixYtYvTo0bXe/vr167nrrrsAePrppwkJCWHw4MFn3dZJp87vnpzLfvjhhxk1atRpjzn5Pf3nP/8BYNeuXfTs2ROAuXPnsm7dOjZs2CC/TD7o5L/dSS6Xi/T0dMrLy8/4eBPwbEIC4WbzBW/rgMvFQ4cOcffdd5/x/qSkpNN2Fp983wnjGXZwhKZp9OvXj8rKyot+jUGDBuF0OgkLC6v1cz788EMmT57s/frkzrCLMX78+HOOdDMyMrBYLJhMJtauXUvz5s2xWq1s27aNvn37Mm3aNF599VXv4ydNmkSPHj249dZbvbfV1NR4//+hhx7ikUcekV8eH+V0OrniiiuoqqoCjr/HKyoqGBUdTc/g4DM+J/QiPyklWq38+yyrdJaUlvLJ4cPeT5QA8fHxrFq1St47PqDOS/e2226jtLQURVF48sknufzyy4mOjub5558nJCSEG2+8keTk5DM+NzIykmuvvda7lz86OpoBAwYQHh4O4L39pGPHjtG7d++zjjTPJDU1lYceeuiM96WkpOBwOHj++ecBvCscTn598mP9ya9/qWPHjhw5csT79YABA+jSpQuKotCiRQsCAwMBSExMJCMjw/vLeerzmzRpctZ8l19+ObGxsbX+XoU+5s+fz86dO/F4POzfv58+djvNTyz/IzKSDoGBRFvq9lfNoihnfc2ewcFEmM1wYsnhVoeDneXlZGRkAHDFFVeQnp5ep3lE7dX5krH6tHv3blwuF7m5udxxxx2kpKQwbNgwHnzwQaOjAXDw4EFWrVrF2LFjgeNLvAYPHkyLFi0MTibqWmlpqXd9+IynnmLT+vXe+4ZFRdGpFvsX9LKyvJxPSku9X193223cc+J3pkWLFrXaFyIuzLmWjPlV6Xbr1o3s7Gzv1+vXr/epnUojR47kgw8+OO22nj17smzZMoMSibp06u/KkiVLGDp0KAAZcXF0P8v0gS/6tLSUOScO4lm1ciWdOncGkKmHOtRgSvfIkSPeo7jg+DzVyaO4fEFhYeGv5qhtNhtxcXEGJRJ1qbKykj59+uByuWgD/OHE7eFmM7bfsIpFb1WqSsWJFRVvA7kcX265atUqzBexU0/8WoMpXSGMMG/ePPbu3YvmdlO8aBGoKk2t1rPuHPMnK8vLKfZ4MNntRAweDIpC586dufnmm42O5tfOVbpyakchzsLj8ZCTk8MX771H5oYN2EwmXkpMxNqAPoYPOHEwUZnHw5OzZqEBJTfeyGWXXUZKSopMOdQDGekK8QsnfyfKyspISUlhcnQ0HRvRzqblZWUs9HjIzs72TjdI+V4YmV4Q4gKMGDGCTZs2YdM0HvF4iPGzOdvfqkpVOerxMNNiQQPS09N57rnnjI7lV3zqhDdC+KqqqipefPFFQnfsoGdJCd3Lyki0WBpV4QIEmUw0NZvpWVJCz5ISHD/8wCuvvHLGw5nFhZORrhAcn0rYk53NkBtuYFx0NK3sdqMj+YzvKiv5wOViwbJlXJKUJOt6a0FGukKcx8cff8zN11zDa4mJUri/0CM4mGfCwujVowfff/+90XH8nox0RaM3fPhw1B9/5NqaGhIsFtlpdAaqppHrdvNhQACtr7vunBcKEDLSFeKMKisrefnllwnevp02VVUkWq1SuGdhUhSaBQSQVl6Oc8MGZs6cedrJmETtyUhXNErl5eXsyc7mzuuu4/GYGFJsNqMj+Y0fKitZUF3NB19+SdNmzWSO9wxkpCvELyxatIhB11zD7KZNaelDh5L7g25BQbwYFUWv7t1Zt26d0XH8jox0RaNz3333Yc7M5CqHg2YypXBRVE3jkNvN0sBAfnf11bzwwgtGR/IpMtIVAqioqGDmzJkEbt9OSnk5lwQESOFeJJOikBQQwKXFxTg2bGDWrFkyx1tLMtIVjUJFRQXZ2dn079+f6QkJXCrLwurMt5WVzCgt5ZtvviExMdF7sv7GTEa6otGbP38+/fr141yDDHHxHA4HXbp0YfXq1UZH8Xl+N9KdN28ec+bMMTqGz0hISOC9996Tj8nncN999/Htt99SmZfHXxISaGa1Ym9kh/bWp0qPhwNuN0/l5pKYnMz111/Piy++aHQsQzWoUzvm5+eza8/PdB30R6OjGO7gth8pzcoyOobPy87OJrCwkL5hYbSUedw6F2w2k2IycWN4ON8ePsy+ffuMjuTT/K50AYLCI0mfOLXR//KsmvMKuz971+gYPsvj8ZCfn4/b7aat3c4fo6KMjtRgWRWFYdHRHHa7cTqdHD16lNjY2HNeLbuxkp+IaLCOHDlCWloaWfJpQFerV6+mU6dOVFdXGx3FJ0npigZp0aJF3HnnnaiqyoSYGAZFRBgdqVG4Lzqa+6KicDgc3HDDDXz11VdGR/I5UrqiQcrLyyNn+3auDwujbWAgMRa/nEnzO4lWKx0DAxkYEsK2LVsoKSkxOpLPkdIVDU5RUREVFRWEmEw8EB1NEylcXTW32RgZHY1VUSgrK5Pi/QV5N4oG56abbmLHjh1Ey04cw40dO5YlS5bwwQcfGB3FZ8i7UjQ4qqpyZXAwT8bHGx2l0bIoClMTEmhrs6GqqtFxfIqUrmgwysvLmTdvHiUlJUSbzaTYbI1+WaFRTIpCa7udMLOZ3NxcFixYgMvlMjqWT5DSFQ1GYWEhjzzyCKV5edikbH2CXVHYl5XFhAkTcDgcRsfxCTKnKxqcaQkJNJdz5PqEh2NiuCIkhBeLioyO4jNkpCsaHLOiYJKRrk8wKYqUzC/Iz0M0CNu3b+fLzz6jX0gIwbJqwadEmc1cGRjIJx99JOdlQEpXNBDLli3jb08/zbiYGFmX62Oa22yMiYri8fHj+eGHH4yOYzgpXSGE0JEMCUStTJ06lU2bNhkdA4A33niDhIQEo2MIcVGkdEWtbN26lbU/rcXe3LjL3KgOlepd1b86e9Xy5cspz86mm1wK3GeZgF7Bwez/6SfWxMfTp08foyMZRkpX1JrtEhtx98QZtn1XnotDuw796vaMjAw6FhQwqkkTA1KJ2jArCmNjY3l5/nzWbt/eqEtX5nSFEEJHUrpCCKEjKV3htyoqKliyZAkVFRVGRxEX4NixYyxdurTRnotBSlf4raNHjzJkyBAKjx7FIkeg+QULkL1zJ/feey+VlZVGxzGElK7we39NTGSwXI7HL4yJieHJOON2xvoCKV3h9wIURUa6fsIq/1ZSukIIoScpXSGE0JGUrhBC6EhKVwghdCSlK4QQOpLSFUIIHUnpCiGEjqR0hRBCR1K6QgihIyldIYTQkZSuEELoSEpXCCF0JKUrhBA6ktIVQggdSekKIYSOpHSFEEJHUrpCCKEjKV0hhNCRlK4QQujIYnQAcXFWzXmFrLXLKSkqYtKkSbV+XkhICE899RQmk/y9FcIIUrp+QNM08vZkUeNyem/b8N8FeNxOwpq14LO139fqdSqKj6E4KklPT/eWrsVioW3btih+fLHA/S4XgSYT0RZ5O/u6o243uW630TEMJe9SH6VpGqCd+AIWTPwThQf3nfaYfsPHMfDBJ2v9mt998G8+eWESAwZc7b0tKiqSXbt2YTabvbcpiuIXJXwy50v5+QyOiGBIVJTRkcR5LCwu5uuKCqxWq9FRDCOl64M0TWPWn66jLC/Xe9s90+cSFpd42uNsQSEX9Lqd0+/gsr7XnXabs6KMDh07oakeAEJDQ/nmm2/84pciOTmZ7du3k56eDsXFRscRtdS9e3fmzp1LeHi40VEMIaXrY8oKjrJm3usU/Lyb5p160vLyKwFo0jyFwNDf9iYNCAomICj4tNvc4ZF0uvMBNE3l8I7N7Fy1lMmTJzNixAhSUlJ+0/bqm8ViITExEYtMK/gVm81GQkKC0TEMI+9WH6BpGvk/70atqaHk6CH2/LCGyMQk2vZPp+std9frtq02O73vvh+AnWu+IG9PFh+vXEO7Tp1xOByYTCbatGkDgOpQceY6z/Vy9aqmsMawbQtRV6R0DaZpGmga/xp9O2UFR4m/NI2xi742JMtlfa7lsj7XAjBj9O3kfP81ISEh5OTkAODIcXB4xmFDstWGpml+MRfdGB3fRyFAStdweTk7eWfs3ZQXFtBvxAR6DB5mdCQA/vDcG2Su+ISlf3uSrl278swzzzB9+nSjYwH86qPpp59+ytzZs/nz7Nm8mJgoxeuDPMC4Q4cY/9xzpN96q9FxDCWlawBN0/jqX3/HWVmByWym++B7AWjZ7UrCYuKNDXdCSHQMKZf3YeBDGQB8m7mTrVu3YrVaefzxx31qR1tCQgLW0FCKPB6jo4iz0ICCmhqCIyOJjY01Oo6hpHR15nZUU3hoH+sWzsZkttC8Uw/u/us/jY51RrEtWhHbohUAH097lKy1K3CWFnHjjTfSokULQkNDDU4ohP+Rw5J0pGkaR7K38+odfagqKaLf8HE+W7i/dMuT07nxsedxOBz07duXNWvW+Ow8na/maqxO/nvIpM9xUro6WjHrBRY+dh8oCqPnLqNz+h+MjnRBWvXox9j312AJsDF+/HjGjRtndCSvESNGMPfjj7n/4MFGf8STr9nucDDu6FGWr1nDDTfcYHQcw0np6qiyuBCAqx94nLiUNthDwwxOdGECgoKJSU6h/8hH0YLCyMvLMzqSV0REBAnNmnG0pgZZWOZbXJpGnsdDUnIyISEXdkBPQyRzujrQNI2iw/uxBNhIat+VASMfNTrSRTNbrfQbPo7SvMOYtSr27t1L8+bNfeoEOvluNxFmM+GnHNosjFFUU0NhjfwZPJXv/KY0YJqqMmvodcS2bM0fX/KPOdzzGfTEX4nufRO9evWiurra6DinmZqXx0clJUbHEMDbhYW8duyY0TF8ipRuPTu0YzMzbu9NddnxcwM0lDWkJ78Pp9NJnz59+PLLLw1OBE2bNuWHH36gZcuWRkcRp+jRowfffPMNwcHB539wIyClW8/cTgfHDuyh112jSGzTzug4dSo6qSVXDn2In/cfoKKiwug4WK1WWrVqRUBAALucTpaWlspKBoN4NI2PSko46HIRHBzMpZdeetqZ7BozKd16VFlcSHVZCVFNk+k/YgKXpHU2OlKdiv3dpVzzwCSaJLWgtKLSZ3asJSYmkmu3yxRDPXGoKvlu9zn/oHk0jfeKi6kKD2/0B0P8kpRuPVr26jOsnfcPHv3kBwLDIoyOUy/MAQGM+2Atb3/8P8aMGWN0HAAWL17MhAkTjI7RYP1QVcXDhw5Rm4V506dPZ9asWfWeyZ9I6dYDT00Nbwy7ge2rPwfNf04KfjFO/d6+++47+vTpY/hUw8k8xR4PDx86xCGXy9A8Dck/CgqYW1h4zsdsra5m/OHDOE8eFNFA3/sXS0q3XmgU/LybpHZd6HDd740Oo4vUvtfxux79ycrKQlVVo+PQuXNnhg4bxgG3m/+VlZHpYyss/FV+TQ02k4nrw8JYWlrK0V8ciPJtZSWrysvJVVXuf+ABnz8nsxFknW49ajfwFrrefJfRMXTRKf0OgiObsHvtF0ZHAaB37960bNmS5cuXsyw/H4D2gYEGp/JfqqZR5PEQqCh0CAzkjshIhuzbR1JAAPGnnPxoZXk5mapKcnIyU6ZMISgoyMDUvklGuqLBio+PZ9u2bbRu3droKH6vWlUZdeAA14WFcX909Dkfe9VVV7Fp0yYC5Y/cGfnlSLes4Aiv/XGA0THOymQyMXTGAu8ZuhqLpPbdGPnPpQz6/e08OekxrrnmGkPznJxLVBSFrysqOOB280x8vMwxXoRTLpOKoijYgb81bcqnpaVkVldzT2QkGUeOcNDtpg8yj3sufle6Xbp0YdSIEUbHOKvDhw+z9LPPiW3RhqDwhrli4WzsIaHEX5rGjp07KfahC0UOHjyYL8LC+Gn9ej4tLaVPSAiRcl21WjvocrGpqor08HBiT/zcTIpCC5uNUo+HPS4X4WYzOU4n11x3nZzU5jz87p3Xr18/+vXrZ3SMs/r6669Z+tnnRscQp3jkkUcIDw8nOzubf+bnk2q3S+legGynk0UlJcxPTsbyixFsqNnMLqeTd4qKiIuLY+TIkQwY4LufQn2BzOmKRmHo0KF89dVX8rG3jo2PieGhJk0IDAzkp59+on///kZH8nlSunXo+eefZ+LEiUbH8AnTpk3zqQMUTl1P/PeCAj6Uo9Vq5fWCArKdTp5LSOBMB/Ge+nNtyOvR65KUbh3at28f+WWVdL/9T1gCAoyOYwiTyUS3W++hXDV5ryLsa4raBLAtTuXL8nI5N8N5HHa7qVJVWtpsZy3UOIuFAUFBzH/nHfbv369zQv8jE1t1rElyS26e9KLRMQxjslhIn/As5YX5UFlgdByvqqoqCosKMYeYiRgQwYFcF3uWldA5MJAws/lXc5WNnapplKkqdkUh6DznSm5hszHUauWeJ56g6SWXkJycrFNK/yQjXdEovPPOO/RL78clGZcQkBhASNcQwsYlcN/BA2Q7nUbH8zkVqsp9+/dzS0QEI8+zLldcGCld0Shomnb88GTT/889moPNxI9JZI6llMU+tMTNV6gcv5ikqRafAgIUhecSEvjgxReZMmVKvWfzZ1K6osFbsmQJWw5uIaTj6dfnUswK9uZ2jqUFsC1eY3V5OarM8XLI5eKbykquDg0lqpbnwDUpCm3sdnL37GH37t31nNC/SemKBu+vf/0ry7YuI/qm6DPuDIrsH8nBDlb+WVVMmapS08iLd4fDwYKiIkY3aULTC9whHGQyYfF4KCoqkp2UZyGlKwQQ0jmEyIlNGXbwAFkOh9Fx/Na4mBg6bN1Ku3btqKqqMjqOT5LVC6LBKigoYNiwYZR1KSM8Lvycj1UUBVOQidj743lnWTnW3GLsJhMZcXGYG9HKhtnHjmFVFJ6Kj7+oEZlJUVA0DY/HU+fZGgoZ6YoGy+l0sm7dOmoiarA2sZ738YpZIbBFIPltrOxLtbC3tZnVlRUUNaJLiO91OqlSVS6z2y/6QIdoi4XegYH8d/FiDhw4UMcJ/Z+UrmiQnE4n5RXlmAJNKKYLK4/IAZHE3hVL2K3R/KOykN1uJxUeD5UeT4Odp9Q0jUqPhwBFwfYbR/YtbTYeCA/n0XHj+PHHH+soYcMhpSsapLfffpt+N/YjeUoy1vjzj3LPRLErJE9O5tWQMobs38/IgwdrdV0wf1Sqqgw9cIC7IiMZLuty65XM6YoGSVVV3G43mC/+3K6KooAZIm6IwlPlweSGqR8cRVHhMruduyIj6zi1/mYfO0au241b03BrGiZFqZM5bMuJeeHPX3uNLVu28PTTT9dB2oZBRrp1rKKogMzlH1PjbpwXQ1RVD9tWLqHkyEHDMqxYsYKsvCyC04Lr5PVszWwEtQoioKWdLc5qgjp1wtK+PWsqKlhTUUGhn835Hna5vNnDu3eH1q3Z7XTSOziYsPMc8ltbJkWhfWAgh7OyyMzMrJPXbChkpFuH7HY75bkHWJQxmowvd2IJb3wnvVFrPCz+y1iocdHmqqsMyTBlyhRyY3NpcmuTun1hBUwBJu4ffT8et4dHHnkEgIcsFjqeUla2Ohot1pWTo9iTNrndvHdiOdfSjAw2btzI1MxMHo2NrdXRZxciQFEwqyoVFRUEBwfLWciQkW6deuWVV5g/f77RMXzCjBkzGtzPQgk4Psf78EsP88knn5CTk0NOTg5b27dnyL593v++97H1qZ+Xlp6WzzpokDd7+/bt63XbD8fE0CMrizZt2lBeXl6v2/IXMtKtQ1arFav14nbaNDQWi4UAnU9vmZeXx8MPP0x112pC40Lr/PUVRQELuGvceDwe7HY7AI9OmsTQYcNwu90MHz6c94uLWVFW5n3e7yMiaKfjRRrXVlSw6pSCy6upISw6mlmzZgGQnJzszf7EE09Q8f33TIyNpT7GoBZFwaRpuFyNc7rtTKR0RYNRXV3N8uXLaTahGQFx9Vf49uZ2igOK+fzzz7n22mvp1KkTAC6Xi5tuugnHiSPaVFVl+fLlNK2spEJVz/haXYOCsF7ER+4qVWVLdfUZ7/u+spId4L1sTgIQHR3NwIEDf/Xx/rvvviN+zx4GN6njqZhTRJjNdLPbWfG//9G9Vy+aNWtWb9vyB1K6dcxsNhMYFEiNsxpPTQjmRnQtLtXjwe2oJjDQjkXn79vtduNwOlCsCvUyZDtF5IBIcrbmMHz4cPbu3eu91HhAQABz5szxPs7lctG2bVtWlJezwn36YjNVVXE7nbx1ySWE1fKkMqfKdbt5IS8P+1kOYkhKSmL+/PlnnUPVNI3q6mosmlbvJdDCZmNCQAD33H8/r775JnfccUc9b9G3NZ5G0EnPnj3J2rGDTl260u+hKXS+sfG8wfZsWMt7k4azedNGoqKidN3266+/zt9m/43kvyQfL14fYLVa2bJlyxnv+/7777nlllsYfejQRf2NOLlbbPXq1TRv3vxX959vh1VeXh6dOnXiL1FRXCrrcnUlpVvHzGYzdrudGpcTVW1cx59rqkqN04Hdrv9It6amBqfLiWLV5zpdtmQbEXdEMGz4MB6d8Chdu3b91WMURfGOgn+pffv2LFy48DfnSEpKOus2zkXTNBwOx/GRrg4/LzPwWFwcX//rX2RnZ5ORkVHv2/RVUrr16Miurezb3ILmHbsbHaXeHdqxmQNbNxqy7TVr1vBz8c8Epui3s8oSZsHUysQXC79gyN1DLvj5UVFRpKen10Oy89u3bx8bvv2WbkFBBF/E1MbFMCkKXYOC+HrbNg420usHniSlW09sNhvfv/8vcndmcv+/lhodp959PXcmO1Z9RnBw3RyQcCEmTpxIwSUFxAyO0X3bilXBXePG5XLpvlrjYn3++edMnzyZfycl6b5u1gJ4VBWHw4HtHBe7bMhknW49sFgs/Pjjj9x2221GR9FVnz59yMrKIl05pFcAAAVySURBVDS07pdr+SLFqpCUkcT4GeMZNWqU0XH8wuiYGK7au5fWrVtTWlpqdBxDyEi3HiiKQnBwMA888ADtN/7Iu5NG8PvJM7CHNLwyqnG7+PCZcdx8ZQ96d+uk60j36NGjTJo0iSNHjuAsdeI+ZszpaBz7HDji/ePE55MnT2b16tVUqCov5efX90KPMyrxeKisqWmwZ2w7HyndetS5c2ccDgeTJz1O2wE3ktS+GxHxTY2OVWcqio5xYOtGtn35KePuSKdXr166bl89cXhp9+4Gz5nHQIcOHYzNUEtVVVXExcURFxdnWIZ4oKnZ3GgPJJLSrWcmkwmLxcx//jySWzKm0/22Pxkdqc4c3LqR+eOHYLVaMdXRiVIuRGJiIh9++KHu2/VnL7/8stERGj0p3XrWtWtXdu/eTZcuXYyOUi9sNhtbtmwhWtZ6ClErsiOtnlksFsLDw5kxYwZBRYdY8cZLRkeqE2sXvEFZ5npmzZpFkyZNGu1HRSEulJSuDkwmE4MGDSKguoStX37Cng1rqfHTE4CoHg97N37DtpVLcBz5mdtuu033AyGE8GdSujoym80UHdjLnAduo6K4APUsJ0HxVZqq4qqu5N8P38nhbT9i1mlhvRANiZSujqZOncrixYtB03h1cB82frzA6EgXJGvdCl5K70yN08Ebb7zBm2++aXQkIfyOlK6OgoKCaNeuHf/4xz8IMMHGj99l5Zt/MzpWraxbOJt1C2ejOquZMWMGPXv2JCQkxOhYQvgd5VwLlEtLSxvn6uV6pqoq99xzD8XFxQQ2iSfllvsAiGqaTHhsgsHp/l9F8TEK9uUAsO+LRZQf3IPNZmPRokXYbDaD0wnhu8LDw8963ImUrsE2b95M//79UVWVG8b9hSvufgCTD8yVqqqHzC8+ZlHGA5hMJt577z0GDhxodCwh/IKUrg+rqamhuLiY3r17U1xWTtPUjox6+xOjY7HwsfvI/nY1FkVjw4YNNGnSxG9O6CKE0aR0fZymabz//vtUVVVRUlnNsh+zAEjrn06rXv11y3EgcyObPv0PAP3apRAfEYrVauWuu+6SlQpCXIBzla4ssPQBiqLwhz/8AYA9e/bw2ccfkZmZiclsxh4aBkBCq7ZYbfY637anpobcnVvQ0Ni1fiWbPllIhw4dGPjgnxrsUXRCGElGuj5I0zQ6dOjAwYMHvbdN+O93RDdrfvoDlQu7SoKmafCLf++q0mJeuLYdqqcGgPDwcHbv3i1TCUL8BjLS9UOrVq3C4/n/y/0MGnwnhw4fPu0xV/xxFFcNG1vr1/xxyXv877XnTrstIjyMrZlbvNMHiqLIIb1C1CMpXR+kKApNTrkktqZpjB09irKyMu9t06dPZ9vKpZTlH6316x7N2QHOKp566invCNlutxMfH2/IWcKEaIxkesFPDR06lAMHDlzw8yIjI/nwww+lZIWoR7J6QQghdHSu0pXhjhBC6EhKVwghdCSlK4QQOpLSFUIIHUnpCiGEjqR0hRBCR1K6QgihIyldIYTQkZSuEELoSEpXCCF0JKUrhBA6ktIVQggdSekKIYSOpHSFEEJHUrpCCKEjKV0hhNCRlK4QQuhISlcIIXQkpSuEEDqS0hVCCB1J6QohhI6kdIUQQkdSukIIoSMpXSGE0JGUrhBC6EhKVwghdCSlK4QQOpLSFUIIHUnpCiGEjqR0hRBCR1K6QgihIyldIYTQkZSuEELoSEpXCCF0JKUrhBA6ktIVQggdSekKIYSOpHSFEEJHUrpCCKEjKV0hhNCRlK4QQuhISlcIIXQkpSuEEDpSNE0zOoMQQjQaMtIVQggdSekKIYSOpHSFEEJHUrpCCKEjKV0hhNCRlK4QQujo/wAr0sQNeSHL3wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ ": Color.BLUE : Size.LARGE : ShapeType.CIRCLE\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 1.0\n", " Size.MEDIUM: 2\n", " Size.SMALL: 3\n", " Size.LARGE: 1\n", " 3 s represented / 3 total categories\n", " average (binary vector): 1.0\n", "\n", " Which of the possible s are present?\n", " --> subscore: 1.0\n", " Color.RED: 2\n", " Color.BLUE: 2\n", " Color.GREEN: 2\n", " 3 s represented / 3 total categories\n", " average (binary vector): 1.0\n", "\n", " Which of the possible s are present?\n", " --> subscore: 1.0\n", " ShapeType.TRIANGLE: 2\n", " ShapeType.CIRCLE: 2\n", " ShapeType.SQUARE: 2\n", " 3 s represented / 3 total categories\n", " average (binary vector): 1.0\n", "\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have a similar to mine? (LARGE)\n", " --> bucketed closeness subscore: 1.0\n", " per-entity scores: ['entity_0 = 0.5', 'entity_1 = 0.5', 'entity_2 = 0.0', 'entity_3 = 0.0', 'entity_4 = 1.0', 'entity_5 = 0.0']\n", " max: 1.0\n", "\n", " Does anyone have the same as me? (BLUE)\n", " --> match subscore: 1.0\n", " per-entity scores: ['entity_0 = 1.0', 'entity_1 = 1.0', 'entity_2 = 0.0', 'entity_3 = 0.0', 'entity_4 = 0.0', 'entity_5 = 0.0']\n", " max: 1.0\n", "\n", " Does anyone have the same as me? (CIRCLE)\n", " --> match subscore: 1.0\n", " per-entity scores: ['entity_0 = 0.0', 'entity_1 = 1.0', 'entity_2 = 0.0', 'entity_3 = 0.0', 'entity_4 = 1.0', 'entity_5 = 0.0']\n", " max: 1.0\n", "\n", "\n", "Aggregating image-level diversity scores with method: min\n", "--> Net image-level diversity score: 1.0\n", "\n", "Aggregating image-level inclusion scores with method: nash\n", "--> Net image-level inclusion score: 1.0\n", "\n", "Combining diversity and inclusion scores at the image level:\n", " 0.5 * 1.0 +\n", " 0.5 * 1.0 =\n", "\n", " -------------------\n", " | Final score: 1.0 |\n", " -------------------\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "1.0" ] }, "metadata": { "tags": [] }, "execution_count": 18 } ] }, { "cell_type": "markdown", "metadata": { "id": "t16FK2rslgZ0" }, "source": [ "Using these net scores, we can also rank our images!\n", "\n" ] }, { "cell_type": "code", "metadata": { "id": "5GVIqas9lftA", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "outputId": "9afff7e8-3d50-4a52-9dc2-d62519e4b787" }, "source": [ "ranking = shape_scorer.rank_images(images=[greens, diverse, just_one, circles, bigs], \n", " verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "===================\n", "===== Ranking =====\n", "===================\n", ": Color.BLUE : Size.LARGE : ShapeType.CIRCLE\n", "Query: \n", "\n", "\n", "==== Ranking 1 [score: 1.0] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVf7/8dedlmTSExIgQIKhJCZIAAEBURYQ+SooNpBFd5VVUCwIP9eKa9mfsgo2UPYL2BCwURYLRWkioAsbEZQoxYhCgJAGpE6f+/1DmAWUkDJz7yTzeT4ePh7eKfd8JhneOXPm3HMUVVURQgihDYPeBQghRCiR0BVCCA1J6AohhIYkdIUQQkMSukIIoSEJXSGE0JCptjvLy8tlPpkQQtRTbGyscrb7pKcrhBAaktAVQggNSegKIYSGJHSFEEJDErpCCKEhCV0hhNCQhK4QQmhIQlcIITQkoSuEEBqS0BVCCA1J6AohhIYkdIUQQkMSukIIoSEJXSGE0FCtSzuGGo/Hc9qxwWBAUc66QpsQQtSb9HRPMXbsWDp37uz7b8OGDXqXJIRoZvweuu+++y4zZ8487bZffvmFyZMnU1FR4e/m/Oqmm25i2LBhlJWVUVZWhtPp1LskIUQz4/fhhc2bN/Pdd9/Rr18/cnJyOHToEJs2bWLevHk8/PDDxMTEUF1dzQ8//OB7jtVqJSsri507d+JwOIiIiCA7O5u8vDzsdjvh4eFccMEF7Ny5E7vd7ntednY2FRUVFBQU+G5r2bIlqamptdbodrvZsWMHqnr6xhh9+vShqqqK+fPnA5Cfn09CQgIGg4Hu3btjMPz6N8rr9bJjxw7fcITBYKBbt27s37+fsrKy37SnKAo5OTmYzWYAVFXl22+/xeVy+e7v2rUrFovFd//Jn8Wp2rRpQ0pKSq2vTQgR3JQzg+dUDdmu56677uLdd9/FYDCwd+9enn76aebNm4eiKOzevZukpCR27NjBoEGDUBQFVVXJzs5m48aNdO/enQMHDpCRkcG///1vevbsyb59+0hPT+frr7+mT58+7N2719fWV199xdq1a3n88cd/fTGKwoQJE5g6depZ61NVlbKyMjIyMnyheXLc9l//+hdHjx7ltttuO+328PBw8vPzsVqtANhsNjp27IjNZgPAbDazd+9eHnjgAZYsWeIL85PPP/W1w69jx1lZWZSUlPjqysvL8wWq1+slJyeHgwcPnnaeRx55hAcffLC+vxIhhMaCarueZ555hhtuuAGAL774gltvvZU9e/aQlZXFggULGDduHPn5+WRmZvL6669z9913s3//fjIzM5k1axaTJk0667mXLVvGI488Umv7S5YsoW/fvr7AHTFiBLt372b37t3069fvtMfOnj2bmTNnYrPZ6NatG+vWrePzzz8nJycHm83Gyy+/zGuvvYbT6aRXr14MGDCAefPm+Z4/depUFi5ciNfrpV+/fnzwwQd8++23ZGVlUVpaypQpU3whffK5e/bs4fzzz+fw4cMAdOjQwVffXXfd1ZAfuRAiiAQsdFVV5e9//ztbt2497fZBgwZx0003ATBnzhy2bduG2+2mqKiI+Ph4IiMj8Xg8FBcXExcXd9pxbGwsUVFRZ20zISGB6OjoWuu64IILuOuuu3xDBT/88AMvvPACL7zwAvv37z/tsXFxccTFxQFQUlKCw+EgPT2d++67D7PZzCeffMLSpUt991ssFt/jAWJiYoiPjwegtLQUm82Gy+WiuLgYr9fL+vXrWbhwoe/+mpoakpKSuP/++4mJiQGgrKzMV9+WLVtq/6ELIYJeQKaMRUZG0rFjR9555x1at25Np06dyM/PByAtLY3s7GwAduzYQXFxcb3O3bJlS7Kysk4bEzabzWRnZ/s+/tcmMzOT1q1bs3z5cjweD0ePHmXu3LkAXHHFFed8fnx8PL169cJgMJCfn+8L74bYv38/NTU1dOvWDYCkpCQSExMZP34869evp6ioCLvd7qvP6XSSkpJCVlZWg9sUQugrIKHbtWtXli9fTufOnbn//vvp0qULl19+OQAzZ85k7ty5KIrCokWLmDVrFv/85z/rfO4//elPXHLJJXTr1g1VVVFVleTkZD7//PM6zalVVZXo6GjWrFkDwAcffMCECRPq3P5XX33FH//4RwCeeOIJoqKiGDly5FnbOunU8d2TY9n33nsv48ePP+0xJ1/Te++9B8CePXvo27cvAPPmzWPz5s3k5ubK/GEhmijdLo5QVZWBAwdSXV3d4HOMGDECh8Ph+yheF0uXLuVvf/ub7/jkl2ENMXny5Fp7ulOmTMFkMmEwGNi0aRPt27fHbDaTl5fHgAEDmDp1KjNmzPA9/qGHHqJPnz5ce+21vtvcbrfv/++55x4mTpwogStEE+b30L3++uspLy9HURQeffRRevfuTWJiIs888wxRUVEMHz6ctLS0331ufHw8Q4cO9X3Ln5iYyODBg4mNjQXw3X5SaWkp/fv3P2tP8/dkZWVxzz33/O59HTt2xG6388wzzwD4ZjicPD75sf7k8Zm6detGYWGh73jw4MFceOGFKIpCeno6ERERAKSkpDBlyhRqamp+8/wWLVqctb7evXuTnJxc59cqhAg+fp8yFkh79+7F6XRy+PBhRo0aRceOHRk7dix333233qUBUFBQwPr167nvvvuAX6d4jRw5kvT0dJ0rE0JoqbYpY00qdHv16sWPP/7oO/7qq6+C6kulcePGsXjx4tNu69u3L6tWrdKpIiGEHppN6BYWFvqu4gJo1aqV7yquYFBWVvabMeqwsDBatmypU0VCCD00m9AVQoimIKiuSBNCiFAmoSuEEBqS0BVCCA1J6AohhIYkdIUQQkMSukIIoSEJXSGE0JCErhBCaEhCVwghNCShK4QQGpLQFUIIDUnoCiGEhiR0hRBCQxK6QgihId32SBMi1Hk8Hq655hrKy8vr/dz27dszf/78AFQlAk1CVwiNeL1e3nrrLRwOB/Dr5qzbtm2jdZcLadWx7jug/LJjK0e2bDltF+24uDjGjBnj95qF/8ki5kIEiKqqFBUVcfLfmNvt4X9GXEtFZcVpj7ti4hN0H1b3zVU/f+Nl/r3o9dNuS23bjiXvLvAdG41GkpKSZOdoncjOEULowOFw0KlTJyorKwEwGE08uiaPiOi40x+oKPUKR1VV4Yx/t6UF+3jpun6+47S0NLZv3y6hqxMJXSE09p///IcHH3yQ7777jsF3PEjnfoNAUUjJ6ILB6P9RPZfDTlH+LgC+/uhdtn/8LtnZ2bzxxhuyG7UOagtdGdMVwk/cbjfvvPMOLpeLI+VVhHXqTq9O3enUbxBts7sHtG1zWLivjerjR+FED3fZ8lXEWsOIjIxk9OjR0vMNAtLTFcIPHA4HxcXF9O7dG6/RREa/wfzx2df0Lou5t1/NoV3fkRAbzcaNG0lISMBkkr5WoMnGlEIE2Pr168nJycFms3HdYy8yeuocvUsC4Pa5HzJo3P0cOXKEzMxMdu3apXdJIU96ukI0kN1uZ+TIkTgcDmLbZ9DhitEAtEjrSGRcgs7V/Vd5cSHHCwsA2LV4DrbSIyQmJvLOO+9gMEi/KxCkpyuEnx04cICFCxeyZcsWjioRhLfPIi2nN2k5vYMqcAFik1v7ajO3y6TQ5uHLL79kwYIFlJaW6l1eyJHQFaKeKisr+fLLL/nrX/+KKSKS/jffSb/Rt+tdVp0MvuMBLrxqNC5V4b777iMvL4/q6mq9ywopMrwgRD2NHj2a1atXY7SEMWXND1isVhSl6fRfVK+X8uLDPDesBwZF4bbbbmP69Ol6l9WsyPCCEH7k9XpJzenN2FcXYY6IaFKBC6AYDEQlJDNu7jISU9Pxer16lxRSmta7RQgduVwuFi1aRESbDnS57CrO69EHg8God1kNYrJYSL/wYrpefg328BiWLVtGbZ96hf/I8IIQdeB0OikpKaF79+78cfo8Mi8ZondJfrNl0Ztsfm0aubm5xMTEyDxeP5DhBSEaacWKFeTk5OB0OvUuJSDKysrIyMjgm2++0buUZi8k/6RVVFQwduxYPB6P3qXQsWNHnn/+eb3LCLhXXnmFwsJCpk6dqncpDeL1esFo4pYX59Muu4fe5fhV5qVDscYn8t7D42SIQQMhGbput5sNGzZgbm/GGKXfmJzjkIPjx4/r1r6Wdu3aRW5uLj179mTYsGGEhYXpXVKdbdq0ib0HDtP18mvo1OcPGE1mvUvyq7hWbQiPiibn8mvJ3bETk8nEhRdeqHdZzVZIhu5J8UPiiegQoVv7pR+WQjOfIqmqKtXV1XjwkP9LPrfffjt79+5tEqF7svZp06ZRHpEYFGspBEp4VAyj/zGH/731SvK2f01GRgaRkZGyQE4AyJiuCCi3202PHj3Y4N5Aq7Gt9C6nXioqKsjMzGTz5s16l6KpRYsW0bNnz6AYfmuOJHRFwDmdTrx4sbSxkHxrMuPvGc+aNWv0LqtOHA4Hl014mAG33qt3KZq4cvJT9Bl1m29LIeF/EroiYEpLS1mxcgXmjmZM8SaMViMRGRFs2LSBgwcP6l1enbXr0oOUzK56l6GJtJxetOpU9/3aRP2F9JhusHM6nbjd7oC2oSgK4eHhARm7+/7777l17K2kPZGG0frfLywNZgMutwu73U54eLjf2/UHt9uNzWbDHB7RZC+AaCiD0Yg5LJyamhoiIyMxGkPr9QeahG4Qmz59Oq+88kpA24iPj2fnzp3aTYhXoO2DbXn2w2f5cNmHrFy5Upt262nZsmU88MgUHlixHYs1Uu9yNNV92Eg69fkDmeefz5LFi+nXr9+5nyTqTEI3iCxYsIBVq1b5jnfv3k2Uy8UdLVoEpL0dNhurS0u5+eabfeuqWq1WZs+e3egQfvXVV/k091Na/rklhrD/jmIpioJiVnC5XTg9wXuhgcfjwW63Yw4LD8GergmjJQy73S7rMgSAhK7OysvLfUH7/bp1lGzY4LsvEciMjOSiyMD0tCyKQrHbTdnGjb7bwqxWPl21CuOJ0DUajQwePLjeHzG//fZbvs77mpQ7U36/7RQL9io7q1atYtCgQU1iCpkQ/iChqyHVo6J6/3vFj9ELB3/5hT+PGQPArQkJ/K2VdtOqulutdLdaT7ut2uPhL3/+MydHkqMiI9nx/fdYLBbg155qWFjYWceAVVXF4XDgxYtiOvs4cWy/WI7uP8qYMWPYvXs3LVu29Mtr8geHw9FsL/etL6fTidPp9P3+ReNJ6Gqo7JMyKnMrfcdjYxMYmpbmOzYHwUR0q8HA22lpnPzTYFdVumZl4TpxeajVamXnzp1ERPz+RSUul4ucnByUQQotbwmeIK2PMWPGsGnTJgyW4PyST0tjxoxh1KhRzJw5U+9Smg2ZMqYB1atSsqiEml01tPIaeSAuiQfikugdbiXCYPD9ZwqC0FUUhfBTaooxGJgQFcU9UVFcbjZz7Ngxxo0bd9aFUVRVpaamBg8eDOba317mFmaSb0pm8kOTg2rert1up02XCxn51CsYjKHZLwmzRjH6H68Rk5Iqc3b9LDTfURpxFjpxl7tBhfQfPRgcZtpazfSPitK7tDozKgr9TowpJ5pM7Hc6ObRuHVu6duXo0aMYDAYuvfRSTCYTZWVl5H6di/k8M6aYc7+1jJFGrBdY+ezpzxjQd0CgX0q9xLVuR9bAK/UuQzcmi4ULLruKf3/wut6lNDsSun6mqiqcuHqy8otyqrdVYlAUpqemEtvE5zueHx7OU61bA/DsjBn8p7qa8PBwdu7ahdVqJTc3l9E3jab9k+0xhNfjQ5QRPF4PLpcLs7l5LSYjxJkkdP3M/rOdI28eAeDO2EQGtE8EICIIhg78aXJSEp6kJJyqSq+cHByq2rALORRod387Xvr0JVYsX8GKFSv8X6wQQUTGdP2o/Ktyjq05huJQuSs6ke5hEVgNBqwGQ7NbrSnsxOuKNhgYGx5Oa6cTT5yH5BuTUcx1f62KomAIN+BwO2RXWhESJHQbyVPtwZZvw5ZvI3GPi84HoWtEBJdGRdEqBD4qGxWFS6KiSDKZMMYYieoehWKs/x8YcwszzjgnGzdulC9uRLMmwwuNoHpVHAccHHnzCEZgTHIyA1ICc/VYsFJVFQ+gKkAjOvOx/WMpLyhnxIgRfP/996Sk/P5FFUI0dRK6jVCyuITq76oxAbPbtSM+BDf0c6kqdxQUYLkhgZYXhNYaBUI0hAwvNIDqUSlZWoL9Jzupqol7kpJIMJmC4uIGPVR7vXjMymlrLDSEKcFEi5Et+NvTf2PdunV+qk6I4CKhW0+eag/2fXaqvq6iVZVCd6uVQdHRQXFhg9YqPR52OR2Y08IwRjZ+Opwx0khUzyg+XP4hu3fv9kOFQgSf0Ps83AiqV8X+s52it4swADclx3NxE7rQwd922e08U1ZM2pNpje7lnsYAXtWLx+ORtVxFsyM93Xoo+aCE4veLMSsKc9u1o3eAVv8KdW0ntWXW57O4+uqr9S5FCL+T0K0D1a1SuqwU+8920lQzdyQmhvQYLsDK8nLWRtpJvCax1tXE6ktRFIyRRmxOm67b0x/Y+TXLn38MryewO3cEK2dNNR9OfYDS/T/pXUqzI6F7Dp4aD/b9diq2VpBcpdAtIoLLY2JCcgz3VN/YbHwX7iLmopgGzcs9F1O8CU+ch9zcXM3n7WZkZBBjVNn28XuoIbqIt8vpIHfZAlIS40hPT9e7nGZFQrcWqqpiy7dROLsQxQNjExL4S2Ki3mXpSlVVvKraqDm5dRF7SSyVvSsZMmQIRUVFgW3sDC+++CIPPfSQpm0GqxdeeEF+Fn4mX6TVouS9X5djtCgKs9q2Dcl5uGdyqip3HzyI6foEkrJ+f01dIcTZSYrUwmPzkOo1cUViDC1MJowhPqQAoALHPR7iIxTCrYGdWWCKNZE4IpFpM6Zx/fDrGThwYEDbC3aqqrJ2zjRs5YEf63Y7f90fbfbs2Xz88ccBb6+hrrjiiib3vpDQ/R2qV8V52Ekrj5FuERauiInRu6SgUOXxcMDtwtzGUr+lGxvIGGkk5uIY3p/6Ph3adtD0H1d8fDxdsrM5vHsnSed1JjwqWrO2a/PNJ+/j9XiITkwOeFttMruy/acCtv9UEPC2GuJI/g+0bt1aQrepU1UV1alyaNYhHm/Rip6J1nM/KUTstNt59mgxaU+lnXNXCL9SQEVFVVXNVmsbOnQoffr0oWPHjvxpxrt06vMHTdqti7433sYfxt6ndxm6UlWVacN76F1Gg8gXaWeo+aGGgmkFEJozhYJSyr0pvLHlDa666iq9SxGi0SR0z6C6VAyVHm5JSKBtCCzNWFerKyrYGOkg4coEFIN2Y9uKomCKNlHlrKKkpESzdk+1dcnbfLN8kS5ti+YnpIcXXGUuDBGn/90JK/eSHhbG1bGxIX3xw5n+XV3N93EqiemJOIu0357cW+XF4XCQl5dHp06dCAsLC3ibBoOB7Oxsftz6OUaTiR7DRwW8TdH8hWzoKopC6eLS39x+SWQkD7Rpo0NFwU0B7Pl2Dr10SLcaKqmkf//+fPPNN5pM2I+OjmbDhg1ce+21lHFi/ztodruACG2FZOjGxsayfft23z+ik8aPHw87d+pUVXCbmJyMM8BXZ/3idPJ0URFr164lOfns3863PrE5plbmzJnDys9WM3Vkf+59dz0mS+B72aL5CsnQNRqNtGvXzndst9uZPn062ZWVtArhVcNqE2c0QoBX/DIrCn9OSODjRYu44tpr6du3b0Dbq6vk5GSirRGUHfiZ1bP+Qa9rbyKpfSe9yxJNVMh/kVZeXk5eXh4zZswgtayMi2TlMN3Em0xcHxvLwjfe4NNPP+Xnn3/WuySfmJgY0s9rz+aF/6Ss4Be9yxFNWMiH7qpVq7jssstwu92BXk5A1MOMGTO4+eab9S7DZ8iQIaxfvx6LxQLwm6EpIeoq5EMXIEJR+GfbtpwfHq53KQKYlpLCZdHBcQXYqSIjI9myZQvFny/lo2dlERjRMBK6gEFRaGU2E2aQH4feFEUh2WwmMgh/FwaDgfPOOw9PTSX7cjez/rUX8Lhdepclmpjge2dr6NChQ1SWlNBKVg8LOjEGA7FeL/n5+bhcwRVsrVq1wmCr4PM3X6bk5x9x1FTrXZJoQkI6dMePH8/K557jxTZtQn5R8mBzQ1wcN1ZU0LNnTwoLC/Uu5zQzZszg+eefx+2wM+PGAfz0n416lySakJAO3ZNksnvwCebfiaIoDBw4kI0bNxIWFsZHzz7Eh1Mf0Lss0UTI52ohGiAqKorMzEwmT56My+WiwmNg/esvAtBl8HCSz+usc4UiWIVk6Ho8Hg4ePIjV7cYqW3wHLZOi0NJk4sjhw8TGxhIbGxuQdgoLC7FYLCTWcysmi8XCww8/DMCGDRuYOHEiBQUFRETHEhEdR3SLwK95K5qekBxeKC8vp0ePHvzhwAH+kpCgdzniLNqazcxt146bhg/nzTffDFg7t9xyC08++WSjzjFgwAC2bt1KREQEHz/3MIufuNc/xYlmJyR7uqcK5rHDUKcoSkAvQjh69ChXXXUV+/btIyMjo1HnUhSFsLAw1q9fj9frZft3efx95CUA/M/Ex8m8ZIg/ShbNQEj2dIUAcLvd7Nq1i15GI2E//8zcuXPxNmJRH4PBQGZmJllZWfS+sDvXDx3EsYJ97Fi1hI1vv8qmBf/E5bD78RWIpijkerrV1dUcPnxY7zJEPZWXl1NUVETLli39cr6qqiqKjhyhhdHINbGx7Nu9m398/z1/+ctfMPjhwoxOnTrx5JNPsm7dOkp3bqV051YUo5GMiwdjsZ6+qFJETBxhVlnzI1SEXOguXbqUiRMn6l2GqKeXX36ZTz/9lC1btvjlfO+//z5PP/QQb6WmYgD2ORx+Oe+pzGbzafW6XC46d+7M8eOn7+Z7zZTnuej6W/zevghOIRe6AFaDgf/furVsx9NEPN6qFR8eP85BP4/vqqqKwq/jsX0iI2nr8TB44EBenjmT7t27N/r8Z35fYDabWb58OR6PBwCv18s111zD56+/xNYlb9fpnJUlRY2uS+grJEPXCKRbLBjlS7SgpygKaRYLcUYjB/10ziVLllCwdStDY2J8K8vFGI20BXbu3El1dWAu61UUhS5duviOvV4v48aNq1d7Cw4Fz3KXomFCMnRFaJs1axatfvqJCS1anHa7AUgwGqk4epTjx48TFxcX0DoMBgOPPfZYnR+vqirLly8PYEVCCzJ7QYgTIg0G3khNZeodd/guehDC3yR0RcgoLS1l0KBBXHH0KNf/ztVtiqL8Og6rqrJIuQgYCV0RMlwuF9u3byfR5SK5li9RL4qMJPLQIebPn+/70ksIf5HQFSGhpqaGY2VlxBqN5/wCdURsLDF79vDYY49J6Aq/k9AVIeHtt9/m6oEDeatdO847sc+ZEHqQ0BUhQVVVvF4vBuq23saFVisPxsRw9bBhbNu2LfAFipAhU8ZEs/fJJ59Q/O23XBoVde4HnxBrNJIO5ObmUlFREbjiRMiR0BXN3rRp0zhv/35uP2Ne7rkoikK00UhNZSVVVVVE1SO0hTgbGV4Q4iysisJbqak8P2ECkyZN0rsc0UxI6Ipmq6SkhOHDh3NlRQXDGrDrhKIoGBUF1ett1JKPQpxKQlc0Ww6Hg82bN9Pa7aZ1IxY3yomIILq4mCVLlsgUMtFoErqiWXI4HFRVVhJpMGBo5MJG18fFkbR3L5MmTcLlcvmpQhGqJHRFs/Taa69xzcCBzE9LI02W8BRBREJXNEterxe3y4UJ/+yDd0F4OP8vJoYxN97I119/3fgCRciS0BXNzpo1azi2eze9I/23BU68yUSWxcKmL76grKzMb+cVoUfm6Ypm5/HHHyfz8GHurOe83HNRgHCDAafdjs1mIyIiwq/nF6FBerpC1FG4ovBWWhqv3nsvd911l97liCZKQlc0G0VFRYwaNYphNhuXRUf7/fyKomBRFNwul8xiEA0moSuaDZvNxurVq0lzu2kXwJXEzg8PJ+bYMVauXCnzdkW9hWToqoBDVfHK7gBBT1VVHF4v54o2l8uFw24nTFEI9Hajo+LjScvP57bbbsPpdAa4NdHchGToVnm9/Hn/fnbZ7XqXIupg0qFDfFxeXutjZs2axQ2DB7MwLY22Mi9XBLGQC92BAwcyZ84c3IqCXE3fNDhVlVGjRzN9+vSzPsbtduN0OLAoSqOvQKuLjLAwJsXFcfvYsTJvV9RLyIVuu3btGDJkiF8mzAvtZGRkcOmll/7ufRs3bqTq55/pquEUrgSTie5hYaz57DOKioo0a1c0fTJPVzR5999/P91KSrg3KUnTdhXAoih4XC6cTicW2QZI1EHI9XSF8BeLovBmaipzJk9m/PjxepcjmoiQ7OlGRUXx5ptv8slLL7G/oIDhDVhrVQRescvFW0eP8sS0afTu3/839x85coSHHnqIwsJCbA4Hh3WaO7vLbidavpQVdRSSoWuxWBgxYgRz586lzOHQuxxxFtVeL19WV/O/l11Gamrqb+73er1UVVVx0UUX6VDdf3UHcnJydK1BNB0hGbqieUhJSWHp0qV6lyFEvciYrhBCaCikQ/evf/0rfcaM4dWSEjxydVpQ2VBZyfrISObOnUtiYqLe5QjhNyEdugMHDiStRw++rK5GIje4/OR0km82M2rUKCL9uC6uEHoL6dA9SQU8qooqvV3dqaqKR9bFEM2YhC5Q4/Vy64ED5Mm0n6Aw8eBBPq2s1LsMIQIi5EO3d+/eTJ8+Hbui4JbeVVCo9nq56tprefTRR/UuRQi/C/kpY+np6SQnJ/PRRx9RfvgwB51O2srlnLqweb3sczrJ6dGD4cOHM2zYML1LEsLvQr6nC79eobZ8+XK+SU3lvWPH9C4nZB1yuXi0sJBZb7/Nddddp3c5QgSEhK4QQmhIQvcUEyZMoO+NN/JaaamM72psU1UVW6KiePHFF4mVtTBEMyahe4qhQ4eScfHFrK6sZI/dTpXsfxVwqqryk8PBV9XV7I2IYOzYsTIvVzRrErpnUBQFJ/BIYSG7ZTGcgFOBp44c4auaGllYXoQECd0zXHnllWzbtk0WpNbYlClTWLJkid5lCBFwIT9l7Ezh4Vhv8HgAAAWcSURBVOG0bduWqVOn8u2773L0wAEuj4nRu6xmqcjl4qOKCu5/9FEGDh0qayzU0a4vPqOi+IjeZeiupvy43iU0iITu77BYLNx+++2MWr2aHT/8QIewMM6zWDTZ8DBUFLtc5NntrKio4IWbb6Zly5Z6l9QkZGVlUVRUhG2PbIaZ1bljk3zfSOiew067nYcPH2ZBWhphIOOOjXRyfYvFx4/zWWUlBoOMcNWVoih88MEHepchGkne8bWYPXs2M2bMwKGq3FlQwHabTe+SmjwVmHjoEBuqqujatSt5eXm0aNFC77KE0IyEbi0SEhLo378/TzzxBBWKwsqKCtZUVOhdVpN1xOXizbIyCl0u/ufqq5k4cSIpKSkYjUa9SxNCMxK659ChQwcmTJhA165d2WU08nlVFb84HLL0YD2VuN3k2e18XFFBekYG1113HTfccIPeZQmhOaW2NWTLy8slWU4xZswYVq5ciVlRWJiWRriiyBjvOZx8f80sKWFdVRVGo5E9e/bIkIJo1mJjY88aDNLTrYcZM2Ywc+ZMXKrK3QUFbJMx3nPyAvcdOsSX1dX06NGDHTt2EB8fr3dZQuhGQrcekpKSuPjii5kyZQrlBgOfVVSwVhbbPqtCl4sFR49yyOVi6NVXc+edd9KuXTsZwxUhTUK3njp06MDEiRPJzs7mB6ORNRUVFDidMsZ7hjK3m+/tdv5VXk5ahw6MHDmSUaNG6V2WELqTMd1GuOWWW/joo48wAQvat8cqY7zAr+O4L5aU8EVVFSaTiR9//FGGFERIkTHdAJk2bRqvvvoqbmDSwYP8p6ZG75J051ZVJh06xNbqanr16kVubi4xchm1ED7S022kffv2sXjxYgCK166l6scfMSgK18XGEhZCV1ttqqrigNOJYjSSduutGCwW0tLSGD16tN6lCaG52nq6Erp+NGXKFJYtW8aRw4eZ3qYNbcxmrM08eFVV5YjbzZzSUnYBaWlprF27lqioKL1LE0I3EroaUVWVjRs3MmLECAAmJiVxWXS0zlUFltPr5eb9+7GrKoMHD2bJkiUyri1CnoSuhmpqajh48CAAr0ydyvoVKzAAz6akEN2MpkrNLi3lW5sNi8XC2ytXEh4RQWRkJG3atNG7NCF0V1voyipjfma1WuncuTMAQ667juQOHUBV+ey99/Da7SSYTE2293vQ6eSr6moA0ocO5fzUVEwmE5nnny+LvgtRR9LT1YDH42HQoEH89NNPJDmdPHJiDdAYozHox3y9qkqJ240KbLfZmF1WRtu2bZk5cyYDBw7UuzwhgpIMLwQBVVWZPHky8+bN8902PjGR4UG+822N18uf9u/HdeJ9EhERwb59+wgPD5exWyHOQkI3SBQWFnLs2DHf8byZM/l06VLf8XVxcQzSeeihxuvl0cOHfVvQR1itzP3oI8wnhg8MBgOdO3eWxceFqIWM6QaJ1q1b07p1a9/xgKuuIiwpyXdc8803/GvXrtOeE2M0BmwM+BeHg2/OWLRHsVgYNm4cnAjVsLAwsrt0wWSSt4oQ/iA93SDy3HPPMX/+fN9xeXk50XY7T6ekBKS9TVVVzD9+nFatWvmGCqKjo9m0aRNmszkgbQoRCmR4oYk483fx1FNP8fLLLwe0zfj4ePbu3fubnqyM1wrRcBK6TVRhYSHFxcUBbcNkMpGVlSUhK4QfSegKIYSGZJUxIYQIEhK6QgihIQldIYTQkISuEEJoSEJXCCE0JKErhBAaktAVQggNSegKIYSGJHSFEEJDErpCCKEhCV0hhNCQhK4QQmhIQlcIITQkoSuEEBqS0BVCCA1J6AohhIYkdIUQQkMSukIIoSEJXSGE0JCErhBCaEhCVwghNFTrbsBCCCH8S3q6QgihIQldIYTQkISuEEJoSEJXCCE0JKErhBAaktAVQggN/R/1t9l/AtGw/wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "\n", "==== Ranking 2 [score: 0.667] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVhVdeIG8PecuwKX/YqCiqmIigsqYqKIEea+NNlK25Q1NY5LajqV1WhpZVO2b/abJsvIpXTKNdcWddQxRSHFbYxSUUHhynLhLuf8/lDvaAEK3nvOXd7P8/g8woF7XvHe9x6+53u+R5BlGUREpAxR7QBERIGEpUtEpCCWLhGRgli6REQKYukSESmIpUtEpCBtfRstFgvnkxERNVB4eLhQ1zYe6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkoHqXdvQkWZYhSZLrY41G4/F9Op3Oyz4WRRGCUOcKbEREbqfaka4sy+jXrx8SExMxcuRIRfZ55513IjEx0fVnx44diuyXiOgitx/p5uTk/K7MWrVqhUmTJl32OUEQMHHiRFRVVSEqKsrdMWr1xz/+EVFRUVi0aBEAwG63K7JfIqKL3F66mzdvRk5ODgAgOTkZx48fh9lsRr9+/ZCcnIyioiKcOnUKANCmTRsAQHBw8GWPYbVakZ+fDwBo3rw5oqKikJeXBwCIjY1FixYt6s1gs9mwd+9eyPLlN77o378/zpw54yrdAwcOwGAwQKvVIjk5GaJ4/sDf6XQiNzfXNfwhiiK6d++OI0eOoKys7Hf7E0UR3bp1cw2RyLKM3NxcOBwOAOffYJKTk6HT6Vzb9+zZ87vSb9myJZo1a1bvv42IfJvw22K6VGNu1zN27Fjk5ORAFEUUFBRg9uzZmD9/PjQaDfbt24e5c+di3rx5AOAqxU6dOmHLli2uzxUUFCAtLQ2CIOCJJ57ALbfcgtTUVAiCgClTpuDpp5+uc/+yLOPEiRPo3Lmz6/EvjtuuWbMGBw4cwIQJEy77fGhoKA4dOgS9Xg8AKC8vR0JCgqsUjUYjDh8+jIcffhirV6/+3ePqdDocPHgQ4eHhAM6Xfvv27WGxWFxfV1BQgCZNmgA4X+pJSUk4ffr0ZY8ze/ZsjB07tqE/ciLyMqrcrkeSJPTp0weLFy9GcnIy9u3bB7PZjOnTp6OgoAD79+9HTEzM777v1VdfxfDhwwEAa9euhdPpxODBgwEAq1atwmOPPVbvfufPn4/+/fu7ijE7OxsFBQUoKChA9+7dL/vaTz75BC+99BLKy8vRpUsXbN26FatWrUJKSgpsNhvef/99vPnmm7BarejWrRtGjx6Nd9991/X9r7/+Oj788EPYbDakpqZi+fLl2LZtG7p06QKLxYIXXngBCxYscP0sFi1ahD179iApKQklJSUAzr/hXMx3//33N/4HTkQ+wWOzFwRBwNixY/HNN9/g6NGjmDt3LqZPn46wsDCEhYVBkqRaZyxUVFTgzJkzAIDo6GgAuOxjk8lU735TUlLwyCOPYPbs2QCA3NxcvPrqqwDwu6PIyMhIVFZWQpZlnD592nWEOn78eDz33HNYtmyZa8ZDcXExjEbjZUMh4eHhrjzFxcWorq6GzWZzHcGuWbPGdfRbUlICq9WK2NhYTJ48GbNnz0Z5eTlOnTrlynet+vfvj2HDhrnlsch3rF27Frm5uZg6dSpn4/gAj5bufffdh19++QXbt2/HvHnzMHnyZISFhXlqlwCALl26IDY2FqtWrYIsyygpKXENZ4wePfqK3x8dHY2UlBQIgoCDBw9eNq2toY4ePYqoqCh069YNAFBVVYUTJ06gV69ervHdS/MBgF4Q0PLCtob41W6HxWJBXFxcrdsTExMREhLSiH8FebudO3fiyy+/xNSpU9WOQlfBo/N0Lx0vvvgOLMuy68+laiu32r7uavYZFRWFDRs2AAD+8Y9/NOjJuHHjRowZMwYA8OKLL6Kmpgb33Xdfnfu6dHz30j+yLGPatGnIzs52ff3kyZMxffr0evffXKfDa1c4UVib8ceOYdGiRa6ThL+1YcOGy4ZXLuYk33bp6+O35xrIO3msdCVJQnp6OsrLy9G1a1csXLgQZrMZs2fPxmeffQYArlkMBw4cQKdOnQAADz30EJYvX44RI0Zg8ODBsFqtDdrvxXHaiyorKxv9b3j00Ufr3T5p0iSIogidToctW7agRYsW0Gg02LNnD9LT0zF9+nTXMAcA14m1utwZEYGhF4YjGur52Fg4a3mDKnU6Mfn4cdxxxx2uo2vg/NS+345xk2+RZRmDBg1CdMoNGPTM2+jUqRNWrlzpmhVE3sntpTt69GhXgV7UtGlT16+9GRkZiIyMrPP7e/fujdatW7vKav369di0adNV779bt24YN25crdvi4+MRGhrqeuzWrVu73ggAICEhAdXV1ZcV5aWSkpJQUFDg+njIkCHo3LkzRFHEddddB71ejyNHjuDDDz9ETU0NampqYKysxLCLQypG4/k/degcFISIRl6ZV9f3BYsixkRFQZYkoKYG1bKMnNJSvPXWW2jWrBmCg4Px5JNPKnJFILlfcXExwmQgKDwSRUVFrmmK5L3cXrpZWVnIysqqc3tGRgYyMjLq3H7y5EmcOHEC/fv3BwDXVK527drBYDBccf/JyclITk6uc3tsbCySkpIu+1yHDh0u+7hdu3a1fm9hYSEKCwtdHyclJWHYsGGIj49HQUEBJEnCT7t2Yc1HHyEWAPR6tNXrcXNExBVze0qQKGLUJfsvdzrx78pK7FqxAgBgCgvDiBEjoNFoEBERccU50OQdqqurcfjwYYTHtYIpKgYavR7N2iXhaOEvCA8PR9OmTdWOSHVw+zzdazVjxgy8/vrrl32ubdu22Llzp+pjVdnZ2Vi1atVln8vKykJOTg4SEhLOD6UYjZhVx8ksb3TO6cR9hYWQANx5xx147/33XdvU/nlT3QoKCtC7d288/tUORLds7fr8q39Iw/23jsIzzzyjYjqqb56uagve1OWxxx7Dgw8+eNnndDqdVxTAG2+8cdl48ZQpU/DDDz/g+p498VJEBAwREdB5Qc6GMIki5rVsCQDYs3UrunbtCgCYN28e0tLS1IxG5Je87kjX2x04cACff/45AKDiu+9gKyyECGBEeLjPFe5vFdps+E9VFQAg9KaboIuNhcFgwNSpU6HVet37c8Bav349Nmzeiv86g3H96PsRFPa/4aMdSz9B8LlTSIwOwYQJE7ziYCUQ1Xeky9JtgGPHjuHbb77BnCeeAACMiY5Gj9+sG+Ev3iouRkF1NYzBwfho+XK0btPmihemkDJmzJiBnK9WYdKXW2rdvvH/5uLgyhzk5uaydFXiU8ML3ubSN6Vp06ah+Ntv8c6FX8f92fgL60RUSRJu7N8fCxYuxMCBAwFwrFdNVzNv/dL/HVmW+f/lZXikewUbNmzA448/DgAY7XSiM4CoAPpVW5JlnHY4sFgUcUQUYTQasWnTJhjrmfpGniHLMoYNG4bolEwkDbkVEc1qn2liPVeG4v8ewJrnx2PJkiVISEhQOCmpsuCNP1i0aBG+XbQIaWVlSCsrQxtJCqjCBQBRENBMp0Oy1Yq0sjL0KCnBqy+/jIMHD6odLSD9+uuvsEGos3ABICgsAqYmzXD06FHYbDYF09HVCKwGuUqSJOHo0aNYmZMDx+7deNRsVjuS6vpfGM+tkSQ8/vbbuK5lS4SGhiI2NlblZIGhpqYGv/76K0Jj4i47cVYXjVaH6Pg2OHHyFGJiYmDmc9hrcHihFhfX0/1rZCRSgoI4JvYbsixj9qlTiMrIwMKFC9WOExDy8/ORnp6OaSt2ITLu6s4pyLKMN27PwO1Db8Lzzz/v4YR0KQ4vNMDatWsxKDMTrzRpgk5GIwu3FoIg4FGzGZ3370d6ejqqLkwzI+/C56530syYMaPOjTU1NXVv9ENLlizBjytXIiw/H2khIdCLfE+qS7AoQuNwoKqsDIdtNpj5K6zHbNq0CWs2foegDj3Rtlc/6AxXfxJTcjohO5349eA+15Kl5HlGo3FmXdvYKjg/hltYWIilH3+MI6tX487ISGj45LyiFno9bgsNxYK5c7Fj82bXqnHkXuvWrcPnS/+FgX95CkGhDVuFrs9dD+OMEIQ333zTQ+mooVi6OD+G27NnT/Q+cgQPX7hbBV0dHYC3WrTAwmef5f3diK5CwJfu6tWrMWrQIPy9aVN04Bhug11cDP3B6Gj0OHwYGRkZqKioUDuWX5BlGSNHjsQJgxl3//2fjX6clJF3YfiMd9CnT19O9fMCAV26X375JbZ/9RXanzyJVjodgjiG22hNtFq0djqRcOwYPnzvPRw4cEDtSD5PlmUcOnQINkELc3zjFyYPiYxGRPPrsH//PlRXV7sxITVGQM7TdTqdOHnyJN577z2EFhRgUi13JaaGa6HX426dDve88AKCw8IQFRXluu08NYzNZsOpU6cQEtUEhuBrX/NC1GgQ3rQ5zpSWoqysDBEqrvEc6AJynu7Zs2eRmJgIh8OBTJOJpetGdlnGPT//DKssIyMjA19//bXakXzS7t27ceONN+Kvq3IRFhN7zcNeF1/nb2XfiJtv7IcXX3zRHTGpDpyne4kVK1Zg+PDhcDgcGGc24+56bh1EDacF8FJcHFKCgvDjjz8iMzMT586dUzuWT5JlAG66gajrRqR+eRjlWwKudM+ePYtD+/djeFgYugQFIaYRtzunugmCgNYGAzJMJnQFkJeXB6fTqXYsn/LDDz9gzcbv0Df7T9AHhbj1sbsNHY1ybQjmz5/f4Dttk3sEVOmeOXMGFosFRkHAmOhoxLJwPSYzNBRDw8IgyzJOnTrFq9YaYPny5VjwxTIMf3wWgkLD3PrYGfeNQ6k+HC+++CJLVyUBVbrZ2dm8d5TCnE4n0tLSsGzZMrWjEHmFgCjd0tJSZGVlIT8/H9cHB+O52NjA+IerrJ3BgJfj4mAEMGfOHEyYMEHtSF5NkiSMHj0aJaY43PbcOx7bT/LgWzDib29jwIABKCgo8Nh+qHYB0T12ux27d+9GV1lG35AQtDUYeBGEAoJFEYkGAwaFhcFZVIRDhw6pHcnr5eXlwa7Vo2nb9h7bR2h0DMyt22PX7t0c9lGB35eu1WrFmTNnAAA3R0TghtBQlRMFFvHC1WodDAbY7XYUFxdDkiS1Y3mdiz+boPBI6AxBHt+fIIowRZlxrrwc5eXlHt8f/Y/fl25OTg769u3LF7oX2LlzJ5KSknD27Fm1o3idPXv2ICmpE7LfWoKeN9/t8f2ZopvgyW/yMH76TMycWeeCWOQBfl+6siwjGOfnjrbS69WOE7Bui4zEo2Yzp4/VQZZlOCUnBFFUZOhLEASIogaSJHEWg8L8unRXrlyJ7du3QwOgg8GAYK6toJo4nQ6tdDrIsozFixdz4ZVLbNu2Dd9v+w9Sb74bOqPnhxYulXTDEFQbw7F48WKWr0L8uoVeeeUV/OuLL2DSaNSOQgA0ggCTIGD6U09hy5YtasfxGgsXLsTHi77ELc+8BqPJvfNyr+TGhyajIrQppk+fziE4hfh16QJAlsmEt1u0AOcqqK+9wYCPW7VCKH/joADm989+QRCgcdP163RtBEHw/ydcAzidTmRnZ6PC3BqjnnxZtRxJNwzB8GfewMhRo7Bv3z7VcgQKv3wNWK1WLFy4EG2qq5FoMKgdhy4hAEg3mXAiNxfr1q1TO47qtm/fDochBM07dlUtQ1hMLOI6dsPWLVs5fUwBflm6ZWVl+POf/4w+FguyOC/Xq4gX7iS8d+lSzJkzR+04qnE4HLBYLNAHm6DVecGsGlGAMTQMVdZqXjDhYQG5iDmR2nbu3ImRo0Zhytc7YYpWf6H3kIhoTF/3E6Y8OhoZyR3w+uuvqx3Jb/nlkS6Rt5MkCXa7HRqtFqKo/uwaQRCg0enhcDo5l9rD/K50Dx06hPWrVyM9JIRTxbxYO4MB8VYrli5dGnD37dq5cyd+3PsTug68GRq9d51zSLg+A46QSCxfvpxTyDzE727X8+677+KVZ57BP+PjOWPByx2orsa0oiIUFBSgadOmasdRzNixY7F13xE88tEKtaPU6j/LFmDT28/h0KFD0Go5AtkYvF0PEZGXYOkSKcThcOCBBx6AvXl7DH7sb2rHqVO7tEwMnz4X2XffjZ9++kntOH7Hr0r3u+++45PEB61duxZHjx5VO4bHybKMTZs2QQqJRKuuqWrHqVNEs+Zo1aMP1q9fj9LSUrXj+B2/GrB5+umnsS8vD004DgWbJMHbT4PYZBmyLGP8+PGYNWsWHnzwQbUjeZTD4YDWYIToAyd4BUGAzhgEm92OmpoaGHiRkdv41Ym0fv36Ib6wEA9ERUEX4Jf+Pn78OH622dSOUS9JluG48HetVuv3J21ErRYTv/w3TFFmiBrv/rfKsgxHTTUWTLoH1ye2wltvvaV2JJ9S34k07/6fbwQNAD0XVIFdlpEcFISBvCLPK/xss+Hzc+XQ6o1eX7jA/4507Q4HbF7+5u1rvP9/nxotVqvF9SEhascIeEdqalAcbEKHbmnQ6HzrJdcqORUQ7Vi3bh2ysrIg8oDmmvEnSORh/7JYsN4ci3tfnQ9DsEntOA0yaNzTkOKTMGbMGDgcjit/A10RS5eISEEsXSIPccoyXjt9GhE334usP01VO06jte7RB8OeeBkP/+kR5Ofnqx3H57F0iTxEArCtshLGDl3RJqWP2nEaLTKuJdr1HYDlK1aguLhY7Tg+j6VL5AGSLMMhy9Do9BD85OSTVqeDw+nk2O418o9nA5GX2Wu1YsyJIoz7agc6Z41QO841M5rC8OQ3+Zj55geYOHGi2nF8GkuXyAMkAFZZgiHEBI1Op3acayYIAoymUFTX2GC1WtWO49NYukRuVmiz4WRQCNr0TPeJCyEaIrZ9ZwhhUdiyZQvX220kli6Rmy0sLcWGZvF48J3F0AcFqx3HrYY+NgO69r1wxx138Eq1RmLpEhEpyL9+9yFSkUOWMa+kBNEjs9E+faDacchL8UiXyE0kWca3FRUwdeuNdmk3qB3HY0Iio9G8Uw/s+M9/uN5uI7B0idxAlmVIAARRA/j5kqIJvTJw1yvz8YfRo7Fjxw614/gcDi8QucFuqxVvWiyY8NUOmMwxaschL8YjXSI3cMgyzjklBEVEQetlt1X3BK3egFF/nYOvN23GRx99pHYcn8LSJbpGx202nA0KQYvOPQJmvVmNTofUP9yDnfn7sXHjRrXj+JTAeIYQedA/z57Fxvh2eOT/vobOGKR2HEUJggAIAiRJQn23/qL/YekSUaPd9vy7CEsbiu7du6OmpkbtOD6BJ9KIGskuy/jk7FnEDL8DHfoMUDuOKoLDIiDojTh9+jSPdK+S3x3pljmdOFpTwycAeZxTlrH63DlE9s5Ex4zAvRgiKDQczRI7Y29eHiwWi9pxvJ7fle73lZWYefKk2jHIz7ne1P18Tu7VaJvaD3989wsMGzESmzdvVjuO1/Or0l26dCmeeOIJtWNQANhltWLc6WJMXLYN7dIy1Y5DPsSvSrdJkyYICwtTOwYFAJsso1SSENqkGXQGo9pxVKfR6TBkwrNYt+1HfPrpp2rH8Wp+VbpESjhpt8NiCEKzhI4QBL6EAECj1aHPXQ9j+959WLlypdpxvJrfPmNkgCfTyCM+KCnBd207Yewn30Br8P+rzxpDlmW+/urgd6V777334su1a/GnY8fwMxdZJlLULc+8huiMm5Gamsrb+tTB7+bphoaGollcHIodDjj4TktuZJdlLCotRbMhoxHRJ0vtOF7JFGWGJigEx44d45FuHfzuSJfIU+yyjGVlZTD3H4rONw5XO47X0gebYG6VgEOHD+PcuXNqx/E6fl+6fLcld+C83KvXNjUdf/p4NQYOHsLFcGrhl6XbpEkT7Nq1C1+ZzficK9uTG+ysqsLUM2cxbskPaNOzr9pxyIf5ZelqtVq0bt0a5RoNLLxNNLmBVZZxyulEVPPr/O4Ov54garTIHDMZW/bsw8KFC9WO41X8snQvVSlJOGm3c5iBGq3E4UCVwYjolq05vHCVNFotbnhwIv6dtx9LlixRO45X8fvS/b6iAlOPHwcrlxrrzeJibO7YA+NzNkKr16sdh3ycX5fu/PnzMW3aNLVjkJ8QeJTbYMMfn43YAbejT58+qKioUDuOV/Dr0o2Pj0dMTAxqZBlLyspQ4nCoHYl8iE2S8EVpKZoNGIlOmUPVjuOTwmNioQuNwOHDhyHx/AoAPy9d4PzFEubmzfFZaSmKWbrUADZZxmelpYgbciu6DvqD2nF8ls5gRERsCxw7doxHuwiA0r399tuxcePGgLlhIJG3ua5HGsYv+h6ZNw3CqlWr1I6jOr+7DPi3Lh2Hm3v6NDJNJmRHRamYiHzBjspKfF5jw9icjYiOb6N2HJ92/jXI8fCLAuLwLzg4GOPGjYNgNuNHqxUrLBZInEJG9aiQJBx3ONG0bXsYgkPUjuPzRI2IPnc9jN2HfsbSpUvVjqOqgChdk8mE5557DikpKSgOCsLnpaWcQkZ1KnM6UaM3ICwmlvNy3UTUaDFw7JPYmn8AH3/8sdpxVBUQpXvRggULMHPmTLVjkJd7+dQpbOuWhseW/ABR4/cjcKSwgCpdQRAgCAIqJQmTjx/H4ZoatSOR1xJczxdyn0F/eQrxQ7KRmZkZsCuQBVTpAkDHjh1x/wMP4Ge7HRvKy7GrqkrtSOQlbJKE5RYLYm8YikTebNIjIuPiYYyMQV5eHpxOp9pxVBFwvzulpqYiISEBa9euxbriYlRJEnoEcwETAqplGf84cwZjbrmPK4l5kEavR6i5KU4XF8NgMCA4wF5/AXekCwARERHIy8tDamqq2lGIAk6r5F6Y/NUOZN40CMuWLVM7juICsnQFQYAoinjttddw08SJmH7iBG/tE+C2VVbihfJKPPLxajTv2FXtOH7t4li5LAfmZcEBWboXJSYmoldmJnrdfjvWVlRwbYYAds7pxM9OJ5ondYMhJFTtOH5PEET0HHU39v9ShBUrVqgdR1EBXboA0LNnT/xt1ix8pdXiv5KESi7KEXDKnU7YdXoER0ZzWq5CRI0GQyfNxL/3HcY777yjdhxFBXzpAucXxcnPz8eWhAS8V1ysdhxS2KyTJ7GrVyYmfbEZgqhROw75OZYuzo8xaTQavPjSSxgyaRKeKSqCnWO8AUMGIAuAKGo4L1dhN4yZhLYj7sPgwYNhsVjUjqOIgJsyVp8OHTqgsrIS+//7X2xavx7djUY00fJH5K9qJAnfV1Qgtu8ARHfvrXacgGSOb4PyM6exY8cO2O12teMogo3yGykpKWjfvj169uyJUKcTJlFEEJeF9EtWWcbbJSV45N6xaJXcS+04AUuj0SI4PBIWiwUmkwlGo1HtSB7FNqlFSEgI8vPz8WNSEt7gGC+RR7XskoK/rt6DzIFDAuLOwTzSrYUgCNBqtfjbjBn4Yc0aPPX3vwMA7omKQpKfvwsHim2VlVgtAWM+WIambTuoHSegCYIAUaOFw+kMiLt2s3Tr0alTJ9hsNuw+eBAAUJiXB0dpKboGBamcjK7VWYcDB2UBt6akQeSMBfUJAroOHIWjp89i3bp1uOmmm9RO5DEcXriC7t2744MPPsAHH3yA4qQkrLHbUeF0osLp5FVsPsoqSXBqdTDyIgivIYoiRj0xB9sOFGLOnDlqx/Eolm4DvP/++3j0tddwb2Eh7i0sxI7KSrUjUSM8W1SE/PRBmPTFFggCXwKkLD7jGkCr1aJ3Whq+WLYMXyxbhl+SkvBsURFmFhWhmley+QwnAEkUodHpOC/Xy/TNfgTtR/0Rt9xyC0pLS9WO4xEc020gs9mMzMzza63u378f1VFRECQJ27Zvh+h0IkKj4Zivl6qRJOyoqkKz1H5o2jFZ7ThUi5g2iaiuPIecTZtgs9nUjuMRLN1rMHbsWACAzWZD165dUVFVhQ46HRIMBtfXaAAYOM/XK1RIEv5++jT+8uo0tOjUXe04VAdR1MAQYkJlVRVsNhv0er3akdyKpesGOp0Oe/bsgSzL2Lp5M+649VbXto5GI2bFxamYjsi3NO+YjKfW/oQBQ9Pw1JRJeOihh9SO5FYsXTcQBAGGC0e3yd27Y8GiRQCA9957Dzu//x4zi4pcX9tCr8eY6GhVcgaybZWV2AgR97/xGaLj26gdh+ohiCK0egNsNrtf3tKHpetm0dHRGDRoEACgsLAQJpPJtW3v3r0oKiqq8wKLVno94nS6Ru0312qF9ZKTeVyi8nKnHA78JGoxsu8ACBzu8Qnt0wfgRFkFvv/+e2RkZKgdx22E+q4AsVgsnIjqRtOmTcOnn35a67bq6mrcGxmJ4WFhjXrsKcePo0iWobtQ2tXV1RgRGoqHzOZG5/UXNknC8nPnsFTUYcq/tnPGgg9Z9drfUHM4F99++63aURokPDy8zicZS1dBNput1l+XZFlGjx49UHzyJDSNLAS7LOPRP/8Zzz77LABgwIABaPvrryxdAI8fP44jNTVwAtAaeBm3L5EcdnTt0sWvSpfDCwqq6yysLMt45513UF1dfU2P36ZNGwRdmK4m8ldol/ujolDlQ8Mt35SX40zTppg1a5baUbxCeHi42hHciqXrBQRBQFZWltox/FYXH5s3nWu1whEejmHDhqkdhTyAh0NERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERApi6RIRKYilS0SkIJYuEZGCWLpERArSqh2APOe7igrkV1erHYMaqMThQBu1Q5DHsHT91G233Yai9HS1Y1AjNW/eXO0I5CGCLMt1brRYLHVvJCKiWoWHhwt1beOYLoqLUHUAAACUSURBVBGRgli6REQKYukSESmIpUtEpCCWLhGRgli6REQKYukSESmIpUtEpCCWLhGRgli6REQKYukSESmIpUtEpCCWLhGRgli6REQKYukSESmIpUtEpCCWLhGRgli6REQKYukSESmIpUtEpKB6b0xJRETuxSNdIiIFsXSJiBTE0iUiUhBLl4hIQSxdIiIFsXSJiBT0/9Rnj3bg42lWAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "\n", "==== Ranking 3 [score: 0.167] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXQUVd4+8Keq13S601kgKyGsAokkEV/4CbI4GDKAIM6oozgqHlBxARQZ1BHFgSPLcVxwZY6IwnvQEURFCAKOgOwYwr6EJQghhJCELJ29t6rfH3npMbJl6a7qTp7POZxD9VL17er007dv3aoryLIMIiJShqh2AUREbQlDl4hIQQxdIiIFMXSJiBTE0CUiUhBDl4hIQdrr3Wmz2TiejIioiaxWq3Ct+9jSJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhB1720IzXkdrsbLIuiCEG45hXciIiuwJZuEzzzzDO46aabPP/WrVundklEFGB80tI9ffo0PvjgA89yQkICpk6d6otNKeovf/kLzGYzFi9eDABwOBwqV0REgcbroXvmzBls3boVS5YsQUpKCvLz89GuXTsMGjQIKSkpKCgoQGFhIQRBQHJyMgoLC3Hx4kXP8zt06IDQ0FAcOXLEc5vRaETv3r09y3a7HYcOHfIs6/V6JCcn49ixY6ipqfEsZ2dno7q6GjqdDikpKY3uCnC73Thw4AAkSWpwe58+feBwODyhe/r0aezZsweiKCI1NRUajQYAIMsyDhw4AJfLBQAQBMGzL4qLi6/Y3uV9odfrPc8/fPgw7Ha75zG9e/eG0Wj0LB8+fBh1dXUN1hMdHY34+PhGvUYiUocgy9eekac50/U888wz+PLLLyGKIk6ePIk5c+ZgyZIlEEURx44dwzvvvINPPvkEAHDkyBF8+OGHWLhwIQRBgCzLmDlzJkaMGIH+/ft7buvSpQuysrI8oXn27Fnccsstnvvj4uJw6NAh/OEPf8ChQ4cQHR2NI0eOID09Hfv370dERASys7Oh1WpvGLyyLKOyshLdunXztGQvP+eLL76AIAgYO3Zsg9t1Oh1OnjwJq9UKoL4F3KNHD9hsNs/jjh8/jrlz52Lp0qW4vM9/W8uRI0cQGxsLAJAkCSkpKTh//rznNe7duxddunTx1NivXz/k5OQ0WM/kyZMxe/bspr5lRORlqk7XM2vWLBw/fhzHjh1Du3btMGPGDKxcuRKyLGPIkCFYunQpevTogezsbMTGxuKdd97BqFGjAAA//vgjnn32WeTm5qJnz544c+YM/vWvfyEtLQ0AsHr1akyfPh0FBQXo1asX5s6di7///e8oKipCYmIiZs6ciZkzZ6KkpARJSUnYv3//Dev94YcfcOutt3oCd9iwYTh+/DiOHz+OoUOHNnjsggULsGjRIjgcDvTt2xdr1qzB7t270bt3b9hsNsydOxfLli2DJEkYMGAAEhMTsXz5cs/zf78vlixZghMnTqBXr164cOECpk2bhoyMDABAeno6Pvroowb7Aqj/ZXC5vunTp7f8DSMin/LZ6AVZljF79mzPT2JBEDBjxgyEhIQgPDwcAHDp0iWMGjUK99xzD6KiovDqq69ixYoV+PnnnwEAixcvxsmTJ+F2u1FUVASXy4Xq6mqUlJQAAJYuXYq8vDxIkoSioiKEhITAYrFcsSzLMoqKiuB0Om9Yd48ePTwtRpfLhVOnTuHtt98GAIwbN67BY61WK8xmMwCguLgYdXV16NixI6ZOnYrZs2dj/fr1ntbvpUuXoNPpEBYW5nm+xWJpsC9qamrgcrlQVFQEANi2bRvOnTsHACgpKUF1dTXCwsIwbdo0vPXWWyguLobNZvPUN2TIENx1111NfKeISEk+Dd19+/ahoKAAJSUlEAQBL7zwAkJCQho8rn///rjvvvsAAA899BBOnjzpCd2DBw/CYDAgNTUVABr0aQL1P8mNRqPnfpPJ1OK6u3XrhscffxwZGRlwOBwoLy/3dIcMGTIEonj9HwchISHo27cvtFotzpw54+mnbY68vDw4HA7P64uOjobVasXEiROxY8cOnDt3Dg6Hw1OfzWZDQkICkpKSOJSNyE95PXQvf9hFUcR3333n6dO9TJZl/LYf+fLy5ef9NiyWLVuGzp07X7HuyxYvXoxevXpd8/7mkGUZRqMR69evBwBkZGTg0UcfbfTz9+3bhzFjxgAA5s2bhy5dumDkyJHX3NbV+tQv9+M+8cQTeP7556+4XZZlzz7Nzc3FLbfcAgBYvnw5duzYgcOHDze6XiJSltdDd86cORg+fDgeeeQRDBw4EJWVlVfc/9sQfvPNN/HTTz9h1apVAICpU6di6NChGD16NIYPHw6t9r8lrl27FhMnTsTgwYORnp6OMWPGQKfTee7/9ttvW1z/unXrMG3aNM/y70cINMWMGTMa1P97c+fO9bSEN2/ejO7du8NgMODo0aNIS0vDu+++i0WLFnkeP3nyZIwcORIjRozw3PbbEzbGjx+Pl156qdn1EpHveT10Q0NDceutt2LOnDkNbhcEAWazGYMHD27QrwkAERERnv+HhIQgKSnpiucDQFhYGCwWC3r27HnV+9u1a4fbb7/dc19cXBx0Op1nuWPHjjesv3v37pg0adJV70tMTAQAz/puvvlm6PV6z3JqaipMJtNVawOAfv36oaamxrM8ePBg9O/f37Pdy/3DMTExePnll6/4wurbty9CQ0OvWd8tt9yCqKioG75GIlKP14eM0bXl5+dj+/btmDhxIgDg+eefx8MPP4xu3bqpXBkRedP1howxdBX0/PPPN+haAepbx5cPHBJR68DQ9ROlpaWoqqpqcJter0d0dLRKFRGRLzB0iYgUpOoZaURE9F8MXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlIQQ5eISEEMXSIiBflsNmAi8p6cnByMHz++Wc998MEH8cwzz3i5Imouhi6RHyoqKsLKlSs9yxcvXsShQ4dg6WuBYGj8rNdV+6qwbt26BrclJydj4MCBXquVmoahS+QHHA4HSkpKPMtHs4/itfmvNXiMJkSDsPQwaEMb/7F1FDiw8+BO7Dy403PbIw8+gq5du3qWTSYTrFZrC6qnpuDMEUR+ICsrC8OGDfMs6+P0iJ0Se9XHCkLjW7pX+3xXbK9A6ZpSz/KECRPw1ltvNaFauhFO10Pkx9555x189dVXOHnyJKInREMMFiHqReij9D7ZnqvSBVe5CwBQ/FUxrJIVycnJWLlyJUSRx9a94Xqhy+4FIhVcvHgRGRkZAICtOVtxMeIiQgaEwNjJCNHo2+DTWrTQWuo/+pb/scBR5kC2lI3FixdDEAT06NEDgwYN8mkNbRlbukQKq6iowPbt2/HQQw9BDBYRMSoClv+xqFqTu8qN8++ch7vajfvvvR/z5s1DREREk7oy6L/YvUDkRyZNmoQvvvgCMmR0fK0jNGaN6uEmyzIgAxc+vgB7rh0RERE4ceIEtFr+GG4Ohi6Ryvbv34+XXnoJAFDauRQ17WoAAIZ4AwSN/7QmHQUOSHYJoktE7IFYQAbS0tLw4osvql1aQGGfLpGKtm3bhg0bNiAzMxPmPmaERNT33fojfUz9wTvJLiH7QDZqsmsgyzI6duyI+++/HxqNRuUKAx9bukQ+IssyysvL8dxzz2F1xmpoTBp0eLEDNEGBE1wXP7+I2pxamA1mZGVlITw8HDqdTu2y/B67F4hU4HA4cNNNN8Fms8HQxYCYJ2IAsWnjbNUmSzKq9laheEUxNBoNvvnmG9xxxx1ql+X3rhe6HJRH5COyLMPtdsM61IqIuyMgaISAClwAEEQBQT2DEDU+Cm7ZfdWTLahp2KdL5AMFBQXYuGkjDEkG6HvqYYg1qF1Ss2ktWohdRZj7mLFt7zYYDAYMGDBA7bICFlu6RF5WU1ODzMxMTJoyCea7zH570KwpRL2IyAcisXD5QixcuBA2m42t3mZi6BJ52eTJk5t9GcZAkJGRgdTUVDgcDrVLCUjsXvAjn332GVavXt2s57733ntISEjwckXUHC6XC5oYDSJHRkLQBVYf7o2EjwxH9eFqODOdbOk2E0NXZWVlZdi0aRMAYP369di6eyuCegQ1+vmyQ0ZNdg2+//57xMXFISgoCCNGjAi4AzatgdvtRkZGBkp0JQjuHYyg7o1/HwOFoYMBskuGvdKO71d/j8GDBiMmJkbtsgIKh4ypyG63IysrC3fddRcEvQAIgD5aj7hJcY1eh7PUifPvnAcAyC4ZMZExyMzMhMlk4kB2BblcLpSXlyM5ORnmP5thTjGrXZJPSXUSzs46i6WLlyI9PR1BQa3vC6YlOGTMT82aNQv33HMPAKDDCx3Q6fVOiH3q6tdQvRZtmBadXu+ETq93Qlh6GAoKCtCtWzdkZ2f7omS6ho0bNyIxMRE1NTVql6KoCRMmcCqgJmLoqkCSJDz55JNYs2YN5BAZ0eOjoQnRQNAJELRN6xYQBKH+eToB5hQz2j/QHna7HX/729+wZMkS37wAuoIkSXC4HYh8JBLGzoE/WuFGBJ2A6HHR0ERr4HQ61S4noLBPV2FlZWXYunUrNmzYgFpTLYJvDoapl8kr69ZF6CAaRZhuNiHzQCasVivi4uKQlpbGPl4FCIIAUw8TREPrb8sIGgGmnibYdtjULiXgtP6/Dj/icDhw9OhRjBs3DhU1FQjpH4KIURFe3YYmWIPocdHQReqw4acNmDRpEmpqauB2u726HbU5nU7U1taqXQaA+r55u9Ne3y/fxghaAZIgoaamhqMZGokH0hQ0ffp0LF26FA6HA/F/j4fWqvXZZf1kpwzbDhtK15bCaDRi/fr1SE1N9cm21PDmm29i1apV2LFjh+qt+Mceewybf92MsAfDIGgD71TflpBdMuqO1aEuow7Hjh3jAbX/w0s7+gmn0wk5REbUqChoLb4LXKC+zy24dzA0wRoUryiGJEk+25aSZFnG1KlTsX37dr9pWTkcDjhdToi6tvfD8XJLt66uTu1SAkbb+ytRgSzL2LhxI/Ly8iCaRAQnBSsyaF4XoUPQTfUtj507d+Lo0aM+36YStmzZglxbLqQYCevXr0dVVZXaJRE1GkNXAZdHK2zcvFH5WQKE+lbvq6++is8++0zZbXuZJNW3qGRRhvlWM1wDXBg7diwuXLigSj2yLKOurq7V9Zc3F/dF4zB0FRQxOgLRT0Qruk2NRYOE1xOgj/PNdN5KysnJQdeuXeG+x43QIaFql4Pa2lokJSXhp59+UrsU1XFfNB5DV0GCVlC8308QBIgGsVUc3JGk+qPk0NbvS02IBpEPR+LVea9i5cqVqtRUXV0N8yAzQu9Q/0tALYaOBrQf2x419hq4XC61y/F7DF0KCDk5Ofhl3y8I6hHkGZolGkSYU8zYvH2zqmfgGTsZ28QJEdeitWoRfHNwq/hiVwJHL1BAWLZsGT768iPET4+/4j5BI0CSJTgcDuj1gd+NQq0bW7oU8GInxeLL41/ijjvu8JthZETXwpYu+TVZlvHyyy8jqzwL4XeFX/UxokGEQ3Kgurpa4eqImo4tXfJ7P/zwA85UnEFwYvA1H6MN00KOkrF161aGL/k1hi75LVmW64+GC6j/dx2WPhbIQ2WMGTMGubm5itRH1BwMXfJbx48fR/fu3YH7gZCBIWqXQ+QVDF3yW263G+Xl5YABjRrfrDFr0P4v7TH/w/n47rvvFKiQqOkYuuSXzp49i0PZh2DsbGz0hd1FgwhLXwvWbVqHffv2+bhCoubh6AUlyYAsyRBE5QaRy7Jcv10E1lCqTz75BJ9+9yk6vNChyc8VBAEyZLjdbohi6zgbj1oPtnQVVJJRgouLLyq6TXelG7mzc+G44FB0u2qKeSoG3+R+gyFDhqhdCtEVGLoKEAQBb7zxBm679TZIdQpf11YGpGoJU5+bivvuu0/ZbTeDLMt4/fXXkVWWhdC05l3PQGPSwO62o6yszMvVXV3FzgrYdrbdaWvs+XaUrCqBLAXWrym1MHQVIIoixo4dix49ekCyS6g7VwfZ7fs/UFeFC/Z8OwBg9OjR6N+/v8+32VKyLGPFihXIqcqBObn505hrQjRAeyArK8tnM/SKoog+ffrAUGRA7Qn/mDpIDa5SF6r3VqNPah+EhYWpXY7fY+gqSBAEuIpcuPDhBbir3D49ZVWWZVQdqELh54UB06cpy3L9PvFCuZY+Fmju0mDYsGHIyclp+Qqvwmg04ocffsCAAQN8sv5AYjAYsHbtWu6LRmDoKmjWrFlYvnw5IAPn3z2Pyj2VPttWwScFKPuxDJGRkcjOzkbv3r19ti1vOXr0KHol9oL2QS1CbuO4XGqdGLoKCgkJQUpKCubPn49gMRhVe6tQ/nO5V7fhrnHj0qpLcBQ4cHvf2zFz5kxERUVBp9N5dTu+4HQ6UVRYBMEkQNS3/E9TDBIRMSYCHy75EKtXr/ZChUQtx9BVWFRUFJ588kn07dsX5gozKvdUwn7e7pU+XlelC/ZcOyp2VKB7h+4YPnw4Hn744YDoXjh//jxOnTkFfQe916Y0Eg0irAOtWLV+FXbt2uWVdV5N586d0SmmU/372MYOJjkvOWFymZCSkgJRZJw0BveSCkRRxLfffot7770XziIn8t/Lh7vS/d8+zSa4/BxZllG1twoXP7sIQRDw6aefYtKkST56Bd73/vvvY9LMSejwXAdozBqvr19G0/dtY82ZMwevPf4aLnx0AbKzbYXupe8vIakiCevXr4fBYFC7nIDAkyNU9MorryAtLQ33338/8t/PBwRAH6VHzJMxjV6Hu9yN/A/zAQCSXUJ0dDQ2btyIyMhIX5UdcGKeiEHGwQzsumMXNm/ezBYZqYqhqyKr1YqUlBTMmTMHQP0lDHcd2IWS1SWNXodUJ8Fd4caUKVMQFRUFs9mM2NjYgOhSAOpb6vPnz8ehykOwDrL6ZBsaswZVUhUKCwt9sv7LZElG6bpSWPpZYIht3a0+2SWjdEMpnEVOoJva1QQWhq7KIiMj8eyzzwKon3ixvLwcKG3iSpKAcePGoWvXrt4v0MdkWcbnn38Od383rAN9E7oAIAaLEMIFHD16FF27doXJZPLq+i0WCxJ7JiJ7ZzaMnY1tInRt221IiEtAx44d1S4noAjX6+ey2Wxtq4OKFCXLMiRJQq9evXweugDgsrlw7o1z2LRpE/r06eP19dfW1qJbt24I/lMwgpNb70SNsixDtsvInZWL5V8uR3p6utol+R2r1XrNN5+dW6Saw4cPIzU1FYa/GmD5H4va5bSY0WjEL7/8gj7VfVC8vFjtcnymJrsGpQtLseeXPRg0aJDa5QQchi6pxm63I+98HgSzANHo+z9F0SgifGQ4Pl/xOdauXev19QuCgA4dOsAgGVB3pg6l60shORW+1oaPVWRWoPKXSkgVEjrEdUBQUJDaJQUchi6p4uLFi8i7kAddpE6xS12KBhGhfwjFiowV2LRpk8+2Exsbi2hTdP2JL26fbUYV1QeqocnToHv37hwF0kzca6SK+fPn4+nXnkb83+J9Mi63MXw1bnfevHlYsGCBz7ejtMuvY9iwYdi8eTP0er3KFQUmhi61OVHjovCf8v8gLS0NkuSbn/8DBgzAL7t+Qfkn5ag55purnClJqpNw/p/n8e5L72L+/PlqlxPQOGSMFCVJEt5//32csJ+A5f+pc/BMG6qFTbKh9lytz1qhJpMJnRI6wV3uRmVWJaQaCZa+gXmw0H7Bjqr9VXBeciIqIoon3rQQW7qkKFmW8cEHH+CU4xRC+ql3JTHRKEKwCjj962nU1dX5ZBuCIKBz584Qz4qo2F0B5yVnwF2bwVXuQu3JWlRurUSXzl144MwLGLrUJln6WBD0UBD69euHgwcP+mQber0eW7ZswYgRI2A/Z0feP/OUnzmkhYr+XYTStaUwm83YvXs3brvtNrVLCngMXVLM/v37cVv/22D6qwnBKcFql6MIQRAwe/ZsfPzxx4AE5H+Qj+rD1WqXdUPuWjfy3sqDPc+OkSNHYvPmzQFxedBAwD5dUkxtbS1OnTqFhLEJ0ASpM2LhtwS9gNChoVjxwwqUl5fjj3/8o0+2ExMTg/79+2PatGkAgG3523Bq4ykAgHWQ1SvXDvaWyqxKuGwuGDQGTP7rZAiCgOTk5IA8xdxfMXRJEZcuXUJRSRG04Vq/+X0lGkSEDw/Hsg+XwVHq8FnoAvXX3H3ttdcAADNnzkTu5lwUFhXC1MMEXTudIieHXI8syXCVu2DbboPepkd8l3i8svQVaLWMCG/zkz9/au1mzpyJp2Y+hfiX4iEGte0/u1mzZuHzzz8HZCD/vXxUHahSuyRI1RLy5ufBke/An/70J2zZsoWB6yPcq6Qof7wITOTYSGz/dTuGDx+OtWvXQqPxbdeHIAhITU3Fzp07AQBv/+ttrH57NQQIiHkqBppg5bpeCpcVwlHoQJg1DNu3bYcoiggNDfXL96m1YOiST0mShEWLFuGsdBbBqf558EwXoUNFTgXKT5QrdvaYyWRCYmIiAGBM2hiItSK+/vpr2LbbIBpEaCwaWG71zbhe+3k7anPqp4yvO1uHxI6JGDVqFBITE3lqrwIYuuRTbrcb8+bNgy5dB+sA3166sSVEvQjZIiM/Px/R0dGKTj1z9913IyEhob7lm/1/N0YArq6uKx6rCdE06VoVrgoX8LtRasI5AUEH6sfbBpmDkJ6ejpdeeqm55VMTMXSJAASnBkO6SULqLalYs3oNBg4cqOj2k5OTceTIEc/y/v37MXTo0CseF/9KPHRhjR+6VbCoAM6Lzga3Pf3005h7ZG7zi6UWYeiSz+zduxfPv/A8zH81Q9Ne/SFi13O5D1OGOmeM/b4PtWfPntiyZYtn+fTp0xg/fjwuLr7YpNmSXZdcePDBB/H00097bmvfvj37bFXE0CWfqaqqwuHDh5Hw5wRoTP4dugAg6ARYB1qR8XMG7HY77rzzTtVqMZnqpzW/LDIyskFwNsWgQYMarIvUxdAlnygvL0dZRRm0IVogQBpVol5ExOgILFm4BGV5ZaqG7u/FxMRg3rx5apdBXsDQJZ948cUXsWbfGsS/Eq92KUR+heNDyCdkWYYsyxAEIeD6D9vd2w5ZQhZGjx4Np9N54ycQNQFbuuRVbrcbX3zxBfLFfAQn+ee43BvRR+pRkVeB4oPFrWbWB/IfbOmSV7lcLsyYMQOnxFOwDvLfcbk3ImgFiMEiLl26BIfDoXY51IowdImuIjg5GNanrLg55Wbs2LFD7XKoFWHoktfs2bMHo+4ehZC/hsB0k0ntclpEEARAACR3YF10nPwfQ5e8xmazYU/WHujidKrN8OtNgkaApZ8FmzM3Y+vWrWqXQ60EQ5e8oqqqClU1VQFxEkRjiXoR7e9tj0UrF2HJkiVql0OtBEcvkFdMnjwZPx7/ER1ndORXOdF18ONBXiFJEiRJgqAJvHG5NxI+KhyHDYdx3333cSQDtRhDl1rE7XZj5cqVKNYVI6hb65ye2xBrgE20Yffu3ZAkHlijlmHoUos4HA5MmTIFOcYchN4RqnY5PiNoBAgGARUVFTxLjVqEoUvUCKYkE8KnhCMpJQk///yz2uVQAGPoUrNlZmbiwYceROjYUBi7GNUux6cEUYAgCnA6nexioBZh6FKzlZSUYOu2rdB31kNraQMDYTSAOdmMzCOZ2LVrl9rVUIBqA58U8oXa2lrU2esgGISAuV5uS4k6EZFjI/Hx4o9xYv8J9O/fX+2SKAAxdKlZJk6ciJ/P/IyEVxOA1nM+BJHPsXuBmsXlcsHldkHQtr5xuTcSlh6Gk5aTePjhh2G329UuhwIMW7rUJC6XCxs2bECZoQyGBOWmKfcnxngjbOU2bNy4kQfVqMnY0qUmcTgcmDBhAn61/oqwoWFql6MesX4iy9raWrhcLrWroQDC0CVqBlMvE9q/0B5JKUn46aef1C6HAghDlxotMzMT4x8fj7AHwmCIb5tdC5cJogBBJ6Cutg5ut1vtciiAMHSp0QoLC7HhPxtg6GGon1q9jRNEAaZeJhw8dRB79+5VuxwKEAxdahSHwwGnywlB17ZGKlyPoBUQ9WgUPlzxIf75z3+qXQ4FCDZXqFEmTJiA7ee3o+MrHSFoGbxEzcWWLjWK3W6H3WmHaBDb3LjcGwn9QyjOhJ7B448/jrq6OrXLIT/H0KXrcrlc2LRpEyqNlTDEte2DZ9di7GREubEcGRkZPKhGN8TQpeuqra3F2LFjcabdGYSlteFxuTciANDU930zeOl6GLpEXmDqYULU36KQcmsK1q1bp3Y55Md4II2uKTMzEwsXLUTon0Ohj9WrXY5fuzyzRGVVJc9Qo+tiS5euKS8vD6tWr4KptwlaK7+fb0QQBBi7GHHi3AkcPnxY7XLITzF06apcLlf97L5i27lebksJWgExE2Lw/sr38Y9//EPtcshPsflCV/XYY4/hl+JfEP/3eF4vl8iL2NKlq6qqqkJNXQ00QRqOy22ikP4hyG+XjylTpqC2tlbtcsjPMHSpAZfLhd27d6M2qBa6SJ3a5QSkoG5BKDeX46uvvuJ07XQFhi41UF1djdGjRyO3Qy7C08PVLidwCfX/3G43L3RODTB0iXwgqGsQol+MRt8BfbFmzRq1yyE/wgNp5LFnzx4sWbYEoXeHQh/JcbktIWgFaEwalJSWcB41aoAtXfI4ffo0/r383zD3NUMbyu/jFhMAQ7wBuYW5OHHihNrVkJ9g6BIAQJIkSLLEMbleJGgExE6MxXvfvofp06erXQ75CTZnCADw6KOPYm/5XnSY3oFfxUQ+xI8XAQDKyspQWVMJrUXLcbleZr7VjKKoIrz88suoqalRuxxSGUO3jXO5XDhw4ADsQXZow/nDxxdMPUywhdvw6aef8qAaMXTbuoqKCtx5553I75aP8D9yXK6vybIMWZbVLoNUxNAlUoCxkxGxL8bi9qG347vvvlO7HFIRf0+2YXv27MHyb5YjbGQYdBE85deXBK0ATYgG5wvP83oMbRxbum1YdnY2PjtKhTUAAALeSURBVPvfz2C53cJxuQrRR+tRUFqAX3/9Ve1SSCUM3TaK/YrKEzQCYp+JxYLvF+DZZ59VuxxSCZs3bdSjjz6KA1UH0GEqx+USKYkftzaqsLAQ5dXl0IZyXK7SzMlmlMWWYfbs2aiurla7HFIYQ7eNcblcOH78OJxBTmhCOCWEGky9TLBF2rBgwQIeVGuDGLptTGlpKQYMGIDCmws5LtdPsH+9bWHoEqnAEG9A3N/ikDYqDd98843a5ZCCeCCtDdm7dy9WrV2F0LRQDhFTmagTIUQIyM3LRWVlpdrlkILY0m1D9u/fj48++QjWO60MXT+hC9ehpLIE58+fV7sUUghDl0glgiggdnIs3lv7Hh577DG1yyGFsLnTRowbNw6Hag8h7tk4Xqjcj3C4XtvDlm4bce7cOZRUlkDXTscPup8x9TShMq4Sb7/9NsfttgEM3VbO5XIhNzcXbqMbYjDfbn8UfHMwKuMr8cYbb6CqqkrtcsjH+Cls5YqLi5GamopLfS8hLD1M7XKI2jyGbhvCbgX/pY/RI25qHMY8MAZff/212uWQD/FAWiu2f/9+rPtpHayDrdBa+Fb7M1EvQhelw6nTp1BaWqp2OeRDbOm2Yrt27cLbH7yNsLvCOC43EAiAJkSDitoKFBUVqV0N+QhDl8hPCIKAuOfi8MF/PsADDzygdjnkI2z+tFLjx4/Hrl274K5yI39BvtrlUBO4bC7I3XkRnNaKodtKpaamIioqSu0yqJni4uLULoF8RLjeZeVsNhu/bomImshqtV5zqBD7dImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgUxdImIFMTQJSJSEEOXiEhBDF0iIgVdd2JKIiLyLrZ0iYgUxNAlIlIQQ5eISEEMXSIiBTF0iYgUxNAlIlLQ/wdO5d7hF2prkQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "\n", "==== Ranking 4 [score: 0.167] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAQGElEQVR4nO3dfWyN9//H8dd1aDtKizo9buZ+6qba2hiZuVm32JZFZoL9sTBzGzdDyBILgoWNjX5ZJCwWMkLipt9l2ZTMphMLkbqfu063YcT0jmrRGz29fn/01+uraLHx7jbPRyKcc51zrs91qef5nE8vOK7rCgBgw1fTAwCAxwnRBQBDRBcADBFdADBEdAHAENEFAEO1q9t49epVricDgAcUGRnpVLWNmS4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCo2n/a8e/KdV2VlZVJkhzHkc/373nvuPXYbuXz+eQ4zj23W7rbWGrVqmU6BuCf5h9ZqwsXLigmJkYxMTH65JNPano4D1V6erp3bLf++P333yVJZ8+evev206dPm481NTW10hjGjBljPgbgn+aRzHR//fVXLV++vNJ9juPogw8+UERExF9+/WAwqNzcXEnS9evX//Lr/Z0EAgHNnj1bkrR161bt3LlTUvkxS1KjRo287d999522bdtWabul9u3ba8aMGZo9e7ZKSkqUn59vPgbgn+aRRPfGjRs6duyYjh49qtLSUoWHh6tz5846ePCg6tWrJ6k8wvHx8crMzNSlS5ckSfHx8QoLC1NhYaGOHz9+x+t26dJFV65c0bFjx7z7Ll26pP3790uSYmNjVbdu3XuOr7S0VEeOHJHrlv/HGLVr11ZCQsJ9L1OUlZXpyJEj9x06x3GUkJCgkJAQSeUfy48ePaqbN2962+Pj4xUaGqpGjRpp5MiRkqRz58550a0QGRnpbc/NzfWie79u3rypo0ePynVd+f1+Pfnkk3c9FxkZGZUi2r59ezVo0MC7nZGRoby8PMXGxv6p5Z2zZ88qOzu70n0VXyeSdP78+UpfF9nZ2frjjz8qPf6JJ55QXFycjh07pqKiIoWFhSkuLk6O4+jSpUs6f/6899hAIKCWLVtKkk6ePKnr168rJCRECQkJSk9P17Vr17zHduzYUcXFxTpz5ox3X6NGjdSuXbsHPk7gdo8kunFxcdqxY4diYmKUm5urTp066dtvv1Xnzp29P0iO4+jEiRNauXKlVqxYIUn66aef1KJFC509e1b9+/f3Xq9irTItLU1ff/21FixY4G3bsmWLkpOTJUm7d+9WXFxctWNzXVd5eXl65ZVXvPXI+vXrKyMjQ6GhofdcF3VdV0VFRRowYIBu3LhRaXy3PuZWPp9P6enp8vv9kspnpW+++Wal6Bw/flzNmjV75OuyeXl5evXVV1VaWqrhw4dr7ty5dz0XM2bM0K5du7x15I0bN+rll1/2bs+ZM0fbt2/3bt8v13Xluq6WLl2qtWvXSvrf+evatatSU1Pluq5WrFihlStXyufz6fDhw1qzZo0+/fTTSvtr166dDhw4oFGjRikjI0OtWrXSkSNHVFZWpk2bNmnu3Lne48ePH6+PPvpIjuNo8uTJOnTokKKionTq1ClNnTpVaWlp3mO/+eYbnT9/XhMnTvTuGzRokFavXi3HcczXzvHvYram6ziOfvzxRw0bNqzaxyUlJWnAgAHe7cmTJys9PV3p6elq06aNxo8fr++//97bPnr0aG97x44d7zmO5ORkPffccwoGg1q3bp0WLVqkgoICxcXFae/evfd8/g8//KCEhAQVFhZKknr37u3tPz09XV988cUdzykrK1OvXr20adMmHT16VJ07d1ZOTo5mzZql5ORkua6rfv363fW5D1tUVJROnDihp59+Wlu2bKn2XLz22ms6ePCgQkNDNX78eE2ZMkVFRUVeHBMTE3XkyBHVqVPnvvdfcS42b94s6X9fF+np6d6bZ79+/bR27Vp16NBBJ0+e1PDhw/X555+rXbt2Sk9PV4sWLardx+uvv67FixerSZMmOnXqlDp27Ki1a9eqb9++cl1Xmzdv1pw5c5Sbm6vY2FgdPnxYQ4YM0b59+1SrVi29/fbbev/991W/fn0dO3ZMvXr10rZt29StWzeVlJT8+ZMP6BFEd/PmzVq5cuUd9zuOI7/ff8+P/3369NE777zj3d67d6+SkpKUlJSky5cvKzw8XFFRUd72unXrKhAIKBAIeB/fqxMXF6eJEyfK5/Np06ZN2r59u1zXVVZW1n39gWrbtq2mTp3q7evs2bPe+C5evKgGDRrI5/Np3rx56t69u1q3bq2FCxeqsLBQhYWFatq0qaZPn67w8HClpqZq/fr1kqScnBxv5vwo+Xw+71wVFRXp+vXrWrhwobp166bExETNmzdPOTk5Kikp0c8//6zly5ertLRUeXl5ysvL885VcXGxwsLCFB0d/UAzP8dxNGHCBMXHx0sqn/l+9tlnSkpK0oYNG+S6rncuateurUAgoCtXrni3o6OjVbt29R/QRowYob59+6qgoED/+c9/lJ2drcLCQuXk5Egqf+OpX7++dyxjx47V8OHD1apVKy1atEgNGzZUfn6+iouLtWzZMp07d07FxcXKzs5+oFk9cDcPfXlh165dOnPmjMaNG/ennt+zZ08FAgFvLfPChQs6ePCgJOn555/Xs88+e8dzSkpKdOLECUny1imr4vf79cwzz0gqv1LgQWcurVu31tixY5WSkqLCwkLl5+dr1apVkqTu3bsrEAjI5/NpxIgROn36tEJCQjRu3DgtWbJEklSvXj316NFDISEhOnfunBeCmhIWFqaxY8d6IQsEAlqwYIHKysqUm5urw4cPe4Fs06bNX95fxbnJyclRcXGxJGnjxo0qLS1VTEyMevfu7a11/1ldunTRgQMHVFJSorS0tHu+mQ0cOFA9e/aUJI0dO1ZfffWVfvvtNwWDQe3fv19+v9+bMPybLk9EzXgka7oV63Z3c+us6PbHVFz32bJlS6WmpkqSFi5cqMWLF0sqn8G89957Gj58eKW1vczMTCUmJkqSJkyYoIULF1Y5tpSUFE2ZMkWStHTpUl24cEETJkyoNO7qZm6u6yokJEQpKSmSpJ07d2ro0KFVn4zbnDx50luvXrRokXr06KEXX3yx0vFXtd/bt916/iq232vN8W6/N7cfd8XPiYmJWr16tfc4x3FUWFhY7e/hvcZQsa9p06Zp2rRpcl1XnTp1UlZWlk6fPq2XXnqpymO8dRx3O5aK26NGjdKpU6fUvHlzpaamqm/fvt43ZivO092ed/vxh4eHa8eOHZU+QbGei7/Kqe7j0tWrVx/4s9TEiROVnJysqKgoZWZmas6cORo2bJgaN25c8ZpKS0vT0KFDFR0drevXr3uXfUVHR99xcX1BQYH3neWUlBTFxcWpbt26ysrK0sCBA3Xx4kWFh4crKytLX375pbp166bIyMgqx7du3TovulFRUQoGg8rLy5NU/h3qQYMGKSkpqcrnp6amatKkSd7t4uJiXb58WZK0atUqBQIBDR48WBkZGZo1a5bOnDmjrVu3KiYmRrNnz1aXLl286EZERCg0NNSb7UZERCg8PNx77fz8fO/c+P3+Oz5WX7t2TQUFBZKkxo0bKyQkRE2aNFFqamqVccjNzVXfvn2VlZWlmzdvynEcNWnSRGvXrlWPHj1UVlamzMxMjRo1SocPH1bDhg295/bv31/Lli1TZmamJk2apD179qhhw4bKzMyU67oKCwtTIBDQnj17VL9+/bvuPxgMqk+fPrpy5Yp3X2ZmpsrKyhQbG6stW7ZIkj7++GNt2LBBfr9fWVlZCgaD6tChg/bt26fs7GwtXbpUq1atUnR0tLe9Vq1aio6OVnZ2tkpLS72llNtvS+WXGlZcnREVFaW33npL8+fPl1S+1LNlyxbNnDlTgUCgUoT37NmjsLCwKr46gHKRkZFVvjs/9Jnu4MGDFRsb691+4YUXvOD+/2CUkJCgDz/88IFfu3Pnzt51vs2aNdP06dO94Enl67XVBVcqXwKobt9PPfVUtc9v06aN3n333btu69q1q+rWrav58+erTp06Gjx4sK5evSrHcTRz5kz16NFDUVFRf+rY71fFJXlVqVOnjiZNmnTHDLJ58+aSyj/+N23aVBMmTKh0yZVUvp5dsX3MmDF3zEornl9dlBzH0ejRo1VUVHTHNr/fr2bNmkmShg4dqpiYGLmuqyVLlnhvjI7jKDo6Wm+88Ua1y0gPqlOnTt6vGzdurMTExEpXyUhSSEgIf+MOf9lDn+kCD0thYaF++eUXDRkyRJmZmWrdurXWr1+v9u3bM9vE31p1M12ii7+t48ePq3fv3nfcf+jQIbVt27YGRgTcH6KLf6SSkhLvL9PcqmnTpvd1eSBQU4guABiqLrpcdAgAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGKpd0wPA4yUYDGrkyJHKz8+v6aGoQYMGWrNmjXw+5h6wQ3Rhbs+ePWpYUKCmISE1NoaLN2/qRGSkXNetsTHg8UR0USNejojQqxERNbb/bVev6r81tnc8zvhcBQCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYql3TA8DjKae0VGeKi2ts/7nBoORjzgF7RBc1YnNenjbn5dXoGPx+f43uH48nogtTPp9Pu3fvVjAYrOmhqFatWvIx24UxogtTjuOoefPmNT0MoMbwNg8AhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIaILAIaILgAYIroAYIjoAoAhogsAhoguABgiugBgiOgCgCGiCwCGiC4AGCK6AGCI6AKAIcd13ZoeAwA8NpjpAoAhogsAhoguABgiugBgiOgCgCGiCwCG/g/OG3+9TjTROgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "\n", "==== Ranking 5 [score: 0.167] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADXCAYAAAC51IK9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWEklEQVR4nO3de3BUZZ7G8e85SbpzD5AbJNwFBQIJDMgKOiJ4ASTOLIPioBkVrAItBmakZtYdorMglylKscBRGUFZdBzKKailcF0uroElKCi3BQkSJI6DQQgBciNXOt1n/0B6gkBIpPvtuD6fqlQlOX3e39vtL0+ffjmeYzmOg4iImGGHegIiIj8kCl0REYMUuiIiBil0RUQMUuiKiBik0BURMSi8uY2VlZU6n0xEpJUSEhKsq23Tka6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETGo2Us7tkU+n48r3cHYtm0sy7rm9mDNJVDjO46Dz+e77PfBmL+ImPe9C91ly5bx4osvXvK7Tp06sX37dgDefPNN5s+ff8n2xMREdu7cSVhYWEDnMnPmTDZu3EhERAS7d+8mLi7uuscsLCwkOzv7st9v2bKFbt26Xff4IhJaQQndL774gj/+8Y+X/M6yLObOnUt8fPx1jX3bbbcRHR0NwKJFiygpKSEyMtK/fejQoTzzzDMAvPjiixQXF2PbwVlFmTBhAoMHD8a2bdxud0DGTE1N9c//vffeIy8vDwCv1xuQ8UUktAIeul9++SX5+fmsWrWKrKwsIiIiqK6u5siRI4wdO5b27dtftk90dDQZGRn+n+vq6igoKPD/HBkZyYABAwDIysoiKysLgOXLl1NSUnLJWBkZGf6x3nrrLYqLi1v9HLxeL/v37/d/zLdtm0GDBmHbNhUVFRw9ehSA2NhY+vfvj23b/qPoqqoqjhw5AkCPHj1wu90UFhb6x46NjaVPnz4cOHAAj8cDXHhDyszMxOVy0aFDByZPngzAsWPH/KH7bQcPHqS+vv6S533xzQjg0KFD1NbWXrJPSkqKjpZFQizgofv888+zevVqbNtmzZo1JCUlsWfPHkaPHs2DDz54xfXWjIwMtm/f7l+z/Pvf/87dd9+NZVk4jkPPnj3Zs2cPlmUFfV3TcRxqamoYO3asPxQjIyMpKioiOjqanTt38tBDD/kfC+ByuSgqKiI+Pp79+/fz05/+FMdxePnll+nRowfjxo3zP5fBgwezefNmJk6cyOnTp/11CwoKSEtLu+bzcxwHx3GYPHkyRUVF/nE//PBD/5uN4zhMmzbN/8Z1cczHH3+cF154IbAvmIi0StDOXvD5fAwfPpw+ffqwcOFCCgsLKSwsJCcn57LHHjlyhL59+3LixAkWL17sX9N8//33mT59OseOHaNPnz58+eWXwZqu34YNGxg8eDDnz5/nT3/6Ey+99BJ1dXUMHDiQvLw8Ro0a5X8uY8aMuWz/W265hYKCAuLj45k9ezY5OTnYts2OHTt48MEHOXDgAP369ePMmTPk5uaydu1aHMdhxIgRrFq16prza/paPPHEE3zwwQcA/OQnP+H555+npKSEvn37+o+uU1JSOHz4MIWFhfz+978P6GslIq0X8CPdiRMnkpWVhc/nY/78+dTU1AAX1ldzc3OJjo4mPT2dGTNmsHDhQm6//XZ69erFkiVL8Hq9/PjHP6auro7Fixfzxhtv8Pnnn+P1eiktLaWxsTHQ071MfX29/wi0Xbt23HjjjSxatAiAXr164Xa7SU1NBbjiOq7L5SIlJcW/FNG9e3d+97vf0b17dx5++GGSkpJ45ZVXAIiLiyMzM9M//uDBg685v8bGRkpLSwH45JNPqKysBKCsrIxz584RFxfHrFmzWLJkCSdPnqS6uprFixdjWRbDhg1j/Pjx1/kKicj1CHjo9ujRg/T0dHr27MmmTZsoKCigtLSU5cuXM2vWLACSk5OZOnUqS5cuZdiwYYwYMYIlS5YA0KVLF//67YEDBygrKwv0FFulffv2DB061P99a3Xq1ImpU6cCcPvttxMVFeUPXYCYmBj/+B07dmzV2CdOnMBxHAYOHAhAeno6sbGxTJs2jd27d1NUVITH42HFihUAnDp1il69etG/f3+dfiYSIgFfXli0aBEzZszAsizWrVvHfffdB9DiP/Lly5fz2GOPAfD222/zyCOP+LddPIf14ldT397WdPvFddCrncPb1LfXjfPz8xk5ciSjRo3io48+uqTO1eo3rfHtuk3HdxyHwsJCRo4cyciRI1mzZs01x784BkBOTg5btmzxf02bNs3/uOXLl7NlyxbWrFnjf/z69euZMGHCFc8DFhEzgnLK2L59+/z/qFNZWUlmZibvvPMOSUlJrRpnzJgx1NXV+X/Ozs4mPPwfU764DHDy5MlLzn646MyZM8CFj94Xj57vv/9+5s2bd9WaY8eOZdeuXdx666088cQTwIV/SNuxYwdpaWls3bqV6dOnA1BeXg7A+fPnGTp0qP/UNMdx/B/79+3bR2ZmJvn5+XTo0IHMzEwKCgoYMWIECxcuxOVyAbB161Z69+7NkSNH/EsAVVVVl7wW4eHhdO7cmUOHDjFu3Dhee+01Vq9e7X/MlClTyMnJ4c477/SHvNfr9X//85//nDlz5gTtFDoRubaAh+6ECRMuC8DU1FTS0tKAC8E5ZMgQLMvi6aefZuDAgXTs2JEFCxbQvn17Ro8eTXJycqCn5de3b99mt0dFRdGtWzeee+45/7mx4eHhdO3alfDwcHr06MEvf/nLVtW0LIuoqCjgwppvWloaubm5l5zS1bt3b2JjY0lKSmp2/ISEBNLS0vjNb35DRUXFJdsGDRpEbGys/03h2/r379/qJQwRCSyruY/blZWVzX8WFxGRyyQkJFx1PVWfM0VEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMCg/1BOQfVq5cybvvvvud9l26dCndunUL8IykLVBf/P+i0A2x8vJytmzZAsCmTZvYnZ/Pj6KiWrx/veOwu7aW9evXk56eTlRUFGPHjsWyrGBNWQz4dl98vGcfNw4f1eL9z9fVUrj9ffVFG2Q5jnPVjZWVlVffKNetoaGBPXv2MG7cOCItCwvo6nLxfHp6i8co9XiYcfw4AB7HIblTJ3bt2kV0dDRhYWFBmrkEU9O+cEVFg2XRsVdfnly1scVjlJ8oZsnE2wEHr8dDSlKi+sKghISEq767KXRDaPbs2axYsQKPx8PyLl3oEB6OBUS04mjEcRw833y/vqKCP5eX43a7ycvLo3///kGZtwTXP/qikd++u4v45E5gW4RHuFo8huM4eM+fB+DDvyxj88sL1BcGNRe6YXPmzLnqjg0NDVffKN+Zz+dj2rRp5OXlEXnuHLNSUujucuGybcJa+fHPsizCvvlKDA+np9vNR1VVfPbZZzQ2NjJw4MAgPQsJtKZ9Yce2Y+L8V+jYO4MIdyR2K49OLcvCDg/HDg8nLrkjaX0yOfjBf6ovDImMjJx7tW1a0zWsvLyc/Px8Nm/eTPu6OobExDAkOjogY3eMiCDathkWHc2BXbtISEggPT2du+66S2t5bVzTvohK6Uy/YXfQ57a7AzJ2YufuRMbGkTFqHPt3bSdh40b1RQhpecGg8+fPs2vXLrKzs3FZFo8nJjI2Pj4otX779dd80dBAYmoq+/btIzIyUmt5bVTTvgh3RzJu1nPc8sDkoNR69dExnDj8KUmJHdQXQdTc8oLO0zUoNzeXn/3sZ1jAK507c09cXNBqLUhLI6dDB06dOsUNN9zAwYMHg1ZLrs/FvsCyeGrtR9w8/hdBqzV1xXrumT5bfRFCCl2DPB4PSY7D7NRU2oeHt3r9tjVclsWwmBhmJidTX1+Pz+cLWi25Ph6Ph/iOXfjF4jeJT0olLDx4q37hLjcZd2Yz4d+Wqi9CRGu6BjiOw5YtWyguLibWtvmnmBgjdTtFROD6Jth37NiB2+0mIyPDSG25tqZ9ERXfjn53jDVSN7FzdyJckWBZ6osQ0JGuAT6fj6lTp/I/eXmtOh0sECwuHPU++8wzrFy50mhtad7Fvti6bRthrpafDhYQFkS4Innm2WfVF4YpdA2akpjInI4djdZsHxbGn7t1o6fpP2ppsXFPPceUl/9qtGZcUgq5eYdJu2mA0bqi5QWjIiwLt232fc6yLKIsS6cGtWFhLhcRkS3/X78DwbJs3NExWIb7UXSkKyJilEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQtcgB/A1c/3ioNR0HLyOA4brSss5Ph8+n9dsTcfB5zVbUy5Q6Br072fPMrekxGjNCq+Xx44d42/f3C9L2p6NS+ayasYkozWrz5ay8J7+nDyi6+maptA1wLIs5s+fz6BbbqHG8PVLfUClz8evnnqK+++/32htad7Fvhg8MJP6c1VGazs+h5qKs/xq5kz1hWEKXQNs22bSpEncdNNN1Pt8HKmvp9HAx/3yxkb+1tAAwH333cewYcOCXlNarmlfNNRV89XBvXg9nmvveJ3OnTnF14WfgqO+CAWFrkGWZVHc2Mi/nDhBpddLc/enu14+xyG/upp5p07pCmNtnGVZnP7b5yx7bCw1FWeD2heOz8eBTf/BW79+GLVFaOjGlAZVVVXx8ccfM3HiRBJsm0c6dODuIN2Y8tkTJ/i8oYG45GTy8/NJTEwkIiIiKLXk+jTti5j2SYye8Qw3//PDQan1+hMTOF6wj4TYaPVFEDV3Y8qwOXPmXHXHhoaGq2+UVnO73cTGxtK5c2fyP/6Ysvp6qrxe+kZGBqxGtdfLqrIyDtTVMejWW5k+fTrDhg3THV/bsKZ98eG2rVSdPU1N+Vm6DRwasBp1VRVsXDqXok+2MSRrgPoiyCIjI+debZuWFwxLTU1l6tSp3HzzzZyKjeWDc+coamgIyBpveWMjhxsaeK+qik69ezNmzBhycnK0vPA90LQvzn11lD3vrub4Z/vxeq7/rJNzZ0o5dmA3O995nS6pyeqLENPyQgg9/fTTvPbaawCs7NqVxG+OOlrzx9D0v9+6ykpWlZVhWRb5+fkMGKBbsXwf/aMvLP51437iUzoCrbv7x4W+uNAb+W+9wqalz6kvDGpueUGhG0KVlZXs2rWLBx54gHZhYdhAF5eLeZ06tXiM042N/PbrrwGo9/mIT00lLy+PlJQUrdV9TzXti9jEFGzbJqXnTTy+bG2Lx6goOc6yR+8FHBpqa2gfF6O+MKi50NU90kIoISGBrKwsFixYAMCGDRs4uHMnb5w92+Ixanw+yrxeZs6cSWpqKrGxsaSlpemj4/fYlfrifw8V8F+Ln23xGPXV56g6fVJ90QYpdEMsJSWF6dOnAxduyV1RUUFRK8fIAB599FFuuOGGgM9PQuNKfVG+f3urxsjIyFBftEFaXhARCbDmlhd09oKIiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDdO0FaZPKysqoqvpuN2tMS0vD5XIFeEYigaHQlTbphRde4NVXX/1O++7YsYN+/foFeEYigaEL3kibcfjwYaZMmQLAqVOnOBd2jtRfpLZ4/8aqRkpeL6Fnz5643W5SU1NZt26dLmcoxul6utLmbdu2jQ0bNnD48GHibonDTrGJi4/D1anlywRhCWEkjEjgLGdpKG7g631f89JLL/HQQw+RnJwcxNmLtJyOdCWkHMehpKSEefPmsfqvqwmLCyN9Rjrh8dd3PFC1s4ry/y7He87L2rVrGTJkCO3atQvQrEWap9v1SJvl8/no27cvp06dwtXZRfrMdKB194m7Esdx8FZ4+WrhVwA8+eST/OEPf7ju+Yq0hK6nK23SoUOHGDFiBGfPniXhjgRSJqVgWa27AePVWJZ14aj51+lEJEewZs0axo8fj8/nC8DMRb47ha6ExLZt23j77bc5ePAgUYOjiO4bjSslsKd5WeEW7nQ3cUPjqI6vZs+ePSxbtozS0tKA1hFpDYWuhMS6detYtnwZYQlhdBjTgaieUUGr1e6OdsT+KJZap5bc3FyOHz8etFoi16LQlZBxp7vpmtsVOyb4bRg3NI70X6cHvY7ItSh0xSiv18v48eN57733AAK2hnstTWs8+eSTLFmyJOg1Ra5EoSvGffrpp1THVxMzIMZoXctlET88nqPHj1JcXGy0tshFCl0JidhBsbS7w+x5s2FRYSSNTyKifYTRuiJNKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNCVkGgobqD281qjNX0eH9X7q/HWeo3WFWlKoSvGxcTEULevjvLN5Ubr+mp9lP6llIi6CNxut9HaIhcpdMUo27bZvXs3kyZNCtkc1q1bx9y5c0NWX37YFLpilGVZuN1uwsLC8JR6KFlZYuTjfk1BDaXvXLhNj8vlIiJCVxqT0FDoSkhkZmYyfMhwag/XUlNQg+esJ2i16o7WUfNZDU6xQ3Z2tm7FLiGlW7BLyOzdu5fs7Gzq6urokN2B+OHx2BGBOw5wHAfH43Di5RP4Tvvo0qULe/fuxbZ1rCHBpVuwS5s0aNAgioqKSE5OpmxDGSeXnwzo+N5KL8fmHOP8yfNMnjyZnTt3Grk1kEhzFLoSMrZtEx0dzbJly7jnrnvwlHgoWVWCt+b613hrCmo4veY0jsdh4cKFTJ48mcjISIWuhFx4qCcgP2yWZXHXXXdx/Phxqqur2bFjB7UZtdjRNnakTdQNLb81u++8j7qjdQDUHKrB+cphzJgx3HvvvXTv3j1Iz0CkdbSmK23GgQMHGD16NAAej4ew5DDSZqa1eP/GskaOv3Acl8uFbdukpaWxd+9eHd2Kcc2t6Sp0pc3wer3U19cDMGfOHFa8vgIrohWB6YDjccjLy6NPnz5YlkVUVJRCV4xT6Mr3zsGDB/niiy++076jRo0iPj4+wDMSaTmFroiIQTplTESkjVDoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikEJXRMQgha6IiEEKXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImKQQldExCCFroiIQQpdERGDFLoiIgYpdEVEDFLoiogYpNAVETFIoSsiYpBCV0TEIIWuiIhBCl0REYMUuiIiBil0RUQMUuiKiBik0BURMUihKyJikOU4TqjnICLyg6EjXRERgxS6IiIGKXRFRAxS6IqIGKTQFRExSKErImLQ/wHJAIXGLjUk8gAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "tags": [], "needs_background": "light" } }, { "output_type": "stream", "text": [ "\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "02CtBFm0ocX6" }, "source": [ "## Part II: People\n", "\n", "When we think about diversity and inclusion, we're often thinking about **social, personal, and cultural** layers as well. In a world of shapes and colors, the choice to prioritize diverse shapes over diverse colors is mostly arbitrary. But when thinking about diversifying images of people, **context matters**.\n", "\n", "What are we using these images for? Is it selecting images for a college brochure? Discovering new hairstyles? Searching for medical advice?\n", "\n", "What impact does an image set being \"more diverse\" or \"more inclusive\" have in this use case? Does representing many ages have the same impact of representing many gender presentations?\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "wohvRKhKoEe2" }, "source": [ "### Toy example: a few images" ] }, { "cell_type": "markdown", "metadata": { "id": "Jkd8ZiVOp70g" }, "source": [ "Supppose we're using the same ideas of inclusion and diversity as before, but now we're using it to rank images of people in an image retriever tool. \n", "\n", "We're going to keep things simple and evaluate diversity and inclusion with respect to a few coarse buckets: perceived **skin type\\***, **age**, and **gender presentation**. \n", "\n", "*This is using the [Fitzpatrick skin type scale](https://en.wikipedia.org/wiki/Fitzpatrick_scale), a scale used in dermatology that roughly corresponds to skin darkness.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "4mo3_xjNrQaI" }, "source": [ "NOTE: **The images below have been tagged for demonstration purposes only; we don't have ground-truth labels for gender, skin tone, or age for these images.**" ] }, { "cell_type": "markdown", "metadata": { "id": "0zE5gAyV4ToR" }, "source": [ "First, enter the demographic information for the image viewer:" ] }, { "cell_type": "code", "metadata": { "id": "xuDYPcBcoIS6", "colab": { "base_uri": "https://localhost:8080/" }, "cellView": "form", "outputId": "19cca5df-56af-4639-fae9-d4f82f5b9c26" }, "source": [ "#@title Enter a viewer's demographics\n", "age = 30#@param {type:\"integer\"}\n", "skin_type = 2 #@param {type:\"slider\", min:1, max:6, step:1}\n", "gender_presentation = \"androgynous\" #@param [\"masculine\", \"feminine\", \"androgynous\"]\n", "\n", "self_params = {\n", " Age:age_to_enum(age),\n", " SkinType:skin_type_to_enum(skin_type),\n", " GenderPresentation:gender_to_enum(gender_presentation),\n", "}\n", "\n", "# me = Person(properties=self_params)\n", "me = Entity(self_params)\n", "scorer = Scorer(me) # Scoring is dependent on the viewer\n", "print(me)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "4DKH6g5zTanS", "colab": { "base_uri": "https://localhost:8080/", "height": 596 }, "cellView": "form", "outputId": "c1e46ea4-66d5-4b11-b4f8-da6d3d9f1db3" }, "source": [ "#@title Display our example images\n", "\n", "test_url_1 = 'http://t0.gstatic.com/images?q=tbn:ANd9GcRvV0wp1mj3YUtZBsVlueDvUt0I0UbL9nIgBoPdTPnOSIZ4UCqEHP1UC5J26ULLAxazBvVv7BUNME0iO1QzimdKicZalaI'\n", "test_url_2 = 'http://t1.gstatic.com/images?q=tbn:ANd9GcQrK6SD6AgFfZlJcdrHhcAzTQSpWmf87ZvAA8DsTOpzwg2SSiKk9VrmA5VhifqVbwt4SgP3B7cFdB8In6NKNs1Q7nAelpU'\n", "test_url_3 = 'https://media.gettyimages.com/photos/picture-id973089468'\n", "\n", "image_1_persons = [\n", " Entity({\n", " Age: age_to_enum(48),\n", " SkinType: skin_type_to_enum(4),\n", " GenderPresentation: gender_to_enum('masculine')\n", " })\n", "]\n", "image_1_params = {'subjects': ['doctor']}\n", "\n", "image_2_persons = [\n", " Entity({\n", " Age: age_to_enum(33),\n", " SkinType: skin_type_to_enum(5),\n", " GenderPresentation: gender_to_enum('feminine')\n", " })\n", "]\n", "image_2_params = {'subjects': ['doctor', 'smiling']}\n", "\n", "image_3_persons = [\n", " Entity({\n", " Age: age_to_enum(19),\n", " SkinType: skin_type_to_enum(1),\n", " GenderPresentation: gender_to_enum('feminine')\n", " }),\n", " Entity({\n", " Age: age_to_enum(23),\n", " SkinType: skin_type_to_enum(1),\n", " GenderPresentation: gender_to_enum('masculine')\n", " }),\n", " Entity({\n", " Age: age_to_enum(45),\n", " SkinType: skin_type_to_enum(2),\n", " GenderPresentation: gender_to_enum('feminine')\n", " })\n", "]\n", "image_3_params = {\n", " 'subjects': ['computer_programmer', 'smiling']\n", "}\n", "\n", "image_1 = EntityImage(image_1_persons, PersonFetcher(test_url_1), image_1_params)\n", "image_2 = EntityImage(image_2_persons, PersonFetcher(test_url_2), image_2_params)\n", "image_3 = EntityImage(image_3_persons, PersonFetcher(test_url_3), image_3_params)\n", "\n", "\n", "image_1.show()\n", "image_2.show()\n", "image_3.show()" ], "execution_count": null, "outputs": [ { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } } ] }, { "cell_type": "markdown", "metadata": { "id": "WuV94V6PaSjl" }, "source": [ "Now we can set a bunch of parameters and see exactly how our net diversity and inclusion scores are calculated!" ] }, { "cell_type": "code", "metadata": { "id": "H7qpL4gra9vh", "cellView": "form" }, "source": [ "#@title How much to weigh out overall diversity vs. inclusion score?\n", "DIVERSITY_VS_INCLUSION_WEIGHT = 0.45 #@param {type:\"slider\", min:0, max:1, step:0.05}\n" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "pKYOn55FbQP2", "cellView": "form" }, "source": [ "#@title How should we aggregate the diversity scores across all people in an image? { run: \"auto\" }\n", "DIVERSITY_AGGREGATE_ACROSS_PEOPLE = \"average\" #@param [\"egalitarian\", \"utilitarian\", \"nash\", \"min\", \"max\", \"average\"]\n" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "FY5Hy7I3b1QX", "cellView": "form" }, "source": [ "#@title How should we weigh the different components of our diversity scores? { run: \"auto\" }\n", "\n", "DIVERSITY_AGE_WEIGHT = 0.1 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "DIVERSITY_GENDER_PRESENTATION_WEIGHT = 0.4 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "DIVERSITY_SKIN_TYPE_WEIGHT = 0.4 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "\n", "DIVERSITY_AGGREGATE_SCORES = {Age : DIVERSITY_AGE_WEIGHT,\n", " GenderPresentation : DIVERSITY_GENDER_PRESENTATION_WEIGHT,\n", " SkinType : DIVERSITY_SKIN_TYPE_WEIGHT}" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "Cy2s_5xlb3wk", "cellView": "form" }, "source": [ "#@title How should we aggregate the inclusion scores across all people in an image? { run: \"auto\" }\n", "INCLUSION_AGGREGATE_ACROSS_PEOPLE = \"average\" #@param [\"egalitarian\", \"utilitarian\", \"nash\", \"min\", \"max\", \"average\"]\n" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "gSYes5lcb_mD", "cellView": "form" }, "source": [ "#@title How should we weigh the different components of our inclusion scores? { run: \"auto\" }\n", "\n", "INCLUSION_AGE_WEIGHT = 0.4 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "INCLUSION_GENDER_PRESENTATION_WEIGHT = 0.1 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "INCLUSION_SKIN_TYPE_WEIGHT = 0.85 #@param {type:\"slider\", min:0, max:1, step:0.05}\n", "\n", "INCLUSION_AGGREGATE_SCORES = {Age : INCLUSION_AGE_WEIGHT,\n", " GenderPresentation : INCLUSION_GENDER_PRESENTATION_WEIGHT,\n", " SkinType : INCLUSION_SKIN_TYPE_WEIGHT}" ], "execution_count": null, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "B0waIDOd9hWD", "colab": { "base_uri": "https://localhost:8080/" }, "cellView": "form", "outputId": "bcd91867-1c1c-49ac-a24c-0acdab547676" }, "source": [ "#@title Roll that all up into a scoring function...\n", "# How to aggregate across different types of subscores (e.g. age vs. gender)\n", "combiner = Combiner(diversity_weight=DIVERSITY_WEIGHT, \n", " diversity_aggregate=DIVERSITY_AGGREGATE_SCORES, \n", " inclusion_aggregate=INCLUSION_AGGREGATE_SCORES)\n", "\n", "scorer.diversity_fn = DiversityFn(\n", " # How to aggregate across *people* in an image\n", " aggregate_method=DIVERSITY_AGGREGATE_ACROSS_PEOPLE, \n", " verbose_output=True)\n", "scorer.inclusion_fn = InclusionFn(\n", " # How to aggregate across *people* in an image\n", " aggregate_method=INCLUSION_AGGREGATE_ACROSS_PEOPLE, \n", " verbose_output=True)\n", "scorer.combiner = combiner\n", "\n", "print('Weighting diversity score at %s, inclusion score at %s' % (\n", " combiner.diversity_weight, combiner.inclusion_weight))\n", "print()\n", "print('Using %s to aggregate across individuals in each image (to get each inclusion subscore).' % scorer.inclusion_fn.aggregate_method)\n", "print('Using %s to aggregate across individuals in each image (to get each diversity subscore).' % scorer.diversity_fn.aggregate_method)\n", "print('Using %s to aggregate inclusion subscores.' % combiner.inclusion_aggregate_name)\n", "print('Using %s to aggregate diversity subscores.' % combiner.diversity_aggregate_name )\n", "print()\n", "print('Viewer:', me)\n", "# print('Query:', query)\n" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Weighting diversity score at 0.5, inclusion score at 0.5\n", "\n", "Using average to aggregate across individuals in each image (to get each inclusion subscore).\n", "Using average to aggregate across individuals in each image (to get each diversity subscore).\n", "Using weighed_average to aggregate inclusion subscores.\n", "Using weighed_average to aggregate diversity subscores.\n", "\n", "Viewer: : Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "HDBh2onCakNm" }, "source": [ "Voila! Now we can see what our function outputs:" ] }, { "cell_type": "code", "metadata": { "id": "PHKa5wF3e01R", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "cellView": "form", "outputId": "bfafe7e0-436a-488b-8874-1f7b60e8700d" }, "source": [ "#@title Example: score an image with one person\n", "individ_score = scorer.score_image(image_1,\n", " verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.167\n", " Age.RANGE_45_64: 1\n", " Age.RANGE_0_17: 0\n", " Age.UNKNOWN: 0\n", " Age.RANGE_25_44: 0\n", " Age.RANGE_18_24: 0\n", " Age.RANGE_65_OVER: 0\n", " 1 s represented / 6 total categories\n", " average (binary vector): 0.167\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.143\n", " SkinType.TYPE_3: 0\n", " SkinType.TYPE_1: 0\n", " SkinType.TYPE_4: 1\n", " SkinType.TYPE_6: 0\n", " SkinType.UNKNOWN: 0\n", " SkinType.TYPE_2: 0\n", " SkinType.TYPE_5: 0\n", " 1 s represented / 7 total categories\n", " average (binary vector): 0.143\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.25\n", " GenderPresentation.ANDROGYNOUS: 0\n", " GenderPresentation.UNKNOWN: 0\n", " GenderPresentation.MASCULINE: 1\n", " GenderPresentation.FEMININE: 0\n", " 1 s represented / 4 total categories\n", " average (binary vector): 0.25\n", "\n", "Note: this image contains a single entity, so diversity is low by definition.\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: average\n", "\n", " Does anyone have a similar to mine? (RANGE_25_44)\n", " --> bucketed closeness subscore: 0.8\n", " per-entity scores: ['entity_0 = 0.8']\n", " average: 0.8\n", "\n", " Does anyone have a similar to mine? (TYPE_2)\n", " --> bucketed closeness subscore: 0.667\n", " per-entity scores: ['entity_0 = 0.667']\n", " average: 0.667\n", "\n", " Does anyone have the same as me? (ANDROGYNOUS)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0']\n", " average: 0.0\n", "\n", "\n", "Aggregating image-level diversity scores with method: weighed_average\n", "--> Net image-level diversity score: 0.174\n", "\n", "Aggregating image-level inclusion scores with method: weighed_average\n", "--> Net image-level inclusion score: 0.887\n", "\n", "Combining diversity and inclusion scores at the image level:\n", " 0.5 * 0.174 +\n", " 0.5 * 0.887 =\n", "\n", " -------------------\n", " | Final score: 0.53 |\n", " -------------------\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "wClPzuq34mku", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "cellView": "form", "outputId": "bbd14ead-097d-432a-e372-bf42db7478b6" }, "source": [ "#@title Example: Score an image with a small group\n", "scorer.diversity_fn = DiversityFn(aggregate_method='average', verbose_output=True)\n", "scorer.inclusion_fn = InclusionFn(aggregate_method='max', verbose_output=True)\n", "scorer.combiner = combiner\n", "individ_score = scorer.score_image(image_3, \n", " # query,\n", " verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.333\n", " Age.RANGE_45_64: 1\n", " Age.RANGE_0_17: 0\n", " Age.UNKNOWN: 0\n", " Age.RANGE_25_44: 0\n", " Age.RANGE_18_24: 2\n", " Age.RANGE_65_OVER: 0\n", " 2 s represented / 6 total categories\n", " average (binary vector): 0.333\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.286\n", " SkinType.TYPE_3: 0\n", " SkinType.TYPE_1: 2\n", " SkinType.TYPE_4: 0\n", " SkinType.TYPE_6: 0\n", " SkinType.UNKNOWN: 0\n", " SkinType.TYPE_2: 1\n", " SkinType.TYPE_5: 0\n", " 2 s represented / 7 total categories\n", " average (binary vector): 0.286\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.5\n", " GenderPresentation.ANDROGYNOUS: 0\n", " GenderPresentation.UNKNOWN: 0\n", " GenderPresentation.MASCULINE: 1\n", " GenderPresentation.FEMININE: 2\n", " 2 s represented / 4 total categories\n", " average (binary vector): 0.5\n", "\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have a similar to mine? (RANGE_25_44)\n", " --> bucketed closeness subscore: 0.8\n", " per-entity scores: ['entity_0 = 0.8', 'entity_1 = 0.8', 'entity_2 = 0.8']\n", " max: 0.8\n", "\n", " Does anyone have a similar to mine? (TYPE_2)\n", " --> bucketed closeness subscore: 1.0\n", " per-entity scores: ['entity_0 = 0.833', 'entity_1 = 0.833', 'entity_2 = 1.0']\n", " max: 1.0\n", "\n", " Does anyone have the same as me? (ANDROGYNOUS)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0', 'entity_1 = 0.0', 'entity_2 = 0.0']\n", " max: 0.0\n", "\n", "\n", "Aggregating image-level diversity scores with method: weighed_average\n", "--> Net image-level diversity score: 0.348\n", "\n", "Aggregating image-level inclusion scores with method: weighed_average\n", "--> Net image-level inclusion score: 1.17\n", "\n", "Combining diversity and inclusion scores at the image level:\n", " 0.5 * 0.348 +\n", " 0.5 * 1.17 =\n", "\n", " -------------------\n", " | Final score: 0.759 |\n", " -------------------\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "NVY2o9qCp0Ou", "colab": { "base_uri": "https://localhost:8080/", "height": 868 }, "cellView": "form", "outputId": "2e008d12-8a7f-4312-97d9-a717853689df" }, "source": [ "#@title Ranking images\n", "image_ranking = scorer.rank_images(images=[image_1, image_2, image_3],\n", " verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "===================\n", "===== Ranking =====\n", "===================\n", ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n", "Query: \n", "\n", "\n", "==== Ranking 1 [score: 0.759] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ "\n", "\n", "==== Ranking 2 [score: 0.53] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ "\n", "\n", "==== Ranking 3 [score: 0.499] ====\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ "\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "Ku1ClTfdPjyU", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "cellView": "form", "outputId": "b1cb8a86-2c8b-4c9f-cd7d-5d9c29cd6c28" }, "source": [ "#@title Example: treat our images as a set and aggregate all of the diversity and inclusion scores\n", "scorer.score_set(images=[image_1, image_2, image_3],\n", " verbose_output=True)" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "\n", "Scoring image 1\n", "\n", "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.167\n", " Age.RANGE_45_64: 1\n", " Age.RANGE_0_17: 0\n", " Age.UNKNOWN: 0\n", " Age.RANGE_25_44: 0\n", " Age.RANGE_18_24: 0\n", " Age.RANGE_65_OVER: 0\n", " 1 s represented / 6 total categories\n", " average (binary vector): 0.167\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.143\n", " SkinType.TYPE_3: 0\n", " SkinType.TYPE_1: 0\n", " SkinType.TYPE_4: 1\n", " SkinType.TYPE_6: 0\n", " SkinType.UNKNOWN: 0\n", " SkinType.TYPE_2: 0\n", " SkinType.TYPE_5: 0\n", " 1 s represented / 7 total categories\n", " average (binary vector): 0.143\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.25\n", " GenderPresentation.ANDROGYNOUS: 0\n", " GenderPresentation.UNKNOWN: 0\n", " GenderPresentation.MASCULINE: 1\n", " GenderPresentation.FEMININE: 0\n", " 1 s represented / 4 total categories\n", " average (binary vector): 0.25\n", "\n", "Note: this image contains a single entity, so diversity is low by definition.\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have a similar to mine? (RANGE_25_44)\n", " --> bucketed closeness subscore: 0.8\n", " per-entity scores: ['entity_0 = 0.8']\n", " max: 0.8\n", "\n", " Does anyone have a similar to mine? (TYPE_2)\n", " --> bucketed closeness subscore: 0.667\n", " per-entity scores: ['entity_0 = 0.667']\n", " max: 0.667\n", "\n", " Does anyone have the same as me? (ANDROGYNOUS)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0']\n", " max: 0.0\n", "\n", "\n", "Scoring image 2\n", "\n", "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.167\n", " Age.RANGE_45_64: 0\n", " Age.RANGE_0_17: 0\n", " Age.UNKNOWN: 0\n", " Age.RANGE_25_44: 1\n", " Age.RANGE_18_24: 0\n", " Age.RANGE_65_OVER: 0\n", " 1 s represented / 6 total categories\n", " average (binary vector): 0.167\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.143\n", " SkinType.TYPE_3: 0\n", " SkinType.TYPE_1: 0\n", " SkinType.TYPE_4: 0\n", " SkinType.TYPE_6: 0\n", " SkinType.UNKNOWN: 0\n", " SkinType.TYPE_2: 0\n", " SkinType.TYPE_5: 1\n", " 1 s represented / 7 total categories\n", " average (binary vector): 0.143\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.25\n", " GenderPresentation.ANDROGYNOUS: 0\n", " GenderPresentation.UNKNOWN: 0\n", " GenderPresentation.MASCULINE: 0\n", " GenderPresentation.FEMININE: 1\n", " 1 s represented / 4 total categories\n", " average (binary vector): 0.25\n", "\n", "Note: this image contains a single entity, so diversity is low by definition.\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have a similar to mine? (RANGE_25_44)\n", " --> bucketed closeness subscore: 1.0\n", " per-entity scores: ['entity_0 = 1.0']\n", " max: 1.0\n", "\n", " Does anyone have a similar to mine? (TYPE_2)\n", " --> bucketed closeness subscore: 0.5\n", " per-entity scores: ['entity_0 = 0.5']\n", " max: 0.5\n", "\n", " Does anyone have the same as me? (ANDROGYNOUS)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0']\n", " max: 0.0\n", "\n", "\n", "Scoring image 3\n", "\n", "\n", "===================\n", "===== Scoring =====\n", "===================\n", "\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ ": Age.RANGE_25_44 : SkinType.TYPE_2 : GenderPresentation.ANDROGYNOUS\n", "\n", "===================================\n", "Diversity subscores (single image):\n", "===================================\n", "\n", " - Aggregating scores across entities in image using method: average\n", " - Using counts or simple binary presence/absence? binary\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.333\n", " Age.RANGE_45_64: 1\n", " Age.RANGE_0_17: 0\n", " Age.UNKNOWN: 0\n", " Age.RANGE_25_44: 0\n", " Age.RANGE_18_24: 2\n", " Age.RANGE_65_OVER: 0\n", " 2 s represented / 6 total categories\n", " average (binary vector): 0.333\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.286\n", " SkinType.TYPE_3: 0\n", " SkinType.TYPE_1: 2\n", " SkinType.TYPE_4: 0\n", " SkinType.TYPE_6: 0\n", " SkinType.UNKNOWN: 0\n", " SkinType.TYPE_2: 1\n", " SkinType.TYPE_5: 0\n", " 2 s represented / 7 total categories\n", " average (binary vector): 0.286\n", "\n", " Which of the possible s are present?\n", " --> subscore: 0.5\n", " GenderPresentation.ANDROGYNOUS: 0\n", " GenderPresentation.UNKNOWN: 0\n", " GenderPresentation.MASCULINE: 1\n", " GenderPresentation.FEMININE: 2\n", " 2 s represented / 4 total categories\n", " average (binary vector): 0.5\n", "\n", "\n", "===================================\n", "Inclusion subscores (single image)\n", "===================================\n", "\n", " - Using aggregation method: max\n", "\n", " Does anyone have a similar to mine? (RANGE_25_44)\n", " --> bucketed closeness subscore: 0.8\n", " per-entity scores: ['entity_0 = 0.8', 'entity_1 = 0.8', 'entity_2 = 0.8']\n", " max: 0.8\n", "\n", " Does anyone have a similar to mine? (TYPE_2)\n", " --> bucketed closeness subscore: 1.0\n", " per-entity scores: ['entity_0 = 0.833', 'entity_1 = 0.833', 'entity_2 = 1.0']\n", " max: 1.0\n", "\n", " Does anyone have the same as me? (ANDROGYNOUS)\n", " --> match subscore: 0.0\n", " per-entity scores: ['entity_0 = 0.0', 'entity_1 = 0.0', 'entity_2 = 0.0']\n", " max: 0.0\n", "\n", "\n", "Aggregating set-level diversity scores with method: average\n", "--> Net set-level diversity score: 0.232\n", "\n", "Aggregating set-level inclusion scores with method: average\n", "--> Net set-level inclusion score: 0.961\n", "\n", "Combining diversity and inclusion scores at the set level:\n", " 0.5 * 0.232 +\n", " 0.5 * 0.961 =\n", "\n", " -------------------\n", " | Final score: 0.596 |\n", " -------------------\n" ], "name": "stdout" }, { "output_type": "execute_result", "data": { "text/plain": [ "0.5961507936507937" ] }, "metadata": { "tags": [] }, "execution_count": 31 } ] }, { "cell_type": "markdown", "metadata": { "id": "765K3WaLxRvm" }, "source": [ "# Conclusion\n", "\n", "As you can see in our toy examples, things can get complex fast, and it's easy to get lost in the details. What categories do we care about the most? How do we label them? What do these scores even mean? There's no obvious \"right\" way to design a ranking system like this. \n", "\n", "What's important to keep in mind is that when trying to quantify concepts like \"diversity\" and \"inclusion\" is this: even design decisions that seem arbitrary or inconsequential can end up encoding priorities, values, and worldviews. These details matter; be thoughtful!\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "lHWL-cSOH-nk" }, "source": [ "# Credits\n", "\n", "This notebook was written by Dylan Baker (`dylanbaker@google.com`).\n", "\n", "Many thanks to Adam Pearce, Ellen Jiang, Meg Mitchell, Timnit Gebru, and Ben Hutchinson for their help and feedback." ] } ] }