Predictive Uncertainty through Quantization

1 INTRODUCTION x < latexit sha1_base64= '' 556K3KfvISjTw9nnZ+SMnXd6wz0= '' > AAAEbHicfZPritNAFMdnt1XXettV/LQIwWVFpJRUBPWDsKgLghVX3NpCU5bJ5KQddi4hM2mbDnkEv+qz+RS+gpM2QnZaPTDhn/M7k3OZTJgwqrTv/9rZbTSvXb+xd7N16/adu/f2D+5/UzJLCfSJZDIdhlgBowL6mmoGwyQFzEMGg/DyXckHM0gVleJc5wmMOZ4IGlOCtXV9DRazi/0jv+OvzNsU3UococrOLg4a50EkScZBaMKwUqOun+ixwammhEHRCjIFCSaXeAIjKwXmoMZmVWvhHVtP5MUytUtob+Wt7zCYK5Xz0EZyrKfKZaVzGxtlOn41NlQkmQZB1onijHlaemXjXkRTIJrlVmCSUlurR6Y4xUTb8WzL0q4qaYf8ak+rmqkoT0fGWkZSte3MExARwUk51raGhVZ0CW80FfnYlDFCalBF69irZyrTaCnZBrCFs8QKm1jAnEjOsYhMcFqYoNwThua0cBiZ/YWxIS5c1ODChUNSQYKZGbq0V6c9ly5rH166MK/B3IWzGpy5cFCDAxfOa3DuwrAGwxK+B/uTpvDJuj4nkGItUxN87BXGrq30mQkmGQ+BKXu6hbH6n3HriPLJ8cJt/j+9J1NaK7N8LQp7D7vurdsU/eed1x3/y4ujk7fVhdxDh+gxeoq66CU6QR/QGeojgiboO/qBfjZ+Nx82D5uP1qG7O9WeB+iKNZ/8Aa7zj1A= < /latexit > < latexit sha1_base64= '' 556K3KfvISjTw9nnZ+SMnXd6wz0= '' > AAAEbHicfZPritNAFMdnt1XXettV/LQIwWVFpJRUBPWDsKgLghVX3NpCU5bJ5KQddi4hM2mbDnkEv+qz+RS+gpM2QnZaPTDhn/M7k3OZTJgwqrTv/9rZbTSvXb+xd7N16/adu/f2D+5/UzJLCfSJZDIdhlgBowL6mmoGwyQFzEMGg/DyXckHM0gVleJc5wmMOZ4IGlOCtXV9DRazi/0jv+OvzNsU3UococrOLg4a50EkScZBaMKwUqOun+ixwammhEHRCjIFCSaXeAIjKwXmoMZmVWvhHVtP5MUytUtob+Wt7zCYK5Xz0EZyrKfKZaVzGxtlOn41NlQkmQZB1onijHlaemXjXkRTIJrlVmCSUlurR6Y4xUTb8WzL0q4qaYf8ak+rmqkoT0fGWkZSte3MExARwUk51raGhVZ0CW80FfnYlDFCalBF69irZyrTaCnZBrCFs8QKm1jAnEjOsYhMcFqYoNwThua0cBiZ/YWxIS5c1ODChUNSQYKZGbq0V6c9ly5rH166MK/B3IWzGpy5cFCDAxfOa3DuwrAGwxK+B/uTpvDJuj4nkGItUxN87BXGrq30mQkmGQ+BKXu6hbH6n3HriPLJ8cJt/j+9J1NaK7N8LQp7D7vurdsU/eed1x3/y4ujk7fVhdxDh+gxeoq66CU6QR/QGeojgiboO/qBfjZ+Nx82D5uP1qG7O9WeB+iKNZ/8Aa7zj1A= < /latexit > < latexit sha1_base64= '' 556K3KfvISjTw9nnZ+SMnXd6wz0= '' > AAAEbHicfZPritNAFMdnt1XXettV/LQIwWVFpJRUBPWDsKgLghVX3NpCU5bJ5KQddi4hM2mbDnkEv+qz+RS+gpM2QnZaPTDhn/M7k3OZTJgwqrTv/9rZbTSvXb+xd7N16/adu/f2D+5/UzJLCfSJZDIdhlgBowL6mmoGwyQFzEMGg/DyXckHM0gVleJc5wmMOZ4IGlOCtXV9DRazi/0jv+OvzNsU3UococrOLg4a50EkScZBaMKwUqOun+ixwammhEHRCjIFCSaXeAIjKwXmoMZmVWvhHVtP5MUytUtob+Wt7zCYK5Xz0EZyrKfKZaVzGxtlOn41NlQkmQZB1onijHlaemXjXkRTIJrlVmCSUlurR6Y4xUTb8WzL0q4qaYf8ak+rmqkoT0fGWkZSte3MExARwUk51raGhVZ0CW80FfnYlDFCalBF69irZyrTaCnZBrCFs8QKm1jAnEjOsYhMcFqYoNwThua0cBiZ/YWxIS5c1ODChUNSQYKZGbq0V6c9ly5rH166MK/B3IWzGpy5cFCDAxfOa3DuwrAGwxK+B/uTpvDJuj4nkGItUxN87BXGrq30mQkmGQ+BKXu6hbH6n3HriPLJ8cJt/j+9J1NaK7N8LQp7D7vurdsU/eed1x3/y4ujk7fVhdxDh+gxeoq66CU6QR/QGeojgiboO/qBfjZ+Nx82D5uP1qG7O9WeB+iKNZ/8Aa7zj1A= < /latexit > z2 < latexit sha1_base64= '' 140MI4KVtB7/uRpqHUXODrPUTWQ= '' > AAAEbnicfZNbi9NAFMdnt1XXetvVBx9EDJaFRUpJF0F9EBZ1QbDiCltbaEqZTE7aYecSMpPehnwGX/Wj+TH8Bk7aCNlp9cCEf87vTM5lMmHCqNK+/2tvv1a/cfPWwe3Gnbv37j84PHr4TcksJdAjksl0EGIFjAroaaoZDJIUMA8Z9MOr9wXvzyBVVIpLvUxgxPFE0JgSrK2rF6xm49PxYdNv+2vztkWnFE1U2sX4qHYZRJJkHIQmDCs17PiJHhmcakoY5I0gU5BgcoUnMLRSYA5qZNbV5t6x9UReLFO7hPbW3uoOg7lSSx7aSI71VLmscO5iw0zHr0eGiiTTIMgmUZwxT0uvaN2LaApEs6UVmKTU1uqRKU4x0XZAu7K0ykpaIb/e07pmKorzkbGWkVQtO/UERERwUgy2pWGhFV3BW03FcmSKGCE1qLxx7FUzFWm0lGwL2MJZYoVNLGBOJOdYRCY4z01Q7AlDc547jMz+wtgQFy4qcOHCASkhwcwMXNqt0q5LV5UPr1y4rMClC2cVOHNhvwL7LpxX4NyFYQWGBfwA9idN4bN1fUkgxVqmJvjUzY1dO+kLE0wyHgJT9nRzY/U/4zYRxZPjhdv8f3pPprRSZvGa5/Yedtxbty16p+03bf/ry+bZu/JCHqAn6Dk6QR30Cp2hj+gC9RBBFH1HP9DP2u/64/rT+rNN6P5euecRumb1kz9QYY/3 < /latexit > < latexit sha1_base64= '' 140MI4KVtB7/uRpqHUXODrPUTWQ= '' > AAAEbnicfZNbi9NAFMdnt1XXetvVBx9EDJaFRUpJF0F9EBZ1QbDiCltbaEqZTE7aYecSMpPehnwGX/Wj+TH8Bk7aCNlp9cCEf87vTM5lMmHCqNK+/2tvv1a/cfPWwe3Gnbv37j84PHr4TcksJdAjksl0EGIFjAroaaoZDJIUMA8Z9MOr9wXvzyBVVIpLvUxgxPFE0JgSrK2rF6xm49PxYdNv+2vztkWnFE1U2sX4qHYZRJJkHIQmDCs17PiJHhmcakoY5I0gU5BgcoUnMLRSYA5qZNbV5t6x9UReLFO7hPbW3uoOg7lSSx7aSI71VLmscO5iw0zHr0eGiiTTIMgmUZwxT0uvaN2LaApEs6UVmKTU1uqRKU4x0XZAu7K0ykpaIb/e07pmKorzkbGWkVQtO/UERERwUgy2pWGhFV3BW03FcmSKGCE1qLxx7FUzFWm0lGwL2MJZYoVNLGBOJOdYRCY4z01Q7AlDc547jMz+wtgQFy4qcOHCASkhwcwMXNqt0q5LV5UPr1y4rMClC2cVOHNhvwL7LpxX4NyFYQWGBfwA9idN4bN1fUkgxVqmJvjUzY1dO+kLE0wyHgJT9nRzY/U/4zYRxZPjhdv8f3pPprRSZvGa5/Yedtxbty16p+03bf/ry+bZu/JCHqAn6Dk6QR30Cp2hj+gC9RBBFH1HP9DP2u/64/rT+rNN6P5euecRumb1kz9QYY/3 < /latexit > < latexit sha1_base64= '' 140MI4KVtB7/uRpqHUXODrPUTWQ= '' > AAAEbnicfZNbi9NAFMdnt1XXetvVBx9EDJaFRUpJF0F9EBZ1QbDiCltbaEqZTE7aYecSMpPehnwGX/Wj+TH8Bk7aCNlp9cCEf87vTM5lMmHCqNK+/2tvv1a/cfPWwe3Gnbv37j84PHr4TcksJdAjksl0EGIFjAroaaoZDJIUMA8Z9MOr9wXvzyBVVIpLvUxgxPFE0JgSrK2rF6xm49PxYdNv+2vztkWnFE1U2sX4qHYZRJJkHIQmDCs17PiJHhmcakoY5I0gU5BgcoUnMLRSYA5qZNbV5t6x9UReLFO7hPbW3uoOg7lSSx7aSI71VLmscO5iw0zHr0eGiiTTIMgmUZwxT0uvaN2LaApEs6UVmKTU1uqRKU4x0XZAu7K0ykpaIb/e07pmKorzkbGWkVQtO/UERERwUgy2pWGhFV3BW03FcmSKGCE1qLxx7FUzFWm0lGwL2MJZYoVNLGBOJOdYRCY4z01Q7AlDc547jMz+wtgQFy4qcOHCASkhwcwMXNqt0q5LV5UPr1y4rMClC2cVOHNhvwL7LpxX4NyFYQWGBfwA9idN4bN1fUkgxVqmJvjUzY1dO+kLE0wyHgJT9nRzY/U/4zYRxZPjhdv8f3pPprRSZvGa5/Yedtxbty16p+03bf/ry+bZu/JCHqAn6Dk6QR30Cp2hj+gC9RBBFH1HP9DP2u/64/rT+rNN6P5euecRumb1kz9QYY/3 < /latexit > z1 < latexit sha1_base64= '' mep7itZ0m4hEPojuKMEVkDGOogU= '' > AAAEbnicfZPdihMxFMez26pr/dhdvfBCxMGysEgpUxHUC2FRFwQrrrC1hU4pmcxpGzaTDEmm7TTMM3irj+Zj+AZm2hFm0+qBDP85v5M5H5mECaNK+/6vvf1a/cbNWwe3G3fu3rt/eHT84JsSqSTQI4IJOQixAkY59DTVDAaJBByHDPrh1fuC9+cgFRX8UmcJjGI85XRCCdbW1QtW83FnfNT02/7avG3RKUUTlXYxPq5dBpEgaQxcE4aVGnb8RI8MlpoSBnkjSBUkmFzhKQyt5DgGNTLranPvxHoibyKkXVx7a291h8GxUlkc2sgY65lyWeHcxYapnrweGcqTVAMnm0STlHlaeEXrXkQlEM0yKzCR1NbqkRmWmGg7oF1ZWmUlrTC+3tO6ZsqL8xETLSKhWnbqCfCI4KQYbEvDUiu6grea8mxkihguNKi8ceJVMxVptBBsC9jCWWKFTcxhQUQcYx6Z4Dw3QbEnDM157jAy/wsnhrhwWYFLFw5ICQlmZuDSbpV2XbqqfHjlwqwCMxfOK3Duwn4F9l24qMCFC8MKDAv4AexPKuGzdX1JQGItpAk+dXNj10763ATTNA6BKXu6ubH6n3GbiOIZ46Xb/H96T2a0Umbxmuf2HnbcW7ctei/ab9r+15fNs3flhTxAj9EzdIo66BU6Qx/RBeohgij6jn6gn7Xf9Uf1J/Wnm9D9vXLPQ3TN6qd/AExbj/Y= < /latexit > < latexit sha1_base64= '' mep7itZ0m4hEPojuKMEVkDGOogU= '' > AAAEbnicfZPdihMxFMez26pr/dhdvfBCxMGysEgpUxHUC2FRFwQrrrC1hU4pmcxpGzaTDEmm7TTMM3irj+Zj+AZm2hFm0+qBDP85v5M5H5mECaNK+/6vvf1a/cbNWwe3G3fu3rt/eHT84JsSqSTQI4IJOQixAkY59DTVDAaJBByHDPrh1fuC9+cgFRX8UmcJjGI85XRCCdbW1QtW83FnfNT02/7avG3RKUUTlXYxPq5dBpEgaQxcE4aVGnb8RI8MlpoSBnkjSBUkmFzhKQyt5DgGNTLranPvxHoibyKkXVx7a291h8GxUlkc2sgY65lyWeHcxYapnrweGcqTVAMnm0STlHlaeEXrXkQlEM0yKzCR1NbqkRmWmGg7oF1ZWmUlrTC+3tO6ZsqL8xETLSKhWnbqCfCI4KQYbEvDUiu6grea8mxkihguNKi8ceJVMxVptBBsC9jCWWKFTcxhQUQcYx6Z4Dw3QbEnDM157jAy/wsnhrhwWYFLFw5ICQlmZuDSbpV2XbqqfHjlwqwCMxfOK3Duwn4F9l24qMCFC8MKDAv4AexPKuGzdX1JQGItpAk+dXNj10763ATTNA6BKXu6ubH6n3GbiOIZ46Xb/H96T2a0Umbxmuf2HnbcW7ctei/ab9r+15fNs3flhTxAj9EzdIo66BU6Qx/RBeohgij6jn6gn7Xf9Uf1J/Wnm9D9vXLPQ3TN6qd/AExbj/Y= < /latexit > < latexit sha1_base64= '' mep7itZ0m4hEPojuKMEVkDGOogU= '' > AAAEbnicfZPdihMxFMez26pr/dhdvfBCxMGysEgpUxHUC2FRFwQrrrC1hU4pmcxpGzaTDEmm7TTMM3irj+Zj+AZm2hFm0+qBDP85v5M5H5mECaNK+/6vvf1a/cbNWwe3G3fu3rt/eHT84JsSqSTQI4IJOQixAkY59DTVDAaJBByHDPrh1fuC9+cgFRX8UmcJjGI85XRCCdbW1QtW83FnfNT02/7avG3RKUUTlXYxPq5dBpEgaQxcE4aVGnb8RI8MlpoSBnkjSBUkmFzhKQyt5DgGNTLranPvxHoibyKkXVx7a291h8GxUlkc2sgY65lyWeHcxYapnrweGcqTVAMnm0STlHlaeEXrXkQlEM0yKzCR1NbqkRmWmGg7oF1ZWmUlrTC+3tO6ZsqL8xETLSKhWnbqCfCI4KQYbEvDUiu6grea8mxkihguNKi8ceJVMxVptBBsC9jCWWKFTcxhQUQcYx6Z4Dw3QbEnDM157jAy/wsnhrhwWYFLFw5ICQlmZuDSbpV2XbqqfHjlwqwCMxfOK3Duwn4F9l24qMCFC8MKDAv4AexPKuGzdX1JQGItpAk+dXNj10763ATTNA6BKXu6ubH6n3GbiOIZ46Xb/H96T2a0Umbxmuf2HnbcW7ctei/ab9r+15fNs3flhTxAj9EzdIo66BU6Qx/RBeohgij6jn6gn7Xf9Uf1J/Wnm9D9vXLPQ3TN6qd/AExbj/Y= < /latexit > y < latexit sha1_base64= '' 4cGYRFkWcBo9hzyw+N7dsLcD1Cg= '' > AAAEbHicfZPritNAFMdnt1XXettV/LQIwWVFpJRUBPWDsKgLghVX3NpCU5bJ5KQddi4hM2mbDnkEv+qz+RS+gpM2QnZaPTDhn/M7k3OZTJgwqrTv/9rZbTSvXb+xd7N16/adu/f2D+5/UzJLCfSJZDIdhlgBowL6mmoGwyQFzEMGg/DyXckHM0gVleJc5wmMOZ4IGlOCtXV9DfLZxf6R3/FX5m2KbiWOUGVnFweN8yCSJOMgNGFYqVHXT/TY4FRTwqBoBZmCBJNLPIGRlQJzUGOzqrXwjq0n8mKZ2iW0t/LWdxjMlcp5aCM51lPlstK5jY0yHb8aGyqSTIMg60RxxjwtvbJxL6IpEM1yKzBJqa3VI1OcYqLteLZlaVeVtEN+tadVzVSUpyNjLSOp2nbmCYiI4KQca1vDQiu6hDeainxsyhghNaiidezVM5VptJRsA9jCWWKFTSxgTiTnWEQmOC1MUO4JQ3NaOIzM/sLYEBcuanDhwiGpIMHMDF3aq9OeS5e1Dy9dmNdg7sJZDc5cOKjBgQvnNTh3YViDYQnfg/1JU/hkXZ8TSLGWqQk+9gpj11b6zASTjIfAlD3dwlj9z7h1RPnkeOE2/5/ekymtlVm+FoW9h1331m2K/vPO647/5cXRydvqQu6hQ/QYPUVd9BKdoA/oDPURQRP0Hf1APxu/mw+bh81H69DdnWrPA3TFmk/+ALL6j1E= < /latexit > < latexit sha1_base64= '' 4cGYRFkWcBo9hzyw+N7dsLcD1Cg= '' > AAAEbHicfZPritNAFMdnt1XXettV/LQIwWVFpJRUBPWDsKgLghVX3NpCU5bJ5KQddi4hM2mbDnkEv+qz+RS+gpM2QnZaPTDhn/M7k3OZTJgwqrTv/9rZbTSvXb+xd7N16/adu/f2D+5/UzJLCfSJZDIdhlgBowL6mmoGwyQFzEMGg/DyXckHM0gVleJc5wmMOZ4IGlOCtXV9DfLZxf6R3/FX5m2KbiWOUGVnFweN8yCSJOMgNGFYqVHXT/TY4FRTwqBoBZmCBJNLPIGRlQJzUGOzqrXwjq0n8mKZ2iW0t/LWdxjMlcp5aCM51lPlstK5jY0yHb8aGyqSTIMg60RxxjwtvbJxL6IpEM1yKzBJqa3VI1OcYqLteLZlaVeVtEN+tadVzVSUpyNjLSOp2nbmCYiI4KQca1vDQiu6hDeainxsyhghNaiidezVM5VptJRsA9jCWWKFTSxgTiTnWEQmOC1MUO4JQ3NaOIzM/sLYEBcuanDhwiGpIMHMDF3aq9OeS5e1Dy9dmNdg7sJZDc5cOKjBgQvnNTh3YViDYQnfg/1JU/hkXZ8TSLGWqQk+9gpj11b6zASTjIfAlD3dwlj9z7h1RPnkeOE2/5/ekymtlVm+FoW9h1331m2K/vPO647/5cXRydvqQu6hQ/QYPUVd9BKdoA/oDPURQRP0Hf1APxu/mw+bh81H69DdnWrPA3TFmk/+ALL6j1E= < /latexit > < latexit sha1_base64= '' 4cGYRFkWcBo9hzyw+N7dsLcD1Cg= '' > AAAEbHicfZPritNAFMdnt1XXettV/LQIwWVFpJRUBPWDsKgLghVX3NpCU5bJ5KQddi4hM2mbDnkEv+qz+RS+gpM2QnZaPTDhn/M7k3OZTJgwqrTv/9rZbTSvXb+xd7N16/adu/f2D+5/UzJLCfSJZDIdhlgBowL6mmoGwyQFzEMGg/DyXckHM0gVleJc5wmMOZ4IGlOCtXV9DfLZxf6R3/FX5m2KbiWOUGVnFweN8yCSJOMgNGFYqVHXT/TY4FRTwqBoBZmCBJNLPIGRlQJzUGOzqrXwjq0n8mKZ2iW0t/LWdxjMlcp5aCM51lPlstK5jY0yHb8aGyqSTIMg60RxxjwtvbJxL6IpEM1yKzBJqa3VI1OcYqLteLZlaVeVtEN+tadVzVSUpyNjLSOp2nbmCYiI4KQca1vDQiu6hDeainxsyhghNaiidezVM5VptJRsA9jCWWKFTSxgTiTnWEQmOC1MUO4JQ3NaOIzM/sLYEBcuanDhwiGpIMHMDF3aq9OeS5e1Dy9dmNdg7sJZDc5cOKjBgQvnNTh3YViDYQnfg/1JU/hkXZ8TSLGWqQk+9gpj11b6zASTjIfAlD3dwlj9z7h1RPnkeOE2/5/ekymtlVm+FoW9h1331m2K/vPO647/5cXRydvqQu6hQ/QYPUVd9BKdoA/oDPURQRP0Hf1APxu/mw+bh81H69DdnWrPA3TFmk/+ALL6j1E= < /latexit > p ( z|x ) < latexit sha1_base64= '' erCiicMLVJdqs4W7M4MLmg5TJrU= '' > AAAFwXiclZTdTtswFMcNoxvrPoDtcjfWKjSYoGoqpCFNSEgUadJ6wSZKC01UOYnTWHXszHYorcmLbLfbQ+1t5oR2JEG7mKVIx+f3P+f4Iz5uTIlUrdbvldVHa7XHT9af1p89f/FyY3Pr1YXkifBwz+OUi4GLJKaE4Z4iiuJBLDCKXIr77uQk4/1rLCTh7FzNYuxEaMxIQDykjGu0uRHvzG/tCKnQDfRNujvabLSarXzAh4a1MBpgMc5GW2vfbZ97SYSZ8iiScmi1YuVoJBTxKE7rdiJxjLwJGuOhMRmKsHR0vvIUbhuPDwMuzMcUzL3lCBUcOpqwOFGYeSYAIkr5FBr//iHM/SX9ueXoLFUuNmMbGggP912i4LmVV5HFAB2aQxECB7k6D8g95jwnZWEi6FKzEEoSxRTD3tcuzI5WYqUIG5eCXM4nCrnyPnsseIBldh+I7n9LECVqBo2E4nI5FMnsVtKqM9/BfTqXGuRyJHyYyaGcRS6n5VRu9H/6iHiCZztKl1FLDx8LFIezkjpIKI2N8W8xw1OPRxFivrYHXqrz381DVA/StEy7Rdqt0k6Rdqr05npB8x+5AgcF+KDqZQFeVuGsAGdVqMICNTOsUFUyLyjmVXhVgFdVOC3AaRWeukvo6lMD69uwgxjB9J2EyPdJ9rrLt6rIZF5J4vuLJCLSfrXChKbDtqM7o79H/rmb2hQHCg5hw4K3t7DRhrYg41BBpxLcL6y9X8783vaIMJ3BH5qnamerGi4b2NGOFyLRzKa7jq4XX5vNuI+HMkQxPrqL3/MFmu4RxrCAZpdHbdN1YJ5gF+qGlX40ZU03s6q966Fx0W5arab15aBxfLDoa+vgDXgLdoAFPoBj8AmcgR7wQAJ+gJ/gV+2kRmpxTdxJV1cWMa9BadT0H/PW+Kw= < /latexit > < latexit sha1_base64= '' erCiicMLVJdqs4W7M4MLmg5TJrU= '' > AAAFwXiclZTdTtswFMcNoxvrPoDtcjfWKjSYoGoqpCFNSEgUadJ6wSZKC01UOYnTWHXszHYorcmLbLfbQ+1t5oR2JEG7mKVIx+f3P+f4Iz5uTIlUrdbvldVHa7XHT9af1p89f/FyY3Pr1YXkifBwz+OUi4GLJKaE4Z4iiuJBLDCKXIr77uQk4/1rLCTh7FzNYuxEaMxIQDykjGu0uRHvzG/tCKnQDfRNujvabLSarXzAh4a1MBpgMc5GW2vfbZ97SYSZ8iiScmi1YuVoJBTxKE7rdiJxjLwJGuOhMRmKsHR0vvIUbhuPDwMuzMcUzL3lCBUcOpqwOFGYeSYAIkr5FBr//iHM/SX9ueXoLFUuNmMbGggP912i4LmVV5HFAB2aQxECB7k6D8g95jwnZWEi6FKzEEoSxRTD3tcuzI5WYqUIG5eCXM4nCrnyPnsseIBldh+I7n9LECVqBo2E4nI5FMnsVtKqM9/BfTqXGuRyJHyYyaGcRS6n5VRu9H/6iHiCZztKl1FLDx8LFIezkjpIKI2N8W8xw1OPRxFivrYHXqrz381DVA/StEy7Rdqt0k6Rdqr05npB8x+5AgcF+KDqZQFeVuGsAGdVqMICNTOsUFUyLyjmVXhVgFdVOC3AaRWeukvo6lMD69uwgxjB9J2EyPdJ9rrLt6rIZF5J4vuLJCLSfrXChKbDtqM7o79H/rmb2hQHCg5hw4K3t7DRhrYg41BBpxLcL6y9X8783vaIMJ3BH5qnamerGi4b2NGOFyLRzKa7jq4XX5vNuI+HMkQxPrqL3/MFmu4RxrCAZpdHbdN1YJ5gF+qGlX40ZU03s6q966Fx0W5arab15aBxfLDoa+vgDXgLdoAFPoBj8AmcgR7wQAJ+gJ/gV+2kRmpxTdxJV1cWMa9BadT0H/PW+Kw= < /latexit > < latexit sha1_base64= '' erCiicMLVJdqs4W7M4MLmg5TJrU= '' > AAAFwXiclZTdTtswFMcNoxvrPoDtcjfWKjSYoGoqpCFNSEgUadJ6wSZKC01UOYnTWHXszHYorcmLbLfbQ+1t5oR2JEG7mKVIx+f3P+f4Iz5uTIlUrdbvldVHa7XHT9af1p89f/FyY3Pr1YXkifBwz+OUi4GLJKaE4Z4iiuJBLDCKXIr77uQk4/1rLCTh7FzNYuxEaMxIQDykjGu0uRHvzG/tCKnQDfRNujvabLSarXzAh4a1MBpgMc5GW2vfbZ97SYSZ8iiScmi1YuVoJBTxKE7rdiJxjLwJGuOhMRmKsHR0vvIUbhuPDwMuzMcUzL3lCBUcOpqwOFGYeSYAIkr5FBr//iHM/SX9ueXoLFUuNmMbGggP912i4LmVV5HFAB2aQxECB7k6D8g95jwnZWEi6FKzEEoSxRTD3tcuzI5WYqUIG5eCXM4nCrnyPnsseIBldh+I7n9LECVqBo2E4nI5FMnsVtKqM9/BfTqXGuRyJHyYyaGcRS6n5VRu9H/6iHiCZztKl1FLDx8LFIezkjpIKI2N8W8xw1OPRxFivrYHXqrz381DVA/StEy7Rdqt0k6Rdqr05npB8x+5AgcF+KDqZQFeVuGsAGdVqMICNTOsUFUyLyjmVXhVgFdVOC3AaRWeukvo6lMD69uwgxjB9J2EyPdJ9rrLt6rIZF5J4vuLJCLSfrXChKbDtqM7o79H/rmb2hQHCg5hw4K3t7DRhrYg41BBpxLcL6y9X8783vaIMJ3BH5qnamerGi4b2NGOFyLRzKa7jq4XX5vNuI+HMkQxPrqL3/MFmu4RxrCAZpdHbdN1YJ5gF+qGlX40ZU03s6q966Fx0W5arab15aBxfLDoa+vgDXgLdoAFPoBj8AmcgR7wQAJ+gJ/gV+2kRmpxTdxJV1cWMa9BadT0H/PW+Kw= < /latexit > < latexit sha1_base64= '' erCiicMLVJdqs4W7M4MLmg5TJrU= '' > AAAFwXiclZTdTtswFMcNoxvrPoDtcjfWKjSYoGoqpCFNSEgUadJ6wSZKC01UOYnTWHXszHYorcmLbLfbQ+1t5oR2JEG7mKVIx+f3P+f4Iz5uTIlUrdbvldVHa7XHT9af1p89f/FyY3Pr1YXkifBwz+OUi4GLJKaE4Z4iiuJBLDCKXIr77uQk4/1rLCTh7FzNYuxEaMxIQDykjGu0uRHvzG/tCKnQDfRNujvabLSarXzAh4a1MBpgMc5GW2vfbZ97SYSZ8iiScmi1YuVoJBTxKE7rdiJxjLwJGuOhMRmKsHR0vvIUbhuPDwMuzMcUzL3lCBUcOpqwOFGYeSYAIkr5FBr//iHM/SX9ueXoLFUuNmMbGggP912i4LmVV5HFAB2aQxECB7k6D8g95jwnZWEi6FKzEEoSxRTD3tcuzI5WYqUIG5eCXM4nCrnyPnsseIBldh+I7n9LECVqBo2E4nI5FMnsVtKqM9/BfTqXGuRyJHyYyaGcRS6n5VRu9H/6iHiCZztKl1FLDx8LFIezkjpIKI2N8W8xw1OPRxFivrYHXqrz381DVA/StEy7Rdqt0k6Rdqr05npB8x+5AgcF+KDqZQFeVuGsAGdVqMICNTOsUFUyLyjmVXhVgFdVOC3AaRWeukvo6lMD69uwgxjB9J2EyPdJ9rrLt6rIZF5J4vuLJCLSfrXChKbDtqM7o79H/rmb2hQHCg5hw4K3t7DRhrYg41BBpxLcL6y9X8783vaIMJ3BH5qnamerGi4b2NGOFyLRzKa7jq4XX5vNuI+HMkQxPrqL3/MFmu4RxrCAZpdHbdN1YJ5gF+qGlX40ZU03s6q966Fx0W5arab15aBxfLDoa+vgDXgLdoAFPoBj8AmcgR7wQAJ+gJ/gV+2kRmpxTdxJV1cWMa9BadT0H/PW+Kw= < /latexit > z̃ < latexit sha1_base64= '' BQ1r9a4mLEbJhupKJFh6aml5+SM= '' > AAAFuXiclZTfT9swEMcNoxtjv2B73Iu1Cg0mqJoKaUgICWlFmrQ+sInSQhMhx3Ear46d2S6lNfkr9jZtf9j+mzmhHUnQHnZSpPN9vndnO/L5CaNKN5u/l5YfrNQePlp9vPbk6bPnL9Y3Xp4pMZaYdLFgQvZ9pAijnHQ11Yz0E0lQ7DPS80cfMt67IlJRwU/1NCFejIachhQjbUPnrqYsIGaWXq7Xm41mbvC+48ydOpjbyeXGyg83EHgcE64xQ0oNnGaiPYOkppiRdM0dK5IgPEJDMrAuRzFRnsl3nMJNGwlgKKT9uIZ5tJyhw33PUJ6MNeHYJkDEmJhAG9/dh3m8pD91PJOVysXWNqGFcH/XpxqeOnkXVUwwkb0MKUmYq/OEPGLvcVQWjiVbaOZCReOEEdj90oHZlSqiNeXDUpIvxEgjX91VT6QIicr+A2K738aIUT2FVsJIuR2KVYx0lFaD+QnuyvnMIl8gGcBMDtU09gUrl/Lj/9PHFEuRnShdZC0iYihREk1L6nDMWGKdf4s5mWARx4gHxu3j1LhZZ4yY6adpmXaKtFOl7SJtV+n11Zz6obmuwn4B3ut6XoDnVTgtwGkV6qhA7YpoVJXMCopZFV4U4EUVTgpwUoXH/gL65tjCtU3YRpwS9lZBFAQ0e9Xlv6rpaFYpEgTzIjI2QbXDiKWDlmfal3+v/FMndRkJNRzAugNvbmC9BV1Jh5GGXiW5V9h7r1z5nYuptJMhGNin6ma7GiwG1+EWjpBsZMttz6wVX5vLRUAGKkIJObzN3wkkmuxQzomE9pSHLTt1YF5gG5q6kx7YtnaaOdXZdd85azWcZsP5vFc/2pvPtVXwGrwBW8AB78ER+AhOQBdgEIPv4Cf4VTuooVpU+3orXV6a57wCJaupP+1S9io= < /latexit > < latexit sha1_base64= '' BQ1r9a4mLEbJhupKJFh6aml5+SM= '' > AAAFuXiclZTfT9swEMcNoxtjv2B73Iu1Cg0mqJoKaUgICWlFmrQ+sInSQhMhx3Ear46d2S6lNfkr9jZtf9j+mzmhHUnQHnZSpPN9vndnO/L5CaNKN5u/l5YfrNQePlp9vPbk6bPnL9Y3Xp4pMZaYdLFgQvZ9pAijnHQ11Yz0E0lQ7DPS80cfMt67IlJRwU/1NCFejIachhQjbUPnrqYsIGaWXq7Xm41mbvC+48ydOpjbyeXGyg83EHgcE64xQ0oNnGaiPYOkppiRdM0dK5IgPEJDMrAuRzFRnsl3nMJNGwlgKKT9uIZ5tJyhw33PUJ6MNeHYJkDEmJhAG9/dh3m8pD91PJOVysXWNqGFcH/XpxqeOnkXVUwwkb0MKUmYq/OEPGLvcVQWjiVbaOZCReOEEdj90oHZlSqiNeXDUpIvxEgjX91VT6QIicr+A2K738aIUT2FVsJIuR2KVYx0lFaD+QnuyvnMIl8gGcBMDtU09gUrl/Lj/9PHFEuRnShdZC0iYihREk1L6nDMWGKdf4s5mWARx4gHxu3j1LhZZ4yY6adpmXaKtFOl7SJtV+n11Zz6obmuwn4B3ut6XoDnVTgtwGkV6qhA7YpoVJXMCopZFV4U4EUVTgpwUoXH/gL65tjCtU3YRpwS9lZBFAQ0e9Xlv6rpaFYpEgTzIjI2QbXDiKWDlmfal3+v/FMndRkJNRzAugNvbmC9BV1Jh5GGXiW5V9h7r1z5nYuptJMhGNin6ma7GiwG1+EWjpBsZMttz6wVX5vLRUAGKkIJObzN3wkkmuxQzomE9pSHLTt1YF5gG5q6kx7YtnaaOdXZdd85azWcZsP5vFc/2pvPtVXwGrwBW8AB78ER+AhOQBdgEIPv4Cf4VTuooVpU+3orXV6a57wCJaupP+1S9io= < /latexit > < latexit sha1_base64= '' BQ1r9a4mLEbJhupKJFh6aml5+SM= '' > AAAFuXiclZTfT9swEMcNoxtjv2B73Iu1Cg0mqJoKaUgICWlFmrQ+sInSQhMhx3Ear46d2S6lNfkr9jZtf9j+mzmhHUnQHnZSpPN9vndnO/L5CaNKN5u/l5YfrNQePlp9vPbk6bPnL9Y3Xp4pMZaYdLFgQvZ9pAijnHQ11Yz0E0lQ7DPS80cfMt67IlJRwU/1NCFejIachhQjbUPnrqYsIGaWXq7Xm41mbvC+48ydOpjbyeXGyg83EHgcE64xQ0oNnGaiPYOkppiRdM0dK5IgPEJDMrAuRzFRnsl3nMJNGwlgKKT9uIZ5tJyhw33PUJ6MNeHYJkDEmJhAG9/dh3m8pD91PJOVysXWNqGFcH/XpxqeOnkXVUwwkb0MKUmYq/OEPGLvcVQWjiVbaOZCReOEEdj90oHZlSqiNeXDUpIvxEgjX91VT6QIicr+A2K738aIUT2FVsJIuR2KVYx0lFaD+QnuyvnMIl8gGcBMDtU09gUrl/Lj/9PHFEuRnShdZC0iYihREk1L6nDMWGKdf4s5mWARx4gHxu3j1LhZZ4yY6adpmXaKtFOl7SJtV+n11Zz6obmuwn4B3ut6XoDnVTgtwGkV6qhA7YpoVJXMCopZFV4U4EUVTgpwUoXH/gL65tjCtU3YRpwS9lZBFAQ0e9Xlv6rpaFYpEgTzIjI2QbXDiKWDlmfal3+v/FMndRkJNRzAugNvbmC9BV1Jh5GGXiW5V9h7r1z5nYuptJMhGNin6ma7GiwG1+EWjpBsZMttz6wVX5vLRUAGKkIJObzN3wkkmuxQzomE9pSHLTt1YF5gG5q6kx7YtnaaOdXZdd85azWcZsP5vFc/2pvPtVXwGrwBW8AB78ER+AhOQBdgEIPv4Cf4VTuooVpU+3orXV6a57wCJaupP+1S9io= < /latexit > < latexit sha1_base64= '' BQ1r9a4mLEbJhupKJFh6aml5+SM= '' > AAAFuXiclZTfT9swEMcNoxtjv2B73Iu1Cg0mqJoKaUgICWlFmrQ+sInSQhMhx3Ear46d2S6lNfkr9jZtf9j+mzmhHUnQHnZSpPN9vndnO/L5CaNKN5u/l5YfrNQePlp9vPbk6bPnL9Y3Xp4pMZaYdLFgQvZ9pAijnHQ11Yz0E0lQ7DPS80cfMt67IlJRwU/1NCFejIachhQjbUPnrqYsIGaWXq7Xm41mbvC+48ydOpjbyeXGyg83EHgcE64xQ0oNnGaiPYOkppiRdM0dK5IgPEJDMrAuRzFRnsl3nMJNGwlgKKT9uIZ5tJyhw33PUJ6MNeHYJkDEmJhAG9/dh3m8pD91PJOVysXWNqGFcH/XpxqeOnkXVUwwkb0MKUmYq/OEPGLvcVQWjiVbaOZCReOEEdj90oHZlSqiNeXDUpIvxEgjX91VT6QIicr+A2K738aIUT2FVsJIuR2KVYx0lFaD+QnuyvnMIl8gGcBMDtU09gUrl/Lj/9PHFEuRnShdZC0iYihREk1L6nDMWGKdf4s5mWARx4gHxu3j1LhZZ4yY6adpmXaKtFOl7SJtV+n11Zz6obmuwn4B3ut6XoDnVTgtwGkV6qhA7YpoVJXMCopZFV4U4EUVTgpwUoXH/gL65tjCtU3YRpwS9lZBFAQ0e9Xlv6rpaFYpEgTzIjI2QbXDiKWDlmfal3+v/FMndRkJNRzAugNvbmC9BV1Jh5GGXiW5V9h7r1z5nYuptJMhGNin6ma7GiwG1+EWjpBsZMttz6wVX5vLRUAGKkIJObzN3wkkmuxQzomE9pSHLTt1YF5gG5q6kx7YtnaaOdXZdd85azWcZsP5vFc/2pvPtVXwGrwBW8AB78ER+AhOQBdgEIPv4Cf4VTuooVpU+3orXV6a57wCJaupP+1S9io= < /latexit > Figure 1 : DLVMs have layers of stochastic latent variables . In high-risk domains , prediction errors come at high costs . Luckily such domains often provide a fail-safe : self-driving cars perform an emergency stop , doctors run another diagnostic test , and industrial processes are temporarily halted . For deep learning models , this can be achieved by rejecting datapoints with a confidence score below a predetermined threshold . This way , a low error rate can be guaranteed at the cost of rejecting some predictions . However , estimating high quality confidence scores from neural networks , which create well-ordered rankings of correct and incorrect predictions , remains an active area of research . Deep Latent Variable Models ( DLVMs , fig . 1 ) approach this by postulating latent variables z for which the uncertainty in p ( z|x ) influences the confidence in the target prediction . Recently , efficient inference algorithms have been proposed in the form of variational inference , where an inference neural network is optimized to predict parameters of a variational distribution that approximates an otherwise intractable distribution ( Kingma & Welling ( 2013 ) ; Rezende et al . ( 2014 ) ; Alemi et al . ( 2016 ) ; Achille & Soatto ( 2016 ) ) . p✓ < latexit sha1_base64= '' omhpCfmSDgPntJKR3V00EOH65fs= '' > AAAFuHiclZTfbtMwFMa9scIo/za45MaimtjQVjUVEhVo0gSdhEQvBlrXbk1UHMdprDp2sJ11XZaX4BJ4Md4GJ2tZkokLLFU6Pb/vO8d25ONGjCrdav1eWb2zVrt7b/1+/cHDR4+fbGw+PVEilpj0sWBCDl2kCKOc9DXVjAwjSVDoMjJwpx8yPjgnUlHBj/U8Ik6IJpz6FCNtUsNobOuAaDTeaLSarXzB24G1CBpgsY7Gm2s/bE/gOCRcY4aUGlmtSDsJkppiRtK6HSsSITxFEzIyIUchUU6SbziFWybjQV9I8+Ma5tmyQ/sdJ6E8ijXh2BggYkzMoMnvdWCeL+mPLSfJSuVis7aggbCz51INj628iyoaksDchZTEz9W5Ic+Ya5yWhbFkS81CqGgYMQL7X3owu1FFtKZ8UjK5Qkw1ctVN9UgKn6jsMyC29y1GjOo5NBJGyu1QqEKkg7SazE9wU85lBrkCSQ9mcqjmoStYuZQb/p8+pFiK7ETp0rXMiIlEUTAvqf2YscgE/xZzMsMiDBH3EnuI08TOOmPEkmGalmmvSHtV2i3SbpVenC+o6ycXVTgswFtdTwvwtArnBTivQh0U6PUDqkouC4rLKjwrwLMqnBXgrAoP3SV0k0MD61uwizgl7KWCyPNo9qjLX1XT6WWliOctisgw8aodpiwdtZ2kO/575Z96qc2Ir+EINix4dQUbbWhLOgk0dCrmQWHvg3LlVzam0kwGb2Seqp3tarScW/vbOECymf3dcZJ68bXZXHhkpAIUkf1r/64n0WyXck4kNKfcb5upA/MCOzBpWOk709ZMM6s6u24HJ+2m1Wpan183Dt4v5to6eA5egG1ggTfgAHwER6APMGDgO/gJftXe1r7WJjV6LV1dWXiegdKqyT96XPWP < /latexit > < latexit sha1_base64= '' omhpCfmSDgPntJKR3V00EOH65fs= '' > AAAFuHiclZTfbtMwFMa9scIo/za45MaimtjQVjUVEhVo0gSdhEQvBlrXbk1UHMdprDp2sJ11XZaX4BJ4Md4GJ2tZkokLLFU6Pb/vO8d25ONGjCrdav1eWb2zVrt7b/1+/cHDR4+fbGw+PVEilpj0sWBCDl2kCKOc9DXVjAwjSVDoMjJwpx8yPjgnUlHBj/U8Ik6IJpz6FCNtUsNobOuAaDTeaLSarXzB24G1CBpgsY7Gm2s/bE/gOCRcY4aUGlmtSDsJkppiRtK6HSsSITxFEzIyIUchUU6SbziFWybjQV9I8+Ma5tmyQ/sdJ6E8ijXh2BggYkzMoMnvdWCeL+mPLSfJSuVis7aggbCz51INj628iyoaksDchZTEz9W5Ic+Ya5yWhbFkS81CqGgYMQL7X3owu1FFtKZ8UjK5Qkw1ctVN9UgKn6jsMyC29y1GjOo5NBJGyu1QqEKkg7SazE9wU85lBrkCSQ9mcqjmoStYuZQb/p8+pFiK7ETp0rXMiIlEUTAvqf2YscgE/xZzMsMiDBH3EnuI08TOOmPEkmGalmmvSHtV2i3SbpVenC+o6ycXVTgswFtdTwvwtArnBTivQh0U6PUDqkouC4rLKjwrwLMqnBXgrAoP3SV0k0MD61uwizgl7KWCyPNo9qjLX1XT6WWliOctisgw8aodpiwdtZ2kO/575Z96qc2Ir+EINix4dQUbbWhLOgk0dCrmQWHvg3LlVzam0kwGb2Seqp3tarScW/vbOECymf3dcZJ68bXZXHhkpAIUkf1r/64n0WyXck4kNKfcb5upA/MCOzBpWOk709ZMM6s6u24HJ+2m1Wpan183Dt4v5to6eA5egG1ggTfgAHwER6APMGDgO/gJftXe1r7WJjV6LV1dWXiegdKqyT96XPWP < /latexit > < latexit sha1_base64= '' omhpCfmSDgPntJKR3V00EOH65fs= '' > AAAFuHiclZTfbtMwFMa9scIo/za45MaimtjQVjUVEhVo0gSdhEQvBlrXbk1UHMdprDp2sJ11XZaX4BJ4Md4GJ2tZkokLLFU6Pb/vO8d25ONGjCrdav1eWb2zVrt7b/1+/cHDR4+fbGw+PVEilpj0sWBCDl2kCKOc9DXVjAwjSVDoMjJwpx8yPjgnUlHBj/U8Ik6IJpz6FCNtUsNobOuAaDTeaLSarXzB24G1CBpgsY7Gm2s/bE/gOCRcY4aUGlmtSDsJkppiRtK6HSsSITxFEzIyIUchUU6SbziFWybjQV9I8+Ma5tmyQ/sdJ6E8ijXh2BggYkzMoMnvdWCeL+mPLSfJSuVis7aggbCz51INj628iyoaksDchZTEz9W5Ic+Ya5yWhbFkS81CqGgYMQL7X3owu1FFtKZ8UjK5Qkw1ctVN9UgKn6jsMyC29y1GjOo5NBJGyu1QqEKkg7SazE9wU85lBrkCSQ9mcqjmoStYuZQb/p8+pFiK7ETp0rXMiIlEUTAvqf2YscgE/xZzMsMiDBH3EnuI08TOOmPEkmGalmmvSHtV2i3SbpVenC+o6ycXVTgswFtdTwvwtArnBTivQh0U6PUDqkouC4rLKjwrwLMqnBXgrAoP3SV0k0MD61uwizgl7KWCyPNo9qjLX1XT6WWliOctisgw8aodpiwdtZ2kO/575Z96qc2Ir+EINix4dQUbbWhLOgk0dCrmQWHvg3LlVzam0kwGb2Seqp3tarScW/vbOECymf3dcZJ68bXZXHhkpAIUkf1r/64n0WyXck4kNKfcb5upA/MCOzBpWOk709ZMM6s6u24HJ+2m1Wpan183Dt4v5to6eA5egG1ggTfgAHwER6APMGDgO/gJftXe1r7WJjV6LV1dWXiegdKqyT96XPWP < /latexit > < latexit sha1_base64= '' omhpCfmSDgPntJKR3V00EOH65fs= '' > AAAFuHiclZTfbtMwFMa9scIo/za45MaimtjQVjUVEhVo0gSdhEQvBlrXbk1UHMdprDp2sJ11XZaX4BJ4Md4GJ2tZkokLLFU6Pb/vO8d25ONGjCrdav1eWb2zVrt7b/1+/cHDR4+fbGw+PVEilpj0sWBCDl2kCKOc9DXVjAwjSVDoMjJwpx8yPjgnUlHBj/U8Ik6IJpz6FCNtUsNobOuAaDTeaLSarXzB24G1CBpgsY7Gm2s/bE/gOCRcY4aUGlmtSDsJkppiRtK6HSsSITxFEzIyIUchUU6SbziFWybjQV9I8+Ma5tmyQ/sdJ6E8ijXh2BggYkzMoMnvdWCeL+mPLSfJSuVis7aggbCz51INj628iyoaksDchZTEz9W5Ic+Ya5yWhbFkS81CqGgYMQL7X3owu1FFtKZ8UjK5Qkw1ctVN9UgKn6jsMyC29y1GjOo5NBJGyu1QqEKkg7SazE9wU85lBrkCSQ9mcqjmoStYuZQb/p8+pFiK7ETp0rXMiIlEUTAvqf2YscgE/xZzMsMiDBH3EnuI08TOOmPEkmGalmmvSHtV2i3SbpVenC+o6ycXVTgswFtdTwvwtArnBTivQh0U6PUDqkouC4rLKjwrwLMqnBXgrAoP3SV0k0MD61uwizgl7KWCyPNo9qjLX1XT6WWliOctisgw8aodpiwdtZ2kO/575Z96qc2Ir+EINix4dQUbbWhLOgk0dCrmQWHvg3LlVzam0kwGb2Seqp3tarScW/vbOECymf3dcZJ68bXZXHhkpAIUkf1r/64n0WyXck4kNKfcb5upA/MCOzBpWOk709ZMM6s6u24HJ+2m1Wpan183Dt4v5to6eA5egG1ggTfgAHwER6APMGDgO/gJftXe1r7WJjV6LV1dWXiegdKqyT96XPWP < /latexit > q < latexit sha1_base64= '' 0IOt3cbERNCZOJ4B7qywM+JNpds= '' > AAAFtniclZTfb9MwEMc9WGGUXxs88mJRTWxoq5oKiUpo0gSdhEQfBlrXbk2oHMdpTB07s911XZb/gVf2n/Hf4GQpSzLxwEmRzvf53p3tyOdGjCrdav1euXd/tfbg4dqj+uMnT589X994cazETGLSx4IJOXSRIoxy0tdUMzKMJEGhy8jAnX5K+eCcSEUFP9KLiDghmnDqU4y0CR2fje0ooOP1RqvZygzedazcaYDcDscbq9e2J/AsJFxjhpQaWa1IOzGSmmJGkro9UyRCeIomZGRcjkKinDjbbgI3TcSDvpDm4xpm0XKG9jtOTHk004RjkwARY2IOTXy3A7N4SX9kOXFaKhMb24QGws6uSzU8srIuqpgQB+YmpCR+ps4Ssoi5xGlZOJNsqcmFioYRI7D/rQfT+1REa8onpSRXiKlGrrqtHknhE5X+BMR2z2aIUb2ARsJIuR0KVYh0kFSD2Qluy7nMIFcg6cFUDtUidAUrl3LD/9OHFEuRnihZZi0jYiJRFCxKan/GWGScf4s5mWMRhoh7sT3ESWynnTFi8TBJyrRXpL0q7RZpt0ovznPq+vFFFQ4L8E7XkwI8qcJFAS6qUAcFalZEo6rksqC4rMLTAjytwnkBzqvwwF1CNz4wsL4Ju4hTwt4oiDyPpk+6/Fc1nV5WinheXkSGsVftMGXJqO3E3fHfK//SS2xGfA1HsGHBqyvYaENb0kmgoVNJHhT2PihXfmtjKs1k8EbmqdrprkbLqbW3hQMkm+ly24nrxddmc+GRkQpQRPZu8nc8ieY7lHMioTnlXttMHZgV2IZxw0o+mLZmmlnV2XXXOW43rVbT+vqusf8xn2tr4BV4DbaABd6DffAZHII+wOAH+Al+getap/a9RmqTG+m9lTznJShZLfoDv470pw== < /latexit > < latexit sha1_base64= '' 0IOt3cbERNCZOJ4B7qywM+JNpds= '' > AAAFtniclZTfb9MwEMc9WGGUXxs88mJRTWxoq5oKiUpo0gSdhEQfBlrXbk2oHMdpTB07s911XZb/gVf2n/Hf4GQpSzLxwEmRzvf53p3tyOdGjCrdav1euXd/tfbg4dqj+uMnT589X994cazETGLSx4IJOXSRIoxy0tdUMzKMJEGhy8jAnX5K+eCcSEUFP9KLiDghmnDqU4y0CR2fje0ooOP1RqvZygzedazcaYDcDscbq9e2J/AsJFxjhpQaWa1IOzGSmmJGkro9UyRCeIomZGRcjkKinDjbbgI3TcSDvpDm4xpm0XKG9jtOTHk004RjkwARY2IOTXy3A7N4SX9kOXFaKhMb24QGws6uSzU8srIuqpgQB+YmpCR+ps4Ssoi5xGlZOJNsqcmFioYRI7D/rQfT+1REa8onpSRXiKlGrrqtHknhE5X+BMR2z2aIUb2ARsJIuR0KVYh0kFSD2Qluy7nMIFcg6cFUDtUidAUrl3LD/9OHFEuRnihZZi0jYiJRFCxKan/GWGScf4s5mWMRhoh7sT3ESWynnTFi8TBJyrRXpL0q7RZpt0ovznPq+vFFFQ4L8E7XkwI8qcJFAS6qUAcFalZEo6rksqC4rMLTAjytwnkBzqvwwF1CNz4wsL4Ju4hTwt4oiDyPpk+6/Fc1nV5WinheXkSGsVftMGXJqO3E3fHfK//SS2xGfA1HsGHBqyvYaENb0kmgoVNJHhT2PihXfmtjKs1k8EbmqdrprkbLqbW3hQMkm+ly24nrxddmc+GRkQpQRPZu8nc8ieY7lHMioTnlXttMHZgV2IZxw0o+mLZmmlnV2XXXOW43rVbT+vqusf8xn2tr4BV4DbaABd6DffAZHII+wOAH+Al+getap/a9RmqTG+m9lTznJShZLfoDv470pw== < /latexit > < latexit sha1_base64= '' 0IOt3cbERNCZOJ4B7qywM+JNpds= '' > AAAFtniclZTfb9MwEMc9WGGUXxs88mJRTWxoq5oKiUpo0gSdhEQfBlrXbk2oHMdpTB07s911XZb/gVf2n/Hf4GQpSzLxwEmRzvf53p3tyOdGjCrdav1euXd/tfbg4dqj+uMnT589X994cazETGLSx4IJOXSRIoxy0tdUMzKMJEGhy8jAnX5K+eCcSEUFP9KLiDghmnDqU4y0CR2fje0ooOP1RqvZygzedazcaYDcDscbq9e2J/AsJFxjhpQaWa1IOzGSmmJGkro9UyRCeIomZGRcjkKinDjbbgI3TcSDvpDm4xpm0XKG9jtOTHk004RjkwARY2IOTXy3A7N4SX9kOXFaKhMb24QGws6uSzU8srIuqpgQB+YmpCR+ps4Ssoi5xGlZOJNsqcmFioYRI7D/rQfT+1REa8onpSRXiKlGrrqtHknhE5X+BMR2z2aIUb2ARsJIuR0KVYh0kFSD2Qluy7nMIFcg6cFUDtUidAUrl3LD/9OHFEuRnihZZi0jYiJRFCxKan/GWGScf4s5mWMRhoh7sT3ESWynnTFi8TBJyrRXpL0q7RZpt0ovznPq+vFFFQ4L8E7XkwI8qcJFAS6qUAcFalZEo6rksqC4rMLTAjytwnkBzqvwwF1CNz4wsL4Ju4hTwt4oiDyPpk+6/Fc1nV5WinheXkSGsVftMGXJqO3E3fHfK//SS2xGfA1HsGHBqyvYaENb0kmgoVNJHhT2PihXfmtjKs1k8EbmqdrprkbLqbW3hQMkm+ly24nrxddmc+GRkQpQRPZu8nc8ieY7lHMioTnlXttMHZgV2IZxw0o+mLZmmlnV2XXXOW43rVbT+vqusf8xn2tr4BV4DbaABd6DffAZHII+wOAH+Al+getap/a9RmqTG+m9lTznJShZLfoDv470pw== < /latexit > < latexit sha1_base64= '' 0IOt3cbERNCZOJ4B7qywM+JNpds= '' > AAAFtniclZTfb9MwEMc9WGGUXxs88mJRTWxoq5oKiUpo0gSdhEQfBlrXbk2oHMdpTB07s911XZb/gVf2n/Hf4GQpSzLxwEmRzvf53p3tyOdGjCrdav1euXd/tfbg4dqj+uMnT589X994cazETGLSx4IJOXSRIoxy0tdUMzKMJEGhy8jAnX5K+eCcSEUFP9KLiDghmnDqU4y0CR2fje0ooOP1RqvZygzedazcaYDcDscbq9e2J/AsJFxjhpQaWa1IOzGSmmJGkro9UyRCeIomZGRcjkKinDjbbgI3TcSDvpDm4xpm0XKG9jtOTHk004RjkwARY2IOTXy3A7N4SX9kOXFaKhMb24QGws6uSzU8srIuqpgQB+YmpCR+ps4Ssoi5xGlZOJNsqcmFioYRI7D/rQfT+1REa8onpSRXiKlGrrqtHknhE5X+BMR2z2aIUb2ARsJIuR0KVYh0kFSD2Qluy7nMIFcg6cFUDtUidAUrl3LD/9OHFEuRnihZZi0jYiJRFCxKan/GWGScf4s5mWMRhoh7sT3ESWynnTFi8TBJyrRXpL0q7RZpt0ovznPq+vFFFQ4L8E7XkwI8qcJFAS6qUAcFalZEo6rksqC4rMLTAjytwnkBzqvwwF1CNz4wsL4Ju4hTwt4oiDyPpk+6/Fc1nV5WinheXkSGsVftMGXJqO3E3fHfK//SS2xGfA1HsGHBqyvYaENb0kmgoVNJHhT2PihXfmtjKs1k8EbmqdrprkbLqbW3hQMkm+ly24nrxddmc+GRkQpQRPZu8nc8ieY7lHMioTnlXttMHZgV2IZxw0o+mLZmmlnV2XXXOW43rVbT+vqusf8xn2tr4BV4DbaABd6DffAZHII+wOAH+Al+getap/a9RmqTG+m9lTznJShZLfoDv470pw== < /latexit > ✓⇤ < latexit sha1_base64= '' LzSxC62qKq7o4xaWWXmaVVswm34= '' > AAAFx3iclZTfbtMwFMY9WGGUfxuIK24sqolt2qqmmsQkNGkSnQSiFwOta7cmVI7jNFadONjOus7LBa/BHbfwRLwNTtayJBMXWIp0cn7f+U7syMeNGZWq1fq9dOfucu3e/ZUH9YePHj95urr27ETyRGDSw5xxMXCRJIxGpKeoYmQQC4JCl5G+O3mX8f45EZLy6FjNYuKEaBxRn2KkTGq0+sLOPbTLEpJCWwVEoS9bo9VGq9nKF7wdWPOgAebraLS2/N32OE5CEinMkJRDqxUrRyOhKGYkrduJJDHCEzQmQxNGKCTS0XnvFK6bjAd9LswTKZhnyxXK33M0jeJEkQibAogY41No8jt7MM+X9MeWozOrXGzWOjQQ7u24VMFjK+8iiwU6MEcjBPFzdV6QZ8ypTsrCRLCFZi6UNIwZgb3PXZgdsCRK0WhcKnI5nyjkyhv3WHCfyOyvILbzNUGMqhk0EkbK7VAoQ6SCtJrMd3Bj5zKDXI6EBzM5lLPQ5axs5Yb/pw8pFjzbUbqoWmT4WKA4mJXUfsJYbIJ/iyMyxTwMUeRpe4BTbWedMWJ6kKZl2i3SbpV2irRTpRfnc+r6+qIKBwV4q+tpAZ5W4awAZ1WoggK9vkJVyWVBcVmFZwV4VoXTApxW4aG7gK4+NLC+DjsoooS9lhB5Hs3uePmvKjq5rJh43txEhNqrdpiwdNh2dGf098g/dlObEV/BIWxY8OoKNtrQFnQcKOhUivuFb++XnbdsTIWZDN7QXFU7+6rhYoztb+AAiWb2uunoevG22RH3yFAGKCb71/XbnkDTbRpFRECzy/22mTowN9iEumGlb01bM82s6uy6HZy0m1araX3abRzszufaCngJXoENYIE34AC8B0egBzDQ4Af4CX7VPtR47bx2cS29szSveQ5Kq/btDwNo+yg= < /latexit > < latexit sha1_base64= '' LzSxC62qKq7o4xaWWXmaVVswm34= '' > AAAFx3iclZTfbtMwFMY9WGGUfxuIK24sqolt2qqmmsQkNGkSnQSiFwOta7cmVI7jNFadONjOus7LBa/BHbfwRLwNTtayJBMXWIp0cn7f+U7syMeNGZWq1fq9dOfucu3e/ZUH9YePHj95urr27ETyRGDSw5xxMXCRJIxGpKeoYmQQC4JCl5G+O3mX8f45EZLy6FjNYuKEaBxRn2KkTGq0+sLOPbTLEpJCWwVEoS9bo9VGq9nKF7wdWPOgAebraLS2/N32OE5CEinMkJRDqxUrRyOhKGYkrduJJDHCEzQmQxNGKCTS0XnvFK6bjAd9LswTKZhnyxXK33M0jeJEkQibAogY41No8jt7MM+X9MeWozOrXGzWOjQQ7u24VMFjK+8iiwU6MEcjBPFzdV6QZ8ypTsrCRLCFZi6UNIwZgb3PXZgdsCRK0WhcKnI5nyjkyhv3WHCfyOyvILbzNUGMqhk0EkbK7VAoQ6SCtJrMd3Bj5zKDXI6EBzM5lLPQ5axs5Yb/pw8pFjzbUbqoWmT4WKA4mJXUfsJYbIJ/iyMyxTwMUeRpe4BTbWedMWJ6kKZl2i3SbpV2irRTpRfnc+r6+qIKBwV4q+tpAZ5W4awAZ1WoggK9vkJVyWVBcVmFZwV4VoXTApxW4aG7gK4+NLC+DjsoooS9lhB5Hs3uePmvKjq5rJh43txEhNqrdpiwdNh2dGf098g/dlObEV/BIWxY8OoKNtrQFnQcKOhUivuFb++XnbdsTIWZDN7QXFU7+6rhYoztb+AAiWb2uunoevG22RH3yFAGKCb71/XbnkDTbRpFRECzy/22mTowN9iEumGlb01bM82s6uy6HZy0m1araX3abRzszufaCngJXoENYIE34AC8B0egBzDQ4Af4CX7VPtR47bx2cS29szSveQ5Kq/btDwNo+yg= < /latexit > < latexit sha1_base64= '' LzSxC62qKq7o4xaWWXmaVVswm34= '' > AAAFx3iclZTfbtMwFMY9WGGUfxuIK24sqolt2qqmmsQkNGkSnQSiFwOta7cmVI7jNFadONjOus7LBa/BHbfwRLwNTtayJBMXWIp0cn7f+U7syMeNGZWq1fq9dOfucu3e/ZUH9YePHj95urr27ETyRGDSw5xxMXCRJIxGpKeoYmQQC4JCl5G+O3mX8f45EZLy6FjNYuKEaBxRn2KkTGq0+sLOPbTLEpJCWwVEoS9bo9VGq9nKF7wdWPOgAebraLS2/N32OE5CEinMkJRDqxUrRyOhKGYkrduJJDHCEzQmQxNGKCTS0XnvFK6bjAd9LswTKZhnyxXK33M0jeJEkQibAogY41No8jt7MM+X9MeWozOrXGzWOjQQ7u24VMFjK+8iiwU6MEcjBPFzdV6QZ8ypTsrCRLCFZi6UNIwZgb3PXZgdsCRK0WhcKnI5nyjkyhv3WHCfyOyvILbzNUGMqhk0EkbK7VAoQ6SCtJrMd3Bj5zKDXI6EBzM5lLPQ5axs5Yb/pw8pFjzbUbqoWmT4WKA4mJXUfsJYbIJ/iyMyxTwMUeRpe4BTbWedMWJ6kKZl2i3SbpV2irRTpRfnc+r6+qIKBwV4q+tpAZ5W4awAZ1WoggK9vkJVyWVBcVmFZwV4VoXTApxW4aG7gK4+NLC+DjsoooS9lhB5Hs3uePmvKjq5rJh43txEhNqrdpiwdNh2dGf098g/dlObEV/BIWxY8OoKNtrQFnQcKOhUivuFb++XnbdsTIWZDN7QXFU7+6rhYoztb+AAiWb2uunoevG22RH3yFAGKCb71/XbnkDTbRpFRECzy/22mTowN9iEumGlb01bM82s6uy6HZy0m1araX3abRzszufaCngJXoENYIE34AC8B0egBzDQ4Af4CX7VPtR47bx2cS29szSveQ5Kq/btDwNo+yg= < /latexit > < latexit sha1_base64= '' LzSxC62qKq7o4xaWWXmaVVswm34= '' > AAAFx3iclZTfbtMwFMY9WGGUfxuIK24sqolt2qqmmsQkNGkSnQSiFwOta7cmVI7jNFadONjOus7LBa/BHbfwRLwNTtayJBMXWIp0cn7f+U7syMeNGZWq1fq9dOfucu3e/ZUH9YePHj95urr27ETyRGDSw5xxMXCRJIxGpKeoYmQQC4JCl5G+O3mX8f45EZLy6FjNYuKEaBxRn2KkTGq0+sLOPbTLEpJCWwVEoS9bo9VGq9nKF7wdWPOgAebraLS2/N32OE5CEinMkJRDqxUrRyOhKGYkrduJJDHCEzQmQxNGKCTS0XnvFK6bjAd9LswTKZhnyxXK33M0jeJEkQibAogY41No8jt7MM+X9MeWozOrXGzWOjQQ7u24VMFjK+8iiwU6MEcjBPFzdV6QZ8ypTsrCRLCFZi6UNIwZgb3PXZgdsCRK0WhcKnI5nyjkyhv3WHCfyOyvILbzNUGMqhk0EkbK7VAoQ6SCtJrMd3Bj5zKDXI6EBzM5lLPQ5axs5Yb/pw8pFjzbUbqoWmT4WKA4mJXUfsJYbIJ/iyMyxTwMUeRpe4BTbWedMWJ6kKZl2i3SbpV2irRTpRfnc+r6+qIKBwV4q+tpAZ5W4awAZ1WoggK9vkJVyWVBcVmFZwV4VoXTApxW4aG7gK4+NLC+DjsoooS9lhB5Hs3uePmvKjq5rJh43txEhNqrdpiwdNh2dGf098g/dlObEV/BIWxY8OoKNtrQFnQcKOhUivuFb++XnbdsTIWZDN7QXFU7+6rhYoztb+AAiWb2uunoevG22RH3yFAGKCb71/XbnkDTbRpFRECzy/22mTowN9iEumGlb01bM82s6uy6HZy0m1araX3abRzszufaCngJXoENYIE34AC8B0egBzDQ4Af4CX7VPtR47bx2cS29szSveQ5Kq/btDwNo+yg= < /latexit > ⇤ < latexit sha1_base64= '' hAnGZAtKFJW9dFFAVofaA1mIVlg= '' > AAAFxHiclZTfbtMwFMY9WGGUf9u45MaimtimrWqqSUxCkybRISR6MdC6dmtC5SROY9Wxg+2s67zwIFxwC6/E2+BkLUsycYGlSCfn933nxI583JgSqVqt30v37i/XHjxceVR//OTps+era+unkifCwz2PUy4GLpKYEoZ7iiiKB7HAKHIp7ruTdxnvX2AhCWcnahZjJ0JjRgLiIWVSo9V1O6+hBfZTaMch+bI9Wm20mq18wbuBNQ8aYL6OR2vL322fe0mEmfIoknJotWLlaCQU8ShO63YicYy8CRrjoQkZirB0dN44hRsm48OAC/MwBfNs2aGCfUcTFicKM88YIKKUT6HJ7+7DPF/Sn1iOzkrlYrM2oIFwf9clCp5YeRdZNOjQnIsQOMjVuSHPmCOdlIWJoAvNXChJFFMMe5+7MDtdiZUibFwyuZxPFHLlbfVY8ADL7Jcguvs1QZSoGTQSisvtUCQjpMK0msx3cFvOpQa5HAkfZnIoZ5HLabmUG/2fPiKe4NmO0oVrkeFjgeJwVlIHCaWxCf4tZnjq8ShCzNf2wEu1nXX2ENWDNC3TbpF2q7RTpJ0qvbyYUzfQl1U4KMA7Xc8K8KwKZwU4q0IVFqh5wwpVJVcFxVUVnhfgeRVOC3BahUfuArr6yMD6BuwgRjB9LSHyfZJd8PJfVWRyVSni+/MiItJ+tcOEpsO2ozujv0f+sZvaFAcKDmHDgtfXsNGGtiDjUEGnYu4Xvr1frrxte0SYyeAPzVW1s68aLmbYwaYXItHMXrccXS/eNptxHw9liGJ8cOPf8QWa7hDGsIBmlwdtM3VgXmAL6oaVvjVtzTSzqrPrbnDablqtpvVpr3G4N59rK+AleAU2gQXegEPwARyDHvDAJfgBfoJftfc1WpO15EZ6b2nueQFKq/btD8Xo+cg= < /latexit > < latexit sha1_base64= '' hAnGZAtKFJW9dFFAVofaA1mIVlg= '' > AAAFxHiclZTfbtMwFMY9WGGUf9u45MaimtimrWqqSUxCkybRISR6MdC6dmtC5SROY9Wxg+2s67zwIFxwC6/E2+BkLUsycYGlSCfn933nxI583JgSqVqt30v37i/XHjxceVR//OTps+era+unkifCwz2PUy4GLpKYEoZ7iiiKB7HAKHIp7ruTdxnvX2AhCWcnahZjJ0JjRgLiIWVSo9V1O6+hBfZTaMch+bI9Wm20mq18wbuBNQ8aYL6OR2vL322fe0mEmfIoknJotWLlaCQU8ShO63YicYy8CRrjoQkZirB0dN44hRsm48OAC/MwBfNs2aGCfUcTFicKM88YIKKUT6HJ7+7DPF/Sn1iOzkrlYrM2oIFwf9clCp5YeRdZNOjQnIsQOMjVuSHPmCOdlIWJoAvNXChJFFMMe5+7MDtdiZUibFwyuZxPFHLlbfVY8ADL7Jcguvs1QZSoGTQSisvtUCQjpMK0msx3cFvOpQa5HAkfZnIoZ5HLabmUG/2fPiKe4NmO0oVrkeFjgeJwVlIHCaWxCf4tZnjq8ShCzNf2wEu1nXX2ENWDNC3TbpF2q7RTpJ0qvbyYUzfQl1U4KMA7Xc8K8KwKZwU4q0IVFqh5wwpVJVcFxVUVnhfgeRVOC3BahUfuArr6yMD6BuwgRjB9LSHyfZJd8PJfVWRyVSni+/MiItJ+tcOEpsO2ozujv0f+sZvaFAcKDmHDgtfXsNGGtiDjUEGnYu4Xvr1frrxte0SYyeAPzVW1s68aLmbYwaYXItHMXrccXS/eNptxHw9liGJ8cOPf8QWa7hDGsIBmlwdtM3VgXmAL6oaVvjVtzTSzqrPrbnDablqtpvVpr3G4N59rK+AleAU2gQXegEPwARyDHvDAJfgBfoJftfc1WpO15EZ6b2nueQFKq/btD8Xo+cg= < /latexit > < latexit sha1_base64= '' hAnGZAtKFJW9dFFAVofaA1mIVlg= '' > AAAFxHiclZTfbtMwFMY9WGGUf9u45MaimtimrWqqSUxCkybRISR6MdC6dmtC5SROY9Wxg+2s67zwIFxwC6/E2+BkLUsycYGlSCfn933nxI583JgSqVqt30v37i/XHjxceVR//OTps+era+unkifCwz2PUy4GLpKYEoZ7iiiKB7HAKHIp7ruTdxnvX2AhCWcnahZjJ0JjRgLiIWVSo9V1O6+hBfZTaMch+bI9Wm20mq18wbuBNQ8aYL6OR2vL322fe0mEmfIoknJotWLlaCQU8ShO63YicYy8CRrjoQkZirB0dN44hRsm48OAC/MwBfNs2aGCfUcTFicKM88YIKKUT6HJ7+7DPF/Sn1iOzkrlYrM2oIFwf9clCp5YeRdZNOjQnIsQOMjVuSHPmCOdlIWJoAvNXChJFFMMe5+7MDtdiZUibFwyuZxPFHLlbfVY8ADL7Jcguvs1QZSoGTQSisvtUCQjpMK0msx3cFvOpQa5HAkfZnIoZ5HLabmUG/2fPiKe4NmO0oVrkeFjgeJwVlIHCaWxCf4tZnjq8ShCzNf2wEu1nXX2ENWDNC3TbpF2q7RTpJ0qvbyYUzfQl1U4KMA7Xc8K8KwKZwU4q0IVFqh5wwpVJVcFxVUVnhfgeRVOC3BahUfuArr6yMD6BuwgRjB9LSHyfZJd8PJfVWRyVSni+/MiItJ+tcOEpsO2ozujv0f+sZvaFAcKDmHDgtfXsNGGtiDjUEGnYu4Xvr1frrxte0SYyeAPzVW1s68aLmbYwaYXItHMXrccXS/eNptxHw9liGJ8cOPf8QWa7hDGsIBmlwdtM3VgXmAL6oaVvjVtzTSzqrPrbnDablqtpvVpr3G4N59rK+AleAU2gQXegEPwARyDHvDAJfgBfoJftfc1WpO15EZ6b2nueQFKq/btD8Xo+cg= < /latexit > < latexit sha1_base64= '' hAnGZAtKFJW9dFFAVofaA1mIVlg= '' > AAAFxHiclZTfbtMwFMY9WGGUf9u45MaimtimrWqqSUxCkybRISR6MdC6dmtC5SROY9Wxg+2s67zwIFxwC6/E2+BkLUsycYGlSCfn933nxI583JgSqVqt30v37i/XHjxceVR//OTps+era+unkifCwz2PUy4GLpKYEoZ7iiiKB7HAKHIp7ruTdxnvX2AhCWcnahZjJ0JjRgLiIWVSo9V1O6+hBfZTaMch+bI9Wm20mq18wbuBNQ8aYL6OR2vL322fe0mEmfIoknJotWLlaCQU8ShO63YicYy8CRrjoQkZirB0dN44hRsm48OAC/MwBfNs2aGCfUcTFicKM88YIKKUT6HJ7+7DPF/Sn1iOzkrlYrM2oIFwf9clCp5YeRdZNOjQnIsQOMjVuSHPmCOdlIWJoAvNXChJFFMMe5+7MDtdiZUibFwyuZxPFHLlbfVY8ADL7Jcguvs1QZSoGTQSisvtUCQjpMK0msx3cFvOpQa5HAkfZnIoZ5HLabmUG/2fPiKe4NmO0oVrkeFjgeJwVlIHCaWxCf4tZnjq8ShCzNf2wEu1nXX2ENWDNC3TbpF2q7RTpJ0qvbyYUzfQl1U4KMA7Xc8K8KwKZwU4q0IVFqh5wwpVJVcFxVUVnhfgeRVOC3BahUfuArr6yMD6BuwgRjB9LSHyfZJd8PJfVWRyVSni+/MiItJ+tcOEpsO2ozujv0f+sZvaFAcKDmHDgtfXsNGGtiDjUEGnYu4Xvr1frrxte0SYyeAPzVW1s68aLmbYwaYXItHMXrccXS/eNptxHw9liGJ8cOPf8QWa7hDGsIBmlwdtM3VgXmAL6oaVvjVtzTSzqrPrbnDablqtpvVpr3G4N59rK+AleAU2gQXegEPwARyDHvDAJfgBfoJftfc1WpO15EZ6b2nueQFKq/btD8Xo+cg= < /latexit > By re-positioning the variational distribution from a potentially limited description of moments , as found in commonly applied conjugate distributions , to a direct expression of probabilities per value , a variety of benefits arise . As the output domain is constrained , the method becomes self-normalizing , relieving the model from hard-to-parallelize batch normalization techniques ( Ioffe & Szegedy ( 2015 ) ) . More interesting priors can be explored and the model is able to learn unique activation functions per neuron . More concretely , the contributions of this work are as follows : • We propose a novel variational inference method by leveraging multinomial distributions on quantized latent variables . • We show that the emerging predicted distributions are multi-modal , motivating the need for flexible distributions in variational inference . • We demonstrate that the proposed method applied to the information bottleneck objective computes competitive uncertainty over the predictions and that this manifests in better performance under strong risk guarantees . 2 BACKGROUND . In this work , we explore deep neural networks for regression and classification . We have data-points consisting of inputs x and targets y in a dataset D = { ( xi , yi ) | i ∈ [ 1 , ... , N ] } and postulate latent variables z that represent the data . We focus on the Information Bottleneck ( IB ) perspective : first proposed by Tishby et al . ( 2000 ) , the information bottleneck objective I ( y , z ; θ ) − βI ( x , z ; θ ) is optimized to maximize the mutual information between z and y , whilst minimizing the mutual information between z and x . The objective can be efficiently optimized using a variational inference scheme as shown concurrently by both Alemi et al . ( 2016 ) and Achille & Soatto ( 2016 ) . Under the Markov assumption P ( z , x , y ) = p ( z|x ) p ( y|x ) p ( x ) , they derive the following lower bound : I ( y , z ; θ ) − βI ( x , z ; θ ) ≥ L = 1 N N∑ n=1 Epθ ( z|xn ) [ log qθ ( yn|z ) ] − βDKL [ pθ ( z|xn ) ‖r ( z ) ] , ( 1 ) where Ep ( z|xn ) is commonly estimated using a single Monte Carlo sample and r ( z ) is a variational approximation to the marginal distribution of z . In practice r ( z ) is fixed to a simple distribution such as a spherical Gaussian . Alemi et al . ( 2016 ) and Achille & Soatto ( 2016 ) continue to show that the Variational Auto Encoder ( VAE ) Evidence Lower Bound ( ELBO ) proposed in Kingma & Ba ( 2014 ) ; Rezende et al . ( 2014 ) is a special case of the IB bound when y = x and β = 1 : I ( z , x ) − βI ( z , i ) ≥ Epθ ( z|xn ) [ log qθ ( xn|z ) ] −DKL [ pθ ( z|xn ) ‖r ( z ) ] , ( 2 ) where i represents the identity of data-point xi . Interestingly , the VAE perspective considers the bound to optimize a variational distribution q ( z|x ) , whilst the IB perspective prescribes that q ( z|x ) in the ELBO is not a variational posterior but the true encoder p ( z|x ) , and instead p ( y|z ) and p ( z ) are the distributions approximated with variational counterparts . However , deriving the ELBO on a conditional distribution p ( y|x ) leads to a KL divergence between the approximate posterior q ( z|x ) and the real posteriorp ( z|x ) , which is not available at test time , rather than the prior p ( z ) . Hence , the information bottleneck perspective provides a more fundamental motivation for the bound in equation 1 . Alternatively , equation 1 can be interpreted as a domain-translating beta-VAE ( Higgins et al . ( 2016 ) ) , where an input image is encoded into a latent space and decoded into the target domain . The Lagrange multiplier β then controls the trade-off between rate and distortion , as argued by Alemi et al . ( 2017 ) . However , this does not naturally arise from deriving the ELBO using Jensen ’ s inequality on the conditional likelihood p ( y|x ) . 3 METHOD . In this work , we follow the IB interpretation of the bound in equation 1 and leave the evaluation of our proposed variational inference scheme in other models such as the VAE for further work . At the heart of our proposal lies the assumption that neuron networks can be effective under strong activation quantization schemes . We start with presenting the derivation if the model in the context of a single latent-layer information bottleneck , following the single data-point loss in equation 1 and dropping the subscript n for clarity , with figure 4 for visual reference : L =Lerror − LKL = Epθ ( z|x ) [ log qθ ( y|z ) ] − βDKL [ pθ ( z|x ) ‖r ( z ) ] . ( 3 ) To impose a flexible , multi-modal distribution over z , we first make a mean-field assumption p ( z|x ) = ∏k p ( zk|x ) . We then quantize the domain of each of the K scalar latent variables zk such that only a small set of potential values remain : zk ∈ v = { v1 , . . . , vC } ∀k with e.g . C = 11 , see fig . 3 . We use the same quantization scheme , defined by v , for all the latents . The specific scheme is considered part of the prior , and is discussed in figure 5 . Although the number of values of z is relatively small , the space of values over all the latent variables grows exponentially . We thus take to a monte-carlo estimation scheme . To optimize the parameters θ with Stochastic Gradient Descent ( SGD ) , we need to derive a fully differentiable sampling scheme that allows us to sample values of z. . To formulate this , we re-parametrize the expectation over z in equation 3 using a set of variables sk ∈ { 1 , . . . , C } which index the value vector v , allowing us to use a softmax function to represent the distribution over z : pθ ( zk = vc|x ) , pθ ( sk = c|x ) = softmax ( Wx+ b ) c. ( 4 ) These indexing values s are then used in conjunction with values v as in input for qθ ( y|z ) , which is modelled with a small network fθ ( · ) : ( abusing notation to indicate element-wise indexing with v [ s ] ) : Lerror = Es∼pθ [ log fθ ( v [ s ] ) ] . To enable sampling from the discrete variables s , we use the Gumbel-Max trick ( Gumbel ( 1954 ) ) , denoted gumb ( ) , re-parameterizing the expectation Es∼p with uniform noise ∼ U ( 0 , 1 ) : Lerror = E [ log fθ ( v [ argmax c gumb ( pθ ( s|x ) , ) ] ) ] . ( 5 ) As the argmax is not differentiable , we approximate this expectation using the Gumbel-Softmax trick ( Maddison et al . ( 2016 ) ; Jang et al . ( 2016 ) ) , which generates samples that smoothly deform into one-hot samples as the softmax temperature τ approaches 0 . Using the inner product ( denoted · ) of the approximate one-hot samples and v , we create samples from z : Lerror ≈ E [ log fθ ( v · softmax c gumb ( pθ ( s|x ) , ) ) ] . ( 6 ) In practice , we anneal τ from 1.0 to 0.5 during the training process , as proposed by Yang et al . ( 2017 ) to reduce gradient variance initially , at the risk of introducing bias . To conclude our derivation , we use a fixed SQUAD distribution to model the variational marginal r ( z ) as shown in figure 5 . We can then derive the KL term analytically following the definition for discrete distributions . Using the fact that the KL divergence is additive for independent variables , we get our final loss : L =E [ log fθ ( v · softmax c gumb ( pθ ( s|x ) , ) ) ] − β K∑ k=1 C∑ c=1 pθ ( sk = c|x ) log pθ ( sk = c|x ) r ( zk = vc ) . ( 7 ) For the remainder of this work , we will refer to the latent variables as z in lieu of s , for clarity . At test time , we can approximate the predictive function p ( y∗|x∗ ) for a new data-point x∗ by taking T samples from the latent variables z i.e . ẑt ∼ p ( z|x∗ ) , and averaging the predictions for y∗ : pθ ( y ∗|x∗ ) ≈ ∫ qθ ( y ∗|z ) pθ ( z|x∗ ) dz ≈ 1 T T∑ t=1 qθ ( y ∗|ẑt ) . ( 8 ) A downside of mean-field variational approximations is that latents are assumed to be uncorrelated . We can bring some correlation to the latents , and extend the flexibility of the proposed model , by creating a hierarchical set of latent variables . We maintain a mean-field assumption for the prior , and the joint posterior distribution of L layers of latents factorizes as follows : pθ ( z1 , . . . , zL|x ) = pθ ( zL|zL−1 ) · · · pθ ( z1|x ) , ( 9 ) With qθ ( y|z1 , . . . , zL ) = qθ ( y|zL ) . This is straightforwardly implemented with a simple ancestral sampling scheme . Interestingly , the strong quantization proposed in our method can itself be considered an additional information bottleneck , as it exactly upper-bounds the number of bits per latent variable . Such bottlenecks are theorized to have a beneficial effect on generalization ( Tishby et al . ( 2000 ) ; Achille & Soatto ( 2016 ) ; Alemi et al . ( 2017 ; 2016 ) ) , and we can directly control this bottleneck by varying the number of quantization bins . The computational complexity of the method , as well as the number of model parameters θ , scale linearly in C , i.e . O ( C ) ( with C the number of quantization bins ) . It is thus suitable for large-scale inference . We would like to stress that the proposed method differs from work that leverages the Gumbel-Softmax trick to model categorical latent variables : our proposal models continuous scalar latent variables by quantizing their domain and modeling belief with a multinomial distribution . Categorical latent variable models would incur a much larger polynomial complexity penalty of O ( C2 ) . The method is easily integrated into existing deep neural network architectures , and SQUAD layer implementations are provided at github.com/anonymized . Matrix-factorization variant In the natural image domain , we anticipate a need for a high amount of information per variable , for which a small number of bins does not suffice . To improving the tractability of using a large number of quantization bins , we propose a variant of SQUAD that uses a matrix factorization scheme to improve the parameter efficiency . Formally , equation 4 becomes : p ( sk = c|x ) = softmax ( w′′k , c ( w′kx+ b′k ) + b′′k , c ) , with full layer weights W′′ and W′ respectively of shape ( K , B , C ) and ( |X| , K , B ) , where K denotes the number of neurons , B the number of factorization inputs , C number of quantization bins and |X| the input dimensionality . To improve the parameter efficiency , we can learn W′ per layer as well , resulting in shape ( |X| , 1 , B ) , which is found to be beneficial for large C by the hyperparameter search presented in section 5 . We depict this alternative model on the right side of figure 4 and will refer to it as SQUAD-factorized . We leave further extensions such as Network-in-Network ( Lin et al . ( 2013 ) ) for future work .

Summary: The authors propose “Stochastic Quantized Activation Distributions” (SQUAD). It quantizes the continuous values of a network activation under a finite number of discrete (non-ordinal) values, and is distributed according to a Gumbel-Softmax distribution. While the topic is interesting, the work could improve by making more precise the benefit of (relaxed) discrete random variables. This will also allow the authors to more precisely display in the experiments why this particular approach is more natural than other baselines (e.g., if multimodality is the issue, compare to a mixture model; if correlation is a difficulty, compare to any structured distribution such as a flow).