| 
|
+| (no preprocessor) | 
| 
|
+|
| 
| 
|
+|
| 
| 
|
+|
| 
| 
|
+|
| 
| 
|
+|
| 
| 
|
+
+### Adapter Examples
+
+| Source | Input | Output |
+|:-------------------------:|:-------------------------:|:-------------------------:|
+| (no preprocessor) | 
| 
|
+| (no preprocessor) | 
| 
|
+| (no preprocessor) | 
| 
|
+| (no preprocessor) | 
| 
|
+| 
| 
| 
|
+| | (clip, non-image) | 
|
+
+Examples by catboxanon, no tweaking or cherrypicking. (Color Guidance)
+
+| Image | Disabled | Enabled |
+|:-------------------------:|:-------------------------:|:-------------------------:|
+| 
| 
| 
|
+| 
| 
| 
|
+
+### Minimum Requirements
+
+* (Windows) (NVIDIA: Ampere) 4gb - with `--xformers` enabled, and `Low VRAM` mode ticked in the UI, goes up to 768x832
+
+### CFG Based ControlNet (Experimental)
+
+The original ControlNet applies control to both conditional (cond) and unconditional (uncond) parts. Enabling this option will make the control only apply to the cond part. Some experiments indicate that this approach improves image quality.
+
+To enable this option, tick `Enable CFG-Based guidance for ControlNet` in the settings.
+
+Note that you need to use a low cfg scale/guidance scale (such as 3-5) and proper weight tuning to get good result.
+
+### Guess Mode (Non-Prompt Mode, Experimental)
+
+Guess Mode is CFG Based ControlNet + Exponential decay in weighting.
+
+See issue https://github.com/Mikubill/sd-webui-controlnet/issues/236 for more details.
+
+Original introduction from controlnet:
+
+The "guess mode" (or called non-prompt mode) will completely unleash all the power of the very powerful ControlNet encoder.
+
+In this mode, you can just remove all prompts, and then the ControlNet encoder will recognize the content of the input control map, like depth map, edge map, scribbles, etc.
+
+This mode is very suitable for comparing different methods to control stable diffusion because the non-prompted generating task is significantly more difficult than prompted task. In this mode, different methods' performance will be very salient.
+
+For this mode, we recommend to **use 50 steps and guidance scale between 3 and 5.**
+
+### Multi-ControlNet / Joint Conditioning (Experimental)
+
+This option allows multiple ControlNet inputs for a single generation. To enable this option, change `Multi ControlNet: Max models amount (requires restart)` in the settings. Note that you will need to restart the WebUI for changes to take effect.
+
+* Guess Mode will apply to all ControlNet if any of them are enabled.
+
+| Source A | Source B | Output |
+|:-------------------------:|:-------------------------:|:-------------------------:|
+| 
| 
| 
|
+
+### Weight and Guidance Strength/Start/End
+
+Weight is the weight of the controlnet "influence". It's analogous to prompt attention/emphasis. E.g. (myprompt: 1.2). Technically, it's the factor by which to multiply the ControlNet outputs before merging them with original SD Unet.
+
+Guidance Start/End is the percentage of total steps the controlnet applies (guidance strength = guidance end). It's analogous to prompt editing/shifting. E.g. \[myprompt::0.8\] (It applies from the beginning until 80% of total steps)
+
+### API/Script Access
+
+This extension can accept txt2img or img2img tasks via API or external extension call. Note that you may need to enable `Allow other scripts to control this extension` in settings for external calls.
+
+To use the API: start WebUI with argument `--api` and go to `http://webui-address/docs` for documents or checkout [examples](https://github.com/Mikubill/sd-webui-controlnet/blob/main/example/api_txt2img.ipynb).
+
+To use external call: Checkout [Wiki](https://github.com/Mikubill/sd-webui-controlnet/wiki/API)
+
+### MacOS Support
+
+Tested with pytorch nightly: https://github.com/Mikubill/sd-webui-controlnet/pull/143#issuecomment-1435058285
+
+To use this extension with mps and normal pytorch, currently you may need to start WebUI with `--no-half`.
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/binary/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/binary/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..2d13ad692ffc109ad95789334bb5524d52794acc
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/binary/__init__.py
@@ -0,0 +1,14 @@
+import cv2
+
+
+def apply_binary(img, bin_threshold):
+ img_gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
+
+ if bin_threshold == 0 or bin_threshold == 255:
+ # Otsu's threshold
+ otsu_threshold, img_bin = cv2.threshold(img_gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
+ print("Otsu threshold:", otsu_threshold)
+ else:
+ _, img_bin = cv2.threshold(img_gray, bin_threshold, 255, cv2.THRESH_BINARY_INV)
+
+ return cv2.cvtColor(img_bin, cv2.COLOR_GRAY2RGB)
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/canny/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/canny/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..ace985839d3fc18dd4947f6c38e9f5d5a2625aca
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/canny/__init__.py
@@ -0,0 +1,5 @@
+import cv2
+
+
+def apply_canny(img, low_threshold, high_threshold):
+ return cv2.Canny(img, low_threshold, high_threshold)
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/clip/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/clip/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..997e90edcf6efcfbd2b4065be2d8de9ced7abb06
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/clip/__init__.py
@@ -0,0 +1,23 @@
+from transformers import CLIPProcessor, CLIPVisionModel
+from modules import devices
+
+version = 'openai/clip-vit-large-patch14'
+clip_proc = None
+clip_vision_model = None
+
+def apply_clip(img):
+ global clip_proc, clip_vision_model
+
+ if clip_vision_model is None:
+ clip_proc = CLIPProcessor.from_pretrained(version)
+ clip_vision_model = CLIPVisionModel.from_pretrained(version)
+
+ clip_vision_model = clip_vision_model.to(devices.get_device_for("controlnet"))
+ style_for_clip = clip_proc(images=img, return_tensors="pt")['pixel_values']
+ style_feat = clip_vision_model(style_for_clip.to(devices.get_device_for("controlnet")))['last_hidden_state']
+ return style_feat
+
+def unload_clip_model():
+ global clip_proc, clip_vision_model
+ if clip_vision_model is not None:
+ clip_vision_model.cpu()
\ No newline at end of file
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/color/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/color/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..f9453add3fb690ef903f0e0d445d09a6085d4858
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/color/__init__.py
@@ -0,0 +1,6 @@
+import cv2
+
+def apply_color(img, res=512):
+ input_img_color = cv2.resize(img, (res//64, res//64), interpolation=cv2.INTER_CUBIC)
+ input_img_color = cv2.resize(input_img_color, (res, res), interpolation=cv2.INTER_NEAREST)
+ return input_img_color
\ No newline at end of file
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/hed/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/hed/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..8a37efbdcb58e94b32c91035f2446de837b86aa2
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/hed/__init__.py
@@ -0,0 +1,148 @@
+from distutils import extension
+import numpy as np
+import cv2
+import torch
+from einops import rearrange
+
+import os
+from modules import devices
+from modules.paths import models_path
+
+class Network(torch.nn.Module):
+ def __init__(self, model_path):
+ super().__init__()
+
+ self.netVggOne = torch.nn.Sequential(
+ torch.nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggTwo = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggThr = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggFou = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=256, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggFiv = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netScoreOne = torch.nn.Conv2d(in_channels=64, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreTwo = torch.nn.Conv2d(in_channels=128, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreThr = torch.nn.Conv2d(in_channels=256, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreFou = torch.nn.Conv2d(in_channels=512, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreFiv = torch.nn.Conv2d(in_channels=512, out_channels=1, kernel_size=1, stride=1, padding=0)
+
+ self.netCombine = torch.nn.Sequential(
+ torch.nn.Conv2d(in_channels=5, out_channels=1, kernel_size=1, stride=1, padding=0),
+ torch.nn.Sigmoid()
+ )
+
+ self.load_state_dict({strKey.replace('module', 'net'): tenWeight for strKey, tenWeight in torch.load(model_path).items()})
+ # end
+
+ def forward(self, tenInput):
+ tenInput = tenInput * 255.0
+ tenInput = tenInput - torch.tensor(data=[104.00698793, 116.66876762, 122.67891434], dtype=tenInput.dtype, device=tenInput.device).view(1, 3, 1, 1)
+
+ tenVggOne = self.netVggOne(tenInput)
+ tenVggTwo = self.netVggTwo(tenVggOne)
+ tenVggThr = self.netVggThr(tenVggTwo)
+ tenVggFou = self.netVggFou(tenVggThr)
+ tenVggFiv = self.netVggFiv(tenVggFou)
+
+ tenScoreOne = self.netScoreOne(tenVggOne)
+ tenScoreTwo = self.netScoreTwo(tenVggTwo)
+ tenScoreThr = self.netScoreThr(tenVggThr)
+ tenScoreFou = self.netScoreFou(tenVggFou)
+ tenScoreFiv = self.netScoreFiv(tenVggFiv)
+
+ tenScoreOne = torch.nn.functional.interpolate(input=tenScoreOne, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreTwo = torch.nn.functional.interpolate(input=tenScoreTwo, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreThr = torch.nn.functional.interpolate(input=tenScoreThr, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreFou = torch.nn.functional.interpolate(input=tenScoreFou, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreFiv = torch.nn.functional.interpolate(input=tenScoreFiv, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+
+ return self.netCombine(torch.cat([ tenScoreOne, tenScoreTwo, tenScoreThr, tenScoreFou, tenScoreFiv ], 1))
+ # end
+# end
+
+netNetwork = None
+remote_model_path = "https://huggingface.co/lllyasviel/ControlNet/resolve/main/annotator/ckpts/network-bsds500.pth"
+modeldir = os.path.join(models_path, "hed")
+old_modeldir = os.path.dirname(os.path.realpath(__file__))
+
+def apply_hed(input_image):
+ global netNetwork
+ if netNetwork is None:
+ modelpath = os.path.join(modeldir, "network-bsds500.pth")
+ old_modelpath = os.path.join(old_modeldir, "network-bsds500.pth")
+ if os.path.exists(old_modelpath):
+ modelpath = old_modelpath
+ elif not os.path.exists(modelpath):
+ from basicsr.utils.download_util import load_file_from_url
+ load_file_from_url(remote_model_path, model_dir=modeldir)
+ netNetwork = Network(modelpath)
+ netNetwork.to(devices.get_device_for("controlnet")).eval()
+
+ assert input_image.ndim == 3
+ input_image = input_image[:, :, ::-1].copy()
+ with torch.no_grad():
+ image_hed = torch.from_numpy(input_image).float().to(devices.get_device_for("controlnet"))
+ image_hed = image_hed / 255.0
+ image_hed = rearrange(image_hed, 'h w c -> 1 c h w')
+ edge = netNetwork(image_hed)[0]
+ edge = (edge.cpu().numpy() * 255.0).clip(0, 255).astype(np.uint8)
+ return edge[0]
+
+def unload_hed_model():
+ global netNetwork
+ if netNetwork is not None:
+ netNetwork.cpu()
+
+def nms(x, t, s):
+ x = cv2.GaussianBlur(x.astype(np.float32), (0, 0), s)
+
+ f1 = np.array([[0, 0, 0], [1, 1, 1], [0, 0, 0]], dtype=np.uint8)
+ f2 = np.array([[0, 1, 0], [0, 1, 0], [0, 1, 0]], dtype=np.uint8)
+ f3 = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]], dtype=np.uint8)
+ f4 = np.array([[0, 0, 1], [0, 1, 0], [1, 0, 0]], dtype=np.uint8)
+
+ y = np.zeros_like(x)
+
+ for f in [f1, f2, f3, f4]:
+ np.putmask(y, cv2.dilate(x, kernel=f) == x, x)
+
+ z = np.zeros_like(y, dtype=np.uint8)
+ z[y > t] = 255
+ return z
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/informative/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/informative/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..fa7697f6e68a1e2899c290242900a750acd98c5e
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/informative/__init__.py
@@ -0,0 +1,131 @@
+from distutils import extension
+import numpy as np
+import cv2
+import torch
+from einops import rearrange
+
+import os
+from modules import devices
+from modules.paths import models_path
+
+class Network(torch.nn.Module):
+ def __init__(self, model_path):
+ super().__init__()
+
+ self.netVggOne = torch.nn.Sequential(
+ torch.nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggTwo = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggThr = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggFou = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=256, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netVggFiv = torch.nn.Sequential(
+ torch.nn.MaxPool2d(kernel_size=2, stride=2),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False),
+ torch.nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, stride=1, padding=1),
+ torch.nn.ReLU(inplace=False)
+ )
+
+ self.netScoreOne = torch.nn.Conv2d(in_channels=64, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreTwo = torch.nn.Conv2d(in_channels=128, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreThr = torch.nn.Conv2d(in_channels=256, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreFou = torch.nn.Conv2d(in_channels=512, out_channels=1, kernel_size=1, stride=1, padding=0)
+ self.netScoreFiv = torch.nn.Conv2d(in_channels=512, out_channels=1, kernel_size=1, stride=1, padding=0)
+
+ self.netCombine = torch.nn.Sequential(
+ torch.nn.Conv2d(in_channels=5, out_channels=1, kernel_size=1, stride=1, padding=0),
+ torch.nn.Sigmoid()
+ )
+
+ self.load_state_dict({strKey.replace('module', 'net'): tenWeight for strKey, tenWeight in torch.load(model_path).items()})
+ # end
+
+ def forward(self, tenInput):
+ tenInput = tenInput * 255.0
+ tenInput = tenInput - torch.tensor(data=[104.00698793, 116.66876762, 122.67891434], dtype=tenInput.dtype, device=tenInput.device).view(1, 3, 1, 1)
+
+ tenVggOne = self.netVggOne(tenInput)
+ tenVggTwo = self.netVggTwo(tenVggOne)
+ tenVggThr = self.netVggThr(tenVggTwo)
+ tenVggFou = self.netVggFou(tenVggThr)
+ tenVggFiv = self.netVggFiv(tenVggFou)
+
+ tenScoreOne = self.netScoreOne(tenVggOne)
+ tenScoreTwo = self.netScoreTwo(tenVggTwo)
+ tenScoreThr = self.netScoreThr(tenVggThr)
+ tenScoreFou = self.netScoreFou(tenVggFou)
+ tenScoreFiv = self.netScoreFiv(tenVggFiv)
+
+ tenScoreOne = torch.nn.functional.interpolate(input=tenScoreOne, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreTwo = torch.nn.functional.interpolate(input=tenScoreTwo, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreThr = torch.nn.functional.interpolate(input=tenScoreThr, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreFou = torch.nn.functional.interpolate(input=tenScoreFou, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+ tenScoreFiv = torch.nn.functional.interpolate(input=tenScoreFiv, size=(tenInput.shape[2], tenInput.shape[3]), mode='bilinear', align_corners=False)
+
+ return self.netCombine(torch.cat([ tenScoreOne, tenScoreTwo, tenScoreThr, tenScoreFou, tenScoreFiv ], 1))
+ # end
+# end
+
+netNetwork = None
+remote_model_path = "https://huggingface.co/lllyasviel/ControlNet/resolve/main/annotator/ckpts/network-bsds500.pth"
+modeldir = os.path.join(models_path, "hed")
+old_modeldir = os.path.dirname(os.path.realpath(__file__))
+
+def apply_hed(input_image):
+ global netNetwork
+ if netNetwork is None:
+ modelpath = os.path.join(modeldir, "network-bsds500.pth")
+ old_modelpath = os.path.join(old_modeldir, "network-bsds500.pth")
+ if os.path.exists(old_modelpath):
+ modelpath = old_modelpath
+ elif not os.path.exists(modelpath):
+ from basicsr.utils.download_util import load_file_from_url
+ load_file_from_url(remote_model_path, model_dir=modeldir)
+ netNetwork = Network(modelpath)
+ netNetwork.to(devices.get_device_for("controlnet")).eval()
+
+ assert input_image.ndim == 3
+ input_image = input_image[:, :, ::-1].copy()
+ with torch.no_grad():
+ image_hed = torch.from_numpy(input_image).float().to(devices.get_device_for("controlnet"))
+ image_hed = image_hed / 255.0
+ image_hed = rearrange(image_hed, 'h w c -> 1 c h w')
+ edge = netNetwork(image_hed)[0]
+ edge = (edge.cpu().numpy() * 255.0).clip(0, 255).astype(np.uint8)
+ return edge[0]
+
+def unload_hed_model():
+ global netNetwork
+ if netNetwork is not None:
+ netNetwork.cpu()
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..2550fe50b9afe0c40effe102ab32c8d82bde310d
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/__init__.py
@@ -0,0 +1,212 @@
+import numpy as np
+import cv2
+import torch
+
+import os
+from modules import devices
+from modules.paths import models_path
+
+import mmcv
+from mmdet.apis import inference_detector, init_detector
+from mmpose.apis import inference_top_down_pose_model
+from mmpose.apis import init_pose_model, process_mmdet_results, vis_pose_result
+
+
+def preprocessing(image, device):
+ # Resize
+ scale = 640 / max(image.shape[:2])
+ image = cv2.resize(image, dsize=None, fx=scale, fy=scale)
+ raw_image = image.astype(np.uint8)
+
+ # Subtract mean values
+ image = image.astype(np.float32)
+ image -= np.array(
+ [
+ float(104.008),
+ float(116.669),
+ float(122.675),
+ ]
+ )
+
+ # Convert to torch.Tensor and add "batch" axis
+ image = torch.from_numpy(image.transpose(2, 0, 1)).float().unsqueeze(0)
+ image = image.to(device)
+
+ return image, raw_image
+
+
+def imshow_keypoints(img,
+ pose_result,
+ skeleton=None,
+ kpt_score_thr=0.1,
+ pose_kpt_color=None,
+ pose_link_color=None,
+ radius=4,
+ thickness=1):
+ """Draw keypoints and links on an image.
+ Args:
+ img (ndarry): The image to draw poses on.
+ pose_result (list[kpts]): The poses to draw. Each element kpts is
+ a set of K keypoints as an Kx3 numpy.ndarray, where each
+ keypoint is represented as x, y, score.
+ kpt_score_thr (float, optional): Minimum score of keypoints
+ to be shown. Default: 0.3.
+ pose_kpt_color (np.array[Nx3]`): Color of N keypoints. If None,
+ the keypoint will not be drawn.
+ pose_link_color (np.array[Mx3]): Color of M links. If None, the
+ links will not be drawn.
+ thickness (int): Thickness of lines.
+ """
+
+ img_h, img_w, _ = img.shape
+ img = np.zeros(img.shape)
+
+ for idx, kpts in enumerate(pose_result):
+ if idx > 1:
+ continue
+ kpts = kpts['keypoints']
+ # print(kpts)
+ kpts = np.array(kpts, copy=False)
+
+ # draw each point on image
+ if pose_kpt_color is not None:
+ assert len(pose_kpt_color) == len(kpts)
+
+ for kid, kpt in enumerate(kpts):
+ x_coord, y_coord, kpt_score = int(kpt[0]), int(kpt[1]), kpt[2]
+
+ if kpt_score < kpt_score_thr or pose_kpt_color[kid] is None:
+ # skip the point that should not be drawn
+ continue
+
+ color = tuple(int(c) for c in pose_kpt_color[kid])
+ cv2.circle(img, (int(x_coord), int(y_coord)),
+ radius, color, -1)
+
+ # draw links
+ if skeleton is not None and pose_link_color is not None:
+ assert len(pose_link_color) == len(skeleton)
+
+ for sk_id, sk in enumerate(skeleton):
+ pos1 = (int(kpts[sk[0], 0]), int(kpts[sk[0], 1]))
+ pos2 = (int(kpts[sk[1], 0]), int(kpts[sk[1], 1]))
+
+ if (pos1[0] <= 0 or pos1[0] >= img_w or pos1[1] <= 0 or pos1[1] >= img_h or pos2[0] <= 0
+ or pos2[0] >= img_w or pos2[1] <= 0 or pos2[1] >= img_h or kpts[sk[0], 2] < kpt_score_thr
+ or kpts[sk[1], 2] < kpt_score_thr or pose_link_color[sk_id] is None):
+ # skip the link that should not be drawn
+ continue
+ color = tuple(int(c) for c in pose_link_color[sk_id])
+ cv2.line(img, pos1, pos2, color, thickness=thickness)
+
+ return img
+
+
+human_det, pose_model = None, None
+det_model_path = "https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth"
+pose_model_path = "https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth"
+
+modeldir = os.path.join(models_path, "keypose")
+old_modeldir = os.path.dirname(os.path.realpath(__file__))
+
+det_config = 'faster_rcnn_r50_fpn_coco.py'
+pose_config = 'hrnet_w48_coco_256x192.py'
+
+det_checkpoint = 'faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth'
+pose_checkpoint = 'hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth'
+det_cat_id = 1
+bbox_thr = 0.2
+
+skeleton = [
+ [15, 13], [13, 11], [16, 14], [14, 12], [11, 12], [5, 11], [6, 12], [5, 6], [5, 7], [6, 8],
+ [7, 9], [8, 10],
+ [1, 2], [0, 1], [0, 2], [1, 3], [2, 4], [3, 5], [4, 6]
+]
+
+pose_kpt_color = [
+ [51, 153, 255], [51, 153, 255], [51, 153, 255], [51, 153, 255], [51, 153, 255],
+ [0, 255, 0],
+ [255, 128, 0], [0, 255, 0], [255, 128, 0], [0, 255, 0], [255, 128, 0], [0, 255, 0],
+ [255, 128, 0],
+ [0, 255, 0], [255, 128, 0], [0, 255, 0], [255, 128, 0]
+]
+
+pose_link_color = [
+ [0, 255, 0], [0, 255, 0], [255, 128, 0], [255, 128, 0],
+ [51, 153, 255], [51, 153, 255], [51, 153, 255], [51, 153, 255], [0, 255, 0],
+ [255, 128, 0],
+ [0, 255, 0], [255, 128, 0], [51, 153, 255], [51, 153, 255], [51, 153, 255],
+ [51, 153, 255],
+ [51, 153, 255], [51, 153, 255], [51, 153, 255]
+]
+
+def find_download_model(checkpoint, remote_path):
+ modelpath = os.path.join(modeldir, checkpoint)
+ old_modelpath = os.path.join(old_modeldir, checkpoint)
+
+ if os.path.exists(old_modelpath):
+ modelpath = old_modelpath
+ elif not os.path.exists(modelpath):
+ from basicsr.utils.download_util import load_file_from_url
+ load_file_from_url(remote_path, model_dir=modeldir)
+
+ return modelpath
+
+def apply_keypose(input_image):
+ global human_det, pose_model
+ if netNetwork is None:
+ det_model_local = find_download_model(det_checkpoint, det_model_path)
+ hrnet_model_local = find_download_model(pose_checkpoint, pose_model_path)
+ det_config_mmcv = mmcv.Config.fromfile(det_config)
+ pose_config_mmcv = mmcv.Config.fromfile(pose_config)
+ human_det = init_detector(det_config_mmcv, det_model_local, device=devices.get_device_for("controlnet"))
+ pose_model = init_pose_model(pose_config_mmcv, hrnet_model_local, device=devices.get_device_for("controlnet"))
+
+ assert input_image.ndim == 3
+ input_image = input_image.copy()
+ with torch.no_grad():
+ image = torch.from_numpy(input_image).float().to(devices.get_device_for("controlnet"))
+ image = image / 255.0
+ mmdet_results = inference_detector(human_det, image)
+
+ # keep the person class bounding boxes.
+ person_results = process_mmdet_results(mmdet_results, det_cat_id)
+
+ return_heatmap = False
+ dataset = pose_model.cfg.data['test']['type']
+
+ # e.g. use ('backbone', ) to return backbone feature
+ output_layer_names = None
+ pose_results, _ = inference_top_down_pose_model(
+ pose_model,
+ image,
+ person_results,
+ bbox_thr=bbox_thr,
+ format='xyxy',
+ dataset=dataset,
+ dataset_info=None,
+ return_heatmap=return_heatmap,
+ outputs=output_layer_names
+ )
+
+ im_keypose_out = imshow_keypoints(
+ image,
+ pose_results,
+ skeleton=skeleton,
+ pose_kpt_color=pose_kpt_color,
+ pose_link_color=pose_link_color,
+ radius=2,
+ thickness=2
+ )
+ im_keypose_out = im_keypose_out.astype(np.uint8)
+
+ # image_hed = rearrange(image_hed, 'h w c -> 1 c h w')
+ # edge = netNetwork(image_hed)[0]
+ # edge = (edge.cpu().numpy() * 255.0).clip(0, 255).astype(np.uint8)
+ return im_keypose_out
+
+
+def unload_hed_model():
+ global netNetwork
+ if netNetwork is not None:
+ netNetwork.cpu()
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/faster_rcnn_r50_fpn_coco.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/faster_rcnn_r50_fpn_coco.py
new file mode 100644
index 0000000000000000000000000000000000000000..a9ad9528b22163ae7ce1390375b69227fd6eafd9
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/faster_rcnn_r50_fpn_coco.py
@@ -0,0 +1,182 @@
+checkpoint_config = dict(interval=1)
+# yapf:disable
+log_config = dict(
+ interval=50,
+ hooks=[
+ dict(type='TextLoggerHook'),
+ # dict(type='TensorboardLoggerHook')
+ ])
+# yapf:enable
+dist_params = dict(backend='nccl')
+log_level = 'INFO'
+load_from = None
+resume_from = None
+workflow = [('train', 1)]
+# optimizer
+optimizer = dict(type='SGD', lr=0.02, momentum=0.9, weight_decay=0.0001)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[8, 11])
+total_epochs = 12
+
+model = dict(
+ type='FasterRCNN',
+ pretrained='torchvision://resnet50',
+ backbone=dict(
+ type='ResNet',
+ depth=50,
+ num_stages=4,
+ out_indices=(0, 1, 2, 3),
+ frozen_stages=1,
+ norm_cfg=dict(type='BN', requires_grad=True),
+ norm_eval=True,
+ style='pytorch'),
+ neck=dict(
+ type='FPN',
+ in_channels=[256, 512, 1024, 2048],
+ out_channels=256,
+ num_outs=5),
+ rpn_head=dict(
+ type='RPNHead',
+ in_channels=256,
+ feat_channels=256,
+ anchor_generator=dict(
+ type='AnchorGenerator',
+ scales=[8],
+ ratios=[0.5, 1.0, 2.0],
+ strides=[4, 8, 16, 32, 64]),
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[.0, .0, .0, .0],
+ target_stds=[1.0, 1.0, 1.0, 1.0]),
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
+ roi_head=dict(
+ type='StandardRoIHead',
+ bbox_roi_extractor=dict(
+ type='SingleRoIExtractor',
+ roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
+ out_channels=256,
+ featmap_strides=[4, 8, 16, 32]),
+ bbox_head=dict(
+ type='Shared2FCBBoxHead',
+ in_channels=256,
+ fc_out_channels=1024,
+ roi_feat_size=7,
+ num_classes=80,
+ bbox_coder=dict(
+ type='DeltaXYWHBBoxCoder',
+ target_means=[0., 0., 0., 0.],
+ target_stds=[0.1, 0.1, 0.2, 0.2]),
+ reg_class_agnostic=False,
+ loss_cls=dict(
+ type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
+ loss_bbox=dict(type='L1Loss', loss_weight=1.0))),
+ # model training and testing settings
+ train_cfg=dict(
+ rpn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.7,
+ neg_iou_thr=0.3,
+ min_pos_iou=0.3,
+ match_low_quality=True,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=256,
+ pos_fraction=0.5,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=False),
+ allowed_border=-1,
+ pos_weight=-1,
+ debug=False),
+ rpn_proposal=dict(
+ nms_pre=2000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ assigner=dict(
+ type='MaxIoUAssigner',
+ pos_iou_thr=0.5,
+ neg_iou_thr=0.5,
+ min_pos_iou=0.5,
+ match_low_quality=False,
+ ignore_iof_thr=-1),
+ sampler=dict(
+ type='RandomSampler',
+ num=512,
+ pos_fraction=0.25,
+ neg_pos_ub=-1,
+ add_gt_as_proposals=True),
+ pos_weight=-1,
+ debug=False)),
+ test_cfg=dict(
+ rpn=dict(
+ nms_pre=1000,
+ max_per_img=1000,
+ nms=dict(type='nms', iou_threshold=0.7),
+ min_bbox_size=0),
+ rcnn=dict(
+ score_thr=0.05,
+ nms=dict(type='nms', iou_threshold=0.5),
+ max_per_img=100)
+ # soft-nms is also supported for rcnn testing
+ # e.g., nms=dict(type='soft_nms', iou_threshold=0.5, min_score=0.05)
+ ))
+
+dataset_type = 'CocoDataset'
+data_root = 'data/coco'
+img_norm_cfg = dict(
+ mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='LoadAnnotations', with_bbox=True),
+ dict(type='Resize', img_scale=(1333, 800), keep_ratio=True),
+ dict(type='RandomFlip', flip_ratio=0.5),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels']),
+]
+test_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(
+ type='MultiScaleFlipAug',
+ img_scale=(1333, 800),
+ flip=False,
+ transforms=[
+ dict(type='Resize', keep_ratio=True),
+ dict(type='RandomFlip'),
+ dict(type='Normalize', **img_norm_cfg),
+ dict(type='Pad', size_divisor=32),
+ dict(type='DefaultFormatBundle'),
+ dict(type='Collect', keys=['img']),
+ ])
+]
+data = dict(
+ samples_per_gpu=2,
+ workers_per_gpu=2,
+ train=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ pipeline=train_pipeline),
+ val=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline),
+ test=dict(
+ type=dataset_type,
+ ann_file=f'{data_root}/annotations/instances_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ pipeline=test_pipeline))
+evaluation = dict(interval=1, metric='bbox')
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/hrnet_w48_coco_256x192.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/hrnet_w48_coco_256x192.py
new file mode 100644
index 0000000000000000000000000000000000000000..9755e6773cd3a8c0d2ac684c612d716cfd44b0ca
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/keypose/hrnet_w48_coco_256x192.py
@@ -0,0 +1,169 @@
+# _base_ = [
+# '../../../../_base_/default_runtime.py',
+# '../../../../_base_/datasets/coco.py'
+# ]
+evaluation = dict(interval=10, metric='mAP', save_best='AP')
+
+optimizer = dict(
+ type='Adam',
+ lr=5e-4,
+)
+optimizer_config = dict(grad_clip=None)
+# learning policy
+lr_config = dict(
+ policy='step',
+ warmup='linear',
+ warmup_iters=500,
+ warmup_ratio=0.001,
+ step=[170, 200])
+total_epochs = 210
+channel_cfg = dict(
+ num_output_channels=17,
+ dataset_joints=17,
+ dataset_channel=[
+ [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
+ ],
+ inference_channel=[
+ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
+ ])
+
+# model settings
+model = dict(
+ type='TopDown',
+ pretrained='https://download.openmmlab.com/mmpose/'
+ 'pretrain_models/hrnet_w48-8ef0771d.pth',
+ backbone=dict(
+ type='HRNet',
+ in_channels=3,
+ extra=dict(
+ stage1=dict(
+ num_modules=1,
+ num_branches=1,
+ block='BOTTLENECK',
+ num_blocks=(4, ),
+ num_channels=(64, )),
+ stage2=dict(
+ num_modules=1,
+ num_branches=2,
+ block='BASIC',
+ num_blocks=(4, 4),
+ num_channels=(48, 96)),
+ stage3=dict(
+ num_modules=4,
+ num_branches=3,
+ block='BASIC',
+ num_blocks=(4, 4, 4),
+ num_channels=(48, 96, 192)),
+ stage4=dict(
+ num_modules=3,
+ num_branches=4,
+ block='BASIC',
+ num_blocks=(4, 4, 4, 4),
+ num_channels=(48, 96, 192, 384))),
+ ),
+ keypoint_head=dict(
+ type='TopdownHeatmapSimpleHead',
+ in_channels=48,
+ out_channels=channel_cfg['num_output_channels'],
+ num_deconv_layers=0,
+ extra=dict(final_conv_kernel=1, ),
+ loss_keypoint=dict(type='JointsMSELoss', use_target_weight=True)),
+ train_cfg=dict(),
+ test_cfg=dict(
+ flip_test=True,
+ post_process='default',
+ shift_heatmap=True,
+ modulate_kernel=11))
+
+data_cfg = dict(
+ image_size=[192, 256],
+ heatmap_size=[48, 64],
+ num_output_channels=channel_cfg['num_output_channels'],
+ num_joints=channel_cfg['dataset_joints'],
+ dataset_channel=channel_cfg['dataset_channel'],
+ inference_channel=channel_cfg['inference_channel'],
+ soft_nms=False,
+ nms_thr=1.0,
+ oks_thr=0.9,
+ vis_thr=0.2,
+ use_gt_bbox=False,
+ det_bbox_thr=0.0,
+ bbox_file='data/coco/person_detection_results/'
+ 'COCO_val2017_detections_AP_H_56_person.json',
+)
+
+train_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='TopDownGetBboxCenterScale', padding=1.25),
+ dict(type='TopDownRandomShiftBboxCenter', shift_factor=0.16, prob=0.3),
+ dict(type='TopDownRandomFlip', flip_prob=0.5),
+ dict(
+ type='TopDownHalfBodyTransform',
+ num_joints_half_body=8,
+ prob_half_body=0.3),
+ dict(
+ type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
+ dict(type='TopDownAffine'),
+ dict(type='ToTensor'),
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406],
+ std=[0.229, 0.224, 0.225]),
+ dict(type='TopDownGenerateTarget', sigma=2),
+ dict(
+ type='Collect',
+ keys=['img', 'target', 'target_weight'],
+ meta_keys=[
+ 'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
+ 'rotation', 'bbox_score', 'flip_pairs'
+ ]),
+]
+
+val_pipeline = [
+ dict(type='LoadImageFromFile'),
+ dict(type='TopDownGetBboxCenterScale', padding=1.25),
+ dict(type='TopDownAffine'),
+ dict(type='ToTensor'),
+ dict(
+ type='NormalizeTensor',
+ mean=[0.485, 0.456, 0.406],
+ std=[0.229, 0.224, 0.225]),
+ dict(
+ type='Collect',
+ keys=['img'],
+ meta_keys=[
+ 'image_file', 'center', 'scale', 'rotation', 'bbox_score',
+ 'flip_pairs'
+ ]),
+]
+
+test_pipeline = val_pipeline
+
+data_root = 'data/coco'
+data = dict(
+ samples_per_gpu=32,
+ workers_per_gpu=2,
+ val_dataloader=dict(samples_per_gpu=32),
+ test_dataloader=dict(samples_per_gpu=32),
+ train=dict(
+ type='TopDownCocoDataset',
+ ann_file=f'{data_root}/annotations/person_keypoints_train2017.json',
+ img_prefix=f'{data_root}/train2017/',
+ data_cfg=data_cfg,
+ pipeline=train_pipeline,
+ dataset_info={{_base_.dataset_info}}),
+ val=dict(
+ type='TopDownCocoDataset',
+ ann_file=f'{data_root}/annotations/person_keypoints_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ data_cfg=data_cfg,
+ pipeline=val_pipeline,
+ dataset_info={{_base_.dataset_info}}),
+ test=dict(
+ type='TopDownCocoDataset',
+ ann_file=f'{data_root}/annotations/person_keypoints_val2017.json',
+ img_prefix=f'{data_root}/val2017/',
+ data_cfg=data_cfg,
+ pipeline=test_pipeline,
+ dataset_info={{_base_.dataset_info}}),
+)
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..ce75553daf861f95dd4e3707b0c9b2ed5de4b1a9
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/__init__.py
@@ -0,0 +1,115 @@
+import cv2
+import numpy as np
+import torch
+import os
+from modules import devices, shared
+from modules.paths import models_path
+from torchvision.transforms import transforms
+
+# AdelaiDepth/LeReS imports
+from .leres.depthmap import estimateleres, estimateboost
+from .leres.multi_depth_model_woauxi import RelDepthModel
+from .leres.net_tools import strip_prefix_if_present
+
+# pix2pix/merge net imports
+from .pix2pix.options.test_options import TestOptions
+from .pix2pix.models.pix2pix4depth_model import Pix2Pix4DepthModel
+
+base_model_path = os.path.join(models_path, "leres")
+old_modeldir = os.path.dirname(os.path.realpath(__file__))
+
+remote_model_path_leres = "https://cloudstor.aarnet.edu.au/plus/s/lTIJF4vrvHCAI31/download"
+remote_model_path_pix2pix = "https://sfu.ca/~yagiz/CVPR21/latest_net_G.pth"
+
+model = None
+pix2pixmodel = None
+
+def unload_leres_model():
+ global model, pix2pixmodel
+ if model is not None:
+ model = model.cpu()
+ if pix2pixmodel is not None:
+ pix2pixmodel = pix2pixmodel.unload_network('G')
+
+def apply_leres(input_image, thr_a, thr_b):
+ global model, pix2pixmodel
+ boost = shared.opts.data.get("control_net_monocular_depth_optim", False)
+
+ if model is None:
+ model_path = os.path.join(base_model_path, "res101.pth")
+ old_model_path = os.path.join(old_modeldir, "res101.pth")
+
+ if os.path.exists(old_model_path):
+ model_path = old_model_path
+ elif not os.path.exists(model_path):
+ from basicsr.utils.download_util import load_file_from_url
+ load_file_from_url(remote_model_path_leres, model_dir=base_model_path)
+ os.rename(os.path.join(base_model_path, 'download'), model_path)
+
+ if torch.cuda.is_available():
+ checkpoint = torch.load(model_path)
+ else:
+ checkpoint = torch.load(model_path,map_location=torch.device('cpu'))
+
+ model = RelDepthModel(backbone='resnext101')
+ model.load_state_dict(strip_prefix_if_present(checkpoint['depth_model'], "module."), strict=True)
+ del checkpoint
+
+ if boost and pix2pixmodel is None:
+ pix2pixmodel_path = os.path.join(base_model_path, "latest_net_G.pth")
+ if not os.path.exists(pix2pixmodel_path):
+ from basicsr.utils.download_util import load_file_from_url
+ load_file_from_url(remote_model_path_pix2pix, model_dir=base_model_path)
+
+ opt = TestOptions().parse()
+ if not torch.cuda.is_available():
+ opt.gpu_ids = [] # cpu mode
+ pix2pixmodel = Pix2Pix4DepthModel(opt)
+ pix2pixmodel.save_dir = base_model_path
+ pix2pixmodel.load_networks('latest')
+ pix2pixmodel.eval()
+
+ if devices.get_device_for("controlnet").type != 'mps':
+ model = model.to(devices.get_device_for("controlnet"))
+
+ assert input_image.ndim == 3
+ height, width, dim = input_image.shape
+
+ with torch.no_grad():
+
+ if boost:
+ depth = estimateboost(input_image, model, 0, pix2pixmodel, max(width, height))
+ else:
+ depth = estimateleres(input_image, model, width, height)
+
+ numbytes=2
+ depth_min = depth.min()
+ depth_max = depth.max()
+ max_val = (2**(8*numbytes))-1
+
+ # check output before normalizing and mapping to 16 bit
+ if depth_max - depth_min > np.finfo("float").eps:
+ out = max_val * (depth - depth_min) / (depth_max - depth_min)
+ else:
+ out = np.zeros(depth.shape)
+
+ # single channel, 16 bit image
+ depth_image = out.astype("uint16")
+
+ # convert to uint8
+ depth_image = cv2.convertScaleAbs(depth_image, alpha=(255.0/65535.0))
+
+ # remove near
+ if thr_a != 0:
+ thr_a = ((thr_a/100)*255)
+ depth_image = cv2.threshold(depth_image, thr_a, 255, cv2.THRESH_TOZERO)[1]
+
+ # invert image
+ depth_image = cv2.bitwise_not(depth_image)
+
+ # remove bg
+ if thr_b != 0:
+ thr_b = ((thr_b/100)*255)
+ depth_image = cv2.threshold(depth_image, thr_b, 255, cv2.THRESH_TOZERO)[1]
+
+ return depth_image
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/LICENSE b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/LICENSE
new file mode 100644
index 0000000000000000000000000000000000000000..e0f1d07d98d4e85e684734d058dfe2515d215405
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/LICENSE
@@ -0,0 +1,23 @@
+https://github.com/thygate/stable-diffusion-webui-depthmap-script
+
+MIT License
+
+Copyright (c) 2023 Bob Thiry
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
\ No newline at end of file
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnet.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnet.py
new file mode 100644
index 0000000000000000000000000000000000000000..f12c9975c1aa05401269be3ca3dbaa56bde55581
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnet.py
@@ -0,0 +1,199 @@
+import torch.nn as nn
+import torch.nn as NN
+
+__all__ = ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101',
+ 'resnet152']
+
+
+model_urls = {
+ 'resnet18': 'https://download.pytorch.org/models/resnet18-5c106cde.pth',
+ 'resnet34': 'https://download.pytorch.org/models/resnet34-333f7ec4.pth',
+ 'resnet50': 'https://download.pytorch.org/models/resnet50-19c8e357.pth',
+ 'resnet101': 'https://download.pytorch.org/models/resnet101-5d3b4d8f.pth',
+ 'resnet152': 'https://download.pytorch.org/models/resnet152-b121ed2d.pth',
+}
+
+
+def conv3x3(in_planes, out_planes, stride=1):
+ """3x3 convolution with padding"""
+ return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
+ padding=1, bias=False)
+
+
+class BasicBlock(nn.Module):
+ expansion = 1
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None):
+ super(BasicBlock, self).__init__()
+ self.conv1 = conv3x3(inplanes, planes, stride)
+ self.bn1 = NN.BatchNorm2d(planes) #NN.BatchNorm2d
+ self.relu = nn.ReLU(inplace=True)
+ self.conv2 = conv3x3(planes, planes)
+ self.bn2 = NN.BatchNorm2d(planes) #NN.BatchNorm2d
+ self.downsample = downsample
+ self.stride = stride
+
+ def forward(self, x):
+ residual = x
+
+ out = self.conv1(x)
+ out = self.bn1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+
+ if self.downsample is not None:
+ residual = self.downsample(x)
+
+ out += residual
+ out = self.relu(out)
+
+ return out
+
+
+class Bottleneck(nn.Module):
+ expansion = 4
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None):
+ super(Bottleneck, self).__init__()
+ self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
+ self.bn1 = NN.BatchNorm2d(planes) #NN.BatchNorm2d
+ self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride,
+ padding=1, bias=False)
+ self.bn2 = NN.BatchNorm2d(planes) #NN.BatchNorm2d
+ self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
+ self.bn3 = NN.BatchNorm2d(planes * self.expansion) #NN.BatchNorm2d
+ self.relu = nn.ReLU(inplace=True)
+ self.downsample = downsample
+ self.stride = stride
+
+ def forward(self, x):
+ residual = x
+
+ out = self.conv1(x)
+ out = self.bn1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+ out = self.relu(out)
+
+ out = self.conv3(out)
+ out = self.bn3(out)
+
+ if self.downsample is not None:
+ residual = self.downsample(x)
+
+ out += residual
+ out = self.relu(out)
+
+ return out
+
+
+class ResNet(nn.Module):
+
+ def __init__(self, block, layers, num_classes=1000):
+ self.inplanes = 64
+ super(ResNet, self).__init__()
+ self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
+ bias=False)
+ self.bn1 = NN.BatchNorm2d(64) #NN.BatchNorm2d
+ self.relu = nn.ReLU(inplace=True)
+ self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+ self.layer1 = self._make_layer(block, 64, layers[0])
+ self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
+ self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
+ self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
+ #self.avgpool = nn.AvgPool2d(7, stride=1)
+ #self.fc = nn.Linear(512 * block.expansion, num_classes)
+
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
+ elif isinstance(m, nn.BatchNorm2d):
+ nn.init.constant_(m.weight, 1)
+ nn.init.constant_(m.bias, 0)
+
+ def _make_layer(self, block, planes, blocks, stride=1):
+ downsample = None
+ if stride != 1 or self.inplanes != planes * block.expansion:
+ downsample = nn.Sequential(
+ nn.Conv2d(self.inplanes, planes * block.expansion,
+ kernel_size=1, stride=stride, bias=False),
+ NN.BatchNorm2d(planes * block.expansion), #NN.BatchNorm2d
+ )
+
+ layers = []
+ layers.append(block(self.inplanes, planes, stride, downsample))
+ self.inplanes = planes * block.expansion
+ for i in range(1, blocks):
+ layers.append(block(self.inplanes, planes))
+
+ return nn.Sequential(*layers)
+
+ def forward(self, x):
+ features = []
+
+ x = self.conv1(x)
+ x = self.bn1(x)
+ x = self.relu(x)
+ x = self.maxpool(x)
+
+ x = self.layer1(x)
+ features.append(x)
+ x = self.layer2(x)
+ features.append(x)
+ x = self.layer3(x)
+ features.append(x)
+ x = self.layer4(x)
+ features.append(x)
+
+ return features
+
+
+def resnet18(pretrained=True, **kwargs):
+ """Constructs a ResNet-18 model.
+ Args:
+ pretrained (bool): If True, returns a model pre-trained on ImageNet
+ """
+ model = ResNet(BasicBlock, [2, 2, 2, 2], **kwargs)
+ return model
+
+
+def resnet34(pretrained=True, **kwargs):
+ """Constructs a ResNet-34 model.
+ Args:
+ pretrained (bool): If True, returns a model pre-trained on ImageNet
+ """
+ model = ResNet(BasicBlock, [3, 4, 6, 3], **kwargs)
+ return model
+
+
+def resnet50(pretrained=True, **kwargs):
+ """Constructs a ResNet-50 model.
+ Args:
+ pretrained (bool): If True, returns a model pre-trained on ImageNet
+ """
+ model = ResNet(Bottleneck, [3, 4, 6, 3], **kwargs)
+
+ return model
+
+
+def resnet101(pretrained=True, **kwargs):
+ """Constructs a ResNet-101 model.
+ Args:
+ pretrained (bool): If True, returns a model pre-trained on ImageNet
+ """
+ model = ResNet(Bottleneck, [3, 4, 23, 3], **kwargs)
+
+ return model
+
+
+def resnet152(pretrained=True, **kwargs):
+ """Constructs a ResNet-152 model.
+ Args:
+ pretrained (bool): If True, returns a model pre-trained on ImageNet
+ """
+ model = ResNet(Bottleneck, [3, 8, 36, 3], **kwargs)
+ return model
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnext_torch.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnext_torch.py
new file mode 100644
index 0000000000000000000000000000000000000000..e5ce4c50a4975acf02079488e42cfd9d686572d3
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnext_torch.py
@@ -0,0 +1,247 @@
+#!/usr/bin/env python
+# coding: utf-8
+import torch.nn as nn
+
+try:
+ from urllib import urlretrieve
+except ImportError:
+ from urllib.request import urlretrieve
+
+__all__ = ['resnext101_32x8d']
+
+
+model_urls = {
+ 'resnext50_32x4d': 'https://download.pytorch.org/models/resnext50_32x4d-7cdf4587.pth',
+ 'resnext101_32x8d': 'https://download.pytorch.org/models/resnext101_32x8d-8ba56ff5.pth',
+}
+
+
+def conv3x3(in_planes, out_planes, stride=1, groups=1, dilation=1):
+ """3x3 convolution with padding"""
+ return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,
+ padding=dilation, groups=groups, bias=False, dilation=dilation)
+
+
+def conv1x1(in_planes, out_planes, stride=1):
+ """1x1 convolution"""
+ return nn.Conv2d(in_planes, out_planes, kernel_size=1, stride=stride, bias=False)
+
+
+class BasicBlock(nn.Module):
+ expansion = 1
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None, groups=1,
+ base_width=64, dilation=1, norm_layer=None):
+ super(BasicBlock, self).__init__()
+ if norm_layer is None:
+ norm_layer = nn.BatchNorm2d
+ if groups != 1 or base_width != 64:
+ raise ValueError('BasicBlock only supports groups=1 and base_width=64')
+ if dilation > 1:
+ raise NotImplementedError("Dilation > 1 not supported in BasicBlock")
+ # Both self.conv1 and self.downsample layers downsample the input when stride != 1
+ self.conv1 = conv3x3(inplanes, planes, stride)
+ self.bn1 = norm_layer(planes)
+ self.relu = nn.ReLU(inplace=True)
+ self.conv2 = conv3x3(planes, planes)
+ self.bn2 = norm_layer(planes)
+ self.downsample = downsample
+ self.stride = stride
+
+ def forward(self, x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.bn1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+ out = self.relu(out)
+
+ return out
+
+
+class Bottleneck(nn.Module):
+ # Bottleneck in torchvision places the stride for downsampling at 3x3 convolution(self.conv2)
+ # while original implementation places the stride at the first 1x1 convolution(self.conv1)
+ # according to "Deep residual learning for image recognition"https://arxiv.org/abs/1512.03385.
+ # This variant is also known as ResNet V1.5 and improves accuracy according to
+ # https://ngc.nvidia.com/catalog/model-scripts/nvidia:resnet_50_v1_5_for_pytorch.
+
+ expansion = 4
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None, groups=1,
+ base_width=64, dilation=1, norm_layer=None):
+ super(Bottleneck, self).__init__()
+ if norm_layer is None:
+ norm_layer = nn.BatchNorm2d
+ width = int(planes * (base_width / 64.)) * groups
+ # Both self.conv2 and self.downsample layers downsample the input when stride != 1
+ self.conv1 = conv1x1(inplanes, width)
+ self.bn1 = norm_layer(width)
+ self.conv2 = conv3x3(width, width, stride, groups, dilation)
+ self.bn2 = norm_layer(width)
+ self.conv3 = conv1x1(width, planes * self.expansion)
+ self.bn3 = norm_layer(planes * self.expansion)
+ self.relu = nn.ReLU(inplace=True)
+ self.downsample = downsample
+ self.stride = stride
+
+ def forward(self, x):
+ identity = x
+
+ out = self.conv1(x)
+ out = self.bn1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+ out = self.relu(out)
+
+ out = self.conv3(out)
+ out = self.bn3(out)
+
+ if self.downsample is not None:
+ identity = self.downsample(x)
+
+ out += identity
+ out = self.relu(out)
+
+ return out
+
+
+class ResNet(nn.Module):
+
+ def __init__(self, block, layers, num_classes=1000, zero_init_residual=False,
+ groups=1, width_per_group=64, replace_stride_with_dilation=None,
+ norm_layer=None):
+ super(ResNet, self).__init__()
+ if norm_layer is None:
+ norm_layer = nn.BatchNorm2d
+ self._norm_layer = norm_layer
+
+ self.inplanes = 64
+ self.dilation = 1
+ if replace_stride_with_dilation is None:
+ # each element in the tuple indicates if we should replace
+ # the 2x2 stride with a dilated convolution instead
+ replace_stride_with_dilation = [False, False, False]
+ if len(replace_stride_with_dilation) != 3:
+ raise ValueError("replace_stride_with_dilation should be None "
+ "or a 3-element tuple, got {}".format(replace_stride_with_dilation))
+ self.groups = groups
+ self.base_width = width_per_group
+ self.conv1 = nn.Conv2d(3, self.inplanes, kernel_size=7, stride=2, padding=3,
+ bias=False)
+ self.bn1 = norm_layer(self.inplanes)
+ self.relu = nn.ReLU(inplace=True)
+ self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
+ self.layer1 = self._make_layer(block, 64, layers[0])
+ self.layer2 = self._make_layer(block, 128, layers[1], stride=2,
+ dilate=replace_stride_with_dilation[0])
+ self.layer3 = self._make_layer(block, 256, layers[2], stride=2,
+ dilate=replace_stride_with_dilation[1])
+ self.layer4 = self._make_layer(block, 512, layers[3], stride=2,
+ dilate=replace_stride_with_dilation[2])
+ #self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
+ #self.fc = nn.Linear(512 * block.expansion, num_classes)
+
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
+ elif isinstance(m, (nn.BatchNorm2d, nn.GroupNorm)):
+ nn.init.constant_(m.weight, 1)
+ nn.init.constant_(m.bias, 0)
+
+ # Zero-initialize the last BN in each residual branch,
+ # so that the residual branch starts with zeros, and each residual block behaves like an identity.
+ # This improves the model by 0.2~0.3% according to https://arxiv.org/abs/1706.02677
+ if zero_init_residual:
+ for m in self.modules():
+ if isinstance(m, Bottleneck):
+ nn.init.constant_(m.bn3.weight, 0)
+ elif isinstance(m, BasicBlock):
+ nn.init.constant_(m.bn2.weight, 0)
+
+ def _make_layer(self, block, planes, blocks, stride=1, dilate=False):
+ norm_layer = self._norm_layer
+ downsample = None
+ previous_dilation = self.dilation
+ if dilate:
+ self.dilation *= stride
+ stride = 1
+ if stride != 1 or self.inplanes != planes * block.expansion:
+ downsample = nn.Sequential(
+ conv1x1(self.inplanes, planes * block.expansion, stride),
+ norm_layer(planes * block.expansion),
+ )
+
+ layers = []
+ layers.append(block(self.inplanes, planes, stride, downsample, self.groups,
+ self.base_width, previous_dilation, norm_layer))
+ self.inplanes = planes * block.expansion
+ for _ in range(1, blocks):
+ layers.append(block(self.inplanes, planes, groups=self.groups,
+ base_width=self.base_width, dilation=self.dilation,
+ norm_layer=norm_layer))
+
+ return nn.Sequential(*layers)
+
+ def _forward_impl(self, x):
+ # See note [TorchScript super()]
+ features = []
+ x = self.conv1(x)
+ x = self.bn1(x)
+ x = self.relu(x)
+ x = self.maxpool(x)
+
+ x = self.layer1(x)
+ features.append(x)
+
+ x = self.layer2(x)
+ features.append(x)
+
+ x = self.layer3(x)
+ features.append(x)
+
+ x = self.layer4(x)
+ features.append(x)
+
+ #x = self.avgpool(x)
+ #x = torch.flatten(x, 1)
+ #x = self.fc(x)
+
+ return features
+
+ def forward(self, x):
+ return self._forward_impl(x)
+
+
+
+def resnext101_32x8d(pretrained=True, **kwargs):
+ """Constructs a ResNet-152 model.
+ Args:
+ pretrained (bool): If True, returns a model pre-trained on ImageNet
+ """
+ kwargs['groups'] = 32
+ kwargs['width_per_group'] = 8
+
+ model = ResNet(Bottleneck, [3, 4, 23, 3], **kwargs)
+ return model
+
+
+
+if __name__ == '__main__':
+ import torch
+ model = resnext101_32x8d(True).cuda()
+
+ rgb = torch.rand((2, 3, 256, 256)).cuda()
+ out = model(rgb)
+ print(len(out))
+
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py
new file mode 100644
index 0000000000000000000000000000000000000000..ebceecbe28ec248f6f96bb65b1c53bdbaf393ecc
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py
@@ -0,0 +1,546 @@
+# Author: thygate
+# https://github.com/thygate/stable-diffusion-webui-depthmap-script
+
+from modules import devices
+from modules.shared import opts
+from torchvision.transforms import transforms
+from operator import getitem
+
+import torch, gc
+import cv2
+import numpy as np
+import skimage.measure
+
+whole_size_threshold = 1600 # R_max from the paper
+pix2pixsize = 1024
+
+def scale_torch(img):
+ """
+ Scale the image and output it in torch.tensor.
+ :param img: input rgb is in shape [H, W, C], input depth/disp is in shape [H, W]
+ :param scale: the scale factor. float
+ :return: img. [C, H, W]
+ """
+ if len(img.shape) == 2:
+ img = img[np.newaxis, :, :]
+ if img.shape[2] == 3:
+ transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.485, 0.456, 0.406) , (0.229, 0.224, 0.225) )])
+ img = transform(img.astype(np.float32))
+ else:
+ img = img.astype(np.float32)
+ img = torch.from_numpy(img)
+ return img
+
+def estimateleres(img, model, w, h):
+ # leres transform input
+ rgb_c = img[:, :, ::-1].copy()
+ A_resize = cv2.resize(rgb_c, (w, h))
+ img_torch = scale_torch(A_resize)[None, :, :, :]
+
+ # compute
+ with torch.no_grad():
+ img_torch = img_torch.to(devices.get_device_for("controlnet"))
+ prediction = model.depth_model(img_torch)
+
+ prediction = prediction.squeeze().cpu().numpy()
+ prediction = cv2.resize(prediction, (img.shape[1], img.shape[0]), interpolation=cv2.INTER_CUBIC)
+
+ return prediction
+
+def generatemask(size):
+ # Generates a Guassian mask
+ mask = np.zeros(size, dtype=np.float32)
+ sigma = int(size[0]/16)
+ k_size = int(2 * np.ceil(2 * int(size[0]/16)) + 1)
+ mask[int(0.15*size[0]):size[0] - int(0.15*size[0]), int(0.15*size[1]): size[1] - int(0.15*size[1])] = 1
+ mask = cv2.GaussianBlur(mask, (int(k_size), int(k_size)), sigma)
+ mask = (mask - mask.min()) / (mask.max() - mask.min())
+ mask = mask.astype(np.float32)
+ return mask
+
+def resizewithpool(img, size):
+ i_size = img.shape[0]
+ n = int(np.floor(i_size/size))
+
+ out = skimage.measure.block_reduce(img, (n, n), np.max)
+ return out
+
+def rgb2gray(rgb):
+ # Converts rgb to gray
+ return np.dot(rgb[..., :3], [0.2989, 0.5870, 0.1140])
+
+def calculateprocessingres(img, basesize, confidence=0.1, scale_threshold=3, whole_size_threshold=3000):
+ # Returns the R_x resolution described in section 5 of the main paper.
+
+ # Parameters:
+ # img :input rgb image
+ # basesize : size the dilation kernel which is equal to receptive field of the network.
+ # confidence: value of x in R_x; allowed percentage of pixels that are not getting any contextual cue.
+ # scale_threshold: maximum allowed upscaling on the input image ; it has been set to 3.
+ # whole_size_threshold: maximum allowed resolution. (R_max from section 6 of the main paper)
+
+ # Returns:
+ # outputsize_scale*speed_scale :The computed R_x resolution
+ # patch_scale: K parameter from section 6 of the paper
+
+ # speed scale parameter is to process every image in a smaller size to accelerate the R_x resolution search
+ speed_scale = 32
+ image_dim = int(min(img.shape[0:2]))
+
+ gray = rgb2gray(img)
+ grad = np.abs(cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize=3)) + np.abs(cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3))
+ grad = cv2.resize(grad, (image_dim, image_dim), cv2.INTER_AREA)
+
+ # thresholding the gradient map to generate the edge-map as a proxy of the contextual cues
+ m = grad.min()
+ M = grad.max()
+ middle = m + (0.4 * (M - m))
+ grad[grad < middle] = 0
+ grad[grad >= middle] = 1
+
+ # dilation kernel with size of the receptive field
+ kernel = np.ones((int(basesize/speed_scale), int(basesize/speed_scale)), float)
+ # dilation kernel with size of the a quarter of receptive field used to compute k
+ # as described in section 6 of main paper
+ kernel2 = np.ones((int(basesize / (4*speed_scale)), int(basesize / (4*speed_scale))), float)
+
+ # Output resolution limit set by the whole_size_threshold and scale_threshold.
+ threshold = min(whole_size_threshold, scale_threshold * max(img.shape[:2]))
+
+ outputsize_scale = basesize / speed_scale
+ for p_size in range(int(basesize/speed_scale), int(threshold/speed_scale), int(basesize / (2*speed_scale))):
+ grad_resized = resizewithpool(grad, p_size)
+ grad_resized = cv2.resize(grad_resized, (p_size, p_size), cv2.INTER_NEAREST)
+ grad_resized[grad_resized >= 0.5] = 1
+ grad_resized[grad_resized < 0.5] = 0
+
+ dilated = cv2.dilate(grad_resized, kernel, iterations=1)
+ meanvalue = (1-dilated).mean()
+ if meanvalue > confidence:
+ break
+ else:
+ outputsize_scale = p_size
+
+ grad_region = cv2.dilate(grad_resized, kernel2, iterations=1)
+ patch_scale = grad_region.mean()
+
+ return int(outputsize_scale*speed_scale), patch_scale
+
+# Generate a double-input depth estimation
+def doubleestimate(img, size1, size2, pix2pixsize, model, net_type, pix2pixmodel):
+ # Generate the low resolution estimation
+ estimate1 = singleestimate(img, size1, model, net_type)
+ # Resize to the inference size of merge network.
+ estimate1 = cv2.resize(estimate1, (pix2pixsize, pix2pixsize), interpolation=cv2.INTER_CUBIC)
+
+ # Generate the high resolution estimation
+ estimate2 = singleestimate(img, size2, model, net_type)
+ # Resize to the inference size of merge network.
+ estimate2 = cv2.resize(estimate2, (pix2pixsize, pix2pixsize), interpolation=cv2.INTER_CUBIC)
+
+ # Inference on the merge model
+ pix2pixmodel.set_input(estimate1, estimate2)
+ pix2pixmodel.test()
+ visuals = pix2pixmodel.get_current_visuals()
+ prediction_mapped = visuals['fake_B']
+ prediction_mapped = (prediction_mapped+1)/2
+ prediction_mapped = (prediction_mapped - torch.min(prediction_mapped)) / (
+ torch.max(prediction_mapped) - torch.min(prediction_mapped))
+ prediction_mapped = prediction_mapped.squeeze().cpu().numpy()
+
+ return prediction_mapped
+
+# Generate a single-input depth estimation
+def singleestimate(img, msize, model, net_type):
+ # if net_type == 0:
+ return estimateleres(img, model, msize, msize)
+ # else:
+ # return estimatemidasBoost(img, model, msize, msize)
+
+def applyGridpatch(blsize, stride, img, box):
+ # Extract a simple grid patch.
+ counter1 = 0
+ patch_bound_list = {}
+ for k in range(blsize, img.shape[1] - blsize, stride):
+ for j in range(blsize, img.shape[0] - blsize, stride):
+ patch_bound_list[str(counter1)] = {}
+ patchbounds = [j - blsize, k - blsize, j - blsize + 2 * blsize, k - blsize + 2 * blsize]
+ patch_bound = [box[0] + patchbounds[1], box[1] + patchbounds[0], patchbounds[3] - patchbounds[1],
+ patchbounds[2] - patchbounds[0]]
+ patch_bound_list[str(counter1)]['rect'] = patch_bound
+ patch_bound_list[str(counter1)]['size'] = patch_bound[2]
+ counter1 = counter1 + 1
+ return patch_bound_list
+
+# Generating local patches to perform the local refinement described in section 6 of the main paper.
+def generatepatchs(img, base_size):
+
+ # Compute the gradients as a proxy of the contextual cues.
+ img_gray = rgb2gray(img)
+ whole_grad = np.abs(cv2.Sobel(img_gray, cv2.CV_64F, 0, 1, ksize=3)) +\
+ np.abs(cv2.Sobel(img_gray, cv2.CV_64F, 1, 0, ksize=3))
+
+ threshold = whole_grad[whole_grad > 0].mean()
+ whole_grad[whole_grad < threshold] = 0
+
+ # We use the integral image to speed-up the evaluation of the amount of gradients for each patch.
+ gf = whole_grad.sum()/len(whole_grad.reshape(-1))
+ grad_integral_image = cv2.integral(whole_grad)
+
+ # Variables are selected such that the initial patch size would be the receptive field size
+ # and the stride is set to 1/3 of the receptive field size.
+ blsize = int(round(base_size/2))
+ stride = int(round(blsize*0.75))
+
+ # Get initial Grid
+ patch_bound_list = applyGridpatch(blsize, stride, img, [0, 0, 0, 0])
+
+ # Refine initial Grid of patches by discarding the flat (in terms of gradients of the rgb image) ones. Refine
+ # each patch size to ensure that there will be enough depth cues for the network to generate a consistent depth map.
+ print("Selecting patches ...")
+ patch_bound_list = adaptiveselection(grad_integral_image, patch_bound_list, gf)
+
+ # Sort the patch list to make sure the merging operation will be done with the correct order: starting from biggest
+ # patch
+ patchset = sorted(patch_bound_list.items(), key=lambda x: getitem(x[1], 'size'), reverse=True)
+ return patchset
+
+def getGF_fromintegral(integralimage, rect):
+ # Computes the gradient density of a given patch from the gradient integral image.
+ x1 = rect[1]
+ x2 = rect[1]+rect[3]
+ y1 = rect[0]
+ y2 = rect[0]+rect[2]
+ value = integralimage[x2, y2]-integralimage[x1, y2]-integralimage[x2, y1]+integralimage[x1, y1]
+ return value
+
+# Adaptively select patches
+def adaptiveselection(integral_grad, patch_bound_list, gf):
+ patchlist = {}
+ count = 0
+ height, width = integral_grad.shape
+
+ search_step = int(32/factor)
+
+ # Go through all patches
+ for c in range(len(patch_bound_list)):
+ # Get patch
+ bbox = patch_bound_list[str(c)]['rect']
+
+ # Compute the amount of gradients present in the patch from the integral image.
+ cgf = getGF_fromintegral(integral_grad, bbox)/(bbox[2]*bbox[3])
+
+ # Check if patching is beneficial by comparing the gradient density of the patch to
+ # the gradient density of the whole image
+ if cgf >= gf:
+ bbox_test = bbox.copy()
+ patchlist[str(count)] = {}
+
+ # Enlarge each patch until the gradient density of the patch is equal
+ # to the whole image gradient density
+ while True:
+
+ bbox_test[0] = bbox_test[0] - int(search_step/2)
+ bbox_test[1] = bbox_test[1] - int(search_step/2)
+
+ bbox_test[2] = bbox_test[2] + search_step
+ bbox_test[3] = bbox_test[3] + search_step
+
+ # Check if we are still within the image
+ if bbox_test[0] < 0 or bbox_test[1] < 0 or bbox_test[1] + bbox_test[3] >= height \
+ or bbox_test[0] + bbox_test[2] >= width:
+ break
+
+ # Compare gradient density
+ cgf = getGF_fromintegral(integral_grad, bbox_test)/(bbox_test[2]*bbox_test[3])
+ if cgf < gf:
+ break
+ bbox = bbox_test.copy()
+
+ # Add patch to selected patches
+ patchlist[str(count)]['rect'] = bbox
+ patchlist[str(count)]['size'] = bbox[2]
+ count = count + 1
+
+ # Return selected patches
+ return patchlist
+
+def impatch(image, rect):
+ # Extract the given patch pixels from a given image.
+ w1 = rect[0]
+ h1 = rect[1]
+ w2 = w1 + rect[2]
+ h2 = h1 + rect[3]
+ image_patch = image[h1:h2, w1:w2]
+ return image_patch
+
+class ImageandPatchs:
+ def __init__(self, root_dir, name, patchsinfo, rgb_image, scale=1):
+ self.root_dir = root_dir
+ self.patchsinfo = patchsinfo
+ self.name = name
+ self.patchs = patchsinfo
+ self.scale = scale
+
+ self.rgb_image = cv2.resize(rgb_image, (round(rgb_image.shape[1]*scale), round(rgb_image.shape[0]*scale)),
+ interpolation=cv2.INTER_CUBIC)
+
+ self.do_have_estimate = False
+ self.estimation_updated_image = None
+ self.estimation_base_image = None
+
+ def __len__(self):
+ return len(self.patchs)
+
+ def set_base_estimate(self, est):
+ self.estimation_base_image = est
+ if self.estimation_updated_image is not None:
+ self.do_have_estimate = True
+
+ def set_updated_estimate(self, est):
+ self.estimation_updated_image = est
+ if self.estimation_base_image is not None:
+ self.do_have_estimate = True
+
+ def __getitem__(self, index):
+ patch_id = int(self.patchs[index][0])
+ rect = np.array(self.patchs[index][1]['rect'])
+ msize = self.patchs[index][1]['size']
+
+ ## applying scale to rect:
+ rect = np.round(rect * self.scale)
+ rect = rect.astype('int')
+ msize = round(msize * self.scale)
+
+ patch_rgb = impatch(self.rgb_image, rect)
+ if self.do_have_estimate:
+ patch_whole_estimate_base = impatch(self.estimation_base_image, rect)
+ patch_whole_estimate_updated = impatch(self.estimation_updated_image, rect)
+ return {'patch_rgb': patch_rgb, 'patch_whole_estimate_base': patch_whole_estimate_base,
+ 'patch_whole_estimate_updated': patch_whole_estimate_updated, 'rect': rect,
+ 'size': msize, 'id': patch_id}
+ else:
+ return {'patch_rgb': patch_rgb, 'rect': rect, 'size': msize, 'id': patch_id}
+
+ def print_options(self, opt):
+ """Print and save options
+
+ It will print both current options and default values(if different).
+ It will save options into a text file / [checkpoints_dir] / opt.txt
+ """
+ message = ''
+ message += '----------------- Options ---------------\n'
+ for k, v in sorted(vars(opt).items()):
+ comment = ''
+ default = self.parser.get_default(k)
+ if v != default:
+ comment = '\t[default: %s]' % str(default)
+ message += '{:>25}: {:<30}{}\n'.format(str(k), str(v), comment)
+ message += '----------------- End -------------------'
+ print(message)
+
+ # save to the disk
+ """
+ expr_dir = os.path.join(opt.checkpoints_dir, opt.name)
+ util.mkdirs(expr_dir)
+ file_name = os.path.join(expr_dir, '{}_opt.txt'.format(opt.phase))
+ with open(file_name, 'wt') as opt_file:
+ opt_file.write(message)
+ opt_file.write('\n')
+ """
+
+ def parse(self):
+ """Parse our options, create checkpoints directory suffix, and set up gpu device."""
+ opt = self.gather_options()
+ opt.isTrain = self.isTrain # train or test
+
+ # process opt.suffix
+ if opt.suffix:
+ suffix = ('_' + opt.suffix.format(**vars(opt))) if opt.suffix != '' else ''
+ opt.name = opt.name + suffix
+
+ #self.print_options(opt)
+
+ # set gpu ids
+ str_ids = opt.gpu_ids.split(',')
+ opt.gpu_ids = []
+ for str_id in str_ids:
+ id = int(str_id)
+ if id >= 0:
+ opt.gpu_ids.append(id)
+ #if len(opt.gpu_ids) > 0:
+ # torch.cuda.set_device(opt.gpu_ids[0])
+
+ self.opt = opt
+ return self.opt
+
+
+def estimateboost(img, model, model_type, pix2pixmodel, max_res=512):
+ global whole_size_threshold
+
+ # get settings
+ if hasattr(opts, 'depthmap_script_boost_rmax'):
+ whole_size_threshold = opts.depthmap_script_boost_rmax
+
+ if model_type == 0: #leres
+ net_receptive_field_size = 448
+ patch_netsize = 2 * net_receptive_field_size
+ elif model_type == 1: #dpt_beit_large_512
+ net_receptive_field_size = 512
+ patch_netsize = 2 * net_receptive_field_size
+ else: #other midas
+ net_receptive_field_size = 384
+ patch_netsize = 2 * net_receptive_field_size
+
+ gc.collect()
+ devices.torch_gc()
+
+ # Generate mask used to smoothly blend the local pathc estimations to the base estimate.
+ # It is arbitrarily large to avoid artifacts during rescaling for each crop.
+ mask_org = generatemask((3000, 3000))
+ mask = mask_org.copy()
+
+ # Value x of R_x defined in the section 5 of the main paper.
+ r_threshold_value = 0.2
+ #if R0:
+ # r_threshold_value = 0
+
+ input_resolution = img.shape
+ scale_threshold = 3 # Allows up-scaling with a scale up to 3
+
+ # Find the best input resolution R-x. The resolution search described in section 5-double estimation of the main paper and section B of the
+ # supplementary material.
+ whole_image_optimal_size, patch_scale = calculateprocessingres(img, net_receptive_field_size, r_threshold_value, scale_threshold, whole_size_threshold)
+
+ # print('wholeImage being processed in :', whole_image_optimal_size)
+
+ # Generate the base estimate using the double estimation.
+ whole_estimate = doubleestimate(img, net_receptive_field_size, whole_image_optimal_size, pix2pixsize, model, model_type, pix2pixmodel)
+
+ # Compute the multiplier described in section 6 of the main paper to make sure our initial patch can select
+ # small high-density regions of the image.
+ global factor
+ factor = max(min(1, 4 * patch_scale * whole_image_optimal_size / whole_size_threshold), 0.2)
+ # print('Adjust factor is:', 1/factor)
+
+ # Check if Local boosting is beneficial.
+ if max_res < whole_image_optimal_size:
+ # print("No Local boosting. Specified Max Res is smaller than R20, Returning doubleestimate result")
+ return cv2.resize(whole_estimate, (input_resolution[1], input_resolution[0]), interpolation=cv2.INTER_CUBIC)
+
+ # Compute the default target resolution.
+ if img.shape[0] > img.shape[1]:
+ a = 2 * whole_image_optimal_size
+ b = round(2 * whole_image_optimal_size * img.shape[1] / img.shape[0])
+ else:
+ a = round(2 * whole_image_optimal_size * img.shape[0] / img.shape[1])
+ b = 2 * whole_image_optimal_size
+ b = int(round(b / factor))
+ a = int(round(a / factor))
+
+ """
+ # recompute a, b and saturate to max res.
+ if max(a,b) > max_res:
+ print('Default Res is higher than max-res: Reducing final resolution')
+ if img.shape[0] > img.shape[1]:
+ a = max_res
+ b = round(max_res * img.shape[1] / img.shape[0])
+ else:
+ a = round(max_res * img.shape[0] / img.shape[1])
+ b = max_res
+ b = int(b)
+ a = int(a)
+ """
+
+ img = cv2.resize(img, (b, a), interpolation=cv2.INTER_CUBIC)
+
+ # Extract selected patches for local refinement
+ base_size = net_receptive_field_size * 2
+ patchset = generatepatchs(img, base_size)
+
+ # print('Target resolution: ', img.shape)
+
+ # Computing a scale in case user prompted to generate the results as the same resolution of the input.
+ # Notice that our method output resolution is independent of the input resolution and this parameter will only
+ # enable a scaling operation during the local patch merge implementation to generate results with the same resolution
+ # as the input.
+ """
+ if output_resolution == 1:
+ mergein_scale = input_resolution[0] / img.shape[0]
+ print('Dynamicly change merged-in resolution; scale:', mergein_scale)
+ else:
+ mergein_scale = 1
+ """
+ # always rescale to input res for now
+ mergein_scale = input_resolution[0] / img.shape[0]
+
+ imageandpatchs = ImageandPatchs('', '', patchset, img, mergein_scale)
+ whole_estimate_resized = cv2.resize(whole_estimate, (round(img.shape[1]*mergein_scale),
+ round(img.shape[0]*mergein_scale)), interpolation=cv2.INTER_CUBIC)
+ imageandpatchs.set_base_estimate(whole_estimate_resized.copy())
+ imageandpatchs.set_updated_estimate(whole_estimate_resized.copy())
+
+ print('Resulting depthmap resolution will be :', whole_estimate_resized.shape[:2])
+ print('Patches to process: '+str(len(imageandpatchs)))
+
+ # Enumerate through all patches, generate their estimations and refining the base estimate.
+ for patch_ind in range(len(imageandpatchs)):
+
+ # Get patch information
+ patch = imageandpatchs[patch_ind] # patch object
+ patch_rgb = patch['patch_rgb'] # rgb patch
+ patch_whole_estimate_base = patch['patch_whole_estimate_base'] # corresponding patch from base
+ rect = patch['rect'] # patch size and location
+ patch_id = patch['id'] # patch ID
+ org_size = patch_whole_estimate_base.shape # the original size from the unscaled input
+ print('\t Processing patch', patch_ind, '/', len(imageandpatchs)-1, '|', rect)
+
+ # We apply double estimation for patches. The high resolution value is fixed to twice the receptive
+ # field size of the network for patches to accelerate the process.
+ patch_estimation = doubleestimate(patch_rgb, net_receptive_field_size, patch_netsize, pix2pixsize, model, model_type, pix2pixmodel)
+ patch_estimation = cv2.resize(patch_estimation, (pix2pixsize, pix2pixsize), interpolation=cv2.INTER_CUBIC)
+ patch_whole_estimate_base = cv2.resize(patch_whole_estimate_base, (pix2pixsize, pix2pixsize), interpolation=cv2.INTER_CUBIC)
+
+ # Merging the patch estimation into the base estimate using our merge network:
+ # We feed the patch estimation and the same region from the updated base estimate to the merge network
+ # to generate the target estimate for the corresponding region.
+ pix2pixmodel.set_input(patch_whole_estimate_base, patch_estimation)
+
+ # Run merging network
+ pix2pixmodel.test()
+ visuals = pix2pixmodel.get_current_visuals()
+
+ prediction_mapped = visuals['fake_B']
+ prediction_mapped = (prediction_mapped+1)/2
+ prediction_mapped = prediction_mapped.squeeze().cpu().numpy()
+
+ mapped = prediction_mapped
+
+ # We use a simple linear polynomial to make sure the result of the merge network would match the values of
+ # base estimate
+ p_coef = np.polyfit(mapped.reshape(-1), patch_whole_estimate_base.reshape(-1), deg=1)
+ merged = np.polyval(p_coef, mapped.reshape(-1)).reshape(mapped.shape)
+
+ merged = cv2.resize(merged, (org_size[1],org_size[0]), interpolation=cv2.INTER_CUBIC)
+
+ # Get patch size and location
+ w1 = rect[0]
+ h1 = rect[1]
+ w2 = w1 + rect[2]
+ h2 = h1 + rect[3]
+
+ # To speed up the implementation, we only generate the Gaussian mask once with a sufficiently large size
+ # and resize it to our needed size while merging the patches.
+ if mask.shape != org_size:
+ mask = cv2.resize(mask_org, (org_size[1],org_size[0]), interpolation=cv2.INTER_LINEAR)
+
+ tobemergedto = imageandpatchs.estimation_updated_image
+
+ # Update the whole estimation:
+ # We use a simple Gaussian mask to blend the merged patch region with the base estimate to ensure seamless
+ # blending at the boundaries of the patch region.
+ tobemergedto[h1:h2, w1:w2] = np.multiply(tobemergedto[h1:h2, w1:w2], 1 - mask) + np.multiply(merged, mask)
+ imageandpatchs.set_updated_estimate(tobemergedto)
+
+ # output
+ return cv2.resize(imageandpatchs.estimation_updated_image, (input_resolution[1], input_resolution[0]), interpolation=cv2.INTER_CUBIC)
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/multi_depth_model_woauxi.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/multi_depth_model_woauxi.py
new file mode 100644
index 0000000000000000000000000000000000000000..822ab0893267042446c2a24ed35b4ea053c9914a
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/multi_depth_model_woauxi.py
@@ -0,0 +1,34 @@
+from . import network_auxi as network
+from .net_tools import get_func
+import torch
+import torch.nn as nn
+from modules import devices
+
+class RelDepthModel(nn.Module):
+ def __init__(self, backbone='resnet50'):
+ super(RelDepthModel, self).__init__()
+ if backbone == 'resnet50':
+ encoder = 'resnet50_stride32'
+ elif backbone == 'resnext101':
+ encoder = 'resnext101_stride32x8d'
+ self.depth_model = DepthModel(encoder)
+
+ def inference(self, rgb):
+ with torch.no_grad():
+ input = rgb.to(self.depth_model.device)
+ depth = self.depth_model(input)
+ #pred_depth_out = depth - depth.min() + 0.01
+ return depth #pred_depth_out
+
+
+class DepthModel(nn.Module):
+ def __init__(self, encoder):
+ super(DepthModel, self).__init__()
+ backbone = network.__name__.split('.')[-1] + '.' + encoder
+ self.encoder_modules = get_func(backbone)()
+ self.decoder_modules = network.Decoder()
+
+ def forward(self, x):
+ lateral_out = self.encoder_modules(x)
+ out_logit = self.decoder_modules(lateral_out)
+ return out_logit
\ No newline at end of file
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/net_tools.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/net_tools.py
new file mode 100644
index 0000000000000000000000000000000000000000..31bb8332b0a1dc16057367f2bed7b145006d5cd8
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/net_tools.py
@@ -0,0 +1,53 @@
+import importlib
+import torch
+import os
+from collections import OrderedDict
+
+
+def get_func(func_name):
+ """Helper to return a function object by name. func_name must identify a
+ function in this module or the path to a function relative to the base
+ 'modeling' module.
+ """
+ if func_name == '':
+ return None
+ try:
+ parts = func_name.split('.')
+ # Refers to a function in this module
+ if len(parts) == 1:
+ return globals()[parts[0]]
+ # Otherwise, assume we're referencing a module under modeling
+ module_name = 'annotator.leres.leres.' + '.'.join(parts[:-1])
+ module = importlib.import_module(module_name)
+ return getattr(module, parts[-1])
+ except Exception:
+ print('Failed to f1ind function: %s', func_name)
+ raise
+
+def load_ckpt(args, depth_model, shift_model, focal_model):
+ """
+ Load checkpoint.
+ """
+ if os.path.isfile(args.load_ckpt):
+ print("loading checkpoint %s" % args.load_ckpt)
+ checkpoint = torch.load(args.load_ckpt)
+ if shift_model is not None:
+ shift_model.load_state_dict(strip_prefix_if_present(checkpoint['shift_model'], 'module.'),
+ strict=True)
+ if focal_model is not None:
+ focal_model.load_state_dict(strip_prefix_if_present(checkpoint['focal_model'], 'module.'),
+ strict=True)
+ depth_model.load_state_dict(strip_prefix_if_present(checkpoint['depth_model'], "module."),
+ strict=True)
+ del checkpoint
+ torch.cuda.empty_cache()
+
+
+def strip_prefix_if_present(state_dict, prefix):
+ keys = sorted(state_dict.keys())
+ if not all(key.startswith(prefix) for key in keys):
+ return state_dict
+ stripped_state_dict = OrderedDict()
+ for key, value in state_dict.items():
+ stripped_state_dict[key.replace(prefix, "")] = value
+ return stripped_state_dict
\ No newline at end of file
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/network_auxi.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/network_auxi.py
new file mode 100644
index 0000000000000000000000000000000000000000..1bd87011a5339aca632d1a10b217c8737bdc794f
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/network_auxi.py
@@ -0,0 +1,417 @@
+import torch
+import torch.nn as nn
+import torch.nn.init as init
+
+from . import Resnet, Resnext_torch
+
+
+def resnet50_stride32():
+ return DepthNet(backbone='resnet', depth=50, upfactors=[2, 2, 2, 2])
+
+def resnext101_stride32x8d():
+ return DepthNet(backbone='resnext101_32x8d', depth=101, upfactors=[2, 2, 2, 2])
+
+
+class Decoder(nn.Module):
+ def __init__(self):
+ super(Decoder, self).__init__()
+ self.inchannels = [256, 512, 1024, 2048]
+ self.midchannels = [256, 256, 256, 512]
+ self.upfactors = [2,2,2,2]
+ self.outchannels = 1
+
+ self.conv = FTB(inchannels=self.inchannels[3], midchannels=self.midchannels[3])
+ self.conv1 = nn.Conv2d(in_channels=self.midchannels[3], out_channels=self.midchannels[2], kernel_size=3, padding=1, stride=1, bias=True)
+ self.upsample = nn.Upsample(scale_factor=self.upfactors[3], mode='bilinear', align_corners=True)
+
+ self.ffm2 = FFM(inchannels=self.inchannels[2], midchannels=self.midchannels[2], outchannels = self.midchannels[2], upfactor=self.upfactors[2])
+ self.ffm1 = FFM(inchannels=self.inchannels[1], midchannels=self.midchannels[1], outchannels = self.midchannels[1], upfactor=self.upfactors[1])
+ self.ffm0 = FFM(inchannels=self.inchannels[0], midchannels=self.midchannels[0], outchannels = self.midchannels[0], upfactor=self.upfactors[0])
+
+ self.outconv = AO(inchannels=self.midchannels[0], outchannels=self.outchannels, upfactor=2)
+ self._init_params()
+
+ def _init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): #NN.BatchNorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+ def forward(self, features):
+ x_32x = self.conv(features[3]) # 1/32
+ x_32 = self.conv1(x_32x)
+ x_16 = self.upsample(x_32) # 1/16
+
+ x_8 = self.ffm2(features[2], x_16) # 1/8
+ x_4 = self.ffm1(features[1], x_8) # 1/4
+ x_2 = self.ffm0(features[0], x_4) # 1/2
+ #-----------------------------------------
+ x = self.outconv(x_2) # original size
+ return x
+
+class DepthNet(nn.Module):
+ __factory = {
+ 18: Resnet.resnet18,
+ 34: Resnet.resnet34,
+ 50: Resnet.resnet50,
+ 101: Resnet.resnet101,
+ 152: Resnet.resnet152
+ }
+ def __init__(self,
+ backbone='resnet',
+ depth=50,
+ upfactors=[2, 2, 2, 2]):
+ super(DepthNet, self).__init__()
+ self.backbone = backbone
+ self.depth = depth
+ self.pretrained = False
+ self.inchannels = [256, 512, 1024, 2048]
+ self.midchannels = [256, 256, 256, 512]
+ self.upfactors = upfactors
+ self.outchannels = 1
+
+ # Build model
+ if self.backbone == 'resnet':
+ if self.depth not in DepthNet.__factory:
+ raise KeyError("Unsupported depth:", self.depth)
+ self.encoder = DepthNet.__factory[depth](pretrained=self.pretrained)
+ elif self.backbone == 'resnext101_32x8d':
+ self.encoder = Resnext_torch.resnext101_32x8d(pretrained=self.pretrained)
+ else:
+ self.encoder = Resnext_torch.resnext101(pretrained=self.pretrained)
+
+ def forward(self, x):
+ x = self.encoder(x) # 1/32, 1/16, 1/8, 1/4
+ return x
+
+
+class FTB(nn.Module):
+ def __init__(self, inchannels, midchannels=512):
+ super(FTB, self).__init__()
+ self.in1 = inchannels
+ self.mid = midchannels
+ self.conv1 = nn.Conv2d(in_channels=self.in1, out_channels=self.mid, kernel_size=3, padding=1, stride=1,
+ bias=True)
+ # NN.BatchNorm2d
+ self.conv_branch = nn.Sequential(nn.ReLU(inplace=True), \
+ nn.Conv2d(in_channels=self.mid, out_channels=self.mid, kernel_size=3,
+ padding=1, stride=1, bias=True), \
+ nn.BatchNorm2d(num_features=self.mid), \
+ nn.ReLU(inplace=True), \
+ nn.Conv2d(in_channels=self.mid, out_channels=self.mid, kernel_size=3,
+ padding=1, stride=1, bias=True))
+ self.relu = nn.ReLU(inplace=True)
+
+ self.init_params()
+
+ def forward(self, x):
+ x = self.conv1(x)
+ x = x + self.conv_branch(x)
+ x = self.relu(x)
+
+ return x
+
+ def init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): # NN.BatchNorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+
+class ATA(nn.Module):
+ def __init__(self, inchannels, reduction=8):
+ super(ATA, self).__init__()
+ self.inchannels = inchannels
+ self.avg_pool = nn.AdaptiveAvgPool2d(1)
+ self.fc = nn.Sequential(nn.Linear(self.inchannels * 2, self.inchannels // reduction),
+ nn.ReLU(inplace=True),
+ nn.Linear(self.inchannels // reduction, self.inchannels),
+ nn.Sigmoid())
+ self.init_params()
+
+ def forward(self, low_x, high_x):
+ n, c, _, _ = low_x.size()
+ x = torch.cat([low_x, high_x], 1)
+ x = self.avg_pool(x)
+ x = x.view(n, -1)
+ x = self.fc(x).view(n, c, 1, 1)
+ x = low_x * x + high_x
+
+ return x
+
+ def init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ # init.normal(m.weight, std=0.01)
+ init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ # init.normal_(m.weight, std=0.01)
+ init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): # NN.BatchNorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+
+class FFM(nn.Module):
+ def __init__(self, inchannels, midchannels, outchannels, upfactor=2):
+ super(FFM, self).__init__()
+ self.inchannels = inchannels
+ self.midchannels = midchannels
+ self.outchannels = outchannels
+ self.upfactor = upfactor
+
+ self.ftb1 = FTB(inchannels=self.inchannels, midchannels=self.midchannels)
+ # self.ata = ATA(inchannels = self.midchannels)
+ self.ftb2 = FTB(inchannels=self.midchannels, midchannels=self.outchannels)
+
+ self.upsample = nn.Upsample(scale_factor=self.upfactor, mode='bilinear', align_corners=True)
+
+ self.init_params()
+
+ def forward(self, low_x, high_x):
+ x = self.ftb1(low_x)
+ x = x + high_x
+ x = self.ftb2(x)
+ x = self.upsample(x)
+
+ return x
+
+ def init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): # NN.Batchnorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+
+class AO(nn.Module):
+ # Adaptive output module
+ def __init__(self, inchannels, outchannels, upfactor=2):
+ super(AO, self).__init__()
+ self.inchannels = inchannels
+ self.outchannels = outchannels
+ self.upfactor = upfactor
+
+ self.adapt_conv = nn.Sequential(
+ nn.Conv2d(in_channels=self.inchannels, out_channels=self.inchannels // 2, kernel_size=3, padding=1,
+ stride=1, bias=True), \
+ nn.BatchNorm2d(num_features=self.inchannels // 2), \
+ nn.ReLU(inplace=True), \
+ nn.Conv2d(in_channels=self.inchannels // 2, out_channels=self.outchannels, kernel_size=3, padding=1,
+ stride=1, bias=True), \
+ nn.Upsample(scale_factor=self.upfactor, mode='bilinear', align_corners=True))
+
+ self.init_params()
+
+ def forward(self, x):
+ x = self.adapt_conv(x)
+ return x
+
+ def init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): # NN.Batchnorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+
+
+# ==============================================================================================================
+
+
+class ResidualConv(nn.Module):
+ def __init__(self, inchannels):
+ super(ResidualConv, self).__init__()
+ # NN.BatchNorm2d
+ self.conv = nn.Sequential(
+ # nn.BatchNorm2d(num_features=inchannels),
+ nn.ReLU(inplace=False),
+ # nn.Conv2d(in_channels=inchannels, out_channels=inchannels, kernel_size=3, padding=1, stride=1, groups=inchannels,bias=True),
+ # nn.Conv2d(in_channels=inchannels, out_channels=inchannels, kernel_size=1, padding=0, stride=1, groups=1,bias=True)
+ nn.Conv2d(in_channels=inchannels, out_channels=inchannels / 2, kernel_size=3, padding=1, stride=1,
+ bias=False),
+ nn.BatchNorm2d(num_features=inchannels / 2),
+ nn.ReLU(inplace=False),
+ nn.Conv2d(in_channels=inchannels / 2, out_channels=inchannels, kernel_size=3, padding=1, stride=1,
+ bias=False)
+ )
+ self.init_params()
+
+ def forward(self, x):
+ x = self.conv(x) + x
+ return x
+
+ def init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): # NN.BatchNorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+
+class FeatureFusion(nn.Module):
+ def __init__(self, inchannels, outchannels):
+ super(FeatureFusion, self).__init__()
+ self.conv = ResidualConv(inchannels=inchannels)
+ # NN.BatchNorm2d
+ self.up = nn.Sequential(ResidualConv(inchannels=inchannels),
+ nn.ConvTranspose2d(in_channels=inchannels, out_channels=outchannels, kernel_size=3,
+ stride=2, padding=1, output_padding=1),
+ nn.BatchNorm2d(num_features=outchannels),
+ nn.ReLU(inplace=True))
+
+ def forward(self, lowfeat, highfeat):
+ return self.up(highfeat + self.conv(lowfeat))
+
+ def init_params(self):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # init.kaiming_normal_(m.weight, mode='fan_out')
+ init.normal_(m.weight, std=0.01)
+ # init.xavier_normal_(m.weight)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.BatchNorm2d): # NN.BatchNorm2d
+ init.constant_(m.weight, 1)
+ init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ init.normal_(m.weight, std=0.01)
+ if m.bias is not None:
+ init.constant_(m.bias, 0)
+
+
+class SenceUnderstand(nn.Module):
+ def __init__(self, channels):
+ super(SenceUnderstand, self).__init__()
+ self.channels = channels
+ self.conv1 = nn.Sequential(nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, padding=1),
+ nn.ReLU(inplace=True))
+ self.pool = nn.AdaptiveAvgPool2d(8)
+ self.fc = nn.Sequential(nn.Linear(512 * 8 * 8, self.channels),
+ nn.ReLU(inplace=True))
+ self.conv2 = nn.Sequential(
+ nn.Conv2d(in_channels=self.channels, out_channels=self.channels, kernel_size=1, padding=0),
+ nn.ReLU(inplace=True))
+ self.initial_params()
+
+ def forward(self, x):
+ n, c, h, w = x.size()
+ x = self.conv1(x)
+ x = self.pool(x)
+ x = x.view(n, -1)
+ x = self.fc(x)
+ x = x.view(n, self.channels, 1, 1)
+ x = self.conv2(x)
+ x = x.repeat(1, 1, h, w)
+ return x
+
+ def initial_params(self, dev=0.01):
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ # print torch.sum(m.weight)
+ m.weight.data.normal_(0, dev)
+ if m.bias is not None:
+ m.bias.data.fill_(0)
+ elif isinstance(m, nn.ConvTranspose2d):
+ # print torch.sum(m.weight)
+ m.weight.data.normal_(0, dev)
+ if m.bias is not None:
+ m.bias.data.fill_(0)
+ elif isinstance(m, nn.Linear):
+ m.weight.data.normal_(0, dev)
+
+
+if __name__ == '__main__':
+ net = DepthNet(depth=50, pretrained=True)
+ print(net)
+ inputs = torch.ones(4,3,128,128)
+ out = net(inputs)
+ print(out.size())
+
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/pix2pix/LICENSE b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/pix2pix/LICENSE
new file mode 100644
index 0000000000000000000000000000000000000000..38b1a24fd389a138b930dcf1ee606ef97a0186c8
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/pix2pix/LICENSE
@@ -0,0 +1,19 @@
+https://github.com/compphoto/BoostingMonocularDepth
+
+Copyright 2021, Seyed Mahdi Hosseini Miangoleh, Sebastian Dille, Computational Photography Laboratory. All rights reserved.
+
+This software is for academic use only. A redistribution of this
+software, with or without modifications, has to be for academic
+use only, while giving the appropriate credit to the original
+authors of the software. The methods implemented as a part of
+this software may be covered under patents or patent applications.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR ''AS IS'' AND ANY EXPRESS OR IMPLIED
+WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
\ No newline at end of file
diff --git a/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/pix2pix/models/__init__.py b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/pix2pix/models/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..f96e5c7f032f2154c6bb433b68fc968d0a19b5a8
--- /dev/null
+++ b/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/pix2pix/models/__init__.py
@@ -0,0 +1,67 @@
+"""This package contains modules related to objective functions, optimizations, and network architectures.
+
+To add a custom model class called 'dummy', you need to add a file called 'dummy_model.py' and define a subclass DummyModel inherited from BaseModel.
+You need to implement the following five functions:
+ -- <__init__>: initialize the class; first call BaseModel.__init__(self, opt).
+ -- Invert colors if your image has white background.
Change your brush width to make it thinner if you want to draw something.
+
+
diff --git a/sd/stable-diffusion-webui/html/extra-networks-no-cards.html b/sd/stable-diffusion-webui/html/extra-networks-no-cards.html
new file mode 100644
index 0000000000000000000000000000000000000000..389358d6c4b383fdc3c5686e029e7b3b1ae9a493
--- /dev/null
+++ b/sd/stable-diffusion-webui/html/extra-networks-no-cards.html
@@ -0,0 +1,8 @@
+
+
+
+
+ {name}
+ -
+ replace preview
+
+
+
diff --git a/sd/stable-diffusion-webui/html/footer.html b/sd/stable-diffusion-webui/html/footer.html
new file mode 100644
index 0000000000000000000000000000000000000000..f26e32e9304aedb5a55b0b46a913396f16375f7a
--- /dev/null
+++ b/sd/stable-diffusion-webui/html/footer.html
@@ -0,0 +1,13 @@
+
+Nothing here. Add some content to the following directories:
+ +-
+{dirs}
+
+
+{versions}
+
diff --git a/sd/stable-diffusion-webui/html/image-update.svg b/sd/stable-diffusion-webui/html/image-update.svg
new file mode 100644
index 0000000000000000000000000000000000000000..3abf12df0f7774c13203e3c49ec3544649df42f4
--- /dev/null
+++ b/sd/stable-diffusion-webui/html/image-update.svg
@@ -0,0 +1,7 @@
+
diff --git a/sd/stable-diffusion-webui/html/licenses.html b/sd/stable-diffusion-webui/html/licenses.html
new file mode 100644
index 0000000000000000000000000000000000000000..f59c352510f95a5d57df7808459c5eb5b21367a9
--- /dev/null
+++ b/sd/stable-diffusion-webui/html/licenses.html
@@ -0,0 +1,419 @@
+
+
+CodeFormer
+Parts of CodeFormer code had to be copied to be compatible with GFPGAN. ++S-Lab License 1.0 + +Copyright 2022 S-Lab + +Redistribution and use for non-commercial purpose in source and +binary forms, with or without modification, are permitted provided +that the following conditions are met: + +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +In the event that redistribution and/or use for commercial purpose in +source or binary forms, with or without modification is required, +please contact the contributor(s) of the work. ++ + +
ESRGAN
+Code for architecture and reading models copied. ++MIT License + +Copyright (c) 2021 victorca25 + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. ++ +
Real-ESRGAN
+Some code is copied to support ESRGAN models. ++BSD 3-Clause License + +Copyright (c) 2021, Xintao Wang +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ++ +
InvokeAI
+Some code for compatibility with OSX is taken from lstein's repository. ++MIT License + +Copyright (c) 2022 InvokeAI Team + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. ++ +
LDSR
+Code added by contirubtors, most likely copied from this repository. ++MIT License + +Copyright (c) 2022 Machine Vision and Learning Group, LMU Munich + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. ++ +
CLIP Interrogator
+Some small amounts of code borrowed and reworked. ++MIT License + +Copyright (c) 2022 pharmapsychotic + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. ++ +
SwinIR
+Code added by contributors, most likely copied from this repository. + ++ Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [2021] [SwinIR Authors] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. ++ +
Memory Efficient Attention
+The sub-quadratic cross attention optimization uses modified code from the Memory Efficient Attention package that Alex Birch optimized for 3D tensors. This license is updated to reflect that. ++MIT License + +Copyright (c) 2023 Alex Birch +Copyright (c) 2023 Amin Rezaei + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. ++ diff --git a/sd/stable-diffusion-webui/javascript/aspectRatioOverlay.js b/sd/stable-diffusion-webui/javascript/aspectRatioOverlay.js new file mode 100644 index 0000000000000000000000000000000000000000..50c81054269c7d75c9e29aa0f18260d1df0939eb --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/aspectRatioOverlay.js @@ -0,0 +1,113 @@ + +let currentWidth = null; +let currentHeight = null; +let arFrameTimeout = setTimeout(function(){},0); + +function dimensionChange(e, is_width, is_height){ + + if(is_width){ + currentWidth = e.target.value*1.0 + } + if(is_height){ + currentHeight = e.target.value*1.0 + } + + var inImg2img = Boolean(gradioApp().querySelector("button.rounded-t-lg.border-gray-200")) + + if(!inImg2img){ + return; + } + + var targetElement = null; + + var tabIndex = get_tab_index('mode_img2img') + if(tabIndex == 0){ // img2img + targetElement = gradioApp().querySelector('div[data-testid=image] img'); + } else if(tabIndex == 1){ //Sketch + targetElement = gradioApp().querySelector('#img2img_sketch div[data-testid=image] img'); + } else if(tabIndex == 2){ // Inpaint + targetElement = gradioApp().querySelector('#img2maskimg div[data-testid=image] img'); + } else if(tabIndex == 3){ // Inpaint sketch + targetElement = gradioApp().querySelector('#inpaint_sketch div[data-testid=image] img'); + } + + + if(targetElement){ + + var arPreviewRect = gradioApp().querySelector('#imageARPreview'); + if(!arPreviewRect){ + arPreviewRect = document.createElement('div') + arPreviewRect.id = "imageARPreview"; + gradioApp().getRootNode().appendChild(arPreviewRect) + } + + + + var viewportOffset = targetElement.getBoundingClientRect(); + + viewportscale = Math.min( targetElement.clientWidth/targetElement.naturalWidth, targetElement.clientHeight/targetElement.naturalHeight ) + + scaledx = targetElement.naturalWidth*viewportscale + scaledy = targetElement.naturalHeight*viewportscale + + cleintRectTop = (viewportOffset.top+window.scrollY) + cleintRectLeft = (viewportOffset.left+window.scrollX) + cleintRectCentreY = cleintRectTop + (targetElement.clientHeight/2) + cleintRectCentreX = cleintRectLeft + (targetElement.clientWidth/2) + + viewRectTop = cleintRectCentreY-(scaledy/2) + viewRectLeft = cleintRectCentreX-(scaledx/2) + arRectWidth = scaledx + arRectHeight = scaledy + + arscale = Math.min( arRectWidth/currentWidth, arRectHeight/currentHeight ) + arscaledx = currentWidth*arscale + arscaledy = currentHeight*arscale + + arRectTop = cleintRectCentreY-(arscaledy/2) + arRectLeft = cleintRectCentreX-(arscaledx/2) + arRectWidth = arscaledx + arRectHeight = arscaledy + + arPreviewRect.style.top = arRectTop+'px'; + arPreviewRect.style.left = arRectLeft+'px'; + arPreviewRect.style.width = arRectWidth+'px'; + arPreviewRect.style.height = arRectHeight+'px'; + + clearTimeout(arFrameTimeout); + arFrameTimeout = setTimeout(function(){ + arPreviewRect.style.display = 'none'; + },2000); + + arPreviewRect.style.display = 'block'; + + } + +} + + +onUiUpdate(function(){ + var arPreviewRect = gradioApp().querySelector('#imageARPreview'); + if(arPreviewRect){ + arPreviewRect.style.display = 'none'; + } + var inImg2img = Boolean(gradioApp().querySelector("button.rounded-t-lg.border-gray-200")) + if(inImg2img){ + let inputs = gradioApp().querySelectorAll('input'); + inputs.forEach(function(e){ + var is_width = e.parentElement.id == "img2img_width" + var is_height = e.parentElement.id == "img2img_height" + + if((is_width || is_height) && !e.classList.contains('scrollwatch')){ + e.addEventListener('input', function(e){dimensionChange(e, is_width, is_height)} ) + e.classList.add('scrollwatch') + } + if(is_width){ + currentWidth = e.value*1.0 + } + if(is_height){ + currentHeight = e.value*1.0 + } + }) + } +}); diff --git a/sd/stable-diffusion-webui/javascript/contextMenus.js b/sd/stable-diffusion-webui/javascript/contextMenus.js new file mode 100644 index 0000000000000000000000000000000000000000..163743c9debc160d46bd3a5a43dec6879e86a43e --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/contextMenus.js @@ -0,0 +1,177 @@ + +contextMenuInit = function(){ + let eventListenerApplied=false; + let menuSpecs = new Map(); + + const uid = function(){ + return Date.now().toString(36) + Math.random().toString(36).substr(2); + } + + function showContextMenu(event,element,menuEntries){ + let posx = event.clientX + document.body.scrollLeft + document.documentElement.scrollLeft; + let posy = event.clientY + document.body.scrollTop + document.documentElement.scrollTop; + + let oldMenu = gradioApp().querySelector('#context-menu') + if(oldMenu){ + oldMenu.remove() + } + + let tabButton = uiCurrentTab + let baseStyle = window.getComputedStyle(tabButton) + + const contextMenu = document.createElement('nav') + contextMenu.id = "context-menu" + contextMenu.style.background = baseStyle.background + contextMenu.style.color = baseStyle.color + contextMenu.style.fontFamily = baseStyle.fontFamily + contextMenu.style.top = posy+'px' + contextMenu.style.left = posx+'px' + + + + const contextMenuList = document.createElement('ul') + contextMenuList.className = 'context-menu-items'; + contextMenu.append(contextMenuList); + + menuEntries.forEach(function(entry){ + let contextMenuEntry = document.createElement('a') + contextMenuEntry.innerHTML = entry['name'] + contextMenuEntry.addEventListener("click", function(e) { + entry['func'](); + }) + contextMenuList.append(contextMenuEntry); + + }) + + gradioApp().getRootNode().appendChild(contextMenu) + + let menuWidth = contextMenu.offsetWidth + 4; + let menuHeight = contextMenu.offsetHeight + 4; + + let windowWidth = window.innerWidth; + let windowHeight = window.innerHeight; + + if ( (windowWidth - posx) < menuWidth ) { + contextMenu.style.left = windowWidth - menuWidth + "px"; + } + + if ( (windowHeight - posy) < menuHeight ) { + contextMenu.style.top = windowHeight - menuHeight + "px"; + } + + } + + function appendContextMenuOption(targetElementSelector,entryName,entryFunction){ + + currentItems = menuSpecs.get(targetElementSelector) + + if(!currentItems){ + currentItems = [] + menuSpecs.set(targetElementSelector,currentItems); + } + let newItem = {'id':targetElementSelector+'_'+uid(), + 'name':entryName, + 'func':entryFunction, + 'isNew':true} + + currentItems.push(newItem) + return newItem['id'] + } + + function removeContextMenuOption(uid){ + menuSpecs.forEach(function(v,k) { + let index = -1 + v.forEach(function(e,ei){if(e['id']==uid){index=ei}}) + if(index>=0){ + v.splice(index, 1); + } + }) + } + + function addContextMenuEventListener(){ + if(eventListenerApplied){ + return; + } + gradioApp().addEventListener("click", function(e) { + let source = e.composedPath()[0] + if(source.id && source.id.indexOf('check_progress')>-1){ + return + } + + let oldMenu = gradioApp().querySelector('#context-menu') + if(oldMenu){ + oldMenu.remove() + } + }); + gradioApp().addEventListener("contextmenu", function(e) { + let oldMenu = gradioApp().querySelector('#context-menu') + if(oldMenu){ + oldMenu.remove() + } + menuSpecs.forEach(function(v,k) { + if(e.composedPath()[0].matches(k)){ + showContextMenu(e,e.composedPath()[0],v) + e.preventDefault() + return + } + }) + }); + eventListenerApplied=true + + } + + return [appendContextMenuOption, removeContextMenuOption, addContextMenuEventListener] +} + +initResponse = contextMenuInit(); +appendContextMenuOption = initResponse[0]; +removeContextMenuOption = initResponse[1]; +addContextMenuEventListener = initResponse[2]; + +(function(){ + //Start example Context Menu Items + let generateOnRepeat = function(genbuttonid,interruptbuttonid){ + let genbutton = gradioApp().querySelector(genbuttonid); + let interruptbutton = gradioApp().querySelector(interruptbuttonid); + if(!interruptbutton.offsetParent){ + genbutton.click(); + } + clearInterval(window.generateOnRepeatInterval) + window.generateOnRepeatInterval = setInterval(function(){ + if(!interruptbutton.offsetParent){ + genbutton.click(); + } + }, + 500) + } + + appendContextMenuOption('#txt2img_generate','Generate forever',function(){ + generateOnRepeat('#txt2img_generate','#txt2img_interrupt'); + }) + appendContextMenuOption('#img2img_generate','Generate forever',function(){ + generateOnRepeat('#img2img_generate','#img2img_interrupt'); + }) + + let cancelGenerateForever = function(){ + clearInterval(window.generateOnRepeatInterval) + } + + appendContextMenuOption('#txt2img_interrupt','Cancel generate forever',cancelGenerateForever) + appendContextMenuOption('#txt2img_generate', 'Cancel generate forever',cancelGenerateForever) + appendContextMenuOption('#img2img_interrupt','Cancel generate forever',cancelGenerateForever) + appendContextMenuOption('#img2img_generate', 'Cancel generate forever',cancelGenerateForever) + + appendContextMenuOption('#roll','Roll three', + function(){ + let rollbutton = get_uiCurrentTabContent().querySelector('#roll'); + setTimeout(function(){rollbutton.click()},100) + setTimeout(function(){rollbutton.click()},200) + setTimeout(function(){rollbutton.click()},300) + } + ) +})(); +//End example Context Menu Items + +onUiUpdate(function(){ + addContextMenuEventListener() +}); diff --git a/sd/stable-diffusion-webui/javascript/dragdrop.js b/sd/stable-diffusion-webui/javascript/dragdrop.js new file mode 100644 index 0000000000000000000000000000000000000000..fe00892481a8ff8b997759b21cb36eb364db788b --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/dragdrop.js @@ -0,0 +1,97 @@ +// allows drag-dropping files into gradio image elements, and also pasting images from clipboard + +function isValidImageList( files ) { + return files && files?.length === 1 && ['image/png', 'image/gif', 'image/jpeg'].includes(files[0].type); +} + +function dropReplaceImage( imgWrap, files ) { + if ( ! isValidImageList( files ) ) { + return; + } + + const tmpFile = files[0]; + + imgWrap.querySelector('.modify-upload button + button, .touch-none + div button + button')?.click(); + const callback = () => { + const fileInput = imgWrap.querySelector('input[type="file"]'); + if ( fileInput ) { + if ( files.length === 0 ) { + files = new DataTransfer(); + files.items.add(tmpFile); + fileInput.files = files.files; + } else { + fileInput.files = files; + } + fileInput.dispatchEvent(new Event('change')); + } + }; + + if ( imgWrap.closest('#pnginfo_image') ) { + // special treatment for PNG Info tab, wait for fetch request to finish + const oldFetch = window.fetch; + window.fetch = async (input, options) => { + const response = await oldFetch(input, options); + if ( 'api/predict/' === input ) { + const content = await response.text(); + window.fetch = oldFetch; + window.requestAnimationFrame( () => callback() ); + return new Response(content, { + status: response.status, + statusText: response.statusText, + headers: response.headers + }) + } + return response; + }; + } else { + window.requestAnimationFrame( () => callback() ); + } +} + +window.document.addEventListener('dragover', e => { + const target = e.composedPath()[0]; + const imgWrap = target.closest('[data-testid="image"]'); + if ( !imgWrap && target.placeholder && target.placeholder.indexOf("Prompt") == -1) { + return; + } + e.stopPropagation(); + e.preventDefault(); + e.dataTransfer.dropEffect = 'copy'; +}); + +window.document.addEventListener('drop', e => { + const target = e.composedPath()[0]; + if (target.placeholder.indexOf("Prompt") == -1) { + return; + } + const imgWrap = target.closest('[data-testid="image"]'); + if ( !imgWrap ) { + return; + } + e.stopPropagation(); + e.preventDefault(); + const files = e.dataTransfer.files; + dropReplaceImage( imgWrap, files ); +}); + +window.addEventListener('paste', e => { + const files = e.clipboardData.files; + if ( ! isValidImageList( files ) ) { + return; + } + + const visibleImageFields = [...gradioApp().querySelectorAll('[data-testid="image"]')] + .filter(el => uiElementIsVisible(el)); + if ( ! visibleImageFields.length ) { + return; + } + + const firstFreeImageField = visibleImageFields + .filter(el => el.querySelector('input[type=file]'))?.[0]; + + dropReplaceImage( + firstFreeImageField ? + firstFreeImageField : + visibleImageFields[visibleImageFields.length - 1] + , files ); +}); diff --git a/sd/stable-diffusion-webui/javascript/edit-attention.js b/sd/stable-diffusion-webui/javascript/edit-attention.js new file mode 100644 index 0000000000000000000000000000000000000000..6e6905daf536d164cc2a246886ca8198603ce61a --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/edit-attention.js @@ -0,0 +1,96 @@ +function keyupEditAttention(event){ + let target = event.originalTarget || event.composedPath()[0]; + if (!target.matches("[id*='_toprow'] textarea.gr-text-input[placeholder]")) return; + if (! (event.metaKey || event.ctrlKey)) return; + + let isPlus = event.key == "ArrowUp" + let isMinus = event.key == "ArrowDown" + if (!isPlus && !isMinus) return; + + let selectionStart = target.selectionStart; + let selectionEnd = target.selectionEnd; + let text = target.value; + + function selectCurrentParenthesisBlock(OPEN, CLOSE){ + if (selectionStart !== selectionEnd) return false; + + // Find opening parenthesis around current cursor + const before = text.substring(0, selectionStart); + let beforeParen = before.lastIndexOf(OPEN); + if (beforeParen == -1) return false; + let beforeParenClose = before.lastIndexOf(CLOSE); + while (beforeParenClose !== -1 && beforeParenClose > beforeParen) { + beforeParen = before.lastIndexOf(OPEN, beforeParen - 1); + beforeParenClose = before.lastIndexOf(CLOSE, beforeParenClose - 1); + } + + // Find closing parenthesis around current cursor + const after = text.substring(selectionStart); + let afterParen = after.indexOf(CLOSE); + if (afterParen == -1) return false; + let afterParenOpen = after.indexOf(OPEN); + while (afterParenOpen !== -1 && afterParen > afterParenOpen) { + afterParen = after.indexOf(CLOSE, afterParen + 1); + afterParenOpen = after.indexOf(OPEN, afterParenOpen + 1); + } + if (beforeParen === -1 || afterParen === -1) return false; + + // Set the selection to the text between the parenthesis + const parenContent = text.substring(beforeParen + 1, selectionStart + afterParen); + const lastColon = parenContent.lastIndexOf(":"); + selectionStart = beforeParen + 1; + selectionEnd = selectionStart + lastColon; + target.setSelectionRange(selectionStart, selectionEnd); + return true; + } + + // If the user hasn't selected anything, let's select their current parenthesis block + if(! selectCurrentParenthesisBlock('<', '>')){ + selectCurrentParenthesisBlock('(', ')') + } + + event.preventDefault(); + + closeCharacter = ')' + delta = opts.keyedit_precision_attention + + if (selectionStart > 0 && text[selectionStart - 1] == '<'){ + closeCharacter = '>' + delta = opts.keyedit_precision_extra + } else if (selectionStart == 0 || text[selectionStart - 1] != "(") { + + // do not include spaces at the end + while(selectionEnd > selectionStart && text[selectionEnd-1] == ' '){ + selectionEnd -= 1; + } + if(selectionStart == selectionEnd){ + return + } + + text = text.slice(0, selectionStart) + "(" + text.slice(selectionStart, selectionEnd) + ":1.0)" + text.slice(selectionEnd); + + selectionStart += 1; + selectionEnd += 1; + } + + end = text.slice(selectionEnd + 1).indexOf(closeCharacter) + 1; + weight = parseFloat(text.slice(selectionEnd + 1, selectionEnd + 1 + end)); + if (isNaN(weight)) return; + + weight += isPlus ? delta : -delta; + weight = parseFloat(weight.toPrecision(12)); + if(String(weight).length == 1) weight += ".0" + + text = text.slice(0, selectionEnd + 1) + weight + text.slice(selectionEnd + 1 + end - 1); + + target.focus(); + target.value = text; + target.selectionStart = selectionStart; + target.selectionEnd = selectionEnd; + + updateInput(target) +} + +addEventListener('keydown', (event) => { + keyupEditAttention(event); +}); \ No newline at end of file diff --git a/sd/stable-diffusion-webui/javascript/extensions.js b/sd/stable-diffusion-webui/javascript/extensions.js new file mode 100644 index 0000000000000000000000000000000000000000..8a0580f706a9511e3391b9170e6684c2655b893a --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/extensions.js @@ -0,0 +1,49 @@ + +function extensions_apply(_, _){ + var disable = [] + var update = [] + + gradioApp().querySelectorAll('#extensions input[type="checkbox"]').forEach(function(x){ + if(x.name.startsWith("enable_") && ! x.checked) + disable.push(x.name.substr(7)) + + if(x.name.startsWith("update_") && x.checked) + update.push(x.name.substr(7)) + }) + + restart_reload() + + return [JSON.stringify(disable), JSON.stringify(update)] +} + +function extensions_check(){ + var disable = [] + + gradioApp().querySelectorAll('#extensions input[type="checkbox"]').forEach(function(x){ + if(x.name.startsWith("enable_") && ! x.checked) + disable.push(x.name.substr(7)) + }) + + gradioApp().querySelectorAll('#extensions .extension_status').forEach(function(x){ + x.innerHTML = "Loading..." + }) + + + var id = randomId() + requestProgress(id, gradioApp().getElementById('extensions_installed_top'), null, function(){ + + }) + + return [id, JSON.stringify(disable)] +} + +function install_extension_from_index(button, url){ + button.disabled = "disabled" + button.value = "Installing..." + + textarea = gradioApp().querySelector('#extension_to_install textarea') + textarea.value = url + updateInput(textarea) + + gradioApp().querySelector('#install_extension_button').click() +} diff --git a/sd/stable-diffusion-webui/javascript/extraNetworks.js b/sd/stable-diffusion-webui/javascript/extraNetworks.js new file mode 100644 index 0000000000000000000000000000000000000000..b15758b91ebd271604ac8ad342a19537b2efc760 --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/extraNetworks.js @@ -0,0 +1,107 @@ + +function setupExtraNetworksForTab(tabname){ + gradioApp().querySelector('#'+tabname+'_extra_tabs').classList.add('extra-networks') + + var tabs = gradioApp().querySelector('#'+tabname+'_extra_tabs > div') + var search = gradioApp().querySelector('#'+tabname+'_extra_search textarea') + var refresh = gradioApp().getElementById(tabname+'_extra_refresh') + var close = gradioApp().getElementById(tabname+'_extra_close') + + search.classList.add('search') + tabs.appendChild(search) + tabs.appendChild(refresh) + tabs.appendChild(close) + + search.addEventListener("input", function(evt){ + searchTerm = search.value.toLowerCase() + + gradioApp().querySelectorAll('#'+tabname+'_extra_tabs div.card').forEach(function(elem){ + text = elem.querySelector('.name').textContent.toLowerCase() + " " + elem.querySelector('.search_term').textContent.toLowerCase() + elem.style.display = text.indexOf(searchTerm) == -1 ? "none" : "" + }) + }); +} + +var activePromptTextarea = {}; + +function setupExtraNetworks(){ + setupExtraNetworksForTab('txt2img') + setupExtraNetworksForTab('img2img') + + function registerPrompt(tabname, id){ + var textarea = gradioApp().querySelector("#" + id + " > label > textarea"); + + if (! activePromptTextarea[tabname]){ + activePromptTextarea[tabname] = textarea + } + + textarea.addEventListener("focus", function(){ + activePromptTextarea[tabname] = textarea; + }); + } + + registerPrompt('txt2img', 'txt2img_prompt') + registerPrompt('txt2img', 'txt2img_neg_prompt') + registerPrompt('img2img', 'img2img_prompt') + registerPrompt('img2img', 'img2img_neg_prompt') +} + +onUiLoaded(setupExtraNetworks) + +var re_extranet = /<([^:]+:[^:]+):[\d\.]+>/; +var re_extranet_g = /\s+<([^:]+:[^:]+):[\d\.]+>/g; + +function tryToRemoveExtraNetworkFromPrompt(textarea, text){ + var m = text.match(re_extranet) + if(! m) return false + + var partToSearch = m[1] + var replaced = false + var newTextareaText = textarea.value.replaceAll(re_extranet_g, function(found, index){ + m = found.match(re_extranet); + if(m[1] == partToSearch){ + replaced = true; + return "" + } + return found; + }) + + if(replaced){ + textarea.value = newTextareaText + return true; + } + + return false +} + +function cardClicked(tabname, textToAdd, allowNegativePrompt){ + var textarea = allowNegativePrompt ? activePromptTextarea[tabname] : gradioApp().querySelector("#" + tabname + "_prompt > label > textarea") + + if(! tryToRemoveExtraNetworkFromPrompt(textarea, textToAdd)){ + textarea.value = textarea.value + " " + textToAdd + } + + updateInput(textarea) +} + +function saveCardPreview(event, tabname, filename){ + var textarea = gradioApp().querySelector("#" + tabname + '_preview_filename > label > textarea') + var button = gradioApp().getElementById(tabname + '_save_preview') + + textarea.value = filename + updateInput(textarea) + + button.click() + + event.stopPropagation() + event.preventDefault() +} + +function extraNetworksSearchButton(tabs_id, event){ + searchTextarea = gradioApp().querySelector("#" + tabs_id + ' > div > textarea') + button = event.target + text = button.classList.contains("search-all") ? "" : button.textContent.trim() + + searchTextarea.value = text + updateInput(searchTextarea) +} \ No newline at end of file diff --git a/sd/stable-diffusion-webui/javascript/generationParams.js b/sd/stable-diffusion-webui/javascript/generationParams.js new file mode 100644 index 0000000000000000000000000000000000000000..95f050939b72a8d09d62de8d725caf1e7d15d3c0 --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/generationParams.js @@ -0,0 +1,33 @@ +// attaches listeners to the txt2img and img2img galleries to update displayed generation param text when the image changes + +let txt2img_gallery, img2img_gallery, modal = undefined; +onUiUpdate(function(){ + if (!txt2img_gallery) { + txt2img_gallery = attachGalleryListeners("txt2img") + } + if (!img2img_gallery) { + img2img_gallery = attachGalleryListeners("img2img") + } + if (!modal) { + modal = gradioApp().getElementById('lightboxModal') + modalObserver.observe(modal, { attributes : true, attributeFilter : ['style'] }); + } +}); + +let modalObserver = new MutationObserver(function(mutations) { + mutations.forEach(function(mutationRecord) { + let selectedTab = gradioApp().querySelector('#tabs div button.bg-white')?.innerText + if (mutationRecord.target.style.display === 'none' && selectedTab === 'txt2img' || selectedTab === 'img2img') + gradioApp().getElementById(selectedTab+"_generation_info_button").click() + }); +}); + +function attachGalleryListeners(tab_name) { + gallery = gradioApp().querySelector('#'+tab_name+'_gallery') + gallery?.addEventListener('click', () => gradioApp().getElementById(tab_name+"_generation_info_button").click()); + gallery?.addEventListener('keydown', (e) => { + if (e.keyCode == 37 || e.keyCode == 39) // left or right arrow + gradioApp().getElementById(tab_name+"_generation_info_button").click() + }); + return gallery; +} diff --git a/sd/stable-diffusion-webui/javascript/hints.js b/sd/stable-diffusion-webui/javascript/hints.js new file mode 100644 index 0000000000000000000000000000000000000000..f1199009b181b83713bb1d136e41d4b29e183634 --- /dev/null +++ b/sd/stable-diffusion-webui/javascript/hints.js @@ -0,0 +1,146 @@ +// mouseover tooltips for various UI elements + +titles = { + "Sampling steps": "How many times to improve the generated image iteratively; higher values take longer; very low values can produce bad results", + "Sampling method": "Which algorithm to use to produce the image", + "GFPGAN": "Restore low quality faces using GFPGAN neural network", + "Euler a": "Euler Ancestral - very creative, each can get a completely different picture depending on step count, setting steps higher than 30-40 does not help", + "DDIM": "Denoising Diffusion Implicit Models - best at inpainting", + "DPM adaptive": "Ignores step count - uses a number of steps determined by the CFG and resolution", + + "Batch count": "How many batches of images to create (has no impact on generation performance or VRAM usage)", + "Batch size": "How many image to create in a single batch (increases generation performance at cost of higher VRAM usage)", + "CFG Scale": "Classifier Free Guidance Scale - how strongly the image should conform to prompt - lower values produce more creative results", + "Seed": "A value that determines the output of random number generator - if you create an image with same parameters and seed as another image, you'll get the same result", + "\u{1f3b2}\ufe0f": "Set seed to -1, which will cause a new random number to be used every time", + "\u267b\ufe0f": "Reuse seed from last generation, mostly useful if it was randomed", + "\u2199\ufe0f": "Read generation parameters from prompt or last generation if prompt is empty into user interface.", + "\u{1f4c2}": "Open images output directory", + "\u{1f4be}": "Save style", + "\u{1f5d1}": "Clear prompt", + "\u{1f4cb}": "Apply selected styles to current prompt", + "\u{1f4d2}": "Paste available values into the field", + "\u{1f3b4}": "Show extra networks", + + + "Inpaint a part of image": "Draw a mask over an image, and the script will regenerate the masked area with content according to prompt", + "SD upscale": "Upscale image normally, split result into tiles, improve each tile using img2img, merge whole image back", + + "Just resize": "Resize image to target resolution. Unless height and width match, you will get incorrect aspect ratio.", + "Crop and resize": "Resize the image so that entirety of target resolution is filled with the image. Crop parts that stick out.", + "Resize and fill": "Resize the image so that entirety of image is inside target resolution. Fill empty space with image's colors.", + + "Mask blur": "How much to blur the mask before processing, in pixels.", + "Masked content": "What to put inside the masked area before processing it with Stable Diffusion.", + "fill": "fill it with colors of the image", + "original": "keep whatever was there originally", + "latent noise": "fill it with latent space noise", + "latent nothing": "fill it with latent space zeroes", + "Inpaint at full resolution": "Upscale masked region to target resolution, do inpainting, downscale back and paste into original image", + + "Denoising strength": "Determines how little respect the algorithm should have for image's content. At 0, nothing will change, and at 1 you'll get an unrelated image. With values below 1.0, processing will take less steps than the Sampling Steps slider specifies.", + "Denoising strength change factor": "In loopback mode, on each loop the denoising strength is multiplied by this value. <1 means decreasing variety so your sequence will converge on a fixed picture. >1 means increasing variety so your sequence will become more and more chaotic.", + + "Skip": "Stop processing current image and continue processing.", + "Interrupt": "Stop processing images and return any results accumulated so far.", + "Save": "Write image to a directory (default - log/images) and generation parameters into csv file.", + + "X values": "Separate values for X axis using commas.", + "Y values": "Separate values for Y axis using commas.", + + "None": "Do not do anything special", + "Prompt matrix": "Separate prompts into parts using vertical pipe character (|) and the script will create a picture for every combination of them (except for the first part, which will be present in all combinations)", + "X/Y/Z plot": "Create grid(s) where images will have different parameters. Use inputs below to specify which parameters will be shared by columns and rows", + "Custom code": "Run Python code. Advanced user only. Must run program with --allow-code for this to work", + + "Prompt S/R": "Separate a list of words with commas, and the first word will be used as a keyword: script will search for this word in the prompt, and replace it with others", + "Prompt order": "Separate a list of words with commas, and the script will make a variation of prompt with those words for their every possible order", + + "Tiling": "Produce an image that can be tiled.", + "Tile overlap": "For SD upscale, how much overlap in pixels should there be between tiles. Tiles overlap so that when they are merged back into one picture, there is no clearly visible seam.", + + "Variation seed": "Seed of a different picture to be mixed into the generation.", + "Variation strength": "How strong of a variation to produce. At 0, there will be no effect. At 1, you will get the complete picture with variation seed (except for ancestral samplers, where you will just get something).", + "Resize seed from height": "Make an attempt to produce a picture similar to what would have been produced with same seed at specified resolution", + "Resize seed from width": "Make an attempt to produce a picture similar to what would have been produced with same seed at specified resolution", + + "Interrogate": "Reconstruct prompt from existing image and put it into the prompt field.", + + "Images filename pattern": "Use following tags to define how filenames for images are chosen: [steps], [cfg], [prompt_hash], [prompt], [prompt_no_styles], [prompt_spaces], [width], [height], [styles], [sampler], [seed], [model_hash], [model_name], [prompt_words], [date], [datetime], [datetime
+Loss: {loss_step:.7f}
+Step: {steps_done}
+Last prompt: {html.escape(batch.cond_text[0])}
+Last saved embedding: {html.escape(last_saved_file)}
+Last saved image: {html.escape(last_saved_image)}
+
Disabled when launched with --hide-ui-dir-config.' if shared.cmd_opts.hide_ui_dir_config else '' + gr.HTML( + f"
Process images in a directory on the same machine where the server is running." +
+ f"
Use an empty output directory to save pictures normally instead of writing to the output directory." +
+ f"
Add inpaint batch mask directory to enable inpaint batch processing."
+ f"{hidden}
{}
" + interp_descriptions = { + "No interpolation": interp_description_css.format("No interpolation will be used. Requires one model; A. Allows for format conversion and VAE baking."), + "Weighted sum": interp_description_css.format("A weighted sum will be used for interpolation. Requires two models; A and B. The result is calculated as A * (1 - M) + B * M"), + "Add difference": interp_description_css.format("The difference between the last two models will be added to the first. Requires three models; A, B and C. The result is calculated as A + (B - C) * M") + } + return interp_descriptions[value] + + with gr.Blocks(analytics_enabled=False) as modelmerger_interface: + with gr.Row().style(equal_height=False): + with gr.Column(variant='compact'): + interp_description = gr.HTML(value=update_interp_description("Weighted sum"), elem_id="modelmerger_interp_description") + + with FormRow(elem_id="modelmerger_models"): + primary_model_name = gr.Dropdown(modules.sd_models.checkpoint_tiles(), elem_id="modelmerger_primary_model_name", label="Primary model (A)") + create_refresh_button(primary_model_name, modules.sd_models.list_models, lambda: {"choices": modules.sd_models.checkpoint_tiles()}, "refresh_checkpoint_A") + + secondary_model_name = gr.Dropdown(modules.sd_models.checkpoint_tiles(), elem_id="modelmerger_secondary_model_name", label="Secondary model (B)") + create_refresh_button(secondary_model_name, modules.sd_models.list_models, lambda: {"choices": modules.sd_models.checkpoint_tiles()}, "refresh_checkpoint_B") + + tertiary_model_name = gr.Dropdown(modules.sd_models.checkpoint_tiles(), elem_id="modelmerger_tertiary_model_name", label="Tertiary model (C)") + create_refresh_button(tertiary_model_name, modules.sd_models.list_models, lambda: {"choices": modules.sd_models.checkpoint_tiles()}, "refresh_checkpoint_C") + + custom_name = gr.Textbox(label="Custom Name (Optional)", elem_id="modelmerger_custom_name") + interp_amount = gr.Slider(minimum=0.0, maximum=1.0, step=0.05, label='Multiplier (M) - set to 0 to get model A', value=0.3, elem_id="modelmerger_interp_amount") + interp_method = gr.Radio(choices=["No interpolation", "Weighted sum", "Add difference"], value="Weighted sum", label="Interpolation Method", elem_id="modelmerger_interp_method") + interp_method.change(fn=update_interp_description, inputs=[interp_method], outputs=[interp_description]) + + with FormRow(): + checkpoint_format = gr.Radio(choices=["ckpt", "safetensors"], value="ckpt", label="Checkpoint format", elem_id="modelmerger_checkpoint_format") + save_as_half = gr.Checkbox(value=False, label="Save as float16", elem_id="modelmerger_save_as_half") + + with FormRow(): + with gr.Column(): + config_source = gr.Radio(choices=["A, B or C", "B", "C", "Don't"], value="A, B or C", label="Copy config from", type="index", elem_id="modelmerger_config_method") + + with gr.Column(): + with FormRow(): + bake_in_vae = gr.Dropdown(choices=["None"] + list(sd_vae.vae_dict), value="None", label="Bake in VAE", elem_id="modelmerger_bake_in_vae") + create_refresh_button(bake_in_vae, sd_vae.refresh_vae_list, lambda: {"choices": ["None"] + list(sd_vae.vae_dict)}, "modelmerger_refresh_bake_in_vae") + + with FormRow(): + discard_weights = gr.Textbox(value="", label="Discard weights with matching name", elem_id="modelmerger_discard_weights") + + with gr.Row(): + modelmerger_merge = gr.Button(elem_id="modelmerger_merge", value="Merge", variant='primary') + + with gr.Column(variant='compact', elem_id="modelmerger_results_container"): + with gr.Group(elem_id="modelmerger_results_panel"): + modelmerger_result = gr.HTML(elem_id="modelmerger_result", show_label=False) + + with gr.Blocks(analytics_enabled=False) as train_interface: + with gr.Row().style(equal_height=False): + gr.HTML(value="See wiki for detailed explanation.
") + + with gr.Row(variant="compact").style(equal_height=False): + with gr.Tabs(elem_id="train_tabs"): + + with gr.Tab(label="Create embedding"): + new_embedding_name = gr.Textbox(label="Name", elem_id="train_new_embedding_name") + initialization_text = gr.Textbox(label="Initialization text", value="*", elem_id="train_initialization_text") + nvpt = gr.Slider(label="Number of vectors per token", minimum=1, maximum=75, step=1, value=1, elem_id="train_nvpt") + overwrite_old_embedding = gr.Checkbox(value=False, label="Overwrite Old Embedding", elem_id="train_overwrite_old_embedding") + + with gr.Row(): + with gr.Column(scale=3): + gr.HTML(value="") + + with gr.Column(): + create_embedding = gr.Button(value="Create embedding", variant='primary', elem_id="train_create_embedding") + + with gr.Tab(label="Create hypernetwork"): + new_hypernetwork_name = gr.Textbox(label="Name", elem_id="train_new_hypernetwork_name") + new_hypernetwork_sizes = gr.CheckboxGroup(label="Modules", value=["768", "320", "640", "1280"], choices=["768", "1024", "320", "640", "1280"], elem_id="train_new_hypernetwork_sizes") + new_hypernetwork_layer_structure = gr.Textbox("1, 2, 1", label="Enter hypernetwork layer structure", placeholder="1st and last digit must be 1. ex:'1, 2, 1'", elem_id="train_new_hypernetwork_layer_structure") + new_hypernetwork_activation_func = gr.Dropdown(value="linear", label="Select activation function of hypernetwork. Recommended : Swish / Linear(none)", choices=modules.hypernetworks.ui.keys, elem_id="train_new_hypernetwork_activation_func") + new_hypernetwork_initialization_option = gr.Dropdown(value = "Normal", label="Select Layer weights initialization. Recommended: Kaiming for relu-like, Xavier for sigmoid-like, Normal otherwise", choices=["Normal", "KaimingUniform", "KaimingNormal", "XavierUniform", "XavierNormal"], elem_id="train_new_hypernetwork_initialization_option") + new_hypernetwork_add_layer_norm = gr.Checkbox(label="Add layer normalization", elem_id="train_new_hypernetwork_add_layer_norm") + new_hypernetwork_use_dropout = gr.Checkbox(label="Use dropout", elem_id="train_new_hypernetwork_use_dropout") + new_hypernetwork_dropout_structure = gr.Textbox("0, 0, 0", label="Enter hypernetwork Dropout structure (or empty). Recommended : 0~0.35 incrementing sequence: 0, 0.05, 0.15", placeholder="1st and last digit must be 0 and values should be between 0 and 1. ex:'0, 0.01, 0'") + overwrite_old_hypernetwork = gr.Checkbox(value=False, label="Overwrite Old Hypernetwork", elem_id="train_overwrite_old_hypernetwork") + + with gr.Row(): + with gr.Column(scale=3): + gr.HTML(value="") + + with gr.Column(): + create_hypernetwork = gr.Button(value="Create hypernetwork", variant='primary', elem_id="train_create_hypernetwork") + + with gr.Tab(label="Preprocess images"): + process_src = gr.Textbox(label='Source directory', elem_id="train_process_src") + process_dst = gr.Textbox(label='Destination directory', elem_id="train_process_dst") + process_width = gr.Slider(minimum=64, maximum=2048, step=8, label="Width", value=512, elem_id="train_process_width") + process_height = gr.Slider(minimum=64, maximum=2048, step=8, label="Height", value=512, elem_id="train_process_height") + preprocess_txt_action = gr.Dropdown(label='Existing Caption txt Action', value="ignore", choices=["ignore", "copy", "prepend", "append"], elem_id="train_preprocess_txt_action") + + with gr.Row(): + process_flip = gr.Checkbox(label='Create flipped copies', elem_id="train_process_flip") + process_split = gr.Checkbox(label='Split oversized images', elem_id="train_process_split") + process_focal_crop = gr.Checkbox(label='Auto focal point crop', elem_id="train_process_focal_crop") + process_multicrop = gr.Checkbox(label='Auto-sized crop', elem_id="train_process_multicrop") + process_caption = gr.Checkbox(label='Use BLIP for caption', elem_id="train_process_caption") + process_caption_deepbooru = gr.Checkbox(label='Use deepbooru for caption', visible=True, elem_id="train_process_caption_deepbooru") + + with gr.Row(visible=False) as process_split_extra_row: + process_split_threshold = gr.Slider(label='Split image threshold', value=0.5, minimum=0.0, maximum=1.0, step=0.05, elem_id="train_process_split_threshold") + process_overlap_ratio = gr.Slider(label='Split image overlap ratio', value=0.2, minimum=0.0, maximum=0.9, step=0.05, elem_id="train_process_overlap_ratio") + + with gr.Row(visible=False) as process_focal_crop_row: + process_focal_crop_face_weight = gr.Slider(label='Focal point face weight', value=0.9, minimum=0.0, maximum=1.0, step=0.05, elem_id="train_process_focal_crop_face_weight") + process_focal_crop_entropy_weight = gr.Slider(label='Focal point entropy weight', value=0.15, minimum=0.0, maximum=1.0, step=0.05, elem_id="train_process_focal_crop_entropy_weight") + process_focal_crop_edges_weight = gr.Slider(label='Focal point edges weight', value=0.5, minimum=0.0, maximum=1.0, step=0.05, elem_id="train_process_focal_crop_edges_weight") + process_focal_crop_debug = gr.Checkbox(label='Create debug image', elem_id="train_process_focal_crop_debug") + + with gr.Column(visible=False) as process_multicrop_col: + gr.Markdown('Each image is center-cropped with an automatically chosen width and height.') + with gr.Row(): + process_multicrop_mindim = gr.Slider(minimum=64, maximum=2048, step=8, label="Dimension lower bound", value=384, elem_id="train_process_multicrop_mindim") + process_multicrop_maxdim = gr.Slider(minimum=64, maximum=2048, step=8, label="Dimension upper bound", value=768, elem_id="train_process_multicrop_maxdim") + with gr.Row(): + process_multicrop_minarea = gr.Slider(minimum=64*64, maximum=2048*2048, step=1, label="Area lower bound", value=64*64, elem_id="train_process_multicrop_minarea") + process_multicrop_maxarea = gr.Slider(minimum=64*64, maximum=2048*2048, step=1, label="Area upper bound", value=640*640, elem_id="train_process_multicrop_maxarea") + with gr.Row(): + process_multicrop_objective = gr.Radio(["Maximize area", "Minimize error"], value="Maximize area", label="Resizing objective", elem_id="train_process_multicrop_objective") + process_multicrop_threshold = gr.Slider(minimum=0, maximum=1, step=0.01, label="Error threshold", value=0.1, elem_id="train_process_multicrop_threshold") + + with gr.Row(): + with gr.Column(scale=3): + gr.HTML(value="") + + with gr.Column(): + with gr.Row(): + interrupt_preprocessing = gr.Button("Interrupt", elem_id="train_interrupt_preprocessing") + run_preprocess = gr.Button(value="Preprocess", variant='primary', elem_id="train_run_preprocess") + + process_split.change( + fn=lambda show: gr_show(show), + inputs=[process_split], + outputs=[process_split_extra_row], + ) + + process_focal_crop.change( + fn=lambda show: gr_show(show), + inputs=[process_focal_crop], + outputs=[process_focal_crop_row], + ) + + process_multicrop.change( + fn=lambda show: gr_show(show), + inputs=[process_multicrop], + outputs=[process_multicrop_col], + ) + + def get_textual_inversion_template_names(): + return sorted([x for x in textual_inversion.textual_inversion_templates]) + + with gr.Tab(label="Train"): + gr.HTML(value="Train an embedding or Hypernetwork; you must specify a directory with a set of 1:1 ratio images [wiki]
") + with FormRow(): + train_embedding_name = gr.Dropdown(label='Embedding', elem_id="train_embedding", choices=sorted(sd_hijack.model_hijack.embedding_db.word_embeddings.keys())) + create_refresh_button(train_embedding_name, sd_hijack.model_hijack.embedding_db.load_textual_inversion_embeddings, lambda: {"choices": sorted(sd_hijack.model_hijack.embedding_db.word_embeddings.keys())}, "refresh_train_embedding_name") + + train_hypernetwork_name = gr.Dropdown(label='Hypernetwork', elem_id="train_hypernetwork", choices=[x for x in shared.hypernetworks.keys()]) + create_refresh_button(train_hypernetwork_name, shared.reload_hypernetworks, lambda: {"choices": sorted([x for x in shared.hypernetworks.keys()])}, "refresh_train_hypernetwork_name") + + with FormRow(): + embedding_learn_rate = gr.Textbox(label='Embedding Learning rate', placeholder="Embedding Learning rate", value="0.005", elem_id="train_embedding_learn_rate") + hypernetwork_learn_rate = gr.Textbox(label='Hypernetwork Learning rate', placeholder="Hypernetwork Learning rate", value="0.00001", elem_id="train_hypernetwork_learn_rate") + + with FormRow(): + clip_grad_mode = gr.Dropdown(value="disabled", label="Gradient Clipping", choices=["disabled", "value", "norm"]) + clip_grad_value = gr.Textbox(placeholder="Gradient clip value", value="0.1", show_label=False) + + with FormRow(): + batch_size = gr.Number(label='Batch size', value=1, precision=0, elem_id="train_batch_size") + gradient_step = gr.Number(label='Gradient accumulation steps', value=1, precision=0, elem_id="train_gradient_step") + + dataset_directory = gr.Textbox(label='Dataset directory', placeholder="Path to directory with input images", elem_id="train_dataset_directory") + log_directory = gr.Textbox(label='Log directory', placeholder="Path to directory where to write outputs", value="textual_inversion", elem_id="train_log_directory") + + with FormRow(): + template_file = gr.Dropdown(label='Prompt template', value="style_filewords.txt", elem_id="train_template_file", choices=get_textual_inversion_template_names()) + create_refresh_button(template_file, textual_inversion.list_textual_inversion_templates, lambda: {"choices": get_textual_inversion_template_names()}, "refrsh_train_template_file") + + training_width = gr.Slider(minimum=64, maximum=2048, step=8, label="Width", value=512, elem_id="train_training_width") + training_height = gr.Slider(minimum=64, maximum=2048, step=8, label="Height", value=512, elem_id="train_training_height") + varsize = gr.Checkbox(label="Do not resize images", value=False, elem_id="train_varsize") + steps = gr.Number(label='Max steps', value=100000, precision=0, elem_id="train_steps") + + with FormRow(): + create_image_every = gr.Number(label='Save an image to log directory every N steps, 0 to disable', value=500, precision=0, elem_id="train_create_image_every") + save_embedding_every = gr.Number(label='Save a copy of embedding to log directory every N steps, 0 to disable', value=500, precision=0, elem_id="train_save_embedding_every") + + use_weight = gr.Checkbox(label="Use PNG alpha channel as loss weight", value=False, elem_id="use_weight") + + save_image_with_stored_embedding = gr.Checkbox(label='Save images with embedding in PNG chunks', value=True, elem_id="train_save_image_with_stored_embedding") + preview_from_txt2img = gr.Checkbox(label='Read parameters (prompt, etc...) from txt2img tab when making previews', value=False, elem_id="train_preview_from_txt2img") + + shuffle_tags = gr.Checkbox(label="Shuffle tags by ',' when creating prompts.", value=False, elem_id="train_shuffle_tags") + tag_drop_out = gr.Slider(minimum=0, maximum=1, step=0.1, label="Drop out tags when creating prompts.", value=0, elem_id="train_tag_drop_out") + + latent_sampling_method = gr.Radio(label='Choose latent sampling method', value="once", choices=['once', 'deterministic', 'random'], elem_id="train_latent_sampling_method") + + with gr.Row(): + train_embedding = gr.Button(value="Train Embedding", variant='primary', elem_id="train_train_embedding") + interrupt_training = gr.Button(value="Interrupt", elem_id="train_interrupt_training") + train_hypernetwork = gr.Button(value="Train Hypernetwork", variant='primary', elem_id="train_train_hypernetwork") + + params = script_callbacks.UiTrainTabParams(txt2img_preview_params) + + script_callbacks.ui_train_tabs_callback(params) + + with gr.Column(elem_id='ti_gallery_container'): + ti_output = gr.Text(elem_id="ti_output", value="", show_label=False) + ti_gallery = gr.Gallery(label='Output', show_label=False, elem_id='ti_gallery').style(grid=4) + ti_progress = gr.HTML(elem_id="ti_progress", value="") + ti_outcome = gr.HTML(elem_id="ti_error", value="") + + create_embedding.click( + fn=modules.textual_inversion.ui.create_embedding, + inputs=[ + new_embedding_name, + initialization_text, + nvpt, + overwrite_old_embedding, + ], + outputs=[ + train_embedding_name, + ti_output, + ti_outcome, + ] + ) + + create_hypernetwork.click( + fn=modules.hypernetworks.ui.create_hypernetwork, + inputs=[ + new_hypernetwork_name, + new_hypernetwork_sizes, + overwrite_old_hypernetwork, + new_hypernetwork_layer_structure, + new_hypernetwork_activation_func, + new_hypernetwork_initialization_option, + new_hypernetwork_add_layer_norm, + new_hypernetwork_use_dropout, + new_hypernetwork_dropout_structure + ], + outputs=[ + train_hypernetwork_name, + ti_output, + ti_outcome, + ] + ) + + run_preprocess.click( + fn=wrap_gradio_gpu_call(modules.textual_inversion.ui.preprocess, extra_outputs=[gr.update()]), + _js="start_training_textual_inversion", + inputs=[ + dummy_component, + process_src, + process_dst, + process_width, + process_height, + preprocess_txt_action, + process_flip, + process_split, + process_caption, + process_caption_deepbooru, + process_split_threshold, + process_overlap_ratio, + process_focal_crop, + process_focal_crop_face_weight, + process_focal_crop_entropy_weight, + process_focal_crop_edges_weight, + process_focal_crop_debug, + process_multicrop, + process_multicrop_mindim, + process_multicrop_maxdim, + process_multicrop_minarea, + process_multicrop_maxarea, + process_multicrop_objective, + process_multicrop_threshold, + ], + outputs=[ + ti_output, + ti_outcome, + ], + ) + + train_embedding.click( + fn=wrap_gradio_gpu_call(modules.textual_inversion.ui.train_embedding, extra_outputs=[gr.update()]), + _js="start_training_textual_inversion", + inputs=[ + dummy_component, + train_embedding_name, + embedding_learn_rate, + batch_size, + gradient_step, + dataset_directory, + log_directory, + training_width, + training_height, + varsize, + steps, + clip_grad_mode, + clip_grad_value, + shuffle_tags, + tag_drop_out, + latent_sampling_method, + use_weight, + create_image_every, + save_embedding_every, + template_file, + save_image_with_stored_embedding, + preview_from_txt2img, + *txt2img_preview_params, + ], + outputs=[ + ti_output, + ti_outcome, + ] + ) + + train_hypernetwork.click( + fn=wrap_gradio_gpu_call(modules.hypernetworks.ui.train_hypernetwork, extra_outputs=[gr.update()]), + _js="start_training_textual_inversion", + inputs=[ + dummy_component, + train_hypernetwork_name, + hypernetwork_learn_rate, + batch_size, + gradient_step, + dataset_directory, + log_directory, + training_width, + training_height, + varsize, + steps, + clip_grad_mode, + clip_grad_value, + shuffle_tags, + tag_drop_out, + latent_sampling_method, + use_weight, + create_image_every, + save_embedding_every, + template_file, + preview_from_txt2img, + *txt2img_preview_params, + ], + outputs=[ + ti_output, + ti_outcome, + ] + ) + + interrupt_training.click( + fn=lambda: shared.state.interrupt(), + inputs=[], + outputs=[], + ) + + interrupt_preprocessing.click( + fn=lambda: shared.state.interrupt(), + inputs=[], + outputs=[], + ) + + def create_setting_component(key, is_quicksettings=False): + def fun(): + return opts.data[key] if key in opts.data else opts.data_labels[key].default + + info = opts.data_labels[key] + t = type(info.default) + + args = info.component_args() if callable(info.component_args) else info.component_args + + if info.component is not None: + comp = info.component + elif t == str: + comp = gr.Textbox + elif t == int: + comp = gr.Number + elif t == bool: + comp = gr.Checkbox + else: + raise Exception(f'bad options item type: {str(t)} for key {key}') + + elem_id = "setting_"+key + + if info.refresh is not None: + if is_quicksettings: + res = comp(label=info.label, value=fun(), elem_id=elem_id, **(args or {})) + create_refresh_button(res, info.refresh, info.component_args, "refresh_" + key) + else: + with FormRow(): + res = comp(label=info.label, value=fun(), elem_id=elem_id, **(args or {})) + create_refresh_button(res, info.refresh, info.component_args, "refresh_" + key) + else: + res = comp(label=info.label, value=fun(), elem_id=elem_id, **(args or {})) + + return res + + components = [] + component_dict = {} + shared.settings_components = component_dict + + script_callbacks.ui_settings_callback() + opts.reorder() + + def run_settings(*args): + changed = [] + + for key, value, comp in zip(opts.data_labels.keys(), args, components): + assert comp == dummy_component or opts.same_type(value, opts.data_labels[key].default), f"Bad value for setting {key}: {value}; expecting {type(opts.data_labels[key].default).__name__}" + + for key, value, comp in zip(opts.data_labels.keys(), args, components): + if comp == dummy_component: + continue + + if opts.set(key, value): + changed.append(key) + + try: + opts.save(shared.config_filename) + except RuntimeError: + return opts.dumpjson(), f'{len(changed)} settings changed without save: {", ".join(changed)}.' + return opts.dumpjson(), f'{len(changed)} settings changed{": " if len(changed) > 0 else ""}{", ".join(changed)}.' + + def run_settings_single(value, key): + if not opts.same_type(value, opts.data_labels[key].default): + return gr.update(visible=True), opts.dumpjson() + + if not opts.set(key, value): + return gr.update(value=getattr(opts, key)), opts.dumpjson() + + opts.save(shared.config_filename) + + return get_value_for_setting(key), opts.dumpjson() + + with gr.Blocks(analytics_enabled=False) as settings_interface: + with gr.Row(): + with gr.Column(scale=6): + settings_submit = gr.Button(value="Apply settings", variant='primary', elem_id="settings_submit") + with gr.Column(): + restart_gradio = gr.Button(value='Reload UI', variant='primary', elem_id="settings_restart_gradio") + + result = gr.HTML(elem_id="settings_result") + + quicksettings_names = [x.strip() for x in opts.quicksettings.split(",")] + quicksettings_names = {x: i for i, x in enumerate(quicksettings_names) if x != 'quicksettings'} + + quicksettings_list = [] + + previous_section = None + current_tab = None + current_row = None + with gr.Tabs(elem_id="settings"): + for i, (k, item) in enumerate(opts.data_labels.items()): + section_must_be_skipped = item.section[0] is None + + if previous_section != item.section and not section_must_be_skipped: + elem_id, text = item.section + + if current_tab is not None: + current_row.__exit__() + current_tab.__exit__() + + gr.Group() + current_tab = gr.TabItem(elem_id="settings_{}".format(elem_id), label=text) + current_tab.__enter__() + current_row = gr.Column(variant='compact') + current_row.__enter__() + + previous_section = item.section + + if k in quicksettings_names and not shared.cmd_opts.freeze_settings: + quicksettings_list.append((i, k, item)) + components.append(dummy_component) + elif section_must_be_skipped: + components.append(dummy_component) + else: + component = create_setting_component(k) + component_dict[k] = component + components.append(component) + + if current_tab is not None: + current_row.__exit__() + current_tab.__exit__() + + with gr.TabItem("Actions"): + request_notifications = gr.Button(value='Request browser notifications', elem_id="request_notifications") + download_localization = gr.Button(value='Download localization template', elem_id="download_localization") + reload_script_bodies = gr.Button(value='Reload custom script bodies (No ui updates, No restart)', variant='secondary', elem_id="settings_reload_script_bodies") + + with gr.TabItem("Licenses"): + gr.HTML(shared.html("licenses.html"), elem_id="licenses") + + gr.Button(value="Show all pages", elem_id="settings_show_all_pages") + + request_notifications.click( + fn=lambda: None, + inputs=[], + outputs=[], + _js='function(){}' + ) + + download_localization.click( + fn=lambda: None, + inputs=[], + outputs=[], + _js='download_localization' + ) + + def reload_scripts(): + modules.scripts.reload_script_body_only() + reload_javascript() # need to refresh the html page + + reload_script_bodies.click( + fn=reload_scripts, + inputs=[], + outputs=[] + ) + + def request_restart(): + shared.state.interrupt() + shared.state.need_restart = True + + restart_gradio.click( + fn=request_restart, + _js='restart_reload', + inputs=[], + outputs=[], + ) + + interfaces = [ + (txt2img_interface, "txt2img", "txt2img"), + (img2img_interface, "img2img", "img2img"), + (extras_interface, "Extras", "extras"), + (pnginfo_interface, "PNG Info", "pnginfo"), + (modelmerger_interface, "Checkpoint Merger", "modelmerger"), + (train_interface, "Train", "ti"), + ] + + css = "" + + for cssfile in modules.scripts.list_files_with_name("style.css"): + if not os.path.isfile(cssfile): + continue + + with open(cssfile, "r", encoding="utf8") as file: + css += file.read() + "\n" + + if os.path.exists(os.path.join(data_path, "user.css")): + with open(os.path.join(data_path, "user.css"), "r", encoding="utf8") as file: + css += file.read() + "\n" + + if not cmd_opts.no_progressbar_hiding: + css += css_hide_progressbar + + interfaces += script_callbacks.ui_tabs_callback() + interfaces += [(settings_interface, "Settings", "settings")] + + extensions_interface = ui_extensions.create_ui() + interfaces += [(extensions_interface, "Extensions", "extensions")] + + with gr.Blocks(css=css, analytics_enabled=False, title="Stable Diffusion") as demo: + with gr.Row(elem_id="quicksettings", variant="compact"): + for i, k, item in sorted(quicksettings_list, key=lambda x: quicksettings_names.get(x[1], x[0])): + component = create_setting_component(k, is_quicksettings=True) + component_dict[k] = component + + parameters_copypaste.connect_paste_params_buttons() + + with gr.Tabs(elem_id="tabs") as tabs: + for interface, label, ifid in interfaces: + with gr.TabItem(label, id=ifid, elem_id='tab_' + ifid): + interface.render() + + if os.path.exists(os.path.join(script_path, "notification.mp3")): + audio_notification = gr.Audio(interactive=False, value=os.path.join(script_path, "notification.mp3"), elem_id="audio_notification", visible=False) + + footer = shared.html("footer.html") + footer = footer.format(versions=versions_html()) + gr.HTML(footer, elem_id="footer") + + text_settings = gr.Textbox(elem_id="settings_json", value=lambda: opts.dumpjson(), visible=False) + settings_submit.click( + fn=wrap_gradio_call(run_settings, extra_outputs=[gr.update()]), + inputs=components, + outputs=[text_settings, result], + ) + + for i, k, item in quicksettings_list: + component = component_dict[k] + + component.change( + fn=lambda value, k=k: run_settings_single(value, key=k), + inputs=[component], + outputs=[component, text_settings], + ) + + text_settings.change( + fn=lambda: gr.update(visible=shared.sd_model and shared.sd_model.cond_stage_key == "edit"), + inputs=[], + outputs=[image_cfg_scale], + ) + + button_set_checkpoint = gr.Button('Change checkpoint', elem_id='change_checkpoint', visible=False) + button_set_checkpoint.click( + fn=lambda value, _: run_settings_single(value, key='sd_model_checkpoint'), + _js="function(v){ var res = desiredCheckpointName; desiredCheckpointName = ''; return [res || v, null]; }", + inputs=[component_dict['sd_model_checkpoint'], dummy_component], + outputs=[component_dict['sd_model_checkpoint'], text_settings], + ) + + component_keys = [k for k in opts.data_labels.keys() if k in component_dict] + + def get_settings_values(): + return [get_value_for_setting(key) for key in component_keys] + + demo.load( + fn=get_settings_values, + inputs=[], + outputs=[component_dict[k] for k in component_keys], + ) + + def modelmerger(*args): + try: + results = modules.extras.run_modelmerger(*args) + except Exception as e: + print("Error loading/saving model file:", file=sys.stderr) + print(traceback.format_exc(), file=sys.stderr) + modules.sd_models.list_models() # to remove the potentially missing models from the list + return [*[gr.Dropdown.update(choices=modules.sd_models.checkpoint_tiles()) for _ in range(4)], f"Error merging checkpoints: {e}"] + return results + + modelmerger_merge.click(fn=lambda: '', inputs=[], outputs=[modelmerger_result]) + modelmerger_merge.click( + fn=wrap_gradio_gpu_call(modelmerger, extra_outputs=lambda: [gr.update() for _ in range(4)]), + _js='modelmerger', + inputs=[ + dummy_component, + primary_model_name, + secondary_model_name, + tertiary_model_name, + interp_method, + interp_amount, + save_as_half, + custom_name, + checkpoint_format, + config_source, + bake_in_vae, + discard_weights, + ], + outputs=[ + primary_model_name, + secondary_model_name, + tertiary_model_name, + component_dict['sd_model_checkpoint'], + modelmerger_result, + ] + ) + + ui_config_file = cmd_opts.ui_config_file + ui_settings = {} + settings_count = len(ui_settings) + error_loading = False + + try: + if os.path.exists(ui_config_file): + with open(ui_config_file, "r", encoding="utf8") as file: + ui_settings = json.load(file) + except Exception: + error_loading = True + print("Error loading settings:", file=sys.stderr) + print(traceback.format_exc(), file=sys.stderr) + + def loadsave(path, x): + def apply_field(obj, field, condition=None, init_field=None): + key = path + "/" + field + + if getattr(obj, 'custom_script_source', None) is not None: + key = 'customscript/' + obj.custom_script_source + '/' + key + + if getattr(obj, 'do_not_save_to_config', False): + return + + saved_value = ui_settings.get(key, None) + if saved_value is None: + ui_settings[key] = getattr(obj, field) + elif condition and not condition(saved_value): + pass + + # this warning is generally not useful; + # print(f'Warning: Bad ui setting value: {key}: {saved_value}; Default value "{getattr(obj, field)}" will be used instead.') + else: + setattr(obj, field, saved_value) + if init_field is not None: + init_field(saved_value) + + if type(x) in [gr.Slider, gr.Radio, gr.Checkbox, gr.Textbox, gr.Number, gr.Dropdown] and x.visible: + apply_field(x, 'visible') + + if type(x) == gr.Slider: + apply_field(x, 'value') + apply_field(x, 'minimum') + apply_field(x, 'maximum') + apply_field(x, 'step') + + if type(x) == gr.Radio: + apply_field(x, 'value', lambda val: val in x.choices) + + if type(x) == gr.Checkbox: + apply_field(x, 'value') + + if type(x) == gr.Textbox: + apply_field(x, 'value') + + if type(x) == gr.Number: + apply_field(x, 'value') + + if type(x) == gr.Dropdown: + def check_dropdown(val): + if getattr(x, 'multiselect', False): + return all([value in x.choices for value in val]) + else: + return val in x.choices + + apply_field(x, 'value', check_dropdown, getattr(x, 'init_field', None)) + + visit(txt2img_interface, loadsave, "txt2img") + visit(img2img_interface, loadsave, "img2img") + visit(extras_interface, loadsave, "extras") + visit(modelmerger_interface, loadsave, "modelmerger") + visit(train_interface, loadsave, "train") + + if not error_loading and (not os.path.exists(ui_config_file) or settings_count != len(ui_settings)): + with open(ui_config_file, "w", encoding="utf8") as file: + json.dump(ui_settings, file, indent=4) + + # Required as a workaround for change() event not triggering when loading values from ui-config.json + interp_description.value = update_interp_description(interp_method.value) + + return demo + + +def reload_javascript(): + head = f'\n' + + inline = f"{localization.localization_js(shared.opts.localization)};" + if cmd_opts.theme is not None: + inline += f"set_theme('{cmd_opts.theme}');" + + for script in modules.scripts.list_scripts("javascript", ".js"): + head += f'\n' + + head += f'\n' + + def template_response(*args, **kwargs): + res = shared.GradioTemplateResponseOriginal(*args, **kwargs) + res.body = res.body.replace(b'', f'{head}'.encode("utf8")) + res.init_headers() + return res + + gradio.routes.templates.TemplateResponse = template_response + + +if not hasattr(shared, 'GradioTemplateResponseOriginal'): + shared.GradioTemplateResponseOriginal = gradio.routes.templates.TemplateResponse + + +def versions_html(): + import torch + import launch + + python_version = ".".join([str(x) for x in sys.version_info[0:3]]) + commit = launch.commit_hash() + short_commit = commit[0:8] + + if shared.xformers_available: + import xformers + xformers_version = xformers.__version__ + else: + xformers_version = "N/A" + + return f""" +python: {python_version} + • +torch: {getattr(torch, '__long_version__',torch.__version__)} + • +xformers: {xformers_version} + • +gradio: {gr.__version__} + • +commit: {short_commit} + • +checkpoint: N/A +""" diff --git a/sd/stable-diffusion-webui/modules/ui_common.py b/sd/stable-diffusion-webui/modules/ui_common.py new file mode 100644 index 0000000000000000000000000000000000000000..21ebb0955eec9604d6d41c22eeb1541f70a82580 --- /dev/null +++ b/sd/stable-diffusion-webui/modules/ui_common.py @@ -0,0 +1,206 @@ +import json +import html +import os +import platform +import sys + +import gradio as gr +import subprocess as sp + +from modules import call_queue, shared +from modules.generation_parameters_copypaste import image_from_url_text +import modules.images + +folder_symbol = '\U0001f4c2' # 📂 + + +def update_generation_info(generation_info, html_info, img_index): + try: + generation_info = json.loads(generation_info) + if img_index < 0 or img_index >= len(generation_info["infotexts"]): + return html_info, gr.update() + return plaintext_to_html(generation_info["infotexts"][img_index]), gr.update() + except Exception: + pass + # if the json parse or anything else fails, just return the old html_info + return html_info, gr.update() + + +def plaintext_to_html(text): + text = "" + "
\n".join([f"{html.escape(x)}" for x in text.split('\n')]) + "
| Extension | +URL | +Version | +Update | +
|---|---|---|---|
| + | {remote} | +{ext.version} | +
| Extension | +Description | +Action | +
|---|---|---|
| {html.escape(name)} {tags_text} |
+ {html.escape(description)} Added: {html.escape(added)} |
+ {install_code} | +
Extension hidden: {hidden}
" + + return code, list(tags) + + +def create_ui(): + import modules.ui + + with gr.Blocks(analytics_enabled=False) as ui: + with gr.Tabs(elem_id="tabs_extensions") as tabs: + with gr.TabItem("Installed"): + + with gr.Row(elem_id="extensions_installed_top"): + apply = gr.Button(value="Apply and restart UI", variant="primary") + check = gr.Button(value="Check for updates") + extensions_disabled_list = gr.Text(elem_id="extensions_disabled_list", visible=False).style(container=False) + extensions_update_list = gr.Text(elem_id="extensions_update_list", visible=False).style(container=False) + + info = gr.HTML() + extensions_table = gr.HTML(lambda: extension_table()) + + apply.click( + fn=apply_and_restart, + _js="extensions_apply", + inputs=[extensions_disabled_list, extensions_update_list], + outputs=[], + ) + + check.click( + fn=wrap_gradio_gpu_call(check_updates, extra_outputs=[gr.update()]), + _js="extensions_check", + inputs=[info, extensions_disabled_list], + outputs=[extensions_table, info], + ) + + with gr.TabItem("Available"): + with gr.Row(): + refresh_available_extensions_button = gr.Button(value="Load from:", variant="primary") + available_extensions_index = gr.Text(value="https://raw.githubusercontent.com/wiki/AUTOMATIC1111/stable-diffusion-webui/Extensions-index.md", label="Extension index URL").style(container=False) + extension_to_install = gr.Text(elem_id="extension_to_install", visible=False) + install_extension_button = gr.Button(elem_id="install_extension_button", visible=False) + + with gr.Row(): + hide_tags = gr.CheckboxGroup(value=["ads", "localization", "installed"], label="Hide extensions with tags", choices=["script", "ads", "localization", "installed"]) + sort_column = gr.Radio(value="newest first", label="Order", choices=["newest first", "oldest first", "a-z", "z-a", "internal order", ], type="index") + + install_result = gr.HTML() + available_extensions_table = gr.HTML() + + refresh_available_extensions_button.click( + fn=modules.ui.wrap_gradio_call(refresh_available_extensions, extra_outputs=[gr.update(), gr.update(), gr.update()]), + inputs=[available_extensions_index, hide_tags, sort_column], + outputs=[available_extensions_index, available_extensions_table, hide_tags, install_result], + ) + + install_extension_button.click( + fn=modules.ui.wrap_gradio_call(install_extension_from_index, extra_outputs=[gr.update(), gr.update()]), + inputs=[extension_to_install, hide_tags, sort_column], + outputs=[available_extensions_table, extensions_table, install_result], + ) + + hide_tags.change( + fn=modules.ui.wrap_gradio_call(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]), + inputs=[hide_tags, sort_column], + outputs=[available_extensions_table, install_result] + ) + + sort_column.change( + fn=modules.ui.wrap_gradio_call(refresh_available_extensions_for_tags, extra_outputs=[gr.update()]), + inputs=[hide_tags, sort_column], + outputs=[available_extensions_table, install_result] + ) + + with gr.TabItem("Install from URL"): + install_url = gr.Text(label="URL for extension's git repository") + install_dirname = gr.Text(label="Local directory name", placeholder="Leave empty for auto") + install_button = gr.Button(value="Install", variant="primary") + install_result = gr.HTML(elem_id="extension_install_result") + + install_button.click( + fn=modules.ui.wrap_gradio_call(install_extension_from_url, extra_outputs=[gr.update()]), + inputs=[install_dirname, install_url], + outputs=[extensions_table, install_result], + ) + + return ui diff --git a/sd/stable-diffusion-webui/modules/ui_extra_networks.py b/sd/stable-diffusion-webui/modules/ui_extra_networks.py new file mode 100644 index 0000000000000000000000000000000000000000..e788319dad4aba4484437af166b7c0fccd651798 --- /dev/null +++ b/sd/stable-diffusion-webui/modules/ui_extra_networks.py @@ -0,0 +1,251 @@ +import glob +import os.path +import urllib.parse +from pathlib import Path + +from modules import shared +import gradio as gr +import json +import html + +from modules.generation_parameters_copypaste import image_from_url_text + +extra_pages = [] +allowed_dirs = set() + + +def register_page(page): + """registers extra networks page for the UI; recommend doing it in on_before_ui() callback for extensions""" + + extra_pages.append(page) + allowed_dirs.clear() + allowed_dirs.update(set(sum([x.allowed_directories_for_previews() for x in extra_pages], []))) + + +def add_pages_to_demo(app): + def fetch_file(filename: str = ""): + from starlette.responses import FileResponse + + if not any([Path(x).absolute() in Path(filename).absolute().parents for x in allowed_dirs]): + raise ValueError(f"File cannot be fetched: {filename}. Must be in one of directories registered by extra pages.") + + ext = os.path.splitext(filename)[1].lower() + if ext not in (".png", ".jpg"): + raise ValueError(f"File cannot be fetched: {filename}. Only png and jpg.") + + # would profit from returning 304 + return FileResponse(filename, headers={"Accept-Ranges": "bytes"}) + + app.add_api_route("/sd_extra_networks/thumb", fetch_file, methods=["GET"]) + + +class ExtraNetworksPage: + def __init__(self, title): + self.title = title + self.name = title.lower() + self.card_page = shared.html("extra-networks-card.html") + self.allow_negative_prompt = False + + def refresh(self): + pass + + def link_preview(self, filename): + return "./sd_extra_networks/thumb?filename=" + urllib.parse.quote(filename.replace('\\', '/')) + "&mtime=" + str(os.path.getmtime(filename)) + + def search_terms_from_path(self, filename, possible_directories=None): + abspath = os.path.abspath(filename) + + for parentdir in (possible_directories if possible_directories is not None else self.allowed_directories_for_previews()): + parentdir = os.path.abspath(parentdir) + if abspath.startswith(parentdir): + return abspath[len(parentdir):].replace('\\', '/') + + return "" + + def create_html(self, tabname): + view = shared.opts.extra_networks_default_view + items_html = '' + + subdirs = {} + for parentdir in [os.path.abspath(x) for x in self.allowed_directories_for_previews()]: + for x in glob.glob(os.path.join(parentdir, '**/*'), recursive=True): + if not os.path.isdir(x): + continue + + subdir = os.path.abspath(x)[len(parentdir):].replace("\\", "/") + while subdir.startswith("/"): + subdir = subdir[1:] + + is_empty = len(os.listdir(x)) == 0 + if not is_empty and not subdir.endswith("/"): + subdir = subdir + "/" + + subdirs[subdir] = 1 + + if subdirs: + subdirs = {"": 1, **subdirs} + + subdirs_html = "".join([f""" + +""" for subdir in subdirs]) + + for item in self.list_items(): + items_html += self.create_html_for_item(item, tabname) + + if items_html == '': + dirs = "".join([f"
+{subdirs_html}
+
+
+{items_html}
+
+"""
+
+ return res
+
+ def list_items(self):
+ raise NotImplementedError()
+
+ def allowed_directories_for_previews(self):
+ return []
+
+ def create_html_for_item(self, item, tabname):
+ preview = item.get("preview", None)
+
+ onclick = item.get("onclick", None)
+ if onclick is None:
+ onclick = '"' + html.escape(f"""return cardClicked({json.dumps(tabname)}, {item["prompt"]}, {"true" if self.allow_negative_prompt else "false"})""") + '"'
+
+ args = {
+ "preview_html": "style='background-image: url(\"" + html.escape(preview) + "\")'" if preview else '',
+ "prompt": item.get("prompt", None),
+ "tabname": json.dumps(tabname),
+ "local_preview": json.dumps(item["local_preview"]),
+ "name": item["name"],
+ "card_clicked": onclick,
+ "save_card_preview": '"' + html.escape(f"""return saveCardPreview(event, {json.dumps(tabname)}, {json.dumps(item["local_preview"])})""") + '"',
+ "search_term": item.get("search_term", ""),
+ }
+
+ return self.card_page.format(**args)
+
+
+def intialize():
+ extra_pages.clear()
+
+
+class ExtraNetworksUi:
+ def __init__(self):
+ self.pages = None
+ self.stored_extra_pages = None
+
+ self.button_save_preview = None
+ self.preview_target_filename = None
+
+ self.tabname = None
+
+
+def pages_in_preferred_order(pages):
+ tab_order = [x.lower().strip() for x in shared.opts.ui_extra_networks_tab_reorder.split(",")]
+
+ def tab_name_score(name):
+ name = name.lower()
+ for i, possible_match in enumerate(tab_order):
+ if possible_match in name:
+ return i
+
+ return len(pages)
+
+ tab_scores = {page.name: (tab_name_score(page.name), original_index) for original_index, page in enumerate(pages)}
+
+ return sorted(pages, key=lambda x: tab_scores[x.name])
+
+
+def create_ui(container, button, tabname):
+ ui = ExtraNetworksUi()
+ ui.pages = []
+ ui.stored_extra_pages = pages_in_preferred_order(extra_pages.copy())
+ ui.tabname = tabname
+
+ with gr.Tabs(elem_id=tabname+"_extra_tabs") as tabs:
+ for page in ui.stored_extra_pages:
+ with gr.Tab(page.title):
+ page_elem = gr.HTML(page.create_html(ui.tabname))
+ ui.pages.append(page_elem)
+
+ filter = gr.Textbox('', show_label=False, elem_id=tabname+"_extra_search", placeholder="Search...", visible=False)
+ button_refresh = gr.Button('Refresh', elem_id=tabname+"_extra_refresh")
+ button_close = gr.Button('Close', elem_id=tabname+"_extra_close")
+
+ ui.button_save_preview = gr.Button('Save preview', elem_id=tabname+"_save_preview", visible=False)
+ ui.preview_target_filename = gr.Textbox('Preview save filename', elem_id=tabname+"_preview_filename", visible=False)
+
+ def toggle_visibility(is_visible):
+ is_visible = not is_visible
+ return is_visible, gr.update(visible=is_visible)
+
+ state_visible = gr.State(value=False)
+ button.click(fn=toggle_visibility, inputs=[state_visible], outputs=[state_visible, container])
+ button_close.click(fn=toggle_visibility, inputs=[state_visible], outputs=[state_visible, container])
+
+ def refresh():
+ res = []
+
+ for pg in ui.stored_extra_pages:
+ pg.refresh()
+ res.append(pg.create_html(ui.tabname))
+
+ return res
+
+ button_refresh.click(fn=refresh, inputs=[], outputs=ui.pages)
+
+ return ui
+
+
+def path_is_parent(parent_path, child_path):
+ parent_path = os.path.abspath(parent_path)
+ child_path = os.path.abspath(child_path)
+
+ return child_path.startswith(parent_path)
+
+
+def setup_ui(ui, gallery):
+ def save_preview(index, images, filename):
+ if len(images) == 0:
+ print("There is no image in gallery to save as a preview.")
+ return [page.create_html(ui.tabname) for page in ui.stored_extra_pages]
+
+ index = int(index)
+ index = 0 if index < 0 else index
+ index = len(images) - 1 if index >= len(images) else index
+
+ img_info = images[index if index >= 0 else 0]
+ image = image_from_url_text(img_info)
+
+ is_allowed = False
+ for extra_page in ui.stored_extra_pages:
+ if any([path_is_parent(x, filename) for x in extra_page.allowed_directories_for_previews()]):
+ is_allowed = True
+ break
+
+ assert is_allowed, f'writing to {filename} is not allowed'
+
+ image.save(filename)
+
+ return [page.create_html(ui.tabname) for page in ui.stored_extra_pages]
+
+ ui.button_save_preview.click(
+ fn=save_preview,
+ _js="function(x, y, z){return [selected_gallery_index(), y, z]}",
+ inputs=[ui.preview_target_filename, gallery, ui.preview_target_filename],
+ outputs=[*ui.pages]
+ )
+
diff --git a/sd/stable-diffusion-webui/modules/ui_extra_networks_checkpoints.py b/sd/stable-diffusion-webui/modules/ui_extra_networks_checkpoints.py
new file mode 100644
index 0000000000000000000000000000000000000000..766e2c2499f0a1866e88424a319b99a9df973fc4
--- /dev/null
+++ b/sd/stable-diffusion-webui/modules/ui_extra_networks_checkpoints.py
@@ -0,0 +1,39 @@
+import html
+import json
+import os
+import urllib.parse
+
+from modules import shared, ui_extra_networks, sd_models
+
+
+class ExtraNetworksPageCheckpoints(ui_extra_networks.ExtraNetworksPage):
+ def __init__(self):
+ super().__init__('Checkpoints')
+
+ def refresh(self):
+ shared.refresh_checkpoints()
+
+ def list_items(self):
+ checkpoint: sd_models.CheckpointInfo
+ for name, checkpoint in sd_models.checkpoints_list.items():
+ path, ext = os.path.splitext(checkpoint.filename)
+ previews = [path + ".png", path + ".preview.png"]
+
+ preview = None
+ for file in previews:
+ if os.path.isfile(file):
+ preview = self.link_preview(file)
+ break
+
+ yield {
+ "name": checkpoint.name_for_extra,
+ "filename": path,
+ "preview": preview,
+ "search_term": self.search_terms_from_path(checkpoint.filename) + " " + (checkpoint.sha256 or ""),
+ "onclick": '"' + html.escape(f"""return selectCheckpoint({json.dumps(name)})""") + '"',
+ "local_preview": path + ".png",
+ }
+
+ def allowed_directories_for_previews(self):
+ return [v for v in [shared.cmd_opts.ckpt_dir, sd_models.model_path] if v is not None]
+
diff --git a/sd/stable-diffusion-webui/modules/ui_extra_networks_hypernets.py b/sd/stable-diffusion-webui/modules/ui_extra_networks_hypernets.py
new file mode 100644
index 0000000000000000000000000000000000000000..5fe6516a71e7ca5203fdacec3f750494d5650efd
--- /dev/null
+++ b/sd/stable-diffusion-webui/modules/ui_extra_networks_hypernets.py
@@ -0,0 +1,36 @@
+import json
+import os
+
+from modules import shared, ui_extra_networks
+
+
+class ExtraNetworksPageHypernetworks(ui_extra_networks.ExtraNetworksPage):
+ def __init__(self):
+ super().__init__('Hypernetworks')
+
+ def refresh(self):
+ shared.reload_hypernetworks()
+
+ def list_items(self):
+ for name, path in shared.hypernetworks.items():
+ path, ext = os.path.splitext(path)
+ previews = [path + ".png", path + ".preview.png"]
+
+ preview = None
+ for file in previews:
+ if os.path.isfile(file):
+ preview = self.link_preview(file)
+ break
+
+ yield {
+ "name": name,
+ "filename": path,
+ "preview": preview,
+ "search_term": self.search_terms_from_path(path),
+ "prompt": json.dumps(f"Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8
") + + pixels = gr.Slider(label="Pixels to expand", minimum=8, maximum=256, step=8, value=128, elem_id=self.elem_id("pixels")) + mask_blur = gr.Slider(label='Mask blur', minimum=0, maximum=64, step=1, value=8, elem_id=self.elem_id("mask_blur")) + direction = gr.CheckboxGroup(label="Outpainting direction", choices=['left', 'right', 'up', 'down'], value=['left', 'right', 'up', 'down'], elem_id=self.elem_id("direction")) + noise_q = gr.Slider(label="Fall-off exponent (lower=higher detail)", minimum=0.0, maximum=4.0, step=0.01, value=1.0, elem_id=self.elem_id("noise_q")) + color_variation = gr.Slider(label="Color variation", minimum=0.0, maximum=1.0, step=0.01, value=0.05, elem_id=self.elem_id("color_variation")) + + return [info, pixels, mask_blur, direction, noise_q, color_variation] + + def run(self, p, _, pixels, mask_blur, direction, noise_q, color_variation): + initial_seed_and_info = [None, None] + + process_width = p.width + process_height = p.height + + p.mask_blur = mask_blur*4 + p.inpaint_full_res = False + p.inpainting_fill = 1 + p.do_not_save_samples = True + p.do_not_save_grid = True + + left = pixels if "left" in direction else 0 + right = pixels if "right" in direction else 0 + up = pixels if "up" in direction else 0 + down = pixels if "down" in direction else 0 + + init_img = p.init_images[0] + target_w = math.ceil((init_img.width + left + right) / 64) * 64 + target_h = math.ceil((init_img.height + up + down) / 64) * 64 + + if left > 0: + left = left * (target_w - init_img.width) // (left + right) + + if right > 0: + right = target_w - init_img.width - left + + if up > 0: + up = up * (target_h - init_img.height) // (up + down) + + if down > 0: + down = target_h - init_img.height - up + + def expand(init, count, expand_pixels, is_left=False, is_right=False, is_top=False, is_bottom=False): + is_horiz = is_left or is_right + is_vert = is_top or is_bottom + pixels_horiz = expand_pixels if is_horiz else 0 + pixels_vert = expand_pixels if is_vert else 0 + + images_to_process = [] + output_images = [] + for n in range(count): + res_w = init[n].width + pixels_horiz + res_h = init[n].height + pixels_vert + process_res_w = math.ceil(res_w / 64) * 64 + process_res_h = math.ceil(res_h / 64) * 64 + + img = Image.new("RGB", (process_res_w, process_res_h)) + img.paste(init[n], (pixels_horiz if is_left else 0, pixels_vert if is_top else 0)) + mask = Image.new("RGB", (process_res_w, process_res_h), "white") + draw = ImageDraw.Draw(mask) + draw.rectangle(( + expand_pixels + mask_blur if is_left else 0, + expand_pixels + mask_blur if is_top else 0, + mask.width - expand_pixels - mask_blur if is_right else res_w, + mask.height - expand_pixels - mask_blur if is_bottom else res_h, + ), fill="black") + + np_image = (np.asarray(img) / 255.0).astype(np.float64) + np_mask = (np.asarray(mask) / 255.0).astype(np.float64) + noised = get_matched_noise(np_image, np_mask, noise_q, color_variation) + output_images.append(Image.fromarray(np.clip(noised * 255., 0., 255.).astype(np.uint8), mode="RGB")) + + target_width = min(process_width, init[n].width + pixels_horiz) if is_horiz else img.width + target_height = min(process_height, init[n].height + pixels_vert) if is_vert else img.height + p.width = target_width if is_horiz else img.width + p.height = target_height if is_vert else img.height + + crop_region = ( + 0 if is_left else output_images[n].width - target_width, + 0 if is_top else output_images[n].height - target_height, + target_width if is_left else output_images[n].width, + target_height if is_top else output_images[n].height, + ) + mask = mask.crop(crop_region) + p.image_mask = mask + + image_to_process = output_images[n].crop(crop_region) + images_to_process.append(image_to_process) + + p.init_images = images_to_process + + latent_mask = Image.new("RGB", (p.width, p.height), "white") + draw = ImageDraw.Draw(latent_mask) + draw.rectangle(( + expand_pixels + mask_blur * 2 if is_left else 0, + expand_pixels + mask_blur * 2 if is_top else 0, + mask.width - expand_pixels - mask_blur * 2 if is_right else res_w, + mask.height - expand_pixels - mask_blur * 2 if is_bottom else res_h, + ), fill="black") + p.latent_mask = latent_mask + + proc = process_images(p) + + if initial_seed_and_info[0] is None: + initial_seed_and_info[0] = proc.seed + initial_seed_and_info[1] = proc.info + + for n in range(count): + output_images[n].paste(proc.images[n], (0 if is_left else output_images[n].width - proc.images[n].width, 0 if is_top else output_images[n].height - proc.images[n].height)) + output_images[n] = output_images[n].crop((0, 0, res_w, res_h)) + + return output_images + + batch_count = p.n_iter + batch_size = p.batch_size + p.n_iter = 1 + state.job_count = batch_count * ((1 if left > 0 else 0) + (1 if right > 0 else 0) + (1 if up > 0 else 0) + (1 if down > 0 else 0)) + all_processed_images = [] + + for i in range(batch_count): + imgs = [init_img] * batch_size + state.job = f"Batch {i + 1} out of {batch_count}" + + if left > 0: + imgs = expand(imgs, batch_size, left, is_left=True) + if right > 0: + imgs = expand(imgs, batch_size, right, is_right=True) + if up > 0: + imgs = expand(imgs, batch_size, up, is_top=True) + if down > 0: + imgs = expand(imgs, batch_size, down, is_bottom=True) + + all_processed_images += imgs + + all_images = all_processed_images + + combined_grid_image = images.image_grid(all_processed_images) + unwanted_grid_because_of_img_count = len(all_processed_images) < 2 and opts.grid_only_if_multiple + if opts.return_grid and not unwanted_grid_because_of_img_count: + all_images = [combined_grid_image] + all_processed_images + + res = Processed(p, all_images, initial_seed_and_info[0], initial_seed_and_info[1]) + + if opts.samples_save: + for img in all_processed_images: + images.save_image(img, p.outpath_samples, "", res.seed, p.prompt, opts.grid_format, info=res.info, p=p) + + if opts.grid_save and not unwanted_grid_because_of_img_count: + images.save_image(combined_grid_image, p.outpath_grids, "grid", res.seed, p.prompt, opts.grid_format, info=res.info, short_filename=not opts.grid_extended_filename, grid=True, p=p) + + return res diff --git a/sd/stable-diffusion-webui/scripts/poor_mans_outpainting.py b/sd/stable-diffusion-webui/scripts/poor_mans_outpainting.py new file mode 100644 index 0000000000000000000000000000000000000000..d39f61c1073376eae210d955ac1e9eba836402da --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/poor_mans_outpainting.py @@ -0,0 +1,146 @@ +import math + +import modules.scripts as scripts +import gradio as gr +from PIL import Image, ImageDraw + +from modules import images, processing, devices +from modules.processing import Processed, process_images +from modules.shared import opts, cmd_opts, state + + +class Script(scripts.Script): + def title(self): + return "Poor man's outpainting" + + def show(self, is_img2img): + return is_img2img + + def ui(self, is_img2img): + if not is_img2img: + return None + + pixels = gr.Slider(label="Pixels to expand", minimum=8, maximum=256, step=8, value=128, elem_id=self.elem_id("pixels")) + mask_blur = gr.Slider(label='Mask blur', minimum=0, maximum=64, step=1, value=4, elem_id=self.elem_id("mask_blur")) + inpainting_fill = gr.Radio(label='Masked content', choices=['fill', 'original', 'latent noise', 'latent nothing'], value='fill', type="index", elem_id=self.elem_id("inpainting_fill")) + direction = gr.CheckboxGroup(label="Outpainting direction", choices=['left', 'right', 'up', 'down'], value=['left', 'right', 'up', 'down'], elem_id=self.elem_id("direction")) + + return [pixels, mask_blur, inpainting_fill, direction] + + def run(self, p, pixels, mask_blur, inpainting_fill, direction): + initial_seed = None + initial_info = None + + p.mask_blur = mask_blur * 2 + p.inpainting_fill = inpainting_fill + p.inpaint_full_res = False + + left = pixels if "left" in direction else 0 + right = pixels if "right" in direction else 0 + up = pixels if "up" in direction else 0 + down = pixels if "down" in direction else 0 + + init_img = p.init_images[0] + target_w = math.ceil((init_img.width + left + right) / 64) * 64 + target_h = math.ceil((init_img.height + up + down) / 64) * 64 + + if left > 0: + left = left * (target_w - init_img.width) // (left + right) + if right > 0: + right = target_w - init_img.width - left + + if up > 0: + up = up * (target_h - init_img.height) // (up + down) + + if down > 0: + down = target_h - init_img.height - up + + img = Image.new("RGB", (target_w, target_h)) + img.paste(init_img, (left, up)) + + mask = Image.new("L", (img.width, img.height), "white") + draw = ImageDraw.Draw(mask) + draw.rectangle(( + left + (mask_blur * 2 if left > 0 else 0), + up + (mask_blur * 2 if up > 0 else 0), + mask.width - right - (mask_blur * 2 if right > 0 else 0), + mask.height - down - (mask_blur * 2 if down > 0 else 0) + ), fill="black") + + latent_mask = Image.new("L", (img.width, img.height), "white") + latent_draw = ImageDraw.Draw(latent_mask) + latent_draw.rectangle(( + left + (mask_blur//2 if left > 0 else 0), + up + (mask_blur//2 if up > 0 else 0), + mask.width - right - (mask_blur//2 if right > 0 else 0), + mask.height - down - (mask_blur//2 if down > 0 else 0) + ), fill="black") + + devices.torch_gc() + + grid = images.split_grid(img, tile_w=p.width, tile_h=p.height, overlap=pixels) + grid_mask = images.split_grid(mask, tile_w=p.width, tile_h=p.height, overlap=pixels) + grid_latent_mask = images.split_grid(latent_mask, tile_w=p.width, tile_h=p.height, overlap=pixels) + + p.n_iter = 1 + p.batch_size = 1 + p.do_not_save_grid = True + p.do_not_save_samples = True + + work = [] + work_mask = [] + work_latent_mask = [] + work_results = [] + + for (y, h, row), (_, _, row_mask), (_, _, row_latent_mask) in zip(grid.tiles, grid_mask.tiles, grid_latent_mask.tiles): + for tiledata, tiledata_mask, tiledata_latent_mask in zip(row, row_mask, row_latent_mask): + x, w = tiledata[0:2] + + if x >= left and x+w <= img.width - right and y >= up and y+h <= img.height - down: + continue + + work.append(tiledata[2]) + work_mask.append(tiledata_mask[2]) + work_latent_mask.append(tiledata_latent_mask[2]) + + batch_count = len(work) + print(f"Poor man's outpainting will process a total of {len(work)} images tiled as {len(grid.tiles[0][2])}x{len(grid.tiles)}.") + + state.job_count = batch_count + + for i in range(batch_count): + p.init_images = [work[i]] + p.image_mask = work_mask[i] + p.latent_mask = work_latent_mask[i] + + state.job = f"Batch {i + 1} out of {batch_count}" + processed = process_images(p) + + if initial_seed is None: + initial_seed = processed.seed + initial_info = processed.info + + p.seed = processed.seed + 1 + work_results += processed.images + + + image_index = 0 + for y, h, row in grid.tiles: + for tiledata in row: + x, w = tiledata[0:2] + + if x >= left and x+w <= img.width - right and y >= up and y+h <= img.height - down: + continue + + tiledata[2] = work_results[image_index] if image_index < len(work_results) else Image.new("RGB", (p.width, p.height)) + image_index += 1 + + combined_image = images.combine_grid(grid) + + if opts.samples_save: + images.save_image(combined_image, p.outpath_samples, "", initial_seed, p.prompt, opts.grid_format, info=initial_info, p=p) + + processed = Processed(p, [combined_image], initial_seed, initial_info) + + return processed + diff --git a/sd/stable-diffusion-webui/scripts/postprocessing_codeformer.py b/sd/stable-diffusion-webui/scripts/postprocessing_codeformer.py new file mode 100644 index 0000000000000000000000000000000000000000..7e337ec41ffffe11fd88fced0ff4f6338d959571 --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/postprocessing_codeformer.py @@ -0,0 +1,36 @@ +from PIL import Image +import numpy as np + +from modules import scripts_postprocessing, codeformer_model +import gradio as gr + +from modules.ui_components import FormRow + + +class ScriptPostprocessingCodeFormer(scripts_postprocessing.ScriptPostprocessing): + name = "CodeFormer" + order = 3000 + + def ui(self): + with FormRow(): + codeformer_visibility = gr.Slider(minimum=0.0, maximum=1.0, step=0.001, label="CodeFormer visibility", value=0, elem_id="extras_codeformer_visibility") + codeformer_weight = gr.Slider(minimum=0.0, maximum=1.0, step=0.001, label="CodeFormer weight (0 = maximum effect, 1 = minimum effect)", value=0, elem_id="extras_codeformer_weight") + + return { + "codeformer_visibility": codeformer_visibility, + "codeformer_weight": codeformer_weight, + } + + def process(self, pp: scripts_postprocessing.PostprocessedImage, codeformer_visibility, codeformer_weight): + if codeformer_visibility == 0: + return + + restored_img = codeformer_model.codeformer.restore(np.array(pp.image, dtype=np.uint8), w=codeformer_weight) + res = Image.fromarray(restored_img) + + if codeformer_visibility < 1.0: + res = Image.blend(pp.image, res, codeformer_visibility) + + pp.image = res + pp.info["CodeFormer visibility"] = round(codeformer_visibility, 3) + pp.info["CodeFormer weight"] = round(codeformer_weight, 3) diff --git a/sd/stable-diffusion-webui/scripts/postprocessing_gfpgan.py b/sd/stable-diffusion-webui/scripts/postprocessing_gfpgan.py new file mode 100644 index 0000000000000000000000000000000000000000..9f7c2baaa28333958818d332324b34bcb8bce3ca --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/postprocessing_gfpgan.py @@ -0,0 +1,33 @@ +from PIL import Image +import numpy as np + +from modules import scripts_postprocessing, gfpgan_model +import gradio as gr + +from modules.ui_components import FormRow + + +class ScriptPostprocessingGfpGan(scripts_postprocessing.ScriptPostprocessing): + name = "GFPGAN" + order = 2000 + + def ui(self): + with FormRow(): + gfpgan_visibility = gr.Slider(minimum=0.0, maximum=1.0, step=0.001, label="GFPGAN visibility", value=0, elem_id="extras_gfpgan_visibility") + + return { + "gfpgan_visibility": gfpgan_visibility, + } + + def process(self, pp: scripts_postprocessing.PostprocessedImage, gfpgan_visibility): + if gfpgan_visibility == 0: + return + + restored_img = gfpgan_model.gfpgan_fix_faces(np.array(pp.image, dtype=np.uint8)) + res = Image.fromarray(restored_img) + + if gfpgan_visibility < 1.0: + res = Image.blend(pp.image, res, gfpgan_visibility) + + pp.image = res + pp.info["GFPGAN visibility"] = round(gfpgan_visibility, 3) diff --git a/sd/stable-diffusion-webui/scripts/postprocessing_upscale.py b/sd/stable-diffusion-webui/scripts/postprocessing_upscale.py new file mode 100644 index 0000000000000000000000000000000000000000..ccec72fcbc72eeffbe24a659bf53ecba71162391 --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/postprocessing_upscale.py @@ -0,0 +1,131 @@ +from PIL import Image +import numpy as np + +from modules import scripts_postprocessing, shared +import gradio as gr + +from modules.ui_components import FormRow + + +upscale_cache = {} + + +class ScriptPostprocessingUpscale(scripts_postprocessing.ScriptPostprocessing): + name = "Upscale" + order = 1000 + + def ui(self): + selected_tab = gr.State(value=0) + + with gr.Tabs(elem_id="extras_resize_mode"): + with gr.TabItem('Scale by', elem_id="extras_scale_by_tab") as tab_scale_by: + upscaling_resize = gr.Slider(minimum=1.0, maximum=8.0, step=0.05, label="Resize", value=4, elem_id="extras_upscaling_resize") + + with gr.TabItem('Scale to', elem_id="extras_scale_to_tab") as tab_scale_to: + with FormRow(): + upscaling_resize_w = gr.Number(label="Width", value=512, precision=0, elem_id="extras_upscaling_resize_w") + upscaling_resize_h = gr.Number(label="Height", value=512, precision=0, elem_id="extras_upscaling_resize_h") + upscaling_crop = gr.Checkbox(label='Crop to fit', value=True, elem_id="extras_upscaling_crop") + + with FormRow(): + extras_upscaler_1 = gr.Dropdown(label='Upscaler 1', elem_id="extras_upscaler_1", choices=[x.name for x in shared.sd_upscalers], value=shared.sd_upscalers[0].name) + + with FormRow(): + extras_upscaler_2 = gr.Dropdown(label='Upscaler 2', elem_id="extras_upscaler_2", choices=[x.name for x in shared.sd_upscalers], value=shared.sd_upscalers[0].name) + extras_upscaler_2_visibility = gr.Slider(minimum=0.0, maximum=1.0, step=0.001, label="Upscaler 2 visibility", value=0.0, elem_id="extras_upscaler_2_visibility") + + tab_scale_by.select(fn=lambda: 0, inputs=[], outputs=[selected_tab]) + tab_scale_to.select(fn=lambda: 1, inputs=[], outputs=[selected_tab]) + + return { + "upscale_mode": selected_tab, + "upscale_by": upscaling_resize, + "upscale_to_width": upscaling_resize_w, + "upscale_to_height": upscaling_resize_h, + "upscale_crop": upscaling_crop, + "upscaler_1_name": extras_upscaler_1, + "upscaler_2_name": extras_upscaler_2, + "upscaler_2_visibility": extras_upscaler_2_visibility, + } + + def upscale(self, image, info, upscaler, upscale_mode, upscale_by, upscale_to_width, upscale_to_height, upscale_crop): + if upscale_mode == 1: + upscale_by = max(upscale_to_width/image.width, upscale_to_height/image.height) + info["Postprocess upscale to"] = f"{upscale_to_width}x{upscale_to_height}" + else: + info["Postprocess upscale by"] = upscale_by + + cache_key = (hash(np.array(image.getdata()).tobytes()), upscaler.name, upscale_mode, upscale_by, upscale_to_width, upscale_to_height, upscale_crop) + cached_image = upscale_cache.pop(cache_key, None) + + if cached_image is not None: + image = cached_image + else: + image = upscaler.scaler.upscale(image, upscale_by, upscaler.data_path) + + upscale_cache[cache_key] = image + if len(upscale_cache) > shared.opts.upscaling_max_images_in_cache: + upscale_cache.pop(next(iter(upscale_cache), None), None) + + if upscale_mode == 1 and upscale_crop: + cropped = Image.new("RGB", (upscale_to_width, upscale_to_height)) + cropped.paste(image, box=(upscale_to_width // 2 - image.width // 2, upscale_to_height // 2 - image.height // 2)) + image = cropped + info["Postprocess crop to"] = f"{image.width}x{image.height}" + + return image + + def process(self, pp: scripts_postprocessing.PostprocessedImage, upscale_mode=1, upscale_by=2.0, upscale_to_width=None, upscale_to_height=None, upscale_crop=False, upscaler_1_name=None, upscaler_2_name=None, upscaler_2_visibility=0.0): + if upscaler_1_name == "None": + upscaler_1_name = None + + upscaler1 = next(iter([x for x in shared.sd_upscalers if x.name == upscaler_1_name]), None) + assert upscaler1 or (upscaler_1_name is None), f'could not find upscaler named {upscaler_1_name}' + + if not upscaler1: + return + + if upscaler_2_name == "None": + upscaler_2_name = None + + upscaler2 = next(iter([x for x in shared.sd_upscalers if x.name == upscaler_2_name and x.name != "None"]), None) + assert upscaler2 or (upscaler_2_name is None), f'could not find upscaler named {upscaler_2_name}' + + upscaled_image = self.upscale(pp.image, pp.info, upscaler1, upscale_mode, upscale_by, upscale_to_width, upscale_to_height, upscale_crop) + pp.info[f"Postprocess upscaler"] = upscaler1.name + + if upscaler2 and upscaler_2_visibility > 0: + second_upscale = self.upscale(pp.image, pp.info, upscaler2, upscale_mode, upscale_by, upscale_to_width, upscale_to_height, upscale_crop) + upscaled_image = Image.blend(upscaled_image, second_upscale, upscaler_2_visibility) + + pp.info[f"Postprocess upscaler 2"] = upscaler2.name + + pp.image = upscaled_image + + def image_changed(self): + upscale_cache.clear() + + +class ScriptPostprocessingUpscaleSimple(ScriptPostprocessingUpscale): + name = "Simple Upscale" + order = 900 + + def ui(self): + with FormRow(): + upscaler_name = gr.Dropdown(label='Upscaler', choices=[x.name for x in shared.sd_upscalers], value=shared.sd_upscalers[0].name) + upscale_by = gr.Slider(minimum=0.05, maximum=8.0, step=0.05, label="Upscale by", value=2) + + return { + "upscale_by": upscale_by, + "upscaler_name": upscaler_name, + } + + def process(self, pp: scripts_postprocessing.PostprocessedImage, upscale_by=2.0, upscaler_name=None): + if upscaler_name is None or upscaler_name == "None": + return + + upscaler1 = next(iter([x for x in shared.sd_upscalers if x.name == upscaler_name]), None) + assert upscaler1, f'could not find upscaler named {upscaler_name}' + + pp.image = self.upscale(pp.image, pp.info, upscaler1, 0, upscale_by, 0, 0, False) + pp.info[f"Postprocess upscaler"] = upscaler1.name diff --git a/sd/stable-diffusion-webui/scripts/prompt_matrix.py b/sd/stable-diffusion-webui/scripts/prompt_matrix.py new file mode 100644 index 0000000000000000000000000000000000000000..51c70998866d4b0853a46e4de73d86c3d9ec9b93 --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/prompt_matrix.py @@ -0,0 +1,111 @@ +import math +from collections import namedtuple +from copy import copy +import random + +import modules.scripts as scripts +import gradio as gr + +from modules import images +from modules.processing import process_images, Processed +from modules.shared import opts, cmd_opts, state +import modules.sd_samplers + + +def draw_xy_grid(xs, ys, x_label, y_label, cell): + res = [] + + ver_texts = [[images.GridAnnotation(y_label(y))] for y in ys] + hor_texts = [[images.GridAnnotation(x_label(x))] for x in xs] + + first_processed = None + + state.job_count = len(xs) * len(ys) + + for iy, y in enumerate(ys): + for ix, x in enumerate(xs): + state.job = f"{ix + iy * len(xs) + 1} out of {len(xs) * len(ys)}" + + processed = cell(x, y) + if first_processed is None: + first_processed = processed + + res.append(processed.images[0]) + + grid = images.image_grid(res, rows=len(ys)) + grid = images.draw_grid_annotations(grid, res[0].width, res[0].height, hor_texts, ver_texts) + + first_processed.images = [grid] + + return first_processed + + +class Script(scripts.Script): + def title(self): + return "Prompt matrix" + + def ui(self, is_img2img): + gr.HTML('') + with gr.Row(): + with gr.Column(): + put_at_start = gr.Checkbox(label='Put variable parts at start of prompt', value=False, elem_id=self.elem_id("put_at_start")) + different_seeds = gr.Checkbox(label='Use different seed for each picture', value=False, elem_id=self.elem_id("different_seeds")) + with gr.Column(): + prompt_type = gr.Radio(["positive", "negative"], label="Select prompt", elem_id=self.elem_id("prompt_type"), value="positive") + variations_delimiter = gr.Radio(["comma", "space"], label="Select joining char", elem_id=self.elem_id("variations_delimiter"), value="comma") + with gr.Column(): + margin_size = gr.Slider(label="Grid margins (px)", minimum=0, maximum=500, value=0, step=2, elem_id=self.elem_id("margin_size")) + + return [put_at_start, different_seeds, prompt_type, variations_delimiter, margin_size] + + def run(self, p, put_at_start, different_seeds, prompt_type, variations_delimiter, margin_size): + modules.processing.fix_seed(p) + # Raise error if promp type is not positive or negative + if prompt_type not in ["positive", "negative"]: + raise ValueError(f"Unknown prompt type {prompt_type}") + # Raise error if variations delimiter is not comma or space + if variations_delimiter not in ["comma", "space"]: + raise ValueError(f"Unknown variations delimiter {variations_delimiter}") + + prompt = p.prompt if prompt_type == "positive" else p.negative_prompt + original_prompt = prompt[0] if type(prompt) == list else prompt + positive_prompt = p.prompt[0] if type(p.prompt) == list else p.prompt + + delimiter = ", " if variations_delimiter == "comma" else " " + + all_prompts = [] + prompt_matrix_parts = original_prompt.split("|") + combination_count = 2 ** (len(prompt_matrix_parts) - 1) + for combination_num in range(combination_count): + selected_prompts = [text.strip().strip(',') for n, text in enumerate(prompt_matrix_parts[1:]) if combination_num & (1 << n)] + + if put_at_start: + selected_prompts = selected_prompts + [prompt_matrix_parts[0]] + else: + selected_prompts = [prompt_matrix_parts[0]] + selected_prompts + + all_prompts.append(delimiter.join(selected_prompts)) + + p.n_iter = math.ceil(len(all_prompts) / p.batch_size) + p.do_not_save_grid = True + + print(f"Prompt matrix will create {len(all_prompts)} images using a total of {p.n_iter} batches.") + + if prompt_type == "positive": + p.prompt = all_prompts + else: + p.negative_prompt = all_prompts + p.seed = [p.seed + (i if different_seeds else 0) for i in range(len(all_prompts))] + p.prompt_for_display = positive_prompt + processed = process_images(p) + + grid = images.image_grid(processed.images, p.batch_size, rows=1 << ((len(prompt_matrix_parts) - 1) // 2)) + grid = images.draw_prompt_matrix(grid, processed.images[0].width, processed.images[1].height, prompt_matrix_parts, margin_size) + processed.images.insert(0, grid) + processed.index_of_first_image = 1 + processed.infotexts.insert(0, processed.infotexts[0]) + + if opts.grid_save: + images.save_image(processed.images[0], p.outpath_grids, "prompt_matrix", extension=opts.grid_format, prompt=original_prompt, seed=processed.seed, grid=True, p=p) + + return processed diff --git a/sd/stable-diffusion-webui/scripts/prompts_from_file.py b/sd/stable-diffusion-webui/scripts/prompts_from_file.py new file mode 100644 index 0000000000000000000000000000000000000000..17c9c967ffeeb3f538c6f95d93ae79c32ca17828 --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/prompts_from_file.py @@ -0,0 +1,177 @@ +import copy +import math +import os +import random +import sys +import traceback +import shlex + +import modules.scripts as scripts +import gradio as gr + +from modules import sd_samplers +from modules.processing import Processed, process_images +from PIL import Image +from modules.shared import opts, cmd_opts, state + + +def process_string_tag(tag): + return tag + + +def process_int_tag(tag): + return int(tag) + + +def process_float_tag(tag): + return float(tag) + + +def process_boolean_tag(tag): + return True if (tag == "true") else False + + +prompt_tags = { + "sd_model": None, + "outpath_samples": process_string_tag, + "outpath_grids": process_string_tag, + "prompt_for_display": process_string_tag, + "prompt": process_string_tag, + "negative_prompt": process_string_tag, + "styles": process_string_tag, + "seed": process_int_tag, + "subseed_strength": process_float_tag, + "subseed": process_int_tag, + "seed_resize_from_h": process_int_tag, + "seed_resize_from_w": process_int_tag, + "sampler_index": process_int_tag, + "sampler_name": process_string_tag, + "batch_size": process_int_tag, + "n_iter": process_int_tag, + "steps": process_int_tag, + "cfg_scale": process_float_tag, + "width": process_int_tag, + "height": process_int_tag, + "restore_faces": process_boolean_tag, + "tiling": process_boolean_tag, + "do_not_save_samples": process_boolean_tag, + "do_not_save_grid": process_boolean_tag +} + + +def cmdargs(line): + args = shlex.split(line) + pos = 0 + res = {} + + while pos < len(args): + arg = args[pos] + + assert arg.startswith("--"), f'must start with "--": {arg}' + assert pos+1 < len(args), f'missing argument for command line option {arg}' + + tag = arg[2:] + + if tag == "prompt" or tag == "negative_prompt": + pos += 1 + prompt = args[pos] + pos += 1 + while pos < len(args) and not args[pos].startswith("--"): + prompt += " " + prompt += args[pos] + pos += 1 + res[tag] = prompt + continue + + + func = prompt_tags.get(tag, None) + assert func, f'unknown commandline option: {arg}' + + val = args[pos+1] + if tag == "sampler_name": + val = sd_samplers.samplers_map.get(val.lower(), None) + + res[tag] = func(val) + + pos += 2 + + return res + + +def load_prompt_file(file): + if file is None: + lines = [] + else: + lines = [x.strip() for x in file.decode('utf8', errors='ignore').split("\n")] + + return None, "\n".join(lines), gr.update(lines=7) + + +class Script(scripts.Script): + def title(self): + return "Prompts from file or textbox" + + def ui(self, is_img2img): + checkbox_iterate = gr.Checkbox(label="Iterate seed every line", value=False, elem_id=self.elem_id("checkbox_iterate")) + checkbox_iterate_batch = gr.Checkbox(label="Use same random seed for all lines", value=False, elem_id=self.elem_id("checkbox_iterate_batch")) + + prompt_txt = gr.Textbox(label="List of prompt inputs", lines=1, elem_id=self.elem_id("prompt_txt")) + file = gr.File(label="Upload prompt inputs", type='binary', elem_id=self.elem_id("file")) + + file.change(fn=load_prompt_file, inputs=[file], outputs=[file, prompt_txt, prompt_txt]) + + # We start at one line. When the text changes, we jump to seven lines, or two lines if no \n. + # We don't shrink back to 1, because that causes the control to ignore [enter], and it may + # be unclear to the user that shift-enter is needed. + prompt_txt.change(lambda tb: gr.update(lines=7) if ("\n" in tb) else gr.update(lines=2), inputs=[prompt_txt], outputs=[prompt_txt]) + return [checkbox_iterate, checkbox_iterate_batch, prompt_txt] + + def run(self, p, checkbox_iterate, checkbox_iterate_batch, prompt_txt: str): + lines = [x.strip() for x in prompt_txt.splitlines()] + lines = [x for x in lines if len(x) > 0] + + p.do_not_save_grid = True + + job_count = 0 + jobs = [] + + for line in lines: + if "--" in line: + try: + args = cmdargs(line) + except Exception: + print(f"Error parsing line {line} as commandline:", file=sys.stderr) + print(traceback.format_exc(), file=sys.stderr) + args = {"prompt": line} + else: + args = {"prompt": line} + + job_count += args.get("n_iter", p.n_iter) + + jobs.append(args) + + print(f"Will process {len(lines)} lines in {job_count} jobs.") + if (checkbox_iterate or checkbox_iterate_batch) and p.seed == -1: + p.seed = int(random.randrange(4294967294)) + + state.job_count = job_count + + images = [] + all_prompts = [] + infotexts = [] + for n, args in enumerate(jobs): + state.job = f"{state.job_no + 1} out of {state.job_count}" + + copy_p = copy.copy(p) + for k, v in args.items(): + setattr(copy_p, k, v) + + proc = process_images(copy_p) + images += proc.images + + if checkbox_iterate: + p.seed = p.seed + (p.batch_size * p.n_iter) + all_prompts += proc.all_prompts + infotexts += proc.infotexts + + return Processed(p, images, p.seed, "", all_prompts=all_prompts, infotexts=infotexts) diff --git a/sd/stable-diffusion-webui/scripts/sd_upscale.py b/sd/stable-diffusion-webui/scripts/sd_upscale.py new file mode 100644 index 0000000000000000000000000000000000000000..dd64d7d385b34f960eebd7aec3233a84ea90609a --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/sd_upscale.py @@ -0,0 +1,101 @@ +import math + +import modules.scripts as scripts +import gradio as gr +from PIL import Image + +from modules import processing, shared, sd_samplers, images, devices +from modules.processing import Processed +from modules.shared import opts, cmd_opts, state + + +class Script(scripts.Script): + def title(self): + return "SD upscale" + + def show(self, is_img2img): + return is_img2img + + def ui(self, is_img2img): + info = gr.HTML("
Will upscale the image by the selected scale factor; use width and height sliders to set tile size
") + overlap = gr.Slider(minimum=0, maximum=256, step=16, label='Tile overlap', value=64, elem_id=self.elem_id("overlap")) + scale_factor = gr.Slider(minimum=1.0, maximum=4.0, step=0.05, label='Scale Factor', value=2.0, elem_id=self.elem_id("scale_factor")) + upscaler_index = gr.Radio(label='Upscaler', choices=[x.name for x in shared.sd_upscalers], value=shared.sd_upscalers[0].name, type="index", elem_id=self.elem_id("upscaler_index")) + + return [info, overlap, upscaler_index, scale_factor] + + def run(self, p, _, overlap, upscaler_index, scale_factor): + if isinstance(upscaler_index, str): + upscaler_index = [x.name.lower() for x in shared.sd_upscalers].index(upscaler_index.lower()) + processing.fix_seed(p) + upscaler = shared.sd_upscalers[upscaler_index] + + p.extra_generation_params["SD upscale overlap"] = overlap + p.extra_generation_params["SD upscale upscaler"] = upscaler.name + + initial_info = None + seed = p.seed + + init_img = p.init_images[0] + init_img = images.flatten(init_img, opts.img2img_background_color) + + if upscaler.name != "None": + img = upscaler.scaler.upscale(init_img, scale_factor, upscaler.data_path) + else: + img = init_img + + devices.torch_gc() + + grid = images.split_grid(img, tile_w=p.width, tile_h=p.height, overlap=overlap) + + batch_size = p.batch_size + upscale_count = p.n_iter + p.n_iter = 1 + p.do_not_save_grid = True + p.do_not_save_samples = True + + work = [] + + for y, h, row in grid.tiles: + for tiledata in row: + work.append(tiledata[2]) + + batch_count = math.ceil(len(work) / batch_size) + state.job_count = batch_count * upscale_count + + print(f"SD upscaling will process a total of {len(work)} images tiled as {len(grid.tiles[0][2])}x{len(grid.tiles)} per upscale in a total of {state.job_count} batches.") + + result_images = [] + for n in range(upscale_count): + start_seed = seed + n + p.seed = start_seed + + work_results = [] + for i in range(batch_count): + p.batch_size = batch_size + p.init_images = work[i * batch_size:(i + 1) * batch_size] + + state.job = f"Batch {i + 1 + n * batch_count} out of {state.job_count}" + processed = processing.process_images(p) + + if initial_info is None: + initial_info = processed.info + + p.seed = processed.seed + 1 + work_results += processed.images + + image_index = 0 + for y, h, row in grid.tiles: + for tiledata in row: + tiledata[2] = work_results[image_index] if image_index < len(work_results) else Image.new("RGB", (p.width, p.height)) + image_index += 1 + + combined_image = images.combine_grid(grid) + result_images.append(combined_image) + + if opts.samples_save: + images.save_image(combined_image, p.outpath_samples, "", start_seed, p.prompt, opts.samples_format, info=initial_info, p=p) + + processed = Processed(p, result_images, seed, initial_info) + + return processed diff --git a/sd/stable-diffusion-webui/scripts/xyz_grid.py b/sd/stable-diffusion-webui/scripts/xyz_grid.py new file mode 100644 index 0000000000000000000000000000000000000000..e457d53de2ade37a53cc200d6902323d3d6f25ce --- /dev/null +++ b/sd/stable-diffusion-webui/scripts/xyz_grid.py @@ -0,0 +1,620 @@ +from collections import namedtuple +from copy import copy +from itertools import permutations, chain +import random +import csv +from io import StringIO +from PIL import Image +import numpy as np + +import modules.scripts as scripts +import gradio as gr + +from modules import images, paths, sd_samplers, processing, sd_models, sd_vae +from modules.processing import process_images, Processed, StableDiffusionProcessingTxt2Img +from modules.shared import opts, cmd_opts, state +import modules.shared as shared +import modules.sd_samplers +import modules.sd_models +import modules.sd_vae +import glob +import os +import re + +from modules.ui_components import ToolButton + +fill_values_symbol = "\U0001f4d2" # 📒 + +AxisInfo = namedtuple('AxisInfo', ['axis', 'values']) + + +def apply_field(field): + def fun(p, x, xs): + setattr(p, field, x) + + return fun + + +def apply_prompt(p, x, xs): + if xs[0] not in p.prompt and xs[0] not in p.negative_prompt: + raise RuntimeError(f"Prompt S/R did not find {xs[0]} in prompt or negative prompt.") + + p.prompt = p.prompt.replace(xs[0], x) + p.negative_prompt = p.negative_prompt.replace(xs[0], x) + + +def apply_order(p, x, xs): + token_order = [] + + # Initally grab the tokens from the prompt, so they can be replaced in order of earliest seen + for token in x: + token_order.append((p.prompt.find(token), token)) + + token_order.sort(key=lambda t: t[0]) + + prompt_parts = [] + + # Split the prompt up, taking out the tokens + for _, token in token_order: + n = p.prompt.find(token) + prompt_parts.append(p.prompt[0:n]) + p.prompt = p.prompt[n + len(token):] + + # Rebuild the prompt with the tokens in the order we want + prompt_tmp = "" + for idx, part in enumerate(prompt_parts): + prompt_tmp += part + prompt_tmp += x[idx] + p.prompt = prompt_tmp + p.prompt + + +def apply_sampler(p, x, xs): + sampler_name = sd_samplers.samplers_map.get(x.lower(), None) + if sampler_name is None: + raise RuntimeError(f"Unknown sampler: {x}") + + p.sampler_name = sampler_name + + +def confirm_samplers(p, xs): + for x in xs: + if x.lower() not in sd_samplers.samplers_map: + raise RuntimeError(f"Unknown sampler: {x}") + + +def apply_checkpoint(p, x, xs): + info = modules.sd_models.get_closet_checkpoint_match(x) + if info is None: + raise RuntimeError(f"Unknown checkpoint: {x}") + modules.sd_models.reload_model_weights(shared.sd_model, info) + + +def confirm_checkpoints(p, xs): + for x in xs: + if modules.sd_models.get_closet_checkpoint_match(x) is None: + raise RuntimeError(f"Unknown checkpoint: {x}") + + +def apply_clip_skip(p, x, xs): + opts.data["CLIP_stop_at_last_layers"] = x + + +def apply_upscale_latent_space(p, x, xs): + if x.lower().strip() != '0': + opts.data["use_scale_latent_for_hires_fix"] = True + else: + opts.data["use_scale_latent_for_hires_fix"] = False + + +def find_vae(name: str): + if name.lower() in ['auto', 'automatic']: + return modules.sd_vae.unspecified + if name.lower() == 'none': + return None + else: + choices = [x for x in sorted(modules.sd_vae.vae_dict, key=lambda x: len(x)) if name.lower().strip() in x.lower()] + if len(choices) == 0: + print(f"No VAE found for {name}; using automatic") + return modules.sd_vae.unspecified + else: + return modules.sd_vae.vae_dict[choices[0]] + + +def apply_vae(p, x, xs): + modules.sd_vae.reload_vae_weights(shared.sd_model, vae_file=find_vae(x)) + + +def apply_styles(p: StableDiffusionProcessingTxt2Img, x: str, _): + p.styles.extend(x.split(',')) + + +def format_value_add_label(p, opt, x): + if type(x) == float: + x = round(x, 8) + + return f"{opt.label}: {x}" + + +def format_value(p, opt, x): + if type(x) == float: + x = round(x, 8) + return x + + +def format_value_join_list(p, opt, x): + return ", ".join(x) + + +def do_nothing(p, x, xs): + pass + + +def format_nothing(p, opt, x): + return "" + + +def str_permutations(x): + """dummy function for specifying it in AxisOption's type when you want to get a list of permutations""" + return x + + +class AxisOption: + def __init__(self, label, type, apply, format_value=format_value_add_label, confirm=None, cost=0.0, choices=None): + self.label = label + self.type = type + self.apply = apply + self.format_value = format_value + self.confirm = confirm + self.cost = cost + self.choices = choices + + +class AxisOptionImg2Img(AxisOption): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.is_img2img = True + +class AxisOptionTxt2Img(AxisOption): + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.is_img2img = False + + +axis_options = [ + AxisOption("Nothing", str, do_nothing, format_value=format_nothing), + AxisOption("Seed", int, apply_field("seed")), + AxisOption("Var. seed", int, apply_field("subseed")), + AxisOption("Var. strength", float, apply_field("subseed_strength")), + AxisOption("Steps", int, apply_field("steps")), + AxisOptionTxt2Img("Hires steps", int, apply_field("hr_second_pass_steps")), + AxisOption("CFG Scale", float, apply_field("cfg_scale")), + AxisOptionImg2Img("Image CFG Scale", float, apply_field("image_cfg_scale")), + AxisOption("Prompt S/R", str, apply_prompt, format_value=format_value), + AxisOption("Prompt order", str_permutations, apply_order, format_value=format_value_join_list), + AxisOptionTxt2Img("Sampler", str, apply_sampler, format_value=format_value, confirm=confirm_samplers, choices=lambda: [x.name for x in sd_samplers.samplers]), + AxisOptionImg2Img("Sampler", str, apply_sampler, format_value=format_value, confirm=confirm_samplers, choices=lambda: [x.name for x in sd_samplers.samplers_for_img2img]), + AxisOption("Checkpoint name", str, apply_checkpoint, format_value=format_value, confirm=confirm_checkpoints, cost=1.0, choices=lambda: list(sd_models.checkpoints_list)), + AxisOption("Sigma Churn", float, apply_field("s_churn")), + AxisOption("Sigma min", float, apply_field("s_tmin")), + AxisOption("Sigma max", float, apply_field("s_tmax")), + AxisOption("Sigma noise", float, apply_field("s_noise")), + AxisOption("Eta", float, apply_field("eta")), + AxisOption("Clip skip", int, apply_clip_skip), + AxisOption("Denoising", float, apply_field("denoising_strength")), + AxisOptionTxt2Img("Hires upscaler", str, apply_field("hr_upscaler"), choices=lambda: [*shared.latent_upscale_modes, *[x.name for x in shared.sd_upscalers]]), + AxisOptionImg2Img("Cond. Image Mask Weight", float, apply_field("inpainting_mask_weight")), + AxisOption("VAE", str, apply_vae, cost=0.7, choices=lambda: list(sd_vae.vae_dict)), + AxisOption("Styles", str, apply_styles, choices=lambda: list(shared.prompt_styles.styles)), +] + + +def draw_xyz_grid(p, xs, ys, zs, x_labels, y_labels, z_labels, cell, draw_legend, include_lone_images, include_sub_grids, first_axes_processed, second_axes_processed, margin_size): + hor_texts = [[images.GridAnnotation(x)] for x in x_labels] + ver_texts = [[images.GridAnnotation(y)] for y in y_labels] + title_texts = [[images.GridAnnotation(z)] for z in z_labels] + + # Temporary list of all the images that are generated to be populated into the grid. + # Will be filled with empty images for any individual step that fails to process properly + image_cache = [None] * (len(xs) * len(ys) * len(zs)) + + processed_result = None + cell_mode = "P" + cell_size = (1, 1) + + state.job_count = len(xs) * len(ys) * len(zs) * p.n_iter + + def process_cell(x, y, z, ix, iy, iz): + nonlocal image_cache, processed_result, cell_mode, cell_size + + def index(ix, iy, iz): + return ix + iy * len(xs) + iz * len(xs) * len(ys) + + state.job = f"{index(ix, iy, iz) + 1} out of {len(xs) * len(ys) * len(zs)}" + + processed: Processed = cell(x, y, z) + + try: + # this dereference will throw an exception if the image was not processed + # (this happens in cases such as if the user stops the process from the UI) + processed_image = processed.images[0] + + if processed_result is None: + # Use our first valid processed result as a template container to hold our full results + processed_result = copy(processed) + cell_mode = processed_image.mode + cell_size = processed_image.size + processed_result.images = [Image.new(cell_mode, cell_size)] + processed_result.all_prompts = [processed.prompt] + processed_result.all_seeds = [processed.seed] + processed_result.infotexts = [processed.infotexts[0]] + + image_cache[index(ix, iy, iz)] = processed_image + if include_lone_images: + processed_result.images.append(processed_image) + processed_result.all_prompts.append(processed.prompt) + processed_result.all_seeds.append(processed.seed) + processed_result.infotexts.append(processed.infotexts[0]) + except: + image_cache[index(ix, iy, iz)] = Image.new(cell_mode, cell_size) + + if first_axes_processed == 'x': + for ix, x in enumerate(xs): + if second_axes_processed == 'y': + for iy, y in enumerate(ys): + for iz, z in enumerate(zs): + process_cell(x, y, z, ix, iy, iz) + else: + for iz, z in enumerate(zs): + for iy, y in enumerate(ys): + process_cell(x, y, z, ix, iy, iz) + elif first_axes_processed == 'y': + for iy, y in enumerate(ys): + if second_axes_processed == 'x': + for ix, x in enumerate(xs): + for iz, z in enumerate(zs): + process_cell(x, y, z, ix, iy, iz) + else: + for iz, z in enumerate(zs): + for ix, x in enumerate(xs): + process_cell(x, y, z, ix, iy, iz) + elif first_axes_processed == 'z': + for iz, z in enumerate(zs): + if second_axes_processed == 'x': + for ix, x in enumerate(xs): + for iy, y in enumerate(ys): + process_cell(x, y, z, ix, iy, iz) + else: + for iy, y in enumerate(ys): + for ix, x in enumerate(xs): + process_cell(x, y, z, ix, iy, iz) + + if not processed_result: + print("Unexpected error: draw_xyz_grid failed to return even a single processed image") + return Processed(p, []) + + sub_grids = [None] * len(zs) + for i in range(len(zs)): + start_index = i * len(xs) * len(ys) + end_index = start_index + len(xs) * len(ys) + grid = images.image_grid(image_cache[start_index:end_index], rows=len(ys)) + if draw_legend: + grid = images.draw_grid_annotations(grid, cell_size[0], cell_size[1], hor_texts, ver_texts, margin_size) + sub_grids[i] = grid + if include_sub_grids and len(zs) > 1: + processed_result.images.insert(i+1, grid) + + sub_grid_size = sub_grids[0].size + z_grid = images.image_grid(sub_grids, rows=1) + if draw_legend: + z_grid = images.draw_grid_annotations(z_grid, sub_grid_size[0], sub_grid_size[1], title_texts, [[images.GridAnnotation()]]) + processed_result.images[0] = z_grid + + return processed_result, sub_grids + + +class SharedSettingsStackHelper(object): + def __enter__(self): + self.CLIP_stop_at_last_layers = opts.CLIP_stop_at_last_layers + self.vae = opts.sd_vae + + def __exit__(self, exc_type, exc_value, tb): + opts.data["sd_vae"] = self.vae + modules.sd_models.reload_model_weights() + modules.sd_vae.reload_vae_weights() + + opts.data["CLIP_stop_at_last_layers"] = self.CLIP_stop_at_last_layers + + +re_range = re.compile(r"\s*([+-]?\s*\d+)\s*-\s*([+-]?\s*\d+)(?:\s*\(([+-]\d+)\s*\))?\s*") +re_range_float = re.compile(r"\s*([+-]?\s*\d+(?:.\d*)?)\s*-\s*([+-]?\s*\d+(?:.\d*)?)(?:\s*\(([+-]\d+(?:.\d*)?)\s*\))?\s*") + +re_range_count = re.compile(r"\s*([+-]?\s*\d+)\s*-\s*([+-]?\s*\d+)(?:\s*\[(\d+)\s*\])?\s*") +re_range_count_float = re.compile(r"\s*([+-]?\s*\d+(?:.\d*)?)\s*-\s*([+-]?\s*\d+(?:.\d*)?)(?:\s*\[(\d+(?:.\d*)?)\s*\])?\s*") + + +class Script(scripts.Script): + def title(self): + return "X/Y/Z plot" + + def ui(self, is_img2img): + self.current_axis_options = [x for x in axis_options if type(x) == AxisOption or x.is_img2img == is_img2img] + + with gr.Row(): + with gr.Column(scale=19): + with gr.Row(): + x_type = gr.Dropdown(label="X type", choices=[x.label for x in self.current_axis_options], value=self.current_axis_options[1].label, type="index", elem_id=self.elem_id("x_type")) + x_values = gr.Textbox(label="X values", lines=1, elem_id=self.elem_id("x_values")) + fill_x_button = ToolButton(value=fill_values_symbol, elem_id="xyz_grid_fill_x_tool_button", visible=False) + + with gr.Row(): + y_type = gr.Dropdown(label="Y type", choices=[x.label for x in self.current_axis_options], value=self.current_axis_options[0].label, type="index", elem_id=self.elem_id("y_type")) + y_values = gr.Textbox(label="Y values", lines=1, elem_id=self.elem_id("y_values")) + fill_y_button = ToolButton(value=fill_values_symbol, elem_id="xyz_grid_fill_y_tool_button", visible=False) + + with gr.Row(): + z_type = gr.Dropdown(label="Z type", choices=[x.label for x in self.current_axis_options], value=self.current_axis_options[0].label, type="index", elem_id=self.elem_id("z_type")) + z_values = gr.Textbox(label="Z values", lines=1, elem_id=self.elem_id("z_values")) + fill_z_button = ToolButton(value=fill_values_symbol, elem_id="xyz_grid_fill_z_tool_button", visible=False) + + with gr.Row(variant="compact", elem_id="axis_options"): + with gr.Column(): + draw_legend = gr.Checkbox(label='Draw legend', value=True, elem_id=self.elem_id("draw_legend")) + no_fixed_seeds = gr.Checkbox(label='Keep -1 for seeds', value=False, elem_id=self.elem_id("no_fixed_seeds")) + with gr.Column(): + include_lone_images = gr.Checkbox(label='Include Sub Images', value=False, elem_id=self.elem_id("include_lone_images")) + include_sub_grids = gr.Checkbox(label='Include Sub Grids', value=False, elem_id=self.elem_id("include_sub_grids")) + with gr.Column(): + margin_size = gr.Slider(label="Grid margins (px)", minimum=0, maximum=500, value=0, step=2, elem_id=self.elem_id("margin_size")) + + with gr.Row(variant="compact", elem_id="swap_axes"): + swap_xy_axes_button = gr.Button(value="Swap X/Y axes", elem_id="xy_grid_swap_axes_button") + swap_yz_axes_button = gr.Button(value="Swap Y/Z axes", elem_id="yz_grid_swap_axes_button") + swap_xz_axes_button = gr.Button(value="Swap X/Z axes", elem_id="xz_grid_swap_axes_button") + + def swap_axes(axis1_type, axis1_values, axis2_type, axis2_values): + return self.current_axis_options[axis2_type].label, axis2_values, self.current_axis_options[axis1_type].label, axis1_values + + xy_swap_args = [x_type, x_values, y_type, y_values] + swap_xy_axes_button.click(swap_axes, inputs=xy_swap_args, outputs=xy_swap_args) + yz_swap_args = [y_type, y_values, z_type, z_values] + swap_yz_axes_button.click(swap_axes, inputs=yz_swap_args, outputs=yz_swap_args) + xz_swap_args = [x_type, x_values, z_type, z_values] + swap_xz_axes_button.click(swap_axes, inputs=xz_swap_args, outputs=xz_swap_args) + + def fill(x_type): + axis = self.current_axis_options[x_type] + return ", ".join(axis.choices()) if axis.choices else gr.update() + + fill_x_button.click(fn=fill, inputs=[x_type], outputs=[x_values]) + fill_y_button.click(fn=fill, inputs=[y_type], outputs=[y_values]) + fill_z_button.click(fn=fill, inputs=[z_type], outputs=[z_values]) + + def select_axis(x_type): + return gr.Button.update(visible=self.current_axis_options[x_type].choices is not None) + + x_type.change(fn=select_axis, inputs=[x_type], outputs=[fill_x_button]) + y_type.change(fn=select_axis, inputs=[y_type], outputs=[fill_y_button]) + z_type.change(fn=select_axis, inputs=[z_type], outputs=[fill_z_button]) + + self.infotext_fields = ( + (x_type, "X Type"), + (x_values, "X Values"), + (y_type, "Y Type"), + (y_values, "Y Values"), + (z_type, "Z Type"), + (z_values, "Z Values"), + ) + + return [x_type, x_values, y_type, y_values, z_type, z_values, draw_legend, include_lone_images, include_sub_grids, no_fixed_seeds, margin_size] + + def run(self, p, x_type, x_values, y_type, y_values, z_type, z_values, draw_legend, include_lone_images, include_sub_grids, no_fixed_seeds, margin_size): + if not no_fixed_seeds: + modules.processing.fix_seed(p) + + if not opts.return_grid: + p.batch_size = 1 + + def process_axis(opt, vals): + if opt.label == 'Nothing': + return [0] + + valslist = [x.strip() for x in chain.from_iterable(csv.reader(StringIO(vals)))] + + if opt.type == int: + valslist_ext = [] + + for val in valslist: + m = re_range.fullmatch(val) + mc = re_range_count.fullmatch(val) + if m is not None: + start = int(m.group(1)) + end = int(m.group(2))+1 + step = int(m.group(3)) if m.group(3) is not None else 1 + + valslist_ext += list(range(start, end, step)) + elif mc is not None: + start = int(mc.group(1)) + end = int(mc.group(2)) + num = int(mc.group(3)) if mc.group(3) is not None else 1 + + valslist_ext += [int(x) for x in np.linspace(start=start, stop=end, num=num).tolist()] + else: + valslist_ext.append(val) + + valslist = valslist_ext + elif opt.type == float: + valslist_ext = [] + + for val in valslist: + m = re_range_float.fullmatch(val) + mc = re_range_count_float.fullmatch(val) + if m is not None: + start = float(m.group(1)) + end = float(m.group(2)) + step = float(m.group(3)) if m.group(3) is not None else 1 + + valslist_ext += np.arange(start, end + step, step).tolist() + elif mc is not None: + start = float(mc.group(1)) + end = float(mc.group(2)) + num = int(mc.group(3)) if mc.group(3) is not None else 1 + + valslist_ext += np.linspace(start=start, stop=end, num=num).tolist() + else: + valslist_ext.append(val) + + valslist = valslist_ext + elif opt.type == str_permutations: + valslist = list(permutations(valslist)) + + valslist = [opt.type(x) for x in valslist] + + # Confirm options are valid before starting + if opt.confirm: + opt.confirm(p, valslist) + + return valslist + + x_opt = self.current_axis_options[x_type] + xs = process_axis(x_opt, x_values) + + y_opt = self.current_axis_options[y_type] + ys = process_axis(y_opt, y_values) + + z_opt = self.current_axis_options[z_type] + zs = process_axis(z_opt, z_values) + + def fix_axis_seeds(axis_opt, axis_list): + if axis_opt.label in ['Seed', 'Var. seed']: + return [int(random.randrange(4294967294)) if val is None or val == '' or val == -1 else val for val in axis_list] + else: + return axis_list + + if not no_fixed_seeds: + xs = fix_axis_seeds(x_opt, xs) + ys = fix_axis_seeds(y_opt, ys) + zs = fix_axis_seeds(z_opt, zs) + + if x_opt.label == 'Steps': + total_steps = sum(xs) * len(ys) * len(zs) + elif y_opt.label == 'Steps': + total_steps = sum(ys) * len(xs) * len(zs) + elif z_opt.label == 'Steps': + total_steps = sum(zs) * len(xs) * len(ys) + else: + total_steps = p.steps * len(xs) * len(ys) * len(zs) + + if isinstance(p, StableDiffusionProcessingTxt2Img) and p.enable_hr: + if x_opt.label == "Hires steps": + total_steps += sum(xs) * len(ys) * len(zs) + elif y_opt.label == "Hires steps": + total_steps += sum(ys) * len(xs) * len(zs) + elif z_opt.label == "Hires steps": + total_steps += sum(zs) * len(xs) * len(ys) + elif p.hr_second_pass_steps: + total_steps += p.hr_second_pass_steps * len(xs) * len(ys) * len(zs) + else: + total_steps *= 2 + + total_steps *= p.n_iter + + image_cell_count = p.n_iter * p.batch_size + cell_console_text = f"; {image_cell_count} images per cell" if image_cell_count > 1 else "" + plural_s = 's' if len(zs) > 1 else '' + print(f"X/Y/Z plot will create {len(xs) * len(ys) * len(zs) * image_cell_count} images on {len(zs)} {len(xs)}x{len(ys)} grid{plural_s}{cell_console_text}. (Total steps to process: {total_steps})") + shared.total_tqdm.updateTotal(total_steps) + + grid_infotext = [None] + + state.xyz_plot_x = AxisInfo(x_opt, xs) + state.xyz_plot_y = AxisInfo(y_opt, ys) + state.xyz_plot_z = AxisInfo(z_opt, zs) + + # If one of the axes is very slow to change between (like SD model + # checkpoint), then make sure it is in the outer iteration of the nested + # `for` loop. + first_axes_processed = 'x' + second_axes_processed = 'y' + if x_opt.cost > y_opt.cost and x_opt.cost > z_opt.cost: + first_axes_processed = 'x' + if y_opt.cost > z_opt.cost: + second_axes_processed = 'y' + else: + second_axes_processed = 'z' + elif y_opt.cost > x_opt.cost and y_opt.cost > z_opt.cost: + first_axes_processed = 'y' + if x_opt.cost > z_opt.cost: + second_axes_processed = 'x' + else: + second_axes_processed = 'z' + elif z_opt.cost > x_opt.cost and z_opt.cost > y_opt.cost: + first_axes_processed = 'z' + if x_opt.cost > y_opt.cost: + second_axes_processed = 'x' + else: + second_axes_processed = 'y' + + def cell(x, y, z): + if shared.state.interrupted: + return Processed(p, [], p.seed, "") + + pc = copy(p) + pc.styles = pc.styles[:] + x_opt.apply(pc, x, xs) + y_opt.apply(pc, y, ys) + z_opt.apply(pc, z, zs) + + res = process_images(pc) + + if grid_infotext[0] is None: + pc.extra_generation_params = copy(pc.extra_generation_params) + pc.extra_generation_params['Script'] = self.title() + + if x_opt.label != 'Nothing': + pc.extra_generation_params["X Type"] = x_opt.label + pc.extra_generation_params["X Values"] = x_values + if x_opt.label in ["Seed", "Var. seed"] and not no_fixed_seeds: + pc.extra_generation_params["Fixed X Values"] = ", ".join([str(x) for x in xs]) + + if y_opt.label != 'Nothing': + pc.extra_generation_params["Y Type"] = y_opt.label + pc.extra_generation_params["Y Values"] = y_values + if y_opt.label in ["Seed", "Var. seed"] and not no_fixed_seeds: + pc.extra_generation_params["Fixed Y Values"] = ", ".join([str(y) for y in ys]) + + if z_opt.label != 'Nothing': + pc.extra_generation_params["Z Type"] = z_opt.label + pc.extra_generation_params["Z Values"] = z_values + if z_opt.label in ["Seed", "Var. seed"] and not no_fixed_seeds: + pc.extra_generation_params["Fixed Z Values"] = ", ".join([str(z) for z in zs]) + + grid_infotext[0] = processing.create_infotext(pc, pc.all_prompts, pc.all_seeds, pc.all_subseeds) + + return res + + with SharedSettingsStackHelper(): + processed, sub_grids = draw_xyz_grid( + p, + xs=xs, + ys=ys, + zs=zs, + x_labels=[x_opt.format_value(p, x_opt, x) for x in xs], + y_labels=[y_opt.format_value(p, y_opt, y) for y in ys], + z_labels=[z_opt.format_value(p, z_opt, z) for z in zs], + cell=cell, + draw_legend=draw_legend, + include_lone_images=include_lone_images, + include_sub_grids=include_sub_grids, + first_axes_processed=first_axes_processed, + second_axes_processed=second_axes_processed, + margin_size=margin_size + ) + + if opts.grid_save and len(sub_grids) > 1: + for sub_grid in sub_grids: + images.save_image(sub_grid, p.outpath_grids, "xyz_grid", info=grid_infotext[0], extension=opts.grid_format, prompt=p.prompt, seed=processed.seed, grid=True, p=p) + + if opts.grid_save: + images.save_image(processed.images[0], p.outpath_grids, "xyz_grid", info=grid_infotext[0], extension=opts.grid_format, prompt=p.prompt, seed=processed.seed, grid=True, p=p) + + return processed diff --git a/sd/stable-diffusion-webui/style.css b/sd/stable-diffusion-webui/style.css new file mode 100644 index 0000000000000000000000000000000000000000..8f58bc9d0cba1f6ec371f211f531530245bf367e --- /dev/null +++ b/sd/stable-diffusion-webui/style.css @@ -0,0 +1,961 @@ +.container { + max-width: 100%; +} + +.token-counter{ + position: absolute; + display: inline-block; + right: 2em; + min-width: 0 !important; + width: auto; + z-index: 100; +} + +.token-counter.error span{ + box-shadow: 0 0 0.0 0.3em rgba(255,0,0,0.15), inset 0 0 0.6em rgba(255,0,0,0.075); + border: 2px solid rgba(255,0,0,0.4) !important; +} + +.token-counter div{ + display: inline; +} + +.token-counter span{ + padding: 0.1em 0.75em; +} + +#sh{ + min-width: 2em; + min-height: 2em; + max-width: 2em; + max-height: 2em; + flex-grow: 0; + padding-left: 0.25em; + padding-right: 0.25em; + margin: 0.1em 0; + opacity: 0%; + cursor: default; +} + +.output-html p {margin: 0 0.5em;} + +.row > *, +.row > .gr-form > * { + min-width: min(120px, 100%); + flex: 1 1 0%; +} + +.performance { + font-size: 0.85em; + color: #444; +} + +.performance p{ + display: inline-block; +} + +.performance .time { + margin-right: 0; +} + +.performance .vram { +} + +#txt2img_generate, #img2img_generate { + min-height: 4.5em; +} + +@media screen and (min-width: 2500px) { + #txt2img_gallery, #img2img_gallery { + min-height: 768px; + } +} + +#txt2img_gallery img, #img2img_gallery img{ + object-fit: scale-down; +} +#txt2img_actions_column, #img2img_actions_column { + margin: 0.35rem 0.75rem 0.35rem 0; +} +#script_list { + padding: .625rem .75rem 0 .625rem; +} +.justify-center.overflow-x-scroll { + justify-content: left; +} + +.justify-center.overflow-x-scroll button:first-of-type { + margin-left: auto; +} + +.justify-center.overflow-x-scroll button:last-of-type { + margin-right: auto; +} + +[id$=_random_seed], [id$=_random_subseed], [id$=_reuse_seed], [id$=_reuse_subseed], #open_folder{ + min-width: 2.3em; + height: 2.5em; + flex-grow: 0; + padding-left: 0.25em; + padding-right: 0.25em; +} + +#hidden_element{ + display: none; +} + +[id$=_seed_row], [id$=_subseed_row]{ + gap: 0.5rem; + padding: 0.6em; +} + +[id$=_subseed_show_box]{ + min-width: auto; + flex-grow: 0; +} + +[id$=_subseed_show_box] > div{ + border: 0; + height: 100%; +} + +[id$=_subseed_show]{ + min-width: auto; + flex-grow: 0; + padding: 0; +} + +[id$=_subseed_show] label{ + height: 100%; +} + +#txt2img_actions_column, #img2img_actions_column{ + gap: 0; + margin-right: .75rem; +} + +#txt2img_tools, #img2img_tools{ + gap: 0.4em; +} + +#interrogate_col{ + min-width: 0 !important; + max-width: 8em !important; + margin-right: 1em; + gap: 0; +} +#interrogate, #deepbooru{ + margin: 0em 0.25em 0.5em 0.25em; + min-width: 8em; + max-width: 8em; +} + +#style_pos_col, #style_neg_col{ + min-width: 8em !important; +} + +#txt2img_styles_row, #img2img_styles_row{ + gap: 0.25em; + margin-top: 0.3em; +} + +#txt2img_styles_row > button, #img2img_styles_row > button{ + margin: 0; +} + +#txt2img_styles, #img2img_styles{ + padding: 0; +} + +#txt2img_styles > label > div, #img2img_styles > label > div{ + min-height: 3.2em; +} + +ul.list-none{ + max-height: 35em; + z-index: 2000; +} + +.gr-form{ + background: transparent; +} + +.my-4{ + margin-top: 0; + margin-bottom: 0; +} + +#resize_mode{ + flex: 1.5; +} + +button{ + align-self: stretch !important; +} + +.overflow-hidden, .gr-panel{ + overflow: visible !important; +} + +#x_type, #y_type{ + max-width: 10em; +} + +#txt2img_preview, #img2img_preview, #ti_preview{ + position: absolute; + width: 320px; + left: 0; + right: 0; + margin-left: auto; + margin-right: auto; + margin-top: 34px; + z-index: 100; + border: none; + border-top-left-radius: 0; + border-top-right-radius: 0; +} + +@media screen and (min-width: 768px) { + #txt2img_preview, #img2img_preview, #ti_preview { + position: absolute; + } +} + +@media screen and (max-width: 767px) { + #txt2img_preview, #img2img_preview, #ti_preview { + position: relative; + } +} + +#txt2img_preview div.left-0.top-0, #img2img_preview div.left-0.top-0, #ti_preview div.left-0.top-0{ + display: none; +} + +fieldset span.text-gray-500, .gr-block.gr-box span.text-gray-500, label.block span{ + position: absolute; + top: -0.7em; + line-height: 1.2em; + padding: 0; + margin: 0 0.5em; + + background-color: white; + box-shadow: 6px 0 6px 0px white, -6px 0 6px 0px white; + + z-index: 300; +} + +.dark fieldset span.text-gray-500, .dark .gr-block.gr-box span.text-gray-500, .dark label.block span{ + background-color: rgb(31, 41, 55); + box-shadow: none; + border: 1px solid rgba(128, 128, 128, 0.1); + border-radius: 6px; + padding: 0.1em 0.5em; +} + +#txt2img_column_batch, #img2img_column_batch{ + min-width: min(13.5em, 100%) !important; +} + +#settings fieldset span.text-gray-500, #settings .gr-block.gr-box span.text-gray-500, #settings label.block span{ + position: relative; + border: none; + margin-right: 8em; +} + +#settings .gr-panel div.flex-col div.justify-between div{ + position: relative; + z-index: 200; +} + +#settings{ + display: block; +} + +#settings > div{ + border: none; + margin-left: 10em; +} + +#settings > div.flex-wrap{ + float: left; + display: block; + margin-left: 0; + width: 10em; +} + +#settings > div.flex-wrap button{ + display: block; + border: none; + text-align: left; +} + +#settings_result{ + height: 1.4em; + margin: 0 1.2em; +} + +input[type="range"]{ + margin: 0.5em 0 -0.3em 0; +} + +#mask_bug_info { + text-align: center; + display: block; + margin-top: -0.75em; + margin-bottom: -0.75em; +} + +#txt2img_negative_prompt, #img2img_negative_prompt{ +} + +/* gradio 3.8 adds opacity to progressbar which makes it blink; disable it here */ +.transition.opacity-20 { + opacity: 1 !important; +} + +/* more gradio's garbage cleanup */ +.min-h-\[4rem\] { min-height: unset !important; } +.min-h-\[6rem\] { min-height: unset !important; } + +.progressDiv{ + position: relative; + height: 20px; + background: #b4c0cc; + border-radius: 3px !important; + margin-bottom: -3px; +} + +.dark .progressDiv{ + background: #424c5b; +} + +.progressDiv .progress{ + width: 0%; + height: 20px; + background: #0060df; + color: white; + font-weight: bold; + line-height: 20px; + padding: 0 8px 0 0; + text-align: right; + border-radius: 3px; + overflow: visible; + white-space: nowrap; + padding: 0 0.5em; +} + +.livePreview{ + position: absolute; + z-index: 300; + background-color: white; + margin: -4px; +} + +.dark .livePreview{ + background-color: rgb(17 24 39 / var(--tw-bg-opacity)); +} + +.livePreview img{ + position: absolute; + object-fit: contain; + width: 100%; + height: 100%; +} + +#lightboxModal{ + display: none; + position: fixed; + z-index: 1001; + padding-top: 100px; + left: 0; + top: 0; + width: 100%; + height: 100%; + overflow: auto; + background-color: rgba(20, 20, 20, 0.95); + user-select: none; + -webkit-user-select: none; +} + +.modalControls { + display: grid; + grid-template-columns: 32px 32px 32px 1fr 32px; + grid-template-areas: "zoom tile save space close"; + position: absolute; + top: 0; + left: 0; + right: 0; + padding: 16px; + gap: 16px; + background-color: rgba(0,0,0,0.2); +} + +.modalClose { + grid-area: close; +} + +.modalZoom { + grid-area: zoom; +} + +.modalSave { + grid-area: save; +} + +.modalTileImage { + grid-area: tile; +} + +.modalClose, +.modalZoom, +.modalTileImage { + color: white; + font-size: 35px; + font-weight: bold; + cursor: pointer; +} + +.modalSave { + color: white; + font-size: 28px; + margin-top: 8px; + font-weight: bold; + cursor: pointer; +} + +.modalClose:hover, +.modalClose:focus, +.modalSave:hover, +.modalSave:focus, +.modalZoom:hover, +.modalZoom:focus { + color: #999; + text-decoration: none; + cursor: pointer; +} + +#modalImage { + display: block; + margin-left: auto; + margin-right: auto; + margin-top: auto; + width: auto; +} + +.modalImageFullscreen { + object-fit: contain; + height: 90%; +} + +.modalPrev, +.modalNext { + cursor: pointer; + position: absolute; + top: 50%; + width: auto; + padding: 16px; + margin-top: -50px; + color: white; + font-weight: bold; + font-size: 20px; + transition: 0.6s ease; + border-radius: 0 3px 3px 0; + user-select: none; + -webkit-user-select: none; +} + +.modalNext { + right: 0; + border-radius: 3px 0 0 3px; +} + +.modalPrev:hover, +.modalNext:hover { + background-color: rgba(0, 0, 0, 0.8); +} + +#imageARPreview{ + position:absolute; + top:0px; + left:0px; + border:2px solid red; + background:rgba(255, 0, 0, 0.3); + z-index: 900; + pointer-events:none; + display:none +} + +#txt2img_generate_box, #img2img_generate_box{ + position: relative; +} + +#txt2img_interrupt, #img2img_interrupt, #txt2img_skip, #img2img_skip{ + position: absolute; + width: 50%; + height: 100%; + background: #b4c0cc; + display: none; +} + +#txt2img_interrupt, #img2img_interrupt{ + left: 0; + border-radius: 0.5rem 0 0 0.5rem; +} +#txt2img_skip, #img2img_skip{ + right: 0; + border-radius: 0 0.5rem 0.5rem 0; +} + +.red { + color: red; +} + +.gallery-item { + --tw-bg-opacity: 0 !important; +} + +#context-menu{ + z-index:9999; + position:absolute; + display:block; + padding:0px 0; + border:2px solid #a55000; + border-radius:8px; + box-shadow:1px 1px 2px #CE6400; + width: 200px; +} + +.context-menu-items{ + list-style: none; + margin: 0; + padding: 0; +} + +.context-menu-items a{ + display:block; + padding:5px; + cursor:pointer; +} + +.context-menu-items a:hover{ + background: #a55000; +} + +#quicksettings { + width: fit-content; +} + +#quicksettings > div, #quicksettings > fieldset{ + max-width: 24em; + min-width: 24em; + padding: 0; + border: none; + box-shadow: none; + background: none; + margin-right: 10px; +} + +#quicksettings > div > div > div > label > span { + position: relative; + margin-right: 9em; + margin-bottom: -1em; +} + +canvas[key="mask"] { + z-index: 12 !important; + filter: invert(); + mix-blend-mode: multiply; + pointer-events: none; +} + + +/* gradio 3.4.1 stuff for editable scrollbar values */ +.gr-box > div > div > input.gr-text-input{ + position: absolute; + right: 0.5em; + top: -0.6em; + z-index: 400; + width: 6em; +} +#quicksettings .gr-box > div > div > input.gr-text-input { + top: -1.12em; +} + +.row.gr-compact{ + overflow: visible; +} + +#img2img_image, #img2img_image > .h-60, #img2img_image > .h-60 > div, #img2img_image > .h-60 > div > img, +#img2img_sketch, #img2img_sketch > .h-60, #img2img_sketch > .h-60 > div, #img2img_sketch > .h-60 > div > img, +#img2maskimg, #img2maskimg > .h-60, #img2maskimg > .h-60 > div, #img2maskimg > .h-60 > div > img, +#inpaint_sketch, #inpaint_sketch > .h-60, #inpaint_sketch > .h-60 > div, #inpaint_sketch > .h-60 > div > img +{ + height: 480px !important; + max-height: 480px !important; + min-height: 480px !important; +} + +/* Extensions */ + +#tab_extensions table{ + border-collapse: collapse; +} + +#tab_extensions table td, #tab_extensions table th{ + border: 1px solid #ccc; + padding: 0.25em 0.5em; +} + +#tab_extensions table input[type="checkbox"]{ + margin-right: 0.5em; +} + +#tab_extensions button{ + max-width: 16em; +} + +#tab_extensions input[disabled="disabled"]{ + opacity: 0.5; +} + +.extension-tag{ + font-weight: bold; + font-size: 95%; +} + +#available_extensions .info{ + margin: 0; +} + +#available_extensions .date_added{ + opacity: 0.85; + font-size: 90%; +} + +#image_buttons_txt2img button, #image_buttons_img2img button, #image_buttons_extras button{ + min-width: auto; + padding-left: 0.5em; + padding-right: 0.5em; +} + +.gr-form{ + background-color: white; +} + +.dark .gr-form{ + background-color: rgb(31 41 55 / var(--tw-bg-opacity)); +} + +.gr-button-tool, .gr-button-tool-top{ + max-width: 2.5em; + min-width: 2.5em !important; + height: 2.4em; +} + +.gr-button-tool{ + margin: 0.6em 0em 0.55em 0; +} + +.gr-button-tool-top, #settings .gr-button-tool{ + margin: 1.6em 0.7em 0.55em 0; +} + + +#modelmerger_results_container{ + margin-top: 1em; + overflow: visible; +} + +#modelmerger_models{ + gap: 0; +} + + +#quicksettings .gr-button-tool{ + margin: 0; + border-color: unset; + background-color: unset; +} + +#modelmerger_interp_description>p { + margin: 0!important; + text-align: center; +} +#modelmerger_interp_description { + margin: 0.35rem 0.75rem 1.23rem; +} +#img2img_settings > div.gr-form, #txt2img_settings > div.gr-form { + padding-top: 0.9em; + padding-bottom: 0.9em; +} +#txt2img_settings { + padding-top: 1.16em; + padding-bottom: 0.9em; +} +#img2img_settings { + padding-bottom: 0.9em; +} + +#img2img_settings div.gr-form .gr-form, #txt2img_settings div.gr-form .gr-form, #train_tabs div.gr-form .gr-form{ + border: none; + padding-bottom: 0.5em; +} + +footer { + display: none !important; +} + +#footer{ + text-align: center; +} + +#footer div{ + display: inline-block; +} + +#footer .versions{ + font-size: 85%; + opacity: 0.85; +} + +#txtimg_hr_finalres{ + min-height: 0 !important; + padding: .625rem .75rem; + margin-left: -0.75em + +} + +#txtimg_hr_finalres .resolution{ + font-weight: bold; +} + +#txt2img_checkboxes, #img2img_checkboxes{ + margin-bottom: 0.5em; + margin-left: 0em; +} +#txt2img_checkboxes > div, #img2img_checkboxes > div{ + flex: 0; + white-space: nowrap; + min-width: auto; +} + +#img2img_copy_to_img2img, #img2img_copy_to_sketch, #img2img_copy_to_inpaint, #img2img_copy_to_inpaint_sketch{ + margin-left: 0em; +} + +#axis_options { + margin-left: 0em; +} + +.inactive{ + opacity: 0.5; +} + +[id*='_prompt_container']{ + gap: 0; +} + +[id*='_prompt_container'] > div{ + margin: -0.4em 0 0 0; +} + +.gr-compact { + border: none; +} + +.dark .gr-compact{ + background-color: rgb(31 41 55 / var(--tw-bg-opacity)); + margin-left: 0; +} + +.gr-compact{ + overflow: visible; +} + +.gr-compact > *{ +} + +.gr-compact .gr-block, .gr-compact .gr-form{ + border: none; + box-shadow: none; +} + +.gr-compact .gr-box{ + border-radius: .5rem !important; + border-width: 1px !important; +} + +#mode_img2img > div > div{ + gap: 0 !important; +} + +[id*='img2img_copy_to_'] { + border: none; +} + +[id*='img2img_copy_to_'] > button { +} + +[id*='img2img_label_copy_to_'] { + font-size: 1.0em; + font-weight: bold; + text-align: center; + line-height: 2.4em; +} + +.extra-networks > div > [id *= '_extra_']{ + margin: 0.3em; +} + +.extra-network-subdirs{ + padding: 0.2em 0.35em; +} + +.extra-network-subdirs button{ + margin: 0 0.15em; +} + +#txt2img_extra_networks .search, #img2img_extra_networks .search{ + display: inline-block; + max-width: 16em; + margin: 0.3em; + align-self: center; +} + +#txt2img_extra_view, #img2img_extra_view { + width: auto; +} + +.extra-network-cards .nocards, .extra-network-thumbs .nocards{ + margin: 1.25em 0.5em 0.5em 0.5em; +} + +.extra-network-cards .nocards h1, .extra-network-thumbs .nocards h1{ + font-size: 1.5em; + margin-bottom: 1em; +} + +.extra-network-cards .nocards li, .extra-network-thumbs .nocards li{ + margin-left: 0.5em; +} + +.extra-network-thumbs { + display: flex; + flex-flow: row wrap; + gap: 10px; +} + +.extra-network-thumbs .card { + height: 6em; + width: 6em; + cursor: pointer; + background-image: url('./file=html/card-no-preview.png'); + background-size: cover; + background-position: center center; + position: relative; +} + +.extra-network-thumbs .card:hover .additional a { + display: block; +} + +.extra-network-thumbs .actions .additional a { + background-image: url('./file=html/image-update.svg'); + background-repeat: no-repeat; + background-size: cover; + background-position: center center; + position: absolute; + top: 0; + left: 0; + width: 24px; + height: 24px; + display: none; + font-size: 0; + text-align: -9999; +} + +.extra-network-thumbs .actions .name { + position: absolute; + bottom: 0; + font-size: 10px; + padding: 3px; + width: 100%; + overflow: hidden; + white-space: nowrap; + text-overflow: ellipsis; + background: rgba(0,0,0,.5); + color: white; +} + +.extra-network-thumbs .card:hover .actions .name { + white-space: normal; + word-break: break-all; +} + +.extra-network-cards .card{ + display: inline-block; + margin: 0.5em; + width: 16em; + height: 24em; + box-shadow: 0 0 5px rgba(128, 128, 128, 0.5); + border-radius: 0.2em; + position: relative; + + background-size: auto 100%; + background-position: center; + overflow: hidden; + cursor: pointer; + + background-image: url('./file=html/card-no-preview.png') +} + +.extra-network-cards .card:hover{ + box-shadow: 0 0 2px 0.3em rgba(0, 128, 255, 0.35); +} + +.extra-network-cards .card .actions .additional{ + display: none; +} + +.extra-network-cards .card .actions{ + position: absolute; + bottom: 0; + left: 0; + right: 0; + padding: 0.5em; + color: white; + background: rgba(0,0,0,0.5); + box-shadow: 0 0 0.25em 0.25em rgba(0,0,0,0.5); + text-shadow: 0 0 0.2em black; +} + +.extra-network-cards .card .actions:hover{ + box-shadow: 0 0 0.75em 0.75em rgba(0,0,0,0.5) !important; +} + +.extra-network-cards .card .actions .name{ + font-size: 1.7em; + font-weight: bold; + line-break: anywhere; +} + +.extra-network-cards .card .actions:hover .additional{ + display: block; +} + +.extra-network-cards .card ul{ + margin: 0.25em 0 0.75em 0.25em; + cursor: unset; +} + +.extra-network-cards .card ul a{ + cursor: pointer; +} + +.extra-network-cards .card ul a:hover{ + color: red; +} + +[id*='_prompt_container'] > div { + margin: 0!important; +} diff --git a/sd/stable-diffusion-webui/test/__init__.py b/sd/stable-diffusion-webui/test/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/sd/stable-diffusion-webui/test/basic_features/__init__.py b/sd/stable-diffusion-webui/test/basic_features/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/sd/stable-diffusion-webui/test/basic_features/extras_test.py b/sd/stable-diffusion-webui/test/basic_features/extras_test.py new file mode 100644 index 0000000000000000000000000000000000000000..0170c511fe54cc6bcf49ec7f75ca7c747de41db5 --- /dev/null +++ b/sd/stable-diffusion-webui/test/basic_features/extras_test.py @@ -0,0 +1,54 @@ +import unittest +import requests +from gradio.processing_utils import encode_pil_to_base64 +from PIL import Image + +class TestExtrasWorking(unittest.TestCase): + def setUp(self): + self.url_extras_single = "http://localhost:7860/sdapi/v1/extra-single-image" + self.extras_single = { + "resize_mode": 0, + "show_extras_results": True, + "gfpgan_visibility": 0, + "codeformer_visibility": 0, + "codeformer_weight": 0, + "upscaling_resize": 2, + "upscaling_resize_w": 128, + "upscaling_resize_h": 128, + "upscaling_crop": True, + "upscaler_1": "None", + "upscaler_2": "None", + "extras_upscaler_2_visibility": 0, + "image": encode_pil_to_base64(Image.open(r"test/test_files/img2img_basic.png")) + } + + def test_simple_upscaling_performed(self): + self.extras_single["upscaler_1"] = "Lanczos" + self.assertEqual(requests.post(self.url_extras_single, json=self.extras_single).status_code, 200) + + +class TestPngInfoWorking(unittest.TestCase): + def setUp(self): + self.url_png_info = "http://localhost:7860/sdapi/v1/extra-single-image" + self.png_info = { + "image": encode_pil_to_base64(Image.open(r"test/test_files/img2img_basic.png")) + } + + def test_png_info_performed(self): + self.assertEqual(requests.post(self.url_png_info, json=self.png_info).status_code, 200) + + +class TestInterrogateWorking(unittest.TestCase): + def setUp(self): + self.url_interrogate = "http://localhost:7860/sdapi/v1/extra-single-image" + self.interrogate = { + "image": encode_pil_to_base64(Image.open(r"test/test_files/img2img_basic.png")), + "model": "clip" + } + + def test_interrogate_performed(self): + self.assertEqual(requests.post(self.url_interrogate, json=self.interrogate).status_code, 200) + + +if __name__ == "__main__": + unittest.main() diff --git a/sd/stable-diffusion-webui/test/basic_features/img2img_test.py b/sd/stable-diffusion-webui/test/basic_features/img2img_test.py new file mode 100644 index 0000000000000000000000000000000000000000..08c5c903e8382ef4b969b01da87bc69fb06ff2b4 --- /dev/null +++ b/sd/stable-diffusion-webui/test/basic_features/img2img_test.py @@ -0,0 +1,66 @@ +import unittest +import requests +from gradio.processing_utils import encode_pil_to_base64 +from PIL import Image + + +class TestImg2ImgWorking(unittest.TestCase): + def setUp(self): + self.url_img2img = "http://localhost:7860/sdapi/v1/img2img" + self.simple_img2img = { + "init_images": [encode_pil_to_base64(Image.open(r"test/test_files/img2img_basic.png"))], + "resize_mode": 0, + "denoising_strength": 0.75, + "mask": None, + "mask_blur": 4, + "inpainting_fill": 0, + "inpaint_full_res": False, + "inpaint_full_res_padding": 0, + "inpainting_mask_invert": False, + "prompt": "example prompt", + "styles": [], + "seed": -1, + "subseed": -1, + "subseed_strength": 0, + "seed_resize_from_h": -1, + "seed_resize_from_w": -1, + "batch_size": 1, + "n_iter": 1, + "steps": 3, + "cfg_scale": 7, + "width": 64, + "height": 64, + "restore_faces": False, + "tiling": False, + "negative_prompt": "", + "eta": 0, + "s_churn": 0, + "s_tmax": 0, + "s_tmin": 0, + "s_noise": 1, + "override_settings": {}, + "sampler_index": "Euler a", + "include_init_images": False + } + + def test_img2img_simple_performed(self): + self.assertEqual(requests.post(self.url_img2img, json=self.simple_img2img).status_code, 200) + + def test_inpainting_masked_performed(self): + self.simple_img2img["mask"] = encode_pil_to_base64(Image.open(r"test/test_files/mask_basic.png")) + self.assertEqual(requests.post(self.url_img2img, json=self.simple_img2img).status_code, 200) + + def test_inpainting_with_inverted_masked_performed(self): + self.simple_img2img["mask"] = encode_pil_to_base64(Image.open(r"test/test_files/mask_basic.png")) + self.simple_img2img["inpainting_mask_invert"] = True + self.assertEqual(requests.post(self.url_img2img, json=self.simple_img2img).status_code, 200) + + def test_img2img_sd_upscale_performed(self): + self.simple_img2img["script_name"] = "sd upscale" + self.simple_img2img["script_args"] = ["", 8, "Lanczos", 2.0] + + self.assertEqual(requests.post(self.url_img2img, json=self.simple_img2img).status_code, 200) + + +if __name__ == "__main__": + unittest.main() diff --git a/sd/stable-diffusion-webui/test/basic_features/txt2img_test.py b/sd/stable-diffusion-webui/test/basic_features/txt2img_test.py new file mode 100644 index 0000000000000000000000000000000000000000..5aa43a44a3e3818d7220a98acf5a6a504cdca3e3 --- /dev/null +++ b/sd/stable-diffusion-webui/test/basic_features/txt2img_test.py @@ -0,0 +1,80 @@ +import unittest +import requests + + +class TestTxt2ImgWorking(unittest.TestCase): + def setUp(self): + self.url_txt2img = "http://localhost:7860/sdapi/v1/txt2img" + self.simple_txt2img = { + "enable_hr": False, + "denoising_strength": 0, + "firstphase_width": 0, + "firstphase_height": 0, + "prompt": "example prompt", + "styles": [], + "seed": -1, + "subseed": -1, + "subseed_strength": 0, + "seed_resize_from_h": -1, + "seed_resize_from_w": -1, + "batch_size": 1, + "n_iter": 1, + "steps": 3, + "cfg_scale": 7, + "width": 64, + "height": 64, + "restore_faces": False, + "tiling": False, + "negative_prompt": "", + "eta": 0, + "s_churn": 0, + "s_tmax": 0, + "s_tmin": 0, + "s_noise": 1, + "sampler_index": "Euler a" + } + + def test_txt2img_simple_performed(self): + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_with_negative_prompt_performed(self): + self.simple_txt2img["negative_prompt"] = "example negative prompt" + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_with_complex_prompt_performed(self): + self.simple_txt2img["prompt"] = "((emphasis)), (emphasis1:1.1), [to:1], [from::2], [from:to:0.3], [alt|alt1]" + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_not_square_image_performed(self): + self.simple_txt2img["height"] = 128 + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_with_hrfix_performed(self): + self.simple_txt2img["enable_hr"] = True + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_with_tiling_performed(self): + self.simple_txt2img["tiling"] = True + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_with_restore_faces_performed(self): + self.simple_txt2img["restore_faces"] = True + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_with_vanilla_sampler_performed(self): + self.simple_txt2img["sampler_index"] = "PLMS" + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + self.simple_txt2img["sampler_index"] = "DDIM" + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_multiple_batches_performed(self): + self.simple_txt2img["n_iter"] = 2 + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + def test_txt2img_batch_performed(self): + self.simple_txt2img["batch_size"] = 2 + self.assertEqual(requests.post(self.url_txt2img, json=self.simple_txt2img).status_code, 200) + + +if __name__ == "__main__": + unittest.main() diff --git a/sd/stable-diffusion-webui/test/basic_features/utils_test.py b/sd/stable-diffusion-webui/test/basic_features/utils_test.py new file mode 100644 index 0000000000000000000000000000000000000000..0bfc28a0d30c070c292ff8154e9b93a74abecb85 --- /dev/null +++ b/sd/stable-diffusion-webui/test/basic_features/utils_test.py @@ -0,0 +1,62 @@ +import unittest +import requests + +class UtilsTests(unittest.TestCase): + def setUp(self): + self.url_options = "http://localhost:7860/sdapi/v1/options" + self.url_cmd_flags = "http://localhost:7860/sdapi/v1/cmd-flags" + self.url_samplers = "http://localhost:7860/sdapi/v1/samplers" + self.url_upscalers = "http://localhost:7860/sdapi/v1/upscalers" + self.url_sd_models = "http://localhost:7860/sdapi/v1/sd-models" + self.url_hypernetworks = "http://localhost:7860/sdapi/v1/hypernetworks" + self.url_face_restorers = "http://localhost:7860/sdapi/v1/face-restorers" + self.url_realesrgan_models = "http://localhost:7860/sdapi/v1/realesrgan-models" + self.url_prompt_styles = "http://localhost:7860/sdapi/v1/prompt-styles" + self.url_embeddings = "http://localhost:7860/sdapi/v1/embeddings" + + def test_options_get(self): + self.assertEqual(requests.get(self.url_options).status_code, 200) + + def test_options_write(self): + response = requests.get(self.url_options) + self.assertEqual(response.status_code, 200) + + pre_value = response.json()["send_seed"] + + self.assertEqual(requests.post(self.url_options, json={"send_seed":not pre_value}).status_code, 200) + + response = requests.get(self.url_options) + self.assertEqual(response.status_code, 200) + self.assertEqual(response.json()["send_seed"], not pre_value) + + requests.post(self.url_options, json={"send_seed": pre_value}) + + def test_cmd_flags(self): + self.assertEqual(requests.get(self.url_cmd_flags).status_code, 200) + + def test_samplers(self): + self.assertEqual(requests.get(self.url_samplers).status_code, 200) + + def test_upscalers(self): + self.assertEqual(requests.get(self.url_upscalers).status_code, 200) + + def test_sd_models(self): + self.assertEqual(requests.get(self.url_sd_models).status_code, 200) + + def test_hypernetworks(self): + self.assertEqual(requests.get(self.url_hypernetworks).status_code, 200) + + def test_face_restorers(self): + self.assertEqual(requests.get(self.url_face_restorers).status_code, 200) + + def test_realesrgan_models(self): + self.assertEqual(requests.get(self.url_realesrgan_models).status_code, 200) + + def test_prompt_styles(self): + self.assertEqual(requests.get(self.url_prompt_styles).status_code, 200) + + def test_embeddings(self): + self.assertEqual(requests.get(self.url_embeddings).status_code, 200) + +if __name__ == "__main__": + unittest.main() diff --git a/sd/stable-diffusion-webui/test/server_poll.py b/sd/stable-diffusion-webui/test/server_poll.py new file mode 100644 index 0000000000000000000000000000000000000000..42d56a4caacfc40d686dc99668d72238392448cd --- /dev/null +++ b/sd/stable-diffusion-webui/test/server_poll.py @@ -0,0 +1,24 @@ +import unittest +import requests +import time + + +def run_tests(proc, test_dir): + timeout_threshold = 240 + start_time = time.time() + while time.time()-start_time < timeout_threshold: + try: + requests.head("http://localhost:7860/") + break + except requests.exceptions.ConnectionError: + if proc.poll() is not None: + break + if proc.poll() is None: + if test_dir is None: + test_dir = "test" + suite = unittest.TestLoader().discover(test_dir, pattern="*_test.py", top_level_dir="test") + result = unittest.TextTestRunner(verbosity=2).run(suite) + return len(result.failures) + len(result.errors) + else: + print("Launch unsuccessful") + return 1 diff --git a/sd/stable-diffusion-webui/test/test_files/empty.pt b/sd/stable-diffusion-webui/test/test_files/empty.pt new file mode 100644 index 0000000000000000000000000000000000000000..02d735ce54e25e46a75d9122e1d9f2ad5d396fb0 --- /dev/null +++ b/sd/stable-diffusion-webui/test/test_files/empty.pt @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:99153ed9b545f0f5af916841c8510b36c9a0a84e88f412678bced8aba994b482 +size 128 diff --git a/sd/stable-diffusion-webui/test/test_files/img2img_basic.png b/sd/stable-diffusion-webui/test/test_files/img2img_basic.png new file mode 100644 index 0000000000000000000000000000000000000000..49a420482d0a70b9f5986d776a66cb3ea39d1a97 Binary files /dev/null and b/sd/stable-diffusion-webui/test/test_files/img2img_basic.png differ diff --git a/sd/stable-diffusion-webui/test/test_files/mask_basic.png b/sd/stable-diffusion-webui/test/test_files/mask_basic.png new file mode 100644 index 0000000000000000000000000000000000000000..0c2e9a6899e5c0381ce7c7364b31d684464ab423 Binary files /dev/null and b/sd/stable-diffusion-webui/test/test_files/mask_basic.png differ diff --git a/sd/stable-diffusion-webui/textual_inversion_templates/hypernetwork.txt b/sd/stable-diffusion-webui/textual_inversion_templates/hypernetwork.txt new file mode 100644 index 0000000000000000000000000000000000000000..91e06890571c7e4974d5a76c30fab62e8587c7d2 --- /dev/null +++ b/sd/stable-diffusion-webui/textual_inversion_templates/hypernetwork.txt @@ -0,0 +1,27 @@ +a photo of a [filewords] +a rendering of a [filewords] +a cropped photo of the [filewords] +the photo of a [filewords] +a photo of a clean [filewords] +a photo of a dirty [filewords] +a dark photo of the [filewords] +a photo of my [filewords] +a photo of the cool [filewords] +a close-up photo of a [filewords] +a bright photo of the [filewords] +a cropped photo of a [filewords] +a photo of the [filewords] +a good photo of the [filewords] +a photo of one [filewords] +a close-up photo of the [filewords] +a rendition of the [filewords] +a photo of the clean [filewords] +a rendition of a [filewords] +a photo of a nice [filewords] +a good photo of a [filewords] +a photo of the nice [filewords] +a photo of the small [filewords] +a photo of the weird [filewords] +a photo of the large [filewords] +a photo of a cool [filewords] +a photo of a small [filewords] diff --git a/sd/stable-diffusion-webui/textual_inversion_templates/none.txt b/sd/stable-diffusion-webui/textual_inversion_templates/none.txt new file mode 100644 index 0000000000000000000000000000000000000000..f77af4612b289a56b718c3bee62c66a6151f75be --- /dev/null +++ b/sd/stable-diffusion-webui/textual_inversion_templates/none.txt @@ -0,0 +1 @@ +picture diff --git a/sd/stable-diffusion-webui/textual_inversion_templates/style.txt b/sd/stable-diffusion-webui/textual_inversion_templates/style.txt new file mode 100644 index 0000000000000000000000000000000000000000..15af2d6b85f259d0bf41fbe0c8ca7a3340e1b259 --- /dev/null +++ b/sd/stable-diffusion-webui/textual_inversion_templates/style.txt @@ -0,0 +1,19 @@ +a painting, art by [name] +a rendering, art by [name] +a cropped painting, art by [name] +the painting, art by [name] +a clean painting, art by [name] +a dirty painting, art by [name] +a dark painting, art by [name] +a picture, art by [name] +a cool painting, art by [name] +a close-up painting, art by [name] +a bright painting, art by [name] +a cropped painting, art by [name] +a good painting, art by [name] +a close-up painting, art by [name] +a rendition, art by [name] +a nice painting, art by [name] +a small painting, art by [name] +a weird painting, art by [name] +a large painting, art by [name] diff --git a/sd/stable-diffusion-webui/textual_inversion_templates/style_filewords.txt b/sd/stable-diffusion-webui/textual_inversion_templates/style_filewords.txt new file mode 100644 index 0000000000000000000000000000000000000000..b3a8159a869a7890bdd42470664fadf015e0658d --- /dev/null +++ b/sd/stable-diffusion-webui/textual_inversion_templates/style_filewords.txt @@ -0,0 +1,19 @@ +a painting of [filewords], art by [name] +a rendering of [filewords], art by [name] +a cropped painting of [filewords], art by [name] +the painting of [filewords], art by [name] +a clean painting of [filewords], art by [name] +a dirty painting of [filewords], art by [name] +a dark painting of [filewords], art by [name] +a picture of [filewords], art by [name] +a cool painting of [filewords], art by [name] +a close-up painting of [filewords], art by [name] +a bright painting of [filewords], art by [name] +a cropped painting of [filewords], art by [name] +a good painting of [filewords], art by [name] +a close-up painting of [filewords], art by [name] +a rendition of [filewords], art by [name] +a nice painting of [filewords], art by [name] +a small painting of [filewords], art by [name] +a weird painting of [filewords], art by [name] +a large painting of [filewords], art by [name] diff --git a/sd/stable-diffusion-webui/textual_inversion_templates/subject.txt b/sd/stable-diffusion-webui/textual_inversion_templates/subject.txt new file mode 100644 index 0000000000000000000000000000000000000000..79f36aa0543fc2151b7f7e28725309c0c9a4912a --- /dev/null +++ b/sd/stable-diffusion-webui/textual_inversion_templates/subject.txt @@ -0,0 +1,27 @@ +a photo of a [name] +a rendering of a [name] +a cropped photo of the [name] +the photo of a [name] +a photo of a clean [name] +a photo of a dirty [name] +a dark photo of the [name] +a photo of my [name] +a photo of the cool [name] +a close-up photo of a [name] +a bright photo of the [name] +a cropped photo of a [name] +a photo of the [name] +a good photo of the [name] +a photo of one [name] +a close-up photo of the [name] +a rendition of the [name] +a photo of the clean [name] +a rendition of a [name] +a photo of a nice [name] +a good photo of a [name] +a photo of the nice [name] +a photo of the small [name] +a photo of the weird [name] +a photo of the large [name] +a photo of a cool [name] +a photo of a small [name] diff --git a/sd/stable-diffusion-webui/textual_inversion_templates/subject_filewords.txt b/sd/stable-diffusion-webui/textual_inversion_templates/subject_filewords.txt new file mode 100644 index 0000000000000000000000000000000000000000..008652a6bf4277f12a1759f5f3c815ae754dcfcf --- /dev/null +++ b/sd/stable-diffusion-webui/textual_inversion_templates/subject_filewords.txt @@ -0,0 +1,27 @@ +a photo of a [name], [filewords] +a rendering of a [name], [filewords] +a cropped photo of the [name], [filewords] +the photo of a [name], [filewords] +a photo of a clean [name], [filewords] +a photo of a dirty [name], [filewords] +a dark photo of the [name], [filewords] +a photo of my [name], [filewords] +a photo of the cool [name], [filewords] +a close-up photo of a [name], [filewords] +a bright photo of the [name], [filewords] +a cropped photo of a [name], [filewords] +a photo of the [name], [filewords] +a good photo of the [name], [filewords] +a photo of one [name], [filewords] +a close-up photo of the [name], [filewords] +a rendition of the [name], [filewords] +a photo of the clean [name], [filewords] +a rendition of a [name], [filewords] +a photo of a nice [name], [filewords] +a good photo of a [name], [filewords] +a photo of the nice [name], [filewords] +a photo of the small [name], [filewords] +a photo of the weird [name], [filewords] +a photo of the large [name], [filewords] +a photo of a cool [name], [filewords] +a photo of a small [name], [filewords] diff --git a/sd/stable-diffusion-webui/webui-user.sh b/sd/stable-diffusion-webui/webui-user.sh new file mode 100644 index 0000000000000000000000000000000000000000..bfa53cb7c67083ec0a01bfa420269af4d85c6c94 --- /dev/null +++ b/sd/stable-diffusion-webui/webui-user.sh @@ -0,0 +1,46 @@ +#!/bin/bash +######################################################### +# Uncomment and change the variables below to your need:# +######################################################### + +# Install directory without trailing slash +#install_dir="/home/$(whoami)" + +# Name of the subdirectory +#clone_dir="stable-diffusion-webui" + +# Commandline arguments for webui.py, for example: export COMMANDLINE_ARGS="--medvram --opt-split-attention" +#export COMMANDLINE_ARGS="" + +# python3 executable +#python_cmd="python3" + +# git executable +#export GIT="git" + +# python3 venv without trailing slash (defaults to ${install_dir}/${clone_dir}/venv) +#venv_dir="venv" + +# script to launch to start the app +#export LAUNCH_SCRIPT="launch.py" + +# install command for torch +#export TORCH_COMMAND="pip install torch==1.12.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113" + +# Requirements file to use for stable-diffusion-webui +#export REQS_FILE="requirements_versions.txt" + +# Fixed git repos +#export K_DIFFUSION_PACKAGE="" +#export GFPGAN_PACKAGE="" + +# Fixed git commits +#export STABLE_DIFFUSION_COMMIT_HASH="" +#export TAMING_TRANSFORMERS_COMMIT_HASH="" +#export CODEFORMER_COMMIT_HASH="" +#export BLIP_COMMIT_HASH="" + +# Uncomment to enable accelerated launch +#export ACCELERATE="True" + +########################################### diff --git a/sd/stable-diffusion-webui/webui.py b/sd/stable-diffusion-webui/webui.py new file mode 100644 index 0000000000000000000000000000000000000000..4c053a3aceeb84d64f63880b87bf1b6afcbaa937 --- /dev/null +++ b/sd/stable-diffusion-webui/webui.py @@ -0,0 +1,286 @@ +import os +import sys +import time +import importlib +import signal +import re +from fastapi import FastAPI +from fastapi.middleware.cors import CORSMiddleware +from fastapi.middleware.gzip import GZipMiddleware +from packaging import version + +import logging +logging.getLogger("xformers").addFilter(lambda record: 'A matching Triton is not available' not in record.getMessage()) + +from modules import import_hook, errors, extra_networks, ui_extra_networks_checkpoints +from modules import extra_networks_hypernet, ui_extra_networks_hypernets, ui_extra_networks_textual_inversion +from modules.call_queue import wrap_queued_call, queue_lock, wrap_gradio_gpu_call + +import torch + +# Truncate version number of nightly/local build of PyTorch to not cause exceptions with CodeFormer or Safetensors +if ".dev" in torch.__version__ or "+git" in torch.__version__: + torch.__long_version__ = torch.__version__ + torch.__version__ = re.search(r'[\d.]+[\d]', torch.__version__).group(0) + +from modules import shared, devices, sd_samplers, upscaler, extensions, localization, ui_tempdir, ui_extra_networks +import modules.codeformer_model as codeformer +import modules.face_restoration +import modules.gfpgan_model as gfpgan +import modules.img2img + +import modules.lowvram +import modules.paths +import modules.scripts +import modules.sd_hijack +import modules.sd_models +import modules.sd_vae +import modules.txt2img +import modules.script_callbacks +import modules.textual_inversion.textual_inversion +import modules.progress + +import modules.ui +from modules import modelloader +from modules.shared import cmd_opts +import modules.hypernetworks.hypernetwork + + +if cmd_opts.server_name: + server_name = cmd_opts.server_name +else: + server_name = "0.0.0.0" if cmd_opts.listen else None + + +def check_versions(): + if shared.cmd_opts.skip_version_check: + return + + expected_torch_version = "1.13.1" + + if version.parse(torch.__version__) < version.parse(expected_torch_version): + errors.print_error_explanation(f""" +You are running torch {torch.__version__}. +The program is tested to work with torch {expected_torch_version}. +To reinstall the desired version, run with commandline flag --reinstall-torch. +Beware that this will cause a lot of large files to be downloaded, as well as +there are reports of issues with training tab on the latest version. + +Use --skip-version-check commandline argument to disable this check. + """.strip()) + + expected_xformers_version = "0.0.16rc425" + if shared.xformers_available: + import xformers + + if version.parse(xformers.__version__) < version.parse(expected_xformers_version): + errors.print_error_explanation(f""" +You are running xformers {xformers.__version__}. +The program is tested to work with xformers {expected_xformers_version}. +To reinstall the desired version, run with commandline flag --reinstall-xformers. + +Use --skip-version-check commandline argument to disable this check. + """.strip()) + + +def initialize(): + check_versions() + + extensions.list_extensions() + localization.list_localizations(cmd_opts.localizations_dir) + + if cmd_opts.ui_debug_mode: + shared.sd_upscalers = upscaler.UpscalerLanczos().scalers + modules.scripts.load_scripts() + return + + modelloader.cleanup_models() + modules.sd_models.setup_model() + codeformer.setup_model(cmd_opts.codeformer_models_path) + gfpgan.setup_model(cmd_opts.gfpgan_models_path) + + modelloader.list_builtin_upscalers() + modules.scripts.load_scripts() + modelloader.load_upscalers() + + modules.sd_vae.refresh_vae_list() + + modules.textual_inversion.textual_inversion.list_textual_inversion_templates() + + try: + modules.sd_models.load_model() + except Exception as e: + errors.display(e, "loading stable diffusion model") + print("", file=sys.stderr) + print("Stable diffusion model failed to load, exiting", file=sys.stderr) + exit(1) + + shared.opts.data["sd_model_checkpoint"] = shared.sd_model.sd_checkpoint_info.title + + shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights())) + shared.opts.onchange("sd_vae", wrap_queued_call(lambda: modules.sd_vae.reload_vae_weights()), call=False) + shared.opts.onchange("sd_vae_as_default", wrap_queued_call(lambda: modules.sd_vae.reload_vae_weights()), call=False) + shared.opts.onchange("temp_dir", ui_tempdir.on_tmpdir_changed) + + shared.reload_hypernetworks() + + ui_extra_networks.intialize() + ui_extra_networks.register_page(ui_extra_networks_textual_inversion.ExtraNetworksPageTextualInversion()) + ui_extra_networks.register_page(ui_extra_networks_hypernets.ExtraNetworksPageHypernetworks()) + ui_extra_networks.register_page(ui_extra_networks_checkpoints.ExtraNetworksPageCheckpoints()) + + extra_networks.initialize() + extra_networks.register_extra_network(extra_networks_hypernet.ExtraNetworkHypernet()) + + if cmd_opts.tls_keyfile is not None and cmd_opts.tls_keyfile is not None: + + try: + if not os.path.exists(cmd_opts.tls_keyfile): + print("Invalid path to TLS keyfile given") + if not os.path.exists(cmd_opts.tls_certfile): + print(f"Invalid path to TLS certfile: '{cmd_opts.tls_certfile}'") + except TypeError: + cmd_opts.tls_keyfile = cmd_opts.tls_certfile = None + print("TLS setup invalid, running webui without TLS") + else: + print("Running with TLS") + + # make the program just exit at ctrl+c without waiting for anything + def sigint_handler(sig, frame): + print(f'Interrupted with signal {sig} in {frame}') + os._exit(0) + + signal.signal(signal.SIGINT, sigint_handler) + + +def setup_cors(app): + if cmd_opts.cors_allow_origins and cmd_opts.cors_allow_origins_regex: + app.add_middleware(CORSMiddleware, allow_origins=cmd_opts.cors_allow_origins.split(','), allow_origin_regex=cmd_opts.cors_allow_origins_regex, allow_methods=['*'], allow_credentials=True, allow_headers=['*']) + elif cmd_opts.cors_allow_origins: + app.add_middleware(CORSMiddleware, allow_origins=cmd_opts.cors_allow_origins.split(','), allow_methods=['*'], allow_credentials=True, allow_headers=['*']) + elif cmd_opts.cors_allow_origins_regex: + app.add_middleware(CORSMiddleware, allow_origin_regex=cmd_opts.cors_allow_origins_regex, allow_methods=['*'], allow_credentials=True, allow_headers=['*']) + + +def create_api(app): + from modules.api.api import Api + api = Api(app, queue_lock) + return api + + +def wait_on_server(demo=None): + while 1: + time.sleep(0.5) + if shared.state.need_restart: + shared.state.need_restart = False + time.sleep(0.5) + demo.close() + time.sleep(0.5) + break + + +def api_only(): + initialize() + + app = FastAPI() + setup_cors(app) + app.add_middleware(GZipMiddleware, minimum_size=1000) + api = create_api(app) + + modules.script_callbacks.app_started_callback(None, app) + + api.launch(server_name="0.0.0.0" if cmd_opts.listen else "127.0.0.1", port=cmd_opts.port if cmd_opts.port else 7861) + + +def webui(): + launch_api = cmd_opts.api + initialize() + + while 1: + if shared.opts.clean_temp_dir_at_start: + ui_tempdir.cleanup_tmpdr() + + modules.script_callbacks.before_ui_callback() + + shared.demo = modules.ui.create_ui() + + if cmd_opts.gradio_queue: + shared.demo.queue(64) + + gradio_auth_creds = [] + if cmd_opts.gradio_auth: + gradio_auth_creds += cmd_opts.gradio_auth.strip('"').replace('\n', '').split(',') + if cmd_opts.gradio_auth_path: + with open(cmd_opts.gradio_auth_path, 'r', encoding="utf8") as file: + for line in file.readlines(): + gradio_auth_creds += [x.strip() for x in line.split(',')] + + app, local_url, share_url = shared.demo.launch( + share=cmd_opts.share, + server_name=server_name, + server_port=cmd_opts.port, + ssl_keyfile=cmd_opts.tls_keyfile, + ssl_certfile=cmd_opts.tls_certfile, + debug=cmd_opts.gradio_debug, + auth=[tuple(cred.split(':')) for cred in gradio_auth_creds] if gradio_auth_creds else None, + inbrowser=cmd_opts.autolaunch, + prevent_thread_lock=True + ) + # after initial launch, disable --autolaunch for subsequent restarts + cmd_opts.autolaunch = False + + # gradio uses a very open CORS policy via app.user_middleware, which makes it possible for + # an attacker to trick the user into opening a malicious HTML page, which makes a request to the + # running web ui and do whatever the attacker wants, including installing an extension and + # running its code. We disable this here. Suggested by RyotaK. + app.user_middleware = [x for x in app.user_middleware if x.cls.__name__ != 'CORSMiddleware'] + + setup_cors(app) + + app.add_middleware(GZipMiddleware, minimum_size=1000) + + modules.progress.setup_progress_api(app) + + if launch_api: + create_api(app) + + ui_extra_networks.add_pages_to_demo(app) + + modules.script_callbacks.app_started_callback(shared.demo, app) + + wait_on_server(shared.demo) + print('Restarting UI...') + + sd_samplers.set_samplers() + + modules.script_callbacks.script_unloaded_callback() + extensions.list_extensions() + + localization.list_localizations(cmd_opts.localizations_dir) + + modelloader.forbid_loaded_nonbuiltin_upscalers() + modules.scripts.reload_scripts() + modules.script_callbacks.model_loaded_callback(shared.sd_model) + modelloader.load_upscalers() + + for module in [module for name, module in sys.modules.items() if name.startswith("modules.ui")]: + importlib.reload(module) + + modules.sd_models.list_models() + + shared.reload_hypernetworks() + + ui_extra_networks.intialize() + ui_extra_networks.register_page(ui_extra_networks_textual_inversion.ExtraNetworksPageTextualInversion()) + ui_extra_networks.register_page(ui_extra_networks_hypernets.ExtraNetworksPageHypernetworks()) + ui_extra_networks.register_page(ui_extra_networks_checkpoints.ExtraNetworksPageCheckpoints()) + + extra_networks.initialize() + extra_networks.register_extra_network(extra_networks_hypernet.ExtraNetworkHypernet()) + + +if __name__ == "__main__": + if cmd_opts.nowebui: + api_only() + else: + webui() diff --git a/sd/stable-diffusion-webui/webui.sh b/sd/stable-diffusion-webui/webui.sh new file mode 100644 index 0000000000000000000000000000000000000000..8cdad22d310fed20f229b09d7a3160aeb1731a85 --- /dev/null +++ b/sd/stable-diffusion-webui/webui.sh @@ -0,0 +1,186 @@ +#!/usr/bin/env bash +################################################# +# Please do not make any changes to this file, # +# change the variables in webui-user.sh instead # +################################################# + +# If run from macOS, load defaults from webui-macos-env.sh +if [[ "$OSTYPE" == "darwin"* ]]; then + if [[ -f webui-macos-env.sh ]] + then + source ./webui-macos-env.sh + fi +fi + +# Read variables from webui-user.sh +# shellcheck source=/dev/null +if [[ -f webui-user.sh ]] +then + source ./webui-user.sh +fi + +# Set defaults +# Install directory without trailing slash +if [[ -z "${install_dir}" ]] +then + install_dir="/home/$(whoami)" +fi + +# Name of the subdirectory (defaults to stable-diffusion-webui) +if [[ -z "${clone_dir}" ]] +then + clone_dir="stable-diffusion-webui" +fi + +# python3 executable +if [[ -z "${python_cmd}" ]] +then + python_cmd="python3" +fi + +# git executable +if [[ -z "${GIT}" ]] +then + export GIT="git" +fi + +# python3 venv without trailing slash (defaults to ${install_dir}/${clone_dir}/venv) +if [[ -z "${venv_dir}" ]] +then + venv_dir="venv" +fi + +if [[ -z "${LAUNCH_SCRIPT}" ]] +then + LAUNCH_SCRIPT="launch.py" +fi + +# this script cannot be run as root by default +can_run_as_root=0 + +# read any command line flags to the webui.sh script +while getopts "f" flag > /dev/null 2>&1 +do + case ${flag} in + f) can_run_as_root=1;; + *) break;; + esac +done + +# Disable sentry logging +export ERROR_REPORTING=FALSE + +# Do not reinstall existing pip packages on Debian/Ubuntu +export PIP_IGNORE_INSTALLED=0 + +# Pretty print +delimiter="################################################################" + +printf "\n%s\n" "${delimiter}" +printf "\e[1m\e[32mInstall script for stable-diffusion + Web UI\n" +printf "\e[1m\e[34mTested on Debian 11 (Bullseye)\e[0m" +printf "\n%s\n" "${delimiter}" + +# Do not run as root +if [[ $(id -u) -eq 0 && can_run_as_root -eq 0 ]] +then + printf "\n%s\n" "${delimiter}" + printf "\e[1m\e[31mERROR: This script must not be launched as root, aborting...\e[0m" + printf "\n%s\n" "${delimiter}" + exit 1 +else + printf "\n%s\n" "${delimiter}" + printf "Running on \e[1m\e[32m%s\e[0m user" "$(whoami)" + printf "\n%s\n" "${delimiter}" +fi + +if [[ -d .git ]] +then + printf "\n%s\n" "${delimiter}" + printf "Repo already cloned, using it as install directory" + printf "\n%s\n" "${delimiter}" + install_dir="${PWD}/../" + clone_dir="${PWD##*/}" +fi + +# Check prerequisites +gpu_info=$(lspci 2>/dev/null | grep VGA) +case "$gpu_info" in + *"Navi 1"*|*"Navi 2"*) export HSA_OVERRIDE_GFX_VERSION=10.3.0 + ;; + *"Renoir"*) export HSA_OVERRIDE_GFX_VERSION=9.0.0 + printf "\n%s\n" "${delimiter}" + printf "Experimental support for Renoir: make sure to have at least 4GB of VRAM and 10GB of RAM or enable cpu mode: --use-cpu all --no-half" + printf "\n%s\n" "${delimiter}" + ;; + *) + ;; +esac +if echo "$gpu_info" | grep -q "AMD" && [[ -z "${TORCH_COMMAND}" ]] +then + export TORCH_COMMAND="pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/rocm5.2" +fi + +for preq in "${GIT}" "${python_cmd}" +do + if ! hash "${preq}" &>/dev/null + then + printf "\n%s\n" "${delimiter}" + printf "\e[1m\e[31mERROR: %s is not installed, aborting...\e[0m" "${preq}" + printf "\n%s\n" "${delimiter}" + exit 1 + fi +done + +if ! "${python_cmd}" -c "import venv" &>/dev/null +then + printf "\n%s\n" "${delimiter}" + printf "\e[1m\e[31mERROR: python3-venv is not installed, aborting...\e[0m" + printf "\n%s\n" "${delimiter}" + exit 1 +fi + +cd "${install_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/, aborting...\e[0m" "${install_dir}"; exit 1; } +if [[ -d "${clone_dir}" ]] +then + cd "${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting...\e[0m" "${install_dir}" "${clone_dir}"; exit 1; } +else + printf "\n%s\n" "${delimiter}" + printf "Clone stable-diffusion-webui" + printf "\n%s\n" "${delimiter}" + "${GIT}" clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git "${clone_dir}" + cd "${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting...\e[0m" "${install_dir}" "${clone_dir}"; exit 1; } +fi + +printf "\n%s\n" "${delimiter}" +printf "Create and activate python venv" +printf "\n%s\n" "${delimiter}" +cd "${install_dir}"/"${clone_dir}"/ || { printf "\e[1m\e[31mERROR: Can't cd to %s/%s/, aborting...\e[0m" "${install_dir}" "${clone_dir}"; exit 1; } +if [[ ! -d "${venv_dir}" ]] +then + "${python_cmd}" -m venv "${venv_dir}" + first_launch=1 +fi +# shellcheck source=/dev/null +if [[ -f "${venv_dir}"/bin/activate ]] +then + source "${venv_dir}"/bin/activate +else + printf "\n%s\n" "${delimiter}" + printf "\e[1m\e[31mERROR: Cannot activate python venv, aborting...\e[0m" + printf "\n%s\n" "${delimiter}" + exit 1 +fi + +if [[ ! -z "${ACCELERATE}" ]] && [ ${ACCELERATE}="True" ] && [ -x "$(command -v accelerate)" ] +then + printf "\n%s\n" "${delimiter}" + printf "Accelerating launch.py..." + printf "\n%s\n" "${delimiter}" + exec accelerate launch --num_cpu_threads_per_process=6 "${LAUNCH_SCRIPT}" "$@" +else + printf "\n%s\n" "${delimiter}" + printf "Launching launch.py..." + printf "\n%s\n" "${delimiter}" + exec "${python_cmd}" "${LAUNCH_SCRIPT}" "$@" +fi diff --git a/sd/stablediffusion/.gitignore b/sd/stablediffusion/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..3b33f062894825a067995b8b9e8224ba4d9ff708 --- /dev/null +++ b/sd/stablediffusion/.gitignore @@ -0,0 +1,165 @@ +# Generated by project +outputs/ + +# Byte-compiled / optimized / DLL files +__pycache__/ +*.py[cod] +*$py.class + +# C extensions +*.so + +# General MacOS +.DS_Store +.AppleDouble +.LSOverride + +# Distribution / packaging +.Python +build/ +develop-eggs/ +dist/ +downloads/ +eggs/ +.eggs/ +lib/ +lib64/ +parts/ +sdist/ +var/ +wheels/ +share/python-wheels/ +*.egg-info/ +.installed.cfg +*.egg +MANIFEST + +# PyInstaller +# Usually these files are written by a python script from a template +# before PyInstaller builds the exe, so as to inject date/other infos into it. +*.manifest +*.spec + +# Installer logs +pip-log.txt +pip-delete-this-directory.txt + +# Unit test / coverage reports +htmlcov/ +.tox/ +.nox/ +.coverage +.coverage.* +.cache +nosetests.xml +coverage.xml +*.cover +*.py,cover +.hypothesis/ +.pytest_cache/ +cover/ + +# Translations +*.mo +*.pot + +# Django stuff: +*.log +local_settings.py +db.sqlite3 +db.sqlite3-journal + +# Flask stuff: +instance/ +.webassets-cache + +# Scrapy stuff: +.scrapy + +# Sphinx documentation +docs/_build/ + +# PyBuilder +.pybuilder/ +target/ + +# Jupyter Notebook +.ipynb_checkpoints + +# IPython +profile_default/ +ipython_config.py + +# pyenv +# For a library or package, you might want to ignore these files since the code is +# intended to run in multiple environments; otherwise, check them in: +# .python-version + +# pipenv +# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. +# However, in case of collaboration, if having platform-specific dependencies or dependencies +# having no cross-platform support, pipenv may install dependencies that don't work, or not +# install all needed dependencies. +#Pipfile.lock + +# poetry +# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. +# This is especially recommended for binary packages to ensure reproducibility, and is more +# commonly ignored for libraries. +# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control +#poetry.lock + +# pdm +# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. +#pdm.lock +# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it +# in version control. +# https://pdm.fming.dev/#use-with-ide +.pdm.toml + +# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm +__pypackages__/ + +# Celery stuff +celerybeat-schedule +celerybeat.pid + +# SageMath parsed files +*.sage.py + +# Environments +.env +.venv +env/ +venv/ +ENV/ +env.bak/ +venv.bak/ + +# Spyder project settings +.spyderproject +.spyproject + +# Rope project settings +.ropeproject + +# mkdocs documentation +/site + +# mypy +.mypy_cache/ +.dmypy.json +dmypy.json + +# Pyre type checker +.pyre/ + +# pytype static type analyzer +.pytype/ + +# Cython debug symbols +cython_debug/ + +# IDEs +.idea/ +.vscode/ diff --git a/sd/stablediffusion/LICENSE b/sd/stablediffusion/LICENSE new file mode 100644 index 0000000000000000000000000000000000000000..58a49c99b2b9151af5e1fee0dbd20307671f47ab --- /dev/null +++ b/sd/stablediffusion/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2022 Stability AI + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/sd/stablediffusion/LICENSE-MODEL b/sd/stablediffusion/LICENSE-MODEL new file mode 100644 index 0000000000000000000000000000000000000000..9684533d88e7d853a55cabf6caa2f1e4a3e6fdc4 --- /dev/null +++ b/sd/stablediffusion/LICENSE-MODEL @@ -0,0 +1,84 @@ +Copyright (c) 2022 Stability AI and contributors + +CreativeML Open RAIL++-M License +dated November 24, 2022 + +Section I: PREAMBLE + +Multimodal generative models are being widely adopted and used, and have the potential to transform the way artists, among other individuals, conceive and benefit from AI or ML technologies as a tool for content creation. + +Notwithstanding the current and potential benefits that these artifacts can bring to society at large, there are also concerns about potential misuses of them, either due to their technical limitations or ethical considerations. + +In short, this license strives for both the open and responsible downstream use of the accompanying model. When it comes to the open character, we took inspiration from open source permissive licenses regarding the grant of IP rights. Referring to the downstream responsible use, we added use-based restrictions not permitting the use of the Model in very specific scenarios, in order for the licensor to be able to enforce the license in case potential misuses of the Model may occur. At the same time, we strive to promote open and responsible research on generative models for art and content generation. + +Even though downstream derivative versions of the model could be released under different licensing terms, the latter will always have to include - at minimum - the same use-based restrictions as the ones in the original license (this license). We believe in the intersection between open and responsible AI development; thus, this License aims to strike a balance between both in order to enable responsible open-science in the field of AI. + +This License governs the use of the model (and its derivatives) and is informed by the model card associated with the model. + +NOW THEREFORE, You and Licensor agree as follows: + +1. Definitions + +- "License" means the terms and conditions for use, reproduction, and Distribution as defined in this document. +- "Data" means a collection of information and/or content extracted from the dataset used with the Model, including to train, pretrain, or otherwise evaluate the Model. The Data is not licensed under this License. +- "Output" means the results of operating a Model as embodied in informational content resulting therefrom. +- "Model" means any accompanying machine-learning based assemblies (including checkpoints), consisting of learnt weights, parameters (including optimizer states), corresponding to the model architecture as embodied in the Complementary Material, that have been trained or tuned, in whole or in part on the Data, using the Complementary Material. +- "Derivatives of the Model" means all modifications to the Model, works based on the Model, or any other model which is created or initialized by transfer of patterns of the weights, parameters, activations or output of the Model, to the other model, in order to cause the other model to perform similarly to the Model, including - but not limited to - distillation methods entailing the use of intermediate data representations or methods based on the generation of synthetic data by the Model for training the other model. +- "Complementary Material" means the accompanying source code and scripts used to define, run, load, benchmark or evaluate the Model, and used to prepare data for training or evaluation, if any. This includes any accompanying documentation, tutorials, examples, etc, if any. +- "Distribution" means any transmission, reproduction, publication or other sharing of the Model or Derivatives of the Model to a third party, including providing the Model as a hosted service made available by electronic or other remote means - e.g. API-based or web access. +- "Licensor" means the copyright owner or entity authorized by the copyright owner that is granting the License, including the persons or entities that may have rights in the Model and/or distributing the Model. +- "You" (or "Your") means an individual or Legal Entity exercising permissions granted by this License and/or making use of the Model for whichever purpose and in any field of use, including usage of the Model in an end-use application - e.g. chatbot, translator, image generator. +- "Third Parties" means individuals or legal entities that are not under common control with Licensor or You. +- "Contribution" means any work of authorship, including the original version of the Model and any modifications or additions to that Model or Derivatives of the Model thereof, that is intentionally submitted to Licensor for inclusion in the Model by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Model, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." +- "Contributor" means Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Model. + +Section II: INTELLECTUAL PROPERTY RIGHTS + +Both copyright and patent grants apply to the Model, Derivatives of the Model and Complementary Material. The Model and Derivatives of the Model are subject to additional terms as described in Section III. + +2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare, publicly display, publicly perform, sublicense, and distribute the Complementary Material, the Model, and Derivatives of the Model. +3. Grant of Patent License. Subject to the terms and conditions of this License and where and as applicable, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this paragraph) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Model and the Complementary Material, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Model to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Model and/or Complementary Material or a Contribution incorporated within the Model and/or Complementary Material constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for the Model and/or Work shall terminate as of the date such litigation is asserted or filed. + +Section III: CONDITIONS OF USAGE, DISTRIBUTION AND REDISTRIBUTION + +4. Distribution and Redistribution. You may host for Third Party remote access purposes (e.g. software-as-a-service), reproduce and distribute copies of the Model or Derivatives of the Model thereof in any medium, with or without modifications, provided that You meet the following conditions: +Use-based restrictions as referenced in paragraph 5 MUST be included as an enforceable provision by You in any type of legal agreement (e.g. a license) governing the use and/or distribution of the Model or Derivatives of the Model, and You shall give notice to subsequent users You Distribute to, that the Model or Derivatives of the Model are subject to paragraph 5. This provision does not apply to the use of Complementary Material. +You must give any Third Party recipients of the Model or Derivatives of the Model a copy of this License; +You must cause any modified files to carry prominent notices stating that You changed the files; +You must retain all copyright, patent, trademark, and attribution notices excluding those notices that do not pertain to any part of the Model, Derivatives of the Model. +You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions - respecting paragraph 4.a. - for use, reproduction, or Distribution of Your modifications, or for any such Derivatives of the Model as a whole, provided Your use, reproduction, and Distribution of the Model otherwise complies with the conditions stated in this License. +5. Use-based restrictions. The restrictions set forth in Attachment A are considered Use-based restrictions. Therefore You cannot use the Model and the Derivatives of the Model for the specified restricted uses. You may use the Model subject to this License, including only for lawful purposes and in accordance with the License. Use may include creating any content with, finetuning, updating, running, training, evaluating and/or reparametrizing the Model. You shall require all of Your users who use the Model or a Derivative of the Model to comply with the terms of this paragraph (paragraph 5). +6. The Output You Generate. Except as set forth herein, Licensor claims no rights in the Output You generate using the Model. You are accountable for the Output you generate and its subsequent uses. No use of the output can contravene any provision as stated in the License. + +Section IV: OTHER PROVISIONS + +7. Updates and Runtime Restrictions. To the maximum extent permitted by law, Licensor reserves the right to restrict (remotely or otherwise) usage of the Model in violation of this License. +8. Trademarks and related. Nothing in this License permits You to make use of Licensors’ trademarks, trade names, logos or to otherwise suggest endorsement or misrepresent the relationship between the parties; and any rights not expressly granted herein are reserved by the Licensors. +9. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Model and the Complementary Material (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Model, Derivatives of the Model, and the Complementary Material and assume any risks associated with Your exercise of permissions under this License. +10. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Model and the Complementary Material (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. +11. Accepting Warranty or Additional Liability. While redistributing the Model, Derivatives of the Model and the Complementary Material thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. +12. If any provision of this License is held to be invalid, illegal or unenforceable, the remaining provisions shall be unaffected thereby and remain valid as if such provision had not been set forth herein. + +END OF TERMS AND CONDITIONS + + + + +Attachment A + +Use Restrictions + +You agree not to use the Model or Derivatives of the Model: + +- In any way that violates any applicable national, federal, state, local or international law or regulation; +- For the purpose of exploiting, harming or attempting to exploit or harm minors in any way; +- To generate or disseminate verifiably false information and/or content with the purpose of harming others; +- To generate or disseminate personal identifiable information that can be used to harm an individual; +- To defame, disparage or otherwise harass others; +- For fully automated decision making that adversely impacts an individual’s legal rights or otherwise creates or modifies a binding, enforceable obligation; +- For any use intended to or which has the effect of discriminating against or harming individuals or groups based on online or offline social behavior or known or predicted personal or personality characteristics; +- To exploit any of the vulnerabilities of a specific group of persons based on their age, social, physical or mental characteristics, in order to materially distort the behavior of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm; +- For any use intended to or which has the effect of discriminating against individuals or groups based on legally protected characteristics or categories; +- To provide medical advice and medical results interpretation; +- To generate or disseminate information for the purpose to be used for administration of justice, law enforcement, immigration or asylum processes, such as predicting an individual will commit fraud/crime commitment (e.g. by text profiling, drawing causal relationships between assertions made in documents, indiscriminate and arbitrarily-targeted use). + diff --git a/sd/stablediffusion/README.md b/sd/stablediffusion/README.md new file mode 100644 index 0000000000000000000000000000000000000000..f41306887b2b06c62b8ae5cff0212a78c0f6e0f5 --- /dev/null +++ b/sd/stablediffusion/README.md @@ -0,0 +1,257 @@ +# Stable Diffusion Version 2 + + + + +This repository contains [Stable Diffusion](https://github.com/CompVis/stable-diffusion) models trained from scratch and will be continuously updated with +new checkpoints. The following list provides an overview of all currently available models. More coming soon. + +## News + +**December 7, 2022** + +*Version 2.1* + +- New stable diffusion model (_Stable Diffusion 2.1-v_, [HuggingFace](https://huggingface.co/stabilityai/stable-diffusion-2-1)) at 768x768 resolution and (_Stable Diffusion 2.1-base_, [HuggingFace](https://huggingface.co/stabilityai/stable-diffusion-2-1-base)) at 512x512 resolution, both based on the same number of parameters and architecture as 2.0 and fine-tuned on 2.0, on a less restrictive NSFW filtering of the [LAION-5B](https://laion.ai/blog/laion-5b/) dataset. +Per default, the attention operation of the model is evaluated at full precision when `xformers` is not installed. To enable fp16 (which can cause numerical instabilities with the vanilla attention module on the v2.1 model) , run your script with `ATTN_PRECISION=fp16 python+[Robin Rombach](https://github.com/rromb)\*, +[Andreas Blattmann](https://github.com/ablattmann)\*, +[Dominik Lorenz](https://github.com/qp-qp)\, +[Patrick Esser](https://github.com/pesser), +[Björn Ommer](https://hci.iwr.uni-heidelberg.de/Staff/bommer)
+_[CVPR '22 Oral](https://openaccess.thecvf.com/content/CVPR2022/html/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.html) | +[GitHub](https://github.com/CompVis/latent-diffusion) | [arXiv](https://arxiv.org/abs/2112.10752) | [Project page](https://ommer-lab.com/research/latent-diffusion-models/)_ + +and [many others](#shout-outs). + +Stable Diffusion is a latent text-to-image diffusion model. +________________________________ + +## Requirements + +You can update an existing [latent diffusion](https://github.com/CompVis/latent-diffusion) environment by running + +``` +conda install pytorch==1.12.1 torchvision==0.13.1 -c pytorch +pip install transformers==4.19.2 diffusers invisible-watermark +pip install -e . +``` +#### xformers efficient attention +For more efficiency and speed on GPUs, +we highly recommended installing the [xformers](https://github.com/facebookresearch/xformers) +library. + +Tested on A100 with CUDA 11.4. +Installation needs a somewhat recent version of nvcc and gcc/g++, obtain those, e.g., via +```commandline +export CUDA_HOME=/usr/local/cuda-11.4 +conda install -c nvidia/label/cuda-11.4.0 cuda-nvcc +conda install -c conda-forge gcc +conda install -c conda-forge gxx_linux-64==9.5.0 +``` + +Then, run the following (compiling takes up to 30 min). + +```commandline +cd .. +git clone https://github.com/facebookresearch/xformers.git +cd xformers +git submodule update --init --recursive +pip install -r requirements.txt +pip install -e . +cd ../stablediffusion +``` +Upon successful installation, the code will automatically default to [memory efficient attention](https://github.com/facebookresearch/xformers) +for the self- and cross-attention layers in the U-Net and autoencoder. + +## General Disclaimer +Stable Diffusion models are general text-to-image diffusion models and therefore mirror biases and (mis-)conceptions that are present +in their training data. Although efforts were made to reduce the inclusion of explicit pornographic material, **we do not recommend using the provided weights for services or products without additional safety mechanisms and considerations. +The weights are research artifacts and should be treated as such.** +Details on the training procedure and data, as well as the intended use of the model can be found in the corresponding [model card](https://huggingface.co/stabilityai/stable-diffusion-2). +The weights are available via [the StabilityAI organization at Hugging Face](https://huggingface.co/StabilityAI) under the [CreativeML Open RAIL++-M License](LICENSE-MODEL). + + + +## Stable Diffusion v2 + +Stable Diffusion v2 refers to a specific configuration of the model +architecture that uses a downsampling-factor 8 autoencoder with an 865M UNet +and OpenCLIP ViT-H/14 text encoder for the diffusion model. The _SD 2-v_ model produces 768x768 px outputs. + +Evaluations with different classifier-free guidance scales (1.5, 2.0, 3.0, 4.0, +5.0, 6.0, 7.0, 8.0) and 50 DDIM sampling steps show the relative improvements of the checkpoints: + + + + + +### Text-to-Image + + + +Stable Diffusion 2 is a latent diffusion model conditioned on the penultimate text embeddings of a CLIP ViT-H/14 text encoder. +We provide a [reference script for sampling](#reference-sampling-script). +#### Reference Sampling Script + +This script incorporates an [invisible watermarking](https://github.com/ShieldMnt/invisible-watermark) of the outputs, to help viewers [identify the images as machine-generated](scripts/tests/test_watermark.py). +We provide the configs for the _SD2-v_ (768px) and _SD2-base_ (512px) model. + +First, download the weights for [_SD2.1-v_](https://huggingface.co/stabilityai/stable-diffusion-2-1) and [_SD2.1-base_](https://huggingface.co/stabilityai/stable-diffusion-2-1-base). + +To sample from the _SD2.1-v_ model, run the following: + +``` +python scripts/txt2img.py --prompt "a professional photograph of an astronaut riding a horse" --ckpt
+ depth2image
+
+
+
+This is the PyTorch code of the BLIP paper [[blog](https://blog.salesforceairesearch.com/blip-bootstrapping-language-image-pretraining/)]. The code has been tested on PyTorch 1.10.
+To install the dependencies, run pip install -r requirements.txt
+
+Catalog:
+- [x] Inference demo
+- [x] Pre-trained and finetuned checkpoints
+- [x] Finetuning code for Image-Text Retrieval, Image Captioning, VQA, and NLVR2
+- [x] Pre-training code
+- [x] Zero-shot video-text retrieval
+- [x] Download of bootstrapped pre-training datasets
+
+
+### Inference demo:
+Run our interactive demo using [Colab notebook](https://colab.research.google.com/github/salesforce/BLIP/blob/main/demo.ipynb) (no GPU needed).
+The demo includes code for:
+1. Image captioning
+2. Open-ended visual question answering
+3. Multimodal / unimodal feature extraction
+4. Image-text matching
+
+Try out the [Web demo](https://huggingface.co/spaces/Salesforce/BLIP), integrated into [Huggingface Spaces 🤗](https://huggingface.co/spaces) using [Gradio](https://github.com/gradio-app/gradio).
+
+Replicate web demo and Docker image is also available at [](https://replicate.com/salesforce/blip)
+
+### Pre-trained checkpoints:
+Num. pre-train images | BLIP w/ ViT-B | BLIP w/ ViT-B and CapFilt-L | BLIP w/ ViT-L
+--- | :---: | :---: | :---:
+14M | Download| - | -
+129M | Download| Download | Download
+
+### Finetuned checkpoints:
+Task | BLIP w/ ViT-B | BLIP w/ ViT-B and CapFilt-L | BLIP w/ ViT-L
+--- | :---: | :---: | :---:
+Image-Text Retrieval (COCO) | Download| - | Download
+Image-Text Retrieval (Flickr30k) | Download| - | Download
+Image Captioning (COCO) | - | Download| Download |
+VQA | Download| Download | -
+NLVR2 | Download| - | -
+
+
+### Image-Text Retrieval:
+1. Download COCO and Flickr30k datasets from the original websites, and set 'image_root' in configs/retrieval_{dataset}.yaml accordingly.
+2. To evaluate the finetuned BLIP model on COCO, run:
+python -m torch.distributed.run --nproc_per_node=8 train_retrieval.py \ +--config ./configs/retrieval_coco.yaml \ +--output_dir output/retrieval_coco \ +--evaluate+3. To finetune the pre-trained checkpoint using 8 A100 GPUs, first set 'pretrained' in configs/retrieval_coco.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base.pth". Then run: +
python -m torch.distributed.run --nproc_per_node=8 train_retrieval.py \ +--config ./configs/retrieval_coco.yaml \ +--output_dir output/retrieval_coco+ +### Image-Text Captioning: +1. Download COCO and NoCaps datasets from the original websites, and set 'image_root' in configs/caption_coco.yaml and configs/nocaps.yaml accordingly. +2. To evaluate the finetuned BLIP model on COCO, run: +
python -m torch.distributed.run --nproc_per_node=8 train_caption.py --evaluate+3. To evaluate the finetuned BLIP model on NoCaps, generate results with: (evaluation needs to be performed on official server) +
python -m torch.distributed.run --nproc_per_node=8 eval_nocaps.py+4. To finetune the pre-trained checkpoint using 8 A100 GPUs, first set 'pretrained' in configs/caption_coco.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_capfilt_large.pth". Then run: +
python -m torch.distributed.run --nproc_per_node=8 train_caption.py+ +### VQA: +1. Download VQA v2 dataset and Visual Genome dataset from the original websites, and set 'vqa_root' and 'vg_root' in configs/vqa.yaml. +2. To evaluate the finetuned BLIP model, generate results with: (evaluation needs to be performed on official server) +
python -m torch.distributed.run --nproc_per_node=8 train_vqa.py --evaluate+3. To finetune the pre-trained checkpoint using 16 A100 GPUs, first set 'pretrained' in configs/vqa.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_capfilt_large.pth". Then run: +
python -m torch.distributed.run --nproc_per_node=16 train_vqa.py+ +### NLVR2: +1. Download NLVR2 dataset from the original websites, and set 'image_root' in configs/nlvr.yaml. +2. To evaluate the finetuned BLIP model, run +
python -m torch.distributed.run --nproc_per_node=8 train_nlvr.py --evaluate+3. To finetune the pre-trained checkpoint using 16 A100 GPUs, first set 'pretrained' in configs/nlvr.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base.pth". Then run: +
python -m torch.distributed.run --nproc_per_node=16 train_nlvr.py+ +### Finetune with ViT-L: +In order to finetune a model with ViT-L, simply change the config file to set 'vit' as large. Batch size and learning rate may also need to be adjusted accordingly (please see the paper's appendix for hyper-parameter details). Gradient checkpoint can also be activated in the config file to reduce GPU memory usage. + +### Pre-train: +1. Prepare training json files where each json file contains a list. Each item in the list is a dictonary with two key-value pairs: {'image': path_of_image, 'caption': text_of_image}. +2. In configs/pretrain.yaml, set 'train_file' as the paths for the json files . +3. Pre-train the model using 8 A100 GPUs: +
python -m torch.distributed.run --nproc_per_node=8 pretrain.py --config ./configs/Pretrain.yaml --output_dir output/Pretrain+ +### Zero-shot video-text retrieval: +1. Download MSRVTT dataset following the instructions from https://github.com/salesforce/ALPRO, and set 'video_root' accordingly in configs/retrieval_msrvtt.yaml. +2. Install [decord](https://github.com/dmlc/decord) with
pip install decord+3. To perform zero-shot evaluation, run +
python -m torch.distributed.run --nproc_per_node=8 eval_retrieval_video.py+ +### Pre-training datasets download: +We provide bootstrapped pre-training datasets as json files. Each json file contains a list. Each item in the list is a dictonary with two key-value pairs: {'url': url_of_image, 'caption': text_of_image}. + +Image source | Filtered web caption | Filtered synthetic caption by ViT-B | Filtered synthetic caption by ViT-L +--- | :---: | :---: | :---: +CC3M+CC12M+SBU | Download| Download| Download +LAION115M | Download| Download| Download + +### Citation +If you find this code to be useful for your research, please consider citing. +
+@inproceedings{li2022blip,
+ title={BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation},
+ author={Junnan Li and Dongxu Li and Caiming Xiong and Steven Hoi},
+ year={2022},
+ booktitle={ICML},
+}
+
+### Acknowledgement
+The implementation of BLIP relies on resources from ALBEF, Huggingface Transformers, and timm. We thank the original authors for their open-sourcing.
diff --git a/sd/stablediffusion/src/blip/SECURITY.md b/sd/stablediffusion/src/blip/SECURITY.md
new file mode 100644
index 0000000000000000000000000000000000000000..8249025739809035264e7776583b2f3ec100553c
--- /dev/null
+++ b/sd/stablediffusion/src/blip/SECURITY.md
@@ -0,0 +1,7 @@
+## Security
+
+Please report any security issue to [security@salesforce.com](mailto:security@salesforce.com)
+as soon as it is discovered. This library limits its runtime dependencies in
+order to reduce the total cost of ownership as much as can be, but all consumers
+should remain vigilant and have their security stakeholders review all third-party
+products (3PP) like this one and their dependencies.
diff --git a/sd/stablediffusion/src/blip/cog.yaml b/sd/stablediffusion/src/blip/cog.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..c1dfcc430a4cab0fdd2a60a682336219a61c4a4f
--- /dev/null
+++ b/sd/stablediffusion/src/blip/cog.yaml
@@ -0,0 +1,17 @@
+build:
+ gpu: true
+ cuda: "11.1"
+ python_version: "3.8"
+ system_packages:
+ - "libgl1-mesa-glx"
+ - "libglib2.0-0"
+ python_packages:
+ - "ipython==7.30.1"
+ - "torchvision==0.11.1"
+ - "torch==1.10.0"
+ - "timm==0.4.12"
+ - "transformers==4.15.0"
+ - "fairscale==0.4.4"
+ - "pycocoevalcap==1.2"
+
+predict: "predict.py:Predictor"
diff --git a/sd/stablediffusion/src/blip/configs/bert_config.json b/sd/stablediffusion/src/blip/configs/bert_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..3ef38aabc7f966b53079e9d559dc59e459cc0051
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/bert_config.json
@@ -0,0 +1,21 @@
+{
+ "architectures": [
+ "BertModel"
+ ],
+ "attention_probs_dropout_prob": 0.1,
+ "hidden_act": "gelu",
+ "hidden_dropout_prob": 0.1,
+ "hidden_size": 768,
+ "initializer_range": 0.02,
+ "intermediate_size": 3072,
+ "layer_norm_eps": 1e-12,
+ "max_position_embeddings": 512,
+ "model_type": "bert",
+ "num_attention_heads": 12,
+ "num_hidden_layers": 12,
+ "pad_token_id": 0,
+ "type_vocab_size": 2,
+ "vocab_size": 30522,
+ "encoder_width": 768,
+ "add_cross_attention": true
+}
diff --git a/sd/stablediffusion/src/blip/configs/caption_coco.yaml b/sd/stablediffusion/src/blip/configs/caption_coco.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..42eab7030c0310ba2f265baf36fa1400aa6e5846
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/caption_coco.yaml
@@ -0,0 +1,33 @@
+image_root: '/export/share/datasets/vision/coco/images/'
+ann_root: 'annotation'
+coco_gt_root: 'annotation/coco_gt'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_caption_capfilt_large.pth'
+
+# size of vit model; base or large
+vit: 'base'
+vit_grad_ckpt: False
+vit_ckpt_layer: 0
+batch_size: 32
+init_lr: 1e-5
+
+# vit: 'large'
+# vit_grad_ckpt: True
+# vit_ckpt_layer: 5
+# batch_size: 16
+# init_lr: 2e-6
+
+image_size: 384
+
+# generation configs
+max_length: 20
+min_length: 5
+num_beams: 3
+prompt: 'a picture of '
+
+# optimizer
+weight_decay: 0.05
+min_lr: 0
+max_epoch: 5
+
diff --git a/sd/stablediffusion/src/blip/configs/med_config.json b/sd/stablediffusion/src/blip/configs/med_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..0ffad0a6f3c2f9f11b8faa84529d9860bb70327a
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/med_config.json
@@ -0,0 +1,21 @@
+{
+ "architectures": [
+ "BertModel"
+ ],
+ "attention_probs_dropout_prob": 0.1,
+ "hidden_act": "gelu",
+ "hidden_dropout_prob": 0.1,
+ "hidden_size": 768,
+ "initializer_range": 0.02,
+ "intermediate_size": 3072,
+ "layer_norm_eps": 1e-12,
+ "max_position_embeddings": 512,
+ "model_type": "bert",
+ "num_attention_heads": 12,
+ "num_hidden_layers": 12,
+ "pad_token_id": 0,
+ "type_vocab_size": 2,
+ "vocab_size": 30524,
+ "encoder_width": 768,
+ "add_cross_attention": true
+}
diff --git a/sd/stablediffusion/src/blip/configs/nlvr.yaml b/sd/stablediffusion/src/blip/configs/nlvr.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..2d1122aadb1a776bd347068233096b0c984f648b
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/nlvr.yaml
@@ -0,0 +1,21 @@
+image_root: '/export/share/datasets/vision/NLVR2/'
+ann_root: 'annotation'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_nlvr.pth'
+
+#size of vit model; base or large
+vit: 'base'
+batch_size_train: 16
+batch_size_test: 64
+vit_grad_ckpt: False
+vit_ckpt_layer: 0
+max_epoch: 15
+
+image_size: 384
+
+# optimizer
+weight_decay: 0.05
+init_lr: 3e-5
+min_lr: 0
+
diff --git a/sd/stablediffusion/src/blip/configs/nocaps.yaml b/sd/stablediffusion/src/blip/configs/nocaps.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..9028135859b94aef5324c85c80e376c609d8a089
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/nocaps.yaml
@@ -0,0 +1,15 @@
+image_root: '/export/share/datasets/vision/nocaps/'
+ann_root: 'annotation'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_caption_capfilt_large.pth'
+
+vit: 'base'
+batch_size: 32
+
+image_size: 384
+
+max_length: 20
+min_length: 5
+num_beams: 3
+prompt: 'a picture of '
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/configs/pretrain.yaml b/sd/stablediffusion/src/blip/configs/pretrain.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..02355ee0228932803c661616485bf315e862b826
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/pretrain.yaml
@@ -0,0 +1,27 @@
+train_file: ['/export/share/junnan-li/VL_pretrain/annotation/coco_karpathy_train.json',
+ '/export/share/junnan-li/VL_pretrain/annotation/vg_caption.json',
+ ]
+laion_path: ''
+
+# size of vit model; base or large
+vit: 'base'
+vit_grad_ckpt: False
+vit_ckpt_layer: 0
+
+image_size: 224
+batch_size: 75
+
+queue_size: 57600
+alpha: 0.4
+
+# optimizer
+weight_decay: 0.05
+init_lr: 3e-4
+min_lr: 1e-6
+warmup_lr: 1e-6
+lr_decay_rate: 0.9
+max_epoch: 20
+warmup_steps: 3000
+
+
+
diff --git a/sd/stablediffusion/src/blip/configs/retrieval_coco.yaml b/sd/stablediffusion/src/blip/configs/retrieval_coco.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..a8569e9b67112fe3605ac25e4fdc0231f7975378
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/retrieval_coco.yaml
@@ -0,0 +1,34 @@
+image_root: '/export/share/datasets/vision/coco/images/'
+ann_root: 'annotation'
+dataset: 'coco'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_retrieval_coco.pth'
+
+# size of vit model; base or large
+
+vit: 'base'
+batch_size_train: 32
+batch_size_test: 64
+vit_grad_ckpt: True
+vit_ckpt_layer: 4
+init_lr: 1e-5
+
+# vit: 'large'
+# batch_size_train: 16
+# batch_size_test: 32
+# vit_grad_ckpt: True
+# vit_ckpt_layer: 12
+# init_lr: 5e-6
+
+image_size: 384
+queue_size: 57600
+alpha: 0.4
+k_test: 256
+negative_all_rank: True
+
+# optimizer
+weight_decay: 0.05
+min_lr: 0
+max_epoch: 6
+
diff --git a/sd/stablediffusion/src/blip/configs/retrieval_flickr.yaml b/sd/stablediffusion/src/blip/configs/retrieval_flickr.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..d75ea4eed87c9a001523c5e5914998c5e737594d
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/retrieval_flickr.yaml
@@ -0,0 +1,34 @@
+image_root: '/export/share/datasets/vision/flickr30k/'
+ann_root: 'annotation'
+dataset: 'flickr'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_retrieval_flickr.pth'
+
+# size of vit model; base or large
+
+vit: 'base'
+batch_size_train: 32
+batch_size_test: 64
+vit_grad_ckpt: True
+vit_ckpt_layer: 4
+init_lr: 1e-5
+
+# vit: 'large'
+# batch_size_train: 16
+# batch_size_test: 32
+# vit_grad_ckpt: True
+# vit_ckpt_layer: 10
+# init_lr: 5e-6
+
+image_size: 384
+queue_size: 57600
+alpha: 0.4
+k_test: 128
+negative_all_rank: False
+
+# optimizer
+weight_decay: 0.05
+min_lr: 0
+max_epoch: 6
+
diff --git a/sd/stablediffusion/src/blip/configs/retrieval_msrvtt.yaml b/sd/stablediffusion/src/blip/configs/retrieval_msrvtt.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..395f62542bb22d706b8e19e2455d2c7298984d0b
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/retrieval_msrvtt.yaml
@@ -0,0 +1,12 @@
+video_root: '/export/share/dongxuli/data/msrvtt_retrieval/videos'
+ann_root: 'annotation'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_retrieval_coco.pth'
+
+# size of vit model; base or large
+vit: 'base'
+batch_size: 64
+k_test: 128
+image_size: 384
+num_frm_test: 8
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/configs/vqa.yaml b/sd/stablediffusion/src/blip/configs/vqa.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..74327e6d0a34672023b44569558fe8beeb052548
--- /dev/null
+++ b/sd/stablediffusion/src/blip/configs/vqa.yaml
@@ -0,0 +1,25 @@
+vqa_root: '/export/share/datasets/vision/VQA/Images/mscoco/' #followed by train2014/
+vg_root: '/export/share/datasets/vision/visual-genome/' #followed by image/
+train_files: ['vqa_train','vqa_val','vg_qa']
+ann_root: 'annotation'
+
+# set pretrained as a file path or an url
+pretrained: 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_vqa_capfilt_large.pth'
+
+# size of vit model; base or large
+vit: 'base'
+batch_size_train: 16
+batch_size_test: 32
+vit_grad_ckpt: False
+vit_ckpt_layer: 0
+init_lr: 2e-5
+
+image_size: 480
+
+k_test: 128
+inference: 'rank'
+
+# optimizer
+weight_decay: 0.05
+min_lr: 0
+max_epoch: 10
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/__init__.py b/sd/stablediffusion/src/blip/data/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..0be209acf415855ea6ef753efedf903b5decb6b9
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/__init__.py
@@ -0,0 +1,101 @@
+import torch
+from torch.utils.data import DataLoader
+from torchvision import transforms
+from torchvision.transforms.functional import InterpolationMode
+
+from data.coco_karpathy_dataset import coco_karpathy_train, coco_karpathy_caption_eval, coco_karpathy_retrieval_eval
+from data.nocaps_dataset import nocaps_eval
+from data.flickr30k_dataset import flickr30k_train, flickr30k_retrieval_eval
+from data.vqa_dataset import vqa_dataset
+from data.nlvr_dataset import nlvr_dataset
+from data.pretrain_dataset import pretrain_dataset
+from transform.randaugment import RandomAugment
+
+def create_dataset(dataset, config, min_scale=0.5):
+
+ normalize = transforms.Normalize((0.48145466, 0.4578275, 0.40821073), (0.26862954, 0.26130258, 0.27577711))
+
+ transform_train = transforms.Compose([
+ transforms.RandomResizedCrop(config['image_size'],scale=(min_scale, 1.0),interpolation=InterpolationMode.BICUBIC),
+ transforms.RandomHorizontalFlip(),
+ RandomAugment(2,5,isPIL=True,augs=['Identity','AutoContrast','Brightness','Sharpness','Equalize',
+ 'ShearX', 'ShearY', 'TranslateX', 'TranslateY', 'Rotate']),
+ transforms.ToTensor(),
+ normalize,
+ ])
+ transform_test = transforms.Compose([
+ transforms.Resize((config['image_size'],config['image_size']),interpolation=InterpolationMode.BICUBIC),
+ transforms.ToTensor(),
+ normalize,
+ ])
+
+ if dataset=='pretrain':
+ dataset = pretrain_dataset(config['train_file'], config['laion_path'], transform_train)
+ return dataset
+
+ elif dataset=='caption_coco':
+ train_dataset = coco_karpathy_train(transform_train, config['image_root'], config['ann_root'], prompt=config['prompt'])
+ val_dataset = coco_karpathy_caption_eval(transform_test, config['image_root'], config['ann_root'], 'val')
+ test_dataset = coco_karpathy_caption_eval(transform_test, config['image_root'], config['ann_root'], 'test')
+ return train_dataset, val_dataset, test_dataset
+
+ elif dataset=='nocaps':
+ val_dataset = nocaps_eval(transform_test, config['image_root'], config['ann_root'], 'val')
+ test_dataset = nocaps_eval(transform_test, config['image_root'], config['ann_root'], 'test')
+ return val_dataset, test_dataset
+
+ elif dataset=='retrieval_coco':
+ train_dataset = coco_karpathy_train(transform_train, config['image_root'], config['ann_root'])
+ val_dataset = coco_karpathy_retrieval_eval(transform_test, config['image_root'], config['ann_root'], 'val')
+ test_dataset = coco_karpathy_retrieval_eval(transform_test, config['image_root'], config['ann_root'], 'test')
+ return train_dataset, val_dataset, test_dataset
+
+ elif dataset=='retrieval_flickr':
+ train_dataset = flickr30k_train(transform_train, config['image_root'], config['ann_root'])
+ val_dataset = flickr30k_retrieval_eval(transform_test, config['image_root'], config['ann_root'], 'val')
+ test_dataset = flickr30k_retrieval_eval(transform_test, config['image_root'], config['ann_root'], 'test')
+ return train_dataset, val_dataset, test_dataset
+
+ elif dataset=='vqa':
+ train_dataset = vqa_dataset(transform_train, config['ann_root'], config['vqa_root'], config['vg_root'],
+ train_files = config['train_files'], split='train')
+ test_dataset = vqa_dataset(transform_test, config['ann_root'], config['vqa_root'], config['vg_root'], split='test')
+ return train_dataset, test_dataset
+
+ elif dataset=='nlvr':
+ train_dataset = nlvr_dataset(transform_train, config['image_root'], config['ann_root'],'train')
+ val_dataset = nlvr_dataset(transform_test, config['image_root'], config['ann_root'],'val')
+ test_dataset = nlvr_dataset(transform_test, config['image_root'], config['ann_root'],'test')
+ return train_dataset, val_dataset, test_dataset
+
+
+def create_sampler(datasets, shuffles, num_tasks, global_rank):
+ samplers = []
+ for dataset,shuffle in zip(datasets,shuffles):
+ sampler = torch.utils.data.DistributedSampler(dataset, num_replicas=num_tasks, rank=global_rank, shuffle=shuffle)
+ samplers.append(sampler)
+ return samplers
+
+
+def create_loader(datasets, samplers, batch_size, num_workers, is_trains, collate_fns):
+ loaders = []
+ for dataset,sampler,bs,n_worker,is_train,collate_fn in zip(datasets,samplers,batch_size,num_workers,is_trains,collate_fns):
+ if is_train:
+ shuffle = (sampler is None)
+ drop_last = True
+ else:
+ shuffle = False
+ drop_last = False
+ loader = DataLoader(
+ dataset,
+ batch_size=bs,
+ num_workers=n_worker,
+ pin_memory=True,
+ sampler=sampler,
+ shuffle=shuffle,
+ collate_fn=collate_fn,
+ drop_last=drop_last,
+ )
+ loaders.append(loader)
+ return loaders
+
diff --git a/sd/stablediffusion/src/blip/data/coco_karpathy_dataset.py b/sd/stablediffusion/src/blip/data/coco_karpathy_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..a34d29205f42aa09695b160ac9c91958ba041bb3
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/coco_karpathy_dataset.py
@@ -0,0 +1,126 @@
+import os
+import json
+
+from torch.utils.data import Dataset
+from torchvision.datasets.utils import download_url
+
+from PIL import Image
+
+from data.utils import pre_caption
+
+class coco_karpathy_train(Dataset):
+ def __init__(self, transform, image_root, ann_root, max_words=30, prompt=''):
+ '''
+ image_root (string): Root directory of images (e.g. coco/images/)
+ ann_root (string): directory to store the annotation file
+ '''
+ url = 'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_train.json'
+ filename = 'coco_karpathy_train.json'
+
+ download_url(url,ann_root)
+
+ self.annotation = json.load(open(os.path.join(ann_root,filename),'r'))
+ self.transform = transform
+ self.image_root = image_root
+ self.max_words = max_words
+ self.prompt = prompt
+
+ self.img_ids = {}
+ n = 0
+ for ann in self.annotation:
+ img_id = ann['image_id']
+ if img_id not in self.img_ids.keys():
+ self.img_ids[img_id] = n
+ n += 1
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ image_path = os.path.join(self.image_root,ann['image'])
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ caption = self.prompt+pre_caption(ann['caption'], self.max_words)
+
+ return image, caption, self.img_ids[ann['image_id']]
+
+
+class coco_karpathy_caption_eval(Dataset):
+ def __init__(self, transform, image_root, ann_root, split):
+ '''
+ image_root (string): Root directory of images (e.g. coco/images/)
+ ann_root (string): directory to store the annotation file
+ split (string): val or test
+ '''
+ urls = {'val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_val.json',
+ 'test':'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_test.json'}
+ filenames = {'val':'coco_karpathy_val.json','test':'coco_karpathy_test.json'}
+
+ download_url(urls[split],ann_root)
+
+ self.annotation = json.load(open(os.path.join(ann_root,filenames[split]),'r'))
+ self.transform = transform
+ self.image_root = image_root
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ image_path = os.path.join(self.image_root,ann['image'])
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ img_id = ann['image'].split('/')[-1].strip('.jpg').split('_')[-1]
+
+ return image, int(img_id)
+
+
+class coco_karpathy_retrieval_eval(Dataset):
+ def __init__(self, transform, image_root, ann_root, split, max_words=30):
+ '''
+ image_root (string): Root directory of images (e.g. coco/images/)
+ ann_root (string): directory to store the annotation file
+ split (string): val or test
+ '''
+ urls = {'val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_val.json',
+ 'test':'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_test.json'}
+ filenames = {'val':'coco_karpathy_val.json','test':'coco_karpathy_test.json'}
+
+ download_url(urls[split],ann_root)
+
+ self.annotation = json.load(open(os.path.join(ann_root,filenames[split]),'r'))
+ self.transform = transform
+ self.image_root = image_root
+
+ self.text = []
+ self.image = []
+ self.txt2img = {}
+ self.img2txt = {}
+
+ txt_id = 0
+ for img_id, ann in enumerate(self.annotation):
+ self.image.append(ann['image'])
+ self.img2txt[img_id] = []
+ for i, caption in enumerate(ann['caption']):
+ self.text.append(pre_caption(caption,max_words))
+ self.img2txt[img_id].append(txt_id)
+ self.txt2img[txt_id] = img_id
+ txt_id += 1
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ image_path = os.path.join(self.image_root, self.annotation[index]['image'])
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ return image, index
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/flickr30k_dataset.py b/sd/stablediffusion/src/blip/data/flickr30k_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..018ab387014ddaf554c4d3184cfc0e2ba8b2d487
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/flickr30k_dataset.py
@@ -0,0 +1,93 @@
+import os
+import json
+
+from torch.utils.data import Dataset
+from torchvision.datasets.utils import download_url
+
+from PIL import Image
+
+from data.utils import pre_caption
+
+class flickr30k_train(Dataset):
+ def __init__(self, transform, image_root, ann_root, max_words=30, prompt=''):
+ '''
+ image_root (string): Root directory of images (e.g. flickr30k/)
+ ann_root (string): directory to store the annotation file
+ '''
+ url = 'https://storage.googleapis.com/sfr-vision-language-research/datasets/flickr30k_train.json'
+ filename = 'flickr30k_train.json'
+
+ download_url(url,ann_root)
+
+ self.annotation = json.load(open(os.path.join(ann_root,filename),'r'))
+ self.transform = transform
+ self.image_root = image_root
+ self.max_words = max_words
+ self.prompt = prompt
+
+ self.img_ids = {}
+ n = 0
+ for ann in self.annotation:
+ img_id = ann['image_id']
+ if img_id not in self.img_ids.keys():
+ self.img_ids[img_id] = n
+ n += 1
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ image_path = os.path.join(self.image_root,ann['image'])
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ caption = self.prompt+pre_caption(ann['caption'], self.max_words)
+
+ return image, caption, self.img_ids[ann['image_id']]
+
+
+class flickr30k_retrieval_eval(Dataset):
+ def __init__(self, transform, image_root, ann_root, split, max_words=30):
+ '''
+ image_root (string): Root directory of images (e.g. flickr30k/)
+ ann_root (string): directory to store the annotation file
+ split (string): val or test
+ '''
+ urls = {'val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/flickr30k_val.json',
+ 'test':'https://storage.googleapis.com/sfr-vision-language-research/datasets/flickr30k_test.json'}
+ filenames = {'val':'flickr30k_val.json','test':'flickr30k_test.json'}
+
+ download_url(urls[split],ann_root)
+
+ self.annotation = json.load(open(os.path.join(ann_root,filenames[split]),'r'))
+ self.transform = transform
+ self.image_root = image_root
+
+ self.text = []
+ self.image = []
+ self.txt2img = {}
+ self.img2txt = {}
+
+ txt_id = 0
+ for img_id, ann in enumerate(self.annotation):
+ self.image.append(ann['image'])
+ self.img2txt[img_id] = []
+ for i, caption in enumerate(ann['caption']):
+ self.text.append(pre_caption(caption,max_words))
+ self.img2txt[img_id].append(txt_id)
+ self.txt2img[txt_id] = img_id
+ txt_id += 1
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ image_path = os.path.join(self.image_root, self.annotation[index]['image'])
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ return image, index
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/nlvr_dataset.py b/sd/stablediffusion/src/blip/data/nlvr_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..a8d6b2d7cd8d3260bd279c7dca80de53bacc691a
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/nlvr_dataset.py
@@ -0,0 +1,78 @@
+import os
+import json
+import random
+
+from torch.utils.data import Dataset
+from torchvision.datasets.utils import download_url
+
+from PIL import Image
+
+from data.utils import pre_caption
+
+class nlvr_dataset(Dataset):
+ def __init__(self, transform, image_root, ann_root, split):
+ '''
+ image_root (string): Root directory of images
+ ann_root (string): directory to store the annotation file
+ split (string): train, val or test
+ '''
+ urls = {'train':'https://storage.googleapis.com/sfr-vision-language-research/datasets/nlvr_train.json',
+ 'val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/nlvr_dev.json',
+ 'test':'https://storage.googleapis.com/sfr-vision-language-research/datasets/nlvr_test.json'}
+ filenames = {'train':'nlvr_train.json','val':'nlvr_dev.json','test':'nlvr_test.json'}
+
+ download_url(urls[split],ann_root)
+ self.annotation = json.load(open(os.path.join(ann_root,filenames[split]),'r'))
+
+ self.transform = transform
+ self.image_root = image_root
+
+
+ def __len__(self):
+ return len(self.annotation)
+
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ image0_path = os.path.join(self.image_root,ann['images'][0])
+ image0 = Image.open(image0_path).convert('RGB')
+ image0 = self.transform(image0)
+
+ image1_path = os.path.join(self.image_root,ann['images'][1])
+ image1 = Image.open(image1_path).convert('RGB')
+ image1 = self.transform(image1)
+
+ sentence = pre_caption(ann['sentence'], 40)
+
+ if ann['label']=='True':
+ label = 1
+ else:
+ label = 0
+
+ words = sentence.split(' ')
+
+ if 'left' not in words and 'right' not in words:
+ if random.random()<0.5:
+ return image0, image1, sentence, label
+ else:
+ return image1, image0, sentence, label
+ else:
+ if random.random()<0.5:
+ return image0, image1, sentence, label
+ else:
+ new_words = []
+ for word in words:
+ if word=='left':
+ new_words.append('right')
+ elif word=='right':
+ new_words.append('left')
+ else:
+ new_words.append(word)
+
+ sentence = ' '.join(new_words)
+ return image1, image0, sentence, label
+
+
+
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/nocaps_dataset.py b/sd/stablediffusion/src/blip/data/nocaps_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..ba0bed06d8af3dbaccf18a56e725f101e585503e
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/nocaps_dataset.py
@@ -0,0 +1,32 @@
+import os
+import json
+
+from torch.utils.data import Dataset
+from torchvision.datasets.utils import download_url
+
+from PIL import Image
+
+class nocaps_eval(Dataset):
+ def __init__(self, transform, image_root, ann_root, split):
+ urls = {'val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/nocaps_val.json',
+ 'test':'https://storage.googleapis.com/sfr-vision-language-research/datasets/nocaps_test.json'}
+ filenames = {'val':'nocaps_val.json','test':'nocaps_test.json'}
+
+ download_url(urls[split],ann_root)
+
+ self.annotation = json.load(open(os.path.join(ann_root,filenames[split]),'r'))
+ self.transform = transform
+ self.image_root = image_root
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ image_path = os.path.join(self.image_root,ann['image'])
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ return image, int(ann['img_id'])
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/pretrain_dataset.py b/sd/stablediffusion/src/blip/data/pretrain_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..703d543ab5267fdc6fe2b7c84ef6a631d8af90ad
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/pretrain_dataset.py
@@ -0,0 +1,59 @@
+import json
+import os
+import random
+
+from torch.utils.data import Dataset
+
+from PIL import Image
+from PIL import ImageFile
+ImageFile.LOAD_TRUNCATED_IMAGES = True
+Image.MAX_IMAGE_PIXELS = None
+
+from data.utils import pre_caption
+import os,glob
+
+class pretrain_dataset(Dataset):
+ def __init__(self, ann_file, laion_path, transform):
+
+ self.ann_pretrain = []
+ for f in ann_file:
+ print('loading '+f)
+ ann = json.load(open(f,'r'))
+ self.ann_pretrain += ann
+
+ self.laion_path = laion_path
+ if self.laion_path:
+ self.laion_files = glob.glob(os.path.join(laion_path,'*.json'))
+
+ print('loading '+self.laion_files[0])
+ with open(self.laion_files[0],'r') as f:
+ self.ann_laion = json.load(f)
+
+ self.annotation = self.ann_pretrain + self.ann_laion
+ else:
+ self.annotation = self.ann_pretrain
+
+ self.transform = transform
+
+
+ def reload_laion(self, epoch):
+ n = epoch%len(self.laion_files)
+ print('loading '+self.laion_files[n])
+ with open(self.laion_files[n],'r') as f:
+ self.ann_laion = json.load(f)
+
+ self.annotation = self.ann_pretrain + self.ann_laion
+
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ image = Image.open(ann['image']).convert('RGB')
+ image = self.transform(image)
+ caption = pre_caption(ann['caption'],30)
+
+ return image, caption
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/utils.py b/sd/stablediffusion/src/blip/data/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..628894844becd462d444584b8b2b01a84ee4b8f7
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/utils.py
@@ -0,0 +1,112 @@
+import re
+import json
+import os
+
+import torch
+import torch.distributed as dist
+
+import utils
+
+def pre_caption(caption,max_words=50):
+ caption = re.sub(
+ r"([.!\"()*#:;~])",
+ ' ',
+ caption.lower(),
+ )
+ caption = re.sub(
+ r"\s{2,}",
+ ' ',
+ caption,
+ )
+ caption = caption.rstrip('\n')
+ caption = caption.strip(' ')
+
+ #truncate caption
+ caption_words = caption.split(' ')
+ if len(caption_words)>max_words:
+ caption = ' '.join(caption_words[:max_words])
+
+ return caption
+
+def pre_question(question,max_ques_words=50):
+ question = re.sub(
+ r"([.!\"()*#:;~])",
+ '',
+ question.lower(),
+ )
+ question = question.rstrip(' ')
+
+ #truncate question
+ question_words = question.split(' ')
+ if len(question_words)>max_ques_words:
+ question = ' '.join(question_words[:max_ques_words])
+
+ return question
+
+
+def save_result(result, result_dir, filename, remove_duplicate=''):
+ result_file = os.path.join(result_dir, '%s_rank%d.json'%(filename,utils.get_rank()))
+ final_result_file = os.path.join(result_dir, '%s.json'%filename)
+
+ json.dump(result,open(result_file,'w'))
+
+ dist.barrier()
+
+ if utils.is_main_process():
+ # combine results from all processes
+ result = []
+
+ for rank in range(utils.get_world_size()):
+ result_file = os.path.join(result_dir, '%s_rank%d.json'%(filename,rank))
+ res = json.load(open(result_file,'r'))
+ result += res
+
+ if remove_duplicate:
+ result_new = []
+ id_list = []
+ for res in result:
+ if res[remove_duplicate] not in id_list:
+ id_list.append(res[remove_duplicate])
+ result_new.append(res)
+ result = result_new
+
+ json.dump(result,open(final_result_file,'w'))
+ print('result file saved to %s'%final_result_file)
+
+ return final_result_file
+
+
+
+from pycocotools.coco import COCO
+from pycocoevalcap.eval import COCOEvalCap
+from torchvision.datasets.utils import download_url
+
+def coco_caption_eval(coco_gt_root, results_file, split):
+ urls = {'val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_val_gt.json',
+ 'test':'https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_test_gt.json'}
+ filenames = {'val':'coco_karpathy_val_gt.json','test':'coco_karpathy_test_gt.json'}
+
+ download_url(urls[split],coco_gt_root)
+ annotation_file = os.path.join(coco_gt_root,filenames[split])
+
+ # create coco object and coco_result object
+ coco = COCO(annotation_file)
+ coco_result = coco.loadRes(results_file)
+
+ # create coco_eval object by taking coco and coco_result
+ coco_eval = COCOEvalCap(coco, coco_result)
+
+ # evaluate on a subset of images by setting
+ # coco_eval.params['image_id'] = coco_result.getImgIds()
+ # please remove this line when evaluating the full validation set
+ # coco_eval.params['image_id'] = coco_result.getImgIds()
+
+ # evaluate results
+ # SPICE will take a few minutes the first time, but speeds up due to caching
+ coco_eval.evaluate()
+
+ # print output evaluation scores
+ for metric, score in coco_eval.eval.items():
+ print(f'{metric}: {score:.3f}')
+
+ return coco_eval
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/data/video_dataset.py b/sd/stablediffusion/src/blip/data/video_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..0a6f8a61105bbd4285f98b3abe9445b73fd4c7ef
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/video_dataset.py
@@ -0,0 +1,110 @@
+from torch.utils.data import Dataset
+from torchvision.datasets.utils import download_url
+
+from PIL import Image
+import torch
+import numpy as np
+import random
+import decord
+from decord import VideoReader
+import json
+import os
+from data.utils import pre_caption
+
+decord.bridge.set_bridge("torch")
+
+class ImageNorm(object):
+ """Apply Normalization to Image Pixels on GPU
+ """
+ def __init__(self, mean, std):
+ self.mean = torch.tensor(mean).view(1, 3, 1, 1)
+ self.std = torch.tensor(std).view(1, 3, 1, 1)
+
+ def __call__(self, img):
+
+ if torch.max(img) > 1 and self.mean.max() <= 1:
+ img.div_(255.)
+ return img.sub_(self.mean).div_(self.std)
+
+def load_jsonl(filename):
+ with open(filename, "r") as f:
+ return [json.loads(l.strip("\n")) for l in f.readlines()]
+
+
+class VideoDataset(Dataset):
+
+ def __init__(self, video_root, ann_root, num_frm=4, frm_sampling_strategy="rand", max_img_size=384, video_fmt='.mp4'):
+ '''
+ image_root (string): Root directory of video
+ ann_root (string): directory to store the annotation file
+ '''
+ url = 'https://storage.googleapis.com/sfr-vision-language-research/datasets/msrvtt_test.jsonl'
+ filename = 'msrvtt_test.jsonl'
+
+ download_url(url,ann_root)
+ self.annotation = load_jsonl(os.path.join(ann_root,filename))
+
+ self.num_frm = num_frm
+ self.frm_sampling_strategy = frm_sampling_strategy
+ self.max_img_size = max_img_size
+ self.video_root = video_root
+ self.video_fmt = video_fmt
+ self.img_norm = ImageNorm(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711))
+
+ self.text = [pre_caption(ann['caption'],40) for ann in self.annotation]
+ self.txt2video = [i for i in range(len(self.annotation))]
+ self.video2txt = self.txt2video
+
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ video_path = os.path.join(self.video_root, ann['clip_name'] + self.video_fmt)
+
+ vid_frm_array = self._load_video_from_path_decord(video_path, height=self.max_img_size, width=self.max_img_size)
+
+ video = self.img_norm(vid_frm_array.float())
+
+ return video, ann['clip_name']
+
+
+
+ def _load_video_from_path_decord(self, video_path, height=None, width=None, start_time=None, end_time=None, fps=-1):
+ try:
+ if not height or not width:
+ vr = VideoReader(video_path)
+ else:
+ vr = VideoReader(video_path, width=width, height=height)
+
+ vlen = len(vr)
+
+ if start_time or end_time:
+ assert fps > 0, 'must provide video fps if specifying start and end time.'
+
+ start_idx = min(int(start_time * fps), vlen)
+ end_idx = min(int(end_time * fps), vlen)
+ else:
+ start_idx, end_idx = 0, vlen
+
+ if self.frm_sampling_strategy == 'uniform':
+ frame_indices = np.arange(start_idx, end_idx, vlen / self.num_frm, dtype=int)
+ elif self.frm_sampling_strategy == 'rand':
+ frame_indices = sorted(random.sample(range(vlen), self.num_frm))
+ elif self.frm_sampling_strategy == 'headtail':
+ frame_indices_head = sorted(random.sample(range(vlen // 2), self.num_frm // 2))
+ frame_indices_tail = sorted(random.sample(range(vlen // 2, vlen), self.num_frm // 2))
+ frame_indices = frame_indices_head + frame_indices_tail
+ else:
+ raise NotImplementedError('Invalid sampling strategy {} '.format(self.frm_sampling_strategy))
+
+ raw_sample_frms = vr.get_batch(frame_indices)
+ except Exception as e:
+ return None
+
+ raw_sample_frms = raw_sample_frms.permute(0, 3, 1, 2)
+
+ return raw_sample_frms
diff --git a/sd/stablediffusion/src/blip/data/vqa_dataset.py b/sd/stablediffusion/src/blip/data/vqa_dataset.py
new file mode 100644
index 0000000000000000000000000000000000000000..92ec1df429b3910316ddd554bfea01c6e7922cae
--- /dev/null
+++ b/sd/stablediffusion/src/blip/data/vqa_dataset.py
@@ -0,0 +1,88 @@
+import os
+import json
+import random
+from PIL import Image
+
+import torch
+from torch.utils.data import Dataset
+from data.utils import pre_question
+
+from torchvision.datasets.utils import download_url
+
+class vqa_dataset(Dataset):
+ def __init__(self, transform, ann_root, vqa_root, vg_root, train_files=[], split="train"):
+ self.split = split
+
+ self.transform = transform
+ self.vqa_root = vqa_root
+ self.vg_root = vg_root
+
+ if split=='train':
+ urls = {'vqa_train':'https://storage.googleapis.com/sfr-vision-language-research/datasets/vqa_train.json',
+ 'vqa_val':'https://storage.googleapis.com/sfr-vision-language-research/datasets/vqa_val.json',
+ 'vg_qa':'https://storage.googleapis.com/sfr-vision-language-research/datasets/vg_qa.json'}
+
+ self.annotation = []
+ for f in train_files:
+ download_url(urls[f],ann_root)
+ self.annotation += json.load(open(os.path.join(ann_root,'%s.json'%f),'r'))
+ else:
+ download_url('https://storage.googleapis.com/sfr-vision-language-research/datasets/vqa_test.json',ann_root)
+ self.annotation = json.load(open(os.path.join(ann_root,'vqa_test.json'),'r'))
+
+ download_url('https://storage.googleapis.com/sfr-vision-language-research/datasets/answer_list.json',ann_root)
+ self.answer_list = json.load(open(os.path.join(ann_root,'answer_list.json'),'r'))
+
+
+ def __len__(self):
+ return len(self.annotation)
+
+ def __getitem__(self, index):
+
+ ann = self.annotation[index]
+
+ if ann['dataset']=='vqa':
+ image_path = os.path.join(self.vqa_root,ann['image'])
+ elif ann['dataset']=='vg':
+ image_path = os.path.join(self.vg_root,ann['image'])
+
+ image = Image.open(image_path).convert('RGB')
+ image = self.transform(image)
+
+ if self.split == 'test':
+ question = pre_question(ann['question'])
+ question_id = ann['question_id']
+ return image, question, question_id
+
+
+ elif self.split=='train':
+
+ question = pre_question(ann['question'])
+
+ if ann['dataset']=='vqa':
+ answer_weight = {}
+ for answer in ann['answer']:
+ if answer in answer_weight.keys():
+ answer_weight[answer] += 1/len(ann['answer'])
+ else:
+ answer_weight[answer] = 1/len(ann['answer'])
+
+ answers = list(answer_weight.keys())
+ weights = list(answer_weight.values())
+
+ elif ann['dataset']=='vg':
+ answers = [ann['answer']]
+ weights = [0.2]
+
+ return image, question, answers, weights
+
+
+def vqa_collate_fn(batch):
+ image_list, question_list, answer_list, weight_list, n = [], [], [], [], []
+ for image, question, answer, weights in batch:
+ image_list.append(image)
+ question_list.append(question)
+ weight_list += weights
+ answer_list += answer
+ n.append(len(answer))
+ return torch.stack(image_list,dim=0), question_list, answer_list, torch.Tensor(weight_list), n
\ No newline at end of file
diff --git a/sd/stablediffusion/src/blip/demo.ipynb b/sd/stablediffusion/src/blip/demo.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..3077a1a42c584f3ef535903f64cf6b3bb722490e
--- /dev/null
+++ b/sd/stablediffusion/src/blip/demo.ipynb
@@ -0,0 +1,301 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2b949f9f",
+ "metadata": {},
+ "source": [
+ "# BLIP: Inference Demo\n",
+ " - [Image Captioning](#Image-Captioning)\n",
+ " - [VQA](#VQA)\n",
+ " - [Feature Extraction](#Feature-Extraction)\n",
+ " - [Image Text Matching](#Image-Text-Matching)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "id": "cbcb066b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# install requirements\n",
+ "import sys\n",
+ "if 'google.colab' in sys.modules:\n",
+ " print('Running in Colab.')\n",
+ " !pip3 install transformers==4.15.0 timm==0.4.12 fairscale==0.4.4\n",
+ " !git clone https://github.com/salesforce/BLIP\n",
+ " %cd BLIP"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "id": "a811a65f",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from PIL import Image\n",
+ "import requests\n",
+ "import torch\n",
+ "from torchvision import transforms\n",
+ "from torchvision.transforms.functional import InterpolationMode\n",
+ "\n",
+ "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
+ "\n",
+ "def load_demo_image(image_size,device):\n",
+ " img_url = 'https://storage.googleapis.com/sfr-vision-language-research/BLIP/demo.jpg' \n",
+ " raw_image = Image.open(requests.get(img_url, stream=True).raw).convert('RGB') \n",
+ "\n",
+ " w,h = raw_image.size\n",
+ " display(raw_image.resize((w//5,h//5)))\n",
+ " \n",
+ " transform = transforms.Compose([\n",
+ " transforms.Resize((image_size,image_size),interpolation=InterpolationMode.BICUBIC),\n",
+ " transforms.ToTensor(),\n",
+ " transforms.Normalize((0.48145466, 0.4578275, 0.40821073), (0.26862954, 0.26130258, 0.27577711))\n",
+ " ]) \n",
+ " image = transform(raw_image).unsqueeze(0).to(device) \n",
+ " return image"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f72f4406",
+ "metadata": {},
+ "source": [
+ "# Image Captioning\n",
+ "Perform image captioning using finetuned BLIP model"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "6835daef",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAERCAIAAAAmJE0sAAEAAElEQVR4nOT9SbMsSXYmiH3nqJq53+FNEfFizIwhByAHFBJAAawBBaBQ1V0sVkt1C4UtLWyyueGK0iv+EG4owh2nRZOU5oIlTaFQqltqLrALKCAHZCLHyJjHF2+87w7ubmaq53ChqubmNrm53/teBNCKxAu/ZjocnT79ztGjavRP//iPDDExExETGyIiYmYKP0DtwOuHAJgZAIM4/AkmhJ9kiAAiApEi/Br4N/wIof7dektEqtp9CyA9AqWfiihEJ8919M1XCgAQgABKf26k7Q0tqbqpet+ORAgZ1hFGkjejDck2LsNIwpD5Tq+uJP+/ROEyVRjvkVbPbi2oG7837R7NvlP8utzxVNPzHIo51D7WIAEZs8E2INt8CABEDLCCANKAPeCIF0rUnqjYBK/mjG3CE/pmaQv1Wq9UFbSJbp1GCHFb8tQNUucffo+3IHYfHL3w1KxvN7feRtiKYlsz2Ts8CQy6Qmj7YqLkJaXaChDj61kYb3sI8IRacoowdYSJkyIES0wcEAzE6Q8CMRE1YIt7YI2IiHWNMcRECGCniZH1c7Huj9637Yd1ZeqHsaieGvYBB21G0/S/EB/YRNUpLdiLSq0IvYKNdNIWvMZazqFovc+ns7O9R/9QaOU2TvqaYL2fDM2ETwLdWhnWrdp9Hp5Mn43dt3WvTa/I0HjbKZNm5KFU46tyN9XWEVgXV+cwReY6gmUOKEZMgXRhjWKIGNbCtfUfNWIFUqNg0i6EdRGqi1xNubtJkPCr3U9AzcW6bdrMWVWDOhwarfFvf8Lxtrt8tNZk2wqUW1Gs9bAbrR4i4wIPTb9WhKbY44rMOHar6pBu0ipoSKRmSzYTdn+PCDkUeinwxFWkJdiuE7ub58gqtTWfZibTgb7b40Oy9SZsVbw1qcchvvt769C1hpQCK6MAZGDShF4gAkF5jVoRqhjKqKkYEcANItaqTOvPrpTjdYj/Dle7a0cbqi0AQFoDdARNuslb87Yu/fJKROvJCJo0Zb5kuUPTvre+vcknIvv00B36Q2KMyD+xiN58uqVMzHan8ISyfRLhL4uotlYf11okM9IyThQM9wHFqI4ZgQwEBQOkaNKx7o/ef0MIv7lWFptkrRF1GxnZeJ+mWf1Amz/GwXS0lDY6j9Sry8DH7Vzd58343TW/9XyI8HdTTWR2LbFHZG4lHMLB7vMpKvOQeFv10OlA/4TweryjL7MItTTxvQXbVYb9Fu+h+g4N+JF8xqmuJYq7mJFhRWSpiRqB4mYlRR00aZQJbTjsIA5DWHPa90IAJ+YV3zYharPafTAU/hKAOw1EgAKSsu9Z8OtGnDImRuJ0Xw2hz8TQ7a3mSJouW3fQN/OZMleHoGpiFfZ43guC42vDFOLWTLLrVLyS0Gp57ItorSVk76xaDT6xWerSx20L42WNxJkSoTeOZQrbmMHcj2D871r6UWNZQLFgVUPcsmxFw0Bz9/yLhkW/jtxJ0orQ+F1vTXahBA3T/m5rfuvVZUbzlPExEmcrRO5X4v/AQy8IXmG7bSXgT0I3b+a8X0X2pmm7pro86vXmY5t4Fb0zNiGsTpmADA1G1iBTHWm6ObSjTQC+bp6pt8AcrNQAwshoLeZ1If2lj7RR60nvyBvPsLmA9FZkaLhPsf1NEX685YdifkHCuMC9T5q/uzpsM9rQ211nzq6Re6doq1KXt7p25850qjVEh6ekHQrd8d80dAxlPhHsWiFiGTNHP4xNFIuusNFABiQgo8iF1lxs5Ac2Z06kXSPCdkTvGxmawFCDI1sdJRG+fUIv7ozH2S+3oVfMvPeA/gKi0hchjMxJmqx17qRPTZFnvIgvQthb/x0Ba2wCwkS4nFii5YaG2Q7JBBUYGYhYQcQAdVXL+s9uBXibvjn0Z1NcIgIEAHOgYCGChPehRpeBnlbRe3Cx6WUN5T8lk654XYp3GVC7vPrTuxG5U857W7Vb5Q61yZQVqzfb6fsMewjf6tkuyO6HoVPwotXgl7Hi4dJYj2kyd0u0HLzIEMCqAUzBzQK0Jl8aUkc3rSaQ9f6bfsTc0BlkXRDsxAlNHKoU/1UNeTaxfyNhd4+vWeHu84lhV+gZL25Itq6dvrt302yuEQH2C91m3DuH7vPL77FcPgwBWWv+dCdkrRZ1UeZqDW1NAS5j/2oG2lTrun+2Infl2aO46ZF7y+qu1uPbApaJkr9FxBQG1fpjgisKzIeRNMQWkFGMsEav+s9RCtY3aGtrvST+RZ0Ibahq5lPn35o5U4ZFF1IvEyZm0ou8TyFsHRy9T8aNUFtL3FHGL1DYT/hdl4QpK99lwq4o83TgbGtWzR9D0GZTpABPGogYc4NbEaITBoG0oVqGhADCJkBzQgZo25SjlqbxRBqIJylREBSN7PvzGXo1nmRoiqIzS8f5xVCEoQk/JeehgtDpzisJNRMcz5M2OWNXts/F0NM0IV9hts3B0EvNur/rHm+xnvrtlCExLszWEjGtH5s/eqUdSbVVS52++A0VNN1k2ZutrX3HCMQAJ/WtpmkJyIgBorUb7YYVrIUaHayJf6ZmTxBWx6m1QmjjjHczh6E/t4LaRKTbkDPINGGrsZXbkKkIfWNiKP/W4tMr/MQwZapvjdA7wj53Q/XQyrQ17KTnjgy8bgcNKaHTDV5TwKgXT7emHQk7ceoRi+F4wikFDWH3UD6t9rQATHDf5xqeGkCWUCcwLxoGmvF/IwXj8Hvtkd8dMVuBbCRc7So9ERn3LnHiKnqZcLWc5crD1rm9a2546iDbHHLd6lxtBa88jDPKoSRPp0Z7GAqtCf4WyT8WtQpJ0YzGDQpGm+5azV7chKeQWDcjA40jROGfVibNamAYTSYCSjfDkci9z5udPaXE3rStyEND//K4c+XjbOJSPDFCl7pOMb1Nr9QlbXbNXt6Dz3blfEK41jtf9si/OctajHKiANP5Zivt9FJ649cP2/aycP6ovkNiDWRIThrhOTaMZZs/gI2bcyj4fCWoWr9F3wjorcP4qynPh+CjG20PQ8BQ6HqHjaBh/fCLs3S38Hc/u/V06Nk6/59mmLLaTclhqApfnF4OYai+O2HiF2oA2xY2EdWbkvGGRYRHa/CS4DEWatFkSem31pnVz1v/og/URgbTTgC3E+RhoCO7JU5cqHshYHwN32/mdMOToDldC+AIwLWarv5zpFDqaDdd9N+VcWwNV55hK3MMTO9uv08RY6vl9PMClNbUGLKrTp9Bu5be4WXdENVJ0PpGHyQHNKDvVNDWNW0ivkwHpumRd8qzFw6GIu+Rf/12Sr9efsp1k0/ZsWr+OYSPTehpytmFPN08xN5Cul4ZxhnreA5TwlOY9t1FcTzaeJyJJaKz0uxR06tVF6ZMyb0Lasm59vaPr9PojPeRgUijT0Yjh0b8YURr5TxewynzfzxC71ScCBlb4+waeiFgqDWeUHhyZV35Alv/nj6se3X58XH1NBt/SggWn63L+VMLveNzD3me8jivgw3Ctgz/rBQ3A7CBYRP/bYXeCL2DbISRjvzZfdW7JI6071bA3XXMXUlftuxWEy2mTTsuDTCm+s9xhXGoT5sMq6s0DbXbRKrSlK37sLfE+mHLft+byVPTxXrbDZ2KtLppuhG9Nn7vhNdbW2Bohk7hj/u1bW93T49ZP7TU+HxReGE0KZTxtFCbU9S/R0Cq+7DVNNOhaqhKQ6E1nbbi1JRX3Qy32sUnRtsaeudks5TmZOgiV3ed6JWnu4oM6adDS3cvy+iVdqQWvdn2itRbQUxD5z0kvJIwPupaSxdGp/dOq8J4id2cW6tRK7fmGBtab6YU1CxrIryOZEhEduNjSwBr2IAkauBRLzSMPOyFsz3CFBgagq39BuX0wTERDYeijYg33rvdGY6BSX61YTo3bIYnIcx+cDmUz1NouolhCEF2DRNhDpPbamRt2FW2KWVNQfDeaJZIkzqJeB8G0sUYm2U0/63f1l4IvXDWSt77djxmb5yRVM1qD0HbThxwerQpK0wrfmuJ6133Wt02olYM8Y6W0b23UkMtPLJK9ebQYn8junlL7Ilrw0ho5txLLnqLRqMjpmvxU+SZLnBTmJFCLwk93ThDHLk3Zu9QGZd5epze4ddLvkYiWMSPGYE1/uRNlbMeZ10gG5Kj9WrKMG1JtnXtnRLtkmEiNu0KYSOhF4yemn3nixku38VX24CfS3f8FR4DQwxjovrZDOkuxnRnP693MvuX5a0r8xB+TcS1cTq296txc0xvJuNcfQqdHA9dttUSrF6Kh/JsRehd50OXdTOZ8mSPMH2d2ylOM3T16xZT2MoImi2zU9FPNGylaS2DWou/j1PR3oKmc8yRWTBkdMPmjB63mXSf7MGRbbjnGum2shaQ9TKyZlv0zqLuxB5nc+OZDNW5W7GJz8fhcisq7ToBhiCj7rPW6NxqpEejxeq+HAHrXjgbb5adwnQDzfQMxyfhiNG6NWJ7tfhuyw/Nn+5cnT7/d63dSNHdQrca3fdYqKaDSG9B43pVd+3p1mh8fNbPWwXVfzKB4/eW4isigBTxX12nmQIuE6WZPu57G+UK5+FfutCq/tbGfAots1OHXj5cCX+cHq68rC/UWN1pwb4qVTewyImStN6OJLRE1Ni1JAIad2QgQhso7m4iROnnaN0/uyIO/dkbf2tuGF4nd8W7iatWHUa4Q1cJGhG1t75bTYcjOwAT5f8ih516qnds9HLb3oRNjtb83coHDVVuSJ69GdwQsR0HlFraS0JMr/xDc6rXNIFOY44X19pyGQnd9h/J0xKSYglqELSNSsYfCiLSxmnMCHGND/0OldRlhj3REIGymxydQTm0sbLTOjPyfI+EvR05DmToUO4Wx9mqbX1xwla6tB+f2i9Vtw17tfiRspr90h17mDAVp0PMxAq25NkJwoaKaCnaW4frFMBtitfb1FPgqRWzK2E3JjdZWP20WVgLMtaf+w0dikjkmvFr9BnKtvUwvRpcby85dZ/CzB9pwPBjfE5+AbHpr0zods301h6P2ezc6bntN557V+76ya4L3lVpi7uGXlY7vTXGLbMWiF+HawJLN0HPQwrfx1yTOQKSo21PDuiMqrYs1B+zN4duVlNsh9RHkrsRtmaCjhrYVWp6jfTNDJt6zXhZTyHsyqqGthpGUm2tS28RV9UCU1Cp+++4JaEp3ghdogYf7I2wX6BpxLA3WqtTxis7XZ5m0TuN5611acnZG4fTScyNYdOaY709XWMYYb1L0CvpELSNyz3+ezzV3tEuM84ub7n4qxd2mhtbB/TnG1qGqumjdOue45WEy4//KYvTFzlwBDKAWGuzf3yXvmKOviULHW5CyUmtGae341tAOWVYdFu2RXmGMmyGEWW7m8NE6Nza5a0IW00247k9oTCxFq35vFNoYf1Wc8xTm0sjbU6bJqTekY8O4br8IJwYto7SXlF3YhVT5kJvwm7RVxWG5LEAqDblN1wzupK1ednAZKa0RdBb3ggETESxZjP1IuxItkMRxvXwKflMR7SmtBM1tSsJU+q4X7YjeY7omBMn4VNAtO5isxVSu3Vp7i3sF3ZKO2JTn7iNOEWYIeyemBs1bC9bC9219ZrVDwl5kyIBtGH36Z2Bg71LEQyVNp93yv6ihZ1km65LXnJwX0m4DJPqhvEVaL9wySb63Fu4FVrL/CXbZ+8VeihMXCFGSGg3ty9ICLyslpsQlM1oCNsOZ+sK17uZGjcT6lr25oDO25HfzdDqjJFe2fqqqe5NHzTNoscXpaFXO8XpDd1leRyz9rajX6E9eChDGtiQGad74xFGwtYaTcy25bQxOC/69ny6EboC7L1R0M25y4yGsh232V9GmCnr+n6l1PFt/xoSvlOi7R5qybdOGBFs4wbaWtlEXx9PGbv7aShbW6TVvrtKUv/Z/HdrWVvDroaMnR6OLAwTJ/beoEYTtob3yLP7uynhiLRXpR9MzKc7h2nz+CRtmjiucAepi6cje6ytJL35DC2Tu4o0UQueCG21VOtv9278aDiuDk2G1jpP6Gxi0oay2ZtJ91Wd7ZUzgktG+4LQ6UtajkcW5PGEl6GQW+NcFb50M/yC9NoXLVx5+1xtDzaxZXrOG/ayNkKTgqRpQWvh17qkpF1uvMImPg4YEVpy15DcG3+EZfSK1xttaOnr5tCM2eUC45Rna+gu183n9dvmn1MsqeMLe3/3PQE02S/sMbXGleut9erdkdwpTBlyrfyHemGPzKeE1hx8Ep0+fcW6JEfB8Ajf4GXNItPvfYbIurH2E7aRT6ugofm/X7ZbO3UKOO4dhrjeyEDvjoZereELAkxPJ1yystNh5UmHpuLZfPiECrrybJ9oGGI/zWB75/PmbNdkE+tHvW4OTWto3EfQfmDbdYmYOP+3Pu8V4PIrxn5hylo9vNh8bmJPD9MlfHIVmSjD1W77joSh/EdMWkODfygM2YJblP+SpewRv/53J7vY1rCxjxn+rb9hvi6bNGwE1AawOn53HrYAjuvzTZvSdy2OzayGpm7zz27M3mjd582+HCE4veFqgWO8ChMF+AICWW//fr7h8mB6JV3fWu+xufC3yuq1Nlw+tMZ8L5R0Eap3xW1G7nZ6c7Jv3WRrJdfOvt9W0ORWK46ACBDcNfqtRZQCOnCjjU2Aca44XtsnHaaU9ZSn5ZXrAl8EWNkvTJT8L5321A1XUoW9kfcyCVtPwgQfAcGrDcyabl7sIAsagLV+S9Ibp1bCW0CGyMn6c+6i78QwErkWtbvUNKsTHj41naIVNIXWw9bv6XJO55V7SLtrGNozuWSeU17tUVxvzr0zc4owE0Nz/Debq/ujd2JeSaixpjvYRuywQ0aeLrUcomMTZRsqZShDS0T1xmEro+7Mjw8bdenywB5RFIRIzdqUbcdR2G2vEWY3Erm7Vjwhq1MrW214FY0nbGH9xPhXGOhybihdFL6qnEfKutrkI0NiaOTsOoq6nTvSbjuF6WJcVYnom9RPKFCfpYiBtWtYE/aGcBGNwwDT22vI0WwKvRoJu4oxnlXvcvQ0wxB7vZKsdg1PQr19omP9KevjT2fGThTmaWoV02flrt09vb69qaJPRvpwSTuLHrKWXNJar7rQQ82gaH5zc5zAD6HYyFTvFaArzB5FbA01Ak7ps53o1WUG6NMH5SnSDqknn68A++XWombTZdhJgBoOWkXstOxNXKR7p/B4ziNZjUg1ZZ7uoScxgLTTqNxydu2vyRgMjYRAzRh79nZvKV8oQ+/4YJoY+QtSo/0w/ZIRnkJ4EvxligJxhaG7Y4jRtt26zHfjt7K98o7bmuF+TRq/XVK76EOh3JNNI3cFlCKhS6k6kbvpoQnGqPXf0YT7suvW8ykwP735tm5Z0Ob2zfhCNNS1410+knCPaN2ixxNOz7YZs15yW2Xtms9l5HlCYWi3bid+0Wr23jHWNNVrn+tGr2DTq/CEwq4dNAXvWpGjr2z9yZI6QhMa+xQfBVQBgmlN7FanbnSkRiJYSzCOCHs8r2Vo7SrU8XflrvVw2Un723sAjdubm4N4YhG7DtCtc2PXMGW+Tcxn77dPIewxUIcid5dhbC4DvZogJqxAU0pvhW7pV5LtFYZaqs2bYwkU7pedOm83jHx1/MEk1FFQ9wpPk9I3B0rr1WVm19DbnRTVzzG0hBnvkadsJv8rFkYm1GVq/eQ65fJ7aE1mOr2Omy7+6XoM1vWJo15dKang6+S9pbbiY+DG4d4QInTnzNZUtcDjIu0UhhRJ7IVKl5x4O9Hvy2e+R247cdg98uyGLxTQt8IVytY7pId2HrbOlK3F7UHKdMeb8i7fOPVM580/I+I0ydoGcWtno1FD7SugT/AemtOt+ZS26DLBlsxbJNk9DI2hPfLfFf52LWK6BWpKEVNMAa3ebAqw1eo3knPz4RcZrcbDTliwU6jzHFHhd0KWKWn3hsheejHU73XM6e3GgNKmK3996f+4cLEsUmIJm5tbQYQasHdV8/ZzDJckLN2wx3DfOjImjrwpGnFrSevNuQVqW3v5qVnBrmRcjU/Uy+e/d6DhL4p3wyXNXrsyr6sNY13w8O6nRETEYfhtJGACEVQTbg2MY0qf10wq5/ZlnMB1oglSjsdprd5X2MpXm9sXNtDnvQm4a5giMG0aNP5yVfAyQQeOmgyxtl1b5kpacusa1sXluhbN6jRjcief9fIbP8+0DYOpT8cckTiCesp+Svi81sNxzevJldsb9sb6rWFKXXoZX3fR2qpi7CFeN9UUgafruVcePl/cpB23uT+X1foyhTZr18ynvldWezJXJd1SamMQo9Y0RyReP9mlLr20tndu7Kpj95bVzaFlmMATxtARwbbGaTHoKwy9Kkyzwder4JUqdDvtau2Hy1cbpuS/67b49DDkP9Rqvcs0wkRMGJFwPFrvvK4hbGgYcMOuoYDQZgBAsmUnMTxR1aaWig6ojQzENgBpO9UIJvZCz/R+Gtq269ZxSs5by73CST6e1ZVsR27NszUoNYWh5LtO1y56NnMYHxgjYWi2X0a2vcOVgxptcyquS3lCcDZ9n31kgvemqoGit4jOtYtoX+kDIJ5tWmPWhkC9f2K0SrT5pq0ObKtXb833mCpXHrYKcMnJPCWTq7X4bg29zb53Wc36PrUO3a+UK5dt4nrZK8nEtbxX2/jcZ81OYaRxGO11vg3b4d9wjrJ3QWug22CpLRWXiCjcm5b+1xEDmNbKdaqJ6N7K+apW1ysPba7aCUOvdmqH8YTjfbF3QSNh4rxqrs/7ifEkeOv0onWbm8tOy8M4YZkCWCNtONJQVzsApmc1pJfYTrxo8EfLsQhAUv4GuyFcpQ3uNYYNyZqeT1pYetnfHg36JAbuCLenzeNHUyJPJOq90XrHX2/MViYtiG8pdC3Jt1ZkouTUd8PyeGgT+R1Ln9ho+4WtFZmoT11JqvrhePKRSbRTuZdhuCMCTBzP3JeB9muaIJ4CUqQg0c0cmvDXP1F1WDnqfXrpcOXUegqHGkKNqxVsep7NYTQRIHr1lN4leuKCcfmZP5J2ykzeL+ehV18QlW0rBDRb5mrbZ9eREPW0vs+/Yxshbf5pgfV3fesxHZXAjgZKSP+kYobaIu4lbL7pRt7gqw3Y6lLCifVphUsuOE8htOjJHgknPu8NIyt5688hFtZdk1vEbVzCPWjd9NBdOXql2kOGPWTeT4G4TPIW4W0lr/uojrafYN3W6/bvk+hi2oQsRJ+M/mr0PRzxJeumJRkiVZfEkSfUOpcMe2sBW98OIU7vArhH245bZ7baFoeG8hRJPpd+7J3VT7nQpxC2WuWejgwTXw1xsemSc0hK/Vaw7ifXCNDaVD9MJpMotDNCUwq9z7GmjZNqODRq9x6+uyoXI9Om++ryVGvEUtsrW6vQ5io3RBibikCTgrVIdzPOuBi15L1dP65ijNdupKyJMT/HcEkhe4Gspce1uruXj08RoNVxQ52+XztPH/xrLRINwwdFhU8BofVhcyIgsDjWtl/FgKxr01uzHXvxiIgYBO3ncvWsG8Lv8dBSkfYevtNL3NqRQ/xlomyhQbrYMZ5Vs9mnzJYmkGkjdHGwNRlay8ZWkN261E1spaHaXSbmxHC1mDie2/TajScZmU3jf47L1ouPE9F572bccmlPkGG/rPeUSZU+v/3yncI442jO4S6Gtvp4P4AeKXcoq/2gfJwubWVAl+y+oUa7TJ7/Aw+0i4lwv+4LY2b64nSZELLaOA0+lLeq7yIaATzBAEQM0G5aIYGaHme43MBtosllbJxDOU+Rbatu2zvnW7x9a8LeON3FsMlwp3PArhit3umqGEOyNXWQOow0Ud3OXdAfilyXOMIFvlBo2EujrgQImg3SW0q3L1oF7YR6rUy2rnPj+UwptG6ocLd/Y8yll01R6sHZLlKp1jS7TbD5c5cdOgoeGv19ucEIhl81Jdl16k4Je1Cb1u9uf3dZfXOG946zZsxWi/UiY6sphkCqK1IvKAzNwBF1pjfnKZO2i5W90erfvUT4CvHrCscSphlb9wtDmbQ6orVgDI3JiWG8T6dnMiVaLW39CZIpqqyidU8ZIdrOtnWGIhjY+i1Em2UoVOtbG0dOO7WFHspw9M6TvcMlqWIXp3YqcSIu7zcZhqq2dYnevvYMyzOyFHfxaI/Gv1pKjmH+eFW5Xa20Q6G3lJEBMCXaEwpbG8QCUK3hrDGYelIqGucBhonYBuSn1RgIH2LS1vM1ymwspNQGqjpmN/ORctGZCZcPQ1OaOvt9mIwLvUO5S4WG4K/5u4vdQzGHBGtlMo4gvbVGozfHadqUOM3hMSTAeGOOP98jjPQFhkfIeIZXJdvW0F0Up6w3vQNyqIjuTNyDUuyUkNId2Z0XHfUtPdaws9l6T33bmn0hHL/cp+carK5tUL/CsAc/6j7vbfpuqomwMiVJb1lb9cShfPZu1ZHlbejJFHn2jrNTcROp7pRSnjJh2TXUNe3yrCcHqU8BrC0rdH2QKWjLHAgYh17Z7GMiSspmxxLR4GBD0q/1WQUwCekpRlcgltmlWlt5ShP+xjTcYSo0Jf6Ut+MTfkTXqOlwb6O18KubeYu51KO5BbLd9mmtHFsRqhVtiLNsJWK7LuPd0Oz6iZEnxt8p50uG/SBma6rufNk6mMejDY2cifJMCeMCMNJ2JAHApurX+NHMK/Q4IC3fMdYe9jTA/wNB6z+zOVaZZv5oT8KR+g9NrWbYFcgm5tOSRBuh+ecUeXp/tx7WE3JI7WpBSXMC9w64Vrkj0naZ6VDkZtGt30OIuV8Y74tu5Ok9/tTI13hTXBLrm5y92Wu9RW/tze6TOs+RkTMemqNiPHlLx2zYttJNs9yQcevICLI39c0t1J16DrGPB9K4OZC2Iq4sbKVIlwxTMmyysKGEI0A2VOgIQk2H0aH5P1GSXcN+Q3965jvJ2Y2897Sc8vCSYaJsrfUDDXI6FH8/ecZ1kasKHFRGViDdgdgdQy3z2WaEjbVXw3GofnMbMFSNCVf/r+ceoqikgGr4dzDnjhlrp3AlndpiH90chphznXA/ybuEtEnc0BnBveJtnX7d/FsyNEFzZI61SNzW2XiZ+TBlke9NgoGWmZjV1mXjkvjY4to7ZUID5pfWmNk1524ptJe600w7JADXdnvaTFn/XjdQXbGNHgVFxW8D0anz8fPBViABJImxYwdETfOJwDxGTQNDIvVG7qIGOgy8d+Vvpe2FA22EcQlHlL7Ww51UiW6SXlTq1rElzE7zZIhvTgmXx8ErzHAkn13n/FYxto7nVne04rTWgJH+Gl+0LrlID2XOG9FS7GbRzVyo9SjFoYYdqy6p+/HzbaFtQdt41zfZWhztSV11NjlMgYAmnO2xntO2HfQw5oYGYmv81QOrN9vW2O2F0d5SmqHFAbth4rAeWifGU10+PIUihsKUqb61eXcqbrzE6ZRqIkhdbdvWWKZEGv342/tQ2EC0HkFrnVKwOTTjFdhJ7nG0Dv8FRZ//raK351goqP7SisZXBMLuOkVv/JEeGiLnI7l11Zb6zy53a0XoNuaIFtClME1om1iFEQWkJUCdc5MwjjRdi3hujTYUroocXW140lC4tXmbYSha74rY+rMLZOP9NRJqmaeIPVREN23AsvrifRAo8ql2FmvmNVS+xlsx2ra2SJ2mCl3j356DIGqrms4a7Bu2dvDEV5cXoDuMWsAxJEwLLlsQuTWH5jyZuP7vSjm7Mn/RwiUh8ikg7HS6NDHo5rbj57tITC99feePqoIEJKDaobULnKNwFu+5VkA2FBbEDLtXbDcnFeoBHWxwa71RGyX3h+7SFDIlSl9HV6WQ/16m1l27szm8htRADLN67dvD7mqC2IZKveqbbrZArwBNitcUY/r63+rZiWGPJHXY2pvjuL9rblcYLg8W03OYUq/mmtf8MVLKdAGGBsb0ru+dCAB4bemK4ioo7UVKAKfWIr+Gs9GSNwlaZHqDdusBqTV9r0m0QQwx2nY1p0iCKCRomgGlUzaqvayt1+p2SZYx3klNXawZv6XBbRWjjtPb0+N65UjO41DY/d3ULluC9cJidzHbb24PCTZUkdbDnbpsj7B1xD6dsBPotEbmSKtubfxuaE38phIwUbzWEwaUoLVTRDQ9pVk+cDhpu75JYY+xo2+Sblc2B3LsaK+jg6N/4CaBKUDjE+POLTAaYkxDD4eGztCKtDVn9EHGUKruIOuFv62/ty7mQ+Hyc/vKkegvS+iiQ2+ciS08Qq6vhKP1yrZ32nBOqTuysf4UU4dw1uXW5cftgp7Zu9YQCRQQc4oNq7UaxOlEAEXvjZH+GF9pt7iyafykQSvt1iZuKXpbV7NmtN5XTZlb+fSOlRH0qV91cxgiVs0II+g5IvwQm6unR1OqPVb1bhihrlPyvCr4m95WGFCsRlajraW3VpTeYbCTIl/PPu04Ibay3TXsgZLdGd2MzPFPlWZGqtHTgQisG94VoI2ppa3bjvoECPpmc8hy7UzWqU8dbaCqGv8XQK2TcIgddBuFatapiY4289mMv7Xvu2K31rShbusdbc0k0yGs+WcXEHujjcz/ZoTBNmwoj80nvQXVENbtqZGih0rfmmrrwycUdiprZDEb6bWnKVhriLZAbSdMHJJqfBa0Inej1Q8bPhkqHZWxrWG2VM5mRep1fEiOtlhYE6wtMTeq2pRwzB8Nm2jb9zq1QtxgUNQAB6zNatPq0i0XOw6+iSNjPM/uIJuCbrtO9en1uhIQueQc/iKEPdphfA170uHKC52+dO0XH+07slUp+k80UUqAyF+0waTSvxu585gW3biFNjxR1NdhD4Uu72gtZcRKvN6B7Wp21HdTWA99Q1BiQZR04RZ2NwttCTlY5fUi1p2QtLlF2K1aU7yhkd0kLyMaX2/aVpyRtbEFiN2YvfStt+9aLd9dlv9qhKEG7IYplP9qZNoxw+5Ia+qbQ8P1MmFoXE0M3EwZr75QgFpmdkGa282QUunwdF7HTT82mZQqAaztM+77BYp3b4RT8WtEu+xU0XSaPYWJB+JbqIrNOb+rbFMIbCtmS8ntzWFIKevlcT1rwAASDUk7ffT/FaBjU8LT1ByvSownuvb0ahVTAkM39gdVFVDqGIzCqUmCkrRdGRI8q6poOlveeL4RM/w32qiajCB5yLJuxKeOf1Ovdh0exmikGg54Ni7TGIKVLmWLT5qqdNi1IKpBm/Ydgl0xhqwPvbrhUJM2F8mdKMBVRe4VmDadM7oVvJLlvbehJqZqtt7W/Hd63pqNV1LBPd6ORN5VpC6n3jX51gjd+T4xtzC0bKQwxFjfS60IgJAuZUwJwisiKClJtDW1ahhyII6noXomraqk4utXANau/qwAkQDUuaxxvGJt6hG/5gRVDUyTKHKsuppNwbZ2FW2kA1RBscY7uXf0qr3o9FC3w9CYeL1rVzOrOv5+xppu8ina0EjkbnX2Xg96e2piNbVztrnVkr3t1m3GoeJ6ozWtDbuGKyRBzZpOrE5vhNYIbObZHcPT5d815gYZWtvLIKTSHX19xixRCEi47e8VUsT/AtvPLaluUKcNWbHrrRl1hkC7UeoiouToWO72CNG+JgFa+nLT9W1wXUFHhB8J3Tjd7m+O1y6n68XHbugCxFbxWgAxJfnec/syNGdraDLcJ1rQEwrTe+pq8+/tl11Xmp1CS0uj4tEn68KI2rsBRAA3SUedOKihBGpcsd0r9IZH10CVmjadntcaHPaJhDaAYNcGaq8tSsRU39Y9klv3bXfdBhEapwYiY9u9B7vKb28VdLMdujjei+wt6oHJbTjUPlsX3pYY09FhbyIzIsYQ1eota+vDq5XwqYUR+jmeapyoXhXu9/K7EUnqhxZAsv8EyJKwiddUCcOR8xgxCk01tSFNV9BqUug28Di+0GE5mupecN9NUoUcEkio1kKEr86N1K0OzT6rZdPkfKsqMft17Ta3cRuZoDEOWkMhRegklUjPwssW16TUfmjg0RRUbanVvWrmSOjVynt5e6/a26z7yIC+5DzvHa/Ts60bamgGtt4OcYpWjZqjqDfbcXmmSP7kQlf4PRTAut/bs2nHMJKqd3x2JWk9ZNKWiqdQaW5jRjDafNg0GlHYiBSlXlVrHS38b0PWviC1SthUD4Ms9ZUblNDx6oZIoyyShh9tN4zpUBtDP4x+pahua9w5ofWdaw21fHPbBJfGgq1hCCy6v1tJmoN4KPlQDp9L6IVdbFNXwyLRZbXdOFco6h7h8xWgXoB3TXgZpao3cOBggG5CdQSmhoiqKlBBd5pR9MJvumiND5HJcmssFyBC8FCLo0cb3nBDp8Q7Vo/meoJON2wiiKqKQjWcTY+7vTVX7Tc/9VewbqrmXbuNKDVmqkJl4wJLqlsjPaXOvkGrj7t/aiO0onV4ZTth/XtIQ2zm3MKLiYN1oorazX9r/DrzFhcbitYVbGQkj3R978DYykH2DkM9OFTWJXXDbhfTZuhG2y/sKmdt+18ThDotxS8tNZ6rQoUCqDXKUI0XIVKDvg2NgcTOpi7pTVwDgFoqAjQehyIo178Hri1rTrleyfrmc/zfOl+S5MW2hoOkPrfp96SQtNt06j5eu5Z08ih5PI1AsZihtur+2apgc9w3oaobsxe2Rt721GxaI3y+tII6SvoIcD85Aa42wyca/wrD1VbcKoTAkYRF5wwAABEUTEiX7dQKUM0XgtdEk6AhTDeoEAUzEIlI33KqsYA0Z3vZRLPCnQiI6BaNXPF9Qx+IhwoUGw4T2ojTVPVb5oCh9lJVCoBC0dhPRKrS4E41nWqH/mxpME50KWnskqYnFJ2ZG5WhAX24TeI6dZzCIIbgsknutq75LbyYOI6nz7TxhaqV4Uj1h2QeejKS+cRwJWhyhbgwpSW1YY4cH04jYYgON7lzM/8tjV8++oB0/YXMyJp0YwEnSp5aKVpTDm0g2votGt73je/S9SHFuohxHMHmHGhpSUkYJTA6+5KR5fSCyY5GXIwymlCMQus2bCccLWDihFQgXcbWCaNmy9iVowb+wYQD0VqdsjW3K2ciO4WhiXFVUg3B+lZ5PsfQFHhveVoN22rPoebdWtz0bmJSAtJ5RoAULGGR1+5MRPSn2sy9GyswBlWKO4/B3tQfohLX2BnYNWzIE+mKpD/iu8AqWUGiPG3Edm1Mm2KvNZF2HFKKV+NKunYtKshEhMEsY7ZbKlhXZyATbSja/XE2V9HLa4LjRGbvrHZ69ZTVtM9XL77a0DKMXBWwdjWe3rBrcSNMmampPaaDhgmDgLQD0Ii2tqDVliJSASR+jamhwgWU4hhX0HXHbVcMNaLV2lCv3F1dph7Q6Yeoikb73fp5SBiMa9whSS3+WPfx0NgdaNkAqOkVqUK0vho3fSFmuBF6rFc9ZWmrF9rPU+EbG6YajqBJfzPuEfoMCP3RWsVtdtakgsZlmI4vzZGAvuadQi1HMh/5c6esnlDCkQx3XZZ6YzaHREth3Ck0x8Z474doNpq9IjsjQAgEcHDoCiahDvdSVZ80x/g2Gae7kYH4SGvFs5VXR9YAZ9RrA+qOvG2KUgJc6jnATqIgSK2Wp9ijSnHPVs7Qn+vnAEiioS9y1xq0u7ct9eXQynlgw6jfBpFaPq4QQNMZsCdyfzf2iNRa1YdCd0DvZ1oaF2bXmBNUbK0ZcGtxnVjclfOdKw9Dg3kPHbnbv7iE8t7UecdzCDGtarjg36whK41lbngLrFOh/luxkQaI3S4AafyQWxqyjboDAnBrrvRBhgJx86FpGroEva/1XGpJxABUKRnQ4z5r3wI71MFd+anPMroRwuVrsXJpvugTG/q6MSWjg4hG/tj08VWsm6du9179f0QB/8uohfVAGxlVIXEK1vDtHWKgvZU8JdspYavxqBV5ugxPJ0w3v16ylHqONMuyQFx7iDjARtCSggU9sC3pePOno+NorveNfJUoHAfYoEK6RoQaB7f1XPrcSMgwOHh1lKoNT8jeptwcJS0Vb0OMaIoKHI02ALtHuj71pLXsd9XhjSfUSlszV23gxxXgGm3UMeL2hjDaiNdpwvURekmXVXK7wXXArN4KW8G6N8JO83wktBj3oFZFLOrc2aeollk+J3UAYA6FiLIjMnOwgQLqu8KP85Qpsv0lCr1aS63Fj4yEZrTwpLffm5NofAzYxkDWADJJ9dFAiACwYouhK1KtRogqTHDO4IHUodDRu8vCFFr7xGuyyFGLr10iBLBqkKMUSEC0wVPW03UTYNqUcveJ15OkWbnhIdGoQq9SPhY2UXVCfE3+cH0NP0ZChwrdK0I3TGzwbdFSYxL75aOHP/2n2fH12cF1d/qZWzxWX3J+zFJIec7zm/bZr2S3XuNrr8DmEIlH/TbL2rs6f9lDF8WGcG2KVtvEuKZNo12EPHwX4PV3SoiDBT6FAEPhH2ooGmvqQetLJ6h+1pJXEb0B0uURLaE3FdVRW1V8tfbmQrQG7UjOJ4zpWtGKOqdQ4zPsEVmjwE1K0sz5qtjEtrD3JvCVhcDod2Vb3ThdA1MTFIbauZXDFlEH8gci7dTlgwe/+OfZ7Jo9++jszT91pw8twRpDRIBnqCGAQfNr5tYr+au/zV/5A3P8PLwCvrsmfJGx7MlZA4b46a6ZTI9M/sE7ACnV12MwwC3lL+qbRAA8Ka3xqBmpqRC1AUvXsEAaLyjr7eCN/YRmfRqqkCpRzQyisWetkKWIVzC3N2tX28MTtAUWW//ZsCmmyYZms2yEK+IRvWkI0ORn16xIm3XukOVEaVFb35BWr3rRQfP3UCnj+fcuEi3Fdj9O1/xTXLH65T9d3fm5O3mIx5/Be3hnLTHEGGuYAWF4ImYQ1In3OLrJ3/iH5pv/mLJD6midU/SjvxphZIHpPpwIo9PhLNyTobw+CKgRr7S+zKepUBA3zThQ1fUWbG0m7trCGlgDgmgLB9ch6Kqi8csC6GV5m39u6FVrS/baX37vAdTMt7l0bzLbmjTUZqNGvEsO3n1GP2mirXWHRL1wQ7onwOMojRWqBcG6wSjJMH57ZYuXTR/Ku9rX+rGPWD/74eLn/3bx2UduuWJj89wY9QIQk0jFxhhjmZkhzExkkWVUnvkf/N/LT39if/e/tNdehLTh7IscnuhGzZRt7vGemk7u1t9hYsg6GepvfGOdFSQsrZxsWIEhhZ9ooVfcr6y1T2o4OoFVEQ91tuVTlbClkBzEfDcO1fMhHVSsJwmDgkk7xcOG5+glzWsEQNaSqyT8CudD02Xa4dx7QGHVtWLf0PC36mJTHg5H68A9tSI0/XgROjf8WR8HDd1dS7sDpmz8S82HGACy7uZJE9HqoH0NuBPcd40sm/Z+0vJx+dF3Tz56/+z0TKAqTrwT9eK9elFRFS++EhEFiVTh0gHKMjs7zj/7ofvX/zt/+rGy6Rb6hSVlW3t2v+Wk26dD2XYXoa2l9DZmy+6+/uhk7ADVzdsR4/EAbiCDNkJLJlUNDquoP1gXZ3U8FUAaj6PHf/tM1wnRBKihrdMKmtpA43fhtL7aoiNX4387ho2M6kxkI0+VdFQzIho3LvPglBKqUKW+duvt2okzYZJZXdegkC7aFUpUKm0xh6MLUvf4FI7TX6KuGaGqatrNCYsjybp2RBuW2uaC3MS1kdKbYnST1znU4NK121J1cfHgnnM0y2fMmTEzXX/cVQGVkJd4VQCZKoXBBmNtfpg/fLv47v9ZLj4LrhvdDferZUBfWHzshulo3ot6LWTszW0DyyK+pCRrYtXCgnRhKgGQWp1Jc2CTB2zM+W4/hgml0kjed/f2Rn1aINLIqmEXityiO3TWZG1dl94wddhpTV8USNlulEjxLDgRsOanih7pLh+2L7NxStfgP5qChrXBRom7kqN0OH7r/vgOYUiMIQTsCUzVarFa2OMbLx8c3TZm5nylIK/sRb2oVxJRcaKiIk6VVI0IIArvlAyztZ+9WX7wR1KeY+wLi1cTpgzRp493NdZcLRvdmhUDm+X1kvANUETSUKI6o+1i0hrYLitRofZqhTTP0fO2U581pWhXhZDmXetsZ+8I7xWxVdbQi5hj8wcS11iHBK66Rrv1OwIRxc/Gd5h5L78YD0OMvU98TRHXYShu49+RDHt6vyHYmlpSWkmQfDuotgmsWRtp8uCNDRpShIHYMErG5S1Mm9QIDR2h9b/0dkO2TUascvLB+9Yczo+fmx3dmh9dB9nSOS9wHs6Lc9559aqV996JqyoRUSH1Kk7ECcjacpV9+AN//lHL6FI3Ql1kzeQ3m2v/9W0nu/uu2e4hWCvh3lWrh2hzrHbniA1+FooRBFUApJQOFrWjRXNVW4+j9ev0Ki3Hw6aitKO6raU1McqGSpLEbFZvWxi9U6JRWFPzGYwSN0BqYG60AOpPFmxmp61KBB1wsvBod+dAruMhbNAOR298HL4eP1QXTqkNaWPIEnEEIqpn7LpB1jAfF0OF6oYIAo1Hu5rUOyBdFy4V3XHVHpDNTOrVcL0qEpvTzz48e/8Xx4aBnDAnXxg7LxaVdz63pKKe1ZCKqGGASeFUocYQxBCzgSqBDO69ox99zx2+YvOjtTJBCuVgid6UlYbgrDYJbtaiv6+oscBffsv4asMT3V5Aqo6Nml1EENZks1hPKEKwj4VfNccItAKqoDULSqbnJtGLqSgQJ2oUHmOlqzSIqO6SZgYdoQGoBkceCrYpbYBEOj61Y9tRfZ5gDUQRfIKDQ6yjSMszngKzkHoVIkDWF3VgY/BpbDA0p9G6qNbkVcQFYN28TQxZS1i7DWuNl81GDI3dmjOhN1RVsT5KQXVsbGSxAWLoIxQN+RPlWg/fjXmlPRFSERFWqZVko5imf8w67xYWrkG2buy6UrrRPvWIknu/+AFLRZkNXmLEGXHmPVWi3rtZZiCwDFVVQyreGKKqzJATkaAyaogZZFCs+P3vu+e/w7e/Qenj2RBEe3RYdBviNNahWqRg1lzva61fb5IDpEUkau7xCY2yk8GG7Zpup/P9ZqrxzGv182oBzhZVpUpsMjA1u7xeEkK7huZaOyCgnkxUu1Fpo5XXbl5o5AZufnMkvmjNL209J6A+802Nrk70dc0LBKkO2uC3scnaM6c/9Lbv5hK3AXj1iNHGMfykhlMNH61JljJqTDAAa+PCRnGNj7msBUgirc1eoaEoARTaWN7sh7p5GmKvswj/TcecGgpCmlcb3LK2iUQk3ew49IeNNm7aVhqKGDWoX/iHGnI1Byk1VtiQjOLarAAgqqCmm3MKcYCEEo0vF9XDT45sPG4pIkTG5vMK7Dwq54nYMKl4NUZVxahCmUCVszb36lWVKSMVKNGjO+beT/21V+zBcWNa1dWOT0I/1bWOXVOPblAboVuXgtWzAI2Ma4cm3WrqHO6VgSc75VMrhr0RLqNy9opn75+vAGWyRKThNHkYB2kAaRw+wS4WM0urHsWjtvFhGFEMgDfVrFqMKET6o965aiJE8o7SeAQ9HFFce/qvB3SI1JzusflaCKnpHpB1BhuNgsacbFSkV3bCeuTFJqpHXrvGAUBTpdKUHxtf7Rm8+Uq1edtFLVZd9VaSersn4UwjbQOCG0+Ig8FaFVBR8YCytSCjAhFtuNFoKqJVZv24bvIantCxxtKa8YXDIcwp03r+00ZjJ1+XZiZNkt9snPpzCYBpDI8EyzGiACAmcZWWFzTPwzsN0phcVL2IV6xKN8+txIs2oSpqyDKrOLAlhVdhrdhmTJmuLrK7P/cv/SbNr9VVrJElejXV4rZ8NlHzgqCQrvtto6La3ipd/1HDogIE2RyxraCjb/cIe9vXLhns6cUCAAgcvxoX77eHKhhMNq6AbKh2Ja/XRqJGG1C4vJkCusV31KA1Gr9Ntx6pRMRpNW46iIf/aJwBlOK2mrumWkQ+mWaGkKJlQYi5aWTmkd2kh+20ARGSyGvojiAh6+0PQrJfN/B2rRIkSZoAs86yObNTWbGWKW5gJtjMtlPlBGGJeKgS8drVawPRYkcxEYiKxfn5yf2Tux+d3vu0XJ6JK6F6dHTw3ItfevbLvzK/+SLszHufVLR6SVvnG7NudFUNl2tUq6UEiNZndYmIyKgqsPY1TdcTpLkd7zTZaKLNugsatwrX6kHsww2PSEr8VaAkVUEgZgtS4JBgSUopy6KoRJSJCucBykzwFmcvYkGqzFApSzbMqmxAEMNGlfn++2b5KW58OWFVooxAKLG+jpkUPsgWhvxaznW79uh6wZ9lY37FG5VVwycpKJTCayBsNBTicL1CFGuGJstuSt7SNFtJ6ghbNdCW7V9VbeUlYA5RzW8kzmtlZo2jVlyAnbUarmA2IZeIFERM4GB5C66joSQ0Br/6BBr1B+Jqnr+2m8e5vbasrC1sVO8EJfxKtlRtal9hltUqXm15r4f4WnuCYm1sijC02aybm+tr4hZrJnH51HD5m5JG7dIDQJqtGr1piRDu54UCytR0wAr/oSbAbZKOhhSNvQSk+gdek6ofES3ianQLWVetHmTG2uXF2afv/uLe+z8/P7m7OHnoypIZxlgmPsvto7uffvzeW7deePWl179+7ZWvi7KKn7L81g2+HtZN7rBJ0+oJXGvNRJR8p8EcuqzeEAhg1AKs6P7FbKJZJGYVFFhKjE/TqhCfMPHd995EsRCUH9+9s1wVGePoYDbLUQlc5WcZO6cK8aK5NVAQqajAMDGcOotcvTOaWVUxSpRV54/N/bfkhd80lGlc7hr9mFZpBZTTgEyW5y68bKRtkvv16YrWPVqUplzdtmlGbK7pI903Eta2hWE07H2+FaTq/LeW3jSAALDKXLeHiTf5EHFYLxlEtQ+a+nCzf/SKEFULpeAsHuEp3FlTn5SMWqLIusiNs0VrjApvw+qdaAPVcTZaImLDWrWTRMnSLK1TERA8VMNaF+RstAVAGl0hwxQPM6leLDWB8nrnieLop7VypqmwqFchasTrho+iN8dW4wW1epeY0Jzz9XNsjHFNGSbsblxku1aWY6pgpmxCMqUA6L1PPvj47R8//vTd8uJUVfJZPp/NvUpmrWG99szzPDs02eG5p3fefuvV8we3f+U7zhyKaFMVr2vROwpF2pekBxaWkAUiYQX1ib1QYmSa/l2LX98lNTziN2q6SRbrxS50KhOYjFlV1aN7n+U3j+998sH8YPb8i8+LcxePz0onpQMzQVgrVUNQFauG2Hsn4q1hhVdSFYjzqmSIDFmpnNx7l32hWT5w94FKGjuJHjQwamjOayQASLwuonHrZk1dW4riA6Qt1B3xq2nQ3CnhUFaXzAQDVbDWZCAO66O1magQETFz3U6aJoMoxSu1lUAQARMxqyoHQ0vQY9KJ4hSZuJ5mafKkedbu4M0/68m2wUtjs27aR+phmhb7NWxroIIgUkjiKqoKcMRrTV+KamsuNb5pja6tzkj7RiE2E6d9hk6l2oBFRDAAmLmZ57rKmxOVa8K1ke9Gc619dhMJajZXc57Et8xQ+fijd++887PTO+8X56dZlpEKsSFm8n51fmKNHMxncvHY2vzwmec1v/HmT36yuP/xl37rd2l+W4Vba2NsfKynUWsaJNlM3cjRwkB1O3GzhbE24KFurmZ71kMKiOBIyfKxYT/qtH+yQxEA7/1r3/rN5U/+SOGevX54dP3GPMtg9CA7fvDgwWJVrUpvmeBcaOdwToXAPvBrJYVjJu89YJQ8ssw72NMHWXmi+XW0TRfhu7Tr4dEcKs2GQmQApuEZQzU6q9S5hpWgPUhqYkHpkptdgaRHt60bsP/sRDt5i76NwNlOJrbuoLIgY0y8+J6ZGSwihuOqqKoiwkxxXeQ4NzhM8yhcWt4aylc9c9bWo0R6gu1GRVSFg/mjueZ2mq/3z+Z0bTSBqraBTzS5sCaboAKixNDEVExNuhLFWytHIadIlDTKudGIDf5YzzTFWjw0Oj5pQOsx16lCu77UubIuybQRmmjYzCR80w8JiNeFsvGuuvPh23c/ent5cg8KO5tHwXxFBDZ2fnRttTg7e3xijV26QqTKj5eL1cUPv/uLg3n+3K9+B8dfFh/uAmiIkTpoo/qBFnQhe23aI+pb/GtDZ3hV87sNDA03ViBw+7gS1LZUaawOKevArUlqKIWSsR4MO3eC+49Ozi8utCqOj2bGZl5WK4E1IJCo5AoFWU7WMqgh9qVYawAoiQWBPWDKs8fz8pREiFg3Oyze9YIpgTQRuI0c1o2OTUq30XyNXHq2NQeSNVJdBY1CYzC3Hu6EX+PBroqVtSYoicaYejYCUs8iZiZiUMOiDAKRVxgwkzaMiNQVOtrYgtyqYcJo0sniMo7ABTecUJrTr4eYNNoIDZxurRj12tucPHVuNQuISVJWNbxqsleFqRC05jD1amjrXXOaFak9f5rCtHqxmcm6BRpzvlXx1rRv8aNGBXuWfWPtxenJuz/97v0Pf3lx8hBk2JhAU4lg87mvVgTMjq+rCKnk126dn58tVpW9ZmcHR3rr9qMHD80H715/uTI3X3di08mzBi0Nlog1FqOmzBoubaJ6y7KHj4RREcx/gvglHRXB5ghJSYSINVKmYOCIQLbZNkg4K7HrUZs91SvK7NpqcZ/s/ONP7hTF6ptfef3m7S8vyvfK+6cEiKi17CvxIl6QZ4YURqFQw0oQJVZAg9EdLs/nbrXC4mO68SugrAZroHai7OHv6z/XQNTcIdekKwHJzhI6vLMGSKxbcwAkl+U1qDUaaq1LpriXB7KRHJrDo7mMtenzANi1sEJV7XJ5EUz4AM3ncWX2XlTFGJNmX7hftWYWGiFAlQyMsd57EdEapxqkqS4s/BaRwBRquAkJiUlEiMN0Wk9+EQ8FM4sqoCrJmVfr/XuBRnNME0ZFpBamyX3q591orZhDnRE3oFLlVKJNP5ywIYCjzr7BROJwCQy1AZAtot4UoIllLYDuStjo1/XdJ3XCOi2xUZV7n7z33l/8ycOP3z4/eeS9miyzs4P54TWvCudsxjbLfLFanT8WV86u33KKoxu3L85OV6siP7rBVXm+qg5XbvGLn778LbbXXvfC2rnoJpSnqrXlK4FIbYNf79qsgQkJcRo6dG1KTfueKVZyPUx0OLY0UVxum41DRMqAj9+ojhJwINTGLS/y5Vm5Ws4Ojr7y6usKfPn1r1nWZ555wX7w8aJwFWummhn2TgQQqGWIqldYoxAf1WGvChUSEmHAn3xGL9UwBCKKH8FKcNQXNI4zAqjebK1roSmOiUoREdBDaTfsjeuna0tEdNiLfdFmIVthrI2eo7aw3rethb+FTc0nWwUgIgsyAZhE1XlP4kPTee9ns7kJIEJkjamzrlylqlk2g6r3Pk4wVS+igIhXUUCZmdkEtgUg4F1TiFoUL+JFAARiyMwqcfIpEDZHFeq9iPdMbJjrdShouGS4eWkHh1KZ0zjYWAGwyfJCfG3wiKZsIScRDZjQxB0m1jgpom4VFrS4igYyweuhFzYSVJUpHILYcE8JLkWEJmZR0thAlGbmAJEJJTOZAB+B+9SVQpLh7OTuhz//wZ1f/qhaPC5WRVWubDbL8iyfZa64MNnc5vlysTDixLlycTY7ug7w8uzM6OPDG88szk6z2Tw/vPbgwZ2bt4vlyunPfvLsl1b25pdofiPJJqrgUMGIM+F52mhLTUgbTmekwbLR6IhGH8ejwxvwFOheZB9IVC8sVFxrEaoanMiYOZl/a2fJNch++oM/Lk4fVKvFLLevfP072bVbWi6cWx4cXbtx7fhk8dAA3qtmZJlLpwoVBpPmlgJfLx2MgaiXUB2qstzIo/thTwxxk5LAkjpF08WmmzMUadkjQGuflc3NEzXr8RNbs/ES0Hrk1dC4WVLULimtsj3gNYWUNejj5LBVsR2HsKH49vj4OPaxwhiGalD2RISNCXM+4YsYY4wxtrKVdwFxAIDIMLMxIsJE3gejNgwb5obrFZExps6tlsNaW8MwM7MxtU2K4iYEMTMUhsiFh+H/mlanBCFr/SXwN1VF8OGuMUhVUSOspoWtyYzSWwqIDHDw9G7RYFFh4rBzEuExOFJCOd1gVcePSlPkAsxsCKQUtmFrqPVehGrjJRkJzE3XVAK1GETx3sqoDlOSeaMuofAwez95889/8u//1fmDT3PLxMaLtzbPstyyyWzmnIp4gs4Pr12cPKgWF8VySdm8evjZ2fni2WefPTt5eOP5V89PTq4994IQf/Lem1/+xnfef/vNs8fnr375wfzrf0soI0CVa76FREAbTJaiG1WCJdXQtnF8N1eLZhuGbq5Rfj23w2igxNyQCuQ6Pyj5gHZMrGQBDYc9CQyosi7u3jl/8/uHB4d6cPDw3mePH91/8doN8ZW60lfljeNjfPbIi4oAEM2tVYH3UMNMcKoMw+QgtZ2CSJ0UN+bHVK4gjsws3SCDhvVCG3CfbA9R/vrscu0O0FURkukwLugAotEmZBS/r01xpWwARASxqK2G1mqqrljbvpPKCWpESAcWIm2k+ttUkVCnOmhikRpKiNIi4V9ymtrQNLF7CAntjes3JDhNpPYgIu6oOaoKa8OV58w801nACFG1xhhr6z3hriqXBE3rJlPNHWIpTAQKGBcaUgFB2Bng+FlJgjHGWDRHuqRdiNTEcYqEVjcc1avAKVOtOcFZnZJVnUi8qT0aB0GAiigRJ9YWEkpUiiNUNude7SCbWFV6plDvfRxsHG/HAIBovDOpOBJ4inhqAiSrguqhHNvMh5WGDBMZpGvRpAGLFJ1jAlYIEZ/c/+jdH/73J599zIY9mGHI5qRkZ3PK5nk+s/lsVXqAraGj6zfPykKxWlyc5wYsJXyZ5cfl8nx+eK1aXsyOb9z75N0Xz885m733ix+/eMPOw4lUXdvmiQim/lhT1BPD9ExTKZk40ybseigr6o2mNU4RB+tZXKvIJ+slE5lU7IaXbFpLTHMRQl1G0PdEZ9dvaj57/+2fZvmsWl3M7rx76zBjw361dOVilttZZs+LyhKVXrSsKLPqVUSsgSqLIgORSgYCCQxBPZGWzul8rr6ijBIqgSRN6mRkhGq8OSTurgo2zI4JEQgNHIi2sgAIUVEMG1qajkBHKETtt4eku1C6ST06giqFr61RvM80HOgM/nvSVAXSmUFN/LEe5xprAa0/ElKfeiGA0j1+oHVVSDnqKxM0yvFARNaY5O+q6rw3xrDhYNMJsBXbMlGbWB6BKHrKWmsDd2qOvJY/ETMbY4lIxCNONg4jPxCZUFYoToK5TgQcqCKrgpiJwCBmCpDnRSjd7G0o9oEhAuDEm4RBaaaoSPi4OhKtWdNDkYDeQS+2lJRiIJmZasVefOyiaDneWEnq7YvQnNFfOJ4J4zgwVb16FlGoj4hvjbFIux9Y0644Iuo1RUTAHPaAoQiuc2m8BozzAQeTZT3CLjG7YlmKZPksm+VsDIG8917Euep4Puf5QUZMXHiyQppZvvncbZAuLs6Xq1VROvv49Pkv3z49fXjjuZeWxQoq2cH1T97/5XOvfu3hL8+z229QdiS+QkIo5XUVNCmE8eI21aA51W2FtN9KHExQXlXJMKfB2ViENN3KqXXezJQu+9R6ra93G0ASOFEQQsJ1nqEBA9slzQ4OXvidv3f3zierxdnxwZG19pOP3r11/bqqOFcQcP1w9nhRhkkqKkSaGYaKKDvylsmJGo5eXoTgRYtVWbIvdXUCewQv4V7aNGLCsDfBEkIRYiR9e1upuRWWDA5JqTQJQ2qfbSEAHBRPD5W68dcolHgTrTcQEpEPGkVcCTMoEwkFx+MahIkonCDQRN+I08/k46sblzysbR+oV24NUyYZz4JRJcXaxKbmzJpC2WyYTuK98z4YrURUAWM4N4aIFZGJqKpLnuVMYW2ENRz0OMMGSqIVUVA6o7JWsxhmJjLJB4eYDHFQBIJXlkpiHQowMaIWqSLhbm7LFG12qhog2AuJeFUJ7iSo0Tbs4ogENZCiqQKqEHHMtrlHEYCGSImk4fdUT0NfVyHQAY4UIFz07ePFueDaUyyATiBizDZ4BTAHoxurqvdeACi81m1bMwhJBvIwjSWWuwYFCBFZA7AX8SpAOFTEUHEiBJDW38LTpFnJvbufutKxMeq9qBpjrM1yY1xVrpyX08cHR9eMNUzkYUtfZHZ+dOMmE+59tviLNz95+YVbb/zab54X7uT+J9df+mpx+jDP7ONH99/45l975Y03TG6VCZ7X34sMi8d63Mf2r/fJWl0QujXgTVRVvPiwywSASEkCLSbmGuDroiJ+1V7ZceaE2R6GB4dRIeKjf7c4ESFiY633/sVv/879D95974//xfnKl7rSqjxbLF967oaIqNL1wwNjzksXjKBKYQdBIywLq0K9qCEwlCBCwpkty4oefIrliTc3vKiqj2bMYCdTIWJig/riZQKQDpxRWvLZpLoJKQABB0IazHKhkQUAyIDrw2oa727hLNoaRTU6FzHSJ9WUTCRvyayM8PUisJIqCYiC6a2meWnzgRo9ELQPRToHqevXpMSaxmLom8a5tAYCtsx5m8g1hbJZw8xsrDHWmGj+D+qeMcHo413lvcznc8PsvIsaEJlg1mc23rswPr1479U5Zzh41Ub7mkq0nqhKYFWSvidIVNtRRLwPm+6GDQWHHFJVie670aRF3jtVSZcYEhGrioeSCiuEIlsWVa8CVSGmeH+GEjjMcQl4ofWSEpZrYmhilLFlaxxHhMoAdpJcfmojiAb9s2bLYUeluWeaaAiQrjiK/m4KkYCYNd0Asw9Vq3lWg/EFTSb+kQwDCGfKkDThtOwTk/nolz/89J2fEoStkarUqnSqNrNsrMlyJq4qr8vl4eGhYWOYV56qqiKTZVl2MJ9dP5p98Mn9P//e9/7ab/2Nux+/f+N2kR9eqxaPws0kX/7aNxafvlNV9vjVb2uWeVd5X0EhjRNFxlisVw6i5t5xTeXCVpKCo3OFQkAkAKJhMfgkGktsjGFt7FPXjRy7T1RFEviDjdH4Rbh4tEUDtw1bVU6ZrSXz2m///sHRoa3KxcOP7n34rlstvHBRrAhirTmYzc6LpSFIsGLCW2PghRnBV8SQliQzywDEe1j1lfdVZbwHGZAnsgxWqjVKMBkC0nXuyZzIJh7+CIyeSbk+8CfiXTiXgzipPIUdZE2k11iwgXiSKg5ONoAJziLhnA9CoQlckhNfUGLqs1ABFVlB4HSkHuuFAqg5VXPGECF6L6UFleu3uuaCa6pFwY9XJ5GvkWBDd1prbZaJSFAYQeyc8+KN4WjYgxomwKiqeHgIMYgsUvUq58NcrKoqs5pzJqre+3CkU9RrJURkraW6KZXj91AkakiqEm60Cya8sCQbkzETc20NI60JCKLC772oeiYKlBAEBsdPrUQuLgoYNqrqxROUyXgSkuA9GwACQhK04OCPEkdMBBHBBpULkoQBuGZztW8qsw1HCFvqdmhPonqLmoggoiJaa4iI89wj6kpxU7IBZxtixKEU1Y3wNlr62Jjl44cf/eLPtSoW5+fMZPIDX5WuKliESLQqRXx2cGSzzCkVZWmJDHMBZdHZtZvPEX9nnn386b2f/OSt6zdvvfSlV4uzhzdffBXVM7I6M/ns1su37/zwz37ys3/2wuvv33rtV2889zLPDr2rsDZXyYaPq0ApTiQN3wQByDASj/Ii3jsAbC0Tiai6yjsXaa8XNhZqlUS9FyiH1dcaJB9G0tCkjkhV1LuKjQnWgxr7mMNgduKcGgXR9eeef5Ad3vn5T84f3jFa3bpxvagqESUmVTqa2XsgJ6qKyguzUfGc27BseBFidYIDyoPpJDC46vRiVhWcWXUcP/mevEoCZSBCIrDB5JRWPjawJtrLgHDOOZwrJSKEzSUiogxhVVavUkEkfOEzGCUbNvB47Dc8UhCIqR4qYfxE8EkIB4RTjMmJLe4grW+FUghrYFVJ9aRkNq85W4Mpo61Irl8lQ1737XSAs0G+aJkmyrNMVUTBAdpB1mZELKKrskrDEQRyXkQcCBDvG0d8mYy1mTHGACZMS1IIvIalkmCMBQWsj8fVgx5EgJqgU0gkpZSW3PjdTCIiY0Sji4aqx3pik1fPLISMDYfLa4LZO4yt2OPB3qEAx20pXtNyiIhzVdRi2WognwHlyAQQEfFEhqhGH9SGZ4lcgBPzABReRESiswmzqho2os2t1bVRLGiQAa+RLGi1Q2+t7dam4eaQSLSuRlImIqnKD375o6oqHt65U1WVEomIODef5WqMzfOAjn61YCJDyLNZ5bwsV4EROScHN55h472visL9yZ/++X/8pdeeuXmzWpyZLL/+7EuczRaPT++frxbIP/3k4/c+eP/42s2vfOOvvfD1X3dKEB9Ov2lSvSWYSANDEhEXvkcDA0PBOhaNXAxCsJcRKcEqyLtK4/ZSaoeoioqqUVU2nPSksAmfKVTgEiWMsBC9QJmD6Z8Q9bmqrO798uenn7x/eDw/PrrOJL4sFCIepXPiPTOVlRhm8V5YPUFEM2YRHyAPqt57a1nCfrNoWZZaLeFdoJsatp05bMQbYlbxlGoF8fF0kyqJsg86ocBDWYlZQcommq4SAIQKK1mAiJyKwpWqmkBz4yhMNHwF/XjNkdZmsSbCxDtg4p9BMybEKxSCzhn2RzfUjlrzjxsSCR+DpT/tztZFBGpdk4M2bE1najZMs0CpiIxzwWoWsJclGp1UCUzMJrF6AKDKe1VhKLEJBjEGZ4Zza5nZsGFKlxGyihrvJQC4SSqjJQIjnVk3pApO1oy0u6pRAQx9IURMEBUR74JNxCQ7d0jofQUYZU4fGpeo0oazUwjgCh+cewMmUVwdgq5HaSc00CqEgyoQUQ1IZ1hUjYio+nB0tXGvLAI+eh9gjk3an11vbkAs2RrIAkQyQ5Vrg10whRuz3oAT8SK+4RIV5qNJehyrrkk7ImXDmz/4o0/f/8Xi8UmwqbiiKorSWDZWRJ2qZNYAyGa5F4di6cuSbZYfHop3y/OC4KvTZW7yLJvdvD6/d3Lx5k9/9vv/wd//7MMPYcy1Z154eFbMdfXo9PzWs68IcXHy6OHjk3v/5r999e2ffPv3/xGObql3QBhdUErb2EQqKirMRJRFoZkpLAts1MZKAYAIrGVjsnwm4oPnA3OwtBsTVX44VxklZmvYhMEaG5aio1/Tmh7uOFJW4hliY3F2cHB8+xk5uZlnnM0ycYX3ogpfVctVcbJ0q0rDjFeiynu2XFZVNssBeFEvDKgTDQgqqlmel2AtlxAHWBABXstCVMhYzWYUFFQIyEBFXanMHEz4rAH3ow1eOTqKrwMphLxT8TAZ2IBZNQMcwkQjE2zZpA7gpDzGPXrESc7JVlJvI4TAia9Jw9IlYT6zpC37uI0RKAnVn5fWJsVbbwhINPEh7YnWkdPuR3qwXt53wLKiLGd5Ho6IQ0Ul+m7VM4SJjSWNTlGsoqJqiL2KDWoXYNkQI2yeMZAZAw5nSiIQM5OCHYmIZw5jDVz7iAXXGwocCt6rEIEjDQnfpItQxoCKIWTMWZYhcRYR8eKR3OgVJN7HVkhbx1CBMlPYawi7UZEcMEVfNCbKs4xqyhRQXFUBCabiCEmUZl9GBO+d946IjbHr7TOo9xJskU3zUKtvNFptolZlrY2nXhv7l41llcMU9b7SeMoibn0GsGbmtAoom+zRpx989uE7xfkFVG2WqYphymcZEVarQq1lEoiwtZnJsvmhzWdsM1F2VaXOzWb5YrGolsXjxVlmMZ/NXn35ubOH9y9OHznx6v1yufRqDl64XVXO+vLwxvOLi4XzhZ8f/7s/+WPrz7/9D/+X3h6LlCwKQdq1DsuDiDgAzBmj/rD0WmuWda1Jg9sjMwnD+zBSpRIKtjCycJ7ryIjYH5EdSUPaUMnTlDUmLQwGokfPvoSLM39yl8iXZbkqC8t0erF8eF48uqhEVMN2AcMLvCiYVpWb5ca5KqzQXlCJEvNF4W4d5hAI8uCMSYGYVYW4AsaKODY5MXN0qWQ1FipKZGxObBBueBBREVDt9SQAh/WVFOAMbEAm7HWCFCYjtkh4kOzuQVUkjbeZRS+OOENq1SbsSySvr9RI6/uJEo9NTyjNrQYupQkdncal0QlUo1QdFangNCES4WjKNRhqiwEAa4wRkcCkALXGBLN4cLUPUMEcbXwa99xEAKI4l7z3qj4nmxtjjIkO7wTRdIwOQsQCGFI2bI1JYBCuQEAkotFRvr5Zn4hJJPS/wjCBwskCQyazsf+Dt7yIMx6qEA3gE8kXUWD0LALxXsWnJg6LCCmUAx2K3q2xhUVENFB8EHNUQkAgWJtFo2ZtEtD4B6UQGjOwSx/cxKKWq3Xr64YpLepNEmz9yVk3blwnC3qdEMg07HCrBJIq0LjTohpNe7768Jc/Xp6eFMsFkYp6a9irgKmsqvl8bm1G6kEa5pLNZrNrt8hmytnMV6vHD88fPrCGK4Koz+eHh4eHN265jz5+8O6bv7z53PPOueXZYyUU7nYgGibPTWapWBoyN5576c1fvvX2R/+Hb/6df/TVb/6ak6r2bvLehUp4L4GhAVAfkaZ2Y0aCs2BeEFEJ/++cqsS7CWxGxoBAhtlkxAziYNekWusJc9B7Cl0ex1Y0ZaYeAzFEsXj4oDq971fnjxanp2ePQ5H3H188vKiKpKDDCVm2TJVXIiq8N56YTOmFyJTeW8sZGRCcIj+c8cEBQFk2B0iNFZNBJSjRRIbYUFAboSbLAtxQvVNBACefCIDUq9QGNQaMruuEiNkRt0FQASUYa7q9EoCgCMQxH01gm599jL3AWg+6OOFY47xNKFYjToQ2XbM8WmMXUUKsNOtjhPQw7IW20KtFzbp/1r9tGDHOCwejiCuYTNBuoADDqQ+3MmU2U6ghIkJZld67PMuIjIHkRBkjM8SkwUmC2DCC0ZoBE7gKyACwxMGEIlJCBSJePLMhNhxMJGCOfNgQh615VS9KEO9Jla2ykjE2OFYh+MOwibuB8eimatCMw+Y0ACKvQiADNmGLMEyV2pKVjPAAmEjIrFuLyFD8EnJabxTJB9gYjnsFcenSePBUPZSiH7zqGupq2hdxh4hMZsM2bCBikixi68NV3Nz7i4SWRJzEb5SGfUCJlTHZ/Y/e+uyjt8uq8s6JiLWGmAROfYANYzJr2UK9MawiIl5FDBtkOc0PD4wlY5ePH+aZPXrhpcMbN42dsYg5uHHnk09Wy4vDaze9e3i4vHj2xVfnh8cZa7B8MbF31bWjg8ePV7j/2ds//NOvfvuvabBhBbcVwxBVVWNsIOdJq4g7NetzGnFxBpMJV6uA2bBJPmrEbNhYEKtGz6zkdrv2c1RSBgf7UZrL4Q6FeKguoadaa2+8+vrPf/zHD+9/dnJx4Zw3rMuVK5wvvQaHZ1WFR6EemTXEpVdLuirdPMuDddALvFcGMmZic35eXF8+ZoAMgw0jM/ksXiSmyQExzm5Vygjr+zTX87WesWrSqsoIm5tU7/lAEfZ8CQj+tQTQ+p4RSPMgU/QUQDLsR5bEtQevkkaDRpQzQk46wELJErWBf4n0Jcxar/fpyTrimom3qhvr2tihro0nI0wtcFE454kVqmVVMTtrrA/marCIOOcIgT97w2SMCVpVBc2NuXEwz5grESkLBC9bISFPCi/eZhnZjFLnqUjpC1H1zjnnCUJhL4vUEhk2mTXEhowBscJ4lco571zUGXxpFGaWsc3FGJDRcCApXKjgHalyYCpp5yBdB0PGWElKBwfzeZzTJi4nTCCjESXIJOAAwARNzrdR5YmG3OTaujafanBkZQDGUuJTddD1CfZg5IpYxumUQjCNJa4n9V5qPSCCeS64DxtjmU3c1FcNp2uZIb766Jc/Xp6frRaLqiwNkRfPzGBbFisQpFpBKp3ZmbUEWGtNlilnXtT6Sr1wfnD03EvZ/KA8fZTN59nsAN6BcZNfZJUH9x9QtVA1n3384XNfeuPg8LoRJ2Vp81nm/er+3YvCHR0ev/jM7Du//zeViK2pHYmbzAu6Pi6WLCdqTNzESAM+6kfx1v6NaQDENb5xojbEofUakLym08IUMzTGhIfRji7effk7f2N2MP/Rf/2//+zk7KLwqnLrKDfOlReFqnivzOxVvFfAZ5ZNBBQtnZtl7EQsGydijS0qx2wJ4h/fZ/HiKrasHG1JlJwRw6SFqIiDKllbz8pkJNk0kgV+llyksd6IBIWDoGuyo7FhBekmd0C5fp9aj9IyHB5wYtBr2AOUJFjxofVJPQo6mKbrUai25iOB6zrXaaFZ19bvdf+utZONYImoKAqwMdGwLwpWQJxja0FkwsFycVIuSLwxGXvOIfPMMgFSodIKqMrSeQ9oZpnYOAcPqGqe55m1YHJOnHdFVTlXBS5cq8QKEvXkq9D3kBIKEMNYFamqKjqjAQRhUMlkDAsxmZzJGGvI5opwqIApXOwdtguNgbHGZmzCkSAE5kQEYktKzGSsUZAkb/vYVRraQkzqlLAwBdzyGthisLZQcpFD3DEKDjtcn4APZ6HqH7UKGSxlADRYF9PhsUANg0NG2LDjOl8QNHxHRKP/HoEVJCKcRieR+ejN73/6wTtl6ZR4Nj/wPlCzzDufHR6piuVwkB9qDWXh6iax1qqxkj5ipKL50S0zu+aWJ8vH92yWszFEms9mLzz/3ME8u3jw4Pozxx/86I+Pj+bPf+kr1fmJUTbizk4f5/Oja0eHL7947drNW9UGiCNQ4TgAlShauOqZWespyTdZSYNLoHoig9qeg2TBaSz/TewjEg6alGhjs4ypvgAStEbG2EP+5gsv337+dvHmhxer8saMROyt44OTi4KZvfjA3Ym08uHMFtjAi3pS53WWcTDBiFcmOrlYHRgjxSmRR6hCPAxCoNpdNa2ZKnBeqWZGiaGJrg/hRgsuoKwUlHAfaRWSWk0cVFdVILyVCr6CMSAbdgACJEQ3isTdKK7yYWT6Wl1NgkqKHFMpUN8QGYc/xcnVOHzaMoxNDdsS9bAz670450grYyzn2WE+M4YJ6tVn3otTFWfUW4KKKNSghIcXD5+LalmuVlCC+qBTiWcmw4CykAGZgtmyCqisvFetwtJBBgRNl/8ABHVSLNjOROFW574qZH21dtyD4rglmI4uxycmaCapUZnYgg0BTGrYmHyezw6y2cyYLKIIMRkTXefC2FblqOPEK1ZqbSQ6CabF3keToaZzxpRuoUE9XU04xxJkFxVIHD0iIt4YG2iC94HyVhqPMXCqXVBQfTLocH0gKWEpifeAQryItTaT8KkgMqxqDD2+98nH7/yMmA7muXNhmmfeOfFiDDORyWaZzQSiomRMPj+aXbtGgF+c2DxXNgIiYhVlm/P8cJY/bw+uuWJVVUtyhT284RanR9dvzth+9N476iW/cQ02d0UV9rpza40vZ4z59VsyvxabQlXTrVC1B3KtboeZpY0AeAMGJJhRRL1qtOKmnqb1kTSQxCMQQWdqGJN1vQ8gBunUPxQ+WCHChOS4wefBpjSHvqqOcnNzRvfOl6dLZ409X1VewUqi8XYDUfXKJARWr/Ci4gVsFewVxtiLZSXXZ8X99/PFIxwfqXj1XoNnfzBSEiniUIIIVL2vOJmtiE3tbBixHorocrC+7CiM21jXQPniUReOO4wEsK2NWYkrJbhJK0lcXCPxC1aUuOm4tpbVuuK60BpelGtB2UTNN5LBtYVeG783UzcY3Ca1q5e39UuthUpiqFpXFG658quLitnms3DWg+FJsRTvvRP16XRQZC0AwskdLx7qOVLWCOqR4EXbnwmX0Xqo96pklLOw/SK+cm5FRBAXrn6VamWzmcK4YuWqwjtHCIuQGCYTzyIQEYkKkWVjCcIEitvG4eBF2KmxsV+MITMz2czk1pjM2sxam1mbZVlus4BnwuTFg42187gSpkYjijeqhWURRCwkquHqQkQ6Fj3kAp4hkfz6plxR9mlXunm/Y31MYH1YvYGJaVmK1rc07aOmxpRFwA1aAccDXkRUFcsP3vrRanGmriIVy0QQArI8c64MlwZbq6Lu6PCI8hnZzJjMKWg2tzajfEak5Cu/Wki5FJuzL2x+aGeH2dFNEcfE2ewQrsLiUVVWy9OTg+MjgLyStZmIU9HDg7kvVzcO8psvfxmzG6jvtg32bFDwumuo26FBNOmDiBaFeNZLEa82iW0VLTg2szbzVZXmgo/TXWuyEElfYHwhoWxchE9E6Vu8CeO8UHX07CwzR4dzX5XEVFTOsKlEyRhiDgbdQPKNsbkxqj5c9VMpUDmizBrjVJ3TsxXwwQfXlnfo+BUVgQhRuGEFSmHnP4xoVSgFoF1byTW6wsXr5zVcEhe3xeP4C84WSsGBozG3KXwPmwlkFUzx7HdwpeU1nmj00Y1qY/Sj4nAKJmEfR+tw0ms0YkF0VeDk0oBIc5MnH9K2KQIPlgijaVMTiRom62hY01zyHUFEXg03nZCqkHeoHsCdEhTZs+AcsoSS/ezhiS8Lt1xaY1TPxZcZG5vNsnwmKuK9iAvmJyZmZu8KFUdJm+ZEG5iITEbM1pANrhvilYKjgAWxqJBhiDhx4p240lcFVFWdihdRqDIXSgxVcU5cpeIQTzupYZh0VSOI2ObMM6hTX1E8jBZWS0tsEQ7lEzFbykCVx4qZLRs21gZOlkMta1C0RCGcZfksWuuSiywbzjLLxgAGShoPGAhRuJ8r8CUfrpcMklM47RZPy4WlSRkwTFASiUcIahZmrY0zGWu6F4za9Rf8ICrp7LRJFzHVgRvmI5B++IufP7zzYbG8KBYX3jlfVYaJMpMZS+GzTyquWDCbalFlemjNDWsznl3j2dy5ikye5zkAM78h5ULKlaoTKdkRxJt8ZmaHbIw5OMhvPMvER4ffL1elKLGrXFWqwldlPj/kPLt2NJ+/+KtErOI04U28FKJh+ECibCmCIQKMEQkecHFjo14kCAaknJnF3bs/+bM/+/bv/8HxtRuqQjDxExNkgvbFzPEcLtfb4bq5YKxDEgBKPH/htfl8ls/yx6UjqypYVH6Wz0Rclh9amxWLs6osGALvxNhZnlmbq/qy8mR5WXqA5pkxzBcrx1VZvP2nB7d/jcxx8AcyCXFElJlhLHG4bouITESk4InEHNsFQpqUTRAofss2sgwNovsAOoiKbgmwsqX1HVbB1kbpTGVCmQ0DGhAN//Fmk7iloAh35yId5gwHMCmJkTxvNaFhfZ1fHK5UncJfwMyIcxKnUqI8g5Ywc7CB99CKxGm1gFsiP6TsGADYqC/hlwCIMyLS8pSqk6ik03ukBuxhr9nF+SJgk2EDZSm9B7FgtVoVq6WqCtSrOPGiyjYTEfFV0KyMNdZkHOetJ3KBbKTbYYmtjU6KzARlNiJOXCW+griw7YVwzi42YkVEogTvxTvxHuJUBOo5IEhYdwyTKY0pOV48K9H2SDDEzJaMhcmCM1bGuapXX6rJFSacZ4NUS1+xKhtjjAFbJaMcNDBia4kYkNzaeT63sxlxFqwTzlWAcJCEDEXPXVUyZK1quPojOdIACtboRswajiqAyTAZGyzbDV8y1nRQiSleYiOqUC8kSYVQiq7Y0byRHJLj8vjxez/76O0fnT9+uDw/Xy2XVVFVZckMJhweHuSznDn4lLA1hkGGmKWSlWP10AOTHYNtWVXkfX5waPJnxFXeLaVYSbmEnKuba7kUQ94YzI4Pnn3l9d/+w89+9t2jw8PFYvXDH/7oYD67eeP60eHhCy/c/srv/UdiDshVwfCsUFA4Fbg2cyEdgE24FlRFDaZGJgMoUzSAItw3BrI2//hnf/H9f/WvPlmcffMP/tAYFu9BwcJgaI2VceEnGICUVclBa/NQaF5RbJDi/ODg2ZdfVTtfLlcXqwImB3FRFTcPb77w0qtf/ZVfZ3tQVI7yzLuiOHtw8uEvTx58XPnyaHYIyGq1MKxetKjk5lF++9WX9fRk+fFbB8t7dHwccSoesI12A2YGbNrpTg3jFVCOurgk6qIcBhgzYIihgTEwwxDCLn0kTw6qIGHE76uAWWEAIY0Xx0QuBAnDDmGjEww2JFW0GfsC4sLVCICA86SjWWULcDTJUfAv94CoK0g8Fg+IDfJjtYcqFZ3f0UdvQ0rk19TO1JVwS/gFgTQ7CIdzyJcqBUDEBmTUZESk2SExq1+SVCAFMWkFcwiTE+WAgjzYQr19+PAzEjXMeZ4ZkHoxTKvlhfNVuVyI88GS5AO7zXJjcwptyiajTJjYs6j3voL4gD7MbLOMidlFJKL6iLWqegdITUBUPUTjxbCqBBIVcV7Vq/PBKYxq/wkiqDCD2BhTxf2psDAwsZKQEgtbtcphIHAmpCzOqXhgDmb1XqsVqScQW0tmBvZsQIbCZUNUlmEWFEQXdmGzWVBWCEQqFkJx+8AQKTGLUpgQSFeHBzYeplagr+E+MnDGxMaED1fFPYBwnSNBlSkco0wHoePtN+KdiA8ZCnHAdHBwnbEwBsxQqsri03d+sjo/PX98RkCW58Hk4r2Dl+WqKMqSIYdHR7PZbLVYmCwry5LwOM8zqBLz/Potc3AEM7P5vFRnQCB1ZeGXZ+X5I3gXMMIYzudHeuNZc/OFay++tvjwF4cZ/8V7H/x//+gHrzx762tfev5rL1x75fd+3+fXyuXCWKsq6TSlIUr+90RRa6h3vwJTCW52yQitSuEz396XwexibfbgnZ//d//1/+2Xy8V/8nf/8Llnb1QXC6lWogRjo69WjU1IyhhHzSdpPPFixsRD6j0EGM5uPvNsmR0sTx8fGFoaC+Wvfvs3n3vmpeWju3fv3j28cXueH7/0+tdtnqn3q1/96x//6I/f/cV3H50/vn44m82PlqtFVTjD3rlqdXZ++9a1j97+4MZnP7Y33gjdGM3kNZRIVPSQTq6BKO4ghBFFEmz2xBw3p9gQGUAIlYoABt5BHGwGzuAdqpKIdL0pGXxYjIonKTWuuAwi+ArOqXoKRq5wMLFagQ28I1cgHBaUitTDzuCdlqeUHyA/UJthcUpEajLlDNWSRCGOoHpxD75SO4PNSQnMZAyyG1CDooQv4FaQCmRIC5CqnRPPAIaxYKtSgWfgjHyl3sWr6NNdDOQWUAfjwTOYTH1F/sye3L2TZ3lms8oa9Q4ieT5njrejePXkldkk0NVwfI5AWZ579VoJVEVFvRNx3lVEmueHSpmTSqpSfQURSpMW0ORAELVCqEKFhSh1pap65xC9QAOchYsxQPEcrxK8sDAZcDw/HrYUgzMRvBMyYbtKqsrMDIi98wRHxooqcaZqoE5ECQ4iUCEJp+iErSU2zFaZvfNanUO9YYIxJsqPyAKIiIMCQ/F4Qhgf0RGNg5t1NIexMdkBGUOcvhVC6SohEZWSwlZbcJ5UH+7PIdSDvDbkKgHEFqAV1Bhmk2ez2dmDz84e3Xt47361Ws1mGTOpUVbKs9y74HnniWm1XBaLZVWWJrPhnENV2FCjxfnZ/ODAZBlMbrKZVGUA62DUUdBqVTBjfnTNHN2io2f54MaBsc9fOzp/8NEn739qrb1YLp6zZ3/3f/pf3Hj9r60WF+DMV6WKV+fijTfBZBa2lYnIWGarKsGNuT7ZGo3c3qv34Rya+tIQcTarqupf/z//q3fe+sXx8dHXv/b6P/u//B9f/rVffe2F2855jUduiawhYlI1gSQbw9E/nokYbMgYYy0RQST5ejIo3IVJ12Z2bumj08WXnr0xz49f+/XfMWQ/fe8XN2489+0//EfCVi5O7n34vsIZ5duvvfHCN3777PTxh+/8+OH54tpBziavfOXF56A7H3763Jd+02Xz85//yTOv/wHscTyV7EVJ4x6leKiQ9wpVNrBZuNcMliPTXJtwFd5R8PzSAn5FqyW5CkQqjhTKlkym4uG8QuKlaeLJl2BWtqSqpAkuPXylvoKrVCFguFKLU62W8GWEuWC45/BpG0U+h1QoL8QeID8mMNQrcW1aRpZTNgNA5hpkSWrUByVuDg6mPQsmEMPkUEA9pAIJ8UxBCFqkXwFBvELhQATKAIEXmIxgAa/iSJzSiuyMRJXYgiAiy2KROSYFA8xk2CizA5x3gBgVRrjrRJ13lYhAvZ9nrgobc1k2IzLely5cSGCttflqVbqqUl+p+vpQPxGFO3lUo0Wp1tGZ4jlLEhXvwvZO/fn4cKCP4rHW4LngmcFCxGAGBJRZJlLx4kWMN8hVxBULiqYtFVUSH65MUO/FeXEVk1eCOK51vWBSDV5H4krxJVE4OaLhWo205U/hoAkoixYFVQo+t4EHqCgY0YgTnLYXkZhwBiJVx2zs7BBsICZMYFIl8QSv4RIkBENtcxdJFcLkQazeB98MPte3fvLd5cX58vw8s+yKZeWCV4yyMd6LcxVBhShs4Jo8F+/JGq9wVcnEomqzzC2K+aGZZSwAWWuYwGyYbTbnfD67CSKy8yOaHUo2L0VvXj967Xd+//v/n//H8cy+fvvWMVW/9tVXn3n5a6UatvOww6FCSuzVUzKKhfnhoOS8MV5UxLvg4hvBm0i8k7JQcUzExrLJLipxXj774Z+99dYvGfLa6195+OOf/eKH3/vW3/27YBL1UC/ekQoqUmIALn7nPGwNssQBR0QUjkmkwRdO1pMymG02y22eFVVF82d+7ff/8dGtl/7Nf/NflavlrRe+cvLg4vlXX1w8Prn9la//+F/+vxdnZw8/++jGM7ef//LXjbUfvvOTk4uTo5kNO67npXu8ohkKzvKff/8nf+u3fqS3vq3xdm4OFjEVDw0+J56gxBmLI2OQ50TBWJZFW5gCKrS6UFmpISoKWa3c8twVBYgzAxB7MILB0VXV6kyd86IEGL8kNjA5WWuyOdlcxcGV8IWKB1g5995DCioXrI7YJC1/pjAgKCllBzq/xszkCzU5OIN4qFdXkC/Yr2AyEKkrEK7qMDmZcEjGoyzjZyeJoFV0vmEGGJxBCq0KIoJUUCUyyA9hs7B9B7YAwa/iLoR40iIeFCUDD0BI1JbFynG4AgwWNMtz78jBgeEAVzkvzhDZ8AEfhhCrihBW/rwqlqLI8oyIs3zGxrAPl2SE6HBVqfEiTU22cFD6Wl3YLUeyqAlIvBcvBCUJzt8Stj3CpS6qyXwbQEfFq0CJES4JiCY6aNwd0uBA7wpXqMkOyFjVuFJRNleQ+oCdwZZTAeFmIVZ1xEazObGRagl18RYPYkqFhyVdRcnO2WYaFM944SkRsYiIL4ksYMhkxmRK8FWBcF1t2DUXT2xE1OQHcYszwJlXVYEEvTjYiUgR/aRApCQEgoJUiOA9ffzhO6eP7harFTOvyrKqSu/ifZDOVd4LAUywmWFFvNMpnECGWsuZNVmW5XkO1cVyyY9PZ7O5zTNrTT6bZbM5E7OxNpsLyLMhV7qTe1WWZ2557aUXbn/918zP/9vf+PL1v/33/vArv/V77uB28BqJvlMQUa/ik4tSuHLRqXgBNPjxe1FjYHV9JwrUZjMn+XnploVXwIlUD+788me/8DbLZ5rPDv703/7L41u3njk+JDtz8hgqxPHGFyQVLvSJipB6jp97BlTViahnir51BAVbZgtjOZtn158le/jcV//6/PiFn/37P1msyoP5cTY7evTZncVn7509Xs5uzE8e3rekdz548/FnH778xtdu3H7p0cnd5Z3VeVHOMhhmFf7gtPj07Q9fuzX/6d3l2Zt/fPSt5ytnvHdhE1u8K6syy+dZlhkmAyWiSj2RmqND8Q6rM2MymBxK6kV85VdLuMI5p8SqplheIJjgXQHvhJnJGkNGnCsuQMzZDCYL18KoVAC8FFgVzpdkrPEunFAmFTKGeI5sDjYAqVQqnuyBEpEKtFIE7zk1oiAnZaGuTEehSCkHLJyjYA1nJgqOWQzKQAxjiBXigvuaike1gi+IAJgQAVCwgZ2DDEAgi8CispmaQ/IOIJBTLyAC5+AcpKoOfmXL1TKsUcThi6kqVeW89yqO4EREnCH2xJaJvAS/BxApUJEnoPSOwN5X4p13pbU5lJ2rxFU+YBni4ecwtonq46tM6iEqzIbDcUunogxSkAZbIwBQOqsXgEZqN6VwIZTWc0ZESSic91SG9xCvql4LBRmaWXvIxrpqpeVSvZNyBUi8bl89wlknAlcEYnjPJgMkKUHC1hDniSmqBjceMjAGysFSo1VF3oEgXsQ7SmdMYWwyD4mKix8WE4URKpeqIGOZTToZDRVR76HCxqiK8w4qYXGO3tdAVDaNXVxcPLr3cbG4eHT/waooV8uVc855UfWx11WZWVTZh7tDFKKGSbwPh9VEtKoq733lRUHz+dxX1fxg7piKi/NslhfZYzufZ7NDZZOub6VsdlAuzqvl+Rvf/I1/ALzw1/9w9vwb3sG5QpxP0YL1MDjN1BcFKMIV58HxNNyiThy/sylcercoitOL5fnFReXFGCvlCuXy4+/96Z99+N7q0cmM6f6H7y/PHv+Pfvf3lg8fzI+OTJapV+XQOGGPDeHEGsJtKL4kX0AkdiATxRsODBGTychYcAa24NmrX/nGB+98tli5n3//3/3oB3/ixL3+2lcfPbg3m2W/fPfN45u3v/+n3yX4m9dv3rx5y2TzsijmN547vH6b7t/zjip1RJTZrCr9n7/34GuvvP61b33t9L03D7/2cHFxsHSuKlblcqXexbPl3lny7EuFVM7ZbMb5nFT0/JHNDB9cY3ugqr5akghDyWTM7EVBZIwlYhioEqtotXBQsTN7cIOtDWScaBbGMJTFC4h5dg1spFqB1cTvOjLncxAJIF4gQnDwFRlDBIVV8bI4YTZKTFWpvmCoCquZazYPB+4JXi2TZmQysOVgzOWMoFqe6Pld+JLIABkYxEbtHGyhjsRrPF5oAFVxFHYVok+Gw/XbKC5ABnycvOGcViX5itQryLqqZGaosLUgI86LlJWrvKoDqTFQ8fACeGIb3JWJAVZGOOXKSuXiolotRJyqIsw6QlWufFV574iSATNtsaczI1V0rCMmazVcZBg94UhEIBJsnQCcc8aYcCdj3KViQ/WBtuTVQOGLbSART85Hr1VidZVTMXZuDg5U1ZVLKZbqS4IKlaThzjkFSJmUmVR9VYqIzeecHwUirQQRH3wCIqMEi3daVWwzJaK4FylwEi5sDq50CnVV9EwiO4Pm4fBguGLbewesSDNlWx8gF++kKsHJZcpX6ipSD2g6VB/2w0lI7n324emjh6cnp2Xli6J0Gk4ksBeELUuKt0sGVxKFCgGWmZlUxXkvCu/AhsSDWZk0n+VZlmV5BgUxC+CrStw5AC8a6ut9dQh1F2Jfeu5Lf+8/W6y0OjmBksJFChmIeDS8R/eadFZLSIOBi4zhVVHev/PZoiiz+aEHiqI6Oz8lhc1sns0Wp4/cxXl175Of/sX3Htx/fPPa4fmiePj4kbL5+JO718z3vvW3/44x1quLgyHkzEwmA9mwA6MygxxAfLyTmuLNv0GVc0reqZdC/JIuFq9947f/5b/40w8/euezTz+syvLll16x+WE2Mx++9ReLovzkzvsPH9+fZfmqXF2//dJLv/rrF3fvZNduv/AGPbh/Z3lxejjPlucPjo6OWN2jC/ev/+KTv/8P/8b9t95y9943N3/TWghlGWXqHcS7YoFq5X2lxQUZouxAaaY0V5Tm6BnNZj6bh4vN2M6NiooXgncVSAyExJHNJT8QEIoLIqYsVzPTLPfMqo5Aykzi4CtxS85mNDvg2dx7DztjyyRC6sWX0ck4fBLQWIaEOzLDKX9PBs6JW8HOmGCI2FhVKMXLuNkSGYN4wwfFW7hIKVxntFqSE8Ao52RyIobNaH5dzQxuAVdSfhisTupWgMLMEO9DFzVAsYDJ1YSLZESdQ1WSO1VfEJTMzDrnLDOcU1eZPDeQ4FHpFY6U1AQ3Re+9U3gmQwgYIiDPbKz1IuIDjfcEXbqqKlb1bpt3DhrOgIXb68PmOnFQGGuTWfB917CARhdKIfKiRAhnkkXC3YgW4sPGbVC+g5sGkVENW4fhAygVrW/p1HB9uS8WjlkBZgObKYcjbD4cCg6uSYH9BT/p4C0TfaI4I3XiKsAheE2QFVWtChVV78gwcxZQWqWK916QECkoj/qOMRTvzgrg7tX7YCAIXFS9ImxiyPrWqsDAFLVjUfDFZRCDTVGcP/j0w2VRemIFzQ/mmfequlquvI93tlvE7wwYAhNEXGZM5SpVMcY68dHorkqkM2st03w2C0AWHEF85aC5sZw8HZyr9ORstXhQuaXe/tI3b66W7mLJ0cKp9fZIcj6KV9aoiIZP+6iCyLBV0MeffvLhxx+tVqvi7LwqyuNbN00+Axv1KrPsoihO7t/HxfmDd39+//TcGi4qd3x44CHvfvLoUfGnX//S37cH14tVcfrogSiyPHciXmANZ9aaLHcCVVhjmUnFR3soaXCdrrwsV6uirMqyLItVnueHh8ff/8Evjm/e/vj9N0X0tTe++fJrv+JFrT+7/+ixzejO3U+YWUSO7fFsdnixEOXDZ195WfSlBx+//eG7P1sWpRddFEVG/OJx/uDBxY+++4tr1+f48z+6/gff9CWYMD++VpWFL4rMGKszOK83noXNK+c9VIi1EmHOzAyqpGEvT2ACezLCK3hH6pWAfA6yRh3zIRsLm4MMSIkzkEK8gNQx2KsB8gOeHcPOVZbQSpwXApERGHWOOWMGaUUEzI7J5oCqL1FVTIQMLIZsRsYC0GwG76g6JylJLbNVeBIPIVVB+Jg8vCrIl5wfanagomALOHUreI+qgCiqAm4FMmRn8CV5B2aIV60i5DkHX4JBYLhCIYBVFRIXuAWqpXVVKQQmNg5wFR8cQNQ751U9KZwLNxOIFwW8FwsyBsHKAFFFtINHv4FwlYr3gBrOZrM5gSpXRpW6tuWnyzMDcjNRpapijYmbj4AGvVNV42lpxDuFmSncTBrtVuHSuGAbJ1JiTRewC9YnkxBPFakvF6owxtpsrpiFI+a+WqIqQw4gQyZjzoiYDLOdIVyrAGjlRYJDBhMsWRZV9YW4UojJWJMdhPvFiCzC1y+lQriHFhRulVLxKk5VFMHwGFCN4zYCQJ6ZWMWD2HCevi3AbPJg2UheSGHvgc/vPqqqyjvvirIsVgxUzjkvlfPee2ssVJ2UJGAim1kRL04rcfG7us6JCoWr3aDzPDPMRFyWBaL3AomqeiGtylVROHlc8GeP/d0FTsr8Jusje3D+z376v/mf//48N2VRhY/ZgojJEFtiluhOE+5ZTBvZIDZZ6eWjjz/45OMPrbVH16/NcuuWS5CXYqFgNubOOx989OFHX37m9sndjz6593DlhJhEdJ6ZR6dnb3z9K3//P/wPf/0P/l7l/Pn5+d2TczAzL1el96rqKiLM89yL2iwnw96Fe1ayslhlhJvXr+WzXMHCxokwm2vHNw4O5p9+ev+jt9+SYjE/OCA2r37tGwc3Xnrp5ed++if//Quv/cpf/Pm/DYZaFZkfHIkXVX9484asTk9OHj8+OymKUtQfH90qLx7l89m3v3J0bNzPPr73+DP3ne98i8tHq0cVspk5uObKlaiabK6aeazAhmzOLCYcysxmKk4AKUuoM6pEopoZkxOBbUZ2RkTpwy2wJicizzY42blVQeyIABWYXMlQfghXiquoOFfviIwaq2RAqmxMdqjiyBjjSxQVTC5kvRctF+RWKo44AxkN+cdr7MUYJjtT76EL9Uziw1lMRQaTIXwWI5wZUJCZgQlupdWC3HnYoQZZFQcoigWKBXyppAhO71ppUARJVVeQJZm5cgYGwGRyGBs2Q1XE+qrygGFWJogUxIZRlaVX9RQszOyYrM1EAXGqJF4z9rAGxNFNxVDwnFCOd0gAqgZcseHAgqugFYXbXVkZxsT5KEoadmjUsCJ4lib/K4XGi/7D+h43EDlsHFBw5EimtKBKiHgmNiZTKK1P/AV8ZCISV1a+MiZjYxXMxho7F3D4Qh2xNTYL/juc5WwzNhbxLJWQVOKr6PklnmyOoECSUVGv6oOniZ2xycI+qIioKz2RqnDY6AmedOJFHJEBs/gygFdwWWA2wTtDEvaFGipEmQhklMPBKSUsFudhA1tcKeKdqHNBb+X5wUycEkn68LIw1BpjZ3CuCi2tIhnPiEikmmU2yzM2BgpXVOo9iIht5XG2LB+v9N65Plzyqcuq55jm8+OL/Ddz+sVy8cHHH/zg3//7v/V3frdYrk4vzk+LEpwdHxwczOdhSbDMXrUsCnWVEpssN/ns5NGjB/furBbnh4eHQbux80OdHRhrq7Iol6vHj07efuvtZ/OZO7nz6OH9u2dLsHHOZVoefOnF//X/9r+cP/d65fT9T+7eyOj+u2+/9+6HbPngxjMvf/VXV2WRzebEfHGx4Mxks7kxpqLSVe7s/GJxfqqqID6onPeSzQ88c2ZsRlw4970/+6GHc5XzSt/4rb/5/Fe/mWfZ3ffenN987u4nb4nS7OC4WC0PZrPj4+unj0+Or91/85cfu+JxfuPF5156496nH68uLm5cuyV0tnL63qn7ve+89Bqbh+9/Yl9+VeY3bXZSsQ2ugrnNTD4jZprNg+cKWWts2LuEwhsib3MJH0BggrGVeJQVM1nLZGy8WjZ8r9Z5mKjQi3PhxCrEc0ac5SY7QD6LBiZfgYL/GtR7lMtwcl4JfvmYqiVlB3pwEyYT70gBcxiwCerhFd6TOHjRPLfZEcgpkcnzeMoxePmpJ1/BexUHZYEwCfmVuiWgambEVnlGxpJTrS7UFTAZjCUyIKNQMnMAsBZE0BnpIWwGk5Gv4g5PsJSTpeyaFe+JSDRemFiImODIrirhY6pESpBws5WXcD+hD054LPAizIjfglYWIB2HU+eLqsryQyKrWpEIjMZbGBHdYpnCN2YkuplqxLDgxRBu7jCGTbiaPdhfkj9qRLUwy8mwzSIFQ7QmcTxfkdwjlcTH20ch3od9RiVfhe8wBvWIjc3Dxerx3ls2BBZxYSUMBiCR4PPCxAYmI3HRD0yUGUjfUtPwpUINJwYKLx7ZoclnNj8gw+qFfRnc/6QqBSXEhREM8UQEEz5JEDzxGFB4R6IgVjIUtlTgq3JVLpfWWmI+Pj4qitJaEbUEUlExjkDW2tk8m2fhtkJ4V62Wq9J5aw0TiRcCrJ2bcLpVdFXJ8ry8KPWs5GVlzjx7xwtvCpobElJDL1r/uMjL8iWe/7hY3Pm0/Kf//O6zB/bFr/yKybNc+HRZWC6ryhVe81lujYHqarkoy9X86NiQXZw9WCzOnfdsLRlriNVXIM5mB+Vq6S8uHt67+5O/+OlN5kMtP7hz78OHp5USiffO//bv/v5//J/+z45v3CrKapbl7/3Jv3n7kzcNuz/+N9+7cDg+PHrjm9/663//H9z80htFVdg8I2OtzQBitvnMOe/4+Hg2n1+/devw8MiH71aIikrhqsf3Hv3sz//02q3nHp2fZQfXX3jt145uvfjsoTz86Ijs6Xtv/yyfzbxz128cPv/cC2zz+Tz/4K2f5sfPnJ2cXZvdVJGiXC1Wy7sPPrt2kJ9dPP7R28vf/Jvfuv2KuWV9XpyX+XW+eZDDg4wak9lgG6F8dhROUCoxRLyrvKjhHExEjqhQkNiMGexKAOp9VVbWeg6fScxnfuV9uKeXMzCxmRt49ZW4SojDqUxrZ2Rn4gutCnVV8LgWt9LFBRjKGQjkQWRYlX2hbE1+qJzBWEhF1QJkOT8gFVcsgjlBfBXWel8WGu7kUGU4SAUVmEzJgDJl1nCEnhgmJwDhU6euJFWyB+Id8QzqAIExxAZsYOLd4kwzQU52BghcmbheBXhSq+rjl76C66oKOVEhDSAQvCZ8NDU7MZEKhK0FESGlcPUIa9hKVWj8qJfGWwsgspzN5myMiI+HN5PCGP18AHiicINXugw+AhSTAeVsDcfzaVJfV4Kwh0BRwSTmBGSGTaBj0eG7DkHVlPjZShCpeoVQvG+MowdiMFoREO46BYwN12YYgg/oBiizpWwOAomYbA7vJV75T0TpYyjep2tlEXm2OHEhr4yMMXwQfIuC4wV8+EaBInyTGMkLOFwrUp/hAYJ1j5iL1cXpg3smz5enZ/lsXlVVNlMtytzkrqwE/vqNAxGZ5ZkxNrOGs0yNLZbFolQV/+B0eefh6aKCMbmBu3H9eIV8Vel5gaXOnLI+A8kMnepvHOSlwU9K/6vXTh6Uh59+PzMGZ0z/3VxeeO3WrdX58dy+/9mnC/XfeOPV5778yp27jyxwURQgE+6NyLL8xuGhJ75Yrk5OT8rVyrsSzAwrorAcrvaunPjK3fnoox9873uyKl568fmPHt7/4OHJymlmDKv/T/8X//lv/O0/yDJTOg9jK9VnX/ry+w/e/fKv/NoLv3j33Q/uLovqvZ///KMPP/of/2f/+Svf/PY1okJ0UVaIrHt2dO0Ycnh8dI0ti3fGZJm1eZYFP7cP33nn4uLs+o1nTs/P3nj+lXd/+mO3XPz8kw9e/Oo33vnuD69dv7lcnMHwl778+osvvVGV1fmD986XS+PuyepiuTp79PBOUa5EpSiL5555tlhdLIrqpz/42W/9xtfKg2v2wOAgOzKH4SS8FxVfQSXLc2OteAnbzc5XNs+JwtaR8+KlcOIrEsvGZLMcmlVlId6JAsaqydXM9cCyP1RmgZIXYqHg5JLPiXM1TCJevBpjTE6cYSYiqjYHWGhBojyzNpv5fB721pQZSloVxCUVDm5FNkd+CF8KAlkIVwESEcRVXpzxy3jW0s7AhrOZ2DmrgDMNR1vZqmZQT/DhmItWC5EKZABFuPDU5jAMUZQXCg8iMjkYlGWQKvozGEPeqQr5EloS2Iqmr6ME8xeCcgqi8Bk/8hIvHmSV8Eh4bYpiRERhMmDy3kHTTdoAwuc1w8c3KWiOJhxi5Gg+ZwJ5VMGDlJk5RmMizplzIhvPPbICFdTFb1vGTy1RYmFEjX9MxhTOw8SzfqFIIN6pSGTYWFWJWwTpbpIAeqoafF1ZRV0pqmQtsYm3MjGTGhibHRxmWS4i3jmpCqpW0a5MTMkfzZgsQGo6KsgIKqc64iyYk0BsaMbGQlzww4geBfWnnICyrM6Wy6IsDfPxbHY0y4LrycN7n5p8JqXL5vNqVS6WK4aQ6vnpmXfVfJZBZXYwn88PiABmIfvw8dmbnyw+fey909PH5xcnD2fPfTl78dbqgw9vPC6ev/bMe6LWZjN2PjOzX+GLN8svAf+AzD8F2JcPVhak33rp4I0vPfulV27fvHXTkBKTr8pitbx7766uzl/9ql47vD7L8xt8/WzlS1ElWpXlycV5VRbeOSfel6t4LL8qma2K+qJwvvKnj88fPvzJj39y9869b7/x5VW5Ol+VCjKMyhV/62/+zd/+e/+TlRObz+Jti4IHH/380/sPn/u1G7fe+NbNg+vv33twcrbwq9X/6//6f/rH/8X/6uu/8TuL1fKiKPN8lmUZWMlmlZOz0nEpUGGQzexsNqtEjo6OP/3kzmq1+uGf/3tjbVmVh9B7H/zyYuXf+uf/5P4n712/du346PrR9esvfOn1+fFzt2f40SfvVK74+M4HLz7/8vG14yzL89lssVyUzsEeXLv+vFud/OiXJ1//lpydXby4OjerR0S3wRbiSR1DyRovUi1XWpXWMOc5VA2RYfLOExT5zIvo8kKcq0QFavNDc3A8I1UVYzNX+bAeq5mHOazGMyNslAV3qDCYvA9fhlZVsX6FaiU2V7KYHWtxzmzN7CjcBSVQdSWqBUsVlCaaH3F2oGTVVyxOpLTZzLD6aqkqho1KOOdTsrtQiGRH6r2Wp+RXUBd8axVe7RGREYrfiWQzA4fzWIhn172j4lzDh4YDdfFnogVKCzsjzhVeycCHA24MIqLMUvgIEaVD8WHbihAONyvigZ1wNgQULNrwkQ2FT/4a0eD8aeBdcH6KdvRgEBMfNhCMsUFfjEe4iY2J24VEhghMCJfaMEcgmxGxCXf9c6UwKsoEVXC8hAzhS9ERFZmZa48bYs/hGFDAhXB+ZY2iFgRmI64MZy2TAmvYxGvUI7RByHtVwBhrDkQzZmPz2ezgaHZ0TKCyKFxZlMtztzwPHu1kLQiMjNjG2484mvOImYNDkEJ8BSEYQ2yz/ICZXVn4siyqsqw8kxgS7/3KyflyVTpniJy4pcdCkOeUO1dUhZ3NH77/oSsLAzEE5/zFxUVZlNaa1arIMgNjZweUzQ8Wjt56/97JRfnLjxer/Ja6kjzlRCDvf+85/mcnr9wrfms2O6/01o37N7n4/qOXlt91VMlD1X9SnBS5/uqz9Mozx6+/9sKzzz0Lwwpgdeo5XDkN8k589emjk4u33zo4usnV4uj4us+OSi+ld+HyEvXOVYUq1Hvv432fTqpquRBflcvVyWefnt67+9pzt+j8NDO0XK1EQSDnPYN//Xd+1zDNMq7EswLqien+w5O/+LPv4+zs6y/eenx9fu5vlGJmRh+v5F/8N//k9utfKR2zOJvbg9zmeUZ0pCDnKu8qJprP5lmWiaiWxcXFBbw11lS+evmVNzibS3X2wYefivr33v/5LMsePXY2f/F3fufv5vkh3Pm9d9+69fwrH3z8rhd1Vbksll6Q54dszgmqnL3y+rcf3H3r/OFn/79/+6Nvv/Hs2enZzZNf4MatVenCvRdhf6Wqymp1od5lTCbLjMnA8Orc+am4yh5dswpRD2Ywe+elPIFqQRR0K1eujM15dkg2V0SnVFXxbDxxOtEc1sXgGuW1WolUrMJW2Fo2x2QNCKWriHMyrOJhjOHj4KRGxqqIc0stz1AuiVQhVbn0xCQF1CO/JvbAZ9cJQm5BZkY2gzilkszcULjm0MEtIU6hUK/wyI8rysBZuuiZSRx5T64AkWYHZEj9iohhDuBLcivgglRQGOIM2QE4V5PBexvO4DMxkwn6jmi60wOwTNBwj7pJV9DEo1eBRRljws1ZIkImfrzLxOsMItkKyGXDgQBrw5dAApqxYSZjjAVUvRDDGsNEM6IZITOcERGxEofFhcGWEO6YY8OGbdgRYEo2NiJO16ipEsMGh9LgEgmioF0SEbNVAomReJ9UxDs2Nuw3R5oWLNfWqjHx2CfPsnye5TM21tiZiffNq3FzqFbFqnCeSdlYy+RcVVaucs5YY7PcEoxWXpdCRGSzfOa9K0U9ZdYwMa1WhVucXxTLx+fn3pUHhqydl8QADjJ7PJ9RnlOW5fnM2NxVj5bL4t/+qz/+0Y9/8ezNo+PD+c3rR847Q0TZ/MKJU338yMmjM3sgBS3PCj2Y5/lR/lWsPoYuTfXX3zj7dz8WXXn7zx5hmc/k4kRJ4IuKHpO5zmdH2f+fqT971jTL8jKxNezhHb7pDH58jDHnqUZqAKopuqFBbahpBpNMSKbmQrpoM5lMJuur/gMk3craZDJZywTCBLSgaKCFRAMFxVDUkJVZQ1ZlRkZExuAR4cNx9zN9wzvtvddaunhPFPhF3HiEux+P873v3mv9fs9Dm7acrvyq8esmLpoqBE9UUrcl500Vybl6ARAAzHkvJUBOeRqN9pKn/ZTI7ch5A0LniJyqWMl5mtQMDTRnA5imaRqGNE6766uv3ll88T/4S1f7Yf///htq2qeciqQiOee33n7zzsNHV9tt0y6NLIk656q6Xj94e3Xnterktfc/fGezrKOWDMAcOGi3318+efzmN38aJHv27B27eRBDzDyZTSkTZ0RjdrGun7/36Xd+41+/eHl+7+w+IHnIH77/Qw7h2bPHapDAqlCtlkeHbf/lH//i1UfvZvTPn78PCE2ziPVCFOvl8ZSz3x0AZEzT2Re+ubmzef+3//Wnz5/91L1q17t1vowuj+KMEW6dEuB98FWDpiYZSxYp/fVFun5u4x7BuFm5xTFXS2QmFdMCebCcjNzsAQ2xNl8xO5OEZVJJjjwgUAi+2RShUpKZhRDJe1PVnIDZuQgIpgnBTEx9xb4iJNVsAICMs4p5OpAV4iDzZcFXSKgye3dQSiKsARQVZgGjcQCOiMrE6CvLk5nZLVK7IEdAIJksT+Ci+qYYYJF5jo46mCUiMgrgHLoIuUMtRh4pgvMAAjKB5fnsYgDoGjQ0GBx+Poy/7YbavMIFB7P3DXG+whKTmyHMTDQjW8mxc26WWd1e6ZiZyNGMa6X5vgaEyERM5Jl5ntCDzsckAEAwZjJAco6dR9OIWIF5BEf4OYUC52Ld7Y5zfsYTId/O1oiYAAF1/hPPvzcTwx8SWQmYAtJcyTZVLTPYi9m5gETswxwTIOdn0BGgs9vUHAI6H2tidysBAswpFenTNIUQUpq6wy5N2VSu94dDylWsfAhoethvS8khVuQCoFSMNYHML0wTGEvOKThHrqSU+767urke9lskjM2ybpfmXAFgxiZW0Xsl8D44H9oqDPvL7/32b//LX/nVjz567EN8vkuyzfRiyuhCuwR2Q9EvsH8O2Ispinc9luELrx0vl8ev/IshH1IeP75qsvQwbOG8gJYf6tjx87c38WizXAauA3hHBBaDN1XJxUoyJlFFIkYGU+cZVIAEAcwwhKYAq2ZNA5JDRJUMgOwjMs/DTS3zi+kWfFVKKtOkohcvXx57/Ymf+2Pu0Tef/KO/CTLXTyCLFpHA9Ed+/heO7z+Kjg3JAOe2jkkZ9/3S+TsP3vjk6vL65vwgoCXvRNjXUy6Xz5995cd/phh750TL0KfD0AFSrNrusL+8vJE0kua7Dx7eXO7/27/+33z0wfuL5fLh61/ZnD7sLj919Wp78zJlZcYipW2aEGO/v/7gt3/T1xtXxZvLF/Pm6ujOIzTb3lzfe/jW2B9MxfKoNi2P72XFjOH8+fVapkdXn/DdG+a7875hjp8bgqGBigyHPPVqqOT96q7b3GXHHCL5SlR1ZpQyU73BlrVMaHo7XTUzySVNlpNpzihE7GMbnEPUnCZVSSWF+XNuqmmccgIklQRSANmIKWdGIDAjkpzRlH001xrNq8siqkjsqzWB4TzsQwJNs1KAyjTHqIA9GliZDMnsD+ficLuoNVNE9TWCkElw0RgtT6BzFNyhTrcl89xjGcESSA9aWVwgONMOdASo0FeIwQRQBtTkqlirFgBzn0+1TQvNBxjHzB6I5oWmIbALzG7G+DATO+fYIQAiFdM8f9cSxrqJzWKx2YRQjUOXx75Mo+aCM7LRDDkQADtnZsyuXW1EhZhXx/em/ZXtbiowB0qAhljACtKthB7/8MIIcPtkZCSM9RKZNY2gAlI+bwMgzDH3+VTGrpjmaVLAV9c317vdom3WiyURVsHbMDJx3TSikIYhOqoad9n149iPU1p5SEAQF5EdmQ45W55AS1W3Ptam4kyAeDLM5IPHaIUKFJGKIK436AOCpVRyKVVdVXXTZ5Fh0KlDQkAvJRPIcVU1J0epbesqro9OY1UT09yVspnhpSUQDfubX/mX/8N3f+PXX//6T7ije1ptpVpM5IjD5HR4GJqrNoz2syjfMP8DK99hftB0P3Xv8rtPli42oWnwdH02dkfeK2wiaCmJaTo+3jw8e+N06aqqbtdHkrOWSVJixwaQUhYRIjJV8h6kmGZkb7dbmVl5pUjEIZgKGqoIzv2w+ewxm70B2XlAVNWckkguYsBBu+4nHh195Wf++HV7l169ODz5GBBFM6BlKT/9J//TL3z1az/x41/ddd29oxURGqBjRuJx7KGunp8/ffw3/59ffXR/GKcB6G7rP70ZxixA9PSTj1WLiA0yTqXkIod+MMCFoWc+2yzIGkBIo/zOt783TXm9OVkfnx3ff+Ps7p0PXj1frE5evXwSQ8xlWrSLqlmUIjDt3//4o+XpyeP3v1eFSqSwi+vje6WUq4vf/+Sjaya4vrmIsX75+Eebk6Ptoas5PO8mfTU8/s4P3jr+Gq2XRQGIkqjdwl6Ab0GuSMQu1LRYOQRQYbSxP5RSbD5oiLFH59EM1MWMzKg6o998Tb7m4A1I1ZRdlzKUIohAJEWT9LcbdyMwQ0bwrQVAAJp/GbhlWgHXc9gRzCSPJgnKhDoSUnYeOXCogbwGB9AwO5bRDuc2zg1wULeEao1QiByQQ+edaQExTaCiyGCiMppdUVxhvZp5Tei8iIAEsERFkBjcEiyDZZg7lM6jX4NUQLO+gEEVgA29CzGq8PyIYWYi1EIwR+di8Mwz00YBjh++uTg6url8tb+8IARiR4SemQCQ0YmyEnnXLlfrk7M7999oFhvHvpRUShr6fR66PA67i3Mp5ej0ATr2sSo5h1gvNicAKiLL5aYMdw5PPobD1rFzi7UQSt/pOJjK7dUUYCpl3/fOx5PjmjgQcwEmo5tDV2kGMwGsYmRPaiBGjChSpjJup2k/JkNOCtXmBHxISGY2DlPX7UOs4jQZuyyQpTSHPpfCRIWqiTlPQ9dd1Y6W0RexouYRdRoJLHhHjkOs1lW7n9J2vwMF7znG6GnhQihmVgqrTgDTfKI2BS2OcFG35HxXihnE6NrVsp8SgJFntUzKgQmZbfb2kANL//Tv/LV33vlh8atPnr5w9VI4gMEm5CTY11GWwC8moOY1xQubeiTNg4q87PzkVqFpp378E8PiVyCONX7jQUt61wwXjV8tF5bGkhMRSynEbDI/mPT2ZD07oRHnjZtJRmIthYMDUCSHBloKzS4rx2RGoWIXbgcTZghsOKtOeWaiADGa5ctnX3/j7sOv/fSVVcMo1bQv/TZ4FqNxmCgu/8Sf+/PtanX30Z2+6w5D7x2wI0SQlIzo537xPwpj9/1/9v/FkndDWqyWhuQZh2kSoPOnz4bDIWG4vrpAwrZdnBwdT7k47wIjVWHRtNnwd37rB1mBfX18uvr5P/3n2uWqf/mkT+MnH/8AzJjdYrFYLJbOV5HtnXd+//js4fd/79dyGaMLZ3fuf/FLXz+9ey/n6XC5+fjjd4ahM9WHq9Pl8ngcx3FKVe33Cb5ysvi1914++NoP9fW3Lycg50K7AYLZx06goKpANvUw9kyW82h50pIptrzYGDr8vPBnCODjraYEiQmZEG4JAmFeuyGaFFEyP5eNTNnN7UrJ01TSiKpMiuQpRGZHCM65JFKK3E7LVWanMZDjqkJaqYpKRpvBD3M2SA3QXOTlXZl6ywOhUtUURMsT6aTFQWLJe8jDjA4DVyGSAoGhjB2kAV1F5LEkAAPywJXdIv8cECEooIDNflsC187PfEBCnGm67BabjSN2zuVpIqIYwzQOzvm2XVZVlfsDM6mCb5o3vvHTi/Xm+vrVp+9/f9jvvA+hrquqlmnw3rsQqaqqdrVYHccqxqqdr5MV1Eykx2emksbhenMKAKvNndvogt1WrA1n+Il69g6AXMCqgcWGQ9W0I+6vcRyRyHu/PRzO93sBV8eQ0TtDmabrwzWyE5FkYAZFcwO88JVzvnYsami+qsKCQ4GZ16pEfBiGbrv1hEerjayOlEilKBGGZijFZFoRmRmFagIKVbUGCFWsHDv2THR5dXWxvcokZ4vVYrFAQiBesSOkPsv1OFWiG6/Xl9uk2NZ1AUpmICp9h7dLJmAGlYlVCFG01BROly0YkmO5PfAaluIYWc0AvPevv/7wxdNPp+r4cdq/XQUCqjB99Ti9f23xSsNLVUKAq79nUyCNiK3mhM37B26aG+/f6HfDq2wHKZ91w6fPL333MmX9yhce/thb+fhoTexBNaeR2ZkZMZlqSZmdM5O5vahmwKyAPE8rAcGA4HZ6O/eUPAVkdr4CmjmlCgiqxZAAUXLJOZUiROZ2lz/zzS+tv/RT+8wBzHtUywKwiG4aJKv9+f/F//Kttx7uD+P17nC8Wh0Mtjc3YJrGw/X1tfPh5OT0p/7H/9NVjL/6D//usm2bGFwV6TB5pznJ1fX2yeP363tf8qFCxFyMiU7WyxADIU5ZRpUXLy8ff/Tp9777by4vz9/6wjeHwe6/dnLz8tmzZx8hWJGyXh+98eaXi2LF8uknH7pYf/jhH3T9wTufUY7uPLhz996rZx8d+t1HH7+72+8WdTtOQ0pTd7jebi+GsfPUPB3Lj90cPrkqL89fnXxtuViucHaAlbkcYoSI6NXAea9p7F89Kal33hEySpbDjZJ3zlMMrl65WJkaoM3eczYFxzZHsVVn2xk5h2zeVUykt4yYUrSUsdexszyhiRJzuyYfzABAZexQbQa364xmUQUrIFm1ICGwR1eBqqmATiTJ0kHYAxo1i3DyICdBK+y95uzqNSKUNOVcLByjS7er+RmyhDNzQcEEQVkFGAyMzGBmN/va5q6e49sMMCBImTmGgAaSkL1qgdy7r/7kz3vnhr5z7BbLdaiqkrObBSQiaRyYGJFdDPViiYgnx2fuq9/a31yzD4vVJga/vXzlnavaRb1YE7mUhpLSZIZInt2hlG7opSRAakL0VdOGoJp3+90wdMGHQNQftuM4MbLmjGmkcWgWaxPor66V2DnWgqOxKTXGvl7cPQJxQQyLUpl6VDlqatJMlQcXssKQpuhCqOq2ik3w6IMAMrGYjlMiROTYlxKZO9BU5Gq/y0jgXBknAPRhiHWzWSxuQULEIYTgVghmyJOI5rLvume73cV+XLfu6sWrk36MMfbjUFVNNsylRB8kjc+3h27osqEbxxhi9Bydr2NoY2TiQOCZprHf9TkpdCKZHNnIRG27dN4RYDElMAPsS7nc7urgv/Gzv/jOxy8vdtZs8Orpi5Ojesr6u58cxlKC9yd3TtatP6odg0getEjJxYDIuaaJ+fLJzYC/MybvKn1AN+fXcLk/ev2LVzl8573nP/+teLJZAXMZeiKaKwczrweJUVFV2QEYuhAQqah48HMaUXJvwEjOEBwTMpOL5AMSATmVQsSWRhUpU0o5pZTLOLx51Lz9C3+sLO6/3I02Q0GQqV40r3158YPfmkL8C//5/+onfuE/zNNQez4MaZqujtatIZy/uhyHwTkvxs8vb7Zjqr/+R77+5LMXjz/YTSM6x0ya0BSGMZ1//MFXHn5lP+m847s57IvkWiukoEhg7uWzV9/5zV9++vTjdrGum+Xzjz949vgH3/v2vwjeNU2zWh3df+3No+NHd+6dffKD33XVcrt9udtvGclUT45ONQ2//E/+wTR23WFnoIToQ61qpmUaDjdXr1QklWKGn24lMF/suzuydfUZIUgRBCo557GbeZBg6hzr0E/TQOwhruZMBnJwVWOSy9gROwPVPIFKRmeSLScixlihC3OnhtDlKQEYmPCM1YMZps6uWkG9QlQUAWJyQcBEVHNvUwemIJmrJbioalAy9FeWBrxlLs6vMEQXKdSiCshURsm9pIMr2biyPJJMYFpcAF9hmTBNMBcfgQ0KipoWY0ZXAwITIoCUEVMCMLFCIMAR6hNkgDIgGYAAIviIYmAArgJ2BgAcwDnU5B688UVCGscBFKq6mlMOjEg4P8UVAGWauv31R+9+r++HsZSjzVEbw2LVENjFxYvzzz4JzrsqrjbrYZo++exxnlL0PhA/unt2yDZOw9LzZ7vegNaVf3C0QZOLy0tGmLheBm95Qna1j5umparxxzyJ3YzTNA4EOpia2SBFDK4lN3VdhYhS2KgK7mpMU5qW7NbtYlnFUFUK2A+9AC0Wy1jdLrVGtf6wn4ZhyhkQqqreDSOoYbX0zjnNr61bpTDlMo3jMHaYBrdoqqqO3hc1A8ySVbVY6qd8GMeU89Fyeff4xPtoaJ4IiIlYRD0gIxxVDpt1Xi7NxBEbspnC1BEYkTktqaTtNJWSV3VT0BHbpm4pNkWFEEfV0nWeeZ5iFPDAvlluHGodN2//3J94/tvvLtL23aefre48ODk7fbG/+Pobpw/urTfLJWhOXT+OhWJImCn6XASR6tohTFevtnS5LWFp7cKub8LyxNXrL9yHd955+hvffefP/sKPu3bpmpbMkJzkiXkwCVIyec+30ikzEzNl9iKCkmkexjMbM6qSC/DvYsyoqkhODYBc6vo8TUTYaHr4+tlrX//Wi53imNq68oRmBuzQB/q5P12G/ps/9vOnX/vJYTjEqlHVwHR107188eqtR3fiwxgZdtubKefQtIaOfNh89ZvDzauqtK/2O7UZYW59Lu+988Nv/MKfWbaLGGNV1cE7QptyFsk+xPe+//t//f/6X58//5TZ37332tgfxn734Ye/b8WKMBJ97cd//o2vfKvbXk+7C/WL1dHpJ5++Owd4mqZlxo8fvw+mSDQDIA3MeY9W+1ABkshty9qAP9qVOtKLJy++/PT96Y3XJfWSE6qaiaohE5FDNLOi5OLpa8RhNmYisa8qxzRtOymifU8pu7kHPUsBspiOOPbkAoIwe65q4OB8mDOdNputDGcWCztnmk0NVHA8aEkmBfIAJatkRiuSAQzjkmNtq7sqtwYA+jxFBeQECcMCJNm0w2qFAABMxBZagorII1rK2VEEh2AG7MBXCAgqoMWsgClqMSUlh65FXgEYlsmsABFJRjEEgnxAm8AAmcFF4AryYLkAobEAknHlrl6et03rQ8wyXby8Rsm1ZyPH3msppuWw314//URLrpkXhKDu/NNPCbVZrg/dbuq2R+v14EMeu8uriz5lBV5UTQW6T2V3GKu6Pjs9VYQDxMvtwZXsyiTkzo5PHpyc7ARTyf1uq0DH6yPPGBBzKaDCTFVwAbUfevZ+vbkTm1pEgMn5aFPWadAyrtfr6yHtlDFpmrbp5gY45JTbumLvnWNn2k/D1ZiHcRQF9JGR9mMualUVHbn9MIJk33NwucsqKmrmCbth3I3TTOIf1FA1hjBXENZNo1JASlGx6eCZ2qquqipHPvRDVgjtyjEqzO4oRyqjlMvd/ub6YspjYH+yWs0bSSXcjTk41pIIbdlUXEV2rApZlQCq4M1sbuWPPnd9txvG7fbmLt10lqqqPrpzLzA09zb3NrXL47gtRDgLkovO8BWIMQBA6kcfw+v3j6rgzl9d7z/eHvpJq8PQ7f7tD93+6c3dH3szpYlCIB8EIMSavJ/2BYD8/GI3I2LH/jbHi+TYgYGYOl+DqomwC+Tj/MGRaQJCIGdIuWgZ+3HoEfGY8Zs/+2PV8cMffnx+KHN5mJrg2LngHOBwUPran/2LLFPevcpcPX/84T/57/5e3493H77eHbqf+aM/s7r78B//f/7hB+/+EE0fPrj35a99442vffPRl7754v0f4OUrj+Hm0GdTEVHVTz592t+8XL3+TTUbRfb9UEo2FUZ88ezV/+u/+b89efJxuzz6yZ/8o/ury6HfPn/+yTiO3nlQrJslhoWrV1+7d/JvfvlXYs3f/94feF+B6aJdBOf6vlu0G2TYby8BAJEJzPuwrJvlehPrlY81u0DsHdp2TD2gvEo/b6n03TgNCkqfi2JR84w2LNMEasTOiMBUi6CnkpMkVY5AUfOkZhYbIJt1nRiAwc8qH+crjjV7x84je2JCRClFBVALmgCQDTuUCSnI1KtmkmSpFxHk4AhFFKZ+hrpYPqALBGBA6Cub9SU+zLQbMDNQq4/moKuogAgRKTAR5GnCkkRGnGngwiATEBkFJCIjxw6oRWZmRFBNk4hhaBFtvsMCKPganDOw2SMDJoAMVkAJRGEY5tid+9f/v7/PDF96801kV4os23aLMys8PX/+0jkfm6YOIfi2WJ6mUdkYNIl98P47ovnOZll7Wh4f90Uurrc+1Ow8mPr15tQ5IicEB0Ao4jQ/OF6tqlhHTqKrdkEhrl1QKceLBQIs23Z2L1VIG+fvGpgpmopokTIqHobRFLAkV/I4TVpEy7TZHKfQwDiI6S6rlbJo3M1hS6lrHdKiLRQJyknroK5cjOSclVszbp9S33eNI/D15ZCdDkg8qdUuTJJ311f9NGUgIm+AMfhlSzXDwjue5Ssq0fliWAcOwRUzINe2i0DovBtzEbWis+nLkcmd1fLuqk2lgEHlOEsx9gY4pakgQsC9SHdz49mxj+QDAILJrtt772OInsik+FhdvXyBu5eboLFavf2Tf5QZbz78/QhlGPpC4J3z3s2liJQzwMxKuu0/SC7MdrxwARdTksNQPXt5c/jshxwXy+Byv3hyvnt0T0O7YA4qBYnYR5MsSRkJHftQiYgUmUvJMOd8VCSPZkDOz1RFdqyqn2PD3bDfD/udATjQ+8vw1Z/48cTLdz86nww2y5X3XgymoTs/P0ez46N1RAlZC3ntdz9857f+h7/3dwNKVdfP37sci/zzF49f/8KXXz158vTZc0T47Pn5t7/7u8GHt7/ytdfvHHXPngUEU90Pg2f2SMM4/Z2//tf/Z/+b/zJu7u2319vrK9OyXC481r/yT39lv99/7cf/+J/8j/+Tj37/D7rQXZw/7vqOCEspq2V7enbv+uKqv3z+nR+8eO3LX/mVf/Q3h6Hz3r/+6MulTPvdflXXm5P7aiVPaRwnRCSwtl2dbI6JPFeL28cZc9vUw25IRoehOHKr9SqmmM3mDPlsd56LxrS6HR/D50Ua5NsuOqcxj8Ps8Z05DszsyELdGEcznWtuapBN8zQ5ljmwZKIGBsNB06B5smFLhNAeIzCAOQ5YOZt6nXqTAX0FHM2KTj1wgJRMM7nAUDi2iL7kvpSCWsD0toFjOpvxzDQXQwwpJyuCJaFMqAKzrcIKIEO1Ih8IBKcBLKMVI9MsyA4wIDYUIoYGrAEsaAgYwXsEtrkKLdkE0AUkNh1ABhh7960vfTGlqWTZ3uyPlos6uIRB03Sz2ykgx2a9WoMPOI3DKG272qw3R6tlVjH9Uqii996xB3bTNJ6tjwSAEQODC7UPFRE6RhFlxHJ6bECHLAIo/e7l9bXqzKIW7zgi5b5ZVXWoqoxw2N3kImaWp7GtayO3P/QvL15RHttYZcD9OPi6GVO+mc65WpY0DiV75op9Uov1IqE92/c7fQm+AtM7y6VzBCKTFBCrQhBEUQvOLdtGDJtpVL0NYRvQkJLnEIIGJCQuop6MiMkxxhBi9FUVmImwGNTM7OZQLs34VNES2Clq5TAlrUIoEZBJSp5KSYL7vk/jNAzbtmlOj9aOnUrZDtPVYUi5Z5689wSW8uQQYwh1dIsq6OFm7LZpf13l66ubq2e7y8W9t149/hEOB79sRdTjrPs00LmkSqVkREwpmWioQiBPaCzFg1ZNWK+a1bL+6NMXl/3u63e//D9vHvyd84uqTXd5UqfgonOeQtBCM7waiATRmJEYkUUF1FBSnqeQhBxrQAMT8jX5WFIau25K16+eP9VS7p+d/ti3vr65//r7zy4fP/3+7uYqVtVyfeKqpomRykCau2H0IawWbT/um4Zf7ftv/+t/VZEuqqqO3nzss/ZD+vi9d771xa+ebja/9t3fVhBmzirv/MHvvQMUg28cVI7VUFTR4VTs+z/88Ff/wd/+y//F/64+Oz09OUGwMpZf/dXvHt1/9Jd//hdXdXj3O/82mRYZh7FnZlXxIWyONl03vH7i3/nOr3G1eO/d7xwOu8Vic3J0YmDPXzz92ld+8mi1On39S1cvX5SSxrFTVef85uy145NTRDp57QvoXb+9MpOT4+UnNy+YcMzQ9aVuWnCO5hhETjruoGRgz6FysTbzmoaSshZRAMgTMhMhqHKshJBUHRqAmmRABnbsfc55ypncDGj1M+lUzDQlVYE8levnMB6IAKQYEcK1X96BUElJhIHa2ppjm/Yz/WZWiwE7kAKSQhUpzK+JnFJCACsJAM1H5QgqlkbQDLPBAAcgIheUHOICEcAF4IBgxM6YAVBMgUaarnUaQTO4CJYIQceMWmGlJqoqiI68J8cmCWZArAsQm7lHaQlxMvDBFaKTVXNTcOFj3Taxqk/rUPk1vXa367p916uIQVrf2TTN/cVyhc6pmRhKSePQ39zcpJTBeURkF5IZmuSc+6vrBMTMIUYiAiAR7XOWXBxCxcjMqchUspgZJKeWxjL4zsUoHMDUezdNGQwvr/ZFxIdwtjpuyKrAh34cciaCo9Vyl2UoChwdMBK5tq68q1pkdknVz50pxF3Wsc+Qd0O/W4TA3hcpZuaIur05FwBpvVp2WVIaSTWYrGo/FVu1rQt+yAJaHFFb19H74FxRBYAspSJSsDRNkQmJRbWUlMZegSZRQozeZSnETtQEyJD7PF50fR6nKU2HnJqmOfQ3Hk3Is/fLqgIpklPwTHULzpN3ijQZUKiG8w8/e/f7F88vhm4s17v67G5omjIwETvngCBLYSMgmFJmvvVFIigwIWIM4XbcwTwOQ2D36P7xogk/+PDVU67+7zcXzw32z+kXGw0yxSURO1MD5xEM2Kkomakau2Dk5iAyIrhZN6UiJUuevK/AZzWahn5/6M6fv+x2u7ffePitr30hnL7xsoOq3dy7S+vlkXfoXABmQJoyW2g9VZfb3bNXF2sPp9b98j/5Z8Or58fLitiJCMIU0FngMZcP3nvn7v2Hf+HP/Mnf+L3f++zpS3bzGwXEdD9ZnzUwOgRNUnt3vKp/+9vfXa7+2l/+X/9vP/vs/Le/81vf/re/6erNycMvxqrF8ZrbTRk+e/rpB94H57yIrFdLBHaEl88+6Ic8XDw7f/7pZnX05S9/48mTD5988LipG3bOOe9CvTo+m7qL3dW5qVZVc//Rm3fu3T/sdymNIimXpJLRHRs6BBGzKZUgue8POWdVKeNYxg6tBB9inSUnIE79DpBcqKJjMGMw59iIBZyUZCIGAERspgCTUlBj5xV5ZjUTM4KZqszspvlDevyAStE8zFhwAFDnkZwxiWRGFjSLS8BZ0T1P/4xcQPGjmQ0TzryzPPCsjCJPhA5EY22hBhXvyIAMREsCQwuzeoYBQctMlDQr2TSDJAODnAn9zKkgQNTRcpLhRg4OfYOhIUog2aRCQPOBDGBGSzKBKrCHaoEG7kfPL1lLE9yd9fLhujmk8tF5Z1rqqrre758+e5bFELGuIpqs27pp25QKg5naME1TKaFq1PlYVW3buKrRNJEMTCwcx2KHmQxpWAUfqpZraB07oiLFN9AAV1VUURKNaKg6pdR325JHlDKmslit6qqJde3IMSKpFhWumnXdAuI4DG1dMQVlzmaEKMw9E5EjAKHb1KyKpMPusLuZ0jiO06KuwVdgEqwsYrzMulqsg+d+dwCDfpg8Yxu9ItZ1yGAll9bTYbRXu9111zV1c7JYoKbLq8v3P/2kaVr2QQ03y2XTtmbEqh7KbpouupSLNFWom9b56JkDQpLkEU/bpg8RaQ3II3qoV4rgiSr+d315RfTemYFo8Wau7EsZ9ofu4uWFEpvB0arRYbs+u//qcA1z/YtARRBAJIOpZKnqepomAwsheM+l5BhjvVrmXIJjJFw31aIKqvYHnzx5aa/DuvpgLydPx598Pco0ECIxO+cVCV0sKZupobIL7CIys3e3CJAZUyUi05DTJOM0jdPV5dX58wu08h/8/M986af/eGmPtqOMU+q6PaoET2MuQxmnnExhnAYiij4q0WK5qJrFxfbyZ/+zvzLtb975x7/EZAUpSSHUxgVHsYf0/Omny9XqP/65n/rk2YvvfP/d/WEwBFBz3ovZWEDNGkdikgqqyMX55f/5//B//K3vfOfQ9/fuvf7ozZPU7Z++972r6/1bX/niu7/7b4lYTb3zr735Vtsury9eNpU/f/6saLl4dd62y5OT048fv3/+/DPvvZrtbi6oTKFdTSmfP3+y328JCYgunj1GGW8OfVs3/e5mnMZx6u8I1ot12r0ShW67X6tUzcIVRTRb6DyPBzBTnXUQFKrZhyQ55d2lTj0isvNcrzDUyGE+e7OjuQIopkQUaTaaoeEtiAPgFo6APoAPljPWrZWieYSUJM0GOQDkbECMTA59gNm+MEPTnScXHBMSFwOdDiwFJSMoUAEkUUEVY29qWQ0RHBFZnkVoJQ0EhqZmxQCBKwi1aSEwQEbfAhiBMoFzHgkhVoQGpiBgYICCRpAGQ8AyAqDNSmNWsIQcQMCA3Dj0kkvv3OX17mq7Xx4dx3aNoRkY16fL03sPRYwJxv6Qp1GKJCmxXbSWQeTuetmsj3yz9IGZmJkUSKXgLAHxseTJgYUQEEgMhpSmaWod9lPK08A+7lPe7/d5HCNaJCYRBCCCtm6C9y6E0LTgPALhXKVPud9v94d9n9Iw9Oq8Q2tXq9gufNMugnPOF+JBIGUTK3NJCczGEKpm4bxXgzJNqgWd76fpkNLJpjXmNoQmOAQ4WmMpEh3tp/GmTzkP3nHl+fnV1a7rDWDRLF8sutMmNqG+e/ZwFFMgILdTf7kbUy55msbh0B92zjkFcj4sF6mtIpqAFArxuG3OVstCVBQZkQjVlMyAKHpXFLKYImURZ+qIkBykw/bZ+1cX55fnL+p21V3vqqbqDp2ExYsnT3LBF90BjmERHYHNyUI1UNGcEiLGGInw1h0AYGrsaLFsTXXsutjUb75+H5F+78NPDnfe2n2j/e0/SGfr6u04FkBk9o3j2FDVWgRAdCFWTeN81FKwJBAxKZonUxZMWlyZBgIahuHq5cuo+Y/9qT+lZ2+9c9Hx1YimpWRAaqoqME4ibd1ueJlTMmmd91XThBjnz6jcf2SmwXvK6Ye//A/QjMyKFjLzHGrPBNYf9n/w++/cPTv7i3/qF7aH3UdPr956683f+M3fuNwexHDdNrnoIYsvednW16P96rf/jWM8Ojo5Or7Tj+PxRt793nePHrzxz//7v6V5Wi43/dAtlkdvf+kbh14e3H/4w9/9zazlxcvni7o5PjrebW9CCDFGZs6lbPfbw/ZmHIe+iImQC1IyIOapAJApbG9uZJhE1UqZxqldHo03LwDx6mbfvHg2KcSq5hDVDAAcRFMZh15KJmYDqmKlxcrYIVJsV4gIJWvqTUtsFiFWxUBU0czRTOu0nDOB2izQmy3aLtxCC0V06i1NyExSShoBDIgpLJ1jchWYgYrlySFQCCKGFAXARLNKAWMg1IwqzjtlRhcUWUpGnkVXCU0AwFIHpZBzLtTIHp3J1GHukQDMCBF5gT6wJskJTRgECRG8GDIGBMnqcBxwvADLSBWFCp0HX1tJoBk0QelADpC3gIj+BNzS3d2szs/PnePNvQexaWJwrx81RpSNyOy4DSnloT8smoibZTG83O76YejZWyooJYiw5GBcOSQtg8JhTEW1n6ZYhUPfpa4TgVg3CDgNXd/1knPVLhbLlU7lMJaU89j1DVMbKo+AYOi4ZiUyTGkQQ2AXY2hbBUxWEpEwh6Z1zaJerIgZUGY+RpfFAWUt+76fNXQUfBUo+tBWnoij51yKqZoUQ0qiKtmx65MagpoVlcOY9sOAAEPKgNTEgEjbsThfvXbaFpzBTvr80EtRMKhizKUgGJm0zrXOTYxHVdCjkxA8qAGjGeRpBFXvQ6zaq7Ec8u6N43UqMuWSchqnMeeiQFUVkd3MelNVUCklO4S7jbt6+vh6e5OyTKJEJFIIyYMEKOfnz6vFqu4LGxBqQ6SiM99TRNmR954IpZSSCwCUXHwMi+XCRPphNIPo+LUHJ8Mw/PDTT8y9nUb+weNXp4u7TUmuXtg4hMY5F6t2kbMAYhFkkHWg48WyqhyY6th3+8NhcD1I3110GRYev/mVN9/+1s9eh5NhTK4MaZwORc3HZrHOHF0IbWy8c4QwmR2mw7jbh/3h7vHRqm0Gka4fUkqMRnffOC/xBAciRLGSM6tEF9ExEaasz8+fPX/x4u7p0X/4Z/7y2z/xcz/1C3/m0x/81m//zm9+8PiZKhbFguCTPHn6dL3epGn0IeYsm5Y/fv8POLYfv/s7V5fPY1U3VXt69/Vv/vTP3lzt33jt7LP33vHt8sWTj5q6bduFqanKlOdZroiUYezYcJyGru/SsD90HSPmXA7d4eLF891uPw43aRz2+62W6ebq1dnJRgGagL2FKUufUirZDKBkH+IBEFVmADwxE3GfRis5DzsrBUNcnpxVzdKj5ZKnPAEAhzkHa3Iru0BhEJV50n8LpJ9x7DnhbNFWI0Jix7EBduhr8g5NRYulpFNvJSUASgmZXWw8k8kkZsxN5SEPaUrjlHtAQi8zmWPO2SIQhso5D7FRKYQWQnS+MjDnSadJFFCSaS4yl1XFTDV1ljtAQ99ybKkMcnODzs/XWwMC4Pkzy7oDK6aT2YDTFqWAFZNkOaHv3Z2To3unx+xD9ME3bcn5+tCzyZhyTgk27XYqL169UqC7Dx6EUBHR0WZztFwG5820LzKIpiROkMyGoc855ZK7oY+Oq3YF9ZGphlgB0nJ9mkvp+44QY9XEyh8TgFlOWZMwIIOBWhrHNAwvdztQKdMIJbd1vVivmkWTDKrVarImpTz2Hcm0PrnjYzXr56ZpPAxjXVW1Yxm7ECvmCKWMJU9F1LRyzocQQkDgw2E/5klT1jL5agGxEQABBIAmuCbWhuBjlYtMKUVvbdsUtbmYXkpyIqkMKZc0DexcDFUdwvGi9oTbqSpqRQTAQDSV/OrySvKomlW0Xayr5aabbDe8BLOh25ecsmrKyRNXMQKC88F7j0CiAmCOXSSplms35svzp1OWKWdQYMf9xfNHD1+btlvZdXsRSdRUDpkZjAnYIwJ6H5z3xOi8l1wkF3OGmaZxUlUzKznX7aKpm7ffuFfy0/fe/1CO3qqqw/vP4zcerXDsEUmlLnnk2JLzMvQ47E6r8sabZ/XbXzBqd9fdbiiH0PfDkOruy6+9Pu2vVw/eeLadng6GWWdeiBq2i1W9XPtY25xuMnOIdVW17eLs9CyXYqKM5mKIRcMmOCCQ/P/4e3//o5syBX3UEgJmhJyzqTK7QA4dIXIp+vTVtt7DO//oX2oZ7x0//Kv/xX/1yQc/unr2o2//zu+1i/VXvvrNT5483733g7g6ald3nOfu5tWQptLvLy7OnfM5pRFpc3y6ufPw3n364Ld/bXe4efrkw9OTu6plGIemWbat64aOkGYQmOTi2DsXYmUgWXZbNb28eNbUjcnoQ5WmAUxUi6iOaQRAUUy5qK/q03uuZEM0EZsmQ3AusnMqeT5Cl7Efx8HKBGIhNhiqaRimfkDv6rqpqmDk0JRNzQAJTCX3E5ohOyCaYe3oGAwIHQUPgKJtmb2utxAJIiaVMi9R0UVsPCFYHm3orIxp7MV7YAfkJYtCAkFlb1ojAIigjkw05wfNO0REKwDiTMAsp7HkAmAZiow9kJdpB9NhRssgAvkGKRqN6Cqr1qAJiLE+tjKgqwUMVDhUcBsdmIAboyUwwXELRWA8YOmV0Vzl+ilhKVWt037bP38aHZuU01WzCFWuY6cUm9XXvnHP2BEYO5cMmVxRFQB2DqAsomOHBEgGtaOS01hkuTwi70qRlp0jlFKcd1VVAzGcHLNz4BhyRsLDYb+/upiG0QE5JDQIsY51m3TvQCi4i64buu5mt11t1sWF4eKqz+Ww21LJSD48eXK03riqXjRV3SwcYkVQxRiXjYJldGA2TtmX7BDykMy0S/n84tUP3383p5TFvONFHckFZm6bpvHu9OgoQihGkkZS81qSApmaasrlpKljuwoEa3fUZ7kY0nYqLgTn/ah40Y83u72BEYCUIqZqtjvs++0FIMXYFOzm4cQ0DlO/Q7PV+oidPz46ioySUxKLMXr2BqYYYqzVoIL9TUrPzl/mXEpRZlYQKZDz9PKD94a+t5S8hgHItHimKjgmUjHvmYikFER27Hxd5TSJqIoMXe+CDyGUkqd+8CG0Tfv2mw+66fGnuyc/gjPYDlVsfuLtlSKY5NR3gaKP7aIOR+wfVFP12usFq2efXTy56adSyjQlySXlbVdAa/d4exA0MC17FQlV3SxXjh2amaS6qqL3nmc2HVeeDSDzbNoxNQwODDGl8vf/9t9+9w9+P6fp3etejqs3NtEhCWgpQqpI6skhOoSyT/hr//SXQrU4unP/7NHPnZfNb/7gU7l+8hf+s7/80fPh99/93mcf/dA5/8abX82Kq+CfPXtMTK/OnwNYkezRr5YbKeXlZ5+erOoupQ8+eLdp2pSmrt+fnZ61y6MHZ2efPP5RybnrdkzkY+XII6JzFTXq3LWUZFb6ft9U1XK5GUOVp845L5JL0RBqZtYyrY5aX9eQGAjRQKtKdQb8G3OcrYwh+NrWKlnSSFKmNMl4qJsFqqZhSP0AaJpHnAXvoKCmOSN7DMHHmpwDZvbRFECSQwsxNlWNvpYiKc8AVM4pmxS79XELaDEpKglAGI2cd4sjHyswMANjsFC55CGU4F0eehfCHB8JxIWciZSU0AqDIZGlXKQHBE09qKAZ5M7KiOxRheqFX6xAtUwekSwns1JyT+QBiVQNGX1tYJQGAkUkEwUCFLapEAIg2cxxlN4Nw2hq2370zGOattfX/X7PIMRM7IhxuVycHm/u379/587Z6XKVRXdTmZPIKeftzXWvetheUUnBu3XbbFYrBY9ElUP0vmqWiOAJCvms2g+jIhaANOVx7Pf7w36/H4dey9Qgr+o6sAdEZq4Wq3EcFnX11mtfKGbIFLwbhiHmdC/GMQ3jft8JilFYLQH0crd/3TOKdCYvc1HVJgZkh8ip5H5/c31zjT5WzWrM+dnTT0qaQtV459tm0QTfp9yXYkkVZby4yHi9aJo2+KO2rj3eTNonudkdLnf7Z6ptUyNT9EFLFiTzsRYpJROQGIhamgYzFTNiunNyevf0TslvIaALXlQd8YzkdUyMlosR2jAmQlif3a/r+va7GlEBplxySemzD55++mQakw9hTB0zqZmIEjJCWaRyJdLEGUqOWYQFCcE7JqJSSoyRiWf6pY+RRXIpPoQ5eRRiJSKkxQAXbf2lN+9N7z8977auWj97//psHV97eArsitl42EYk6q/WbVl++Vu4fHT55PrlITG7ihyGwOxUtR+nfd/tpomI6hjb5tQQ6hBDCNFxdG4SCd7NCXJCRAYyJWJAUzORWVCsQfUf/f3/7t/8i19mxjylouUHL3dFFm8d1Y5QFUpRpEzOHLEgTql0u+31xdOTB6+PSX7tn/3D73/nX6Sx++CT53W9uHr1xMCmachqq9V63G0fvf317/3WL5vq7CH0ITTt0kSvn374/e98ur253Gw2Fxcvh7EPIXgfQ92ePnqz64eqbZ9+9mGRUlXLOjTHZ69tdwfNzvuIAGBMyDmV5WpDvn757EdAjjiw8+uTe3VV3asFaVF5ryqzgIMdA9ItkdnUSi4ppXGAMoGqqeZpNABC108peHU+lFKccxwqM3ChMWR2fsYuMqPCLAnLkjozAaSxiO12SBSCZ2YTqaLHUJVUcF4yqsK4x6lnEPYVxEZKLiZQimhPCCjFkSJxKUigqslSl7rL0t8AEbkKLAGyiwsfKpE0966894YkWvRwgQgUa2PmUIe6ShBTEgAzVyMyaLLSE0YDIhfBBWZC9hgqyh5yD5oQnRmZTjSOEALECIVBEuTODeNUeecJFp7ONid050SQZgRNMWyqql4snI9oEpmGnPeTANA0JSm5G4bd1RWVZIjDoU/TFIkqz7nbOkBQa2O13iz3fe9CrE/vV3cejKVQqBSK845is2K/Ojo2wFxyIGxCCHh7NxaVTajEzEw9opn0w9CnnJF23SglKzrzbrVYFc2WJlc1L4uWLFCGYUpILqNXTTOoP1F98mDjvY+OD92e8U0mi/WirSsHYOQ8ASJE5xZtkxV3qcxqvZtcLnMh4GIQm8WG/DD0GaAu2QiJMKWpDnHsuu3QB+9NJTAunA+hqaqqrpuqabx3s0wNAcaccpGUctd1w9jJ1InacrEC5uBjmRFOhEyemJIURN9S+uj805ILMU25eO/HcZimJFlFxcjU8zJrARB0qlaKQsB5qkuOGQnAfPQzSZOIYl2FoiVndixFgA0ARTSEAGanx5svvSH7956OOXyR1/zDm/3J0eY4MlVaJjfctOVQ1ytbPjjcpOtBOFSRnZnmkopoMiDv75yc5pSY3aqtGWeKOyJhxejQmF3ORUDZ11kBAZIUADGwqUgR9d5try5/6W/8jR+9833nME1JpKiKAfzw5b6YvLluwRTBVKFMyTlmZBFJeWLnp93L7/2rf/Dkkx/5EJyPIrnvrr/09W/92f/0z/21/8t/nabp6sWz5XL18ukHAFBVzTgNdV2dHN/JabLcvf/x46vri6PNcS4l58l7H52v6ubo5F5c36mXL/o0laJFStMsV6uT9ck9810eYgifzruV5fpOIDazUC+rZu3clYq4UC+O71F9dHF9dbKst6/OBYB9MDMpeY7+z8VJNLJSmJiblSGAgVcVyQTgHM8lpIAUHCFILiqippJKgpwADYZsacjDDkStJAoRASm26CuVPKYEoN47dkQKCqil4GzpZifkSsmWJlTEEFEtpQImiIimkHvIA0iaadeOvUq2OdCbJ3SemrXVGyEWY9FkYlgGBHPOVyePVMUvm357yKUotobOyMgEiUALqEFcmYpjRh+Q0catjTecowGCDCgZYYC4wGZNcQHeWU4mgxmhc+7l+XmeBjOrYmiaxdHJnbt3z1br9Wa5AICrq8uS85SzEO8+eQLs+mEIjoNzF5eXu8NepQTCzclpu1yuV0suidlV602IVUFf1dWqictpHKQMWWgaTTOzVaEJwTnHxYiIc1E0rqoIZpYFsrFDAFfKOMPHqhAYafJusWxn2LQDzSVPWVMupXDvuBhQrOLCFdGm0UBUB48AbfSqWlQPUx7HlK2sAk7Bn1/fpKstykQlA7sQ/HFTbbsDsYPQ+HZ51C4JVDgQsY/VKjgHNk4JzG66/bTfqmZmDibpcHOYspkStc75drk0pKw2ZM3SlTTeXbXMnIoSYsmpTHldVScna7XVNKVSigAOabq5vrq4eKkGsW42i/Z4vQ6xiYH3zz7t9ttYx0Es5SHnLKI6X19VwWAAqapQEyWzsVifZNGii87MCLFZtpJLSXkWfs57zFgFZoL5w6PqQ4UE7F2eEhHdv3t6dn752cXzH7Gvb6L8zuM//x+tgWwZq5i7anPsj852nX56fTMmISbQwoieCQEcOyRUVYfBMY5DN01JDNC5RbtgcM6TR0SmLJjTmKYxl5SKhbqtfSglA2Hq+l/7V7/64fvvi5QiUkqe9RGzIfj9l92U7Y1NBaZqqGYpK5Kcntx7te9Sml4+f0Lkf+KP/OKXv/ljHz99uVwuL598ePHi4+/+xq8ichUb7935Z+/v95chhJLzen28XKyZHKO8//7390P3xqM31+3COIxj33X7onJ5eeHCR9OYJB26w0FUc8pVEx88ev3NL7190DBcv7h59Xy/v4IiZ4/eXFbN6cNH2z4fS7/fXo3Mm9O7R/cfHd+7d/7qR8Ssw8EQTXMWA1Uo4zglQOJYhaoFJGSHzM4xwBwU86pmZkSKqmoiGQnBI0ZPiE7VyYhaJq4DNdVUVYyqhuSr+YBj5EwKex+rCp035DElwkSABgpaAD23xzijfszQMQCqZgIy0+AdlKxSrCS5HbkBpcnyBCAIAOSNuIz7nNLnjTc0UzRUUw2R6xaJ6wUZWBpGgsmQ2fH8vxWdA80YHEi2rIoejIBrK4IIqIhcu1Dz+lQUc9+h9AhKqjBdgIJ7++E9ZL9cLtqqqquQgTBEBbwcRVWxOQKVPPX9fn91cZ5TMbOqaTfr1erk7PjeQ1IpUjyqcWDN3cXBBSyISI6btvg4eG+hhpyWTFkhjUXGfL19uTvsneOSi6SplLys4luvv6ZGBIg2n49h6tOYBldXpa4WlUP23TCFGBjB2CVyXZE+la7rhZ2ask1VRU1dIVLJaTdMU1HY93kaD/stGIYQvHfGy9VRu9ocd8MYGBfRD2nqx3SnDW+jXuz7i24ig34YUQtwQqI8djuwbNyN451Fe1rHa21HsUWo/Ab7XNZIy7ZFFxQAAZMUnjKaTP3h/Obw4qVNuSRRNRAVAHIxVlUTvR+mIU0DAx4tWhGtY+V8iFXd1JHAagcB5PzyWYjVdj9MQ5rGUUpOqQCic2Tmpykt25AKjsJr1ePXvphimG4+bpHAJKckpbBzYMpzxxiBAFXEBZ/HqUjx3hPP33k0G1eqiG8+OivpyafpstOlf3z1Mx8df/OnvlWji0ZhtYb2aFLvHNUks7jTsWMKJlpK6odhSmnou5ub6yK6WG7qdrFsIgFsu/5VyaZK7GIIhAiKzKFiQqJp9h4a/sH339lvt6fHRx9+fImzP2u2882AIYTH14dDSl84ah2JGaiCFL2etmbKTHW7/Nk//ZcChd2rDx/dO/nB7/3e0N2EevVbv/btR6+9dbyM11eXh8PW+5Bzruv63mtfWiyPr55/+MknHyWRo9WmbVdVVV9ebwm5lKJm+8O+2b3SokfrlZYcfBiGw+XFizYuPefiFpb7nKeSEwKoZh9jbCpOCqi5JAWNwZnp3bv36ENg76jdoJRYNw5McmZr44ocz15ImqcMKkJmJomZjD15BDNvkxSbxMh7dr4UmaYhDwfNk+XJxg5KT7ExjugcKmh3IOfgc62XI0zM3nsOVUoZyCHemnaQ1FlBHzk4RAQEETFAMCXvCdF8YGKOnsl2l1cldaAC8yW9TAA8r0iBeK6fG7uZLEnemauykWUtqZALzXIFKgBYbFYHmeURJRuiWUYdbBhAherVvM0wawjNIMvFx6aG5I0rAJTccR6Bonvy7FmaxllJWcWoiCFWVb04Oj1pYojR3T09wdXJTdsuj45R1RE550IVxCx33Z1VyGokBdIAaUA2zv3ZnQdbgd04dLvt0B9GwVwE0XJO3f4wD3FExDnWPB2vN+hCr/jB84sYY+t8reKRPNOijuyW4Gno+lfn1251JLFGSdGEvDNyuUh0fHz/bkHOZklExMYpiUHK+fLiFYE1ISyjj6tV8H61WoQQi1maRjBbVD7WTWDHoYq+H0RHo+QqZTnsd+1ydeg6Yl4u156clWxgJ0fHQ5JX4tr1WcNewaac0WvtGbx3SAaW0iSlDFOfcgbTxWpT+yAqV10vc4sYnUPbLBeOHWPbRG9GIQRmVwWvpo7QO+cQJtGSOgTtxymXkqZRRdKUTY09ajEAY6a6ijXg/noPi+Ozb3zt+UWXh5tp2i2WNQPllLim+UHG7KZp8BUTsxZxzt1Kp5AALQ3DnI30sXr94d1y2G9G+eHFrn+9/aXf+f7X3n7QPnjw8iq/dreJ7erZ9c0wiotVW1ee3JjLrktSSpEyFZGivl6eNitAqEIkoqyS0oQGqZRSxHsAQkJSs5JLkeK8b+rm8Tvf/9F7Pzp//qImG8e+5OK9+1xHdXsyAzBCfHVIfdLXN/XCo6qOBV5sX4gaEr16/vi93/zHZ2dn3/3Ob1XBL9Z3Tk7vvP21b427m+jw8Yfv7rvDMPab9ZFzoV1szu49HLub5+dPi2pgx0jbm2tclnHYA82OES0ldYetZKmcpGHH7FWl5KLIIpBLktQXLcM0kMHN1avWx5fP8PLy4vrlp0N/MNNpHK/Onx9v1oslOnLieNXWggRiaOaI5rWdRyKElJOo5lJMBCShCIUKERUIZts0USoGmNL+SqeDjAcysJJmUw5OBT2zzC8AZQIkQgNDSoaqKFMqhw6cD7FBZrLMCCWXLEWniQg51FKSGKCI824ZYzHKkhXQYYDgw1HIY29mt90SSURILmYFlQwi+DnOTgwUGLKy56wOmBBZxJgYAcvhQqY9IcLuKRLirDpzEdkRe+13xG4uIVjpFMQoAHvgCK6Z8yvmKyvJffXH/0gpaeh6NFmu14FvBe4qJaAtPbrUpZxrQu/x8vKmN6ibpjtId+hEyhXjqo7DlFqC/mbLCNvr68uuW917zZZH0Tuo6g1xyQm0XF++cpHvnBy/OIyb9SbWTZ8KzNJdgMO+x24MPj46Pa3QnKPFyXEQEFU62jSLtpATdgpQANmRdw7FxGAEGlPOaez6/rDboeYqxlVb39ssYtVWVeWdG3MmJGW+2G4lT7t+ErNlHRdqDtDKkIrU3t85PuZYXe4PDsE5bwDdlJNBVVWEJArOs4kaYFEJwTHisq4OQ0+mJDkbOkKvkrvrE1N/dIQATRXBbEy5ib6pmxg9I5no8bIxoizmEJHRgMQsi4wZB5EXu5vDzcWu688a7PfbcRxLmdEXikSOoJRCiGDgHM9ehXrZXPQH/9E77dmbN6v7w1VXZyNnIpKnpISA6EMEgDSOoWpUpeTMziFiTskHDwDMzjkfYzha1PnBaXu9dxX9hisvDP7uP/vuf/4Xf/Fgfrcfl2d0ud0JcJRcoYL3RNxUYSpMxTWNL1oAwDmeHSVEWBPOmhtYLhFx7pZJkVksZmAhVjdX19/+9W+ff/ZJ8H7y/tB1AEBE+jlJ4vZpZjab+w65vHuxe7BaomRFOjo+vri6nlJCxDTsDlvwntWw5LHfbp+89167PL66fhVjLCUVKX3f3X/45o/9kT/26Y/e+dG730tpJLAiOeWR2am1Ivk2XDxjdVIOvgzjRIQ+eCI3DIf99qpxpmEhucs5IaCoDEOfjSi07LqikFIiBGQei7z54M7R6xuNATj0UxaVlFPJGZHyNFi51Uiz8z4GJiQ0ZFdyxjwwuxArqhrnAzOLWlaoqnjbSwdD/fyZbzoDl5AZYXZeAwDN60FTQTBvOsenBdEQgJCYTMxyynksWcCQnQNy5v2QhRiYufLeAKYxmQHH2kz1NikNTLeaEAR3GzqbAQQla0oGqglmsYGBls/tRVgydBeaD6AFZQRiimtABiRjh8jzn5BNoYwWN+oCmhrXMy2DyIurjbN788EdFSmih34aStlvt9vr62kapmlkohi9gcUQ1qtlFWMIfuxGK6kKcdE0CBiqUHu/LAKKq9WdYb8P67sWfPY+CxiRiwHYT0rRw/pYH96/D76CY+wPh8OQEG2cRnIhxmpRLwAU2W9Fi3OqzEkCMcfoQU6P1lngcrd1aMvF6ubQ92kKde28l1LylFCVDc42ayYeh8Nuu18d3xGz691uHX1OkyEC+9pz0y43q1VRzKoI0k+T91GIBsarqZSxT0WKyGlLTCyiU0qsEmJdxVAH109ZzZhw6g+Vj1eH/fmrC4/UVkEJHblu7F+9eF6mwcfKxbZZrJGg73cyDgCA7KrgPLsqxhBr8m7ZLs+ONpsm7rruqutUrJ8Vub45PdvE/ZMX+900pqEfDYCZbB4uAEjOzKwixKiiTe33Y37y0UdfaBfV8jhNdw7D+aL2JakWCZ7JsTfvmMEgjWMpCRFExTs/L9CcY/beeWdSqKru3zvL03h0trn6wcv3XqRXdv7+D3/3jW/+8Rh9NitZ+jSOzu3H3LaNIdTMYFB7z1iMwBCZybPT28ijesasZoDMLACgFhwggCIi8asXL37zN7/Lc4N16p0FLUXNTE1EiW4xxLcdL4CZpQWGnfLN7nB0dLTg4Jhd07RV/eTps08/e7ZYLphIjQpY1hybBrZw/vxxLnn+JC1Xx5999N7jj95BU0fcpRERp5Q8T/1wsDmCimimYDoX9ZF92yz7lOuqSWm4unwq45Wgy2Xc72+mcQSw/fby/LMPoYyHvuv22ymNhDAc9ttXz7cnD19/48d2Q5e2l7VjI45Mra/RTILzPjBzykWk5Jx1mvI4iOQ0jUhMzoWiy8WqCoZgsw/X2DH7Wd44n75V1dSAEGcyrCkhIii72R+sYFSKEBCaKpACMDPYDFAzF0KE1ko2VQBjImQENVUUFUABANOihkiMpp4DMqsUQy4iSASA5CKCAaKBAXuQwiqINrcRyHRWUhqYeaD1a3p4Aakj18Ks3y0TajHJSEbNKdUrkLHwXeAKNQPMiglD9lASphEA3A9++F4RGadsZs67NI55HEpJ7Hy7XB+dnCKj5CQq+6Q2DTlnG8zhZMRV00bFXRkXbZP7HpkxNEJOmYaSVDOQ09QF71RKEhSBNEkNsqkqKZXl1EZuT49uMphzznkfghqAKYWA7PdFLPWQ0jISg/Vd3+fSjVM3fkJVnRCR/d3jNbMrScUKse9zRswuROeq0ei0rsNisXDW8KIAX02lCd7ArJhDrNk7hFTy3I+KxJFBgMg5xyxSGAl99CmLSkEa+mHX91PWrJZzykUAu5RGc4EcF+fMAIhc3d594wtsttteqyE7FzwvqzsBIUvZjamO0TObFnSuaRZ10/ZiNOZF8ONA131valXlmqqpPOxeXI/DpAB2+11ozrmSCyEas5rNC/VZALhexPOr7vlHH3zhp37+Zbqb8z6XgRFFtQgxyNj3IYZZ7DCfcECknwZ2QUpORE1dEQLVLTIvl4tVWxfEP/nTbz3959+/3Mrf+uUf/CVd/dRf/Z9szT2/2grhZnMcQhzE2qpWBJC0O3RDSjlPWgQBQuA2xvXmGJw3s8gISIxoRKkIAjFY7fmTp+e/9Ev/cNjesMl62Z4ftn0aEQwRxHQ+dHx+x7RbGQrM1Hvsh369XoHqi5fnqrperbzzrz96cP7i1dXVpXOOaLvfhaef/khFionOGkWk1Xrz8vnHl6/OEbEKnghmc72ImGnKxTOORZm4qCISkVtv7rTrs7tHm5dX12qyu3mZS1IJ4JyJ5DSJFgBUKfv9ZWBJCjkl1QKEY3/YXb746L3xy9/6SjfmV08/dUSxadvNsQuNgSK6YhCAwMcpSxZTQyFP7H1cAJALnpC6KfdDLzkRIxiEqkEfnI+AlHMhRL41YyB7VlMAYxDPvqiqSBU8k08AY7cdDvu2adrFMpmiimMGQJXkCH0VZt0PEc+5Vr39m4FSCiI6ZnZehUTNAUTnxDS4Wg1FMiCZFNNC6NATOIrseFZjGmgZJWc1I8cUKrAFt6cwHqx0hABaQMVAGTI3G+TahQrDWorKNAAENcCSZsY3wFwETm447LtxdC60wZmkRV2NYIKVGUiR8+fPZ5aLC26W8IpBiFVRrdtVyUl31+vlERjFxQLYg/fMrGCU0zh0QOh8BSpCLIjiKwEqgqVoXCwMEJ3HOh6zKyJ9343DGIOPziPSYRz6rkOzYHp1vRuuL3LfgXOT2pBTYa6P79BiMx2y98gEgX0dqyo475ypKjECZimBaZtMK8+EgcFrTob7rm+8K2X3ohuymRSZ0qgKTDCHKRCgjiE6but6FUJWTMjE5JnW7ACgqKQiRQrjhtk7MCAUs+CcZwJAEaFHDz1B14/XN1dd16UQwPmq8T5EVSPwVR0DQQ0yjtO21z1x9OHRSShqqqKWqN8fbi6LzTBXYGRjEBHCWTJ/+0NV5vtD5XnVhOurm4tPP3SLu9t9dlGaJuQiAIbozcSVYio4T/tV2DkFRLOSRgDsykS4ClWVc64rF2KlqRyfHP/sj739z3/9HVN97/c+HP/KlEbzMa7a1aKpmVlUCABAV3U8amoxUBVR9UwKpgKVZ/vcV5gkFxHvKBVTUwO83B/+7a/+6rS9oTyUkolw0dbzrpCJENDM/r0v93ZkdrsHAGAmAhinMeVMiOMwvPXaF8acDV6ZmaqCFQJfwICAFFWNHC2a9nDYpqGLjoeUD2NZVJUjVDU1nf96m3ZJkyDQods6JGZu2sZ5Xh9v9ofu6Pi0P1zXzQqZgObhDM77PQMMoSbi+ewMgMw+xoqcv7i82vkvt4++kpMO+5tx6KlqmrhQVUYsBlKyc8FVcY7sG5iUomZoxrOMzAWCSlNiZlM1Iod8Ky5xbGBaCqMhQi5FVAk5FektmRQw3W+TTIPmUaYBFPqr8+hYAXXqiQyJTQuZOR9CXZeS2VcutqFZcGyMHBE1nhK6qVhwbIxmRhhUlUCrGBAJIQx9Nw6D98GzgYqaQJ5ymkqeiB2H6BeLJGZazBTNTAuECn0wYkLFMmKZHGZCQA+gQlWkEOgmyzDOJmBgZxghBIKMuXefPXsuAk3b7BBzGp333oUQIxFzG9dNS8Q+eHJkObPzoWkQCBCKaB7IO66bBpGMSZ2byyVIhOx9uxbAImVKoxqw41BVyFyFColVhcCI4Pr6OudpGMeUVcxUoW7bWZzBxABqpTC69t5r66qKdUPOa5lMc103nfFkOKbkET0iaErTsNvnPKUhZxFBpOBDqGsfg0N2oF6mF7vu5cWF5slM0cXV0bGZ5iIu1M4HyLkv2A+dlky3MxqM0QPhZrXZrNeb1YoRPaCPwcyr2TClyNTEWNRExUqeRMeiY8r77c1ue+N8aNqFjyF4v3KuraIj1lKK5H6aPn72rO92JU3eeQQk7xixit75ivaf7bc3RVRV1ZRuLzs2U1xUjQhVZ84iGIIZbNqqH/NnH3wQxx/GIvnuuoQZ2XOrkRdVR94MiGnOehOh5GRgRIxIJSXLkxY/ZXPeWypDtp/61td/9/sfuEN58Gz89t/658f/yf/o/oMHoELEZrBaLKJHBFBDIqiYHMJUSlF0RAjgyRBRAAGdd0xFwcyzqJr3br991Vx+2owXB4hFxJIQcT9O/TgyMyLNX6Aq3J7FPpfezLw5kTJlTDkzkXOuSOnHjtjnPCc5jImY2Dnz3hFyyRkJU5qGYVg0zRwqHqZJRKsY+2FgpBCq1fLo7PT08uKibpppODCSqYxDx2GR1Jqmcs3rr148Wx/dXwbqi7FDvrpwLPOTd7k+PTo54WGyC6Obq7pdL1ZHrq4vzy8uhvygWR2vvJ6coAGFMNfCb3PCImWack7OOwCQXEw1VBEMQBRptsqCc8zMYASIKU0gk3NepRQRMBhyEjUtyUwgT5ozmM0iHiS2WWPOAR0BhCxiZVIxGTqaFRehSSWNQwZUwsHoBgHJM8UFxSV7H5vWhYrRhmkCmz1QoKqDCCJqzv1uW6SQGzVNSAgy6djBeABL5BuOTdycGHozm3OCJhOKaO7IudgsgMgvN8ZRczJNmItsr9B5dg5l0N0Lk4QIiGShVWSz4r709psx1otFE70bp5Tma6yPwI6IUhqlWN22TLg6jYwoRMOYCNEjxkVYNi2qpTy5WBXnCzISiiqZTTlv+0nJ7pzdAdMh5cP+ZkrThNwPkwJKKXns9vu9qKHzVbtsl0dNvajrxnuOIYTgCSBLCTN6iQkBi6kQMbU3peSSx1TGaZyHGmRiKnXdbNpFmzM6DwZTKV3XX19fSSnzrJHZoQ9TMeddXdepFGL2dbto1967EHxdRYfg0YhJDYmZPncXO56BKgZgNVFkpwi1cx5VDHLORiQKhhw9R+dWdXPv7CyGaGDzyVxEFbCAGdGiWW7Wq7TZIAIyZzE2USRC8uxkvPn0s9/rhmma8uHQzWkyFStFReZzFs4fZmY2lduCG9OmDYfDSGhI1N/0VHlsfEQGAETSedFFRGizctE5RwBFCqLNV8+p79oqIKOpGtBU9Liq//TPff03/tU7Ofj//p/9+s+dLF7/2o83wVXtclLohn4cLYSQBczMobWVr3wIjFltHnsTkYiNaZxSJjVkmoolKc3+5slv/Vqd9q/58elhf6nVkMrl1dWhPwR2olpmnSDA/AV+Ltz8d+e0IqJq8+2z5KImP3jvB+t2sWpqRPLBBxdWq816czRNk3e83908ef4kpQQAqkbeOcPoQyr5ZHNUSgEEM4nRE/sYnLKfQ3OERBwevP7W0b1HNYeLbnQ+1quT9bK1fqAOnK9KzvNqL03J+di6FUG5ePnMh/rs/ptJZJg++uzTz17/aTfmnIuYpFob75iYRdRK9swQvYthZlswOVMBg5zGEGsphRyzczV7xhkdYuoYAMkRiud5wghIOZkpOw8uQNQ5dUEuAgGxB0kGQOzmWQOYaCmaR0kjqIJmU0MkirFerJgduTDbmo38/MxlBTHzPoLpOE2eaUw5TROCSs4ghZwHX5GriBklZWADsKmTnBWmcvmKfEREco6IZzZRiEvnfTKylHM+IO6gJDDh2MoEyKyWEMTqjY0HnXZoAiJgigCu2RxZzmiaSwnOmSgTh+iT2bC7mrKMWXY31+ujo2nsApOa7ftxGgfLuTJxhui9omLdivPd9ppBRTWEiCG2qyNyYTtdLptqGqYW4dG9s6sEizUUtTwOppuzR0ENmF0xa+u6aRYiJTqqYgBEEQVTAmMEKGLMkiYxy4pSFE09WVw0VaxjjERERAq3EHJHBApIkHK+vLwc+p4Qh7FfLldn9+4ZICN4x1WI7AN7VzvnEQS5qHiee0Y0nwqYGAhADUwDIRCVklgLEU2lMOIkoGDGPCs14+ewRiZSQQXx7D0hIhKS3R6uNMmsKXWiyoQI5n00uF1zby8+6w43qjYOo6ipmJoJgKoSkarMfhlE8I5UMZdCRKrW1jF6HsZ8OBQRSNspVDGLIiL7MD+ZEQxM2TlVnVtEoDhfAlXKKGXs3DKQAWRByyWn8WS9Oi/93+nMUPp/+et/9QtfSfEoFkk5MzpAENE2OCKHxMVslxSRFKwUSaUggpVyc3O1vbkuOUkp5MMmcP74ve2nH8jY1UQPg6b9/vluKpKrECrPU5GpGxWsiMy+tH/vUGa3zzUDNb29bpsCACHuusOU/Hp9tN6cOHLtYrVZtZcXr56/eHpx8SqLzIvREKr1et33XWV6tb02s+BDLklyRjQM8fU339r1ZTjsrq9fGcL2+uLV08dXl1c1ydNnL6dpuvPgwdnp2QmH7fkn+6vLPZFIOb378OjoQbNc6kTOn9XNMjaL5d2HRcX/6PvXF9cheGOOBmiRCT0zAjrmCUFKAQTnAswvTSRE9sR1DFWMZjamVEoZRQkJUYkZgHDWx3pmAFAxtyAA1SNDNFEzQ5qZ2lZyIvKKRIR0S7WecWnoeQ1mjn1gUBFRM1UkZCZEAjMwZQZRMPLMKKKiwGQKWBQcO4qIzGCWSiEEycm0SBrZM4XK2ImroYxA3tBMCoCJ5JInIuRmqexLmmTYokxoCXztqjWSk5LBgMDYB5kGFUWO1twxUwQCE5Xs3v3R42a5kJR9DGbYNDUhjsOL2js0OUx53ulOr16FKqZxGIehrioCw5xC9MaBmBQ4GYlAXG6Gwy6XBA67XTcW7Yb+4tUFEk3DsFgsXnvrrdff/NKijlMqulwgggE5do5ZtETn6hCvt3sG8EjFQM0cOzNz7PZTT5NFtoboop9EJZVsau1yKTqLmZGREEzNEA1MeTZtONc8fDjz4UytmJlJZI8zFofIAJkITE1FVZbezQwBk1wHDwCIWRUZASTv9l1SOL/ZgkrrCc2Wy42SQyJmDs4TMyKhmZbs0LN3kaiYGThRCQ4ZIAkWNMcOVJoqmFnOOXp0HpGcqGkedy8+6buh6/qu70tRABORMk9MEch5UEVUhzjfOlXnk5oxETnvGgKDbp/SMO1vhuqsAfTzcUbN0JCdY6JSyvzah5mfh4QKPrhpHKTUSakgOcPc9482i5//0t1ff3zFr1cvuosqHUa+n1RO24ZDzIZdypMg5NRP0zBOIkLIBUzMTLXkHJgX7SLWzTROCob9Ln/w/ZtPPyjDnrSAZcdwr9ZhkpcUh1xUhZDmXa3MIHxARFIT/PyCOa81/3CcNgvu5o9uKuXq5moch7au97vLZ0/k6ua6H4b5iP2Hm4Tlcp2mkTgQYs6yWiwO3YGIEJk51u3q+YvH6EM39LMBJMYYq83p0XI6XIPJy+ePcRpO3/pCbFsfg/OeGTcnd07uPbr32p2X1730FGMVq7hcLV1TBx9E1DMCORGdUp7jE6YGzGImagagNuvgbB6KJVUEwJwYrQ5emYsKIqoaoiGhAkgphCQAIiIimhOwN1AiBgCG2yC08xEAXQhIICmzY7vNqhpoMQMxmQp4ZkZl70zVEEVVAFQMckFE74kc1TEMu21/+Sp1O7Ai02DkwuoOuuDrhfNuyiAA5r2xs5yROSzXrIvovDEzCCIVZFAB50TA0pByhwaEDusNNxsDgzyAAhGoSsrgq4VnyH1n3TWkgxEiR9Dizh4+8qEKnh1T3SxSTiVPR3fOxmEglQgaqqo7DAZICHW7OHEuON+2DeYcvUdiY06qh5xTTs55rldE7ENYIZY0teyOH30xODbTxWLpHDvvEdF7zmNBJM8wHG5205izFNEYQlU3m5OTjJhSnmdzZorsqlihSl033vu7i0IAOvPnZuYJUynFCBEgem+mUuanlqkVRAzsAqGqVI6ZGADAcMw5iVQhACgTErtgxgSilpMES6+2l9uxeOeQaEjl6ub6ars/Pj1j5984WaHKxXb7yePHN9u95qmuau9d3bRVUx8tV0ftoq68IhqCJ1RDM2K0yAwEDXksBTzXFaecG3crmiwiwcebq6e768sp5+7Q5yJgoCozosfMippDnF+rYoqiJsZIZf4XUD2S8z6EDJHSfupfXG1N3cMNTLmweEL0jjwTURV9zkjBl5S0FJViRA4cGKRxEItAToqMQ3/v7PS/+t//lf/y//Tfvtf19ytshk8vd3de9Yc09MG706PNWIRdYLBUBECbunGAqoJIPrQK6BjqunbEIXoY++//i29fPv6RpRGkqGZVIYDW+y8e4aqXjzu7OhQwCZ6zlNuZP96CPv8wbjb/899Ln/17oQ0EEdl3h313+MOfYuf+/f9wyunuwzdUNJUcw5WasK+LaknT8/OnWWDcr3bXL7KJqc6Tqf6wB3DdXvO0Ny3DYT9Wq+7y8vr6Yhi6cexV5PriVXCLOtJ224+78+6wRcfPP/no6P7DJPnxZ6/2N9fN5mQUMeSxpFGUANjMAMk5AFARAAihAhMDKKWUnMfJQO12jEs4O8glZ8dujmoFB1UMRUoRf9C5LmHzK07KXPQEpFtyFYoREbs58oWICILztQaRjFAVixQTiSF670ilAFIIEdQQgHj8/5P1J7G2tdt5HjaKr5jFKnZ1yr+6NQuREklVtCRSgmNFkQRZliIrSSOBgiSAe3YjCAIjSTdA0jXScsduOIocBFYKO04URVIo01KsSOTlveSt+N+/OP+pdrnWmnN+1RgjjXn+S1JunI2zsYC9gYW1vzm+d7zv8845He/T6diAKOzAjT5EdkHJlSZVjJzv42CtVTGMg4FCa/2m90zFrCZRBSRjIkfATErRhSvER2BgpgCAoBa3CGYtwfrHXCqTxn4jfW/HN6AN/cgxuifPHpdSWxMVqSW1lACk1rrpB/YBiLwP+yu3Wp8lp5oX50OZl8jkfUDvpnkmB5tIzrl+6FJ2SDQXCV2MZwMAmItZjRA2Xe955RZxFuFNbE2qtkPKtSoC+37ALvq+I0RAGhzFEGUt6SJq/p0Bb865SfMhOOQ1mqOq0ppzbqVrogkADjEgWBML5Lz3BGBgDR0RizTP6MgNgRltvWKpSFWcW1lEjqncT8vN7U3LOfZjjBFBjZxxfP7ebrfZ5FrnIpvon15cfv2Dj6poLtlUvI9mOvQ9EnfOmQm9y2WBR1xHwVkszfNxSQ/H6fru5mq/A+eZWVpbUhKVzsczfbssy+EwqxkT1dII0RCJKOcEhmAQ/BpMQUMCtnd6GdHqMAIj5xxsSQHK/fLw+g5VLj+4CMHV2hCAwGIIPnqKQUUUUQCYHTAhoEoDkRA8GeZSGGxelstvfvXP/uI3fvB//ifhg3F8/MGH+/NDGlboH5rsiYE9Em2IHfEQQxGtUk3VsUN20lpae6nuH17++t9//f3val7WtotaZV1EqBkh7gM8LbowHKsRQHRuXcWuLtl3BY34brD6/Y4zNcPfN6z9/pfWy9TvvQRoaKXmt9fXS6lVW1Mt81xbFZEmYseHcdh4IrBmIgDYVMjsdHxAQscirYDp3fWr3eZyyAXAMXs1MMCUM/kIFHzPUk+ADMCOgg/x4smzz35wNx0eGkcjR8TSzDmEVYdFYMfaBBFErImAKjEQUfBeDQ1UaiUkYDYwrQ3JmfMGpqK1lWoWvfOex7FvrTV9pxwYrJ8KECnSGiMhYSSFhxMMG0AiEWI0twbLzBExw2rzV1U0dUQMZCZNGjsPVRSgv3h6+fwjYDfPi9RiLbVSipDVYkQiWRcgk2KI65uvemqz5YkRnHP92aWiU9UlLc73PkRC0FocgbkA0lrOJorMxh6UAEykqZiUewMz3oJzII0KuOPhOAS3pAXA5brcvf6izlOupaYkqqHvY9d34whIecnWqoFGFy7PzrZDJ6VaiNX5U6qe4WwccnOGNqJddNx3FPvYTEXVyAUfKlJr0tQUwDN1jBkhNXzvvQ+JEA2bigIQgCkQ2hDC0qRJRaYEUGsFxNok+mCA07wMXQ+guZTWJITQr0txMAZTs2pQSg2es5oArQbXpTTFRoAOWnBNpQFQMUBVh2DEBrYJgV0c++Gjx1fBcUc4VSHCy6GbqubW9n1YVXwzbYbs2FTUOodUzBwYAWS1qkKmDi03bUCgpgj303I/nVTs7uEBRDl0GZjVjsspuKDMuZRox+n45nRaRHT9fBORmdbaRARhBYZaLVnUHL1TKJjZuXWvp+S9mUXyThQMVFXmdvcwmw9Pnl/0fXQEQCSqaEBMDokAiB0TNWmMykxKNGx2B62oFRE+//zl9b//d37r298JbH/m5z48u/yg+v6sH5pAbZWZnWNSXbeOqlBb9cHH0K8bhlLXJICHlj//r/7Rm+/9FpSEUlSKqoqtb6etFZapqUe4CGCKyTA4ePdWAKjaO8Mw/IFj6r/+7R84yH5PXVvfTFuJXa3Wm5vXIG1OqTU10xXspWa5lOPhHhE9WiuFHddcEazULK1O80wIgHy4e/O5gdZUAZY05ZwM7HB/83D3hmRp3Of52EQPh9vD8SEez3Iq3bifD4s7g5orszjvCUxVkRgBTJTQkL7cxBCqrqYsAzRQdc6FGHXt7CNenzprfSUaNbVymg3ReWb0Wpfl7tV8uOU4sg+RcbPdUNwYIhPVWs130KRpQWKpBijeB1HVmhGE2O+GQARLKmDmYxA1BAKDmhcEbK0+PNyBSF0mMy3zSfLJua7lhVChVWmFY+QwADpi0po0L2it64b++UdFKLeMhghccmJrqhUIVQhaQx+6Yawp1byg80CIoABk1hpFXJMK7MA1NHEs5fXbh5SrD50BdpszcLFOh+U0b7puvz8fhj50nfmoajF2XT92zIN3jhkAKITFtGOyJue7AdkdU12LvJpzjn3vXZOWGwBi57ioNAFFEG0b78bOH7Pk1tSI1geuiJllUwSqxqk2VIsxNpNUpItekeZcHLsq9jDNS04p1+iDHqfTaQrBsyMwYB824zB0/f3h6H0AaIgQogfRIq2J5lzUzDsXvQPiKgIATBqYTjUDUiBCM2IqqpvIpcHdnBjBRGqx3nkgWqpFQjJtpmJ0qpURBcyZriJsMmhiQH5pMi+J2XEIO9xG5vefPFklDzJzq2bfBSIoVdLdZz/44jv3D8fWVr87MlOt0mqrTZiQEU0VTRkIAU0FaZ1IEWBNNQEh1QZq4D332+Hk6/FU59upAH3to4swBgbzzgORiQIisQvM2hoTqSlSoG7rxr1LR1WcT/c/fvny//3/+0E1+8azy1/6o39Mz9+PSyUyimxC1UCMgJlRHRIg5qJgVlVE2uA9R18FoOYv/uk/eP3dfyFlRimg695JqmoRVVNVq2KlaVXzCBtWbJDMCM3eGc1MFPjL8+hfOrD+ZdXsD776k1lund0UQM1Ox/shRmltJVSbNVp9Wwi5pCXNvu9ryc45SAaIVRoi+jCeXVzd3R9Vb3OapsO1G7Y1zbUVU1kQDzdfQB6y2ny8n6YHJPjkR99xrA931/e31w83N0++9VNYqpmwCRObd6Jrgl6JAMxCCGBqost0AlDvPSPE2Bn79SQSkaZK3pmoM/FEzZjAiAne1YhUQ8K4iTvqhsEBEGij6NgTmnNETK0pgZJxYBIDMBBphEAxlCpSy8NJPBMhN2lSCiI54iWlVrLVxLg+I1BUSVs/brGP8/Fo2mqZ0VRrUalS2lqXR74L+20fAzlvYasi0Koxq6ktD4ebIxKxcyjF+UBhwNADALEHJEQFJWCEdcozQ2ux3yCaqrqz/X672c1iZqAizEzswMDHiKbDbidNHBMgppSDw23fsUqHREyKuCgwYURA8EWNQKJjEXTRBaLc2nHJfXBJQFLyzBS6LhIBHJZ22/IYw8AGQFUBDRShiKgoMTFYK7W2lkudlymEkIq21t6FStJ8FniZT/Ph1Lg7PDyUkkXBwFrO49ifnV9IiAskacoOANTUWmoAaIZNbRgG711t1lohMwJsIp6dAU6pOEbyDoyWOUVmi5hrW6r0wYHqSesR25KzW1ksRkygyMlIzZBJRJ1BCFyaIhNiMSDm1dEB59uNKhiCa0ZMVbSZIUKTVufUalpuXrz47Ivb22M/xPVW3qSVUphJTdFARM3MhXe5E4BVFQey9U/0S4fCuiUgdJ4HM6faDkt6lV6W/OG3noddz0wKxohgxs4BYVMLPqhp6CK4MfSbfhAQ+a1PXvzT3/xeFd1G/gt/9o+e/eFfuXmYVMQcE9smMLdGBLnp6XQE17F3ZDpGTyrs3FKbEGteXv2zf/j2t37DcmItZk21FdEiWlVLUzWook0sixTRqoaGXtLTD77aXz79L3/914mIEFVNQOndZfMPzF+rcKa/by77yYlGv6ejvRPX1hONaA2HAq0rHgJcpzyDJqJNDImZx3i+pOSQnPP99nx3/vTRxW5Jv5uzb1Kn6eGj9795d/Pm7v5m/RViKqred02aqjgOTaSkZbc/e/npj9Iy9x5XmL2sDILYA0LOBZFW+w1araVcf/Hi4fqtmQIiMw7j5vLJky4G4G7sO0FiZiBKy5LT0nI2E6lFDGI/un50zo+PHq3BiUAo0gwweBe8V9Ncm2eHJgGptQYARMCOg2MViY4JiFysrY1DD0S1ZGlSW2UEc06pL8vsCfvOx/5RUxU1R7odL1uaTWvJBVoGKc656Bm9c9wpuTKd6uk0nU7EZKbE7HxA14XzLSBhy5huLD+06S36gbut9ntQJCYkB1pBlQk5Bu+4LofVn+wiQiPm6Jvh+ulvVTh6T+hDl1MKDjt0a5rhzPktiaoyYxe4IZmatNqFDpGCSq41OL+05pRUbei7yNJqCQbQdac5BaqjCwiw7XypTczSmidRdd6RNmAm53NJX7x9qzV7QlXAVu6aqguK5Hx0wV+/flPy8uHzZ9v9OZArIgoYnOu7/jQvzntAyCJLWjaxFxXvAyISUa2NCB1zU7XaEMAxxRhMjQDVjBGpiwh2zHmaUwgBEU1VRItIa0VbLSmp6ul0VBFQQUSBtTA8bnbbq6ur87Nzh2hEnk1UmKg1UwA0RNXWpBrAyvIn8mgC+Obu4f7+bkrp2YZe//ZvffH5F2kpMbqi0pq0Wr5MYqOqNlHvnCgQERkQARCtSeImzVQQANkxIiORB4BWEcyhDbEd8+2bB2zywU+9Z7s+BiYERlwpgOzY1ACJXaTYk4tdF+f7u/H88Z/4pf7pxeYbX/vw5/74L//mx68FoO96IKpNTFtg50Poo9+MIyIuTRvQbcq9cx3hdtNZa9/+tX/05jv/XGsiySZVpBaRXDU1raJNrYoWETXMYkWgquVcPvzmT//Nf+d/9ub129/8zd+cDkfnHSKoChLT75vO1nfmD1w1f9+Mhv+1/cBP/h+7zX53luW21lJrBQIiRiAzY+e32/OzR0/HYWqIx+MdqiIhIsQYu3HDTMzeiAm9mT7+ys+mNN+++ZydZ98RB+OOXAfIBhj7sTZjH5B5Oh03wZMZsZPgVaSUTESeUA2cp3mRw2muOU3zosQq2NJS0zwdp+Nx2l5eDruLualKk7QgACG2mjXNraRpSSiyuXh0OW6lpjnVruuICMibGjG1WsnMee5DMLMG5J1zCV7/+Hun+1tA0ppUxIWIyN3+0pzLm2HoBvbBEfSbQWozhNZUtjuRhmD9MNQmpYmJIkK/v1Q0yRm1AYMU0dqQgRyneRYf2XdWFyMmDtaKASIHUUWoBub8znc7h1WOb9rd79qhR+eVHfoOgYhJrEG3a0CWJ2iLmblD0Twdh90uBBd8EGmzwuCZVEYWCy5Pc4eR2J7uYh9jdCwKzoSJcqto0DNGh6np4Gnru+BZeg+IxQzQgsdFKTjyIZ4PoSo8nObA7GO4n4WabPuuqhWD+XDaR7yIEcwqR370yNjfPzwAUc7Fiwx913Xx9jgvKZ8/fnJ/PL1eWidLjIHZOUcuBiDcjkMIzpNr2lY6oJoRGCAQo0MHZgDqiVZDearl/vZ6maZaGxCx80vOJmLIYggAwXvnvXccnDvbnw2OHLRcm+GzsevQTKTdp9YAnePtZhu9h3cFONhUg/cESKgqCoBZpGVTEU9oYFOpIu205JRybg2JXXn49OMfi6hjktoMLOVislplxTObGREhArMDUGRSVVIwNCVVVVoDSmDI3ERKlZRqSnXONZVmbNFAHub2o1fPP7zabkNA67sAaOyirVcI58BFpdhE9wwXH7z37A//aQDooz/Oy6//8DVQKDm54FXEezfuL54+ebzbDL2nwBSZNypLlQa+iFZ0mJbv/4P/++vf/k0smSWZ1abamua2HmS2VM2tGUBVa2JZNJVaqvzsL//pP//f/Vvq+w++9o1/+9/9X/1f/uP/w4++/zunw4GYBRRWyBbYmrcnIgMQkZ+cVrgeZD/xbax6Kvy+qU1ld355cXaZwJeWW62I4H1QVVMJPmzOLvpxf9b3t6fJO9dqcc6Fro+7s7Mnzx5dXxvi3d217zYOLY6X3/rFX/neP/sHOc/jxfPt0C8N9hdyd/NGTUK33V08aXdvun5TGi5paaItFXbsmV0Mq3dszpW9Q+/7zWY1eSBxSQlMvA9EnHMm78h7EWVi6rdoStZidLDdAWC/LDklF+M0J6mlthrq6jICdM45MjFtlZiRuI+uzKfj/V10sOl70P3d/UOt0nLBYn7YWG2M7ub64W19bVp96MaLq67rg2MDHKKH2DnvakomuumC984IpGkrmYfgw0bBUsqoUEXYuYC++g5MTAZkByoIQAAqFc1MKjrXFASJqA/nX/P70o7Xmg+Q74G9ASkS9pc1C1qxdLI2o6nbQH3/+XmqZqjHZe4d+7qgpv1mI4B3c3vxyecjCXl3Ohy7fhOc88zahPthVptMY+wC8el4OO/9xdWVJ1TV/dBxNzhmdOy8V4DcxNQY7dHZ9n7OuUrv3BCcJ4hEcXB1DNO8VLHoSKVebDeietFfrSU2O9aBpBmk7f4ouBg/vboEwi54RyTSHDt654O3VIvmqbV6zJUQjinneU7TBFr6cZOazsdjiKEpOOfENKcktSmR906lgTkX/Aq62253RAQqNaUKeDzcM0PvuOWs0i4uzs7Orh6d7x8xdcyz6lxE1ZqqiBKSghJgUQns2JGoMq7oOTjm0kQAwBN77wnxbL9zjufvf7KkpCLe+ybNOWbClNe4D4ms5WGgaiKNmZjWx5QCkOlqGUVibM1Kq7nU45RKqat2Hh1XsoR6anJze7ot+vNfO//w6UV3fsWh01Y2BCK1mFHoYzd8cDm+/9WvVdz91hezqKnZzsenjx83ETHru67zDsCQ427bd86B6lyKVAnBBwZWcdHVefruP/hP3/7wu06KWAGr0lppUtSKWhbNTbNIUauyimWSSr16/8M/85f++s//yX8FV3kb5Ctf/9q//T//d9+8fvWd3/zNX/t7/9nDy8+956wwV0F4ZxtG/D3v2O+/e9qqkcEf/AeABt45x+j9l74cJOdcLRmRAAgdK7rQBy/kfUfI3odhs+/GvWCnRk2N2D358OvvffTBxBvHMt2+un39WejH/dMPeoO7F+JcNFSOI3dDt9sjO/Rdmw6fffZpMx7PHnHw+/1uGxwwxsEbOEUMMRZxNm7YsZo5JpHWqpqqQ6srchdBDKSqGIMQE4xD8Az73dZ3sdbWJDYFBaw5I77jYbR3hEUAkfubu3K4K/NpNr0n7vqN317Fyw4AWs3MDAqtikqz0qRlZC9NlmWpPiJAU0Os3jtpamYIGbUZO1B1RE1VDdP1a2MOwyayq6UQIjunSsjunXZJBGbkuX8ZuwAA8JRJREFUHBNqmdn34KuB0mrVdWzDJY+PDN8F2Fb8hi0PWhKaAJIBuX/0//2NLvrN5SX7cH9/dEido2HwkW7csKFhK6H//O4hzfdg2NWM5T4C+Bjrze3tPM01iWjOyfsOEboYO+e8DzF2SQGYN5txt9koArnoEJtKcKjAm83m2dmOAnZdLyqkMiCcnY1IJAYwATI5woiqrjPRoNm5ziOcBf+MvSI3kZulPuTaWlGFKc8IcFiWnOq0LCJNpJW8tJy7bkgpxeDaMkvJz957j7c9mExLZudM7cTo2I19F/vOO+e9N3JJLPjYBa8iMfiSZmv1MC2qCiKxG4T40EinfL/kJjo6vDrbBXYC6tHEERMbUFVDYlUzBDEDwFIleLfxXkUNTEWWnM92O+9cj/njPPkQQ1tnCDCDVldQBMjqLXrXCYaoSIQqSo6RSJqtTu7WpDYAgzmXw7TU0jrP5/suBu+dK1WOU8rK43h19fjxz/zyL73/9a9ligq4d/jVM19uPv2Nb3+79hdff7z/2i98Q7ePrj+b33+6c4wK4AAZLC1LQWai0EWPMKd0nPONLNZqFzuRiik7F435+ObVi//i750++xFbBSnaaqmtNSmquWmpWsRy09w0NyuiKZc4bv61v/I3fvUv/ZXNdpuXxTGK41ostcrOffjBez/7za//2V/+Y3/73/vfvvzkk4L8Si03xfURYbZuQdTMTNcj61/acq50jZ+wNx49eXY+7kuXbt9+arUiUddv1tMu9pvLJx/srt4bNN3Nn5ZatdVS0svPf1wrnO5ujrdv7+9u8ny/nG5PN/GEDymdHu7ezMvp7s0nz9//6KOPPtpfnh3v39y/eXH57Nmzr35teLj97Pu/tb26cv24f/zckIr51trt/aF0HqW1ZZqXJK1FtqHvK/nT6Wiqm/25C9E57rquGQFoxyqAjkjYGWBpsiz5+jBpSr7rRsPoOZXaRMEAEYhZVVsthFjmh0qOAJiQh613vuWspc6nY02v2QeOwzrqcoih6xC8djj0F2EYmVBEvOMuhi6GVrM4BsLeew7O1hYKh+wCSjNt24vzqoZq3jOHAacMoOCCga0oFDDVWtY1NqFHpBXwGB35fsMotfVYllqTLJPWAmBiBCorl5OaGao7CRjGdH9suTSRYRwF4nRIreTgH8i0KQKAlExEVl2bJzGxCadlrjE2Q3Lx0fnVZuhjP+w247brmxrHTkXKO4NM3TiaSmlIq8dG1JroJ29umKDVOi+LqVbyiGAqTFxKQeeQPIGSo8Dcxb6L8XI3Rk6m5pj2nQe1VvW4FB8CsF9KRuY5n/abgbWBypxc/+QxszPis+12G72oxa6LDPsuujUZC6REc0qEOJfmnBs8e+9PTVJpnQuClqsE773jyyesptExmXbBLzmbQh8YkOfalLj3vOQyxDjnBWohpOhiBfiyNomGMapakxpCBIDWWm2t7yIiMogcrmtJ/TiWZq01U/1SkaW1cwwZCZGQ1ZSZV9+s1qaqgLgObSJam6ZScqnR0dV+tx3iuOkMcF4KD/03f/kX3vvat7ZXT5g9ET+AqgiCTcA3zX3w1Z//Y9vLF59+crmJ1m/TElI5Jm2gxqvfogkgTDkDYl/EWiXHBOCYkUlNgw8A4Ls4Xb/85B/+35bPf0zWRItprU1Lk6aWqi5VmlkWSU1SsyW3VOsf+oVf+pt/63/05IMPT0uSZQrMpWbvuBu6rrmHORlANnny9Mn/+H/6v/hP/qP/8J//+q/tPdwZr2ggA7NVI1svmPb7XLU/mcW+DAysfhcRvbl/WBTa+vYhEbMBmKiKHB4OS0GXj/e3b2opTQqAHg93m2EL0tJylFbmefr8k9/xVsdHH0k1Zldzfvvix99Vig7Vh77vXi2nN5/98L333x/HPjiHCE3l9vY+BN8UU1qY/avDXZ6n7e6s1ayIx+uXvXMX++049knx9euaawUV5wOFmB4eEMwPYwjRhbDb7foYzncjgNJurGZoRmCXu81pSbnmMk3UjUBsKgKK5Mt8QqLYD+iDNUFn7LoyTaKzLDPkJfQjD9t+HEXVb3YEpq3lXMl7IjdN08PNmzJPokbDpovdGCiGLqul+eQ9d8NuWZZWM6FprTUlJgJURKemiAA1YUvkHBKZqus2sKbfJZP30Kq0rGmyOHpC3j7qCCDPLS3akpXUSq5lXX0joXfvvf9BFx2aAvApF2kteB+Ca6UikUOrpbFjx5cpzZ1jIOv7jhwX8osZdcN2tw9d55nJeRNTMBBVwsVqQahWwVB8VNI55Trnw+evzYyJFKyUQoRmwMQ+RAM0AEIQaS5ENImxZ+eQ3Qj+VOWzt7cAto1x3Ay1SfB+txlj8KlJU8ulMtrZ/qy0Jsx9759c9WMXilgqlYJ7ezqJUTsuyzyl6WA5TUuaco7syDtg148b9sGz956HvheR3rvt2O+6gIin0jpv0XtVWIvKt9ErMIKZwb73BhiYi/q5WTOH7EW1ldZUQDXXWmsBM3znP1lt3u9GhmleYoxn6UZFlnlZj7kvR+oVVgieqErzziEiA634dluBgYBrg6Gq5NbSUlXtbBPP99thiM6RiB7mcv78q9/8o798/uQ5IVdV0UogIMVqA7Bk/uVDbqIfPf3Kh8BeHszIck1Ns0JkYsZAXNm11py33rnBczFJUkPwnUciYgAmIh/efvK73/7P/5Py9qUnMG26hjAVqmqqWkSzWKotNZmLnOY07s7+wn/v3/yVP//nYwz3Dw+eUEwZwBNAyae01FJ3sZvSfCuKxATwV/+H/9ZP/8Iv/Wf/x7+TPv18AnoXxDVQ1He7y3eAoHcn2h/wyuKa6Hf3Nzd1ThnwdDpIa0QkanmZwUwN7q9f+W72bW4lr0ZNAsjLMp3uc0mSTtqaqJbS7o+T0utDSnmZck4IdjzcLKeD64e8zICQ5vnzH37/2de/1qR9/vHvHr96+fE/+8eK4IfNw+11jP0yL90w1PnI7LaXj7phd7i9PhwPMURk2l48IheRXKuCmqo0a82IATnPR8nzvUkfw3Z3ZmZd8HOVudSLzehNl2VSg5wX7xwTAfvQDew9M7PzqBKJpDppbeh2uhtUGoWuiQBxadLS3GpR1diPrVWcp1bmlpLUoqb9/rIDzPMpn1REXL9DNNUiMKuhAOeUTEQVmqnWDO0AbdGamVnTEeuskgnMdVuKAzOjCwSG5BKz73e935qnNJ+wVcZ1V+h4cxZVFGCeZikLmLgQu6XkdXAdXai1dn3fj72VhkyMxI4dc/CBmcboURoCEFMxOJWagUqTvMxVFVSjc330HrDrd110Rnw4nlqDWuvb29tWxTlGomU6aavsfecDEZghe2cAa0Xb7vIqMB2myYehtpZT7gcWsNqkmEkup9OUXnwRQ3AhxBCGvjvb7ZqqmlUA57HrOs/QhQAr5Lq1t29fH2+ul5zKPBtAiOF4f7e6t4hpJvKxA6RW2/78bDvGoevHzeZqGKKDU9OHadn2fR+5iaBIM00CDbB3HD0VsVOuqoJoBOC9P6XcWltykVpBZduFwdNyfLh+mKqaIJUlA5GZsmcVkVqJ8PHZfs/T/c3tNC/47nIEIkqEovpO4jYzMEdUW1O1EDwYVJE1cd2aKkAuwozn22637XbbEQjvD9OS67d+/o+891M/EzYbLQu4HrQxgtbaalERQ1Dxxu5lK3OtZ5vzKzfku/zidtKyOAqpyM3D3X7sh802Ote5UEXmnGLwj0Nw3uemrVUgitp+8Ou/9tu/9vcpnwigNjWVXGpqImalShFIrU1FU21TLrnqT//RX/7X/tt/88OvfKXW6msJ3teU7uYEzkdHI6EYZHDWIAY/MlelQy6vj6f3//Af+1tf+6n//G//B3//H/5aMv7JmfXlcpPsy2QIffkS/H4yLbkmdUmnYrZ2oaupa1VVwKzWkpaDiKiVeUm27rMRoZVSFgV0SGpqZqfTg3dhOtxktbKcUprMVK6/eP3FZ+P+/Obtq5zz8fDgQ//2izcXV09bKobcXTw+PdwvS+l3VzUlYrecjul08N5Px9thd8GxDzF2230rZZoLe0Wz4Di6cP70ypkGx+l09D0oJHOR4nCXmtQafaPQxX4UtWKEcUO1OceqYgZd6AwgDgQAKmJAvh/6YWS0oQ/SpLZ2mnNpotKQOXS9mQBQq01Vynws0wGJOcYYoqkc33xuBt3uzFQ9KppJKX3Xbc7OxCCllOZFVcBUsrOMis0zITs3bAGASSU9WJ6QFAwJGrk+bM7csPfdxqSJkvfOtLWUiRygmYrUStZicBLOgNCtavGainZizjvvPZg5z57fzSZjcIOngLDbBinYcgKHxdSpfHZ9++LNTZ5Py7x4wnEYPVLfdxiDurDdjCHGR9vhbLfTp5ev7w8Fea5Sa3NkZ9sNk8vL0kcvAGdDPwRen5jR+dyk1lJEjL1nTk2yMYJimj9/+Sq6AZ27ONtfbHddFy6GPqsmWcPk/rRk50lVpdqyTD/83nfTMp8OMzuoOV2en//Uhx88+cU/fDoe3tzcXD9M5tzZbvv82XvjMLgQe8+d90DkALpAzDyXhsyDJ1OHgLkJqM65ESJAy6U+nCZAVLDT4YiEOWdSobWpV/TOOTLb9OFst0dQh3b02I2jKCiRKXTBPb66hHK4//6PXr167QlyLi6EJeUV6s+EiJByI8YVxEjEzO88imYIAGoqImLmmTa9343RB59LfX17PJ3mn/25b5xdnWurbTnxsG3l9E5LkmIlg1ZUa6oKJv0gNc8lHzf7bilvX7748cc/5uCBnCK/CaGLrjWNMQIAe2dNvEMy88HvdmdB8m/8F3//89/5TQ+iiNLEEJpIE8lNS9MqlprMpc1VT0vaXjz6a3/j3/y5P/ErjdzdcSIwtxmbVB/CVReZqDZRtbbM237UNUDvw6Ayxk40llplu/03/tb/ZHv5+O/+3f/rUhvRu1lsDeO+82qYrYW+7/abv5cNgJrTkk5NQaUaACpWqe8wka2m+QDggsNa8k9muhVCQKpGsPb1lpKrVI8GyMRMhCJYa769ftUFn/Kc0vLm9aevPv/hdn/x6ac/enr1RM02j98LZ4/yPNXawiBMePP6i7cvPtmM45KWaS6b/SVgq/e3Pg7o3OXV1TBuQghFNOeshMbUX27HPpxynYuaITnnYyetmSqbEa98J0aiJkI+gJpIXVeaAMYIBDjPU6sFVW8O7IIDVQRyIZo5ACDXSc0ibXtx2UTc1eO6nEpVadn7WNO0wrAQwXWDIYlUoHj7ME1L3m+Hoev3mwtClCZNBFpZuXskVWoyz9SaylWMHW+2oI1cBPJEqMaSlnKcoObT21spGcpCzhsSmGo5mSohx91lOLt0ZuKIvfOe2cC2Q+cJj1PyQ8/M96c5TdOLkjCXcejJzK/+666/T+nt4eHwcFiz9ezcIk2XPPhQbcFc5tpeXhN3w9Nnz9+j+NWLzbP9cNvgMOclJSSKIT7My36/CQhEqK064n0I0zzPDXPTJhqYY+d3fTcE/5BqErt8cv6LX/vgNpWbLAqmTYBoNuhj1NqmJbXcnHOguJwmRsxpDv12Li1u3cVuHDfbMG4mpocK+/Mnv/DNbwDBq4d0tt/MVUX1mMqSageW2TdpJtSQU8pzWrTkVUNVQAPLuX5eynw6llrQRxHbbUZViH3cbDZ1nqZ5XkqrKrvYH6epII8js/MK8PjR5mzsx3E00+C4tpbFoOqPb96WtMLtSFVXG6eosHOpFEAj4iamWkPwzvvWmpiowpr/ISbP7Bm7QM65pvry+nh9+/Ctjx6fne3QTEtp6JhneEfPUW255WytWS0tzUjY785olLkuOZ2GYffe86dpSR9/9nk/7sbdJtd8/fr1/HBjKsNmG4KvTbph2203fYu3v/Pt+x9+J51uwazoaiNHFa1NishSpIhV1bnIcclV6Rf/1K/+N//6f2c8O59zYu8cs4oeHw7Be99FMVuWkkoeu24zDKC1GSjStGRJs7Z2d3tbqhyOJyL45T/zKwjwd/5Pf3clavxEGVsT1msByu/tN81W+WwVChEJUH7SjAIG74Cuqjln9kujoAAuREgLIzFz12277dXgsQnmkqXVVWpzzpk5Imcm7FxtKiJPnn14vL+tJR0P95dXj1utX3z2u/l49LFbluRij9TUJC/zePbIxXE53XXd8OTZ04urJ8G7ptYHFmQcz+c037y9WQVARNJW2Ec6UD/0xPxl+R5yjIwIBsmQ7R3kjpmlVQUspTrn0rKM0SNirdl5JyqmhkS1KYiAZEEKsXOO0Jqp+BBKWhDdXBYiGvY7aAWJht2OwNYnqYqSc4BAxGiKiIcl23xEq4QIxC0lSSdNJ8mT5gNIBVMwQRPnAxG5EIbLD7aXlwy63N2WVqVUlkXLiXyEliSvihuAVjNTBHl7nF//0HV1+uDxZdhsjupuj9NxTmWeTtOspk0kH4+OGRD7fqOujV3nvVuOB/a+24wfnO1Hhpvbu2lO1SAv6XK3/ei9Z+z9lLP3tPXsx20z/Ozm4e3dPSEQUS3ZDLb7PcxZ1KaUc1qcc86Ht0s778OG8XJDrwzvk5qAa+22lrNorVXv/bHUE7msQIgqIEhVDIrU0oAwF021Eabf/eH3AODi8uLq7Oy950/P9/tpSb7rWlv9lDQ3tSq/+frhauzRAIilpVeH5bDUZZkIMTVhk+lwVDAFbLWWtBCzNCFGzwSq2+22EQN6MiLPFTB2Yeji8fBQRc832wsmFyK6UC/OgbiWXJog8Uno/n7qpyy19Q42fc8hHF9+8nB/v3L9ibi2CmqmAmi1VRVlJlMrtbLjJqKaVEHaenECInbrZgCMiBXg7pjeXt+9/2R/eXXO7KHVlqZAWK2uatFayaF5aTmbFANy7KWWkk7Uota8iNpmc35x/tnt0bybl9T3nW3ONmeXtZXWWuiGPvab3TYcbw/f/qfp9Y+pFUZaSpb2DtSjps1gKbo0zbWl0k6p7B+/96f/8l/91s//4SKNTgdkn4+n4Fw2KqVO6bDdbEDleH87Hw8lLa216f6OmVNaus3uo298c56W+7tbNtBW5iXdvL25ePLer/7qn/1//r/+XvBuPYx+ssD8l5Qy+/JQq9JiP8QkaG21p4EBM0tDeFeBFMbNxePHj/T1G9WWlnllMsYujtttgOZ9YKKGFruRyXy34dD541GXhcj1210c9ufkdy8/Pdzf5JKbyP7i0Rcff09EwzDEsTXVpqc6LdK0H8fd+Vnsvj449PsLQxJAZhJCaWKlMHPsIgAZMYCZRDMNMRI7RGDniGi1cDvvCUhNalNEFTVQBTQkx96ZoQFM8wzEiBAdUwiNvYGYQVFFHxmgpDlLY0biYIg+9rQiyU1BG4CSSiBWVTFjBEDQnMykqXR+bSkGQiqlSk4tHcvhWsoMOYE1Zg+I7AMAgJihkiRtjfzb+dWPWpkcIwD64cxMyXVGARnRTFsB50wFrRkgoDNQ9zs/+PEnn73ouyEb5VZbqaptJbjHGIOPPsa+H5x3nWNSCeyGi535sNntfAxXg4eP3hOkqYgBcKv3dw/kORe82I5IjERW24dX5+QQkXKVUnLsBlsLhMCamGxGQPLeK2BqbSnNFS1iQxeQKJWSa71RIyLJCRGAyBMxaPCeV6qYShWQUoPH25sbNXHOP3r6lBFl1ciJrx4/9iGIwdVu6B0P3rdWamuZPDHenuZdFx9t8dkZI16Y6SdfvDmVNjy6Oh4nYJ5SJh8RLTCWlHzoxs14Ng7eORfCYZpiCMh+XtJSq7nIILdT7oKrD6fOUz8M6AIZdiGIaG01eldrraKvbg7ojk83vrz6eJ4XACDCXFqtbX1IrvRhIiJClRUUQSLWTFe77+rL9Y4JQVSQmRyflnZ3Nz2+GB9d7vu+B9NWFofQErLzREyOVVRq0ZxEZBUWCIHYWxOtkzonarfX7tHVs298I6ytBjEE75yZLXm5vz/u97t2vL35zj+5/vi7WCZCRNSU8zG1XJuqREeGMFedsmSRZSkYh5//1X/1j/zKn3Oxe3tzz+zM1FTZBTSrIuzYe/7ke799//YNrL0KtXRd143bVuv26unTDz7anJ2HPvWbkVpxJlXFmrDzX//qh2l6+Ee//k/WXervj2SuxXQ/iQoQ0UpQ2p2dm4VGcPv2xYq72Z09Ot7fiBRGjDF0fbc7vzodjrLKq4hm1moxFUE1NPbeSgHyrus3l09zntM0A9wgou+Gp1//qeubm/1nPyy1IjpFDv2WXAjBWfTgLkC1dJ3uz8jx6e4mzadWSx3Gbk4xBjNgIfLvLla1ViACtRjj2rIMoK00ZZNcgdb5yy2H02TQjYOUSgxMRMitJhd7rZmJVMW5oIxqIjm3DDFGz2gK7L03qTVXIyISxSbgnAM1mWdYn5QqJi2fDqHrGK211m33xzeft5xNtabJhY68C8MOkbvtznUbH3vr+64fW5pqmus8GWjX9SpNapY2A3dNIIZRMJgfXRhwxRFqQxekZmj5nbmsZZPGPoiCiSBkMHTbi/MYY0Tbgk5JjoBqMQ6bbhgeP7qMMWqtu83QOeq6jthbbYQQoh87X3NqDY61HKf55jD7cdNOp9vr21zb3d1NtxkuznfB+1mITDuGYRycweX52Ra9d476vjQJnjvnEHCuNXrHaHPTVKVD80CHlKNzhOydU2uI1sdAzIc5JdUm2oXQTEyaY5e1gcEwbjbj+OyZm5YFifq+Y6b9mTGxZz4t83FeFseLbxHNWo3kAlMfow8+NFEzBzAE98e/+ZEAgdWcxQhPRXOTrgvesdWWc01ItRQKvjS9iF2rNQbfhSAiLjgHqIitFCKejwcRcJ5idIjEiKSNzSrytu8uhni57evhzXfvb5sYIOVaa2uiRmAIuHLrkckUSq2OHSGJqCGIrhIBv+tUJWJCYqhNlqWMkR9fbPu+BzCpVZxXm1WFOboQSBkUtLWVuE3EoPJOSdJmRCCCUm/vrjeb7ePLy4c5g4EhVREw67sB43T7nX96/cPfyrfX3qGJGGAp9bjUubQiMgRvCFMuxyxz09bsG3/kj/0rf/GvPv7wI22iqgBnZmKmtVQwXOfNUlvLqZRCzpei7L0fxu351bjd7s525/vdzcvPv/tPvp3nGQyWaQpD/+i9D3abYRyHovQX/vK//rufffHFixdrc+gfdJb93okGsO7b0TGSI3p3M0Uijl2/BGepwvrcYFSzGHzzfd/1CoqAIcSzy8dd302nOaVlmk7duHny5P1HX/+p2zevrc61zGBa0mnYjj/z/rP7lz9My8l3W3Rx3F/2wxD7XkDzdHLO932vYK1V1w2di877YbvzTNYKEabapFZTiZsNh+C8l1IRjZnNEMmH0FEI6TSZCRMhYux6ZDIAMQVwtLaLUWfvytxYmyxpAbAVOokuNMJpWVQUsZqptUbO98OmH7paW2uy5lzXohwkkqbkfFVLJYNJef15Ph0FwBRc2FCMplLmxF2/nE6whhQJESNthuE8MEhkA2nNsJWsUolIazapdb5HBdEGuo6E3mqBmtl7A9OaAYlibAqILNpAkV1w3/jZP4SqxC6YXO37syG+OrWDQisN2U3zBMQ3p6XlonkhNSnN0MyHY17NsFxrM7DaCuWGIpePnyzTfEql+HD36UsBdb5TWAkh4JnRh24t4I0xxBEAzvbbPgYjeu98dznEsy4o6iLwUNVa8Y6NYC6LitRcRVofArRqqnOD1kSJA6N3biCvZsTOO4+IZ7ttEyFCBgzeidmcy/FwUtHzi7NS6v3dLYDdPtz1seuH0XfDkpaxH47TnEoaPGFeWmsiuhkG7jpr7VSKKpq0oQ+ltWHcKwD76IMnpmVZQgzsPJp1jNFT5rA0O99t3Ip/ZGrkailLysfTSYCIOTp3nE7neoMuVLXazBRrFXZoCk2aiCKiIyi1rZh7ERFTIvIrrx1UVNBAVZ0jJG7NEGy37fqhQ0IDQCIzRXQmolBAQJVMVFWJWUXABJGZnEkFYiJGE6uFO3d3d3sZ90PfWS1sDZGWh7svvvftF9/7ren+znumVqUZEEXHzlnfByDshIkpVZ0aHXLZbve/+lf+2s/8yV9Vs/l48s4xc85L8I6IjCmlrOqIaYg+7LbjZjNPE4IS0bDZra0LjvGLTz/5zj/7r46311ILAF2c7z68uHo86uH4ytphO/TOD3/tv/UX/oP//d+eU0b8vaslAOhakmT26Orq/v6+tsrsnjx9j0M6pqMLkdkz8bg7X+bjSj10vr948sHu8fPOhQehMGzycjK1U0qH+7t57k7H+2meCLHrvPfIWqaH2+PhZjodAOyLH3//B9/+zW/+9LdQGxI9fv786r2P7u9uzs4euRCoCgKp1KrmvfMIj54+Jtc5MGm5GqRa7q6vn3zwIQJqkziOpWTnQ9f1a1mNcy66d/p+gUrM75BQ7AEpzVNGYyKQGhyi62tOIOIgDR2z6ytwMwVRI2pNnQ8YUVtb3a6quqRcagFpxBxib9qAGQHKdEJE9CHGgLu9lXy6fY0+khhFh0yiCFUMVGHJtRB9+TEE6YKncUtdv9zdNxeIGZG579EkOAak1m9NirW0Zo21ZgTDKK0sUGfqzgD92oqu1sBt2Hdg5rYMGQC03R7nuehLOk5rEZVqKXWZppRSTfn9b/7sw9sbnR6Y2Ds37redY47xfOiY3ZxzbsIiu2EIXdcdo3oWRNpuAHATeNeFLvjrh1OqImYAcL7dbvb752eb4N3Y9475LtXa2pIBTdTFm9N0PnbDtm+IMTrneiSac21NHOFZ5wghlTqlXBCboIp1fcg5R+dKbT5Gz2QI0nReplRyCLHU1kTmaXp4uJmPh5RziH1TY5831eiUxr57uL5lwlbb/cNcSybEcRxvru9zTtbasNs+3D+gSpkPngmRybk4jj72TCzSxs2uibCP0mQ/9k+uzrfjyBSqaAUAIIcIIZTaxu1OVbuuq00IoJ5EVI+HSURaEyJm5tLqWm9ra/XB6kddyY7kAFaKsqkKIRKTmREhIzapnrAL3gwQgJ0DRFEFlRX9pepU2socRVk98sropFXV5kK3JlilZAGYTHfzuQOud68//t53S9UvPvmxLCeH4AlRrVRBgkjs2ak0BPDOMWpTq2rHUr7+tQ//8v/g3+ovn5VclpQQsdZi5p1zq/veB8/Oi6h3rta0TMfY9UN/Ka12XVTV0zQD4u3D/e319fmTZ/1ml+dpuxk/eHJ5/ebl93/0ceg3yzztOj7rxpefff7e+f5Hr64BvyzSfBfHfHfrzDm7ELAUM725uXk4wZyPpgCMyOz70fmu0gKIrWnO+eXL1zAfru/u53leTgdAbCJMPnab1kpOSWp9/eJTrIJdtywzO4dIBlZyPhwf5qXUJmmZmUTKUss8dL4butNhYUc3r18uhyMj5rQAmI8xOtfF8Pjpk0cXTzbbLSA6x+Cd97ztttEhmzIhEaecoMp0PBITMqE4NZinU4wxlfbmzduW5lYyd0OI0ZCcD0CwPNzqfOy7Dp33sXOxC3FAQjZFJu8DuEDOmahzVEsxVOcdYxMCVFXVbhwccxXN08khOh+63eWsN60UFwdCZB/NzGomk37Txd6RYj9umiEhAruqoE2tVd9vAJSdB4O6GIeOvcpycP3A7AEAWkNGEHUtg4gxAyCyN61WMyKg661V9yzWudohmbvYNZG3t6fD/f08zczMlj27/Wbnzva73u2eP92E55uxd8x9F81HdUwmYhgJi5op7js3dKGVcpiPlfxBeSn1G/v4ZL8ptR2Xer+k60aHVPd9BB8WkWHoQwzR82YwaG0GfHlKc7pL81weYNyOs4K7uWGCo/lZsYk83N/dvH07Dj2YHQ4HNH385HHsR2IexxHZiSgS1VZSKv0wotn1zS05l9OyLBOTY0Yw5NAhub4P+7ML7/3t3Z1n6rtunicDY/Zx2w9Dv9tuVPThcKgl11piv/Wh63aXUhOCqbQslg5HQNxdXKgqu74PYfdo9ERVreZkMfbegfPHVKSJIRLT2bAlgOhYABzidKxvX72dp4mdd96JNVNjJufcUgsYAOK7npF1wU5ICK01M3COVxipERIjAjAT4uqeY+c8I6oIE9GXnJu1ZWe1+BCiSlVFFzp4B8yxVhISioosBWt6uHl5++kXbz/5+OHNKy3JEQXHtRREcIhDF1NrSy5oUmtbC+wICUAQcNfHv/iv/6Xu/LHW5D0ShSWV9eLMjoILq8ehSVvT8o5dyRm/xGGLiEqTkrt+OD8/i94T4nw6bcaBpLz+9JO0VOAOgAz5Ry+uT6ePr7Y7LXkb+VgF3vkz3h1k6+3y4XBY0xRg9sUnv7tomKaHaTlyds65u7evToe7vExrd+Lm9Qvkric93d+aShNBxFpTzlMTqXlprRhYzktOS5qXUmtrWmtZmwBf/PgHkfjs4sp7X1I+PNy/+Pj7/XIbrJIPogkouNjPx5lcPN7dtJvbvMwA1n2//+q3fuan/9DPso/RU80pzdMnL19AK2bQWiX2DVFqqyWhj87HPB1B2wqiYnYlZ0lH9s5VCaUyO3YpbvZxfzUrHlvDkuv9gX2HhC1Na08ZEfbbs/PHzzebvo9evB87N3QBzA7zIqW6NTUEouCL25SUpnl2Pgz7M98aKICZc4zMofcdA5iAFOe7vu8M/VzqsiQVYR/CMBIoCBAAM/G4LSk5NNeNlhdkr62hC0gE2Ezbu05UhLac3sUzDLFlIHb/4uUEyALoi+3H/moII23nIZa8WCmbLoTRs4+dpOFsA0Cnqmju7aGkfGrMRZSc8851BOfRP5zmpdT7lFOtTWrXjwJ6/bo8vTgfh/76OE9iHIdt388ikXXJtRh8/+X18XTM01RTQh8pBEIsKaVpfvT0cXAOAE6nU02pLsfd+Xmu9Xh/B1pPdzcqCuyNoBQVNTQlRy54FQCEYbt//PT5bhyGvsutjUO/326aggu+Sbs8O9sNg5oCIhgMnV/mueuCttKPY8p1sxmic2AyteqcV7VUhEM0QgTH2DOz98REV+f7q7Md+U6bxs6HEHIptFaiGZRSFmmuWa7FEfcxBscMRkiEsO2cSPv4s0/evr1mAhNBpi4GRFDTnJsjUjNGWgvWGIAYzFDBAMF7DoGJWYgdAYCtMGnHzOyYmZhMV8ZQY3bGRty9S434gAitZARExFaKC5GZzBoYIDlVaaV4hHk+Tjevrz/+gV8nQ20oZkjOu7OxC8xzSsdlvaw4VEHTlZgnCkwuNXf/8Y+wFu9RxM6fvDdsNlUEiUuuSOCcj0RqKk1MWmDOhwcFOz083Lx88flnH0MTH+Lh+DCfjqDy6L2P/uSf/JMvPvsEiX3Xd2ppnj758ceH03S526RWwrjpclakqVT80v2/NnSs0Mp3Fg2Dw/2NhU1ZTu9aucFqmqWWJhUAEfPp4RY5qMe8HEWaqqw/p+REteW8IJgaLPPp/uFWP9X7h0NOD6lm1QZgx/s3n/+uffVb36xVajMkz767fX1kZ3WZog9P3v/g7RcvT8d5OR4BQA3jsKnSKPSI8Pnnn/swpHxKtbVmYJbnYzrcO7cSUxx73407JK9qvutvX77oh5GIVCqYgPPsIxBjiN24RQRVi/0wnF/l6QSmbrCakqq4fitSl7SA6pyLIi9T3EbSnO+tpSIp5VxbWSYmbGbe+XF3zqahD+NmFyNLHA3AOVJgR2uBdJnnuSi1bAjH64cHZh63W+ciBmdI0sR5n8tcaiFbW6e1mrrYC7m2zM53AaymCcEAjENvxCAV2aspgAIwaAVT93opjsghDojXt7c3n3+SU2qt1VyQnSH58Nb74JD24/CN959e7nbUxwk6F8NJYDme2jxJq6ks9zmVKhJi9v7+cGytaqs+RgP60aubzWaz2e1D8PsBCU2lvfziOvbd8TodT8daqkhzzjnE4D2z22xG9+Sx976lxQy2m03rhwPB61evHdN8OkhJwC4MW0XMzQyhH8daSi0514V85x1vd/vtZozB7Z4/HYex1ppSGoderHU+MEJrLQuumuTFZkhjn4qWVEKIxOwIgmdpdr7Z7AbJtSqcSxPvue86RDwejvvNdhi61Y095QqeRPU4TaI6dISqTEhDV0s1U+ccqrVWTUUAvXPGfH9KWI+31288U6ut63yq7zLngEhoffQGaGrOkYqtqAwmNhNEQMY1Jc2OiFCattpMzQVa4dqqKtK8iwCm0oj8muP1IYJpq9VaA2IAJe9gLTdc0QWiWlurBcBKGMZN3wde0/iISACOuO9677ypjD727I8pNVUCZBMmcMqHZdoNg0d4OBzv3r5RbZvtltJpd35+tt8LOQ4ekE2XLvhZcCo25Xy8vX3z+ecANk3z/fWrVuvpcD9Np2EYAPH58yc//Yd++vb1i5xSNwz399evXr89naZUWpV2//BwnKau71Nrg/civLT2+4BlvxfPXL9My+SUcp6/DFfUkqaVqAaAVm1ZHpACBl/LYmAKsBaxgalqa7JyGnRaJmZmFDN6t3RWMtPWSm4VjC6fPLt+/SKl+XD39nA8zoe7NOurNw9PPvrw7NGj0MebVy/n+2NpBw7dkydP+hhOp/u761fXr18/+vBrT7/yDQ+GRLLblrPzKlJKrcvSalmWidlzCKEbLp5/dLq/A0RQQ/ab/QW4NWHoaslq5hyn40MruSxTTcmsMTtDRAqAHDd7bZWsvfnx96xmM2utgjZgcqEHAG0FTLzvQz9O8wRArt9096ddF4bttt/uPEVECJ76fijLgo4oi9vtW2tpmZAInSvpJK0hoBi24AMRMpdSiYjYOybHDrseJYCtOBlDIkO3PsYVEJnQEJuhY2lJ07Te6hXJzafjm88+SacHMCUkQnKo3Tj4riMDIj7m9sMXry8OpyXlLKrs/O78cDrm6WTL4kABSBAX0URohLU2NCyldl232e43u/35xYUZpNJyKsTc7c63mwH5UFpNSyq1UggCMGz33rk+uBA8gVnfqZrUTMQfPL5a8vL27n7YX1ZV7wMSbTbbGCMzxND1jlOttdXYdbHrEVBEiOjm7v7TTz4NXdfU9tsNM0u9i46L6vn+LJVS1EBPwTnP7tHFWdZGHKS1KZW1FbSPYbsZmAiRRCHVOi9L1/Wptds314y03fTesQG2ZmbahSBq0oQADGoXfHAMKqICzKUaIMwp5em02Z+TIbPz3gFQiP44H0oR59g5dkxr+wmAAjgzUGNVMwV512L9zmeAhk3X2cLIMQe/qjbr8q6V4ry3VQWrhRFqBiI0FRDV1SHKDRoiEYCqKjLVVtOcXBD2qfeui14MVkRPa0qOG0BT7b3fBK85d8GjmYksayoXbBPDOHblcNzunji41NaCo9sXn9+9fPlwdfX6ixdsGoP/9IuXX/3oQ3AunD3aP37CWs7PNiklSfboyZPY9Z1H1prSEjd7NH396e8e74/nT56l+ThPp8uLi29+7avLPC1LOs2nt29vSq0p11rKV549+uLu8HCa7Cc9dO++4DtDrIgHU2n2pYlDVlamrYsCbbX64FVFTZFWCRsNwZCc7/pOfdDj8d50rb4DJCb2TE5AkVgNXBiKwPOv/czLH/42gJ6Od/fH48vPPy90fpxO7eMfd8PY9f2zD74CH7pas7SaDvc3b14haJnnZ1/5xsWz99lMtKFRN46b3Q6ZUyqH2+vpeASz+XjcnJ37EAGgk7aautmt7TYKpjVnHzqQery/iyFIKQqG7E1gnmdml1MGUADSmgEQyGHnTaUbvUoD0NAN0pqIxhjgywo7Ju9cBKKb43yfxR+m3jsfguvGwIf9ZnDOBdFS8pITlyTs7k4H8r6lTAQKMIawtGqtaS1SU4gdjmM+PRA7kEpo5CNIA0FkJyKAtnaDIZiZQppKmiWdHDpGxDevXp1uromwGUqWcQg+REQMIa5Ou82wIaRSyzG1YbM7vH57e/1FtY+9d9aaI1ZEck58uD8cDtPy6L3nT589n+f07MOPhmEYul6amKqqhlWgAWuitw8PQ9e9//x53/XzktG73TjudyOIlVbmRWqr0iR6t9tswayKXFw9fv7ee10IpbWltLnUJZfb+wcm70MI0TfA0A8hRgTzjMF10lofwr1oXlIXgnNO1Zri8TAhc6q3JtJ1seu6ptC0emZrWlVMrakIEyMuafHOm+l2HGqppdbjkjzTMI7eh+idKYhhLSUE7zkaqLbWamHHona8m4fgrRZ0Dl3zTE0VgajfHObFafax6/qI1NbNS5Pm/FoxvCr6QMQiWkWZyTHm0lbqtqpVUVNdRw9CJOIYPDO/y/G8wz/gGj9UNQJspTgPAGyiIg1EjBhrUVJmBlVdW/lqW1LqiarUgXnKcreUpUoudanNkMa+68KyiX7fxY5wDH7Tx5KSGp+WtCJZhnHTbzdVGwVPQyy5jlfPlmn6jd/4jbIs2z7mZT7OE7Y6nF/+9PtfsXyC080ZqwTZXwwUujnN5ML1yzebiyvvw+l0bEq7y6tSsuTT1fl+s7tSIoz95pK2y/Lk+Ydoenv9utV0fXN7XvtS67TkL2NMBPBlBgDMzOKw53kiYgAjotANpVVIaW2hQ3Ycus3ZucJd07ZMJyRCpO3u0ocuH3Ep2Tm3dhgi+67ftRr86dSawHoqdENtOoyX28vHWrIPfW12d/22eElzKtP0ANfORx/9Zne2zPMQ3H47Xj76QwrIxOjYkGpKvt+S41SLA5aSx3G4CM8uHz8ztNPDwYXQDT05d/HosuRCzK3kNXHpY2RiMCNHpmoA6XRaptPp7rY1NQoAKFoBFIEMCJm1NHLeqJZamb3vOva+H30cNtqqinAIw/5ifR4QU6slT1NJ0/QwO3K+62PXPxxPgBBCHIdhHMfC1EoK/aCAyJWdh5prmk1bW078zmzU8nRCVW1l7a41LCtzAls1a+QiOzRDq8VEwCqTYYwupVzm08PDgxpyMw7RhZ6dE1XvnYgisHfB+857T8Om72If/QcfjVePHxPqdrMJiIdp+s63v/PNn/mmdt15bsL+7OKSiJeUVC30QzM0lLosisjMzjuHEAMtqaQq3vHTZ8+6GJdlWduVl7QY2GqR34z9mkiP3jfVuTZBEigOYBtd71D78GS/NURCdExnuy0iliaiyqie6P6UHl3snj9+lEpBQiZCwCIioillIDK0lgsTtVqHLgbndmNfxaZlPi2SS90MnaggEjER824bifAJEiGZyZSSqKFBYELHtZSGhYCqCDQprTnv1ez17X3fxYBMWpJZ33WeicC8966m1sp2uz2erl9f36lZE6mlqQgjAGIMfqVuOc+tiamJKBgAG6DV2kSUeS0ucZ3j6B0TO2Y1WVG6RIiEa/8zMtlaL4aIvE6aigA1F4AUYtdqa6JAkHOrTaKptpZK+93b6fo0L7V1ITx9dPGVD97f7XYhhmlOJS1LSlXadFrYQI0WAUEsgGee6JPvg0KrLW42290+PPvwze3Ds/qhD6ELcZmOtZYnT58BaJsPdw/3bT7Vzt0fHsY4uN3V7tHTPsbf+e3vvHh7G/thmpbrN6+GwCFEM/3gww8oehFhz9YEwZZlIpNa0+vXb5dcpdVnF9tPX7fchBEBFAHRdMWbqcH2/OrheONzQIQQurNHz4C45ISICNT122F3cXbxqOYqBPe3b3BFpGkzsM3Z+f0nP5RW1TTXepqPERAA2Dm3Xu5C3F487vtxc3bRwUcPb15dPn7++Sc/Sg2atXQ4gRURUQUfu7ZMj5893V5e1aYCwMS1NTQwLc59WdXctEiOXTwejq1WQuLgu+3GOcc+MCGoduOAACFGUJtPJ23i+pWXB0DEzPurR7vzi0fPnpeS5sPB1FQlpaXlTESByXU9EEktqjqM226zzTkvx/tcKjJR8OScthqGEYiZyIVuGDc5LdP9XTod8uFhergzs2G7ZXa3quN27z0zmHds7BxvnXMivWk1YSJuKamBDx2aKapzHZmYCgCpVhMBE9VmVXGawERbRedUmrVEhE7NpIlzXomlNlVlgNzEEXl0YdhE72Pst7vtbju6NSGRU/Dh8vKSERHM9/3YZP/sw+zDseSnXd/M0pIAIHTd4e5BVYN3xl7UiHBKeU45BI8GQx/Pui61Zqq11hhDa03BtrtNLs1yBsBSKzpuCnlOCtpyrqXUnJZpaikbiA/hbL8nNBEZx40Bdn3nY6y5iqkpAMDrN9daCjuO/eAdGbJj573vd2NubUkVkAggdiGXJLqS1HQI4Wwcc6uAWBZ0jn3wqFKq5VJVBUXHzbB1uO+7+zm1Voq0vCRBZqJaKpj5wAiw3Wy247jWNJRS0ryw94A4dh2CHd58tyxJFF7fHO8eZmJyTEyqhoLIjEuqTNT1wVTXVBCAiRispTuAaqZN1/uo8wzv3DfmkaU1RBRmrLLmb9Z2jzUsjUhAROwJCVBFdF6mdxBWNSJH7NSAwa5fvpXUnj179vjxo0fn+7HrQj+UUvphGIYtOZrnJS2pLvNxnksqMviUEwxd//gDGTelVn+xMx8n55fT6fzy/OLq8vNPf3zx7Nmw+2lTMm3H+9s8P7Dv3pzefPydT589e4/7ITT1y/T6sx8j8Xa33ezPuzGTc9Phnr0/u7hUhd/5rd8otW63u4fDQZoA2OF4fDgc+hhj8GbWWtv14WGpVRoCrhnMVWgUESIEs7XBxHnvfYghfkkLUkRj513Xx9hB7NYblgEi0+bs4tUnP5iXU1MDhFKWV68ngxerGxeBxuhH9sFmj0Hn+9LasN1KjP24bRYe5qpq05xM2zgMH773ZAj+cHyYUuq60QxKKUAUQkjziZ3bnl3FkQ1Ra6W+i/3oOyOi1pqa5pQxVyZWrc6HWnLoe0IKfed8YMfSCgIhQZqXBqWW4kLg0O8fj0xcS2GmEIKqtFZNAQj13bBJNZc4boftdjqeSlqI0HU9MrUmCIIhtlrZcezH0A0Iz8t0qqc7SZNJhbacnZ2Hjpx3XTeo2ZwLALZaTBuxA1TnA5MDqVYK9QPUsl5sPIGBAiERAzhUJ00VGhBRP4IaIwAPZuYcatiNu/0WmR1T9KHVBmKeXRdD9B7Uur6vOUvKceMc+2M+LYcleAci/XZbDO6nJavG87PaJMK7FpzSWozh8eNLAog+IMLY93PKrJpzNoPYdVXEtaZNFEFUNkOHYAiIqgjqEAXUpE6HnHJO06ks83w6Sq3OR98PahpjXE7pYUogUluBlZPJawEHbPdnRq7lpE3AjJikFtcShWChP9vuvPfb3WYYNtraotoPHQDX0hYARGBEQquleqKzoS81l2VhRAXLot65udW3X7xW1YtNv+k6NBm6btP19cv15Xrp88Sd49YaMPfBx01fN/2cFs/oCOrp5vqT7xPhJ59dv747RecBrIhSE0TwzE2UiLzHVKoZECGza9qcI6TfQwHpWvLqPCABgq3Cv6mt7WQAq4pP9E5jEwAkUhDvvZmWlERljWiu2WDnIxqy82v74us3Ny/e3P47/+of33zlp75IlEs6pjLuzhpCqdNuexH94rda83LlfFPwaF0Mfd8NZxe+G7A2dk7VIiCS1Zzykij2Syrq0vxw3/Udxn6zuxhy+vzVmzjuwA/UbwDhRz/4wecvXjnn9xyIaRxiWkJKgUP49LPPTKW2CgalNABUbTf39/O8DF1vAKLGjqsIMz672Ly8PRQRQlp9Z2ZA5M4fPbl+87mKIGK/vdg9/UiU3OvPHTl27uzx+xfv/9Tz995zgAnZ/7iTVg3t/v7mxWc/aqUgk6w+PkAi/okv12EtWfJRrr///1k224ubp+Pu4qYOz3/q558+fX7x6FE+wVU3bo8xdPH86pEL8eF4bMZQWisHH0M3bvvNVs1C1/nYiTQA2OzPtFUm8jGqNjPs+6HUrNIQEEzTqbTanONyPHTbHTERAyC2KoiKouwcIfkY87LUnJwLzSoxl9qOhwMzWpNWqphKy924YeelNsjZxy7Ejpl1zX8hETMgIBF5T4iAIK310Q8X5+OTyzFw8M4QXrx6+3D3kO8PANdiCNaYXIg9IBAhAZB3uH6jaLU4YiAEViMyExc6yUVraWnysWcfAMHEgNEwQE2+G90Xn3xCYpv9GTh/eXU+OOJxP+cGZuhDKpWYZZ4lZ+r7eph2e96dn2ltaZ7Bh2wCjaEfkPB+Wvq+r7XWFfGJJKK7cRMYfQzrdmzXd810CV4N2bEnX0si5CXlaVmOBz+djseHWyLXWs3LYghlXojZhzCfJhe8AQ7jFokM0LmwTIupxmEwYscu9iuj24PBagvU2vI05yWFYUilgkIxT0J7pm2gt8eH4F0wDT4M221ali74ods0kVyKY9cxzWYKqGD7zea4LLJSXVTMrN9trs72r67vGrpGbMC1GUFzCIAIhK1K8E5MW2u51tKap+308HYTPOUiYqJyfPlxOh1F2osv3qZUt5cdIbQqSARgTU3NUEEMsIpfuy8Jmbk1QYPaWm26HmQrGgjfQRlVRNZ2JmlNib8cfuO6YwNAQCHmWqsDaqWJKSAQM6g553yMrama5SWdPX/y5/70H//+9z/+9/7j/8d//1de/tKf+nOvn3w4gZPWgMx7/3B7fby7S+n0cDwx4tgNYM17V1vr+uHy8or7cdjsTQUJ+nHbmoZx3MdQUj7cH1qVMPoVNFZEP/rpn/vwmz9bSyHnHMI+F/FbJigp/fN/8S9Ox0Mr9fHjR7vt/tF7w9APNWfv+cmzJ59+8uLt9e3Vk3GZT8eHg9ZWSgWzZppqHbv45GL38ubQRN6Z7QCqtOk0n6bTel9DYgh9f/7Yuw7QDHieUri//6zKw5vXc63zstS66PpmevSO86qLGa6tKKuTLbK9t6HXM1zPTWyqKZXDbTL/0S/+N7TWj549vnryJHdNVYevfqUf+ynl1qRXM5U4bNhxiF0TCV0vquaDmQ6bkdgjoQI003o6xhidI6nJExmHFRwzbPc1La3WZT42g64fy5IoeoaVHJW6zdiaWHsHbV/miZyLPKLV2EVRAWRQ885z5el0ImIm140jvguuG3+5QamlSCuFXOi6pkIACLDkVJCmQ0GEuqTz891uv4/eHx7u8jzXWk6nh1KK7wZib1K9d3Gzc84zgHNcWwnd4EOU3KQWcr7Msws9GlCE2goBIBIgYq0r67wpuD/xrY8SeGU/dP5sOzrTUyMN2NSMnR9dF3xExOUo5MAo1VIrOu81BBdCroUYi7YqWEtuTYJ3yCilVbNhv717eIgxDmKeafQsJiDWETVVqfXt4eFwd+dDSCk7pvuc52kCZrOcUwIw5z11A4Ll1uJur7V6H0pTszb0vQ++5NRtt10I+912GPux6x2zEROxtlZang4n18ru+RPysdRSWy1ViQhqvj0usd+Gzflc5g0gAZLzTSCX6hD6EJpKqrbr+7FzYjA4yNrMx1N+N+D0Zpsudk8fVVOsFZCJyJgDg6r14qw3h+CZbu8PPeOzq13OlaOfioAL7Hxe5pqWXFJKGX6vbsOcf5clFJUmRqirRbaJMaGZrlbPWmWFUAbHMQbnSFUFhNkpQBNBgXEzqFlrTcUAV78Ar3QaIm4iaCaS1FYXmvpIKzm61WYGjt1S50fn+0DDX//ln/+tH332H/693/jH/+V3/sZf/LPf+uU/9aLbHquKaK1tqbmWWnJd3Xyt1SAiSqlOr16/vbx69Oyjr4YQvA/zvAAyqnofiJko8/oZ8F6tEOI4bjiGZZq0ianuHz07e8ZoBkhPPvpKy6WV2lqNMcSu64fewADw9u0Nx+Hxs0iEKaenIjktyzy31mot56qtFFuWs21/OKamiohgtOT51YsfLqejSEVEOty9/vEPrWku+Z0t481nau2eoqbD/d3bJZ1W0zITNzFZewYA1k6aFdPNAAR0vYAqMOFUoYoK4Nvj9CzXl5/++JtXQ2vWB8dMzoW0pJxKiD4OQy0lzRM79jFudnszja5X0VqSmq3GZmZSEQVQlVJaqxUA+nFDxNEHMQ1MgOi6IU+naZqG3UZrI+eQeNjvHZOwOh8AQaWtP6rk0mqV1gwsxg4QybHjwfejmVkT8s6FoK0JExOBqYgQE2FgF9bNCYioak0ZAYnZ1vacF68dQ2RgRlPpg3/04XMA4tDND7c3N3M9nVKaDJmYnPNdPyCApMlaamkiwDDuQ+yUPcQAaCpgrRCzipOaTNhawf/o3//fvbhfYvRj9FWxGRhS7OJhSYaMZqLmiDqywN7UQhcR0NTYsZgVE2Ne0lqQF+elVtVcCoXAxM7xOsiLGBNt+7jkFJ1jMGRkDvOylNamJd/f3uZldkStFAHyQ9eaaJNaMnkfY99ttl0/lJo9syMaulBK9sy7/X4YN47QO9bWAJCZ2DlVTalIq8TIACH25LjV4gmLkUiLPjjHJtrH0AzMmgOLMUqp1TRX01YGT+ZCKs2kOR+9tcfb3thPuXSOOu9MBAAULCmU0kpOTDxsNgYWCEttVS1NEzm31NI5d96HORWO8XQ8mVmV1scotx+//ME///GnL3/02VtR7LpYW1EBpHfytKg5jwgEZjE475gIwUDNcimICIDeUfSBCVb5a+ijY66teeZ+07cquZSVwBG8Z+aV0+18EFWRSujIMQKKKRE7x0TsYxc9T6mVXH7+61979cX1t7/3oz/+R3/h9sXrf/Ht7372409/4SvP/sKf+PmLD7+yPPvGm+NSa2ZiBj0dHlprzrn10nF3e+vZXzx71g9DNwzjuBGRWlsquaYECMy+idB6KBCJitbmQyi5LMs8bDbEbCLLNMWuV1MCAMdpXgih1Sa2ovkFwIL3iGsyuh0Oh9ev356Op5xTyTnnVHOdl6m1mpY8lbLGwlRluz2ruVSpCBi6YXf+BAAO1y9KSQAYYjw7uyp5OR3vU07MBPCu7kkNzBTeFXCsAhutgMOwhrzRxshoOlcExDmXn/sjv/z8vQ//jT/+/NlXvr6Az6kcsjzMqeSsavvL81IaI7Jz6+o89L0hIXrVqirIzgChtSYNibz3pmqqTRohheBBhX0EQseO3RpKk9ZERVcRFcG460vKpuqCRxUy4xCYmZhbrdPpaKt3et0IlaIinpmZWi3rBizEDomsCjpsKcWuVxAzzMtiTdhhXmYmMrUw9FKbtqoq8/2t866mKXZhM+42+/0wDpJmRlQDlbZMx6ZG7ImgplnzJGVpywlAw7CPu8s4bNCUAJRWZlq1Wnw3MCH+L//X/xsmxFod4dluIwBZbNd33lMFXprUJmoUHYcYeuahi1JLzmW/HYjpfi73ub6z66j5Lq7gZiPKpQqAiuWcOERGi97Pp1nMtmM/RE+EuVS33h9TrbXmWk6HoyLutlsXPCGejvOwGUOMXRfAIOe82wyB2DlWtVJySrnreyAys4hIqMQ8zamLgZ03EWJUUTFYvZ/StOs7JkQkZtQq7Kgp5JybioiNXRy6bsnltCQ0Dd577xFRagNC7xkRmSgCkpSpmZmlvCaTIDjene2YqJYSQJpCAySkUpsjQKTampqZqpXFhdjE2DmX71799j/+3vc/fjjl07wsy1qCDGtxFgI6ZiJQQ8/oPRORd+SI18KEddZ3jhgwxBVoSH3f0bsIp3rvmqiIEuNq2ida/xLRYJVU7f/P05/8WNZu+XnY6t5m79NERGZ++TW3qXtvdSRFWmwkAZRokRIEmzIsuBFAG7DlgQbWyAY88sCGRwZs2VP7v7DhmQcGPJAgCaJkyqRYVS6y6tat23xdZkZGxDln7/12ay0PdpI5zEECkbHPPu+71u/3PI6YpsQkqp8cuhJijDGir83c/fd//KN/+k/+/Jffv/+X/9bfQsUY4uOf/tP/+j//L37zy1//1sPx3/6X//Lv/J2/9Xz/4+eq6NZ1mOPoXVUf7o6AqMimjmCt9a3UEEIUISEzB7PhFnKOISDiXgFO8wyIOsb15QJuISft5qbuHqe89yslxFK2VlsrpfeRDrP15gDqbg7v3727vLysy/L88clUzXT3h9vQ3lstpY9RahtD1QzcU0ilV9gV5YczIrWytFb3DQARug/bfUYIiET7bweRmREJEQiRiEbvgsDk6ijMTMhMgmDuaq5mMR3/1b/2l/+l3/1ypcPd2y9QwtZ0Ot+Zac7T9XYLMQKSMGnvXQ0AjudTnI/WOzCFENy9lVJr633wLtYcHUnMTHX3e6bArO4iIiKEHkMY5qUU08HMMU2tbGN0lvCJewHe20DGIGHfWxAhII6hZV0AsV6uLhRETNXBYppijOYObm46+iBEIgwxIaBq77W0uvmwkLK7OXwCZI7RCbFvC8FQd5HYtyuh7P+PTBCmmYi0lb5crK7aNm0beXM3kpiOryQfJeYQEzGbdlNFMBsq756e0DmkYK3eXp4f3r69dnv/dAkxxcjuGFIy9GVZaquM/Op8jOC11tvT0/k0W8zaGyIBy7qVM+E5hXkKax/bGGMoieSUEDFPE7jd3Z1TFB29DiWEpn5bt9OJcs4xpTPTm1f34EjMQy0EuTudeCfkITBASKFvpZhN0xRCkMB38bhuxRENsLS+rYubHY/Hx5frNGUhyikgko1+dzgquwvkFNEMCfpQ9R0ApTGIVaBI7l5qRabz6eAO6KajA2FMUmptTQOLErxsNZOWrQBzqaOs9eHNnSFebwsRja4SOInUsgExInW1LGBqSBACh3wvzDvVw6PXbiHl0ExaH1r2lT8RAsAYaqq9KyC5cR8mgm7S6RODZR+fuSMwEhIRCTO4l1Z3ikkfA/b5AqATm1kfivgpmz7G2Ge3sEFK6OCmgzkBoLq13m3AF28e7mf+5vv36PD08SWn7ECvf/f3/95f/Ivf/eIXf//v/5f/l//oH/z4D/7kv/ev/ZWf/s7P3j38rCu8mkIjLDg/vSwOkIK4KyF28+kwL8u6rktZlsPd3fnuPCHGVntdfT4xo40GlkhCnPLhcKi1ltq25TnEZNpLqb12B8uHoxkYEYV4mGdwaK2a+zbs+eV6XbfSuyM4Qmk1iJh5b+MTxUcEAGJKaTrMx+N8OiNQCHy+vze1p48fvvv2ewBF4fzPXrKH4/FwPCFaiCmGAOBlK+42TVOMqW4rs/TWMrZXPLZSvn26ff1xRQQW2mvbAigxnM93PpZffPvhsT/nd8/5eOYQ7kqNacrqQUIbSlGGegw5ZTTV1vp0dM6p1Xa9XERknqa782kr9batCD7UTLvgnjdydxvqkpKO0XqzMWKQnOYpT0O7da3rojpCDAhIQUwN3ac5m5urgyowqWGKEd1pnpD5eJiHmUgopYBZlADgQwdKAN3RGlZLrbUhoLsi4Xw4melOkDJAHUNCCEwEyAi3x+96XYnIgRAbujOiauNrJCJhcm06xujNzMkIwb2Pcn2cWehwarUScwh7GNJAGP+D//X/dpRtjD7U+rrlaUIEcKTA5phiQIkGwERpmoAopTjHUMv6/HIz89dv3qjZblhtrdfWYs6ff/bq/pCH08fnCyDOhwMhEOJhmlSH7Mk6JzXtuhuhkIWZeZIgQQgscGi9m5uZphgZvJu1ruZAhMI0hg8zIjqmWFrlmITJ1dpO43I3pDknIQLEbSvCNOWUU0QHdSu1mhkDUghmFpljlForE+ac3X0MHQZqZg7kykymllIcvdfW9xAcmuUQlBABt9JL3bSP0+kQYhDiWpvqAKA8TwSOAIl5x5w7IIMTASGR0Dd/9J//wd//T4v6VurLy3UrHRERIcawC+V6b74H9gFVnXnXljsLgwMRiEiMIcWQhHhPHADUWkOIap8mayEI4j8Pu0MI8ol8j2QIMUZEijESoQSREB0I3H/n7eEY5PVnX/xXf/SL/9d/8o9utSGF3//h27/9N/+l8zS9/a2fPL1cjyH+43/8h//p//s//u5P/+y3T/L3/u7fPP2df3ujwzzJyza+e75ta9mZJa3WIPzm9V0O8bpsgFAvlw//9E/qy8vTt98i02d/+18nDsfTmRENgJjLVhC9D1vK5o6HeT6eTtt6G2Pkw2G0vpW6h3rJ/fLy1EbblFOe1qW8vDx9/PD9tq5gvl+qx2j7RZAQX795/bu///vnh4d1Lc6cRHw0ZAohiHB3Xktf14K2PwVovXGQ3lqe596aD0XCMUpdt2meCGBbFgMA7XdTOES6p/6Pfv71N08LEpoZOgwb0zQfz3fn4/l4OISUAGmMAYgppulwBKYpynycHz57G0LettWGnu7vhamPLiFGCX0PSSAjgqkyk4MLi5lt28YiZr1t1YaeHh4IIeY8+lBTQgLmUdsnJjWhsKQYFdFUR22AYAhojoimSsxRGMHNHZGBUHWvNgwYBrhP3juAhxj2BgkiJaFW2rJtqr2VNXBAZjA7HGZHcAfX4eAhhHq9PH/4bg+UoEOvSyt1zokIADFIcFcfXftmY6DbDjYS4ePDZzKd94QAovdlURshTfg//vf/56BmCPuMiT8tAXE+HQDI3fbALjOfcgwS1KG3WmpNOQtD78YiFBK6nVJ42upwDEFen+aY0rWU29bcsZZiZvd3JwKIgduwOSdm2VofY6QUkVAN0I0IACiFoDrM3AElMOgARAc8froegiDDjtPTkXN0g6G6E2zMQRDUTNXUFIlrV3Ko622ep/P51Ht3Iu3WeyUmQJjytEeQzIxFsoipxhja0DZs6P5L9CwkCEDU1RyglMKIOacUI7gPhxQCIfTWa9mGe0zJHUw1MquZEAlDChnBYwpl20S4Pv36P/t//t/ffbyGGM395eVatmLmIQoiAsKOAEUkd9g5hbuIgxkZCRBZOAjlGPIUQXVX+u0Gd1OTKAhoZsxMLPvXNhIFYQAHZAcw0/1CGWMiohCDRAEUVP03/8ZP73PSeP4P/6//t//65788zMcfffbqX/yLvytEh8N8nLMj3929Or160Lr9w3/wD58G/LW/8Dtf/uyneT6eZ8laH6/bz99dt61FBvn43Xd//Cfv3r27uheDk/fL9x9v19t1Ss+lvU3yo7/6L7Z1m1KQkO5Tmr766qLmqqfz3XDDkNw9CK/LtfURmThkZHbtOkbZNkRqOrqCOezzsrqtTDLPKU85xKStgFsvRUR+/OPfulwuqoNTOp1P27L2ruBuo4aY57sHByAJ7u5mIOH28ly3xZ0IoZZCiCwcU3K30dpe7pEgKScg1t6D65cz/vz721LbdJgkBABMzMfDnOfj64dzd7gttbW2bav2YdbruiK4mzHj+XR68+VXr7/8AXLYe2xm0FoPQVRHjEGC2BgsMQipqupwAxZurXU1dGQGQmQEYmq1swiHYGO00bT36XCwWgPhPE2B3IB3xcxSuw3dN0KjtRgD7U04JjBnkdG7mYE7gLuZmhHT3lcbY9gYSERmCEYhurkTah+9bDZ6iMEBQNVGOdy9ArDYNxzlZSnoY13rVgoH8TEQzM1E2Ha95ujMgowpxHQ4AzIQjbp5L0Ac84QkEqYppUzMgBRSFOb9GU9TdnM1q+va+8t8mMrtdjydU4zqxCG1PoZLrf0QEgE6QGA65XRpWoZ98/hynicSOk8psFwYnl6uj0/POafJMzFXNQatrSKi7K9RMEAspW615RgPh4Oz6zByNwcbQ4eWrd6djwiw3V7ynJ241jqVcMxpEqlDibD0jlEOTBzjc+m1dwQoY6ikBlyGzjkzYoXmKOiQU2y1ITM4uHktXUVNh5oBUo7BjNysmy7XNaQYBZIIIpwOD6Pr3oQJOSZEUxujxxRzTvu4AR1ab2spqkMNOczNB/VOjCFNOMq3//Qfla2oGqs5YgiinXdH3C4uIQDJCUmGDi8WmAxhT40ZOCKYmRuC+x4pMDMdigg7PXt05f3173uFzIMIuI+hBhAimdo+PHZ3QJAURAQo9NYm4SlKR1k6bsvtf/R3/82/8Zd+Z5SNgrj5+XR6WhvFmae52/hY+2d/5a98kebFxp8/Xeh6s1rz9eP09PWXd/e//PXz08vtP//Hf/DHv/zN1bURB/MT4RcpOeK1j9r617W+/Gf/RSBYJYC5XC8/+st/+ad/828h+mXd5uNJYiprUdM4n+eYJcj1+eJuo2uajoET+sA+oKsaIrWHV2+Ox0MMcr08A2LZltEqmn388J6IpxgdtLSqj+37Px/T+Xy8ezMc5sMp50QxlGXTWmOQ3tvxfH8+nXpObtZqCSlsa9uWlx3AO+W5lXJ+darbioCt9SnlKYVHh1c/fPWGaWdapBg/n8Pr0wyAH6ueU7473zlJG7qU8uH770dtxDwdDrWs33333eOHDz+r9ce//xdrqSGEoU4iHEIfvdSGtbhjSAAQ3H2ojW7Q2jRNIXIrm+oAkeXlkuZZRMAdAWIK05R6qa22Vgckebpc96PA/XG6P80EuNKw5iIiKZqahDBGZ0QgAFMfo9uwvVmvigg8cI86MjGQiQRwq2VFb8QsSGmeulCvdYyxk1dGa6O/J8YJ7TDnN68PMZIbXK/X2kbrfTjVbXXrrqP35goc+Pz2yxhn145uwEIkmA8UExJq3eT152/RUZhrrQCgfThAyIncT8cDijyHoGoKDkybY+sjxTBLvJ+Cur+XeGvNSmWAGEKe8inAZVlK1+d1zdOUgAjGOUfzkwGqGQAys6lNU94HQ631obrn3B1pnufRe2lNmFJK7mbqABRTZPRa6tYHAbBZYEkpL6WU1l8d56Gm7iKBOH5cFhHZUygSeE9e6VA3vy7bthZ1m3M+H9JwkBiHWW39/nRW09o7EO/5q2YWEIcBOD7c3ccUCGGY9tpHHTEKCX/CyZjtTMR9PV9bi0xTislJprnJqKO5WSll1AJbzSn3l+/rugCj6mjNJWViQkIGAgA3R3IWYgIWQg5gXrwZudm+QYP9LQXgw2zUPdYPgIDuoACAzND7CEFsv7juYGhAYASEMZSIkAX2tA4RAnKImCYFSpGJ8HbbfvwXfu8/+Hv/TpjOw/rD61cAMB0OmGe/bavStfWtbigizNeXRwBDZjc3s8Oc/vQf/fLv/yf/jx8c5o/P7ZmCnE5Zx9QHp/zAsC5lCC211TEIoGhFJmaeI72L0xenu8DkIQQSBYjAx/MZmJF49D6ahhgICfIMiBwzaGsRwn7+JLjd1lrX6+Nza0Nt1PXmNlxNYs4xfXj//es3r/pQDtP5PJXl2tva1dl6X17CdAhxmk8nNbfR3U3dQghA4e71Azq+vFyvMYQUCR2A7u7ulrXknGOIGObWh0wBx2AWFgHXOfAs/Hi9vbtup8NcIcqo4COmycwOSeYf/aB99hrAFSgEAe1pmiQEdz8eDiHGVsswMNPDYUYi7WNv/FYwJk4xaVv3ALAEDIfJAcwsvnpgJDQlNyRolxeKfDqewxyQhJnMxvNSt2FjtG1dUeIUYxK6rQ0IWaTU2moB4hQDAZJQsv2BcVdDwv2cAYTsioyunUKY5kPZVqEAaih2mOcmsleDe2uMd72sFGIpa+3MY4znYnWdUwjCISRCqgG6gY4ZbDCiCDEFR2SR/UCIRByS60BgZpbM3HojYDfbV7zMlOYpE90nOmV+HU/fL22pffcheu+fn5ITqXkQmgLV6qYGQZ7Xkh1yCA+nyU9zqX04uPtaK6h2w3UrccrgPmrNMd6WrdUaRHKOOWQHDyIOcLve8pznPOHe8VeXKa/LOrSCkBDdHQ9laGkVgCJRjnJIiQnVISMB0fW2Docco7YWUmxdGai1jrtZkvH+4bwvsIZZH7qu9XQ6HHKSQKM0Yc4hEEHXkUmQ0NwjSyBBAhsK4IAQYwhBAGDs83U3QGbhWtuw4eYapLUuxAQ+R46Sam0xZkBqt9u3v/qz5eO7oO5DwUFVl5crgKl5a2N3XnbhKLtPZDgR77RnM3dnFlUXIgnUh3rrDkBECIZIzIT7Zw8AkdScEcGdkADZfBDs/4Kqqog4kAOZGYrQdAzToTtNU2jXRWEWDC8Vnj6+d4dJ4PO3D8/ePn7/0oY1GzIdiXC01SUut6uZPz8+qVsQihMgp1/y3S8LTKTNOkv4HGCa80VCX24rmAFExyzEQELYQzwe8/mrH8zN318v/+A//o/+pb/1t3FOeZokBBZat0JsPgYKC+c+lJncIQZhmAI6mI3WMcYMECIS8Qz47ptv+gAmOd/fHY/H6/NHHUGm8+cPn398923Ztul4JOIoEYSG0tP7j2+/+mFmAYTp/BAY3ay1Os+hlhqZ37y+Px2Py7Ye5rwsRREPpzuBnnMuw46nGYhqadrbFHmtupRmQfj0EEWYCerYtu045ZTSMHh+fkH00TWmFFLsZgihF43qy21BlpgyuyFxLQuoAUCeJwfiGPqywTA/nfJhqrUsZcPGCM78iYEeg8Qg5BQIcJq6+rKsxyg58bpuc45f3h0HeNnWUjsCai3dBgCOoRwEiRjZiWrvgWgKGdjH/oy6MbAjCtMhhZDYHK+tg6sBpHm2Plqry/U5cDic71hCWW91WT5Vl3QTliyMhCNIRVj6QHP2Yb2iK3FgJuBIAH20DpXKpjaYhBGAEMDB1K3b6HJMUtCRieWobqqeQrw/zMH6UUBb064MsB83WEgNh8NnxxxDUPdb08Pp0NogEWYiolqrQDwcpihc2yitOXNzH675MAWiecq9DzXvve3COkTKKTgguPc+7u/PZuam6EYiHFiYbARr/XiYWGQM3S7L3d3dcHhebjlPZWgBbKWGKO6QcgpO19tW1yUdjiJSdSBynlJtrdQaRIR5jjLUWzeJYZgLkfZ+nKfRR2ltraOOMRIKk5mV1glaDpxEEosgGUJrnZmcMKdEbkJibvORzXyoNx1tjLUUG+N8PE4x9lokUAy5aLdXr+5fv/H68de/+rXZrau/XEvTAQ616VDd07JMezcTkVCYP+ECAZB017XUgTpU9pkKmfCOnN178Ajg5u4u9CmCgYhogA5gwwD3cx26g7srEqccpgNwdODjnP/pn/zi85/87uPT49s3d/X6q7svfhByXNSX69LGAISug3xwOlLMyOHtDw/E9PmP/XZ5+eUf/MG3l/qjL376P/y3P//mu3ePT8/f/Pqby9PHCLSBLa1aHwVAxuiOApDIlUN3649PerpTpLqt27DrVs93r4HZiT++XISl1y1KSMTLuqWchbm2ttZqZmM0GBaCdO11K+guwpePj4x+OBxce9lueY7pOKdwxym8f//u+++++62f/vb9lz843T0EibfbjUUePu/pMPnQGJARxxgShUNSMzfYxhi6Hg7HNzOj9tfp7qmO6oiURkjEKimAw+F4N8YgxHjw0boTlVpxtC4cQ3LHjnwrnTk081E3Rs7HuCxrDMHc67a5K1EAAKbNECTGutwEab1d0DqHgEzau/du2j//6svD+S5Px73W1hWRRUJwt7UUNk2bUZqvtZa6PYowCQDC+4+B8Bj57nQ0wOAjszsQkiRCEl6W1QFGH0jYTLEWCWHdapwymg8dO6Pl5XrDUVmicuhj5JR435sj1To2LW3o6Xw+Ho95ysttMffR2vL8vAbO00xEjhhzAnekICnrthCxu4IrSgASN++tIVHTRgDgTlCcSEdPkQV0MMAYen88iPDaxkRwD/Vlqx9WayTXAYAISKrqNo7zxCKt9SRyWdZlWSFPJDzGYIrmLhJqVy7tMMXEFAJvwx2ZzOZ5GrWNMcyc988/ERM19X5dcoqBOYVgbq22NgbqiCIxhKI7w1SGWh+NiQDdtLv5lHMUyYH7GBQCENdtI5EpcY7zyKmNQeA7VysKgpG77NrBbdie2OrqO67DwKwruOeUDvM8zBFxjLGW4QAsvPWxNg3CusdcdbALIrbRhBnBzI3A9mlVQBKJS+0h59r7tm3oqrYFptPdmWPsrfucjm++fLy1shYKgRyGqiMYQB2uZg5K8MlbLrRPJhF3KAjpDhakPWhAOzoM9iSnBPpn5FR08B1Iu4e2ANGYwPdhHzAjR04SkGNTj2qIENP87Xfvbx9u39VfXnr4G3/zb4VXb59Kf77durWBMEblmPPhoGBjNOSAOny03rybMqGjP97W2y9+9fJyMeKPl6WbgePNhhoXx20oI6CwCydEG6pbZTOM4cP3H8JxZpSvfvf3+NXrtZRufbst8/HMKTmCASpQyBkQbuvKEkpppr3Vfnp4MNPeWiktT1Nv9XK53m63FAO5p8jb5ZnSYbnc1tvLcPwr/8p/87O3byFE3cEsEsxsPp0AgWIEwF6K6dhTZnk+ttaiEAGU2jjAMWYfejjksQ1yS8TdfAxnwlrr6TCzj4B+ujszkQEsdVxKrbVzDGo++uCux8P0ie3rLsyOGJjj+WTqhyR3EVwSIAchtrOQ3eqbb95/eLosMEa/rfnhYZTtFz//c7Ru2lNOHBIgTtPExF+8fj3f3RdViaK92+gEvi6bpAnBtucXDOGdA3+4chJtI4RPPMQoDKObeTqfEwCDb9umBsE95rBHEU2HqoEE4ORGpkqubrSWNnpLIR6mWQBab9rH5eXleqUc5f7hjY+6u36Q2FURzUf/xC1x1NbcjeIUcray7sojJ2gr9qHMsidDtRUE4hB1NPw//If/x5zj81oRIEuoXWfGH95PHtI3l21tHdNUSkUJaH48TG/vDhPY/RQY4etr+7CWgdLUSikxJXSfpwkJooTzIc8xuNvWtKoPNQkcQvjEKQYzRyKMIiSybltgqqU7ces1hoBAXXuUYMstxtTBppxTCATW3bshM885CMBQ22pVxx0bjYAS9lkkqikRgVlKyRF760Iw5ZSIALyrvSzFAFJMvbUQJKUksOdZKbqe52yMaLY17e6tGwu6E4EigKs5uCEwcesjxGCqKSYdfagOHZE4SjCwbS2ltWEmRNba3f0pihCRtvXrP/vD//q//Acfn597H6P3PrqZmVrr3cx612EOO5UMcT9z7V7VT1ISphhYiPaMPYCnFBmRmQkBEAhJzdRcRFgYAZgIifad+k55ZJY0TSFNKGE+n0+v37TWl49PH3718y8+f/30dP2rf+2vf/nljzbzQWSqrr2ty+X9+/nhVZhmN7vdLstaVU2Yeu+tje+/f38UxtFNMiBJiJfLpdXym1/8so/x2Q9+PH/2ee/t/de/vr48i/ZgICSvf/CD86tXz+tSzaf7V7/1279zd3eSGBiJd9mU275A3LH9IkJM1g0Q8zy1XpelzHMW5t76tm2X27out9FHq1u9vhD489P7n/327z28fbvfrAHo/OrBTJlDrx3QEdAACFFbAdWQ8jB1t5DScrnkHImobqv2Eac5Smi9M0Kc5hDTcrvNOSIxMgdmsfEQ/PFaV46uZqbHKeac58BuJjFuqrXZMO99uFmKwRC3ZQW37F18zKTl+vzNt9/N5zOR4HQ8ne+mwD76w91xUFouz68yPV7L++fr01JYt1no43Ux8GWr27LkFGOafvDbv8cxpxgCszMPVQBYbwsRxpyFiETKugFCyLPWaq5gtjelYsocBAmFSFtfL7e71w8xxT36u2+c9kQBEgix6gDHWrY+ugCAjhDj+fUrM2/bOloLMZ7n9PnDgSS8vFyvW3MbvVWtDehTO91UR2scgqkBeM5p1IroUZBYDKAsq5nuIxdCGnWT989XtrHVkedJDvnukFvH7zuf0X77zd2U5NbtXT1+93I1UwTXPhSVMDsMIJrneWvdkTuzjhFD6KMjUR9j29bT4XCc8mlKs3sd1s0JcfS+KzPAXM0Jsdc6WldAA2BwdKi1IbMQtzGqWogxEE5JgpsitmoKMKwTQhZJIvdzasNfSiulsITRu9Ju6LCUWA3Wy42ZHg4zgR0Z5yiJERE/P06RQR3ePW6ufX1++fycIeYNyW63HwnIfFCKz7ftUsezQlcN5GaWCHKWNw93Y/StjrWPaj4A6rbqGJJSztmGDbfemoOfTkciUoDt+lLWdQGaxR6//cV/9V/+f14+PvcxwD3GAGCmYISA4EOjiIHbMHAw+NRcnnPctZa9m5vvof9PybVdksmgqmOXlJCb+lA1c1YjQg8CZkQkLMzCEhyhmbXaWd3XNczl8vHx63/6Tx7Oh+++/zDleTH/5ft3h9PJkR3USrk9fXx+ftpU0+m8XJfn50vTHQ4t63J1VS3r4XTEfJ7uHsI0IeDx8x8g4Wc//m0AwBCn0xmYfvJ7v+dq7kAAKBLzhMI/TbnVQgDCLISB2dV6707Sh0YJpi7uiKjmMccBwwG3ZTmcjzGm3XVStkVHz+zTm9fuXpbLx9Fjzm9/8IM3b96oupYbS8R0ABIAZBFzYKLWG5mHnN3BeTgicijLDZAP968QnFzb8ClTSHn0gQLqUMagIBJC7TrNwc2GuzBtqnfnE1Nqqn306u5tYCu7s299efn23QcEJCYh8hynaT6Cal3actu2a83pH//BP8FpTsWm47x8+/3hdI75WNab1fX1Od+f71oQcT+R3j1E6LDV9pf+8s9M9UPxb65N3dM0G7KOYYhGOMc5ErmpBNmWxVRL1wQg5Mf7V+bQEVRH2UovNeWJY+ylIKGxjGE8560UG32aM4EZAqKN2oHZzWvbiFlbQ9fT8dTq2nRoH+3DY45REEKQdV3qcoVWfvf3f/s8yeP3H65LuZXr3WE6352sFsl3GKb1trzcbqUU4tDMFIkRttrQapomIQAgFLHRwX0g4f/if/O/OwSpralh7eP+1QOYqoOP/nCcf/LZvfbxrsN12KjDzabjNAk/5PRmYkz5ca1r68NwKa32PUnsIYQ9U84shxiPc3JAQCCg4b7VWkqpXdetINE0zwieg8QYdxKLmbnZnLMiqFmr3c1Pc06BT0EOKTSHqnYttQ/bv5+j8BwDoJeqRhgY3WFPGmShAB6EA2ISPB0mc1jaAJKPj4+Xp49k45v3j4/Ldvfm9XK9kOsU8+V2c7MvTqff+fGXPznK/PoNu22t/+KbD24uKV0hLFsV9LFtD6/uX795hQC3bgoEROVTpYOGGaoDeRBx7cvtyjCe3n17fXm+fHz/4d2H5ba03m0MQAAwHap9qJup7a0p9J3cCQbuDlutZjAlQXdz36Gyh7y3yh12H8ouryZEIgfXoQCwt+2YSIIAuMheUgqSJ0lJncpWYp6Q2VWf372/vTwhYkrpd//KX4nz+XA6IaCOgciEULZtXbehQCHK4fDZ51/2WnurffhAw2HldultHO7u4zTr0JwTIaace29mgERm+/lVUxAw6rWe3r6JeVqXm/VBjCw82oghADrSnikl6yPGSEwSRN21m6CpKcUMDiGyqvbSurZWS60NAFUHOlyfn6bjiQGnw4GFtfW6XpDgsx/8NOZpjP3RBSbcq4iqujeTAEBVmbmVjkxjDAbotTIZEdteIXDnKPM01XXdNVT3U5iFbm2g23DMkYvxIWCy7mbPt5Xn6Xa9rqU+fvhw+fgohO7aypby5KMjejqc1+XKgKoqEtyd0VU1z3OaJnOEtp3nFPP8+jxvWz0G/kd/+MfX0r784s1/42c/enx8+vyzV6+/+OI3lzEodZk4Sq0VzKx3DjJPc8rTGM3dWu+j9d5bzrOr4q746l1SdAckBFURQUQ3q8sVwWWaD0HOSZLQtfS1VEVswzmwq4IDmhGh9lpNERjM+hijroEZiKCXOEqIadTy7t27pbXRCns9RhmjHY/nL7/86vXdnXGYppjPr94/3dZS2xgIHpj66KMWliASrBU3GL3j//7/9H/+0cNBwD9W22pjFiSUEJ5u2xTCm2Ma5ksdA7kDLGtVt8OUDzmFXn/r89enQ2pdl+HfXbdL1a462pAgwzSEkGKMOfEneAAcUxJBAWfhPRnQHd5fbmsbIYZERAAppmGj1EYOIYZ5nvc9LuGnU1tAOJ+P7GAAa6toqICGcIoiiGttIJyDDLNTSsdAvnMKzd1UXKOP2sevPjx/d7n+/E/+dLjlFK/Pl+PdWRjL9UYxkUgr5XA6pfmQXF+d7zLjD075y89fv5rTy8ttc7jcrt98XOY5O/J0Ombi+ylBnrrjVhswqgEQsatp224vy+V5vV23bW29r+u2XC86xk7muV6uAmg2mo5WmqqNMeJOvFBDAOJ9qsX7SeS2lt6Gu+2NS3MXJmE2cySgXT/IuC8N9omYCIcQ9vA3EoYQJIikHFIiCSRBHa4vL7Vpb2273cbwtz/+yenhs1evX7/64gtTDzEGCdZ7SqmVAuDdLB8Oy+UmKTx++21d12meJeY0T6Yap2m7rSIcpxkAAhMQ7fIBIpIgal62qqPneeq1hRhDkE/naREH77WmGACgj65mQjzGQPCUMofwyWcBNIbundN8OGzXBQmG6la21hog2lDcM3e9whhmFlIiZjQfvd+dD6dXn5mjueZpcnMmlMC9D0IAZELa9aNgNtTPk0TT5+u1Oy61kYRpyrsIpdV2vn+IUZbbrbV2iOHLYxLmrfWivmwlMCJg72NtvRuEENh7K/X9h3fEtC7Ly4f3vawxJgfXXpED6OAgEqK1prrP/nQ+nvI8sYQ3p9Rbv66btnpb1r/4W1+8PaTrup7vX49WV4jxcFqeXv7qv/Cz+/u7x4pfX9uOCQOkoTr6CDGMPnSMmCLtJRAAHapuO6QPzQGdiVtrrWwphvOUk9BxSjFGZoQx2A2JQkqU88f3H6/LtvSxtWFma9nQPeTsY5Tl1nu7vTxdPrxnoVE3dBOJ091ZexvbZV3W5XZLkVKMQpRiTCmLyDHHzz7/PE6HAaENdVciMRtt24iYQkTvIsHU5Lpuv3F7k+Scw8/uz2L9wzbel55Z5iSR7LMpnl5NyOHm9PXL9u1lbcOwjYnju+eFwV8f8g/vwptZ/uzj9t3L6gQhCAKjQ6u19h5EYhAAuKwLut3P6fPTnbi+nhMR/Fmkr6/1VltpjZm8g4jkaXI3ZtlqAzcCd2J1u5bq4N8+vZzmec7pboqR4f40dbWI3gwU43C6li6E17GWlJ5u623bLi/PZdtaKZcP77W3ZVu2ywUJD6ezNSHG5w8fgohM+XR3d7289N51jOePHyVEPp5Hnv9w0T/80/d/7fPTcx2XDjic8vGxW2C8PV7iYf664XnTE+uyXsp6DSEs27rerqOUrbUxrGwbpUjMOnqIEQDHGCGn2bxvm6r3bhJTQKy15CCq3dTMgNiIJMYEiBLkcOfX63J5vtTW9mmCmnV1AGdm2qtrSOaupkgYRPYDGgAMNVNUtByIgIq69raVS2/a1TkmycdXr758ePP2q9/6CYfEiIfjsbdqZhITpklEnENXvT4+ynQAkZByPp4Op3Oepp2K5qzCkg/zLiJGolLb8XwmFtMRYyThSDzUQ4oxRSIKMQEiWYVPzjdKOfpQB0gpD4cpxVrqGGOYjVqQKIiojjSnMbyNgaV+/PDy+vPP0oROzNGG6u35Yy+LBAGwVqvrLmlBd83TRASuI+SJOcGnK0topdTeT6dDjKGsW2sjxIhMU4RhNueU1a+3zZCDSK9dArvZznX8+PHFRvfRH5cV4X6O4eE8fZYS9Fpaf1e0iedoGZEIe2938/z08rys6/H+TZ7PH37zq1ZXJGpdA7Cpt7GNy5UQQwjYFFAMOKTp/pC37bb1jkyl9jzP71YF7y+r/vr24SHRosv1u/fzPL//+LFeLz7KF29+65tlMLiqUUhJ+M1xWmp/vlytVSOcgqQQPIiD71V57z2n0Ax0zmOl4Z6jAOCt9Km185wUaO2KWuvzlUJcy2Y2LrfS1eq2lXUZOrR3pF1MYcTihL13c+xtyBi1bSlGcnj96nXKR7VOOlqvZm0opHkezW/fPSW5nO7uYgwhBmT2bhKDmgP4GEPNEUFK7aZjXchD+P7p9qNjqI7X6uuwx+v1Q5D7Of3+6/nVEZPQBvXRx3DcliUe523g01Yft3bIMRImoeOctz7UDB32qpBI6GMgQJ7S6INCeO4Wr0sCR7PfenP6nVd5jvJUczfo+5NqNtRiigDAaCFNtamEXfgdd3y7AlSzx7WC++OyTjHlGJR46Ta0O/itqIO1x+dvvv6NjdrKtjw/l9u1LDcAb62ZOYcQp2n03raVYxy1zIFq3YTlzc9+B4nv7l+FGLdlRZa6XTmkP3jcxtBh6kDgOhFNr46R/BAxYoftw+PT+3cfn9c6DEDdy7YyAIaASBijxMQIHbH0oUOZaJpnRxlmUxSHLeXIjNOY0Hqv2FrDvW6Zc5zmAYRI2Ho6MpbOKG1Zzcz2yggho7KQMKs6s+/nWTXcmoJbCAGZEXF0azBsWfZzewjxix/+8Ec//HGejylPwnt7Orh7nqY+dG/o3pbl7u68lY2J0jyFS+xbZaayrvP5LMKn43kMHdpbbSJhOgizmHstFdL+PJMbScq1VmbiEHBHdcTQSs1TVlOJCRxaazmHocbMIUiSYKppSjKCg9vQNKVSaoiTCDPDlJO5vf3RFwjeWnNHCQEAJE0AdLs8m6owAkIbI8ac83Sc5zFqWV5CKxRiPp7ckZlSCuDe2nCzndfY+5imfQznT0VTym9Senx6EREGoBBqbcRg3qcc3eN6vaWUi4GM7hWI0REPx8MDtXJZh6sNV1cbPZ6OP/vt3/75z3/RWo/TfP78q/V60dGGQe/NhiMyB44piQRiIqTT/fmrLz8XHe9ad0U1k+NdYMbp8Mtl1eHaytMC14/vGCHG8Kc43PAXf/yH/92/+2+l+59suvPVvdTWh0amc4prqzZ81SYcbss1hHAQToIft9r6GA4ofDcflnX79uPCOVrvE/nj8wU5ABO6vX+51dZVbdvWsl60tV7LcrmUWuf5MB2PtRRGyFHK5fn28pxSTIwS5PWrux+8fX13mqfp2Eg+Xm4wOvZtqf2laD6e9tlKb/X55QUB5uMxSDQbIcYQk/XOLKU2RhTvbatjUavI3+r4U/B5mgZCqd0dXoSv1/DxI/744fz5w/npVnpXB05JBlhDeSp6nLOHeCll67Z7c1vvrbU8zZKTqnfVUrbldospAjO4r2tB8G9j2Pp4O8cvpyBu/+T7F0Ywwl6bxbyWKkynwOAqjLMQMwqxq3X0tbS1KTEFhkDyXNeU4vl4cHA1u63L9XJbXp4ev/tGTbVstWzr5cIxmEMrW2/d3amPOB+Red1qUCNhWsqXP3w4f/aWSHBvmI+ViNoYIU97AitliQAR8WHiNwHm2ce2vv/w/lffv3t8vqxbZREHI5b5/r4vWxvjmOfSOhHpp0tcPd2dynKbDtmJOca7h/tem4QENlT7NOfRdrQG61AiAhILeVm25XZdrouqXZctMEPMBE6wA2cYHKbjtLtsCcABJEhv4/pycbcMIgYchDDGeFRzwXiY59/6nd+9f/0Z58zICBBiNDcgNrO9RMyScgh124Z5G8bkkcb54X653tCAiIF4DKtDl+t6vDue8mRuwsIcgsh8NHPQ3hBR8lRbc8RaW0zB3JgEHW/bZSs1TQEAaq8ppaHGQUTEiRC8t8YxSYyINpgRSSTklLqOWjcwK6Wkabq8XGMMQBxikhBOpxOczuf7+9v1st1uQ0eM+Xg+xiBI6G6qvaz17u3nZubmQ9nNHLwstx6E9iWVg6kTIjKx8G4w3k03lOLoY6dIlW1D5BDDfJwkxDFGcfj+1m5r/dEXnzn499dt3bZa+7audV3PD/fbuKQYf/Sz317W9fHd+1YbUJjPxxCnsl133dHoI87HaZqISHsPKV3W+vL4kZhCnFB41FqXG5Y1pVgJJE3a6puvfrhtW7kujxqOdw+f/aW/fvGYBW14b03AW9nm+7ObIwuFpK0j4bJuSLJ1ZffT8ZQ8XLetDrWmZkBEknPtzXTUouu27Yh2BJQYQp5QdWIxt6U/1a5OolaGeiv1MOfzFOdAP3k1qX51nqa3n53jdCSSUdfr5ZYTP8zzIUUwf/v5w0D8+S/efXi+lFoBfJonQkCWMWz0FcH2eNon+hJR7x3/3f/Jv+cO7oCMTDSG+hjEDEw6hiAhM6ekZbub53A4Yspg7jpYJDBzkMNhfpUDgee7e+BwK61027ZtnqbD+dx6PwsExpelruplqKpRCIgwpZSYxPT3P7uLkf/hN8/bMGIGsBTSPr0mRCG6y+EuioTwodTnl4WjcAhraSjsbQATuR+Px22rt+X2/Py83q7L89P16f16vdR10VrjPLGk6XTkEEspvTYHsNEPpxMQ9a2OMZjxL/7Vv5YPx9u1HM7nu7vzMNXR21ZCzrvrZA7847v59QRYL4/vv3/38enx5fayrLUPcGt9sMg/V2OHPNU+BDzNh1LHdDwiACAkoRDjtt5SjKrmqugwRuvbtlwvTOgOu207kC3XFQhvSyuKj49POjoSS0wS03w6EeA0T0AkzCyRmPM86Rgxz3VdzO1wd9dq0WFlW4PIfD6f7x+AudfGLNM05eMhxVRrc3BtTZjNMaSduzkFIh0mQaZ5Uh07lEKHHk8n1aEOvXetLcQgQYhova0hRreuZkFinqbR1dG19WmeDUFYxhgAnnJGgNE7EbuZMA0dpfXIAmC1rK7Kwo4sIr21rhpjImJEc0R3n1N0xyDpZbmM1nprwNxK660iSZwmZkZzQASznIMO3epABCJDVd4v5KOe7x4kJB2j1iIhhhR2EFgMCQm1qwHFKDA05ohErdSYplI3HX06zODoYFHkdlu7obuBdkIBdCTU1rbb7fOH07/w06++fqnffHy5Xa/buo7RT+fzbqA5HI7H87lt5Xa7fv3LXyJhXcvoW12uMU/5cESEtm0xhfv7u+Px4EitdrMxzwcOYbTWyla3Zb0tRACurTQJMp+P03yXD6dpmtR0CnJ3OufDkZjX1kfvaO6jELETobmpqVk37737aOc55+PZ3NQBhbEW1/a0tN77erlMp4Mj22huFkOSIPNhJnR0F3DcnqtCGVDLpm2DUSODDntZ1mVZYwwIkIPsoW4YbS1rFjrM05vPPjvmsNxuP/ny7euvfnjZ+m1ZzCCGsK7Ly7a1bmRjXa7qCMigGqCnaerq+N/+d/77ALDPNYiJiEkE9iQvgITALCKhlm20FkNgEUA3cyKOUx61obCElFN+89lrBhvdjMVY3r55vZVi7rEsmVGmg82nAcTCiPh6zudIH5dSuxKSxKjMBK7qwyyIzCkw09rUxjgzAmEDilGYqA3lEFobA9zUhukYOvqovV8vL08fHrfLy+3x/Rijbmtv22iViNUs5sOXP/udbblNU57Pd3vdOqfsgHGaDsfD6e4eiVOMn04UIqWUWlvOmRBnts+izeP6q1/+2a+/f1xaB8TRxxij9ZZz6mq9jWmeQ57G6NPhcLtt8xQlzTshL89T2VaBMc1nB/ehW1mFxUYvZTMHZl6ul8t16+o6Btp4frki0bpVDJkkxiiAnNI0HY75eCAW7d3cXQeFGIQBcD4eOQTVDgDWjYOk6bBt6+l83quajt5KFwlMhELMAfaQRyl5nmKMvTZkJkIJaeioWzvMU0zBYUewERKB+yiVYxhjsLCI9FrdIcTYeyvLmo+HFELZtlIWAoopIRGBg7sZphQkBWEpWymlSJTeems6pSigBr77PftQQCLiNCVT1d6RKOa8y1oCkcRcW1PT0Xobo9R6eX4GszylvlwJnUJGN9D2+ssf3G6Fp0OvdYpyfzqnw2QOrmP0ziGM3pAIzIkw5hSYkdjN1q2WWnPKOUckAoMphqa91WqAqjrH+HA8mNulja20Vup0mMt6Q4QxtK6lbuubu8PPfvSVsvzm/dOHDx/HGILkBMwShdV8mucQYx+93G6j9ZfLtWxLTEmY19vtOKc3n7/dH3Tt4/r0IR8P5fISQkCJ+XQk4l4Kovc+WtOUJOY5TTO42bCHzz4XkfV2jXlKMfgYe/zl5fHDNCVAIo5tNHA3tT0nCGDIgjtDcdR1XdWst6a9997n0z0F0VoBPKZISDGIjs7gPvry+O3xdJdP973Xl48fnt6/L9tatqW0BvBJer9tawxxnrJE0d4QcJ6Ph+NBvPfej2n6yQ+/nM8P8xS//OK1EKrqspWldCF8fHq5LlvtXVt7fnoeDpJn/O/8u3/PzdwcPvH/6JMUNggzIZCpjdY5sLkK8Z4731dRpmZDHYBjSvPRVEercZpTnuI8789tTMnAg8hhng/zdD+lKadA+DrxjOPrS/nYXFnUPKe46+y6GSBNKZJZM8s570XlFORuzsL8stWylX0gNVSnnC+3dVtXYRyqpRQiBLUQ47ZurW4fP7xHgJTT519+9er1a5IwHw7uNsY4zJPp/oFBBwfznGJOQdVKrTrU++CcrPcvzimv77775ldff/f++XZTwNZ73aowE6OOwSEaESGd7u45RNOR5nld6nyYAIEAOIYQUt22IEQSltvym1/86s0PfyiAXXUrTVI+nO8+fHi/3FZ0zMcTMYWYJMieHQt5alvjGBCRGWtpo/eYc8qJiNxcmImpbGWakpCDw+W2iHBK2dV662Vb0zwjSa/tMM8xpz0GtVNqTdVVR68ppxAnc5sOh7JsihiEp5R67ynlXeRkZrfrVWLc3y+19ZhSCLItC3JAUCYKMW6l9NYQoJbibttyzTmTROvN9VMeGIny4TjG+PKLt0TYW69bQSYwu12v+Xi20d+8/Xy5XSUIuiMJc3AwIVcDc9/Wiky7+H1Z1u368vLh23e/+nPt/c0XX51fPQACmqtpyrOkacrpzdu3IGnnu7RapmlG8Mv1KhIYydyJIOcJEYHotqyt9ciERAY45yRChAhm5LYUnVLspkgUoyxLqbXMh7nV6oYppx1ru16fqZbzYZ5ev75t7eVyuXz8uJf/1mUJMc6nO3ADHXGa0b3WKiHEnATxMGdXBZbeu5ndXl7GvnJ+ekICEZqORwBiifP5DMCEULeVkGKekDildDxMpTYkKssKAIfDFFM0UwcSCUPH6D0GHn0M9dHr/v0E7sK43G47knsMHbWMrrjrJYchOqCDGhE5QLtd+2huY71cEMHNW7m5DtXRax2973143evoZjGnnNP5eDyc70+nkwGN0cFNJBIAEZ3Ox/vj/DrzMWEMIc6TY1iut9Hr09oQaStb6bo1xf/B//TfdwA33auREiIiGgAgutro3U1H7xKDm9Kn3A2A2W58ZpE0HWOMDmDmgEAsIU+n+3sknFLelZp5nqaUTklOTIdA4nZt9rjVzaDUzqBTysfToam1MRDIiXSMu+Ok5lNM4G5g6uAGOYiZXrfCIrV1YkoxCWOe8raW2oewSCA3ZyYCYsJhhjpsWwn8fDqmlJtbJHIbiuG6VSb87DznIOwqLEQwWhf6xNYBwo8fv3t+95tf/urX7z48GSIQbbX1Nphpnqda1iAhzAdiIRscUzqcEUBSGmbTPJmZ9gGAvXVgDjF98/V3777/AAY/+t3fEQl1q2k+5PNJQnSw0bv17qY7MR0IdfSybSGku1evr5erAzCLRDHVkPN+wQPQvm0f379DotP51NdFTdetTtMUUvz+V79hYUO6f/OWkO9eP8yH4+3lEqfISL12YWy9LpfrGP3+/r6PkaZJCF1trc1HyznFfEAEYZkOR5HwqZzfh6qt6xpTZtlZ/AHcAOG2LNtWx+itlnmegWi93qYopRR1Wy7P5Jjn+XQ83b+6d22jtZBybc3MAdGBR2uvv/zyw3ffCFNIh5hSCDxaR+JpngHcxliXZYyxK2zLVoaN5eVle3myPUnv3sqGbu59tPHx++/efvXVFz/84Rc/+pli2F+Ot8v1cJh2gNLuX/vw8elwnHPKxBKFEZGEfYzedauVkIgJkD67P/70Ln681da7pHzd2tbH1nXbNmZm5hQEcV8pxFa7mpatCHOKEgi30ilK3crttiDhfJgRERHNfQ+DmDkFAffR+uF0bKWyMBBJCHUrZSvEtLw8LZeXsq5vvvpBTJOODgaIzsxmyszHu7vR+vl03Kmu13Xtrbnpbr0CACQZvS8vTzHKdDhO85FFeq/gxiza1lpKqRUcOcZ1ucWURh/uDubr7SpRtA/rVUJGtF5rK+Xl8V1ZrhIjgIMZIpmp9tpqAQDTrsMQIU/T8Xh888UXd3cPDsAiPlTHcDDtnRBCCI6EplPASdi1FwUkmQKlKZXSgmAO9O5llek4qzoCxJzANeWJSdRUh/b+aZjBx8O61eFubswiIaqNEEJMKeWU5tMhZ05BJBB460MBQ4z0ieAFKSUCD20TkHzI92TXZs/Nngc44BDaqi2lPfchhGxOzByjjrFtTc36LuzCT2R608FMam5D9y/JOnQt4+W2ImBO0d1GU0BQhSjSat+28v79+zEGE+fDVRBbbXfHww9O6c3dec60dt2WZSDMQu+3/uFy3eqgVn77s+P9BN+/++6P/uwXj5db67pt204dQZJpTkioDkSc5oljdNVhvly3N/OJgyzrcnu+xHk2A1UrdaTjWQRpOB1e/fAvfBVTGr2DyJQPMU8hR3ZfLrfb86P2Rnsf3FBS3nVxrdVWq7ZNHeLxLEQhTzFGBG9l+fDrP3/++Hi9XOfjydprZL5dru5+fX6OKV6XWwgxpLSt6/39GV2fnp7Qxu37j8fzvY/eEXaoWUxp24r6XhLgVuvHdx/ADdDMEZBSzMf7e+F4//CQU3ThXYa0LSu5mQ0A66XEFLMwRC4+5DAh+Oh9Psy9bGYOSA+v37794otAlKdsptr7U/lwe3zctlJru3v92Xx37waPHx6BI+es7kPdfLQ+JGA0Y+Y4HxRwe/8+pTzUDalsdZhxyuzWer9dXhjMVEX4+vyc5kM6vQqHM0lY18bCrXZzc/Sh3loP0edpevPmzV7OV1VzDRx6bYc554lSDbUNAEf33vqfPOq2FdI+43L/cI4Ax/mgp8N1rdfbzc1yjsKUhc+BTjk0u9uuy9bViU732d2nPJ3Od0PNXSUwANRSkVDV63LTtYnEmNMwN4DI0segiPM0MRGKxJTu3r7VNspWEHE+n0OM5FCXVVJEQpFwmKedX2ajn+fE5wOD7+3d4d571xwCU+t9AJRSYoxExAznw9S7rMxq5s7qgChlrWnK7mA6OEZTRSJzdKbWuvaBwpKnYNrKlmJK84GEwWGaEzGLMLmN0UOIMUZiYZFtuanqNM2wh9qYTbWua11uAAjgNyJH8N53NrKpMgEhvjqfjodYtor//v/yfzXU1IyJSEg4hPBpWkSMvfa3x+m3PrsrY/zJtx+vpXXHGAUATg/3vXRTi/MchM9TJgBkvF5uHx6fdHQw3ZFhZkaEIchXb17/K3/hJz8+xa/X8Q+/vbxsLeWUc4pMBrjW1kcHg3w8aOuttxjSIccUwxiac0KH2puOAQClDhISYQ6x9N67igi7rrdbbzVKyPNctnUrpbVubimmlBMhQgiXl8uuyIHez4zH43xtOkjcjBC3Us1ttP7qIG9D/f5XP3/38Xk4IhES1tYBPrW4Y0oxpzEgMsQpj50EudXSATkASxs2FA8Pr9I0p/m4lXo4n9wciWLMIoIErVVGjimA+3a7aW+1bd//8pduTTjEwxEQGRkYAdDGYKH18pKm49svf3h+9ar3joTkHoJ8++s//+7Xv3z58Dif73/4s5+M1p4eH0Ftu73Md/dmhMLa+8ObN3f35zzPvQO41m2xVlKeJOWQJiDS1sZo2huCmWqptWwN3LflwiwhT4fDEZDy8Q5M94nq6XxiFpLgo67XCxNenz+OPiSkPM/DTZgRRXIKIe4eo3k+xJz3lffxMJuO3m2ff66luDpLkBCEaVmLmYYQibD3gQCqQ6IQ7mQWHEN7a09Pz8e7U91qK+Xpw/tyu86nQ0pxvS2jFjCLOccYPv/qq7df/nDoEKZSewgRwBAgpaSqIUR3c1MJQYdv29Z6Z6Ep51Z6zmGaJus9Cwv5F6f5bk5LqX/+/kVHy9aX4XMMd4fJOZxOp63UbYzlekspzgGPOTHLrer9ebZa1q4VUA2eb9s29DBNy7oCYggREIRFVZuOVpuZ1doICcHzNHUd3ruEAIj7S2foYOYxBiKaGxEx4Bgd/JPQOMRoOgJhShlc1ZwBgqACmNPOROjdDNB06Oh7bjZHDqBN3d37UEXqXYHYVc1cRxu9A4KqrrcLAQLRKJvVOuoS0F/fH1OK8XRvJMNBh45WGKnU0ktptTi4j97W1QFClBBjynMQ6WVzorYVJgBVtUHM7m46Wql7lQXB3Zw+qYQQEYWYyXezhZO5grHoUF/XTUJgplvpZa2/8+X9KaefP9enMmoptbUP331AxMP57Kaj2zSn8yRbHx+W6/r0OEZ3tyRyOJ2JGJAoJgjxF19/X+boKb8+HSCktbfR+wR0SnGSvI1o7hyktHZ3PBLh6AMAeq232y1PUwy0q6623gjkJAxjkOrYlkvZrs8v5s6MhJSuqbcmU5acxGyY2bqlaUK1PJ+IyUdrY7wr7TdPL9qa6UBTYJEQzOwYaQ7+//ujP7kuG4rsEp29QrDn5neoppv12uNpXpZtua7d2eM0nV/lu1f5eAoxEYnpCNOkvatfY5oYkYPsRWJ1R4QQZC9ajt4/fPuthBDn0+PXv0bCI4oDmup0PDAHQi5rCdNdOh7j6VxqR7DAoW4LQZzyVLbt8vQUUrp8fCzLqmbXxw+t1jifJEotG7q76eXx0Xp3G61s5bZqr4DgQMe7++kwl62a2bZcY47LbZXpePfZ59u6Jg5IJERbs8PdsesAd9MBvU+HWfsAqjZq2VYmNGQjLzq8VUIiicfzPTGBQ5rznJKbg+oUY2DuXVVNe2dmByaU06vzLt2cp2meZza/bRsyxxBySjpGryUloTDVsjnY3etXrx7uS+0lVJun+4f72+V6ujsL2LJuU06HwyHmHERGH4CIpqO342GurQeRsq4xJEaSwLUokwhzDMTCALhuZRikKY3eWq0++olDfbn+0fv3o3dCv6xrG87CY38fIKScP3v98MOH88Px9MUxC9rH63ZZ25uzoI3H67pWfb7enJiYRm2Uorkd5qm1TogkTIDElFPm00lVS2sI2EdDR0RHJieqWxEmcARzR4whCJK5I4O2Mec8hq5bMYfhLQRZW7ttBdEJHFB24HiICd2CkAgtWx2tIUJZV0DUEQ6CQKSqTBAYWR29DZbAwCn2zo7QSjk+3AfExNZKbOZlPeTjcYyhxM9r723p26q7a7o1IhytmY3dUw6IQ4cVb7WrGhPbGCiCRMjIMZKpqwGAxBTiNLT31vY6jGpHG4fMTCy9d3RwHbviApBVkWJAdgNEpJeuf/ayKsKrSX50lPsstxYupTaFZq5DL5ebjtGX5X4KATFGef3l564+dtkncqu11wpb7Xh9T/yHZgT++vWrME0A4L2xhO/fvUdCihlDIofjPF2X1d0NYL1ct3UFgJRLDIROAF5rQYTvfvnSS9ltuEjIImme3IFiNGIQ6bWN1s39cDoKUC01HUMIAoTXlwVJeDqQORLVZUFiR3QgRj1S+fkf//y2dvU9U797i5D3VpqjAwDismxm9u13azfJ54fp/k0+nY+nu2GWUg4pqY4gwkKMEh7ukSgyOSIKO34CYMCeRXc73t9JkPl4RJGXn/x0u1wkxjzNbfTT6YzoAdFMh8E85W1by+gxhG25gOnysoxW6lrqVkzH+9/8Jk4HCrENvbxcpvMtzbpcXmyMPE1MxCIfv/0mH2aSADHv3/C363VrXfJsSDjfgQhP4fBwPxTSdCKKIQZ3ZKJXr+6fX17mKROACEsMaDgc3n//nrUc5nlZlsNhlpiJOM+ThBRS6r2Z+raWKSZ1IyIzm3LcFR6meS/Hn6ZJAYjZRIKwmhnuTSzxf/YnTjOBjd4B964WglmKMuUwWmfhz9686q3v4axAmFIcQ+vttqsOtTcJ4lBNjYkdSAJvW23XdbfALctqaohICLfL8144G7UCAAf5ZpizOFjftjTlXnpMSQ0opIBUR9/Uvr+UX7+/HE/H05xPgY9TagZ9afcCCQcyyKu7bfhSq4s4wlYrIju4uKNZ6V1i6LWniKoKZo54Ph4BfPTBIZj5mKY+Rvtn61cRUjcCIGCJcWK/v59LTWvrTRWBNpBB0npzZkAA96HetyJMvasQCCJFGb3FyKP3UgYI3x/nN4fJR2fEyzDRvo6ahUdpgfCUQ5qn43E+n+Y58vNl/fXz9uG6rtUUeu21lnJ7fh6tINIYnZkIBIjcWccA0xAjEiI6mLVWU55Qwi6e3rY6ehdhQhra97GVmgHsmQuWnNj7wyFEYfx7/97/TIjnKPenwyFFYykYnETdbdeamUWhY5CM9uU5Z+/T8fRStatelS6X5ft1mOptW9tQtJFSjCkBiuQM7gguzK8SModu/sNzfuy4tM45t+7qJq663CqJDo15enj94O7CbGZC1NVKqQ5GMWpXdUADQNxaXS9P16envm1mFuZDiCFIlCAOPs2H490DoINqmlII2cFbbbX1EIKNQcJlWXQoENXrjZmQKUZBJBv6JuvzL//o2+/eG2DtioghhBACMezAIjMn4TH05eWqitP96/svvkrzSVJydwlRDabjMU+Tth6CTPOBhN2dwAFw6DgeDvt8BBF7G2N0QLDWD+fT7vWSKL1bDKHUom6n+WCjr7fbti6H0xlGXZcbhURIpq1shYViSk+PT7/+0z/t2wboknKaD+vttjw/SQjT8XB7fCzb+tVPfpZSOt7fbds2nc4hRrBxe3pxs/n+Ph1O8/0DA40xRh/MNE3TsmwANk35eDylGBjpcDysy8IhbMtStnW5XA6n0/HuYVm3QL68PPbWgrBLPJ7vU0puBkCACABlq0Sf3LKAlMI+gIc85XUrajrFqGpIpKoEwIHdofceYjJTQLQ+HBzGwF3q4b6uFREQnQBYsBsyUuu9bCXlDIiBqbSKQOttWUt58/pBRHKeai3Wez7MqmbqT8/Px+MxJR6919qQSFtttfbWSCjEoF3NzdTjPIlIX4u5pZx2oQkyC5K5pTSRBAdANwAKUVpr2qqZEyK5IzmObjokZzNk4TE6U4h5BkYmcrPWG5gj+DTPDmAOZoaAxPjw8OAOQdgd3Gyo1la1KUcRIhtjEnyVOQmfM5sjhVhqXZoZYhQ0zu9eluGDdwOJG6HXbWtDAT6R8IgI0RNYZAL3smy3oevo+3ixrZuZSYxJ+HQ8ZKFTks/O0xTDfuH47vH63dOlqb7c1uePj701MOAYzToC9bIBuANGkRAlhOA6dPSYwul0Hr23WojY3a7PT7UUVeu9pRhiShRCijGEgEillO16Ve1uiv/Wf+vvkrAIn+bDj18df/zmzGkC4m8qPq6t10LIqgPAhDmDoo435zkxHwjevnk4JfnF8/pyXb5Z9VrVwFlYWE6vP8vzrNviNl4dD77eDlM29y8PoR3uv71286Gqw5BD6OZJJAQxQHUstfXWYpTjlFPO5rZtxcyCsAIKiwOs67Zt2+16aaUBeJznsqwIOB0P8/EwT/PhOO9x7RQE4dOpbThcLsv1ejPVEKTXSsJ5noUFAMpyQ3Crqzz/8k/+8R9sahxiSIkDu9nu2lIzUx3m21avt5Lm4+n1m3w4pflw/+azfDwuLxdHMbe3P/gqhMhELMzEgBCYVW3oIABmUXNEF5FS6164a7UiM6CDOSIQBcS9GUemvWyl9w5mEoL1hgjq6g5jaO9jmicwc4fa6vtvvmMhJwoSOMrH776/PT+1dXl59z2F+Lt/7a+X6wXM7t68ceL5dE8E6+0WRCTk6XicjydhPh6P4D7NOYUgIqM3ZkHEPgYCmvvL5TrPWYRbbZfLS0rp/uHV6L23vm1LW2/Xp8f71697bXdvPg85M9G6ljzloQoOxDTGEGYdOk+ZCUZTjpGZ3EZtPUhgkbKsHKWVJjGw8L4sf3l+maccouxig1I2YiGCsjUHa+s1xEwiRDwM3I2DCJLEoKOb2p7jC8Ipht0fCDrUnGJYbguzCOO2bhIDAlyen9RtdAVEbVVEJBAiuhMAtq2kOTFL3YqbS2RwI8fD+WTDOEVhQebRS69K5CxBVVspksJ2ve7WrhBTrw0ZkSiF6P/cYmcmTHW5MYOrskSOiUjy8SAxigQGHK0RYYhBJIQYWm3M7Dqo1/tDZvBD4kPOzTwFmkRYSETC8e7PfvHrd9el9waOrQ9i2LY2H2ZzV8dP+gozETHtxEFEzHVXNLVSu/aUMrq5qgTebVWjK5p9eT//3tsjsXz9dP1w2W5d1bFt204vbrVq32u1lvOUApsORDQzMNtpJaVs8GnNqqOst8v15XJRd0YKOc2HA+ggQALoZmXb2rrkHPFv/51/I6QIzAiEAJHpbs53OY40P9eRmD6/P70s21KaCIurcWC0U4qK/Pbu/MPXp1FWHePXFbpM6mimpyk3EtQ+C8nhEBF92Gdv7nm9PpwPNc7v1vHy8rLcbiwBQo4pCmLKaZi/XG592LLcck4pSmDRMcwgzNN+5DkfJgIYpl2BmQghT4mQ1q3syGdmarWJiI6urQORtf7w8BACMRK4pSBjqANGocttCTE31bW155frqCXjWB9/8/M/+uM+bD6dQopluYWYkOj2/ASITtwUJB3uP//i/OpNPszz4RDSJCKHwwwObYyhGjgwooO7W2tdh0oKbgrugQUIdRgzIpHt/XEidxtqQl5qB4AYEhEwA6iVsoGOkKfeh41elnU6HlurYwwCRGEdivSJTtP72K7X0VrZFnNgCciyXV6e379/88MfgFmrxVUPd/ecMiHGlELMOYXT3d1hPoSQEFGEVDXnxMS9dyaapryum4SIAGq2ax9jDPsLWoeCgzCVWoMw2Lg+PbZtU/eHt1/EPO26AlX9tGjiTy9xBJQoo/eylDjlaZ4QQLW7/XOCLoyhiD66qeo0ZUdUsyiChK310eqOmTLAbSvDBjiEGJmJkWopxCQsIkxotQ4Jcb/e+j5UBmegEEIfAxGFuffuiG5qQy+X56aqw7alAGOKUtcFiXr3NE+jFEQLIXHMRBCFW9nGGESc8iRBoGuc0vF86upDlRG22225XhwxpcRx0t4cQJh7KxKklVq3lZhabTFnGCpJ6m2p65XBQ86qOs0zc3AiiWm0YWoShJmned6nyaec7ucohN0REA+BMuKbc54DO2HZqgJ0ta2M7y/L4219enxC1/nuLs+H03EWAlMb7ju2RM2ImRCnaW6ttK3simkH7/tSJSZCMB1A5IDkOoO9Pk4Ph/j1y3qpw4DLtpkrIbZP87KORKfTobVu2vfQCSCMVohExzAdptp7RbcYAiC8PL8Acc4Tu4r3LESI161yCHVd0B3/9b/zt3dxPCAikCNJTIgw5zyfzj94e//Vw+HxZX2qUMxba4iMLDHI8e6cUnxI8tOH+RC4q31Y2/frWBVBmxOvt9thyiTct1LVmHly/fLVnebDzamsqzrkKUvK2jozno7HKMHMulnpbVkKCbFE3VZkmY6HEAITImJikhjcXIjH6H2oyCecHiL01tW1dZUQEpObE+EhxXMkULsLfphyzOm5+lPpz5eFmDHIWpq57U7J223Z1lvKOabUau21AAsitnVVIMk55WmOAgiq2M0A3c3IMQQJwgMxIJmaI4w+JISdv3673hwhMGfh0up+V93TrZJmJETwPhTBXp7XPE+H40RIaJ0JL9drb+14OpVSR+/bsuR56rUAEAIOHSjcSo1TbusmMTLz5cOH5XLdtjXmNJ/uyvUCSOlwqMs1xBRzBoB4PMLQ4/H49osvDodD3TZwnw4TIl0v15QSIRKTAwjinhIABEZy99pqiJEQWMIYAwG0d0BMSepW123TXsv15XA8pMPRDJatBLBtOOy5OZL5MAOiDp0Pc9k7AyIUBA16ayknM2OEIMEAai3f/fnXJPTw2av5eGQJ2jshDVPGnbmHbioSDNHGADMMTEhg5u4ijMSI0PvYxy69D2R0sxiklp6miZm2dUspuiuAj9odiaMQYCsFERTwcJhv15u7o4R1WUcfpuN8f+dmjAhuo/d5ntS8toYso/WYou+08k+tC2tbHe4hhhQzII5W9+iVCJFwbd1URy3r9QUAidhsILi2ggRMJBIJqfdGLAhExPPdmZDcLOY0z3NCz0nmGG4grSsDBPQvTvlhisfMl+vSDdxMguTA3z69fP20Pt82J56n2XSkKDlIjBKDtNbb0OEISKYWyE1HVyUi0N5qzYEBSJGSBButj3E4TMtWehuTj3g83JYSYgACQgLEXmsQ7r2NobUUU1XthDRGB1MbLc9HN+u1IqOZEdI8T1HITRnxPIdR2sttaX2Y47puWy2td3DHf/Vf+1fdXYcSMQnHPLOIiMSUQkzn4xyCGDIQU8ruYEjah8S4nwbJ4Rz5p589/PUfvw5kf/yx/slLK20AQGnN3FtvzCGmuBcXdlft+eFeRBwBHXaoFRHt35tBJOWkDqX2oaru3RSAhCjIfopxRhDiFGKMclu3EEJg3rMaCm4GDkYkOSXXEYRH76A6sz89fvReBpAQNeR0Pt+d7ziGdaul97qVvtbjq3OpwxFyjIzQex9uw2B/cRgRDCXy9XK5f3ioXQ0Q3AAMHCWI6wgxn8+nbauA+18G7c0Ay1ZSSmZmOmKObgC7XoAwxChBVHcotiJRDHF/uwmiqbWyQZBeeopBwUYftaylVjePMbe2icTWGoC7WpA0HQ8SiHwneXueD6OW1sc0zXnKn/RyDr3WmFIIQdVSjKWUnJOpAuEutdvWgkSIHnauJBAyAfjhMBNAKY0/ef+cmeq2vby8MFrM+Xq59tFarYHQzWNKl2Vbb9dh6L2ElLfa5ynPp8P1cvvqBz/Yf/AcQk6xbGXvM7hpr+X+9euyVUSIh0NI2YaO2vI8jTEIYajllLT3/VeRghiAINVWACgEbq2LMCCp2pSzu5VSiVlE+hh1KwlBEZFlnqcxxrZtU44G4I5lq0yQpuzuvEdY910Hi4MT834uHqq9td46k7cxjofDYcqA2FrvrQWRwDRKUaTbWpTDsixBJB8PQfjy9JJzIiJV3R3mo4+yrr3X0UevxVvdBTYMYNpziCFnBDycjgowTTMiTFPKgArIRIyovVV1Q3SiPZuZRILbj18dXk30dN2+u2596HUpp4inKYUQIYRraQNlLR2ZUyAwhTE4cFfftsZC4E4ESLxtq6lZH46ehf/Cl3cphN+81AnGdSuH4yEgbG3kKFsfy1azoKOEENX1+flSSuljrOu63Za63UwVEW20HY09HybigIAhBGKKKeYUaIwU6NUh/fCz8/NSv/5wfXdZWcRUS9n6MHTbZWKOjBITh4Ash9MJzHfxyTZg1cEBhCCIs4jWbjqM2QYRIgcZIXy79f/vN5fffTMfp/hQ/Yq09MFEwng4Hg7T1EppYwBJKYVZ1m1DpJynEEPf24ytPjzczzmbe60tiLDrthVgijG0rgNcGzCRDVM3ME9phMCmXmsPQXZVoupw932MUstWtu3p6enl+Xno6Mtt1MY5U2B0CDEFfvfm7dvD+QSA6jgMl9Z5q8ISopRlvdyuSDTN2Q1qqcwhH6Zd93Z8/RkzTYIYWFtVcw7JeocQc8qAxPu9z2GfozJSF2m1MnOI6XCYW21uThPXdUNwBAwcYgiqhky99k++mP9/U3/WY1m2pulCXzfGmM1ay8ybaPbemVk9VUVJdQniXHODhGiEkOh0jgrED+QHcIPEDUdAQXV5yFNZmbn3jtbdzWw1c47ma7iYvktHCskj5KGQeZitOcf4vvd9HkAkWp6emLBSRQK0KLmsy6zuo5sIj7HkaYoAHypZAHC7XRlxXlcKZ+HRx+n53d4GMydiRKh1/9o5Y6w6nNK88BhqalPJAJBSMvRpPlx/qbev+clcSqtdhwpjzukgRCamodp6c7fr9ZXc7OuSBp0EKD59/jyvy/L8/Pbycr/f09DR+8tPf1zfPalGvb1NyxJITIR4nEWY901SYkl/+MPv69bSNH/4oFPZw9wCJSdhdlNXV9ZABIQylbZXHX2eJwskgmFuHgKYcsExau+MpObgbmZTyeV8en15zVMRYUQws5RTSgUgDvAUwldCsRKbqRBat0iBQF8RGnHMlVIg9bq32lnSsJhzIsI8T2Pb35/O3344TUKPbn+87vepBAQgsPD0/omJgVDHUDVBGq4bQ2sCAL0WtzEt8+l0IoBAXNc1J6l7m5ZpjDGGah/X19tRpjsv0/NpCnGZlmC5bvVaW+/jUvg8lR9e7m8bLwIBeO92tfj80vzztp6Wb9YpoV9O+bvTad/2X+8P4xRmYABu7hbdAJFFlpLcUqsdUybBrv5XP76ccnrZR05cGB1wFSxq4XHvdsRGdVQgdLPa9+vrVT0CY7hdrzfTwcxhZtqJeKillFgkHo+67wAeNhjw6d3z9vHD672WROvEeZOIyCW7KYQCslw+fgMBnNLxamWRXGYzA2JmJkkiUtbZh5GIOwDisa3rvR3OApT8pW8vt+2nx+nDeX1f6HmZXwZU89rH41F/ffslCVPOBLae1inlVHLK2dwDYGAARs4JEdVtytnNa2sl53kFABSRzCqSEPFQnvTWPWKeMwLEV2ot3GtNnCRxqEqS4f52u/34d7+/Xa8B4WZMDCmX9TT6IGZImVLK6+IIY6+OMJ/OnJ/fXq5zkjlytQEpj2E8jBHX9TSGHpocZAqEt+ttLjO7t9pP5xOl9Lbt5U/vWPMIh3mZAAAIiDnlGIThwIRt2wA5JdHwnBNFJAxiMNXH/YYePE9LOQEkZtExInyZV3K/Ph7mCASZWNXWtYw+5mlGogPXs7cqklMqeZoee51zSSTLkoYaMQUEiQijhSVJvbdR9wCSed62SkQs0twEBdXCfSqTgf/y0+dpSpfLfIhRoMRe98fWp5xymcBs7623HY5k//nUt+3TL78+ffgYAa8vb6d1YaLr9Tafn0Qyp1z3drimRhtSpn3bhnaWXLfHtCznD9+hyP3xsPEG7tZ6WpaEuLf+8nJ19/fffCTmo8s5l3z026ecj6uAA3p8bRaLiEgKj/vtfj6fo3eHkCTHXMwDS+ZpmQFpjHE4SRnJXJkFMdaliMhoo/Vq7toH5WTu7lBK6W2MMUrOAYoA4ZbnOYi62nFSYyLO2QE/bePlvstQIORpWs+TEJuqAxCmxNJafXo6gYe66dDLaYVwFEYggFAdGFBy3ltzt3DnLC9fXrf7fV7Wp/fPJPzydmUYnPgbms+X089froD0m3MRKDfit0cTomD54d7XjEQcyHMp67rc3m6tjddUiOjL59v7tXy3cMxyB+kKYwxiLvPa7w8WatqvrxUQ3a3u+6GGunpcqyE4MpZcfnndXsNJcK/bgK8Sw8de1aPVnREC0LWrOTiUeXUbwtJaS6Uc9BQiMh1brUPV3ELHPJVgeav+5fHA+CpCyyJuysy5oAjL6XKZ52Wa5sDYHxsScylI0lt1d2TGQ2GZ2NwQ6ZgySk6jj2meJcnoo7cqKe1DP1+vNxFMmXJ2i0xI64Tn1dzbUIpYTysBmlvbKxAicWGZS9HeBYE83J0QHcjM748tAnwYgH/77cdCuO0tizytEws7kJoHwFDbe9PaOykwzikJWdt2Ifzm24+lCATsrTtgzpMkfozb04cPp/MpZ/YIUN/6SMQ+1MPOSwZ3dUgpEVqkTATruoRDniynxEdRIyI9PxOzjZ5Lmefp0RoSEAARDrWUk4IDopvmzHttWRIT1j7MI8KW0+xugihJ9sdmOgK+Gmjqvk/hYFGW2cZoOnIut7e30ToCEDpqKIeO4aM70jRzuKdSPBw5eQCRzPNECCLy2HeAmKdFCFLKw0bdOxHtWlNKeVqXdY6Ix2OHMFeHlM1DR48IdZ+mMs8ZDxzosdhyA0Biud633MfptLgPG13dDMDVyrp++E12dXdtvUf4+en8+sNPo48+bN+ruTFJXs4sjObTenKPIwaxPWrbB4SrmuvI03L58GG9nJ+fn5fTGZAO+XGtjZPIlLs5jWF9mNmB6ycEizjC4McTjRJJkqEjIHLOEHGgutXMAyUlhyjTSkS11t7HxKX3nnPWMYQ4lwwExzhvmOWUR+8WruoirKaI5BGHyi/nst3vI5znhVJiomMzeHs0AEci0W2aMgaaux3Dp6GppNWjHBwuYdWx17rMM4sAQO9DiIaNMHPzZr2OUbctTWU+rYeRoLaOqup23/bfffP+dL78ctv++q9/YcTTeZ3nqQYViieOXocKA4SEZ8nT01oSh+Rtr4706dHuQ2aCPmo1M9NSplCLsPHYXCSlDMimIyfhPJecCXyM4W7usW9tJnt/mrmUR6IxBhDsA22ea++90TC93+5hOk1ThAIYRIArhYXp/TpE2IYhgbmnlC7z+bQu6zqfTqcxdIxhZmYKhrWrqrEQMweASMrPy5QYSQp5mNo6T4acCPsYLBkYEEE4hTsxmx3inxR+nOsBEXIpiHi/P27aex8okuel1z5N+XQ+lWnea+1DS8n7/ZZyCYgxTFJOiSRsYfaJo3UI7/uwgIB4tBEe6zylhEvJM8UMvdW7SLpQgqrn8yyn5THgFfwiuZwyQXyq2hFbrdu2tdbMNOVCAHk5yTwLUe8dPUpmM923Zqpq7g49cX8Zrh0Be+/5fE4pa+tlWcBxe2zhLqVcH9euyolyKhSopto7At0fW2vdh259uKkDBqCO8dMff56WKU9luz9ykvV8tjGW02kqMwLUfd9uNx217TunBMxZUgTknMPj9fXlpGOYUcqjDxyt9+Fu99e3AJBpOl3OAQgBOnp73O7X15ym+fJMhNMy2+hTFpakww5mCRF/ZeeXMvpATsxiYbW2cE+J96o2NHPSXi0gkBCh7TsLlZIBcWjsbXMbyzyvSwaAoXpwU0pOFAnIzNTciXj46K1TLvd97/riAeBY5uXy7oOOMc3z+enpfFqTCDI9blvYGKqItK6LMA81wiCWMVSEEWkqWc0OxKt57HvLWcpUMICmrxBwFl7KbHAsKOCYCyLicRdWs2PmlSUBITH33s39iOzP8ywsVJCI1UP9cMi4qULAsUCIAD4i6RAiTEyjD0npQPgGRK1tnuc+VIRdlafMJK03kgQY4La3se17KeUokEou5rFvdd/ru3cXMGUCJCRJW+3zQtq7mSJla+PzTz9zzu8+vkfhiFiWGQkOYH9OqakBEEr58hj3/e3y/hkA+xhBHDoS+Qz+L/7hN2+P+m/++OXL222/3xHgdFq+Oc9lQSE0i0vB22P7hIzgo7aIMIs8ld7t+npnkZzSej5FQBFJ3qaI371b17LUkPt9ezplGJ1NT6dET/nx2Bzirz9vd/ecJJZJzYmo9zr2hpROlycfo96umShkyiw6epAzQ855mZfz+XQ6rc+n8u2S98djV2JJ6GMYdI2ttm6+Nd1rFfaYwJZE17qX8HlJTwsrppewUI0wwtSbQvQk4oHuVqaCECJf8SNH16S2Hm697trG8vwMJPO6DB2/fvpCiNqqqkaEq07LvK6naZpO335o2/2+Pe6S7tvuAWmeDuUSQ7iDkVwf29M6G3DtOk20nM+z0Bo1oP3ZNE8LfNmVtV+WaRV428atBwAE4XR89kbHiAM0zsRfPn3po499f71fARB7T1Mp65pznuZZSExz6y2GHr3ZMk/m9th2M5eUYGtj9ESUo/T6cDck8mGIse9CTAg4VHWrESYpmweIBAuyTMu8zBNzMh19v7cdHrf7H//Tf5yX+dtvP3773TfD7dPPvzSL+XzudbMASrn1DoTQ6+i9PW4sycL++Df/7VC9vP/w+UccY5RpwZQfr59ef/rx/O7D+v4dc5qXlfP029/9biqeixCyEDmENys5+7AkSd3XZWmtPmo11Ry0rGuoeoSU5BbDw3TcbzeZ5+vbaynT+ekJgIbh0DHGSFmQ8OXldcoSGlLKOvFQteEzAj1fejdEvN3uCC6SkHmapnVdAYKIh6qrJmFiSc9JVXNOKUlCRITH3lkkXHVYyqkfbTYz32sSKUnGUECq2w6AeSp2xPRF1PwY7UfE4QPDCABURIg40hhjdGRC4iPFIszAoqrH6zkAcko2BgIO8yQcfzp2mdno3SGO3OVQTUkkAgCEBRDWldCdMJhoqIabB5QkLDzUbm+7u085FSGyURhz5lLO21aZYAztrU64HNnsSACB4ZBTqn1cb/fT5UzCpRQmDote9asfG/k3335Qh73WAHia07kwhaLQazVhFuKh/uPjsU7lt5fln3/7HCz386n2ziKbG+s4Jb6rIlAS3NURwRDWzOc5tXDIPM1FciLiCcMLF28L6OfN/3qv//0/e//NyrwPVhvqP77c91+uj8deW6NSalcDRCIILzmDCXgqa9IwNyVO0caBxpzmQgjCMk1SUm616ei993ob14YvX673Pr77cP7+qeQyk9ApnQPor396/flK+L/9P/yXCWMq5d7VzC9T/nBe1rlsDr9srbdR5hWIh6kkkZRM7askm6X3fhAcc04B0LcN3HOelstpXpZlykL4eOyf3976Vol49JZLef7wnhlVbbtvL59f8umUp6LmKSdiqvedhUtK63nN6xkBs1CEz0TnxOphph8KMkIMVY8aQMvZIFob133svTngelqKJA0HQACotbEQEl2vt9FHBDiQlFKY3r9/nudiQK11BChZRMTMWx9gLiXttSFizvn4bKgZmBJGLtNQc4s2GgEmlvU09W5ER2eTeutlmlLiMLu/bZJpqL78/PPby5dwBSQb2vbHX/yjf3C5XLa3FxF5PLba6nK+jN63t+vzd99Pl6f79Z4Q1tPyuL6+fXkNob/9D3/Z635+focEJDKtT/PlYqp//a//n+d379599507LKen999/t8wzAK6XdyycCT0iALJIOATB4/EoKYmk4dZqXZd5qIWbjz7UT+fTNE17rUfWfNsrEyah3jqxpCStNRHiVPatQQwmIkkl595q3TdtfVkXydP7D++OnhAyjaGPxwMgeh+SSwRsj0cRWdcllwThbkbMrba5ZJJU21DVdZ5LSULk4VvvECBEZt7NWY4lCR7qY48Yveecp5wjwtyJKNzXZQEAj0D3gDjOaAjhgXutJNJqA4RlKnbcCiDAw9wlCSKVLAhgHm4GhxWevqrq4Ot/yHWYuZkFgU1Mz0sh190AWKzVIyK/18oR82n1PpZMp7kE8qPblNPe+tv1GmAdEkpy8zBrrZ3P62Vdr7crhENEd+CUAtDGADNDevvyMp+XWbhtG4nU3r2Pp2X+7t0JiLvHLy+3x17nks19AIYbu5GbEhOifo0ogmtPLGWZKacj5xEQTLTdrt5rD1T1MToxPWf+Z9+sv39te28zGnK61zanfC58njhMHyMqpYfhbk7EKQm4qWrbW29tPa2HhVqYI0LHsMPQ4+E61tNKiHXbzRQiGOHjzBNFHzpLoI/f//J6735KdFqX8zL9g4+zQPynX66/PoZMJWVGcMiEx9awWmxvN2E6pQIplSQ58a+v+37frSx9DDSdpslSVjVwR0pDDYmkzMJ4ADas18+3V7BxHJ7n0yqCBAIYL59+3bcHIj62zimLjrZpEqlvd2A2j9aiumq7l8ctPHofgVhKQveunpfTH5a5t9a2SokxYsr39bxOJacpDfC362OYJiJOkkuZS5my1FoDcZrXD9+sSRIgEB7+Q3TA2karfV4mC5xYksDeOwBttbn6PBXEYHApqVZvPaalEPjj8Xg6X5LwPE8B0EfLJbsqAgDilNNc0gFNvd/vdd9v9zuiQy6P143Amcs3f/H3p6n8/Iffp1wuy5rdP//68/XzFyccfXAu27Zvtdbr7en56fR0ppJR0rvf/tlPf/PXfZgkIeT9/jbGyPOiFq+fv0zn5zSvspyC0svL24cP73JiSdK2moSJqO3Hk51P63J9eywLah9msO9dhAAYKNVx9/vW1VxtnTIgFOFSsqqeTus0Tb3V7bpvt8F5Ws6XYXx9/UKMMUYgtm1HIg8T2R3scd9yLqfzaS6lJPn8+XNEpJSOdDER37c9HncGyKUEDPd4bLdpmVwHBlYMxAlLaa3tey8l195yyZPkgMACEGBjBLHrEElTmXpvKWc3O/663u/LsoD7GAPicIMmRsTwy2ltQ1tEQBDhruoRxKR9TDkfNPl9b0QYEK31ZZ7DRuYcgIAx5YQQ1tvELuQdHRCnJN/M8mfPpy/b2DUuH6Za6zZ0k6IG65Kny1xb89afLydCoPY4Z/715dO//09/ePfb3333zYfn56dwj3UKdwqdUrre7u/mlJgpSaudfazsspyjprE/pnmJlLvZqZRv3q3P67SNYHJ3/+05/ew9i5+W6W1ENWTC69ub1iYUBQFDxxjVvJkFk2+7mgPA7fVLmmbKxfo4kjcJJKX0NNHf+80Hl7u5vV/ku3en1320oU9zev/uaT0tfX/86//fD3/96b711g+7ZxJTPchVRHQ4NJMIEcKc+1bVrYjUx2O0FmamysJzTh9W+affn1hb7ZqZlgzfP5X/9OlB4F823163bd++v5T/3u+e+Mcv+L/5V/+nIlzkaHvCQXlW10T0/HQ+n1dB/P6URx+f67gP2PrYrlcphZezedDxwUUws2mesmCv++N23+93Ux1qpnqsR8GDhE3NejcbTCzreX56R24eBhGSJ0Bx8F4b+OAwQgwEZGZOpRRgRJlymeGwcuTkpm+fX95/+81pXcidEPI8D4e67cdw5DDXntc15cTTpMAQMZUy6p5L3tsIh3mZRZAAt1qF08f37/ZWP336nMokWfp+xP+mwtSHugWLMMO+7RpxWRdCyiVvWx3aT/MytAeS5IwRo7c6vNVa93q/vgHg+vzsgL3WMk8iwq7j7dP17frtb3+bc7l+/rTXer/f6lalFGJ8vL56hDsQBQKenp+e3n88v//wyx//+NiqpKnVrd1ep2V9//1v9ttNezu//1DmZT2dP3zzbW8tMxJimpcxRhKapzK6pSR77YTugEUEGbet1q0v67yuEyG13kQOMD9MOZkNV2NmdbDRhYmFmKl1+/nnXx/7rr2padtq2x7r03m/307nc5jWbVtOJwso87yez5fz8/m8/PjjL9vjvqwnycV1iEggTfMszGUqSYQAP7+89qG97giIhPNUSs5Pz8/7Xok5AlTHNBVAZDygvknVdAx3B0IESEnCQ82QiIgiAnSYWiAOs5JLyWI65jKpH1wFrq0NtQMkRETaKosEwOhjXZfemntI4tCvx8y9VfE4P51Ih4/2Yc0n4SD+w88v51OZwh4KNaI284iS+FSEp/nz632ecirl9z/9+mGdFcDGuF7vHaG7A8t5Ku+fni/r9K7wnHi4j66fH9u9uxqclvK8lG9PeUZX80PlXbuZx5fbvgFZa89FcsmIft37r7f26eWl7/XP//w3gfTl7X6Qkyl0ztR6vN32CHXwXvWb92cGvzdXs6PDlMuEzJISIVnbSxZ0U/UIIKtr4m/P8zpNTK69f/vxfL/X//iHn//t3/18fey919GHiABETgkAc86pZDc1tflg8xKFOxGv82SjT+vCRD66jZoRetvJVIce99PvLyXMfn3oy6aN8ONa3k34z3/7/h/+/Q9/9e//E/4v/vf/FREy0lTyt6cyJ3prfmujaQSAEOWpnEqaCDw8ABNT76MF1KBAYmE3Azcb2ltr+2Nsu4ePMQCQU0p/Akuaf+V899alJM6JJRExMyELS5ZpZpH9em2tAYT3KoxIDEgYkHKilFByWdZwz/OSStHW8lSScL1vOsbxxZhayllKTqWIJFV7/vA+iUC4pNyHIctU0rzML29XG7asy2mdrffW6zrN52X5+eW1tc5CFvjyel2XMpVyv90weDmvSEhhTGwA5FZr0wCIQEQGsLB9r8u8lHkKBHXQodr6MD0iYEcTgJnCFKzfX15SSSXlaZmHwjRPj+v1en0bY7z9+uuvP/wxTdNoVfuQkqd5Tin/5h/8/R//9u/my/Pp/XvrioTzaf3u+++XMksWJOqtbbfH0/OTSNLRdQwHPN4ofPwN8zh+HiGyiJn2rkQU4cSEAR6Qkrj60N5bJyLtbds3RlId5+cnN89Cp3UdZq+3+8vPP7x8/kzzab/dt9sLEwlRmJVpYmKDsNbm0/r88ePlcv786+fboz59+LCs59FauOZSJKdwZ+Kn5/frsiCRR4wxJKUDZcwk4YEUzFL3dkQUc8nh9vLTrwEwn9Y0FzcPwr7XnPO6TNM0mbuZ9d6stfa4y3Kqe0MihxAhjMhlmueJSVrvOTEGBgQz1r0O8zRlsgCicBOinJNZIOI8Z9T2yy+/PrZ91KqIwmkWWda59a6PWx2DyskZWXIuZZomCp8ZX673bfTn8xlF1CzUHreblGyqGB4RZV3nafpmnSd0GN1t/PLrp9v19vSb38yXJ6EE7r+Z+bROu8aJgcAd8Lws85J673fF11vNDBH+tJSfXre/+uXtjz9/6abn04IBABDu81ISRCLsQ4d2KbMNfb/m9+u0abw8Gnh07YiYy0QRAT7GSKmYjq3tERAIrsY23s35n337dB86z+Wc8S//8Mu/++sfrtvmcaCC99GauwMEIumRPVcjQgwXSZxkXZZ5WYT5dFpLKWGjPm73x+NRe92aOwSYUCRiQiRAJ+Qk79eylvTteT4n3ari/+q//FcRAAEBccny/WVOwiLyMHjtdn9Udw+IP1GePQsTsrkHImeJwHCz0U1NTX2oH7RikQCUlEhYWw0z1QEBGIAieZkRkFmAUHJBQHcr8xpubd81DtqhSxKWbB5ySBAJ1/OZmAHR7dDfSi65t25jMECe8qO211+/LOuyrKvqCID5dF7PJ0Rs98fzhw8pJwIIomEQocKIgL279nY6zc/ns2qvXT3C3W+Ph2pIotv1pqMTgKSUS04sgJynabS91TatK1hs95u7xTEGTolYiGlopJJMR5iV9QKAEE4EjKB9jFFNVYiXZX3++PGwmfz4ww+np+d3755bqwcO+/PPP+3btpzO+2PTXgnxh//0N7/9h/8oz+Xdx4/f/fZ3ANC7Csn5sqpqbw2QGCCXdLgnzDylBADbVnPOKZGpY5LWB4YHwFfAE8Ch1+xqxyi99qZmYV7rTkxg7u5PT+cANPfMvMyTDjWtf/Xv/90Pv/9BzdOcttcXQmRJR9kzL7PWaqrLaV7W80+//wOX6XR+fv7uu/X81Ot+e/k1WsvzFMQfvvleynQ6n5/OlwAgotY7Iq1zcbM+NOc0hiLg0OER5q571WHLukynSdWQjtZ6chtMHBGt94PnnlJKTODexwCiY6aWmSUlNT9Cv72PnCX8qIOSe7S9onAWJvQkEh7ddL9vU4x2u/4//s1fpmlezycklJyZEEwPJs1hIyVTjJBpplT2x+NxfcuZhIgkpVJ0jLZXZjSNcjrnlFJKkghUH29v+7bVx73u99vtJsLLsggzUpKc3r97F2MgRE5CzCTlssznKX/7/vm0Lh8uMwltj3ZZ5p9v9YeXt7/95dWIjFiHFmYpyTxGawQQpvM0bdv2zZq/fToxYa1VTQ149J6FtTbh+Ln6UJ+W2QLaUDMfY4y6W6uJGUUg4h9+WP7J95e//vHz3/z05e16//TYR61Hgt3Ujgtc3WvfH6Y95zLPCxIkSSI8TdMyldqaqplra0pw8LQ13BHBjyxpBNKB4AIATDkJckqC/+v/6l+5QwC4ewCsU0mI35wLBtwMb63X2sw93OHwZ7i7x0ElJyYdGuGAwSzEDBFqCgGc8+FAiQgwtyO7wBTmeZ7SNNuwlMvhepFcfAwkdFUdAw4991SEmIi5TAiBCJLSPC8RjozgqGachADUXXvXx4PA0rKqeS4TsFjXQCAmJnAPIDpdnhAQesOc6zCKECZOvMwrCk059b3utU7zVHKutW/7vreuavWx6Wiu3cZYL5fz5ZLnGSPaY2v7LlkcMMxYEqeUkrhb3yol4VLmZWn3DcKxzAQArqM1CJcs2hoiny+Xpw8fiAXdWq3/zb//D//oX/zzkpfR2zxP61Tw2CcARKC71dYtIjGF+/l8IiQEqK0ZQErZw4/H0GgNiYWl1g2QwH2eCwBGgJmLcCq51qY6WDhx+iqr+VP2qo+hZuqBTG5gQ0EIMSKCwLMkOhQ2wm5OYNfr9d/+f/7NT3/4PTPVx4MQiFhKmqa5bo/D8BYRo26//vHH3/zDfzyvp/OH9/N6KWX65Yffb18+panM5+dvfvPbaV4Q4Hw6pyRmjsQHUIQJ6dAhMocbBAaCuR/vGIRAcLe4PR6t9Wkqamatl7kgs3AaqjklgDiW70SYJOWU1A0jiNA9VANEwA3iq7kuMx5u55xZR48ANRg6Xl6v+75dlvJ47F9er3XfrO55zlPJRDSfLgQw1MKGPm5MAAT1sY3eI0xyDgtJKRzUFFnMY3l+9+7DR5YEphAGEPXx+PTjjyK43TcPe/n5Z21tOi3TPHNKkpN1VffT5ZJyBuBUprJe8jSv8/yupL848d/73bdA8rKNMcYsgAQ/3P33n17vW53W2cP5CKYGCLGOhoBgPie8zEnr7kgY9iFD68bz/Bhx26sCmgdINg8Pt956bR7BzOYDTb9Z8t//cP7t+yX69n/5v/+7P3y56fCqw/qI8HDvYxzfA2IsaSollZKzUCm57vvtdotwSTkCVPWwgrrZMSiICAAnxOMawURHe/TD0yruwcIRwSxj6FYbE85FLnMWGwmB16X3XvfqHlKyqekhOkIcw8M9IoDwoHRDBBIjEng4OAIiExAJZZZEIto7Ebk5Ens42MGKYORwG+5AqbAIp+QBHjhN5fR0EqQyTUkkl2LubYxt24moVgUMsK6jUxJ3qtuOzJ0GRwTF6IOMFABZCOl+vQOCjw6SUp44sYFPaeqjR/fb29jvj1yKI5kaES6nU1ngdrvbca9gyQstl1OapvVy7rX14TG0qR8UrpTF1a6fP5sbiQi4u1NEAJbT5Zgozes69h3B2GWon5/Pl6fLup5qrfdtb70u51NJEgAK8fnzS5unp3dP5l7vd3TjUnKe1MZ+f3TVrrYsE1iYBSfxoSRsgHXfiQgDzNWB1CICUH0q6aBZMHPv3d0QQEQIQ0dvHsLMCBox3G9vV84JkHpXkeTDgSAJbfc7IYCD5EyE7pBSulze/Q/+i//i85cvnz+/vHz+tN8fiPDxN9//vb/4iz/+3d91j7o9dOjLTz+ev/nN5bvfIpIqmkMdvrz7lsspzfP79+/B7PXzlyT8/PTUh0qSeSrHvUHNEgYmNtMpZ0SobWRmCL9e32ptYwxkftxuakpIpaRe69LmJFLKVMqEbsN876OPMU2lDqe9zXMRJkRaClsOB2hVWcQBHvvuRm3fKSXApHq4kWSayrunk7u9PeqyLH/27t31evv5D39Epj5MslAqNion6T5c0rZt9f6mrWFK8+lcnj+aAQshs7hPpRATIrqH1RYYDB5mhOjhj3tt+74/7siEScy81pYC1KLubX1+5jJZH4ER0SxuhkQiGvDLffvi8rv3J3T7cm9frvc/e7e8X+fpu6ffv+Wq3vae1iUL99bGaHzkkJO+m9Pf/3D+y7+rL00D6GVTsPE7yZe5uMPLY3d3wjHaQMKckqlyQIRhwLD4dbfHT29b7X/v4/I//Gd/8ecvj74//ps//Fp9fjyqmuaccy7ECOFTScs6C1NrfXs8tm0bY0TEGMYix/OLWI4+H7ibm0cIi0Asif7ifSk5pVyeLyv+7/6P/2dAOOpEqjb6QIjM/HRec5JqnkoJgFZ77x1TgojH9WamHgCHiyni+PVIanBK4R5mcNSXCX0oIlKS47JDyP9ZNoOELJlETC0iWHg9nySlsMAk6zIJ8QHwS5KYeV1nYmFJe91rG19er6aqddPeiCjlaVrmQDyCQqZm7oAgxJwTEiMyYoy9Ls9P87LUx8OGni5PcbQAkQCxzCXMrDdhOawKvbWvI5ucTLXf7wCAzJRE9biKwKi1Xa+n8zr6uF2vJHJIA48bcV7PZTnttzcIZ0ltezBDkkQi01QmltO7d3vTPrSPkVN693RBpjH69ctbLuXp/RMDbrcr+eBcWGTfa0jampra99++77UxiYZzxLrOqmYRCNhaz1OBCEJwJAJMWZAozCJi36sIpyQ2zHQEogFe5sKI98djqD62HXNqdbTWhRAPHkVvow9AmOc5lwmFVb3k6bzO6zwxk6ptrW3bph4lp9M8B1EEbvdb2x+t62OvPnpZ5pLn0/nsAClJzomQvPeXl1divFxOp3X1ADiU7MwIoKqEmFNy95xkDLUI8BDCX19eau/7Y4/j/UFoqhghOeWSRhsYkEte5uW7bz5Kks+v92GmOlR1Om4AAERMjImptRHhp3Uy865a9713PaRHrQ1KlAmnuXSN2kZvlQACqfVe9z0RqCpJwvBUplY3iAiPMRozu0cqebm8c3XwkUshFnTIUwGAum9hLlNG1/G49fp43O+ff/7JTW/XKwRISqmUMpVU5jStX37+ZZ7yfDrF4RAVQZLnjx9FUsq5rDO4F4To7fX19ng8SpYiMpVsKM6cJL/7+A51gDZEFEJzZ5aL4D/53TeP2/3T/fHusj6v5T/84eXxuJ+WKUhue21dnXmY9XocVJUQjk3Rkct3d4JIhIIxM/6DD0uY7kNfBm3DbXRAZOZtf2jXl0+f+lDH2B/bfr+bu6pKEjy4IgjTNE/TNHofrfUxAFGYvznnf/Hnz//yz9798W381ZcuxBKqQHS4p5gZCyEhQVxbj20HANx2Fs4pA6Kp5pSXyzmOvq25JIHD36rDzEgSMZnZQarq+xYBph4QDEhIgATMrmamLAk5AZKUaTpnYSLE9bwSUkSknMf+uL688TxLybV1ALher9M8TfM85XxaJiBsrT/ACQBY8umcc9HenQ+SnAYShNfW2TzPcy68v70BUgDuWzXHCA4WTgncbfRj3wruXf1+v+UsgITEOWfAYGIdezsiV4Qk6UBKqKpuFTBut5uZyTxLKWhhNkjSwX2+fflMjBQweq/btpzmiCjzfADX749appKnYh6ErG6hAyDW82qmL58+99YPeYlIOz89yTQDpfcThjsBliwACCFuNsz8T8G6LFyOzbw7MxwXNCYKiFZb+vptDSlZmdRDEMzdHE7n0+Px2PY+um33u0wTpYSI+/V19GYORITqznaeT31UYLxum0GcliUClmkp0/zYt9Hb3jsAllLev3um9885Z3fftt3cp2kS5lYrM0XgVLKs5bv3Zwtk5qEWAHtt3Q2HESMiHk1xQGiqj8fjUPIU4Zzz3rsjELGOYU0BwsfAWg9j0FeunIdGfP/xw2kuTbV1YmYAbGMwIgu4mpdyr5VT6rdtuz8kpa66bxu4g2GYI3jrDRCDhIS9wjD1PhzpGKGaWWsDEaTMANRbHb3Py6zmow8ibve7B+SShFlyCoTRKiGOWgMhmhGiQ3z68Qd1e9weaqrDUinECUnGsDH2p9O7+XQ2Ha0b5UwoHkAQ9/v9cn7SML/dwe1t2wujYXx5fQ3wkvN0ukRA3TZG+MMf8jRN58vZRxeCiCDm+vHDp7/5sm8b696Gsa0z6J99e3K37hiDt9b3asQERAHUHtt2fzjCVCQiRutqA4K+5hZs/HTbP8xpSdyIRyClguFt6P1eb7fr2+ubmwNFmHvAOByFEWF+xFeZOCcZvZkbswDAUvKHd88Py/+3//Z+HbANM2+y7zvndByyze04nzgAOFgABBDRqGMcKVCPpg7EaZqnnDH8aE0zCyJ7+NF/g3AWDo9ea++j7XX0jkRAyEmmeRmthzsS56lMy1zmJeeMANu+2bARGh51r71WYkYiN48IjwD3cb/drrd1Xcs8l2liIjMDIkqynC/CAsL68LBIy6KtmXmYSZ4IyNTyPAmlxJJKKaUIcyrp8IaYIBNzSqaUHAIQCcODWTglN3OPrgpJfAwIcHNtPdwdwdwgQtVMe7JIZbIDTcckSGYKjB6u+845T1NBJJJEzM9P73LOCuQ6IpyIEKLW7mFJ5HG755xPy7wRG7gOqsO4juXEbmNOuamBgiDc9pbmaW89AAiCCJdlhQh0kyyINMzNPDTq/lVAfX66JEZzBMRjyuNu+94sgg3neTZ3VTvNH5hTmUpOMt4/taGO0Pb6eNR5WdZ5Pp/OW2297jp8f9Rc5LQse+voOOWp5DzcjqiqsLg7E4tIJkoiIoxREPH17Y2FnMjNzGydpykJEjKCeWy1uQdAAITVnQ+AogghmVtTFaallIN6Olo/yM+AZKr19TotC+dMKfWhse+f3m5Py3KaC7OoKSI/tscxLhweFP7x+VS7OiCl5IAQKCkT0+N2J0Ttg4lf3x7TXADpsHnaGGYWCISU11Ns+2j7vj0g4kh8WCDnydwDcIwxzbOImJrpjkzbvtvXBZ/mnLSPacoB7A7n99+keSk5+xjh/fZ6UwuZJ1MFSUgpElMuRAJuo3UAzCWL9bFtj9tLktzr/vzhOQm+frmOUoZ7yqXtDwBDoH0rj8fjmIQ/PT0lUXm7peVUW3/c9x9v7V//7a+J8Ek8dHieyvlCZdrv19vtypIQEMJqrQ4AWMx8tCZMy3nVPpy0pKnV/W+3neKgJROJeB/MgIc4nMV8mPo0zchJ3fGYXUEgIbG4+/3+MB2ACODTVEj4xy/bT9wwSZLjs2lSR8+ICUEYwy0AjyMNERMxEmGS8ACPw/ZCETkLMRBjADHzgSRHDGZRVVUlETAngPPlIsxDda/HuK0ggDATUSoFPMYYyASAddvqvtuxpDjoo6plWXLOEG4BHuBmx+8CwN6aAez7HnGwWRIG+L7h+TwvMzMyopnZXOq2Tx/fny7P22NLOeWUzT0nKTnpGKP3er8SU1lPOU8H65xznuaJiQDQdNTaUciP/YGkqA2QzNT74JLBAkyDBAEgQHiWnFutWntaJrvdfej6/l3vQ2tPmXTbdIw8r6fndx/fv1+XNSIS8/3eSykBx40kVMcYipKW07pMJc9z63qEe3WoOzDGTz/8dL8+Tpe1b9sIfP/9N+s0ixAEBOD9tnkoI2RhJNYIADTzfdt1WEocHhp2EFkPcU0fFuGt7m0LROi9ExFE+GjDuksCQIHIuWSISZhTYrSShDE9wpgPGqW4WWZ+Oi9wcAT6aNoHk+YDatSOJcIwc3dmioCnywXwGFlASrn2QaDTVAgpHxY6oNpaAKhZaw0AIAKJDuzyPE3zNJ1O6xjjss4WcL1eP//y6zENqLUWwMTMIh9Op+elkKC6hgcjq6qZKULrbmbbVr/58HyasgaWLBFQ99r6cPP5vADiY4y6d0pkDqrtaB2gpL7vSKR9mLukVFjutysjWThE+H1bni8pF0lpKlPKyc1q69O6ksjTvKrqaI0Ap2UicMF49+G9h0/T3LueS14TsY/W+s/XHUQYorV2f3l9vd66ORAgS2bS3r788AMRhdvj9tZrDYDeq1m0bbu+fJnXUy6H6F49IOduo67L6dtz+c27QgTrKT9dpvsEv2+PZS3ndV4I1ozh+Def3lj3pay6ZtAMkkarxPz88f3oAwByxiSifbAk5oQNSsk2OoXfXl7dLedyzKO+XsSYScRbR6I+BiJO0/wnNkyIMEB4AEYgsrszY87ZA2rbU0qFSGGEh6oKeJjrbVfpAkRw6HnhoKRnYXZXAw8EIk4siQ8w9VEhMhdOKbsZRWCEm+kY6OFMQhzREY45mhABRNRarY/z8xMI970hIhoSM8uxMdBjJhIRQBzHnwPwK0DnyDSmIoftq2TtzfqotzsgDNWyrk4kSYiImJFRH0okScroA836fWgagBBejkFfa73XvpzPZsGMEbBvdZonAdJhZs3Ne++2jdZanhZAAmREcFcIDMfeu+TMwQAGoyOzO/RawU0iR+Dh/hp1lNMaoTZGnpbzu3dPT8+c0t4GQiSxJNyHRgARnU4LuhvQ3tr753MW+fJ6JaQwG+bzugjzz3/8cW/j+ftvOCUj1t5r6wgYqgigESlPEd63W2HhnANRzRJzSgkBEKD3HhGtVQDoQ5npMC1EBLqnnERk3zZi0t6BekpJxzg6Hsw42kDAVPI0lVKKECFA3Tf3AhGIgEgpMQALc9fjQQxmfnA3zay3lkTMaYyRc2aEPBVwNx2qYRDRurohoJsd3oZpKn1YuKtb6+3wTdNMrpqyTEnmRO/WeZh/8/75si51DHMffSDRuixLyc9Pp/dLeXTbNZgQAByBEW+3u6SckoT76/Xh7qWUY06HxCWjuQ3t53U5C7fWDckt/rNhZJhRxOXdu+1+H72Hu4eHx/1xy/PEQpGo1YaAkjOnRCJlWeCgzX6dCLNE4pRRhAHq/RquOsa+dSll1DZyOs3p44f3l6fx5e1xPs0AyN+9Yx+/vtUvTrW20eunX5v2vjydOLjE6cH88vr6+edPy1QuS8E5T/OUz+d3794TxH67MdPlcjqX/BfvTv/kzz4+mhKl7759rm382YmnIufTHH0gwDqV35wJCEnyX/8U73k1kS+b9G2PRHR6sgAiCDdTZWILz2k2Mw8/uDuApGphdvwwuDszT/NyiARGbQEhOVGEt+YR6MwskpIORfJ5mpZlFuFDK3OkoI8wWLgLAoQHAqqOY999MEVVVTyMCMKJGZgDIEwjiUWADkmEAWNvYHGoaiMAAdwsPIhKEDkCS/Khx/XzQL9yTmqmjw2JhKX3ThHTPK/r2rsOU9U+hlJKfd+NB3FCJimFqDAnklSmSYfmJGWa6t4kFzOn3qd1yTkHAEsGiLCYz5ekRgSAmOdFdUhORDTNc9ubCF+en5mplNxaF6GcshUhlqHWbfSux4R+ezza3s1iWtdUcm9NSmmPvd6voeYePrqb1se2Pj+t56cP33wsU86lYOAYbdt3pBSBSH46n0QyCzNh673W4aaX8xIWGiRZPKL3QQClMM9Ta73uNacUEPdbBcT9ceckIfT0/l0pBcLp6XQC2h77y+ub1Yphy/k0Tdncax2DVUyvX96QkcEpTAPfffPdMOutmQ4PZ+aw0NEisCwzhLdbXc/nlAszI3LtDd2BiBLWbZec1T2JoPD1/oDrjfmgtowylfV0Pp3WnFIQpZScnBEJMCVxdkRiJlVFIDdXs4PNY+77vmMAM8/LPFS3vSKAe6TEGhFI5pBEIqdosUzzISrNOWNmU7vf77o/lmVyoDqGR5jqXms/NOA6bJ6ajh8D5mWd5pkJmUKEiDmXknNmonutQnQwxBHJ3A5uogNWZ9s7EYzWJaXhkUueZZ7nEh6n08zM798/a+vMxAzm8Prltda2t1bmKb4qaWheT3M5jsPRx1DzXqv3fr6sxIgsvffusF0fSJjnBIGqet17rvnnXW179NbwS2pdc5Y///jum2+Xb/L0qGOh/uuHJ3LNmdnteebT0+Vv//jLH374VHK6rGXK6fz09OtuR70Un5+IuLZ+8/g3v9Sfx+uHdTrLyAQly1rSXtvnvf983Xuv5wQfz+XD8+XDb95d1uXXXz9/ern/7XXb9pqzrOciDNq1tSbCCEiEIowQBICAkoupfo2mIg1zQgwAznm9PEVo32vvPeVEyG5x/FjmlIiQD2MDoyQxN/dAoJwLMaMZMwch/k/+Z/9LTgIeX1UoRFIyIn5dRiAhITOTCBJbb0jMOQMEcxJJOgYz55ICQCRDgLo5YP4qhslIWO+Pw5J7DDxSShBBzABfmQSIB0MFAjCI3Kz1joGjbixpOl8oCQsfUFNCiXDTUeapb7XtlZMEgLlN85RSLtOUUtofdzc3AAhMwjkXTtL6PvZWSinLTCRmGnE8zmGrPee0zIUQwuOIO5i5SB691Vpb62O0gLBWR+syzdpH33f3kDIJoSSZT8uHbz7O03zwXnIpR5oLEIKlDTuYNq210VqMsVzOHkCEJXFOGZhNjYnMPVyZEACzpDhahWbq8djqY9vXdcly7Kw53NyNiWvrtTYkCncMzzlL4u3REKNM+fXL27QsY3voqPOyzOdLWNgYSOThh59Q3Y8zwiGATrmklExH33aZpjwVHbrtOzNLyV9Fsx5jDGHWMVSVc5IkTMSAU57meb48XVhY7SCMWSklAkT4fntMU8o5qZqpp3yAunrJSVhKSnZ8e44kkvt2fzw9PR3kWdfhEOYQFkfZg+kYdOD15SUwMKBH3G51q7W11vddSmaWUrL2DgHzXE6n83I+U0QWPljSrmruasAYktPoqkORqeRMiETY1VhY++itl6k4ABJCBCGZWd0e2/Z49+GbdZmO5vk8FWHe9vrlei9TMT1wVgEAetznzRCiTPPt7U1Vf/Pb79QBgTzisT227aGtl8PagxCATNi2u6ozSRCVaQIMHab7tq4zMe/XKwKc1mnfam/7JeH3pyKMmubny5Oa//HW9m7dtNZWa0WEMs9q7mZlyoxYwuac6+N+zrQu5cvLTd1+uu/e26g1i1xOy/dr+ovv3z+ta3X/f/1w7wBhwUxHadEdEmPYcAjJmRF73W+3++O+9d7dgwi+RspFkjASogMz2NAAcNPWuschSAEKhwBzZ2IiQIgxhqnmnFMpeoD0IcJdiFDNwz2ldFQy6SuimA/vE8vXZweie4DWlomI2L15eKhFOAkyipsdITdkAXBtlSE4CwMEIQQGYkqJRExVmFtrpppS4pwRgJkcAAidv04HU75QyillFpaUgKiUjIC9tmVZSNh4pCmnnCMgIpBQ+8hJuo3wI2gBaSopyeFnWqZ5D9737bDIqKoPZSImliJJGABbazaGIxIxiqh2C0s5A/EYfXt9HXUnZmKBgOn8xISny+ndhw/CTIgAMVpv1ZEoSI72YikFXE9zaZ0DgZkq0n2ouYvkMANAjfDWWuvEHDoej7uqzvM8TdPBLDvyHDTPYNbevtxbDdcwDwQfXpYp3NVivjyrBVLUVlPK+/3BiNpTmrKZQypTKdrr/fWlTAsngQAfQJlKmaY01X0jxK8L3EAh6HWvtV6WGRA5Z1R1M20tAlptGIFEtffH61ueJi7FPADAwrXtzW1rrZTCTMe3aaj11olRI8yttUDErztKdzPz4N67mx05odZGStx6V/PWex9dWNpey5RbV1PLU6KAxCSJBGNaFkUYtaNZmUobQ9WdCDlN8zzN5cvj0Vvvo99vt/ntVObptJ7mKa8lUyINbuaZQIdmgjzlgx/UevNuOSUBkJwYYJrLvu9uRELax+H9VPVWKxMRuHrUpqe1BGBOOWdWcDViQR9d/2QvHr2mIu8+vKtbGxbuvt3e+jDKkqdJJNkYgdDdwlyEgZgTMwtJYiZJiUWZaLjNc6bTaXR/AI+MLuXTaL98blm4rMbb6zrNzqxCwyLPS17XVvdwTzkRZgIYvbWIq9W69593pGvbrlfVrq2qdmIm97cX+7vP8P/+4e3d5TwLVrWn98/TJHPJY9CXqzk4EjHJGKO3AREQsa6rMN+31ls100PmokOtD9NxAObMHMIPRQ4zE4H7sVMxQKIsfQwbgwmnqQRQbWOMAQCIMHSI+ldYirkT4VHJFjgOC8ncAdGOt3UEcwIk8CCmI1SGiGFmapjYVCHCQflofKl5hKgw4KHzUXMbY4wh8+wRTCTTJCIAQCIsnIhUtW07k6R5QkJE0t4TzznnAAz7mthGRGudhTiVOJAF4bU2VettlyTr+SLMdBxdiBAhpVSSjD5YeNs28JCSRlda5rLMwmSjt1Z1jCPg4UdZeQxOEoCIuJ4vhLRvD9NBInOZnt6/z1kIiSD6XpGOQ41EeHhsWwWIVMpee6t7Tp2FVU1Yzuu8npbX17dtu4Lr9kAgJogDAtH27UiuA8D9djc3BwjAulVRH60O11H3vm/MmEoew9q1u/l8uozeDIKcUp6OwzUy1q3OlwyEnFJOqYaLMKXk4blMmICYMWVCOJ3PqibCvVZCQuLlfDIPA+y1fg18EW+PO5fpEIpra0DIOZGwu0NAHyrMJU2O2N2191b7us5ZJIEP7WuZ3fXxaJI4J1FVYq619d6HTmZxXE9yzs1s23dVdWSsrbXONHofQdT7IJFt2+vWLudlUhlHbxAIJJnaUSk5PydzFZE5pyzy/ptvbrddpgxmADCGOsBQgwJqZu6JydWY8FF7kkTkmRhy3nRHIo0giIBoQ1UdwoXSsc08P13KNB2I12kuOUtr47G182lZl4kJz5fy08+fmxmEb49tuz/yXFIpb19eIKLkKVRb70HECZkZEPf6OBJOcmyiHObzEyO6uYj0uhPAVIrMM4Kf1kWHPbbNIjRgDK0bhZTz5UwQfd+rW2GZp5xSqnVDxFLKfn8sa57n+XG7WziTAMK8zMx8RCLq7f62bWZHwYQROc9FEO+1bxiFAer2fjp9s+DH9+//4w9vf3y99THUouScs2gfYRgRc1rXZfr0S793HX0ceTSmo6fk4REIBw3YIVpr4cFEqiPAiSia+5+eeoFkBuZ+NIvcwy1EwzEAAd09ApEQHIe7mYr8SQ3K7BFEBOCcxCMQoaQC4AjQ+9A+yMEPUePoY3ROwpQcoHdFiON4iCzm7hFJBFmY2d0BIOXESb5eQSNEWEQCUDUITTLTsWM1NVNTw5RdBwCQsKsFBAKGueSU58mHIrOa7duevvqi8LDRWa/adiZyZoseEcvllHMGBHXvfRwuxYO7zci9j9EHqh5W2rKsl/fvl/P5kHKWUg51YwQCuEMkzoR+bCLLVFpXRKyttb2GuepRAjNXm86naSrMpGPc3l6JwvqYlkWYv56VAuq2H5Tq8djSPAMx8EGkJ0wTOOQyee99dEoT5+QRKEIlEWAEcc4xOjD30ed1CbfRKyCrjJzLseDre5/XNJcZic3ddIBHBCBJnuapJB9DRNziy+vb6AqAy2Wt2x7EbdusD8riBgRAkkDE3G1oILo5IOVSSmYCkFLqMDVPHjoUedQ6zO39ejGzMbQQtdaAWS0sLBQdQsdgSZh4Xue9DWSe5wUgKCViOpep69h1OOLhJ0QMU9vaaOp701wmKcmtIaL3/rrdmehyefrm2495yn3fBUF1aNvA+NXqy+eX3loWySWfTqe3fYwxiIiQTudz3TYlAmYUYpZELDmJJHCfhIngoCeIyFBrraeUw72rvd03QgS3nOl2u3769dPhGBxDaZecc7ib2TzNp/MieTKP07KUeTpiA0O19t5rKyUf7atWGzOnJFySufe9u+lpXR9bhfCSJcLVQ1DQsplBOELkkoloqHvbJaXLMo8xdu1JODy2beujIeIYx3LMknAQnU6nkvI8z8MMEJBoyfnEkcJY6HJaEfw5x0LhdetXiO0mfU+pXN0Y/LuV8ykxehGu6q9bY3suKb2+vD1UAcHVDuY1HoloBFePiN66JCEkDw+IOFbeCMKcpiJ59jYwgBhU9Su9HQAjAhE8PAyIiDNTEmQ6svyqg4gA0d35q4MbItxdRQgBmQ9VhyGRcAai0Vs4gGASOcqgruoRiMqS1nlOOR9ImWOVRoBoFoc5J0JECDEsGEGSAGA317rrXutjI6JpXY7AV54KIXJKRHx8eAIAcwoPd0PEcDsqA+G2Px6jNaA/PZeJwcO/fuWM6JITp0RIyKRmRFTcbahH9NanZUk5BwDNRZgBQN3i2NIdHTo8WmLk7ghMzAgG7m4xz8s05QjX4XutytZqH2OcL5ckycMfb68RIMvJh9reZcoB0VpTNRK2gKgDqbu6lxIOaMopAZI7MCeIMI2yTGFea4cASaXXXf1Q64J6jG33iDIXFAEW8DBFKfMYiqzEQcQemHM6nRIBHCDCIJKUkghJal3HcBQaBVxbDBsRsFVi6RreWlkWV+VUADE88mUuOSNSymlKiRmPN1/Kxc2nqbTWwqyUklNBjOeniwXMpYRbANTa3Y9rB4zWizAzgUMfJoxJeCpZBiaEtZTz+TQJInon9trNTd29N1UbY6zrbKqqo0xTLtlteI0SDtp//vHn4QFhItJGf/3lF2ae13VZZwMe6knE3T/9+hMhJslmDkwpT+ijTHNZTsI0lVRKSYzuxkSTcACHGyDIlOtW931/upyi+3I6l/vjcb+rmg7NnILFQh2je9zrYIMkTCKIuJTMEGFu7o/axlAdve6NGbsq7HtK2QH2velQQCnFe+vMwgQl56lITkf+y11HuKeSCejx2MbozBkZlzIZCSbp2g+CQyQwNxs41OjrqpFO754J6UAqFYr3M0+gZP0ff+R/+4fHf/iiEfp6rW2/DzVVZSYPEMLbr2lJ+TLnf/K7d2eB5q2IfPfN+6fL+fZ4mNrj/ni73tz0eKa4k6q6+Rj9mNS7AxEm4ZxSmdJpKshSh7s3OOJgAIiAiMJfJYqGRMiAhOaOEYf0Ied8LNe/tsQB6NhMuYWFsEiSnLND4PHA4zTUxiGiCAAPYSQWTOLuIvlIvqkO0+POHMyEiKoa5nkqiXOYA1EwsDsRDVU11T7cLa2nJJIyA4CHMxHSV5krAIUrIh0LGiJgQkIgBAIMIvDglJBE+4hjcozBSGbRWyO0nBJLEknTVPro5oHhmAQA+OmSS95qMzURhgAAEGL72g8bwkdMCk3dzEjI1CTJkWOYlzkLMrEXD7PB/BXoHCEY333zsT2db29XMy/zHOkIcBIEtdqXy4nI676ba8oTAQWEm1ISYJIyaW0OBu6jjwBkhPAgdjeglJAMmYCES1lKKcscAK4KCMvTxXQwwKFJPorWhDHqjhEG+PnXu6pezqcylXWeSrI8laF+Y1af4gzX6w0hRm9125G4b21ZZ8lJluW0LKd5zjm5WWLOOTEjEz/2utXOCE+niZ5Odd8ZME9JVZOw2hHmwJQki9TW4bgOuEsSIgyPUhIAJCY3BdMYXQjBddudCYl4mmdOmkS+1p5OqyQpKbXajigyQZQkM6P16l2H2U9//MOmpu6ttvvry+n5eVtXG0oivbVUinms6zqfzz7cFLdtD9ec91KHJEGIZZ6eni455eOsjYgMkRB768sy9dFa7ymlofb8/sN8Opvq9tjM/XQ658NJ5uZmSRILu3ur1UenJFkSIaUsAah9tN5JZNt37UPHJinlec5zbPfHaCyJgUm7ITKaRngiPvYbrpAIiyBEue84uqopI0LEIbUhpGMZF+6dR69d3RiPyaYj4hHE393+9qGt7eI2dPz6sA0TYsak3nggVBta62g1TH8h4ZTnUv6/f/Pr8yzL02UELZOcl3Re3veh42lZ13y7PWrtex3hjgBEOJVZciLEqaSSZZ3LUiameCpsgD92BQgWJg/OGCLHeuTrUwoBJcmxBft6M/6TyYZFDkLG8c+IyCwWOvoIBzMlYeIDiE7HxsfDY/QeCikzIDG6R20dceSSzQwRzQYhSkrVlCghYuv9SPmj/CkscpAog1ikSDnsjimng3sRiKaqrqiWUh5x+CsoMMYwRhZ266OnxCklYWYm5ixkAcTSx1Fc9N47ISK4hSLSGN3UjroouBOi9da1Y0AiZkJAOs6zx7iaAAwgIJBou+2MeFrONjQlZkY30tYF5aDFi0gRahrX6xXdRMjMSkrp44e2D8lJ5GveaoxxdKS2650lI9OxVA5IrUbdK3w1dMSBW7CBqUxAKZVEzDllADCLJedpXQhpXmZ3u11vo7ayLjklO26zSBAArojU23h7uZJQmWZAIcLWdagjs5k+Eaxlmt+du7m7n9dZPVQHqOa5EBIjuZu6MTEToNuUxc0IwdTaaI/7rgDzlLatBkJv3SIcAcNbGw5Qch69D7NSykE9cLOUMwK4OTMfBRAmSuDeNSiGh94fQMxEahUQiWgqE2BEjcNpb2YsguETE0KoeUxrlvTbPz9dCv/Lv/ju9eX1drv94Tz/++1xf7vV2qwPThIRqSsSASCk6XjQjDHADYhoNA/z8H2vavH+aU2eCosDuGpKKcxcxzoVD1B3TpIJy1Qi4HQ+H3Pk8/nsh797DCbuvT+2PQkZM6ru7uF/2vplucg8zVOd0u1236uySM6p7hU8HvtdSi7FEXFvlRAgrHkwUU6SiFjtbavBxIwRgRA6wk0BIU8FhSGAmFRHbOOwvmKSA4dDACkLp1xre9Q+ugHRf/3HB0WUoud1+ngqOPFPr3dXC4sRqOophevYbFielvM8k2QEgbgIIsGX7o+m2nUq5St4KZwJCVFNmQUBMBwgIMJtcMDTlDzgj2bMBBgAwJwB0N3lwGh8fWwhppQw4Mh0RDghBcCR5jdVIjr+t3JiRDyafxHGkg8Qx7wsJImI3MHV5Ku2y1nZ3ABQhG00R3IPJDCzvj2AUPKMX3MbEQCSMhGJJFAgTkJJUiLAcBeS8LAxxlAjROZcCkgQAXu0vSMoHGBQFiJE8CA8QHCIkJiIWNwJEAnNo+4VEfNUWFKEuruatdYP80WoESExqmFESAIOAgw3PV4gGBSA+94AsGTMOXFKNkzHwIg00TIXV3X3urcgBiJBAMFSShAN17Y3qGM5n6Z1TsIpJUYsOakpQLhHXxeWJCyuet+24b49csu5t6p9hCqSEBfJmSUfOhXhVOYZwnNOy7IwERMBUWvGRMvT5fJ0XkqOkggDgYaqufcjr3A6O0RGYjaZs5R8vd0z5WkqjtRUiYiYIGLKYgE0l8QM4YQsjEO1DzU/fiih5GJuQ9XMzIFLIvfRx1EZX89nZux9lCSSxNSTiLlrhO6NAI7+5nH+BUSPGKPX2myZn+YijNDCA8cYw2o4UE6mxoAppW3fkDHlAmYQQYyJWTC7+72qA13WycKqxvvT5fnp3ZzwX263f/zbj3/5l3/19narCRmR5lNZ1qammNZ5CYjXz1+YCd1TRGFE15TSx+dLkLyfy58/zyTpbfjLdTfyyALIWGuac1PrPTis1YapuDnnRMzu1ts4Nk3okDKLMACY+faoaubHGV8SbLDMUzcToimlp/OpDRUMHiDnRT48b49Hb52EjyjL8b5lkW1vRLQuJQmDuxyDP0RFc0B1i9oifJpmIRThdDlPKQ+3AAgEUUPCkhOobe5BNJ9OhFQfN7PogS+3XTO+z/Tbp+k08U+f3sQzTbLMs0WQ5I/Pp989LwFw3bua/fxyS4m/3Nvnt/tj2wFCj6MWs2LIIT/t7cjZhgcTJqYkXFuHiDoCANw9MRJB76qq+D/+n/7PERERSilHjowABI/ObYyhw8whDubEMQw6ghRIaOYHswwIIcLVcskpZRJGEvdwd0D46rYJd3OIQDhGMxERNtTdJCeWzMyAYOZIxJJSmeg/B1GAyjJLKXkq67IM1frYAYFEAuAQi6QjkfSfPzMplynb0CSEKbsampd5kiM4AQAA6tC7jjH80LjFEdSinIWYzdzNEUKYJWczOywBjAhEpooALAkD1FQ9gAgcdGieS0mJAI4+2DKVlKTulYgopd4auLMIiThArW3fKxHlnBjp6FOs81RK3h4bICRJdnwUhclBVae5AFHrum3b43HXcZC+F3O3gEDcbveU87zMIuxmJaXnp0sSaWPc7g8mWqZ8SMjj2EW4j945J1MTkT7MPKxWdz1fLuu6ttG3bc8pTVPe9iYiOafex7HHxIOpyxQeB2B6r9XMmCjlnHM6EpKEPNTmeUKI1oeqiiRhUrPa+pEd7b0v69pVj85QFg4AInSzQxNi5kiwbfskfCkSoZ9e71QmYLk+HqpOxAhwfPhH11QkZUH1peRpnoapm/fWc8kAwMzW27nIOSezOJ2WE0ZgsFXfHt26t/4wms/nEXB9tHkp9fH4/Q+/XtaFCV0yAEzTlJL84++f3gb+zafrhPH++WmZ8gjYzcdwQ9wfWxtD3R3wyI5KSkPtOC1GwHFTAfAxBiNxSgdlprZ28Fvu+34kigGwj45jOOLltKD7NKUlZQegJH9KeEUd1oceD3EUGarCBK6EIHSUrx2ZQs3cltNJR9/uGwNclgkxHOkoEqkqCGvddfS55GZx29px5bQ+emuQRFjqtoUb2aDR5ql093DjiNNc9q5BkhLrvj3aaF9xiSFJlkkA4vHYbVhrTVVHHxEmRBqO9JXxe9RIIoKQv+bFmAAdAYqwmdWuiCARjkiIZGZI5KoEaD4QwR1URzeFCBsGhEzsYTZ01IoI5iaSkIiQ3M1U6wZH6QARAQn5gHchHBaJg+kY8HWxcNxtEakyABzHqNE68kEvIQBiYmBConTNkqfzu3dzKZIk52Tm+vWhwEcnzt2naQKAbmFmOjQiLIjMzC0RqVnv3U1zSutpxa9fAOx7D3cWdhth7nrsHIQp4L+j7cEARByqETRUCSEBlpwSp4QYSDo8AKwPZ8wlh7sOrUqmeqRSzdRUkUQQIbwwScmHjQ4OAKwHIZr6vT/6GOaesk/CJUsg9qEpS0oy1BGilLRM74eZm0Jg3WuMMS3LMn9kQmGCcAg+L8uUCAnd6el8IiS17m4RwAxt2PX1bbs/yjqvpwURW2tMtJxXQGARQBDCdZoAwS2OGF0f2nqH8Jzk2LFoOBMd6YqS8zANj4horekwIAxXM0OMknM+ppB0JHMPFgkfJDUdQ4cyAGEws0awEHio6fGZd8DwQCKLqNs21JA0LHrto6v2lqYpT2WYmVu0yCmf1vI0T0NHM/eIbkpKLKKmb7fbp09tLdMwX07zJacUMC/ZYfbISvZWG7/2lGR0N4p5Pn37u+zIoF0daxtbVR6RvtQ8TwPo1+vjU4fvnpYpJXUjZnQQIYdkrSXipt0BOYAJEydiGubCMsYgxBCutfveWJgpBIGQ5mkqhcdwM6u9E1IzI6LrY2eibdidGgJwYgqY51JKKjlabedpyUhfqnUPd1OlsOHuyOTh5IAAWYQiiqROWO+PznF5WtuIMCXwCOPAKYtapb7XDkxMgMjAKMzkdERYCLN45HrHoQCOHsSS1KjbCNWoXYcSkx3kSxEDvO8Dw4/+1pQXN9v3unIUipeqQKJBAcHgjMFEgexIE6Fqv3VHxK03czvaMnIwTxDZPcYYgIgRYQ7hh9ESAD287xURmBjCLA4G7XE7HYhfj15mDgAa0AkP6BgdIcjDjXok3I/RmxlEGAQEMLEOhQhAJREMcDV3EBZkgJQlTxCeynR693y+XETSkX9xdw1wJAdAJDUjRBH2iEJghEciGAIAjlkTRwQKo4MHmDtEJAQuKSLMzLqGmiEgMg5zd3RAJvPISeJPA0Quxc0BDvOxv71eDwmj5MkDEJFzJuLean1sQ41bBgcESMzIvKyzMAdA78dTjRPL0G6mzFxKARzHDTcQSQTcA/m+dXeflyK5GMB9r61WwjjNsw4NIBFGkfF4uPlX55v7MW8iRHcnYURCxGEqIof9t9cmnC7PZ0lJskyl7L2Hw/E66b2PVkMN3A7urFkkwtp7NwtVTgyA0zRtj4eZUc61tpTkcjn77hqKB8TVPUtSHwjRWj9eOapahxITM7uqe0pJInz0DscBBNmGAgASEzP6cfUECzsyXIxFgWptWhsSh0WYIgS4j70fnpFlysuUhPn18XjcH3hkGUTMrNW273tA7K1TmZZ13rp2C61N7gJEW91TzrW2um9lmggZH2Oak5tvW3MbBzzjgIxf+8jM4JaY9n37w2hZaPraarBpysSUmHPJOOjteqt9LOuifWitRFRK/lp9UTO3iMiSw+1+e6SSTa3kvBRBFEYfGlM671v92sYH2McgCHbutX369HmeyrzMAL4mupzzGDqyMOfeZW94pBXlMJFlJjpYjEPVIKc90O59tE4IQhjuqjEV4Ty33s2HTNxrI8ecsmRwiD4oMSKxg2dmG8MjXO1AfqoqRFjEsi5I0HbUoZKEiFut1jsgMBI5YEDJ6ZxxEpAZmPg+ANwumZ6WzISbGgH80w/8x8/4X/9QAQMAgVIA6FBBJIhwVwAEB3fHiIhwHWN0M0PAgNDeCNCIDgwQBJopoEM4BBxWAkcGCAwg5GMXEOEAfExqgY4wMGdmd4gwcQcDzEwH9BqAU5KUPYBEciksiUVYEiHN6zyfTiTSRz8iP4TA7qbmiAZGEcy4b01ySsKmGqY6DHJMeUan49YjxDgLAboDhEcEEydBHRbqOgawcAZzDzOKECnApBGINNSJkCOIkAnd1DQCOYjaMCD7Eze8jU2vb6/77b4+XZ7WlZlHH0F8HCGZqY2xbRujLOus4eYOga4OBU/rXJGCSdUCwHQgiUUjodYHErFIrX273sB0rP2xd87ldFokyfndEwK2NtQ0l5ISm3v3qNd7mVLJBZCPCFieBACbqgRMU04p9d4BIDEvp5xLfux127sOa7WnxKwGHjnLNBVmJrcgYuIIdPMjJ3gQLbd9P64wB0CYmY28t8bCRykCIWiechZTIyZAcLc+VEQQ6aAtH8d6BweP3vrw0DEkcYBOpYTwttXrY1+WaT6pGwjTXLIPTRhCpBYNsawnER5m42CHMiNi2/cyTzpGeLBkD53KnHMxDxtDRCgnSKnX7gG9d2Fal0Vy5py1Nh1HmmmwJCQyszAl4cfjsUVEOIcDiwMS4jyX0cde69Pz82lZiKKNexFZS9q7jj4AYtt3AiTGAyAhwvGwYQ7UwMwRSNKj9Ucdy5TWuZymMoZbfK21uEdrbdSKiMtpkZLHGI/a2hg+xuOxfbkuSADM00GxNTOzzBIQ1sYA6GZDBxOHh6oFUVdHHeuciuDjWkFSDb3t/ThWSwQChh+wMwJ3oaDEB6un5OSEgRxmYSPc4fjXAJgJmeSIjwqrGhEFc7gDoZoPVfTYFJmwCAvjGCPMwskABIHc/8Vv1//RP738X/+NiwyC6G7mNrqa6v8fhVAgsfiBI9sAAAAASUVORK5CYII=\n",
+ "text/plain": [
+ "
+ 
+
[](https://huggingface.co/spaces/sczhou/CodeFormer) [](https://replicate.com/sczhou/codeformer) 
+
+
+
+[Shangchen Zhou](https://shangchenzhou.com/), [Kelvin C.K. Chan](https://ckkelvinchan.github.io/), [Chongyi Li](https://li-chongyi.github.io/), [Chen Change Loy](https://www.mmlab-ntu.com/person/ccloy/)
+
+S-Lab, Nanyang Technological University
+
+
+
+
+:star: If CodeFormer is helpful to your images or projects, please help star this repo. Thanks! :hugs:
+
+**[News]**: :whale: *Due to copyright issues, we have to delay the release of the training code (expected by the end of this year). Please star and stay tuned for our future updates!*
+### Update
+- **2022.10.05**: Support video input `--input_path [YOUR_VIDOE.mp4]`. Try it to enhance your videos! :clapper:
+- **2022.09.14**: Integrated to :hugs: [Hugging Face](https://huggingface.co/spaces). Try out online demo! [](https://huggingface.co/spaces/sczhou/CodeFormer)
+- **2022.09.09**: Integrated to :rocket: [Replicate](https://replicate.com/explore). Try out online demo! [](https://replicate.com/sczhou/codeformer)
+- **2022.09.04**: Add face upsampling `--face_upsample` for high-resolution AI-created face enhancement.
+- **2022.08.23**: Some modifications on face detection and fusion for better AI-created face enhancement.
+- **2022.08.07**: Integrate [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) to support background image enhancement.
+- **2022.07.29**: Integrate new face detectors of `['RetinaFace'(default), 'YOLOv5']`.
+- **2022.07.17**: Add Colab demo of CodeFormer.
+- **2022.07.16**: Release inference code for face restoration. :blush:
+- **2022.06.21**: This repo is created.
+
+### TODO
+- [ ] Add checkpoint for face inpainting
+- [ ] Add checkpoint for face colorization
+- [ ] Add training code and config files
+- [x] ~~Add background image enhancement~~
+
+#### :panda_face: Try Enhancing Old Photos / Fixing AI-arts
+[
](https://imgsli.com/MTI3NTE2) [
](https://imgsli.com/MTI3NTE1) [
](https://imgsli.com/MTI3NTIw)
+
+#### Face Restoration
+
+
+
+
+#### Face Color Enhancement and Restoration
+
+
+
+#### Face Inpainting
+
+
+
+
+
+### Dependencies and Installation
+
+- Pytorch >= 1.7.1
+- CUDA >= 10.1
+- Other required packages in `requirements.txt`
+```
+# git clone this repository
+git clone https://github.com/sczhou/CodeFormer
+cd CodeFormer
+
+# create new anaconda env
+conda create -n codeformer python=3.8 -y
+conda activate codeformer
+
+# install python dependencies
+pip3 install -r requirements.txt
+python basicsr/setup.py develop
+```
+
+
+### Quick Inference
+
+#### Download Pre-trained Models:
+Download the facelib pretrained models from [[Google Drive](https://drive.google.com/drive/folders/1b_3qwrzY_kTQh0-SnBoGBgOrJ_PLZSKm?usp=sharing) | [OneDrive](https://entuedu-my.sharepoint.com/:f:/g/personal/s200094_e_ntu_edu_sg/EvDxR7FcAbZMp_MA9ouq7aQB8XTppMb3-T0uGZ_2anI2mg?e=DXsJFo)] to the `weights/facelib` folder. You can manually download the pretrained models OR download by running the following command.
+```
+python scripts/download_pretrained_models.py facelib
+```
+
+Download the CodeFormer pretrained models from [[Google Drive](https://drive.google.com/drive/folders/1CNNByjHDFt0b95q54yMVp6Ifo5iuU6QS?usp=sharing) | [OneDrive](https://entuedu-my.sharepoint.com/:f:/g/personal/s200094_e_ntu_edu_sg/EoKFj4wo8cdIn2-TY2IV6CYBhZ0pIG4kUOeHdPR_A5nlbg?e=AO8UN9)] to the `weights/CodeFormer` folder. You can manually download the pretrained models OR download by running the following command.
+```
+python scripts/download_pretrained_models.py CodeFormer
+```
+
+#### Prepare Testing Data:
+You can put the testing images in the `inputs/TestWhole` folder. If you would like to test on cropped and aligned faces, you can put them in the `inputs/cropped_faces` folder.
+
+
+#### Testing on Face Restoration:
+[Note] If you want to compare CodeFormer in your paper, please run the following command indicating `--has_aligned` (for cropped and aligned face), as the command for the whole image will involve a process of face-background fusion that may damage hair texture on the boundary, which leads to unfair comparison.
+
+🧑🏻 Face Restoration (cropped and aligned face)
+```
+# For cropped and aligned faces
+python inference_codeformer.py -w 0.5 --has_aligned --input_path [input folder]
+```
+
+:framed_picture: Whole Image Enhancement
+```
+# For whole image
+# Add '--bg_upsampler realesrgan' to enhance the background regions with Real-ESRGAN
+# Add '--face_upsample' to further upsample restorated face with Real-ESRGAN
+python inference_codeformer.py -w 0.7 --input_path [image folder/image path]
+```
+
+:clapper: Video Enhancement
+```
+# For video clips
+python inference_codeformer.py --bg_upsampler realesrgan --face_upsample -w 1.0 --input_path [video path]
+```
+
+
+Fidelity weight *w* lays in [0, 1]. Generally, smaller *w* tends to produce a higher-quality result, while larger *w* yields a higher-fidelity result.
+
+The results will be saved in the `results` folder.
+
+### Citation
+If our work is useful for your research, please consider citing:
+
+ @inproceedings{zhou2022codeformer,
+ author = {Zhou, Shangchen and Chan, Kelvin C.K. and Li, Chongyi and Loy, Chen Change},
+ title = {Towards Robust Blind Face Restoration with Codebook Lookup TransFormer},
+ booktitle = {NeurIPS},
+ year = {2022}
+ }
+
+### License
+
+This project is licensed under NTU S-Lab License 1.0. Redistribution and use should follow this license.
+
+### Acknowledgement
+
+This project is based on [BasicSR](https://github.com/XPixelGroup/BasicSR). Some codes are brought from [Unleashing Transformers](https://github.com/samb-t/unleashing-transformers), [YOLOv5-face](https://github.com/deepcam-cn/yolov5-face), and [FaceXLib](https://github.com/xinntao/facexlib). We also adopt [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) to support background image enhancement. Thanks for their awesome works.
+
+### Contact
+If you have any question, please feel free to reach me out at `shangchenzhou@gmail.com`.
diff --git a/sd/stablediffusion/src/codeformer/assets/CodeFormer_logo.png b/sd/stablediffusion/src/codeformer/assets/CodeFormer_logo.png
new file mode 100644
index 0000000000000000000000000000000000000000..024cb724f43c2b5cff7039c69b78f261a5a4898c
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/CodeFormer_logo.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/color_enhancement_result1.png b/sd/stablediffusion/src/codeformer/assets/color_enhancement_result1.png
new file mode 100644
index 0000000000000000000000000000000000000000..34433db6378b37cb47a1e544217e4d7f679f7038
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/color_enhancement_result1.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/color_enhancement_result2.png b/sd/stablediffusion/src/codeformer/assets/color_enhancement_result2.png
new file mode 100644
index 0000000000000000000000000000000000000000..228690ac9b1453e67e0212ab2952bea887543a09
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/color_enhancement_result2.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/imgsli_2.jpg b/sd/stablediffusion/src/codeformer/assets/imgsli_2.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..42dd7f43867aabdf1790ee66e73d8b13fd31ec32
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/imgsli_2.jpg differ
diff --git a/sd/stablediffusion/src/codeformer/assets/imgsli_3.jpg b/sd/stablediffusion/src/codeformer/assets/imgsli_3.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c3f67d9d3526233f5cd5122a571db48135827325
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/imgsli_3.jpg differ
diff --git a/sd/stablediffusion/src/codeformer/assets/inpainting_result1.png b/sd/stablediffusion/src/codeformer/assets/inpainting_result1.png
new file mode 100644
index 0000000000000000000000000000000000000000..2c6fa68ad4340c0281e096f7928d28be831c00c1
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/inpainting_result1.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/inpainting_result2.png b/sd/stablediffusion/src/codeformer/assets/inpainting_result2.png
new file mode 100644
index 0000000000000000000000000000000000000000..2945f9f91c93c329c5e66d4e8519dbb3f90fa1b5
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/inpainting_result2.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/network.jpg b/sd/stablediffusion/src/codeformer/assets/network.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..5aaa6bd1b0f71bf28e5f175c2cda1e7b34b8aa5f
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/network.jpg differ
diff --git a/sd/stablediffusion/src/codeformer/assets/restoration_result1.png b/sd/stablediffusion/src/codeformer/assets/restoration_result1.png
new file mode 100644
index 0000000000000000000000000000000000000000..8fd3b67ec9a5c9b7606ea0515a5b071c1e7a1118
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/restoration_result1.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/restoration_result2.png b/sd/stablediffusion/src/codeformer/assets/restoration_result2.png
new file mode 100644
index 0000000000000000000000000000000000000000..a2ff282701b6c66a612b3b669512e8d99595ee9f
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/restoration_result2.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/restoration_result3.png b/sd/stablediffusion/src/codeformer/assets/restoration_result3.png
new file mode 100644
index 0000000000000000000000000000000000000000..022d764266b4d43f4ffea6b1f7ccca63b32e180c
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/restoration_result3.png differ
diff --git a/sd/stablediffusion/src/codeformer/assets/restoration_result4.png b/sd/stablediffusion/src/codeformer/assets/restoration_result4.png
new file mode 100644
index 0000000000000000000000000000000000000000..5e965076c7b5fae051dc2df354f74c0864ec4214
Binary files /dev/null and b/sd/stablediffusion/src/codeformer/assets/restoration_result4.png differ
diff --git a/sd/stablediffusion/src/codeformer/basicsr/VERSION b/sd/stablediffusion/src/codeformer/basicsr/VERSION
new file mode 100644
index 0000000000000000000000000000000000000000..1892b926767774e9ba91f1e584fa71b4c56abb69
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/VERSION
@@ -0,0 +1 @@
+1.3.2
diff --git a/sd/stablediffusion/src/codeformer/basicsr/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..c7ffcccd7fc0f33b59d99d73d0436d60e561b0fc
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/__init__.py
@@ -0,0 +1,11 @@
+# https://github.com/xinntao/BasicSR
+# flake8: noqa
+from .archs import *
+from .data import *
+from .losses import *
+from .metrics import *
+from .models import *
+from .ops import *
+from .train import *
+from .utils import *
+from .version import __gitsha__, __version__
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/archs/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..cfb1e4d7bb221c429082bd389d9140e5b1cc07b0
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/__init__.py
@@ -0,0 +1,25 @@
+import importlib
+from copy import deepcopy
+from os import path as osp
+
+from basicsr.utils import get_root_logger, scandir
+from basicsr.utils.registry import ARCH_REGISTRY
+
+__all__ = ['build_network']
+
+# automatically scan and import arch modules for registry
+# scan all the files under the 'archs' folder and collect files ending with
+# '_arch.py'
+arch_folder = osp.dirname(osp.abspath(__file__))
+arch_filenames = [osp.splitext(osp.basename(v))[0] for v in scandir(arch_folder) if v.endswith('_arch.py')]
+# import all the arch modules
+_arch_modules = [importlib.import_module(f'basicsr.archs.{file_name}') for file_name in arch_filenames]
+
+
+def build_network(opt):
+ opt = deepcopy(opt)
+ network_type = opt.pop('type')
+ net = ARCH_REGISTRY.get(network_type)(**opt)
+ logger = get_root_logger()
+ logger.info(f'Network [{net.__class__.__name__}] is created.')
+ return net
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/arcface_arch.py b/sd/stablediffusion/src/codeformer/basicsr/archs/arcface_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..fe5afb7bd2b359e0c2b7efdf628ab10b63964d87
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/arcface_arch.py
@@ -0,0 +1,245 @@
+import torch.nn as nn
+from basicsr.utils.registry import ARCH_REGISTRY
+
+
+def conv3x3(inplanes, outplanes, stride=1):
+ """A simple wrapper for 3x3 convolution with padding.
+
+ Args:
+ inplanes (int): Channel number of inputs.
+ outplanes (int): Channel number of outputs.
+ stride (int): Stride in convolution. Default: 1.
+ """
+ return nn.Conv2d(inplanes, outplanes, kernel_size=3, stride=stride, padding=1, bias=False)
+
+
+class BasicBlock(nn.Module):
+ """Basic residual block used in the ResNetArcFace architecture.
+
+ Args:
+ inplanes (int): Channel number of inputs.
+ planes (int): Channel number of outputs.
+ stride (int): Stride in convolution. Default: 1.
+ downsample (nn.Module): The downsample module. Default: None.
+ """
+ expansion = 1 # output channel expansion ratio
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None):
+ super(BasicBlock, self).__init__()
+ self.conv1 = conv3x3(inplanes, planes, stride)
+ self.bn1 = nn.BatchNorm2d(planes)
+ self.relu = nn.ReLU(inplace=True)
+ self.conv2 = conv3x3(planes, planes)
+ self.bn2 = nn.BatchNorm2d(planes)
+ self.downsample = downsample
+ self.stride = stride
+
+ def forward(self, x):
+ residual = x
+
+ out = self.conv1(x)
+ out = self.bn1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+
+ if self.downsample is not None:
+ residual = self.downsample(x)
+
+ out += residual
+ out = self.relu(out)
+
+ return out
+
+
+class IRBlock(nn.Module):
+ """Improved residual block (IR Block) used in the ResNetArcFace architecture.
+
+ Args:
+ inplanes (int): Channel number of inputs.
+ planes (int): Channel number of outputs.
+ stride (int): Stride in convolution. Default: 1.
+ downsample (nn.Module): The downsample module. Default: None.
+ use_se (bool): Whether use the SEBlock (squeeze and excitation block). Default: True.
+ """
+ expansion = 1 # output channel expansion ratio
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None, use_se=True):
+ super(IRBlock, self).__init__()
+ self.bn0 = nn.BatchNorm2d(inplanes)
+ self.conv1 = conv3x3(inplanes, inplanes)
+ self.bn1 = nn.BatchNorm2d(inplanes)
+ self.prelu = nn.PReLU()
+ self.conv2 = conv3x3(inplanes, planes, stride)
+ self.bn2 = nn.BatchNorm2d(planes)
+ self.downsample = downsample
+ self.stride = stride
+ self.use_se = use_se
+ if self.use_se:
+ self.se = SEBlock(planes)
+
+ def forward(self, x):
+ residual = x
+ out = self.bn0(x)
+ out = self.conv1(out)
+ out = self.bn1(out)
+ out = self.prelu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+ if self.use_se:
+ out = self.se(out)
+
+ if self.downsample is not None:
+ residual = self.downsample(x)
+
+ out += residual
+ out = self.prelu(out)
+
+ return out
+
+
+class Bottleneck(nn.Module):
+ """Bottleneck block used in the ResNetArcFace architecture.
+
+ Args:
+ inplanes (int): Channel number of inputs.
+ planes (int): Channel number of outputs.
+ stride (int): Stride in convolution. Default: 1.
+ downsample (nn.Module): The downsample module. Default: None.
+ """
+ expansion = 4 # output channel expansion ratio
+
+ def __init__(self, inplanes, planes, stride=1, downsample=None):
+ super(Bottleneck, self).__init__()
+ self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, bias=False)
+ self.bn1 = nn.BatchNorm2d(planes)
+ self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False)
+ self.bn2 = nn.BatchNorm2d(planes)
+ self.conv3 = nn.Conv2d(planes, planes * self.expansion, kernel_size=1, bias=False)
+ self.bn3 = nn.BatchNorm2d(planes * self.expansion)
+ self.relu = nn.ReLU(inplace=True)
+ self.downsample = downsample
+ self.stride = stride
+
+ def forward(self, x):
+ residual = x
+
+ out = self.conv1(x)
+ out = self.bn1(out)
+ out = self.relu(out)
+
+ out = self.conv2(out)
+ out = self.bn2(out)
+ out = self.relu(out)
+
+ out = self.conv3(out)
+ out = self.bn3(out)
+
+ if self.downsample is not None:
+ residual = self.downsample(x)
+
+ out += residual
+ out = self.relu(out)
+
+ return out
+
+
+class SEBlock(nn.Module):
+ """The squeeze-and-excitation block (SEBlock) used in the IRBlock.
+
+ Args:
+ channel (int): Channel number of inputs.
+ reduction (int): Channel reduction ration. Default: 16.
+ """
+
+ def __init__(self, channel, reduction=16):
+ super(SEBlock, self).__init__()
+ self.avg_pool = nn.AdaptiveAvgPool2d(1) # pool to 1x1 without spatial information
+ self.fc = nn.Sequential(
+ nn.Linear(channel, channel // reduction), nn.PReLU(), nn.Linear(channel // reduction, channel),
+ nn.Sigmoid())
+
+ def forward(self, x):
+ b, c, _, _ = x.size()
+ y = self.avg_pool(x).view(b, c)
+ y = self.fc(y).view(b, c, 1, 1)
+ return x * y
+
+
+@ARCH_REGISTRY.register()
+class ResNetArcFace(nn.Module):
+ """ArcFace with ResNet architectures.
+
+ Ref: ArcFace: Additive Angular Margin Loss for Deep Face Recognition.
+
+ Args:
+ block (str): Block used in the ArcFace architecture.
+ layers (tuple(int)): Block numbers in each layer.
+ use_se (bool): Whether use the SEBlock (squeeze and excitation block). Default: True.
+ """
+
+ def __init__(self, block, layers, use_se=True):
+ if block == 'IRBlock':
+ block = IRBlock
+ self.inplanes = 64
+ self.use_se = use_se
+ super(ResNetArcFace, self).__init__()
+
+ self.conv1 = nn.Conv2d(1, 64, kernel_size=3, padding=1, bias=False)
+ self.bn1 = nn.BatchNorm2d(64)
+ self.prelu = nn.PReLU()
+ self.maxpool = nn.MaxPool2d(kernel_size=2, stride=2)
+ self.layer1 = self._make_layer(block, 64, layers[0])
+ self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
+ self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
+ self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
+ self.bn4 = nn.BatchNorm2d(512)
+ self.dropout = nn.Dropout()
+ self.fc5 = nn.Linear(512 * 8 * 8, 512)
+ self.bn5 = nn.BatchNorm1d(512)
+
+ # initialization
+ for m in self.modules():
+ if isinstance(m, nn.Conv2d):
+ nn.init.xavier_normal_(m.weight)
+ elif isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.BatchNorm1d):
+ nn.init.constant_(m.weight, 1)
+ nn.init.constant_(m.bias, 0)
+ elif isinstance(m, nn.Linear):
+ nn.init.xavier_normal_(m.weight)
+ nn.init.constant_(m.bias, 0)
+
+ def _make_layer(self, block, planes, num_blocks, stride=1):
+ downsample = None
+ if stride != 1 or self.inplanes != planes * block.expansion:
+ downsample = nn.Sequential(
+ nn.Conv2d(self.inplanes, planes * block.expansion, kernel_size=1, stride=stride, bias=False),
+ nn.BatchNorm2d(planes * block.expansion),
+ )
+ layers = []
+ layers.append(block(self.inplanes, planes, stride, downsample, use_se=self.use_se))
+ self.inplanes = planes
+ for _ in range(1, num_blocks):
+ layers.append(block(self.inplanes, planes, use_se=self.use_se))
+
+ return nn.Sequential(*layers)
+
+ def forward(self, x):
+ x = self.conv1(x)
+ x = self.bn1(x)
+ x = self.prelu(x)
+ x = self.maxpool(x)
+
+ x = self.layer1(x)
+ x = self.layer2(x)
+ x = self.layer3(x)
+ x = self.layer4(x)
+ x = self.bn4(x)
+ x = self.dropout(x)
+ x = x.view(x.size(0), -1)
+ x = self.fc5(x)
+ x = self.bn5(x)
+
+ return x
\ No newline at end of file
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/arch_util.py b/sd/stablediffusion/src/codeformer/basicsr/archs/arch_util.py
new file mode 100644
index 0000000000000000000000000000000000000000..bad45ab34e901c47fb539152fca714a3795b0de2
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/arch_util.py
@@ -0,0 +1,318 @@
+import collections.abc
+import math
+import torch
+import torchvision
+import warnings
+from distutils.version import LooseVersion
+from itertools import repeat
+from torch import nn as nn
+from torch.nn import functional as F
+from torch.nn import init as init
+from torch.nn.modules.batchnorm import _BatchNorm
+
+from basicsr.ops.dcn import ModulatedDeformConvPack, modulated_deform_conv
+from basicsr.utils import get_root_logger
+
+
+@torch.no_grad()
+def default_init_weights(module_list, scale=1, bias_fill=0, **kwargs):
+ """Initialize network weights.
+
+ Args:
+ module_list (list[nn.Module] | nn.Module): Modules to be initialized.
+ scale (float): Scale initialized weights, especially for residual
+ blocks. Default: 1.
+ bias_fill (float): The value to fill bias. Default: 0
+ kwargs (dict): Other arguments for initialization function.
+ """
+ if not isinstance(module_list, list):
+ module_list = [module_list]
+ for module in module_list:
+ for m in module.modules():
+ if isinstance(m, nn.Conv2d):
+ init.kaiming_normal_(m.weight, **kwargs)
+ m.weight.data *= scale
+ if m.bias is not None:
+ m.bias.data.fill_(bias_fill)
+ elif isinstance(m, nn.Linear):
+ init.kaiming_normal_(m.weight, **kwargs)
+ m.weight.data *= scale
+ if m.bias is not None:
+ m.bias.data.fill_(bias_fill)
+ elif isinstance(m, _BatchNorm):
+ init.constant_(m.weight, 1)
+ if m.bias is not None:
+ m.bias.data.fill_(bias_fill)
+
+
+def make_layer(basic_block, num_basic_block, **kwarg):
+ """Make layers by stacking the same blocks.
+
+ Args:
+ basic_block (nn.module): nn.module class for basic block.
+ num_basic_block (int): number of blocks.
+
+ Returns:
+ nn.Sequential: Stacked blocks in nn.Sequential.
+ """
+ layers = []
+ for _ in range(num_basic_block):
+ layers.append(basic_block(**kwarg))
+ return nn.Sequential(*layers)
+
+
+class ResidualBlockNoBN(nn.Module):
+ """Residual block without BN.
+
+ It has a style of:
+ ---Conv-ReLU-Conv-+-
+ |________________|
+
+ Args:
+ num_feat (int): Channel number of intermediate features.
+ Default: 64.
+ res_scale (float): Residual scale. Default: 1.
+ pytorch_init (bool): If set to True, use pytorch default init,
+ otherwise, use default_init_weights. Default: False.
+ """
+
+ def __init__(self, num_feat=64, res_scale=1, pytorch_init=False):
+ super(ResidualBlockNoBN, self).__init__()
+ self.res_scale = res_scale
+ self.conv1 = nn.Conv2d(num_feat, num_feat, 3, 1, 1, bias=True)
+ self.conv2 = nn.Conv2d(num_feat, num_feat, 3, 1, 1, bias=True)
+ self.relu = nn.ReLU(inplace=True)
+
+ if not pytorch_init:
+ default_init_weights([self.conv1, self.conv2], 0.1)
+
+ def forward(self, x):
+ identity = x
+ out = self.conv2(self.relu(self.conv1(x)))
+ return identity + out * self.res_scale
+
+
+class Upsample(nn.Sequential):
+ """Upsample module.
+
+ Args:
+ scale (int): Scale factor. Supported scales: 2^n and 3.
+ num_feat (int): Channel number of intermediate features.
+ """
+
+ def __init__(self, scale, num_feat):
+ m = []
+ if (scale & (scale - 1)) == 0: # scale = 2^n
+ for _ in range(int(math.log(scale, 2))):
+ m.append(nn.Conv2d(num_feat, 4 * num_feat, 3, 1, 1))
+ m.append(nn.PixelShuffle(2))
+ elif scale == 3:
+ m.append(nn.Conv2d(num_feat, 9 * num_feat, 3, 1, 1))
+ m.append(nn.PixelShuffle(3))
+ else:
+ raise ValueError(f'scale {scale} is not supported. Supported scales: 2^n and 3.')
+ super(Upsample, self).__init__(*m)
+
+
+def flow_warp(x, flow, interp_mode='bilinear', padding_mode='zeros', align_corners=True):
+ """Warp an image or feature map with optical flow.
+
+ Args:
+ x (Tensor): Tensor with size (n, c, h, w).
+ flow (Tensor): Tensor with size (n, h, w, 2), normal value.
+ interp_mode (str): 'nearest' or 'bilinear'. Default: 'bilinear'.
+ padding_mode (str): 'zeros' or 'border' or 'reflection'.
+ Default: 'zeros'.
+ align_corners (bool): Before pytorch 1.3, the default value is
+ align_corners=True. After pytorch 1.3, the default value is
+ align_corners=False. Here, we use the True as default.
+
+ Returns:
+ Tensor: Warped image or feature map.
+ """
+ assert x.size()[-2:] == flow.size()[1:3]
+ _, _, h, w = x.size()
+ # create mesh grid
+ grid_y, grid_x = torch.meshgrid(torch.arange(0, h).type_as(x), torch.arange(0, w).type_as(x))
+ grid = torch.stack((grid_x, grid_y), 2).float() # W(x), H(y), 2
+ grid.requires_grad = False
+
+ vgrid = grid + flow
+ # scale grid to [-1,1]
+ vgrid_x = 2.0 * vgrid[:, :, :, 0] / max(w - 1, 1) - 1.0
+ vgrid_y = 2.0 * vgrid[:, :, :, 1] / max(h - 1, 1) - 1.0
+ vgrid_scaled = torch.stack((vgrid_x, vgrid_y), dim=3)
+ output = F.grid_sample(x, vgrid_scaled, mode=interp_mode, padding_mode=padding_mode, align_corners=align_corners)
+
+ # TODO, what if align_corners=False
+ return output
+
+
+def resize_flow(flow, size_type, sizes, interp_mode='bilinear', align_corners=False):
+ """Resize a flow according to ratio or shape.
+
+ Args:
+ flow (Tensor): Precomputed flow. shape [N, 2, H, W].
+ size_type (str): 'ratio' or 'shape'.
+ sizes (list[int | float]): the ratio for resizing or the final output
+ shape.
+ 1) The order of ratio should be [ratio_h, ratio_w]. For
+ downsampling, the ratio should be smaller than 1.0 (i.e., ratio
+ < 1.0). For upsampling, the ratio should be larger than 1.0 (i.e.,
+ ratio > 1.0).
+ 2) The order of output_size should be [out_h, out_w].
+ interp_mode (str): The mode of interpolation for resizing.
+ Default: 'bilinear'.
+ align_corners (bool): Whether align corners. Default: False.
+
+ Returns:
+ Tensor: Resized flow.
+ """
+ _, _, flow_h, flow_w = flow.size()
+ if size_type == 'ratio':
+ output_h, output_w = int(flow_h * sizes[0]), int(flow_w * sizes[1])
+ elif size_type == 'shape':
+ output_h, output_w = sizes[0], sizes[1]
+ else:
+ raise ValueError(f'Size type should be ratio or shape, but got type {size_type}.')
+
+ input_flow = flow.clone()
+ ratio_h = output_h / flow_h
+ ratio_w = output_w / flow_w
+ input_flow[:, 0, :, :] *= ratio_w
+ input_flow[:, 1, :, :] *= ratio_h
+ resized_flow = F.interpolate(
+ input=input_flow, size=(output_h, output_w), mode=interp_mode, align_corners=align_corners)
+ return resized_flow
+
+
+# TODO: may write a cpp file
+def pixel_unshuffle(x, scale):
+ """ Pixel unshuffle.
+
+ Args:
+ x (Tensor): Input feature with shape (b, c, hh, hw).
+ scale (int): Downsample ratio.
+
+ Returns:
+ Tensor: the pixel unshuffled feature.
+ """
+ b, c, hh, hw = x.size()
+ out_channel = c * (scale**2)
+ assert hh % scale == 0 and hw % scale == 0
+ h = hh // scale
+ w = hw // scale
+ x_view = x.view(b, c, h, scale, w, scale)
+ return x_view.permute(0, 1, 3, 5, 2, 4).reshape(b, out_channel, h, w)
+
+
+class DCNv2Pack(ModulatedDeformConvPack):
+ """Modulated deformable conv for deformable alignment.
+
+ Different from the official DCNv2Pack, which generates offsets and masks
+ from the preceding features, this DCNv2Pack takes another different
+ features to generate offsets and masks.
+
+ Ref:
+ Delving Deep into Deformable Alignment in Video Super-Resolution.
+ """
+
+ def forward(self, x, feat):
+ out = self.conv_offset(feat)
+ o1, o2, mask = torch.chunk(out, 3, dim=1)
+ offset = torch.cat((o1, o2), dim=1)
+ mask = torch.sigmoid(mask)
+
+ offset_absmean = torch.mean(torch.abs(offset))
+ if offset_absmean > 50:
+ logger = get_root_logger()
+ logger.warning(f'Offset abs mean is {offset_absmean}, larger than 50.')
+
+ if LooseVersion(torchvision.__version__) >= LooseVersion('0.9.0'):
+ return torchvision.ops.deform_conv2d(x, offset, self.weight, self.bias, self.stride, self.padding,
+ self.dilation, mask)
+ else:
+ return modulated_deform_conv(x, offset, mask, self.weight, self.bias, self.stride, self.padding,
+ self.dilation, self.groups, self.deformable_groups)
+
+
+def _no_grad_trunc_normal_(tensor, mean, std, a, b):
+ # From: https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/layers/weight_init.py
+ # Cut & paste from PyTorch official master until it's in a few official releases - RW
+ # Method based on https://people.sc.fsu.edu/~jburkardt/presentations/truncated_normal.pdf
+ def norm_cdf(x):
+ # Computes standard normal cumulative distribution function
+ return (1. + math.erf(x / math.sqrt(2.))) / 2.
+
+ if (mean < a - 2 * std) or (mean > b + 2 * std):
+ warnings.warn(
+ 'mean is more than 2 std from [a, b] in nn.init.trunc_normal_. '
+ 'The distribution of values may be incorrect.',
+ stacklevel=2)
+
+ with torch.no_grad():
+ # Values are generated by using a truncated uniform distribution and
+ # then using the inverse CDF for the normal distribution.
+ # Get upper and lower cdf values
+ low = norm_cdf((a - mean) / std)
+ up = norm_cdf((b - mean) / std)
+
+ # Uniformly fill tensor with values from [low, up], then translate to
+ # [2l-1, 2u-1].
+ tensor.uniform_(2 * low - 1, 2 * up - 1)
+
+ # Use inverse cdf transform for normal distribution to get truncated
+ # standard normal
+ tensor.erfinv_()
+
+ # Transform to proper mean, std
+ tensor.mul_(std * math.sqrt(2.))
+ tensor.add_(mean)
+
+ # Clamp to ensure it's in the proper range
+ tensor.clamp_(min=a, max=b)
+ return tensor
+
+
+def trunc_normal_(tensor, mean=0., std=1., a=-2., b=2.):
+ r"""Fills the input Tensor with values drawn from a truncated
+ normal distribution.
+
+ From: https://github.com/rwightman/pytorch-image-models/blob/master/timm/models/layers/weight_init.py
+
+ The values are effectively drawn from the
+ normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)`
+ with values outside :math:`[a, b]` redrawn until they are within
+ the bounds. The method used for generating the random values works
+ best when :math:`a \leq \text{mean} \leq b`.
+
+ Args:
+ tensor: an n-dimensional `torch.Tensor`
+ mean: the mean of the normal distribution
+ std: the standard deviation of the normal distribution
+ a: the minimum cutoff value
+ b: the maximum cutoff value
+
+ Examples:
+ >>> w = torch.empty(3, 5)
+ >>> nn.init.trunc_normal_(w)
+ """
+ return _no_grad_trunc_normal_(tensor, mean, std, a, b)
+
+
+# From PyTorch
+def _ntuple(n):
+
+ def parse(x):
+ if isinstance(x, collections.abc.Iterable):
+ return x
+ return tuple(repeat(x, n))
+
+ return parse
+
+
+to_1tuple = _ntuple(1)
+to_2tuple = _ntuple(2)
+to_3tuple = _ntuple(3)
+to_4tuple = _ntuple(4)
+to_ntuple = _ntuple
\ No newline at end of file
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/codeformer_arch.py b/sd/stablediffusion/src/codeformer/basicsr/archs/codeformer_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..4d0d8027c8c4ffb26af6f4ba361514e93e320e8d
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/codeformer_arch.py
@@ -0,0 +1,276 @@
+import math
+import numpy as np
+import torch
+from torch import nn, Tensor
+import torch.nn.functional as F
+from typing import Optional, List
+
+from basicsr.archs.vqgan_arch import *
+from basicsr.utils import get_root_logger
+from basicsr.utils.registry import ARCH_REGISTRY
+
+def calc_mean_std(feat, eps=1e-5):
+ """Calculate mean and std for adaptive_instance_normalization.
+
+ Args:
+ feat (Tensor): 4D tensor.
+ eps (float): A small value added to the variance to avoid
+ divide-by-zero. Default: 1e-5.
+ """
+ size = feat.size()
+ assert len(size) == 4, 'The input feature should be 4D tensor.'
+ b, c = size[:2]
+ feat_var = feat.view(b, c, -1).var(dim=2) + eps
+ feat_std = feat_var.sqrt().view(b, c, 1, 1)
+ feat_mean = feat.view(b, c, -1).mean(dim=2).view(b, c, 1, 1)
+ return feat_mean, feat_std
+
+
+def adaptive_instance_normalization(content_feat, style_feat):
+ """Adaptive instance normalization.
+
+ Adjust the reference features to have the similar color and illuminations
+ as those in the degradate features.
+
+ Args:
+ content_feat (Tensor): The reference feature.
+ style_feat (Tensor): The degradate features.
+ """
+ size = content_feat.size()
+ style_mean, style_std = calc_mean_std(style_feat)
+ content_mean, content_std = calc_mean_std(content_feat)
+ normalized_feat = (content_feat - content_mean.expand(size)) / content_std.expand(size)
+ return normalized_feat * style_std.expand(size) + style_mean.expand(size)
+
+
+class PositionEmbeddingSine(nn.Module):
+ """
+ This is a more standard version of the position embedding, very similar to the one
+ used by the Attention is all you need paper, generalized to work on images.
+ """
+
+ def __init__(self, num_pos_feats=64, temperature=10000, normalize=False, scale=None):
+ super().__init__()
+ self.num_pos_feats = num_pos_feats
+ self.temperature = temperature
+ self.normalize = normalize
+ if scale is not None and normalize is False:
+ raise ValueError("normalize should be True if scale is passed")
+ if scale is None:
+ scale = 2 * math.pi
+ self.scale = scale
+
+ def forward(self, x, mask=None):
+ if mask is None:
+ mask = torch.zeros((x.size(0), x.size(2), x.size(3)), device=x.device, dtype=torch.bool)
+ not_mask = ~mask
+ y_embed = not_mask.cumsum(1, dtype=torch.float32)
+ x_embed = not_mask.cumsum(2, dtype=torch.float32)
+ if self.normalize:
+ eps = 1e-6
+ y_embed = y_embed / (y_embed[:, -1:, :] + eps) * self.scale
+ x_embed = x_embed / (x_embed[:, :, -1:] + eps) * self.scale
+
+ dim_t = torch.arange(self.num_pos_feats, dtype=torch.float32, device=x.device)
+ dim_t = self.temperature ** (2 * (dim_t // 2) / self.num_pos_feats)
+
+ pos_x = x_embed[:, :, :, None] / dim_t
+ pos_y = y_embed[:, :, :, None] / dim_t
+ pos_x = torch.stack(
+ (pos_x[:, :, :, 0::2].sin(), pos_x[:, :, :, 1::2].cos()), dim=4
+ ).flatten(3)
+ pos_y = torch.stack(
+ (pos_y[:, :, :, 0::2].sin(), pos_y[:, :, :, 1::2].cos()), dim=4
+ ).flatten(3)
+ pos = torch.cat((pos_y, pos_x), dim=3).permute(0, 3, 1, 2)
+ return pos
+
+def _get_activation_fn(activation):
+ """Return an activation function given a string"""
+ if activation == "relu":
+ return F.relu
+ if activation == "gelu":
+ return F.gelu
+ if activation == "glu":
+ return F.glu
+ raise RuntimeError(F"activation should be relu/gelu, not {activation}.")
+
+
+class TransformerSALayer(nn.Module):
+ def __init__(self, embed_dim, nhead=8, dim_mlp=2048, dropout=0.0, activation="gelu"):
+ super().__init__()
+ self.self_attn = nn.MultiheadAttention(embed_dim, nhead, dropout=dropout)
+ # Implementation of Feedforward model - MLP
+ self.linear1 = nn.Linear(embed_dim, dim_mlp)
+ self.dropout = nn.Dropout(dropout)
+ self.linear2 = nn.Linear(dim_mlp, embed_dim)
+
+ self.norm1 = nn.LayerNorm(embed_dim)
+ self.norm2 = nn.LayerNorm(embed_dim)
+ self.dropout1 = nn.Dropout(dropout)
+ self.dropout2 = nn.Dropout(dropout)
+
+ self.activation = _get_activation_fn(activation)
+
+ def with_pos_embed(self, tensor, pos: Optional[Tensor]):
+ return tensor if pos is None else tensor + pos
+
+ def forward(self, tgt,
+ tgt_mask: Optional[Tensor] = None,
+ tgt_key_padding_mask: Optional[Tensor] = None,
+ query_pos: Optional[Tensor] = None):
+
+ # self attention
+ tgt2 = self.norm1(tgt)
+ q = k = self.with_pos_embed(tgt2, query_pos)
+ tgt2 = self.self_attn(q, k, value=tgt2, attn_mask=tgt_mask,
+ key_padding_mask=tgt_key_padding_mask)[0]
+ tgt = tgt + self.dropout1(tgt2)
+
+ # ffn
+ tgt2 = self.norm2(tgt)
+ tgt2 = self.linear2(self.dropout(self.activation(self.linear1(tgt2))))
+ tgt = tgt + self.dropout2(tgt2)
+ return tgt
+
+class Fuse_sft_block(nn.Module):
+ def __init__(self, in_ch, out_ch):
+ super().__init__()
+ self.encode_enc = ResBlock(2*in_ch, out_ch)
+
+ self.scale = nn.Sequential(
+ nn.Conv2d(in_ch, out_ch, kernel_size=3, padding=1),
+ nn.LeakyReLU(0.2, True),
+ nn.Conv2d(out_ch, out_ch, kernel_size=3, padding=1))
+
+ self.shift = nn.Sequential(
+ nn.Conv2d(in_ch, out_ch, kernel_size=3, padding=1),
+ nn.LeakyReLU(0.2, True),
+ nn.Conv2d(out_ch, out_ch, kernel_size=3, padding=1))
+
+ def forward(self, enc_feat, dec_feat, w=1):
+ enc_feat = self.encode_enc(torch.cat([enc_feat, dec_feat], dim=1))
+ scale = self.scale(enc_feat)
+ shift = self.shift(enc_feat)
+ residual = w * (dec_feat * scale + shift)
+ out = dec_feat + residual
+ return out
+
+
+@ARCH_REGISTRY.register()
+class CodeFormer(VQAutoEncoder):
+ def __init__(self, dim_embd=512, n_head=8, n_layers=9,
+ codebook_size=1024, latent_size=256,
+ connect_list=['32', '64', '128', '256'],
+ fix_modules=['quantize','generator']):
+ super(CodeFormer, self).__init__(512, 64, [1, 2, 2, 4, 4, 8], 'nearest',2, [16], codebook_size)
+
+ if fix_modules is not None:
+ for module in fix_modules:
+ for param in getattr(self, module).parameters():
+ param.requires_grad = False
+
+ self.connect_list = connect_list
+ self.n_layers = n_layers
+ self.dim_embd = dim_embd
+ self.dim_mlp = dim_embd*2
+
+ self.position_emb = nn.Parameter(torch.zeros(latent_size, self.dim_embd))
+ self.feat_emb = nn.Linear(256, self.dim_embd)
+
+ # transformer
+ self.ft_layers = nn.Sequential(*[TransformerSALayer(embed_dim=dim_embd, nhead=n_head, dim_mlp=self.dim_mlp, dropout=0.0)
+ for _ in range(self.n_layers)])
+
+ # logits_predict head
+ self.idx_pred_layer = nn.Sequential(
+ nn.LayerNorm(dim_embd),
+ nn.Linear(dim_embd, codebook_size, bias=False))
+
+ self.channels = {
+ '16': 512,
+ '32': 256,
+ '64': 256,
+ '128': 128,
+ '256': 128,
+ '512': 64,
+ }
+
+ # after second residual block for > 16, before attn layer for ==16
+ self.fuse_encoder_block = {'512':2, '256':5, '128':8, '64':11, '32':14, '16':18}
+ # after first residual block for > 16, before attn layer for ==16
+ self.fuse_generator_block = {'16':6, '32': 9, '64':12, '128':15, '256':18, '512':21}
+
+ # fuse_convs_dict
+ self.fuse_convs_dict = nn.ModuleDict()
+ for f_size in self.connect_list:
+ in_ch = self.channels[f_size]
+ self.fuse_convs_dict[f_size] = Fuse_sft_block(in_ch, in_ch)
+
+ def _init_weights(self, module):
+ if isinstance(module, (nn.Linear, nn.Embedding)):
+ module.weight.data.normal_(mean=0.0, std=0.02)
+ if isinstance(module, nn.Linear) and module.bias is not None:
+ module.bias.data.zero_()
+ elif isinstance(module, nn.LayerNorm):
+ module.bias.data.zero_()
+ module.weight.data.fill_(1.0)
+
+ def forward(self, x, w=0, detach_16=True, code_only=False, adain=False):
+ # ################### Encoder #####################
+ enc_feat_dict = {}
+ out_list = [self.fuse_encoder_block[f_size] for f_size in self.connect_list]
+ for i, block in enumerate(self.encoder.blocks):
+ x = block(x)
+ if i in out_list:
+ enc_feat_dict[str(x.shape[-1])] = x.clone()
+
+ lq_feat = x
+ # ################# Transformer ###################
+ # quant_feat, codebook_loss, quant_stats = self.quantize(lq_feat)
+ pos_emb = self.position_emb.unsqueeze(1).repeat(1,x.shape[0],1)
+ # BCHW -> BC(HW) -> (HW)BC
+ feat_emb = self.feat_emb(lq_feat.flatten(2).permute(2,0,1))
+ query_emb = feat_emb
+ # Transformer encoder
+ for layer in self.ft_layers:
+ query_emb = layer(query_emb, query_pos=pos_emb)
+
+ # output logits
+ logits = self.idx_pred_layer(query_emb) # (hw)bn
+ logits = logits.permute(1,0,2) # (hw)bn -> b(hw)n
+
+ if code_only: # for training stage II
+ # logits doesn't need softmax before cross_entropy loss
+ return logits, lq_feat
+
+ # ################# Quantization ###################
+ # if self.training:
+ # quant_feat = torch.einsum('btn,nc->btc', [soft_one_hot, self.quantize.embedding.weight])
+ # # b(hw)c -> bc(hw) -> bchw
+ # quant_feat = quant_feat.permute(0,2,1).view(lq_feat.shape)
+ # ------------
+ soft_one_hot = F.softmax(logits, dim=2)
+ _, top_idx = torch.topk(soft_one_hot, 1, dim=2)
+ quant_feat = self.quantize.get_codebook_feat(top_idx, shape=[x.shape[0],16,16,256])
+ # preserve gradients
+ # quant_feat = lq_feat + (quant_feat - lq_feat).detach()
+
+ if detach_16:
+ quant_feat = quant_feat.detach() # for training stage III
+ if adain:
+ quant_feat = adaptive_instance_normalization(quant_feat, lq_feat)
+
+ # ################## Generator ####################
+ x = quant_feat
+ fuse_list = [self.fuse_generator_block[f_size] for f_size in self.connect_list]
+
+ for i, block in enumerate(self.generator.blocks):
+ x = block(x)
+ if i in fuse_list: # fuse after i-th block
+ f_size = str(x.shape[-1])
+ if w>0:
+ x = self.fuse_convs_dict[f_size](enc_feat_dict[f_size].detach(), x, w)
+ out = x
+ # logits doesn't need softmax before cross_entropy loss
+ return out, logits, lq_feat
\ No newline at end of file
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/rrdbnet_arch.py b/sd/stablediffusion/src/codeformer/basicsr/archs/rrdbnet_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..49a2d6c204557cba53ada7550deb587541855cfb
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/rrdbnet_arch.py
@@ -0,0 +1,119 @@
+import torch
+from torch import nn as nn
+from torch.nn import functional as F
+
+from basicsr.utils.registry import ARCH_REGISTRY
+from .arch_util import default_init_weights, make_layer, pixel_unshuffle
+
+
+class ResidualDenseBlock(nn.Module):
+ """Residual Dense Block.
+
+ Used in RRDB block in ESRGAN.
+
+ Args:
+ num_feat (int): Channel number of intermediate features.
+ num_grow_ch (int): Channels for each growth.
+ """
+
+ def __init__(self, num_feat=64, num_grow_ch=32):
+ super(ResidualDenseBlock, self).__init__()
+ self.conv1 = nn.Conv2d(num_feat, num_grow_ch, 3, 1, 1)
+ self.conv2 = nn.Conv2d(num_feat + num_grow_ch, num_grow_ch, 3, 1, 1)
+ self.conv3 = nn.Conv2d(num_feat + 2 * num_grow_ch, num_grow_ch, 3, 1, 1)
+ self.conv4 = nn.Conv2d(num_feat + 3 * num_grow_ch, num_grow_ch, 3, 1, 1)
+ self.conv5 = nn.Conv2d(num_feat + 4 * num_grow_ch, num_feat, 3, 1, 1)
+
+ self.lrelu = nn.LeakyReLU(negative_slope=0.2, inplace=True)
+
+ # initialization
+ default_init_weights([self.conv1, self.conv2, self.conv3, self.conv4, self.conv5], 0.1)
+
+ def forward(self, x):
+ x1 = self.lrelu(self.conv1(x))
+ x2 = self.lrelu(self.conv2(torch.cat((x, x1), 1)))
+ x3 = self.lrelu(self.conv3(torch.cat((x, x1, x2), 1)))
+ x4 = self.lrelu(self.conv4(torch.cat((x, x1, x2, x3), 1)))
+ x5 = self.conv5(torch.cat((x, x1, x2, x3, x4), 1))
+ # Emperically, we use 0.2 to scale the residual for better performance
+ return x5 * 0.2 + x
+
+
+class RRDB(nn.Module):
+ """Residual in Residual Dense Block.
+
+ Used in RRDB-Net in ESRGAN.
+
+ Args:
+ num_feat (int): Channel number of intermediate features.
+ num_grow_ch (int): Channels for each growth.
+ """
+
+ def __init__(self, num_feat, num_grow_ch=32):
+ super(RRDB, self).__init__()
+ self.rdb1 = ResidualDenseBlock(num_feat, num_grow_ch)
+ self.rdb2 = ResidualDenseBlock(num_feat, num_grow_ch)
+ self.rdb3 = ResidualDenseBlock(num_feat, num_grow_ch)
+
+ def forward(self, x):
+ out = self.rdb1(x)
+ out = self.rdb2(out)
+ out = self.rdb3(out)
+ # Emperically, we use 0.2 to scale the residual for better performance
+ return out * 0.2 + x
+
+
+@ARCH_REGISTRY.register()
+class RRDBNet(nn.Module):
+ """Networks consisting of Residual in Residual Dense Block, which is used
+ in ESRGAN.
+
+ ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks.
+
+ We extend ESRGAN for scale x2 and scale x1.
+ Note: This is one option for scale 1, scale 2 in RRDBNet.
+ We first employ the pixel-unshuffle (an inverse operation of pixelshuffle to reduce the spatial size
+ and enlarge the channel size before feeding inputs into the main ESRGAN architecture.
+
+ Args:
+ num_in_ch (int): Channel number of inputs.
+ num_out_ch (int): Channel number of outputs.
+ num_feat (int): Channel number of intermediate features.
+ Default: 64
+ num_block (int): Block number in the trunk network. Defaults: 23
+ num_grow_ch (int): Channels for each growth. Default: 32.
+ """
+
+ def __init__(self, num_in_ch, num_out_ch, scale=4, num_feat=64, num_block=23, num_grow_ch=32):
+ super(RRDBNet, self).__init__()
+ self.scale = scale
+ if scale == 2:
+ num_in_ch = num_in_ch * 4
+ elif scale == 1:
+ num_in_ch = num_in_ch * 16
+ self.conv_first = nn.Conv2d(num_in_ch, num_feat, 3, 1, 1)
+ self.body = make_layer(RRDB, num_block, num_feat=num_feat, num_grow_ch=num_grow_ch)
+ self.conv_body = nn.Conv2d(num_feat, num_feat, 3, 1, 1)
+ # upsample
+ self.conv_up1 = nn.Conv2d(num_feat, num_feat, 3, 1, 1)
+ self.conv_up2 = nn.Conv2d(num_feat, num_feat, 3, 1, 1)
+ self.conv_hr = nn.Conv2d(num_feat, num_feat, 3, 1, 1)
+ self.conv_last = nn.Conv2d(num_feat, num_out_ch, 3, 1, 1)
+
+ self.lrelu = nn.LeakyReLU(negative_slope=0.2, inplace=True)
+
+ def forward(self, x):
+ if self.scale == 2:
+ feat = pixel_unshuffle(x, scale=2)
+ elif self.scale == 1:
+ feat = pixel_unshuffle(x, scale=4)
+ else:
+ feat = x
+ feat = self.conv_first(feat)
+ body_feat = self.conv_body(self.body(feat))
+ feat = feat + body_feat
+ # upsample
+ feat = self.lrelu(self.conv_up1(F.interpolate(feat, scale_factor=2, mode='nearest')))
+ feat = self.lrelu(self.conv_up2(F.interpolate(feat, scale_factor=2, mode='nearest')))
+ out = self.conv_last(self.lrelu(self.conv_hr(feat)))
+ return out
\ No newline at end of file
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/vgg_arch.py b/sd/stablediffusion/src/codeformer/basicsr/archs/vgg_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..23bb0103c8b14ef2588028f7177753db9af62cae
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/vgg_arch.py
@@ -0,0 +1,161 @@
+import os
+import torch
+from collections import OrderedDict
+from torch import nn as nn
+from torchvision.models import vgg as vgg
+
+from basicsr.utils.registry import ARCH_REGISTRY
+
+VGG_PRETRAIN_PATH = 'experiments/pretrained_models/vgg19-dcbb9e9d.pth'
+NAMES = {
+ 'vgg11': [
+ 'conv1_1', 'relu1_1', 'pool1', 'conv2_1', 'relu2_1', 'pool2', 'conv3_1', 'relu3_1', 'conv3_2', 'relu3_2',
+ 'pool3', 'conv4_1', 'relu4_1', 'conv4_2', 'relu4_2', 'pool4', 'conv5_1', 'relu5_1', 'conv5_2', 'relu5_2',
+ 'pool5'
+ ],
+ 'vgg13': [
+ 'conv1_1', 'relu1_1', 'conv1_2', 'relu1_2', 'pool1', 'conv2_1', 'relu2_1', 'conv2_2', 'relu2_2', 'pool2',
+ 'conv3_1', 'relu3_1', 'conv3_2', 'relu3_2', 'pool3', 'conv4_1', 'relu4_1', 'conv4_2', 'relu4_2', 'pool4',
+ 'conv5_1', 'relu5_1', 'conv5_2', 'relu5_2', 'pool5'
+ ],
+ 'vgg16': [
+ 'conv1_1', 'relu1_1', 'conv1_2', 'relu1_2', 'pool1', 'conv2_1', 'relu2_1', 'conv2_2', 'relu2_2', 'pool2',
+ 'conv3_1', 'relu3_1', 'conv3_2', 'relu3_2', 'conv3_3', 'relu3_3', 'pool3', 'conv4_1', 'relu4_1', 'conv4_2',
+ 'relu4_2', 'conv4_3', 'relu4_3', 'pool4', 'conv5_1', 'relu5_1', 'conv5_2', 'relu5_2', 'conv5_3', 'relu5_3',
+ 'pool5'
+ ],
+ 'vgg19': [
+ 'conv1_1', 'relu1_1', 'conv1_2', 'relu1_2', 'pool1', 'conv2_1', 'relu2_1', 'conv2_2', 'relu2_2', 'pool2',
+ 'conv3_1', 'relu3_1', 'conv3_2', 'relu3_2', 'conv3_3', 'relu3_3', 'conv3_4', 'relu3_4', 'pool3', 'conv4_1',
+ 'relu4_1', 'conv4_2', 'relu4_2', 'conv4_3', 'relu4_3', 'conv4_4', 'relu4_4', 'pool4', 'conv5_1', 'relu5_1',
+ 'conv5_2', 'relu5_2', 'conv5_3', 'relu5_3', 'conv5_4', 'relu5_4', 'pool5'
+ ]
+}
+
+
+def insert_bn(names):
+ """Insert bn layer after each conv.
+
+ Args:
+ names (list): The list of layer names.
+
+ Returns:
+ list: The list of layer names with bn layers.
+ """
+ names_bn = []
+ for name in names:
+ names_bn.append(name)
+ if 'conv' in name:
+ position = name.replace('conv', '')
+ names_bn.append('bn' + position)
+ return names_bn
+
+
+@ARCH_REGISTRY.register()
+class VGGFeatureExtractor(nn.Module):
+ """VGG network for feature extraction.
+
+ In this implementation, we allow users to choose whether use normalization
+ in the input feature and the type of vgg network. Note that the pretrained
+ path must fit the vgg type.
+
+ Args:
+ layer_name_list (list[str]): Forward function returns the corresponding
+ features according to the layer_name_list.
+ Example: {'relu1_1', 'relu2_1', 'relu3_1'}.
+ vgg_type (str): Set the type of vgg network. Default: 'vgg19'.
+ use_input_norm (bool): If True, normalize the input image. Importantly,
+ the input feature must in the range [0, 1]. Default: True.
+ range_norm (bool): If True, norm images with range [-1, 1] to [0, 1].
+ Default: False.
+ requires_grad (bool): If true, the parameters of VGG network will be
+ optimized. Default: False.
+ remove_pooling (bool): If true, the max pooling operations in VGG net
+ will be removed. Default: False.
+ pooling_stride (int): The stride of max pooling operation. Default: 2.
+ """
+
+ def __init__(self,
+ layer_name_list,
+ vgg_type='vgg19',
+ use_input_norm=True,
+ range_norm=False,
+ requires_grad=False,
+ remove_pooling=False,
+ pooling_stride=2):
+ super(VGGFeatureExtractor, self).__init__()
+
+ self.layer_name_list = layer_name_list
+ self.use_input_norm = use_input_norm
+ self.range_norm = range_norm
+
+ self.names = NAMES[vgg_type.replace('_bn', '')]
+ if 'bn' in vgg_type:
+ self.names = insert_bn(self.names)
+
+ # only borrow layers that will be used to avoid unused params
+ max_idx = 0
+ for v in layer_name_list:
+ idx = self.names.index(v)
+ if idx > max_idx:
+ max_idx = idx
+
+ if os.path.exists(VGG_PRETRAIN_PATH):
+ vgg_net = getattr(vgg, vgg_type)(pretrained=False)
+ state_dict = torch.load(VGG_PRETRAIN_PATH, map_location=lambda storage, loc: storage)
+ vgg_net.load_state_dict(state_dict)
+ else:
+ vgg_net = getattr(vgg, vgg_type)(pretrained=True)
+
+ features = vgg_net.features[:max_idx + 1]
+
+ modified_net = OrderedDict()
+ for k, v in zip(self.names, features):
+ if 'pool' in k:
+ # if remove_pooling is true, pooling operation will be removed
+ if remove_pooling:
+ continue
+ else:
+ # in some cases, we may want to change the default stride
+ modified_net[k] = nn.MaxPool2d(kernel_size=2, stride=pooling_stride)
+ else:
+ modified_net[k] = v
+
+ self.vgg_net = nn.Sequential(modified_net)
+
+ if not requires_grad:
+ self.vgg_net.eval()
+ for param in self.parameters():
+ param.requires_grad = False
+ else:
+ self.vgg_net.train()
+ for param in self.parameters():
+ param.requires_grad = True
+
+ if self.use_input_norm:
+ # the mean is for image with range [0, 1]
+ self.register_buffer('mean', torch.Tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
+ # the std is for image with range [0, 1]
+ self.register_buffer('std', torch.Tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
+
+ def forward(self, x):
+ """Forward function.
+
+ Args:
+ x (Tensor): Input tensor with shape (n, c, h, w).
+
+ Returns:
+ Tensor: Forward results.
+ """
+ if self.range_norm:
+ x = (x + 1) / 2
+ if self.use_input_norm:
+ x = (x - self.mean) / self.std
+ output = {}
+
+ for key, layer in self.vgg_net._modules.items():
+ x = layer(x)
+ if key in self.layer_name_list:
+ output[key] = x.clone()
+
+ return output
diff --git a/sd/stablediffusion/src/codeformer/basicsr/archs/vqgan_arch.py b/sd/stablediffusion/src/codeformer/basicsr/archs/vqgan_arch.py
new file mode 100644
index 0000000000000000000000000000000000000000..5ac692633ae67c616a8588ee8f7f41dcf4fc35f4
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/archs/vqgan_arch.py
@@ -0,0 +1,434 @@
+'''
+VQGAN code, adapted from the original created by the Unleashing Transformers authors:
+https://github.com/samb-t/unleashing-transformers/blob/master/models/vqgan.py
+
+'''
+import numpy as np
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import copy
+from basicsr.utils import get_root_logger
+from basicsr.utils.registry import ARCH_REGISTRY
+
+def normalize(in_channels):
+ return torch.nn.GroupNorm(num_groups=32, num_channels=in_channels, eps=1e-6, affine=True)
+
+
+@torch.jit.script
+def swish(x):
+ return x*torch.sigmoid(x)
+
+
+# Define VQVAE classes
+class VectorQuantizer(nn.Module):
+ def __init__(self, codebook_size, emb_dim, beta):
+ super(VectorQuantizer, self).__init__()
+ self.codebook_size = codebook_size # number of embeddings
+ self.emb_dim = emb_dim # dimension of embedding
+ self.beta = beta # commitment cost used in loss term, beta * ||z_e(x)-sg[e]||^2
+ self.embedding = nn.Embedding(self.codebook_size, self.emb_dim)
+ self.embedding.weight.data.uniform_(-1.0 / self.codebook_size, 1.0 / self.codebook_size)
+
+ def forward(self, z):
+ # reshape z -> (batch, height, width, channel) and flatten
+ z = z.permute(0, 2, 3, 1).contiguous()
+ z_flattened = z.view(-1, self.emb_dim)
+
+ # distances from z to embeddings e_j (z - e)^2 = z^2 + e^2 - 2 e * z
+ d = (z_flattened ** 2).sum(dim=1, keepdim=True) + (self.embedding.weight**2).sum(1) - \
+ 2 * torch.matmul(z_flattened, self.embedding.weight.t())
+
+ mean_distance = torch.mean(d)
+ # find closest encodings
+ min_encoding_indices = torch.argmin(d, dim=1).unsqueeze(1)
+ # min_encoding_scores, min_encoding_indices = torch.topk(d, 1, dim=1, largest=False)
+ # [0-1], higher score, higher confidence
+ # min_encoding_scores = torch.exp(-min_encoding_scores/10)
+
+ min_encodings = torch.zeros(min_encoding_indices.shape[0], self.codebook_size).to(z)
+ min_encodings.scatter_(1, min_encoding_indices, 1)
+
+ # get quantized latent vectors
+ z_q = torch.matmul(min_encodings, self.embedding.weight).view(z.shape)
+ # compute loss for embedding
+ loss = torch.mean((z_q.detach()-z)**2) + self.beta * torch.mean((z_q - z.detach()) ** 2)
+ # preserve gradients
+ z_q = z + (z_q - z).detach()
+
+ # perplexity
+ e_mean = torch.mean(min_encodings, dim=0)
+ perplexity = torch.exp(-torch.sum(e_mean * torch.log(e_mean + 1e-10)))
+ # reshape back to match original input shape
+ z_q = z_q.permute(0, 3, 1, 2).contiguous()
+
+ return z_q, loss, {
+ "perplexity": perplexity,
+ "min_encodings": min_encodings,
+ "min_encoding_indices": min_encoding_indices,
+ "mean_distance": mean_distance
+ }
+
+ def get_codebook_feat(self, indices, shape):
+ # input indices: batch*token_num -> (batch*token_num)*1
+ # shape: batch, height, width, channel
+ indices = indices.view(-1,1)
+ min_encodings = torch.zeros(indices.shape[0], self.codebook_size).to(indices)
+ min_encodings.scatter_(1, indices, 1)
+ # get quantized latent vectors
+ z_q = torch.matmul(min_encodings.float(), self.embedding.weight)
+
+ if shape is not None: # reshape back to match original input shape
+ z_q = z_q.view(shape).permute(0, 3, 1, 2).contiguous()
+
+ return z_q
+
+
+class GumbelQuantizer(nn.Module):
+ def __init__(self, codebook_size, emb_dim, num_hiddens, straight_through=False, kl_weight=5e-4, temp_init=1.0):
+ super().__init__()
+ self.codebook_size = codebook_size # number of embeddings
+ self.emb_dim = emb_dim # dimension of embedding
+ self.straight_through = straight_through
+ self.temperature = temp_init
+ self.kl_weight = kl_weight
+ self.proj = nn.Conv2d(num_hiddens, codebook_size, 1) # projects last encoder layer to quantized logits
+ self.embed = nn.Embedding(codebook_size, emb_dim)
+
+ def forward(self, z):
+ hard = self.straight_through if self.training else True
+
+ logits = self.proj(z)
+
+ soft_one_hot = F.gumbel_softmax(logits, tau=self.temperature, dim=1, hard=hard)
+
+ z_q = torch.einsum("b n h w, n d -> b d h w", soft_one_hot, self.embed.weight)
+
+ # + kl divergence to the prior loss
+ qy = F.softmax(logits, dim=1)
+ diff = self.kl_weight * torch.sum(qy * torch.log(qy * self.codebook_size + 1e-10), dim=1).mean()
+ min_encoding_indices = soft_one_hot.argmax(dim=1)
+
+ return z_q, diff, {
+ "min_encoding_indices": min_encoding_indices
+ }
+
+
+class Downsample(nn.Module):
+ def __init__(self, in_channels):
+ super().__init__()
+ self.conv = torch.nn.Conv2d(in_channels, in_channels, kernel_size=3, stride=2, padding=0)
+
+ def forward(self, x):
+ pad = (0, 1, 0, 1)
+ x = torch.nn.functional.pad(x, pad, mode="constant", value=0)
+ x = self.conv(x)
+ return x
+
+
+class Upsample(nn.Module):
+ def __init__(self, in_channels):
+ super().__init__()
+ self.conv = nn.Conv2d(in_channels, in_channels, kernel_size=3, stride=1, padding=1)
+
+ def forward(self, x):
+ x = F.interpolate(x, scale_factor=2.0, mode="nearest")
+ x = self.conv(x)
+
+ return x
+
+
+class ResBlock(nn.Module):
+ def __init__(self, in_channels, out_channels=None):
+ super(ResBlock, self).__init__()
+ self.in_channels = in_channels
+ self.out_channels = in_channels if out_channels is None else out_channels
+ self.norm1 = normalize(in_channels)
+ self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=1, padding=1)
+ self.norm2 = normalize(out_channels)
+ self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3, stride=1, padding=1)
+ if self.in_channels != self.out_channels:
+ self.conv_out = nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=1, padding=0)
+
+ def forward(self, x_in):
+ x = x_in
+ x = self.norm1(x)
+ x = swish(x)
+ x = self.conv1(x)
+ x = self.norm2(x)
+ x = swish(x)
+ x = self.conv2(x)
+ if self.in_channels != self.out_channels:
+ x_in = self.conv_out(x_in)
+
+ return x + x_in
+
+
+class AttnBlock(nn.Module):
+ def __init__(self, in_channels):
+ super().__init__()
+ self.in_channels = in_channels
+
+ self.norm = normalize(in_channels)
+ self.q = torch.nn.Conv2d(
+ in_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0
+ )
+ self.k = torch.nn.Conv2d(
+ in_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0
+ )
+ self.v = torch.nn.Conv2d(
+ in_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0
+ )
+ self.proj_out = torch.nn.Conv2d(
+ in_channels,
+ in_channels,
+ kernel_size=1,
+ stride=1,
+ padding=0
+ )
+
+ def forward(self, x):
+ h_ = x
+ h_ = self.norm(h_)
+ q = self.q(h_)
+ k = self.k(h_)
+ v = self.v(h_)
+
+ # compute attention
+ b, c, h, w = q.shape
+ q = q.reshape(b, c, h*w)
+ q = q.permute(0, 2, 1)
+ k = k.reshape(b, c, h*w)
+ w_ = torch.bmm(q, k)
+ w_ = w_ * (int(c)**(-0.5))
+ w_ = F.softmax(w_, dim=2)
+
+ # attend to values
+ v = v.reshape(b, c, h*w)
+ w_ = w_.permute(0, 2, 1)
+ h_ = torch.bmm(v, w_)
+ h_ = h_.reshape(b, c, h, w)
+
+ h_ = self.proj_out(h_)
+
+ return x+h_
+
+
+class Encoder(nn.Module):
+ def __init__(self, in_channels, nf, emb_dim, ch_mult, num_res_blocks, resolution, attn_resolutions):
+ super().__init__()
+ self.nf = nf
+ self.num_resolutions = len(ch_mult)
+ self.num_res_blocks = num_res_blocks
+ self.resolution = resolution
+ self.attn_resolutions = attn_resolutions
+
+ curr_res = self.resolution
+ in_ch_mult = (1,)+tuple(ch_mult)
+
+ blocks = []
+ # initial convultion
+ blocks.append(nn.Conv2d(in_channels, nf, kernel_size=3, stride=1, padding=1))
+
+ # residual and downsampling blocks, with attention on smaller res (16x16)
+ for i in range(self.num_resolutions):
+ block_in_ch = nf * in_ch_mult[i]
+ block_out_ch = nf * ch_mult[i]
+ for _ in range(self.num_res_blocks):
+ blocks.append(ResBlock(block_in_ch, block_out_ch))
+ block_in_ch = block_out_ch
+ if curr_res in attn_resolutions:
+ blocks.append(AttnBlock(block_in_ch))
+
+ if i != self.num_resolutions - 1:
+ blocks.append(Downsample(block_in_ch))
+ curr_res = curr_res // 2
+
+ # non-local attention block
+ blocks.append(ResBlock(block_in_ch, block_in_ch))
+ blocks.append(AttnBlock(block_in_ch))
+ blocks.append(ResBlock(block_in_ch, block_in_ch))
+
+ # normalise and convert to latent size
+ blocks.append(normalize(block_in_ch))
+ blocks.append(nn.Conv2d(block_in_ch, emb_dim, kernel_size=3, stride=1, padding=1))
+ self.blocks = nn.ModuleList(blocks)
+
+ def forward(self, x):
+ for block in self.blocks:
+ x = block(x)
+
+ return x
+
+
+class Generator(nn.Module):
+ def __init__(self, nf, emb_dim, ch_mult, res_blocks, img_size, attn_resolutions):
+ super().__init__()
+ self.nf = nf
+ self.ch_mult = ch_mult
+ self.num_resolutions = len(self.ch_mult)
+ self.num_res_blocks = res_blocks
+ self.resolution = img_size
+ self.attn_resolutions = attn_resolutions
+ self.in_channels = emb_dim
+ self.out_channels = 3
+ block_in_ch = self.nf * self.ch_mult[-1]
+ curr_res = self.resolution // 2 ** (self.num_resolutions-1)
+
+ blocks = []
+ # initial conv
+ blocks.append(nn.Conv2d(self.in_channels, block_in_ch, kernel_size=3, stride=1, padding=1))
+
+ # non-local attention block
+ blocks.append(ResBlock(block_in_ch, block_in_ch))
+ blocks.append(AttnBlock(block_in_ch))
+ blocks.append(ResBlock(block_in_ch, block_in_ch))
+
+ for i in reversed(range(self.num_resolutions)):
+ block_out_ch = self.nf * self.ch_mult[i]
+
+ for _ in range(self.num_res_blocks):
+ blocks.append(ResBlock(block_in_ch, block_out_ch))
+ block_in_ch = block_out_ch
+
+ if curr_res in self.attn_resolutions:
+ blocks.append(AttnBlock(block_in_ch))
+
+ if i != 0:
+ blocks.append(Upsample(block_in_ch))
+ curr_res = curr_res * 2
+
+ blocks.append(normalize(block_in_ch))
+ blocks.append(nn.Conv2d(block_in_ch, self.out_channels, kernel_size=3, stride=1, padding=1))
+
+ self.blocks = nn.ModuleList(blocks)
+
+
+ def forward(self, x):
+ for block in self.blocks:
+ x = block(x)
+
+ return x
+
+
+@ARCH_REGISTRY.register()
+class VQAutoEncoder(nn.Module):
+ def __init__(self, img_size, nf, ch_mult, quantizer="nearest", res_blocks=2, attn_resolutions=[16], codebook_size=1024, emb_dim=256,
+ beta=0.25, gumbel_straight_through=False, gumbel_kl_weight=1e-8, model_path=None):
+ super().__init__()
+ logger = get_root_logger()
+ self.in_channels = 3
+ self.nf = nf
+ self.n_blocks = res_blocks
+ self.codebook_size = codebook_size
+ self.embed_dim = emb_dim
+ self.ch_mult = ch_mult
+ self.resolution = img_size
+ self.attn_resolutions = attn_resolutions
+ self.quantizer_type = quantizer
+ self.encoder = Encoder(
+ self.in_channels,
+ self.nf,
+ self.embed_dim,
+ self.ch_mult,
+ self.n_blocks,
+ self.resolution,
+ self.attn_resolutions
+ )
+ if self.quantizer_type == "nearest":
+ self.beta = beta #0.25
+ self.quantize = VectorQuantizer(self.codebook_size, self.embed_dim, self.beta)
+ elif self.quantizer_type == "gumbel":
+ self.gumbel_num_hiddens = emb_dim
+ self.straight_through = gumbel_straight_through
+ self.kl_weight = gumbel_kl_weight
+ self.quantize = GumbelQuantizer(
+ self.codebook_size,
+ self.embed_dim,
+ self.gumbel_num_hiddens,
+ self.straight_through,
+ self.kl_weight
+ )
+ self.generator = Generator(
+ self.nf,
+ self.embed_dim,
+ self.ch_mult,
+ self.n_blocks,
+ self.resolution,
+ self.attn_resolutions
+ )
+
+ if model_path is not None:
+ chkpt = torch.load(model_path, map_location='cpu')
+ if 'params_ema' in chkpt:
+ self.load_state_dict(torch.load(model_path, map_location='cpu')['params_ema'])
+ logger.info(f'vqgan is loaded from: {model_path} [params_ema]')
+ elif 'params' in chkpt:
+ self.load_state_dict(torch.load(model_path, map_location='cpu')['params'])
+ logger.info(f'vqgan is loaded from: {model_path} [params]')
+ else:
+ raise ValueError(f'Wrong params!')
+
+
+ def forward(self, x):
+ x = self.encoder(x)
+ quant, codebook_loss, quant_stats = self.quantize(x)
+ x = self.generator(quant)
+ return x, codebook_loss, quant_stats
+
+
+
+# patch based discriminator
+@ARCH_REGISTRY.register()
+class VQGANDiscriminator(nn.Module):
+ def __init__(self, nc=3, ndf=64, n_layers=4, model_path=None):
+ super().__init__()
+
+ layers = [nn.Conv2d(nc, ndf, kernel_size=4, stride=2, padding=1), nn.LeakyReLU(0.2, True)]
+ ndf_mult = 1
+ ndf_mult_prev = 1
+ for n in range(1, n_layers): # gradually increase the number of filters
+ ndf_mult_prev = ndf_mult
+ ndf_mult = min(2 ** n, 8)
+ layers += [
+ nn.Conv2d(ndf * ndf_mult_prev, ndf * ndf_mult, kernel_size=4, stride=2, padding=1, bias=False),
+ nn.BatchNorm2d(ndf * ndf_mult),
+ nn.LeakyReLU(0.2, True)
+ ]
+
+ ndf_mult_prev = ndf_mult
+ ndf_mult = min(2 ** n_layers, 8)
+
+ layers += [
+ nn.Conv2d(ndf * ndf_mult_prev, ndf * ndf_mult, kernel_size=4, stride=1, padding=1, bias=False),
+ nn.BatchNorm2d(ndf * ndf_mult),
+ nn.LeakyReLU(0.2, True)
+ ]
+
+ layers += [
+ nn.Conv2d(ndf * ndf_mult, 1, kernel_size=4, stride=1, padding=1)] # output 1 channel prediction map
+ self.main = nn.Sequential(*layers)
+
+ if model_path is not None:
+ chkpt = torch.load(model_path, map_location='cpu')
+ if 'params_d' in chkpt:
+ self.load_state_dict(torch.load(model_path, map_location='cpu')['params_d'])
+ elif 'params' in chkpt:
+ self.load_state_dict(torch.load(model_path, map_location='cpu')['params'])
+ else:
+ raise ValueError(f'Wrong params!')
+
+ def forward(self, x):
+ return self.main(x)
\ No newline at end of file
diff --git a/sd/stablediffusion/src/codeformer/basicsr/data/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/data/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..c6adb4bb6a926af7a46aaec4794eee95fda02a33
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/data/__init__.py
@@ -0,0 +1,100 @@
+import importlib
+import numpy as np
+import random
+import torch
+import torch.utils.data
+from copy import deepcopy
+from functools import partial
+from os import path as osp
+
+from basicsr.data.prefetch_dataloader import PrefetchDataLoader
+from basicsr.utils import get_root_logger, scandir
+from basicsr.utils.dist_util import get_dist_info
+from basicsr.utils.registry import DATASET_REGISTRY
+
+__all__ = ['build_dataset', 'build_dataloader']
+
+# automatically scan and import dataset modules for registry
+# scan all the files under the data folder with '_dataset' in file names
+data_folder = osp.dirname(osp.abspath(__file__))
+dataset_filenames = [osp.splitext(osp.basename(v))[0] for v in scandir(data_folder) if v.endswith('_dataset.py')]
+# import all the dataset modules
+_dataset_modules = [importlib.import_module(f'basicsr.data.{file_name}') for file_name in dataset_filenames]
+
+
+def build_dataset(dataset_opt):
+ """Build dataset from options.
+
+ Args:
+ dataset_opt (dict): Configuration for dataset. It must constain:
+ name (str): Dataset name.
+ type (str): Dataset type.
+ """
+ dataset_opt = deepcopy(dataset_opt)
+ dataset = DATASET_REGISTRY.get(dataset_opt['type'])(dataset_opt)
+ logger = get_root_logger()
+ logger.info(f'Dataset [{dataset.__class__.__name__}] - {dataset_opt["name"]} ' 'is built.')
+ return dataset
+
+
+def build_dataloader(dataset, dataset_opt, num_gpu=1, dist=False, sampler=None, seed=None):
+ """Build dataloader.
+
+ Args:
+ dataset (torch.utils.data.Dataset): Dataset.
+ dataset_opt (dict): Dataset options. It contains the following keys:
+ phase (str): 'train' or 'val'.
+ num_worker_per_gpu (int): Number of workers for each GPU.
+ batch_size_per_gpu (int): Training batch size for each GPU.
+ num_gpu (int): Number of GPUs. Used only in the train phase.
+ Default: 1.
+ dist (bool): Whether in distributed training. Used only in the train
+ phase. Default: False.
+ sampler (torch.utils.data.sampler): Data sampler. Default: None.
+ seed (int | None): Seed. Default: None
+ """
+ phase = dataset_opt['phase']
+ rank, _ = get_dist_info()
+ if phase == 'train':
+ if dist: # distributed training
+ batch_size = dataset_opt['batch_size_per_gpu']
+ num_workers = dataset_opt['num_worker_per_gpu']
+ else: # non-distributed training
+ multiplier = 1 if num_gpu == 0 else num_gpu
+ batch_size = dataset_opt['batch_size_per_gpu'] * multiplier
+ num_workers = dataset_opt['num_worker_per_gpu'] * multiplier
+ dataloader_args = dict(
+ dataset=dataset,
+ batch_size=batch_size,
+ shuffle=False,
+ num_workers=num_workers,
+ sampler=sampler,
+ drop_last=True)
+ if sampler is None:
+ dataloader_args['shuffle'] = True
+ dataloader_args['worker_init_fn'] = partial(
+ worker_init_fn, num_workers=num_workers, rank=rank, seed=seed) if seed is not None else None
+ elif phase in ['val', 'test']: # validation
+ dataloader_args = dict(dataset=dataset, batch_size=1, shuffle=False, num_workers=0)
+ else:
+ raise ValueError(f'Wrong dataset phase: {phase}. ' "Supported ones are 'train', 'val' and 'test'.")
+
+ dataloader_args['pin_memory'] = dataset_opt.get('pin_memory', False)
+
+ prefetch_mode = dataset_opt.get('prefetch_mode')
+ if prefetch_mode == 'cpu': # CPUPrefetcher
+ num_prefetch_queue = dataset_opt.get('num_prefetch_queue', 1)
+ logger = get_root_logger()
+ logger.info(f'Use {prefetch_mode} prefetch dataloader: ' f'num_prefetch_queue = {num_prefetch_queue}')
+ return PrefetchDataLoader(num_prefetch_queue=num_prefetch_queue, **dataloader_args)
+ else:
+ # prefetch_mode=None: Normal dataloader
+ # prefetch_mode='cuda': dataloader for CUDAPrefetcher
+ return torch.utils.data.DataLoader(**dataloader_args)
+
+
+def worker_init_fn(worker_id, num_workers, rank, seed):
+ # Set the worker seed to num_workers * rank + worker_id + seed
+ worker_seed = num_workers * rank + worker_id + seed
+ np.random.seed(worker_seed)
+ random.seed(worker_seed)
diff --git a/sd/stablediffusion/src/codeformer/basicsr/data/data_sampler.py b/sd/stablediffusion/src/codeformer/basicsr/data/data_sampler.py
new file mode 100644
index 0000000000000000000000000000000000000000..575452d9f844a928f7f42296c81635cfbadec7c2
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/data/data_sampler.py
@@ -0,0 +1,48 @@
+import math
+import torch
+from torch.utils.data.sampler import Sampler
+
+
+class EnlargedSampler(Sampler):
+ """Sampler that restricts data loading to a subset of the dataset.
+
+ Modified from torch.utils.data.distributed.DistributedSampler
+ Support enlarging the dataset for iteration-based training, for saving
+ time when restart the dataloader after each epoch
+
+ Args:
+ dataset (torch.utils.data.Dataset): Dataset used for sampling.
+ num_replicas (int | None): Number of processes participating in
+ the training. It is usually the world_size.
+ rank (int | None): Rank of the current process within num_replicas.
+ ratio (int): Enlarging ratio. Default: 1.
+ """
+
+ def __init__(self, dataset, num_replicas, rank, ratio=1):
+ self.dataset = dataset
+ self.num_replicas = num_replicas
+ self.rank = rank
+ self.epoch = 0
+ self.num_samples = math.ceil(len(self.dataset) * ratio / self.num_replicas)
+ self.total_size = self.num_samples * self.num_replicas
+
+ def __iter__(self):
+ # deterministically shuffle based on epoch
+ g = torch.Generator()
+ g.manual_seed(self.epoch)
+ indices = torch.randperm(self.total_size, generator=g).tolist()
+
+ dataset_size = len(self.dataset)
+ indices = [v % dataset_size for v in indices]
+
+ # subsample
+ indices = indices[self.rank:self.total_size:self.num_replicas]
+ assert len(indices) == self.num_samples
+
+ return iter(indices)
+
+ def __len__(self):
+ return self.num_samples
+
+ def set_epoch(self, epoch):
+ self.epoch = epoch
diff --git a/sd/stablediffusion/src/codeformer/basicsr/data/data_util.py b/sd/stablediffusion/src/codeformer/basicsr/data/data_util.py
new file mode 100644
index 0000000000000000000000000000000000000000..63b1bce8e089485182c962e830a163d6d0059da8
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/data/data_util.py
@@ -0,0 +1,305 @@
+import cv2
+import numpy as np
+import torch
+from os import path as osp
+from torch.nn import functional as F
+
+from basicsr.data.transforms import mod_crop
+from basicsr.utils import img2tensor, scandir
+
+
+def read_img_seq(path, require_mod_crop=False, scale=1):
+ """Read a sequence of images from a given folder path.
+
+ Args:
+ path (list[str] | str): List of image paths or image folder path.
+ require_mod_crop (bool): Require mod crop for each image.
+ Default: False.
+ scale (int): Scale factor for mod_crop. Default: 1.
+
+ Returns:
+ Tensor: size (t, c, h, w), RGB, [0, 1].
+ """
+ if isinstance(path, list):
+ img_paths = path
+ else:
+ img_paths = sorted(list(scandir(path, full_path=True)))
+ imgs = [cv2.imread(v).astype(np.float32) / 255. for v in img_paths]
+ if require_mod_crop:
+ imgs = [mod_crop(img, scale) for img in imgs]
+ imgs = img2tensor(imgs, bgr2rgb=True, float32=True)
+ imgs = torch.stack(imgs, dim=0)
+ return imgs
+
+
+def generate_frame_indices(crt_idx, max_frame_num, num_frames, padding='reflection'):
+ """Generate an index list for reading `num_frames` frames from a sequence
+ of images.
+
+ Args:
+ crt_idx (int): Current center index.
+ max_frame_num (int): Max number of the sequence of images (from 1).
+ num_frames (int): Reading num_frames frames.
+ padding (str): Padding mode, one of
+ 'replicate' | 'reflection' | 'reflection_circle' | 'circle'
+ Examples: current_idx = 0, num_frames = 5
+ The generated frame indices under different padding mode:
+ replicate: [0, 0, 0, 1, 2]
+ reflection: [2, 1, 0, 1, 2]
+ reflection_circle: [4, 3, 0, 1, 2]
+ circle: [3, 4, 0, 1, 2]
+
+ Returns:
+ list[int]: A list of indices.
+ """
+ assert num_frames % 2 == 1, 'num_frames should be an odd number.'
+ assert padding in ('replicate', 'reflection', 'reflection_circle', 'circle'), f'Wrong padding mode: {padding}.'
+
+ max_frame_num = max_frame_num - 1 # start from 0
+ num_pad = num_frames // 2
+
+ indices = []
+ for i in range(crt_idx - num_pad, crt_idx + num_pad + 1):
+ if i < 0:
+ if padding == 'replicate':
+ pad_idx = 0
+ elif padding == 'reflection':
+ pad_idx = -i
+ elif padding == 'reflection_circle':
+ pad_idx = crt_idx + num_pad - i
+ else:
+ pad_idx = num_frames + i
+ elif i > max_frame_num:
+ if padding == 'replicate':
+ pad_idx = max_frame_num
+ elif padding == 'reflection':
+ pad_idx = max_frame_num * 2 - i
+ elif padding == 'reflection_circle':
+ pad_idx = (crt_idx - num_pad) - (i - max_frame_num)
+ else:
+ pad_idx = i - num_frames
+ else:
+ pad_idx = i
+ indices.append(pad_idx)
+ return indices
+
+
+def paired_paths_from_lmdb(folders, keys):
+ """Generate paired paths from lmdb files.
+
+ Contents of lmdb. Taking the `lq.lmdb` for example, the file structure is:
+
+ lq.lmdb
+ ├── data.mdb
+ ├── lock.mdb
+ ├── meta_info.txt
+
+ The data.mdb and lock.mdb are standard lmdb files and you can refer to
+ https://lmdb.readthedocs.io/en/release/ for more details.
+
+ The meta_info.txt is a specified txt file to record the meta information
+ of our datasets. It will be automatically created when preparing
+ datasets by our provided dataset tools.
+ Each line in the txt file records
+ 1)image name (with extension),
+ 2)image shape,
+ 3)compression level, separated by a white space.
+ Example: `baboon.png (120,125,3) 1`
+
+ We use the image name without extension as the lmdb key.
+ Note that we use the same key for the corresponding lq and gt images.
+
+ Args:
+ folders (list[str]): A list of folder path. The order of list should
+ be [input_folder, gt_folder].
+ keys (list[str]): A list of keys identifying folders. The order should
+ be in consistent with folders, e.g., ['lq', 'gt'].
+ Note that this key is different from lmdb keys.
+
+ Returns:
+ list[str]: Returned path list.
+ """
+ assert len(folders) == 2, ('The len of folders should be 2 with [input_folder, gt_folder]. '
+ f'But got {len(folders)}')
+ assert len(keys) == 2, ('The len of keys should be 2 with [input_key, gt_key]. ' f'But got {len(keys)}')
+ input_folder, gt_folder = folders
+ input_key, gt_key = keys
+
+ if not (input_folder.endswith('.lmdb') and gt_folder.endswith('.lmdb')):
+ raise ValueError(f'{input_key} folder and {gt_key} folder should both in lmdb '
+ f'formats. But received {input_key}: {input_folder}; '
+ f'{gt_key}: {gt_folder}')
+ # ensure that the two meta_info files are the same
+ with open(osp.join(input_folder, 'meta_info.txt')) as fin:
+ input_lmdb_keys = [line.split('.')[0] for line in fin]
+ with open(osp.join(gt_folder, 'meta_info.txt')) as fin:
+ gt_lmdb_keys = [line.split('.')[0] for line in fin]
+ if set(input_lmdb_keys) != set(gt_lmdb_keys):
+ raise ValueError(f'Keys in {input_key}_folder and {gt_key}_folder are different.')
+ else:
+ paths = []
+ for lmdb_key in sorted(input_lmdb_keys):
+ paths.append(dict([(f'{input_key}_path', lmdb_key), (f'{gt_key}_path', lmdb_key)]))
+ return paths
+
+
+def paired_paths_from_meta_info_file(folders, keys, meta_info_file, filename_tmpl):
+ """Generate paired paths from an meta information file.
+
+ Each line in the meta information file contains the image names and
+ image shape (usually for gt), separated by a white space.
+
+ Example of an meta information file:
+ ```
+ 0001_s001.png (480,480,3)
+ 0001_s002.png (480,480,3)
+ ```
+
+ Args:
+ folders (list[str]): A list of folder path. The order of list should
+ be [input_folder, gt_folder].
+ keys (list[str]): A list of keys identifying folders. The order should
+ be in consistent with folders, e.g., ['lq', 'gt'].
+ meta_info_file (str): Path to the meta information file.
+ filename_tmpl (str): Template for each filename. Note that the
+ template excludes the file extension. Usually the filename_tmpl is
+ for files in the input folder.
+
+ Returns:
+ list[str]: Returned path list.
+ """
+ assert len(folders) == 2, ('The len of folders should be 2 with [input_folder, gt_folder]. '
+ f'But got {len(folders)}')
+ assert len(keys) == 2, ('The len of keys should be 2 with [input_key, gt_key]. ' f'But got {len(keys)}')
+ input_folder, gt_folder = folders
+ input_key, gt_key = keys
+
+ with open(meta_info_file, 'r') as fin:
+ gt_names = [line.split(' ')[0] for line in fin]
+
+ paths = []
+ for gt_name in gt_names:
+ basename, ext = osp.splitext(osp.basename(gt_name))
+ input_name = f'{filename_tmpl.format(basename)}{ext}'
+ input_path = osp.join(input_folder, input_name)
+ gt_path = osp.join(gt_folder, gt_name)
+ paths.append(dict([(f'{input_key}_path', input_path), (f'{gt_key}_path', gt_path)]))
+ return paths
+
+
+def paired_paths_from_folder(folders, keys, filename_tmpl):
+ """Generate paired paths from folders.
+
+ Args:
+ folders (list[str]): A list of folder path. The order of list should
+ be [input_folder, gt_folder].
+ keys (list[str]): A list of keys identifying folders. The order should
+ be in consistent with folders, e.g., ['lq', 'gt'].
+ filename_tmpl (str): Template for each filename. Note that the
+ template excludes the file extension. Usually the filename_tmpl is
+ for files in the input folder.
+
+ Returns:
+ list[str]: Returned path list.
+ """
+ assert len(folders) == 2, ('The len of folders should be 2 with [input_folder, gt_folder]. '
+ f'But got {len(folders)}')
+ assert len(keys) == 2, ('The len of keys should be 2 with [input_key, gt_key]. ' f'But got {len(keys)}')
+ input_folder, gt_folder = folders
+ input_key, gt_key = keys
+
+ input_paths = list(scandir(input_folder))
+ gt_paths = list(scandir(gt_folder))
+ assert len(input_paths) == len(gt_paths), (f'{input_key} and {gt_key} datasets have different number of images: '
+ f'{len(input_paths)}, {len(gt_paths)}.')
+ paths = []
+ for gt_path in gt_paths:
+ basename, ext = osp.splitext(osp.basename(gt_path))
+ input_name = f'{filename_tmpl.format(basename)}{ext}'
+ input_path = osp.join(input_folder, input_name)
+ assert input_name in input_paths, (f'{input_name} is not in ' f'{input_key}_paths.')
+ gt_path = osp.join(gt_folder, gt_path)
+ paths.append(dict([(f'{input_key}_path', input_path), (f'{gt_key}_path', gt_path)]))
+ return paths
+
+
+def paths_from_folder(folder):
+ """Generate paths from folder.
+
+ Args:
+ folder (str): Folder path.
+
+ Returns:
+ list[str]: Returned path list.
+ """
+
+ paths = list(scandir(folder))
+ paths = [osp.join(folder, path) for path in paths]
+ return paths
+
+
+def paths_from_lmdb(folder):
+ """Generate paths from lmdb.
+
+ Args:
+ folder (str): Folder path.
+
+ Returns:
+ list[str]: Returned path list.
+ """
+ if not folder.endswith('.lmdb'):
+ raise ValueError(f'Folder {folder}folder should in lmdb format.')
+ with open(osp.join(folder, 'meta_info.txt')) as fin:
+ paths = [line.split('.')[0] for line in fin]
+ return paths
+
+
+def generate_gaussian_kernel(kernel_size=13, sigma=1.6):
+ """Generate Gaussian kernel used in `duf_downsample`.
+
+ Args:
+ kernel_size (int): Kernel size. Default: 13.
+ sigma (float): Sigma of the Gaussian kernel. Default: 1.6.
+
+ Returns:
+ np.array: The Gaussian kernel.
+ """
+ from scipy.ndimage import filters as filters
+ kernel = np.zeros((kernel_size, kernel_size))
+ # set element at the middle to one, a dirac delta
+ kernel[kernel_size // 2, kernel_size // 2] = 1
+ # gaussian-smooth the dirac, resulting in a gaussian filter
+ return filters.gaussian_filter(kernel, sigma)
+
+
+def duf_downsample(x, kernel_size=13, scale=4):
+ """Downsamping with Gaussian kernel used in the DUF official code.
+
+ Args:
+ x (Tensor): Frames to be downsampled, with shape (b, t, c, h, w).
+ kernel_size (int): Kernel size. Default: 13.
+ scale (int): Downsampling factor. Supported scale: (2, 3, 4).
+ Default: 4.
+
+ Returns:
+ Tensor: DUF downsampled frames.
+ """
+ assert scale in (2, 3, 4), f'Only support scale (2, 3, 4), but got {scale}.'
+
+ squeeze_flag = False
+ if x.ndim == 4:
+ squeeze_flag = True
+ x = x.unsqueeze(0)
+ b, t, c, h, w = x.size()
+ x = x.view(-1, 1, h, w)
+ pad_w, pad_h = kernel_size // 2 + scale * 2, kernel_size // 2 + scale * 2
+ x = F.pad(x, (pad_w, pad_w, pad_h, pad_h), 'reflect')
+
+ gaussian_filter = generate_gaussian_kernel(kernel_size, 0.4 * scale)
+ gaussian_filter = torch.from_numpy(gaussian_filter).type_as(x).unsqueeze(0).unsqueeze(0)
+ x = F.conv2d(x, gaussian_filter, stride=scale)
+ x = x[:, :, 2:-2, 2:-2]
+ x = x.view(b, t, c, x.size(2), x.size(3))
+ if squeeze_flag:
+ x = x.squeeze(0)
+ return x
diff --git a/sd/stablediffusion/src/codeformer/basicsr/data/prefetch_dataloader.py b/sd/stablediffusion/src/codeformer/basicsr/data/prefetch_dataloader.py
new file mode 100644
index 0000000000000000000000000000000000000000..5088425050d4cc98114a9b93eb50ea60273f35a0
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/data/prefetch_dataloader.py
@@ -0,0 +1,125 @@
+import queue as Queue
+import threading
+import torch
+from torch.utils.data import DataLoader
+
+
+class PrefetchGenerator(threading.Thread):
+ """A general prefetch generator.
+
+ Ref:
+ https://stackoverflow.com/questions/7323664/python-generator-pre-fetch
+
+ Args:
+ generator: Python generator.
+ num_prefetch_queue (int): Number of prefetch queue.
+ """
+
+ def __init__(self, generator, num_prefetch_queue):
+ threading.Thread.__init__(self)
+ self.queue = Queue.Queue(num_prefetch_queue)
+ self.generator = generator
+ self.daemon = True
+ self.start()
+
+ def run(self):
+ for item in self.generator:
+ self.queue.put(item)
+ self.queue.put(None)
+
+ def __next__(self):
+ next_item = self.queue.get()
+ if next_item is None:
+ raise StopIteration
+ return next_item
+
+ def __iter__(self):
+ return self
+
+
+class PrefetchDataLoader(DataLoader):
+ """Prefetch version of dataloader.
+
+ Ref:
+ https://github.com/IgorSusmelj/pytorch-styleguide/issues/5#
+
+ TODO:
+ Need to test on single gpu and ddp (multi-gpu). There is a known issue in
+ ddp.
+
+ Args:
+ num_prefetch_queue (int): Number of prefetch queue.
+ kwargs (dict): Other arguments for dataloader.
+ """
+
+ def __init__(self, num_prefetch_queue, **kwargs):
+ self.num_prefetch_queue = num_prefetch_queue
+ super(PrefetchDataLoader, self).__init__(**kwargs)
+
+ def __iter__(self):
+ return PrefetchGenerator(super().__iter__(), self.num_prefetch_queue)
+
+
+class CPUPrefetcher():
+ """CPU prefetcher.
+
+ Args:
+ loader: Dataloader.
+ """
+
+ def __init__(self, loader):
+ self.ori_loader = loader
+ self.loader = iter(loader)
+
+ def next(self):
+ try:
+ return next(self.loader)
+ except StopIteration:
+ return None
+
+ def reset(self):
+ self.loader = iter(self.ori_loader)
+
+
+class CUDAPrefetcher():
+ """CUDA prefetcher.
+
+ Ref:
+ https://github.com/NVIDIA/apex/issues/304#
+
+ It may consums more GPU memory.
+
+ Args:
+ loader: Dataloader.
+ opt (dict): Options.
+ """
+
+ def __init__(self, loader, opt):
+ self.ori_loader = loader
+ self.loader = iter(loader)
+ self.opt = opt
+ self.stream = torch.cuda.Stream()
+ self.device = torch.device('cuda' if opt['num_gpu'] != 0 else 'cpu')
+ self.preload()
+
+ def preload(self):
+ try:
+ self.batch = next(self.loader) # self.batch is a dict
+ except StopIteration:
+ self.batch = None
+ return None
+ # put tensors to gpu
+ with torch.cuda.stream(self.stream):
+ for k, v in self.batch.items():
+ if torch.is_tensor(v):
+ self.batch[k] = self.batch[k].to(device=self.device, non_blocking=True)
+
+ def next(self):
+ torch.cuda.current_stream().wait_stream(self.stream)
+ batch = self.batch
+ self.preload()
+ return batch
+
+ def reset(self):
+ self.loader = iter(self.ori_loader)
+ self.preload()
diff --git a/sd/stablediffusion/src/codeformer/basicsr/data/transforms.py b/sd/stablediffusion/src/codeformer/basicsr/data/transforms.py
new file mode 100644
index 0000000000000000000000000000000000000000..aead9dc73ed063e1c5865040eaa2652b26aa3ad3
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/data/transforms.py
@@ -0,0 +1,165 @@
+import cv2
+import random
+
+
+def mod_crop(img, scale):
+ """Mod crop images, used during testing.
+
+ Args:
+ img (ndarray): Input image.
+ scale (int): Scale factor.
+
+ Returns:
+ ndarray: Result image.
+ """
+ img = img.copy()
+ if img.ndim in (2, 3):
+ h, w = img.shape[0], img.shape[1]
+ h_remainder, w_remainder = h % scale, w % scale
+ img = img[:h - h_remainder, :w - w_remainder, ...]
+ else:
+ raise ValueError(f'Wrong img ndim: {img.ndim}.')
+ return img
+
+
+def paired_random_crop(img_gts, img_lqs, gt_patch_size, scale, gt_path):
+ """Paired random crop.
+
+ It crops lists of lq and gt images with corresponding locations.
+
+ Args:
+ img_gts (list[ndarray] | ndarray): GT images. Note that all images
+ should have the same shape. If the input is an ndarray, it will
+ be transformed to a list containing itself.
+ img_lqs (list[ndarray] | ndarray): LQ images. Note that all images
+ should have the same shape. If the input is an ndarray, it will
+ be transformed to a list containing itself.
+ gt_patch_size (int): GT patch size.
+ scale (int): Scale factor.
+ gt_path (str): Path to ground-truth.
+
+ Returns:
+ list[ndarray] | ndarray: GT images and LQ images. If returned results
+ only have one element, just return ndarray.
+ """
+
+ if not isinstance(img_gts, list):
+ img_gts = [img_gts]
+ if not isinstance(img_lqs, list):
+ img_lqs = [img_lqs]
+
+ h_lq, w_lq, _ = img_lqs[0].shape
+ h_gt, w_gt, _ = img_gts[0].shape
+ lq_patch_size = gt_patch_size // scale
+
+ if h_gt != h_lq * scale or w_gt != w_lq * scale:
+ raise ValueError(f'Scale mismatches. GT ({h_gt}, {w_gt}) is not {scale}x ',
+ f'multiplication of LQ ({h_lq}, {w_lq}).')
+ if h_lq < lq_patch_size or w_lq < lq_patch_size:
+ raise ValueError(f'LQ ({h_lq}, {w_lq}) is smaller than patch size '
+ f'({lq_patch_size}, {lq_patch_size}). '
+ f'Please remove {gt_path}.')
+
+ # randomly choose top and left coordinates for lq patch
+ top = random.randint(0, h_lq - lq_patch_size)
+ left = random.randint(0, w_lq - lq_patch_size)
+
+ # crop lq patch
+ img_lqs = [v[top:top + lq_patch_size, left:left + lq_patch_size, ...] for v in img_lqs]
+
+ # crop corresponding gt patch
+ top_gt, left_gt = int(top * scale), int(left * scale)
+ img_gts = [v[top_gt:top_gt + gt_patch_size, left_gt:left_gt + gt_patch_size, ...] for v in img_gts]
+ if len(img_gts) == 1:
+ img_gts = img_gts[0]
+ if len(img_lqs) == 1:
+ img_lqs = img_lqs[0]
+ return img_gts, img_lqs
+
+
+def augment(imgs, hflip=True, rotation=True, flows=None, return_status=False):
+ """Augment: horizontal flips OR rotate (0, 90, 180, 270 degrees).
+
+ We use vertical flip and transpose for rotation implementation.
+ All the images in the list use the same augmentation.
+
+ Args:
+ imgs (list[ndarray] | ndarray): Images to be augmented. If the input
+ is an ndarray, it will be transformed to a list.
+ hflip (bool): Horizontal flip. Default: True.
+ rotation (bool): Ratotation. Default: True.
+ flows (list[ndarray]: Flows to be augmented. If the input is an
+ ndarray, it will be transformed to a list.
+ Dimension is (h, w, 2). Default: None.
+ return_status (bool): Return the status of flip and rotation.
+ Default: False.
+
+ Returns:
+ list[ndarray] | ndarray: Augmented images and flows. If returned
+ results only have one element, just return ndarray.
+
+ """
+ hflip = hflip and random.random() < 0.5
+ vflip = rotation and random.random() < 0.5
+ rot90 = rotation and random.random() < 0.5
+
+ def _augment(img):
+ if hflip: # horizontal
+ cv2.flip(img, 1, img)
+ if vflip: # vertical
+ cv2.flip(img, 0, img)
+ if rot90:
+ img = img.transpose(1, 0, 2)
+ return img
+
+ def _augment_flow(flow):
+ if hflip: # horizontal
+ cv2.flip(flow, 1, flow)
+ flow[:, :, 0] *= -1
+ if vflip: # vertical
+ cv2.flip(flow, 0, flow)
+ flow[:, :, 1] *= -1
+ if rot90:
+ flow = flow.transpose(1, 0, 2)
+ flow = flow[:, :, [1, 0]]
+ return flow
+
+ if not isinstance(imgs, list):
+ imgs = [imgs]
+ imgs = [_augment(img) for img in imgs]
+ if len(imgs) == 1:
+ imgs = imgs[0]
+
+ if flows is not None:
+ if not isinstance(flows, list):
+ flows = [flows]
+ flows = [_augment_flow(flow) for flow in flows]
+ if len(flows) == 1:
+ flows = flows[0]
+ return imgs, flows
+ else:
+ if return_status:
+ return imgs, (hflip, vflip, rot90)
+ else:
+ return imgs
+
+
+def img_rotate(img, angle, center=None, scale=1.0):
+ """Rotate image.
+
+ Args:
+ img (ndarray): Image to be rotated.
+ angle (float): Rotation angle in degrees. Positive values mean
+ counter-clockwise rotation.
+ center (tuple[int]): Rotation center. If the center is None,
+ initialize it as the center of the image. Default: None.
+ scale (float): Isotropic scale factor. Default: 1.0.
+ """
+ (h, w) = img.shape[:2]
+
+ if center is None:
+ center = (w // 2, h // 2)
+
+ matrix = cv2.getRotationMatrix2D(center, angle, scale)
+ rotated_img = cv2.warpAffine(img, matrix, (w, h))
+ return rotated_img
diff --git a/sd/stablediffusion/src/codeformer/basicsr/losses/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/losses/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..2b184e74c861e6fca0c548692a9a949a6100b0aa
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/losses/__init__.py
@@ -0,0 +1,26 @@
+from copy import deepcopy
+
+from basicsr.utils import get_root_logger
+from basicsr.utils.registry import LOSS_REGISTRY
+from .losses import (CharbonnierLoss, GANLoss, L1Loss, MSELoss, PerceptualLoss, WeightedTVLoss, g_path_regularize,
+ gradient_penalty_loss, r1_penalty)
+
+__all__ = [
+ 'L1Loss', 'MSELoss', 'CharbonnierLoss', 'WeightedTVLoss', 'PerceptualLoss', 'GANLoss', 'gradient_penalty_loss',
+ 'r1_penalty', 'g_path_regularize'
+]
+
+
+def build_loss(opt):
+ """Build loss from options.
+
+ Args:
+ opt (dict): Configuration. It must constain:
+ type (str): Model type.
+ """
+ opt = deepcopy(opt)
+ loss_type = opt.pop('type')
+ loss = LOSS_REGISTRY.get(loss_type)(**opt)
+ logger = get_root_logger()
+ logger.info(f'Loss [{loss.__class__.__name__}] is created.')
+ return loss
diff --git a/sd/stablediffusion/src/codeformer/basicsr/losses/loss_util.py b/sd/stablediffusion/src/codeformer/basicsr/losses/loss_util.py
new file mode 100644
index 0000000000000000000000000000000000000000..744eeb46d1f3b5a7b4553ca23237ddd9c899a698
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/losses/loss_util.py
@@ -0,0 +1,95 @@
+import functools
+from torch.nn import functional as F
+
+
+def reduce_loss(loss, reduction):
+ """Reduce loss as specified.
+
+ Args:
+ loss (Tensor): Elementwise loss tensor.
+ reduction (str): Options are 'none', 'mean' and 'sum'.
+
+ Returns:
+ Tensor: Reduced loss tensor.
+ """
+ reduction_enum = F._Reduction.get_enum(reduction)
+ # none: 0, elementwise_mean:1, sum: 2
+ if reduction_enum == 0:
+ return loss
+ elif reduction_enum == 1:
+ return loss.mean()
+ else:
+ return loss.sum()
+
+
+def weight_reduce_loss(loss, weight=None, reduction='mean'):
+ """Apply element-wise weight and reduce loss.
+
+ Args:
+ loss (Tensor): Element-wise loss.
+ weight (Tensor): Element-wise weights. Default: None.
+ reduction (str): Same as built-in losses of PyTorch. Options are
+ 'none', 'mean' and 'sum'. Default: 'mean'.
+
+ Returns:
+ Tensor: Loss values.
+ """
+ # if weight is specified, apply element-wise weight
+ if weight is not None:
+ assert weight.dim() == loss.dim()
+ assert weight.size(1) == 1 or weight.size(1) == loss.size(1)
+ loss = loss * weight
+
+ # if weight is not specified or reduction is sum, just reduce the loss
+ if weight is None or reduction == 'sum':
+ loss = reduce_loss(loss, reduction)
+ # if reduction is mean, then compute mean over weight region
+ elif reduction == 'mean':
+ if weight.size(1) > 1:
+ weight = weight.sum()
+ else:
+ weight = weight.sum() * loss.size(1)
+ loss = loss.sum() / weight
+
+ return loss
+
+
+def weighted_loss(loss_func):
+ """Create a weighted version of a given loss function.
+
+ To use this decorator, the loss function must have the signature like
+ `loss_func(pred, target, **kwargs)`. The function only needs to compute
+ element-wise loss without any reduction. This decorator will add weight
+ and reduction arguments to the function. The decorated function will have
+ the signature like `loss_func(pred, target, weight=None, reduction='mean',
+ **kwargs)`.
+
+ :Example:
+
+ >>> import torch
+ >>> @weighted_loss
+ >>> def l1_loss(pred, target):
+ >>> return (pred - target).abs()
+
+ >>> pred = torch.Tensor([0, 2, 3])
+ >>> target = torch.Tensor([1, 1, 1])
+ >>> weight = torch.Tensor([1, 0, 1])
+
+ >>> l1_loss(pred, target)
+ tensor(1.3333)
+ >>> l1_loss(pred, target, weight)
+ tensor(1.5000)
+ >>> l1_loss(pred, target, reduction='none')
+ tensor([1., 1., 2.])
+ >>> l1_loss(pred, target, weight, reduction='sum')
+ tensor(3.)
+ """
+
+ @functools.wraps(loss_func)
+ def wrapper(pred, target, weight=None, reduction='mean', **kwargs):
+ # get element-wise loss
+ loss = loss_func(pred, target, **kwargs)
+ loss = weight_reduce_loss(loss, weight, reduction)
+ return loss
+
+ return wrapper
diff --git a/sd/stablediffusion/src/codeformer/basicsr/losses/losses.py b/sd/stablediffusion/src/codeformer/basicsr/losses/losses.py
new file mode 100644
index 0000000000000000000000000000000000000000..1bcf272cfb756d99451a3005567ea4d4c9059067
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/losses/losses.py
@@ -0,0 +1,455 @@
+import math
+import lpips
+import torch
+from torch import autograd as autograd
+from torch import nn as nn
+from torch.nn import functional as F
+
+from basicsr.archs.vgg_arch import VGGFeatureExtractor
+from basicsr.utils.registry import LOSS_REGISTRY
+from .loss_util import weighted_loss
+
+_reduction_modes = ['none', 'mean', 'sum']
+
+
+@weighted_loss
+def l1_loss(pred, target):
+ return F.l1_loss(pred, target, reduction='none')
+
+
+@weighted_loss
+def mse_loss(pred, target):
+ return F.mse_loss(pred, target, reduction='none')
+
+
+@weighted_loss
+def charbonnier_loss(pred, target, eps=1e-12):
+ return torch.sqrt((pred - target)**2 + eps)
+
+
+@LOSS_REGISTRY.register()
+class L1Loss(nn.Module):
+ """L1 (mean absolute error, MAE) loss.
+
+ Args:
+ loss_weight (float): Loss weight for L1 loss. Default: 1.0.
+ reduction (str): Specifies the reduction to apply to the output.
+ Supported choices are 'none' | 'mean' | 'sum'. Default: 'mean'.
+ """
+
+ def __init__(self, loss_weight=1.0, reduction='mean'):
+ super(L1Loss, self).__init__()
+ if reduction not in ['none', 'mean', 'sum']:
+ raise ValueError(f'Unsupported reduction mode: {reduction}. ' f'Supported ones are: {_reduction_modes}')
+
+ self.loss_weight = loss_weight
+ self.reduction = reduction
+
+ def forward(self, pred, target, weight=None, **kwargs):
+ """
+ Args:
+ pred (Tensor): of shape (N, C, H, W). Predicted tensor.
+ target (Tensor): of shape (N, C, H, W). Ground truth tensor.
+ weight (Tensor, optional): of shape (N, C, H, W). Element-wise
+ weights. Default: None.
+ """
+ return self.loss_weight * l1_loss(pred, target, weight, reduction=self.reduction)
+
+
+@LOSS_REGISTRY.register()
+class MSELoss(nn.Module):
+ """MSE (L2) loss.
+
+ Args:
+ loss_weight (float): Loss weight for MSE loss. Default: 1.0.
+ reduction (str): Specifies the reduction to apply to the output.
+ Supported choices are 'none' | 'mean' | 'sum'. Default: 'mean'.
+ """
+
+ def __init__(self, loss_weight=1.0, reduction='mean'):
+ super(MSELoss, self).__init__()
+ if reduction not in ['none', 'mean', 'sum']:
+ raise ValueError(f'Unsupported reduction mode: {reduction}. ' f'Supported ones are: {_reduction_modes}')
+
+ self.loss_weight = loss_weight
+ self.reduction = reduction
+
+ def forward(self, pred, target, weight=None, **kwargs):
+ """
+ Args:
+ pred (Tensor): of shape (N, C, H, W). Predicted tensor.
+ target (Tensor): of shape (N, C, H, W). Ground truth tensor.
+ weight (Tensor, optional): of shape (N, C, H, W). Element-wise
+ weights. Default: None.
+ """
+ return self.loss_weight * mse_loss(pred, target, weight, reduction=self.reduction)
+
+
+@LOSS_REGISTRY.register()
+class CharbonnierLoss(nn.Module):
+ """Charbonnier loss (one variant of Robust L1Loss, a differentiable
+ variant of L1Loss).
+
+ Described in "Deep Laplacian Pyramid Networks for Fast and Accurate
+ Super-Resolution".
+
+ Args:
+ loss_weight (float): Loss weight for L1 loss. Default: 1.0.
+ reduction (str): Specifies the reduction to apply to the output.
+ Supported choices are 'none' | 'mean' | 'sum'. Default: 'mean'.
+ eps (float): A value used to control the curvature near zero.
+ Default: 1e-12.
+ """
+
+ def __init__(self, loss_weight=1.0, reduction='mean', eps=1e-12):
+ super(CharbonnierLoss, self).__init__()
+ if reduction not in ['none', 'mean', 'sum']:
+ raise ValueError(f'Unsupported reduction mode: {reduction}. ' f'Supported ones are: {_reduction_modes}')
+
+ self.loss_weight = loss_weight
+ self.reduction = reduction
+ self.eps = eps
+
+ def forward(self, pred, target, weight=None, **kwargs):
+ """
+ Args:
+ pred (Tensor): of shape (N, C, H, W). Predicted tensor.
+ target (Tensor): of shape (N, C, H, W). Ground truth tensor.
+ weight (Tensor, optional): of shape (N, C, H, W). Element-wise
+ weights. Default: None.
+ """
+ return self.loss_weight * charbonnier_loss(pred, target, weight, eps=self.eps, reduction=self.reduction)
+
+
+@LOSS_REGISTRY.register()
+class WeightedTVLoss(L1Loss):
+ """Weighted TV loss.
+
+ Args:
+ loss_weight (float): Loss weight. Default: 1.0.
+ """
+
+ def __init__(self, loss_weight=1.0):
+ super(WeightedTVLoss, self).__init__(loss_weight=loss_weight)
+
+ def forward(self, pred, weight=None):
+ y_diff = super(WeightedTVLoss, self).forward(pred[:, :, :-1, :], pred[:, :, 1:, :], weight=weight[:, :, :-1, :])
+ x_diff = super(WeightedTVLoss, self).forward(pred[:, :, :, :-1], pred[:, :, :, 1:], weight=weight[:, :, :, :-1])
+
+ loss = x_diff + y_diff
+
+ return loss
+
+
+@LOSS_REGISTRY.register()
+class PerceptualLoss(nn.Module):
+ """Perceptual loss with commonly used style loss.
+
+ Args:
+ layer_weights (dict): The weight for each layer of vgg feature.
+ Here is an example: {'conv5_4': 1.}, which means the conv5_4
+ feature layer (before relu5_4) will be extracted with weight
+ 1.0 in calculting losses.
+ vgg_type (str): The type of vgg network used as feature extractor.
+ Default: 'vgg19'.
+ use_input_norm (bool): If True, normalize the input image in vgg.
+ Default: True.
+ range_norm (bool): If True, norm images with range [-1, 1] to [0, 1].
+ Default: False.
+ perceptual_weight (float): If `perceptual_weight > 0`, the perceptual
+ loss will be calculated and the loss will multiplied by the
+ weight. Default: 1.0.
+ style_weight (float): If `style_weight > 0`, the style loss will be
+ calculated and the loss will multiplied by the weight.
+ Default: 0.
+ criterion (str): Criterion used for perceptual loss. Default: 'l1'.
+ """
+
+ def __init__(self,
+ layer_weights,
+ vgg_type='vgg19',
+ use_input_norm=True,
+ range_norm=False,
+ perceptual_weight=1.0,
+ style_weight=0.,
+ criterion='l1'):
+ super(PerceptualLoss, self).__init__()
+ self.perceptual_weight = perceptual_weight
+ self.style_weight = style_weight
+ self.layer_weights = layer_weights
+ self.vgg = VGGFeatureExtractor(
+ layer_name_list=list(layer_weights.keys()),
+ vgg_type=vgg_type,
+ use_input_norm=use_input_norm,
+ range_norm=range_norm)
+
+ self.criterion_type = criterion
+ if self.criterion_type == 'l1':
+ self.criterion = torch.nn.L1Loss()
+ elif self.criterion_type == 'l2':
+ self.criterion = torch.nn.L2loss()
+ elif self.criterion_type == 'mse':
+ self.criterion = torch.nn.MSELoss(reduction='mean')
+ elif self.criterion_type == 'fro':
+ self.criterion = None
+ else:
+ raise NotImplementedError(f'{criterion} criterion has not been supported.')
+
+ def forward(self, x, gt):
+ """Forward function.
+
+ Args:
+ x (Tensor): Input tensor with shape (n, c, h, w).
+ gt (Tensor): Ground-truth tensor with shape (n, c, h, w).
+
+ Returns:
+ Tensor: Forward results.
+ """
+ # extract vgg features
+ x_features = self.vgg(x)
+ gt_features = self.vgg(gt.detach())
+
+ # calculate perceptual loss
+ if self.perceptual_weight > 0:
+ percep_loss = 0
+ for k in x_features.keys():
+ if self.criterion_type == 'fro':
+ percep_loss += torch.norm(x_features[k] - gt_features[k], p='fro') * self.layer_weights[k]
+ else:
+ percep_loss += self.criterion(x_features[k], gt_features[k]) * self.layer_weights[k]
+ percep_loss *= self.perceptual_weight
+ else:
+ percep_loss = None
+
+ # calculate style loss
+ if self.style_weight > 0:
+ style_loss = 0
+ for k in x_features.keys():
+ if self.criterion_type == 'fro':
+ style_loss += torch.norm(
+ self._gram_mat(x_features[k]) - self._gram_mat(gt_features[k]), p='fro') * self.layer_weights[k]
+ else:
+ style_loss += self.criterion(self._gram_mat(x_features[k]), self._gram_mat(
+ gt_features[k])) * self.layer_weights[k]
+ style_loss *= self.style_weight
+ else:
+ style_loss = None
+
+ return percep_loss, style_loss
+
+ def _gram_mat(self, x):
+ """Calculate Gram matrix.
+
+ Args:
+ x (torch.Tensor): Tensor with shape of (n, c, h, w).
+
+ Returns:
+ torch.Tensor: Gram matrix.
+ """
+ n, c, h, w = x.size()
+ features = x.view(n, c, w * h)
+ features_t = features.transpose(1, 2)
+ gram = features.bmm(features_t) / (c * h * w)
+ return gram
+
+
+@LOSS_REGISTRY.register()
+class LPIPSLoss(nn.Module):
+ def __init__(self,
+ loss_weight=1.0,
+ use_input_norm=True,
+ range_norm=False,):
+ super(LPIPSLoss, self).__init__()
+ self.perceptual = lpips.LPIPS(net="vgg", spatial=False).eval()
+ self.loss_weight = loss_weight
+ self.use_input_norm = use_input_norm
+ self.range_norm = range_norm
+
+ if self.use_input_norm:
+ # the mean is for image with range [0, 1]
+ self.register_buffer('mean', torch.Tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
+ # the std is for image with range [0, 1]
+ self.register_buffer('std', torch.Tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
+
+ def forward(self, pred, target):
+ if self.range_norm:
+ pred = (pred + 1) / 2
+ target = (target + 1) / 2
+ if self.use_input_norm:
+ pred = (pred - self.mean) / self.std
+ target = (target - self.mean) / self.std
+ lpips_loss = self.perceptual(target.contiguous(), pred.contiguous())
+ return self.loss_weight * lpips_loss.mean()
+
+
+@LOSS_REGISTRY.register()
+class GANLoss(nn.Module):
+ """Define GAN loss.
+
+ Args:
+ gan_type (str): Support 'vanilla', 'lsgan', 'wgan', 'hinge'.
+ real_label_val (float): The value for real label. Default: 1.0.
+ fake_label_val (float): The value for fake label. Default: 0.0.
+ loss_weight (float): Loss weight. Default: 1.0.
+ Note that loss_weight is only for generators; and it is always 1.0
+ for discriminators.
+ """
+
+ def __init__(self, gan_type, real_label_val=1.0, fake_label_val=0.0, loss_weight=1.0):
+ super(GANLoss, self).__init__()
+ self.gan_type = gan_type
+ self.loss_weight = loss_weight
+ self.real_label_val = real_label_val
+ self.fake_label_val = fake_label_val
+
+ if self.gan_type == 'vanilla':
+ self.loss = nn.BCEWithLogitsLoss()
+ elif self.gan_type == 'lsgan':
+ self.loss = nn.MSELoss()
+ elif self.gan_type == 'wgan':
+ self.loss = self._wgan_loss
+ elif self.gan_type == 'wgan_softplus':
+ self.loss = self._wgan_softplus_loss
+ elif self.gan_type == 'hinge':
+ self.loss = nn.ReLU()
+ else:
+ raise NotImplementedError(f'GAN type {self.gan_type} is not implemented.')
+
+ def _wgan_loss(self, input, target):
+ """wgan loss.
+
+ Args:
+ input (Tensor): Input tensor.
+ target (bool): Target label.
+
+ Returns:
+ Tensor: wgan loss.
+ """
+ return -input.mean() if target else input.mean()
+
+ def _wgan_softplus_loss(self, input, target):
+ """wgan loss with soft plus. softplus is a smooth approximation to the
+ ReLU function.
+
+ In StyleGAN2, it is called:
+ Logistic loss for discriminator;
+ Non-saturating loss for generator.
+
+ Args:
+ input (Tensor): Input tensor.
+ target (bool): Target label.
+
+ Returns:
+ Tensor: wgan loss.
+ """
+ return F.softplus(-input).mean() if target else F.softplus(input).mean()
+
+ def get_target_label(self, input, target_is_real):
+ """Get target label.
+
+ Args:
+ input (Tensor): Input tensor.
+ target_is_real (bool): Whether the target is real or fake.
+
+ Returns:
+ (bool | Tensor): Target tensor. Return bool for wgan, otherwise,
+ return Tensor.
+ """
+
+ if self.gan_type in ['wgan', 'wgan_softplus']:
+ return target_is_real
+ target_val = (self.real_label_val if target_is_real else self.fake_label_val)
+ return input.new_ones(input.size()) * target_val
+
+ def forward(self, input, target_is_real, is_disc=False):
+ """
+ Args:
+ input (Tensor): The input for the loss module, i.e., the network
+ prediction.
+ target_is_real (bool): Whether the targe is real or fake.
+ is_disc (bool): Whether the loss for discriminators or not.
+ Default: False.
+
+ Returns:
+ Tensor: GAN loss value.
+ """
+ if self.gan_type == 'hinge':
+ if is_disc: # for discriminators in hinge-gan
+ input = -input if target_is_real else input
+ loss = self.loss(1 + input).mean()
+ else: # for generators in hinge-gan
+ loss = -input.mean()
+ else: # other gan types
+ target_label = self.get_target_label(input, target_is_real)
+ loss = self.loss(input, target_label)
+
+ # loss_weight is always 1.0 for discriminators
+ return loss if is_disc else loss * self.loss_weight
+
+
+def r1_penalty(real_pred, real_img):
+ """R1 regularization for discriminator. The core idea is to
+ penalize the gradient on real data alone: when the
+ generator distribution produces the true data distribution
+ and the discriminator is equal to 0 on the data manifold, the
+ gradient penalty ensures that the discriminator cannot create
+ a non-zero gradient orthogonal to the data manifold without
+ suffering a loss in the GAN game.
+
+ Ref:
+ Eq. 9 in Which training methods for GANs do actually converge.
+ """
+ grad_real = autograd.grad(outputs=real_pred.sum(), inputs=real_img, create_graph=True)[0]
+ grad_penalty = grad_real.pow(2).view(grad_real.shape[0], -1).sum(1).mean()
+ return grad_penalty
+
+
+def g_path_regularize(fake_img, latents, mean_path_length, decay=0.01):
+ noise = torch.randn_like(fake_img) / math.sqrt(fake_img.shape[2] * fake_img.shape[3])
+ grad = autograd.grad(outputs=(fake_img * noise).sum(), inputs=latents, create_graph=True)[0]
+ path_lengths = torch.sqrt(grad.pow(2).sum(2).mean(1))
+
+ path_mean = mean_path_length + decay * (path_lengths.mean() - mean_path_length)
+
+ path_penalty = (path_lengths - path_mean).pow(2).mean()
+
+ return path_penalty, path_lengths.detach().mean(), path_mean.detach()
+
+
+def gradient_penalty_loss(discriminator, real_data, fake_data, weight=None):
+ """Calculate gradient penalty for wgan-gp.
+
+ Args:
+ discriminator (nn.Module): Network for the discriminator.
+ real_data (Tensor): Real input data.
+ fake_data (Tensor): Fake input data.
+ weight (Tensor): Weight tensor. Default: None.
+
+ Returns:
+ Tensor: A tensor for gradient penalty.
+ """
+
+ batch_size = real_data.size(0)
+ alpha = real_data.new_tensor(torch.rand(batch_size, 1, 1, 1))
+
+ # interpolate between real_data and fake_data
+ interpolates = alpha * real_data + (1. - alpha) * fake_data
+ interpolates = autograd.Variable(interpolates, requires_grad=True)
+
+ disc_interpolates = discriminator(interpolates)
+ gradients = autograd.grad(
+ outputs=disc_interpolates,
+ inputs=interpolates,
+ grad_outputs=torch.ones_like(disc_interpolates),
+ create_graph=True,
+ retain_graph=True,
+ only_inputs=True)[0]
+
+ if weight is not None:
+ gradients = gradients * weight
+
+ gradients_penalty = ((gradients.norm(2, dim=1) - 1)**2).mean()
+ if weight is not None:
+ gradients_penalty /= torch.mean(weight)
+
+ return gradients_penalty
diff --git a/sd/stablediffusion/src/codeformer/basicsr/metrics/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/metrics/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..19d55cc8321f124c918d78465b053aef67f13a33
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/metrics/__init__.py
@@ -0,0 +1,19 @@
+from copy import deepcopy
+
+from basicsr.utils.registry import METRIC_REGISTRY
+from .psnr_ssim import calculate_psnr, calculate_ssim
+
+__all__ = ['calculate_psnr', 'calculate_ssim']
+
+
+def calculate_metric(data, opt):
+ """Calculate metric from data and options.
+
+ Args:
+ opt (dict): Configuration. It must constain:
+ type (str): Model type.
+ """
+ opt = deepcopy(opt)
+ metric_type = opt.pop('type')
+ metric = METRIC_REGISTRY.get(metric_type)(**data, **opt)
+ return metric
diff --git a/sd/stablediffusion/src/codeformer/basicsr/metrics/metric_util.py b/sd/stablediffusion/src/codeformer/basicsr/metrics/metric_util.py
new file mode 100644
index 0000000000000000000000000000000000000000..4d18f0f7816431bed6af9d58319c6435bdf5c971
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/metrics/metric_util.py
@@ -0,0 +1,45 @@
+import numpy as np
+
+from basicsr.utils.matlab_functions import bgr2ycbcr
+
+
+def reorder_image(img, input_order='HWC'):
+ """Reorder images to 'HWC' order.
+
+ If the input_order is (h, w), return (h, w, 1);
+ If the input_order is (c, h, w), return (h, w, c);
+ If the input_order is (h, w, c), return as it is.
+
+ Args:
+ img (ndarray): Input image.
+ input_order (str): Whether the input order is 'HWC' or 'CHW'.
+ If the input image shape is (h, w), input_order will not have
+ effects. Default: 'HWC'.
+
+ Returns:
+ ndarray: reordered image.
+ """
+
+ if input_order not in ['HWC', 'CHW']:
+ raise ValueError(f'Wrong input_order {input_order}. Supported input_orders are ' "'HWC' and 'CHW'")
+ if len(img.shape) == 2:
+ img = img[..., None]
+ if input_order == 'CHW':
+ img = img.transpose(1, 2, 0)
+ return img
+
+
+def to_y_channel(img):
+ """Change to Y channel of YCbCr.
+
+ Args:
+ img (ndarray): Images with range [0, 255].
+
+ Returns:
+ (ndarray): Images with range [0, 255] (float type) without round.
+ """
+ img = img.astype(np.float32) / 255.
+ if img.ndim == 3 and img.shape[2] == 3:
+ img = bgr2ycbcr(img, y_only=True)
+ img = img[..., None]
+ return img * 255.
diff --git a/sd/stablediffusion/src/codeformer/basicsr/metrics/psnr_ssim.py b/sd/stablediffusion/src/codeformer/basicsr/metrics/psnr_ssim.py
new file mode 100644
index 0000000000000000000000000000000000000000..bbd950699c2495880236883861d9e199f900eae8
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/metrics/psnr_ssim.py
@@ -0,0 +1,128 @@
+import cv2
+import numpy as np
+
+from basicsr.metrics.metric_util import reorder_image, to_y_channel
+from basicsr.utils.registry import METRIC_REGISTRY
+
+
+@METRIC_REGISTRY.register()
+def calculate_psnr(img1, img2, crop_border, input_order='HWC', test_y_channel=False):
+ """Calculate PSNR (Peak Signal-to-Noise Ratio).
+
+ Ref: https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio
+
+ Args:
+ img1 (ndarray): Images with range [0, 255].
+ img2 (ndarray): Images with range [0, 255].
+ crop_border (int): Cropped pixels in each edge of an image. These
+ pixels are not involved in the PSNR calculation.
+ input_order (str): Whether the input order is 'HWC' or 'CHW'.
+ Default: 'HWC'.
+ test_y_channel (bool): Test on Y channel of YCbCr. Default: False.
+
+ Returns:
+ float: psnr result.
+ """
+
+ assert img1.shape == img2.shape, (f'Image shapes are differnet: {img1.shape}, {img2.shape}.')
+ if input_order not in ['HWC', 'CHW']:
+ raise ValueError(f'Wrong input_order {input_order}. Supported input_orders are ' '"HWC" and "CHW"')
+ img1 = reorder_image(img1, input_order=input_order)
+ img2 = reorder_image(img2, input_order=input_order)
+ img1 = img1.astype(np.float64)
+ img2 = img2.astype(np.float64)
+
+ if crop_border != 0:
+ img1 = img1[crop_border:-crop_border, crop_border:-crop_border, ...]
+ img2 = img2[crop_border:-crop_border, crop_border:-crop_border, ...]
+
+ if test_y_channel:
+ img1 = to_y_channel(img1)
+ img2 = to_y_channel(img2)
+
+ mse = np.mean((img1 - img2)**2)
+ if mse == 0:
+ return float('inf')
+ return 20. * np.log10(255. / np.sqrt(mse))
+
+
+def _ssim(img1, img2):
+ """Calculate SSIM (structural similarity) for one channel images.
+
+ It is called by func:`calculate_ssim`.
+
+ Args:
+ img1 (ndarray): Images with range [0, 255] with order 'HWC'.
+ img2 (ndarray): Images with range [0, 255] with order 'HWC'.
+
+ Returns:
+ float: ssim result.
+ """
+
+ C1 = (0.01 * 255)**2
+ C2 = (0.03 * 255)**2
+
+ img1 = img1.astype(np.float64)
+ img2 = img2.astype(np.float64)
+ kernel = cv2.getGaussianKernel(11, 1.5)
+ window = np.outer(kernel, kernel.transpose())
+
+ mu1 = cv2.filter2D(img1, -1, window)[5:-5, 5:-5]
+ mu2 = cv2.filter2D(img2, -1, window)[5:-5, 5:-5]
+ mu1_sq = mu1**2
+ mu2_sq = mu2**2
+ mu1_mu2 = mu1 * mu2
+ sigma1_sq = cv2.filter2D(img1**2, -1, window)[5:-5, 5:-5] - mu1_sq
+ sigma2_sq = cv2.filter2D(img2**2, -1, window)[5:-5, 5:-5] - mu2_sq
+ sigma12 = cv2.filter2D(img1 * img2, -1, window)[5:-5, 5:-5] - mu1_mu2
+
+ ssim_map = ((2 * mu1_mu2 + C1) * (2 * sigma12 + C2)) / ((mu1_sq + mu2_sq + C1) * (sigma1_sq + sigma2_sq + C2))
+ return ssim_map.mean()
+
+
+@METRIC_REGISTRY.register()
+def calculate_ssim(img1, img2, crop_border, input_order='HWC', test_y_channel=False):
+ """Calculate SSIM (structural similarity).
+
+ Ref:
+ Image quality assessment: From error visibility to structural similarity
+
+ The results are the same as that of the official released MATLAB code in
+ https://ece.uwaterloo.ca/~z70wang/research/ssim/.
+
+ For three-channel images, SSIM is calculated for each channel and then
+ averaged.
+
+ Args:
+ img1 (ndarray): Images with range [0, 255].
+ img2 (ndarray): Images with range [0, 255].
+ crop_border (int): Cropped pixels in each edge of an image. These
+ pixels are not involved in the SSIM calculation.
+ input_order (str): Whether the input order is 'HWC' or 'CHW'.
+ Default: 'HWC'.
+ test_y_channel (bool): Test on Y channel of YCbCr. Default: False.
+
+ Returns:
+ float: ssim result.
+ """
+
+ assert img1.shape == img2.shape, (f'Image shapes are differnet: {img1.shape}, {img2.shape}.')
+ if input_order not in ['HWC', 'CHW']:
+ raise ValueError(f'Wrong input_order {input_order}. Supported input_orders are ' '"HWC" and "CHW"')
+ img1 = reorder_image(img1, input_order=input_order)
+ img2 = reorder_image(img2, input_order=input_order)
+ img1 = img1.astype(np.float64)
+ img2 = img2.astype(np.float64)
+
+ if crop_border != 0:
+ img1 = img1[crop_border:-crop_border, crop_border:-crop_border, ...]
+ img2 = img2[crop_border:-crop_border, crop_border:-crop_border, ...]
+
+ if test_y_channel:
+ img1 = to_y_channel(img1)
+ img2 = to_y_channel(img2)
+
+ ssims = []
+ for i in range(img1.shape[2]):
+ ssims.append(_ssim(img1[..., i], img2[..., i]))
+ return np.array(ssims).mean()
diff --git a/sd/stablediffusion/src/codeformer/basicsr/models/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/models/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..00bde45f003698a5b15d3517ae47b59ef1d86e0c
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/models/__init__.py
@@ -0,0 +1,30 @@
+import importlib
+from copy import deepcopy
+from os import path as osp
+
+from basicsr.utils import get_root_logger, scandir
+from basicsr.utils.registry import MODEL_REGISTRY
+
+__all__ = ['build_model']
+
+# automatically scan and import model modules for registry
+# scan all the files under the 'models' folder and collect files ending with
+# '_model.py'
+model_folder = osp.dirname(osp.abspath(__file__))
+model_filenames = [osp.splitext(osp.basename(v))[0] for v in scandir(model_folder) if v.endswith('_model.py')]
+# import all the model modules
+_model_modules = [importlib.import_module(f'basicsr.models.{file_name}') for file_name in model_filenames]
+
+
+def build_model(opt):
+ """Build model from options.
+
+ Args:
+ opt (dict): Configuration. It must constain:
+ model_type (str): Model type.
+ """
+ opt = deepcopy(opt)
+ model = MODEL_REGISTRY.get(opt['model_type'])(opt)
+ logger = get_root_logger()
+ logger.info(f'Model [{model.__class__.__name__}] is created.')
+ return model
diff --git a/sd/stablediffusion/src/codeformer/basicsr/ops/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/ops/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
diff --git a/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/__init__.py b/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/__init__.py
new file mode 100644
index 0000000000000000000000000000000000000000..32e3592f896d61b4127e09d0476381b9d55e32ff
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/__init__.py
@@ -0,0 +1,7 @@
+from .deform_conv import (DeformConv, DeformConvPack, ModulatedDeformConv, ModulatedDeformConvPack, deform_conv,
+ modulated_deform_conv)
+
+__all__ = [
+ 'DeformConv', 'DeformConvPack', 'ModulatedDeformConv', 'ModulatedDeformConvPack', 'deform_conv',
+ 'modulated_deform_conv'
+]
diff --git a/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/deform_conv.py b/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/deform_conv.py
new file mode 100644
index 0000000000000000000000000000000000000000..734154f9ed9447d585eae7df6886acb136f8a3cf
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/deform_conv.py
@@ -0,0 +1,377 @@
+import math
+import torch
+from torch import nn as nn
+from torch.autograd import Function
+from torch.autograd.function import once_differentiable
+from torch.nn import functional as F
+from torch.nn.modules.utils import _pair, _single
+
+try:
+ from . import deform_conv_ext
+except ImportError:
+ import os
+ BASICSR_JIT = os.getenv('BASICSR_JIT')
+ if BASICSR_JIT == 'True':
+ from torch.utils.cpp_extension import load
+ module_path = os.path.dirname(__file__)
+ deform_conv_ext = load(
+ 'deform_conv',
+ sources=[
+ os.path.join(module_path, 'src', 'deform_conv_ext.cpp'),
+ os.path.join(module_path, 'src', 'deform_conv_cuda.cpp'),
+ os.path.join(module_path, 'src', 'deform_conv_cuda_kernel.cu'),
+ ],
+ )
+
+
+class DeformConvFunction(Function):
+
+ @staticmethod
+ def forward(ctx,
+ input,
+ offset,
+ weight,
+ stride=1,
+ padding=0,
+ dilation=1,
+ groups=1,
+ deformable_groups=1,
+ im2col_step=64):
+ if input is not None and input.dim() != 4:
+ raise ValueError(f'Expected 4D tensor as input, got {input.dim()}' 'D tensor instead.')
+ ctx.stride = _pair(stride)
+ ctx.padding = _pair(padding)
+ ctx.dilation = _pair(dilation)
+ ctx.groups = groups
+ ctx.deformable_groups = deformable_groups
+ ctx.im2col_step = im2col_step
+
+ ctx.save_for_backward(input, offset, weight)
+
+ output = input.new_empty(DeformConvFunction._output_size(input, weight, ctx.padding, ctx.dilation, ctx.stride))
+
+ ctx.bufs_ = [input.new_empty(0), input.new_empty(0)] # columns, ones
+
+ if not input.is_cuda:
+ raise NotImplementedError
+ else:
+ cur_im2col_step = min(ctx.im2col_step, input.shape[0])
+ assert (input.shape[0] % cur_im2col_step) == 0, 'im2col step must divide batchsize'
+ deform_conv_ext.deform_conv_forward(input, weight,
+ offset, output, ctx.bufs_[0], ctx.bufs_[1], weight.size(3),
+ weight.size(2), ctx.stride[1], ctx.stride[0], ctx.padding[1],
+ ctx.padding[0], ctx.dilation[1], ctx.dilation[0], ctx.groups,
+ ctx.deformable_groups, cur_im2col_step)
+ return output
+
+ @staticmethod
+ @once_differentiable
+ def backward(ctx, grad_output):
+ input, offset, weight = ctx.saved_tensors
+
+ grad_input = grad_offset = grad_weight = None
+
+ if not grad_output.is_cuda:
+ raise NotImplementedError
+ else:
+ cur_im2col_step = min(ctx.im2col_step, input.shape[0])
+ assert (input.shape[0] % cur_im2col_step) == 0, 'im2col step must divide batchsize'
+
+ if ctx.needs_input_grad[0] or ctx.needs_input_grad[1]:
+ grad_input = torch.zeros_like(input)
+ grad_offset = torch.zeros_like(offset)
+ deform_conv_ext.deform_conv_backward_input(input, offset, grad_output, grad_input,
+ grad_offset, weight, ctx.bufs_[0], weight.size(3),
+ weight.size(2), ctx.stride[1], ctx.stride[0], ctx.padding[1],
+ ctx.padding[0], ctx.dilation[1], ctx.dilation[0], ctx.groups,
+ ctx.deformable_groups, cur_im2col_step)
+
+ if ctx.needs_input_grad[2]:
+ grad_weight = torch.zeros_like(weight)
+ deform_conv_ext.deform_conv_backward_parameters(input, offset, grad_output, grad_weight,
+ ctx.bufs_[0], ctx.bufs_[1], weight.size(3),
+ weight.size(2), ctx.stride[1], ctx.stride[0],
+ ctx.padding[1], ctx.padding[0], ctx.dilation[1],
+ ctx.dilation[0], ctx.groups, ctx.deformable_groups, 1,
+ cur_im2col_step)
+
+ return (grad_input, grad_offset, grad_weight, None, None, None, None, None)
+
+ @staticmethod
+ def _output_size(input, weight, padding, dilation, stride):
+ channels = weight.size(0)
+ output_size = (input.size(0), channels)
+ for d in range(input.dim() - 2):
+ in_size = input.size(d + 2)
+ pad = padding[d]
+ kernel = dilation[d] * (weight.size(d + 2) - 1) + 1
+ stride_ = stride[d]
+ output_size += ((in_size + (2 * pad) - kernel) // stride_ + 1, )
+ if not all(map(lambda s: s > 0, output_size)):
+ raise ValueError('convolution input is too small (output would be ' f'{"x".join(map(str, output_size))})')
+ return output_size
+
+
+class ModulatedDeformConvFunction(Function):
+
+ @staticmethod
+ def forward(ctx,
+ input,
+ offset,
+ mask,
+ weight,
+ bias=None,
+ stride=1,
+ padding=0,
+ dilation=1,
+ groups=1,
+ deformable_groups=1):
+ ctx.stride = stride
+ ctx.padding = padding
+ ctx.dilation = dilation
+ ctx.groups = groups
+ ctx.deformable_groups = deformable_groups
+ ctx.with_bias = bias is not None
+ if not ctx.with_bias:
+ bias = input.new_empty(1) # fake tensor
+ if not input.is_cuda:
+ raise NotImplementedError
+ if weight.requires_grad or mask.requires_grad or offset.requires_grad \
+ or input.requires_grad:
+ ctx.save_for_backward(input, offset, mask, weight, bias)
+ output = input.new_empty(ModulatedDeformConvFunction._infer_shape(ctx, input, weight))
+ ctx._bufs = [input.new_empty(0), input.new_empty(0)]
+ deform_conv_ext.modulated_deform_conv_forward(input, weight, bias, ctx._bufs[0], offset, mask, output,
+ ctx._bufs[1], weight.shape[2], weight.shape[3], ctx.stride,
+ ctx.stride, ctx.padding, ctx.padding, ctx.dilation, ctx.dilation,
+ ctx.groups, ctx.deformable_groups, ctx.with_bias)
+ return output
+
+ @staticmethod
+ @once_differentiable
+ def backward(ctx, grad_output):
+ if not grad_output.is_cuda:
+ raise NotImplementedError
+ input, offset, mask, weight, bias = ctx.saved_tensors
+ grad_input = torch.zeros_like(input)
+ grad_offset = torch.zeros_like(offset)
+ grad_mask = torch.zeros_like(mask)
+ grad_weight = torch.zeros_like(weight)
+ grad_bias = torch.zeros_like(bias)
+ deform_conv_ext.modulated_deform_conv_backward(input, weight, bias, ctx._bufs[0], offset, mask, ctx._bufs[1],
+ grad_input, grad_weight, grad_bias, grad_offset, grad_mask,
+ grad_output, weight.shape[2], weight.shape[3], ctx.stride,
+ ctx.stride, ctx.padding, ctx.padding, ctx.dilation, ctx.dilation,
+ ctx.groups, ctx.deformable_groups, ctx.with_bias)
+ if not ctx.with_bias:
+ grad_bias = None
+
+ return (grad_input, grad_offset, grad_mask, grad_weight, grad_bias, None, None, None, None, None)
+
+ @staticmethod
+ def _infer_shape(ctx, input, weight):
+ n = input.size(0)
+ channels_out = weight.size(0)
+ height, width = input.shape[2:4]
+ kernel_h, kernel_w = weight.shape[2:4]
+ height_out = (height + 2 * ctx.padding - (ctx.dilation * (kernel_h - 1) + 1)) // ctx.stride + 1
+ width_out = (width + 2 * ctx.padding - (ctx.dilation * (kernel_w - 1) + 1)) // ctx.stride + 1
+ return n, channels_out, height_out, width_out
+
+
+deform_conv = DeformConvFunction.apply
+modulated_deform_conv = ModulatedDeformConvFunction.apply
+
+
+class DeformConv(nn.Module):
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ kernel_size,
+ stride=1,
+ padding=0,
+ dilation=1,
+ groups=1,
+ deformable_groups=1,
+ bias=False):
+ super(DeformConv, self).__init__()
+
+ assert not bias
+ assert in_channels % groups == 0, \
+ f'in_channels {in_channels} is not divisible by groups {groups}'
+ assert out_channels % groups == 0, \
+ f'out_channels {out_channels} is not divisible ' \
+ f'by groups {groups}'
+
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.kernel_size = _pair(kernel_size)
+ self.stride = _pair(stride)
+ self.padding = _pair(padding)
+ self.dilation = _pair(dilation)
+ self.groups = groups
+ self.deformable_groups = deformable_groups
+ # enable compatibility with nn.Conv2d
+ self.transposed = False
+ self.output_padding = _single(0)
+
+ self.weight = nn.Parameter(torch.Tensor(out_channels, in_channels // self.groups, *self.kernel_size))
+
+ self.reset_parameters()
+
+ def reset_parameters(self):
+ n = self.in_channels
+ for k in self.kernel_size:
+ n *= k
+ stdv = 1. / math.sqrt(n)
+ self.weight.data.uniform_(-stdv, stdv)
+
+ def forward(self, x, offset):
+ # To fix an assert error in deform_conv_cuda.cpp:128
+ # input image is smaller than kernel
+ input_pad = (x.size(2) < self.kernel_size[0] or x.size(3) < self.kernel_size[1])
+ if input_pad:
+ pad_h = max(self.kernel_size[0] - x.size(2), 0)
+ pad_w = max(self.kernel_size[1] - x.size(3), 0)
+ x = F.pad(x, (0, pad_w, 0, pad_h), 'constant', 0).contiguous()
+ offset = F.pad(offset, (0, pad_w, 0, pad_h), 'constant', 0).contiguous()
+ out = deform_conv(x, offset, self.weight, self.stride, self.padding, self.dilation, self.groups,
+ self.deformable_groups)
+ if input_pad:
+ out = out[:, :, :out.size(2) - pad_h, :out.size(3) - pad_w].contiguous()
+ return out
+
+
+class DeformConvPack(DeformConv):
+ """A Deformable Conv Encapsulation that acts as normal Conv layers.
+
+ Args:
+ in_channels (int): Same as nn.Conv2d.
+ out_channels (int): Same as nn.Conv2d.
+ kernel_size (int or tuple[int]): Same as nn.Conv2d.
+ stride (int or tuple[int]): Same as nn.Conv2d.
+ padding (int or tuple[int]): Same as nn.Conv2d.
+ dilation (int or tuple[int]): Same as nn.Conv2d.
+ groups (int): Same as nn.Conv2d.
+ bias (bool or str): If specified as `auto`, it will be decided by the
+ norm_cfg. Bias will be set as True if norm_cfg is None, otherwise
+ False.
+ """
+
+ _version = 2
+
+ def __init__(self, *args, **kwargs):
+ super(DeformConvPack, self).__init__(*args, **kwargs)
+
+ self.conv_offset = nn.Conv2d(
+ self.in_channels,
+ self.deformable_groups * 2 * self.kernel_size[0] * self.kernel_size[1],
+ kernel_size=self.kernel_size,
+ stride=_pair(self.stride),
+ padding=_pair(self.padding),
+ dilation=_pair(self.dilation),
+ bias=True)
+ self.init_offset()
+
+ def init_offset(self):
+ self.conv_offset.weight.data.zero_()
+ self.conv_offset.bias.data.zero_()
+
+ def forward(self, x):
+ offset = self.conv_offset(x)
+ return deform_conv(x, offset, self.weight, self.stride, self.padding, self.dilation, self.groups,
+ self.deformable_groups)
+
+
+class ModulatedDeformConv(nn.Module):
+
+ def __init__(self,
+ in_channels,
+ out_channels,
+ kernel_size,
+ stride=1,
+ padding=0,
+ dilation=1,
+ groups=1,
+ deformable_groups=1,
+ bias=True):
+ super(ModulatedDeformConv, self).__init__()
+ self.in_channels = in_channels
+ self.out_channels = out_channels
+ self.kernel_size = _pair(kernel_size)
+ self.stride = stride
+ self.padding = padding
+ self.dilation = dilation
+ self.groups = groups
+ self.deformable_groups = deformable_groups
+ self.with_bias = bias
+ # enable compatibility with nn.Conv2d
+ self.transposed = False
+ self.output_padding = _single(0)
+
+ self.weight = nn.Parameter(torch.Tensor(out_channels, in_channels // groups, *self.kernel_size))
+ if bias:
+ self.bias = nn.Parameter(torch.Tensor(out_channels))
+ else:
+ self.register_parameter('bias', None)
+ self.init_weights()
+
+ def init_weights(self):
+ n = self.in_channels
+ for k in self.kernel_size:
+ n *= k
+ stdv = 1. / math.sqrt(n)
+ self.weight.data.uniform_(-stdv, stdv)
+ if self.bias is not None:
+ self.bias.data.zero_()
+
+ def forward(self, x, offset, mask):
+ return modulated_deform_conv(x, offset, mask, self.weight, self.bias, self.stride, self.padding, self.dilation,
+ self.groups, self.deformable_groups)
+
+
+class ModulatedDeformConvPack(ModulatedDeformConv):
+ """A ModulatedDeformable Conv Encapsulation that acts as normal Conv layers.
+
+ Args:
+ in_channels (int): Same as nn.Conv2d.
+ out_channels (int): Same as nn.Conv2d.
+ kernel_size (int or tuple[int]): Same as nn.Conv2d.
+ stride (int or tuple[int]): Same as nn.Conv2d.
+ padding (int or tuple[int]): Same as nn.Conv2d.
+ dilation (int or tuple[int]): Same as nn.Conv2d.
+ groups (int): Same as nn.Conv2d.
+ bias (bool or str): If specified as `auto`, it will be decided by the
+ norm_cfg. Bias will be set as True if norm_cfg is None, otherwise
+ False.
+ """
+
+ _version = 2
+
+ def __init__(self, *args, **kwargs):
+ super(ModulatedDeformConvPack, self).__init__(*args, **kwargs)
+
+ self.conv_offset = nn.Conv2d(
+ self.in_channels,
+ self.deformable_groups * 3 * self.kernel_size[0] * self.kernel_size[1],
+ kernel_size=self.kernel_size,
+ stride=_pair(self.stride),
+ padding=_pair(self.padding),
+ dilation=_pair(self.dilation),
+ bias=True)
+ self.init_weights()
+
+ def init_weights(self):
+ super(ModulatedDeformConvPack, self).init_weights()
+ if hasattr(self, 'conv_offset'):
+ self.conv_offset.weight.data.zero_()
+ self.conv_offset.bias.data.zero_()
+
+ def forward(self, x):
+ out = self.conv_offset(x)
+ o1, o2, mask = torch.chunk(out, 3, dim=1)
+ offset = torch.cat((o1, o2), dim=1)
+ mask = torch.sigmoid(mask)
+ return modulated_deform_conv(x, offset, mask, self.weight, self.bias, self.stride, self.padding, self.dilation,
+ self.groups, self.deformable_groups)
diff --git a/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/src/deform_conv_cuda.cpp b/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/src/deform_conv_cuda.cpp
new file mode 100644
index 0000000000000000000000000000000000000000..5d9424908ed2dbd4ac3cdb98d13e09287a4d2f2d
--- /dev/null
+++ b/sd/stablediffusion/src/codeformer/basicsr/ops/dcn/src/deform_conv_cuda.cpp
@@ -0,0 +1,685 @@
+// modify from
+// https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/blob/mmdetection/mmdet/ops/dcn/src/deform_conv_cuda.c
+
+#include 
+🔥 CodeFormer is a robust face restoration algorithm for old photos or AI-generated faces.
+🤗 Try CodeFormer for improved stable-diffusion generation!
+""" +article = r""" +If CodeFormer is helpful, please help to ⭐ the Github Repo. Thanks! +[](https://github.com/sczhou/CodeFormer) + +--- + +📝 **Citation** + +If our work is useful for your research, please consider citing: +```bibtex +@inproceedings{zhou2022codeformer, + author = {Zhou, Shangchen and Chan, Kelvin C.K. and Li, Chongyi and Loy, Chen Change}, + title = {Towards Robust Blind Face Restoration with Codebook Lookup TransFormer}, + booktitle = {NeurIPS}, + year = {2022} +} +``` + +📋 **License** + +This project is licensed under S-Lab License 1.0. +Redistribution and use for non-commercial purposes should follow this license. + +📧 **Contact** + +If you have any questions, please feel free to reach me out at shangchenzhou@gmail.com. + +
+ 🤗 Find Me:
+ 
+ 
+
+
+
+
++[Patrick Esser](https://github.com/pesser)\*, +[Robin Rombach](https://github.com/rromb)\*, +[Björn Ommer](https://hci.iwr.uni-heidelberg.de/Staff/bommer)
+\* equal contribution + +**tl;dr** We combine the efficiancy of convolutional approaches with the expressivity of transformers by introducing a convolutional VQGAN, which learns a codebook of context-rich visual parts, whose composition is modeled with an autoregressive transformer. + + +[arXiv](https://arxiv.org/abs/2012.09841) | [BibTeX](#bibtex) | [Project Page](https://compvis.github.io/taming-transformers/) + + +### News +#### 2022 +- More pretrained VQGANs (e.g. a f8-model with only 256 codebook entries) are available in our new work on [Latent Diffusion Models](https://github.com/CompVis/latent-diffusion). +- Added scene synthesis models as proposed in the paper [High-Resolution Complex Scene Synthesis with Transformers](https://arxiv.org/abs/2105.06458), see [this section](#scene-image-synthesis). +#### 2021 +- Thanks to [rom1504](https://github.com/rom1504) it is now easy to [train a VQGAN on your own datasets](#training-on-custom-data). +- Included a bugfix for the quantizer. For backward compatibility it is + disabled by default (which corresponds to always training with `beta=1.0`). + Use `legacy=False` in the quantizer config to enable it. + Thanks [richcmwang](https://github.com/richcmwang) and [wcshin-git](https://github.com/wcshin-git)! +- Our paper received an update: See https://arxiv.org/abs/2012.09841v3 and the corresponding changelog. +- Added a pretrained, [1.4B transformer model](https://k00.fr/s511rwcv) trained for class-conditional ImageNet synthesis, which obtains state-of-the-art FID scores among autoregressive approaches and outperforms BigGAN. +- Added pretrained, unconditional models on [FFHQ](https://k00.fr/yndvfu95) and [CelebA-HQ](https://k00.fr/2xkmielf). +- Added accelerated sampling via caching of keys/values in the self-attention operation, used in `scripts/sample_fast.py`. +- Added a checkpoint of a [VQGAN](https://heibox.uni-heidelberg.de/d/2e5662443a6b4307b470/) trained with f8 compression and Gumbel-Quantization. + See also our updated [reconstruction notebook](https://colab.research.google.com/github/CompVis/taming-transformers/blob/master/scripts/reconstruction_usage.ipynb). +- We added a [colab notebook](https://colab.research.google.com/github/CompVis/taming-transformers/blob/master/scripts/reconstruction_usage.ipynb) which compares two VQGANs and OpenAI's [DALL-E](https://github.com/openai/DALL-E). See also [this section](#more-resources). +- We now include an overview of pretrained models in [Tab.1](#overview-of-pretrained-models). We added models for [COCO](#coco) and [ADE20k](#ade20k). +- The streamlit demo now supports image completions. +- We now include a couple of examples from the D-RIN dataset so you can run the + [D-RIN demo](#d-rin) without preparing the dataset first. +- You can now jump right into sampling with our [Colab quickstart notebook](https://colab.research.google.com/github/CompVis/taming-transformers/blob/master/scripts/taming-transformers.ipynb). + +## Requirements +A suitable [conda](https://conda.io/) environment named `taming` can be created +and activated with: + +``` +conda env create -f environment.yaml +conda activate taming +``` +## Overview of pretrained models +The following table provides an overview of all models that are currently available. +FID scores were evaluated using [torch-fidelity](https://github.com/toshas/torch-fidelity). +For reference, we also include a link to the recently released autoencoder of the [DALL-E](https://github.com/openai/DALL-E) model. +See the corresponding [colab +notebook](https://colab.research.google.com/github/CompVis/taming-transformers/blob/master/scripts/reconstruction_usage.ipynb) +for a comparison and discussion of reconstruction capabilities. + +| Dataset | FID vs train | FID vs val | Link | Samples (256x256) | Comments +| ------------- | ------------- | ------------- |------------- | ------------- |------------- | +| FFHQ (f=16) | 9.6 | -- | [ffhq_transformer](https://k00.fr/yndvfu95) | [ffhq_samples](https://k00.fr/j626x093) | +| CelebA-HQ (f=16) | 10.2 | -- | [celebahq_transformer](https://k00.fr/2xkmielf) | [celebahq_samples](https://k00.fr/j626x093) | +| ADE20K (f=16) | -- | 35.5 | [ade20k_transformer](https://k00.fr/ot46cksa) | [ade20k_samples.zip](https://heibox.uni-heidelberg.de/f/70bb78cbaf844501b8fb/) [2k] | evaluated on val split (2k images) +| COCO-Stuff (f=16) | -- | 20.4 | [coco_transformer](https://k00.fr/2zz6i2ce) | [coco_samples.zip](https://heibox.uni-heidelberg.de/f/a395a9be612f4a7a8054/) [5k] | evaluated on val split (5k images) +| ImageNet (cIN) (f=16) | 15.98/15.78/6.59/5.88/5.20 | -- | [cin_transformer](https://k00.fr/s511rwcv) | [cin_samples](https://k00.fr/j626x093) | different decoding hyperparameters | +| | | | || | +| FacesHQ (f=16) | -- | -- | [faceshq_transformer](https://k00.fr/qqfl2do8) +| S-FLCKR (f=16) | -- | -- | [sflckr](https://heibox.uni-heidelberg.de/d/73487ab6e5314cb5adba/) +| D-RIN (f=16) | -- | -- | [drin_transformer](https://k00.fr/39jcugc5) +| | | | | || | +| VQGAN ImageNet (f=16), 1024 | 10.54 | 7.94 | [vqgan_imagenet_f16_1024](https://heibox.uni-heidelberg.de/d/8088892a516d4e3baf92/) | [reconstructions](https://k00.fr/j626x093) | Reconstruction-FIDs. +| VQGAN ImageNet (f=16), 16384 | 7.41 | 4.98 |[vqgan_imagenet_f16_16384](https://heibox.uni-heidelberg.de/d/a7530b09fed84f80a887/) | [reconstructions](https://k00.fr/j626x093) | Reconstruction-FIDs. +| VQGAN OpenImages (f=8), 256 | -- | 1.49 |https://ommer-lab.com/files/latent-diffusion/vq-f8-n256.zip | --- | Reconstruction-FIDs. Available via [latent diffusion](https://github.com/CompVis/latent-diffusion). +| VQGAN OpenImages (f=8), 16384 | -- | 1.14 |https://ommer-lab.com/files/latent-diffusion/vq-f8.zip | --- | Reconstruction-FIDs. Available via [latent diffusion](https://github.com/CompVis/latent-diffusion) +| VQGAN OpenImages (f=8), 8192, GumbelQuantization | 3.24 | 1.49 |[vqgan_gumbel_f8](https://heibox.uni-heidelberg.de/d/2e5662443a6b4307b470/) | --- | Reconstruction-FIDs. +| | | | | || | +| DALL-E dVAE (f=8), 8192, GumbelQuantization | 33.88 | 32.01 | https://github.com/openai/DALL-E | [reconstructions](https://k00.fr/j626x093) | Reconstruction-FIDs. + + +## Running pretrained models + +The commands below will start a streamlit demo which supports sampling at +different resolutions and image completions. To run a non-interactive version +of the sampling process, replace `streamlit run scripts/sample_conditional.py --` +by `python scripts/make_samples.py --outdir