internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: azavea/raster-vision

Issue:
Not able to use default stride with sliding window semseg option



Discussion:

Resolving patch:
diff --git a/rastervision/task/semantic_segmentation.py b/rastervision/task/semantic_segmentation.py
--- a/rastervision/task/semantic_segmentation.py
+++ b/rastervision/task/semantic_segmentation.py
@@ -77,6 +77,7 @@ def filter_windows(windows):
             stride = chip_options.stride
             if stride is None:
                 stride = chip_size / 2
+            stride = int(round(stride))
 
             return list(
                 filter_windows((extent.get_windows(chip_size, stride))))
diff --git a/rastervision/task/semantic_segmentation_config.py b/rastervision/task/semantic_segmentation_config.py
--- a/rastervision/task/semantic_segmentation_config.py
+++ b/rastervision/task/semantic_segmentation_config.py
@@ -95,6 +95,10 @@ def from_proto(self, msg):
         negative_survival_probability = conf.chip_options \
                                             .negative_survival_probability
 
+        stride = conf.chip_options.stride
+        if stride == 0:
+            stride = None
+
         return self.with_classes(list(conf.class_items)) \
                 .with_predict_batch_size(msg.predict_batch_size) \
                 .with_predict_package_uri(msg.predict_package_uri) \
@@ -107,7 +111,7 @@ def from_proto(self, msg):
                     negative_survival_probability=negative_survival_probability,
                     chips_per_scene=conf.chip_options.chips_per_scene,
                     target_count_threshold=conf.chip_options.target_count_threshold,
-                    stride=conf.chip_options.stride)
+                    stride=stride)
 
     def validate(self):
         super().validate()


Test patch:
Repository: azavea/raster-vision

Issue:
Debug chips for semseg not tagged by split id
This means all but the last split gets over-written.


Discussion:

Resolving patch:
diff --git a/rastervision/backend/tf_deeplab.py b/rastervision/backend/tf_deeplab.py
--- a/rastervision/backend/tf_deeplab.py
+++ b/rastervision/backend/tf_deeplab.py
@@ -549,32 +549,26 @@ def process_sceneset_results(self, training_results: List[str],
         upload_or_copy(validation_record_path_local, validation_record_path)
 
         if self.backend_config.debug:
-            training_zip_path = join(base_uri, '{}'.format(TRAIN))
-            training_zip_path_local = get_local_path(training_zip_path,
-                                                     tmp_dir)
-            validation_zip_path = join(base_uri, '{}'.format(VALIDATION))
-            validation_zip_path_local = get_local_path(validation_zip_path,
-                                                       tmp_dir)
-
-            training_debug_dir = join(tmp_dir, 'training-debug')
-            make_debug_images(
-                training_record_path_local, training_debug_dir, self.class_map,
-                self.task_config.chip_options.debug_chip_probability)
-            shutil.make_archive(training_zip_path_local, 'zip',
-                                training_debug_dir)
-
-            validation_debug_dir = join(tmp_dir, 'validation-debug')
-            make_debug_images(
-                validation_record_path_local, validation_debug_dir,
-                self.class_map,
-                self.task_config.chip_options.debug_chip_probability)
-            shutil.make_archive(validation_zip_path_local, 'zip',
-                                validation_debug_dir)
-
-            upload_or_copy('{}.zip'.format(training_zip_path_local),
-                           '{}.zip'.format(training_zip_path))
-            upload_or_copy('{}.zip'.format(validation_zip_path_local),
-                           '{}.zip'.format(validation_zip_path))
+
+            def _make_debug_chips(split, record_path_local):
+                zip_path = join(base_uri, '{}-debug'.format(split),
+                                chip_suffix)
+                zip_path_local = get_local_path(zip_path, tmp_dir)
+
+                debug_dir = join(tmp_dir, '{}-debug'.format(split),
+                                 chip_suffix)
+                make_debug_images(
+                    record_path_local, debug_dir, self.class_map,
+                    self.task_config.chip_options.debug_chip_probability)
+                shutil.make_archive(zip_path_local, 'zip', debug_dir)
+
+                upload_or_copy('{}.zip'.format(zip_path_local),
+                               '{}.zip'.format(zip_path))
+
+            if training_results:
+                _make_debug_chips(TRAIN, training_record_path_local)
+            if validation_results:
+                _make_debug_chips(VALIDATION, validation_record_path_local)
 
     def train(self, tmp_dir: str) -> None:
         """Train a DeepLab model the task and backend config.


Test patch:
Repository: azavea/raster-vision

Issue:
Get Sphinx to render Google-style API docs correctly
In the API docs on RTD https://docs.rastervision.io/en/latest/api.html, the line breaks in the docs aren't rendered correctly. In particular, the `lr` argument for the PyTorch backends is on the same line as the `batch_size` argument, which obscures the `lr` argument. I think this is because we are using Google style PyDocs which aren't natively supported by Sphinx. However, there is a plugin for doing this:

https://stackoverflow.com/questions/7033239/how-to-preserve-line-breaks-when-generating-python-docs-using-sphinx


Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -44,7 +44,8 @@
     'sphinx.ext.autodoc',
     'sphinx.ext.intersphinx',
     'pallets_sphinx_themes',
-    'sphinxcontrib.programoutput'
+    'sphinxcontrib.programoutput',
+    'sphinxcontrib.napoleon',
 ]
 
 # https://read-the-docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
diff --git a/rastervision/backend/pytorch_chip_classification_config.py b/rastervision/backend/pytorch_chip_classification_config.py
--- a/rastervision/backend/pytorch_chip_classification_config.py
+++ b/rastervision/backend/pytorch_chip_classification_config.py
@@ -73,9 +73,8 @@ def with_train_options(self,
                 details.
             num_epochs: (int) number of epochs (sweeps through training set) to
                 train model for
-            model_arch: (str) classification model backbone to use for UNet
-                architecture. Any option in torchvision.models is valid, for
-                example, resnet18.
+            model_arch: (str) Any classification model option in
+                torchvision.models is valid, for example, resnet18.
             sync_interval: (int) sync training directory to cloud every
                 sync_interval epochs.
             debug: (bool) if True, save debug chips (ie. visualizations of


Test patch:
Repository: azavea/raster-vision

Issue:
Verbose flag is not passed to remote jobs
The `-v` flag works with the inprocess runner, but doesn't seem to get forwarded to remote jobs when using the batch runner.


Discussion:

Resolving patch:
diff --git a/rastervision_aws_batch/rastervision/aws_batch/aws_batch_runner.py b/rastervision_aws_batch/rastervision/aws_batch/aws_batch_runner.py
--- a/rastervision_aws_batch/rastervision/aws_batch/aws_batch_runner.py
+++ b/rastervision_aws_batch/rastervision/aws_batch/aws_batch_runner.py
@@ -155,10 +155,13 @@ def run(self,
             job_name = f'{pipeline_run_name}-{command}-{uuid.uuid4()}'
 
             if not external:
-                cmd = [
-                    'python', '-m', 'rastervision.pipeline.cli run_command',
-                    cfg_json_uri, command, '--runner', AWS_BATCH
-                ]
+                cmd = ['python', '-m', 'rastervision.pipeline.cli']
+                if rv_config.get_verbosity() > 1:
+                    cmd.append('-' + 'v' * (rv_config.get_verbosity() - 1))
+                cmd.extend([
+                    'run_command', cfg_json_uri, command, '--runner', AWS_BATCH
+                ])
+
                 if command in pipeline.split_commands and num_splits > 1:
                     num_array_jobs = num_splits
                     cmd += ['--num-splits', str(num_splits)]


Test patch:
Repository: azavea/raster-vision

Issue:
Semantic Segmentation Scenes w/ 0 Chips Fail
Semantic segmentation scenes with zero chips cause raster-vision to halt.  This can be an issue when using finely-cropped scenes and filtering by nodata.


Discussion:
```
...
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py", line 128, in chip
    return super().chip(*args, **kwargs)                              
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/rv_pipeline.py", line 110, in chip
    chip_scene(s.build(class_cfg, self.tmp_dir), VALIDATION)                  
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/rv_pipeline.py", line 93, in chip_scene
    windows = self.get_train_windows(scene)
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py", line 136, in get_train_windows
    chip_nodata_threshold=self.config.chip_nodata_threshold)
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py", line 79, in get_train_windows                                                                                            
    a_window = windows[0]
```


Resolving patch:
diff --git a/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py b/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py
--- a/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py
+++ b/rastervision_core/rastervision/core/rv_pipeline/semantic_segmentation.py
@@ -75,11 +75,15 @@ def should_use_window(window: Box) -> bool:
 
     if co.window_method == SemanticSegmentationWindowMethod.sliding:
         stride = co.stride or int(round(chip_size / 2))
-        windows = list(filter_windows((extent.get_windows(chip_size, stride))))
-        a_window = windows[0]
-        windows = list(filter(should_use_window, windows))
-        if len(windows) == 0:
-            windows = [a_window]
+        unfiltered_windows = extent.get_windows(chip_size, stride)
+        windows = list(filter_windows(unfiltered_windows))
+        if len(windows) > 0:
+            a_window = windows[0]
+            windows = list(filter(should_use_window, windows))
+            if len(windows) == 0:
+                windows = [a_window]
+        elif len(windows) == 0:
+            return [unfiltered_windows[0]]
     elif co.window_method == SemanticSegmentationWindowMethod.random_sample:
         windows = []
         attempts = 0


Test patch:
Repository: azavea/raster-vision

Issue:
Upgrade to latest PyTorch
Currently we're at v1.2 https://github.com/azavea/raster-vision/blob/master/rastervision_pytorch_learner/requirements.txt

This should probably be upgraded right before the next release so that we can make sure all the examples work with it.


Discussion:

Resolving patch:
diff --git a/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_utils.py b/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_utils.py
--- a/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_utils.py
+++ b/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_utils.py
@@ -341,7 +341,12 @@ def resnet_fpn_backbone(backbone_name, pretrained):
         if 'layer2' not in name and 'layer3' not in name and 'layer4' not in name:
             parameter.requires_grad_(False)
 
-    return_layers = {'layer1': 0, 'layer2': 1, 'layer3': 2, 'layer4': 3}
+    return_layers = {
+        'layer1': '0',
+        'layer2': '1',
+        'layer3': '2',
+        'layer4': '3'
+    }
 
     out_channels = 256
     in_channels_list = get_out_channels(backbone)
diff --git a/rastervision_pytorch_learner/rastervision/pytorch_learner/utils/torch_hub.py b/rastervision_pytorch_learner/rastervision/pytorch_learner/utils/torch_hub.py
--- a/rastervision_pytorch_learner/rastervision/pytorch_learner/utils/torch_hub.py
+++ b/rastervision_pytorch_learner/rastervision/pytorch_learner/utils/torch_hub.py
@@ -77,7 +77,7 @@ def torch_hub_load_github(repo: str, hubconf_dir: str, tmp_dir: str,
         Any: The output from calling the entrypoint.
     """
     torch.hub.set_dir(tmp_dir)
-    out = torch.hub.load(github=repo, model=entrypoint, *args, **kwargs)
+    out = torch.hub.load(repo, entrypoint, *args, source='github', **kwargs)
 
     orig_dir = join(tmp_dir, _repo_name_to_dir_name(repo))
     shutil.move(orig_dir, hubconf_dir)
@@ -139,26 +139,5 @@ def torch_hub_load_uri(uri: str, hubconf_dir: str, entrypoint: str,
 
 def torch_hub_load_local(hubconf_dir: str, entrypoint: str, *args,
                          **kwargs) -> Any:
-    """Same as torch.hub.load(), minus the downloading part.
-
-    Args:
-        hubconf_dir (str): A directory containing a hubconf.py file.
-        entrypoint (str): Name of a callable present in hubconf.py.
-
-    Returns:
-        Any: The output from calling the entrypoint.
-    """
-    from torch.hub import (sys, import_module, MODULE_HUBCONF,
-                           _load_entry_from_hubconf)
-
-    sys.path.insert(0, hubconf_dir)
-
-    hub_module = import_module(MODULE_HUBCONF, join(hubconf_dir,
-                                                    MODULE_HUBCONF))
-
-    entry = _load_entry_from_hubconf(hub_module, entrypoint)
-    out = entry(*args, **kwargs)
-
-    sys.path.remove(hubconf_dir)
-
-    return out
+    return torch.hub.load(
+        hubconf_dir, entrypoint, *args, source='local', **kwargs)


Test patch:
Repository: azavea/raster-vision

Issue:
Predictor does not reset the scene's aoi_geometries and the raster source's extent_crop
Currently, the `Predictor` re-uses a `SceneConfig` from the pipeline config in the bundle (instead of creating a new one) and resets its `label_source` and `aoi_uris`.
https://github.com/azavea/raster-vision/blob/master/rastervision_core/rastervision/core/predictor.py#L70-L71

However, it should also do this for `raster_source.extent_crop` (#1030) and `aoi_geometries` (#1033). In general, it should be done for every field that cannot be safely assumed to be the same for the input scene.

Instead of having to add to this every time something new is added to the `SceneConfig` or any of its member classes, it might be better to create a new scene in the predictor with options from the command line.


Discussion:

Resolving patch:
diff --git a/rastervision_core/rastervision/core/predictor.py b/rastervision_core/rastervision/core/predictor.py
--- a/rastervision_core/rastervision/core/predictor.py
+++ b/rastervision_core/rastervision/core/predictor.py
@@ -69,6 +69,9 @@ def __init__(self,
 
         self.scene.label_source = None
         self.scene.aoi_uris = None
+        self.scene.aoi_geometries = None
+        self.scene.raster_source.extent_crop = None
+
         self.config.dataset.train_scenes = [self.scene]
         self.config.dataset.validation_scenes = [self.scene]
         self.config.dataset.test_scenes = []


Test patch:
Repository: azavea/raster-vision

Issue:
CPU Batch jobs cannot pull container
After making a large number of dependency upgrades suggested by Dependabot, I started getting the following error when running any job that runs in the CPU compute environment on Batch. The GPU ones work fine. 

See https://aws.amazon.com/premiumsupport/knowledge-center/batch-job-failure-disk-space/

```
CannotPullContainerError: failed to register layer: ApplyLayer exit status 1 stdout: stderr: write /root/.cache/pip/http/d/c/1/1/c/dc11c115dbb602e6636493addd92f61ac957da65f9166a69f6820fbc: no space left on device
```


Discussion:

Resolving patch:
diff --git a/rastervision_pytorch_learner/rastervision/pytorch_learner/semantic_segmentation_learner.py b/rastervision_pytorch_learner/rastervision/pytorch_learner/semantic_segmentation_learner.py
--- a/rastervision_pytorch_learner/rastervision/pytorch_learner/semantic_segmentation_learner.py
+++ b/rastervision_pytorch_learner/rastervision/pytorch_learner/semantic_segmentation_learner.py
@@ -34,7 +34,7 @@ def build_model(self) -> nn.Module:
         out_classes = len(self.cfg.data.class_names)
         if self.cfg.solver.ignore_last_class:
             out_classes -= 1
-        model = models.segmentation.segmentation._segm_resnet(
+        model = models.segmentation.segmentation._segm_model(
             'deeplabv3',
             self.cfg.model.get_backbone_str(),
             out_classes,


Test patch:
diff --git a/integration_tests/semantic_segmentation/config.py b/integration_tests/semantic_segmentation/config.py
--- a/integration_tests/semantic_segmentation/config.py
+++ b/integration_tests/semantic_segmentation/config.py
@@ -23,6 +23,7 @@ def get_path(part):
             return join(dirname(__file__), part)
 
     class_config = ClassConfig(names=['red', 'green'], colors=['red', 'green'])
+    class_config.ensure_null_class()
 
     def make_scene(id, img_path, label_path):
         raster_source = RasterioSourceConfig(
Repository: azavea/raster-vision

Issue:
Local runner should write makefile to temporary dir
Instead, it writes it to the `root_uri` which might be an S3 URI, and `make`, which is used by the local runner cannot handle that.
Makefile error when `root_uri` is an S3 path
## 🐛 Bug
When running training command and having `root_uri` set to an S3 folder, this error shows up:
```
make: s3://<random_bucket_name>/predictions/Makefile: No such file or directory
make: *** No rule to make target 's3://<random_bucket_name>/predictions/Makefile'.  Stop.
```

This error disappears when `root_uri` is a local path. AWS config is right as it is able to read and write the files.

## To Reproduce

Steps to reproduce the behavior:

1. I ran the following command inside the container:
`python -m rastervision.pipeline.cli run local code/local_exp.py -a raw_uri s3://<random_bucket_name>/datafortesting/data/ -a root_uri s3://<random_bucket_name>/predictions -a test False`

<!-- Please provide the command executed, source of the get_config() function, error messages, and/or full stack traces if at all possible -->

## Expected behavior

It should run normally like it is running when `root_uri` is a local path.

## Environment

Running with docker.  **Image**: quay.io/azavea/raster-vision:pytorch-v0.13.1

## Additional context

This might be a relevant issue: #991 



Discussion:

Yes, this is the same as #991.

Two possible workarounds:
1. Use [`inprocess`](https://docs.rastervision.io/en/0.13/runners.html?highlight=inprocess#inprocess) instead of `local`: `python -m rastervision.pipeline.cli run inprocess ...`
2. Change this line so that `makefile_path` is a local path (e.g. in a temporary directory): https://github.com/azavea/raster-vision/blob/b735cffe76b2918a48cfe26b2f8047626716ec65/rastervision_pipeline/rastervision/pipeline/runner/local_runner.py#L69
Will this error still occur if I use `batch` in the run command?
No. This is specific to `local`.

Resolving patch:
diff --git a/rastervision_pipeline/rastervision/pipeline/runner/local_runner.py b/rastervision_pipeline/rastervision/pipeline/runner/local_runner.py
--- a/rastervision_pipeline/rastervision/pipeline/runner/local_runner.py
+++ b/rastervision_pipeline/rastervision/pipeline/runner/local_runner.py
@@ -2,7 +2,7 @@
 from os.path import dirname, join
 from subprocess import Popen
 
-from rastervision.pipeline.file_system import str_to_file
+from rastervision.pipeline.file_system import str_to_file, download_if_needed
 from rastervision.pipeline.runner.runner import Runner
 from rastervision.pipeline.utils import terminate_at_exit
 
@@ -68,7 +68,8 @@ def run(self,
 
         makefile_path = join(dirname(cfg_json_uri), 'Makefile')
         str_to_file(makefile, makefile_path)
-        process = Popen(['make', '-j', '-f', makefile_path])
+        makefile_path_local = download_if_needed(makefile_path)
+        process = Popen(['make', '-j', '-f', makefile_path_local])
         terminate_at_exit(process)
         exitcode = process.wait()
         if exitcode != 0:


Test patch:
Repository: azavea/raster-vision

Issue:
Allow `neg_ratio` to be None in `ObjectDetectionGeoDataWindowConfig`
## 🚀 Feature

See the following discussion on Gitter: https://gitter.im/azavea/raster-vision?at=6298bff6ef00bd1dc60fc7ee

`ObjectDetectionGeoDataWindowConfig` should allow `neg_ratio` to be `None` since `ObjectDetectionRandomWindowGeoDataset` allows it.

## Motivation

It's a feature that already exists. This will allow users to actually make use of it.

## Pitch

`ObjectDetectionGeoDataWindowConfig.neg_ratio`'s type hint will need to be changed to `Optional[float]`.

## Alternatives

N/A

## Additional context

N/A

Allow `neg_ratio` to be None in `ObjectDetectionGeoDataWindowConfig`
## 🚀 Feature

See the following discussion on Gitter: https://gitter.im/azavea/raster-vision?at=6298bff6ef00bd1dc60fc7ee

`ObjectDetectionGeoDataWindowConfig` should allow `neg_ratio` to be `None` since `ObjectDetectionRandomWindowGeoDataset` allows it.

## Motivation

It's a feature that already exists. This will allow users to actually make use of it.

## Pitch

`ObjectDetectionGeoDataWindowConfig.neg_ratio`'s type hint will need to be changed to `Optional[float]`.

## Alternatives

N/A

## Additional context

N/A



Discussion:

Resolving patch:
diff --git a/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_learner_config.py b/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_learner_config.py
--- a/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_learner_config.py
+++ b/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_learner_config.py
@@ -71,14 +71,14 @@ class ObjectDetectionGeoDataWindowConfig(GeoDataWindowConfig):
         False,
         description='Clip bounding boxes to window limits when retrieving '
         'labels for a window.')
-    neg_ratio: float = Field(
-        1.0,
+    neg_ratio: Optional[float] = Field(
+        None,
         description='The ratio of negative chips (those containing no '
         'bounding boxes) to positive chips. This can be useful if the '
         'statistics of the background is different in positive chips. For '
         'example, in car detection, the positive chips will always contain '
         'roads, but no examples of rooftops since cars tend to not be near '
-        'rooftops. Defaults to 1.0.')
+        'rooftops. Defaults to None.')
     neg_ioa_thresh: float = Field(
         0.2,
         description='A window will be considered negative if its max IoA with '


Test patch: