Repository: keras-team/keras-cv

Issue:
Retinanet Input Shape
Hi! I think I found an issue with the RetinaNet class.
Referring to the last part of the example at:
[Object Detection with KerasCV](https://keras.io/guides/keras_cv/object_detection_keras_cv/#data-loading) 

This example runs, but when I change the model's backbone and I change the resizing operation to output (224, 224) shaped images, like this:

```
augmenter = keras.Sequential(
    layers=[
        keras_cv.layers.RandomFlip(
            mode="horizontal", 
            bounding_box_format="xywh"
        ),
        keras_cv.layers.JitteredResize(
            target_size=(224, 224), 
            scale_factor=(0.75, 1.3), 
            bounding_box_format="xywh"
        ),
    ]
)

model = keras_cv.models.RetinaNet.from_preset(
    "resnet18",
    num_classes=len(class_mapping),
    # For more info on supported bounding box formats, visit
    # https://keras.io/api/keras_cv/bounding_box/
    bounding_box_format="xywh",
)
```
I get the following error:
```
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/gianlucasama/.vscode/extensions/ms-python.python-2023.8.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/gianlucasama/.vscode/extensions/ms-python.python-2023.8.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/gianlucasama/.vscode/extensions/ms-python.python-2023.8.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/gianlucasama/.vscode/extensions/ms-python.python-2023.8.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/gianlucasama/.vscode/extensions/ms-python.python-2023.8.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/gianlucasama/.vscode/extensions/ms-python.python-2023.8.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/gianlucasama/Github/Deckard/od.py", line 158, in <module>
    model.fit(
  File "/home/gianlucasama/.local/lib/python3.10/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/tmp/__autograph_generated_filec1rr6auo.py", line 15, in tf__train_function
    retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
  File "/tmp/__autograph_generated_fileatd89d7s.py", line 10, in tf__call
    difference = ag__.ld(y_true) - ag__.ld(y_pred)
ValueError: in user code:

    File "/home/gianlucasama/.local/lib/python3.10/site-packages/keras/engine/training.py", line 1284, in train_function  *
        return step_function(self, iterator)
    File "/home/gianlucasama/.local/lib/python3.10/site-packages/keras_cv/losses/smooth_l1.py", line 38, in call  *
        difference = y_true - y_pred

    ValueError: Dimensions must be equal, but are 9351 and 9441 for '{{node SmoothL1Loss/sub}} = Sub[T=DT_FLOAT](retina_net_label_encoder/strided_slice_23, retina_net/box/concat)' with input shapes: [1,9351,4], [1,9441,4].
```
Am I missing something? Shouldn't the "resnet18" backbone have a (224, 224) input shape? I also tried with other backbones and the only input shape that seems to work is (640, 640).


Discussion:
ianstenbit: /gcbrun
ianstenbit: Tests show that this seems to have broken `models.legacy.FasterRCNN` -- I'm going to see if there's a reasonable fix to acommodate both.
jbischof: If it turns out to be difficult feel free to fork the current code and FasterRCNN or just turn off legacy tests for now.
ianstenbit: > If it turns out to be difficult feel free to fork the current code and FasterRCNN or just turn off legacy tests for now.

It turns out that FasterRCNN and RetinaNet basically disagree on whether to include anchors centered outside of the image. (RetinaNet does, FasterRCNN doesn't). So we probably just need to include this as a parameter in the `AnchorGenerator`, but for now I left this as a TODO and updated the FasterRCNN test.
ianstenbit: /gcbrun

Patch:
diff --git a/keras_cv/layers/object_detection/anchor_generator.py b/keras_cv/layers/object_detection/anchor_generator.py
index afebe8b..427a007 100644
--- a/keras_cv/layers/object_detection/anchor_generator.py
+++ b/keras_cv/layers/object_detection/anchor_generator.py
@@ -262,11 +262,15 @@ class _SingleAnchorGenerator:
         # make sure range of `cx` is within limit of `image_width` with
         # `stride`, also for sizes where `image_width % stride != 0`.
         # [W]
-        cx = tf.range(0.5 * stride, (image_width // stride) * stride, stride)
+        cx = tf.range(
+            0.5 * stride, tf.math.ceil(image_width / stride) * stride, stride
+        )
         # make sure range of `cy` is within limit of `image_height` with
         # `stride`, also for sizes where `image_height % stride != 0`.
         # [H]
-        cy = tf.range(0.5 * stride, (image_height // stride) * stride, stride)
+        cy = tf.range(
+            0.5 * stride, tf.math.ceil(image_height / stride) * stride, stride
+        )
         # [H, W]
         cx_grid, cy_grid = tf.meshgrid(cx, cy)
         # [H, W, 1]


Test patch:
diff --git a/keras_cv/layers/object_detection/anchor_generator_test.py b/keras_cv/layers/object_detection/anchor_generator_test.py
index de58a9d..1419532 100644
--- a/keras_cv/layers/object_detection/anchor_generator_test.py
+++ b/keras_cv/layers/object_detection/anchor_generator_test.py
@@ -71,11 +71,12 @@ class AnchorGeneratorTest(tf.test.TestCase, parameterized.TestCase):
         boxes = anchor_generator(image=image)
         boxes = tf.concat(list(boxes.values()), axis=0)
 
-        expected_box_shapes = (
-            (image_shape[0] // tf.constant(strides))
-            * (image_shape[1] // tf.constant(strides))
+        expected_box_shapes = tf.cast(
+            tf.math.ceil(image_shape[0] / tf.constant(strides))
+            * tf.math.ceil(image_shape[1] / tf.constant(strides))
             * len(scales)
-            * len(aspect_ratios)
+            * len(aspect_ratios),
+            tf.int32,
         )
 
         sum_expected_shape = (expected_box_shapes.numpy().sum(), 4)
@@ -102,11 +103,12 @@ class AnchorGeneratorTest(tf.test.TestCase, parameterized.TestCase):
         boxes = anchor_generator(image_shape=image_shape)
         boxes = tf.concat(list(boxes.values()), axis=0)
 
-        expected_box_shapes = (
-            (image_shape[0] // tf.constant(strides))
-            * (image_shape[1] // tf.constant(strides))
+        expected_box_shapes = tf.cast(
+            tf.math.ceil(image_shape[0] / tf.constant(strides))
+            * tf.math.ceil(image_shape[1] / tf.constant(strides))
             * len(scales)
-            * len(aspect_ratios)
+            * len(aspect_ratios),
+            tf.int32,
         )
 
         sum_expected_shape = (expected_box_shapes.numpy().sum(), 4)
diff --git a/keras_cv/models/legacy/object_detection/faster_rcnn/faster_rcnn_test.py b/keras_cv/models/legacy/object_detection/faster_rcnn/faster_rcnn_test.py
index 44956ed..08fd76c 100644
--- a/keras_cv/models/legacy/object_detection/faster_rcnn/faster_rcnn_test.py
+++ b/keras_cv/models/legacy/object_detection/faster_rcnn/faster_rcnn_test.py
@@ -28,10 +28,15 @@ from keras_cv.models.object_detection.__test_utils__ import (
 
 
 class FasterRCNNTest(tf.test.TestCase, parameterized.TestCase):
+    # TODO(ianstenbit): Make FasterRCNN support shapes that are not multiples
+    # of 128, perhaps by adding a flag to the anchor generator for whether to
+    # include anchors centered outside of the image. (RetinaNet does use those,
+    # while FasterRCNN doesn't). For more context on why this is the case, see
+    # https://github.com/keras-team/keras-cv/pull/1882
     @parameterized.parameters(
-        ((2, 640, 480, 3),),
+        ((2, 640, 384, 3),),
         ((2, 512, 512, 3),),
-        ((2, 224, 224, 3),),
+        ((2, 128, 128, 3),),
     )
     def test_faster_rcnn_infer(self, batch_shape):
         model = FasterRCNN(
@@ -46,9 +51,9 @@ class FasterRCNNTest(tf.test.TestCase, parameterized.TestCase):
         self.assertAllEqual([2, 1000, 4], outputs[0].shape)
 
     @parameterized.parameters(
-        ((2, 640, 480, 3),),
+        ((2, 640, 384, 3),),
         ((2, 512, 512, 3),),
-        ((2, 224, 224, 3),),
+        ((2, 128, 128, 3),),
     )
     def test_faster_rcnn_train(self, batch_shape):
         model = FasterRCNN(