internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: google/pytype

Issue:
Automatically added git ignore file to sub directory
This PR automatically creates a .gitignore file to the output directory of pytype/tools/analyze_project/main.py

Issue #759 


Discussion:

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: **Please visit <https://cla.developers.google.com/> to sign.**

Once you've signed (or fixed any issues), please reply here with `@googlebot I signed it!` and we'll verify it.

----

#### What to do if you already signed the CLA

##### Individual signers

*   It's possible we don't have your GitHub username or you're using a different email address on your commit. Check [your existing CLA data](https://cla.developers.google.com/clas) and verify that your [email is set on your git commits](https://help.github.com/articles/setting-your-email-in-git/).

##### Corporate signers

*   Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to [go/cla#troubleshoot](http://go/cla#troubleshoot) ([Public version](https://opensource.google/docs/cla/#troubleshoot)).
*   The email used to register you as an authorized contributor must be the email used for the Git commit. Check [your existing CLA data](https://cla.developers.google.com/clas) and verify that your [email is set on your git commits](https://help.github.com/articles/setting-your-email-in-git/).
*   The email used to register you as an authorized contributor must also be [attached to your GitHub account](https://github.com/settings/emails).
		

ℹ️ **Googlers: [Go here](https://goto.google.com/prinfo/https%3A%2F%2Fgithub.com%2Fgoogle%2Fpytype%2Fpull%2F770) for more info**.

<!-- need_sender_cla -->
@googlebot I signed it!

Resolving patch:
diff --git a/pytype/tools/analyze_project/main.py b/pytype/tools/analyze_project/main.py
--- a/pytype/tools/analyze_project/main.py
+++ b/pytype/tools/analyze_project/main.py
@@ -1,6 +1,7 @@
 """Analyze an entire project using pytype."""
 
 import logging
+import os
 import sys
 import tempfile
 
@@ -74,6 +75,8 @@ def main():
   logging.info('Source tree:\n%s',
                importlab.output.formatted_deps_list(import_graph))
   tool_utils.makedirs_or_die(conf.output, 'Could not create output directory')
+  with open(os.path.join(conf.output, '.gitignore'), 'w') as f:
+    f.write('# Automatically created by pytype\n*')
   deps = pytype_runner.deps_from_import_graph(import_graph)
   runner = pytype_runner.PytypeRunner(conf, deps)
   return runner.run()


Test patch:
Repository: google/pytype

Issue:
Get the line number of first line in function body
Resolves #783


Discussion:

Resolving patch:
diff --git a/pytype/blocks.py b/pytype/blocks.py
--- a/pytype/blocks.py
+++ b/pytype/blocks.py
@@ -340,7 +340,8 @@ def visit_code(self, code):
         fn_code = code.co_consts[code_op.arg]
         if not _is_function_def(fn_code):
           continue
-        end_line = fn_code.co_code[0].line  # First line of code in body.
+        # First line of code in body.
+        end_line = min(op.line for op in fn_code.co_code)
         self.make_function_ops[op.line] = (end_line, op)
       elif (isinstance(op, STORE_OPCODES) and
             op.line not in self.make_function_ops):


Test patch:
diff --git a/pytype/tests/py3/test_type_comments.py b/pytype/tests/py3/test_type_comments.py
--- a/pytype/tests/py3/test_type_comments.py
+++ b/pytype/tests/py3/test_type_comments.py
@@ -32,6 +32,7 @@ def g(xs: List[str]) -> List[str]: ...
 
 
 class Py3TypeCommentTest(test_base.TargetPython3FeatureTest):
+  """Type comment tests that use Python 3-only features."""
 
   def test_ignored_comment(self):
     self.CheckWithErrors("""
@@ -39,5 +40,23 @@ def f():
         v: int = None  # type: str  # ignored-type-comment
     """)
 
+  def test_first_line_of_code(self):
+    # In Python 3.8, the line number of this method's first opcode is the second
+    # line in the method body, not the first. This test verifies that we are
+    # still able to determine the first line number in the body. If we get the
+    # wrong line, the `Dict[str, int]` annotation is associated to the method
+    # signature, generating a spurious [redundant-function-type-comment] error.
+    self.Check("""
+      from typing import Dict
+      def f() -> Dict[str, int]:
+        # some_var = ''
+        # something more
+        cast_type: Dict[str, int] = {
+          'one': 1,
+          'two': 2,
+          'three': 3,
+        }
+        return cast_type
+    """)
 
 test_base.main(globals(), __name__ == "__main__")
Repository: google/pytype

Issue:
Add support for PEP 604 (Union |) in stub files
Part of #785, though not a complete fix because this doesn't allow the `|` syntax at runtime (I think).

I was pleasantly surprised by how easy this change was. I hope that doesn't mean I missed something.


Discussion:
Wow! That really is a shockingly easy change.

@martindemello Could you review this PR, since you wrote pyi/parser.py and are the most familiar with it?
looks great, thanks for the fix!
Great, I'll go ahead and import this PR. Thanks, Martin!

Resolving patch:
diff --git a/pytype/pyi/parser.py b/pytype/pyi/parser.py
--- a/pytype/pyi/parser.py
+++ b/pytype/pyi/parser.py
@@ -190,6 +190,12 @@ def visit_Attribute(self, node):
     annotation = _attribute_to_name(node).id
     return self.defs.new_type(annotation)
 
+  def visit_BinOp(self, node):
+    if isinstance(node.op, ast3.BitOr):
+      return self.defs.new_type("typing.Union", [node.left, node.right])
+    else:
+      raise ParseError(f"Unexpected operator {node.op}")
+
   def visit_BoolOp(self, node):
     if isinstance(node.op, ast3.Or):
       raise ParseError("Deprecated syntax `x or y`; use `Union[x, y]` instead")


Test patch:
diff --git a/pytype/pyi/parser_test.py b/pytype/pyi/parser_test.py
--- a/pytype/pyi/parser_test.py
+++ b/pytype/pyi/parser_test.py
@@ -2925,6 +2925,22 @@ def f(x: Callable[..., R]) -> Callable[..., R]: ...
     """)
 
 
+class UnionOrTest(_ParserTestBase):
+
+  def test_basic(self):
+    self.check("""
+      def f(x: int | str) -> None: ...
+      def g(x: bool | str | float) -> None: ...
+      def h(x: str | None) -> None: ...
+    """, """
+      from typing import Optional, Union
+
+      def f(x: Union[int, str]) -> None: ...
+      def g(x: Union[bool, str, float]) -> None: ...
+      def h(x: Optional[str]) -> None: ...
+    """)
+
+
 class TypeGuardTest(_ParserTestBase):
 
   def test_typing_extensions(self):
Repository: google-parfait/tensorflow-federated

Issue:
`tff.learning.from_keras_model` does not accept subclassed Keras model
**Describe the bug**
`tff.learning.from_keras_model()` does not accept subclassed keras model. See this [colab](https://colab.research.google.com/drive/1dPJlo3e6vVHXS9O4Pm3_Qq2c_h0VtRaP?usp=sharing).

**Environment (please complete the following information):**
* OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Colab
* Python package versions (e.g., TensorFlow Federated, TensorFlow): Colab TF-nightly TFF-nightly
* Python version: python 3.6
* Bazel version (if building from source): N/A
* CUDA/cuDNN version: N/A
* What TensorFlow Federated execution stack are you using? N/A

**Expected behavior**
`tff.learning.from_keras_model()` properly accepts subclassed keras model, or provide info on the doc if this is not possible?

**Additional context**
The issue seems to be due to `from_keras_model()` consuming `tf.keras.model.output` (see [pointer](https://github.com/tensorflow/federated/blob/v0.16.1/tensorflow_federated/python/learning/keras_utils.py#L260)) which is not written when the model is subclassed.


Discussion:
Thanks Honglin! One option is to write your own keras model wrapper, or tff.learning.Model. It should not be very difficult, but you may loose some flexibility on things like keras metricss. Examples can be found at https://github.com/tensorflow/federated/blob/master/tensorflow_federated/python/examples/simple_fedavg/simple_fedavg_tf.py#L38
and https://github.com/tensorflow/federated/blob/master/tensorflow_federated/python/examples/simple_fedavg/simple_fedavg_test.py#L137
This behavior is definitely TFF working as intended. TFF is strongly typed and neds to be able to reason about the output shapes and types of the functions it invokes.

But definitely we should update our documentation. Writing such a change now.

Resolving patch:
diff --git a/tensorflow_federated/python/learning/keras_utils.py b/tensorflow_federated/python/learning/keras_utils.py
--- a/tensorflow_federated/python/learning/keras_utils.py
+++ b/tensorflow_federated/python/learning/keras_utils.py
@@ -51,6 +51,11 @@ def from_keras_model(
   which the generated `tff.learning.Model` will appear, TFF needs the type `y`
   in addition to the type `x`.
 
+  Note: This function does not currently accept subclassed `tf.keras.Models`,
+  as it makes assumptions about presence of certain attributes which are
+  guaranteed to exist through the functional or Sequential API but are
+  not necessarily present for subclassed models.
+
   Args:
     keras_model: A `tf.keras.Model` object that is not compiled.
     loss: A single `tf.keras.losses.Loss` or a list of losses-per-output. If a


Test patch:
Repository: google-parfait/tensorflow-federated

Issue:
Error in loading the GLDv2 dataset
Hello! Thanks for the fantastic library. I have run into an error while loading the GLDv2 dataset via TFF. 

Please see the snippet below:
```
>>> import tensorflow_federated as tff
>>> dataset = tff.simulation.datasets.gldv2.load_data()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/path/to/python3.8/site-packages/tensorflow_federated/python/simulation/datasets/gldv2.py", line 396, in load_data
    qh = logging.handlers.QueueHandler(q)
AttributeError: module 'logging' has no attribute 'handlers'
```

## The root cause of this error
The error appears to be because of a missing import in `gldv2.py`. We can isolate this error to the following lines which occur within `gldv2.py`. Compare this snippet which does not work
```
>>> import logging, multiprocessing
>>> q = multiprocessing.Queue(-1)
>>> qh = logging.handlers.QueueHandler(q)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'logging' has no attribute 'handlers'
```
to this one which does
```
>>> import logging, multiprocessing
>>> import logging.handlers            # Add this import statement
>>> q = multiprocessing.Queue(-1)
>>> qh = logging.handlers.QueueHandler(q)  # works!
```

## Fixing the error
Adding an import statement `import logging.handlers` in `gldv2.py` fixes the issue -- I've tested this out locally. I can send a pull request. 

**Environment (please complete the following information):**
* OS Platform and Distribution: Ubuntu 20.04
* Python package versions (e.g., TensorFlow Federated, TensorFlow): TFF: 0.19.0 and TF: 2.5.0. The same error also occurs in the nightly version: TFF: 0.19.0.dev20210821 and TF: 2.7.0-dev20210822.
* Python version: 3.8
* Bazel version (if building from source): N/A (installed via pip)
* CUDA/cuDNN version: N/A
* What TensorFlow Federated execution stack are you using? simulation 


**Expected behavior**
I expect the data to be loaded correctly.




Discussion:

Resolving patch:
diff --git a/tensorflow_federated/python/simulation/datasets/gldv2.py b/tensorflow_federated/python/simulation/datasets/gldv2.py
--- a/tensorflow_federated/python/simulation/datasets/gldv2.py
+++ b/tensorflow_federated/python/simulation/datasets/gldv2.py
@@ -15,6 +15,7 @@
 
 import collections
 import logging
+import logging.handlers
 import multiprocessing.pool
 import os
 import shutil


Test patch:
Repository: cisagov/manage.get.gov

Issue:
Address local runs of tests in `dev`
We need a way for people to run the things that run in CI locally so they don't have to go through CI over and over. Options: 

- A template pre-commit hook that runs the same things as in CI (would have to be added individually) 
- Use something like [`act`](https://github.com/nektos/act) to run the github actions locally
- A script that is used locally and in CI
- A requirements-dev.txt file and freeze may be necessary in all cases to pin requirements in dev and CI. 


Discussion:

Resolving patch:
diff --git a/src/registrar/management/__init__.py b/src/registrar/management/__init__.py
new file mode 100644
diff --git a/src/registrar/management/commands/__init__.py b/src/registrar/management/commands/__init__.py
new file mode 100644
diff --git a/src/registrar/management/commands/lint.py b/src/registrar/management/commands/lint.py
new file mode 100644
--- /dev/null
+++ b/src/registrar/management/commands/lint.py
@@ -0,0 +1,63 @@
+from subprocess import run, CalledProcessError  # nosec
+
+from django.core.management.base import BaseCommand, CommandError
+
+
+class Command(BaseCommand):
+    """
+    Helper command for running installed linters.
+
+    Run using `./manage.py lint`.
+    """
+
+    help = "Runs linters: flake8 black mypy bandit"
+
+    # Add new linters here.
+    #
+    # To maintain security while using subprocess, avoid passing user input
+    # and, in all cases, make sure to use a list (not a string) for flags/args
+    # as this will quote the output.
+    linters = {
+        "flake8": {
+            "purpose": "Linting",
+            "args": ["flake8", ".", "--count", "--show-source", "--statistics"],
+        },
+        "black": {
+            "purpose": "Formatting",
+            "args": ["black", "--check", "."],
+        },
+        "mypy": {
+            "purpose": "Type checking",
+            "args": ["mypy", "."],
+        },
+        "bandit": {
+            "purpose": "Security scanning",
+            "args": ["bandit", "-r", "."],
+        },
+    }
+
+    def add_arguments(self, parser):
+        parser.add_argument("linters", nargs="*", type=str)
+
+    def handle(self, *args, **options):
+        try:
+            for linter in self.linters.values():
+                self.stdout.write(f"[manage.py lint] {linter['purpose']}. . .")
+                result = run(linter["args"])
+                if result.returncode:
+                    self.stderr.write(
+                        self.style.NOTICE(
+                            "[manage.py lint] Re-try with: [docker-compose exec app] "
+                            f"{' '.join(linter['args'])}"
+                        )
+                    )
+                    break
+                else:
+                    self.stdout.write(
+                        f"[manage.py lint] {linter['purpose']} completed with success!"
+                    )
+        except CalledProcessError as e:
+            raise CommandError(e)
+        self.stdout.write(
+            self.style.SUCCESS("[manage.py lint] All linters ran successfully.")
+        )


Test patch:
Repository: cisagov/manage.get.gov

Issue:
Setup new cloud.gov prototyping org
We recently signed an agreement for a more robust prototyping org with cloud.gov that will give us the ability to have multiple spaces and invite members from DHS. 

- [ ] Setup org and make sure everyone who needs access has the appropriate access
- [ ] Setup spaces `unstable` and `staging` according to rules
- [ ] Change Github actions access to CD
- [ ] Replace org and space name in settings (actions, documentation, scripts, etc.)


Discussion:

Resolving patch:
diff --git a/src/registrar/config/settings.py b/src/registrar/config/settings.py
--- a/src/registrar/config/settings.py
+++ b/src/registrar/config/settings.py
@@ -388,6 +388,7 @@
 # web server configurations.
 ALLOWED_HOSTS = [
     "getgov-unstable.app.cloud.gov",
+    "getgov-staging.app.cloud.gov",
     "get.gov",
 ]
 


Test patch:
Repository: cisagov/manage.get.gov

Issue:
Add initial security scanning for Django application code
- Turn on dependabot alerts 
- Investigate various options for security scanning through github actions https://github.com/marketplace/actions/django-security-check
- Contemplate adding Zap scanning:  https://github.com/18F/django-template/blob/main/django_template/templates/github/workflows/owasp-scan.yml.jinja#L44-L51


Discussion:
This will be an add on, if necessary after #81 
From a team member in Slack: 

OWASP publishes docker images for Zap and if you already have your app set up in docker-compose, then you can add their images under a separate profile (so that container won’t start when just running the app locally) and then [run zap scans whenever you want with docker-compose run owasp](https://github.com/18F/django-template/pull/11/commits/1b212e486851a426eef482cb6ab0003cf8b8fe44#diff-0f463ed90dd8d656d46cc46c6f229d255c62d754813acf83c2a7866d9bcde5f1).

They also publish [Github actions for scans](https://github.com/18F/django-template/blob/main/django_template/templates/github/workflows/owasp-scan.yml.jinja#L44-L51).

Resolving patch:
diff --git a/src/registrar/config/settings.py b/src/registrar/config/settings.py
--- a/src/registrar/config/settings.py
+++ b/src/registrar/config/settings.py
@@ -228,6 +228,16 @@
 # endregion
 # region: Headers-----------------------------------------------------------###
 
+# Content-Security-Policy configuration
+# this can be restrictive because we have few external scripts
+allowed_sources = ("'self'",)
+CSP_DEFAULT_SRC = allowed_sources
+# Most things fall back to default-src, but these two do not and should be
+# explicitly set
+CSP_FRAME_ANCESTORS = allowed_sources
+CSP_FORM_ACTION = allowed_sources
+
+
 # Content-Length header is set by django.middleware.common.CommonMiddleware
 
 # X-Frame-Options header is set by


Test patch:
Repository: cisagov/manage.get.gov

Issue:
Reconfigure OIDC logout to send client_id
Login.gov recently changed their logout method to take `client_id` instead of the previous parameter `id_token_hint`. We need to change our code to match.

![Screen Shot 2022-10-20 at 15 26 58](https://user-images.githubusercontent.com/443389/197051545-59d60ba9-af91-42d7-8d00-d515c9210d4c.png)

@SSPJ knows this code the best of any of us.


Discussion:

Resolving patch:
diff --git a/src/djangooidc/views.py b/src/djangooidc/views.py
--- a/src/djangooidc/views.py
+++ b/src/djangooidc/views.py
@@ -84,8 +84,7 @@ def logout(request, next_page=None):
     try:
         username = request.user.username
         request_args = {
-            # it is perfectly fine to send the token, even if it is expired
-            "id_token_hint": request.session["id_token_raw"],
+            "client_id": CLIENT.client_id,
             "state": request.session["state"],
         }
         if (


Test patch:
diff --git a/src/djangooidc/tests/test_views.py b/src/djangooidc/tests/test_views.py
--- a/src/djangooidc/tests/test_views.py
+++ b/src/djangooidc/tests/test_views.py
@@ -81,7 +81,6 @@ def test_login_callback_raises(self, mock_auth, mock_client):
     def test_logout_redirect_url(self, mock_client):
         # setup
         session = self.client.session
-        session["id_token_raw"] = "TEST"  # nosec B105
         session["state"] = "TEST"  # nosec B105
         session.save()
         # mock
@@ -92,12 +91,13 @@ def test_logout_redirect_url(self, mock_client):
         mock_client.provider_info = {
             "end_session_endpoint": "http://example.com/log_me_out"
         }
+        mock_client.client_id = "TEST"
         # test
         with less_console_noise():
             response = self.client.get(reverse("logout"))
         # assert
         expected = (
-            "http://example.com/log_me_out?id_token_hint=TEST&state"
+            "http://example.com/log_me_out?client_id=TEST&state"
             "=TEST&post_logout_redirect_uri=http%3A%2F%2Fexample.com%2Fback"
         )
         actual = response.url
Repository: cisagov/manage.get.gov

Issue:
Add a DEMO SITE banner to unstable and staging
As we develop on public cloud.gov sites, the USWDS theme encourages people to believe that this is an official government site. But of course it is not. We should  add a LARGE RED BANNER with text explaining that these are only demo sites. And configure that banner to appear in the `unstable` and `staging` environments, but it won't appear on a production environment.


Discussion:
GSA's guidance says that sites should "Clearly indicate that it is a staging site" if the site is "publicly accessible". They liink to the [USWDS site alert component](https://designsystem.digital.gov/components/site-alert/) 

Resolving patch:
diff --git a/src/registrar/config/settings.py b/src/registrar/config/settings.py
--- a/src/registrar/config/settings.py
+++ b/src/registrar/config/settings.py
@@ -146,7 +146,6 @@
     BASE_DIR / "assets",
 ]
 
-# TODO: decide on template engine and document in ADR
 TEMPLATES = [
     {
         "BACKEND": "django.template.backends.django.DjangoTemplates",
@@ -168,11 +167,16 @@
                 "django.contrib.messages.context_processors.messages",
                 "registrar.context_processors.language_code",
                 "registrar.context_processors.canonical_path",
+                "registrar.context_processors.is_demo_site",
             ],
         },
     },
 ]
 
+# IS_DEMO_SITE controls whether or not we show our big red "TEST SITE" banner
+# underneath the "this is a real government website" banner.
+IS_DEMO_SITE = True
+
 # endregion
 # region: Database----------------------------------------------------------###
 
diff --git a/src/registrar/context_processors.py b/src/registrar/context_processors.py
--- a/src/registrar/context_processors.py
+++ b/src/registrar/context_processors.py
@@ -21,3 +21,13 @@ def canonical_path(request):
     template itself, so we do it here and pass the information on.
     """
     return {"CANONICAL_PATH": request.build_absolute_uri(request.path)}
+
+
+def is_demo_site(request):
+    """Add a boolean if this is a demo site.
+
+    To be able to render or not our "demo site" banner, we need a context
+    variable for the template that indicates if this banner should or
+    should not appear.
+    """
+    return {"IS_DEMO_SITE": settings.IS_DEMO_SITE}


Test patch: