internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: Netflix/lemur

Issue:
Add 'replacedBy' field in certificate response.
In addition to the 'replaces' field, we should also have a field of 'replacedBy' so we can easily determine if a certificate has been replaced and should no longer be used.


Discussion:
yes, having a field that links/points to the most recent version of the cert would be very helpful. Say if cert 2 is replaced by cert 4 with is then renewed/reissued by cert 55. The replaced by  field in cert 2 should either point to cert 55 or show that it was replaced by cert 4 then 55. ie. Replaced By: 4 -> 55

This would help to show the history of the cert along with what the current version is. I see that in the API output cert 55 contains the certs it replaces which is handy.


Field has been named `replaced`.

Resolving patch:
diff --git a/lemur/certificates/schemas.py b/lemur/certificates/schemas.py
--- a/lemur/certificates/schemas.py
+++ b/lemur/certificates/schemas.py
@@ -175,6 +175,7 @@ class CertificateOutputSchema(LemurOutputSchema):
     authority = fields.Nested(AuthorityNestedOutputSchema)
     roles = fields.Nested(RoleNestedOutputSchema, many=True)
     endpoints = fields.Nested(EndpointNestedOutputSchema, many=True, missing=[])
+    replaced = fields.Nested(CertificateNestedOutputSchema, many=True)
 
 
 class CertificateUploadInputSchema(CertificateCreationSchema):
diff --git a/lemur/certificates/views.py b/lemur/certificates/views.py
--- a/lemur/certificates/views.py
+++ b/lemur/certificates/views.py
@@ -99,6 +99,7 @@ def get(self):
                         "name": "*.test.example.net"
                     }],
                     "replaces": [],
+                    "replaced": [],
                     "name": "WILDCARD.test.example.net-SymantecCorporation-20160603-20180112",
                     "roles": [{
                         "id": 464,
@@ -520,6 +521,7 @@ def get(self, certificate_id):
                     "name": "*.test.example.net"
                 }],
                 "replaces": [],
+                "replaced": [],
                 "name": "WILDCARD.test.example.net-SymantecCorporation-20160603-20180112",
                 "roles": [{
                     "id": 464,
@@ -720,6 +722,7 @@ def get(self, notification_id):
                         "name": "*.test.example.net"
                     }],
                     "replaces": [],
+                    "replaced": [],
                     "name": "WILDCARD.test.example.net-SymantecCorporation-20160603-20180112",
                     "roles": [{
                         "id": 464,
@@ -824,6 +827,7 @@ def get(self, certificate_id):
                         "name": "*.test.example.net"
                     }],
                     "replaces": [],
+                    "replaced": [],
                     "name": "WILDCARD.test.example.net-SymantecCorporation-20160603-20180112",
                     "roles": [{
                         "id": 464,


Test patch:
Repository: Netflix/lemur

Issue:
Fix notification filter in UI.



Discussion:

Resolving patch:
diff --git a/lemur/certificates/service.py b/lemur/certificates/service.py
--- a/lemur/certificates/service.py
+++ b/lemur/certificates/service.py
@@ -9,7 +9,7 @@
 from datetime import timedelta
 
 from flask import current_app
-from sqlalchemy import func, or_
+from sqlalchemy import func, or_, cast, Boolean
 
 from cryptography import x509
 from cryptography.hazmat.backends import default_backend
@@ -294,6 +294,8 @@ def render(args):
 
         elif 'destination' in terms:
             query = query.filter(Certificate.destinations.any(Destination.id == terms[1]))
+        elif 'notify' in filt:
+            query = query.filter(Certificate.notify == cast(terms[1], Boolean))
         elif 'active' in filt:
             query = query.filter(Certificate.active == terms[1])
         elif 'cn' in terms:


Test patch:
Repository: Netflix/lemur

Issue:
Better expose the certificate ID in the Lemur UI
Our application integrated with Lemur using its API and we use Lemur's primary key -- the certificate id -- throughout our system.

I'd like to be able to search and filter by certificate ID in the UI.

This is especially important for frequently-renewed certificates that may have tens or hundreds of entries in Lemur.


Discussion:

Resolving patch:
diff --git a/lemur/certificates/service.py b/lemur/certificates/service.py
--- a/lemur/certificates/service.py
+++ b/lemur/certificates/service.py
@@ -9,7 +9,7 @@
 from datetime import timedelta
 
 from flask import current_app
-from sqlalchemy import func, or_, not_, cast, Boolean
+from sqlalchemy import func, or_, not_, cast, Boolean, Integer
 
 from cryptography import x509
 from cryptography.hazmat.backends import default_backend
@@ -299,6 +299,8 @@ def render(args):
                     Certificate.domains.any(Domain.name.ilike('%{0}%'.format(terms[1])))
                 )
             )
+        elif 'id' in terms:
+            query = query.filter(Certificate.id == cast(terms[1], Integer))
         else:
             query = database.filter(query, Certificate, terms)
 


Test patch:
Repository: Netflix/lemur

Issue:
Ensure rotation column == 'False' during migration.
Null values creates problems during validation.


Discussion:

Resolving patch:
diff --git a/lemur/migrations/versions/131ec6accff5_.py b/lemur/migrations/versions/131ec6accff5_.py
--- a/lemur/migrations/versions/131ec6accff5_.py
+++ b/lemur/migrations/versions/131ec6accff5_.py
@@ -16,7 +16,7 @@
 
 def upgrade():
     # ### commands auto generated by Alembic - please adjust! ###
-    op.add_column('certificates', sa.Column('rotation', sa.Boolean(), nullable=True))
+    op.add_column('certificates', sa.Column('rotation', sa.Boolean(), nullable=False, server_default=False))
     op.add_column('endpoints', sa.Column('last_updated', sa.DateTime(), server_default=sa.text('now()'), nullable=False))
     # ### end Alembic commands ###
 


Test patch:
Repository: Netflix/lemur

Issue:
[Web-UI] Active button for authority is not working
I have list of authorities created on the lemur. I want to disable creation of new certificates using one of the authority. Clicking on **active** button against that authority should make that authority inactive which is not the case.

Even the PUT call to the `http://example.com/api/1/authorities/5` with active key's value set to false returns true in response. Please find the request and response for the API in the file given below.
[active.txt](https://github.com/Netflix/lemur/files/878813/active.txt)



Discussion:

Resolving patch:
diff --git a/lemur/authorities/schemas.py b/lemur/authorities/schemas.py
--- a/lemur/authorities/schemas.py
+++ b/lemur/authorities/schemas.py
@@ -70,7 +70,7 @@ def ensure_dates(self, data):
 class AuthorityUpdateSchema(LemurInputSchema):
     owner = fields.Email(required=True)
     description = fields.String()
-    active = fields.Boolean()
+    active = fields.Boolean(missing=True)
     roles = fields.Nested(AssociatedRoleSchema(many=True))
 
 
diff --git a/lemur/authorities/service.py b/lemur/authorities/service.py
--- a/lemur/authorities/service.py
+++ b/lemur/authorities/service.py
@@ -29,9 +29,7 @@ def update(authority_id, description=None, owner=None, active=None, roles=None):
     if roles:
         authority.roles = roles
 
-    if active:
-        authority.active = active
-
+    authority.active = active
     authority.description = description
     authority.owner = owner
     return database.update(authority)


Test patch:
Repository: Netflix/lemur

Issue:
Set lemur to log to stdout
When running lemur inside docker I would like to have it log everything to `stdout` so that I can forward logs to splunk. At the moment `lemur.config.py` has a `LEMUR_LOG` parameter that expects a filename. Is there a way to configure lemur to log to stdout instead of a file?


Discussion:
Looks like it is configured to also stream to stdout but for some reason it does not have a predefined `LOG_LEVEL` like the file handler one: https://github.com/Netflix/lemur/blob/02991c70a9878f8ba9a10da13a18532affaa1cb5/lemur/factory.py#L155

Resolving patch:
diff --git a/lemur/factory.py b/lemur/factory.py
--- a/lemur/factory.py
+++ b/lemur/factory.py
@@ -153,7 +153,7 @@ def configure_logging(app):
     app.logger.addHandler(handler)
 
     stream_handler = StreamHandler()
-    stream_handler.setLevel(app.config.get('LOG_LEVEL'))
+    stream_handler.setLevel(app.config.get('LOG_LEVEL', 'DEBUG'))
     app.logger.addHandler(stream_handler)
 
 


Test patch:
Repository: Netflix/lemur

Issue:
Adding domain fails on unselectable "sensitive"
Client side

![screenshot_2017-05-12_11-06-54](https://cloud.githubusercontent.com/assets/445200/25991405/528d417a-3703-11e7-9e6c-d70beb6d38e2.png)


Server side

```
May 12 09:05:48 lemur supervisord: lemur-web [2017-05-12 09:05:48,892] ERROR in schema: 'sensitive'
May 12 09:05:48 lemur supervisord: lemur-web Traceback (most recent call last):
May 12 09:05:48 lemur supervisord: lemur-web   File "/var/www/lemur/lemur/common/schema.py", line 158, in decorated_function
May 12 09:05:48 lemur supervisord: lemur-web     resp = f(*args, **kwargs)
May 12 09:05:48 lemur supervisord: lemur-web   File "/var/www/lemur/lemur/domains/views.py", line 126, in post
May 12 09:05:48 lemur supervisord: lemur-web     return service.create(data['name'], data['sensitive'])
May 12 09:05:48 lemur supervisord: lemur-web KeyError: 'sensitive'
May 12 09:05:48 lemur supervisord: lemur-web
May 12 09:05:48 lemur supervisord: lemur-web 'sensitive'
May 12 09:05:48 lemur supervisord: lemur-web Traceback (most recent call last):
May 12 09:05:48 lemur supervisord: lemur-web   File "/var/www/lemur/lemur/common/schema.py", line 158, in decorated_function
May 12 09:05:48 lemur supervisord: lemur-web     resp = f(*args, **kwargs)
May 12 09:05:48 lemur supervisord: lemur-web   File "/var/www/lemur/lemur/domains/views.py", line 126, in post
May 12 09:05:48 lemur supervisord: lemur-web     return service.create(data['name'], data['sensitive'])
May 12 09:05:48 lemur supervisord: lemur-web KeyError: 'sensitive'
```


Discussion:
Made a PR for this is at #795 -- but can't search for the domain in the UI.
^^
This was just the python fix, didn't touch the UI part.

Resolving patch:
diff --git a/lemur/domains/schemas.py b/lemur/domains/schemas.py
--- a/lemur/domains/schemas.py
+++ b/lemur/domains/schemas.py
@@ -15,7 +15,7 @@
 class DomainInputSchema(LemurInputSchema):
     id = fields.Integer()
     name = fields.String(required=True)
-    sensitive = fields.Boolean()
+    sensitive = fields.Boolean(missing=False)
     certificates = fields.Nested(AssociatedCertificateSchema, many=True, missing=[])
 
 


Test patch:
Repository: Netflix/lemur

Issue:
Return error other than 401 when a user doesn't have access to issuer.
Right now we return a 401 when a user attempts to use an issuer they don't have access to. During a clone operation, if an issuer is no longer available this status code, sends the user into a login redirect loop. 


Discussion:

Resolving patch:
diff --git a/lemur/auth/views.py b/lemur/auth/views.py
--- a/lemur/auth/views.py
+++ b/lemur/auth/views.py
@@ -164,17 +164,17 @@ def post(self):
                 algo = header_data['alg']
                 break
         else:
-            return dict(message='Key not found'), 403
+            return dict(message='Key not found'), 401
 
         # validate your token based on the key it was signed with
         try:
             jwt.decode(id_token, secret.decode('utf-8'), algorithms=[algo], audience=args['clientId'])
         except jwt.DecodeError:
-            return dict(message='Token is invalid'), 403
+            return dict(message='Token is invalid'), 401
         except jwt.ExpiredSignatureError:
-            return dict(message='Token has expired'), 403
+            return dict(message='Token has expired'), 401
         except jwt.InvalidTokenError:
-            return dict(message='Token is invalid'), 403
+            return dict(message='Token is invalid'), 401
 
         user_params = dict(access_token=access_token, schema='profile')
 
diff --git a/lemur/certificates/views.py b/lemur/certificates/views.py
--- a/lemur/certificates/views.py
+++ b/lemur/certificates/views.py
@@ -271,7 +271,7 @@ def post(self, data=None):
             data['creator'] = g.user
             return service.create(**data)
 
-        return dict(message="You are not authorized to use {0}".format(data['authority'].name)), 403
+        return dict(message="You are not authorized to use the authority: {0}".format(data['authority'].name)), 403
 
 
 class CertificatesUpload(AuthenticatedResource):


Test patch:
Repository: Netflix/lemur

Issue:
Allow aws plugin to upload certificates to cloudfront.
Only real change needed is the ability to set the "path" parameter when uploading the certificate to IAM.
I would envision this as an option for the AWS destination plugin and have explicit sources for certificates. Additionally we may need to modify the the name of the cloudfront certificate, if the certificate is used by both ELB and cloudfront as there will be a name collision otherwise. 



Discussion:

Resolving patch:
diff --git a/lemur/plugins/lemur_aws/iam.py b/lemur/plugins/lemur_aws/iam.py
--- a/lemur/plugins/lemur_aws/iam.py
+++ b/lemur/plugins/lemur_aws/iam.py
@@ -53,7 +53,7 @@ def create_arn_from_cert(account_number, region, certificate_name):
 
 @sts_client('iam')
 @retry(retry_on_exception=retry_throttled, stop_max_attempt_number=7, wait_exponential_multiplier=100)
-def upload_cert(name, body, private_key, cert_chain=None, **kwargs):
+def upload_cert(name, body, private_key, path, cert_chain=None, **kwargs):
     """
     Upload a certificate to AWS
 
@@ -61,12 +61,14 @@ def upload_cert(name, body, private_key, cert_chain=None, **kwargs):
     :param body:
     :param private_key:
     :param cert_chain:
+    :param path:
     :return:
     """
     client = kwargs.pop('client')
     try:
         if cert_chain:
             return client.upload_server_certificate(
+                Path=path,
                 ServerCertificateName=name,
                 CertificateBody=str(body),
                 PrivateKey=str(private_key),
@@ -74,6 +76,7 @@ def upload_cert(name, body, private_key, cert_chain=None, **kwargs):
             )
         else:
             return client.upload_server_certificate(
+                Path=path,
                 ServerCertificateName=name,
                 CertificateBody=str(body),
                 PrivateKey=str(private_key)
diff --git a/lemur/plugins/lemur_aws/plugin.py b/lemur/plugins/lemur_aws/plugin.py
--- a/lemur/plugins/lemur_aws/plugin.py
+++ b/lemur/plugins/lemur_aws/plugin.py
@@ -161,6 +161,12 @@ class AWSDestinationPlugin(DestinationPlugin):
             'required': True,
             'validation': '/^[0-9]{12,12}$/',
             'helpMessage': 'Must be a valid AWS account number!',
+        },
+        {
+            'name': 'path',
+            'type': 'str',
+            'default': '/',
+            'helpMessage': 'Path to upload certificate.'
         }
     ]
 
@@ -172,6 +178,7 @@ class AWSDestinationPlugin(DestinationPlugin):
 
     def upload(self, name, body, private_key, cert_chain, options, **kwargs):
         iam.upload_cert(name, body, private_key,
+                        self.get_option('path', options),
                         cert_chain=cert_chain,
                         account_number=self.get_option('accountNumber', options))
 


Test patch:
Repository: Netflix/lemur

Issue:
TypeError: IPv4Address('192.168.0.1') is not JSON serializable
During create new certificate with Subj Alt Name IPAddress
`2017-05-25 16:48:16,503 ERROR: Exception on /api/1/certificates [GET] [in /opt/lemur/lib/python3.4/site-packages/flask/app.py:1560]
Traceback (most recent call last):
  File "/opt/lemur/lib/python3.4/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/lemur/lib/python3.4/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/lemur/lib/python3.4/site-packages/flask_restful/__init__.py", line 481, in wrapper
    return self.make_response(data, code, headers=headers)
  File "/opt/lemur/lib/python3.4/site-packages/flask_restful/__init__.py", line 510, in make_response
    resp = self.representations[mediatype](data, *args, **kwargs)
  File "/opt/lemur/lib/python3.4/site-packages/flask_restful/representations/json.py", line 20, in output_json
    dumped = dumps(data, **settings) + "\n"
  File "/usr/lib64/python3.4/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib64/python3.4/json/encoder.py", line 192, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib64/python3.4/json/encoder.py", line 250, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib64/python3.4/json/encoder.py", line 173, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: IPv4Address('192.168.0.1') is not JSON serializable
`
![2017-05-25 15_07_49-lemur - opera](https://cloud.githubusercontent.com/assets/17988004/26466256/0f888a8e-415c-11e7-94c1-5aae50cef0e2.png)







Discussion:

Resolving patch:
diff --git a/lemur/common/fields.py b/lemur/common/fields.py
--- a/lemur/common/fields.py
+++ b/lemur/common/fields.py
@@ -327,7 +327,12 @@ def _serialize(self, value, attr, obj):
                     name_type = 'DNSName'
 
                 elif isinstance(name, x509.IPAddress):
-                    name_type = 'IPAddress'
+                    if isinstance(value, ipaddress.IPv4Network):
+                        name_type = 'IPNetwork'
+                    else:
+                        name_type = 'IPAddress'
+
+                    value = str(value)
 
                 elif isinstance(name, x509.UniformResourceIdentifier):
                     name_type = 'uniformResourceIdentifier'


Test patch:
Repository: Netflix/lemur

Issue:
Missing 'default' rotation policy
When trying to create a certificate, the error message is displayed:

`{"_schema":"Unable to find <class 'lemur.policies.models.RotationPolicy'> with name: default"}`


Discussion:

Resolving patch:
diff --git a/lemur/manage.py b/lemur/manage.py
--- a/lemur/manage.py
+++ b/lemur/manage.py
@@ -251,7 +251,7 @@ def run(self, password):
             days=days
         ))
 
-        policy_service.create(days=days)
+        policy_service.create(days=days, name='default')
         sys.stdout.write("[/] Done!\n")
 
 


Test patch:
Repository: Netflix/lemur

Issue:
Allow Digicert CIS to specify end time as well as end date
Currently we truncate the the times sent to digicert, we should respect the times given to us:

https://github.com/Netflix/lemur/blob/master/lemur/plugins/lemur_digicert/plugin.py#L160

Valid format for `valid_to` is `YYYY-MM-DDTHH:MM:SSZ`

cc @andrewachen 


Discussion:

Resolving patch:
diff --git a/lemur/plugins/lemur_digicert/plugin.py b/lemur/plugins/lemur_digicert/plugin.py
--- a/lemur/plugins/lemur_digicert/plugin.py
+++ b/lemur/plugins/lemur_digicert/plugin.py
@@ -157,7 +157,7 @@ def map_cis_fields(options, csr):
         "csr": csr,
         "signature_hash": signature_hash(options.get('signing_algorithm')),
         "validity": {
-            "valid_to": options['validity_end'].format('YYYY-MM-DD')
+            "valid_to": options['validity_end'].format('YYYY-MM-DDTHH:MM:SSZ')
         },
         "organization": {
             "name": options['organization'],


Test patch:
diff --git a/lemur/plugins/lemur_digicert/tests/test_digicert.py b/lemur/plugins/lemur_digicert/tests/test_digicert.py
--- a/lemur/plugins/lemur_digicert/tests/test_digicert.py
+++ b/lemur/plugins/lemur_digicert/tests/test_digicert.py
@@ -103,7 +103,7 @@ def test_map_cis_fields(app):
         'signature_hash': 'sha256',
         'organization': {'name': 'Example, Inc.', 'units': ['Example Org']},
         'validity': {
-            'valid_to': arrow.get(2017, 5, 7).format('YYYY-MM-DD')
+            'valid_to': arrow.get(2017, 5, 7).format('YYYY-MM-DDTHH:MM:SSZ')
         },
         'profile_name': None
     }
@@ -132,7 +132,7 @@ def test_map_cis_fields(app):
             'signature_hash': 'sha256',
             'organization': {'name': 'Example, Inc.', 'units': ['Example Org']},
             'validity': {
-                'valid_to': arrow.get(2018, 11, 3).format('YYYY-MM-DD')
+                'valid_to': arrow.get(2018, 11, 3).format('YYYY-MM-DDTHH:MM:SSZ')
             },
             'profile_name': None
         }
Repository: Netflix/lemur

Issue:
Running init more than once will create multiple 'default' rotation policies
When you run `init`, it checks for default roles, default users, etc. and skips creating them if those are found.  However, default rotation policy ('default': 30) is created and inserted into the database every time.  This causes an issue when you try to create a certificate without a rotation policy, then the default rotation policy is used (named: "default").  This commonly occurs if you use lemur-docker to test, and then only restart the web container and keep the existing lemur-postgres container with the existing data.

The fix is to either go into the DB and remove the duplicate.  We should check if 'default' already exists (name should probably be set to unique as well).  


Discussion:
This makes sense, rotation policy was added quite a bit after the init stuff was done. I may have missed checking for existence of the policy. 

Resolving patch:
diff --git a/lemur/manage.py b/lemur/manage.py
--- a/lemur/manage.py
+++ b/lemur/manage.py
@@ -251,12 +251,17 @@ def run(self, password):
         recipients = current_app.config.get('LEMUR_SECURITY_TEAM_EMAIL')
         notification_service.create_default_expiration_notifications("DEFAULT_SECURITY", recipients=recipients)
 
-        days = current_app.config.get("LEMUR_DEFAULT_ROTATION_INTERVAL", 30)
-        sys.stdout.write("[+] Creating default certificate rotation policy of {days} days before issuance.\n".format(
-            days=days
-        ))
+        _DEFAULT_ROTATION_INTERVAL = 'default'
+        default_rotation_interval = policy_service.get_by_name(_DEFAULT_ROTATION_INTERVAL)
+
+        if default_rotation_interval:
+            sys.stdout.write("[-] Default rotation interval policy already created, skipping...!\n")
+        else:
+            days = current_app.config.get("LEMUR_DEFAULT_ROTATION_INTERVAL", 30)
+            sys.stdout.write("[+] Creating default certificate rotation policy of {days} days before issuance.\n".format(
+                days=days))
+            policy_service.create(days=days, name=_DEFAULT_ROTATION_INTERVAL)
 
-        policy_service.create(days=days, name='default')
         sys.stdout.write("[/] Done!\n")
 
 
diff --git a/lemur/policies/service.py b/lemur/policies/service.py
--- a/lemur/policies/service.py
+++ b/lemur/policies/service.py
@@ -18,6 +18,15 @@ def get(policy_id):
     return database.get(RotationPolicy, policy_id)
 
 
+def get_by_name(policy_name):
+    """
+    Retrieves policy by its name.
+    :param policy_name:
+    :return:
+    """
+    return database.get_all(RotationPolicy, policy_name, field='name').all()
+
+
 def delete(policy_id):
     """
     Delete a rotation policy.


Test patch: