internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: paperless-ngx/paperless-ngx

Issue:
[Other] Allow access to webhooks for readthedocs
I recently set up the [readthedocs site](https://paperless-ngx.readthedocs.io/en/latest/). Unfortunately, I don't have access to the webhooks settings of the project. 
I requested access to it from the project owner. When granted, the docs will automatically update.
Also we should change some items in https://github.com/paperless-ngx/paperless-ngx/blob/master/docs/conf.py 
(namely `project = u'Paperless-ng'` and `copyright = u'2021, Daniel Quinn, Jonas Winkler'`)

Should we just add `paperless-ngx team` to it?


Discussion:
Great, thanks! I just approved access to readthedocs.

> Also we should change some items in https://github.com/paperless-ngx/paperless-ngx/blob/master/docs/conf.py
(namely project = u'Paperless-ng' and copyright = u'2021, Daniel Quinn, Jonas Winkler')

We can create a PR if it's just those two lines. What should the copyright change to?
maybe `paperless-ngx team`? 

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -27,8 +27,8 @@
 master_doc = 'index'
 
 # General information about the project.
-project = u'Paperless-ng'
-copyright = u'2021, Daniel Quinn, Jonas Winkler'
+project = u'Paperless-ngx'
+copyright = u'2015-2022, Daniel Quinn, Jonas Winkler, and the paperless-ngx team'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[Other] Update name to paperless-ngx in docs
https://github.com/paperless-ngx/paperless-ngx/issues/4#issuecomment-1039890021

> I would go through the places where the project name is mentioned and add the x to it. Only actually code (variable names) could remain unchanged for backwards compatibility.


Discussion:
`vars/paperless-ng.yml` and similar will have to stay as they are for backwards compatibility.
Some links can't be changed yet, for example `https://github.com/jonaswinkler/paperless-ng.git.master` doesn't exist on ngx.
We don't have a release at the moment; `image: jonaswinkler/paperless-ng:0.9.x`.

I can go through whatever can be updated though, then do the rest gradually.

---

Contains 'paperless-ng' or 'jonaswinkler/paperless-ng':
- [ ] https://github.com/paperless-ngx/paperless-ngx/blob/master/.github/workflows/ansible.yml (file structure)
- [ ] https://github.com/paperless-ngx/paperless-ngx/blob/master/ansible/ (various, file structure)
- [ ] https://github.com/paperless-ngx/paperless-ngx/tree/master/docker/compose (various, wait for release)
- [ ] https://github.com/paperless-ngx/paperless-ngx/tree/master/docs (file structure, some repo links)

Not yet inspected:
/scripts
/src
/src-ui

To remain as 'paperless-ng' for backwards compatibility:
- todo

Resolving patch:
diff --git a/src/paperless/urls.py b/src/paperless/urls.py
--- a/src/paperless/urls.py
+++ b/src/paperless/urls.py
@@ -132,8 +132,8 @@
 ]
 
 # Text in each page's <h1> (and above login form).
-admin.site.site_header = "Paperless-ng"
+admin.site.site_header = "Paperless-ngx"
 # Text at the end of each page's <title>.
-admin.site.site_title = "Paperless-ng"
+admin.site.site_title = "Paperless-ngx"
 # Text at the top of the admin index page.
-admin.site.index_title = _("Paperless-ng administration")
+admin.site.index_title = _("Paperless-ngx administration")


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Auto-detected date is day before receipt date
**Describe the bug**
When the system automatically detects the date on the receipt, the "Date Created" field is set to the day before.

**To Reproduce**
Steps to reproduce the behavior:

1. Have system timezone set to UTC-6
2. Scan document that has approved date type
3. See that date is day before date in receipt.

**Expected behavior**
Date should be exact date on receipt.

**Webserver logs**
N/A

**Relevant information**
 - Host OS of the machine running paperless: Docker/Ubuntu 18.04
 - Browser: Chrome
 - Version 1.5.0
 - Installation method: docker
 - Any configuration changes:
```
PAPERLESS_TIME_ZONE=America/Chicago
PAPERLESS_DATE_ORDER=MDY
```


Discussion:
Thanks for the report. Possibly related:

https://github.com/jonaswinkler/paperless-ng/issues/1301
https://github.com/jonaswinkler/paperless-ng/issues/1417
https://github.com/jonaswinkler/paperless-ng/issues/1344
https://github.com/jonaswinkler/paperless-ng/issues/872

Resolving patch:
diff --git a/src/documents/parsers.py b/src/documents/parsers.py
--- a/src/documents/parsers.py
+++ b/src/documents/parsers.py
@@ -224,6 +224,7 @@ def __parser(ds, date_order):
                 "DATE_ORDER": date_order,
                 "PREFER_DAY_OF_MONTH": "first",
                 "RETURN_AS_TIMEZONE_AWARE": True,
+                "TIMEZONE": settings.TIME_ZONE,
             },
         )
 


Test patch:
diff --git a/src/documents/tests/test_consumer.py b/src/documents/tests/test_consumer.py
--- a/src/documents/tests/test_consumer.py
+++ b/src/documents/tests/test_consumer.py
@@ -288,7 +288,7 @@ def get_test_archive_file(self):
         shutil.copy(src, dst)
         return dst
 
-    @override_settings(PAPERLESS_FILENAME_FORMAT=None)
+    @override_settings(PAPERLESS_FILENAME_FORMAT=None, TIME_ZONE="America/Chicago")
     def testNormalOperation(self):
 
         filename = self.get_test_file()
@@ -316,6 +316,8 @@ def testNormalOperation(self):
 
         self._assert_first_last_send_progress()
 
+        self.assertEqual(document.created.tzinfo.zone, "America/Chicago")
+
     @override_settings(PAPERLESS_FILENAME_FORMAT=None)
     def testDeleteMacFiles(self):
         # https://github.com/jonaswinkler/paperless-ng/discussions/1037
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Mail account login via STARTTLS broken
### Description

Mail account login via STARTTLS is broken. See the error in the log file.

### Expected behavior

STARTTLS mail accounts should work normally

### Steps to reproduce

1. Setup a mail account with STARTTLS security and enable a mail rule for synchronization
2. Wait for the task to trigger

### Webserver logs

```bash
10:52:20 [Q] INFO Process-1 created a task from schedule [Check all e-mail accounts]
10:52:20 [Q] INFO Process-1:13 processing [wisconsin-music-enemy-sierra]
10:52:20 [Q] INFO Process-1:13 stopped doing work
10:52:20 [Q] ERROR Failed [wisconsin-music-enemy-sierra] - __init__() got an unexpected keyword argument 'starttls' : Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django_q/cluster.py", line 432, in worker
    res = f(*task["args"], **task["kwargs"])
  File "/usr/src/paperless/src/paperless_mail/tasks.py", line 15, in process_mail_accounts
    total_new_documents += MailAccountHandler().handle_mail_account(account)
  File "/usr/src/paperless/src/paperless_mail/mail.py", line 158, in handle_mail_account
    with get_mailbox(
  File "/usr/src/paperless/src/paperless_mail/mail.py", line 95, in get_mailbox
    mailbox = MailBox(server, port, starttls=True)
TypeError: __init__() got an unexpected keyword argument 'starttls'

10:52:21 [Q] INFO recycled worker Process-1:13
10:52:21 [Q] INFO Process-1:17 ready for work at 574
```


### Screenshots

_No response_

### Paperless-ngx version

1.7.0-rc1

### Host OS

Debian 11

### Installation method

Docker

### Browser

Firefox

### Configuration changes

n.a.

### Other

_No response_


Discussion:
could you add a debug output as well?
Has it worked with ngx-1.6.0 before?
This was a breaking change in imap-tools 52:

https://github.com/ikvk/imap_tools/releases/tag/v0.52.0

Bumped here in #236. This will need to be fixed

Edit: thank you for reporting
I'm working on it right now

Resolving patch:
diff --git a/src/paperless_mail/mail.py b/src/paperless_mail/mail.py
--- a/src/paperless_mail/mail.py
+++ b/src/paperless_mail/mail.py
@@ -18,6 +18,7 @@
 from imap_tools import MailBoxUnencrypted
 from imap_tools import MailMessage
 from imap_tools import MailMessageFlags
+from imap_tools.mailbox import MailBoxTls
 from paperless_mail.models import MailAccount
 from paperless_mail.models import MailRule
 
@@ -92,7 +93,7 @@ def get_mailbox(server, port, security):
     if security == MailAccount.ImapSecurity.NONE:
         mailbox = MailBoxUnencrypted(server, port)
     elif security == MailAccount.ImapSecurity.STARTTLS:
-        mailbox = MailBox(server, port, starttls=True)
+        mailbox = MailBoxTls(server, port)
     elif security == MailAccount.ImapSecurity.SSL:
         mailbox = MailBox(server, port)
     else:


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] CONSUME_SUBDIRS_AS_TAGS not working when using CONSUMER_ENABLE_BARCODES
### Description

my scanner puts files into a specific subdirectory of the `CONSUMPTION_DIR`. However when `CONSUMER_ENABLE_BARCODES` is enabled and a barcode is detected the split documents don't have the expected tags.

It looks like this is caused by `save_to_dir()` in `src/documents/tasks.py` and the fact that it always stores the new documents into `CONSUMPTION_DIR`

### Steps to reproduce

1. enable CONSUMER_ENABLE_BARCODES and CONSUME_SUBDIRS_AS_TAGS 
2. put document with correct Barcode into <CONSUMPTION_DIR>/foo
3. wait for import
4. imported documents don't have the tag `foo`

### Webserver logs

_No response_

### Paperless-ngx version

1.7.1

### Host OS

Debian

### Installation method

Bare metal

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
Thank you, that seems pretty clearly to be an unintended interaction.

Resolving patch:
diff --git a/src/documents/tasks.py b/src/documents/tasks.py
--- a/src/documents/tasks.py
+++ b/src/documents/tasks.py
@@ -87,6 +87,8 @@ def consume_file(
     override_created=None,
 ):
 
+    path = Path(path).resolve()
+
     # check for separators in current document
     if settings.CONSUMER_ENABLE_BARCODES:
 
@@ -122,7 +124,11 @@ def consume_file(
                         newname = f"{str(n)}_" + override_filename
                     else:
                         newname = None
-                    barcodes.save_to_dir(document, newname=newname)
+                    barcodes.save_to_dir(
+                        document,
+                        newname=newname,
+                        target_dir=path.parent,
+                    )
 
                 # if we got here, the document was successfully split
                 # and can safely be deleted


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Internal server error @ fresh install
### Description

Get "Internal server error" with no description after a fresh install.  Have I missed a step, or is this a bug?

### Steps to reproduce

...

### Webserver logs

```bash
Nothing shows up using 'docker-compose logs -f'

webserver_1  | [2022-08-30 23:21:18 +0000] [44] [CRITICAL] WORKER TIMEOUT (pid:47)
webserver_1  | [2022-08-30 23:21:18 +0000] [44] [WARNING] Worker with pid 47 was terminated due to signal 6
```


### Paperless-ngx version

1.8.0

### Host OS

CentOS 7.9 -fully patched with latest patches

### Installation method

Docker - official image

### Browser

Firefox

### Configuration changes

none

### Other

...


Discussion:
I just installed for the first time. Had the same error in the logs. Took a while for paperless to show the website, but finally came up.
Im not sure theres a true bug, no. Please see https://github.com/paperless-ngx/paperless-ngx/discussions/1417 (and many other discussions of this), in other cases the worker getting killed was a resource limitation e.g. OOM. Have you investigated that aspect? https://github.com/paperless-ngx/paperless-ngx/search?q=worker+timeout
I see this occasionally when first starting the container, even in an environment with a lot of resources.  However, it's never affected anything I can tell.  Is it affecting something for you?

The general solutions I've found suggest increasing the gunicorn timeout, but with it already set to 120, that doesn't seem like much of a solution.

Resolving patch:
diff --git a/gunicorn.conf.py b/gunicorn.conf.py
--- a/gunicorn.conf.py
+++ b/gunicorn.conf.py
@@ -1,9 +1,17 @@
 import os
 
+# See https://docs.gunicorn.org/en/stable/settings.html for
+# explanations of settings
+
 bind = f'{os.getenv("PAPERLESS_BIND_ADDR", "[::]")}:{os.getenv("PAPERLESS_PORT", 8000)}'
+
 workers = int(os.getenv("PAPERLESS_WEBSERVER_WORKERS", 1))
 worker_class = "paperless.workers.ConfigurableWorker"
 timeout = 120
+preload_app = True
+
+# https://docs.gunicorn.org/en/stable/faq.html#blocking-os-fchmod
+worker_tmp_dir = "/dev/shm"
 
 
 def pre_fork(server, worker):


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] permission error if no consume share is mapped
### Description

starting with Paperless-ngx 1.8.0, Paperless on docker does not start when no consume share is mapped:

`
  SystemCheckError: System check identified some issues:
  ERRORS:
  ?: PAPERLESS_CONSUMPTION_DIR is not writeable
	HINT: Set the permissions of 
  drwxr-xr-x /usr/src/paperless/consume
  to be writeable by the user running the Paperless services

`

I've some containers running without a mapped share, as I don't need a consumption folder. Was no issue in previous versions; I assume the permissions of the folder in the docker container has changed.

### Steps to reproduce

1. create docker-compose without mapped consumption folder
2. look at the error message

### Webserver logs

```bash
SystemCheckError: System check identified some issues:
ERRORS:
?: PAPERLESS_CONSUMPTION_DIR is not writeable
	HINT: Set the permissions of 
drwxr-xr-x /usr/src/paperless/consume
 to be writeable by the user running the Paperless services
```


### Paperless-ngx version

1.8,0

### Host OS

docker

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
That's curious indeed.  If you get a shell inside the container, what does `ls -ahl /usr/src/paperless` output?

Resolving patch:
diff --git a/src/paperless/checks.py b/src/paperless/checks.py
--- a/src/paperless/checks.py
+++ b/src/paperless/checks.py
@@ -1,4 +1,6 @@
+import grp
 import os
+import pwd
 import shutil
 import stat
 
@@ -32,12 +34,15 @@ def path_check(var, directory):
                 with open(test_file, "w"):
                     pass
             except PermissionError:
+                dir_stat = os.stat(directory)
+                dir_mode = stat.filemode(dir_stat.st_mode)
+                dir_owner = pwd.getpwuid(dir_stat.st_uid).pw_name
+                dir_group = grp.getgrgid(dir_stat.st_gid).gr_name
                 messages.append(
                     Error(
                         writeable_message.format(var),
                         writeable_hint.format(
-                            f"\n{stat.filemode(os.stat(directory).st_mode)} "
-                            f"{directory}\n",
+                            f"\n{dir_mode} {dir_owner} {dir_group} " f"{directory}\n",
                         ),
                     ),
                 )


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] 1.9 OCR output mode "skip_noarchive" is not valid
### Description

With version 1.9 the following env trow errors:
```ini
PAPERLESS_OCR_MODE=skip_noarchive
```

### Steps to reproduce

1. set above docker env variable
2. start container

### Webserver logs

```bash
Paperless-ngx docker container starting...
Mapping UID and GID for paperless:paperless to 50014:50014
Creating directory /tmp/paperless
Adjusting permissions of paperless files. This may take a while.
Waiting for PostgreSQL to start...
Waiting for Redis...
Connected to Redis broker.
Apply database migrations...
SystemCheckError: System check identified some issues:
ERRORS:
?: OCR output mode "skip_noarchive" is not valid
```


### Paperless-ngx version

1.9

### Host OS

Ubuntu 22.04 x86_64

### Installation method

Docker - official image

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
Thanks for the report, I think https://github.com/paperless-ngx/paperless-ngx/pull/1551
So the option is removed or just a failure? The [documentation](https://paperless-ngx.readthedocs.io/en/latest/configuration.html?highlight=PAPERLESS_OCR_MODE#ocr-settings) still lists this value. Hope it's not removed, though.
No, it's still valid, I missed that setting as an option.

Resolving patch:
diff --git a/src/paperless/checks.py b/src/paperless/checks.py
--- a/src/paperless/checks.py
+++ b/src/paperless/checks.py
@@ -127,10 +127,10 @@ def _ocrmypdf_settings_check():
                 Error(f'OCR output type "{settings.OCR_OUTPUT_TYPE}" is not valid'),
             )
 
-        if settings.OCR_MODE not in {"force", "skip", "redo_ocr"}:
+        if settings.OCR_MODE not in {"force", "skip", "redo", "skip_noarchive"}:
             msgs.append(Error(f'OCR output mode "{settings.OCR_MODE}" is not valid'))
 
-        if settings.OCR_CLEAN not in {"clean", "clean_final"}:
+        if settings.OCR_CLEAN not in {"clean", "clean-final"}:
             msgs.append(Error(f'OCR clean mode "{settings.OCR_CLEAN}" is not valid'))
         return msgs
 


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Wrong version number string within docker 1.9.1
### Description

After a successful pull and deploy via docker with https://ghcr.io/paperless-ngx/paperless-ngx:1.9.1 the version string on the paperless-ngx Web-UI is still 1.9.0.

![Bildschirmfoto 2022-09-27 um 16 33 06](https://user-images.githubusercontent.com/48019302/192555664-f7aca659-8180-4d50-b4b8-147b54ca8769.png)


### Steps to reproduce

1. Pull the new version via docker, docker-compose or portainer via https://ghcr.io/paperless-ngx/paperless-ngx and tag 1.9.1
2. Access the Web-UI.
3. Login
4. Find the version string on the lower left side.

### Webserver logs

_No response_

### Paperless-ngx version

1.9.1

### Host OS

Alpine Linux x86-64

### Installation method

Docker - official image

### Browser

Chrome

### Configuration changes

_No response_

### Other

_No response_


Discussion:
Yeah, looks like I forgot to update those.  I'm not positive it's worth a .2 release, but I'll let others chime in.
> Yeah, looks like I forgot to update those. I'm not positive it's worth a .2 release, but I'll let others chime in.

I updated yesterday between the 1.9.0 release and the 1.9.1 release and got confused after updating two times and the message not going away. I decided just to wait what happens. 

A .2 would minimize confusion and possible support.

Resolving patch:
diff --git a/src/paperless/version.py b/src/paperless/version.py
--- a/src/paperless/version.py
+++ b/src/paperless/version.py
@@ -1,7 +1,7 @@
 from typing import Final
 from typing import Tuple
 
-__version__: Final[Tuple[int, int, int]] = (1, 9, 0)
+__version__: Final[Tuple[int, int, int]] = (1, 9, 2)
 # Version string like X.Y.Z
 __full_version_str__: Final[str] = ".".join(map(str, __version__))
 # Version string like X.Y


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] 1.9.1 OCR output mode "none" is not valid
### Description

Very similar to #1643 

Updating via portainer, and with the following ENV:

PAPERLESS_OCR_CLEAN=none

the startup of the web server goes into a loop (see logs below).



### Steps to reproduce

1. Set PAPERLESS_OCR_CLEAN=none (from 1.8.0 install)
2. Update docker via portainer to latest paperless-ngx
3. Start container

### Webserver logs

```bash
Paperless-ngx docker container starting...
Creating directory /tmp/paperless
Adjusting permissions of paperless files. This may take a while.
Waiting for PostgreSQL to start...
Waiting for Redis...
Connected to Redis broker.
Apply database migrations...
SystemCheckError: System check identified some issues:
ERRORS:
?: OCR clean mode "none" is not valid
```


### Paperless-ngx version

1.9.1

### Host OS

Raspberry Pi 4B

### Installation method

Docker - official image

### Browser

Firefox

### Configuration changes

_No response_

### Other

_No response_


Discussion:

Resolving patch:
diff --git a/src/paperless/checks.py b/src/paperless/checks.py
--- a/src/paperless/checks.py
+++ b/src/paperless/checks.py
@@ -130,7 +130,7 @@ def _ocrmypdf_settings_check():
         if settings.OCR_MODE not in {"force", "skip", "redo", "skip_noarchive"}:
             msgs.append(Error(f'OCR output mode "{settings.OCR_MODE}" is not valid'))
 
-        if settings.OCR_CLEAN not in {"clean", "clean-final"}:
+        if settings.OCR_CLEAN not in {"clean", "clean-final", "none"}:
             msgs.append(Error(f'OCR clean mode "{settings.OCR_CLEAN}" is not valid'))
         return msgs
 


Test patch:
Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Incorrectly listing new documents as duplicates
### Description

I use dockerized version 1.9.2  (image: ghcr.io/paperless-ngx/paperless-ngx:latest)
and use usually use a consume folder for feeding paperless. I just recently updated to that version.
I have now several documents that are kept in that folder and are shown in the log as: "Not consuming" as "it is a duplicate". 

However that is not correct, there are no duplicates to be found. From the task view I have only the option to abort, but no option to overwrite and ignore. I also seem to have no option to delete the "false original".

### Steps to reproduce

add document into consume folder.
Receive error

### Webserver logs

```bash
[2022-09-30 10:29:23,621] [ERROR] [paperless.consumer] Not consuming 1802140_Stromabrechnung_2020.pdf: It is a duplicate.
```


### Paperless-ngx version

1.9.2

### Host OS

Synology Docker

### Installation method

Other (please describe above)

### Browser

_No response_

### Configuration changes

_No response_

### Other

_No response_


Discussion:
Paperless checks for duplicates by comparing and md5 hash so I think its unlikely that its wrong, tbh see https://github.com/paperless-ngx/paperless-ngx/blob/5639659b6384fc675569fbec044bc2757a6696f5/src/documents/consumer.py#L111 and its not something thats been reported by others.

Have you looked through your existing docs to make absolutely certain that its incorrect?
This does present a nice enhancement. It should be very easy to get the title of the already stored duplicate. That's be a good qol
Yea had the same thought!
Yes, I can confirm that those documents did not exist. Simply as I had just recieved them and stored them in the consumption folder. Nevertheless did I try to find them with some keywords, but no luck, otherwise I wouldn't have raised the ticket.
I restarted the container now, didn't change the situation for those documents, but newer ones are now processed again.
To get those in question into paperless I re-printed them as PDFs and afterwards I was able to feed them.

Resolving patch:
diff --git a/src/documents/consumer.py b/src/documents/consumer.py
--- a/src/documents/consumer.py
+++ b/src/documents/consumer.py
@@ -111,14 +111,16 @@ def pre_check_file_exists(self):
     def pre_check_duplicate(self):
         with open(self.path, "rb") as f:
             checksum = hashlib.md5(f.read()).hexdigest()
-        if Document.objects.filter(
+        existing_doc = Document.objects.filter(
             Q(checksum=checksum) | Q(archive_checksum=checksum),
-        ).exists():
+        )
+        if existing_doc.exists():
             if settings.CONSUMER_DELETE_DUPLICATES:
                 os.unlink(self.path)
             self._fail(
                 MESSAGE_DOCUMENT_ALREADY_EXISTS,
-                f"Not consuming {self.filename}: It is a duplicate.",
+                f"Not consuming {self.filename}: It is a duplicate of"
+                f" {existing_doc.get().title} (#{existing_doc.get().pk})",
             )
 
     def pre_check_directories(self):


Test patch: