Repository: inventree/InvenTree

Issue:
News Feed task doesn't work behind proxy, impacting performance
### Please verify that this bug has NOT been raised before.

- [X] I checked and didn't find a similar issue

### Describe the bug*

The `update_news_feed` task attempts to fetch the RSS/Atom feed once daily. This, however, doesn't work behind a proxy server.

The result is that these tasks occupy workers all the time, and never complete.
Each worker is terminated roughly every 90 seconds due to this.

### Steps to Reproduce

1. Put the InvenTree backend on a network unable to reach `INVENTREE_NEWS_URL`
2. Trigger the task
3. Task will lead to continuous timeout termination of workers

### Expected behaviour

Task should finish with no new News entries added if URL is unreachable.

### Deployment Method

- [ ] Docker
- [X] Bare metal

### Version Information

0.12.10

### Please verify if you can reproduce this bug on the demo site.

- [ ] I can reproduce this bug on the demo site.

### Relevant log output

_No response_


Discussion:
@LavissaWoW we have been discussing removing the news feed entirely - it is very rarely updated anyway. Happy to run with an interim fix until we make that change though. 

In this case do you know what is blocking? Is it simply a connection timeout to the remote server?
@SchrodingersGat My guess is a network timeout. My prod instance sits behind a proxy, and from what I see, feedparser doesn't really account for timeouts when trying to use it for requests as well.

I can't really mess a whole lot around and experiment with my prod server, so not too much done on that front. Looking at the feedparser package though, it seems that the intended use of it is only parsing returned data from I.e the requests library. Not running the request itself. Saw some discussions on their GH page about removing the ability for it to do HTTP request by itself.

On my prod, I've deleted that task schedule from my admin view as a workaround. But I suppose that can pop back into existence after a reboot.
@LavissaWoW do you see the same problem in the `check_for_updates` task?
Deleting the scheduled task didn't work. The task is simply re-added to "scheduled tasks" on a daily basis.

![image](https://github.com/inventree/InvenTree/assets/7506688/6fb28de1-3411-4ba6-b5e9-2d12189d7b75)

Only the news_feed is problematic here, And I think the issue lies in how it's implemented. 
As I mentioned above, the maintainer for the `feedparser` package has stated that the ability to do requests with the library _will_ be deprecated in due time.

Move the requests call out of feedparser, and the requests library should handle timeouts etc and cause the task to at least finish, even though it didn'treach the target URL.

So, in essence, something like this:
```python
    # Fetch and parse feed
    try:
        req = requests.get(settings.INVENTREE_NEWS_URL)
        d = feedparser.parse(req)
    except Exception as entry:  # pragma: no cover
        logger.warning('update_news_feed: Error parsing the newsfeed', entry)
        return
```

This simple change will cause the task to properly catch `requests.exceptions.RequestException`s
If you want to be even more fancy, add a global setting for http and https proxy servers and use them with http requests. 
What about to also just ignoring the newsfeed feature if the url is empty, which can be used for such cases?
@LavissaWoW if you can submit a PR with your suggested change, that would be great :)

Resolving patch:
diff --git a/InvenTree/common/tasks.py b/InvenTree/common/tasks.py
--- a/InvenTree/common/tasks.py
+++ b/InvenTree/common/tasks.py
@@ -10,6 +10,7 @@
 from django.utils import timezone
 
 import feedparser
+import requests
 
 from InvenTree.helpers_model import getModelsWithMixin
 from InvenTree.models import InvenTreeNotesMixin
@@ -47,11 +48,16 @@ def update_news_feed():
         logger.info("Could not perform 'update_news_feed' - App registry not ready")
         return
 
+    # News feed isn't defined, no need to continue
+    if not settings.INVENTREE_NEWS_URL or type(settings.INVENTREE_NEWS_URL) != str:
+        return
+
     # Fetch and parse feed
     try:
-        d = feedparser.parse(settings.INVENTREE_NEWS_URL)
-    except Exception as entry:  # pragma: no cover
-        logger.warning('update_news_feed: Error parsing the newsfeed', entry)
+        feed = requests.get(settings.INVENTREE_NEWS_URL)
+        d = feedparser.parse(feed.content)
+    except Exception:  # pragma: no cover
+        logger.warning('update_news_feed: Error parsing the newsfeed')
         return
 
     # Get a reference list


Test patch:
diff --git a/InvenTree/common/test_tasks.py b/InvenTree/common/test_tasks.py
--- a/InvenTree/common/test_tasks.py
+++ b/InvenTree/common/test_tasks.py
@@ -1,8 +1,9 @@
 """Tests for tasks in app common."""
 
+from django.conf import settings
 from django.test import TestCase
 
-from common.models import NotificationEntry
+from common.models import NewsFeedEntry, NotificationEntry
 from InvenTree.tasks import offload_task
 
 from . import tasks as common_tasks
@@ -16,3 +17,77 @@ def test_delete(self):
         # check empty run
         self.assertEqual(NotificationEntry.objects.all().count(), 0)
         offload_task(common_tasks.delete_old_notifications)
+
+
+class NewsFeedTests(TestCase):
+    """Tests for update_news_feed task.
+
+    Tests cover different networking and addressing possibilities.
+    """
+
+    def setUp(self):
+        """Setup for tests."""
+        # Needs to be set to allow SQLite to store entries
+        settings.USE_TZ = True
+
+        # Store setting to restore on teardown
+        self.news_url = settings.INVENTREE_NEWS_URL
+
+        NewsFeedEntry.objects.all().delete()
+
+    def tearDown(self):
+        """Teardown for tests."""
+        # Restore proper setting
+        settings.INVENTREE_NEWS_URL = self.news_url
+
+        NewsFeedEntry.objects.all().delete()
+
+    def test_valid_url(self):
+        """Tests that news feed is updated when accessing a valid URL."""
+        try:
+            common_tasks.update_news_feed()
+        except Exception as ex:
+            self.fail(f'News feed raised exceptions: {ex}')
+
+        self.assertNotEqual(NewsFeedEntry.objects.all().count(), 0)
+
+    def test_connection_error(self):
+        """Test connecting to an unavailable endpoint.
+
+        This also emulates calling the endpoint behind a blocking proxy.
+        """
+        settings.INVENTREE_NEWS_URL = 'http://10.255.255.1:81'
+
+        with self.assertLogs('inventree', level='WARNING'):
+            common_tasks.update_news_feed()
+
+        self.assertEqual(NewsFeedEntry.objects.all().count(), 0)
+
+    def test_unset_url(self):
+        """Test that no call is made to news feed if URL setting is invalid."""
+        settings.INVENTREE_NEWS_URL = ''
+
+        self.assertTrue(
+            offload_task(common_tasks.update_news_feed)
+        )  # Task considered complete
+        self.assertEqual(
+            NewsFeedEntry.objects.all().count(), 0
+        )  # No Feed entries created
+
+        settings.INVENTREE_NEWS_URL = 0
+
+        self.assertTrue(
+            offload_task(common_tasks.update_news_feed)
+        )  # Task considered complete
+        self.assertEqual(
+            NewsFeedEntry.objects.all().count(), 0
+        )  # No Feed entries created
+
+        settings.INVENTREE_NEWS_URL = None
+
+        self.assertTrue(
+            offload_task(common_tasks.update_news_feed)
+        )  # Task considered complete
+        self.assertEqual(
+            NewsFeedEntry.objects.all().count(), 0
+        )  # No Feed entries created