Repository: fossasia/open-event-server

Issue:
Image resizing is not working
https://github.com/fossasia/open-event-server/blob/9bb9f68305c261713ee486f3840ebcf55dc70d23/app/api/helpers/files.py#L80

urlib is throwing 403 Forbidden error for almost all images. 

Please use requests and verify image resizing works. Also, create a management command which finds all events which have no resized images and resizes them


Discussion:
@codedsun Can you please take this up?
Taking this up
@iamareebjamal I learnt that this task creates resized image and stores in static folder. How do i test that for 403 error via which api `upload image?` Please guide me how should I proceed?

As when I am hitting through the frontend and selecting the image like event background, event logo it works fine. 

And for management command do I need to add the task in the celery?
Send this as original image URL in Postman - https://storage.eventyay.com/eventyay.com/events/758/background/Vk0zSTdyZX/1517e947-29df-4a3c-9580-373db3ba9ba9.png

Also, working fine means the image is resized and all image URLs are correctly set. Is that happening?

For management command, no
@iamareebjamal 

![image](https://user-images.githubusercontent.com/20026440/69666656-9c41c300-10b2-11ea-8871-edccba04100e.png)

They are null
@iamareebjamal When the event is created the files.py function is run. What is expected that large_image_url, thumbnail_image_url - should be created with a valid url and saved in the db? Am I right?
Yes
@iamareebjamal Okay I will debug it tomorrow.
Status? PATCH endpoint is not as important. That is atleast working with its limitation. This is completely broken
@iamareebjamal Sorry I missed that due to health issues. I am resuming and working on it now.
@iamareebjamal I am finding a way to check that celery worker is running or not on my machine also how do I turn on logs for celery.. I see that function is called, but I m debugging more about it.
celery won't run if you've not run it. It's simple
Did you follow the instructions and run celery? https://github.com/fossasia/open-event-server/blob/development/docs/installation/basic.md
Yeah, while running `celery worker -A app.celery`

I am having error
```
[2019-12-05 17:39:30,349: WARNING/ForkPoolWorker-3] HEAD http://localhost:9200/event [status:N/A request:0.004s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 238, in perform_request
    method, url, body, retries=Retry(False), headers=request_headers, **kw
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 344, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 181, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x10db7e750>: Failed to establish a new connection: [Errno 61] Connection refused
```
ENABLE_ELASTICSEARCH=False in your .env
@iamareebjamal For the locally saved images on the server it's working fine for me.i.e. From frontend I tried to upload the images, they are getting resized and stored in the events table. 

Now as per the requirement, that creating an event by using a postman request with imageurl of not a locally saved file, it's throwing 403 error. I am debugging it now.
No need to debug. The cause and solution both are written in the issue itself
@iamareebjamal Lol, what should I do then? 
Follow what's written in the issue
>Also, create a management command which finds all events which have no resized images and resizes them?

How do we write a management command, please guide me? 
Also, What should be done for 403 error?


What i understood is that, the image can't be accessed by urllib but that's right as from fronted while uploading images we are uploading that to db and returning a locally saved file URL.

> urlib is throwing 403 Forbidden error for almost all images. 
> 
> Please use requests and verify image resizing works.

I have literally pasted what I wrote in the issue. Now tell me what you don't understand
>No need to debug. The cause and solution both are written in the issue itself

What's the solution here? I need to fix 403 error of urllib

>urlib is throwing 403 Forbidden error for almost all images.

It's throwing for the files which are not saved locally on the server, do I need to fix this for the images which are from the source other than local machine or test for which are locally saved on the server?

Also when I am uploading the images from the frontend , it's working fine. They are not working for images not saved on local machine


Sorry for asking such question but help me where I am not able to understand. Thanks
> Please use [requests](https://github.com/psf/requests) and verify image resizing works.
That hyperlink made a ton of difference for a beginner like me. I thought request as a hitting api.

>Please use requests and verify image resizing works.

For this you meant to replace using urllib and use requests instead to open the file and then perform the resizing. Am I correct here?
Yes
@iamareebjamal If that hyperlink would have been present, it wouldn't have taken a day. Thanks :) 

Resolving patch:
diff --git a/app/api/helpers/files.py b/app/api/helpers/files.py
--- a/app/api/helpers/files.py
+++ b/app/api/helpers/files.py
@@ -5,6 +5,7 @@
 import urllib.parse
 import urllib.request
 import uuid
+import requests
 
 import PIL
 from PIL import Image
@@ -77,7 +78,7 @@ def create_save_resized_image(image_file, basewidth=None, maintain_aspect=None,
     if not image_file:
         return None
     filename = '{filename}.{ext}'.format(filename=get_file_name(), ext=ext)
-    data = urllib.request.urlopen(image_file).read()
+    data = requests.get(image_file).content
     image_file = io.BytesIO(data)
     try:
         im = Image.open(image_file)
@@ -129,7 +130,14 @@ def create_save_image_sizes(image_file, image_sizes_type, unique_identifier=None
     try:
         image_sizes = ImageSizes.query.filter_by(type=image_sizes_type).one()
     except NoResultFound:
-        image_sizes = ImageSizes(image_sizes_type, 1300, 500, True, 100, 75, 30, True, 100, 500, 200, True, 100)
+        image_sizes = ImageSizes(image_sizes_type, full_width=1300,
+                                 full_height=500, full_aspect=True, full_quality=80,
+                                 icon_width=75, icon_height=30, icon_aspect=True,
+                                 icon_quality=80, thumbnail_width=500, thumbnail_height=200,
+                                 thumbnail_aspect=True, thumbnail_quality=80, logo_width=500,
+                                 logo_height=200, icon_size_width_height=35, icon_size_quality=80,
+                                 small_size_width_height=50, small_size_quality=80,
+                                 thumbnail_size_width_height=500)
 
     # Get an unique identifier from uuid if not provided
     if unique_identifier is None:


Test patch: