Repository: modin-project/modin

Issue:
Psycopg2 connections not supported in parallel
copied from #694.

I did install from master branch 
```

In [1]: import pandas as pd

In [2]: import psycopg2
   ...:
   ...:
   ...: CONN = psycopg2.connect(dbname="tmpdb", user="xx",
   ...:                             password="xx", host="xx")

In [3]: df = pd.read_sql("SELECT time FROM tmpdb", CONN)

In [4]: import modin.pandas as pd

In [5]: df = pd.read_sql("SELECT time FROM tmpdb", CONN)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/stream_exit/lib/python3.6/site-packages/ray/worker.py in put_object(self, object_id, value)
    382         try:
--> 383             self.store_and_register(object_id, value)
    384         except pyarrow.PlasmaObjectExists:

~/stream_exit/lib/python3.6/site-packages/ray/worker.py in store_and_register(self, object_id, value, depth)
    316                         serialization_context=self.get_serialization_context(
--> 317                             self.task_driver_id))
    318                 break

~/stream_exit/lib/python3.6/site-packages/ray/utils.py in _wrapper(*args, **kwargs)
    478                     with self.lock:
--> 479                         return orig_attr(*args, **kwargs)
    480

~/stream_exit/lib/python3.6/site-packages/ray/pyarrow_files/pyarrow/_plasma.pyx in pyarrow._plasma.PlasmaClient.put()

~/stream_exit/lib/python3.6/site-packages/ray/pyarrow_files/pyarrow/serialization.pxi in pyarrow.lib.serialize()

~/stream_exit/lib/python3.6/site-packages/ray/pyarrow_files/pyarrow/serialization.pxi in pyarrow.lib.SerializationContext._serialize_callback()

~/stream_exit/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py in dumps(obj, protocol)
    951         cp = CloudPickler(file, protocol=protocol)
--> 952         cp.dump(obj)
    953         return file.getvalue()

~/stream_exit/lib/python3.6/site-packages/ray/cloudpickle/cloudpickle.py in dump(self, obj)
    266         try:
--> 267             return Pickler.dump(self, obj)
    268         except RuntimeError as e:

/usr/local/var/pyenv/versions/3.6.4/lib/python3.6/pickle.py in dump(self, obj)
    408             self.framer.start_framing()
--> 409         self.save(obj)
    410         self.write(STOP)

/usr/local/var/pyenv/versions/3.6.4/lib/python3.6/pickle.py in save(self, obj, save_persistent_id)
    495             if reduce is not None:
--> 496                 rv = reduce(self.proto)
    497             else:

TypeError: can't pickle psycopg2.extensions.connection objects
```
I get this error


Discussion:
Thanks @devin-petersohn . Last time i tried to pickle, i was blocked as i couldnt do it . Interested to see how you you will be able to do it . 
may be this helps 
https://stackoverflow.com/questions/52537741/multiprocessing-psycopg2-typeerror-cant-pickle-thread-rlock-objects
Thanks much again
any update on the fix > I see the milestone 0.6.0 is today. Can we expect the fix by EOD?
Yes, we are planning to release today, but many of the bigger improvements took longer than expected (Windows support particularly) so it won't be 0.6.0, but probably a micro release 0.5.4. We will get several of these bugfixes in today and test and release.
@qxlsz I came up with a bit of a solution for us to use internally, but would you see if this works for you?

```python
In [1]: import pandas as pd

In [2]: import psycopg2
   ...:
   ...:
   ...: CONN = psycopg2.connect(dbname="tmpdb", user="xx",
   ...:                             password="xx", host="xx")

In [3]: CONN = "postgresql+psycopg2://{}:{}@{}{}/{}".format(  # Table in DB
   ...:                        CONN.info.user,  # <Username>: for DB
   ...:                        CONN.info.password,  # Password for DB
   ...:                        CONN.info.host if CONN.info.host != "/tmp" else "",  # @<Hostname>
   ...:                        (":" + str(CONN.info.port)) if CONN.info.host != "/tmp" else "",  # <port>
   ...:                        CONN.info.dbname,  # Table in DB
   ...:                    )

In [4]: df = pd.read_sql("SELECT time FROM tmpdb", CONN)

In [5]: import modin.pandas as pd

In [6]: df = pd.read_sql("SELECT time FROM tmpdb", CONN)
```

It should work in Modin if you create the string as outlined above. I will open a PR to add this functionality, but I will wait for you to confirm if it is working or not to merge. Let me know!

Resolving patch:
diff --git a/modin/engines/ray/generic/io.py b/modin/engines/ray/generic/io.py
--- a/modin/engines/ray/generic/io.py
+++ b/modin/engines/ray/generic/io.py
@@ -455,6 +455,21 @@ def read_sql(cls, sql, con, index_col=None, **kwargs):
 
         import sqlalchemy as sa
 
+        try:
+            import psycopg2 as pg
+
+            if isinstance(con, pg.extensions.connection):
+                con = "postgresql+psycopg2://{}:{}@{}{}/{}".format(  # Table in DB
+                    con.info.user,  # <Username>: for DB
+                    con.info.password,  # Password for DB
+                    con.info.host if con.info.host != "/tmp" else "",  # @<Hostname>
+                    (":" + str(con.info.port))
+                    if con.info.host != "/tmp"
+                    else "",  # <port>
+                    con.info.dbname,  # Table in DB
+                )
+        except ImportError:
+            pass
         # In the case that we are given a SQLAlchemy Connection or Engine, the objects
         # are not pickleable. We have to convert it to the URL string and connect from
         # each of the workers.


Test patch: