Repository: InstaPy/InstaPy

Issue:
load_followers_data(...) crashes on month change.
I'm playing with InstaPy trying to understand IG user behavior. But for now, i was only tracking my followers and unfollowers on a daily basis (sometimes two or tree times, if i saw some bigger relation change.

I started around mid January, and on February, 1st the docker_quickstart.py that i was using started to crash on the "session.pick_unfollowers(...)" call. 

My call is:
`all_unfollowers, active_unfollowers = bot.pick_unfollowers(username="ig_id", compare_by="earliest", compare_track="first", live_match=True, store_locally=True, print_out=True)`


I've checked if no corrupted json or something strange was present in the "~/InstaPy/logs/<my_id>/relationship_data/<ig_id>/followers" , and found nothing wrong with the data present there. 

**If you can create and assign the issue to me, i'll provide a suggested solution as soon i've it tested.**

## Expected Behavior

Collect and store follower/unfollower information.

## Current Behavior

Crashing to collect unfollower users after month changed from January to February.
Crash occurs at: https://github.com/timgrossmann/InstaPy/blob/0c8a86057aa2959af47a38bca4f1c25edf92a1cd/instapy/relationship_tools.py#L1088

## Possible Solution (optional)

I think i already found the problem ( as it crashes trying to "check" for a dictionary entry of the new month before setting it),  and i'm currently testing a solution. I can provide a PR with it as soon i've done some more testing.

## InstaPy configuration

Currently running the docker version, but problem is reproducible, with a very simple setup.


###
To reproduce the problem:
---
```
cd *directory where InstaPy is installed*
mkdir -p "logs/my_id/relationship_data/my_id/followers/"

touch '01-02-2019~full~11.json'
touch '30-01-2019~full~11.json'
touch '31-01-2019~full~11.json'

```

Then create a docker_quickstart.py with:

---
```
from instapy import InstaPy

bot = InstaPy(username='my_id', password='pass',
              selenium_local_session=False)

#get the unfollowers since beggining
all_unfollowers, active_unfollowers = bot.pick_unfollowers(username="my_id", compare_by="earliest", compare_track="first", live_match=True, store_locally=True, print_out=True)
bot.end()
```
---
*Crash traceback
```
web_1       | ___________________________________________________________________________
web_1       | INFO [2019-02-02 12:48:08] [my_id]  Starting to pick Unfollowers of my_id..
web_1       | """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
web_1       | Traceback (most recent call last):
web_1       |   File "code/docker_quickstart.py", line 7, in <module>
web_1       |     all_unfollowers, active_unfollowers = bot.pick_unfollowers(username="my_id", compare_by="earliest", compare_track="first", live_match=True, store_locally=True, print_out=True)
web_1       |   File "/code/instapy/instapy.py", line 3906, in pick_unfollowers
web_1       |     self.logfolder)
web_1       |   File "/code/instapy/relationship_tools.py", line 566, in get_unfollowers
web_1       |     logfolder)
web_1       |   File "/code/instapy/relationship_tools.py", line 1080, in load_followers_data
web_1       |     structured_entries["years"][entry_year]["months"][entry_month][
web_1       | KeyError: '02'
instapy_web_1 exited with code 1
```




Discussion:
Found out that the error also occurs on year change, so i've to refactor the code some more, as my first solution doesn't work correctly for all (known) situation.

Should have a working solution latter today, i expect :)

Resolving patch:
diff --git a/instapy/relationship_tools.py b/instapy/relationship_tools.py
--- a/instapy/relationship_tools.py
+++ b/instapy/relationship_tools.py
@@ -1060,39 +1060,12 @@ def load_followers_data(username, compare_by, compare_track, logger,
     structured_entries = {}
 
     for entry in sorted_filenames:
-        entry_year = entry[-4:]
-        entry_month = entry[-7:5]
-        entry_day = entry[:2]
-
-        if not structured_entries:
-            structured_entries = {
-                "years": {entry_year: {"months": {entry_month: {
-                    "days": {entry_day: {"entries": [entry]}}}}}}}
-            continue
-
-        existing_years = list(
-            year for year, month in structured_entries["years"].items())
-        existing_months = list(month for month, day in
-                               structured_entries["years"][entry_year][
-                                   "months"].items())
-        existing_days = list(
-            day for day, entry in
-            structured_entries["years"][entry_year]["months"][entry_month][
-                "days"].items())
-
-        if entry_year not in existing_years:
-            structured_entries["years"][entry_year] = {
-                "months": {
-                    entry_month: {"days": {entry_day: {"entries": [entry]}}}}}
-        elif entry_month not in existing_months:
-            structured_entries["years"][entry_year]["months"][entry_month] = {
-                "days": {entry_day: {"entries": [entry]}}}
-        elif entry_day not in existing_days:
-            structured_entries["years"][entry_year]["months"][entry_month][
-                "days"][entry_day] = {"entries": [entry]}
-        else:
-            structured_entries["years"][entry_year]["months"][entry_month][
-                "days"][entry_day]["entries"].append(entry)
+        entry_day, entry_month, entry_year = entry.split('-')
+
+        structured_entries.setdefault("years", {}).setdefault(entry_year, {}).\
+            setdefault("months", {}).setdefault(entry_month, {}).\
+            setdefault("days", {}).setdefault(entry_day, {}).\
+            setdefault("entries", []).append(entry)
 
     if compare_by == "latest":
         selected_filename = sorted_filenames[-1]


Test patch: