Repository: InstaPy/InstaPy

Issue:
Cannot focus element error
I'm running the script on mac and after some time i get this error:

Message: unknown error: cannot focus element
  (Session info: chrome=61.0.3163.100)
  (Driver info: chromedriver=2.32.498537 (cb2f855cbc7b82e20387eaf9a43f6b99b6105061),platform=Mac OS X 10.12.3 x86_64)

Now I'm trying to update chromedriver with all of the packages to check whether they are the reason, but does anybody else get this error?


Discussion:
Im getting this same issue. Im failing on comments with the following stack trace and error:

```
File "/Users/alexmattson/Desktop/InstaPy/instapy/instapy.py", line 614, in like_by_tags
    self.browser, comments)
  File "/Users/alexmattson/Desktop/InstaPy/instapy/comment_util.py", line 25, in comment_image
    comment_input[0].send_keys("\b")
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 349, in send_keys
    'value': keys_to_typing(value)})
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 493, in _execute
    return self._parent.execute(command, params)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in execute
    self.error_handler.check_response(response)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot focus element
  (Session info: chrome=61.0.3163.100)
  (Driver info: chromedriver=2.31.488774 (7e15618d1bf16df8bf0ecf2914ed1964a387ba0b),platform=Mac OS X 10.12.2 x86_64)
```

@diveu were you able to find a solution?
I'm getting the same error on Windows 10 with chromedriver 2.33.506120
upgraded to 2.33.506106 and still getting error on mac 
I still get the failure when I run on the firefox webdriver but a different error:

```
  File "/Users/alexmattson/Desktop/InstaPy/instapy/instapy.py", line 614, in like_by_tags
    self.browser, comments)
  File "/Users/alexmattson/Desktop/InstaPy/instapy/comment_util.py", line 26, in comment_image
    comment_input[0].submit()
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 82, in submit
    form = self.find_element(By.XPATH, "./ancestor-or-self::form")
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 510, in find_element
    {"using": by, "value": value})['value']
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webelement.py", line 493, in _execute
    return self._parent.execute(command, params)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in execute
    self.error_handler.check_response(response)
  File "/Library/Python/2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <textarea class="_bilrf"> stale: either the element is no longer attached to the DOM or the page has been refreshed
```
@alexmattson I haven't found anything yet. But I'll look into it later today, and if I find smth I'll write it here.
I managed to fix this issue on Chrome by changing `instapy/comment_util.py`

I moved the find element code from `comment_image` into its own function
```
def get_comment_input(browser):
    comment_input = browser.find_elements_by_xpath(
        '//textarea[@placeholder = "Add a comment…"]')
    if len(comment_input) <= 0:
        comment_input = browser.find_elements_by_xpath(
            '//input[@placeholder = "Add a comment…"]')
    return comment_input
```

And changed the beginning of the `comment_image` function to the following
```
def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    comment_input = get_comment_input(browser)
    comment_input[0].clear()
    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
```

It seems that clearing the comment textarea (`comment_input[0].clear()`) and finding the element again fixes the issue.
@ateich thx, it worked!
@ateich You forgot small part which produces error sometimes

Instead
```
def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    comment_input = get_comment_input(browser)
    comment_input[0].clear()
    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
```
use
```
def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    comment_input = get_comment_input(browser)
    comment_input[0].clear()
    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')

    print("--> Commented: {}".format(rand_comment.encode('utf-8')))
    sleep(2)

    return 1
```
@HammerSpb Sorry if I wasn't clear. I didn't intend for my `comment_image` function to be a full replacement for the existing one. Instead, I was suggesting changing the first part of the function to the code I posted.

Thanks for posting the entire function. It should go a long way to alleviate any confusion for anyone else with the issue.
@HammerSpb Also, I noticed your `comment_image` function is missing the following lines from the original. Are you able to post comments without them?

```
# An extra space is added here and then deleted.
# This forces the input box to update the reactJS core
comment_input[0].send_keys("\b")
comment_input[0].submit()
```

Here is what my entire `comment_util.py` looks like 

```
# -*- coding: utf-8 -*-
"""Module which handles the commenting features"""
from random import choice
from .time_util import sleep
import emoji

def get_comment_input(browser):
    comment_input = browser.find_elements_by_xpath(
        '//textarea[@placeholder = "Add a comment…"]')
    if len(comment_input) <= 0:
        comment_input = browser.find_elements_by_xpath(
            '//input[@placeholder = "Add a comment…"]')
    return comment_input

def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    comment_input = get_comment_input(browser)
    comment_input[0].clear()
    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
        # An extra space is added here and then deleted.
        # This forces the input box to update the reactJS core
        comment_input[0].send_keys("\b")
        comment_input[0].submit()
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')

    print("--> Commented: {}".format(rand_comment.encode('utf-8')))
    sleep(2)

    return 1

```
@ateich You are absolutely right
But sometimes it doesn't work
I found a better solution and tested it few times. 
```
comment_input[0].submit()
comment_input[0].send_keys("\b")
comment_input[0].submit()
```

Sometimes `send_keys("\b")` overwrite comment! But if you add `submit()` before calling `send_keys("\b")` everything works fine. Atleast for now

```
def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    comment_input = get_comment_input(browser)
    if len(comment_input) > 0:
        comment_input[0].clear()
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')
        return False

    comment_input = get_comment_input(browser)
    if len(comment_input) > 0:
        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
        comment_input[0].submit()
        comment_input[0].send_keys("\b")
        comment_input[0].submit()
        
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')
        return False

    print("--> Commented: {}".format(rand_comment.encode('utf-8')))
    sleep(2)

    return True
```
@HammerSpb `send_keys("\b")` is the equivalent of pressing the BACKSPACE key. It is there to delete an extra space at the end of the following line of code and force reactJS to update
```
browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
```

Note that there is a space between the `"` and the `'` before the semicolon
`+ " ';"`

The `send_keys("\b")` forces the input box to update the reactJS core of the Instagram website. The extra space at the end is there to make sure that the blank space is the deleted character. Maybe the reactJS core update is no longer necessary since we cleared the `textarea` earlier?

I haven't had a comment overwritten yet, but I'm glad to hear that you found a solution that is working for you 😃 
@ateich Thanks for the help debugging! But I keep getting a new error with your revised code.

Now an IndexError: list index out of range

Also any idea on the /instapy.py, line 614, in like_by_tags? I was looking at issue #700 but seems slightly different.

![image](https://user-images.githubusercontent.com/15362634/32037477-483fb54a-b9da-11e7-9e0c-12dea9729053.png)

comment_util.py
![image](https://user-images.githubusercontent.com/15362634/32037530-8ceec686-b9da-11e7-9e9d-3f322f8c678b.png)



@HammerSpb @Xander8 HammerSpb, i copied your comment code and it seems to be working fine. I did run into 1 error ( the original one) but i think thats because i moved the window as it was trying to go to another post.

@Xander8 copy, Hammerspb's code exactly and your code should work - its a good fix for now. 
@Xander8 It looks like `comment_input` contains no items, which means that the `Add a comment...` textarea/input is not visible on screen. This is most likely the result of the browser window not being wide enough. (Instagram hides the `Add a comment...` element on smaller width screens)

Changing `comment_image` to the following will stop the `list index out of range` exception from being thrown, but you will then get `Warning: Comment Action Likely Failed: Comment Element not found`

```
def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        comment_input[0].clear()
        comment_input = get_comment_input(browser)

        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
        # An extra space is added here and then deleted.
        # This forces the input box to update the reactJS core
        comment_input[0].send_keys("\b")
        comment_input[0].submit()
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')

    print("--> Commented: {}".format(rand_comment.encode('utf-8')))
    sleep(2)

    return 1
```

To actually get the element back on the screen, you need to open chrome in a wider window. You can make chrome enter fullscreen on launch by adding `chrome_options.add_argument("--start-fullscreen");` to `instapy/instapy.py` after line 129

`instapy/instapy.py` lines 128 to 133 would now look like
```
chromedriver_location = './assets/chromedriver'
chrome_options = Options()
chrome_options.add_argument("--start-fullscreen");
chrome_options.add_argument('--dns-prefetch-disable')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--lang=en-US')
```

I'm not sure if this will work in VirtualBox, but I think it is your best bet. Let me know how it goes
@Xander8 After some more thought, here is a better solution that doesn't require Chrome to be fullscreen. (No need to change `instapy/instapy.py`)

I added the function `open_comment_section`, which will press the on screen comment (message bubble) button. This causes the `comment_input` textarea/input to appear regardless of window size.

So all we need to do now is call `open_comment_section` before `get_comment_input`, as is done in the code below

`instapy/comment_utl.py`

```
# -*- coding: utf-8 -*-
"""Module which handles the commenting features"""
from random import choice
from .time_util import sleep
import emoji

def get_comment_input(browser):
    comment_input = browser.find_elements_by_xpath(
        '//textarea[@placeholder = "Add a comment…"]')
    if len(comment_input) <= 0:
        comment_input = browser.find_elements_by_xpath(
            '//input[@placeholder = "Add a comment…"]')
    return comment_input

def open_comment_section(browser):
    comment_elem = browser.find_elements_by_xpath(
        "//a[@role='button']/span[text()='Comment']/..")
    if len(comment_elem) > 0:
        comment_elem[0].click()
    else:
        print('--> Warning: Comment Button Not Found:'
              ' May cause issues with browser windows of smaller widths')

def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    open_comment_section(browser)
    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        comment_input[0].clear()
        comment_input = get_comment_input(browser)

        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
        # An extra space is added here and then deleted.
        # This forces the input box to update the reactJS core
        comment_input[0].send_keys("\b")
        comment_input[0].submit()
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')

    print("--> Commented: {}".format(rand_comment.encode('utf-8')))
    sleep(2)

    return 1
```



Changed the comment_util.py to the last @ateich suggestion, but still getting the same freaking error..any idea?
@AlessioBucaioni Could you post the error log you are receiving?
@ateich

Have been running your latest revision on my local MacBook Pro and so far the fix seems fairly stable. So far it's made it through 2 tags without failing so I'll report back later on today if it's completely acceptable. 

This might even be the fix for my follow/unfollow issue. I'll report back on that one as well. 
@RomeoJuliett I'm not sure if this is related to your unfollow issue, but I ran into a problem unfollowing users when the browser window is smaller. Specifically, at smaller window widths the "following" link tag moves outside of the header and breaks `unfollow_util.py` line 125

Replacing lines 125 to 127 fixed this issue for me

before
```
following_link = browser.find_elements_by_xpath(
                '//header/div[2]//li[3]')
following_link[0].click()
```

after
```
following_link = browser.find_elements_by_xpath(
                '//a[@href="/' + username + '/followers/"]')
following_link[0].send_keys("\n")
```
@ateich I've tried your code that you posted above, the one where you added `open_comment_section` but I sometimes get this error:

```
Traceback (most recent call last):
File "/Users/katieywu/Katie/Code/InstaPy/examples/katwu.py", line 91, in <module>
    amount=10)
File "/Users/katieywu/Katie/Code/InstaPy/instapy/instapy.py", line 531, in like_by_tags
    commented += comment_image(self.browser, comments)
File "/Users/katieywu/Katie/Code/InstaPy/instapy/comment_util.py", line 50, in comment_image
    open_comment_section(browser)
File "/Users/katieywu/Katie/Code/InstaPy/instapy/comment_util.py", line 26, in open_comment_section
    comment_elem[0].click()
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 72, in click
    self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py", line 461, in _execute
    return self._parent.execute(command, params)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Element is not clickable at point (732, 593)
  (Session info: chrome=61.0.3163.100)
  (Driver info: chromedriver=2.31.488774 (7e15618d1bf16df8bf0ecf2914ed1964a387ba0b),platform=Mac OS X 10.11.5 x86_64)
```
@ateich @HammerSpb Thank god for the duo debug team! Appreciate it guys! 
@ateich 

It unfollows just fine actually, it's when it tries to follow someone that it breaks. I reported back in the other thread as well.
@katieywu Based on the error log, it looks like the comment button is sometimes either not visible or off screen when we call `comment_elem[0].click()` in `open_comment_section`

Try replacing the `open_comment_section` function with the following
```
def open_comment_section(browser):
    missing_comment_elem_warning = (
        '--> Warning: Comment Button Not Found:'
        ' May cause issues with browser windows of smaller widths')
    comment_elem = browser.find_elements_by_xpath(
        "//a[@role='button']/span[text()='Comment']/..")
    if len(comment_elem) > 0:
        try:
            browser.execute_script(
                "arguments[0].click();", comment_elem[0])
        except WebDriverException:
            print(missing_comment_elem_warning)
    else:
        print(missing_comment_elem_warning)
```

Instead of `comment_elem[0].click()` the above uses
```
try:
     browser.execute_script(
          "arguments[0].click();", comment_elem[0])
except WebDriverException:
     print(missing_comment_elem_warning)
```
which should work even if the comment_button is not visible and, if all else fails, will catch the WebDriverException you are getting so the script won't crash


@ateich  thanks that's fixed it for me
@ateich Would you be interested in proposing a fix via PR? :smile: 
@ateich got another one for ya. ~~~On line \~68, the line is `comment_input[0].submit()` but I get this exception when it attempts to comment: `selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document` ...very odd. A cursory google search says the reference to the element (`comment_input`) is not valid if the DOM refreshes, and it seems to be happening between the the `comment_input[0].send_keys("\b")` line and the submit line. If that line is removed it obviously doesn't actually comment, but it also doesn't throw the StaleElementReferenceException anymore...~~

Update: I ended up fixing it with a one liner. If you get the StaleElementException it seems the easy thing to do is just to re-fetch the reference. Here is the updated code:
```
    if len(comment_input) > 0:
        comment_input[0].clear()
        comment_input = get_comment_input(browser)

        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
        # An extra space is added here and then deleted.
        # This forces the input box to update the reactJS core
        comment_input[0].send_keys("\b")
        comment_input = get_comment_input(browser) #added this line to avoid StaleElementException
        comment_input[0].submit()
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')
```

still getting the error @ateich 

 File "quickstart.py", line 19, in <module>
    session.like_by_tags(['jokes', 'funny', 'lmao', 'jokes', 'lol' ,'laugh' ,'crazy','joking','silly','laughing','epic','lmfao','haha','humor','wacky','hilarious'], amount=50)
  File "/root/InstaPy/instapy/instapy.py", line 571, in like_by_tags
    self.like_by_followers_lower_limit)
  File "/root/InstaPy/instapy/like_util.py", line 413, in check_link
    "return window._sharedData.entry_data."
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 465, in execute_script
    'args': converted_args})['value']
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Cannot read property 'entry_data' of undefined
  (Session info: chrome=62.0.3202.62)
  (Driver info: chromedriver=2.32.498513 (2c63aa53b2c658de596ed550eb5267ec5967b351),platform=Linux 2.6.32-042stab125.3 x86_64)

Changed the file with this and the comments are working now in my case:

```python
# -*- coding: utf-8 -*-
"""Module which handles the commenting features"""
from random import choice
from .time_util import sleep
import emoji

def get_comment_input(browser):
    comment_input = browser.find_elements_by_xpath(
        '//textarea[@placeholder = "Add a comment…"]')
    if len(comment_input) <= 0:
        comment_input = browser.find_elements_by_xpath(
            '//input[@placeholder = "Add a comment…"]')
    return comment_input

def open_comment_section(browser):
    missing_comment_elem_warning = (
        '--> Warning: Comment Button Not Found:'
        ' May cause issues with browser windows of smaller widths')
    comment_elem = browser.find_elements_by_xpath(
        "//a[@role='button']/span[text()='Comment']/..")
    if len(comment_elem) > 0:
        try:
            browser.execute_script(
                "arguments[0].click();", comment_elem[0])
        except WebDriverException:
            print(missing_comment_elem_warning)
    else:
        print(missing_comment_elem_warning)

def comment_image(browser, comments):
    """Checks if it should comment on the image"""
    rand_comment = (choice(comments))
    rand_comment = emoji.demojize(rand_comment)
    rand_comment = emoji.emojize(rand_comment, use_aliases=True)

    open_comment_section(browser)
    comment_input = get_comment_input(browser)

    if len(comment_input) > 0:
        comment_input[0].clear()
        comment_input = get_comment_input(browser)

        browser.execute_script(
            "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
        # An extra space is added here and then deleted.
        # This forces the input box to update the reactJS core
        comment_input[0].send_keys("\b")
        comment_input[0].submit()
    else:
        print('--> Warning: Comment Action Likely Failed:'
              ' Comment Element not found')

    print("--> Commented: {}".format(rand_comment.encode('utf-8')))
    sleep(2)

    return 1

```

Resolving patch:
diff --git a/instapy/comment_util.py b/instapy/comment_util.py
--- a/instapy/comment_util.py
+++ b/instapy/comment_util.py
@@ -2,8 +2,31 @@
 """Module which handles the commenting features"""
 from random import choice
 from .time_util import sleep
+from selenium.common.exceptions import WebDriverException
 import emoji
 
+def get_comment_input(browser):
+    comment_input = browser.find_elements_by_xpath(
+        '//textarea[@placeholder = "Add a comment…"]')
+    if len(comment_input) <= 0:
+        comment_input = browser.find_elements_by_xpath(
+            '//input[@placeholder = "Add a comment…"]')
+    return comment_input
+
+def open_comment_section(browser):
+    missing_comment_elem_warning = (
+        '--> Warning: Comment Button Not Found:'
+        ' May cause issues with browser windows of smaller widths')
+    comment_elem = browser.find_elements_by_xpath(
+        "//a[@role='button']/span[text()='Comment']/..")
+    if len(comment_elem) > 0:
+        try:
+            browser.execute_script(
+                "arguments[0].click();", comment_elem[0])
+        except WebDriverException:
+            print(missing_comment_elem_warning)
+    else:
+        print(missing_comment_elem_warning)
 
 def comment_image(browser, comments):
     """Checks if it should comment on the image"""
@@ -11,18 +34,19 @@ def comment_image(browser, comments):
     rand_comment = emoji.demojize(rand_comment)
     rand_comment = emoji.emojize(rand_comment, use_aliases=True)
 
-    comment_input = browser.find_elements_by_xpath(
-        '//textarea[@placeholder = "Add a comment…"]')
-    if len(comment_input) <= 0:
-        comment_input = browser.find_elements_by_xpath(
-            '//input[@placeholder = "Add a comment…"]')
+    open_comment_section(browser)
+    comment_input = get_comment_input(browser)
 
     if len(comment_input) > 0:
+        comment_input[0].clear()
+        comment_input = get_comment_input(browser)
+
         browser.execute_script(
             "arguments[0].value = '" + rand_comment + " ';", comment_input[0])
         # An extra space is added here and then deleted.
         # This forces the input box to update the reactJS core
         comment_input[0].send_keys("\b")
+        comment_input = get_comment_input(browser)
         comment_input[0].submit()
     else:
         print('--> Warning: Comment Action Likely Failed:'


Test patch: