Repository: Flexget/Flexget

Issue:
AttributeError: 'NoneType' object has no attribute 'find'
### Expected behaviour:

Properly rename movie files from imdb_lookup

### Actual behaviour:

### Steps to reproduce:
`flexget --test inject "The.Iron.Giant.1999.Signature.Edition.1080p.BluRay.x264.AC3.mp4" execute --tasks movies_rename`

#### Config:
```yaml
--- config from task: movies_rename
accept_all: true
deluge:
  content_filename: '{% if imdb_name is defined %}{{imdb_name|replace('':'','''')}}
    ({{imdb_year}}) {% else %}{{title}} {% endif %}{{quality.resolution}}'
  host: XXXX
  label: movies_renamed
  main_file_only: true
  move_completed_path: ~/torrents/completed/_Movies/_renamed
  password: XXXX
  port: 2134
  username: XXXX
from_deluge:
  filter:
    label: movies
    state: Seeding
  host: XXXX
  password: XXXX
  port: 2134
  username: XXXX
if:
  - imdb_url == None: reject
imdb_lookup: true
no_entries_ok: true
require_field: imdb_id
template: no_global

---
```
  
#### Log:
<details>
 <summary>(click to expand)</summary>

```
2021-05-26 07:31:18 DEBUG    imdb.utils    movies_rename   SUCCESS: only one movie remains
2021-05-26 07:31:18 VERBOSE  imdb_lookup   movies_rename   Found https://www.imdb.com/title/tt0129167/
2021-05-26 07:31:18 VERBOSE  imdb_lookup   movies_rename   Parsing imdb for `The.Iron.Giant.1999.Signature.Edition.1080p.BluRay.x264.AC3-PxHD.mp4`
2021-05-26 07:31:18 DEBUG    utils.requests movies_rename   Waiting 2.26 seconds until next request to imdb.com
2021-05-26 07:31:20 DEBUG    utils.requests movies_rename   GETing URL https://www.imdb.com/title/tt0129167/ with args () and kwargs {'allow_redirects': True, 'timeout': 30}
2021-05-26 07:31:22 DEBUG    imdb.utils    movies_rename   No year found for tt0129167
2021-05-26 07:31:22 ERROR    lazy_lookup   movies_rename   Unhandled error in lazy lookup plugin: 'NoneType' object has no attribute 'find'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/flexget/utils/lazy_dict.py", line 41, in __getitem__
    callee.func(self.store, *(callee.args or []), **(callee.kwargs or {}))
  File "/usr/local/lib/python3.6/dist-packages/flexget/components/imdb/imdb_lookup.py", line 69, in lazy_loader
    self.lookup(entry)
  File "/usr/local/lib/python3.6/dist-packages/flexget/plugin.py", line 126, in wrapped_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/flexget/utils/database.py", line 30, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/flexget/components/imdb/imdb_lookup.py", line 222, in lookup
    movie = self._parse_new_movie(entry['imdb_url'], session)
  File "/usr/local/lib/python3.6/dist-packages/flexget/components/imdb/imdb_lookup.py", line 267, in _parse_new_movie
    parser.parse(imdb_url)
  File "/usr/local/lib/python3.6/dist-packages/flexget/components/imdb/utils.py", line 327, in parse
    original_name_elem = title_wrapper.find('div', {'class': 'originalTitle'})
AttributeError: 'NoneType' object has no attribute 'find'
```
</details>

### Additional information:

- FlexGet version: 3.1.128
- Python version: 3.6
- Installation method: unsure (managed seedbox)
- Using daemon (yes/no): yes
- OS and version:  Ubuntu 18.04.4 LTS

I think IMDb updated their HTML again as that page doesn't have the originalTitle tag or the title_wrapper class that:
https://github.com/Flexget/Flexget/blob/97ca6d2219c43f723be67bfb7ef2021dbe3c1f72/flexget/components/imdb/utils.py#L327
is looking for. Let me know if there's any more info you need.


Discussion:

Resolving patch:
diff --git a/flexget/components/imdb/utils.py b/flexget/components/imdb/utils.py
--- a/flexget/components/imdb/utils.py
+++ b/flexget/components/imdb/utils.py
@@ -285,6 +285,16 @@ def parse(self, imdb_id, soup=None):
             soup = get_soup(page.text)
 
         title_wrapper = soup.find('div', attrs={'class': 'title_wrapper'})
+        if not title_wrapper:
+            # New layout, transitional
+            title_wrapper = soup.find(
+                'div', {"class": re.compile("^TitleBlock__TitleContainer.?")}
+            )
+
+        if not title_wrapper:
+            raise plugin.PluginError(
+                'IMDB parser needs updating, imdb format changed. Please report on Github.'
+            )
 
         data = json.loads(soup.find('script', {'type': 'application/ld+json'}).text)
 
@@ -304,6 +314,12 @@ def parse(self, imdb_id, soup=None):
             )
 
         year = soup.find('span', attrs={'id': 'titleYear'})
+        if not year:
+            # Test new layout
+            year = title_wrapper.find(
+                'span', {"class": re.compile("^TitleBlockMetaData__ListItemText.?")}
+            )
+
         if year:
             m = re.search(r'([0-9]{4})', year.text)
             if m:
@@ -324,11 +340,24 @@ def parse(self, imdb_id, soup=None):
         else:
             logger.debug('No photo found for {}', self.imdb_id)
 
+        strip_pre_text = False
         original_name_elem = title_wrapper.find('div', {'class': 'originalTitle'})
+        if not original_name_elem:
+            # Test new layout
+            strip_pre_text = True
+            original_name_elem = title_wrapper.find(
+                'div', {"class": re.compile("^OriginalTitle.?")}
+            )
+
         if original_name_elem:
             self.name = title_wrapper.find('h1').contents[0].strip()
             self.original_name = original_name_elem.contents[0].strip().strip('"')
-        else:
+            if strip_pre_text:
+                striped_text = re.search(r"([^\:]*)\:? (.*)", self.original_name)
+                if len(striped_text.groups()) == 2:
+                    self.original_name = striped_text.group(2)
+
+        if not original_name_elem:
             logger.debug('No original title found for {}', self.imdb_id)
 
         votes_elem = data.get('aggregateRating', {}).get('ratingCount')
@@ -344,6 +373,10 @@ def parse(self, imdb_id, soup=None):
             logger.debug('No score found for {}', self.imdb_id)
 
         meta_score_elem = soup.find(attrs={'class': 'metacriticScore'})
+        if not meta_score_elem:
+            # Test new layout
+            meta_score_elem = soup.find('span', attrs={'class': 'score-meta'})
+
         if meta_score_elem:
             self.meta_score = str_to_int(meta_score_elem.text)
         else:
@@ -375,6 +408,10 @@ def parse(self, imdb_id, soup=None):
 
         # Details section
         title_details = soup.find('div', attrs={'id': 'titleDetails'})
+        if not title_details:
+            # Test new layout
+            title_details = soup.find('div', attrs={'data-testid': 'title-details-section'})
+
         if title_details:
             # get languages
             for link in title_details.find_all(
@@ -402,6 +439,17 @@ def parse(self, imdb_id, soup=None):
                 self.plot_keywords = [
                     keyword_elem.text.strip() for keyword_elem in keyword_elem.find_all("a")[:-1]
                 ]
+        else:
+            # Test new layout
+            storyline = soup.find('div', attrs={'data-testid': 'storyline-plot-summary'})
+            if storyline:
+                self.plot_outline = storyline.text
+
+            keyword_elem = soup.find('div', attrs={'data-testid': 'storyline-plot-keywords'})
+            if keyword_elem:
+                self.plot_keywords = [
+                    keyword_elem.text.strip() for keyword_elem in keyword_elem.find_all("a")[:-1]
+                ]
 
         genres = data.get('genre', [])
         if not isinstance(genres, list):
@@ -419,3 +467,14 @@ def parse(self, imdb_id, soup=None):
                 if isinstance(actor_name, Tag):
                     actor_name = None
                 self.actors[actor_id] = actor_name
+        else:
+            # Test new layout
+            cast = soup.find_all('a', attrs={'data-testid': 'title-cast-item__actor'})
+            if cast:
+                for actor in cast:
+                    actor_id = extract_id(actor['href'])
+                    actor_name = actor.text.strip()
+                    # tag instead of name
+                    if isinstance(actor_name, Tag):
+                        actor_name = None
+                    self.actors[actor_id] = actor_name


Test patch: