Repository: j-andrews7/kenpompy (language: python)

Issue:
Update docs to go back to 1999 season
Looks like more data is available than previously, most stuff goes back to 1999 now rather than 2002/2001. Docstrings need to be updated and potentially a few tests tweaked.
Cloudflare Challenge Page 
With the season around the corner, I was reviewing the changes I made over the summer for #92 wanting to finally tidy it up, get it pushed, and open the PR. However, I seem to be hitting the Cloudflare challenge page during login. I'm not sure if its just me or if kenpom changed something over the offseason. The challenge I'm hitting seems to be the [javascript challenge](https://developers.cloudflare.com/waf/reference/cloudflare-challenges/). 

Since mechanicalsoup can't execute javascript, I'm not sure there will be a workaround except for moving to something like selenium or trying [cloudscraper](https://github.com/VeNoMouS/cloudscraper/tree/master) maybe.
Update for predicted possessions and new date matching 
Passes all tests locally. Thirds times the charm (hopefully).


Discussion / hints:

Ugh, yeah, I was kind of worried about continually running into this given
the issues that started really cropping up last season. Selenium is pretty
heavy in terms of solution.

On Thu, Oct 24, 2024, 12:47 PM Sean Kim ***@***.***> wrote:

> With the season around the corner, I was reviewing the changes I made over
> the summer for #92 <https://github.com/j-andrews7/kenpompy/issues/92>
> wanting to finally tidy it up, get it pushed, and open the PR. However, I
> seem to be hitting the Cloudflare challenge page during login. I'm not sure
> if its just me or if kenpom turned on managed challenges over the
> offseason. The challenge I'm hitting is the javascript challenge
> <https://developers.cloudflare.com/waf/reference/cloudflare-challenges/>.
>
> Since mechanicalsoup can't execute javascript, I'm not sure there will be
> a workaround except for moving to something like selenium or trying
> cloudscraper <https://github.com/VeNoMouS/cloudscraper/tree/master> maybe.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/j-andrews7/kenpompy/issues/93>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACOAQNDPXYHMCQ5MRXP6JYDZ5EXCXAVCNFSM6AAAAABQRTVDA2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGYYTEMRSGM2TMNA>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>

Yeah selenium is definitely not an elegant or lightweight solution and potential overkill. I don't really know much about cloudscraper (just came across it doing some googling), I can try some preliminary experimenting/digging and see if it could be a potential solution. From what I can tell it seems like it was built to be a drop in replacement for the `requests` library and supports both a pure python backend to replicate the javascript execution for the challenge and external js engines like node. 
I may also try to reach out to Ken directly about this. He has answered in
the past, so who knows, maybe he'll be able and willing to help us out.

On Thu, Oct 24, 2024, 1:01 PM Sean Kim ***@***.***> wrote:

> Yeah selenium is definitely not an elegant or lightweight solution and
> potential overkill. I don't really know much about cloudscraper (just came
> across it doing some googling), I can try some preliminary
> experimenting/digging and see if it could be a potential solution. From
> what I can tell it seems like it was built to be a drop in replacement for
> the requests library and supports both a pure python backend to replicate
> the javascript execution for the challenge and external js engines like
> node.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/j-andrews7/kenpompy/issues/93#issuecomment-2436018034>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACOAQNCOFWVZODG736RNH63Z5EYW3AVCNFSM6AAAAABQRTVDA2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZWGAYTQMBTGQ>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

Oh that'd be awesome. In the meantime, I did mock out a rewrite of the login function using cloudscraper and it worked, was able to get through and hit the fanmatch page. 
Ok after a little trial and error, its a relatively simple drop in fix. The `utils` file with cloudscraper looks like this: 

```py
import cloudscraper

def login(email, password):

    # Create a Cloudscraper session
    scraper = cloudscraper.create_scraper()

    scraper.get('https://kenpom.com/index.php')

    form_data = {
        'email': email,
        'password': password,
        'submit': 'Login!',
    }

    scraper.post(
        'https://kenpom.com/handlers/login_handler.php',
        data=form_data,
        allow_redirects=True,
    )

    # Check if login was successful
    home_page = scraper.get('https://kenpom.com/')
    if 'Logout' not in home_page.text:
        raise Exception('Logging in failed - check your credentials.')

    return scraper
```

I still get the same response page error but further navigation works fine and shows me as logged in. 

For the `FanMatch` class I just had to change this: 

https://github.com/j-andrews7/kenpompy/blob/e28f4b0d5376415781de70efcf0f7918ec2c0428/kenpompy/FanMatch.py#L51-L52

To this: 

```py
response = browser.get(self.url)
fm = BeautifulSoup(response.content, "html.parser")
```

And the rest of the functionality can remain as is. I haven't dug too much into the other classes in the library since I mainly just use the fanmatch functionality but I assume it'd be a similarly simple refactor to take a `CloudScraper` object instead of a `StatefulBrowser` and then dump the return html into a `BeautifulSoup` instance. 
Any chance you wanna pop in a PR even if it's not quite all the way there?

On Thu, Oct 24, 2024, 4:23 PM Sean Kim ***@***.***> wrote:

> Ok after a little trial and error, its a relatively simple drop in fix.
> The utils file with cloudscraper looks like this:
>
> import cloudscraper
> def login(email, password):
>
>     # Create a Cloudscraper session
>     scraper = cloudscraper.create_scraper()
>
>     scraper.get('https://kenpom.com/index.php')
>
>     form_data = {
>         'email': email,
>         'password': password,
>         'submit': 'Login!',
>     }
>
>     scraper.post(
>         'https://kenpom.com/handlers/login_handler.php',
>         data=form_data,
>         allow_redirects=True,
>     )
>
>     # Check if login was successful
>     home_page = scraper.get('https://kenpom.com/')
>     if 'Logout' not in home_page.text:
>         raise Exception('Logging in failed - check your credentials.')
>
>     return scraper
>
> I still get the same response page error but further navigation works fine
> and shows me as logged in.
>
> For the FanMatch class I just had to change this:
>
>
> https://github.com/j-andrews7/kenpompy/blob/e28f4b0d5376415781de70efcf0f7918ec2c0428/kenpompy/FanMatch.py#L51-L52
>
> To this:
>
> response = browser.get(self.url)fm = BeautifulSoup(response.content, "html.parser")
>
> And the rest of the functionality can remain as is. I haven't dug too much
> into the other classes in the library since I mainly just use the fanmatch
> functionality but I assume it'd be a similarly simple refactor to take a
> CloudScraper object instead of a StatefulBrowser and then dump the return
> html into a BeautifulSoup instance.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/j-andrews7/kenpompy/issues/93#issuecomment-2436361465>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACOAQND37DVMCPEMXHQNL4TZ5FQNHAVCNFSM6AAAAABQRTVDA2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZWGM3DCNBWGU>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

Yeah I can open one. I'll open one today 


Fix patch:
diff --git a/docs/requirements.txt b/docs/requirements.txt
index 6c5d5d4..d9ae056 100644
--- a/docs/requirements.txt
+++ b/docs/requirements.txt
@@ -1,1 +1,4 @@
 sphinx-rtd-theme
+pandas
+bs4
+cloudscraper
diff --git a/kenpompy/FanMatch.py b/kenpompy/FanMatch.py
index 58a86f9..553edba 100644
--- a/kenpompy/FanMatch.py
+++ b/kenpompy/FanMatch.py
@@ -4,6 +4,12 @@
 
 import pandas as pd
 from io import StringIO
+import re
+from datetime import datetime
+from cloudscraper import CloudScraper
+from bs4 import BeautifulSoup
+from typing import Optional
+from .utils import get_html
 
 class FanMatch:
     """Object to hold FanMatch page scraping results.
@@ -11,7 +17,7 @@ class FanMatch:
     This class scrapes the kenpom FanMatch page when a new instance is created. 
 
     Args:
-        browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+        browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
             by the `login` function.
         date (str): Date to scrape, in format "YYYY-MM-DD", such as "2020-01-29".
 
@@ -28,10 +34,10 @@ class FanMatch:
         record_favs (str): Record of favorites for the day.
         expected_record_favs (str): Expected record of favorites for the day.
         exact_mov (str): Number of games where margin of victory was accurately predicted out of total played.
-        fm_df (pandas dataframe): Pandas dataframe containing parsed FanMatch table.
+        fm_df (pandas dataframe or None): Pandas dataframe containing parsed FanMatch table. If there are no games that day, fm_df will be None.
     """
 
-    def __init__(self, browser, date = None):
+    def __init__(self, browser: CloudScraper, date: Optional[str]=None):
         self.url = 'https://kenpom.com/fanmatch.php'
         self.date = date
         self.lines_o_night = None
@@ -44,12 +50,24 @@ def __init__(self, browser, date = None):
         self.record_favs = None
         self.expected_record_favs = None
         self.exact_mov = None
+        self.fm_df = None
         
         if self.date is not None:
             self.url = self.url + "?d=" + self.date
 
-        browser.open(self.url)
-        fm = browser.get_current_page()
+        fm = BeautifulSoup(get_html(browser, self.url), "html.parser")
+        if "Sorry, no games today." in fm.text:
+            return
+        if date is not None:
+            date_text = fm.find("div", class_="lh12").get_text()
+            date_match = re.search(r"for \w+, (\w+ \d{1,2}[a-z]{2})", date_text)
+            if date_match:
+                extracted_date_str = re.sub(r"(st|nd|rd|th)", "", date_match.group(1))
+                extracted_date = datetime.strptime(extracted_date_str, "%B %d")
+                extracted_mmdd = extracted_date.strftime("%m-%d")
+                user_mmdd = datetime.strptime(date, "%Y-%m-%d").strftime("%m-%d")
+                if extracted_mmdd != user_mmdd:
+                    return
         table = fm.find_all("table")[0]
         fm_df = pd.read_html(StringIO(str(table)))
         fm_df = fm_df[0]
@@ -112,13 +130,11 @@ def __init__(self, browser, date = None):
         pos = fm_df.Game.str.split(r" \[").str[1]
         fm_df["Game"], fm_df["Possessions"] = fm_df.Game.str.split(r" \[").str[0], pos.astype("str")
         fm_df.Possessions = fm_df.Possessions.str.rstrip(r"\] ")
-        predict_info = fm_df.Prediction.str.split()
-        pred_winner = fm_df.Prediction.astype("str").str.split().str[0:-2].tolist()
-        pred_winner = [" ".join(i) if not any(pd.isnull(i)) else float("nan") for i in pred_winner]
-        fm_df["PredictedWinner"] = pred_winner
-        fm_df["PredictedScore"] = fm_df.Prediction.str.split().str[-2]
-        fm_df["WinProbability"] = fm_df.Prediction.str.split().str[-1]
-        fm_df.WinProbability = fm_df.WinProbability.str.strip("()")
+        fm_df["PredictedWinner"] = fm_df["Prediction"].str.extract(r"^(.+?) \d+-\d+")[0]
+        fm_df["PredictedScore"] = fm_df["Prediction"].str.extract(r" (\d+-\d+)")[0]
+        fm_df["WinProbability"] = fm_df["Prediction"].str.extract(r"\((\d+%)\)")[0]
+        fm_df["PredictedPossessions"] = fm_df["Prediction"].str.extract(r"\[(\d+)\]")[0].astype(float)
+        fm_df["Possessions"] = fm_df["Possessions"].where(fm_df["Possessions"] != "", fm_df["PredictedPossessions"])
 
         fm_df["PredictedMOV"] = [(int(x[0]) - int(x[1])) if len(x) > 1 else float("nan") for x 
                                  in fm_df.PredictedScore.astype("str").str.split("-")]
@@ -128,11 +144,12 @@ def __init__(self, browser, date = None):
         # Parse predicted loser.
         teams = fm_df.Game.str.split(", ").tolist()
         teams_np = fm_df.Game.str.split(" at ").tolist()
+        pred_winner = fm_df["PredictedWinner"].tolist()
         
         i = 0
         pred_loser = []
-        for x in teams:
-            if not len(x) == 2:
+        for i, x in enumerate(teams):
+            if len(x) != 2:
                 x = teams_np[i]
                 
                 # Account for neutral games.
diff --git a/kenpompy/conference.py b/kenpompy/conference.py
index f5a41d7..8d0120f 100644
--- a/kenpompy/conference.py
+++ b/kenpompy/conference.py
@@ -5,15 +5,19 @@
 
 import pandas as pd
 from io import StringIO
+from cloudscraper import CloudScraper
+from bs4 import BeautifulSoup
+from typing import Optional
+from .utils import get_html
 
-def get_valid_conferences(browser, season=None):
+def get_valid_conferences(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the conferences (https://kenpom.com/conf.php) into a list.
 
 	Args:
-		browser (mechanicalsoul StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		conference_list (list): List containing all valid conferences for the given season on kenpom.com.
@@ -23,8 +27,7 @@ def get_valid_conferences(browser, season=None):
 	url = url + '?c=B10'
 	if(season):
 		url = url + '&y=' + str(season)
-	browser.open(url)
-	confs = browser.get_current_page()
+	confs = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = confs.find_all('table')[-1]
 	links = table.find_all('a')
 	conf_list = []
@@ -34,15 +37,15 @@ def get_valid_conferences(browser, season=None):
 	return conf_list
 
 
-def get_aggregate_stats(browser, conf=None, season=None):
+def get_aggregate_stats(browser: CloudScraper, conf: Optional[str]=None, season: Optional[str]=None):
 	"""
 	Scrapes a given conference's stats (https://kenpom.com/conf.php or https://kenpom.com/confstats.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoul StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
 		conf (str: optional): conference abbreviation (ie B10, P12). If None, it will grab the table from https://kenpom.com/confstats.php instead of https://kenpom.com/conf.php
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		conference_df (dataframe): Dataframe containing aggregate stats of the conference for the given season on kenpom.com.
@@ -52,8 +55,7 @@ def get_aggregate_stats(browser, conf=None, season=None):
 		url = url + f'?c={conf}'
 		if(season):
 			url = url + '&y=' + str(season)
-		browser.open(url)
-		confs = browser.get_current_page()
+		confs = BeautifulSoup(get_html(browser, url), "html.parser")
 		#get first table
 		table = confs.find_all('table')[-3]
 		conf_df = pd.read_html(StringIO(str(table)))[0]
@@ -72,8 +74,7 @@ def get_aggregate_stats(browser, conf=None, season=None):
 		url = "https://kenpom.com/confstats.php"
 		if(season):
 			url = url + '?y=' + str(season)
-		browser.open(url)
-		confs = browser.get_current_page()
+		confs = BeautifulSoup(get_html(browser, url), "html.parser")
 		#get table
 		table = confs.find_all('table')[0]
 		conf_df = pd.read_html(StringIO(str(table)))[0]
@@ -82,15 +83,15 @@ def get_aggregate_stats(browser, conf=None, season=None):
 		conf_df.columns = [stat[:-1] + 'Rank' if '.1' in stat else stat for stat in conf_df.columns]
 		return conf_df
 
-def get_standings(browser, conf, season=None):
+def get_standings(browser: CloudScraper, conf: str, season: Optional[str]=None):
 	"""
 	Scrapes a given conference's standing stats (https://kenpom.com/conf.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoul StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
 		conf (str): conference abbreviation (ie B10, P12)
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		conference_df (dataframe): Dataframe containing standing stats of the conference for the given season on kenpom.com.
@@ -100,8 +101,7 @@ def get_standings(browser, conf, season=None):
 	url = url + f'?c={conf}'
 	if(season):
 		url = url + '&y=' + str(season)
-	browser.open(url)
-	confs = browser.get_current_page()
+	confs = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = confs.find_all('table')[0]
 	conf_df = pd.read_html(StringIO(str(table)))[0]
 	# Parse out seed
@@ -114,15 +114,15 @@ def get_standings(browser, conf, season=None):
 	return conf_df
 
 
-def get_offense(browser, conf, season=None):
+def get_offense(browser: CloudScraper, conf: str, season: Optional[str]=None):
 	"""
 	Scrapes a given conference's offense only stats (https://kenpom.com/conf.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoul StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
 		conf (str): conference abbreviation (ie B10, P12)
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		conference_df (dataframe): Dataframe containing offensive stats of the conference for the given season on kenpom.com.
@@ -132,8 +132,7 @@ def get_offense(browser, conf, season=None):
 	url = url + f'?c={conf}'
 	if(season):
 		url = url + '&y=' + str(season)
-	browser.open(url)
-	confs = browser.get_current_page()
+	confs = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = confs.find_all('table')[1]
 	conf_df = pd.read_html(StringIO(str(table)))[0]
 
@@ -143,15 +142,15 @@ def get_offense(browser, conf, season=None):
 	return conf_df
 
 
-def get_defense(browser, conf, season=None):
+def get_defense(browser: CloudScraper, conf: str, season: Optional[str]=None):
 	"""
 	Scrapes a given conference's defense only stats (https://kenpom.com) into a dataframe.
 
 	Args:
-		browser (mechanicalsoul StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
 		conf (str): conference abbreviation (ie B10, P12)
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		conference_df (dataframe): Dataframe containing defensive stats of the conference for the given season on kenpom.com.
@@ -161,8 +160,7 @@ def get_defense(browser, conf, season=None):
 	url = url + f'?c={conf}'
 	if(season):
 		url = url + '&y=' + str(season)
-	browser.open(url)
-	confs = browser.get_current_page()
+	confs = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = confs.find_all('table')[2]
 	conf_df = pd.read_html(StringIO(str(table)))[0]
 
diff --git a/kenpompy/misc.py b/kenpompy/misc.py
index f460221..2a9b5a3 100644
--- a/kenpompy/misc.py
+++ b/kenpompy/misc.py
@@ -6,44 +6,47 @@
 import pandas as pd
 from io import StringIO
 import re
-import mechanicalsoup
+from cloudscraper import CloudScraper
+from bs4 import BeautifulSoup
+from typing import Optional
+from .utils import get_html
 
-def get_current_season(browser: mechanicalsoup.StatefulBrowser):
+def get_current_season(browser: CloudScraper):
 	"""
 	Scrapes the KenPom homepage to get the latest season year that has data published
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
             by the `login` function.
 
 	Returns:
 		current_season (int): Number corresponding to the last season year that has data published
 	"""
-	browser.open('https://kenpom.com/index.php')
-	page_title = browser.page.select_one('#content-header h2').text
+	url = 'https://kenpom.com/index.php'
+	content = BeautifulSoup(get_html(browser, url), "html.parser")
+	page_title = content.select_one('#content-header h2').text
 	YEAR_PATTERN = r'^(\d{4})'
 	return int(re.match(YEAR_PATTERN, page_title).group(0))
 
-def get_pomeroy_ratings(browser, season=None):
+def get_pomeroy_ratings(browser: CloudScraper, season: Optional[str]=None):
     """
     Scrapes the Pomeroy College Basketball Ratings table (https://kenpom.com/index.php) into a dataframe.
 
     Args:
-        browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+        browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
             by the `login` function.
-        season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+        season (str, optional): Used to define different seasons. 1999 is the earliest available season.
             Most recent season is the default.
     Returns:
         refs_df (pandas dataframe): Pandas dataframe containing the Pomeroy College Basketball Ratings table from kenpom.com.
     Raises:
-        ValueError: If `season` is less than 2002.
+        ValueError: If `season` is less than 1999.
     """
     url = 'https://kenpom.com/index.php'
-    if season and int(season) < 2002:
-        raise ValueError("season cannot be less than 2002")
+    if season and int(season) < 1999:
+        raise ValueError("season cannot be less than 1999")
     url += '?y={}'.format(season)
-    browser.open(url)
-    page = browser.get_current_page()
+    page = BeautifulSoup(get_html(browser, url), "html.parser")
     table = page.find_all('table')[0]
     ratings_df = pd.read_html(StringIO(str(table)))
     # Dataframe tidying.
@@ -66,12 +69,12 @@ def get_pomeroy_ratings(browser, season=None):
     return ratings_df
 
 
-def get_trends(browser):
+def get_trends(browser: CloudScraper):
 	"""
 	Scrapes the statistical trends table (https://kenpom.com/trends.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 
 	Returns:
@@ -80,8 +83,7 @@ def get_trends(browser):
 
 	url = 'https://kenpom.com/trends.php'
 
-	browser.open(url)
-	trends = browser.get_current_page()
+	trends = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = trends.find_all('table')[0]
 	trends_df = pd.read_html(StringIO(str(table)))
 
@@ -92,12 +94,12 @@ def get_trends(browser):
 	return trends_df
 
 
-def get_refs(browser, season=None):
+def get_refs(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the officials rankings table (https://kenpom.com/officials.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		season (str, optional): Used to define different seasons. 2016 is the earliest available season.
 			Most recent season is the default.
@@ -117,8 +119,7 @@ def get_refs(browser, season=None):
 				'season cannot be less than 2016, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 
-	browser.open(url)
-	refs = browser.get_current_page()
+	refs = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = refs.find_all('table')[0]
 	refs_df = pd.read_html(StringIO(str(table)))
 
@@ -131,14 +132,13 @@ def get_refs(browser, season=None):
 	return refs_df
 
 
-def get_hca(browser):
+def get_hca(browser: CloudScraper):
 	"""
 	Scrapes the home court advantage table (https://kenpom.com/hca.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
-		season (str, optional): Used to define different seasons. 2010 is the earliest available season.
 
 	Returns:
 		hca_df (pandas dataframe): Pandas dataframe containing the home court advantage table from kenpom.com.
@@ -146,8 +146,7 @@ def get_hca(browser):
 
 	url = 'https://kenpom.com/hca.php'
 
-	browser.open(url)
-	hca = browser.get_current_page()
+	hca = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = hca.find_all('table')[0]
 	hca_df = pd.read_html(StringIO(str(table)))
 
@@ -160,12 +159,12 @@ def get_hca(browser):
 	return hca_df
 
 
-def get_arenas(browser, season=None):
+def get_arenas(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the arenas table (https://kenpom.com/arenas.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		season (str, optional): Used to define different seasons. 2010 is the earliest available season.
 			Most recent season is the default.
@@ -185,8 +184,7 @@ def get_arenas(browser, season=None):
 				'season cannot be less than 2010, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 
-	browser.open(url)
-	arenas = browser.get_current_page()
+	arenas = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = arenas.find_all('table')[0]
 	arenas_df = pd.read_html(StringIO(str(table)))
 
@@ -201,12 +199,12 @@ def get_arenas(browser, season=None):
 	return arenas_df
 
 
-def get_gameattribs(browser, season=None, metric='Excitement'):
+def get_gameattribs(browser: CloudScraper, season: Optional[str]=None, metric: str='Excitement'):
 	"""
 	Scrapes the Game Attributes tables (https://kenpom.com/game_attrs.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		season (str, optional): Used to define different seasons. 2010 is the earliest available season.
 			Most recent season is the default.
@@ -247,8 +245,7 @@ def get_gameattribs(browser, season=None, metric='Excitement'):
 			)
 		url = url + '&y=' + str(season)
 
-	browser.open(url)
-	playerstats = browser.get_current_page()
+	playerstats = BeautifulSoup(get_html(browser, url), "html.parser")
 
 	table = playerstats.find_all('table')[0]
 	ga_df = pd.read_html(StringIO(str(table)))
@@ -263,12 +260,12 @@ def get_gameattribs(browser, season=None, metric='Excitement'):
 	return ga_df
 
 
-def get_program_ratings(browser):
+def get_program_ratings(browser: CloudScraper):
 	"""
 	Scrapes the program ratings table (https://kenpom.com/programs.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 
 	Returns:
@@ -277,8 +274,7 @@ def get_program_ratings(browser):
 
 	url = 'https://kenpom.com/programs.php'
 
-	browser.open(url)
-	programs = browser.get_current_page()
+	programs = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = programs.find_all('table')[0]
 	programs_df = pd.read_html(StringIO(str(table)))
 	programs_df = programs_df[0]
diff --git a/kenpompy/summary.py b/kenpompy/summary.py
index 21c5721..1cb5106 100644
--- a/kenpompy/summary.py
+++ b/kenpompy/summary.py
@@ -6,34 +6,37 @@
 import pandas as pd
 import re
 from io import StringIO
+from cloudscraper import CloudScraper
+from bs4 import BeautifulSoup
+from typing import Optional
+from .utils import get_html
 
-def get_efficiency(browser, season=None):
+def get_efficiency(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the Efficiency stats table (https://kenpom.com/summary.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season but 
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season but 
 			possession length data wasn't available until 2010. Most recent season is the default.
 
 	Returns:
 		eff_df (pandas dataframe): Pandas dataframe containing the summary efficiency/tempo table from kenpom.com.
 
 	Raises:
-		ValueError: If `season` is less than 2002.
+		ValueError: If `season` is less than 1999.
 	"""
 
 	url = 'https://kenpom.com/summary.php'
 
 	if season:
-		if int(season) < 2002:
+		if int(season) < 1999:
 			raise ValueError(
-				'season cannot be less than 2002, as data only goes back that far.')
+				'season cannot be less than 1999, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 
-	browser.open(url)
-	eff = browser.get_current_page()
+	eff = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = eff.find_all('table')[0]
 	eff_df = pd.read_html(StringIO(str(table)))
 
@@ -65,33 +68,32 @@ def get_efficiency(browser, season=None):
 	return eff_df
 
 
-def get_fourfactors(browser, season=None):
+def get_fourfactors(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the Four Factors table (https://kenpom.com/stats.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 			Most recent season is the default.
 
 	Returns:
 		ff_df (pandas dataframe): Pandas dataframe containing the summary Four Factors table from kenpom.com.
 
 	Raises:
-		ValueError: If `season` is less than 2002.
+		ValueError: If `season` is less than 1999.
 	"""
 
 	url = 'https://kenpom.com/stats.php'
 
 	if season:
-		if int(season) < 2002:
+		if int(season) < 1999:
 			raise ValueError(
-				'season cannot be less than 2002, as data only goes back that far.')
+				'season cannot be less than 1999, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 
-	browser.open(url)
-	ff = browser.get_current_page()
+	ff = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = ff.find_all('table')[0]
 	ff_df = pd.read_html(StringIO(str(table)))
 
@@ -113,23 +115,23 @@ def get_fourfactors(browser, season=None):
 	return ff_df
 
 
-def get_teamstats(browser, defense=False, season=None):
+def get_teamstats(browser: CloudScraper, defense: Optional[bool]=False, season: Optional[str]=None):
 	"""
 	Scrapes the Miscellaneous Team Stats table (https://kenpom.com/teamstats.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		defense (bool, optional): Used to flag whether the defensive teamstats table is wanted or not. False by 
 			default.
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 			Most recent season is the default.
 
 	Returns:
 			ts_df (pandas dataframe): Pandas dataframe containing the Miscellaneous Team Stats table from kenpom.com.
 
 	Raises:
-			ValueError: If `season` is less than 2002.
+			ValueError: If `season` is less than 1999.
 	"""
 
 	url = 'https://kenpom.com/teamstats.php'
@@ -137,9 +139,9 @@ def get_teamstats(browser, defense=False, season=None):
 
 	# Create URL.
 	if season:
-		if int(season) < 2002:
+		if int(season) < 1999:
 			raise ValueError(
-				'season cannot be less than 2002, as data only goes back that far.')
+				'season cannot be less than 1999, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 		if defense:
 			url = url + '&od=d'
@@ -148,8 +150,7 @@ def get_teamstats(browser, defense=False, season=None):
 		url = url + '?od=d'
 		last_cols = ['AdjDE', 'AdjDE.Rank']
 
-	browser.open(url)
-	ts = browser.get_current_page()
+	ts = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = ts.find_all('table')[0]
 	ts_df = pd.read_html(StringIO(str(table)))
 
@@ -170,34 +171,33 @@ def get_teamstats(browser, defense=False, season=None):
 	return ts_df
 
 
-def get_pointdist(browser, season=None):
+def get_pointdist(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the Team Points Distribution table (https://kenpom.com/pointdist.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 			Most recent season is the default.
 
 	Returns:
 		dist_df (pandas dataframe): Pandas dataframe containing the Team Points Distribution table from kenpom.com.
 
 	Raises:
-		ValueError: If `season` is less than 2002.
+		ValueError: If `season` is less than 1999.
 	"""
 
 	url = 'https://kenpom.com/pointdist.php'
 
 	# Create URL.
 	if season:
-		if int(season) < 2002:
+		if int(season) < 1999:
 			raise ValueError(
-				'season cannot be less than 2002, as data only goes back that far.')
+				'season cannot be less than 1999, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 
-	browser.open(url)
-	dist = browser.get_current_page()
+	dist = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = dist.find_all('table')[0]
 	dist_df = pd.read_html(StringIO(str(table)))
 
@@ -217,12 +217,12 @@ def get_pointdist(browser, season=None):
 	return dist_df
 
 
-def get_height(browser, season=None):
+def get_height(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the Height/Experience table (https://kenpom.com/height.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		season (str, optional): Used to define different seasons. 2007 is the earliest available season but 
 			continuity data wasn't available until 2008. Most recent season is the default.
@@ -242,8 +242,7 @@ def get_height(browser, season=None):
 				'Season cannot be less than 2007, as data only goes back that far.')
 		url = url + '?y=' + str(season)
 
-	browser.open(url)
-	height = browser.get_current_page()
+	height = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = height.find_all('table')[0]
 	h_df = pd.read_html(StringIO(str(table)))
 
@@ -274,12 +273,12 @@ def get_height(browser, season=None):
 	return h_df
 
 
-def get_playerstats(browser, season=None, metric='EFG', conf=None, conf_only=False):
+def get_playerstats(browser: CloudScraper, season: Optional[str]=None, metric: str='EFG', conf: Optional[str]=None, conf_only: bool=False):
 	"""
 	Scrapes the Player Leaders tables (https://kenpom.com/playerstats.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		season (str, optional): Used to define different seasons. 2004 is the earliest available season. 
 			Most recent season is the default.
@@ -338,8 +337,7 @@ def get_playerstats(browser, season=None, metric='EFG', conf=None, conf_only=Fal
 	if conf:
 		url = url + '&f=' + conf
 
-	browser.open(url)
-	playerstats = browser.get_current_page()
+	playerstats = BeautifulSoup(get_html(browser, url), "html.parser")
 	if metric == 'ORTG':
 		ps_dfs = []
 		tables = playerstats.find_all('table')
@@ -380,12 +378,12 @@ def get_playerstats(browser, season=None, metric='EFG', conf=None, conf_only=Fal
 	return ps_df
 
 
-def get_kpoy(browser, season=None):
+def get_kpoy(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the kenpom Player of the Year tables (https://kenpom.com/kpoy.php) into dataframes.
 
 	Args:
-		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function.
 		season (str, optional): Used to define different seasons. 2011 is the earliest available season.
 			Most recent season is the default.
@@ -410,8 +408,7 @@ def get_kpoy(browser, season=None):
 	else:
 		season = 2013
 
-	browser.open(url)
-	kpoy = browser.get_current_page()
+	kpoy = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = kpoy.find_all('table')[0]
 	df = pd.read_html(StringIO(str(table)))
 
@@ -443,4 +440,4 @@ def get_kpoy(browser, season=None):
 
 		kpoy_dfs.append(mvp_df)
 
-	return kpoy_dfs
\ No newline at end of file
+	return kpoy_dfs
diff --git a/kenpompy/team.py b/kenpompy/team.py
index 651e1d6..00d66b6 100644
--- a/kenpompy/team.py
+++ b/kenpompy/team.py
@@ -7,17 +7,20 @@
 from io import StringIO
 from .misc import get_current_season
 import re
+from cloudscraper import CloudScraper
 from bs4 import BeautifulSoup
 from codecs import encode, decode
+from typing import Optional
+from .utils import get_html
 
-def get_valid_teams(browser, season=None):
+def get_valid_teams(browser: CloudScraper, season: Optional[str]=None):
 	"""
 	Scrapes the teams (https://kenpom.com) into a list.
 
 	Args:
-		browser (mechanicalsoup.StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		team_list (list): List containing all valid teams for the given season on kenpom.com.
@@ -26,8 +29,7 @@ def get_valid_teams(browser, season=None):
 	url = "https://kenpom.com"
 	url = url + '?y=' + str(season)
 
-	browser.open(url)
-	teams = browser.get_current_page()
+	teams = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = teams.find_all('table')[0]
 	team_df = pd.read_html(StringIO(str(table)))
 	# Get only the team column.
@@ -42,21 +44,21 @@ def get_valid_teams(browser, season=None):
 
 	return team_list
 
-def get_schedule(browser, team=None, season=None):
+def get_schedule(browser: CloudScraper, team: Optional[str]=None, season: Optional[str]=None):
 	"""
 	Scrapes a team's schedule from (https://kenpom.com/team.php) into a dataframe.
 
 	Args:
-		browser (mechanicalsoup.StatefulBrowser): Authenticated browser with full access to kenpom.com generated
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
 			by the `login` function
-		team: Used to determine which team to scrape for schedule.
-		season (str, optional): Used to define different seasons. 2002 is the earliest available season.
+		team (str, optional): Used to determine which team to scrape for schedule.
+		season (str, optional): Used to define different seasons. 1999 is the earliest available season.
 
 	Returns:
 		team_df (pandas dataframe): Dataframe containing a team's schedule for the given season.
 
 	Raises:
-		ValueError if `season` is less than 2002.
+		ValueError if `season` is less than 1999.
 		ValueError if `season` is greater than the current year.
 		ValueError if `team` is not in the valid team list.
 	"""
@@ -65,9 +67,9 @@ def get_schedule(browser, team=None, season=None):
 	current_season = get_current_season(browser)
 
 	if season:
-		if int(season) < 2002:
+		if int(season) < 1999:
 			raise ValueError(
-				'season cannot be less than 2002, as data only goes back that far.')
+				'season cannot be less than 1999, as data only goes back that far.')
 		if int(season) > int(current_season):
 			raise ValueError(
 				'season cannot be greater than the current year.')
@@ -84,8 +86,7 @@ def get_schedule(browser, team=None, season=None):
 	url = url + "?team=" + str(team)
 	url = url + "&y=" + str(season)
 
-	browser.open(url)
-	schedule = browser.get_current_page()
+	schedule = BeautifulSoup(get_html(browser, url), "html.parser")
 	table = schedule.find_all('table')[1]
 	schedule_df = pd.read_html(StringIO(str(table)))
 
@@ -119,21 +120,21 @@ def get_schedule(browser, team=None, season=None):
 
 	return schedule_df.reset_index(drop=True)
 
-def get_scouting_report(browser, team=None, season=None, conference_only=False):
+def get_scouting_report(browser: CloudScraper, team: str, season: Optional[int]=None, conference_only: bool=False):
 	"""
     Retrieves and parses team scouting report data from (https://kenpom.com/team.php) into a dictionary.
 
     Args:
-    	browser (mechanicalsoup.StatefulBrowser): The mechanize browser object for web scraping.
+    	browser (CloudScraper): The mechanize browser object for web scraping.
     	team (str): team: Used to determine which team to scrape for schedule.
-    	season (int, optional): Used to define different seasons. 2002 is the earliest available season.
+    	season (int, optional): Used to define different seasons. 1999 is the earliest available season.
     	conference_only (bool, optional): When True, only conference-related stats are retrieved; otherwise, all stats are fetched.
 
     Returns:
     	dict: A dictionary containing various team statistics.
 
     Raises:
-    	ValueError if the provided season is earlier than 2002 or greater than the current year
+    	ValueError if the provided season is earlier than 1999 or greater than the current year
 		ValueError if the team name is invalid or not found in the specified year
 	"""
 
@@ -142,9 +143,9 @@ def get_scouting_report(browser, team=None, season=None, conference_only=False):
 	current_season = get_current_season(browser)
 
 	if season:
-		if int(season) < 2002:
+		if int(season) < 1999:
 			raise ValueError(
-				'season cannot be less than 2002, as data only goes back that far.')
+				'season cannot be less than 1999, as data only goes back that far.')
 		if int(season) > current_season:
 			raise ValueError(
 				'season cannot be greater than the current year.')
@@ -161,8 +162,8 @@ def get_scouting_report(browser, team=None, season=None, conference_only=False):
 	url = url + "?team=" + str(team)
 	url = url + "&y=" + str(season)
 
-	browser.open(url)
-	scouting_report_scripts = browser.page.find("script", { "type": "text/javascript", "src": ""} )
+	report = BeautifulSoup(get_html(browser, url), "html.parser")
+	scouting_report_scripts = report.find("script", { "type": "text/javascript", "src": ""} )
 
 	extraction_pattern = re.compile(r"\$\(\"td#(?P<token>[A-Za-z0-9]+)\"\)\.html\(\"(.+)\"\);")
 	if conference_only:
@@ -177,4 +178,4 @@ def get_scouting_report(browser, team=None, season=None, conference_only=False):
 	for stat in stats:
 		stats_df[stat[0]] = stat[1]
 		stats_df[stat[0]+'.Rank'] = stat[2]
-	return stats_df
\ No newline at end of file
+	return stats_df
diff --git a/kenpompy/utils.py b/kenpompy/utils.py
index 9e81b02..c40f32c 100644
--- a/kenpompy/utils.py
+++ b/kenpompy/utils.py
@@ -2,11 +2,10 @@
 The utils module provides utility functions, such as logging in.
 """
 
-import mechanicalsoup
-from requests import Session
-from ._DESAdapter import DESAdapter, environment_requires_DES_adapter
+import cloudscraper
+from cloudscraper import CloudScraper
 
-def login(email, password):
+def login(email: str, password: str):
 	"""
 	Logs in to kenpom.com using user credentials.
 
@@ -18,33 +17,44 @@ def login(email, password):
 		browser (mechanicalsoup StatefulBrowser): Authenticated browser with full access to kenpom.com.
 	"""
 
-	# Fix for Cloudflare SSL profiling (https://github.com/j-andrews7/kenpompy/issues/33) provided by Nick Ostendorf (@nickostendorf)
-	session = Session()
-	if environment_requires_DES_adapter():
-		session.mount('https://kenpom.com/', DESAdapter())
+	browser = cloudscraper.create_scraper()
+	browser.get('https://kenpom.com/index.php')
 
-	browser = mechanicalsoup.StatefulBrowser(session)
-	browser.set_user_agent('Mozilla/5.0')
-	browser.open('https://kenpom.com/index.php')
-
-	if 'Cloudflare' in browser.page.title.string:
-		raise Exception(
-			'Opening kenpom.com failed - request was intercepted by Cloudflare protection')
+	form_data = {
+		'email': email,
+		'password': password,
+		'submit': 'Login!',
+	}
 
 	# Response page actually throws an error but further navigation works and will show you as logged in.
-	browser.get_current_page()
-	browser.select_form('form[action="handlers/login_handler.php"]')
-	browser['email'] = email
-	browser['password'] = password
-
-	response = browser.submit_selected()
+	browser.post(
+		'https://kenpom.com/handlers/login_handler.php',
+		data=form_data, 
+		allow_redirects=True
+	)
 
-	if response.status_code != 200 or 'PHPSESSID=' not in response.headers['set-cookie']:
-		raise Exception(
-			'Logging in to kenpom.com failed - check that the site is available and your credentials are correct.')
-	
-	if 'subscription expired' in str(browser.get('https://kenpom.com/index.php').content):
-		raise Exception(
-			'Logging in to kenpom.com failed - account subscription is expired')
+	home_page = browser.get('https://kenpom.com/')
+	if 'Logout' not in home_page.text:
+		raise Exception('Logging in failed - check your credentials')
 
 	return browser
+
+def get_html(browser: CloudScraper, url: str):
+	"""
+	Performs a get request on the specified url.
+
+	Args:
+		browser (CloudScraper): Authenticated browser with full access to kenpom.com generated
+            by the `login` function.
+		url (str): The url to perform the get request on.
+	
+	Returns:
+		html (Bytes | Any): The return content.
+	
+	Raises:
+		Exception if get request gets a non-200 response code.
+	"""	
+	response = browser.get(url)
+	if response.status_code != 200:
+		raise Exception(f'Failed to retrieve {url} (status code: {response.status_code})')
+	return response.content
\ No newline at end of file
diff --git a/requirements.txt b/requirements.txt
index 229629c..1ace735 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,4 +1,5 @@
-mechanicalsoup
 pandas
 bs4
-sphinx>2.2.0
\ No newline at end of file
+sphinx>2.2.0
+cloudscraper
+lxml
diff --git a/setup.py b/setup.py
index 5a75f23..ce4cfd6 100644
--- a/setup.py
+++ b/setup.py
@@ -5,7 +5,7 @@
 
 setuptools.setup(
     name="kenpompy",
-    version="0.3.5",
+    version="0.4.0",
     author="Jared Andrews",
     author_email="jared.andrews07@gmail.com",
     description="A python package for scraping kenpom.com NCAA basketball data.",
@@ -22,6 +22,6 @@
         "Intended Audience :: Science/Research",
         "Intended Audience :: Developers"
     ],
-    install_requires = ["mechanicalsoup", "pandas", "bs4"],
+    install_requires = ["mechanicalsoup", "pandas", "bs4", "cloudscraper"],
     python_requires='>=3.8',
 )
\ No newline at end of file


Test patch:
diff --git a/tests/test_conference.py b/tests/test_conference.py
index dab4992..b3c2e33 100644
--- a/tests/test_conference.py
+++ b/tests/test_conference.py
@@ -67,8 +67,8 @@ def test_get_standings(browser):
 	
 	confs_2021 = kpconf.get_standings(browser, 'B10', season = '2021')
 	assert confs_2021.iloc[0, :]['Team'] == expectedTeam1
-	assert confs_2021.iloc[0, :]['AdjO'] == expectedTeam1AdjO
-	assert confs_2021.iloc[0, :]['AdjO.Rank'] == expectedTeam1AdjORank
+	assert confs_2021.iloc[0, :]['ORtg'] == expectedTeam1AdjO
+	assert confs_2021.iloc[0, :]['ORtg.Rank'] == expectedTeam1AdjORank
 
 	expectedTeam1 = 'Wisconsin'
 	expectedTeam1AdjO = 114.4
@@ -76,8 +76,8 @@ def test_get_standings(browser):
 
 	confs_2003 = kpconf.get_standings(browser, 'B10', season = '2003')
 	assert confs_2003.iloc[0, :]['Team'] == expectedTeam1
-	assert confs_2003.iloc[0, :]['AdjO'] == expectedTeam1AdjO
-	assert confs_2003.iloc[0, :]['AdjO.Rank'] == expectedTeam1AdjORank
+	assert confs_2003.iloc[0, :]['ORtg'] == expectedTeam1AdjO
+	assert confs_2003.iloc[0, :]['ORtg.Rank'] == expectedTeam1AdjORank
 
 
 def test_get_offense(browser):
diff --git a/tests/test_fanmatch.py b/tests/test_fanmatch.py
index 4549cd7..9c693d5 100644
--- a/tests/test_fanmatch.py
+++ b/tests/test_fanmatch.py
@@ -17,6 +17,7 @@ def test_fanmatch(browser):
 				"Marquette",
 				"73-72",
 				"51%",
+                "nan",
 				"1",
 				"Xavier",
 				"2OT",
@@ -54,4 +55,14 @@ def test_fanmatch(browser):
 	date = "2016-04-02"
 	fm = FanMatch(browser, date)
 	assert fm.fm_df.Tournament.loc[0] == "NCAA"
-	assert "NCAA" not in fm.fm_df.Game.loc[0]
\ No newline at end of file
+	assert "NCAA" not in fm.fm_df.Game.loc[0]
+
+    # Test for a day with no game that is in season
+	date = "2020-12-24"
+	fm = FanMatch(browser, date)
+	assert fm.fm_df is None
+
+    # Test for a day with no games that is during the offseason
+	date = "2024-10-30"
+	fm = FanMatch(browser, date)
+	assert fm.fm_df is None
diff --git a/tests/test_misc.py b/tests/test_misc.py
index 4fe91f9..196beed 100644
--- a/tests/test_misc.py
+++ b/tests/test_misc.py
@@ -89,5 +89,5 @@ def test_get_gameattribs(browser):
 
 def test_get_program_ratings(browser):
 	df = kpmisc.get_program_ratings(browser)
-	expected = (len(browser.page.select("tr:not(:has(th))")), 17)
+	expected = (362, 17)
 	assert df.shape == expected
\ No newline at end of file
diff --git a/tests/test_summary.py b/tests/test_summary.py
index 47843c8..f8c05c5 100644
--- a/tests/test_summary.py
+++ b/tests/test_summary.py
@@ -16,7 +16,7 @@ def test_get_efficiency(browser):
 	assert [str(i) for i in df[df.Team == 'Louisville'].iloc[0].to_list()] == expected
 
 	with pytest.raises(ValueError):
-		kpsum.get_efficiency(browser, season = '2001')
+		kpsum.get_efficiency(browser, season = '1998')
 
 
 def test_get_fourfactors(browser):
@@ -27,7 +27,7 @@ def test_get_fourfactors(browser):
 	assert [str(i) for i in df[df.Team == 'Louisville'].iloc[0].to_list()] == expected
 
 	with pytest.raises(ValueError):
-		kpsum.get_fourfactors(browser, season = '2001')
+		kpsum.get_fourfactors(browser, season = '1998')
 
 
 def test_get_teamstats(browser):
@@ -44,7 +44,7 @@ def test_get_teamstats(browser):
 	assert [str(i) for i in df[df.Team == 'Louisville'].iloc[0].to_list()] == expected
 
 	with pytest.raises(ValueError):
-		kpsum.get_teamstats(browser, season = '2001')
+		kpsum.get_teamstats(browser, season = '1998')
 
 
 def test_get_pointdist(browser):
@@ -55,7 +55,7 @@ def test_get_pointdist(browser):
 	assert [str(i) for i in df[df.Team == 'Louisville'].iloc[0].to_list()] == expected
 
 	with pytest.raises(ValueError):
-		kpsum.get_pointdist(browser, season = '2001')
+		kpsum.get_pointdist(browser, season = '1998')
 
 
 def test_get_height(browser):
diff --git a/tests/test_team.py b/tests/test_team.py
index 9dcad93..8330561 100644
--- a/tests/test_team.py
+++ b/tests/test_team.py
@@ -49,7 +49,7 @@ def test_get_schedule(browser):
 	nextYear = str(int(currentYear)+1)
 
 	with pytest.raises(ValueError):
-		kpteam.get_schedule(browser, team="Iowa", season = '2001')
+		kpteam.get_schedule(browser, team="Iowa", season = '1998')
 
 	with pytest.raises(ValueError):
 		kpteam.get_schedule(browser, team="Kansas", season = nextYear)
@@ -98,4 +98,4 @@ def test_get_scouting_report(browser):
 	assert data['OE'] == 109.5
 	assert data['APLD'] == ''
 	# Not including SoS and Personnel (88 with)
-	assert len(data) == 70
\ No newline at end of file
+	assert len(data) == 70