Repository: google/budoux

Issue:
CVE-2007-4559 Patch
# Patching CVE-2007-4559

Hi, we are security researchers from the Advanced Research Center at [Trellix](https://www.trellix.com). We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this [blog](https://www.trellix.com/en-us/about/newsroom/stories/research/tarfile-exploiting-the-world.html).

If you have further questions you may contact us through this projects lead researcher [Kasimir Schulz](mailto:kasimir.schulz@trellix.com).



Discussion:
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this [failed invocation](https://github.com/google/budoux/pull/94/checks?check_run_id=9569053252) of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Resolving patch:
diff --git a/scripts/load_knbc.py b/scripts/prepare_knbc.py
similarity index 80%
rename from scripts/load_knbc.py
rename to scripts/prepare_knbc.py
--- a/scripts/load_knbc.py
+++ b/scripts/prepare_knbc.py
@@ -15,17 +15,11 @@
 
 import argparse
 import os
-import tarfile
 import typing
-import urllib.error
-import urllib.request
 from html.parser import HTMLParser
 
 from budoux import utils
 
-RESOURCE_URL = (
-    'https://nlp.ist.i.kyoto-u.ac.jp/kuntt/KNBC_v1.0_090925_utf8.tar.bz2')
-
 
 class KNBCHTMLParser(HTMLParser):
   """Parses the HTML files in the KNBC corpus and outputs the chunks."""
@@ -95,25 +89,9 @@ def postprocess(chunks: typing.List[str]) -> typing.List[str]:
   return chunks
 
 
-def download_knbc(target_dir: str) -> None:
-  """Downloads the KNBC corpus and extracts files.
-
-  Args:
-    target_dir: A path to the directory to expand files.
-  """
-  os.makedirs(target_dir, exist_ok=True)
-  download_file_path = os.path.join(target_dir, 'knbc.tar.bz2')
-  try:
-    urllib.request.urlretrieve(RESOURCE_URL, download_file_path)
-  except urllib.error.HTTPError:
-    print(f'\033[91mResource unavailable: {RESOURCE_URL}\033[0m')
-    raise
-  with tarfile.open(download_file_path, 'r:bz2') as t:
-    t.extractall(path=target_dir)
-
-
 def parse_args() -> argparse.Namespace:
   parser = argparse.ArgumentParser(description=__doc__)
+  parser.add_argument('source_dir', help='Path to the KNBC corpus directory.')
   parser.add_argument(
       '-o',
       '--outfile',
@@ -125,10 +103,9 @@ def parse_args() -> argparse.Namespace:
 
 def main() -> None:
   args = parse_args()
+  source_dir = args.source_dir
   outfile = args.outfile
-  html_dir = 'data/KNBC_v1.0_090925_utf8/html/'
-  if not os.path.isdir(html_dir):
-    download_knbc('data')
+  html_dir = os.path.join(source_dir, 'html')
   with open(outfile, 'w') as f:
     for file in sorted(os.listdir(html_dir)):
       if file[-11:] != '-morph.html':


Test patch: