neulab/docprompting-conala huggingface dataset:
urllib.robotparser — Parser for robots.txt Source code: Lib/urllib/robotparser.py This module provides a single class, RobotFileParser, which answers questions about whether or not a particular user agent can fetch a URL on the Web site that published the robots.txt file. For more details on the structure of robots.txt files, see http://www.robotstxt.org/orig.html.  
class urllib.robotparser.RobotFileParser(url='')  
This class provides methods to read, parse and answer questions about the robots.txt file at url.  
set_url(url)  
Sets the URL referring to a robots.txt file. 
  
read()  
Reads the robots.txt URL and feeds it to the parser. 
  
parse(lines)  
Parses the lines argument. 
  
can_fetch(useragent, url)  
Returns True if the useragent is allowed to fetch the url according to the rules contained in the parsed robots.txt file. 
  
mtime()  
Returns the time the robots.txt file was last fetched. This is useful for long-running web spiders that need to check for new robots.txt files periodically. 
  
modified()  
Sets the time the robots.txt file was last fetched to the current time. 
  
crawl_delay(useragent)  
Returns the value of the Crawl-delay parameter from robots.txt for the useragent in question. If there is no such parameter or it doesn’t apply to the useragent specified or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.6.  
  
request_rate(useragent)  
Returns the contents of the Request-rate parameter from robots.txt as a named tuple RequestRate(requests, seconds). If there is no such parameter or it doesn’t apply to the useragent specified or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.6.  
  
site_maps()  
Returns the contents of the Sitemap parameter from robots.txt in the form of a list(). If there is no such parameter or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.8.  
 
 The following example demonstrates basic use of the RobotFileParser class: >>> import urllib.robotparser
>>> rp = urllib.robotparser.RobotFileParser()
>>> rp.set_url("http://www.musi-cal.com/robots.txt")
>>> rp.read()
>>> rrate = rp.request_rate("*")
>>> rrate.requests
3
>>> rrate.seconds
20
>>> rp.crawl_delay("*")
6
>>> rp.can_fetch("*", "http://www.musi-cal.com/cgi-bin/search?city=San+Francisco")
False
>>> rp.can_fetch("*", "http://www.musi-cal.com/")
True
class urllib.robotparser.RobotFileParser(url='')  
This class provides methods to read, parse and answer questions about the robots.txt file at url.  
set_url(url)  
Sets the URL referring to a robots.txt file. 
  
read()  
Reads the robots.txt URL and feeds it to the parser. 
  
parse(lines)  
Parses the lines argument. 
  
can_fetch(useragent, url)  
Returns True if the useragent is allowed to fetch the url according to the rules contained in the parsed robots.txt file. 
  
mtime()  
Returns the time the robots.txt file was last fetched. This is useful for long-running web spiders that need to check for new robots.txt files periodically. 
  
modified()  
Sets the time the robots.txt file was last fetched to the current time. 
  
crawl_delay(useragent)  
Returns the value of the Crawl-delay parameter from robots.txt for the useragent in question. If there is no such parameter or it doesn’t apply to the useragent specified or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.6.  
  
request_rate(useragent)  
Returns the contents of the Request-rate parameter from robots.txt as a named tuple RequestRate(requests, seconds). If there is no such parameter or it doesn’t apply to the useragent specified or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.6.  
  
site_maps()  
Returns the contents of the Sitemap parameter from robots.txt in the form of a list(). If there is no such parameter or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.8.
can_fetch(useragent, url)  
Returns True if the useragent is allowed to fetch the url according to the rules contained in the parsed robots.txt file.
crawl_delay(useragent)  
Returns the value of the Crawl-delay parameter from robots.txt for the useragent in question. If there is no such parameter or it doesn’t apply to the useragent specified or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.6.
modified()  
Sets the time the robots.txt file was last fetched to the current time.
mtime()  
Returns the time the robots.txt file was last fetched. This is useful for long-running web spiders that need to check for new robots.txt files periodically.
request_rate(useragent)  
Returns the contents of the Request-rate parameter from robots.txt as a named tuple RequestRate(requests, seconds). If there is no such parameter or it doesn’t apply to the useragent specified or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.6.
site_maps()  
Returns the contents of the Sitemap parameter from robots.txt in the form of a list(). If there is no such parameter or the robots.txt entry for this parameter has invalid syntax, return None.  New in version 3.8.
uu — Encode and decode uuencode files Source code: Lib/uu.py This module encodes and decodes files in uuencode format, allowing arbitrary binary data to be transferred over ASCII-only connections. Wherever a file argument is expected, the methods accept a file-like object. For backwards compatibility, a string containing a pathname is also accepted, and the corresponding file will be opened for reading and writing; the pathname '-' is understood to mean the standard input or output. However, this interface is deprecated; it’s better for the caller to open the file itself, and be sure that, when required, the mode is 'rb' or 'wb' on Windows. This code was contributed by Lance Ellinghouse, and modified by Jack Jansen. The uu module defines the following functions:  
uu.encode(in_file, out_file, name=None, mode=None, *, backtick=False)  
Uuencode file in_file into file out_file. The uuencoded file will have the header specifying name and mode as the defaults for the results of decoding the file. The default defaults are taken from in_file, or '-' and 0o666 respectively. If backtick is true, zeros are represented by '`' instead of spaces.  Changed in version 3.7: Added the backtick parameter.  
  
uu.decode(in_file, out_file=None, mode=None, quiet=False)  
This call decodes uuencoded file in_file placing the result on file out_file. If out_file is a pathname, mode is used to set the permission bits if the file must be created. Defaults for out_file and mode are taken from the uuencode header. However, if the file specified in the header already exists, a uu.Error is raised. decode() may print a warning to standard error if the input was produced by an incorrect uuencoder and Python could recover from that error. Setting quiet to a true value silences this warning. 
  
exception uu.Error  
Subclass of Exception, this can be raised by uu.decode() under various situations, such as described above, but also including a badly formatted header, or truncated input file. 
  See also  
Module binascii


Support module containing ASCII-to-binary and binary-to-ASCII conversions.
uu.decode(in_file, out_file=None, mode=None, quiet=False)  
This call decodes uuencoded file in_file placing the result on file out_file. If out_file is a pathname, mode is used to set the permission bits if the file must be created. Defaults for out_file and mode are taken from the uuencode header. However, if the file specified in the header already exists, a uu.Error is raised. decode() may print a warning to standard error if the input was produced by an incorrect uuencoder and Python could recover from that error. Setting quiet to a true value silences this warning.
uu.encode(in_file, out_file, name=None, mode=None, *, backtick=False)  
Uuencode file in_file into file out_file. The uuencoded file will have the header specifying name and mode as the defaults for the results of decoding the file. The default defaults are taken from in_file, or '-' and 0o666 respectively. If backtick is true, zeros are represented by '`' instead of spaces.  Changed in version 3.7: Added the backtick parameter.
exception uu.Error  
Subclass of Exception, this can be raised by uu.decode() under various situations, such as described above, but also including a badly formatted header, or truncated input file.
uuid.getnode()  
Get the hardware address as a 48-bit positive integer. The first time this runs, it may launch a separate program, which could be quite slow. If all attempts to obtain the hardware address fail, we choose a random 48-bit number with the multicast bit (least significant bit of the first octet) set to 1 as recommended in RFC 4122. “Hardware address” means the MAC address of a network interface. On a machine with multiple network interfaces, universally administered MAC addresses (i.e. where the second least significant bit of the first octet is unset) will be preferred over locally administered MAC addresses, but with no other ordering guarantees.  Changed in version 3.7: Universally administered MAC addresses are preferred over locally administered MAC addresses, since the former are guaranteed to be globally unique, while the latter are not.
uuid.NAMESPACE_DNS  
When this namespace is specified, the name string is a fully-qualified domain name.
uuid.NAMESPACE_OID  
When this namespace is specified, the name string is an ISO OID.
uuid.NAMESPACE_URL  
When this namespace is specified, the name string is a URL.
uuid.NAMESPACE_X500  
When this namespace is specified, the name string is an X.500 DN in DER or a text output format.
uuid.RFC_4122  
Specifies the UUID layout given in RFC 4122.
class uuid.SafeUUID  
 New in version 3.7.   
safe  
The UUID was generated by the platform in a multiprocessing-safe way. 
  
unsafe  
The UUID was not generated in a multiprocessing-safe way. 
  
unknown  
The platform does not provide information on whether the UUID was generated safely or not.
class uuid.UUID(hex=None, bytes=None, bytes_le=None, fields=None, int=None, version=None, *, is_safe=SafeUUID.unknown)  
Create a UUID from either a string of 32 hexadecimal digits, a string of 16 bytes in big-endian order as the bytes argument, a string of 16 bytes in little-endian order as the bytes_le argument, a tuple of six integers (32-bit time_low, 16-bit time_mid, 16-bit time_hi_version, 8-bit clock_seq_hi_variant, 8-bit clock_seq_low, 48-bit node) as the fields argument, or a single 128-bit integer as the int argument. When a string of hex digits is given, curly braces, hyphens, and a URN prefix are all optional. For example, these expressions all yield the same UUID: UUID('{12345678-1234-5678-1234-567812345678}')
UUID('12345678123456781234567812345678')
UUID('urn:uuid:12345678-1234-5678-1234-567812345678')
UUID(bytes=b'\x12\x34\x56\x78'*4)
UUID(bytes_le=b'\x78\x56\x34\x12\x34\x12\x78\x56' +
              b'\x12\x34\x56\x78\x12\x34\x56\x78')
UUID(fields=(0x12345678, 0x1234, 0x5678, 0x12, 0x34, 0x567812345678))
UUID(int=0x12345678123456781234567812345678)
 Exactly one of hex, bytes, bytes_le, fields, or int must be given. The version argument is optional; if given, the resulting UUID will have its variant and version number set according to RFC 4122, overriding bits in the given hex, bytes, bytes_le, fields, or int. Comparison of UUID objects are made by way of comparing their UUID.int attributes. Comparison with a non-UUID object raises a TypeError. str(uuid) returns a string in the form 12345678-1234-5678-1234-567812345678 where the 32 hexadecimal digits represent the UUID.
UUID.bytes  
The UUID as a 16-byte string (containing the six integer fields in big-endian byte order).
UUID.bytes_le  
The UUID as a 16-byte string (with time_low, time_mid, and time_hi_version in little-endian byte order).
UUID.fields  
A tuple of the six integer fields of the UUID, which are also available as six individual attributes and two derived attributes:   
Field Meaning   
time_low the first 32 bits of the UUID  
time_mid the next 16 bits of the UUID  
time_hi_version the next 16 bits of the UUID  
clock_seq_hi_variant the next 8 bits of the UUID  
clock_seq_low the next 8 bits of the UUID  
node the last 48 bits of the UUID  
time the 60-bit timestamp  
clock_seq the 14-bit sequence number
UUID.is_safe  
An enumeration of SafeUUID which indicates whether the platform generated the UUID in a multiprocessing-safe way.  New in version 3.7.
UUID.variant  
The UUID variant, which determines the internal layout of the UUID. This will be one of the constants RESERVED_NCS, RFC_4122, RESERVED_MICROSOFT, or RESERVED_FUTURE.
UUID.version  
The UUID version number (1 through 5, meaningful only when the variant is RFC_4122).
uuid.uuid1(node=None, clock_seq=None)  
Generate a UUID from a host ID, sequence number, and the current time. If node is not given, getnode() is used to obtain the hardware address. If clock_seq is given, it is used as the sequence number; otherwise a random 14-bit sequence number is chosen.
uuid.uuid3(namespace, name)  
Generate a UUID based on the MD5 hash of a namespace identifier (which is a UUID) and a name (which is a string).
uuid.uuid5(namespace, name)  
Generate a UUID based on the SHA-1 hash of a namespace identifier (which is a UUID) and a name (which is a string).
exception ValueError  
Raised when an operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError.
vars([object])  
Return the __dict__ attribute for a module, class, instance, or any other object with a __dict__ attribute. Objects such as modules and instances have an updateable __dict__ attribute; however, other objects may have write restrictions on their __dict__ attributes (for example, classes use a types.MappingProxyType to prevent direct dictionary updates). Without an argument, vars() acts like locals(). Note, the locals dictionary is only useful for reads since updates to the locals dictionary are ignored. A TypeError exception is raised if an object is specified but it doesn’t have a __dict__ attribute (for example, if its class defines the __slots__ attribute).
venv.create(env_dir, system_site_packages=False, clear=False, symlinks=False, with_pip=False, prompt=None)  
Create an EnvBuilder with the given keyword arguments, and call its create() method with the env_dir argument.  New in version 3.3.   Changed in version 3.4: Added the with_pip parameter   Changed in version 3.6: Added the prompt parameter
create(env_dir)  
Create a virtual environment by specifying the target directory (absolute or relative to the current directory) which is to contain the virtual environment. The create method will either create the environment in the specified directory, or raise an appropriate exception. The create method of the EnvBuilder class illustrates the hooks available for subclass customization: def create(self, env_dir):
    """
    Create a virtualized Python environment in a directory.
    env_dir is the target directory to create an environment in.
    """
    env_dir = os.path.abspath(env_dir)
    context = self.ensure_directories(env_dir)
    self.create_configuration(context)
    self.setup_python(context)
    self.setup_scripts(context)
    self.post_setup(context)
 Each of the methods ensure_directories(), create_configuration(), setup_python(), setup_scripts() and post_setup() can be overridden.
create_configuration(context)  
Creates the pyvenv.cfg configuration file in the environment.
ensure_directories(env_dir)  
Creates the environment directory and all necessary directories, and returns a context object. This is just a holder for attributes (such as paths), for use by the other methods. The directories are allowed to exist already, as long as either clear or upgrade were specified to allow operating on an existing environment directory.
install_scripts(context, path)  
path is the path to a directory that should contain subdirectories “common”, “posix”, “nt”, each containing scripts destined for the bin directory in the environment. The contents of “common” and the directory corresponding to os.name are copied after some text replacement of placeholders:  
__VENV_DIR__ is replaced with the absolute path of the environment directory. 
__VENV_NAME__ is replaced with the environment name (final path segment of environment directory). 
__VENV_PROMPT__ is replaced with the prompt (the environment name surrounded by parentheses and with a following space) 
__VENV_BIN_NAME__ is replaced with the name of the bin directory (either bin or Scripts). 
__VENV_PYTHON__ is replaced with the absolute path of the environment’s executable.  The directories are allowed to exist (for when an existing environment is being upgraded).
post_setup(context)  
A placeholder method which can be overridden in third party implementations to pre-install packages in the virtual environment or perform other post-creation steps.
setup_python(context)  
Creates a copy or symlink to the Python executable in the environment. On POSIX systems, if a specific executable python3.x was used, symlinks to python and python3 will be created pointing to that executable, unless files with those names already exist.
setup_scripts(context)  
Installs activation scripts appropriate to the platform into the virtual environment.
upgrade_dependencies(context)  
Upgrades the core venv dependency packages (currently pip and setuptools) in the environment. This is done by shelling out to the pip executable in the environment.  New in version 3.9.
class warnings.catch_warnings(*, record=False, module=None)  
A context manager that copies and, upon exit, restores the warnings filter and the showwarning() function. If the record argument is False (the default) the context manager returns None on entry. If record is True, a list is returned that is progressively populated with objects as seen by a custom showwarning() function (which also suppresses output to sys.stdout). Each object in the list has attributes with the same names as the arguments to showwarning(). The module argument takes a module that will be used instead of the module returned when you import warnings whose filter will be protected. This argument exists primarily for testing the warnings module itself.  Note The catch_warnings manager works by replacing and then later restoring the module’s showwarning() function and internal list of filter specifications. This means the context manager is modifying global state and therefore is not thread-safe.
warnings.filterwarnings(action, message='', category=Warning, module='', lineno=0, append=False)  
Insert an entry into the list of warnings filter specifications. The entry is inserted at the front by default; if append is true, it is inserted at the end. This checks the types of the arguments, compiles the message and module regular expressions, and inserts them as a tuple in the list of warnings filters. Entries closer to the front of the list override entries later in the list, if both match a particular warning. Omitted arguments default to a value that matches everything.
warnings.formatwarning(message, category, filename, lineno, line=None)  
Format a warning the standard way. This returns a string which may contain embedded newlines and ends in a newline. line is a line of source code to be included in the warning message; if line is not supplied, formatwarning() will try to read the line specified by filename and lineno.
warnings.resetwarnings()  
Reset the warnings filter. This discards the effect of all previous calls to filterwarnings(), including that of the -W command line options and calls to simplefilter().
warnings.showwarning(message, category, filename, lineno, file=None, line=None)  
Write a warning to a file. The default implementation calls formatwarning(message, category, filename, lineno, line) and writes the resulting string to file, which defaults to sys.stderr. You may replace this function with any callable by assigning to warnings.showwarning. line is a line of source code to be included in the warning message; if line is not supplied, showwarning() will try to read the line specified by filename and lineno.
warnings.simplefilter(action, category=Warning, lineno=0, append=False)  
Insert a simple entry into the list of warnings filter specifications. The meaning of the function parameters is as for filterwarnings(), but regular expressions are not needed as the filter inserted always matches any message in any module as long as the category and line number match.
warnings.warn(message, category=None, stacklevel=1, source=None)  
Issue a warning, or maybe ignore it or raise an exception. The category argument, if given, must be a warning category class; it defaults to UserWarning. Alternatively, message can be a Warning instance, in which case category will be ignored and message.__class__ will be used. In this case, the message text will be str(message). This function raises an exception if the particular warning issued is changed into an error by the warnings filter. The stacklevel argument can be used by wrapper functions written in Python, like this: def deprecation(message):
    warnings.warn(message, DeprecationWarning, stacklevel=2)
 This makes the warning refer to deprecation()’s caller, rather than to the source of deprecation() itself (since the latter would defeat the purpose of the warning message). source, if supplied, is the destroyed object which emitted a ResourceWarning.  Changed in version 3.6: Added source parameter.
warnings.warn_explicit(message, category, filename, lineno, module=None, registry=None, module_globals=None, source=None)  
This is a low-level interface to the functionality of warn(), passing in explicitly the message, category, filename and line number, and optionally the module name and the registry (which should be the __warningregistry__ dictionary of the module). The module name defaults to the filename with .py stripped; if no registry is passed, the warning is never suppressed. message must be a string and category a subclass of Warning or message may be a Warning instance, in which case category will be ignored. module_globals, if supplied, should be the global namespace in use by the code for which the warning is issued. (This argument is used to support displaying source for modules found in zipfiles or other non-filesystem import sources). source, if supplied, is the destroyed object which emitted a ResourceWarning.  Changed in version 3.6: Add the source parameter.
exception wave.Error  
An error raised when something is impossible because it violates the WAV specification or hits an implementation deficiency.
wave.open(file, mode=None)  
If file is a string, open the file by that name, otherwise treat it as a file-like object. mode can be:  
'rb' 

Read only mode.  
'wb' 

Write only mode.   Note that it does not allow read/write WAV files. A mode of 'rb' returns a Wave_read object, while a mode of 'wb' returns a Wave_write object. If mode is omitted and a file-like object is passed as file, file.mode is used as the default value for mode. If you pass in a file-like object, the wave object will not close it when its close() method is called; it is the caller’s responsibility to close the file object. The open() function may be used in a with statement. When the with block completes, the Wave_read.close() or Wave_write.close() method is called.  Changed in version 3.4: Added support for unseekable files.
Wave_read.close()  
Close the stream if it was opened by wave, and make the instance unusable. This is called automatically on object collection.
Wave_read.getcompname()  
Human-readable version of getcomptype(). Usually 'not compressed' parallels 'NONE'.
Wave_read.getcomptype()  
Returns compression type ('NONE' is the only supported type).
Wave_read.getnchannels()  
Returns number of audio channels (1 for mono, 2 for stereo).
Wave_read.getparams()  
Returns a namedtuple() (nchannels, sampwidth,
framerate, nframes, comptype, compname), equivalent to output of the get*() methods.
Wave_read.readframes(n)  
Reads and returns at most n frames of audio, as a bytes object.