Module Expr Str ImportFrom alias ImportFrom alias TryExcept body ImportFrom alias handlers ExceptHandler type NameLoad body ImportFrom alias ImportFrom alias FunctionDef arguments args NameParam defaults body Expr Str Return Call AttributeLoad Str attr GeneratorExp NameLoad comprehension NameStore NameLoad CompareLt Call NameLoad NameLoad Num decorator_list ClassDef bases NameLoad body Expr Str FunctionDef arguments args NameParam NameParam defaults body Expr Str Expr Call AttributeLoad Call NameLoad NameLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam defaults body Expr Str TryExcept body If CompareEq NameLoad Str body Return AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad NameLoad attr NameLoad If CompareEq NameLoad Str body Assign NameStore SubscriptLoad Call NameLoad AttributeLoad NameLoad attr Index Num Assign NameStore Call AttributeLoad Str attr ListComp Call AttributeLoad Call NameLoad NameLoad attr comprehension NameStore Call AttributeLoad NameLoad attr Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr Str Str attr orelse Assign NameStore Call AttributeLoad SubscriptLoad Call NameLoad AttributeLoad NameLoad attr Index Num attr NameLoad If CompareIn NameLoad ListLoad Str Str body Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Return NameLoad handlers ExceptHandler type NameLoad body If CompareIsNot NameLoad Str body Return NameLoad Raise Call NameLoad BinOpAdd Str NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr For NameStore NameLoad body Assign NameStore Call NameLoad AttributeLoad NameLoad attr Call AttributeLoad NameLoad attr Str Expr Yield Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults Dict Str ListLoad Str Str Str Str Str Num body Expr Str Assign NameStore ListLoad If CompareIn Call NameLoad NameLoad ListLoad NameLoad NameLoad body TryExcept body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr Assign NameStore ListComp BinOpAdd BinOpAdd NameLoad AttributeLoad NameLoad attr NameLoad comprehension NameStore NameLoad If CompareEq Call NameLoad NameLoad Num body Raise Call NameLoad BinOpAdd Str NameLoad handlers ExceptHandler type NameLoad body Raise Call NameLoad BinOpAdd Str NameLoad orelse Assign NameStore ListComp BinOpAdd BinOpAdd NameLoad NameLoad NameLoad comprehension NameStore NameLoad If CompareNotIn NameLoad ListLoad Str Str body Raise Call NameLoad Str If CompareEq NameLoad Str body TryExcept body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign NameStore Call NameLoad AttributeLoad NameLoad attr If CompareGtE Call NameLoad NameLoad Call NameLoad NameLoad body ImportFrom alias Assign NameStore Call NameLoad NameLoad NameLoad keyword NameLoad orelse ImportFrom alias Assign NameStore Call NameLoad NameLoad NameLoad For TupleStore NameStore NameStore NameLoad body If CompareGt NameLoad Num body Expr Call NameLoad Str NameLoad Str keyword Str keyword Str If CompareEq NameLoad Str body TryExcept body Assign NameStore Call AttributeLoad Str attr ListComp Call AttributeLoad Call NameLoad NameLoad attr comprehension NameStore Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Assign NameStore NameLoad Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr Str Str attr orelse Assign NameStore Call AttributeLoad NameLoad attr NameLoad If CompareIn NameLoad ListLoad Str Str body Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Raise Call NameLoad BinOpAdd Str NameLoad orelse TryExcept body If CompareIn Call NameLoad NameLoad ListLoad NameLoad NameLoad body Assign NameStore ListLoad NameLoad orelse If CompareEq Call NameLoad NameLoad NameLoad body Assign NameStore NameLoad orelse Raise Call NameLoad Str ImportFrom alias alias Assign NameStore Call NameLoad NameLoad NameLoad Assign NameStore Dict For TupleStore NameStore NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign SubscriptStore NameLoad Index NameLoad Call NameLoad AttributeLoad NameLoad attr TryExcept body For NameStore Call NameLoad keyword Num body Assign NameStore Call AttributeLoad NameLoad attr For NameStore Call AttributeLoad NameLoad attr body If CompareGt NameLoad Num body Expr Call NameLoad Str NameLoad Str keyword Str keyword Str Assign NameStore SubscriptLoad SubscriptLoad NameLoad Index NameLoad Index NameLoad If CompareEq NameLoad Str body TryExcept body Assign NameStore Call AttributeLoad Str attr ListComp Call AttributeLoad Call NameLoad NameLoad attr comprehension NameStore Call AttributeLoad NameLoad attr handlers ExceptHandler type NameLoad body Assign NameStore NameLoad Assign NameStore Call AttributeLoad Call AttributeLoad NameLoad attr Str Str attr orelse Assign NameStore Call AttributeLoad NameLoad attr NameLoad If CompareIn NameLoad ListLoad Str Str body Assign NameStore Call NameLoad AttributeLoad NameLoad attr NameLoad Assign SubscriptStore NameLoad Index NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad handlers ExceptHandler type NameLoad body Pass handlers ExceptHandler type NameLoad body Raise Call NameLoad BinOpAdd Str NameLoad Return TupleLoad NameLoad NameLoad decorator_list decorator_list
N/A N/A 
scrapple.selectors.css
~~~~~~~~~~~~~~~~~~~~~~

 __future__ print_function lxml cssselect N/A N/A urlparse urljoin N/A N/A N/A ImportError N/A urllib.parse urljoin scrapple.selectors.selector Selector make_ascii N/A N/A s N/A N/A N/A 
    Convert text to ASCII
     N/A N/A N/A join N/A i N/A i s N/A N/A ord i 128 N/A CssSelector N/A Selector N/A N/A 
	The ``CssSelector`` object defines CSS selector expressions.

	 __init__ N/A N/A self url N/A N/A N/A 
		The ``Selector`` class acts as the super class for this class.

		 N/A N/A N/A N/A super CssSelector self __init__ url N/A extract_content N/A N/A self selector attr default N/A N/A N/A 
		Method for performing the content extraction for the given CSS selector.

		The cssselect library is used to handle CSS selector expressions. 		XPath expressions have a higher speed of execution, so the given CSS selector 		expression is translated into the corresponding XPath expression, by the 		``cssselect.CSSSelector`` class. This selector can be used to extract content 		from the element tree corresponding to the fetched web page.

		If the selector is "url", the URL of the current web page is returned.
		Otherwise, the selector expression is used to extract content. The particular 		attribute to be extracted ("text", "href", etc.) is specified in the method 		arguments, and this is used to extract the required content. If the content 		extracted is a link (from an attr value of "href" or "src"), the URL is parsed 		to convert the relative path into an absolute path.

		If the selector does not fetch any content, the default value is returned. 		If no default value is specified, an exception is raised.

		:param selector: The CSS selector expression
		:param attr: The attribute to be extracted from the selected tag
		:param default: The default value to be used if the selector does not return any data
		:return: The extracted content

		 N/A N/A N/A N/A selector url N/A N/A N/A self url N/A sel N/A N/A cssselect CSSSelector selector N/A N/A attr text N/A N/A tag N/A N/A sel N/A self tree N/A 0 N/A content N/A N/A join N/A N/A N/A N/A make_ascii x strip N/A x N/A N/A tag itertext N/A content N/A N/A N/A N/A content replace strip N/A N/A content N/A N/A N/A N/A sel N/A self tree N/A 0 get attr N/A N/A attr N/A href src N/A N/A content N/A urljoin N/A self url content N/A content N/A N/A N/A IndexError N/A N/A N/A default N/A N/A default N/A N/A Exception N/A There is no content for the selector  selector N/A extract_links N/A N/A self selector N/A N/A N/A 
		Method for performing the link extraction for the crawler implementation.

		As in the extract_content method, the cssselect library is used to translate 		the CSS selector expression into an XPath expression. 

		The selector passed as the argument is a selector to point to the anchor tags 		that the crawler should pass through. A list of links is obtained, and the links 		are iterated through. The relative paths are converted into absolute paths and 		a ``CssSelector`` object is created with the URL of the next page as the argument 		and this created object is yielded. 

		The extract_links method basically generates ``CssSelector`` objects for all of 		the links to be crawled through.

		:param selector: The selector for the anchor tags to be crawled through
		:return: A ``CssSelector`` object for every page to be crawled through 
		
		 N/A sel N/A N/A cssselect CSSSelector selector N/A links N/A sel N/A self tree N/A link links N/A N/A next_url N/A urljoin N/A self url N/A N/A link get href N/A N/A N/A CssSelector next_url N/A extract_tabular N/A N/A self result table_type header prefix suffix selector attr default verbosity N/A N/A rows N/A text 0 N/A N/A 
		Method for performing the extraction of tabular data.

		As in the extract_content method, the cssselect library is used to translate 		the CSS selector expression into an XPath expression. 

		:param result:
		:param table_type:
		:param header:
		:param prefix:
		:param suffix:
		:param selector:
		:param attr:
		:param default:
		:param verbosity:
		:return: A 2-tuple containing the list of all the column headers extracted and the list of 		dictionaries which contain (header, content) pairs
		 N/A result_list N/A N/A N/A N/A type header N/A str unicode N/A N/A N/A N/A sel N/A N/A cssselect CSSSelector header N/A header_list N/A sel N/A self tree N/A table_headers N/A N/A N/A prefix N/A h text suffix N/A h header_list N/A N/A N/A len table_headers 0 N/A N/A N/A Exception N/A Invalid CSS selector  header N/A N/A N/A TypeError N/A N/A N/A Exception N/A Selector expression string to be provided. Got  selector N/A N/A table_headers N/A N/A N/A prefix h suffix N/A h header N/A N/A table_type N/A rows columns N/A N/A N/A Exception Specify 'rows' or 'columns' in table_type N/A N/A table_type rows N/A N/A N/A N/A sel N/A N/A cssselect CSSSelector selector N/A values N/A sel N/A self tree N/A N/A N/A len table_headers N/A len values N/A itertools izip_longest N/A pairs N/A izip_longest table_headers values fillvalue default N/A itertools izip N/A pairs N/A izip table_headers values N/A N/A head val pairs N/A N/A N/A verbosity 1 N/A N/A N/A print 
Extracting head attribute sep end N/A N/A attr text N/A N/A N/A N/A content N/A N/A join N/A N/A N/A N/A make_ascii x strip N/A x N/A N/A val itertext N/A N/A N/A Exception N/A N/A content default N/A content N/A N/A N/A N/A content replace strip N/A N/A content N/A N/A val get attr N/A N/A attr N/A href src N/A N/A content N/A urljoin N/A self url content N/A N/A result N/A head content N/A N/A N/A result_list append result N/A N/A N/A TypeError N/A N/A N/A Exception N/A Selector expression string to be provided. Got  selector N/A N/A N/A N/A N/A N/A type selector N/A str unicode N/A N/A selectors N/A selector N/A N/A N/A N/A type selector list N/A N/A selectors selector N/A N/A N/A Exception Use a list of selector expressions for the various columns itertools izip count N/A pairs N/A izip table_headers selectors N/A columns N/A N/A N/A head selector pairs N/A N/A sel N/A N/A cssselect CSSSelector selector N/A N/A columns N/A head N/A sel N/A self tree N/A N/A N/A i N/A count start 0 N/A N/A r N/A N/A result copy N/A head N/A N/A columns keys N/A N/A N/A verbosity 1 N/A N/A N/A print 
Extracting head attribute sep end N/A col N/A N/A columns N/A head N/A i N/A N/A attr text N/A N/A N/A N/A content N/A N/A join N/A N/A N/A N/A make_ascii x strip N/A x N/A N/A col itertext N/A N/A N/A Exception N/A N/A content default N/A content N/A N/A N/A N/A content replace strip N/A N/A content N/A N/A col get attr N/A N/A attr N/A href src N/A N/A content N/A urljoin N/A self url content N/A N/A r N/A head content N/A N/A N/A result_list append r N/A N/A N/A IndexError N/A N/A N/A N/A N/A TypeError N/A N/A N/A Exception N/A Selector expression string to be provided. Got  selector N/A N/A table_headers result_list N/A N/A