Repository: apify/crawlee-python
File: src/crawlee/_types.py (Python)

Code before:
        transform_request_function: Callable[[RequestOptions], RequestOptions | RequestTransformAction] | None = None,
        requests: Sequence[str | Request] | None = None,
        **kwargs: Unpack[EnqueueLinksKwargs],
    ) -> Coroutine[None, None, None]:
        """Call enqueue links function.

        Args:
            selector: A selector used to find the elements containing the links. The behaviour differs based
                on the crawler used:
                - `PlaywrightCrawler` supports CSS and XPath selectors.
                - `ParselCrawler` supports CSS selectors.
                - `BeautifulSoupCrawler` supports CSS selectors.
            label: Label for the newly created `Request` objects, used for request routing.
            user_data: User data to be provided to the newly created `Request` objects.
            transform_request_function: A function that takes `RequestOptions` and returns either:
                - Modified `RequestOptions` to update the request configuration,
                - `'skip'` to exclude the request from being enqueued,
                - `'unchanged'` to use the original request options without modification.
            requests: Requests to be added to the `RequestManager`.
            **kwargs: Additional keyword arguments.
        """


@docs_group('Functions')
class ExtractLinksFunction(Protocol):
    """A function for extracting URLs to crawl based on elements selected by a given selector."""

    def __call__(
        self,
        *,
        selector: str,
        label: str | None = None,
        user_data: dict[str, Any] | None = None,
        transform_request_function: Callable[[RequestOptions], RequestOptions | RequestTransformAction] | None = None,
        **kwargs: Unpack[EnqueueLinksKwargs],
    ) -> Coroutine[None, None, list[str | Request]]:
        """Call extract links function.

        Args:
            selector: A selector used to find the elements containing the links. The behaviour differs based
                on the crawler used:
                - `PlaywrightCrawler` supports CSS and XPath selectors.
                - `ParselCrawler` supports CSS selectors.
                - `BeautifulSoupCrawler` supports CSS selectors.
            label: Label for the newly created `Request` objects, used for request routing.
            user_data: User data to be provided to the newly created `Request` objects.
            transform_request_function: A function that takes `RequestOptions` and returns either:
                - Modified `RequestOptions` to update the request configuration,
                - `'skip'` to exclude the request from being enqueued,
                - `'unchanged'` to use the original request options without modification.
            **kwargs: Additional keyword arguments.

Review comment (refactor):
Could we expand the docs? similar to what we have in `enqueue_links`?

extract links:
![image](https://github.com/user-attachments/assets/65f96a88-1ca0-4b57-83b5-68b8861445d3)

vs enqueue links:
![image](https://github.com/user-attachments/assets/3e504c73-47ce-47b6-ba9e-12bb776a421f)


Code after:
        selector: str | None = None,
        label: str | None = None,
        user_data: dict[str, Any] | None = None,
        transform_request_function: Callable[[RequestOptions], RequestOptions | RequestTransformAction] | None = None,
        requests: Sequence[str | Request] | None = None,
        **kwargs: Unpack[EnqueueLinksKwargs],
    ) -> Coroutine[None, None, None]:
        """Call enqueue links function.

        Args:
            selector: A selector used to find the elements containing the links. The behaviour differs based
                on the crawler used:
                - `PlaywrightCrawler` supports CSS and XPath selectors.
                - `ParselCrawler` supports CSS selectors.
                - `BeautifulSoupCrawler` supports CSS selectors.
            label: Label for the newly created `Request` objects, used for request routing.
            user_data: User data to be provided to the newly created `Request` objects.
            transform_request_function: A function that takes `RequestOptions` and returns either:
                - Modified `RequestOptions` to update the request configuration,
                - `'skip'` to exclude the request from being enqueued,
                - `'unchanged'` to use the original request options without modification.
            requests: Requests to be added to the `RequestManager`.
            **kwargs: Additional keyword arguments.
        """


@docs_group('Functions')
class ExtractLinksFunction(Protocol):
    """A function for extracting URLs to crawl based on elements selected by a given selector.

    It extracts URLs from the current page and allows filtering through selectors and other options. You can also
    specify labels and user data to be associated with the newly created `Request` objects.
    """

    def __call__(
        self,
        *,
        selector: str = 'a',
        label: str | None = None,
        user_data: dict[str, Any] | None = None,
        transform_request_function: Callable[[RequestOptions], RequestOptions | RequestTransformAction] | None = None,
        **kwargs: Unpack[EnqueueLinksKwargs],
    ) -> Coroutine[None, None, list[Request]]:
        """Call extract links function.

        Args:
            selector: A selector used to find the elements containing the links. The behaviour differs based
                on the crawler used:
                - `PlaywrightCrawler` supports CSS and XPath selectors.
                - `ParselCrawler` supports CSS selectors.
                - `BeautifulSoupCrawler` supports CSS selectors.