Package: pybas-automation
py-bas-automationNote: This library is currently in active development, and its API may undergo changes without notice at any time.Note:This project originally started as aworking proof of conceptand does not aim to offer extensive support or
documentation. It serves as a fundamental demonstration of the concept and should be considered a foundation for further
development or experimentation.DescriptionThis library enables you to work with BAS (BrowserAutomationStudio) using headless Chromium browsers and a
customizable Windows GUI program, while controlling it with Python ❤️.Key FeaturesBrowserAutomationStudio Integration:Run BAS seamlessly with headless browsers while enjoying the convenience of a
user-friendly and customizable Windows GUI program
throughBrowserAutomationStudio.Unique Fingerprint Feature:The application includes a unique feature that assigns afingerprintto each
browser instance usingFingerprintSwitcher. Please be aware that this ispaidservice.Proxy Support:The application supports proxy providers such asBrightdata. Please
note that this is apaidservice.Executing Browser Automation Studio (BAS) Actions from Python: Implement BAS actions from Python using the
un-documented API. This includes actions such as retrieving page source,emulating mouse movements, etc. (Note: Not
all functions are currently supported).Playwright Control:The application leveragesPlaywrightto efficiently manage
and control BAS.Understanding the WorkflowThe functioning of BAS (Browser Automation Studio) involves the following steps:Initial Execution:Upon initiation, BAS runscmd_initial.py. This script is responsible for
creating tasks and storing them on the disk for later use.Data Acquisition and Browser Configuration:BAS then retrieves the necessary data, configures, and launches
browser instances based on the tasks provided earlier.Task Execution:Following the browser setup, BAS executescmd_worker.pyusing thetask_idandremote-debugging-portnumber as command-line parameters.Task Handling:cmd_worker.pyobtains both thews_endpointandremote-debugging-portfrom the command line. It then manages complex tasks usingPlaywright. These tasks
can range from opening a webpage to filling out forms or even taking screenshots.Task Completion:Once the tasks have been completed, BAS terminates the browser instances and exits.The result of runningcmd_worker.pyis as follows:{"tasks_file":"C:\\Users\\Administrator\\AppData\\Local\\PyBASProfilesTasks\\tasks.json"}This is an example of the createdtasks_file:[{"task_id":"9683607e-2458-4adb-9b14-7e99123bf34d","browser_settings":{"components":{"widevine":"enable","safe_browsing":"enable","components":"enable"},"network":{"enable_qiuc_protocol":true},"rendering":{"maximum_fps":30},"browser_version":"default","command_line":["--disk-cache-size=104857600","--disable-gpu-program-cache","--disable-gpu-shader-disk-cache","--disable-features=GpuProcessHighPriorityWin,GpuUseDisplayThreadPriority","--lang=en"],"profile":{"profile_folder_path":"C:\\Users\\Administrator\\AppData\\Local\\PyBASProfiles\\tmp3az8nj96","always_load_fingerprint_from_profile_folder":false,"always_load_proxy_from_profile_folder":false},"proxy":{"server":"brd.superproxy.io","port":"22225","type":"http","login":"brd-customer-hl___redacted__","password":"__redacted__"},"fingerprint":{"safe_canvas":true,"use_perfect_canvas":true,"safe_webgl":true,"safe_audio":true,"safe_battery":true,"use_font_pack":true,"safe_element_size":false,"emulate_sensor_api":true,"emulate_device_scale_factor":true}}}]This file contains task details such as browser settings, network configurations, rendering options, and fingerprint
settings, among other things.System RequirementsFor the optimal running of this application, the following system requirements are necessary:Operating System:The application is compatible with Windows 10/11 and Windows Server 2022 (tested on 21H2).Python:Ensure you have Python 3.11 or higher installed. If not, you can download it from the official
Pythonwebsite.Poetry:This is a necessary tool for managing Python dependencies. You can find the installation guide on the
official Poetrydocumentation.Git:The application requires Git for version control. If it's not already installed on your system, you can
download it from the official Gitwebsite.Make:This is an optional tool, it can be downloaded from the
Chocolateywebsite.FingerprintSwitcher License:Please note that this is apaidfeature. You will need a valid license
forFingerprintSwitcherto access its functionalities.Installation GuideInstalling the Current Development VersionTo work with the most recent development version ofpybas-automation, follow the steps outlined below:Clone the Repository:Clone thepy-bas-automationrepository from GitHub.Navigate to the Directory:Once cloned, navigate to thepy-bas-automationdirectory on your local system.Install Dependencies:With Poetry, install all the necessary dependencies.Here are the corresponding commands for each step:gitclonegit@github.com:sergerdn/py-bas-automation.gitcdpy-bas-automation
gitlfspull
poetryinstallInstalling the Latest Release from pypi.org (Currently not recommended)If you wish to incorporatepybas-automationinto your project, execute the following command:poetryaddpybas-automationPlease note that this is not currently recommended as the latest release may have unresolved issues.How to Run the ApplicationDownload the BAS Program:Begin by downloading the latest version of the compiled BAS program,
calledPyBasFree.zip. You can find this file in the project directory
underPyBasFree.zip. After downloading, extract the contents and
executePyBasFree.exe.Set Variables in the BAS GUI:After running the BAS program, proceed to set the necessary variables within the
BAS graphical user interface (GUI).Set Up Proxy Provider:If you are using a proxy provider, you will need to configure it within the BAS GUI. This
can be accomplished by navigating to theProxy Settingsoption in the vertical menu and selecting the appropriate
provider.Start the Program:Once all variables have been set, click the "OK" button to initiate the program.Advanced UsageThis guide introduces a Python script that integrates theBrowser Automation Studio(BAS) withpy-bas-automation.
The
purpose is to handle the creation of browser profiles through FingerprintSwitcher and manage tasks related to these
profiles.Initial script: cmd_initial.pyDescription:This script facilitates the integration betweenBAS (Browser Automation Studio)andpy-bas-automation. It manages
the creation of browser profiles usingFingerprintSwitcherand generates tasks associated with these profiles.Overview:Initialization: Import essential modules and configure logging.Browser Profiles: UtilizeFingerprintSwitcher's fingerprint key to generate or manage browser profiles.Proxy Support: Configure proxy settings for each browser profile in full-automatic mode by handling proxy
providers. Note: at the moment onlyBrightdatais supported.Tasks Generation: Generate an associated task for each browser profile and store it."""This script facilitates the integration between BAS (Browser Automation Studio) and `py-bas-automation`.It handles the creation of browser profiles using FingerprintSwitcher and manages tasks associated with these profiles."""importjsonimportloggingimportosimportclickfrompydanticimportFilePathfrompybas_automation.browser_profileimportBrowserProfileStoragefrompybas_automation.taskimportBasTask,TaskStorage,TaskStorageModeEnumlogger=logging.getLogger("[cmd_worker]")defrun(fingerprint_key:str,count_profiles:int)->FilePath:"""Initialize and run the script.:param fingerprint_key: Personal fingerprint key from FingerprintSwitcher.:param count_profiles: Number of profiles to be created.:return: Path to the generated tasks file."""# Initialize task storage with read-write access and clear it.# The default storage location is C:\Users\{username}\AppData\Local\PyBASTaskstask_storage=TaskStorage(mode=TaskStorageModeEnum.READ_WRITE)task_storage.clear()# Initialize browser profiles using the given fingerprint key.# The default profile storage location is C:\Users\{username}\AppData\Local\PyBASProfilesbrowser_profile_storage=BrowserProfileStorage(fingerprint_key=fingerprint_key)needs=count_profiles-browser_profile_storage.count()# Create any additional profiles if necessaryifneeds>0:for_inrange(needs):browser_profile=browser_profile_storage.new()logger.debug("Created new profile:%s",browser_profile.profile_dir)# Generate tasks corresponding to each profileforbrowser_profileinbrowser_profile_storage.load_all()[:count_profiles]:task=BasTask()task.browser_settings.profile.profile_folder_path=browser_profile.profile_dirtask_storage.save(task=task)logger.info("Total tasks generated:%d",task_storage.count())task_storage.save_all()returntask_storage.task_file_path@click.command()@click.option("--bas_fingerprint_key",help="Your personal fingerprint key of FingerprintSwitcher.",required=True,)@click.option("--count_profiles",help="Number of profiles.",default=10,)defmain(bas_fingerprint_key:str,count_profiles:int)->None:"""Entry point of the script. Sets up logging, validates the fingerprint key,triggers the primary function, and prints the path to the tasks file.:param bas_fingerprint_key: Personal fingerprint key from FingerprintSwitcher.:param count_profiles: Number of profiles to be created.:return: None."""importmultiprocessingprocess=multiprocessing.current_process()# Configure logging settingslogging.basicConfig(level=logging.DEBUG,format=f"%(asctime)s{process.pid}%(levelname)s %(name)s %(message)s",filename=os.path.join(os.path.dirname(__file__),"logs","cmd_initial.log"),)logger.info("Script cmd_initial has started.")# Ensure the fingerprint key is presentbas_fingerprint_key=bas_fingerprint_key.strip()ifnotbas_fingerprint_key:raiseValueError("bas_fingerprint_key is not provided")# Invoke the main function to get the path to the tasks filetask_file_path=run(fingerprint_key=bas_fingerprint_key,count_profiles=count_profiles)# Print the path for potential use in BASprint(json.dumps({"tasks_file":str(task_file_path)},indent=4))logger.info("cmd_initial script execution completed.")if__name__=="__main__":main()Worker script: cmd_worker.pyDescription:This script demonstrates how to execute tasks using thePlaywrightPython library in conjunction with thepybas_automationpackage. The primary goal is to fetch task data, connect to an existing browser instance usingPlaywright, and perform actions on a webpage.Overview:Initialization: Import necessary libraries and set up our task id and debugging port.Task Storage: Fetch a specific task from our task storage.Remote Browser Connection: Use the remote debugging port to get a WebSocket endpoint, which allows us to connect
to an existing browser instance.Executing Browser Automation Studio (BAS) Actions from Python: Implement BAS actions from Python using the
un-documented API. This includes actions such as retrieving page source, emulating mouse movements, etc. (Note: Not
all functions are currently supported).importasynciofromuuidimportUUIDfromplaywright.async_apiimportasync_playwrightfrompybas_automation.taskimportTaskStorage,TaskStorageModeEnumfrompybas_automation.browser_automatorimportBrowserAutomatorasyncdefmain():# 1. Initialization# For demonstration purposes, we're using hardcoded values. In a real scenario, these will be fetched dynamically.task_id=UUID("some_task_id_that_we_getting_from_cmd_line_from_BAS")remote_debugging_port=9222# A unique identifier for the `Worker.exe` process. Retrieved from the command line  argument `--unique-process-id`.unique_process_id="some_unique_process_id"# 2. Task Storage# Create a new task storage instance in READ mode to fetch tasks.task_storage=TaskStorage(mode=TaskStorageModeEnum.READ)found_task=task_storage.get(task_id=task_id)# Note: You can manipulate or inspect the `found_task` as needed.# 3. Remote Browser ConnectionasyncwithBrowserAutomator(remote_debugging_port=remote_debugging_port,unique_process_id=unique_process_id)asautomator:# Variant 1: Work with the BrowserAutomator APIawaitautomator.page.goto("https://playwright.dev/python/")ifunique_process_id:# With Automator, you can call function from the BrowserAutomationStudio API.print("Unique process ID:%s",unique_process_id)page_content=awaitautomator.bas_get_page_content()elem=automator.page.locator("xpath=//a[@class='getStarted_Sjon']")awaitautomator.bas_move_mouse_to_elem(elem=elem)awaitelem.click()print("Page content from BAS_SAFE api:%s...",page_content[:100])# Variant 1: Work with the Playwright API directly.ws_endpoint=automator.get_ws_endpoint()asyncwithasync_playwright()aspw:# Connect to an existing browser instance using the fetched WebSocket endpoint.browser=awaitpw.chromium.connect_over_cdp(ws_endpoint)# Access the main page of the connected browser instance.page=browser.contexts[0].pages[0]# Perform actions using Playwright, like navigating to a webpage.awaitpage.goto("https://playwright.dev/python/")if__name__=="__main__":asyncio.run(main())Planned Improvements:Add Proxy support.Develop end-to-end tests to thoroughly assess the entire workflow.Include build scripts for converting Python files to executable format.Expand the repository with more illustrative examples.ContributingYour ideas and contributions are highly valued. Please do not hesitate to open
anissueif you have suggestions, questions, or if you would
like to contribute to its enhancement.LicenseLICENSE