Package: simpleautogui
Introduction tosimpleautoguiRead the docssimpleautoguiis a Python library designed to automate
interaction with graphical user interfaces on Windows operating
systems through scripting. Leveraging the power of several
underlying libraries, it provides a set of tools to simulate
human interaction with the computer in a straightforward and
efficient manner.Modules and FunctionalityThe library is organized into several modules, each providing
distinct functionalities:Base: Defines the fundamental classes likePointandRegionfor screen regions and coordinates.Screen: Contains utilities for image and color recognition,
waiting for elements to appear.Win: Provides functions to manipulate window geometry, search
for windows by title, and arrange windows in a grid
pattern.The functionwaitImagewaits for a specific image to
appear on the screen, while waitImages handles multiple images.waitColorandwaitColorsperform similar operations for color
detection. The Region class allows for capturing
screenshots, finding text, and clicking within specific screen regions.
The win module'sset_window_geometryenables
precise control over window size and position, essential for creating a
consistent automated environment.This library is under the MIT License, ensuring open-source
accessibility and the freedom to integrate it into various
projects. The combination of its robust feature set and the ease
of use makes simpleautogui a valuable tool for
automating GUI interactions on Windows.Dependenciessimpleautoguirelies on the following dependencies:Python 3.11 or higherPyAutoGUIOpenCV-PythonPyWin32PyTesseractMouseWebcolorsKeyboardQuick StartTo get started withsimpleautogui, install the package using pip:pipinstallsimpleautoguiClassessimpleautoguiprovides two main classes:PointandRegion,
which facilitate screen interaction.PointThePointclass represents a specific location on the screen.
It provides methods to perform actions like clicking or moving
the mouse to its coordinates.fromsimpleautoguiimportPoint# Create a points with x and y coordinatesp=Point(100,100)p2=Point(500,500)# Click at first pointp.click()# Move the mouse to first pointp.moveTo()# Drag and drop 'p' to 'p2'.p.dragDropTo(p2.x,p2.y)# Drag and drop by offsetp.dragDropRel(400,400)Let me remind you that these are wrapper functions aroundpyautoguifunctions.
Read more about arguments in thepyautoguidocumentation.fromsimpleautoguiimportPointasPP(100,100).click(oX=0,# Y offsetoY=0,# X offsetclicks=2,# double clickinterval=0.0,# interval between clicks in secondsbutton='right',# 'left', 'middle'logScreenshot=True,# screenshot logging)P(100,100).moveTo(oX=0,oY=0,# **move_kwargsduration=0.0,logScreenshot=False,)# dragDropTo, dragDropRel sameRegionTheRegionclass represents a rectangular area on the screen.
It allows you to perform operations within this specified area,
like taking screenshots or searching for text.fromsimpleautoguiimportRegion# Define a region starting at x=100, y=200 with width=300 and height=400.r=Region(100,200,300,400)# Click to the center at this region.r.click()# r.click(center=False) u can.# Move the mouse to region center.r.moveTo()# center=False u can.# Show this region (save screenshot like .png in local/temp and open it).r.show()# Recognizes and returns the text in the specified area of the screen.# We find the Tesseract-OCR path in the path variable to tesseract.exe.# Example with default arguments:r.text(lang:str='eng+rus',# Language(s) for OCR, separated by a plus sign (e.g., 'eng+rus').contrast=0,# Level of contrast enhancement to apply to the image. 0 means no enhancement.resize=0,# Scaling factor to apply to the image. 0 means no scaling.sharpen=True,# Whether to apply a sharpening filter to the image.**image_to_string_kwargs# Additional keyword arguments for pytesseract.image_to_string.)# Use something like this:if'some text'inr.text():pass# Search for text throughout the entire screen.# Return the list of Region objects containing the text.result:list[Region]=Region().findText('example')# With args...result:list[Region]=Region().findText(text='example',lang='eng+rus',contrast=0,resize=0,sharpen=True,case_sensitive=False,**image_to_string_kwargs)Both classes streamline the process of screen automation by providing a set of intuitive methods to interact with the GUI elements.ModulesScreenwaitImageWaits for a specified image or images to appear on the screen within a timeout.fromsimpleautoguiimportscreen# Waits for the image to appear and clicks on its center.waitImage('image.png').click()# Waits for one of several images to appear and clicks...waitImage(('image.png','image2.png')).click()The same function but with all the arguments.
Subsequent functions will be shown more briefly, please read the detailed docs for more details.# Same thing, but more detailsresult:Region|None=waitImage(paths='image.png',# Path or tuple of paths to the images to be searched.timeout=10000,# Time in milliseconds to wait for the images.confidence=0.9,# The confidence with which to match the images.error_dialog=False,# If True, shows an error dialog if the images are not found.# Displays an error dialog and asks the user whether to continue exec code or stop.region=(0,0,size().width,size().height),# The region of the screen to search in.check_interval=100,# Interval in milliseconds between checks.)result.click()waitImagesWaits for multiple images to appear on the screen within a specified timeout.fromsimpleautoguiimportscreenimages=waitImages('image.png')forimginimages:img.click()# with argsimages:Region|[]=waitImages(paths='image.png',# Path or tuple of paths to the images to be searched.timeout=10000,# Time in milliseconds to wait for the images.confidence=0.9,# The confidence with which to match the images.error_dialog=False,# If True, shows an error dialog if the images are not found.region=(0,0,size().width,size().height),# The region of the screen to search in.check_interval=100,# Interval in milliseconds between checks.proximity_threshold_px=2,# Pixel distance to consider images as distinct.min_matches=1# Minimum number of matches.)waitColor and waitColorsWaits for a specified color or colors to appear on the screen within a timeout.frompyautoguiimportsizefromsimpleautoguiimportPointfromsimpleautogui.screenimportwaitColor,waitColorspoint:Point=waitColor('red')ifpoint:print(f"Found red color at{point.x},{point.y}")else:print("Red color not found")point:Point=waitColor(color=('#00ff00','rgb(255, 0, 0)'),timeout=10000,confidence=0.9,error_dialog=False,region=(0,0,size().width,size().height),check_interval=100)ifpoint:point.click()else:print("Green color not found")points:list[Point]=waitColors(color=('rgb(255, 0, 0)','#00ff00','blue'),timeout=10000,confidence=0.9,error_dialog=False,region=(0,0,size().width,size().height),check_interval=100,proximity_threshold_px=2,min_matches=0# if 0 return all founded else count first founded)ifpoints:forpointinpoints:point.click()else:print("Specified colors not found")