Package: autopy
For more information, see theGitHub Repository.AutoPy Introduction and TutorialIntroductionAutoPy is a simple, cross-platform GUI automation library for Python. It
includes functions for controlling the keyboard and mouse, finding colors and
bitmaps on-screen, and displaying alerts.Currently supported on macOS, Windows, and X11 with the XTest extension.Getting StartedRequirementsPython 2.7, or Python 3.5 and up.Rust 1.23.0-nightly 2019-02-06 or later (unless using a binary wheel
distribution).macOS 10.6 and up.Windows 7 and up.X11 with the XTest extension.InstallationFirst, see if a binary wheel is available for your machine by running:$ pip install -U autopyIf that fails, installrustupand then run:$ rustup default nightly-2019-10-05
$ pip install -U setuptools-rust
$ pip install -U autopyAnother option is to build from the latest source on the GitHub repository:$ git clone git://github.com/autopilot-rs/autopy-rs.git
$ cd autopy
$ make
$ make installNote: AutoPy currently requires the2019-10-05Rust nightly in order to
build from source. This is to maintain compatibility with an older version of
PyO3, as the latest version has dropped Python 2 support. Python 2 support will
likely be dropped from AutoPy as well sometime later this year, depending on
how necessary it is to upgrade to a more recent version of PyO3 or Rust. In the
meantime, it may be necessary to install the required nightly via the following
when building locally:rustup install nightly 2019-10-05 --forceThis is due to rustup complaining that it doesn't include certain components
such asrustfmt.Additional instructions for installing from source on Windows are availablehere.Hello WorldThe following is the source for a "hello world" script in autopy. Running this
code will cause an alert dialog to appear on every major platform:importautopydefhello_world():autopy.alert.alert("Hello, world")hello_world()TutorialsControlling the MouseAutoPy includes a number of functions for controlling the mouse. For a full
list, consult theAPI
Reference. E.g.,
to immediately "teleport" the mouse to the top left corner of the screen:>>> import autopy
>>> autopy.mouse.move(1, 1)To move the mouse a bit more realistically, we could use:>>> import autopy
>>> autopy.mouse.smooth_move(1, 1)Even better, we could write our own function to move the mouse across the screen
as a sine wave:importautopyimportmathimporttimeimportrandomimportsysTWO_PI=math.pi*2.0defsine_mouse_wave():"""Moves the mouse in a sine wave from the left edge ofthe screen to the right."""width,height=autopy.screen.size()height/=2height-=10# Stay in the screen bounds.forxinrange(int(width)):y=int(height*math.sin((TWO_PI*x)/width)+height)autopy.mouse.move(x,y)time.sleep(random.uniform(0.001,0.003))sine_mouse_wave()Controlling the KeyboardThe following will enter the keys from the string "Hello, world!" in the
currently focused input at 100 WPM:importautopyautopy.key.type_string("Hello, world!",wpm=100)Alternatively, individual keys can be entered using the following:importautopyautopy.key.tap(autopy.key.Code.TAB,[autopy.key.Modifier.META])autopy.key.tap("w",[autopy.key.Modifier.META])Working with BitmapsAll of autopy's bitmap routines can be found in the moduleautopy.bitmap. A
useful way to explore autopy is to use Python's built-inhelp()function, for
example inhelp(autopy.bitmap.Bitmap). AutoPy's functions are documented with
descriptive docstrings, so this should show a nice overview.>>> import autopy
>>> autopy.bitmap.capture_screen()
<Bitmap object at 0x12278>This takes a screenshot of the main screen, copies it to a bitmap, displays its
memory address, and then immediately destroys it. Let's do something more
useful, like look at its pixel data:>>> import autopy
>>> autopy.bitmap.capture_screen().get_color(1, 1)
15921906AutoPy uses a coordinate system with its origin starting at the top-left, so
this should return the color of pixel at the top-left corner of the screen. The
number shown looks a bit unrecognizable, but we can format it with Python's
built-inhexfunction:>>> import autopy
>>> hex(autopy.bitmap.capture_screen().get_color(1, 1))
'0xF2F2F2'Alternatively, we can use:>>> import autopy
>>> autopy.color.hex_to_rgb(autopy.screen.get_color(1, 1))
(242, 242, 242)which converts that hex value to a tuple of(r, g, b)values. (Note thatautopy.screen.get_color(), used here, is merely a more convenient and
efficient version ofautopy.bitmap.capture_screen().get_color().)To save the screen capture to a file, we can use:>>> import autopy
>>> autopy.bitmap.capture_screen().save('screengrab.png')The filetype is either parsed automatically from the filename, or given as an
optional parameter. Currently only jpeg and png files are supported.>>> import autopy
>>> autopy.bitmap.Bitmap.open('needle.png')
<Bitmap object at 0x1001d5378>Aside from analyzing a bitmap's pixel data, the main use for loading a bitmap is
finding it on the screen or inside another bitmap. For example, the following
prints the coordinates of the first image found in a bitmap (scanned from left
to right, top to bottom):importautopydeffind_image_example():needle=autopy.bitmap.Bitmap.open('needle.png')haystack=autopy.bitmap.Bitmap.open('haystack.png')pos=haystack.find_bitmap(needle)ifpos:print("Found needle at:%s"%str(pos))find_image_example()It's also possible to do a bounded search by passing a tuple((x, y), (width, height)):haystack.find_bitmap(needle,rect=((10,10),(100,100)))Projects using AutoPyAutoPyDriverServer, AutoPy
through WebDriver or a webdriver-compatible server.guibot, A tool for GUI automation using
a variety of computer vision and desktop control backends.spynner, Programmatic web browsing
module with AJAX support for Python.SUMO, An open source, highly portable,
microscopic and continuous road traffic simulation package designed to handle
large road networks.API ReferenceHope you enjoy using autopy! For a more in depth overview, see theAPI
Reference.ContributingIf you are interested in this project, please consider contributing. Here are a
few ways you can help:Report issues.Fix bugs andsubmit pull requests.Write, clarify, or fix documentation.Suggest or add new features.LicenseThis project is licensed under either theApache-2.0orMITlicense, at your option.Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.