cyandrocel 1.1.7
Android Automation Framework for real devices without root access
License: MIT
Keywords: Android, Automation, Cython, Pandas, Adb, Numpy


# 📱 **CyAndroCel: Advanced Android Automation Library**

<!-- PROJECT LOGO -->
<br />
<div align="center">
  <a href="https://github.com/hansalemaos/cyandrocel">
    <img src="https://github.com/hansalemaos/cyandrocel/blob/main/logo.png?raw=true" alt="Logo" width="400" height="400">
  </a>

  <h3 align="center">Python / Cython Automation for Android Emulators</h3>

  <p align="center">
    Android automations without ADB - directly on emulators
  </p>
</div>


## About The Project

**CyAndroCel** is a powerful [**Python**](https://www.python.org/) library designed for advanced Android automation. For performance reasons, it’s primarily written in [**C++**](https://cplusplus.com/) and [**Cython**](https://cython.readthedocs.io/), providing multiple backends for interacting with UI elements, handling input events, and efficiently extracting screen data.

**CyAndroCel** is designed to run on non-rooted devices and depends on [**ADB**](https://developer.android.com/tools/adb). If you need a version that runs directly and independently on an emulator or rooted device without relying on ADB, check outt [**CyAndroEmu**](https://github.com/hansalemaos/cyandroemu)

---

## 🚀 **Why Choose CyAndroCel?**


- **Low-Level Performance:**
  Thanks to its C++ core, CyAndroCel achieves unparalleled speed and efficiency, even in resource-intensive environments.

- **Multi-Backend Support:**
  Choose from various parsers and input methods tailored to different Android environments (real devices, emulators, custom ROMs) without needing root.

- **Seamless Emulator Integration:**
  Designed to work flawlessly with any Android Device.

- **Linux/Windows support**
  Run your automation scripts on both Windows and Linux.


---

## Getting Started

### Installation

#### Before installing, ensure you have a C++20 compiler installed ([MSVC on Windows](https://visualstudio.microsoft.com/de/vs/features/cplusplus/) / [GCC on Linux](https://www.gnu.org/)), run

#### `pip install cyandrocel`

The library will be compiled the first time you import it.

After compilation, four parsers will be downloaded and built using [Zig's C++ compiler](https://ziglang.org/):


[**Uiautomator2 Parser**](https://github.com/hansalemaos/uiautomator2tocsv)


[**Uiautomator Classic Parser**](https://github.com/hansalemaos/uiautomator_dump_to_csv)


[**Fragment Parser**](https://github.com/hansalemaos/android_fragment_parser)


[**Tesseract Parser**](https://github.com/hansalemaos/tesseract_hocr_to_csv)


***
***


## Writing Python scripts

CyAndroCel is primarily written in [C++](https://cplusplus.com/)  (4 out of 5 parsers) and [Cython](https://cython.readthedocs.io/) (1 parser and the Python interface), but all automation is handled using the Python class CyAndroCel.

***

### The Constructor

#### May Look Intimidating, but It’s Easy to Understand

```py
CyAndroCel:
    str adb_exe,
    str device_id,
    int subprocess_timeout = 30,
    bint add_input_tap = True,
    str input_cmd_tap = "input tap",
    str input_cmd_text = "input text",
    str input_cmd_swipe= "input swipe",
    int screen_height = 2400,
    int screen_width = 1080,
    str sh_exe = "sh",
    str su_exe = "su",
    str path_exe_tesseract = "tesseract",
    object tesseract_args = ("-l", "por+eng", "--oem", "3"),
    int tesseract_word_group_limit=20,
    bint tesseract_delete_tmp_files=True,
    str ui2_download_link1="https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test_with_hidden_elements.apk",
    str ui2_download_link2="https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator_with_hidden_elements.apk",
    str adb_keyboard_link = "https://github.com/hansalemaos/ADBKeyBoard/raw/refs/heads/master/ADBKeyboard.apk",
    object valid_adb_keyboards = ("com.android.adbkeyboard/.AdbIME", "com.github.uiautomator/.AdbKeyboard",),
    object kwargs=None,
```


### Each constructor argument explained

***

#### `adb_exe : str`

Path to the ADB executable.
On Windows systems, paths for adb_exe and path_exe_tesseract are shortened using get_short_path_name.

#### `device_id : str`

Unique identifier for the target Android device.
use `adb devices` to get the serial number

#### `subprocess_timeout : int, optional`

Timeout in seconds for subprocess calls (default 30).

#### `add_input_tap : bint, optional`

If True, enable input tap functionality (default True).

#### `input_cmd_tap : str, optional`

Command used to simulate tap events (default "input tap").

#### `input_cmd_text : str, optional`

Command used to simulate text input (default "input text").

#### `input_cmd_swipe : str, optional`

Command used to simulate text swipe (default "input swipe").

#### `screen_height : int, optional`

Expected screen height in pixels (default 2400).

#### `screen_width : int, optional`

Expected screen width in pixels (default 1080).

#### `sh_exe : str, optional`

Shell executable to use (default "sh").

#### `su_exe : str, optional`

Superuser command to use (default "su").
Some predefined shell commands require `su`, but the core functionality of the library — parsing screen elements and interacting with them — does not require `su`.

#### `path_exe_tesseract : str, optional`

Path to the Tesseract executable (default "tesseract").
On Windows systems, paths for adb_exe and path_exe_tesseract are shortened using get_short_path_name.

#### `tesseract_args : tuple or list, optional`

Arguments for Tesseract OCR (default ("-l", "por+eng", "--oem", "3")).

#### `tesseract_word_group_limit : int, optional`

Limit in pixel for grouping words in OCR output (default 20).

#### `tesseract_delete_tmp_files : bint, optional`

Whether to delete temporary OCR files after processing (default True).

#### `ui2_download_link1 : str, optional`

URL for the first UIAutomator2 APK download.

(default  "https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test_with_hidden_elements.apk")

I forked the Uiautomator2 project and modified it to include non-visible objects in the parsed data. This enhancement lets you, for example, read notifications without having to open the swipe-down menu.

#### `ui2_download_link2 : str, optional`

URL for the second UIAutomator2 APK download.

(default  "https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator_with_hidden_elements.apk")

I forked the Uiautomator2 project and modified it to include non-visible objects in the parsed data. This enhancement lets you, for example, read notifications without having to open the swipe-down menu.

#### `adb_keyboard_link : str, optional`

Backup APK from the [ADBKeyboard project](https://github.com/senzhk/ADBKeyBoard). It allows sending Unicode characters but may not work on all devices. In most cases, there’s no need to modify it.


URL for downloading the ADB keyboard APK.

(default  "https://github.com/hansalemaos/ADBKeyBoard/raw/refs/heads/master/ADBKeyboard.apk")


#### `valid_adb_keyboards : tuple|list, optional`

A tuple|list containing valid ADB keyboard identifiers.

(default  ("com.android.adbkeyboard/.AdbIME", "com.github.uiautomator/.AdbKeyboard"))


#### `kwargs : dict, optional`

Additional keyword arguments for subprocess execution.

(default None)



***
***


## Creating an instance

```py
from cyandrocel import CyAndroCel
import shutil


adb_exe = shutil.which("adb")
device_id = "YOUR DEVICE ID HERE"
cyandro = CyAndroCel(
    adb_exe=adb_exe,
    device_id=device_id,
    subprocess_timeout=30,
    add_input_tap=True,
    input_cmd_tap="input tap",
    input_cmd_text="input text",
    screen_height=2400,
    screen_width=1080,
    sh_exe="sh",
    su_exe="su",
    path_exe_tesseract="tesseract",
    tesseract_args=("-l", "por+eng", "--oem", "3"),
    tesseract_word_group_limit=20,
    tesseract_delete_tmp_files=True,
    ui2_download_link1="https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test_with_hidden_elements.apk",
    ui2_download_link2="https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator_with_hidden_elements.apk",
    adb_keyboard_link="https://github.com/hansalemaos/ADBKeyBoard/raw/refs/heads/master/ADBKeyboard.apk",
    valid_adb_keyboards=(
        "com.android.adbkeyboard/.AdbIME",
        "com.github.uiautomator/.AdbKeyboard",
    ),
    kwargs=None,
)
```
***

## Automation - it's all about pandas DataFrames

Most automation tools implement their own logic for finding elements, which often looks like this:

`self.find(text="Some text").children[0].click()`

That looks beautiful and very Pythonic, but most of the time, it’s just unreliable — mainly when the attributes and positions of objects vary, or when the objects aren’t loaded on the screen yet.

Instead of bothering you with my own "great" ideas, I decided to rely on battle-tested solutions: [pandas DataFrames](https://pandas.pydata.org/).

**With DataFrames, you can leverage powerful features like:**

* Fast sorting algorithms
* All kinds of comparisons
* Regular expressions (regex)
* Custom filter functions

If you know how to work with pandas DataFrames, you basically already know everything you need to automate whatever you want.
If not, you can learn everything from the official documentation: https://pandas.pydata.org/docs/
That’s where I got 95% of my pandas knowledge from.

***

### Pretty Print


The default `__repr__` and `__str__` functions in pandas are not ideal for displaying DataFrames, which is why they’ve been monkey-patched to ensure the entire DataFrame is pretty-printed.

```py
df = cyandro.get_df_uiautomator2(with_screenshot=True)
print(df)
```

Now, when you print the DataFrame, you’ll see the complete content neatly formatted, making it easier to analyze UI elements.
For a line-break-free experience, I recommend using the [VTM terminal](https://github.com/directvt/vtm)

***

#### Example of a pandas DataFrame in the VTM terminal

[![Video 1](https://img.youtube.com/vi/e-lqgiDWDNQ/0.jpg)](https://www.youtube.com/watch?v=e-lqgiDWDNQ)

***

#### Getting screenshots of each element

In the example above, the elements are very easy to locate. However, this is not always the case.
You might want to see a screenshot of each element to write your script faster. In this case, you can use the following command on the device:

```py
df.bb_save_screenshots_as_png("/sdcard/uiautomator2screenshots")
```

The screenshot filenames correspond to the location of the elements in the DataFrame.
For example, if you want to locate the third element (index 2), simply run:

```py
df.iloc[2]
```
This will extract the element’s information, allowing you to build your pandas query from there.


***
***

#### Python Code


```py
from cyandrocel import CyAndroCel
import shutil


adb_exe = shutil.which("adb")
device_id = "xxxx" # use adb devices -l
screen_width, screen_height = CyAndroCel.get_resolution_of_screen(
    adb_exe=adb_exe, device_id=device_id
)
print(screen_width, screen_height)

cyandro = CyAndroCel(
    adb_exe=adb_exe,
    device_id=device_id,
    subprocess_timeout=30,
    add_input_tap=True,
    input_cmd_tap="input tap",
    input_cmd_text="input text",
    input_cmd_swipe="input swipe",
    screen_height=screen_height,
    screen_width=screen_width,
    sh_exe="sh",
    su_exe="su",
    path_exe_tesseract="tesseract",
    tesseract_args=("-l", "por+eng", "--oem", "3"),
    tesseract_word_group_limit=20,
    tesseract_delete_tmp_files=True,
    ui2_download_link1="https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test_with_hidden_elements.apk",
    ui2_download_link2="https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator_with_hidden_elements.apk",
    adb_keyboard_link="https://github.com/hansalemaos/ADBKeyBoard/raw/refs/heads/master/ADBKeyboard.apk",
    valid_adb_keyboards=(
        "com.android.adbkeyboard/.AdbIME",
        "com.github.uiautomator/.AdbKeyboard",
    ),
    kwargs=None,
)

# disables the uiautomator2 server automatically if it is activated
df2 = cyandro.get_df_uiautomator_classic(with_screenshot=True)
print(df2)

# make sure that tesseract is installed
# choco install tesseract
# or https://tesseract-ocr.github.io/
df1 = cyandro.get_df_tesseract()
print(df1)
# levenshtein search afterwards
df1 = df1.d_fm_damerau_levenshtein_distance_2ways(
    ["YouTube : Musicas Gospel"], "aa_text", "aa_searchfortext"
).sort_values(by=["damerau_levenshtein_distance_2ways_match"], ascending=False)

# fragment parser
df3 = cyandro.get_df_fragments(with_screenshot=True)
print(df3)

# window parser
df4 = cyandro.get_df_window_dump(with_screenshot=True)
print(df4)


# call cyandro.download_and_install_uiautomator2_apks() to download and install the apks first
# first parse with uiautomator2 takes around 5 seconds to activate the server

df5 = cyandro.get_df_uiautomator2(with_screenshot=True)
print(df5)

# clicking on a button - same system for all parsers
# df5.loc[df5.aa_text == "Spotify"].aa_input_tap.iloc[0]()

naturaltext = cyandro.get_cmd_send_text_natural("Hallo")
shell = cyandro.open_shell()

# sends unicode, may not work with every device
# Supported letters are:
# á, é, í, ó, ú, ý, Á, É, Í, Ó, Ú, Ý, ç, Ç, â, ê, î, ô, û, Â, Ê, Î, Ô, Û, ã, ñ, õ, Ã, Ñ, Õ, ß, ẞ, ä, ë, ï, ö, ü, ÿ, Ä, Ë, Ï, Ö, Ü, Ÿ, à, è, ì, ò, ù, À, È, Ì, Ò, Ù
acaotext = cyandro.get_cmd_send_text_unicode_natural("Ação")
acaotext()

# creates a new column "black_color" with the color search results
df5.bb_search_for_colors([[0, 0, 0], [255, 255, 255]], result_column="black_color")



```
#### Check out [CyAndoEmu](https://github.com/hansalemaos/cyandroemu) for more detailed examples - the API is the same (except for the aditional root functionality that cyandroemu has)

## The Backends

***

### [Uiautomator2 Backend](https://github.com/hansalemaos/uiautomator2tocsv/tree/d5022bd0bb5864ba3b36cfb9c444f6557c2381c8)

| **Advantages ✅**                               | **Disadvantages ❌**                                      |
|------------------------------------------------|----------------------------------------------------------|
| Very fast parsing                              | Blocks other parsers like the Logcat parser and traditional UIAutomator |
| Never fails due to excessive screen movement (unlike UIAutomator) | Requires APKs to be installed                              |
| Includes both element text and description     | The server needs to run in the background                 |
| Great for web scraping and interacting with web elements | Automation might be detected when used with installed apps |
| Runs on any device                           | Useless with specific engines like the Unreal Engine                 |
| Finds also hidden items                           | -                |

***

### [Tesseract Backend](https://github.com/hansalemaos/screencap2tesseract2csv)

| **Advantages ✅**                                              | **Disadvantages ❌**                                           |
|---------------------------------------------------------------|----------------------------------------------------------------|
| Detects (in theory) all text on the screen                    | Higher CPU usage compared to other parsers                     |
| Works with Unreal Engine and Roblox games                     | May produce inaccurate results (mostly resolved by fast C++-implemented Levenshtein distance calculation) |
| Can categorize elements (image, text, line, etc.)             | Requires additional downloads                                 |
| Undetectable                                                  | Needs a screenshot to function                                |
| Can be used alongside any other parser                        | —                                                              |

***

### [Fragment Parser Backend](https://github.com/hansalemaos/android_fragment_parser)

| **Advantages ✅**                                              | **Disadvantages ❌**                                           |
|---------------------------------------------------------------|----------------------------------------------------------------|
| The fastest of all parsers                                    | Results don’t include the element’s text                       |
| Very low CPU usage                                            | Sometimes misses elements that UIAutomator-based parsers can detect |
| Can be used alongside any other parser                        | —                                                              |
| Provides useful additional information that other parsers don’t have | —                                                              |
| Can find elements that UIAutomator-based parsers can’t detect | —                                                              |
| Undetectable                                                  | —                                                              |
| Highly reliable—never fails                                  | —                                                              |


***

### [UIAutomator Classic Backend](https://github.com/hansalemaos/uiautomator_dump_to_csv)

| **Advantages ✅**                                              | **Disadvantages ❌**                                           |
|---------------------------------------------------------------|----------------------------------------------------------------|
| Includes both element text and description                    | Higher CPU usage compared to UIAutomator2                      |
| Undetectable                                                  | May fail (idle state error) if there’s too much activity on the screen |
| Can be used alongside any other parser (except UIAutomator2)  | —                                                              |

***

### Window Dump Backend

| **Advantages ✅**                                              | **Disadvantages ❌**                                           |
|---------------------------------------------------------------|----------------------------------------------------------------|
| Very fast parser                                              | Results don’t include the element’s text                       |
| Very low CPU usage                                            | Doesn’t work everywhere (e.g., the start page of the launcher) |
| Can be used alongside any other parser                        | —                                                              |
| Provides lots of additional information that other parsers don’t have | —                                                              |
| Can find elements that UIAutomator-based parsers can’t detect | —                                                              |
| Undetectable                                                  | —                                                              |

***

## Color Search and other image tools

### Color Search

#### If you decide to add a screenshot to your DataFrame, you can perform a very fast color search (uses fast C++ vectors and structs for the results internally).

```py

# Pass RGB colors and the name of the column
df.bb_search_for_colors([[255, 255, 255], [0, 0, 0]], result_column="white_and_black")

# An new column will be added
df.white_and_black

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
iloc | index   | white_and_black                                                                                                                                                                                                                                                                                           |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0    | 0       | [{'x': 140, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 20 |
1    | 1       | [{'x': 140, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 20 |
2    | 2       | [{'x': 140, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 20 |
3    | 3       | [{'x': 128, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 129, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 130, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 131, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 132, 'y': 100, 'count': 4 |
4    | 4       | [{'x': 128, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 129, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 130, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 131, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 132, 'y': 100, 'count': 665,  |
5    | 5       | [{'x': 248, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 249, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 250, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 251, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 252, 'y': 100, 'count': 1 |
6    | 6       | [{'x': 470, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 471, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 472, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 473, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 474, 'y': 100, 'count': 830,  |
7    | 7       | [{'x': 117, 'y': 412, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 118, 'y': 412, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 119, 'y': 412, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 118, 'y': 413, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 119, 'y': 413, 'count': 311,  |
8    | 8       | [{'x': 339, 'y': 412, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 340, 'y': 412, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 341, 'y': 412, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 340, 'y': 413, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 341, 'y': 413, 'count': 402,  |
9    | 9       | [{'x': 568, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 569, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 580, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 589, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 597, 'y': 497, 'count': 62, 'r':  |
10   | 10      | [{'x': 151, 'y': 729, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 152, 'y': 729, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 85, 'y': 730, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 86, 'y': 730, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 149, 'y': 730, 'count': 197, 'r |
11   | 11      | [{'x': 362, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 363, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 364, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 365, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 366, 'y': 638, 'count': 627,  |
12   | 12      | [{'x': 140, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 17 |
13   | 13      | [{'x': 136, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 77, 'count': 100, 'r':  |
14   | 14      | [{'x': 150, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 151, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 152, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 153, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 154, 'y': 65, 'count': 15 |
15   | 15      | [{'x': 568, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 569, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 570, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 571, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 572, 'y': 77, 'count': 177, 'r':  |
16   | 16      | []                                                                                                                                                                                                                                                                                                           |
17   | 17      | []                                                                                                                                                                                                                                                                                                           |
18   | 18      | []                                                                                                                                                                                                                                                                                                           |
19   | 19      | [{'x': 344, 'y': 1129, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 333, 'y': 1130, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 334, 'y': 1130, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 331, 'y': 1131, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 334, 'y': 1131, 'count': 250, 'r': 0, 'g': 0, 'b' |
20   | 20      | []                                                                                                                                                                                                                                                                                                           |
21   | 21      | [{'x': 599, 'y': 1147, 'count': 1, 'r': 255, 'g': 255, 'b': 255}]                                                                                                                                                                                                                                            |
22   | 22      | []

```

### count colors

```py
df.bb_count_colors(result_column="aa_color_count_all")

# Counts all colors in each element and adds a new column to the DataFrame

# {(36, 7, 3): 1640403, (37, 8, 4): 63475, (255, 255, 255): 44447, (38, 9, 5): 31206, (39, 10, 6): 25333, ... }

```

### count colors - percentage

```py
df.bb_count_colors_percentage(result_column="aa_color_count_all_percentage")

# Counts all colors in each element and adds a new column to the DataFrame

# {(36, 7, 3): 79.10894097222221, (37, 8, 4): 3.061101466049383, (255, 255, 255): 2.1434702932098766, ... }

```


### calculate color average

```py
df.bb_calculate_color_average(result_column="aa_average_color")

# Calculate the color average of each element and adds a new column to the DataFrame

# (16, 20, 46)

```


### get unique colors

```py
df.bb_get_unique_colors(result_column="aa_unique_colors")

# Gets all unique colors of each element and adds a new column to the DataFrame

# ((0, 0, 0), (0, 0, 1), (0, 0, 2), (0, 0, 3), ... )

```


### get twins with own index

```py
df.bb_get_twins(result_column="aa_twins_with_own_index", with_own_index=True)

# Finds equal elements (index for df.loc) and adds a new column to the DataFrame

# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

```


### get twins

```py
df.bb_get_twins(result_column="aa_twins", with_own_index=False)

# Finds equal elements (index for df.loc) and adds a new column to the DataFrame

# [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

```


### template matching with scipy

```py
df.bb_match_template_scipy(result_column="aa_template_matching_result", needle=df.aa_screenshot[37])

# Does a template matching

# needs scikit-image and pyfftw

# pip install scikit-image
# pip install pyfftw

# Match percentage, start x, start y
# (0.2471, 1, 5)

```


## 🎯 **Input Events Without Element Parsing**

To interact with elements—such as clicking, tapping, scrolling, writing text, or pressing keys—you can create instances of a collection of input classes. The advantage is that these instances are **callable** and can be **reused**, which saves overhead.

Most of these classes are implemented in **Cython** using low-overhead `cdef` classes for maximum performance.


### 📱 Using the System's `input` Command

```py
cmd_input_tap = cyandro.get_cmd_input_tap(100, 100)
cmd_input_tap()
```

***


### ✍️ Using the System's `input` Command (Latin Letters with Accents Normalized)

```py
# Sends the whole text
cmd_send_text = cyandro.get_cmd_send_text("Hi there m'y frönd")
cmd_send_text()

# sends each letter of the text
cmd_send_text_natural = cyandro.get_cmd_send_text_natural("Hi there my frönd")
cmd_send_text_natural()

# sends each letter of the text - unicode - supports á, é, í, ó, ú, ý, Á, É, Í, Ó, Ú, Ý, ç, Ç, â, ê, î, ô, û, Â, Ê, Î, Ô, Û, ã, ñ, õ, Ã, Ñ, Õ, ß, ẞ, ä, ë, ï, ö, ü, ÿ, Ä, Ë, Ï, Ö, Ü, Ÿ, à, è, ì, ò, ù, À, È, Ì, Ò, Ù
# Uses Androids input keycombination - contribs for more letters are welcome!
cmd_send_text_natural_with_unicode = cyandro.get_cmd_send_text_unicode("Hi there my frönd")
cmd_send_text_natural_with_unicode()

cmd_send_text_natural_with_unicode_each_letter = cyandro.get_cmd_send_text_unicode_natural("Hi there my frönd")
cmd_send_text_natural_with_unicode_each_letter()
```

***

### ⌨️ Keycodes (Using the System's input Command)

```py
cyandro.KeyCodes.long_press_KEYCODE_BACKSLASH()
cyandro.KeyCodes.short_press_KEYCODE_BACKSLASH()
```
***

### [🌐 ADBKeyboard - For Unicode Text (Not Supported on Every Device)](https://github.com/senzhk/ADBKeyBoard)
```py
cyandro.adb_keyboard.download_adbkeyboard_apk()
cyandro.adb_keyboard.install_adbkeyboard_apk()
cyandro.adb_keyboard.activate_adb_keyboard()
cyandro.adb_keyboard.send_unicode_text_with_delay(
 text="Hello myç friend",
 delay_range=(0.05, 0.3),
)
cyandro.adb_keyboard.send_unicode_text("Hello myç friend")
cyandro.disable_adb_keyboard()
```

***

## 🖥️ **The Interactive Shell**

Sometimes, you need to do more than just parse and interact with elements on the screen—you need to execute commands directly on the device.
**CyAndroCel** provides an interactive shell that doesn’t rely on either the Python `subprocess` module or Python threads.

This shell is implemented **100% in C++**, runs in **nogil mode**, and offers **excellent reaction times**, making it perfect for high-performance automation tasks.

***

### 🚀 Key Benefits of the Interactive Shell:

1. **Real-Time Interaction:** Fast command execution with immediate feedback.
2. **Root Access Control:** Easily switch between normal and superuser modes.
3. **DataFrame Integration:** Collect system data in a structured format for automation workflows.
4. **Optimized for Performance:** Runs in C++ with nogil mode for high responsiveness.

***

### 🚀 **Starting an Interactive Shell**

### To start a shell, simply use:

#### For a C++ shell:

| **Advantages ✅**                                              | **Disadvantages ❌**                                           |
|---------------------------------------------------------------|----------------------------------------------------------------|
| No-Gil                                                        | Segfaults if there is no ADB connection (Python crashes)                                                             |
| Very low CPU usage                                            | —                                                              |
| The C++ destructor closes the shell                           | —                                                              |
| Uses C++ threads to capture stdout/stderr                     | —                                                              |

```python
myshell = cyandro.open_shell(
    buffer_size=40960,        # The max size of the C++ vector—works like collections.deque (ring buffer)
    exit_command=b"exit",     # The command to exit the shell, required by the C++ destructor.
                              # Executed automatically when 'myshell' goes out of scope.
    print_stdout=False,       # Prints stdout to the screen—useful for debugging purposes
    print_stderr=False,       # Prints stderr to the screen—useful for debugging purposes
    use_py_subproc=False      # Doesn't use Python's subprocess
)

```

#### For a Python shell:

| **Advantages ✅**                                              | **Disadvantages ❌**                                           |
|---------------------------------------------------------------|----------------------------------------------------------------|
| Throws an exception if there is no ADB connection                                                        | Higher CPU usage                                                              |
| —                     | Shell must be closed manually using self.close_py_subproc()                                                            |
| —                     | —                                                              |
| —                     | —                                                              |

```python
myshell = cyandro.open_shell(
    buffer_size=40960,        # ignored
    exit_command=b"exit",     # ignored
    print_stdout=False,       # ignored
    print_stderr=False,        # ignored
    use_py_subproc=True,      # uses Python's subprocess
)
# Do some stuff
...
# close the shell manually
myshell.close_py_subproc()

***

## ⚡ **Interacting Directly with the Shell**

Once the interactive shell is running, you can execute commands directly and capture their output effortlessly.

***

### 📋 **Executing Commands**

```python
>>> stdout, stderr = myshell.write_and_wait("ls -lZ")
# stdout and stderr contain the command’s output in bytes.
# Use .decode() to convert the output to a human-readable string.
>>> print(stdout.decode())
total 48
-rw-r--r--  1 root    root    u:object_r:app_data_file:s0                     12264 2025-02-06 20:29 None
drwxr-x--- 16 root    root    u:object_r:app_data_file:s0                      4096 2025-02-04 22:57 cyandrocel
-rw-r--r--  1 root    root    u:object_r:app_data_file:s0                         0 2025-02-02 01:40 pckldf.pkl
-rw-r-----  1 root    root    u:object_r:app_data_file:s0                     17446 2025-02-04 22:51 pyshell.py
drwx------  5 u0_a286 u0_a286 u:object_r:app_data_file:s0:c30,c257,c512,c768   4096 2025-02-01 23:45 tessdata
drwxr-xr-x  5 root    root    u:object_r:app_data_file:s0                      4096 2025-02-02 00:13 tessdata_best
drwxr-xr-x  5 root    root    u:object_r:app_data_file:s0                      4096 2025-02-02 00:11 tessdata_fast
```

***

## Predefined Shell Commands

### The shell comes with a variety of predefined commands to simplify common tasks.

#### These commands follow two naming conventions:

1. get_df_* → Returns data as a pandas DataFrame for easy analysis.
2. sh_* → Executes shell commands and usually returns a tuple[bytes, bytes] containing stdout and stderr,
but not always—some commands may return different structures.

### ✅ Example: Checking the Current User

```py
# This command checks the current user, and the result is returned in byte format (b'root').
>>> myshell.sh_whoami()
b'shell'
```
***

## Important: Some of the commands below may not work due to missing root rights

### ✅ Example: Getting the build props as a pandas DataFrame

```py
>>> buildprops=myshell.get_df_build_props()
>>> print(buildprops)

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
iloc  | index | aa_file                                            | aa_line   | aa_line_content                                                                                                                                                                                                                                                                                              |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0     | 0     | '/data/adb/modules/make_writeable/system.prop'     | 0         | b'# This file will be read by resetprop'                                                                                                                                                                                                                                                                     |
1     | 1     | '/data/adb/modules/make_writeable/system.prop'     | 1         | b'# Example: Change dpi'                                                                                                                                                                                                                                                                                     |
2     | 2     | '/data/adb/modules/make_writeable/system.prop'     |