Skill: jordaneparis/desktop-automation-ultra (version 2.0.0)

# Desktop Automation Skill v2.0 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![OpenClaw](https://img.shields.io/badge/OpenClaw-Skill-blue)](https://github.com/openclaw/openclaw) **Complete desktop automation for Windows/macOS/Linux. Zero-error edition.** --- ## ⚠️ **Privacy & Security** **CRITICAL**: This skill captures ALL keyboard and mouse events. - **NEVER record while entering passwords, credit cards, or secrets** - Recorded macros are stored as JSON in `recorded_macro/` directory - Always use **dry_run=true** to test before actual execution - Store macros in secure locations only - Enable safe mode by default (it is) --- ## 🎯 **What It Does** Automate desktop interactions without APIs: - ✅ Click, type, drag, scroll - ✅ Capture screenshots - ✅ Recognize images (OpenCV template matching) - ✅ Extract text (Tesseract OCR) - ✅ Record and replay macros - ✅ Find windows by title - ✅ Clipboard operations - ✅ Safe mode with dry_run for testing --- ## 🔐 **Safety Features (Built-In)** ### 1. **Safe Mode** (Default: ON) Blocks dangerous actions when enabled: - `type`, `press_key`, `click`, `drag` are monitored - Parameters are scanned for dangerous patterns: `rm `, `del `, `C:\Windows\`, `/etc/`, `sudo`, etc. - Blocked actions are logged ### 2. **Dry-Run Mode** All actions support `dry_run=true`: - Action is logged but NOT executed - Use for testing before running real automation ### 3. **Audit Logging** Every action logged to `~/.openclaw/skills/desktop-automation-logs/automation_YYYY-MM-DD.log` ### 4. **Thread Safety** All modules use locks to prevent race conditions. --- ## 📦 **Installation** ### 1. Extract Files Place `desktop-automation-ultra-local/` in: - Windows: `C:\Users\<User>\.openclaw\workspace\skills\` - Linux/macOS: `~/.openclaw/workspace/skills/` ### 2. Install Dependencies ```bash pip install -r requirements.txt ``` ### 3. Optional: Tesseract for OCR For `find_text_on_screen` functionality: - **Windows**: Download installer from https://github.com/UB-Mannheim/tesseract/wiki - **Linux**: `sudo apt install tesseract-ocr` - **macOS**: `brew install tesseract` ### 4. Restart OpenClaw ```bash openclaw gateway restart ``` --- ## 🚀 **Quick Start** ### Basic Click ```yaml action: click params: x: 100 y: 100 dry_run: true # Test first! ``` ### Type Text ```yaml action: type params: text: "Hello World" interval: 0.05 # Delay between keys dry_run: false ``` ### Find Image ```yaml action: find_image params: template_path: "templates/button.png" confidence: 0.95 ``` ### Extract Text (OCR) ```yaml action: read_text_ocr params: lang: "fra" # French ``` --- ## 📖 **Core Actions** ### **Mouse & Keyboard** | Action | Parameters | Returns | |--------|------------|---------| | `click` | `x`, `y`, `button="left"`, `dry_run` | `{status, x, y}` | | `type` | `text`, `interval=0.05`, `dry_run` | `{status, text}` | | `press_key` | `key`, `dry_run` | `{status, key}` | | `move_mouse` | `x`, `y`, `duration=0.5`, `dry_run` | `{status, x, y}` | | `scroll` | `amount=5`, `dry_run` | `{status, amount}` | | `drag` | `start_x`, `start_y`, `end_x`, `end_y`, `duration=0.5`, `dry_run` | `{status}` | | `copy_to_clipboard` | `text`, `dry_run` | `{status}` | | `paste_from_clipboard` | `dry_run` | `{status, length}` | ### **Screenshots & Windows** | Action | Parameters | Returns | |--------|------------|---------| | `screenshot` | `path="~/Desktop/screenshot.png"`, `dry_run` | `{status, path}` | | `get_active_window` | `dry_run` | `{status, title, x, y, width, height}` | | `list_windows` | `dry_run` | `{status, windows[], count}` | | `activate_window` | `title_substring`, `dry_run` | `{status, title}` | ### **Image Recognition** (requires OpenCV) | Action | Parameters | Returns | |--------|------------|---------| | `find_image` | `template_path`, `confidence=0.9`, `dry_run` | `{status, x, y, confidence}` | | `find_image_multiscale` | `template_path`, `confidence`, `scale_factors`, `dry_run` | `{status, x, y, confidence, scale}` | | `wait_for_image` | `template_path`, `timeout=30.0`, `interval=0.5`, `confidence=0.9`, `dry_run` | `{status, x, y, confidence}` | ### **OCR / Text Recognition** (requires Tesseract) | Action | Parameters | Returns | |--------|------------|---------| | `find_text_on_screen` | `text`, `lang="fra"`, `dry_run` | `{status, locations[], count}` | | `find_all_text_on_screen` | `text`, `lang="fra"`, `dry_run` | `{status, data[], count}` | | `read_text_ocr` | `lang="fra"`, `dry_run` | `{status, text, length}` | | `read_text_region` | `x`, `y`, `width`, `height`, `lang="fra"`, `dry_run` | `{status, text, length}` | | `extract_screen_data` | `region={}`, `output_format="json"`, `lang="fra"`, `dry_run` | `{status, data[], count}` | ### **Macros** | Action | Parameters | Returns | |--------|------------|---------| | `play_macro` | `macro_path`, `speed=1.0`, `dry_run` | `{status, executed, total, errors[]}` | | `stop_macro` | — | `{status}` | | `play_macro_with_subroutines` | `macro_path`, `speed=1.0`, `sub_macros_dir`, `dry_run` | `{status, executed, total, errors[]}` | ### **Safety Management** | Action | Parameters | Returns | |--------|------------|---------| | `set_safe_mode` | `enabled=true` | `{status, safe_mode}` | | `get_safety_status` | — | `{status, safe_mode_enabled, dangerous_patterns, dangerous_actions[]}` | --- ## 📝 **Macro Format** Recorded macros are JSON with this structure: ```json { "events": [ { "action": "click", "params": {"x": 100, "y": 50}, "wait": 500 }, { "action": "type", "params": {"text": "Hello"}, "wait": 200 }, { "action": "press_key", "params": {"key": "return"}, "wait": 100 } ] } ``` - `action` — action name - `params` — action parameters - `wait` — milliseconds to wait before next action --- ## 🔧 **Advanced: Mouse Move Debouncing** To avoid recording hundreds of `move_mouse` events during a smooth drag, the recorder uses **debouncing**: - When you move the mouse, events are **suppressed** during movement - After you **stop moving** for `N` seconds (default: 1 sec), the **final position** is recorded - This reduces macro size dramatically while preserving intended end positions - Configurable via GUI: set debounce time (0.1–10 seconds) **Example:** - Fast horizontal line → 1 `move_mouse` event (end coordinates) - Slow, stop-and-go → multiple `move_mouse` events (one per "stop") --- ## 🧪 **Testing** Run the unit test suite: ```bash python scripts/test_automation.py ``` Output: ``` test_dry_run_click ... ok test_get_active_window ... ok test_safe_mode_blocks_dangerous ... ok ... Ran 13 tests OK ``` --- ## 📊 **Logging** All actions logged to: `~/.openclaw/skills/desktop-automation-logs/automation_YYYY-MM-DD.log` Example: ``` [2026-03-15 10:23:45] [INFO] ActionManager: ActionManager initialized with safe_mode=True [2026-03-15 10:23:46] [INFO] ActionManager: Clicked at (100, 50) with left button [2026-03-15 10:23:47] [INFO] ActionManager: Typed: Hello World ``` --- ## ⚙️ **Configuration** ### Environment Variables ```bash # Override log directory export AUTOMATION_LOG_DIR=~/my_logs # Disable safe mode globally (NOT recommended) export AUTOMATION_SAFE_MODE=false ``` --- ## 🐛 **Troubleshooting** ### "pyautogui failsafe triggered" Move mouse to corner of screen to stop. ### OCR returns empty text - Ensure Tesseract is installed correctly - Check image quality (high contrast helps) - Try `read_text_ocr` instead of `find_text_on_screen` ### Image recognition not finding template - Ensure template image exists and is correct format (PNG, JPG) - Try lower confidence threshold (e.g., 0.85 instead of 0.95) - Use `find_image_multiscale` to detect at different scales ### Actions blocked by safe mode This is intentional. To run dangerous actions: ```yaml action: set_safe_mode params: enabled: false ``` Then execute your action. Re-enable safe mode immediately after: ```yaml action: set_safe_mode params: enabled: true ``` --- ## 📄 **License** MIT License. See LICENSE file. --- ## 📚 **Files Structure** ``` desktop-automation-ultra-local/ ├── SKILL.md (This file) ├── requirements.txt (Python dependencies) ├── lib/ │ ├── actions.py (Core click/type/drag actions) │ ├── image_recognition.py (OpenCV template matching) │ ├── ocr_engine.py (Tesseract OCR) │ ├── macro_player.py (Record/playback macros) │ ├── safety_manager.py (Safe mode, blocking) │ └── utils.py (Logging, helpers) ├── scripts/ │ └── test_automation.py (Unit tests) └── recorded_macro/ (Output: saved macros) ``` --- ## ✅ **Validation Checklist** - [x] All modules have proper error handling - [x] Thread safety implemented (locks) - [x] Safe mode enabled by default - [x] Dry-run mode on all actions - [x] Comprehensive logging - [x] Unit tests (13 tests) - [x] UTF-8 encoding for all text - [x] No hardcoded paths (uses expanduser) - [x] Graceful fallbacks for missing dependencies - [x] Documentation complete **Status: PRODUCTION READY** ✅ --- *Last updated: 2026-03-15* *Version: 2.0.0*

path: AUTHORS.md
content: # Authors

## Original Creator

**Jordane Guemara** — [GitHub](https://github.com/JordaneParis) — Freelance developer, automation expert, OpenClaw enthusiast.

---

## Contributors

Thanks to everyone who has improved this skill!

- [Add your name here after first PR]

---

## How to be listed

Any significant contribution (code, docs, tests, bug reports) leads to being added to this list. Minor contributions (typos) are appreciated but not automatically listed.

To appear:
1. Submit an accepted PR
2. Your name will be added in the Contributors section

---

## Citation

If you use this skill in a project, please cite:

```bibtex
@misc{desktop-automation-ultra-local,
  title = {Desktop Automation Skill for OpenClaw},
  author = {Jordane Guemara and contributors},
  year = {2026},
  url = {https://github.com/JordaneParis/desktop-automation-ultra-local}
}
```


path: CONTRIBUTING.md
content: # Contributing to Desktop Automation Skill

Thank you for your interest! This skill is open-source under the MIT license. All contributions are welcome — bug reports, features, documentation, tests.

---

## 🛠️ Development Environment

### Prerequisites
- Python 3.10+
- Git
- Pip + virtualenv (recommended)

### Setup
```bash
git clone https://github.com/JordaneParis/desktop-automation-ultra-local.git
cd desktop-automation-ultra-local
python -m venv venv
venv\Scripts\activate  # Windows
# or source venv/bin/activate  # Linux/macOS
pip install -r requirements.txt
```

### Test
```bash
# Check module imports
python -c "import pyautogui, pygetwindow, PIL; print('OK')"

# Dry-run a macro (if test file exists)
python scripts/play_macro.py recorded_macro/example.json --dry-run
```

---

## 📝 Code Style

- **PEP 8** — follow Python conventions (4 spaces, camelCase for functions)
- **Type hints** — add for new functions (not required for minor changes)
- **Docstrings** — every function/class must have a docstring in English
- **Logging** — use `logging` instead of `print`. Levels:
  - `DEBUG`: technical details
  - `INFO`: important user actions
  - `WARNING`: recoverable problems
  - `ERROR`: blocking errors
- **Exceptions** — raise clear exceptions with informative messages
- **JSON** — always `ensure_ascii=False`, `indent=2` for readability

---

## 🔄 Contribution Workflow

1. **Fork** the repository
2. **Create a branch**:
   ```bash
   git checkout -b feature/my-new-feature
   ```
   or
   ```bash
   git checkout -b fix/bug-fix
   ```
3. **Develop** with clear commits:
   ```bash
   git commit -am "Add resize_window action"
   ```
4. **Test** locally (if applicable)
5. **Push** to your fork:
   ```bash
   git push origin feature/my-new-feature
   ```
6. **Open a Pull Request**:
   - Clear title (e.g., "Add drag-and-drop action", "Fix window monitor race condition")
   - Full description: what, why, how
   - Link related issues
   - Attach screenshots/outputs if relevant

---

## 📋 PR Checklist

- [ ] Code conforms to style (PEP 8 + rules above)
- [ ] Docstrings added/updated
- [ ] README.md updated if needed (new action, parameter)
- [ ] SKILL.md updated (action list)
- [ ] Tests performed (manual or automated)
- [ ] No stray `print()` statements (use `logging`)
- [ ] UTF-8 everywhere (accents preserved)
- [ ] JSON files well-formed
- [ ] No external dependencies added without discussion (prefer optional)

---

## 🧪 Testing

Currently the skill has no automated tests. If you add them:
- Place tests in `tests/`
- Use `pytest` if possible
- Test pure functions (e.g., timestamp parsing, JSON validation)
- For GUI actions: document manual tests in the PR

---

## 📚 Documentation

- **SKILL.md** — short description, action list, usage example. Update for every new action.
- **README.md** — complete user guide (install, troubleshooting, examples). Update for major changes.
- **Docstrings** — serve as developer reference.

---

## 🔐 Security

- **No network access** — the skill must remain local only
- **No external code execution** (no `eval`, `exec`, `os.system` without sandbox)
- **No credentials** — never request or store passwords
- **Respect permissions** — warn if admin rights are needed

---

## ❓ Questions

- Open an *Issue* to discuss before large modifications
- For bugs: provide steps to reproduce, OS, Python/package versions
- For features: explain the use case

---

## 📄 License

All contributions are under the MIT license (see `LICENSE`). By submitting a PR, you agree that your changes will be distributed under this license.

---

**Thanks for making this skill better!** 🚀


path: DEPENDENCIES.md
content: # Dependencies — Desktop Automation Skill

This document explains each dependency and its purpose.

## Core (Required)

| Package | Version | Purpose |
|---------|---------|---------|
| `pyautogui` | 0.9.54 | Main automation library: mouse moves, clicks, keyboard typing, screenshots, scrolling |
| `pygetwindow` | 0.0.9 | Window management: get active window, list windows, activate by title |
| `Pillow` | 10.4.0 | Image handling (used by pyautogui for screenshots) |
| `pynput` | 1.7.6 | Low-level input listeners for macro recording (keyboard + mouse events) |

## Optional — Vision (Image & OCR)

| Package | Version | Purpose | Required for actions |
|---------|---------|---------|---------------------|
| `opencv-python` | 4.10.0.84 | Computer vision: template matching, image recognition | `find_image`, `wait_for_image` |
| `pytesseract` | 0.3.10 | Optical Character Recognition (OCR) | `find_text_on_screen`, `extract_screen_data` |

**Note on pytesseract**: You also need the [Tesseract binary](https://github.com/tesseract-ocr/tesseract) installed on your system (not just the Python package). On Windows, download the installer and add it to PATH.

## Optional — Clipboard

| Package | Version | Purpose | Required for actions |
|---------|---------|---------|---------------------|
| `pyperclip` | 1.9.0 | Cross-platform clipboard access | `copy_to_clipboard`, `paste_from_clipboard` |

**Platform notes**:
- Windows/macOS: usually works out of the box
- Linux: may require `xclip` or `xsel` system packages

## Optional — Data Integration (Excel/CSV)

| Package | Version | Purpose | Required for actions |
|---------|---------|---------|---------------------|
| `openpyxl` | 3.1.5 | Read/write Excel `.xlsx` files | `excel_read`, `excel_write` |
| `pandas` | 2.2.3 | DataFrames, CSV I/O | `data_to_csv` (and used inside excel functions) |

---

## Installing Selectively

You can install only the dependencies you need:

```bash
# Minimal: basic automation only
pip install pyautogui pygetwindow Pillow pynput

# + Image recognition
pip install opencv-python

# + OCR
pip install pytesseract

# + Clipboard
pip install pyperclip

# + Excel/CSV
pip install openpyxl pandas
```

The skill will automatically detect which dependencies are available and enable the corresponding actions. Missing optional dependencies will cause the related actions to return an error with a clear message (e.g., "openpyxl not installed").

---

## Full Install

To install everything at once:

```bash
pip install -r requirements.txt
```

The `requirements.txt` in this skill pins the versions that have been tested together.

---

## Troubleshooting

- **ImportError: No module named 'cv2'** → `opencv-python` not installed. Install it or avoid using `find_image`.
- **pytesseract.pytesseract.TesseractNotFoundError** → Tesseract binary missing. Install from https://github.com/tesseract-ocr/tesseract and add to PATH.
- **pyperclip.PyperclipException** on Linux → install `xclip` (`sudo apt-get install xclip`) or `xsel`.

---

**Last updated**: 2026-03-15


path: LICENSE.txt
content: **MIT License**

Copyright (c) 2026 Jordane Guemara & contributeurs

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


path: README.md
content: # Desktop Automation Skill for OpenClaw

**âš ï¸  PRIVACY WARNING**: The macro recorder captures **ALL** keyboard events (including passwords, credit card numbers, private messages) and window titles. **Never record while entering credentials.** Only use for non-sensitive workflows. Store recorded macros securely.

Full desktop automation: control mouse, keyboard, windows, OCR, image recognition â€” all local.

- **PyAutoGUI** for basic actions
- **OpenCV** for image recognition
- **pytesseract** for OCR

---

## ðŸ” Security & Privacy

### âš ï¸ Keyboard Recording Warning
The **macro recorder** captures **ALL** keyboard events, including passwords, credit card numbers, and personal data. **Never** record macros while entering credentials. Use only for non-sensitive workflows. Recorded macro files contain raw keystrokes â€” store them securely.

**Summary**:
- No network access â€” completely local
- No credential storage (unless you record them)
- All dependencies are standard Python packages
- You are responsible for macro file security

---

## Installation

### Prerequisites
- Python 3.10+
- Pip

### Install dependencies

```bash
pip install -r requirements.txt
```

### ðŸ¤– Advanced Features (v2+)

The skill includes powerful advanced features for power users.

---

### Multi-Scale Image Detection

`find_image_multiscale` searches for a template at multiple scales. Useful when the UI element size varies (e.g., high-DPI displays).

```javascript
sessions_spawn({
  task: 'find_image_multiscale',
  params: {
    template_path: "C:/templates/button.png",
    confidence: 0.85,
    scale_factors: [0.5, 0.75, 1.0, 1.25, 1.5]  // optional, default includes these
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{
  "status": "ok",
  "x": 400,
  "y": 300,
  "confidence": 0.92,
  "scale": 1.25
}
```

---

### Find All Text Occurrences

`find_all_text_on_screen` returns every match, not just the first. Great for extracting tables or lists.

```javascript
sessions_spawn({
  task: 'find_all_text_on_screen',
  params: {
    text: "Total",
    lang: "eng"
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{
  "status": "ok",
  "matches": [
    {"text": "Total", "left": 100, "top": 200, "width": 50, "height": 20, "conf": 98.5},
    {"text": "Total", "left": 300, "top": 200, "width": 50, "height": 20, "conf": 97.2}
  ],
  "count": 2
}
```

---

### Detect UI Elements

`detect_ui_elements` uses shape heuristics to find common UI components.

```javascript
sessions_spawn({
  task: 'detect_ui_elements',
  params: {
    element_type: "button"  // omit for all types
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{
  "status": "ok",
  "elements": [
    {"type": "button", "x": 120, "y": 80, "w": 80, "h": 30, "area": 2400},
    {"type": "field", "x": 120, "y": 130, "w": 200, "h": 25, "area": 5000}
  ],
  "count": 2
}
```

---

### Conditional Monitoring with Logic

`monitor_screen_with_logic` extends `monitor_screen` with AND/OR logic between conditions.

```javascript
sessions_spawn({
  task: 'monitor_screen_with_logic',
  params: {
    conditions: [
      {
        type: "image",
        template_path: "C:/templates/ok.png",
        confidence: 0.9,
        logic: "AND",
        actions: [
          {"action": "click", "params": {}}
        ]
      },
      {
        type: "text",
        text: "Success",
        logic: "OR",
        actions: [
          {"action": "press_key", "params": {"key": "enter"}}
        ]
      }
    ],
    timeout: 60,
    interval: 0.5
  },
  label: 'desktop-automation-ultra-local'
});
```

---

### Macros with Subroutines

`play_macro_with_subroutines` allows nested macro calls. Create a macro file that contains:
```json
{
  "events": [
    {"action": "type", "params": {"text": "Hello"}},
    {"action": "call_macro", "params": {"macro_file": "submacro.json"}}
  ]
}
```

Run it:
```javascript
sessions_spawn({
  task: 'play_macro_with_subroutines',
  params: {
    macro_path: "C:/macros/main.json",
    speed: 1.0,
    sub_macros_dir: "C:/macros/subs"
  },
  label: 'desktop-automation-ultra-local'
});
```

---

### Protected Macros (Password-Locked)

Create a macro encrypted with AES using a password:

```javascript
sessions_spawn({
  task: 'create_protected_macro',
  params: {
    output_path: "C:/macros/secure.enc",
    [REDACTED_SECRET]",
    macro_events: [
      {"action": "click", "params": {"x":100,"y":200}}
    ]
  },
  label: 'desktop-automation-ultra-local'
});
```

Load and decrypt:
```javascript
sessions_spawn({
  task: 'load_and_decrypt_protected_macro',
  params: {
    encrypted_path: "C:/macros/secure.enc",
    [REDACTED_SECRET]"
  },
  label: 'desktop-automation-ultra-local'
});
// Returns {"status":"ok", "events": [...]} â€” you can then play them
```

---

### Macro Execution Reports

`generate_macro_report` creates HTML and JSON logs of macro execution:

```javascript
sessions_spawn({
  task: 'generate_macro_report',
  params: {
    macro_path: "C:/macros/my_macro.json",
    execution_log: {
      "status": "ok",
      "actions": [
        {"action": "click", "params": {"x":100,"y":200}, "result": {"status":"ok"}}
      ],
      "elapsed": 12.34
    }
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{
  "status": "ok",
  "report_json": "C:/macros/reports/report_2026-03-15_00-54-00.json",
  "report_html": "C:/macros/reports/report_2026-03-15_00-54-00.html"
}
```

---

### Macro Stop Hotkey (Safety)

The `MacroStopManager` class (used internally) can stop a running macro via a hotkey combination. This is a safety feature to abort long-running or misbehaving macros. (In the current version, hotkey is configurable but not exposed as a direct action; it's used by the framework.)

---

### Dependencies Explained
The skill uses several Python packages. Here's a quick summary:

| Package | Purpose | Actions that need it |
|---------|---------|---------------------|
| pyautogui | mouse/keyboard control, screenshots | all basic actions |
| pygetwindow | window management | activate_window, list_windows, get_active_window |
| Pillow | image processing | used by pyautogui |
| pynput | macro recording (listeners) | record_macro |
| opencv-python | image recognition | find_image, wait_for_image, find_image_multiscale, find_on_screen, click_image |
| pytesseract | OCR (text from images) | find_text_on_screen, extract_screen_data, find_all_text_on_screen, read_text_ocr, read_text_region |
| pyperclip | clipboard access | copy_to_clipboard, paste_from_clipboard |
| openpyxl | Excel read/write | excel_read, excel_write |
| pandas | CSV/DataFrame conversion | data_to_csv (and used by excel functions) |
| cryptography | AES encryption for protected macros | create_protected_macro, load_and_decrypt_protected_macro |
| mss | fast screen capture | screenshot_mss, vision functions (fallback if missing) |
| numpy | array operations (OpenCV, mss) | used by opencv-python and mss |

**Optional packages**: If a package is missing, related actions will return an error. You can install selectively:
```bash
# Minimal
pip install pyautogui pygetwindow Pillow pynput
# + Image recognition
pip install opencv-python
# + OCR
pip install pytesseract
# + Clipboard
pip install pyperclip
# + Excel/CSV
pip install openpyxl pandas
```

For full details, see `DEPENDENCIES.md` in this skill folder.

---

### Enable skill
Place the folder `desktop-automation-ultra-local` in your OpenClaw skills directory:
```bash
pip install -r requirements.txt
```

### Enable skill
Place the folder `desktop-automation-ultra-local` in your OpenClaw skills directory:
- **Windows**: `C:\Users\<YourUsername>\.openclaw\workspace\skills\`
- **Linux/macOS**: `~/.openclaw/workspace/skills/`

Then restart OpenClaw:
```bash
openclaw gateway restart
```

---

## Available Actions

### Basic Actions
- `click` â€” click at coordinates
- `type` â€” type text
- `screenshot` â€” capture screen
- `get_active_window` â€” get active window info
- `list_windows` â€” list all windows
- `activate_window` â€” activate window by title substring
- `move_mouse` â€” move cursor
- `press_key` â€” press a single key
- `scroll` â€” scroll screen
- `copy_to_clipboard` â€” copy text to clipboard
- `paste_from_clipboard` â€” paste from clipboard
- `drag` â€” drag from start to end

### Advanced Actions
- `find_image` â€” locate image on screen (OpenCV)
- `wait_for_image` â€” wait until image appears
- `monitor_screen` â€” conditional monitoring: watch for images/text and trigger actions
- `find_text_on_screen` â€” OCR text search
- `extract_screen_data` â€” extract structured text data (OCR) from screen or region
- `excel_read` â€” read data from an Excel file into list of dicts
- `excel_write` â€” write list of dicts/lists to an Excel file
- `data_to_csv` â€” convert list of dicts to CSV (string or file)

---

## Examples

```javascript
// Click and type
sessions_spawn({ task: 'click {"x":100,"y":200}', label: 'desktop-automation-ultra-local' });
sessions_spawn({ task: 'type {"text":"Hello World"}', label: 'desktop-automation-ultra-local' });

// Take screenshot
sessions_spawn({ task: 'screenshot {"path":"~/Desktop/screen.png"}', label: 'desktop-automation-ultra-local' });

// Activate Notepad
sessions_spawn({ task: 'activate_window {"title_substring":"Notepad"}', label: 'desktop-automation-ultra-local' });

// Find an image
sessions_spawn({
  task: 'find_image {"template_path":"C:/button.png","confidence":0.9}',
  label: 'desktop-automation-ultra-local'
});
```

---

## Macro Recording

### GUI Recorder
Run the Tkinter GUI to record macros:
```bash
python scripts/record_macro.py
```
Records: mouse moves, clicks, scrolling, keyboard, window switches.
Saves JSON files to `recorded_macro/`.

#### Mouse Move Debouncing
To avoid recording hundreds of `move_mouse` events during a smooth drag, the recorder uses **debouncing**:

- **Configurable debounce time** (default: 1 second) via the GUI entry field
- When you move the mouse, events are suppressed during movement
- After you **stop moving** for the debounce period, the **final position** is recorded
- This reduces macro size dramatically while preserving intended end positions

**Example:**
- Fast horizontal line â†’ 1 `move_mouse` event (end coordinates)
- Slow, stop-and-go â†’ multiple `move_mouse` events (one per "stop")

Adjust the debounce time in the GUI to suit your workflow (0.1â€“10 seconds).

### CLI Player
Replay a recorded macro:
```bash
python scripts/play_macro.py recorded_macro/macro_2026-03-14_22-00-00.json 1.0
```
Speed factor optional (1.0 = normal).

---

## ðŸ” Monitor Screen (Conditional Automation)

The `monitor_screen` action watches the screen for specific visual elements (images or text) and executes actions when they appear.

### Parameters

```json
{
  "checks": [
    {
      "type": "image" | "text",
      "template_path": "path/to/template.png",
      "confidence": 0.9,
      "text": "string to find",
      "lang": "fra",
      "action": "click",
      "action_params": { "button": "left" }
    }
  ],
  "timeout": 60,
  "interval": 0.5,
  "stop_condition": "first_match",
  "fallback_confidence": 0.85
}
```

- `checks`: array of conditions to monitor
- `timeout`: max seconds to monitor (default 60)
- `interval`: check every N seconds (default 0.5)
- `stop_condition`:
  - `"first_match"`: stop after first condition matches (default)
  - `"all_matched"`: stop when all conditions have matched at least once
  - `null` or omitted: run until timeout
- `fallback_confidence`: if image not found at initial confidence, automatically retry with lower thresholds (0.85, 0.8, 0.75, ...). Set to `null` to disable.

### Example: Click a pop-up as soon as it appears

```javascript
sessions_spawn({
  task: 'monitor_screen',
  params: {
    checks: [
      {
        type: "image",
        template_path: "C:/templates/ok_button.png",
        confidence: 0.9,
        action: "click"
      }
    ],
    timeout: 30,
    stop_condition: "first_match"
  },
  label: 'desktop-automation-ultra-local'
});
```

### Example: Type credentials when fields appear

```javascript
sessions_spawn({
  task: 'monitor_screen',
  params: {
    checks: [
      {
        type: "text",
        text: "Email",
        lang: "eng",
        action: "type",
        action_params: { "text": "[REDACTED_SECRET]", "interval": 0.05 }
      },
      {
        type: "text",
        text: "Password",
        lang: "eng",
        action: "type",
        action_params: { "text": "secret123", "interval": 0.05 }
      }
    ],
    timeout: 20,
    stop_condition: "all_matched"
  },
  label: 'desktop-automation-ultra-local'
});
```

---

## Configuration

### OCR (pytesseract)
- Install [Tesseract](https://github.com/tesseract-ocr/tesseract) on your OS.
- On Windows, add to PATH or set:
  ```python
  pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'
  ```
- Install language packs (`fra`, `eng`, etc.)

### Pyperclip
- Linux: install `xclip` or `xsel`
- Windows/macOS: usually works out of the box

---

## ðŸ“Š Data Integration

### extract_screen_data
Extract structured text data from the screen (or a specific region) using Tesseract OCR. Returns bounding boxes, text, and confidence scores.

```javascript
sessions_spawn({
  task: 'extract_screen_data',
  params: {
    region: {x: 0, y: 0, width: 800, height: 600},  // optional
    output_format: 'json'  // default
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{
  "status": "ok",
  "data": [
    {"text": "Total", "left": 100, "top": 200, "width": 50, "height": 15, "conf": 98.5},
    {"text": "$123.45", "left": 200, "top": 200, "width": 60, "height": 15, "conf": 99.1}
  ],
  "count": 2
}
```

### excel_read
Read an Excel file (.xlsx) and return rows as an array of objects (keys = column headers).

```javascript
sessions_spawn({
  task: 'excel_read',
  params: {
    filepath: "C:/data/input.xlsx",
    sheet_name: 0,      // or "Sheet1"
    range: "A1:C10"     // optional; reads whole sheet if omitted
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{
  "status": "ok",
  "data": [
    {"Name": "Alice", "Age": 30},
    {"Name": "Bob", "Age": 25}
  ],
  "columns": ["Name", "Age"],
  "rows": 2
}
```

### excel_write
Write an array of objects (or array of arrays) to an Excel file.

```javascript
sessions_spawn({
  task: 'excel_write',
  params: {
    filepath: "C:/data/output.xlsx",
    data: [
      {"Product": "Widget", "Price": 9.99},
      {"Product": "Gadget", "Price": 19.99}
    ],
    sheet_name: "Results",
    start_cell: "A1"
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response**:
```json
{"status":"ok","filepath":"C:/data/output.xlsx","rows":2,"columns":2}
```

### data_to_csv
Convert a list of dictionaries to CSV format. Returns CSV string or writes to a file.

```javascript
sessions_spawn({
  task: 'data_to_csv',
  params: {
    data: [{"a":1,"b":2},{"a":3,"b":4}],
    filepath: "C:/data/out.csv"  // optional; if omitted, CSV string is returned
  },
  label: 'desktop-automation-ultra-local'
});
```

**Response** (without filepath):
```json
{
  "status": "ok",
  "csv": "a,b\n1,2\n3,4\n",
  "rows": 2,
  "columns": 2
}
```

---

## Safety & Best Practices

âš ï¸ **This skill controls mouse and keyboard. Use with extreme caution.**

### Safe Mode (enabled by default)
- Blocks risky actions (`type`, `press_key`, `click`, `drag`) to prevent accidental damage.
- Disable with: `sessions_spawn({ task: 'set_safe_mode', params: {enabled: false} })`
- Also blocks dangerous patterns (e.g., `rm `, `del `, system paths).

### Dry-Run (Sandbox)
Add `"dry_run": true` to any action to simulate without side effects. Great for testing.

### Audit Logging
All actions are logged to daily files under:
- **Windows**: `C:\Users\<User>\.openclaw\skills\desktop-automation-logs\`
- **Linux/macOS**: `~/.openclaw/skills/desktop-automation-logs/`

Use logs to debug or audit activity.

### General
- Verify coordinates before clicking.
- Test macros in dry-run or slow speed first.
- `activate_window` requires exact or partial title match.
- No network access â€” completely local.
- May require admin rights for certain applications on Windows.

---

## Troubleshooting

| Issue | Fix |
|-------|-----|
| `pyautogui.FailSafeException` | Move mouse to corner (0,0) to disable failsafe, or set `pyautogui.FAILSAFE = False` |
| OCR not working | Ensure Tesseract is installed and in PATH. Check language pack. |
| `find_image` fails | Template must match screen exactly (scale, colors). Lower confidence if needed (0.7â€“0.95). |
| `activate_window` can't find window | Use `list_windows` to get exact titles. |
| Missing modules | Run `pip install -r requirements.txt` |
| Tkinter GUI not opening | Linux: `sudo apt-get install python3-tk`. Windows: usually already installed. |

---

## License

MIT Â© 2026 Jordane Guemara & contributors

See `LICENSE` file.

---

## Contributing

See `CONTRIBUTING.md` to contribute to this skill.

---

## Author

Created by **Jordane Guemara** â€” [@JordaneParis](https://github.com/JordaneParis)





path: requirements.txt
content: pyautogui>=0.9.53
pygetwindow>=0.0.9
Pillow>=8.0.0
opencv-python>=4.5.0
pytesseract>=0.3.10
pyperclip>=1.8.2


path: skill.js
content: #!/usr/bin/env node
/**
 * Desktop Automation Skill — OpenClaw Integration
 * Usage: node skill.js <action> '<json args>'
 * Example: node skill.js click '{"x":100,"y":200}'
 */

const { spawnSync } = require('child_process');
const path = require('path');

function runPython(action, params = {}) {
  const script = path.join(__dirname, 'lib', 'automation.py');
  const args = [script, action, JSON.stringify(params)];
  const result = spawnSync('python', args, { encoding: 'utf-8' });

  if (result.status !== 0) {
    return { status: 'error', message: result.stderr || 'Python error' };
  }

  try {
    return JSON.parse(result.stdout);
  } catch (e) {
    return { status: 'error', message: 'Invalid JSON from Python: ' + e.message };
  }
}

// OpenClaw sends tasks via command line arguments
if (process.argv.length < 3) {
  console.error(JSON.stringify({ status: 'error', message: 'Missing action' }));
  process.exit(1);
}

const action = process.argv[2];
const params = process.argv[3] ? JSON.parse(process.argv[3]) : {};

const output = runPython(action, params);
console.log(JSON.stringify(output));
process.exit(output.status === 'ok' ? 0 : 1);


path: skill.yaml
content: name: desktop-automation-ultra
version: 2.0.0
description: Full desktop automation for OpenClaw — control mouse, keyboard, windows, OCR, image recognition. Record and replay macros.
author: Jordane Guemara
license: MIT
repository: https://github.com/JordaneParis/desktop-automation-ultra-local
tags:
  - automation
  - macro
  - ocr
  - windows
  - desktop
entryPoint: skill.js
requirements:
  - python>=3.10
  - pyautogui==0.9.54
  - pygetwindow==0.0.9
  - Pillow==10.4.0
  - opencv-python==4.10.0.84
  - pyperclip==1.9.0
  - pytesseract==0.3.10
  - pynput==1.7.6
  - openpyxl==3.1.5
  - pandas==2.2.3
  - cryptography==44.0.1
  - mss==9.0.1
  - numpy==1.26.4


path: scripts/play_macro.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Macro Player — Version ultra-robuste
Usage: python play_macro.py <macro_json_path> [speed_factor]
"""
import sys, json, time, os, logging
import pyautogui
import pygetwindow as gw

# ============ CONFIGURATION LOGGING ============
logging.basicConfig(
    level=logging.INFO,
    format='[%(levelname)s] %(asctime)s — %(message)s',
    handlers=[logging.StreamHandler(sys.stderr)]
)
logger = logging.getLogger(__name__)

# ============ VALIDATION ET PRÉPARATION ============
def validate_macro_file(path):
    """Validation exhaustive du fichier macro"""
    if not os.path.exists(path):
        raise FileNotFoundError(f"Fichier introuvable : {path}")

    if not os.path.isfile(path):
        raise ValueError(f"Ce n'est pas un fichier : {path}")

    try:
        size = os.path.getsize(path)
        if size == 0:
            raise ValueError("Fichier vide")
    except Exception as e:
        raise IOError(f"Impossible de lire le fichier : {e}")

    return True

def load_macro(path):
    """Chargement sécurisé du fichier JSON"""
    validate_macro_file(path)

    try:
        with open(path, 'r', encoding='utf-8') as f:
            data = json.load(f)
    except json.JSONDecodeError as e:
        raise ValueError(f"JSON invalide : {e} (ligne {e.lineno}, colonne {e.colno})")
    except UnicodeDecodeError as e:
        raise ValueError(f"Erreur d'encodage (doit être UTF-8) : {e}")
    except Exception as e:
        raise IOError(f"Erreur lecture fichier : {e}")

    # Validation de la structure minimale
    if not isinstance(data, dict):
        raise ValueError("Macro invalide : racine doit être un objet JSON")

    if 'events' not in data:
        raise ValueError("Macro invalide : champ 'events' manquant")

    if not isinstance(data['events'], list):
        raise ValueError("Macro invalide : 'events' doit être un tableau")

    # Vérifier chaque événement
    for i, event in enumerate(data['events']):
        if not isinstance(event, dict):
            raise ValueError(f"Événement {i}: doit être un objet")

        if 'action' not in event:
            raise ValueError(f"Événement {i}: champ 'action' manquant")

        if 'timestamp' not in event:
            raise ValueError(f"Événement {i}: champ 'timestamp' manquant")

        if 'params' not in event:
            raise ValueError(f"Événement {i}: champ 'params' manquant")

    logger.info(f"Macro chargée : {len(data['events'])} événements")
    return data

# ============ EXÉCUTION DES ACTIONS ============
def execute_action(action, params, dry_run=False):
    """Exécution robuste d'une action avec fallbacks"""
    try:
        if action == 'click':
            x = params['x']
            y = params['y']
            button = params.get('button', 'left')
            if not dry_run:
                pyautogui.click(x=x, y=y, button=button)
            logger.debug(f"Click: ({x},{y}) button={button}")

        elif action == 'move_mouse':
            x = params['x']
            y = params['y']
            if not dry_run:
                pyautogui.moveTo(x, y)
            logger.debug(f"Move: ({x},{y})")

        elif action == 'press_key':
            key = params['key']
            if not dry_run:
                pyautogui.press(key)
            logger.debug(f"Press key: {key}")

        elif action == 'scroll':
            amount = params['amount']
            if not dry_run:
                pyautogui.scroll(amount)
            logger.debug(f"Scroll: {amount}")

        elif action == 'type':
            text = params['text']
            interval = params.get('interval', 0.0)
            if not dry_run:
                pyautogui.typewrite(text, interval=interval)
            logger.debug(f"Type: '{text}' interval={interval}")

        elif action == 'activate_window':
            title_sub = params['title_substring']
            wins = gw.getWindowsWithTitle(title_sub)
            if wins:
                if not dry_run:
                    wins[0].activate()
                logger.debug(f"Activate window: {title_sub}")
            else:
                logger.warning(f"Fenêtre non trouvée : '{title_sub}'")

        elif action == 'drag':
            start_x = params['start_x']
            start_y = params['start_y']
            end_x = params['end_x']
            end_y = params['end_y']
            duration = params.get('duration', 0.5)
            button = params.get('button', 'left')
            if not dry_run:
                pyautogui.moveTo(start_x, start_y)
                pyautogui.dragTo(end_x, end_y, duration=duration, button=button)
            logger.debug(f"Drag: ({start_x},{start_y})→({end_x},{end_y})")

        elif action == 'copy_to_clipboard':
            text = params['text']
            try:
                import pyperclip
                if not dry_run:
                    pyperclip.copy(text)
                logger.debug(f"Copy to clipboard: '{text[:50]}...'")
            except ImportError:
                logger.error("Module pyperclip non installé pour copy_to_clipboard")
                raise

        elif action == 'paste_from_clipboard':
            if not dry_run:
                pyautogui.hotkey('ctrl', 'v')
            logger.debug("Paste from clipboard")

        elif action == 'find_image':
            template_path = params['template_path']
            confidence = params.get('confidence', 0.9)
            if dry_run:
                logger.debug(f"Find image: {template_path} (confidence={confidence})")
                return {'status': 'ok', 'x': 0, 'y': 0, 'confidence': 1.0}
            try:
                import cv2
                import numpy as np
                template = cv2.imread(template_path, cv2.IMREAD_GRAYSCALE)
                if template is None:
                    raise ValueError(f"Image template introuvable : {template_path}")
                screen = pyautogui.screenshot()
                screen_np = np.array(screen)
                screen_gray = cv2.cvtColor(screen_np, cv2.COLOR_BGR2GRAY)
                res = cv2.matchTemplate(screen_gray, template, cv2.TM_CCOEFF_NORMED)
                _, max_val, _, max_loc = cv2.minMaxLoc(res)
                if max_val >= confidence:
                    h, w = template.shape
                    center_x = max_loc[0] + w // 2
                    center_y = max_loc[1] + h // 2
                    logger.debug(f"Image trouvée : ({center_x},{center_y}) conf={max_val:.3f}")
                    return {'status': 'ok', 'x': center_x, 'y': center_y, 'confidence': float(max_val)}
                else:
                    logger.debug(f"Image non trouvée (conf={max_val:.3f} < {confidence})")
                    return {'status': 'not_found', 'confidence': float(max_val)}
            except ImportError:
                logger.error("OpenCV non installé pour find_image")
                raise

        elif action == 'find_text_on_screen':
            text = params['text']
            lang = params.get('lang', 'fra')
            if dry_run:
                logger.debug(f"Find text: '{text}' lang={lang}")
                return {'status': 'ok', 'x': 0, 'y': 0}
            try:
                import pytesseract
                img = pyautogui.screenshot()
                data = pytesseract.image_to_data(img, lang=lang, output_type=pytesseract.Output.DICT)
                n = len(data['text'])
                for i in range(n):
                    if text.lower() in data['text'][i].lower():
                        x = data['left'][i] + data['width'][i] // 2
                        y = data['top'][i] + data['height'][i] // 2
                        logger.debug(f"Texte trouvé : '{data['text'][i]}' at ({x},{y})")
                        return {'status': 'ok', 'x': x, 'y': y, 'matched_text': data['text'][i]}
                logger.debug(f"Texte non trouvé : '{text}'")
                return {'status': 'not_found'}
            except ImportError:
                logger.error("pytesseract non installé pour find_text_on_screen")
                raise

        elif action == 'wait_for_image':
            template_path = params['template_path']
            timeout = params.get('timeout', 30.0)
            interval = params.get('interval', 1.0)
            confidence = params.get('confidence', 0.9)
            start = time.time()
            while time.time() - start < timeout:
                result = execute_action('find_image', {
                    'template_path': template_path,
                    'confidence': confidence
                }, dry_run=dry_run)
                if result.get('status') == 'ok':
                    logger.info(f"Image trouvée après {time.time() - start:.2f}s")
                    return result
                time.sleep(interval)
            logger.warning(f"Timeout : image non trouvée après {timeout}s")
            return {'status': 'timeout', 'message': 'Image not found within timeout'}

        else:
            logger.warning(f"Action inconnue ou non implémentée : {action}")
            return {'status': 'error', 'message': f'Unknown action: {action}'}

        return {'status': 'ok'}

    except KeyError as e:
        logger.error(f"Paramètre manquant pour action '{action}' : {e}")
        raise
    except Exception as e:
        logger.error(f"Erreur exécution action '{action}' : {e}")
        raise

# ============ LECTURE DE MACRO ============
def play_macro(macro_path, speed=1.0, dry_run=False):
    """
    Lecture d'une macro avec gestion robuste

    Args:
        macro_path: chemin vers le fichier .json
        speed: facteur de vitesse (1.0 = normal, 2.0 = 2x plus rapide)
        dry_run: mode simulation (vérifie la validité sans exécuter)
    """
    # Validation arguments
    if speed <= 0:
        raise ValueError("Le facteur de vitesse doit être > 0")

    # Chargement
    data = load_macro(macro_path)
    events = data.get('events', [])

    logger.info(f"Lecture macro : {len(events)} événements, speed={speed}x, dry_run={dry_run}")

    if dry_run:
        logger.info("Mode dry-run : vérification seulement")
        for i, event in enumerate(events):
            try:
                execute_action(event['action'], event.get('params', {}), dry_run=True)
            except Exception as e:
                logger.error(f"Événement {i} en erreur : {e}")
                raise
        logger.info("Dry-run terminé avec succès")
        return {'status': 'ok', 'dry_run': True}

    # Exécution réelle
    start_time = time.time()
    event_index = 0
    errors = []

    while event_index < len(events):
        now = time.time() - start_time
        # Chercher le prochain événement dont le timestamp / speed <= now
        for i in range(event_index, len(events)):
            target_time = events[i]['timestamp'] / speed
            if now >= target_time:
                event = events[i]
                action = event['action']
                params = event.get('params', {})

                try:
                    execute_action(action, params, dry_run=False)
                except Exception as e:
                    logger.error(f"Erreur événement {i} (t={event['timestamp']:.3f}s, action={action}) : {e}")
                    errors.append({'index': i, 'timestamp': event['timestamp'], 'action': action, 'error': str(e)})
                event_index = i + 1
                break
        else:
            # Pas d'événement prêt, petite pause
            time.sleep(0.001)

    total_time = time.time() - start_time
    logger.info(f"Macro terminée en {total_time:.2f}s (vitesse {speed}x) — {len(errors)} erreur(s)")

    if errors:
        return {'status': 'partial', 'errors': errors, 'total_events': len(events)}
    return {'status': 'ok', 'total_events': len(events), 'duration': total_time}

# ============ POINT D'ENTRÉE ============
def main():
    if len(sys.argv) < 2:
        print("Usage: play_macro.py <macro.json> [speed]")
        sys.exit(1)

    macro_path = sys.argv[1]
    speed = float(sys.argv[2]) if len(sys.argv) > 2 else 1.0

    try:
        result = play_macro(macro_path, speed=speed)
        if result['status'] == 'ok':
            print("✅ Macro terminée avec succès")
            sys.exit(0)
        elif result['status'] == 'partial':
            print(f"⚠️  Macro terminée avec {len(result['errors'])} erreur(s)")
            sys.exit(1)
        else:
            print("❌ Macro terminée avec erreurs")
            sys.exit(1)
    except FileNotFoundError as e:
        logger.error(str(e))
        print(f"❌ {e}")
        sys.exit(2)
    except ValueError as e:
        logger.error(str(e))
        print(f"❌ {e}")
        sys.exit(3)
    except Exception as e:
        logger.critical(f"Erreur fatale : {e}", exc_info=True)
        print(f"❌ Erreur fatale : {e}")
        sys.exit(1)

if __name__ == '__main__':
    main()


path: scripts/record_macro.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Macro Recorder GUI with Tkinter — Version ultra-robuste
Usage: python record_macro.py [output_path]

⚠️  PRIVACY WARNING: This recorder captures ALL keyboard events, including
passwords, credit card numbers, and other sensitive data. Use with extreme caution.
Only record macros for non-sensitive workflows. Never record while entering credentials.
"""
import tkinter as tk
from tkinter import filedialog, messagebox, scrolledtext, ttk
import json, os, time, threading, logging, sys
from datetime import datetime
from pynput import mouse, keyboard
from pynput.mouse import Button
from pynput.keyboard import Key

# ============ CONFIGURATION LOGGING ============
logging.basicConfig(
    level=logging.INFO,
    format='[%(levelname)s] %(asctime)s — %(message)s',
    handlers=[logging.StreamHandler(sys.stderr)]
)
logger = logging.getLogger(__name__)

# ============ HANDLER TKINTER POUR LOGS ============
class TkinterLogHandler(logging.Handler):
    """Handler qui écrit dans un widget ScrolledText de Tkinter"""
    def __init__(self, text_widget, max_lines=100):
        super().__init__()
        self.text_widget = text_widget
        self.max_lines = max_lines
        self.setFormatter(logging.Formatter('%(asctime)s — %(levelname)s: %(message)s', '%H:%M:%S'))

    def emit(self, record):
        try:
            if self.text_widget is None:
                return
            msg = self.format(record)
            # Utiliser after_idle pour éviter de bloquer l'UI
            def append_log():
                self.text_widget.configure(state='normal')
                self.text_widget.insert(tk.END, msg + '\n')
                self.text_widget.configure(state='disabled')
                self.text_widget.yview(tk.END)
                # Limiter le nombre de lignes
                line_count = int(self.text_widget.index('end-1c').split('.')[0])
                if line_count > self.max_lines:
                    self.text_widget.delete(1.0, 2.0)
            self.text_widget.after_idle(append_log)
        except Exception:
            self.handleError(record)

# ============ MACRO RECORDER CLASS (Thread-safe) ============
class MacroRecorder:
    def __init__(self):
        self.recording = False
        self.events = []
        self.start_time = None
        self.output_path = None
        self.mouse_listener = None
        self.keyboard_listener = None
        self.last_window = None
        self.window_check_interval = 0.5
        self.window_thread = None
        self.stop_event = threading.Event()
        self.lock = threading.RLock()
        self.window_import_success = False
        
        # Debounce pour mousemove
        self.debounce_seconds = 1.0  # Timer par défaut
        self.pending_move = None  # Position en attente
        self.debounce_timer = None  # Timer thread
        self.debounce_lock = threading.Lock()  # Lock pour le debounce

    def _safe_import_pygetwindow(self):
        """Import pygetwindow de manière safe et cache le résultat"""
        if not self.window_import_success:
            try:
                import pygetwindow as gw
                self.gw = gw
                self.window_import_success = True
                return gw
            except ImportError as e:
                logger.error(f"Impossible d'importer pygetwindow : {e}")
                self.gw = None
                return None
        return self.gw

    def _append_event_safe(self, event):
        """Ajout thread-safe d'un événement"""
        with self.lock:
            self.events.append(event)

    def start(self, output_path):
        """Démarrer l'enregistrement avec validation"""
        # Validation du chemin de sortie
        if not output_path:
            raise ValueError("Chemin de sortie obligatoire")

        output_dir = os.path.dirname(output_path) or os.getcwd()
        if not os.path.isabs(output_dir):
            output_dir = os.path.abspath(output_dir)

        # Vérifier/create le dossier
        try:
            os.makedirs(output_dir, exist_ok=True)
            test_file = os.path.join(output_dir, '.write_test')
            with open(test_file, 'w') as f:
                f.write('test')
            os.remove(test_file)
        except Exception as e:
            raise IOError(f"Dossier de sortie inaccessible ({output_dir}) : {e}")

        # Reset state
        self.output_path = output_path
        self.recording = True
        self.events = []
        self.start_time = time.time()
        self.stop_event.clear()

        # Import pygetwindow une première fois
        self._safe_import_pygetwindow()

        # Start listeners avec gestion d'erreur
        try:
            self.mouse_listener = mouse.Listener(
                on_move=self._on_move_wrapper,
                on_click=self._on_click_wrapper,
                on_scroll=self._on_scroll_wrapper
            )
            self.keyboard_listener = keyboard.Listener(
                on_press=self._on_press_wrapper,
                on_release=self._on_release_wrapper
            )
            self.mouse_listener.start()
            self.keyboard_listener.start()
        except Exception as e:
            logger.error(f"Erreur démarrage listeners : {e}")
            self.stop()
            raise

        # Start window monitor thread
        self.window_thread = threading.Thread(target=self._window_monitor, daemon=True)
        self.window_thread.start()

        logger.info(f"Enregistrement démarré → {output_path}")

    def _window_monitor(self):
        """Surveillance de changement de fenêtre (thread séparé)"""
        last_title = None
        logger.debug("Window monitor thread démarré")
        while not self.stop_event.is_set():
            try:
                gw = self._safe_import_pygetwindow()
                if gw:
                    win = gw.getActiveWindow()
                    if win and win.title and win.title != last_title:
                        event = {
                            "timestamp": time.time() - self.start_time,
                            "action": "activate_window",
                            "params": {"title_substring": win.title}
                        }
                        self._append_event_safe(event)
                        last_title = win.title
                        logger.debug(f"Window change détectée : {win.title}")
            except Exception as e:
                logger.warning(f"Erreur dans window monitor (non bloquant) : {e}")
            time.sleep(self.window_check_interval)
        logger.debug("Window monitor thread arrêté")

    def _on_move_wrapper(self, x, y):
        """Gère le mouvement de souris avec debounce (1s par défaut)"""
        if not self.recording:
            return
        
        # Coordonnées arrondies
        x_int = int(round(x))
        y_int = int(round(y))
        
        with self.debounce_lock:
            # Annuler le timer précédent s'il existe
            if self.debounce_timer and self.debounce_timer.is_alive():
                self.debounce_timer.cancel()
            
            # Stocker la position actuelle
            self.pending_move = (x_int, y_int, time.time() - self.start_time)
            
            # Démarrer un nouveau timer
            self.debounce_timer = threading.Timer(self.debounce_seconds, self._flush_move)
            self.debounce_timer.daemon = True
            self.debounce_timer.start()
    
    def _flush_move(self):
        """Appelé après le debounce : enregistre la position finale"""
        with self.debounce_lock:
            if self.pending_move:
                x, y, timestamp = self.pending_move
                self._append_event_safe({
                    "timestamp": timestamp,
                    "action": "move_mouse",
                    "params": {"x": x, "y": y}
                })
                self.pending_move = None

    def _on_click_wrapper(self, x, y, button, pressed):
        if not self.recording or not pressed:
            return
        btn = 'left' if button == Button.left else 'right' if button == Button.right else 'middle'
        self._append_event_safe({
            "timestamp": time.time() - self.start_time,
            "action": "click",
            "params": {"x": int(round(x)), "y": int(round(y)), "button": btn}
        })

    def _on_scroll_wrapper(self, x, y, dx, dy):
        if not self.recording:
            return
        amount = int(dy * 120)  # Unités PyAutoGUI
        self._append_event_safe({
            "timestamp": time.time() - self.start_time,
            "action": "scroll",
            "params": {"amount": amount}
        })

    def _on_press_wrapper(self, key):
        if not self.recording:
            return
        try:
            k = key.char
        except AttributeError:
            k = str(key).replace('Key.', '')
        self._append_event_safe({
            "timestamp": time.time() - self.start_time,
            "action": "press_key",
            "params": {"key": k}
        })

    def _on_release_wrapper(self, key):
        # Pas enregistré pour l'instant
        pass

    def stop(self, timeout=5.0):
        """Arrêt propre avec attente des threads"""
        if not self.recording:
            return

        logger.info("Arrêt de l'enregistrement...")
        self.recording = False
        self.stop_event.set()

        # Annuler le timer de debounce s'il est en cours
        with self.debounce_lock:
            if self.debounce_timer and self.debounce_timer.is_alive():
                self.debounce_timer.cancel()
            # Vider le pending move si présent
            if self.pending_move:
                x, y, timestamp = self.pending_move
                self._append_event_safe({
                    "timestamp": timestamp,
                    "action": "move_mouse",
                    "params": {"x": x, "y": y}
                })
                self.pending_move = None

        # Arrêter les listeners
        if self.mouse_listener:
            self.mouse_listener.stop()
        if self.keyboard_listener:
            self.keyboard_listener.stop()

        # Attendre le thread window
        if self.window_thread and self.window_thread.is_alive():
            self.window_thread.join(timeout=timeout)
            if self.window_thread.is_alive():
                logger.warning("Window monitor thread toujours vivant après timeout")

        # Calcul duration
        duration = time.time() - self.start_time if self.start_time else 0
        logger.info(f"Enregistrement arrêté — {len(self.events)} événements, durée: {duration:.2f}s")

    def save(self):
        """Sauvegarde atomique des événements"""
        if not self.output_path:
            raise RuntimeError("Aucun chemin de sortie défini")

        # Préparer données
        meta = {
            "duration": time.time() - self.start_time if self.start_time else 0,
            "recorded_at": datetime.now().isoformat(),
            "event_count": len(self.events)
        }

        # Copie thread-safe des événements
        with self.lock:
            events_copy = list(self.events)

        data = {"meta": meta, "events": events_copy}

        # Écriture atomique
        temp_path = self.output_path + '.tmp'
        try:
            with open(temp_path, 'w', encoding='utf-8') as f:
                json.dump(data, f, indent=2, ensure_ascii=False)
            # Renommage atomique (POSIX rename est atomique)
            os.replace(temp_path, self.output_path)
            logger.info(f"Macro sauvegardée : {self.output_path}")
            return self.output_path
        except Exception as e:
            # Nettoyage du fichier temporaire
            if os.path.exists(temp_path):
                try:
                    os.remove(temp_path)
                except:
                    pass
            raise IOError(f"Échec sauvegarde : {e}")

# ============ GUI TKINTER ============
class RecorderGUI:
    def __init__(self):
        self.recorder = MacroRecorder()
        self.output_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'recorded_macro'))
        os.makedirs(self.output_dir, exist_ok=True)

        self.root = tk.Tk()
        self.root.title("Macro Recorder — Ultra Robuste")
        self.root.geometry("420x350")
        self.root.resizable(False, False)

        self.status_label = None
        self.start_button = None
        self.stop_button = None
        self.folder_label = None
        self.events_tree = None  # Treeview pour les événements
        self.debounce_var = None  # Variable Tkinter pour le debounce

        self._build_ui()

    def _build_ui(self):
        """Construction de l'interface"""
        pad = 10

        # Boutons principaux
        button_frame = tk.Frame(self.root)
        button_frame.pack(pady=pad)

        self.start_button = tk.Button(
            button_frame,
            text="🚀 Démarrer l'enregistrement",
            command=self._start_recording,
            width=30,
            bg='#4CAF50',
            fg='white',
            font=('Segoe UI', 10, 'bold')
        )
        self.start_button.pack(pady=5)

        self.stop_button = tk.Button(
            button_frame,
            text="🛑 Arrêter",
            command=self._stop_recording,
            width=30,
            bg='#f44336',
            fg='white',
            font=('Segoe UI', 10, 'bold'),
            state='disabled'
        )
        self.stop_button.pack(pady=5)

        tk.Button(
            button_frame,
            text="📁 Choisir dossier",
            command=self._choose_folder,
            width=30
        ).pack(pady=5)

        # Configuration du debounce
        debounce_frame = tk.Frame(self.root)
        debounce_frame.pack(pady=5)

        tk.Label(
            debounce_frame,
            text="Débounce mousemove (sec) :",
            font=('Segoe UI', 9)
        ).pack(side=tk.LEFT, padx=5)

        self.debounce_var = tk.StringVar(value="1.0")
        debounce_entry = tk.Entry(
            debounce_frame,
            textvariable=self.debounce_var,
            width=6,
            justify='center'
        )
        debounce_entry.pack(side=tk.LEFT, padx=5)

        tk.Button(
            debounce_frame,
            text="Appliquer",
            command=self._apply_debounce,
            bg='#2196F3',
            fg='white',
            font=('Segoe UI', 9, 'bold')
        ).pack(side=tk.LEFT, padx=5)

        # Dossier
        self.folder_label = tk.Label(
            self.root,
            text=self.output_dir,
            wraplength=350,
            fg='#666'
        )
        self.folder_label.pack(pady=5)

        # Statut
        self.status_label = tk.Label(
            self.root,
            text="✅ Prêt",
            fg='#2E7D32',
            font=('Segoe UI', 9)
        )
        self.status_label.pack(pady=5)

        # Zone de logs (légère, avec limitation automatique)
        log_frame = tk.LabelFrame(self.root, text="Logs en direct", padx=5, pady=5)
        log_frame.pack(fill=tk.BOTH, expand=True, padx=10, pady=10)

        self.log_text = scrolledtext.ScrolledText(
            log_frame,
            height=8,
            width=50,
            state='disabled',
            wrap=tk.WORD,
            font=('Consolas', 8)
        )
        self.log_text.pack(fill=tk.BOTH, expand=True)

        # Cadre pour les événements enregistrés (affichage temps réel)
        events_frame = tk.LabelFrame(self.root, text="Événements enregistrés (temps réel)", padx=5, pady=5)
        events_frame.pack(fill=tk.BOTH, expand=True, padx=10, pady=10)

        # Treeview pour afficher les événements
        self.events_tree = ttk.Treeview(
            events_frame,
            columns=("temps", "action", "details"),
            show='headings',
            height=6
        )
        self.events_tree.heading("temps", text="Temps (s)")
        self.events_tree.heading("action", text="Action")
        self.events_tree.heading("details", text="Détails")
        self.events_tree.column("temps", width=60, minwidth=50)
        self.events_tree.column("action", width=80, minwidth=70)
        self.events_tree.column("details", width=200, minwidth=150)

        # Scrollbar pour le Treeview
        events_scroll = ttk.Scrollbar(events_frame, orient=tk.VERTICAL, command=self.events_tree.yview)
        self.events_tree.configure(yscrollcommand=events_scroll.set)

        self.events_tree.pack(side=tk.LEFT, fill=tk.BOTH, expand=True)
        events_scroll.pack(side=tk.RIGHT, fill=tk.Y)

        # Info label
        tk.Label(
            self.root,
            text="⚠️  PRIVACY: records ALL keystrokes (including passwords). Use with caution.",
            wraplength=350,
            fg='#D32F2F',
            font=('Segoe UI', 8, 'bold')
        ).pack(pady=5)

        tk.Label(
            self.root,
            text="Enregistre : souris, clic, défilement, clavier, changements de fenêtre",
            wraplength=350,
            fg='#666',
            font=('Segoe UI', 8)
        ).pack(pady=5)

        # Attacher le handler de logging
        self._attach_log_handler()

    def _attach_log_handler(self):
        """Attache le handler Tkinter au logger pour afficher les logs dans l'UI"""
        handler = TkinterLogHandler(self.log_text, max_lines=100)
        logger.addHandler(handler)

    def _update_events_list(self):
        """Met à jour la liste des événements enregistrés en temps réel"""
        if self.events_tree is None:
            return
        
        # Effacer les anciennes entrées
        self.events_tree.delete(*self.events_tree.get_children())
        
        # Copie thread-safe des événements
        with self.recorder.lock:
            events = list(self.recorder.events)
        
        # Afficher les 100 derniers événements
        for event in events[-100:]:
            t = event.get('timestamp', 0)
            action = event.get('action', '?')
            params = event.get('params', {})
            
            # Formater les détails selon l'action
            if action == 'click':
                details = f"({params.get('x')},{params.get('y')}) [{params.get('button')}]"
            elif action == 'press_key':
                details = f"'{params.get('key')}'"
            elif action == 'type':
                text = params.get('text', '')
                details = f"'{text[:30]}{'...' if len(text)>30 else ''}'"
            elif action == 'move_mouse':
                details = f"({params.get('x')},{params.get('y')})"
            elif action == 'scroll':
                details = f"{params.get('amount')}"
            elif action == 'drag':
                start = params.get('start', (0,0))
                end = params.get('end', (0,0))
                details = f"({start})→({end})"
            elif action == 'activate_window':
                title = params.get('title_substring', '')
                details = f"'{title[:30]}{'...' if len(title)>30 else ''}'"
            else:
                details = str(params)[:40]
            
            self.events_tree.insert("", tk.END, values=(f"{t:.3f}", action, details))
        
        # Programmer la prochaine mise à jour si enregistrement en cours
        if self.recorder.recording:
            self.root.after(250, self._update_events_list)

    def _start_recording(self):
        """Callback bouton Démarrer"""
        try:
            timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
            filename = f"macro_{timestamp}.json"
            full_path = os.path.join(self.output_dir, filename)

            self.recorder.start(full_path)
            self.status_label.config(text="🔴 Enregistrement en cours...", fg='#D32F2F')
            self.start_button.config(state='disabled')
            self.stop_button.config(state='normal')
            # Démarrer l'affichage des événements en temps réel
            self._update_events_list()
        except Exception as e:
            logger.error(f"Échec démarrage : {e}")
            messagebox.showerror("Erreur", f"Impossible de démarrer : {e}")

    def _stop_recording(self):
        """Callback bouton Arrêter"""
        try:
            self.recorder.stop()
            saved_path = self.recorder.save()
            messagebox.showinfo("✅ Enregistrement terminé", f"Macro sauvegardée :\n{saved_path}")
            self.status_label.config(text="✅ Prêt", fg='#2E7D32')
            self.start_button.config(state='normal')
            self.stop_button.config(state='disabled')
        except Exception as e:
            logger.error(f"Échec arrêt/sauvegarde : {e}")
            messagebox.showerror("Erreur", f"Impossible de sauvegarder : {e}")
            self.status_label.config(text="❌ Erreur", fg='#C62828')
            self.start_button.config(state='normal')
            self.stop_button.config(state='disabled')

    def _choose_folder(self):
        """Choix du dossier de sortie"""
        folder = filedialog.askdirectory(initialdir=self.output_dir)
        if folder:
            self.output_dir = folder
            self.folder_label.config(text=folder)

    def _apply_debounce(self):
        """Applique le paramètre de debounce depuis l'Entry"""
        try:
            value = float(self.debounce_var.get())
            if value < 0.1 or value > 10.0:
                raise ValueError("Entre 0.1 et 10 secondes")
            self.recorder.debounce_seconds = value
            logger.info(f"Débounce configuré à {value}s")
        except ValueError as e:
            messagebox.showerror("Valeur invalide", f"Le debounce doit être un nombre entre 0.1 et 10 secondes.\nErreur: {e}")

    def run(self):
        """Lancement de la boucle Tkinter"""
        self.root.mainloop()

# ============ POINT D'ENTRÉE ============
if __name__ == '__main__':
    try:
        app = RecorderGUI()
        app.run()
    except Exception as e:
        logger.critical(f"Erreur fatale : {e}")
        messagebox.showerror("Erreur fatale", str(e))
        sys.exit(1)


path: scripts/test_automation.py
content: """
Test Automation Script

Unit tests for desktop automation modules.
"""

import sys
import os
import json
import unittest
from pathlib import Path

# Add lib to path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))

from lib.actions import ActionManager
from lib.safety_manager import SafetyManager
from lib.macro_player import MacroPlayer
from lib.image_recognition import ImageRecognition
from lib.ocr_engine import OCREngine
from lib.utils import setup_logging


class TestActionManager(unittest.TestCase):
    """Test ActionManager."""
    
    def setUp(self):
        self.manager = ActionManager(safe_mode=True)
    
    def test_dry_run_click(self):
        """Test click in dry_run mode."""
        result = self.manager.click(100, 100, dry_run=True)
        self.assertEqual(result['status'], 'ok')
        self.assertTrue(result['dry_run'])
    
    def test_safe_mode_blocks_dangerous(self):
        """Test that safe mode blocks dangerous patterns."""
        result = self.manager.click(100, 100)
        self.assertIn(result['status'], ['ok', 'blocked'])
    
    def test_dry_run_type(self):
        """Test type in dry_run mode."""
        result = self.manager.type("test", dry_run=True)
        self.assertEqual(result['status'], 'ok')
        self.assertTrue(result['dry_run'])
    
    def test_get_active_window(self):
        """Test getting active window."""
        result = self.manager.get_active_window()
        self.assertIn(result['status'], ['ok', 'error'])


class TestSafetyManager(unittest.TestCase):
    """Test SafetyManager."""
    
    def setUp(self):
        self.manager = SafetyManager(safe_mode_enabled=True)
    
    def test_safe_mode_enabled(self):
        """Test safe mode is enabled."""
        self.assertTrue(self.manager.is_safe_mode_enabled())
    
    def test_dangerous_pattern_detection(self):
        """Test dangerous pattern detection."""
        self.assertTrue(self.manager.contains_dangerous_pattern("rm /etc/"))
        self.assertFalse(self.manager.contains_dangerous_pattern("echo hello"))
    
    def test_check_action_safety(self):
        """Test action safety checking."""
        safe, reason = self.manager.check_action_safety("click", {"x": 100, "y": 100})
        self.assertTrue(safe)
    
    def test_disable_safe_mode(self):
        """Test disabling safe mode."""
        self.manager.disable_safe_mode()
        self.assertFalse(self.manager.is_safe_mode_enabled())
    
    def test_get_safety_status(self):
        """Test getting safety status."""
        status = self.manager.get_safety_status()
        self.assertEqual(status['status'], 'ok')
        self.assertIn('safe_mode_enabled', status)


class TestMacroPlayer(unittest.TestCase):
    """Test MacroPlayer."""
    
    def setUp(self):
        self.action_manager = ActionManager(safe_mode=True)
        self.player = MacroPlayer(self.action_manager)
    
    def test_load_nonexistent_macro(self):
        """Test loading non-existent macro."""
        macro = self.player.load_macro("/nonexistent/path.json")
        self.assertIsNone(macro)
    
    def test_create_and_load_macro(self):
        """Test creating and loading a test macro."""
        test_macro = {
            "events": [
                {"action": "click", "params": {"x": 100, "y": 100}},
                {"action": "type", "params": {"text": "hello"}}
            ]
        }
        
        # Save test macro
        import tempfile
        with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
            json.dump(test_macro, f)
            temp_path = f.name
        
        try:
            # Load it back
            loaded = self.player.load_macro(temp_path)
            self.assertIsNotNone(loaded)
            self.assertEqual(len(loaded['events']), 2)
        finally:
            os.unlink(temp_path)


class TestImageRecognition(unittest.TestCase):
    """Test ImageRecognition."""
    
    def setUp(self):
        self.recognizer = ImageRecognition()
    
    def test_find_nonexistent_image(self):
        """Test finding non-existent image."""
        result = self.recognizer.find_image("/nonexistent/image.png", dry_run=True)
        self.assertIn(result['status'], ['ok', 'error'])
    
    def test_dry_run_find_image(self):
        """Test find_image in dry_run mode."""
        result = self.recognizer.find_image("test.png", dry_run=True)
        self.assertIn(result['status'], ['ok', 'error'])


class TestOCREngine(unittest.TestCase):
    """Test OCREngine."""
    
    def setUp(self):
        self.ocr = OCREngine()
    
    def test_dry_run_find_text(self):
        """Test find_text in dry_run mode."""
        result = self.ocr.find_text_on_screen("test", dry_run=True)
        self.assertIn(result['status'], ['ok', 'error'])


def main():
    """Run all tests."""
    setup_logging()
    
    # Create test suite
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    
    suite.addTests(loader.loadTestsFromTestCase(TestActionManager))
    suite.addTests(loader.loadTestsFromTestCase(TestSafetyManager))
    suite.addTests(loader.loadTestsFromTestCase(TestMacroPlayer))
    suite.addTests(loader.loadTestsFromTestCase(TestImageRecognition))
    suite.addTests(loader.loadTestsFromTestCase(TestOCREngine))
    
    # Run tests
    runner = unittest.TextTestRunner(verbosity=2)
    result = runner.run(suite)
    
    # Return exit code
    return 0 if result.wasSuccessful() else 1


if __name__ == '__main__':
    exit(main())


path: lib/actions.py
content: """
Desktop Automation Actions Module

Core actions for mouse, keyboard, and window control.
Thread-safe, error handling, logging integrated.
"""

import pyautogui
import pygetwindow as gw
import time
import threading
import logging
from typing import Dict, Tuple, Optional, Any, List
from PIL import Image

logger = logging.getLogger(__name__)


class ActionManager:
    """Manages desktop automation actions with safety and logging."""
    
    def __init__(self, safe_mode: bool = True):
        self.safe_mode = safe_mode
        self.lock = threading.Lock()
        pyautogui.FAILSAFE = True
        logger.info("ActionManager initialized with safe_mode=%s", self.safe_mode)
    
    def set_safe_mode(self, enabled: bool) -> Dict[str, Any]:
        """Enable or disable safe mode."""
        with self.lock:
            self.safe_mode = enabled
            logger.info("Safe mode set to %s", enabled)
            return {"status": "ok", "safe_mode": self.safe_mode}
    
    def _check_safe(self, action: str, params: Dict) -> bool:
        """Check if action is safe to execute."""
        if not self.safe_mode:
            return True
        
        dangerous_actions = {"type", "press_key", "click", "drag"}
        dangerous_patterns = ["rm ", "del ", "C:\\Windows\\", "/etc/", "sudo"]
        
        if action not in dangerous_actions:
            return True
        
        # Check parameters for dangerous patterns
        for key, value in params.items():
            if isinstance(value, str):
                for pattern in dangerous_patterns:
                    if pattern.lower() in value.lower():
                        logger.warning("Safe mode blocked action %s: dangerous pattern '%s'", action, pattern)
                        return False
        
        return True
    
    def click(self, x: int, y: int, button: str = "left", dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Click at coordinates."""
        params = {"x": x, "y": y, "button": button}
        
        if not self._check_safe("click", params):
            return {"status": "blocked", "reason": "Safe mode active"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] click at (%d, %d) button=%s", x, y, button)
                return {"status": "ok", "dry_run": True, "x": x, "y": y}
            
            with self.lock:
                pyautogui.click(x, y, button=button)
                logger.info("Clicked at (%d, %d) with %s button", x, y, button)
            return {"status": "ok", "x": x, "y": y}
        except Exception as e:
            logger.error("click failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def type(self, text: str, interval: float = 0.05, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Type text."""
        params = {"text": text}
        
        if not self._check_safe("type", params):
            return {"status": "blocked", "reason": "Safe mode active"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] type '%s' with interval %.2f", text, interval)
                return {"status": "ok", "dry_run": True, "text": text}
            
            with self.lock:
                pyautogui.typewrite(text, interval=interval)
                logger.info("Typed: %s", text)
            return {"status": "ok", "text": text}
        except Exception as e:
            logger.error("type failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def press_key(self, key: str, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Press a single key."""
        params = {"key": key}
        
        if not self._check_safe("press_key", params):
            return {"status": "blocked", "reason": "Safe mode active"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] press key '%s'", key)
                return {"status": "ok", "dry_run": True, "key": key}
            
            with self.lock:
                pyautogui.press(key)
                logger.info("Pressed key: %s", key)
            return {"status": "ok", "key": key}
        except Exception as e:
            logger.error("press_key failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def move_mouse(self, x: int, y: int, duration: float = 0.5, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Move mouse to coordinates."""
        try:
            if dry_run:
                logger.info("[DRY RUN] move mouse to (%d, %d)", x, y)
                return {"status": "ok", "dry_run": True, "x": x, "y": y}
            
            with self.lock:
                pyautogui.moveTo(x, y, duration=duration)
                logger.info("Moved mouse to (%d, %d)", x, y)
            return {"status": "ok", "x": x, "y": y}
        except Exception as e:
            logger.error("move_mouse failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def scroll(self, amount: int = 5, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Scroll up/down."""
        try:
            if dry_run:
                logger.info("[DRY RUN] scroll %d", amount)
                return {"status": "ok", "dry_run": True, "amount": amount}
            
            with self.lock:
                pyautogui.scroll(amount)
                logger.info("Scrolled %d", amount)
            return {"status": "ok", "amount": amount}
        except Exception as e:
            logger.error("scroll failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def drag(self, start_x: int, start_y: int, end_x: int, end_y: int, 
             duration: float = 0.5, button: str = "left", dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Drag from one position to another."""
        params = {"start": (start_x, start_y), "end": (end_x, end_y)}
        
        if not self._check_safe("drag", params):
            return {"status": "blocked", "reason": "Safe mode active"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] drag from (%d,%d) to (%d,%d)", start_x, start_y, end_x, end_y)
                return {"status": "ok", "dry_run": True}
            
            with self.lock:
                pyautogui.drag(end_x - start_x, end_y - start_y, duration=duration, button=button)
                logger.info("Dragged from (%d,%d) to (%d,%d)", start_x, start_y, end_x, end_y)
            return {"status": "ok"}
        except Exception as e:
            logger.error("drag failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def screenshot(self, path: str = "~/Desktop/screenshot.png", dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Take a screenshot."""
        try:
            if dry_run:
                logger.info("[DRY RUN] screenshot to %s", path)
                return {"status": "ok", "dry_run": True, "path": path}
            
            import os
            full_path = os.path.expanduser(path)
            os.makedirs(os.path.dirname(full_path), exist_ok=True)
            
            with self.lock:
                screenshot = pyautogui.screenshot()
                screenshot.save(full_path)
                logger.info("Screenshot saved to %s", full_path)
            return {"status": "ok", "path": full_path}
        except Exception as e:
            logger.error("screenshot failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def get_active_window(self, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Get active window info."""
        try:
            with self.lock:
                active = gw.getActiveWindow()
                if active:
                    result = {
                        "status": "ok",
                        "title": active.title,
                        "x": active.left,
                        "y": active.top,
                        "width": active.width,
                        "height": active.height
                    }
                    logger.info("Active window: %s", active.title)
                    return result
                return {"status": "error", "message": "No active window"}
        except Exception as e:
            logger.error("get_active_window failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def list_windows(self, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """List all windows."""
        try:
            with self.lock:
                windows = []
                for w in gw.getAllWindows():
                    windows.append({
                        "title": w.title,
                        "x": w.left,
                        "y": w.top,
                        "width": w.width,
                        "height": w.height,
                        "is_active": w == gw.getActiveWindow()
                    })
                logger.info("Listed %d windows", len(windows))
                return {"status": "ok", "windows": windows, "count": len(windows)}
        except Exception as e:
            logger.error("list_windows failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def activate_window(self, title_substring: str, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Activate window by title substring."""
        try:
            if dry_run:
                logger.info("[DRY RUN] activate window containing '%s'", title_substring)
                return {"status": "ok", "dry_run": True}
            
            with self.lock:
                for w in gw.getAllWindows():
                    if title_substring.lower() in w.title.lower():
                        w.activate()
                        logger.info("Activated window: %s", w.title)
                        return {"status": "ok", "title": w.title}
                return {"status": "error", "message": f"Window '{title_substring}' not found"}
        except Exception as e:
            logger.error("activate_window failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def copy_to_clipboard(self, text: str, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Copy text to clipboard."""
        try:
            if dry_run:
                logger.info("[DRY RUN] copy to clipboard: %s", text[:50])
                return {"status": "ok", "dry_run": True}
            
            try:
                import pyperclip
                with self.lock:
                    pyperclip.copy(text)
                    logger.info("Copied to clipboard (%d chars)", len(text))
                return {"status": "ok"}
            except ImportError:
                return {"status": "error", "message": "pyperclip not installed"}
        except Exception as e:
            logger.error("copy_to_clipboard failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def paste_from_clipboard(self, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Paste from clipboard."""
        try:
            try:
                import pyperclip
                with self.lock:
                    text = pyperclip.paste()
                    pyautogui.typewrite(text, interval=0.01)
                    logger.info("Pasted from clipboard (%d chars)", len(text))
                return {"status": "ok", "length": len(text)}
            except ImportError:
                return {"status": "error", "message": "pyperclip not installed"}
        except Exception as e:
            logger.error("paste_from_clipboard failed: %s", str(e))
            return {"status": "error", "message": str(e)}


path: lib/advanced_automation.py
content: #!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Advanced Automation Features — Do not modify existing basic functions.
These are additive, more powerful features with explicit names.
"""
import os, time, json, logging
import pyautogui
import cv2
import numpy as np
import pytesseract

logger = logging.getLogger(__name__)

# ============================================
# 1. MULTI-SCALE IMAGE DETECTION
# ============================================

def find_image_multiscale(template_path, confidence=0.9, scale_factors=None):
    """
    Trouve une image en testant plusieurs échelles (pyramide).

    Args:
        template_path: chemin vers le template
        confidence: seuil minimum (0.0-1.0)
        scale_factors: liste de facteurs, ex: [0.5, 0.75, 1.0, 1.25, 1.5]
                      Si None, default: [0.5, 0.75, 1.0, 1.25, 1.5]

    Returns:
        {"status":"ok", "x":int, "y":int, "confidence":float, "scale":float}
        ou {"status":"not_found", "best_confidence":float}
    """
    if scale_factors is None:
        scale_factors = [0.5, 0.75, 1.0, 1.25, 1.5]

    template = cv2.imread(template_path, cv2.IMREAD_GRAYSCALE)
    if template is None:
        return {"status": "error", "message": f"Template not found: {template_path}"}

    screen = pyautogui.screenshot()
    screen_np = np.array(screen)
    screen_gray = cv2.cvtColor(screen_np, cv2.COLOR_BGR2GRAY)

    best_match = None
    best_conf = -1

    for scale in scale_factors:
        # Redimensionner le template
        new_w = int(template.shape[1] * scale)
        new_h = int(template.shape[0] * scale)
        if new_w < 5 or new_h < 5:
            continue
        resized = cv2.resize(template, (new_w, new_h), interpolation=cv2.INTER_LINEAR)

        # Matching
        res = cv2.matchTemplate(screen_gray, resized, cv2.TM_CCOEFF_NORMED)
        _, max_val, _, max_loc = cv2.minMaxLoc(res)

        if max_val > best_conf:
            best_conf = max_val
            best_match = {
                'x': max_loc[0] + new_w // 2,
                'y': max_loc[1] + new_h // 2,
                'scale': scale
            }

    if best_conf >= confidence and best_match:
        logger.debug(f"Multi-scale match: ({best_match['x']},{best_match['y']}) scale={best_match['scale']} conf={best_conf:.3f}")
        return {"status": "ok", "x": best_match['x'], "y": best_match['y'], "confidence": float(best_conf), "scale": best_match['scale']}
    else:
        logger.debug(f"Multi-scale image not found (best conf={best_conf:.3f})")
        return {"status": "not_found", "best_confidence": float(best_conf)}

# ============================================
# 2. FIND ALL TEXT OCCURRENCES
# ============================================

def find_all_text_on_screen(text, lang='fra'):
    """
    Retourne TOUTES les occurrences du texte à l'écran, pas seulement la première.

    Args:
        text: chaîne à rechercher (ne sera pas sensible à la casse)
        lang: langue Tesseract (défaut 'fra')

    Returns:
        {"status":"ok", "matches": [{"text":str, "x":int, "y":int, "w":int, "h":int, "conf":float}, ...]}
    """
    if pytesseract is None:
        return {"status": "error", "message": "pytesseract not installed"}

    try:
        img = pyautogui.screenshot()
        data = pytesseract.image_to_data(img, lang=lang, output_type=pytesseract.Output.DICT)

        matches = []
        n = len(data['text'])
        for i in range(n):
            word = data['text'][i].strip()
            if word and text.lower() in word.lower():
                matches.append({
                    'text': word,
                    'left': int(data['left'][i]),
                    'top': int(data['top'][i]),
                    'width': int(data['width'][i]),
                    'height': int(data['height'][i]),
                    'conf': float(data['conf'][i])
                })

        logger.debug(f"Found {len(matches)} occurrences of '{text}'")
        return {"status": "ok", "matches": matches, "count": len(matches)}
    except Exception as e:
        logger.error(f"find_all_text_on_screen error: {e}")
        return {"status": "error", "message": str(e)}

# ============================================
# 3. UI ELEMENTS DETECTION
# ============================================

def detect_ui_elements(element_type=None):
    """
    Détecte des éléments UI courants par reconnaissance d'image / heuristiques.

    Args:
        element_type: 'button', 'field', 'slider', ou None pour tout détecter

    Returns:
        {"status":"ok", "elements": [{"type":..., "x":..., "y":..., "w":..., "h":...}, ...]}
    """
    # Pour l'instant, une détection basique par formes/contours (à améliorer)
    try:
        screen = pyautogui.screenshot()
        screen_np = np.array(screen)
        gray = cv2.cvtColor(screen_np, cv2.COLOR_BGR2GRAY)
        edges = cv2.Canny(gray, 50, 150)

        contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

        elements = []
        for cnt in contours:
            x, y, w, h = cv2.boundingRect(cnt)
            area = w * h
            # Filtrer par taille (éviter bruit)
            if area < 100 or area > 10000:
                continue
            # Heuristique simple : ratio width/height pour deviner le type
            ratio = w / max(h, 1)
            etype = 'unknown'
            if 2.5 < ratio < 6 and h < 40:
                etype = 'button'
            elif 1.5 < ratio < 5 and h > 30:
                etype = 'field'
            elif 0.8 < ratio < 1.2 and 30 < h < 200:
                etype = 'slider'

            if element_type is None or etype == element_type:
                elements.append({
                    'type': etype,
                    'x': x,
                    'y': y,
                    'w': w,
                    'h': h,
                    'area': area
                })

        logger.info(f"Detected {len(elements)} UI elements")
        return {"status": "ok", "elements": elements, "count": len(elements)}
    except Exception as e:
        logger.error(f"detect_ui_elements error: {e}")
        return {"status": "error", "message": str(e)}

# ============================================
# 4. CONDITIONAL MONITOR WITH LOGIC
# ============================================

def monitor_screen_with_logic(conditions, timeout=60, interval=0.5):
    """
    Surveillance avancée avec logique combinatoire (AND/OR).

    Args:
        conditions: liste de conditions, chaque condition:
            {
                "type": "image" ou "text",
                "template_path": "...",
                "text": "...",
                "confidence": 0.9,
                "lang": "fra",
                "logic": "AND" | "OR",  # pour combiner plusieurs conditions dans un groupe
                "actions": [ {"action": "...", "params": {...}}, ... ]
            }
        timeout: secondes max
        interval: délai entre vérifications

    Exemple:
    conditions=[
        {"type":"image", "template_path":"ok.png", "confidence":0.9, "logic":"AND",
         "actions": [{"action":"click", "params":{...}}]},
        {"type":"text", "text":"Success", "logic":"OR", "actions":[...]}
    ]

    Retourne les actions exécutées et le timing.
    """
    start_time = time.time()
    executed_actions = []
    satisfied_groups = set()

    # Grouper par index (chaque condition est un groupe pour l'instant)
    while time.time() - start_time < timeout:
        for idx, cond in enumerate(conditions):
            # Vérifier la condition
            cond_ok = False
            if cond.get('type') == 'image':
                res = find_image_multiscale(cond['template_path'], cond.get('confidence', 0.9))
                cond_ok = res.get('status') == 'ok'
            elif cond.get('type') == 'text':
                res = find_all_text_on_screen(cond['text'], cond.get('lang', 'fra'))
                cond_ok = res.get('status') == 'ok' and res.get('count', 0) > 0
            else:
                logger.warning(f"Unknown condition type: {cond.get('type')}")
                continue

            # Si condition vérifiée, exécuter les actions
            if cond_ok:
                for act in cond.get('actions', []):
                    action_name = act['action']
                    params = act.get('params', {}).copy()
                    # Injecter coordonnées si présentes dans la détection
                    if 'x' in params and 'y' in params and 'location' in res:
                        params['x'] = res['location']['x']
                        params['y'] = res['location']['y']
                    # Exécuter l'action (seulement actions de base)
                    safe_actions = {'click', 'type', 'press_key', 'scroll', 'move_mouse', 'activate_window', 'drag', 'copy_to_clipboard', 'paste_from_clipboard'}
                    if action_name in safe_actions and action_name in globals():
                        try:
                            result = globals()[action_name](**params)
                            executed_actions.append({
                                'condition_index': idx,
                                'action': action_name,
                                'result': result,
                                'timestamp': time.time() - start_time
                            })
                            logger.info(f"Condition {idx} satisfied, executed {action_name}")
                        except Exception as e:
                            logger.error(f"Action {action_name} failed: {e}")
                satisfied_groups.add(idx)

        # Si toutes les conditions ont été satisfaites au moins une fois, on peut arrêter
        if len(satisfied_groups) == len(conditions):
            logger.info("All conditions satisfied, stopping monitor")
            break

        time.sleep(interval)

    total_elapsed = time.time() - start_time
    return {
        "status": "ok" if total_elapsed < timeout else "timeout",
        "executed_actions": executed_actions,
        "satisfied_conditions": list(satisfied_groups),
        "elapsed": total_elapsed
    }

# ============================================
# 5. PLAY MACRO WITH SUBROUTINES
# ============================================

def play_macro_with_subroutines(macro_path, speed=1.0, sub_macros_dir=None):
    """
    Lecture de macro avec support de sous-macros.

    Format de macro attendu:
    {
      "meta": {...},
      "events": [
        {"action": "call_macro", "params": {"macro_file": "submacro1.json"}},
        {"action": "type", "params": {"text": "Hello"}},
        ...
      ]
    }

    Args:
        macro_path: chemin macro principale
        speed: facteur vitesse
        sub_macros_dir: dossier où chercher les sous-macros (défaut: même dossier que macro_path)
    """
    if sub_macros_dir is None:
        sub_macros_dir = os.path.dirname(macro_path) or os.getcwd()

    # Charger la macro principale
    try:
        with open(macro_path, 'r', encoding='utf-8') as f:
            macro_data = json.load(f)
        events = macro_data.get('events', [])
    except Exception as e:
        return {"status": "error", "message": f"Failed to load macro: {e}"}

    start_time = time.time()
    event_index = 0
    sub_macro_recursion = {}

    while event_index < len(events):
        now = time.time() - start_time
        # Chercher le prochain événement dont timestamp / speed <= now
        for i in range(event_index, len(events)):
            target_time = events[i]['timestamp'] / speed
            if now >= target_time:
                event = events[i]
                action = event.get('action')
                params = event.get('params', {}).copy()

                # Gestion spéciale de call_macro
                if action == 'call_macro':
                    sub_macro_file = params.get('macro_file')
                    sub_macro_path = os.path.join(sub_macros_dir, sub_macro_file)
                    # Charger et jouer la sous-macro (sans call_macro imbriqué infini)
                    if sub_macro_path in sub_macro_recursion and sub_macro_recursion[sub_macro_path] > 5:
                        logger.error(f"Recursion limit exceeded for {sub_macro_file}")
                        event_index = i + 1
                        break
                    sub_macro_recursion[sub_macro_file] = sub_macro_recursion.get(sub_macro_file, 0) + 1
                    # Lecture directe (on pourrait factoriser avec un appel interne)
                    try:
                        with open(sub_macro_path, 'r', encoding='utf-8') as sf:
                            sub_data = json.load(sf)
                        sub_events = sub_data.get('events', [])
                        # Insérer les événements de la sous-macro dans le flux
                        # On décale les timestamps pour qu'ils s'insèrent naturellement
                        base_ts = now * speed  # timestamp dans la macro original
                        for se in sub_events:
                            se['timestamp'] = base_ts + se['timestamp']
                        events = events[:i+1] + sub_events + events[i+1:]
                        # On ne change pas event_index car la liste a été étendue
                        continue
                    except Exception as e:
                        logger.error(f"Failed to load sub-macro {sub_macro_file}: {e}")
                        event_index = i + 1
                        break
                else:
                    # Exécution normale
                    try:
                        if action in globals():
                            globals()[action](**params)
                        else:
                            logger.warning(f"Unknown action: {action}")
                    except Exception as e:
                        logger.error(f"Error in macro event {i} action={action}: {e}")
                    event_index = i + 1
                    break
        else:
            time.sleep(0.001)

    total_time = time.time() - start_time
    return {"status": "ok", "duration": total_time, "events_processed": event_index}

# ============================================
# 6. PROTECTED MACRO (Password-Locked)
# ============================================

def create_protected_macro(output_path, password, macro_events=None):
    """
    Crée une macro protégée par mot de passe (chiffrement simple).

    Args:
        output_path: fichier de sortie .json
        [REDACTED_SECRET] de passe (sera hashée)
        macro_events: liste d'événements (optionnel, sinon enregistrement direct depuis recorder)

    Retourne le chemin du fichier protégé.
    """
    import hashlib, base64
    from cryptography.fernet import Fernet

    # Si macro_events n'est pas fourni, on lance l'enregistreur
    if macro_events is None:
        # Dummy call: dans la vraie vie on appellerait record_macro()
        return {"status": "error", "message": "macro_events required for now"}

    # Dériver une cléFernet du mot de passe
    key = base64.urlsafe_b64encode(hashlib.sha256(password.encode()).digest()[:32])
    f = Fernet(key)

    # Sérialiser les événements
    data = json.dumps({"events": macro_events}, ensure_ascii=False, indent=2).encode('utf-8')
    encrypted = f.encrypt(data)

    # Écrire le fichier (format: {encrypted_base64}.enc)
    with open(output_path, 'wb') as f_out:
        f_out.write(encrypted)

    logger.info(f"Protected macro saved to {output_path}")
    return {"status": "ok", "path": output_path}

def load_and_decrypt_protected_macro(encrypted_path, password):
    """
    Charge et déchiffre une macro protégée.

    Args:
        encrypted_path: fichier .enc
        password: mot de passe

    Returns:
        {"status":"ok", "events": [...]} ou erreur
    """
    import hashlib, base64
    from cryptography.fernet import Fernet, InvalidToken

    key = base64.urlsafe_b64encode(hashlib.sha256(password.encode()).digest()[:32])
    f = Fernet(key)

    try:
        with open(encrypted_path, 'rb') as f_in:
            encrypted = f_in.read()
        decrypted = f.decrypt(encrypted)
        data = json.loads(decrypted.decode('utf-8'))
        return {"status": "ok", "events": data.get('events', [])}
    except InvalidToken:
        return {"status": "error", "message": "Invalid password or corrupted file"}
    except Exception as e:
        return {"status": "error", "message": str(e)}

# ============================================
# 7. MACRO HOTKEY STOP
# ============================================

class MacroStopManager:
    """Gère l'arrêt global d'une macro via hotkey (sécurité)"""
    def __init__(self, stop_hotkey='ctrl+alt+esc'):
        self.stop_hotkey = stop_hotkey
        self.stop_requested = False
        self.listener = None

    def start_listener(self):
        """Démarre l'écoute de la hotkey dans un thread séparé"""
        from pynput import keyboard

        def on_press(key):
            try:
                if key == keyboard.Key.ctrl_l or key == keyboard.Key.ctrl_r:
                    self.ctrl_pressed = True
                if key == keyboard.Key.alt_l or key == keyboard.Key.alt_r:
                    self.alt_pressed = True
                if hasattr(key, 'char') and key.char == '[':
                    self.bracket_pressed = True  # dummy
            except:
                pass

        def on_release(key):
            try:
                if key == keyboard.Key.ctrl_l or key == keyboard.Key.ctrl_r:
                    self.ctrl_pressed = False
                if key == keyboard.Key.alt_l or key == keyboard.Key.alt_r:
                    self.alt_pressed = False
                if hasattr(key, 'char') and key.char == '[':
                    self.bracket_pressed = False
            except:
                pass

        self.ctrl_pressed = False
        self.alt_pressed = False
        self.bracket_pressed = False

        self.listener = keyboard.Listener(on_press=on_press, on_release=on_release)
        self.listener.start()
        logger.info(f"Macro stop hotkey listener started: {self.stop_hotkey}")

    def check_stop(self):
        """Vérifie si la hotkey a été pressée"""
        # Simple check: ctrl+alt+somekey (ex: ctrl+alt+esc)
        # On implement a simple combo check
        # Note: for a real implementation, we'd track the exact key sequence
        return self.stop_requested

    def stop(self):
        self.stop_requested = True
        if self.listener:
            self.listener.stop()

# ============================================
# 8. MACRO REPORTING
# ============================================

def generate_macro_report(macro_path, execution_log):
    """
    Génère un rapport HTML/JSON d'exécution de macro.

    Args:
        macro_path: chemin de la macro
        execution_log: dict avec timestamps, actions, résultats

    Returns:
        {"status":"ok", "report_path": "...", "summary": {...}}
    """
    import datetime
    report_dir = os.path.join(os.path.dirname(macro_path), 'reports')
    os.makedirs(report_dir, exist_ok=True)

    timestamp = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
    report_base = f"report_{timestamp}"

    # JSON report
    json_path = os.path.join(report_dir, report_base + '.json')
    with open(json_path, 'w', encoding='utf-8') as f:
        json.dump({
            'macro': macro_path,
            'generated_at': timestamp,
            'log': execution_log
        }, f, indent=2, ensure_ascii=False)

    # HTML report (simple)
    html_path = os.path.join(report_dir, report_base + '.html')
    with open(html_path, 'w', encoding='utf-8') as f:
        f.write(f"""<!DOCTYPE html>
<html>
<head>
<title>Macro Report — {os.path.basename(macro_path)}</title>
<style>
body {{ font-family: sans-serif; margin: 20px; }}
table {{ border-collapse: collapse; width: 100%; }}
th, td {{ border: 1px solid #ccc; padding: 8px; text-align: left; }}
th {{ background: #f0f0f0; }}
.success {{ color: green; }}
.error {{ color: red; }}
</style>
</head>
<body>
<h1>Macro Execution Report</h1>
<p><strong>Macro:</strong> {macro_path}</p>
<p><strong>Generated:</strong> {timestamp}</p>
<h2>Summary</h2>
<ul>
<li>Total actions: {len(execution_log.get('actions', []))}</li>
<li>Status: {execution_log.get('status', 'unknown')}</li>
<li>Duration: {execution_log.get('elapsed', 0):.2f}s</li>
</ul>
<h2>Actions Log</h2>
<table>
<tr><th>#</th><th>Action</th><th>Params</th><th>Result</th></tr>
""")
        for i, act in enumerate(execution_log.get('actions', [])):
            result_class = 'success' if act.get('result', {}).get('status') == 'ok' else 'error'
            f.write(f"<tr><td>{i+1}</td><td>{act.get('action')}</td><td>{act.get('params')}</td><td class='{result_class}'>{act.get('result')}</td></tr>\n")
        f.write("""
</table>
</body>
</html>
""")

    logger.info(f"Macro report generated: {json_path} & {html_path}")
    return {"status": "ok", "report_json": json_path, "report_html": html_path}


path: lib/image_recognition.py
content: """
Image Recognition Module

Uses OpenCV for template matching and multi-scale detection.
Thread-safe, robust error handling.
"""

import logging
import threading
import os
from typing import Dict, Tuple, Optional, List, Any

logger = logging.getLogger(__name__)


class ImageRecognition:
    """Image recognition using OpenCV."""
    
    def __init__(self):
        self.lock = threading.Lock()
        try:
            import cv2
            self.cv2 = cv2
            logger.info("OpenCV initialized")
        except ImportError:
            logger.warning("OpenCV not installed - image recognition unavailable")
            self.cv2 = None
    
    def find_image(self, template_path: str, confidence: float = 0.9, dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Find template image on screen."""
        if not self.cv2:
            return {"status": "error", "message": "OpenCV not installed"}
        
        if not os.path.exists(template_path):
            return {"status": "error", "message": f"Template not found: {template_path}"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] find_image: %s (confidence=%.2f)", template_path, confidence)
                return {"status": "ok", "dry_run": True}
            
            with self.lock:
                import pyautogui
                
                # Take screenshot
                screenshot = pyautogui.screenshot()
                screen_array = self.cv2.cvtColor(
                    self.cv2.UMat(self.cv2.imread(template_path, 0)).get(),
                    self.cv2.COLOR_RGB2BGR
                )
                
                # Load template
                template = self.cv2.imread(template_path, 0)
                if template is None:
                    return {"status": "error", "message": "Failed to load template"}
                
                # Template matching
                result = self.cv2.matchTemplate(
                    self.cv2.cvtColor(
                        self.cv2.UMat(self.cv2.cvtColor(
                            self.cv2.UMat(screenshot).get(),
                            self.cv2.COLOR_RGB2BGR
                        )).get(),
                        self.cv2.COLOR_BGR2GRAY
                    ),
                    template,
                    self.cv2.TM_CCOEFF_NORMED
                )
                
                min_val, max_val, min_loc, max_loc = self.cv2.minMaxLoc(result)
                
                if max_val >= confidence:
                    x, y = max_loc
                    logger.info("Found image at (%d, %d) with confidence %.2f", x, y, max_val)
                    return {
                        "status": "ok",
                        "x": x,
                        "y": y,
                        "confidence": float(max_val)
                    }
                return {"status": "not_found", "message": f"Template not found (max_confidence={max_val:.2f})"}
        except Exception as e:
            logger.error("find_image failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def find_image_multiscale(self, template_path: str, confidence: float = 0.9, 
                              scale_factors: Optional[List[float]] = None, 
                              dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Find image at multiple scales."""
        if not self.cv2:
            return {"status": "error", "message": "OpenCV not installed"}
        
        if scale_factors is None:
            scale_factors = [0.5, 0.75, 1.0, 1.25, 1.5]
        
        if not os.path.exists(template_path):
            return {"status": "error", "message": f"Template not found: {template_path}"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] find_image_multiscale: %s (scales=%s)", template_path, scale_factors)
                return {"status": "ok", "dry_run": True}
            
            with self.lock:
                import pyautogui
                
                screenshot = pyautogui.screenshot()
                template = self.cv2.imread(template_path, 0)
                
                if template is None:
                    return {"status": "error", "message": "Failed to load template"}
                
                for scale in scale_factors:
                    resized = self.cv2.resize(template, None, fx=scale, fy=scale)
                    result = self.cv2.matchTemplate(
                        self.cv2.cvtColor(
                            self.cv2.UMat(screenshot).get(),
                            self.cv2.COLOR_RGB2GRAY
                        ),
                        resized,
                        self.cv2.TM_CCOEFF_NORMED
                    )
                    
                    min_val, max_val, min_loc, max_loc = self.cv2.minMaxLoc(result)
                    
                    if max_val >= confidence:
                        x, y = max_loc
                        logger.info("Found image at scale %.2f (%.2f confidence)", scale, max_val)
                        return {
                            "status": "ok",
                            "x": x,
                            "y": y,
                            "confidence": float(max_val),
                            "scale": scale
                        }
                
                return {"status": "not_found", "message": "Template not found at any scale"}
        except Exception as e:
            logger.error("find_image_multiscale failed: %s", str(e))
            return {"status": "error", "message": str(e)}
    
    def wait_for_image(self, template_path: str, timeout: float = 30.0, 
                       interval: float = 0.5, confidence: float = 0.9, 
                       dry_run: bool = False, **kwargs) -> Dict[str, Any]:
        """Wait for image to appear on screen."""
        if not os.path.exists(template_path):
            return {"status": "error", "message": f"Template not found: {template_path}"}
        
        try:
            if dry_run:
                logger.info("[DRY RUN] wait_for_image: %s (timeout=%.1fs)", template_path, timeout)
                return {"status": "ok", "dry_run": True}
            
            import time
            start = time.time()
            
            while time.time() - start < timeout:
                result = self.find_image(template_path, confidence=confidence)
                if result["status"] == "ok":
                    logger.info("Image appeared after %.1fs", time.time() - start)
                    return result
                time.sleep(interval)
            
            return {"status": "timeout", "message": f"Image not found within {timeout}s"}
        except Exception as e:
            logger.error("wait_for_image failed: %s", str(e))
            return {"status": "error", "message": str(e)}


Security scan verdicts — ClawScan: suspicious | static analysis: suspicious | VirusTotal: clean | SkillSpector: suspicious (CRITICAL)