Oliver Nitsche commited on
Commit
b5a13fe
·
0 Parent(s):

Initial commit

Browse files
.claude/settings.local.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "permissions": {
3
+ "allow": [
4
+ "Bash(pip show *)",
5
+ "Bash(python -c ' *)",
6
+ "Bash(ping -c 2 reachy-mini.local)",
7
+ "Bash(ssh *)",
8
+ "Bash(lsof -i :8000 -sTCP:LISTEN)",
9
+ "Read(//Users/oliver/.ssh/**)",
10
+ "Bash(ssh-keygen -F reachy-mini.local)",
11
+ "Bash(lsof -i :8042 -sTCP:LISTEN)",
12
+ "Bash(curl -s http://localhost:8042/status)",
13
+ "Bash(curl -s -X POST http://localhost:8042/set_name -H \"Content-Type: application/json\" -d '{\"name\":\"test\"}')",
14
+ "Bash(curl -s http://localhost:8000/)",
15
+ "Bash(curl -s http://localhost:8000/docs)",
16
+ "Bash(curl -s http://localhost:8000/status)",
17
+ "Bash(curl -s http://localhost:8000/openapi.json)",
18
+ "Bash(python3 -c \"import json,sys; d=json.load\\(sys.stdin\\); [print\\(k\\) for k in d.get\\('paths',{}\\).keys\\(\\)]\")",
19
+ "Bash(curl -s http://localhost:8000/api/apps/list-available)",
20
+ "Bash(python3 -c \"import json,sys; d=json.load\\(sys.stdin\\); [print\\(a.get\\('name','?'\\), '-', a.get\\('description',''\\)\\) for a in \\(d if isinstance\\(d,list\\) else d.get\\('apps', []\\)\\)]\")",
21
+ "Bash(curl -s http://localhost:8000/api/apps/current-app-status)",
22
+ "Bash(python3 -m json.tool)",
23
+ "Bash(curl -s -X POST \"http://localhost:8000/api/apps/start-app/recognizer\")",
24
+ "Bash(curl -sv -X POST \"http://localhost:8000/api/apps/start-app/recognizer\")"
25
+ ]
26
+ }
27
+ }
.gitignore ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ __pycache__/
2
+ *.egg-info/
3
+ build/
CLAUDE.md ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # CLAUDE.md
2
+
3
+ This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4
+
5
+ ## Environment Setup
6
+
7
+ The virtual environment lives one level up at `../reachy_mini_env`. Always activate it first:
8
+
9
+ ```bash
10
+ source ../reachy_mini_env/bin/activate
11
+ ```
12
+
13
+ Install the package in editable mode (required for entry-point registration):
14
+
15
+ ```bash
16
+ pip install -e .
17
+ ```
18
+
19
+ ### System dependency (Raspberry Pi / Reachy Mini wireless)
20
+
21
+ ```bash
22
+ sudo apt-get install espeak-ng # text-to-speech synthesis
23
+ pip install face-recognition # compiles dlib from source (~15 min on Pi)
24
+ ```
25
+
26
+ ## Running the App
27
+
28
+ Run directly (connects to a live Reachy Mini robot):
29
+
30
+ ```bash
31
+ python recognizer/main.py
32
+ ```
33
+
34
+ Or via the daemon entry point (used when the robot's daemon manages app lifecycle):
35
+
36
+ ```bash
37
+ reachy-mini-app run recognizer
38
+ ```
39
+
40
+ The control panel web UI is served at `http://0.0.0.0:8042` while the app runs.
41
+
42
+ ## Publishing
43
+
44
+ ```bash
45
+ reachy-mini-app check # validate the app before publishing
46
+ reachy-mini-app publish # publish to Hugging Face Spaces
47
+ ```
48
+
49
+ ## Architecture
50
+
51
+ This is a **Reachy Mini robot app** — a Python package that plugs into the `reachy_mini` SDK.
52
+
53
+ **Entry point**: `recognizer/main.py` — `Recognizer` class inheriting from `ReachyMiniApp` (ABC from `reachy_mini`).
54
+
55
+ **App lifecycle** (handled by `ReachyMiniApp.wrapped_run()`):
56
+ 1. Spawns a FastAPI/uvicorn server on `custom_app_url` (port 8042) in a background thread
57
+ 2. Connects to the robot daemon (auto-detects localhost vs. network → LOCAL backend on wireless robot)
58
+ 3. Calls `Recognizer.run(reachy_mini, stop_event)` — the main state-machine loop
59
+ 4. On stop: sets `stop_event`, shuts down the web server
60
+
61
+ **State machine** (`recognizer/main.py`):
62
+
63
+ ```
64
+ SLEEPING →(speech detected × 3)→ WAKING → ACTIVE → SLEEPING
65
+ ↓ (unknown face)
66
+ ENROLLING → SLEEPING
67
+ ```
68
+
69
+ - **SLEEPING**: polls `media.get_DoA()` at 5 Hz; robot stays in sleep pose. Three consecutive `speech_detected=True` readings (debounced) trigger a wake-up.
70
+ - **WAKING**: calls `wake_up()` (built-in animation + sound), then `look_at_world()` toward the DoA angle.
71
+ - **ACTIVE**: captures camera frames every 0.5 s, runs `face_recognition.face_locations()` + `face_recognition.face_encodings()` (HOG model, 2× downsampled for speed). Gentle head-scan idle animation via `set_target()`. 15 s timeout → back to sleep.
72
+ - **ENROLLING**: robot has detected an unrecognised face; waits for name to be submitted via the web UI (`POST /set_name`). Stores encoding in `face_db.json`, says "Nice to meet you, <name>!", then sleeps.
73
+
74
+ **Helper modules**:
75
+ - `recognizer/face_db.py` — load/save/query face encodings. Database at `recognizer/face_db.json` (gitignored). `find_match()` tolerance = 0.55.
76
+ - `recognizer/tts.py` — synthesises text via `espeak-ng -s 140 -w <tmp.wav>`, plays via `media.play_sound()`, then sleeps to let playback finish.
77
+
78
+ **Settings UI** (`recognizer/static/`):
79
+ - `index.html` / `main.js` / `style.css` — polls `GET /status` every second to show current state; reveals a name-entry form when state is `"enrolling"`.
80
+ - REST endpoints defined in `run()` via `self.settings_app` (FastAPI): `GET /status`, `POST /set_name`.
81
+
82
+ **Root-level `index.html` / `style.css`**: HuggingFace Spaces landing page — separate from the in-app UI in `recognizer/static/`.
83
+
84
+ **Entry-point registration** in `pyproject.toml`:
85
+ ```toml
86
+ [project.entry-points."reachy_mini_apps"]
87
+ recognizer = "recognizer.main:Recognizer"
88
+ ```
89
+
90
+ ## Key APIs
91
+
92
+ ```python
93
+ # Direction of Arrival from the ReSpeaker mic array
94
+ # Returns (angle_radians, speech_detected) or None
95
+ # 0 rad = left, π/2 = front/back, π = right
96
+ doa = reachy_mini.media.get_DoA()
97
+
98
+ # Camera frame (BGR uint8 numpy array)
99
+ frame = reachy_mini.media.get_frame()
100
+
101
+ # Built-in animations (blocking)
102
+ reachy_mini.wake_up()
103
+ reachy_mini.goto_sleep()
104
+
105
+ # Smooth head movement (blocking)
106
+ reachy_mini.look_at_world(x, y, z, duration=0.5) # forward=+x, right=+y
107
+
108
+ # Immediate head pose (non-blocking, use set_target for idle animation)
109
+ reachy_mini.set_target(head=pose_4x4)
110
+
111
+ # Audio
112
+ reachy_mini.media.play_sound("/abs/path/to/file.wav") # async; sleep afterward
113
+ ```
README.md ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Recognizer
3
+ emoji: 👋
4
+ colorFrom: red
5
+ colorTo: blue
6
+ sdk: static
7
+ pinned: false
8
+ short_description: Write your description here
9
+ tags:
10
+ - reachy_mini
11
+ - reachy_mini_python_app
12
+ ---
index.html ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!doctype html>
2
+ <html>
3
+
4
+ <head>
5
+ <meta charset="utf-8" />
6
+ <meta name="viewport" content="width=device-width" />
7
+ <title> Recognizer </title>
8
+ <link rel="stylesheet" href="style.css" />
9
+ </head>
10
+
11
+ <body>
12
+ <div class="hero">
13
+ <div class="hero-content">
14
+ <div class="app-icon">🤖⚡</div>
15
+ <h1> Recognizer </h1>
16
+ <p class="tagline">Enter your tagline here</p>
17
+ </div>
18
+ </div>
19
+
20
+ <div class="container">
21
+ <div class="main-card">
22
+ <div class="app-preview">
23
+ <div class="preview-image">
24
+ <div class="camera-feed">🛠️</div>
25
+ </div>
26
+ </div>
27
+ </div>
28
+ </div>
29
+
30
+ <div class="footer">
31
+ <p>
32
+ 🤖 Recognizer •
33
+ <a href="https://github.com/pollen-robotics" target="_blank">Pollen Robotics</a> •
34
+ <a href="https://huggingface.co/spaces/pollen-robotics/reachy-mini-landing-page#apps" target="_blank">Browse More
35
+ Apps</a>
36
+ </p>
37
+ </div>
38
+ </body>
39
+
40
+ </html>
pyproject.toml ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [build-system]
2
+ requires = ["setuptools>=61.0"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+
6
+ [project]
7
+ name = "recognizer"
8
+ version = "0.1.0"
9
+ description = "Add your description here"
10
+ readme = "README.md"
11
+ requires-python = ">=3.10"
12
+ dependencies = [
13
+ "reachy-mini",
14
+ "face-recognition",
15
+ "scipy",
16
+ ]
17
+ keywords = ["reachy-mini-app"]
18
+
19
+ [project.entry-points."reachy_mini_apps"]
20
+ recognizer = "recognizer.main:Recognizer"
21
+
22
+ [tool.setuptools]
23
+ package-dir = { "" = "." }
24
+ include-package-data = true
25
+
26
+ [tool.setuptools.packages.find]
27
+ where = ["."]
28
+
29
+ [tool.setuptools.package-data]
30
+ recognizer = ["**/*"] # Also include all non-.py files
recognizer/__init__.py ADDED
File without changes
recognizer/face_db.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {}
recognizer/face_db.py ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Face database: persist face encodings keyed by name."""
2
+
3
+ import json
4
+ from pathlib import Path
5
+ from typing import Optional
6
+
7
+ import numpy as np
8
+
9
+ try:
10
+ import face_recognition
11
+ except ImportError as exc:
12
+ raise ImportError(
13
+ "face-recognition is required: pip install face-recognition"
14
+ ) from exc
15
+
16
+
17
+ DB_PATH = Path(__file__).parent / "face_db.json"
18
+
19
+
20
+ def load() -> dict[str, list[list[float]]]:
21
+ if DB_PATH.exists():
22
+ return json.loads(DB_PATH.read_text())
23
+ return {}
24
+
25
+
26
+ def save(db: dict[str, list[list[float]]]) -> None:
27
+ DB_PATH.write_text(json.dumps(db, indent=2))
28
+
29
+
30
+ def find_match(
31
+ encoding: np.ndarray,
32
+ db: dict[str, list[list[float]]],
33
+ tolerance: float = 0.55,
34
+ ) -> Optional[str]:
35
+ for name, enc_list in db.items():
36
+ known = [np.array(e) for e in enc_list]
37
+ if any(face_recognition.compare_faces(known, encoding, tolerance=tolerance)):
38
+ return name
39
+ return None
40
+
41
+
42
+ def add_face(
43
+ name: str,
44
+ encoding: np.ndarray,
45
+ db: dict[str, list[list[float]]],
46
+ max_per_person: int = 5,
47
+ ) -> None:
48
+ if name not in db:
49
+ db[name] = []
50
+ if len(db[name]) < max_per_person:
51
+ db[name].append(encoding.tolist())
52
+ save(db)
recognizer/main.py ADDED
@@ -0,0 +1,223 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Recognizer app for Reachy Mini wireless.
2
+
3
+ State machine:
4
+ SLEEPING → polls DoA for noise; robot stays in sleep pose
5
+ WAKING → plays wake-up animation, looks toward the sound source
6
+ ACTIVE → captures camera frames, runs face recognition (timeout: 15 s)
7
+ ENROLLING → unknown face detected; waits for name entry via control panel
8
+ """
9
+
10
+ import logging
11
+ import math
12
+ import threading
13
+ import time
14
+ from enum import Enum, auto
15
+ from typing import Optional
16
+
17
+ import numpy as np
18
+ from pydantic import BaseModel
19
+ from reachy_mini import ReachyMini, ReachyMiniApp
20
+
21
+ from recognizer.face_db import add_face, find_match
22
+ from recognizer.face_db import load as load_face_db
23
+ from recognizer.tts import speak
24
+
25
+ try:
26
+ import face_recognition # noqa: F401 – checked at import time
27
+ except ImportError as exc:
28
+ raise ImportError(
29
+ "face-recognition is required: pip install face-recognition"
30
+ ) from exc
31
+
32
+ logger = logging.getLogger(__name__)
33
+
34
+ ACTIVE_TIMEOUT = 15.0 # seconds before returning to sleep with no face
35
+ DOA_DEBOUNCE = 3 # consecutive speech-detected readings to wake up
36
+ FACE_INTERVAL = 0.5 # seconds between face-recognition attempts
37
+ SCAN_AMPLITUDE = 0.35 # head scan amplitude (world-y, metres) during ACTIVE
38
+
39
+
40
+ class State(Enum):
41
+ SLEEPING = auto()
42
+ WAKING = auto()
43
+ ACTIVE = auto()
44
+ ENROLLING = auto()
45
+
46
+
47
+ class Recognizer(ReachyMiniApp):
48
+ custom_app_url: str | None = "http://0.0.0.0:8042"
49
+ request_media_backend: str | None = None
50
+
51
+ def run(self, reachy_mini: ReachyMini, stop_event: threading.Event) -> None:
52
+ import face_recognition as fr
53
+
54
+ # --- Shared mutable state (main loop ↔ FastAPI handlers) ---
55
+ _lock = threading.Lock()
56
+ _shared: dict = {
57
+ "state": "sleeping",
58
+ "pending_name": None, # set by /set_name when ENROLLING
59
+ }
60
+
61
+ # --- Settings-app REST endpoints ---
62
+ class NamePayload(BaseModel):
63
+ name: str
64
+
65
+ @self.settings_app.post("/set_name")
66
+ def set_name(payload: NamePayload):
67
+ with _lock:
68
+ if _shared["state"] == "enrolling":
69
+ _shared["pending_name"] = payload.name.strip()
70
+ return {"ok": True}
71
+
72
+ @self.settings_app.get("/status")
73
+ def get_status():
74
+ with _lock:
75
+ return {"state": _shared["state"]}
76
+
77
+ # --- Initialise ---
78
+ face_db = load_face_db()
79
+ state = State.SLEEPING
80
+ doa_angle = math.pi / 2 # default: facing front
81
+ speech_count = 0
82
+ active_start = 0.0
83
+ last_face_check = 0.0
84
+ pending_enc: Optional[np.ndarray] = None
85
+ scan_t0 = 0.0 # reference time for head-scan idle animation
86
+
87
+ reachy_mini.goto_sleep()
88
+
89
+ while not stop_event.is_set():
90
+
91
+ # ---------- SLEEPING ----------
92
+ if state == State.SLEEPING:
93
+ with _lock:
94
+ _shared["state"] = "sleeping"
95
+
96
+ doa = reachy_mini.media.get_DoA()
97
+ if doa is not None:
98
+ angle, speech = doa
99
+ if speech:
100
+ speech_count += 1
101
+ if speech_count >= DOA_DEBOUNCE:
102
+ doa_angle = angle
103
+ state = State.WAKING
104
+ speech_count = 0
105
+ else:
106
+ speech_count = max(0, speech_count - 1)
107
+
108
+ time.sleep(0.2)
109
+
110
+ # ---------- WAKING ----------
111
+ elif state == State.WAKING:
112
+ with _lock:
113
+ _shared["state"] = "waking"
114
+
115
+ reachy_mini.wake_up()
116
+
117
+ # Look toward the sound: DoA 0=left π/2=front π=right → world-y
118
+ y = math.sin(doa_angle - math.pi / 2) * 0.6
119
+ reachy_mini.look_at_world(1.0, y, 0.0, duration=0.5)
120
+
121
+ active_start = time.time()
122
+ scan_t0 = active_start
123
+ last_face_check = 0.0
124
+ pending_enc = None
125
+ state = State.ACTIVE
126
+
127
+ # ---------- ACTIVE ----------
128
+ elif state == State.ACTIVE:
129
+ with _lock:
130
+ _shared["state"] = "active"
131
+
132
+ now = time.time()
133
+
134
+ # Gentle head scan so the robot looks alive while waiting
135
+ elapsed = now - scan_t0
136
+ y_scan = math.sin(2 * math.pi * 0.15 * elapsed) * SCAN_AMPLITUDE
137
+ reachy_mini.set_target(
138
+ head=_look_direction(1.0, y_scan, 0.0)
139
+ )
140
+
141
+ # Throttled face recognition
142
+ if now - last_face_check >= FACE_INTERVAL:
143
+ last_face_check = now
144
+ frame = reachy_mini.media.get_frame()
145
+ if frame is not None:
146
+ rgb = frame[::2, ::2, ::-1] # 2× downsample + BGR→RGB
147
+ locs = fr.face_locations(rgb, model="hog")
148
+ if locs:
149
+ # Scale locations back to full-res for accurate encoding
150
+ full_locs = [(t*2, r*2, b*2, l*2) for t, r, b, l in locs]
151
+ encs = fr.face_encodings(frame[:, :, ::-1], full_locs)
152
+ if encs:
153
+ enc = encs[0]
154
+ name = find_match(enc, face_db)
155
+ if name:
156
+ speak(f"Hi {name}!", reachy_mini)
157
+ reachy_mini.goto_sleep()
158
+ state = State.SLEEPING
159
+ else:
160
+ speak(
161
+ "I don't know you yet. "
162
+ "Please enter your name on the control panel.",
163
+ reachy_mini,
164
+ )
165
+ pending_enc = enc
166
+ with _lock:
167
+ _shared["pending_name"] = None
168
+ state = State.ENROLLING
169
+
170
+ # Timeout: nobody showed up
171
+ if state == State.ACTIVE and time.time() - active_start > ACTIVE_TIMEOUT:
172
+ reachy_mini.goto_sleep()
173
+ state = State.SLEEPING
174
+
175
+ time.sleep(0.05)
176
+
177
+ # ---------- ENROLLING ----------
178
+ elif state == State.ENROLLING:
179
+ with _lock:
180
+ _shared["state"] = "enrolling"
181
+ name = _shared.get("pending_name")
182
+
183
+ if name:
184
+ with _lock:
185
+ _shared["pending_name"] = None
186
+ if pending_enc is not None:
187
+ add_face(name, pending_enc, face_db)
188
+ speak(f"Nice to meet you, {name}!", reachy_mini)
189
+ reachy_mini.goto_sleep()
190
+ state = State.SLEEPING
191
+
192
+ time.sleep(0.2)
193
+
194
+ reachy_mini.goto_sleep()
195
+
196
+
197
+ def _look_direction(x: float, y: float, z: float) -> np.ndarray:
198
+ """Return a 4×4 head-pose matrix that points the head toward (x, y, z)."""
199
+ from scipy.spatial.transform import Rotation as R
200
+
201
+ target = np.array([x, y, z], dtype=float)
202
+ target /= np.linalg.norm(target)
203
+ forward = np.array([1.0, 0.0, 0.0])
204
+
205
+ axis = np.cross(forward, target)
206
+ axis_norm = np.linalg.norm(axis)
207
+ if axis_norm < 1e-8:
208
+ rot_mat = np.eye(3) if np.dot(forward, target) > 0 else R.from_euler("y", 180, degrees=True).as_matrix()
209
+ else:
210
+ angle = np.arccos(np.clip(np.dot(forward, target), -1.0, 1.0))
211
+ rot_mat = R.from_rotvec(angle * axis / axis_norm).as_matrix()
212
+
213
+ pose = np.eye(4)
214
+ pose[:3, :3] = rot_mat
215
+ return pose
216
+
217
+
218
+ if __name__ == "__main__":
219
+ app = Recognizer()
220
+ try:
221
+ app.wrapped_run()
222
+ except KeyboardInterrupt:
223
+ app.stop()
recognizer/static/index.html ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!DOCTYPE html>
2
+ <html lang="en">
3
+
4
+ <head>
5
+ <meta charset="UTF-8">
6
+ <title>Recognizer – Control Panel</title>
7
+ <meta name="viewport" content="width=device-width, initial-scale=1">
8
+ <link rel="stylesheet" href="/static/style.css">
9
+ </head>
10
+
11
+ <body>
12
+ <h1>Recognizer – Control Panel</h1>
13
+
14
+ <div id="status-box">
15
+ State: <strong id="state-label">—</strong>
16
+ </div>
17
+
18
+ <div id="enroll-section" style="display:none;">
19
+ <p>A new face was detected. Enter the person's name:</p>
20
+ <div id="enroll-form">
21
+ <input type="text" id="name-input" placeholder="Enter name…" autocomplete="off">
22
+ <button id="submit-name-btn">Submit</button>
23
+ </div>
24
+ <div id="enroll-status"></div>
25
+ </div>
26
+
27
+ <script src="/static/main.js"></script>
28
+ </body>
29
+
30
+ </html>
recognizer/static/main.js ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ const STATE_LABELS = {
2
+ sleeping: "😴 Sleeping – listening for noise",
3
+ waking: "⚡ Waking up…",
4
+ active: "👀 Active – looking for faces",
5
+ enrolling: "🆕 New face detected – waiting for name",
6
+ };
7
+
8
+ let currentState = "";
9
+
10
+ async function pollStatus() {
11
+ try {
12
+ const resp = await fetch("/status");
13
+ const data = await resp.json();
14
+ const newState = data.state;
15
+
16
+ if (newState !== currentState) {
17
+ currentState = newState;
18
+
19
+ const label = document.getElementById("state-label");
20
+ label.textContent = STATE_LABELS[newState] ?? newState;
21
+
22
+ const enrollSection = document.getElementById("enroll-section");
23
+ enrollSection.style.display = newState === "enrolling" ? "block" : "none";
24
+
25
+ if (newState === "enrolling") {
26
+ document.getElementById("name-input").value = "";
27
+ document.getElementById("enroll-status").textContent = "";
28
+ document.getElementById("name-input").focus();
29
+ }
30
+ }
31
+ } catch (e) {
32
+ document.getElementById("state-label").textContent = "⚠ Backend unreachable";
33
+ }
34
+ }
35
+
36
+ async function submitName() {
37
+ const input = document.getElementById("name-input");
38
+ const name = input.value.trim();
39
+ if (!name) return;
40
+
41
+ try {
42
+ const resp = await fetch("/set_name", {
43
+ method: "POST",
44
+ headers: { "Content-Type": "application/json" },
45
+ body: JSON.stringify({ name }),
46
+ });
47
+ const data = await resp.json();
48
+ if (data.ok) {
49
+ document.getElementById("enroll-status").textContent =
50
+ `✓ Name "${name}" submitted – the robot will greet you shortly.`;
51
+ input.value = "";
52
+ }
53
+ } catch (e) {
54
+ document.getElementById("enroll-status").textContent = "Error submitting name.";
55
+ }
56
+ }
57
+
58
+ document.getElementById("submit-name-btn").addEventListener("click", submitName);
59
+
60
+ document.getElementById("name-input").addEventListener("keydown", (e) => {
61
+ if (e.key === "Enter") submitName();
62
+ });
63
+
64
+ // Poll every second
65
+ setInterval(pollStatus, 1000);
66
+ pollStatus();
recognizer/static/style.css ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ * { box-sizing: border-box; margin: 0; padding: 0; }
2
+
3
+ body {
4
+ font-family: system-ui, sans-serif;
5
+ background: #f5f5f5;
6
+ color: #222;
7
+ padding: 2rem;
8
+ max-width: 540px;
9
+ margin: 0 auto;
10
+ }
11
+
12
+ h1 {
13
+ font-size: 1.4rem;
14
+ margin-bottom: 1.5rem;
15
+ color: #111;
16
+ }
17
+
18
+ #status-box {
19
+ background: #fff;
20
+ border: 1px solid #ddd;
21
+ border-radius: 8px;
22
+ padding: 1rem 1.25rem;
23
+ margin-bottom: 1.5rem;
24
+ font-size: 1rem;
25
+ }
26
+
27
+ #state-label {
28
+ color: #1a73e8;
29
+ }
30
+
31
+ #enroll-section {
32
+ background: #fff;
33
+ border: 1px solid #fbc02d;
34
+ border-radius: 8px;
35
+ padding: 1.25rem;
36
+ }
37
+
38
+ #enroll-section p {
39
+ margin-bottom: 0.75rem;
40
+ font-weight: 500;
41
+ }
42
+
43
+ #enroll-form {
44
+ display: flex;
45
+ gap: 0.5rem;
46
+ }
47
+
48
+ #name-input {
49
+ flex: 1;
50
+ padding: 0.55rem 0.75rem;
51
+ border: 1px solid #ccc;
52
+ border-radius: 6px;
53
+ font-size: 1rem;
54
+ }
55
+
56
+ #name-input:focus {
57
+ outline: none;
58
+ border-color: #1a73e8;
59
+ }
60
+
61
+ button {
62
+ padding: 0.55rem 1.1rem;
63
+ background: #1a73e8;
64
+ color: #fff;
65
+ border: none;
66
+ border-radius: 6px;
67
+ font-size: 1rem;
68
+ cursor: pointer;
69
+ }
70
+
71
+ button:hover { background: #1558b0; }
72
+
73
+ #enroll-status {
74
+ margin-top: 0.75rem;
75
+ font-size: 0.9rem;
76
+ color: #2e7d32;
77
+ }
recognizer/tts.py ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Text-to-speech via espeak-ng → WAV file → Reachy Mini audio device."""
2
+
3
+ import logging
4
+ import os
5
+ import subprocess
6
+ import tempfile
7
+ import time
8
+
9
+ logger = logging.getLogger(__name__)
10
+
11
+
12
+ def speak(text: str, reachy_mini, words_per_second: float = 2.5) -> None:
13
+ """Synthesize *text* with espeak-ng and play it through the robot's speakers.
14
+
15
+ Blocks until playback should be finished.
16
+ Requires: sudo apt-get install espeak-ng
17
+ """
18
+ with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
19
+ wav_path = f.name
20
+ try:
21
+ subprocess.run(
22
+ ["espeak-ng", "-s", "140", "-w", wav_path, "--", text],
23
+ check=True,
24
+ timeout=15,
25
+ capture_output=True,
26
+ )
27
+ reachy_mini.media.play_sound(wav_path)
28
+ # play_sound() returns immediately; wait for GStreamer playback to finish.
29
+ estimated = len(text.split()) / words_per_second + 0.8
30
+ time.sleep(max(estimated, 1.0))
31
+ except FileNotFoundError:
32
+ logger.warning("espeak-ng not found — install with: sudo apt-get install espeak-ng")
33
+ except subprocess.CalledProcessError as exc:
34
+ logger.warning("espeak-ng failed: %s", exc.stderr.decode(errors="replace"))
35
+ finally:
36
+ try:
37
+ os.unlink(wav_path)
38
+ except OSError:
39
+ pass
style.css ADDED
@@ -0,0 +1,411 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ * {
2
+ margin: 0;
3
+ padding: 0;
4
+ box-sizing: border-box;
5
+ }
6
+
7
+ body {
8
+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
9
+ line-height: 1.6;
10
+ color: #333;
11
+ background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
12
+ min-height: 100vh;
13
+ }
14
+
15
+ .hero {
16
+ background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
17
+ color: white;
18
+ padding: 4rem 2rem;
19
+ text-align: center;
20
+ }
21
+
22
+ .hero-content {
23
+ max-width: 800px;
24
+ margin: 0 auto;
25
+ }
26
+
27
+ .app-icon {
28
+ font-size: 4rem;
29
+ margin-bottom: 1rem;
30
+ display: inline-block;
31
+ }
32
+
33
+ .hero h1 {
34
+ font-size: 3rem;
35
+ font-weight: 700;
36
+ margin-bottom: 1rem;
37
+ background: linear-gradient(45deg, #fff, #f0f9ff);
38
+ background-clip: text;
39
+ -webkit-background-clip: text;
40
+ -webkit-text-fill-color: transparent;
41
+ }
42
+
43
+ .tagline {
44
+ font-size: 1.25rem;
45
+ opacity: 0.9;
46
+ max-width: 600px;
47
+ margin: 0 auto;
48
+ }
49
+
50
+ .container {
51
+ max-width: 1200px;
52
+ margin: 0 auto;
53
+ padding: 0 2rem;
54
+ position: relative;
55
+ z-index: 2;
56
+ }
57
+
58
+ .main-card {
59
+ background: white;
60
+ border-radius: 20px;
61
+ box-shadow: 0 20px 40px rgba(0, 0, 0, 0.1);
62
+ margin-top: -2rem;
63
+ overflow: hidden;
64
+ margin-bottom: 3rem;
65
+ }
66
+
67
+ .app-preview {
68
+ background: linear-gradient(135deg, #1e3a8a, #3b82f6);
69
+ padding: 3rem;
70
+ color: white;
71
+ text-align: center;
72
+ position: relative;
73
+ }
74
+
75
+ .preview-image {
76
+ background: #000;
77
+ border-radius: 15px;
78
+ padding: 2rem;
79
+ max-width: 500px;
80
+ margin: 0 auto;
81
+ position: relative;
82
+ overflow: hidden;
83
+ }
84
+
85
+ .camera-feed {
86
+ font-size: 4rem;
87
+ margin-bottom: 1rem;
88
+ opacity: 0.7;
89
+ }
90
+
91
+ .detection-overlay {
92
+ position: absolute;
93
+ top: 50%;
94
+ left: 50%;
95
+ transform: translate(-50%, -50%);
96
+ width: 100%;
97
+ }
98
+
99
+ .bbox {
100
+ background: rgba(34, 197, 94, 0.9);
101
+ color: white;
102
+ padding: 0.5rem 1rem;
103
+ border-radius: 8px;
104
+ font-size: 0.9rem;
105
+ font-weight: 600;
106
+ margin: 0.5rem;
107
+ display: inline-block;
108
+ border: 2px solid #22c55e;
109
+ }
110
+
111
+ .app-details {
112
+ padding: 3rem;
113
+ }
114
+
115
+ .app-details h2 {
116
+ font-size: 2rem;
117
+ color: #1e293b;
118
+ margin-bottom: 2rem;
119
+ text-align: center;
120
+ }
121
+
122
+ .template-info {
123
+ display: grid;
124
+ grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
125
+ gap: 2rem;
126
+ margin-bottom: 3rem;
127
+ }
128
+
129
+ .info-box {
130
+ background: #f0f9ff;
131
+ border: 2px solid #e0f2fe;
132
+ border-radius: 12px;
133
+ padding: 2rem;
134
+ }
135
+
136
+ .info-box h3 {
137
+ color: #0c4a6e;
138
+ margin-bottom: 1rem;
139
+ font-size: 1.2rem;
140
+ }
141
+
142
+ .info-box p {
143
+ color: #0369a1;
144
+ line-height: 1.6;
145
+ }
146
+
147
+ .how-to-use {
148
+ background: #fefce8;
149
+ border: 2px solid #fde047;
150
+ border-radius: 12px;
151
+ padding: 2rem;
152
+ margin-top: 3rem;
153
+ }
154
+
155
+ .how-to-use h3 {
156
+ color: #a16207;
157
+ margin-bottom: 1.5rem;
158
+ font-size: 1.3rem;
159
+ text-align: center;
160
+ }
161
+
162
+ .steps {
163
+ display: flex;
164
+ flex-direction: column;
165
+ gap: 1.5rem;
166
+ }
167
+
168
+ .step {
169
+ display: flex;
170
+ align-items: flex-start;
171
+ gap: 1rem;
172
+ }
173
+
174
+ .step-number {
175
+ background: #eab308;
176
+ color: white;
177
+ width: 2rem;
178
+ height: 2rem;
179
+ border-radius: 50%;
180
+ display: flex;
181
+ align-items: center;
182
+ justify-content: center;
183
+ font-weight: bold;
184
+ flex-shrink: 0;
185
+ }
186
+
187
+ .step h4 {
188
+ color: #a16207;
189
+ margin-bottom: 0.5rem;
190
+ font-size: 1.1rem;
191
+ }
192
+
193
+ .step p {
194
+ color: #ca8a04;
195
+ }
196
+
197
+ .download-card {
198
+ background: white;
199
+ border-radius: 20px;
200
+ box-shadow: 0 20px 40px rgba(0, 0, 0, 0.1);
201
+ padding: 3rem;
202
+ text-align: center;
203
+ }
204
+
205
+ .download-card h2 {
206
+ font-size: 2rem;
207
+ color: #1e293b;
208
+ margin-bottom: 1rem;
209
+ }
210
+
211
+ .download-card>p {
212
+ color: #64748b;
213
+ font-size: 1.1rem;
214
+ margin-bottom: 2rem;
215
+ }
216
+
217
+ .dashboard-config {
218
+ margin-bottom: 2rem;
219
+ text-align: left;
220
+ max-width: 400px;
221
+ margin-left: auto;
222
+ margin-right: auto;
223
+ }
224
+
225
+ .dashboard-config label {
226
+ display: block;
227
+ color: #374151;
228
+ font-weight: 600;
229
+ margin-bottom: 0.5rem;
230
+ }
231
+
232
+ .dashboard-config input {
233
+ width: 100%;
234
+ padding: 0.75rem 1rem;
235
+ border: 2px solid #e5e7eb;
236
+ border-radius: 8px;
237
+ font-size: 0.95rem;
238
+ transition: border-color 0.2s;
239
+ }
240
+
241
+ .dashboard-config input:focus {
242
+ outline: none;
243
+ border-color: #667eea;
244
+ }
245
+
246
+ .install-btn {
247
+ background: linear-gradient(135deg, #667eea, #764ba2);
248
+ color: white;
249
+ border: none;
250
+ padding: 1.25rem 3rem;
251
+ border-radius: 16px;
252
+ font-size: 1.2rem;
253
+ font-weight: 700;
254
+ cursor: pointer;
255
+ transition: all 0.3s ease;
256
+ display: inline-flex;
257
+ align-items: center;
258
+ gap: 0.75rem;
259
+ margin-bottom: 2rem;
260
+ box-shadow: 0 8px 25px rgba(102, 126, 234, 0.3);
261
+ }
262
+
263
+ .install-btn:hover:not(:disabled) {
264
+ transform: translateY(-3px);
265
+ box-shadow: 0 15px 35px rgba(102, 126, 234, 0.4);
266
+ }
267
+
268
+ .install-btn:disabled {
269
+ opacity: 0.7;
270
+ cursor: not-allowed;
271
+ transform: none;
272
+ }
273
+
274
+ .manual-option {
275
+ background: #f8fafc;
276
+ border-radius: 12px;
277
+ padding: 2rem;
278
+ margin-top: 2rem;
279
+ }
280
+
281
+ .manual-option h3 {
282
+ color: #1e293b;
283
+ margin-bottom: 1rem;
284
+ font-size: 1.2rem;
285
+ }
286
+
287
+ .manual-option>p {
288
+ color: #64748b;
289
+ margin-bottom: 1rem;
290
+ }
291
+
292
+ .btn-icon {
293
+ font-size: 1.1rem;
294
+ }
295
+
296
+ .install-status {
297
+ padding: 1rem;
298
+ border-radius: 8px;
299
+ font-size: 0.9rem;
300
+ text-align: center;
301
+ display: none;
302
+ margin-top: 1rem;
303
+ }
304
+
305
+ .install-status.success {
306
+ background: #dcfce7;
307
+ color: #166534;
308
+ border: 1px solid #bbf7d0;
309
+ }
310
+
311
+ .install-status.error {
312
+ background: #fef2f2;
313
+ color: #dc2626;
314
+ border: 1px solid #fecaca;
315
+ }
316
+
317
+ .install-status.loading {
318
+ background: #dbeafe;
319
+ color: #1d4ed8;
320
+ border: 1px solid #bfdbfe;
321
+ }
322
+
323
+ .install-status.info {
324
+ background: #e0f2fe;
325
+ color: #0369a1;
326
+ border: 1px solid #7dd3fc;
327
+ }
328
+
329
+ .manual-install {
330
+ background: #1f2937;
331
+ border-radius: 8px;
332
+ padding: 1rem;
333
+ margin-bottom: 1rem;
334
+ display: flex;
335
+ align-items: center;
336
+ gap: 1rem;
337
+ }
338
+
339
+ .manual-install code {
340
+ color: #10b981;
341
+ font-family: 'SF Mono', 'Monaco', 'Inconsolata', 'Roboto Mono', monospace;
342
+ font-size: 0.85rem;
343
+ flex: 1;
344
+ overflow-x: auto;
345
+ }
346
+
347
+ .copy-btn {
348
+ background: #374151;
349
+ color: white;
350
+ border: none;
351
+ padding: 0.5rem 1rem;
352
+ border-radius: 6px;
353
+ font-size: 0.8rem;
354
+ cursor: pointer;
355
+ transition: background-color 0.2s;
356
+ }
357
+
358
+ .copy-btn:hover {
359
+ background: #4b5563;
360
+ }
361
+
362
+ .manual-steps {
363
+ color: #6b7280;
364
+ font-size: 0.9rem;
365
+ line-height: 1.8;
366
+ }
367
+
368
+ .footer {
369
+ text-align: center;
370
+ padding: 2rem;
371
+ color: white;
372
+ opacity: 0.8;
373
+ }
374
+
375
+ .footer a {
376
+ color: white;
377
+ text-decoration: none;
378
+ font-weight: 600;
379
+ }
380
+
381
+ .footer a:hover {
382
+ text-decoration: underline;
383
+ }
384
+
385
+ /* Responsive Design */
386
+ @media (max-width: 768px) {
387
+ .hero {
388
+ padding: 2rem 1rem;
389
+ }
390
+
391
+ .hero h1 {
392
+ font-size: 2rem;
393
+ }
394
+
395
+ .container {
396
+ padding: 0 1rem;
397
+ }
398
+
399
+ .app-details,
400
+ .download-card {
401
+ padding: 2rem;
402
+ }
403
+
404
+ .features-grid {
405
+ grid-template-columns: 1fr;
406
+ }
407
+
408
+ .download-options {
409
+ grid-template-columns: 1fr;
410
+ }
411
+ }