DJ-Goanna-Coding commited on
Commit
e15895e
·
verified ·
1 Parent(s): 151c05f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -480
README.md CHANGED
@@ -1,489 +1,22 @@
1
- # VAMGUARD TITAN 🚀
2
-
3
- **Automated GitHub to HuggingFace Space Synchronization with AI Agent**
4
-
5
- VAMGUARD TITAN is a comprehensive system for managing, synchronizing, and automating file transfers between GitHub repositories and HuggingFace Spaces. It includes a Streamlit-based web interface with an integrated AI agent powered by Claude for intelligent file management and code assistance.
6
-
7
- ## Features
8
-
9
- - 🤖 **AI Agent Interface**: Interactive Claude-powered assistant for code help and file management
10
- - 🔄 **Automated Sync**: Seamless synchronization between GitHub and HuggingFace Spaces
11
- - 📊 **File Auditing**: Complete file system auditing with JSON inventory and compressed archives
12
- - 📁 **File Management**: Web-based file browser, upload, and management interface
13
- - ⚙️ **Automation**: GitHub Actions workflows for scheduled syncing and deployment
14
- - 🗂️ **Mapping & Inventory**: Integration with mapping-and-inventory repository for backup and organization
15
-
16
- ## Quick Start
17
-
18
- ### Prerequisites
19
-
20
- - Python 3.11+
21
- - HuggingFace account with API token
22
- - Anthropic API key (for AI agent)
23
- - GitHub repository (for automation)
24
-
25
- ### Installation
26
-
27
- 1. **Clone the repository**
28
- ```bash
29
- git clone https://github.com/DJ-Goana-Coding/VAMGUARD_TITAN.git
30
- cd VAMGUARD_TITAN
31
- ```
32
-
33
- 2. **Install dependencies**
34
- ```bash
35
- pip install -r requirements.txt
36
- ```
37
-
38
- 3. **Configure environment variables**
39
- ```bash
40
- cp .env.example .env
41
- # Edit .env with your API keys and configuration
42
- ```
43
-
44
- 4. **Update configuration**
45
- Edit `config.yaml` to match your spaces and preferences.
46
-
47
- ### Running the Application
48
-
49
- #### Streamlit Web Interface
50
-
51
- ```bash
52
- streamlit run app.py
53
- ```
54
-
55
- The web interface provides:
56
- - Dashboard with space overview
57
- - AI agent chat interface
58
- - File management tools
59
- - Space synchronization controls
60
- - Audit and archive creation
61
-
62
- #### Command Line Automation
63
-
64
- ```bash
65
- # Run file audit
66
- python automation.py audit
67
-
68
- # Sync all configured spaces
69
- python automation.py sync
70
-
71
- # Sync specific space
72
- python automation.py sync-space --space vamguard-titan
73
-
74
- # Backup to mapping-and-inventory
75
- python automation.py backup
76
-
77
- # Run full automation (audit + sync + backup)
78
- python automation.py full
79
- ```
80
-
81
- ## Configuration
82
-
83
- ### Environment Variables (.env)
84
-
85
- ```bash
86
- # HuggingFace Configuration
87
- HF_TOKEN=your_huggingface_token_here
88
- HF_USERNAME=your-username
89
-
90
- # Anthropic API (for AI agent)
91
- ANTHROPIC_API_KEY=your_anthropic_api_key_here
92
-
93
- # GitHub (optional)
94
- GITHUB_TOKEN=your_github_token_here
95
-
96
- # Agent Settings
97
- AGENT_MODEL=claude-sonnet-4-5-20250929
98
- MAX_TOKENS=4096
99
- ```
100
-
101
- ### Configuration File (config.yaml)
102
-
103
- ```yaml
104
- spaces:
105
- primary:
106
- name: "vamguard-titan"
107
- type: "streamlit"
108
- auto_sync: true
109
-
110
- mapping_inventory:
111
- name: "mapping-and-inventory"
112
- type: "streamlit"
113
- auto_sync: true
114
-
115
- sync:
116
- enabled: true
117
- interval_minutes: 30
118
- exclude_patterns:
119
- - "*.pyc"
120
- - "__pycache__"
121
- - ".git"
122
- - ".env"
123
- ```
124
-
125
- ## Architecture
126
-
127
- ### Core Components
128
-
129
- 1. **genesis_boiler.py**: File auditing and archive creation
130
- - Scans directories and creates file inventories
131
- - Generates compressed tar.gz archives
132
- - Produces JSON metadata for all files
133
-
134
- 2. **hf_space_sync.py**: HuggingFace Space synchronization
135
- - Creates and manages HF Spaces
136
- - Uploads files and directories
137
- - Handles bulk synchronization
138
-
139
- 3. **app.py**: Streamlit web interface
140
- - Interactive dashboard
141
- - AI agent chat interface
142
- - File management UI
143
- - Space sync controls
144
-
145
- 4. **automation.py**: Automation and CLI tools
146
- - Scheduled synchronization
147
- - Batch operations
148
- - Logging and reporting
149
-
150
- ### GitHub Actions Workflows
151
-
152
- #### HuggingFace Space Sync (.github/workflows/hf_sync.yml)
153
- - Runs on push to main branches
154
- - Scheduled execution every 6 hours
155
- - Manual trigger with sync type selection
156
- - Uploads logs and archives as artifacts
157
-
158
- #### Deploy to HuggingFace Space (.github/workflows/deploy_spaces.yml)
159
- - Deploys to primary space on push to main
160
- - Automatically creates spaces if they don't exist
161
- - Deploys to mapping-and-inventory space
162
-
163
- ## Usage Examples
164
-
165
- ### Using the AI Agent
166
-
167
- The AI agent can help with:
168
-
169
- ```
170
- User: "Help me organize these Python files by module"
171
- Agent: I'll analyze your Python files and suggest an organization structure...
172
-
173
- User: "Create a backup of all .py files"
174
- Agent: I'll use the genesis_boiler to create an archive of Python files...
175
-
176
- User: "Sync the latest changes to HuggingFace"
177
- Agent: I'll initiate a sync to your configured HuggingFace space...
178
- ```
179
-
180
- ### File Auditing
181
-
182
- ```python
183
- from genesis_boiler import GenesisBoiler
184
-
185
- boiler = GenesisBoiler()
186
- results = boiler.run_full_audit()
187
- # Creates: inventory.json and genesis_archive_TIMESTAMP.tar.gz
188
- ```
189
-
190
- ### Space Synchronization
191
-
192
- ```python
193
- from hf_space_sync import HFSpaceSync
194
-
195
- sync = HFSpaceSync()
196
- result = sync.sync_directory("vamguard-titan", ".")
197
- print(f"Uploaded {result['uploaded']} files")
198
- ```
199
-
200
- ### Automation
201
-
202
- ```bash
203
- # Via cron (every 6 hours)
204
- 0 */6 * * * cd /path/to/VAMGUARD_TITAN && python automation.py full
205
-
206
- # Via GitHub Actions (automatic on push)
207
- git push origin main # Triggers deployment workflow
208
- ```
209
-
210
- ## GitHub Secrets Setup
211
-
212
- For GitHub Actions to work, configure these secrets in your repository:
213
-
214
- 1. Go to Settings > Secrets and variables > Actions
215
- 2. Add the following secrets:
216
- - `HF_TOKEN`: Your HuggingFace API token
217
- - `HF_USERNAME`: Your HuggingFace username
218
- - `ANTHROPIC_API_KEY`: Your Anthropic API key (for AI agent)
219
-
220
- ## File Structure
221
-
222
- ```
223
- VAMGUARD_TITAN/
224
- ├── .github/
225
- │ └── workflows/
226
- │ ├── hf_sync.yml # Sync automation workflow
227
- │ └── deploy_spaces.yml # Deployment workflow
228
- ├── app.py # Streamlit web interface
229
- ├── automation.py # CLI automation script
230
- ├── genesis_boiler.py # File auditing and archiving
231
- ├── hf_space_sync.py # HuggingFace sync utilities
232
- ├── config.yaml # Configuration file
233
- ├── requirements.txt # Python dependencies
234
- ├── .env.example # Environment template
235
- └── README.md # This file
236
- ```
237
-
238
- ## Advanced Features
239
-
240
- ### Custom Sync Filters
241
-
242
- Edit `config.yaml` to customize sync behavior:
243
-
244
- ```yaml
245
- sync:
246
- exclude_patterns:
247
- - "*.log"
248
- - "temp/*"
249
- - "__pycache__"
250
- include_extensions:
251
- - ".py"
252
- - ".md"
253
- - ".yaml"
254
- ```
255
-
256
- ### Multi-Space Management
257
-
258
- Configure multiple spaces for different purposes:
259
-
260
- ```yaml
261
- spaces:
262
- production:
263
- name: "vamguard-prod"
264
- type: "streamlit"
265
- auto_sync: true
266
-
267
- development:
268
- name: "vamguard-dev"
269
- type: "streamlit"
270
- auto_sync: false
271
-
272
- archive:
273
- name: "mapping-and-inventory"
274
- type: "streamlit"
275
- auto_sync: true
276
- ```
277
-
278
- ### Scheduled Backups
279
-
280
- The system automatically backs up to the mapping-and-inventory space, which serves as:
281
- - Central repository for all code
282
- - Historical archive of changes
283
- - Recovery point for accidentally deleted spaces
284
-
285
- ## Troubleshooting
286
-
287
- ### Common Issues
288
-
289
- **HuggingFace Token Error**
290
- - Ensure `HF_TOKEN` is set in `.env`
291
- - Verify token has write permissions
292
- - Check token hasn't expired
293
-
294
- **AI Agent Not Working**
295
- - Verify `ANTHROPIC_API_KEY` is set correctly
296
- - Check API key has sufficient credits
297
- - Ensure model name is correct
298
-
299
- **Sync Failures**
300
- - Check network connectivity
301
- - Verify space names match HuggingFace
302
- - Review exclude patterns in config
303
-
304
- ### Logs and Debugging
305
-
306
- - Check `automation_log_YYYYMMDD.txt` for detailed logs
307
- - Review `automation_report_*.json` for sync results
308
- - Inspect `inventory.json` for file audit details
309
-
310
- ## Contributing
311
-
312
- Contributions are welcome! Please:
313
-
314
- 1. Fork the repository
315
- 2. Create a feature branch
316
- 3. Make your changes
317
- 4. Submit a pull request
318
-
319
- ## License
320
-
321
- This project is open source and available under the MIT License.
322
-
323
- ## Support
324
-
325
- For issues, questions, or suggestions:
326
- - Open an issue on GitHub
327
- - Check the documentation in `config.yaml`
328
- - Review the automation logs
329
-
330
- ## Roadmap
331
-
332
- - [ ] Multi-repository sync support
333
- - [ ] Web UI for configuration editing
334
- - [ ] Advanced AI agent capabilities
335
- - [ ] Real-time sync notifications
336
- - [ ] Space health monitoring
337
- - [ ] Automated testing integration
338
-
339
  ---
340
-
341
- **Built with:**
342
- - 🐍 Python
343
- - 🎈 Streamlit
344
- - 🤗 HuggingFace Hub
345
- - 🤖 Claude AI (Anthropic)
346
- - ⚡ GitHub Actions
347
-
348
- *Automate your way to organized code spaces!* 🚀
349
- ---
350
- title: TIA-ARCHITECT-CORE
351
- emoji: 🧠
352
- colorFrom: indigo
353
- colorTo: purple
354
  sdk: streamlit
355
  sdk_version: 1.42.0
 
356
  app_file: app.py
357
  pinned: false
358
- license: mit
359
  ---
360
 
361
- # 🧠 TIA-ARCHITECT-CORE
362
-
363
- **The Intelligence Architect** — Central Reasoning Hub for Q.G.T.N.L. Citadel Mesh
364
-
365
- ## Overview
366
 
367
- TIA-ARCHITECT-CORE is a Sovereign AI Oracle & RAG System providing:
368
-
369
- - **RAG Intelligence** — Vector search & synthesis across Citadel documents
370
- - **Model Management** — Deploy & monitor AI models from the registry
371
- - **Worker Orchestration** — Coordinate automation workers across the mesh
372
- - **Knowledge Mesh** — Connect all Citadel nodes via persistent knowledge graphs
373
-
374
- ## Features
375
-
376
- ### 🏠 Dashboard
377
- System overview with real-time status, district topology mapping, and connection health.
378
-
379
- ### 🤖 Models Registry
380
- Browse, download, and manage AI models including:
381
- - Gemma 4 (2B, 4B) — Multimodal, edge-ready
382
- - Qwen 3.5 (7B, 14B) — Multilingual code specialist
383
- - DeepSeek V4 — Reasoning & code expert
384
- - FinBERT, CryptoBERT — Financial sentiment models
385
-
386
- ### ⚙️ Workers Constellation
387
- Automated task execution via:
388
- - Apps Script Toolbox — Google Sheets integration
389
- - Worker Watchdog — Monitor & restart workers
390
- - Self-Healing Worker — Auto-recovery system
391
-
392
- ### 📚 Knowledge Base & RAG
393
- Retrieval-Augmented Generation with:
394
- - FAISS vector store
395
- - Sentence-transformers embeddings
396
- - District artifact knowledge base
397
-
398
- ### 🔧 Tools & Utilities
399
- System diagnostics, configuration export, cache management.
400
-
401
- ## Architecture
402
-
403
- ```
404
- ┌─────────────────────────────────────┐
405
- │ TIA-ARCHITECT-CORE (HF) │
406
- │ Streamlit UI + RAG + Orchestration │
407
- └──────────────┬──────────────────────┘
408
-
409
- ┌──────────┼──────────┐
410
- ▼ ▼ ▼
411
- ┌────────┐ ┌────────┐ ┌────────┐
412
- │ Models │ │Workers │ │ RAG │
413
- │Registry│ │Constel.│ │ Store │
414
- └────────┘ └────────┘ └────────┘
415
- ```
416
-
417
- ## District System
418
-
419
- | District | Domain |
420
- |----------|--------|
421
- | D01 | Core Infrastructure |
422
- | D02 | Data Processing |
423
- | D03 | Security & Authentication |
424
- | D04 | ML Models & Training |
425
- | D05 | API & Integration |
426
- | D06 | Random Futures Trading |
427
-
428
- ## Double-N Rift
429
-
430
- - **GitHub:** DJ-Goana-Coding (single N)
431
- - **HuggingFace:** DJ-Goanna-Coding (double N)
432
-
433
- ## Development
434
-
435
- ```bash
436
- # Install dependencies
437
- pip install -r requirements.txt
438
-
439
- # Run locally
440
- streamlit run app.py
441
- ```
442
-
443
- ## Testing
444
-
445
- Comprehensive test suite with 150+ test cases and ~85% code coverage.
446
-
447
- ```bash
448
- # Install test dependencies
449
- pip install -r requirements-test.txt
450
-
451
- # Run all tests
452
- pytest -v --cov=. --cov-report=term-missing
453
-
454
- # Run specific test file
455
- pytest tests/test_genesis_boiler.py -v
456
-
457
- # Generate HTML coverage report
458
- pytest --cov=. --cov-report=html
459
- ```
460
-
461
- See [TESTING.md](TESTING.md) for detailed testing documentation.
462
-
463
- ### Test Coverage
464
-
465
- - ✅ **genesis_boiler.py** - 95% coverage (25+ tests)
466
- - ✅ **worker_watchdog.py** - 90% coverage (30+ tests)
467
- - ✅ **self_healing_worker.py** - 90% coverage (35+ tests)
468
- - ✅ **apps_script_toolbox.py** - 85% coverage (20+ tests)
469
- - ✅ **download scripts** - 80% coverage (15+ tests)
470
- - ✅ **app.py** - 75% coverage (25+ tests)
471
-
472
- ### CI/CD
473
-
474
- Tests run automatically on:
475
- - Push to main, develop, or claude/* branches
476
- - Pull requests to main
477
- - Supports Python 3.10, 3.11, 3.12, 3.13
478
-
479
- ## Tech Stack
480
-
481
- - **SDK:** Streamlit ≥1.42.0
482
- - **Python:** 3.13 compatible
483
- - **Embeddings:** sentence-transformers (all-MiniLM-L6-v2)
484
- - **Vector Store:** FAISS
485
- - **LLM Tools:** llama-index, smolagents
486
-
487
- ---
488
 
489
- *TIA-ARCHITECT-CORE v25.0.OMNI++ | Citadel Mesh Coordination System*
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: oppo-node
3
+ emoji: 💻
4
+ colorFrom: green
5
+ colorTo: gray
 
 
 
 
 
 
 
 
 
 
6
  sdk: streamlit
7
  sdk_version: 1.42.0
8
+ python_version: '3.13'
9
  app_file: app.py
10
  pinned: false
 
11
  ---
12
 
13
+ # 📱 OPPO-NODE
14
+ Sovereign Interface for the Q.G.T.N.L. Citadel Mesh.
 
 
 
15
 
16
+ ## 🏛️ SYSTEM ROLE
17
+ This node serves as the **Mobile Telemetry Faceplate**. It is specifically aligned to visualize data shards harvested from the Oppo hardware environment and bridged to the **Librarian Vault**.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
 
19
+ ## 🚀 ACTIVE PROTOCOLS
20
+ - **Telemetry visualization**: Real-time display of district stability.
21
+ - **RAG Integration**: Modular search across the 321GB Substrate.
22
+ - **Alignment**: Standardized to 777.1122.