Hugging Face's logo

Spaces:
Thadillo
/
participatory-planner
Sleeping

App Files Community
participatory-planner / README.md
thadillo
Add Claude Code attribution for AI-enhanced development
cb2979b
preview code
|
raw
history blame contribute delete
9.94 kB
metadata
title: Participatory Planning Application
emoji: 🏙️
colorFrom: blue
colorTo: purple
sdk: docker
pinned: false
license: mit

Participatory Planning Application

An AI-powered collaborative urban planning platform for multi-stakeholder engagement sessions with advanced sentence-level categorization and fine-tuning capabilities.

Author & Copyright

Created by: Marcos Thadeu Queiroz Magalhães 📧 thadillo@gmail.com

Copyright © 2024-2025 Marcos Thadeu Queiroz Magalhães. All rights reserved.

This software must be cited and credited to Marcos Thadeu Queiroz Magalhães whenever used or employed in any capacity.

Developed using Claude Code for AI-enhanced development.

Features

Core Features

  • 🎯 Token-based access - Self-service registration for participants
  • 🤖 AI categorization - Automatic classification using BART zero-shot models (free & offline)
  • 📝 Sentence-level analysis - Each sentence categorized independently for multi-topic submissions
  • 🗺️ Geographic mapping - Interactive visualization of geotagged contributions
  • 📊 Analytics dashboard - Real-time charts with submission and sentence-level aggregation
  • 💾 Session management - Export/import for pause/resume workflows
  • 👥 Multi-stakeholder - Government, Community, Industry, NGO, Academic, Other

Advanced AI Features

  • 🧠 Model Fine-tuning - Train custom models with LoRA or head-only methods
  • 📈 Real-time training progress - Detailed epoch/step/loss tracking during training
  • 🔄 Training data management - Export, import, and clear training examples
  • 🎛️ Multiple training modes - Head-only (fast, <100 examples) or LoRA (better, >100 examples)
  • 📦 Model deployment - Deploy fine-tuned models with one click
  • 🗑️ Force delete - Remove stuck or problematic training runs

Sentence-Level Categorization

  • ✂️ Smart segmentation - Handles abbreviations, bullet points, and complex punctuation
  • 🎯 Independent classification - Each sentence gets its own category
  • 📊 Category distribution - View breakdown of categories within submissions
  • 🔄 Backward compatible - Falls back to submission-level for legacy data
  • ✏️ Sentence editing - Edit individual sentence categories in UI

Categories

The system classifies text into six strategic planning categories:

  1. Vision - Long-term aspirational goals and ideal future states
  2. Problem - Current issues, challenges, and gaps
  3. Objectives - Specific, measurable goals and targets
  4. Directives - High-level mandates and policy directions
  5. Values - Guiding principles and community priorities
  6. Actions - Concrete implementation steps and projects

Quick Start

Basic Setup

  1. Access the application at http://localhost:5000
  2. Login with admin token: <your-admin-token-from-startup>
  3. Go to Registration to get the participant signup link
  4. Share the link with stakeholders
  5. Collect submissions and analyze with AI

Sentence-Level Analysis Workflow

  1. Collect Submissions - Participants submit via web form
  2. Run Analysis - Click "Analyze All" in Admin → Submissions
  3. Review Sentences - Click "View Sentences" on any submission
  4. Correct Categories - Edit sentence categories as needed (creates training data)
  5. Train Model - Once you have 20+ sentence corrections, train a custom model
  6. Deploy Model - Activate your fine-tuned model for better accuracy

Default Login

  • Admin Token: <your-admin-token-from-startup>
  • Admin Access: Full dashboard, analytics, moderation, AI training

Tech Stack

  • Backend: Flask (Python web framework)
  • Database: SQLite with sentence-level schema
  • AI Models:
    • BART-large-MNLI (default, 400M parameters)
    • DeBERTa-v3-base-MNLI (fast, 86M parameters)
    • DistilBART-MNLI (balanced, 134M parameters)
  • Fine-tuning: LoRA (Low-Rank Adaptation) with PEFT
  • Frontend: Bootstrap 5, Leaflet.js, Chart.js
  • Deployment: Docker support

AI Training

Training Data Management

Export Training Examples

  • Download all training data as JSON
  • Option to export only sentence-level examples
  • Use for backups or sharing datasets

Import Training Examples

  • Load training data from JSON files
  • Automatically skips duplicates
  • Useful for migrating between environments

Clear Training Examples

  • Remove unused examples to clean up
  • Option to clear only sentence-level data
  • Safe defaults prevent accidental deletion

Training Modes

Head-Only Training (Recommended for <100 examples)

  • Faster training (2-5 minutes)
  • Lower memory usage
  • Good for small datasets
  • Only trains classification layer

LoRA Fine-tuning (Recommended for >100 examples)

  • Better accuracy on larger datasets
  • Parameter-efficient (trains adapter layers)
  • Configurable rank, alpha, dropout
  • Takes 5-15 minutes depending on data size

Progress Tracking

During training, you'll see:

  • Current epoch / total epochs
  • Current step / total steps
  • Real-time loss values
  • Precise progress percentage
  • Estimated time remaining

Model Management

  • Deploy models with one click
  • Rollback to base model anytime
  • Export trained models as ZIP files
  • Force delete stuck or failed runs
  • View detailed training metrics

Demo Data

The app starts empty. You can:

  1. Generate tokens for test users
  2. Submit sample contributions (multi-sentence for best results)
  3. Run AI sentence-level analysis
  4. Correct sentence categories to build training data
  5. Train a custom fine-tuned model
  6. View analytics in submission or sentence mode

File Structure 

participatory_planner/
├── app/
│   ├── analyzer.py              # AI classification engine
│   ├── sentence_segmenter.py    # Sentence splitting logic
│   ├── models/
│   │   └── models.py            # Database models (Submission, SubmissionSentence, etc.)
│   ├── routes/
│   │   ├── admin.py             # Admin dashboard and API endpoints
│   │   └── main.py              # Public submission forms
│   ├── fine_tuning/
│   │   ├── trainer.py           # LoRA fine-tuning engine
│   │   └── model_manager.py     # Model deployment/rollback
│   └── templates/
│       ├── admin/
│       │   ├── submissions.html # Sentence-level UI
│       │   ├── dashboard.html   # Analytics with dual modes
│       │   └── training.html    # Fine-tuning interface
│       └── submit.html          # Public submission form
├── migrations/
│   └── migrate_to_sentence_level.py
├── models/
│   ├── finetuned/              # Trained model checkpoints
│   └── zero_shot/              # Base BART models
├── data/
│   └── app.db                  # SQLite database
└── README.md

Environment Variables 

SECRET_KEY=your-secret-key-here
MODELS_DIR=models/finetuned
ZERO_SHOT_MODELS_DIR=models/zero_shot

API Endpoints

Public

  • POST /submit - Submit new contribution
  • GET /register/:token - Participant registration

Admin (requires auth)

  • POST /admin/api/analyze - Analyze submissions with sentences
  • POST /admin/api/update-sentence-category/:id - Edit sentence category
  • GET /admin/api/export-training-examples - Export training data
  • POST /admin/api/import-training-examples - Import training data
  • POST /admin/api/clear-training-examples - Clear training data
  • POST /admin/api/start-fine-tuning - Start model training
  • GET /admin/api/training-status/:id - Get training progress
  • POST /admin/api/deploy-model/:id - Deploy fine-tuned model
  • DELETE /admin/api/force-delete-training-run/:id - Force delete run

Database Schema

Key Tables

submissions

  • Core submission data
  • sentence_analysis_done flag for tracking
  • Backward compatible with old category field

submission_sentences

  • Individual sentences from submissions
  • Each sentence has its own category
  • Linked to parent submission via foreign key

training_examples

  • Admin corrections for fine-tuning
  • Supports both sentence and submission-level
  • Tracks usage in training runs

fine_tuning_runs

  • Training job metadata and results
  • Real-time progress tracking fields
  • Model paths and deployment status

Troubleshooting

Training stuck at 0% progress?

  • Check if CUDA is available or forcing CPU mode
  • Reduce batch size if out of memory
  • Check training logs for errors

Sentences not being categorized?

  • Run database migration: python migrations/migrate_to_sentence_level.py
  • Ensure sentence_analysis_done column exists
  • Check that sentence segmenter is working

Can't delete training run?

  • Use "Force Delete" button for active/training runs
  • Type "DELETE" to confirm force deletion
  • Check model files aren't locked

License

MIT License - See LICENSE file for full details.

Copyright © 2024-2025 Marcos Thadeu Queiroz Magalhães (thadillo@gmail.com)

Citation Requirement

Any use, modification, or distribution of this software must include proper attribution to:

Marcos Thadeu Queiroz Magalhães (thadillo@gmail.com)

This includes:

  • Source code and documentation
  • Academic publications or presentations
  • Derivative works and commercial applications
  • User interfaces and credits sections

Contributing

Contributions welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Maintain attribution to the original author in all contributions
  4. Submit a pull request with clear description

Support

For issues or questions:

  1. Check existing documentation files
  2. Review troubleshooting section above
  3. Contact: thadillo@gmail.com
  4. Open an issue with detailed description