Spaces:

usman-ai-dev
/

ai-code-maintainability-engine

Sleeping

File size: 14,159 Bytes

38bc0dc

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>AI Code Maintainability Scoring & Refactoring Engine — Revision Notes</title>
<style>
  :root {
    --bg: #0d1117;
    --card: #161b22;
    --border: #30363d;
    --text: #e6edf3;
    --muted: #8b949e;
    --accent: #58a6ff;
    --green: #3fb950;
    --orange: #d29922;
    --red: #f85149;
    --purple: #bc8cff;
  }
  * { box-sizing: border-box; }
  body {
    background: var(--bg);
    color: var(--text);
    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
    line-height: 1.6;
    margin: 0;
    padding: 0;
  }
  .wrap { max-width: 880px; margin: 0 auto; padding: 32px 20px 80px; }
  h1 {
    font-size: 1.7em;
    border-bottom: 2px solid var(--accent);
    padding-bottom: 12px;
    margin-bottom: 4px;
  }
  .subtitle { color: var(--muted); margin-bottom: 28px; font-size: 0.95em; }
  h2 {
    color: var(--accent);
    font-size: 1.25em;
    margin-top: 42px;
    border-left: 4px solid var(--accent);
    padding-left: 12px;
  }
  h3 {
    color: var(--purple);
    font-size: 1.05em;
    margin-top: 26px;
  }
  .card {
    background: var(--card);
    border: 1px solid var(--border);
    border-radius: 8px;
    padding: 16px 20px;
    margin: 14px 0;
  }
  .pill {
    display: inline-block;
    background: rgba(88,166,255,0.15);
    color: var(--accent);
    border-radius: 5px;
    padding: 2px 8px;
    font-size: 0.85em;
    margin-right: 6px;
  }
  table { width: 100%; border-collapse: collapse; margin: 14px 0; }
  th, td {
    border: 1px solid var(--border);
    padding: 8px 12px;
    text-align: left;
    font-size: 0.95em;
  }
  th { background: #1c2229; color: var(--accent); }
  tr:nth-child(even) { background: rgba(255,255,255,0.02); }
  code, pre {
    background: #010409;
    color: #79c0ff;
    border-radius: 6px;
    font-family: "SF Mono", Consolas, monospace;
  }
  code { padding: 2px 6px; font-size: 0.9em; }
  pre {
    padding: 14px 16px;
    overflow-x: auto;
    border: 1px solid var(--border);
    line-height: 1.5;
    font-size: 0.88em;
  }
  .flow {
    background: var(--card);
    border: 1px solid var(--border);
    border-radius: 8px;
    padding: 18px 22px;
    font-family: Consolas, monospace;
    color: var(--green);
    white-space: pre;
    font-size: 0.9em;
  }
  .q {
    color: var(--orange);
    font-weight: 600;
    margin-top: 18px;
  }
  .summary-box {
    background: linear-gradient(135deg, rgba(88,166,255,0.1), rgba(188,140,255,0.1));
    border: 1px solid var(--accent);
    border-radius: 8px;
    padding: 18px 22px;
    font-style: italic;
    margin: 16px 0;
  }
  .warn {
    background: rgba(248,81,73,0.1);
    border: 1px solid var(--red);
    border-radius: 8px;
    padding: 12px 18px;
    color: #ffb3ac;
    font-size: 0.92em;
    margin: 14px 0;
  }
  .num-table td:first-child { color: var(--muted); }
  .num-table td:last-child { color: var(--green); font-weight: 600; }
  ul { padding-left: 22px; }
  li { margin: 4px 0; }
</style>
</head>
<body>
<div class="wrap">

<h1>AI Code Maintainability Scoring & Refactoring Engine</h1>
<div class="subtitle">Revision Notes — Light-touch interview prep (2–3 sentences + one technical detail per topic)</div>

<div class="warn">
<strong>Scope reminder:</strong> this project was deliberately scoped as light-touch, not a deep-dive like Urdu Sentiment or Medical Image. Use the "One-Line Summary" and "Quick Answers" sections as your primary prep. The deeper sections exist so you're not caught flat-footed on a follow-up, not so you memorize all of it.
</div>

<h2>What Is This Project? (Elevator Pitch)</h2>
<div class="card">
"I built a two-phase AI system that scores the structural quality of Python code and then autonomously refactors risky code to improve it. Phase 1 uses AST parsing + a Random Forest classifier to predict a maintainability risk score. Phase 2 uses a CodeT5 deep learning model to generate and iteratively select better versions of risky code."
</div>

<h2>The Problem It Solves</h2>
<div class="card">
Traditional linters (PyLint, Flake8) use rigid, hand-written rules. This engine instead <strong>learns</strong> what "risky" structure looks like from data — deep nesting, high complexity, large functions — and then goes a step further than any linter by actually generating improved code, not just flagging problems.
</div>

<h2>Full Pipeline Flow</h2>
<div class="flow">Input Code
   │
   ▼
PHASE 1 — Evaluator
  AST Parsing → Feature Extraction → ML Scoring → Explanation
   │
   ▼
Risk Score (0–100) + Top 3 Reasons
   │
   ▼
PHASE 2 — Refactorer (only if risky)
  Generate Candidates (CodeT5) → Validate → Re-score via Phase 1 → Select Best
   │  (loop until target score or max iterations)
   ▼
Improved Code</div>

<h2>Phase 1 — The Evaluator</h2>

<h3>1. AST Analyzer — 11 Structural Features</h3>
<div class="card">
Parses code into an Abstract Syntax Tree (not line-by-line text reading — actual structural traversal via <code>ast.walk()</code>) and extracts 11 signals:
<table>
<tr><th>Feature</th><th>What it measures</th></tr>
<tr><td>max_nesting_depth</td><td><strong>Strongest predictor</strong> — deepest if/for/while/try nesting</td></tr>
<tr><td>cyclomatic_complexity</td><td>1 + every decision point (if/for/while/except/with/assert/bool-op)</td></tr>
<tr><td>avg_function_length</td><td>Mean lines per function</td></tr>
<tr><td>num_functions, num_loops, num_if, num_try_except, num_return</td><td>Raw structural counts</td></tr>
<tr><td>line_count</td><td>Total file size</td></tr>
<tr><td>recursion_flag</td><td>1 if any function calls itself, else 0</td></tr>
<tr><td>global_variable_count</td><td>Count of <code>global</code> declarations — hidden state / tight coupling</td></tr>
</table>
<strong>One technical detail worth knowing cold:</strong> nesting depth is computed via a recursive traversal that increments depth only when it enters a defined set of "nesting nodes" (<code>If, For, While, With, Try, FunctionDef, AsyncFunctionDef, ClassDef</code>) — everything else keeps the current depth.
</div>

<h3>2. Dataset Generation</h3>
<div class="card">
Synthetic dataset generated from code templates, not scraped real-world code. <strong>220 total samples, perfectly balanced: 110 labeled Clean (0), 110 labeled Risky (1).</strong> Reproducible via <code>random.seed(42)</code> — same dataset every run.
</div>

<h3>3. Feature Pipeline</h3>
<div class="card">
Converts the feature dictionary into a fixed-order numeric vector (<code>FEATURE_SCHEMA</code> defines the order — this order must stay identical between training and inference or predictions break silently). Scales with <code>StandardScaler</code> (mean=0, std=1) so no single feature like <code>line_count</code> dominates just because its raw numbers are bigger. Scaler is pickled and reused at inference time — never re-fit on new data.
</div>

<h3>4. Model Training</h3>
<div class="card">
Two models trained side by side on an 80/20 stratified split (176 train / 44 test):
<table>
<tr><th>Model</th><th>Key hyperparameters</th></tr>
<tr><td>Random Forest <span class="pill">primary / production</span></td><td>200 trees, max_depth=10, min_samples_split=4, class_weight="balanced"</td></tr>
<tr><td>XGBoost <span class="pill">comparison only</span></td><td>200 estimators, max_depth=6, learning_rate=0.1, subsample=0.8</td></tr>
</table>
<strong>Random Forest is the one actually used in the live Scoring API</strong> — XGBoost is trained and evaluated for comparison but not deployed.
</div>
<div class="warn">
No saved accuracy/F1 number exists in the repo — the evaluation function prints it live but doesn't persist it to a file. Run <code>python model_trainer.py</code> before your interview and note the actual number rather than guessing one.
</div>

<h3>5. Explanation Engine</h3>
<div class="card">
Answers "why is this risky" without recomputing anything new. Logic: for each of the 11 features, check if the code's actual value exceeds a fixed threshold (e.g. <code>max_nesting_depth > 3</code>, <code>cyclomatic_complexity > 5</code>, <code>global_variable_count > 1</code>). Of the features that exceed threshold, rank by the model's feature importance and return the <strong>top 3</strong> as plain-English sentences (e.g. "Deep nesting detected (depth: 7)").
</div>

<h3>6. Scoring API — Final Output</h3>
<div class="card">
Single entry point: <code>evaluate(code)</code>. Runs the model's predicted probability of the "risky" class × 100 as the risk score.
<table>
<tr><th>Risk Score</th><th>Level</th></tr>
<tr><td>0–30</td><td style="color:var(--green)">Low</td></tr>
<tr><td>31–60</td><td style="color:var(--orange)">Medium</td></tr>
<tr><td>61–100</td><td style="color:var(--red)">High</td></tr>
</table>
<pre>{
  "risk_score": 82,
  "risk_level": "High",
  "confidence": 0.82,
  "top_risk_factors": ["Deep nesting detected (depth: 7)", ...]
}</pre>
Properties worth naming if asked: deterministic, stateless, fast (no training happens at inference time).
</div>

<h2>Phase 2 — The Refactorer</h2>

<div class="card">
Only runs on code Phase 1 flagged as risky. Uses <strong>CodeT5</strong> (a deep learning code-generation model) to produce multiple refactored candidates, then uses Phase 1's own <code>evaluate()</code> as a reward/ranking function to pick the best one.
</div>

<h3>1. Candidate Generation — 3 Strategies</h3>
<div class="card">
Generates one candidate per strategy, each with a different prompt and temperature:
<table>
<tr><th>Strategy</th><th>Temperature</th></tr>
<tr><td>Improve readability and clarity</td><td>0.5 (conservative)</td></tr>
<tr><td>Reduce nesting and simplify logic</td><td>0.7</td></tr>
<tr><td>Refactor for strict maintainability best practices</td><td>0.85 (more creative)</td></tr>
</table>
<strong>One technical detail:</strong> temperature controls how much the model deviates from the "safe" rewrite — low temperature stays close to minimal edits, high temperature takes bigger structural risks.
</div>

<h3>2. Validation → Selection → Iteration</h3>
<div class="card">
Each candidate is checked to <strong>actually compile</strong> (no syntax errors) before it's even considered. Valid candidates are re-scored by feeding them back through the Phase 1 <code>evaluate()</code> API, and the lowest risk score wins that round.
<br><br>
<strong>Iterative loop defaults:</strong> <code>target_score=20</code>, <code>max_iterations=3</code>. Stops early if the target is hit, or after 3 rounds regardless. Starts by initializing the best-known score to <code>float('inf')</code> — a simple trick meaning "anything found is automatically an improvement over nothing."
</div>

<h2>Tech Stack</h2>
<div class="card">
<table>
<tr><th>Component</th><th>Technology</th></tr>
<tr><td>Structural Analysis</td><td>Python <code>ast</code> module</td></tr>
<tr><td>Risk Prediction</td><td>Random Forest (primary), XGBoost (comparison) — scikit-learn</td></tr>
<tr><td>Refactoring</td><td>CodeT5 (Transformers / PyTorch)</td></tr>
<tr><td>API</td><td>Unified <code>main.py</code> entry point + <code>api_server.py</code></td></tr>
<tr><td>Deployment</td><td>Live at ai-code-maintainability.hmuhammadusman.com</td></tr>
</table>
</div>

<h2>Numbers to Remember</h2>
<div class="card">
<table class="num-table">
<tr><td>Structural features extracted</td><td>11</td></tr>
<tr><td>Total dataset samples</td><td>220 (110 Clean / 110 Risky — balanced)</td></tr>
<tr><td>Train / test split</td><td>176 / 44 (80/20, stratified)</td></tr>
<tr><td>Random Forest trees</td><td>200 (max_depth=10)</td></tr>
<tr><td>Risk score range</td><td>0–100</td></tr>
<tr><td>Top risk factors surfaced</td><td>3</td></tr>
<tr><td>Refactor candidate strategies</td><td>3 (temps 0.5 / 0.7 / 0.85)</td></tr>
<tr><td>Default optimizer target / max iterations</td><td>20 / 3</td></tr>
<tr><td>Reproducibility seed</td><td>42</td></tr>
</table>
</div>

<h2>Quick Answers — Likely Questions</h2>

<div class="q">"Why Random Forest over a simpler rule-based linter?"</div>
<div class="card">Rule-based linters need every threshold hand-tuned per rule. A trained model learns which combinations of features actually correlate with risk from data, and can weigh 11 signals together instead of checking them independently.</div>

<div class="q">"Why two models (RF + XGBoost) if only one is deployed?"</div>
<div class="card">Comparison during development — training both and comparing accuracy tells you whether the extra complexity of boosting is worth it on this dataset size before committing to one in production.</div>

<div class="q">"Isn't 220 samples very small for ML?"</div>
<div class="card">Yes — be upfront about this if asked. It's a synthetic, template-generated dataset, not real-world code, which is both the honest limitation and a natural "what I'd improve next" answer (train on real open-source repos labeled by actual maintainability metrics).</div>

<div class="q">"How does Phase 2 know a candidate is actually better, not just different?"</div>
<div class="card">It doesn't trust the language model's own judgment — every candidate gets re-scored through the exact same Phase 1 <code>evaluate()</code> pipeline used on the original code, so "better" is measured by the same objective risk score, not by how the refactor looks.</div>

<div class="q">"What would you improve?"</div>
<div class="card">Real-world training data instead of synthetic templates; persisting evaluation metrics instead of only printing them; and multi-file / cross-function analysis, since right now every file is scored in isolation.</div>

<h2>One-Line Summary to Open With</h2>
<div class="summary-box">
"I built a two-phase AI system — Phase 1 uses AST parsing and a Random Forest model to score Python code's maintainability risk out of 100 and explain why, and Phase 2 uses a CodeT5 deep learning model to iteratively generate and select better refactored versions of risky code, using the Phase 1 score itself as the judge of improvement."
</div>

</div>
</body>
</html>