Skill: ivangdavila/storage (version 1.0.0)

--- name: Storage description: Choose and architect storage systems for applications with the right tradeoffs. metadata: {"clawdbot":{"emoji":"💾","os":["linux","darwin","win32"]}} --- ## Object vs Block vs File - Object storage (S3, R2, GCS) for immutable blobs: images, videos, backups, logs — cheap, scales infinitely, but no partial updates - Block storage (EBS, Persistent Disks) for databases and apps needing filesystem semantics — faster, but tied to single instance - Network file systems (NFS, EFS) when multiple instances need shared filesystem access — convenient but latency and cost add up - Default to object storage for user uploads — block storage for database files only ## When SQL vs NoSQL - SQL when you need joins, transactions, or complex queries — fighting against NoSQL for relational data wastes months - Document stores (MongoDB, Firestore) for nested/variable schemas where you always fetch the whole document - Key-value (Redis, DynamoDB) for simple lookups by ID at massive scale — not for complex queries - Time-series databases (InfluxDB, TimescaleDB) for metrics with timestamp-based queries — regular SQL struggles with retention policies - Start with PostgreSQL unless you have a specific reason not to — it handles JSON, full-text search, and scales further than most assume ## Local vs Cloud Storage - Local disk for ephemeral data: temp files, build artifacts, caches — assume it disappears on restart - Cloud storage for anything that must survive instance termination — never store user data only on local disk - Local SSD for databases in production — network-attached storage adds latency to every query - Hybrid: local cache in front of cloud storage for frequently accessed files ## CDN Patterns - Put CDN in front of static assets always — origin requests are slower and more expensive - Set long cache TTLs with versioned URLs (`style.abc123.css`) — cache invalidation is slow and unreliable - CDN for dynamic content only if latency matters more than freshness — adds complexity for marginal gains - Edge caching for API responses works but cache keys get tricky — start simple, add only when needed ## Upload Handling - Never accept uploads directly to app server disk in production — use presigned URLs to cloud storage - Set file size limits at load balancer level, not just application — prevents memory exhaustion attacks - Generate unique keys for uploads (UUIDs) — user-provided filenames cause collisions and path traversal risks - Validate file types by content (magic bytes), not extension — extensions are trivially spoofed ## Data Locality - Keep compute and storage in same region — cross-region data transfer adds latency and cost - Replicate data to regions where users are, not where developers are - Multi-region storage adds complexity — single region with backups elsewhere usually sufficient - Database read replicas in user regions for read-heavy workloads ## Retention and Lifecycle - Define retention policy before storing data — "keep everything" becomes expensive and legally risky - Automate deletion of temporary data — manual cleanup never happens consistently - Tiered storage for aging data: hot → warm → cold → archive — but check retrieval costs before archiving - Separate storage for logs vs business data — different retention, different compliance requirements ## Cost Traps - Egress fees dominate cloud storage costs — calculate before choosing provider - Many small files cost more than few large files — batch small writes when possible - Minimum storage duration on cold tiers — early deletion still charges full period - API request costs matter at scale — millions of LIST operations add up ## Backup Strategy - 3-2-1 rule: 3 copies, 2 different media types, 1 offsite — cloud counts as one location - Test restores regularly — untested backups are not backups - Point-in-time recovery for databases — daily snapshots lose a day of data - Version important files — deletion or corruption often discovered late


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)