django-formatted-autofield 0.2.0
Django field that automatically generates formatted IDs with concurrency-safe sequence management
License: MIT
Requires Python: >=3.8
Keywords: django, field, autofield, sequence, formatted

# django-formatted-autofield

A Django field that automatically generates formatted IDs with concurrency-safe sequence management.

[![PyPI version](https://badge.fury.io/py/django-formatted-autofield.svg)](https://badge.fury.io/py/django-formatted-autofield)
[![Python versions](https://img.shields.io/pypi/pyversions/django-formatted-autofield.svg)](https://pypi.org/project/django-formatted-autofield/)
[![Django versions](https://img.shields.io/pypi/djversions/django-formatted-autofield.svg)](https://pypi.org/project/django-formatted-autofield/)

## Features

- **Auto-incrementing formatted IDs**: Generate IDs like `INV-2024-00001`, `PO-12345`, or any custom format
- **Concurrency-safe**: Uses database row-level locking to prevent duplicate IDs
- **Flexible formatting**: Support for custom format strings with placeholders
- **Dynamic placeholders**: Use callables (lambdas) for dynamic values like current year
- **Custom starting values**: Start sequences at any number
- **Manual override support**: Optionally set values manually when needed
- **Thread and process safe**: Works correctly in multi-threaded/multi-process environments

## Installation

```bash
pip install django-formatted-autofield
```

Add `formatted_autofield` to your `INSTALLED_APPS`:

```python
INSTALLED_APPS = [
    ...
    'formatted_autofield',
    ...
]
```

Run migrations to create the sequence table:

```bash
python manage.py migrate formatted_autofield
```

## Quick Start

```python
from django.db import models
from datetime import datetime
from formatted_autofield import FormattedAutoField

class PurchaseOrder(models.Model):
    order_number = FormattedAutoField(
        format_string="PO-{year}-{seq:05d}",
        placeholders={
            "year": lambda: datetime.now().year
        },
        primary_key=True
    )
    vendor = models.CharField(max_length=100)
    total = models.DecimalField(max_digits=10, decimal_places=2)

# Create instances
po1 = PurchaseOrder.objects.create(vendor="Acme Corp", total=1500.00)
print(po1.order_number)  # Output: PO-2026-00001

po2 = PurchaseOrder.objects.create(vendor="Widget Inc", total=2500.00)
print(po2.order_number)  # Output: PO-2026-00002
```

## Usage Examples

### Basic Sequential IDs

```python
class Order(models.Model):
    order_id = FormattedAutoField(
        format_string="ORD-{seq:05d}",
        primary_key=True
    )
```

Creates: `ORD-00001`, `ORD-00002`, `ORD-00003`, ...

### Custom Starting Value

```python
class Invoice(models.Model):
    invoice_number = FormattedAutoField(
        format_string="INV-{seq}",
        start_at=1000,
        primary_key=True
    )
```

Creates: `INV-1000`, `INV-1001`, `INV-1002`, ...

### Dynamic Placeholders

```python
from datetime import datetime

class Ticket(models.Model):
    ticket_number = FormattedAutoField(
        format_string="{year}-{month}-{seq:04d}",
        placeholders={
            "year": lambda: datetime.now().year,
            "month": lambda: datetime.now().strftime("%m")
        },
        primary_key=True
    )
```

Creates: `2026-02-0001`, `2026-02-0002`, ...

### Static Placeholders

```python
class Product(models.Model):
    sku = FormattedAutoField(
        format_string="{category}-{seq:06d}",
        placeholders={
            "category": "WIDGET"
        },
        max_length=50
    )
    name = models.CharField(max_length=200)
```

Creates: `WIDGET-000001`, `WIDGET-000002`, ...

### Manual Override

You can manually set the field value before saving to override auto-generation:

```python
# Auto-generated
order1 = Order.objects.create()  # Gets ORD-00001

# Manual override
order2 = Order(order_id="ORD-SPECIAL")
order2.save()  # Keeps ORD-SPECIAL

# Back to auto-generated
order3 = Order.objects.create()  # Gets ORD-00002
```

## Field Parameters

### `format_string` (required)

Python format string with placeholders. **Must include `{seq}`** for the sequence number.

- Use format specifications for padding: `{seq:05d}` (5 digits, zero-padded)
- Combine with other text: `"PREFIX-{seq}-SUFFIX"`
- Use multiple placeholders: `"{year}/{month}/{seq:04d}"`

**Examples:**
```python
format_string="ID-{seq}"           # ID-1, ID-2, ID-3
format_string="{seq:04d}"          # 0001, 0002, 0003
format_string="INV-{year}-{seq}"   # INV-2026-1, INV-2026-2
```

### `placeholders` (optional)

Dictionary mapping placeholder names to values or callables.

- **Static values**: `{"prefix": "ABC"}`
- **Callables**: `{"year": lambda: datetime.now().year}`
- Callables are evaluated at generation time (not at field definition)

**Example:**
```python
placeholders={
    "dept": "SALES",
    "year": lambda: datetime.now().year,
    "user": lambda: get_current_user().username
}
```

### `start_at` (optional, default: 1)

The first number in the sequence.

```python
start_at=1     # Default: 1, 2, 3, ...
start_at=100   # Starts at: 100, 101, 102, ...
start_at=1000  # Starts at: 1000, 1001, 1002, ...
```

### `max_length` (optional, default: 100)

Maximum length of the formatted string (CharField limitation).

```python
max_length=50   # For shorter IDs
max_length=200  # For longer formatted strings
```

## How It Works

### Sequence Management

Each `FormattedAutoField` maintains its own sequence counter in the database. The sequence is identified by:

```
app_label.model_name.field_name
```

For example: `myapp.purchaseorder.order_number`

### Concurrency Safety

The library uses Django's `select_for_update()` with `transaction.atomic()` to ensure thread-safety:

```python
with transaction.atomic():
    sequence = Sequence.objects.select_for_update().get_or_create(key=field_key)
    sequence.last_value += 1
    sequence.save()
    next_value = sequence.last_value
```

This provides database-level row locking, ensuring no duplicate IDs even with:
- Multiple Django processes (Gunicorn workers, Celery workers)
- Multiple threads
- High-concurrency scenarios

### When Values Are Generated

- **On INSERT**: Values are generated when creating new records
- **On UPDATE**: Existing values are preserved (not regenerated)
- **Manual override**: If you set a value before saving, auto-generation is skipped

## Database Compatibility

### Fully Supported (Row-level locking)
- **PostgreSQL** ✅
- **MySQL / MariaDB** ✅
- **Oracle** ✅

### Limited Support
- **SQLite** ⚠️ - Works, but uses database-level locking (less concurrent)

## Testing

The library includes comprehensive tests covering:
- Basic sequencing
- Custom placeholders (static and callable)
- Format string validation
- Concurrency (10+ simultaneous threads)
- Manual overrides
- Update operations

Run tests:

```bash
python manage.py test tests
```

## Migration Considerations

### Important: Callable Placeholders in Migrations

Django cannot serialize lambda functions in migrations. When you define a field with callable placeholders:

```python
order_number = FormattedAutoField(
    format_string="PO-{year}-{seq:05d}",
    placeholders={"year": lambda: datetime.now().year}
)
```

**The lambda will NOT be included in the migration file.** You must ensure the field definition with the callable remains in your model file. This is intentional and safe - the callable is evaluated at runtime, not migration time.

### Migrating Existing Models

If you're adding `FormattedAutoField` to an existing model with data:

1. **Add the field as non-primary, nullable** first:
```python
legacy_id = models.IntegerField(primary_key=True)  # Existing
order_number = FormattedAutoField(
    format_string="ORD-{seq:05d}",
    null=True,  # Temporarily nullable
    blank=True
)
```

2. **Run a data migration** to populate values
3. **Make it the primary key** in a subsequent migration if desired

## Performance Considerations

### Transaction Overhead

Each ID generation requires:
- One database transaction
- One row lock (SELECT FOR UPDATE)
- One update operation

For high-throughput scenarios:
- Consider bulk creation patterns where possible
- Use database connection pooling
- Ensure proper indexing (automatically created)

### Sequence Table Size

The `Sequence` table has one row per unique field. Even with thousands of models, this table remains small and fast.

## API Reference

### `FormattedAutoField`

```python
class FormattedAutoField(models.CharField):
    def __init__(
        self,
        format_string="{seq}",
        placeholders=None,
        start_at=1,
        *args,
        **kwargs
    ):
        ...
```

**Inherits from:** `django.db.models.CharField`

**Automatic settings:**
- `blank=True` - Always set (value is auto-generated)
- `editable=False` - Field not shown in forms

### `Sequence` Model

Internal model for tracking sequence values. Generally, you don't need to interact with this directly.

**Fields:**
- `key` (CharField): Unique identifier for the sequence
- `last_value` (PositiveIntegerField): Last number issued
- `created_at` (DateTimeField): When sequence was created
- `updated_at` (DateTimeField): Last increment time

## Common Patterns

### Year-Based Sequences

Reset sequence each year by using year in the key:

```python
# This approach creates separate sequences per year automatically
class Invoice(models.Model):
    invoice_number = FormattedAutoField(
        format_string="{year}-{seq:05d}",
        placeholders={"year": lambda: datetime.now().year}
    )
```

Result: `2026-00001`, `2026-00002`, ... then in 2027: `2027-00001`, `2027-00002`, ...

### Department-Specific Sequences

```python
class Request(models.Model):
    department = models.CharField(max_length=50)
    request_id = FormattedAutoField(
        format_string="{dept}-{seq:04d}",
        placeholders={"dept": lambda: get_current_department()}
    )
```

### Multi-tenant Applications

```python
class Order(models.Model):
    tenant = models.ForeignKey(Tenant, on_delete=models.CASCADE)
    order_number = FormattedAutoField(
        format_string="{tenant_code}-{seq:06d}",
        placeholders={"tenant_code": lambda: get_current_tenant().code}
    )
```

## Troubleshooting

### "Cannot serialize function: lambda" error in migrations

**Cause:** Django tries to serialize callable placeholders
**Solution:** This is expected. The lambda will be excluded from migrations automatically. Keep the field definition with the lambda in your model file.

### Duplicate IDs appearing

**Cause:** Not using database with row-level locking support
**Solution:** Use PostgreSQL, MySQL, or Oracle for production. SQLite is only recommended for development.

### IDs have gaps

**Cause:** Manual overrides or deleted records
**Solution:** This is normal behavior. Sequences are monotonically increasing but not necessarily contiguous.

### Sequence starts from 1 instead of `start_at` value

**Cause:** Sequence already exists from a previous migration/test
**Solution:** Delete the sequence record or manually set its value:

```python
from formatted_autofield.models import Sequence
Sequence.objects.filter(key='myapp.mymodel.myfield').delete()
```

## Contributing

Contributions are welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

## License

MIT License - see LICENSE file for details.

## Links

- **GitHub**: https://github.com/Perpay/django-formatted-autofield
- **PyPI**: https://pypi.org/project/django-formatted-autofield/
- **Issues**: https://github.com/Perpay/django-formatted-autofield/issues

## Changelog

### 0.1.0 (2026-02-02)

- Initial release
- FormattedAutoField with custom format strings
- Concurrency-safe sequence management
- Support for static and callable placeholders
- Django 4.2 LTS support
- Comprehensive test suite