fhi/smartflexpack-morl

SmartFlexPack MORL: Sequential Attention Policy for Packaging Scheduling

Trained model weights for the paper:

Multi-Objective Reinforcement Learning for Adaptive Packaging Scheduling via Sequential Attention Policy
L. Venturoso, G. Egger, F. Visentin, M. Todescato
Fraunhofer Italia Research EFRE1054 Smartflexpack
Manuscript in preparation

Model description

SeqAttentionActorCritic (12,355 parameters) decides, for each order, between inline printing and labelling. At each planning event a one-layer multi-head self-attention encoder (4 heads, d=32) encodes the pending orders; for each decision a 54→64→64 MLP combines that embedding with ten live decision features and twelve global features, including the preference vector (tardiness, cost, CO2), and outputs two action logits and a value.

A single trained policy covers the whole trade-off: the operator sets the preference at inference time, with no retraining.

Results

In the reference plant configuration (3 label lines, inline 0.45 s/pack) the policy exceeds the hypervolume of the full family of 32 realisable reference controllers by +0.058 (95% CI [+0.025, +0.095], about 7%), replicated on an independent seed set (+0.050). The advantage grows with a slower inline line (12–15% at 0.60 s/pack); with a single label line, and with the fastest inline line, the policy ties the reference family or falls slightly below it.

Repository structure

ppo/seed_{1..5}/policy_200.pt   # final PPO checkpoints (5 independent seeds)
ppo/seed_{1..5}/metrics.csv     # training metrics
bc/bc_seq_morl.pt               # behavioural-cloning warm start
configs/paper.json              # training configuration
normalisation/                  # observation and reward normalisers
inference/seq_morl/             # minimal code to load the policy
manifest.json                   # source revision and SHA-256 of every file

Inference

import sys, torch
from pathlib import Path
from huggingface_hub import snapshot_download

release = Path(snapshot_download("fhi/smartflexpack-morl"))
sys.path.insert(0, str(release / "inference"))
from seq_morl.policies.attention import SeqAttentionActorCritic

model = SeqAttentionActorCritic()
model.load_state_dict(torch.load(release / "ppo/seed_1/policy_200.pt",
                                 map_location="cpu", weights_only=True))
model.eval()

with torch.no_grad():
    ctx = model.encode_sequence(order_feats)              # [N, 32], pending orders
    logits, value = model.forward_single(ctx[i], extra)   # extra: 10 decision + 12 global
    action = logits.argmax(dim=-1).item()                 # 0 = inline, 1 = label

Simulation and evaluation: experiments/evaluate_policy.py in the project repository; it also needs the separate DataGenerator.

Training

Behavioural-cloning warm start, then 5 independent seeds of 200 PPO updates, 10 parallel episodes of 600 orders, lr=3e-4 (Adam). About 3.8 h CPU per seed.

Training code: experiments/train_bc_warm_start.py, experiments/train_policy.py

Citation

@unpublished{venturoso2026morl,
  title  = {Multi-Objective Reinforcement Learning for Adaptive Packaging
            Scheduling via Sequential Attention Policy},
  author = {Venturoso, Leonardo and Egger, Georg and
            Visentin, Francesco and Todescato, Marco},
  year   = {2026},
  note   = {Manuscript in preparation}
}

License

CC BY 4.0 Fraunhofer Italia Research, EFRE1054 Smartflexpack project (funded under FESR 2021–27, Priority 1 "Smart" R&I programme).