moostral: Fine-tuning of Mistral for Multi-objective optimization problem translation

moostral is a fine-tuning on Mistral 7B with the goal of translating multi-objective optimization problems from a textual representation to a jMetal implementation automatically.

A graphical tool for using the model, the code for training and inference can be found at gitlab.com/jfaldanam-phd/moostral.

The latest release of the model weights can be found at huggingface.co/jfaldanam/moostral-7B.

Get started

The easiest way to get started using moostral is via the provided dashboard and using the latest release of the model from Hugging Face.

$ git clone https://gitlab.com/jfaldanam-phd/moostral
$ cd moostral
$ pip install ".[dashboard]"
$ python -m moostral dashboard

  You can now view your Streamlit app in your browser.

  Network URL: http://127.0.0.1:8501

moostral is trained on the following format:

### Textual problem:
{Problem definition in natural language}
### Java implementation:
{jMetal implementation of the textual problem}

A complete example could be:

### Textual problem:
f(x) = exp(x[1]) + x[2]^2 + sin(x[3])

g(x) = x[1]^2 + exp(x[2]) + cos(x[3])

h(x) = tanh(x[1]) - x[2]*x[3] + sin(x[1])

Value ranges:
x[1] in [-3, 3]
x[2] in [-3, 3]
x[3] in [0, 2*PI]
### Java implementation:
package xyz.jfaldanam;

import java.util.List;
import org.uma.jmetal.problem.doubleproblem.impl.AbstractDoubleProblem;
import org.uma.jmetal.solution.doublesolution.DoubleSolution;

public class ProblemY extends AbstractDoubleProblem {

  /** Constructor */
  public ProblemY() {
    int numberOfVariables = 3;
    numberOfObjectives(3);
    numberOfConstraints(0);
    name("ProblemY");

    List<Double> lowerLimit = List.of(-3.0, -3.0, 0.0);
    List<Double> upperLimit = List.of(3.0, 3.0, 6.283);

    variableBounds(lowerLimit, upperLimit);
  }

  /** Evaluate() method */
  @Override
  public DoubleSolution evaluate(DoubleSolution solution) {
    double x1 = solution.variables().get(0);
    double x2 = solution.variables().get(1);
    double x3 = solution.variables().get(2);
    
    solution.objectives()[0] = Math.exp(x1) + Math.pow(x2, 2) + Math.sin(x3);
    solution.objectives()[1] = Math.pow(x1, 2) + Math.exp(x2) + Math.cos(x3);
    solution.objectives()[2] = Math.tanh(x1) - x2 * x3 + Math.sin(x1);
    
    return solution;
  }
}

Technical details

Fine-tuning

This project is not meant as a super-configurable tool for fine-tuning LLMs, but as a practical tool for research, expect not a lot of parameters to be available through the public APIs.

For fine-tuning, a training dataset is expected as a folder in the format provided by jfaldanam-phd/syntheticai. Fine-tuning is achieved using the Hugging Face transformers library.

$ python -m moostral finetune [-h] --dataset DATASET [--base-model BASE_MODEL] [--output OUTPUT]

After fine-tuning, the new version of the model can be uploaded to Hugging Face:

$ python -m moostral push2hf --checkpoint-folder CHECKPOINT_FOLDER --hf-token HF_TOKEN [--repository REPOSITORY]

Inference

There are 3 main ways to use the model for inference:

Command line: Providing either with the latest release of hugging face or via a local checkpoint folder.

$ python -m moostral inference [-h] [--checkpoint-folder CHECKPOINT_FOLDER] "$PROMPT"

Importing the project: Using the provided Python functions

from moostral.inference import inference

PROMPT = "..."
inference(PROMPT, "jfaldanam/moostral-7B")

Without importing the project: The latest version of the fine-tuned model is available on Hugging Face under jfaldanam/moostral-7B and can be used with the standard interface of the transformers library.

from peft import PeftModel, PeftConfig
from transformers import AutoTokenizer, AutoModelForCausalLM
config = PeftConfig.from_pretrained("jfaldanam/moostral-7B")
base_model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path)
model = PeftModel.from_pretrained(base_model, "jfaldanam/moostral-7B")
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

Validation of the generated problems

The Python modules includes code to automatically validate the generated problems. This is automatically done when using the provided dashboard, but the code is also available under the moostral.validation module.

Acknowledgements

Mistral fine-tuning code inspired from https://github.com/brevdev/notebooks/blob/main/mistral-finetune-own-data.ipynb

jfaldanam
/

moostral-7B