Package: llm-connect
LLMConnect APITable of ContentsIntroductionFeaturesInstallationPrerequisitesCommandsPredefined tasksCommandOutputCommand examplesAuthorLicenseIntroductionLLMConnect API is a developer-friendly Python-based CLI utility designed to manage and evaluate Language Models including LLMs and SLMs on local servers or clusters. It enables users to run a variety of standard and custom tasks with popular models such as Llama-2, Mistral, Falcon, etc., and also supports the integration of new LLMs.FeaturesTask Versatility: Execute predefined tasks like NER, Sentiment Analysis, Summarisation, or craft your own.Model Selection: Choose and add your custom LLMs and SLMs from HuggingFace.Adaptable Environments: Operate seamlessly on local servers and extend to local network clusters.Hardware Compatibility: Ensure efficient LLM functioning with GPU compatibility checks and memory monitoring.CLI Interface:Navigate tasks, models, and hardware diagnostics with simple, intuitive commands.Development & Security:Developed in Python 3.x, emphasising seamless LLM integration, and detailed documentation.Features enhanced input validation for secure, reliable operations.Deployment:Eager to experience the power of Large Language Models through a Python-based Command Line Interface? LLMConnect API is your gateway to harnessing this technology on your local systems!InstallationRequired librariesPython 3.10click==8.1.7setuptools~=68.2.0transformerstorch~=2.1.0acceleratebitsandbytescoloramaCommandslc listList all available tasks or models.

  Usage:
    lc list [OPTIONS] COMMAND [ARGS]

  Options:
    -h, --help  Show this message and exit.

  Commands:
    models  List available models
    tasks   List available taskslc addAdd new Hugging Face Model. 

  Usage:
    lc add [OPTIONS]
    Model format: repo_id/model_id

  Options:
    --model TEXT  [required]
    -h, --help    Show this message and exit.lc removeRemove an existing HuggingFace Model.

  Usage:
    lc remove [OPTIONS]
    Model format: repo_id/model_id

  Options:
    --model TEXT  [required]
    -h, --help    Show this message and exit.lc hardwareCheck hardware compatibility for given Hugging Face model.

  Usage: 
    lc hardware [OPTIONS]
    Model format: repo_id/model_id.

  Options:
    --model TEXT  Model name in format: repoID/modelID  [required]
    -h, --help    Show this message and exit.lc execExecute an input prompt with given model and given task.

  Usage: 
    lc exec [OPTIONS]

  Options:
    --task TEXT   Specify the task name  [required]
    --model TEXT  Specify the model name (repoID/modelID)  [required]
    --input TEXT  Specify input text (optional)
    -h, --help    Show this message and exit.lc fetchFetch the logs of previous sessions.

  Usage:
    lc fetch [OPTIONS]

  Options:
    -h, --help  Show this message and exit.Predefined tasksCommandlc list tasksOutputAvailable Tasks:NERSummaryAnalyseSentimentDetectBiasTagTopicCustomCommand exampleslc list modelslc list taskslc add --model ceadar-ie/FinanceConnect-13Blc remove --model ceadar-ie/FinanceConnect-13Blc hardware --model ceadar-ie/FinanceConnect-13Blc exec --task NER --model ceadar-ie/FinanceConnect-13B --input "Hi! I'm LLMConnect API"lc fetchAuthorCeADAR Connect GroupLicenseAPACHE 2.0