cloudos-cli 2.80.0
Python package for interacting with CloudOS
Requires Python: >=3.9

# cloudos-cli

[![CI_tests](https://github.com/lifebit-ai/cloudos-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/lifebit-ai/cloudos-cli/actions/workflows/ci.yml)

Python package for interacting with CloudOS

---

## Table of Contents

- [cloudos-cli](#cloudos-cli)
  - [Table of Contents](#table-of-contents)
  - [Requirements](#requirements)
  - [Installation](#installation)
    - [From PyPI](#from-pypi)
    - [Docker Image](#docker-image)
    - [From Github](#from-github)
  - [Usage](#usage)
  - [Configuration](#configuration)
    - [Configure Default Profile](#configure-default-profile)
    - [Configure Named Profile](#configure-named-profile)
    - [Change the Default Profile](#change-the-default-profile)
    - [List Profiles](#list-profiles)
    - [Remove Profile](#remove-profile)
  - [Commands](#commands)
    - [Configure](#configure)
    - [Project](#project)
      - [List Projects](#list-projects)
      - [Create Projects](#create-projects)
    - [Queue](#queue)
      - [List Queues](#list-queues)
    - [Workflow](#workflow)
      - [List All Available Workflows](#list-all-available-workflows)
      - [Import a Nextflow Workflow](#import-a-nextflow-workflow)
    - [Nextflow Jobs](#nextflow-jobs)
      - [Submit a Job](#submit-a-job)
      - [Check Job Status](#check-job-status)
      - [List Jobs](#list-jobs)
      - [Get Job Results](#get-job-results)
      - [Clone or Resume Job](#clone-or-resume-job)
      - [Abort Jobs](#abort-jobs)
        - [Basic Usage](#basic-usage)
        - [Force Abort](#force-abort)
        - [Additional Options](#additional-options)
      - [Get Job Details](#get-job-details)
      - [Get Job Workdir](#get-job-workdir)
      - [Get Job Logs](#get-job-logs)
      - [Get Job Costs](#get-job-costs)
      - [Get Job Related Analyses](#get-job-related-analyses)
      - [Delete Job Results](#delete-job-results)
      - [Archive Jobs](#archive-jobs)
      - [Unarchive Jobs](#unarchive-jobs)
    - [Bash Jobs](#bash-jobs)
      - [Send Array Job](#send-array-job)
      - [Submit a Bash Array Job](#submit-a-bash-array-job)
        - [Options](#options)
          - [Array File](#array-file)
          - [Separator](#separator)
          - [List Columns](#list-columns)
          - [Array File Project](#array-file-project)
          - [Disable Column Check](#disable-column-check)
          - [Array Parameter](#array-parameter)
          - [Custom Script Path](#custom-script-path)
          - [Custom Script Project](#custom-script-project)
      - [Use multiple projects for files in `--parameter` option](#use-multiple-projects-for-files-in---parameter-option)
    - [Datasets](#datasets)
      - [List Files](#list-files)
      - [Move Files](#move-files)
      - [Rename Files](#rename-files)
      - [Copy Files](#copy-files)
      - [Link S3 Folders to Interactive Analysis](#link-s3-folders-to-interactive-analysis)
      - [Create Folder](#create-folder)
      - [Remove Files or Folders](#remove-files-or-folders)
    - [Link](#link)
      - [Link Folders to Interactive Analysis](#link-folders-to-interactive-analysis)
    - [Procurement](#procurement)
      - [List Procurement Images](#list-procurement-images)
      - [Set Procurement Organization Image](#set-procurement-organization-image)
      - [Reset Procurement Organization Image](#reset-procurement-organization-image)
    - [Cromwell and WDL Pipeline Support](#cromwell-and-wdl-pipeline-support)
      - [Manage Cromwell Server](#manage-cromwell-server)
      - [Run WDL Workflows](#run-wdl-workflows)
  - [Python API Usage](#python-api-usage)
      - [Running WDL pipelines using your own scripts](#running-wdl-pipelines-using-your-own-scripts)
  - [Unit Testing](#unit-testing)

---

## Requirements

CloudOS CLI requires Python 3.9 or higher and several key dependencies for API communication, data processing, and user interface functionality.

```
click>=8.0.1
pandas>=1.3.4
numpy>=1.26.4
requests>=2.26.0
rich_click>=1.8.2
```

---

## Installation

CloudOS CLI can be installed in multiple ways depending on your needs and environment. Choose the method that best fits your workflow.

### From PyPI

The repository is also available from [PyPI](https://pypi.org/project/cloudos-cli/):

```bash
pip install cloudos-cli
```

To update CloudOS CLI to the latest version using pip, you can run:

```bash
pip install --upgrade cloudos-cli
```

To check your current version:

```bash
cloudos --version
```

### Docker Image

It is recommended to install it as a docker image using the `Dockerfile` and the `environment.yml` files provided.

To run the existing docker image at `quay.io`:

```bash
docker run --rm -it quay.io/lifebitaiorg/cloudos-cli:latest
```


### From Github

You will need Python >= 3.9 and pip installed.

Clone the repo and install it using pip:

```bash
git clone https://github.com/lifebit-ai/cloudos-cli
cd cloudos-cli
pip install -r requirements.txt
pip install .
```

> NOTE: To be able to call the `cloudos` executable, ensure that the local clone of the `cloudos-cli` folder is included in the `PATH` variable, using for example the command `export PATH="/absolute/path/to/cloudos-cli:$PATH"`.

---

## Usage

CloudOS CLI can be used both as a command-line interface tool for interactive work and as a Python package for scripting and automation.

To get general information about the tool:

```bash
cloudos --help
```
```console
                                                                                                                        
 Usage: cloudos [OPTIONS] COMMAND [ARGS]...                                                                             
                                                                                                                        
 CloudOS python package: a package for interacting with CloudOS.                                                        
                                                                                                                        
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --debug        Show detailed error information and tracebacks                                                        │
│ --version      Show the version and exit.                                                                            │
│ --help         Show this message and exit.                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ bash               CloudOS bash functionality.                                                                       │
│ configure          CloudOS configuration.                                                                            │
│ cromwell           Cromwell server functionality: check status, start and stop.                                      │
│ datasets           CloudOS datasets functionality.                                                                   │
│ job                CloudOS job functionality: run, check and abort jobs in CloudOS.                                  │
│ procurement        CloudOS procurement functionality.                                                                │
│ project            CloudOS project functionality: list and create projects in CloudOS.                               │
│ queue              CloudOS job queue functionality.                                                                  │
│ workflow           CloudOS workflow functionality: list and import workflows.                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

This will tell you the implemented commands. Each implemented command has its own subcommands with its own `--help`:

```bash
cloudos job list --help
```
```console                                                                                                      Usage: cloudos job list [OPTIONS]                                                                          
                                                                                                            
 Collect workspace jobs from a CloudOS workspace in CSV or JSON format.                                     
                                                                                                            
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *  --apikey                    -k  TEXT        Your CloudOS API key [required]                           │
│ *  --cloudos-url               -c  TEXT        The CloudOS url you are trying to access to.              │
│                                                Default=https://cloudos.lifebit.ai.                       │
│                                                [required]                                                │
│ *  --workspace-id                  TEXT        The specific CloudOS workspace id. [required]             │
│    --output-basename               TEXT        Output file base name to save jobs list. Default=joblist  │
│    --output-format                 [csv|json]  The desired file format (file extension) for the output.  │
│                                                For json option --all-fields will be automatically set to │
│                                                True. Default=csv.                                        │
│    --all-fields                                Whether to collect all available fields from jobs or just │
│                                                the preconfigured selected fields. Only applicable when   │
│                                                --output-format=csv. Automatically enabled for json       │
│                                                output.                                                   │
│    --last-n-jobs                   TEXT        The number of last workspace jobs to retrieve. You can    │
│                                                use 'all' to retrieve all workspace jobs. Default=30.     │
│    --page                          INTEGER     Response page to retrieve. If --last-n-jobs is set, then  │
│                                                --page value corresponds to the first page to retrieve.   │
│                                                Default=1.                                                │
│    --archived                                  When this flag is used, only archived jobs list is        │
│                                                collected.                                                │
│    --filter-status                 TEXT        Filter jobs by status (e.g., completed, running, failed,  │
│                                                aborted).                                                 │
│    --filter-job-name               TEXT        Filter jobs by job name ( case insensitive ).             │
│    --filter-project                TEXT        Filter jobs by project name.                              │
│    --filter-workflow               TEXT        Filter jobs by workflow/pipeline name.                    │
│    --last                                      When workflows are duplicated, use the latest imported    │
│                                                workflow (by date).                                       │
│    --filter-job-id                 TEXT        Filter jobs by specific job ID.                           │
│    --filter-only-mine                          Filter to show only jobs belonging to the current user.   │
│    --filter-queue                  TEXT        Filter jobs by queue name. Only applies to jobs running   │
│                                                in batch environment. Non-batch jobs are preserved in     │
│                                                results.                                                  │
│    --filter-owner                  TEXT        Filter jobs by owner username.                            │
│    --verbose                                   Whether to print information messages or not.             │
│    --disable-ssl-verification                  Disable SSL certificate verification. Please, remember    │
│                                                that this option is not generally recommended for         │
│                                                security reasons.                                         │
│    --ssl-cert                      TEXT        Path to your SSL certificate file.                        │
│    --debug                                     Show detailed error information and tracebacks               │
│    --profile                       TEXT        Profile to use from the config file                          │
│    --help                                      Show this message and exit.                                  │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

In the same way, each implemented command has its own subcommands with its own `--debug` flag, that will print the full traceback for detailed error debugging. When this flag is not activated, the errors are presented in short descriptive format.

---

## Configuration

CloudOS CLI uses a profile-based configuration system to store your credentials and settings securely. This eliminates the need to provide authentication details with every command and allows you to work with multiple CloudOS environments.

Configuration will be saved in the $HOME path folder regardless of operating system. Here, a new folder named `.cloudos` will be created, with files `credentials` and `config` also being created. The structure will look like:

```console
$HOME
    └── .cloudos/
          ├── credentials     # Stores API keys
          └── config          # Stores all other parameters
```

### Configure Default Profile

To facilitate the reuse of required parameters, you can create profiles. 

To generate a profile called `default`, use the following command:

```bash
cloudos configure
```

This will prompt you for API key, platform URL, project name, platform executor, repository provider, workflow name (if any), and session ID for interactive analysis. This becomes the default profile if no other profile is explicitly set. The default profile allows running all subcommands without adding the `--profile` option.

### Configure Named Profile

To generate a named profile, use the following command:

```bash
cloudos configure --profile {profile-name}
```

The same prompts will appear. If a profile with the same name already exists, the current parameters will appear in square brackets and can be overwritten or left unchanged by pressing Enter/Return.

> [!NOTE]
> When there is already at least 1 previous profile defined, a new question will appear asking to make the current profile as default

### Change the Default Profile

Change the default profile with:

```bash
cloudos configure --profile {other-profile} --make-default
```

### List Profiles

View all configured profiles and identify the default:

```bash
cloudos configure list-profiles
```

The response will look like:

```console
Available profiles:
 - default (default)
 - second-profile
 - third-profile
```

### Remove Profile

Remove any profile with:

```bash
cloudos configure remove-profile --profile second-profile
```

---

## Commands

### Configure

See [Configuration](#configuration) section above for detailed information on setting up profiles and managing your CloudOS CLI configuration.



### Project

Projects in CloudOS provide logical separation of datasets, workflows, and results, making it easier to manage complex research initiatives. You can list all available projects or create new ones using the CLI.

#### List Projects

You can get a summary of all available workspace projects in two different formats:
- **CSV**: A table with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
- **JSON**: All available information from projects in JSON format

To get a CSV table with all available projects for a given workspace:

```bash
cloudos project list --profile my_profile --output-format csv --all-fields
```

The expected output is something similar to:

```console
Executing list...
	Project list collected with a total of 320 projects.
	Project list saved to project_list.csv
```

To get the same information in JSON format:

```bash
cloudos project list --profile my_profile --output-format json
```

#### Create Projects

You can create a new project in your CloudOS workspace using the `project create` command. This command requires the name of the new project and will return the project ID upon successful creation.

```bash
cloudos project create --profile my_profile --new-project "My New Project"
```

The expected output is something similar to:

```console
	Project "My New Project" created successfully with ID: 64f1a23b8e4c9d001234abcd
```


### Queue

Job queues are required for running jobs using AWS batch executor. The available job queues in your CloudOS workspace are listed in the "Compute Resources" section in "Settings". You can get a summary of all available workspace job queues in two formats:
- **CSV**: A table with a selection of the available job queue information. You can get all information using the `--all-fields` flag
- **JSON**: All available information from job queues in JSON format

#### List Queues

This command allows you to view available computational queues and their configurations. Example command for getting all available job queues in JSON format:

```bash
cloudos queue list --profile my_profile --output-format json --output-basename "available_queues"
```

```console
Executing list...
	Job queue list collected with a total of 5 queues.
	Job queue list saved to available_queues.json
```

This command will output the list of available job queues in JSON format and save it to a file named `available_queues.json`. You can use `--output-format csv` for a CSV file, or omit `--output-basename` to print to the console.

> NOTE: The queue name that is visible in CloudOS and must be used with the `--job-queue` parameter is the one in the `label` field.

**Job queues for platform workflows**

Platform workflows (those provided by CloudOS in your workspace as modules) run on separate and specific AWS batch queues. Therefore, CloudOS will automatically assign the valid queue and you should not specify any queue using the `--job-queue` parameter. Any attempt to use this parameter will be ignored. Examples of such platform workflows are "System Tools" and "Data Factory" workflows.


### Workflow

#### List All Available Workflows

You can get a summary of all available workspace workflows in two different formats:
- **CSV**: A table with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
- **JSON**: All available information from workflows in JSON format

To get a CSV table with all available workflows for a given workspace:

```bash
cloudos workflow list --profile my_profile --output-format csv --all-fields
```

The expected output is something similar to:

```console
Executing list...
	Workflow list collected with a total of 609 workflows.
	Workflow list saved to workflow_list.csv
```

To get the same information in JSON format:

```bash
cloudos workflow list --profile my_profile --output-format json
```

```console
Executing list...
	Workflow list collected with a total of 609 workflows.
	Workflow list saved to workflow_list.json
```

The collected workflows are those that can be found in the "WORKSPACE TOOLS" section in CloudOS.

#### Import a Nextflow Workflow

You can import new workflows to your CloudOS workspaces. The requirements are:

- The workflow must be a Nextflow pipeline
- The workflow repository must be located at GitHub, GitLab or BitBucket Server (specified by the `--repository-platform` option. Available options: `github`, `gitlab` and `bitbucketServer`)
- If your repository is private, you must have access to the repository and have linked your GitHub, Gitlab or Bitbucket server accounts to CloudOS

**Usage of the workflow import command**

To import GitHub workflows to CloudOS:

```bash
# Example workflow to import: https://github.com/lifebit-ai/DeepVariant
cloudos workflow import --profile my_profile --workflow-url "https://github.com/lifebit-ai/DeepVariant" --workflow-name "new_name_for_the_github_workflow" --repository-platform github
```

The expected output will be:

```console
CloudOS workflow functionality: list and import workflows.

Executing workflow import...

	Only Nextflow workflows are currently supported.

	Workflow test_import_github_3 was imported successfully with the following ID: 6616a8cb454b09bbb3d9dc20
```

Optionally, you can add a link to your workflow documentation by providing the URL using the `--workflow-docs-link` parameter:

```bash
cloudos workflow import --profile my_profile --workflow-url "https://github.com/lifebit-ai/DeepVariant" --workflow-name "new_name_for_the_github_workflow" --workflow-docs-link "https://github.com/lifebit-ai/DeepVariant/blob/master/README.md" --repository-platform github
```

> NOTE: Importing workflows using cloudos-cli is not yet available in all CloudOS workspaces. If you try to use this feature in a non-prepared workspace you will get the following error message: `It seems your API key is not authorised. Please check if your workspace has support for importing workflows using cloudos-cli`.


### Nextflow Jobs

The job commands allow you to submit, monitor, and manage computational workflows on CloudOS. This includes both Nextflow pipelines and bash scripts, with support for various execution platforms.

#### Submit a Job

You can submit Nextflow workflows to CloudOS using either configuration files or command-line parameters. Jobs can be configured with specific compute resources, execution platforms, parameters, etc.

First, configure your local environment to ease parameter input. We will try to submit a small toy example already available:

```bash
cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable
```

As you can see, a file with the job parameters is used to configure the job. This file could be a regular `nextflow.config` file or any file with the following structure:

```
params {
        reads   = s3://lifebit-featured-datasets/pipelines/rnatoy-data
        annot   = s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.bed.gff
}
```

In addition, parameters can also be specified using the command-line `-p` or `--parameter`. For instance:

```bash
cloudos job run \
  --profile my_profile \
  --workflow-name rnatoy \
  --parameter reads=s3://lifebit-featured-datasets/pipelines/rnatoy-data \
  --parameter genome=s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.Ggal71.500bpflank.fa \
  --parameter annot=s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.bed.gff \
  --resumable
```

**Params file**

You can pass a Nextflow-style params file using `--params-file` (only JSON or YAML):

```bash
cloudos job run \
  --profile my_profile \
  --workflow-name rnatoy \
  --params-file Data/params.json \
  --resumable
```

Example JSON params file:

```json
{
  "reads": "s3://lifebit-featured-datasets/pipelines/rnatoy-data",
  "genome": "s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.Ggal71.500bpflank.fa",
  "annot": "s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.bed.gff"
}
```

Example YAML params file:

```yaml
reads:
  s3://lifebit-featured-datasets/pipelines/rnatoy-data
genome:
  s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.Ggal71.500bpflank.fa
annot:
  s3://lifebit-featured-datasets/pipelines/rnatoy-data/ggal_1_48850000_49020000.bed.gff
```

> NOTE: options `--job-config`, `--parameter` and `--params-file` are completely compatible and complementary, so you can use a `--job-config` or `--params-file` and add additional parameters using `--parameter` in the same call.

> NOTE: when using `--params-file`, the value must be an S3 URI or a File Explorer relative path (e.g., `Data/file.json`). Local file paths are not supported.

If everything went well, you should see something like:

```console
Executing run...
	Job successfully launched to CloudOS, please check the following link: https://cloudos.lifebit.ai/app/advanced-analytics/analyses/62c83a1191fe06013b7ef355
	Your assigned job id is: 62c83a1191fe06013b7ef355
	Your current job status is: initializing
	To further check your job status you can either go to https://cloudos.lifebit.ai/app/advanced-analytics/analyses/62c83a1191fe06013b7ef355 or use the following command:
    cloudos job status \
        --apikey $MY_API_KEY \
        --cloudos-url https://cloudos.lifebit.ai \
        --job-id 62c83a1191fe06013b7ef355
```

As you can see, the current status is `initializing`. This will change while the job progresses. To check the status, just apply the suggested command.

Another option is to set the `--wait-completion` parameter, which runs the same job run command but waits for its completion:

```bash
cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable --wait-completion
```

When setting this parameter, you can also set `--request-interval` to a bigger number (default is 30s) if the job is quite large. This will ensure that the status requests are not sent too close from each other and recognized as spam by the API.

If the job takes less than `--wait-time` (3600 seconds by default), the previous command should have an output similar to:

```console
Executing run...
	Job successfully launched to CloudOS, please check the following link: https://cloudos.lifebit.ai/app/advanced-analytics/analyses/62c83a6191fe06013b7ef363
	Your assigned job id is: 62c83a6191fe06013b7ef363
	Please, wait until job completion or max wait time of 3600 seconds is reached.
	Your current job status is: initializing.
	Your current job status is: running.
	Your job took 420 seconds to complete successfully.
```

When there are duplicate `--workflow-name` in the platform, you can add the `--last` flag to use the latest import of that pipeline in the workspace, based on the date.

_For example, the pipeline `lifebit-process` was imported on May 23 2025 and again on May 30 2025; with the `--last` flag, it will use the import of May 30, 2025._

**AWS Executor Support**

CloudOS supports [AWS batch](https://www.nextflow.io/docs/latest/executor.html?highlight=executors#aws-batch) executor by default.
You can specify the AWS batch queue to use from the ones available in your workspace (see [here](#list-job-queues)) by specifying its name with the `--job-queue` parameter. If none is specified, the most recent suitable queue in your workspace will be selected by default.

Example command:

```bash
cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable
```

> Note: From cloudos-cli 2.7.0, the default executor is AWS batch. The previous Apache [ignite](https://www.nextflow.io/docs/latest/ignite.html#apache-ignite) executor is being removed progressively from CloudOS, so most likely will not be available in your CloudOS. Cloudos-cli still supports ignite during this period by adding the `--ignite` flag to the `cloudos job run` command. Please note that if you use the `--ignite` flag in a CloudOS without ignite support, the command will fail.

**Azure Execution Platform Support**

CloudOS can also be configured to use Microsoft Azure compute platforms. If your CloudOS is configured to use Azure, you will need to take into consideration the following:

- When sending jobs to CloudOS using `cloudos job run` command, please use the option `--execution-platform azure`
- Due to the lack of AWS batch queues in Azure, `cloudos queue list` command is not working

Other than that, `cloudos-cli` will work very similarly. For instance, this is a typical send job command:

```bash
cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable --execution-platform azure
```

**HPC Execution Support**

CloudOS is also prepared to use an HPC compute infrastructure. For such cases, you will need to take into account the following for your job submissions using `cloudos job run` command:

- Use the following parameter: `--execution-platform hpc`
- Indicate the HPC ID using: `--hpc-id XXXX`

Example command:

```bash
cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --execution-platform hpc --hpc-id $YOUR_HPC_ID
```

Please note that HPC execution does not support the following parameters and all of them will be ignored:

- `--job-queue`
- `--resumable | --do-not-save-logs`
- `--instance-type` | `--instance-disk` | `--cost-limit`
- `--storage-mode` | `--lustre-size`
- `--wdl-mainfile` | `--wdl-importsfile` | `--cromwell-token`

#### Check Job Status

To check the status of a submitted job, use the following command:

```bash
cloudos job status --profile my_profile --job-id 62c83a1191fe06013b7ef355
```

The expected output should be something similar to:

```console
Executing status...
	Your current job status is: completed

	To further check your job status you can either go to https://cloudos.lifebit.ai/app/advanced-analytics/analyses/62c83a1191fe06013b7ef355 or repeat the command you just used.
```

#### List Jobs

View your workspace jobs in a clean, formatted table directly in your terminal. The table automatically adapts to your terminal width, showing different column sets for optimal viewing. By default, jobs are displayed as a rich table with job IDs and colored visual status indicators.

**Output Formats**

CloudOS CLI provides three output formats for job listings:

- **Table (default)**: Rich formatted table displayed in the terminal with pagination information
- **CSV**: Tabular format with predefined or all available columns using `--all-fields`
- **JSON**: Complete job information in JSON format (`--all-fields` is always enabled)

**Default Behavior**

By default, the command displays the 10 most recent jobs in a formatted table:

```bash
cloudos job list --profile my_profile
```

The output shows a rich table with job information and pagination details:

```console
Executing list...

                                                    Job List                                                    
┏━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Status ┃ Name         ┃ Project     ┃ Owner    ┃ Pipeline     ┃ ID                      ┃ Submit time  ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ ✓      │ analysis_run │ test-proj   │ John     │ rnatoy       │ 692ee71c40e98ed6ed529e43│ 2025-12-02   │
│        │              │             │ Doe      │              │                         │ 15:30:45     │
│ ◐      │ test_job     │ research    │ Jane     │ VEP          │ 692ee81d50f98ed7fe639f54│ 2025-12-02   │
│        │              │             │ Smith    │              │                         │ 14:20:30     │
└────────┴──────────────┴─────────────┴──────────┴──────────────┴─────────────────────────┴──────────────┘

Showing 10 of 45 total jobs | Page 1 of 5
```

**Status Indicators**

Jobs are displayed with colored visual status indicators:
- **Green ✓** Completed
- **Grey ◐** Running
- **Red ✗** Failed
- **Orange ■** Aborted
- **Grey ○** Initialising

**Clickable Job IDs**

Job IDs in the table are clickable hyperlinks (when supported by your terminal) that open the job details page in CloudOS.

**Job Listing Control Options**

CloudOS CLI provides two ways to control the number of jobs retrieved:

1. **Pagination Control (Default)**: Use `--page` and `--page-size` for precise pagination
2. **Last N Jobs**: Use `--last-n-jobs` for retrieving the most recent jobs

> [!IMPORTANT]
> **These options are mutually exclusive**. When `--last-n-jobs` is specified, it takes precedence and `--page`/`--page-size` parameters are ignored. A warning message will be displayed if both are provided.

**Pagination Examples**

Retrieve specific pages using `--page` and `--page-size`:

```bash
# Get page 2 with 15 jobs per page
cloudos job list --profile my_profile --page 2 --page-size 15

# Get page 5 with maximum 100 jobs per page (maximum allowed)
cloudos job list --profile my_profile --page 5 --page-size 100
```

> [!NOTE]
> `--page-size` has a maximum limit of 100 jobs per page. Attempting to use a larger value will result in an error.

**Last N Jobs Examples**

Use `--last-n-jobs` to get the most recent jobs:

```bash
# Get the last 50 jobs
cloudos job list --profile my_profile --last-n-jobs 50

# Get all workspace jobs
cloudos job list --profile my_profile --last-n-jobs all
```

**Customizing Table Columns**

You can customize which columns are displayed in the table using the `--table-columns` option:

```bash
# Show only status, name, and cost columns
cloudos job list --profile my_profile --table-columns status,name,cost

# Show a minimal view
cloudos job list --profile my_profile --table-columns status,name,id,submit_time
```

Available columns: `status`, `name`, `project`, `owner`, `pipeline`, `id`, `submit_time`, `end_time`, `run_time`, `commit`, `cost`, `resources`, `storage_type`

> [!NOTE]
> The `--table-columns` option only applies when using the default table output format (stdout).

**File Output Formats**

To save job lists to files instead of displaying them in the terminal:

```bash
# Save as CSV with default columns
cloudos job list --profile my_profile --output-format csv

# Save as CSV with all available fields
cloudos job list --profile my_profile --output-format csv --all-fields

# Save as JSON with complete job data
cloudos job list --profile my_profile --output-format json
```

The expected output for file formats:

```console
Executing list...
	Job list collected with a total of 10 jobs.
	Job list saved to joblist.csv
```

**Filtering Jobs**

You can find specific jobs within your workspace using the filtering options. Filters can be combined to narrow down results and work with all output formats.

**Available filters:**

- **`--filter-status`**: Filter jobs by execution status (e.g., completed, running, failed, aborted, initialising)
- **`--filter-job-name`**: Filter jobs by job name (case insensitive partial matching)
- **`--filter-project`**: Filter jobs by project name (exact match required)
- **`--filter-workflow`**: Filter jobs by workflow/pipeline name (exact match required)
- **`--filter-job-id`**: Filter jobs by specific job ID (exact match required)
- **`--filter-only-mine`**: Show only jobs belonging to the current user
- **`--filter-owner`**: Show only jobs for the specified owner (exact match required, e.g., "John Doe")
- **`--filter-queue`**: Filter jobs by queue name (only applies to batch jobs)

**Filtering Examples**

Using pagination approach (default):

```bash
# Get completed jobs from page 1 (default 10 jobs)
cloudos job list --profile my_profile --filter-status completed

# Get completed jobs from page 2 with 20 jobs per page
cloudos job list --profile my_profile --page 2 --page-size 20 --filter-status completed
```

Using last-n-jobs approach:

```bash
# Get all completed jobs from the last 50 jobs
cloudos job list --profile my_profile --last-n-jobs 50 --filter-status completed
```

Find jobs with "analysis" in the name from a specific project:

```bash
# Using pagination (gets first 10 matching jobs)
cloudos job list --profile my_profile --filter-job-name analysis --filter-project "My Research Project"

# Using last-n-jobs
cloudos job list --profile my_profile --last-n-jobs 100 --filter-job-name analysis --filter-project "My Research Project"
```

Get all jobs using a specific workflow and queue:

```bash
# Using pagination with larger page size
cloudos job list --profile my_profile --page-size 50 --filter-workflow rnatoy --filter-queue high-priority-queue

# Using last-n-jobs to search all jobs
cloudos job list --profile my_profile --last-n-jobs all --filter-workflow rnatoy --filter-queue high-priority-queue
```

> [!NOTE]
> - Project and workflow names must match exactly (case sensitive)
> - Job name filtering is case insensitive and supports partial matches
> - The `--last` flag can be used with `--filter-workflow` when multiple workflows have the same name
> - When filters are applied, pagination information reflects the filtered results

#### Get Job Results

The following command allows you to get the path where CloudOS stores the output files for a job. This can be used only on your user's jobs and for jobs with "completed" status.

Example:
```bash
cloudos job results --profile my_profile --job-id "12345678910"
```
```console
Executing results...
results: s3://path/to/location/of/results/results/
```

You can also link all result directories to an interacti