testzeus-sdk 0.0.15
Python SDK for TestZeus testing platform
License: MIT
Requires Python: >=3.11

# TestZeus SDK

Python SDK for the TestZeus testing platform.

## Installation

Install the package using pip:

```bash
pip install testzeus-sdk
```

Or use Poetry:

```bash
poetry add testzeus-sdk
```

## Components

### Python SDK
The TestZeus SDK provides programmatic access to the TestZeus testing platform through a Python interface.

### Command Line Interface (CLI)
The TestZeus CLI provides a command-line interface for interacting with the TestZeus platform. For detailed CLI documentation, see the [CLI README](testzeus-cli/README.md).

```bash
# Install CLI
pip install testzeus-cli

# Login to TestZeus
testzeus login

# List tests
testzeus tests list
```

## Getting Started with SDK

```python
import asyncio
from testzeus_sdk import TestZeusClient

async def main():
    # Create a client with email/password
    client = TestZeusClient(email="your-email", password="your-password")
    
    # Use as a context manager
    async with client:
        # Your code here
        pass

# Run the example
asyncio.run(main())
```

## Authentication

The SDK supports three authentication methods:

### Email/Password

```python
client = TestZeusClient(email="your-email", password="your-password")
```

### Environment Variables

```
export TESTZEUS_EMAIL="your-email"
export TESTZEUS_PASSWORD="your-password"
```

Then create the client without parameters:

```python
client = TestZeusClient()
```

## Core Functionality

### Tests Management

#### List Tests
```python
# Get list of tests with filters and sorting
tests = await client.tests.get_list(
    expand='tags',  # Expand related entities
    sort='id',      # Sort by field
    filters={       # Filter results
        'id': 'y9b88f17vabx476'
    }
)
print(tests)
print(tests['items'][0].data)  # Access test data
```

#### Advanced Filtering

The SDK supports powerful filtering capabilities using PocketBase filter syntax. All managers that extend `BaseManager` support these filtering options:

##### Simple Filters (Backward Compatible)
```python
# Basic field matching
filters = {
    "name": "Test Name",
    "status": "active",
    "priority": 5
}

# List values (OR condition)
filters = {
    "status": ["active", "pending", "draft"]  # status = "active" OR status = "pending" OR status = "draft"
}
```

##### Advanced Operators
```python
# Use complex operators with value objects
filters = {
    "created": {"operator": ">", "value": "2023-01-01"},           # created > "2023-01-01"
    "priority": {"operator": ">=", "value": 3},                   # priority >= 3
    "name": {"operator": "~", "value": "test"},                   # name LIKE "%test%"
    "description": {"operator": "!~", "value": "old"}            # description NOT LIKE "%old%"
}
```

##### Supported Operators

**Comparison Operators:**
- `=` - Equal (default when no operator specified)
- `!=` - Not equal
- `>` - Greater than
- `>=` - Greater than or equal
- `<` - Less than
- `<=` - Less than or equal

**String Operators:**
- `~` - Like/Contains (auto-wraps with % for wildcard matching)
- `!~` - Not Like/Contains

**Array Operators (for multi-value fields):**
- `?=` - Any/At least one equal
- `?!=` - Any/At least one not equal
- `?>` - Any/At least one greater than
- `?>=` - Any/At least one greater than or equal
- `?<` - Any/At least one less than
- `?<=` - Any/At least one less than or equal
- `?~` - Any/At least one like/contains
- `?!~` - Any/At least one not like/contains

##### Array Operators with Lists
```python
# Check if any tag matches the values
filters = {
    "tags": {"operator": "?=", "value": ["urgent", "important"]}
}
# Result: (tags ?= "urgent" || tags ?= "important")
```

##### Logical Grouping

**AND Conditions:**
```python
filters = {
    "$and": [
        {"status": "active"},
        {"priority": {"operator": ">", "value": 3}},
        {"created": {"operator": ">", "value": "2023-01-01"}}
    ]
}
# Result: (status = "active" && priority > 3 && created > "2023-01-01")
```

**OR Conditions:**
```python
filters = {
    "$or": [
        {"status": "urgent"},
        {"priority": {"operator": ">=", "value": 8}}
    ]
}
# Result: (status = "urgent" || priority >= 8)
```

**Complex Combinations:**
```python
filters = {
    "tenant": "abc123",  # Always filter by current tenant
    "$or": [
        {"status": "active"},
        {
            "$and": [
                {"status": "draft"},
                {"modified": {"operator": ">", "value": "2023-01-01"}},
                {"tags": {"operator": "?=", "value": ["review", "pending"]}}
            ]
        }
    ]
}
# Result: tenant = "abc123" && (status = "active" || (status = "draft" && modified > "2023-01-01" && (tags ?= "review" || tags ?= "pending")))
```

##### Practical Examples

**Filter tests by date range:**
```python
tests = await client.tests.get_list(
    filters={
        "$and": [
            {"created": {"operator": ">=", "value": "2023-01-01"}},
            {"created": {"operator": "<", "value": "2023-12-31"}}
        ]
    }
)
```

**Filter by multiple statuses and search in name:**
```python
tests = await client.tests.get_list(
    filters={
        "status": ["active", "pending"],
        "name": {"operator": "~", "value": "integration"}
    }
)
```

**Filter test runs by status and date:**
```python
test_runs = await client.test_runs.get_list(
    filters={
        "$and": [
            {"status": {"operator": "!=", "value": "draft"}},
            {"start_time": {"operator": ">", "value": "2023-01-01T00:00:00Z"}}
        ]
    },
    sort="-created"  # Sort by created date descending
)
```

**Filter environments by tags:**
```python
environments = await client.environments.get_list(
    filters={
        "tags": {"operator": "?=", "value": ["production", "staging"]}
    }
)
```

#### Get Single Test
```python
# Get test by ID
test = await client.tests.get_one('311137kown88nd6')
print(test.data)
```

#### Create Test
```python
# Create a new test
new_test = await client.tests.create(
    name="New Test",
    test_feature="Example feature",
    status="draft",  # Optional: 'draft', 'ready', 'deleted'
    test_data=["data_id1", "data_id2"],  # Optional: List of test data IDs
    tags=["tag1", "tag2"],  # Optional: List of tag IDs
    environment="env_id"  # Optional: Environment ID
)
```

#### Update Test
```python
# Update test properties
updated_test = await client.tests.update(
    'test_id',
    name='Updated Test Name'
)
print(updated_test.data)
```

#### Delete Test
```python
# Delete a test
await client.tests.delete('test_id')
```

### Test Runs Management


#### Track Test Run Status
```python
# Get test run by ID
test_run = await client.test_runs.get_one('test_run_id')

# Check test run status
if test_run.is_running():
    print("Test is currently running")
elif test_run.is_completed():
    print("Test has completed successfully")
elif test_run.is_failed():
    print("Test has failed")
elif test_run.is_crashed():
    print("Test has crashed")
elif test_run.is_cancelled():
    print("Test was cancelled")
elif test_run.is_pending():
    print("Test is pending")

# Get test run duration
duration = test_run.get_duration()
if duration:
    print(f"Test run took {duration} seconds")
```

#### Get Detailed Test Run Information
```python
# Get expanded test run details including all outputs, steps, and attachments
expanded_test_run = await client.test_runs.get_expanded('test_run_id')

# Access different components of the test run
test_run_data = expanded_test_run['test_run']
test_run_dashs = expanded_test_run['test_run_dashs']
test_run_dash_outputs = expanded_test_run['test_run_dash_outputs']
test_run_dash_output_steps = expanded_test_run['test_run_dash_output_steps']
test_run_dash_outputs_attachments = expanded_test_run['test_run_dash_outputs_attachments']

# Print test run details
print(f"Test Run Name: {test_run_data['name']}")
print(f"Status: {test_run_data['status']}")
print(f"Start Time: {test_run_data['start_time']}")
print(f"End Time: {test_run_data['end_time']}")

# Print test run steps
for step in test_run_dash_output_steps:
    print(f"Step: {step['name']}")
    print(f"Status: {step['status']}")
    print(f"Is Passed: {step['is_passed']}")
    print(f"Assert Summary: {step['assert_summary']}")
```

#### Cancel Test Run

```python
# Cancel a running test
try:
    cancelled_test = await client.test_runs.cancel('test_run_id')
    print(f"Test run cancelled: {cancelled_test.status}")
except ValueError as e:
    print(f"Cannot cancel test: {str(e)}")

# Cancel test run using user email instead of user ID
try:
    cancelled_test = await client.test_runs.cancel_with_email(
        'test_run_id',
        'user@example.com'
    )
    print(f"Test run cancelled by {cancelled_test.modified_by}")
except ValueError as e:
    print(f"Cannot cancel test: {str(e)}")
```

#### Download Test Run Attachments
```python
# Download all attachments for a test run
download_attachment = await client.test_runs.download_all_attachments(
    'test_run_id',
    'local/path/to/save'
)

# Download specific attachment
attachment = await client.test_run_dash_outputs_attachments.download_attachment(
    'attachment_id',
    'local/path/to/save'
)
```


### Test Data Management

```python
# Create test data
test_data = await client.test_data.create({
    "name": "Test Data",
    "type": "test",  # Optional: defaults to "test"
    "status": "draft"  # Optional: defaults to "draft"
})
```

### Environment Management

```python
# List environments
environments = await client.environments.get_list()

# Get environment by ID
environment = await client.environments.get_one('env_id')
```

### Tags Management

```python
# List tags
tags = await client.tags.get_list()

# Create tag
tag = await client.tags.create({
    "name": "New Tag"
})
```

### Schedule Management

#### Create Test Report Schedule

```python
# Create a schedule with cron expression
schedule = await client.test_report_schedules.create_test_report_schedule(
    name="Daily Test Report",
    is_active=True,
    cron_expression="0 9 * * *",  # Run at 9 AM daily
    filter_name_pattern="test-*",  # Filter test runs by name pattern
    filter_tags=["production", "critical"],  # Filter by tags
    notification_channels=["email-channel", "slack-channel"]  # Send to these channels
)
```

#### Create Schedule with Time Intervals

```python
from testzeus_sdk.managers.test_report_schedule_manager import FilterTimeIntervals

# Create a schedule with specific time intervals
time_intervals: FilterTimeIntervals = {
    "start_time": "2025-01-01 00:00:00",
    "end_time": "2025-01-01 23:59:59"
}

schedule = await client.test_report_schedules.create_test_report_schedule(
    name="Hourly Report",
    is_active=True,
    filter_time_intervals=time_intervals,
    filter_env=["staging", "production"],  # Filter by environments
    filter_test_data=["dataset1", "dataset2"]  # Filter by test data
)
```

#### Update Schedule

```python
# Update an existing schedule
updated_schedule = await client.test_report_schedules.update_test_report_schedule(
    id_or_name="schedule_id_or_name",
    name="Updated Schedule Name",
    is_active=False,
    cron_expression="0 12 * * *"  # Change to run at noon
)
```

#### Schedule Validation Rules

The schedule manager enforces several validation rules:

- **Name is required** for all schedules
- **At least one filter** must be provided (name pattern, time intervals, cron, tags, environments, or test data)
- **Mutually exclusive filters**:
  - Cannot use both `filter_time_intervals` and `cron_expression`
  - Cannot use both `filter_tags` and `filter_tag_pattern`
  - Cannot use both `filter_env` and `filter_env_pattern`
  - Cannot use both `filter_test_data` and `filter_test_data_pattern`

### Notification Channel Management

#### Create Email Notification Channel

```python
# Create a basic email notification channel
channel = await client.notification_channels.create_notification_channel(
    name="Email Alerts",
    display_name="Email Alert Channel",
    emails=["admin@example.com", "team@example.com"],
    is_active=True,
    is_default=False
)
```

#### Create Webhook Notification Channel

```python
# Create a notification channel with webhooks
channel = await client.notification_channels.create_notification_channel(
    name="Webhook Alerts",
    emails=["admin@example.com"],
    webhooks=["https://webhook.example.com/alerts", "https://backup-webhook.example.com"],
    is_active=True
)
```

#### Update Notification Channel

```python
# Update notification channel settings
updated_channel = await client.notification_channels.update_notification_channel(
    id_or_name="channel_id_or_name",
    name="Updated Channel Name",
    emails=["new-admin@example.com"],
    is_active=False
)
```

#### Remove Channel Configurations

```python
# Remove specific configuration types from a channel
await client.notification_channels.remove_config(
    id_or_name="channel_id",
    config_type="webhook"  # Options: "webhook", "slack", "jira"
)
```

#### Channel Configuration Methods

```python
# Get a notification channel
channel = await client.notification_channels.get_one("channel_id")

# Check what configurations are enabled
if channel.has_email_config():
    print("Email notifications enabled")
    
if channel.has_webhook_config():
    print("Webhook notifications enabled")

# Get list of enabled channel types
enabled_channels = channel.get_enabled_channels()
print(f"Enabled channels: {enabled_channels}")
```

### Test Run Groups Management

Test Run Groups allow you to execute multiple tests together as a coordinated batch with shared configuration and monitoring.

#### Create and Execute Test Run Group

```python
# Create and execute a test run group with specific test IDs
test_run_group = await client.test_run_groups.create_and_execute(
    name="Regression Test Suite",
    test_ids=["test_id_1", "test_id_2", "test_id_3"],
    execution_mode="strict",  # or "lenient"
    environment="production",
    notification_channels=["email-channel", "slack-channel"]
)
print(f"Test run group created: {test_run_group.id}")
```

#### Create Test Run Group by Tags

```python
# Create and execute a test run group using tags to filter tests
test_run_group = await client.test_run_groups.create_and_execute(
    name="Critical Tests",
    tags=["critical", "smoke"],  # Run all tests with these tags
    execution_mode="strict",
    environment="staging",
    notification_channels=["webhook-alerts"]
)
```

#### Execute and Monitor Test Run Group

```python
# Execute a test run group and monitor until completion with automatic report download
test_run_group = await client.test_run_groups.execute_and_monitor(
    name="Full Integration Suite",
    test_ids=["test1", "test2", "test3"],
    execution_mode="lenient",
    environment="staging",
    sleep_interval=30,  # Check status every 30 seconds
    output_dir="reports",
    format="ctrf",  # Options: "ctrf", "pdf", "csv", "zip"
    filename="integration-report.json"
)
```

#### Cancel Test Run Group

```python
# Cancel all running test runs in a group
try:
    cancelled_group = await client.test_run_groups.cancel_group("group_id")
    print(f"Test run group cancelled: {cancelled_group.status}")
except ValueError as e:
    print(f"Cannot cancel group: {str(e)}")
```

#### Download Test Run Group Reports

```python
# Download the consolidated report for a test run group
from pathlib import Path

report_path = await client.test_run_groups.download_report(
    id_or_name="group_id",
    output_dir="reports",
    format="pdf"  # Options: "ctrf", "pdf", "csv", "zip"
)

if report_path:
    print(f"Report downloaded to: {report_path}")
```

#### Download All Test Run Group Attachments

```python
# Download all attachments from all test runs in the group
downloaded_attachments = await client.test_run_groups.download_all_attachments(
    id_or_name="group_id",
    output_dir="attachments"
)

# Structure: attachments/<group-name>/<test-run-name>/files...
for test_run_name, files in downloaded_attachments.items():
    print(f"Test run '{test_run_name}': {len(files)} files downloaded")
```

#### Test Run Group Validation Rules

- **Name is required** for all test run groups
- **Must specify either test_ids OR tags** (mutually exclusive)
- **Cannot specify both test_ids and tags**
- Only groups in "pending" or "running" status can be cancelled

### Extensions Management

Extensions provide additional functionality and integrations for TestZeus.

```python
# List all available extensions
extensions = await client.extensions.get_list()
for extension in extensions["items"]:
    print(f"Extension: {extension.name}")

# Get specific extension
extension = await client.extensions.get_one("extension_id")
print(f"Extension details: {extension.data}")


### AI Test Generator

The AI Test Generator helps create test cases automatically using artificial intelligence.

```python
# List AI test generator configurations
ai_generators = await client.tests_ai_generator.get_list()
for generator in ai_generators["items"]:
    print(f"AI Generator: {generator.name}")

# Get specific AI generator
ai_generator = await client.tests_ai_generator.get_one("generator_id")
print(f"Generator config: {ai_generator.data}")

# Generate tests using AI
generated_tests = await client.tests_ai_generator.generate_tests(
    generator_id="generator_id",
    specification={
        "api_endpoint": "https://api.example.com",
        "methods": ["GET", "POST", "PUT", "DELETE"],
        "authentication": "bearer_token"
    }
)
```

### Bulk Test Operations

Run multiple tests efficiently using the test manager's bulk operations:

```python
# Run multiple tests as a coordinated group
test_run_result = await client.tests.run_tests(
    ids_or_names=["test1", "test2", "test3"],  # Can use names or IDs
    name="Nightly Regression",
    execution_mode="strict",
    environment="production",
    tags=["regression", "critical"],
    notification_channels=["email-alerts", "slack-notifications"]
)

print(f"Test run group created: {test_run_result}")
```

## Available Managers

The SDK provides managers for all TestZeus collections:

- `client.tests` - Tests
- `client.test_runs` - Test Runs
- `client.test_run_groups` - Test Run Groups
- `client.test_run_dashs` - Test Run Dashboards
- `client.test_data` - Test Data
- `client.environments` - Environments
- `client.tags` - Tags
- `client.test_report_schedules` - Test Report Schedules
- `client.notification_channels` - Notification Channels
- `client.extensions` - Extensions
- `client.tests_ai_generator` - AI Test Generator
- `client.agent_configs` - Agent Configurations
- `client.test_devices` - Test Devices
- `client.test_designs` - Test Designs
- `client.test_run_dash_outputs` - Test Run Dashboard Outputs
- `client.test_run_dash_output_steps` - Test Run Dashboard Output Steps
- `client.tenant_consumption` - Tenant Consumption
- `client.tenant_consumption_logs` - Tenant Consumption Logs

## Contributing

1. Clone the repository
2. Install dependencies with Poetry: `poetry install`
3. Run tests: `poetry run pytest`

## License

MIT

## Release Process

This project uses automated releases through GitHub Actions. The process is:

1. To create a new release, use one of the following make commands:
   ```bash
   # Interactive version bump (will prompt for version type)
   make release

   # Specific version bumps
   make release-patch   # Increments the patch version (e.g., 1.0.0 -> 1.0.1)
   make release-minor   # Increments the minor version (e.g., 1.0.0 -> 1.1.0)
   make release-major   # Increments the major version (e.g., 1.0.0 -> 2.0.0)
   make release-custom  # Set a custom version (will prompt for version)
   ```

2. The command will:
   - Update the version in pyproject.toml
   - Commit the change with a release message
   - Create a git tag for the version (e.g., v1.0.1)
   - Push the commit and tag to GitHub

3. The GitHub Actions workflow will automatically:
   - Detect the new tag
   - Build the package
   - Run tests
   - Publish the package to PyPI

Note: The release workflow only runs on tagged releases (not on pushes to the main branch).