vpcrawler 0.1.0
A CLI tool to validate AWS VPC connectivity configuration between two VPCs
Requires Python: >=3.9

# AWS VPC Networking Test Tool

A Python CLI tool that validates AWS VPC connectivity configuration between two VPCs and runs local network connectivity tests. Use it to troubleshoot cross-VPC connectivity, verify peering and Transit Gateway setup, and test reachability from your local machine to targets inside your VPCs.

## Overview

The tool combines two capabilities:

1. **Configuration validation** — Uses the AWS API (boto3) to analyze route tables, security groups, network ACLs, VPC peering, and Transit Gateway attachments. It determines whether the *configuration* allows traffic to flow between two VPCs.

2. **Connectivity testing** — Runs ping, DNS resolution, and TCP port checks from your local machine to verify actual reachability. Targets can be specified manually or discovered automatically from EC2 instances in the VPCs.

All commands run from your local machine using your AWS profile. No agents or deployments inside AWS are required.

## Features

- **VPC configuration analysis** — CIDR overlap, route tables, security groups, NACLs, peering, Transit Gateway
- **IP range reporting** — VPC and subnet CIDR blocks with availability zones
- **Auto-discovery** — Find running EC2 instances in both VPCs and test them automatically
- **Local network tests** — Ping, DNS (dig/nslookup), TCP port checks
- **JSON output** — Machine-readable output for scripting and CI/CD
- **Cross-platform** — Works on macOS, Linux, and Windows

## Requirements

- Python 3.9+
- AWS credentials configured (profile or environment)
- Network access to AWS APIs and to targets you test (VPN, bastion, or direct if testing public endpoints)

## Installation

```bash
# Clone or navigate to the project
cd vpcrawler

# Install in editable mode
pip install -e .

# Or install dependencies only
pip install -r requirements.txt
```

### Dependencies

- **boto3** — AWS SDK for Python
- **typer** — CLI framework
- **rich** — Formatted console output

## Quick Start

```bash
# Validate configuration between two VPCs
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --profile myprofile

# Validate and automatically test EC2 instances in both VPCs
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --profile myprofile --auto-test
```

## Usage Modes

### 1. Configuration Validation Only

Validates that the AWS configuration allows connectivity between two VPCs. No network tests are run.

```bash
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --profile myprofile --region us-east-1
```

Use this to:
- Audit peering or Transit Gateway setup
- Verify route tables and security groups
- Check for CIDR overlap before peering

### 2. Auto-Test Mode

Discovers running EC2 instances in both VPCs and automatically runs ping and port checks against their private IPs.

```bash
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --profile myprofile --auto-test
```

With options:

```bash
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --auto-test \
  --max-targets-per-vpc 3 \
  --auto-test-ports 22,443,8080
```

**Note:** Auto-test requires your local machine to reach the EC2 private IPs (e.g., via VPN or bastion host). If you cannot reach private IPs from your machine, use manual targets instead.

### 3. Manual Connectivity Tests

Specify exact targets for ping, DNS, and port checks.

```bash
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 \
  --ping 10.0.1.50 \
  --dns internal.example.com \
  --port 10.0.2.100:443
```

You can use any combination of `--ping`, `--port`, and `--dns`.

### 4. Combined Usage

Auto-test and manual targets can be used together. All tests run and appear in the output.

```bash
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --auto-test \
  --ping 10.0.1.1 \
  --dns api.internal.example.com
```

## Command Reference

| Option | Required | Default | Description |
|--------|----------|---------|-------------|
| `--vpc-a` | Yes | — | First VPC ID |
| `--vpc-b` | Yes | — | Second VPC ID |
| `--profile` | No | `AWS_PROFILE` env | AWS profile name |
| `--region` | No | `us-east-1` | AWS region |
| `--auto-test` | No | `false` | Discover EC2 instances and run ping/port checks |
| `--max-targets-per-vpc` | No | `5` | Max instances to test per VPC with `--auto-test` |
| `--auto-test-ports` | No | `22,443` | Ports to check with `--auto-test` (comma-separated) |
| `--ping` | No | — | IP or hostname to ping |
| `--dns` | No | — | Hostname to resolve via DNS |
| `--port` | No | — | `host:port` to test (e.g., `10.0.2.100:443`) |
| `--json` | No | `false` | Output results as JSON |

## Configuration Validation

The tool runs these checks to determine if connectivity *should* work between the two VPCs:

### CIDR Overlap

VPC peering and Transit Gateway require non-overlapping CIDR blocks. If the VPCs overlap, peering cannot be established.

- **PASS** — CIDRs do not overlap
- **FAIL** — CIDRs overlap; connectivity not possible

### VPC Peering

Checks for an active VPC peering connection between the two VPCs.

- **PASS** — Active peering exists
- **FAIL** — No peering (and no Transit Gateway fallback)

### Transit Gateway

If no peering exists, checks whether both VPCs are attached to the same Transit Gateway.

- **PASS** — Both VPCs attached to same TGW
- **FAIL** — No shared Transit Gateway

### Route Tables

When peering or TGW exists, verifies that route tables in both VPCs have routes for the peer VPC’s CIDR.

- **PASS** — Routes exist in both directions
- **FAIL** — Missing routes in one or both VPCs

### Security Groups

Checks whether any security groups allow traffic between the VPC CIDRs (ingress and egress).

- **PASS** — Rules allow cross-VPC traffic
- **WARN** — May restrict traffic; manual verification recommended

### Network ACLs

Looks for NACL deny rules that could block cross-VPC traffic.

- **PASS** — No blocking rules found
- **WARN** — Deny rules may block traffic; manual verification recommended

### Connectivity Path

Overall result: **VALID** if no checks fail, **INVALID** if any check fails.

## Local Connectivity Tests

Tests run from your local machine. Your machine must have network path to the targets (VPN, bastion, or direct access).

### Ping

- **Command:** `ping -c 4` (Linux/macOS) or `ping -n 4` (Windows)
- **Purpose:** ICMP reachability
- **Timeout:** 15 seconds

### DNS

- **Command:** `dig +short` (preferred) or `nslookup`
- **Purpose:** Hostname resolution
- **Timeout:** 10 seconds
- **Note:** Requires `dig` or `nslookup` on your system

### Port Check

- **Command:** `nc -zv` (netcat) or Python socket fallback
- **Purpose:** TCP connectivity
- **Timeout:** 5 seconds
- **Note:** Uses `nc` when available; otherwise uses a built-in socket check

## Output

### Console Output

- **VPC Configuration** — VPC IDs, CIDR blocks, region
- **IP Ranges** — VPC and subnet CIDRs with availability zones
- **Validation Checks** — Table of checks with PASS/FAIL/WARN
- **Connectivity Path** — Overall VALID or INVALID
- **Local Network Tests** — Table of ping, DNS, and port results

Failed network tests include the raw command output for debugging.

### JSON Output

Use `--json` for machine-readable output:

```bash
vpcrawler --vpc-a vpc-0abc123 --vpc-b vpc-0def456 --json
```

Structure:

```json
{
  "config": {
    "vpc_a": { "vpc_id": "...", "cidr_block": "...", "subnets": [...] },
    "vpc_b": { "vpc_id": "...", "cidr_block": "...", "subnets": [...] },
    "checks": [{ "name": "...", "status": "pass|fail|warn", "message": "..." }],
    "connectivity_path_valid": true,
    "error": null
  },
  "network_tests": [
    { "test_type": "ping", "target": "...", "status": "pass", "message": "...", "duration_ms": 123 }
  ]
}
```

## AWS Permissions

The tool uses these EC2 API operations:

- `DescribeVpcs`
- `DescribeSubnets`
- `DescribeRouteTables`
- `DescribeSecurityGroups`
- `DescribeNetworkAcls`
- `DescribeVpcPeeringConnections`
- `DescribeTransitGatewayAttachments`
- `DescribeInstances` (when using `--auto-test`)

A minimal IAM policy:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeVpcs",
        "ec2:DescribeSubnets",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeNetworkAcls",
        "ec2:DescribeVpcPeeringConnections",
        "ec2:DescribeTransitGatewayAttachments",
        "ec2:DescribeInstances"
      ],
      "Resource": "*"
    }
  ]
}
```

## Troubleshooting

### "VPC(s) not found"

- Confirm VPC IDs are correct
- Ensure `--region` matches the VPCs
- Verify AWS credentials and profile

### "No active VPC peering or Transit Gateway connection"

- Check that peering is in `active` state
- For Transit Gateway, ensure both VPCs are attached to the same TGW

### Ping/port checks fail from local machine

- Private IPs are only reachable from within the VPC or via VPN/bastion
- Ensure your machine has a path to the target (VPN, SSH tunnel, etc.)
- Firewall or security groups may block ICMP or the tested ports

### DNS resolution fails

- `dig` or `nslookup` must be installed
- Private hostnames require DNS resolution that can reach your private zones (e.g., Route 53 Resolver, VPN split-tunnel DNS)

## Limitations

- **Single region** — Both VPCs must be in the same region (cross-region peering not yet supported)
- **Local tests** — Connectivity tests run from your machine; they do not test traffic between instances inside the VPCs
- **Private IPs** — Auto-test uses private IPs; your machine must be able to reach them (VPN, etc.)

## Development

```bash
# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest tests/ -v
```

### Project Structure

```
vpcrawler/
├── src/aws_vpc_net_tester/
│   ├── cli.py              # CLI entry point
│   ├── config_validator.py # boto3 VPC analysis + EC2 discovery
│   ├── models.py           # Data classes
│   └── network_tester.py   # Ping, DNS, port checks
├── tests/
├── pyproject.toml
└── requirements.txt
```

## Publishing to PyPI

Releases are published automatically via [GitHub Actions](https://github.com/marketplace/actions/pypi-publish) when you push a version tag.

### Setup (one-time)

1. **Create a PyPI project** — Register `vpcrawler` on [pypi.org](https://pypi.org) if it does not exist.

2. **Configure trusted publishing** — In your PyPI project settings, add a trusted publisher:
   - Publisher: GitHub Actions
   - Owner: your GitHub org/username
   - Repository: your repo name
   - Workflow: `publish.yml`
   - Environment: `pypi`

3. **Create the `pypi` environment** — In your GitHub repo: Settings → Environments → New environment → name it `pypi`.

### Releasing

1. Update the version in `pyproject.toml`.
2. Commit, then create and push a tag:

   ```bash
   git tag v0.1.0
   git push origin v0.1.0
   ```

3. The workflow builds the package and publishes to PyPI. No API tokens are required when using trusted publishing.