npm package: @galihvsx/gmr-scraper
Description: Modern, production-ready Google Maps review scraper with TypeScript, retry logic, caching, and CLI support
Author: Galih Putro Aji
Latest version: 1.0.1

# 🗺️ GMR-Scraper

<div align="center">

![npm version](https://img.shields.io/npm/v/%40galihvsx%2Fgmr-scraper?style=for-the-badge)
![license](https://img.shields.io/npm/l/%40galihvsx%2Fgmr-scraper?style=for-the-badge)
![downloads](https://img.shields.io/npm/dw/%40galihvsx%2Fgmr-scraper?style=for-the-badge)
![build](https://img.shields.io/github/actions/workflow/status/galihvsx/gmr-scraper/ci.yml?style=for-the-badge)

**Modern, production-ready Google Maps review scraper**

Built with TypeScript • Powered by Bun • Feature-rich CLI

[Features](#-features) • [Installation](#-installation) • [Quick Start](#-quick-start) • [Documentation](#-documentation) • [Examples](#-examples)

</div>

---

## ✨ Features

### Core Capabilities

- 🎯 **Scrape Google Maps Reviews** - Extract reviews from any Google Maps location
- 🔄 **Smart Pagination** - Automatic pagination with configurable limits
- 🌐 **Multi-Language Support** - Scrape reviews in any language
- 🔍 **Advanced Filtering** - Filter by rating, text, images, keywords
- 📊 **Built-in Analytics** - Calculate ratings, distributions, and insights

### Production-Ready

- ⚡ **Retry Logic** - Exponential backoff with configurable attempts
- 💾 **Caching Layer** - Optional in-memory cache with TTL
- 🚦 **Rate Limiting** - Token bucket algorithm to prevent API blocks
- ⏱️ **Timeout Control** - Configurable request timeouts
- 🎨 **Progress Callbacks** - Real-time scraping progress updates

### Advanced Features

- 📦 **Batch Processing** - Scrape multiple locations concurrently
- 🌊 **Streaming API** - Memory-efficient processing for large datasets
- 🖥️ **CLI Tool** - Command-line interface with multiple output formats
- 🔧 **Full TypeScript** - Complete type safety and IntelliSense support
- 🎯 **Custom Errors** - Detailed error classes for better debugging

---

## 📦 Installation

### Using npm

```bash
npm install @galihvsx/gmr-scraper
```

### Using Bun

```bash
bun add @galihvsx/gmr-scraper
```

### Using Yarn

```bash
yarn add @galihvsx/gmr-scraper
```

### CLI (Global)

```bash
npm install -g @galihvsx/gmr-scraper
```

---

## 🚀 Quick Start

### Basic Usage

```typescript
import { scraper } from "@galihvsx/gmr-scraper";

const reviews = await scraper("https://www.google.com/maps/place/...", {
  sort_type: "newest",
  pages: 5,
  clean: true,
  lang: "en",
});

console.log(`Found ${reviews.length} reviews`);
```

### With Advanced Features

```typescript
import { scraper } from "@galihvsx/gmr-scraper";

const reviews = await scraper(url, {
  sort_type: "newest",
  pages: 10,
  clean: true,

  cache: {
    enabled: true,
    ttl: 300000,
  },

  retry: {
    maxAttempts: 5,
    initialDelay: 2000,
  },

  rateLimit: {
    requestsPerSecond: 2,
  },

  onProgress: (current, total) => {
    console.log(`Scraping page ${current}/${total}`);
  },
});
```

### CLI Usage

```bash
gmr-scraper scrape "https://www.google.com/maps/place/..." \
  --sort newest \
  --pages 5 \
  --clean \
  --output table
```

---

## 📖 Documentation

### API Options

| Option         | Type                                                            | Default      | Description                            |
| -------------- | --------------------------------------------------------------- | ------------ | -------------------------------------- |
| `sort_type`    | `'relevant' \| 'newest' \| 'highest_rating' \| 'lowest_rating'` | `'relevant'` | Sort order for reviews                 |
| `pages`        | `number \| 'max'`                                               | `'max'`      | Number of pages to scrape              |
| `search_query` | `string`                                                        | `''`         | Filter reviews by text                 |
| `clean`        | `boolean`                                                       | `false`      | Return parsed objects vs raw data      |
| `lang`         | `string`                                                        | `'en'`       | Language code (e.g., 'en', 'id', 'es') |
| `cache`        | `CacheOptions`                                                  | `undefined`  | Enable caching with TTL                |
| `retry`        | `RetryOptions`                                                  | `undefined`  | Configure retry logic                  |
| `rateLimit`    | `RateLimitOptions`                                              | `undefined`  | Configure rate limiting                |
| `timeout`      | `number`                                                        | `30000`      | Request timeout in ms                  |
| `onProgress`   | `function`                                                      | `undefined`  | Progress callback                      |

### Complete Guides

- 📘 [API Documentation](./docs/API.md)
- 🔄 [Migration Guide](./docs/MIGRATION.md) - From google-maps-review-scraper
- 🛠️ [Troubleshooting](./docs/TROUBLESHOOTING.md)
- 💻 [CLI Usage](./examples/cli-usage.md)

---

## 💡 Examples

### Batch Processing

```typescript
import { batchScraper } from "@galihvsx/gmr-scraper";

const results = await batchScraper([url1, url2, url3], {
  concurrency: 3,
  includeAnalytics: true,
  onProgress: (completed, total, url) => {
    console.log(`${completed}/${total}: ${url}`);
  },
});
```

### Streaming API

```typescript
import { scrapeStream } from "@galihvsx/gmr-scraper";

for await (const review of scrapeStream(url, { clean: true })) {
  console.log(`${review.author.name}: ${review.review.rating}⭐`);
}
```

### Analytics & Filtering

```typescript
import { calculateAnalytics, filterReviews } from "@galihvsx/gmr-scraper";

const reviews = await scraper(url, { clean: true });

const analytics = calculateAnalytics(reviews);
console.log(`Average rating: ${analytics.averageRating}`);

const highRated = filterReviews(reviews, {
  minRating: 4,
  hasText: true,
  keywords: ["excellent", "great"],
});
```

### More Examples

- [Basic Usage](./examples/basic.ts)
- [Advanced Features](./examples/advanced.ts)
- [Batch Processing](./examples/batch.ts)
- [Streaming API](./examples/streaming.ts)

---

## 🆚 Comparison with google-maps-review-scraper

| Feature                | google-maps-review-scraper | @galihvsx/gmr-scraper   |
| ---------------------- | -------------------------- | ----------------------- |
| **Language**           | JavaScript                 | ✅ TypeScript           |
| **Runtime**            | Node.js only               | ✅ Bun + Node.js        |
| **Type Safety**        | ❌ No types                | ✅ Full TypeScript      |
| **Retry Logic**        | ❌ No                      | ✅ Exponential backoff  |
| **Rate Limiting**      | ❌ No                      | ✅ Token bucket         |
| **Caching**            | ❌ No                      | ✅ In-memory cache      |
| **Progress Callbacks** | ❌ No                      | ✅ Real-time updates    |
| **CLI Tool**           | ❌ No                      | ✅ Full-featured CLI    |
| **Batch Processing**   | ❌ No                      | ✅ Concurrent scraping  |
| **Streaming API**      | ❌ No                      | ✅ Memory-efficient     |
| **Analytics**          | ❌ No                      | ✅ Built-in insights    |
| **Filtering**          | ❌ No                      | ✅ Advanced filters     |
| **Error Handling**     | Basic                      | ✅ Custom error classes |
| **Documentation**      | Basic                      | ✅ Comprehensive        |
| **Examples**           | Limited                    | ✅ Extensive            |
| **Build System**       | ❌ No                      | ✅ tsup bundling        |
| **Dependencies**       | 1                          | ✅ 0 (runtime)          |

---

## 🎯 Why Choose GMR-Scraper?

### 1. **Production-Ready**

Built with enterprise features like retry logic, rate limiting, and caching out of the box.

### 2. **Developer Experience**

Full TypeScript support, comprehensive documentation, and extensive examples.

### 3. **Performance**

Optimized for Bun runtime with streaming API for memory-efficient processing.

### 4. **Flexibility**

Use as a library or CLI tool. Batch processing or streaming. Your choice.

### 5. **Modern Stack**

Latest TypeScript, modern async/await patterns, and best practices.

---

## 🛠️ Development

### Setup

```bash
git clone https://github.com/galihvsx/gmr-scraper.git
cd gmr-scraper
bun install
```

### Build

```bash
bun run build
```

### Test

```bash
bun test
bun run test:coverage
```

### Lint

```bash
bun run lint
```

---

## 🤝 Contributing

Contributions are welcome! Please read our [Contributing Guide](./CONTRIBUTING.md) for details.

### Contributors

Thanks to all contributors who have helped make this project better!

---

## 📄 License

MIT License - see [LICENSE](./LICENSE) for details.

---

## 🙏 Acknowledgments

- Inspired by [google-maps-review-scraper](https://github.com/YasogaN/google-maps-review-scraper) by [@YasogaN](https://github.com/YasogaN)
- Special thanks to [@marin-m](https://github.com/marin-m) for [pbtk](https://github.com/marin-m/pbtk) research

---

## ⚖️ Legal Disclaimer

This project is not affiliated with, endorsed by, or associated with Google LLC. All product and company names are trademarks of their respective holders.

**Educational Purpose:** This project is created for educational purposes and proof of concept. It demonstrates technical approaches for API integration and data processing.

**Non-Commercial:** This is a non-commercial, open-source project shared with the community to foster learning and collaboration.

**Responsible Use:** Users are responsible for complying with Google's Terms of Service and applicable laws when using this tool.

---

<div align="center">

**Made with ❤️ by [Galih Putro Aji](https://github.com/galihvsx)**

⭐ Star this repo if you find it useful!

[Report Bug](https://github.com/galihvsx/gmr-scraper/issues) • [Request Feature](https://github.com/galihvsx/gmr-scraper/issues) • [Documentation](./docs/API.md)

</div>