webinfo-retriever-ai 2.0.0
🤖 AI-Powered Web Intelligence System - Real-time research, comprehensive analysis, and intelligent insights using Google Gemini 2.0 Flash
License: MIT
Requires Python: >=3.8
Keywords: ai, artificial intelligence, web intelligence, research, automation, gemini, google ai, web scraping, data analysis, nlp, search engine, real-time, comprehensive analysis, intelligent insights, web research, market research, competitive analysis, news monitoring, trend analysis, content creation, academic research, financial analysis, cryptocurrency, technology trends, cli tool, python api, async, fast, reliable

# 🧠 WebIntel - Advanced Web Intelligence System

[![PyPI version](https://badge.fury.io/py/webintel.svg)](https://badge.fury.io/py/webintel)
[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![GitHub stars](https://img.shields.io/github/stars/JustM3Sunny/webintel.svg)](https://github.com/JustM3Sunny/webintel/stargazers)

WebIntel is a **superfast AI-powered web intelligence system** that provides real-time web research, comprehensive analysis, and intelligent insights using Google Gemini 2.0 Flash. Get comprehensive research results in **10-20 seconds** with **90%+ success rate**.

## 🚀 Key Features

- **🔍 Multi-Engine Search**: Searches across DuckDuckGo, Bing, and Google simultaneously with intelligent fallback mechanisms
- **🧠 AI-Powered Analysis**: Uses Google Gemini 2.0 Flash for intelligent content analysis, synthesis, and insights generation
- **⚡ Lightning Fast**: Optimized for speed with parallel processing, smart caching, and efficient content extraction (10-20s results)
- **🛡️ Reliable & Robust**: Built-in error handling, retry mechanisms, and fallback strategies ensure consistent performance
- **🔧 Highly Configurable**: Customizable output formats (JSON, Markdown, Rich), configurable result counts, and flexible API integration
- **📊 Advanced Analytics**: Provides relevance scoring, source credibility analysis, confidence levels, and comprehensive performance metrics

## 📦 Installation

### Using pip (Recommended)
```bash
pip install webintel
```

### Using pipx (For CLI tools)
```bash
pipx install webintel
```

### From Source
```bash
git clone https://github.com/JustM3Sunny/webintel.git
cd webintel
pip install -r requirements.txt
pip install -e .
```

## 🔑 API Key Setup

WebIntel requires a Google Gemini API key for AI-powered analysis:

1. Get your free API key from [Google AI Studio](https://makersuite.google.com/app/apikey)
2. Set your API key:

```bash
# Linux/macOS
export GEMINI_API_KEY="your-api-key-here"

# Windows
set GEMINI_API_KEY=your-api-key-here
```

## 🎯 Quick Start

```bash
# Basic search
webintel search "artificial intelligence trends 2024"

# Get comprehensive results
webintel search "machine learning frameworks" --max-results 10

# JSON output for integration
webintel search "blockchain technology" --format json

# Save results to file
webintel search "climate change research" --save --output-dir ./results
```

## 💻 Usage Examples

### Command Line Interface

```bash
# Market research
webintel search "electric vehicle market 2024" --max-results 5

# Technology research
webintel search "best python frameworks 2024" --format json

# Academic research
webintel search "climate change impact studies" --save

# News and trends
webintel search "AI breakthrough 2024" --max-results 15
```

### Python API

```python
from webintel import DataProcessor
from webintel.config import get_default_config

# Initialize WebIntel
config = get_default_config()
processor = DataProcessor(config)

# Perform comprehensive search
results = await processor.process_query(
    query="artificial intelligence trends 2024",
    max_results=10
)

# Access AI-generated insights
print("🤖 AI Analysis:")
print(results['synthesis']['executive_summary'])

print(f"\n📊 Statistics:")
print(f"Sources found: {len(results['sources'])}")
print(f"Processing time: {results['processing_time']}s")

print(f"\n🔍 Key Insights:")
for insight in results['synthesis']['key_findings']:
    print(f"• {insight}")

print(f"\n📋 Top Sources:")
for source in results['sources'][:3]:
    print(f"• {source['title']}")
    print(f"  🔗 {source['url']}")
    print(f"  📊 Relevance: {source['relevance_score']:.2f}")
```

## 🌟 Real-World Use Cases

### 📈 Market Research
```bash
webintel search "electric vehicle market trends 2024" --max-results 10
```

### 💻 Technology Research
```bash
webintel search "best AI frameworks comparison" --format json
```

### 🎓 Academic Research
```bash
webintel search "climate change impact studies" --save --output-dir ./research
```

### 📰 News & Trends
```bash
webintel search "latest AI breakthroughs 2024" --max-results 15
```

## ⚙️ Configuration

### Environment Variables
```bash
export GEMINI_API_KEY="your-api-key"
export WEBINTEL_MAX_RESULTS=10
export WEBINTEL_OUTPUT_FORMAT="rich"
export WEBINTEL_TIMEOUT=30
```

### Configuration File
WebIntel uses a YAML configuration file located at `~/.webintel/config.yaml`:

```yaml
# Google Gemini AI Configuration
gemini:
  api_key: "your-api-key-here"
  model_name: "gemini-2.0-flash"
  max_tokens: 8192
  temperature: 0.7

# Web Scraping Configuration
scraping:
  max_concurrent_requests: 10
  request_timeout: 30
  retry_attempts: 3

# Output Configuration
output:
  format: "rich"
  save_to_file: false
  include_sources: true
```

## 📊 Performance Metrics

- **Response Time**: 10-20 seconds average
- **Success Rate**: 90%+ for most queries
- **Search Engines**: 3+ engines with intelligent fallback
- **Concurrent Requests**: Up to 10 parallel requests
- **Cache Hit Rate**: 85%+ for repeated queries

## 🛠️ Requirements

- **Python**: 3.8 or higher
- **API Key**: Google Gemini API key (free tier available)
- **Internet**: Stable internet connection
- **Memory**: 512MB RAM minimum
- **Storage**: 100MB for installation

## 🤝 Contributing

We welcome contributions! Here's how you can help:

1. **Fork the repository**
2. **Create a feature branch**: `git checkout -b feature/amazing-feature`
3. **Commit your changes**: `git commit -m 'Add amazing feature'`
4. **Push to the branch**: `git push origin feature/amazing-feature`
5. **Open a Pull Request**

### Development Setup
```bash
git clone https://github.com/JustM3Sunny/webintel.git
cd webintel
pip install -r requirements.txt
pip install -e .
```

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- **Google Gemini 2.0 Flash** for AI-powered analysis
- **Multiple Search Engines** for comprehensive web coverage
- **Open Source Community** for inspiration and support

## 📞 Support

- **GitHub Issues**: [Report bugs or request features](https://github.com/JustM3Sunny/webintel/issues)
- **PyPI Package**: [WebIntel on PyPI](https://pypi.org/project/webintel/)

---

**Made with ❤️ by JustM3Sunny. Star ⭐ this repo if you find it useful!**