semvec/pypi-packages huggingface dataset:
vibescraper 0.1.6
A semantic web scraping and summarization toolkit.
Requires Python: <4.0,>=3.13

# vibescraper

**vibescraper** is a Python utility that generates an AI summary of web results based on a search query.

This is very much a work in progress. The results of each search may vary depending on how well the parser handles the pages. If you find that some results are not sufficient, you can try increasing the number of sites to return in the search api using the domain_count argument in the vibe_search function.

The number of sites will slow the operation down, so you may need to experiment and find the optimal tradeoff that suits your needs, between speed and accuracy of results.

In order to select the search api you need to run the following in your projects python environment:
    - change-search-api <google/brave>


The output files and folders from the search operations can be used or discarded as you need.
In future I may change this so that the datbase can be saved or discarded according to an argument flag in the vibe_search function or as part of another python config script.

---





## How it works

- **Web Search:** Searched the web using search engine API.
- **Semantic Chunking:** Extracts html from each url in the search results, then divides the html page content into meaningful chunks.
- **Embedding Generation:** Generate vector embeddings for each chunk using openai embedding models.
- **Similarity Search:** Find the most relevant content chunks for a given search query.
- **AI Summarization:** Generates concise summaries of relevant content, then combines all summaries into a combined summary.
- **JSON Export:** Saves results and summaries as JSON files for easy inspection or downstream use.

---

## Installation

Install via pip (from PyPI):

`pip install vibescraper`

Or install locally with Poetry:

`poetry install`

NOTE:
You need either a Beave or Google search API key to use this, as well as an open AI API key.


to import the keys into config.py after installing run:
    `check-api-keys`

in your python project env.

---

## Requirements

- Python 3.12+
- [OpenAI API key](https://platform.openai.com/) (for embedding and summarization)
- Other dependencies: numpy, requests, beautifulsoup4, pandas, sqlalchemy, openai, tiktoken, google-api-python-client, html5lib

---

## Usage

- Install the package into your python project, then import the vibe_search function

```python
from vibescraper.vibe_search import vibe_search

ai_summary = await vibe_search(query='What is the state of the software development job market in 2025?', domain_count=10, text_model='gpt-4o')
```

## Environment Variables

You must set your OpenAI API key (and either a Google or Brave yea keys)  as environment variables

---

## License

MIT License

---

## Contributing

Pull requests and issues are welcome!

---
webui2 2.5.6
Use any web browser as GUI
Requires Python: >=3.7

# Python WebUI v2.5.6

> Use any web browser as GUI, with Python in the backend and HTML5 in the frontend, all in a lightweight Python package.

## Features

- Portable (*Needs only a web browser at runtime*)
- Lightweight (*Few Kb library*) & Small memory footprint
- Fast binary communication protocol
- Multi-platform & Multi-Browser
- Using private profile for safety
- Original library is written in Pure C

## Documentation

* [Online Documentation](https://webui.me/docs/#/python_api)

## Install

```sh
pip install webui2
```

## Example

```python
from webui import webui

MyWindow = webui.window()
MyWindow.show('<html><script src="webui.js"></script> Hello World! </html>')
webui.wait()
```
cs-typingutils 20250503
Trite hacks for use with typing.
Keywords: python3

Trite hacks for use with typing.

*Latest release 20250503*:
is_optional: bugfix: Optional[T] has a get_args of (T,<NoneType>), not (T,None).

Short summary:
* `is_optional`: Check if `annotation` is an `Optional[type]`. Return `type` if so, `None` otherwise.
* `subtype`: Construct a `TypeVar` for subtypes of the type `t`.

Module contents:
- <a name="is_optional"></a>`is_optional(annotation)`: Check if `annotation` is an `Optional[type]`.
  Return `type` if so, `None` otherwise.
- <a name="subtype"></a>`subtype(t, name=None)`: Construct a `TypeVar` for subtypes of the type `t`.

  Parameters:
  * `t`: the type which bounds the `TypeVar`
  * `name`: optional name for the `TypeVar`,
    default `t.__name__ + 'SubType'`

# Release Log



*Release 20250503*:
is_optional: bugfix: Optional[T] has a get_args of (T,<NoneType>), not (T,None).

*Release 20250428*:
New is_optional(annotation) function to recognise Optional[type] annotations, returning type if true, None otherwise.

*Release 20230331*:
Initial release.