semvec/pypi-packages huggingface dataset:
desearch-py 1.0.9
A Python SDK for interacting with the Desearch API service.
License: MIT
Requires Python: >=3.6

# Desearch


Desearch API in Python

https://console.desearch.ai/

## Installation

`pip install desearch-py`

## Usage

Import the package and initialize the Desearch client with your API key:

```python
    from desearch_py import Desearch

    desearch = Desearch(api_key="your-api-key")
```

## Common requests

```python
    
    # Desearch AI Search
    result = desearch.ai_search(
        prompt="Bittensor",
        tools=[
            "web",
            "hackernews",
            "reddit",
            "wikipedia",
            "youtube",
            "twitter",
            "arxiv"
        ],
        date_filter="PAST_24_HOURS",
        streaming=False,
        result_type="LINKS_WITH_SUMMARIES",
        system_message="",
        count=10,
    )

    #Desearch Twitter post search
    result = desearch.twitter_links_search(
        prompt="Bittensor", 
        count=10,
    )

    #Desearch Web links search
    result = desearch.web_links_search(
        prompt="Bittensor",
        tools=[
            "web",
            "hackernews",
            "reddit",
            "wikipedia",
            "youtube",
            "arxiv"
        ],
        count=10,
    )

    #Basic Twitter search
    result = desearch.basic_twitter_search(
        query="Whats going on with Bittensor",
        sort="Top",
        user="elonmusk",
        start_date="2024-12-01",
        end_date="2025-02-25",
        lang="en",
        verified=True,
        blue_verified=True,
        is_quote=True,
        is_video=True,
        is_image=True,
        min_retweets=1,
        min_replies=1,
        min_likes=1,
        count=10
    )

    #Basic Web search
    result = desearch.basic_web_search(
        query="latest news on AI",
        num=10,
        start=0
    )

    #Web crawl
    result = desearch.web_crawl(
        "https://docs.desearch.ai/docs/desearch-api"
    )

    #Fetch Tweets by URLs
    result = desearch.twitter_by_urls(
        urls=["https://twitter.com/elonmusk/status/1613000000000000000"]
    )

    #Fetch Tweets by ID
    result = desearch.twitter_by_id(id="123456789")

    #Fetch Tweets by User
    result = desearch.tweets_by_user(
        user="elonmusk",
        query="Bittensor",
        count=10
    )

    #Fetch Latest Tweets
    result = desearch.latest_tweets(
        user="elonmusk",
        count=10
    )

    #Fetch Tweets and Replies by User
    result = desearch.tweets_and_replies_by_user(
        user="elonmusk",
        query="Bittensor",
        count=10
    )

    #Fetch Replies by Post
    result = desearch.twitter_replies_post(
        post_id="123456789",
        count=10
        query="Bittensor"
    )

    #Fetch Retweets by Post
    result = desearch.twitter_retweets_post(
        post_id="123456789",
        count=10,
        query="Bittensor"
    )

    #Fetch Tweeter User
    result = desearch.tweeter_user(
        user="elonmusk")

```
run-vast 0.3.0
A tool for running commands on vast.ai instances
License: MIT
Requires Python: >=3.6

# `run_vast`

A command-line tool. Lets you put bash commands in markdown files, and runs them in parallel on many vast.ai instances.

Uses a `waiting`/`running`/`fail`/`succeed` state machine to represent every command. All state is contained in the markdown file, in human-readable and human-editable form.

I like to provision 10-20 Vast instances, usually with 4x4090s each, at the beginning of the day, with the same custom Dockerfile.

Vast lets me keep these nodes idle for very cheap. So by default, all instances are idle.

Then later, I will add a new ML experiment to my `journal.md` file. Every training run in the experiment is a bash command in a triple-backtick ```` ```vast```` code block.

Then I run `rv journal.md` to run them all in parallel. Each Vast instance will go idle when its command succeeds.

If a run fails, the code block will be marked as ```` ```vast:fail/012345````, where `012345` is the instance ID of the machine it ran on. I can then ssh into the instance and debug my training run.

If a run starts up successfully, the code block will be marked as ```` ```vast:running/012345````.

## Installation

```bash
pip install run_vast
rv journal.md
```

## Usage

#### Make a list of commands you want to run

You should put these in a markdown file. Each command gets its own triple-backtick code block, annotated with `vast`.

For example, to train nanogpt with two different lrs:

````markdown
# Train nanogpt with different lrs

lr=0.5 and lr=1.5:

```vast
git clone https://github.com/karpathy/nanogpt && \
cd nanogpt && \
pip install torch numpy transformers datasets tiktoken wandb tqdm && \
python data/shakespeare_char/prepare.py &&
python train.py config/train_shakespeare_char.py --min_lr=0.5e-4
```

```vast
git clone https://github.com/karpathy/nanogpt && \
cd nanogpt && \
pip install torch numpy transformers datasets tiktoken wandb tqdm && \
python data/shakespeare_char/prepare.py &&
python train.py config/train_shakespeare_char.py --min_lr=1.5e-4
```
````

#### Set up your Vast account

You need to make an SSH key to connect to Vast instances.

Register your SSH key on the vast website, then put the private key in `~/.ssh/id_vast`.

#### Run `rv my_training_runs.md`

`rv` will prompt you to provision two Vast instances, so it can run both commands in parallel.

Important: in the vast.ai web UI, before provisioning Vast instances, you must edit the instance template to set the environment variable `IS_FOR_AUTORUNNING=1`.

Remember to press the "+" button to save the environment variable.

#### Go to the Vast dashboard and wait for your instances to be "Connected"

This should take a minute or so.

Then, return to the `rv` prompt and press Enter to continue.

#### Wait for your commands to finish

You should track your runs via i.e. wandb. `rv` doesn't handle any logging for you.

Once your commands have finished, run `rv journal.md`.

It will move them from the `vast:running/0123456` state to the `vast:finished` state.

## License

MIT License