semvec/pypi-packages huggingface dataset:
MainContentExtractor 0.0.4
A library to extract the main content from html. Developed for information on LLM and for feeding data into LangChain and LlamaIndex.
License: MIT
Requires Python: >=3.6

# main_content_extractor

## Description

This library is designed for extracting only the main content from HTML.<br>
It was developed for obtaining information related to LLM and for data input to LangChain and LlamaIndex.<br>
<br>
Since this library contains element information and hierarchy information of HTML, it is useful when utilizing them.<br>
For example, it can be helpful in obtaining a list of links or headers from the main content.<br>
<br>
While `trafilatura` is an excellent library for main content extraction, it has issues such as missing necessary data or inability to output HTML.<br>
To address these problems, this library exists.<br>
<br>
The sequence of main content extraction is as follows:<br>
<br>
![image](https://raw.githubusercontent.com/HawkClaws/main_content_extractor/main/content_extraction_sequence.png)
<br>
In addition to HTML format, output in Text format and Markdown format is also supported. This is to make it easier to output data in a format that is more convenient for LLM.<br>
<br>
The extraction of main content uses `trafilatura`.<br>
Since `trafilatura` cannot output in HTML format, it is output in XML format containing HTML information and then converted to HTML.<br>
The conversion from XML to HTML is irreversible and does not perfectly match the original data.<br>
<br>

## Installation

`pip install MainContentExtractor`

## HowToUse

```python
import requests
from main_content_extractor import MainContentExtractor

# Get HTML using requests
url = "https://developer.mozilla.org/ja/docs/Web"
response = requests.get(url)
response.encoding = 'utf-8'
content = response.text

# Get HTML with main content extracted from HTML
extracted_html = MainContentExtractor.extract(content)

# Get HTML with main content extracted from Markdown
extracted_markdown = MainContentExtractor.extract(content, output_format="markdown")
```
notepads-dev 2023.12.10.10
Create runtime folders, files, and code notes. All stored in notepads environment.
License: MIT
Keywords: notes,notepad,notepads,np,nps

<h1 align="center">
  <a href="https://pypi.org/project/notepads"><img src="https://i.ibb.co/D71Jnvz/sketch1702189741549-modified.png" width="400"></a>
</h1>


# [notepads - PyPi](https://pypi.org/project/notepads)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/)
[![Python Versions](https://img.shields.io/badge/python-3.7%20|%203.8%20|%203.9%20|%203.10%20|%203.11%20|%203.12%20-blue)](https://www.python.org/downloads/)

```
notepads: V2023.12.10.10
```

## Installing
```shell
# Linux/macOS
python3 pip install -U notepads

# Windows
py -3 -m pip install -U notepads
```

Create runtime only directory
    [1] -> Make folders
    [2] -> Make files
    [3] -> Make notes for code

========================================

# Directory Example Layout:
```ruby
Directory["/notepads/"]
  🗀  examples
   ↳ 🗎  example.py
  🗎  main.py
```

========================================

# Create note:
```python
notepads.note(name='notename', content='notecontent', author='OPTIONAL_noteauthor')
```
# Find note
```python
notepads.get_note(name='notename')

# or

notepads.notes()[<index>]
```
========================================

# Create folder:
```python
notepads.folder(name='foldername', version='OPTIONAL_0.0.0', description='OPTIONAL_a test folder', author='OPTIONAL_test author'))

# name, *, version=None, description=None, author=None
```

# Find folder:
```python
notepads.get(name='foldername')
```

# Update folder:
```python
folder.update(name='newfoldername', version='0.0.1', description='a new test folder', author='test author')
```

# Delete folder:
```python
folder.delete()
```

========================================

# Create file:
```python
notepads.file(name='filename', description='OPTIONAL_a test file', author='OPTIONAL_test author', parent='OPTIONAL_folder name')

# name, *, version=None, description=None, author=None, parent: NotepadsFolder=None
```

# Find file:
```python
notepads.get(name='filename')
```

# Update file:
```python
file.update(name='newfilename', description='a new test file', author='test author)
```

# Load file:
```python
file.load()
```

# Write file:
```python
file.write(content='newfilecontent')
```

# Delete file:
```python
file.delete()
```