Saravanan666/pypi-org huggingface dataset:
Package: extract-dtb
extract-dtbTool to split a kernel image with appended dtbs into separated kernel and dtb files.A Device Tree is a data structure for describing hardware. They are used in a lot of
ARM devices (e.g. Android), otherwise these would not be able to boot.This tool is similar tosplit-appended-dtbbut it is written in Python and its code is simpler and almost 3x shorter. Moreover, it doesn't
require any external Python library.If you want to learn more about DTB you can have a look at theDevice Tree Reference.Install & Usageextract-dtbis available on PyPI!$ pip install extract-dtbTheextract-dtbcommand is now available:$ extract-dtb --help
usage: extract-dtb [-h] [-o OUTPUT_DIR] [-n] [-V] filename

Extract dtbs from kernel images.

positional arguments:
  filename       Android kernel image

optional arguments:
  -h, --help     show this help message and exit
  -o OUTPUT_DIR  Output directory
  -n             Do not extract, just output information
  -V, --version  show program's version number and exitExample:$ extract-dtb -n /tmp/postmarketOS-export/vmlinuz-motorola-titan
Found 9 appended dtbs

$ extract-dtb /tmp/postmarketOS-export/vmlinuz-motorola-titan -o /tmp/dtb
Dumped kernel, start=0 end=7534024
Dumped dtbdump_01_Motorola_QC-MSM8226_Titan.dtb, start=7534024 end=7728853
Dumped dtbdump_02_Motorola_QC-MSM8226_Titan.dtb, start=7728853 end=7923682
Dumped dtbdump_03_Motorola_QC-MSM8226_Titan.dtb, start=7923682 end=8118511
Dumped dtbdump_04_Motorola_QC-MSM8226_Titan.dtb, start=8118511 end=8313340
Dumped dtbdump_05_Motorola_QC-MSM8226_Titan.dtb, start=8313340 end=8508169
Dumped dtbdump_06_Motorola_QC-MSM8226_Thea.dtb, start=8508169 end=8700762
Dumped dtbdump_07_Motorola_QC-MSM8226_Thea.dtb, start=8700762 end=8894086
Dumped dtbdump_08_Motorola_QC-MSM8226_Thea.dtb, start=8894086 end=9087470
Dumped dtbdump_09_Motorola_QC-MSM8226_Thea.dtb, start=9087470 end=9280854
Extracted 9 appended dtbs + kernel to /tmp/dtb

$ ls -l /tmp/dtb/
total 9088
-rw-rw-r-- 1 pablo pablo  194829 Aug 21 01:48 dtbdump_01_Motorola_QC-MSM8226_Titan.dtb
-rw-rw-r-- 1 pablo pablo  194829 Aug 21 01:48 dtbdump_02_Motorola_QC-MSM8226_Titan.dtb
-rw-rw-r-- 1 pablo pablo  194829 Aug 21 01:48 dtbdump_03_Motorola_QC-MSM8226_Titan.dtb
-rw-rw-r-- 1 pablo pablo  194829 Aug 21 01:48 dtbdump_04_Motorola_QC-MSM8226_Titan.dtb
-rw-rw-r-- 1 pablo pablo  194829 Aug 21 01:48 dtbdump_05_Motorola_QC-MSM8226_Titan.dtb
-rw-rw-r-- 1 pablo pablo  192593 Aug 21 01:48 dtbdump_06_Motorola_QC-MSM8226_Thea.dtb
-rw-rw-r-- 1 pablo pablo  193324 Aug 21 01:48 dtbdump_07_Motorola_QC-MSM8226_Thea.dtb
-rw-rw-r-- 1 pablo pablo  193384 Aug 21 01:48 dtbdump_08_Motorola_QC-MSM8226_Thea.dtb
-rw-rw-r-- 1 pablo pablo  193384 Aug 21 01:48 dtbdump_09_Motorola_QC-MSM8226_Thea.dtb
-rw-rw-r-- 1 pablo pablo 7534024 Aug 21 01:48 kernelThis tool can also be run directly onboot.imgimages.
Package: extractemail
Extract E-MailExtract E-Mail from StringScreenshotInstall$ pip install extractemailUsageimportextractemailtext="""Yoginth is very productive person he has multiple emailslike yoginth@zoho.com but he likes yoginth@gmail.combut the username has already taken, yoginth wants toget yoginth@gitlab.com but he want to work there."""extractemail.extract(text)#=> ['yoginth@zoho.com', 'yoginth@gmail.com', 'yoginth@gitlab.com']Get HelpThere are few ways to get help:Pleasepost questions on Stack Overflow. You can open issues with questions, as long you add a link to your Stack Overflow question.For bug reports and feature requests, open issues.For direct and quick help, you canemail me.How to contributeHave an idea? Found a bug? Seehow to contribute.Thanks!LicenseMIT
Package: extract-emails
Extract EmailsExtract emails and linkedins profiles from a given websiteSupport the project with BTC:bc1q0cxl5j3se0ufhr96h8x0zs8nz4t7h6krrxkd6lDocumentationRequirementsPython >= 3.9Installationpipinstallextract_emails[all]# orpipinstallextract_emails[requests]# orpipinstallextract_emails[selenium]Simple UsageAs libraryfrompathlibimportPathfromextract_emailsimportDefaultFilterAndEmailFactoryasFactoryfromextract_emailsimportDefaultWorkerfromextract_emails.browsers.requests_browserimportRequestsBrowserasBrowserfromextract_emails.data_saversimportCsvSaverwebsites=["website1.com","website2.com",]browser=Browser()data_saver=CsvSaver(save_mode="a",output_path=Path("output.csv"))forwebsiteinwebsites:factory=Factory(website_url=website,browser=browser,depth=5,max_links_from_page=1)worker=DefaultWorker(factory)data=worker.get_data()data_saver.save(data)As CLI tool$extract-emails--help

$extract-emails--urlhttps://en.wikipedia.org/wiki/Email-ofoutput.csv-d1$catoutput.csv
email,page,website
bob@b.org,https://en.wikipedia.org/wiki/Email,https://en.wikipedia.org/wiki/Email
Package: extracteur-de-fou-malade-pour-charles-le-charlo
PDF Data extractorSimple package wrapper that allows us to retrieve both the year of publication and a summary of a PDF.The package mainly relies on 3 other packages :textractto convert PDF to plain textpdfminer3to extract the date from a PDF filesumyto summarize textUsagefrom pdf_extractor import pdf_extractorextractor = pdf_extractor.PDFExtractor()pdf_path = "./test.pdf"extractor.extract_data(pdf_path, 10)
Package: extract-fasta-seq
extract_fasta_seq1 Introductionextract_fasta_seqis a tool to extract specific fasta sequences from a fasta file. By Guanliang MENG, seehttps://github.com/linzhi2013/extract_fasta_seq2 Installationpip install extract_fasta_seqThere will be a commandextract_fasta_seqcreated under the same directory as yourpipcommand.3 Usage$ extract_fasta_seq
usage: extract_fasta_seq.py [-h] [-q <str> [<str> ...]] [-f <query file>]
                            [-s [<subject file>]] [-s1 <pattern>]
                            [-s2 <pattern>] [-d1 [<int>]] [-d2 [<int>]]
                            [-o [<outfile>]] [-v] [-V] [--lazy] [--version]

To extract specific fasta sequences from a fasta file. By Guanliang MENG, see
https://github.com/linzhi2013

optional arguments:
  -h, --help            show this help message and exit
  -q <str> [<str> ...]  query list. "-s1" and "-d1" have no effect on this
                        option.
  -f <query file>       query list file, one line should contain only one
                        queryid! (but can be mixed with others, and we can use
                        "-s1" and "-d1" option to extract the queryid)
  -s [<subject file>]   subject file [stdin]
  -s1 <pattern>         query file sep_pattern [\s+]
  -s2 <pattern>         subject file sep_pattern [\s+]
  -d1 [<int>]           which field in the query_file is to used? [0]
  -d2 [<int>]           which field in the subject_file is to used? useful for
                        finding out all sequences in the subject_file whose
                        seqids equal to the queryids [0]
  -o [<outfile>]        outfile [stdout]
  -v                    invert the output [False]
  -V                    verbose output
  --lazy                Stop searching once each required seqid has at least
                        one sequence found, which can be problems if some
                        required seqids have more than sequences! Works only
                        for non-invert mode. [False]
  --version             show program's version number and exit4 AuthorGuanliang MENG5 CitationCurrently I have no plan to publishextract_fasta_seq.
Package: extractfq
extractfq1 Introductionextractfqis a tool to extract some fastq reads from the beginning of the files.2 Installationpip install extractfqThere will be a commandextractfqcreated under the same directory as yourpipcommand.3 Usage$ extractfq
usage: extractfq.py [-h] [-fq1 <str>] [-fq2 <str>] [-outfq1 <str>]
                    [-outfq2 <str>] [-size_required <float>] [-rl <int>] [-gz]
                    [-cache_num <int>]

Extract some fastq reads from the beginning of the files. Author: Guanliang
Meng, see https://github.com/linzhi2013/extractfq. This script is part of the
package `MitoZ`, when you use the script in your work, please cite: MitoZ: A
toolkit for mitochondrial genome assembly, annotation and visualization with
NGS data. Guangliang Meng, Yiyuan Li, Chentao Yang, Shanlin Liu (in
manuscript)

optional arguments:
  -h, --help            show this help message and exit
  -fq1 <str>            input fastq 1 file
  -fq2 <str>            input fastq 2 file
  -outfq1 <str>         output fastq 1 file
  -outfq2 <str>         output fastq 2 file
  -size_required <float>
                        size required in Gigabase. [3]
  -rl <int>             read length required. discard the smaller ones, and
                        cut the longer ones to this length [None]
  -gz                   gzip output. [False]
  -cache_num <int>      the cache number of reads before writing to the file,
                        to speed up. the larger of cache_num, the more memory
                        (default is ca. 2G) will be used. [1500000]AuthorGuanliang MENGCitationThis script is part of the packageMitoZ, when you use the script in your work, please cite:Guanliang Meng, Yiyuan Li, Chentao Yang, Shanlin Liu. MitoZ: A toolkit for mitochondrial genome assembly, annotation and visualization; doi: https://doi.org/10.1101/489955
Package: extract-from-url
DescriptionTake advantage oflibarchiveto download and extract files without having to store the archive first. Works with ZIP files too!It supportspyfilesystem2in order to save the files to cloud services, remote resource, as well as local directories.Dependencieslibarchive-cwhich requireslibarchive-develorlibarchive-devto be builttqdmfor progress bars (optional)You can usepoetryorpipto install the dependencies.UsageAs standalone programPlease seecli.py--helpAs libraryAvailable as package onPyPI.
Package: extract_genome_region
docstestspackageGiven a CSV file of variable information defining the regions of interest, return a file that contains a fasta-formatted representation of these regions.Free software: BSD licenseUsage$ extract_genome_region --help
Usage: extract_genome_region [OPTIONS] REGIONS IN_FASTA OUT_FASTA

  Given a CSV file of variable information defining the regions of interest
  along with input and output fasta file paths, write a file that contains a
  fasta-formatted representation of these regions.

  Structure of the `regions` CSV file:

    record_name   The name you want the seq to have in the new fasta.
       scaffold   The name of the seq record in the source fasta (chromosome, scaffold, contig, etc).
          start   The first bp of the seq feature you want in the new fasta.
           stop   The last bp of the seq feature you want in the new fasta.
       left_bfr   How many "extra" bp with coords smaller than `start` you want (0 for none).
      right_bfr   How many "extra" bp with coords larger than `stop` you want (0 for none).

  Naming Strategies:

            csv   use only the contents of the `record_name` field in the csv file (>CPR23).
      seq_range   use only the `scaffold` name and sequence range (>scaffold1:230-679).
  csv_seq_range   use both the contents of the `record_name` field and the `scaffold`.
                  name and sequence range (>CPR23 scaffold1:230-679).

Options:
  -n, --naming [csv|seq_range|csv_seq_range]
                                  Options regarding how each new fasta record
                                  will be named. See main help-text for
                                  explainations of options. [default='csv']
  --help                          Show this message and exit.Installation$ conda install -c bioconda -c gusdunn extract_genome_regionOr$ pip install extract_genome_regionDocumentationhttps://extract-genome-region.readthedocs.org/DevelopmentTo run the all tests run:toxChangelog0.0.3 (2016-05-24)altered meaning of ‘csv_seq_range’ to allow splitting fasta key on whitespace to ignore scaffold:range info0.0.2 (2016-05-24)expanded the help textbuilt the docs0.0.1 (2016-05-23)First release on anaconda.org/gusdunn.
Package: extract-html-diff
This package allows you to extract a difference between two html pages:
given pages A and B, it will try to extract parts of A that are changed in B.
It useslxml.html.diffunder the hood. but provides only changed parts as HTML.It requires Python 3 currently.License is MIT.InstallatonYou can install the package from PyPI:pip install extract-html-diffUsageYou can extract diff as text:import extract_html_diff

html = '<div> <h1>My site</h1> <div>My content</div> </div>'
other_html = '<div> <h1>My site</h1> <div>Other content</div> </div>'

extract_html_diff.as_string(html, other_html)this will give you:'<div><div>My content</div>  </div>'You can also get diff as a tree (anlxml.html.HtmlElement) if
you plan to do additional transformations or change serialization:extract_html_diff.as_tree(html, other_html)You can pass input html asstrorbytes(it will be parsed withlxml.html.fromstringin this case), or as an already parsedlxml.html.HtmlElement.
Package: extractify
ExtractifyExtractify is a command-line tool for converting documents in various formats (.pdf, .doc, .docx, .xlsx, .txt) to plain text. The tool works withboth local directories and S3 buckets.For local directories, the tool creates a 'txt' subdirectory within the specified input directory and saves the plain text files with the same filenames but with a .txt extension. For S3 buckets, it saves the plain text files in a 'txt' folder under the specified prefix.InstallationInstall Extractify using pip:pipinstallextractifyUsageLocallyTo use Extractifywith a local directory, run the following command:extractify<input_dir>(Replace<input_dir>with the path to the directory containing the documents you want to convert.)In S3To use Extractifywith an S3 bucket, run the following command:extractifys3://bucket-name/prefixReplacebucket-nameandprefixwith the appropriate values for your S3 bucket.Omit PDF files from the processTo omit PDF file formats, add the--omit-pdfflag:extractify<input_dirors3_bucket_address>--omit-pdfOutputExtractify will create a 'txt' subdirectory within the input directory and save the plain text files there.Supported FormatsExtractify currently supports the following document formats:.pdf.doc.docx.xlsx.txtDependenciesExtractify requires the following Python libraries:tikaopenpyxlargparsetqdmboto3
Package: extract-img-txt
Text-Image-ExtractorThis package helps to extract text from clear images such as screenshots, scanned pictures etc.The following steps demonstrate how to use this package:PrerequisitesPytesseract must be installed on your local devices.For Linux users, you must install pytesseract OCR using sudoInstall packageTo install, use the code below.pip install extract-img-textExtract textTo extract the text into your terminal, run the followingfrom image_extract.ext import ImgExt

#uncomment if you are a window user 

#pytesseract.tesseract_cmd = path_to_tesseract_on _your_device 

img = ImgExt(image_path="image_path",file_name="test")

# To return text

text=img.get_text()

print(text)

# To convert text to docx
img.convert_to_word()
Package: extractini
extractiniis a command-line application to extract a single option from an INI file with a structure
as described in thepython documentation.InstallInstall withpip.pip install extractiniUsageRunextractiniwith positional arguments for the path to the ini-file, the name of the ini file section, and the name of the ini file option.extractini [OPTIONS] CONFIGFILE SECTION OPTIONFor example, extractingtypefromchiliconfig.iniprintscapsicum:$ cat chiliconfig.ini
[habanero]
type=capsicum
heat=very hot
scovilles=100,000 - 350,000

$ extractini chiliconfig.ini habanero type
capsicumTestingextractinicomes with a few unit tests which can be run withpytest.ContributingIf you find something wrong feel free to submit a PR or raise an Issue on Github.LicenseCopyright 2018 Matt MaginPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Package: extract_iocs
extract_iocs is a Python module that extracts indicators of compromise (IOCs),
including domain names, IPv4 addresses, email addresses, and hashes, from
text. It uses some huge and ugly regexes, has special handling to identify
domain names with a relatively low false-positive rate, and does some magic to
try to extract IOCs across line breaks.This script was inspired by and initially based on Stephen Brannon’s
IOCextractor (https://github.com/stephenbrannon/IOCextractor), but turned into
a complete rewrite. extract_iocs provides no GUI and does not support any kind
of analyst workflow. It is intended to be used for triage or automation
purposes where a relatively high FP rate (as well as the occational false
negative) are acceptable.
Package: extractipedia
ExtractipediaExtractipedia aims to create plain text Wikipedia pages in order to use indexing while training machine learning models.Input file:Wikimedia XLS file which can be found onWikimedia Dumps.Output file:SQLite database file.Installation:pip install extractipediaBasic Usage:python -m extractipedia.Extraction -f file_name.xml[-f, --file_name] ==> File Name(str): Name of the Wikipedia Dump File (.xml)Tuning into the Script (Advanced):python -m extractipedia.Extraction -f file_name.xml -b batch_size -d database_file.db
-t table_name -n num_workers -s [first_sentence][-b, --batch_size] ==> Batch Size(int): RAM usage increases as the batch size gets bigger. (default = 2500)
[-d, --database_file] ==> Database File(str): Name of the SQLite database. The script will create for you if the file does not exist. (default = 'new_database.db')
[-t, --table_name] ==> Table Name(str): Name of the table for the database above. It will be created if it does not exist. (default = 'new_table')
[-n, --num_workers] ==> Number of Workers(int): Each process runs on different core. So the maximum process number equals to the cores that your machine has. But it is advisable that you should at least exclude 1 core in order to give your machine breathing room. You can give the core number directly. (default = max - 2)
[-s, --first_sentence] ==> First Sentence(bool): If you need just the first sentence of a page, just use -s flag. It's memory-friendly. (default = False)Check out your database once it is created:You can check out your database with the command below. CheckDatabase module is just for checking out.python -m extractipedia.CheckDatabase -f YOUR_DATABASE.db -t YOUR_TABLE -c chunk_size -r [random](optional) [-c, --chunk_size] ==> Chunk Size(int): It will retrieve the first n items from your database, don't type a large number which you might run into a memory problem. (default = 10)
(optional) [-r, --random] ==> Random(bool): If you want to retrieve random n items, just use -r flag. (default = False)Getting Help:[-h, --help] ==> It will print out the necessary information.Potential Improvements:Increase speed, using multiprocessing. (Done!)Progress bar. (Coming soon!)Simplify the regex, more human-readable.Get the templates and split the entire .xls into seperate and desired files.Get the tables (if there is any) and process them.Let there be extraction!
Package: extractlinks
extractlinksextract and repair links from Requests objects, including redirects and final landing pageInstallationpip install extractlinks
python3 -m pip install extractlinksUsageimport requests
from extractlinks import ExtractLinks
URL = "http://cnn.com/"
r = requests.get(URL, allow_redirects=True)
e = ExtractLinks(content=r)
print(e.json)Example Output[
	{
		"@timestamp": "2021-06-26T16:33:20.384Z",
		"url": {
			"full": "https://www.cnn.com/",
			"original": "https://www.cnn.com/",
			"scheme": "https",
			"domain": "www.cnn.com",
			"path": "/"
		},
		"http": {
			"response": {
			"status_code": 200,
			"status_code_reason": "OK",
			"body_bytes": 1110460
		},
		"chainitem": 2,
		"pguid": "1ff26fce-21a0-401a-9d53-1f863c6e3e31",
		"guid": "59dcfa56-b6d2-4924-bae1-70dbcd9d8309"
		"count": 324,
		"types": [
			"a-href",
			"form-action",
			"link-href",
			"meta-content",
			"script-src"
		],
		"tags": [
			"script",
			"meta",
			"a",
			"form",
			"link"
		],
		"attributes": [
			"action",
			"content",
			"src",
			"href"
		],
		"links": [
			"https://www.cnn.com/specials/cnn-investigates",
			"https://www.cnn.com/specials/tech/innovate",
			"https://www.cnn.com/travel/news",
			"https://www.i.cdn.cnn.com/.a/fonts/cnn/3.9.0/cnnsans-italic.woff2"
		...Objects# primary list-of-dictionaries / JSON dump
# these contain the full link extractions, including items not recognized as URLs or mobile links
output # list of dictionaries
json # JSON string

# lists
links_all # this only contains full links and any relative links "repaired" back to full-link format (ex. /images becomes https://www.cnn.com/images
types_all # ex. "a-href", "img-src", etc
tags_all # ex. "a", "img"
attributes_all # ex. "href", "src"

# generators, if urlbreakdown module is installed; runs URLBreakdown on every link in links_all
urlbreakdown_generator_dict()
urlbreakdown_generator_json()Notesselect URL and HTTP output fields align to the Elastic Common Schemalinks_count is not reflective of a unique count, and includes all objects identified including non-URLs in otherwise link-related tag attributes
Package: extract-mailchimp
MailChimp Extractor by Directory IDDownload mailchimp campaign data by campaign directory (folder) id
Origin bymailchimp-marketing-pythonRequirementsPython 3.6+Installationpip installpipinstallextract-mailchimp==0.0.7EndpointsMethodrequirementreturnget_campaign_raw_data_by_folder_id{campaign_folder_id}JSONget_campaign_id_by_folder_id{campaign_folder_id}listcampaign_information_to_pandas{campaign_id}pd.DataFrameclick_details_to_pandas{campaign_id}pd.DataFrameUsageyou must make.envfile with your mailchimp api key and serverfromextract_mailchimpimportMailChimpif__name__=="__main__":campaign_folder_id={campaign_folder_id}mc=MailChimp()print(mc.get_campaign_id_by_folder_id(campaign_folder_id))# [['{campaign_id}', '{campaign_title}']]