Saravanan666/pypi-org huggingface dataset:
Package: osin
osin ·There are existing systems (e.g., neptune.ai, sacred) helping you organize, log data of your experiments. However, typically, the tasks of running the experiments are your responsible to bear. If you update your code and need to re-run your experiments, you may want to delete previous runs, which would be painful to have to do manually many times.We rethink the experimenting process. Why don't we start with specifying the designed report (e.g., charts) and how to run/query to get the numbers to fill the report? This would free ones from manually starting/running the experiments and managing the experiment data.osinis a tool that helps you to achieve that goal.Note: this tool is expected to use locally or inside VPN network as it doesn't provide any protection against attackers.Quick startStart the application:DBFILE=%PATH_TO_DBFILE%python-mosin.mainOr start the services manually:exportDBFILE=%PATH_TO_DBFILE%
python-mosin.worker# start worker to run jobspython-mosin.server# start the server so clients can send job resultstreamlitrunosin/ui/dashboard.py# start a dashboard to view/create reportsYou will start by designing the output that your experiments will produce. For example:
Package: osint
Build your custom OSINT tools and APIs with this python package! It includes different OSINT modules for performing reconnaissance on the target, and a built-in database for mapping and visuialzing some of the reconnaissance results.The final results is a json output that can be intergrated with other projectsInstallsudopip3installosintUsage Example - Scan ips or domains for http and httpsfromosintimportQBDns,QBScantargets=QBDns().convert_to_ips(["https://test.com","1.2.3.4"])targets=QBScan().run(targets,[80,443])print(targets)Usage Example - Extract text from domainsfromosintimportQBDns,QBHost,QBExtracttargets=QBDns().convert_to_ips(["https://test.com"])targets=QBHost().run(targets)targets=QBExtract().run(targets,function="text")print(targets)Usage Example - Interact with the built-in databaseprint(QBGetInfo().cursor.execute(("SELECT * FROM ports WHERE port=?"),(80,)).fetchone())Usage Example - Test targetsudopython3-mosint--test"https://test.com"Current modulesQBDns() - Dns lookupsQBDns().convert_to_ips(targets)targetList of target domains or ips, the results is needed for the rest of modules e.g. [”http://test…”,”1.2.3.4”]QBHost() - Extract host information and certQBHost().run(targets,function)targetfrom QBDns().convert_to_ips() functionfunctionall, cert or contentQBCached() - Check archive.org from snapshotsQBCached().run(targets,from_date_in,to_date_in)targetfrom QBDns().convert_to_ips() functionfrom_date_in#start date as month/year e.g. 12/2020to_date_in#end date as month/year e.g. 12/2021QBExtract() - Extract text from pagesQBExtract().run(targets,function)targetfrom QBDns().convert_to_ips() functionfunctionall, text, metadata, links, image or languageQBScan() - Extract text from pagesQBScan.run(targets,ports,function)targetfrom QBDns().convert_to_ips() functionportsports to scan e.g. [80,443]functionall, sync, tcp, xmas, fin, null, ack, window or udpQBTraceRoute() - Extract text from pagesQBTraceRoute.run(targets)targetfrom QBDns().convert_to_ips() functionQBPing() - Ping hostQBPing.run(targets,function)targetfrom QBDns().convert_to_ips() functionfunction#all, arp, icmp or udpQBWhois() - Whois informationQBWhois.run(targets)targetfrom QBDns().convert_to_ips() functionQBICS() - Industrial Control Systems ScanningQBICS.run(targets)targetfrom QBDns().convert_to_ips() functionQBICS() module is not available and currently under testingBuilt-in Databasecountries_ids (country text, ctry text, cntry text, cid int, latitude int, longitude int, flag text)
countries_ips (ipfrom bigint, ipto bigint, registry text, assigned int, ctry text, cntry text, country text)
dns_servers (dns text, description text)
languages (ctry text, language text)
ports (port int, protocol text, service text, description text)
reserved_ips (ipfrom bigint, ipto bigint, description text)
temp_emails (email text, description text, blocked boolean)
url_shorteners (URL text, description text)AcknowledgementBy using this framework, you are accepting the license terms of all these packages:scapy tld netifaces dnspython beautifulsoup4 requests pyOpenSSL lxml langdetect
Package: osint-cli-tool-skeleton
OSINT cli tool skeletonTemplate for new OSINT command-line tools.Press button "Use this template" to generate your own tool repository.SeeINSTALL.mdfor further setup.FeaturesDetailed readmeProcess N targets from args, text files, stdinMake TXT, CSV reportsProxy supportReady to publish Python packageUsage$python3-mosint-cli-tool-skeleton<target># or simply$osint_cli_tool_skeleton<target>TargetsSpecify targets one or more times:$osint_cli_tool_skeletonwww.google.comreddit.compatreon.com

Target:www.google.com
Resultsfound:11)Value:Google
Code:200------------------------------
Target:patreon.com
Resultsfound:11)Value:Bestwayforartistsandcreatorstogetsustainableincomeandconnectwithfans|Patreon
Code:200------------------------------
Target:reddit.com
Resultsfound:11)Value:Reddit-Diveintoanything
Code:200------------------------------
Totalfound:3Or use a file with targets list:$osint_cli_tool_skeleton--target-listtargets.txtOr combine tool with other through input/output pipelining:$catlist.txt|osint_cli_tool_skeleton--targets-from-stdinReportsThe skeleton implements CSV reports:$osint_cli_tool_skeletonwww.google.comreddit.compatreon.com-oCresults.csv
...
Resultsweresavedtofileresults.csv

$moreresults.csv"Target","Value","Code""www.google.com","Google","200""patreon.com","Best way for artists and creators to get sustainable income and connect with fans | Patreon","200""reddit.com","Reddit - Dive into anything","200"And can save console output to text file separately:osint_cli_tool_skeletonwww.google.comreddit.compatreon.com-oTresults.txt
...
$head-n4results.txt
Target:www.google.com
Resultsfound:11)Value:Google
Code:200ProxyThe tool supports proxy:$osint_cli_tool_skeletonwww.google.com--proxyhttp://localhost:8080InstallationMake sure you have Python3 and pip installed.ManuallyClone ordownloadrespository$gitclonehttps://github.com/soxoj/osint-cli-tool-skeletonInstall dependencies$pip3install-rrequirements.txtAs a the packageYou can clone/download repo and install it from the directory to use as a Python package.$pip3install.Also you can install it from the PyPI registry:$pip3installhttps://github.com/soxoj/osint-cli-tool-skeleton
Package: osint-geo-extractor
geo_extractorLibrary to extract geo-related information from databases such asBellingcat,Cen4InfoRes,DefMon3,GeoConfirmedandTexty.org.ua.InstallationInstall fromPyPI:pip install osint-geo-extractorUsagefromgeo_extractorimportget_bellingcat_dataevents=get_bellingcat_data()foreinevents[:10]:print(f"{e.id}: [{e.latitude},{e.longitude}]")See also theexamples/folder.DocsConvenience functions:get_bellingcat_dataget_ceninfores_dataget_defmon_dataget_defmon_spreadsheet_dataget_geoconfirmed_dataget_texty_dataData is returned as a list ofEventobjects:FieldTypeidstrdatedatetimelatitudefloatlongitudefloatplace_descstrtitlestrdescriptionstrsourcestrlinksList[str]Data formats: Seedataformats/Exporting toGeoJSON: Useextractors.format_as_geojson(data){"type":"FeatureCollection","features":[{"type":"Feature","id":"CIV0001","geometry":{"type":"Point","coordinates":[36.659031,49.85005]},"properties":{"title":"<title>","date":"2022-02-24","description":"<desc>","links":["https://twitter.com/Michael1Sheldon/status/1496717647089651716","https://twitter.com/AFP/status/1496768532448788482"],"source":"BELLINGCAT"}}]}TestsTo obtain necessary testing data not shipped with the repository, install the
package into yourvirtualenvviapip install -e ., navigate totestsand runpython generate_fixtures.py.The tests can be run viapytest:$ pip install pytest
$ PYTHONPATH=. pytestTo run a single test, usepytest -k <test name>.To run thedownloadtests that require an internet connection,
usepytest --online -k download.LicenseMIT
Package: osint-python-starter-service
python-starter-serviceThis is a starter service project template.
Use this template to create a new service project.python3-mvenv.venvsource.venv/bin/activate
pipinstall-rrequirements.txt# Or instead of `pip`, use `pip3`Getting startedFirst you need to provide ENV variables for the service to run.ENV variablesMain ENV variables:CLIENT_ID- client id of the serviceREST_API_ENABLED- enable/disable REST API (default:true)KafkaCONSUME- comma separated list of topics to consumePRODUCE- comma separated list of topics to produceKAFKA_HOST- Kafka hostSCHEMA_REGISTRY- schema registry hostMAX_POLL_INTERVAL_MS- max poll interval in ms (default:600000)SESSION_TIMEOUT_MS- session timeout in ms (default:600000)UsageCheck the provided examples in theexamplesfolder.Examplefrom starter_service import StarterService

class ExampleService(StarterService):

    def health(self):
        """ Health check endpoint """
        return "OK"

    def ready(self):
        """ Ready check endpoint """    
        return True

    def process(self, message):
        """ Process message """

    @API.post(consumer="article_raw_en", producer="metadata_item_key_en", doc="Process raw article and return metadata")
    def handle_message(self, message: dict):
        """ Process raw article and return metadata """
        return {
            "articleId": message['id'],
            "origin": "string",
            "data": [
                {
                    "type": "string",
                    "value": "string",
                    "confidence": 0,
                    "metadata": {
                        "string": "string"
                    }
                }
            ]
        }

    if __name__ == '__main__':
        ExampleService()Uploading to PyPiEnsure you have the necessary tools installed: Make sure you havesetuptoolsandwheelinstalled. You can install them usingpip:# Build the distribution files: In the root directory of your project, run the following command to build the distribution files (wheel and source distribution):pipinstallsetuptoolswheeltwineBuild the distribution files: In the root directory of your project, run the following command to build the distribution files (wheel and source distribution):# This command will generate the distribution files inside the dist directory.# Remember to FIRST update the version number in your `setup.py` file for each new release to avoid conflicts.pythonsetup.pysdistbdist_wheelThis command will generate the distribution files inside the dist directory.Register an account on PyPI: If you haven't done so already, create an account on PyPI and verify your email address.Upload the package to PyPI: Use twine to upload the distribution files to PyPI:# This command will prompt you to enter your PyPI username and password. Once provided, twine will upload the distribution files to PyPI.twineuploaddist/*This command will prompt you to enter your PyPI username and password. Once provided, twine will upload the distribution files to PyPI.Verify the package on PyPI: Visit your package page onPyPIto ensure that the package has been successfully uploaded and published.
Package: osint-python-test-bed-adapter
NAAS-PYTHON-KAFKAThis is the Kafka adapter for Python: it allows you to easily connect Python
services to Apache Kafka via Python.The implementation is a wrapper aroundConfluent-Kafka-Python:AVRO schema's and messages: both key's and values should have a schema.
as explainedhere.Kafka consumer and producer for the test-bed topics.ManagementHeartbeat (topic: system-heartbeat), so you know which clients are online.
Each time the test-bed-adapter is executed, it starts a heartbeat process to notify
the its activity to other clients.InstallationYou need to installPython 3+.To run the examples you will need to install the dependencies specified on the filerequirements.txt.For that, runpython3-mvenv.venvsource.venv/bin/activate
pipinstall-rrequirements.txt# Or instead of `pip`, use `pip3`from the project folder.Examples and usageurl_producer: Creates a message with 4 URLs to RSS feeds on the topic (system_rss_urls)rss_producer: Listens to URL messages (system_rss_urls) and produces RSS messages (system_rss_urls)rss_consumer: Listens to RSS messages (system_rss_urls) and prints them to console.Uploading to PyPiEnsure you have the necessary tools installed: Make sure you havesetuptoolsandwheelinstalled. You can install them usingpip:# Build the distribution files: In the root directory of your project, run the following command to build the distribution files (wheel and source distribution):pipinstallsetuptoolswheelBuild the distribution files: In the root directory of your project, run the following command to build the distribution files (wheel and source distribution):# This command will generate the distribution files inside the dist directory.pythonsetup.pysdistbdist_wheelThis command will generate the distribution files inside the dist directory.Register an account on PyPI: If you haven't done so already, create an account on PyPI and verify your email address.Install and configuretwine: Installtwine, a tool used to upload packages to PyPI, usingpip:# Upload the package to PyPI: Use twine to upload the distribution files to PyPI:pipinstalltwineUpload the package to PyPI: Use twine to upload the distribution files to PyPI:# This command will prompt you to enter your PyPI username and password. Once provided, twine will upload the distribution files to PyPI.twineuploaddist/*This command will prompt you to enter your PyPI username and password. Once provided, twine will upload the distribution files to PyPI.Verify the package on PyPI: Visit your package page onPyPIto ensure that the package has been successfully uploaded and published.Remember to update the version number in yoursetup.pyfile for each new release to avoid conflicts.
Package: osipi
osipiis the authorative python package for perfusion MRI.For more detail please see thedocumentation.osipiis developed by the Open Source Initiative for Perfusion ImagingOSIPI, an initiative of the perfusion study group of the International Society for Magnetic Resonance in MedicineISMRM.Theosipipackage structure and logic follows thelexicondefined by OSIPI, and wraps around selected implementations collected in thecode contributionsof OSIPI.
Package: osirispy
OSIRISPYosirispyis a python package for reading simulation
output for the particle-in-cell code OSIRIS. It utilizes the numpy interface to provide
a simple way to readsimulation output.Installing osirispyosirispyis available on PyPI. You can install it by running the following
command inside your terminalpipinstallosirispy

pip3installosirispyUsing osirispyimport the package in pythonimportosirispyasospyYou can use the functionospy.read()to read osiris files. It will give a different output based on the type of file that is being opened.grid filesGrid files are the output of vdf objects and can have between 1 and 3 dimensions. They can represent charge density, current, E.M. fields, phasespaces, etc...ospy.read("/path/to/grid.h5")will output agridobject when reading a grid-type file.
The data is a numpy array accessible atgrid.dataThe object also contains a list of axis containing the physical dimentions of the grid file atgrid.axis.particle (aka RAW) filesParticle files contain the  output of the RAW diagnostic for a given species at a given time-step.
RAW files contain information on the quantitiesx1,x2,x3,p1,p2,p3,ene.ospy.read(filepath,req_quantitites)will output arawobject with the required quantities specified with a list of strings on thereq_quantitiesparameter.The data is a python dictionary of numpy arrays accessible at grid data. An example is given below:# read raw datarawdata=ospy.read("/path/to/raw.h5",("x1","x2"))#access x1 arrayx1=rawdata.data["x1"]The object also contains a dictionary with the label of each quantity accessible atraw.label.track  filesTrack files contain the output of the tracks diagnostic for a given species .
Track files contain information on the quantitiest,x1,x2,x3,p1,p2,p3,ene.ospy.read(filepath,req_quantitites)will output atracksobjectThe data is a python dictionary of a list of numpy arrays accessible at grid data. An example is given below:# read tracks datatrackdata=ospy.read("/path/to/track.h5",("x1","x2"))#access x1 array of particle ix1=trackdata.data["x1"][i]The object also contains a dictionary with the label of each quantity accessible atraw.label.
Package: osirisvalidator
OSIRIS VALIDATOROsiris Validator is a set of decorators for validation with SQLAlchemy(Readme is under construction...)Getting StartedInstallingpip install osirisvalidatorUsageTo use the decorators, thevalidates()decorator from SQLAlchemy must be used before, and the pattern must be followed.The parameter "field" is mandatory and you can set a custom message.fromsqlalchemy.ormimportvalidatesfromosirisvalidator.stringimport*fromosirisvalidator.internetimportvalid_emailclassUser(db.Model):id=db.Column(db.Integer,primary_key=True)name=db.Column(db.String(200),nullable=False)email=db.Column(db.String(60),nullable=False)@validates('name')@not_blank(field='name',message='Custom message')@is_alpha_space(field='name')@string_len(field='name',min=3,max=100)defvalidate_name(self,key,name):returnname@validates('email')@not_blank(field='email')@valid_email(field='email')defvalidate_email(self,key,email):returnemailWhen a validation error occurs, aValidationExceptionis thrown.Flask example@app.route('/saveuser',methods=['POST'])defsaveuser():try:user=User()user.name=request.form['name']user.email=request.form['email']db.session.add(user)db.session.commit()exceptValidationExceptionasve:flash(ve.errors)returnredirect(url_for('newuser'))returnredirect(url_for('index'))Flask-restless exampleThe parametervalidation_exceptionsinAPIManager.create_api()from Flask-restless must be set to use osiris'ValidationException.fromosirisvalidator.exceptionsimportValidationException[...]manager=APIManager(app,flask_sqlalchemy_db=db)manager.create_api(User,validation_exceptions=[ValidationException],methods=['GET','POST'])See about in:https://flask-restless.readthedocs.io/en/stable/customizing.html#capturing-validation-errorsList of validatorsosirisvalidator.stringnot_emptynot_blankis_alphais_alpha_space (alpha characters and space)is_alnumis_alnum_space (alphanumeric characters and space)is_digitstring_len (mandatory parameters:minandmax)match_regex (mandatory parameter:regex)osirisvalidator.numbermin_max (mandatory parameters:minandmax)osirisvalidator.internetvalid_emailosiris.intl.brvalid_cpfvalid_cnpj
Package: osirixgrpc
osirixgrpcWelcome to the osirixgrpc library. This provides the protobuf definitions for interacting with the osirixgrpc plugin. This code is autogenerated by the grpc tools using theproto template filesprovided in the mainosirixgrpc project.For day-day usage we suggest using the dedicatedpyosirix packageinstead.ExamplesEstablishing a connection with grpcimport grpc
import osirix_pb2_grpc

port = 12345  # Must match activated port in OsiriX plugin
server_url_localhost = 'localhost:' + str(port)
channel_opt = [('grpc.max_send_message_length', 512 * 1024 * 1024), ('grpc.max_receive_message_length', 512 * 1024 * 1024)]
channel = grpc.insecure_channel(server_url_localhost, options=channel_opt)
stub = osirix_pb2_grpc.OsiriXServiceStub(channel)Obtain a copy of the current browserimport utilities_pb2

request = utilities_pb2.Empty()  # For functions with no input, use an empty request
response = stub.OsirixCurrentBrowser(request)
if response.status.status == 0:
    raise Exception("Could not get browser.  Reason: %s" % response.status.message)
browser_controller = response.browser_controllerGet the current database selection within the browser controllerimport browsercontroller_pb2

response = stub.BrowserControllerDatabaseSelection(browser_controller)
series = response.series
studies = response.studiesObtain the names of the selected studiesimport dicomstudy_pb2

for dicom_study in studies:
    response = stub.DicomStudyName(dicom_study)
    print("Study name: ", response.name)
Package: osis-book-tools
OSIS Book ToolsThis packages contains tools for converting between OSIS abbreviations
(both IDs and paratext abbreviations) for books in the canonical Old and
New Testaments and localized book names.The core of this is a sqlite database which does all the mappings, and ensures basic
data integrity. Data is currently loaded once during module init. After that, all
operations are in-memory.This abbreviations were compiled fromhttps://wiki.crosswire.org/OSIS_Book_Abbreviations(fetched 2019-03-25).Usage ExamplesReference a book via enumeration valueOSISBook.Gen# GenesisOSISBook._1Sam# 1 Samuel (note the leading underscore to make the identifier valid)Construct a book from a paratext valueOSISBook.from_paratext("GEN")Get the paratext abbreviation for a bookOSISBook.Gen.paratext_abbreviationGet the localized name of a bookThis function expects the shortest possible ISO 639 code available for the language. English
is guaranteed to be available, so use this as a fallback.OSISBook.Gen.localized_name("en")Get the localized chapter descriptor for a bookThis function returns the chapter descriptor, which may be slightly different from the name. Use this when
constructing a reference string. For example, you writePsalm 23, notPsalms 23. This is the difference
betweenlocalized_nameandchapter_descriptor.Likelocalized_name, this function expects the shortest possible ISO 639 code available for the language. English
is guaranteed to be available, so use this as a fallback. If a localized name exists, a chapter descriptor must
also exist.OSISBook.Gen.chapter_descriptor("en")
Package: osjsonrpc
Over-Simple JSON RPC ImplementationOver-simpleJSON-RPCPython implementation foraiohttpInstallpip install osjsonrpcUsageExample:fromaiohttpimportwebfromosjsonrpcimportJsonRpcEndpointdefping():return"pong"defmultiply(a,b):return{"a":a,"b":b,"a*b":a*b}rpc=(JsonRpcEndpoint().register_method(ping).register_method(multiply))app=web.Application()app.add_routes([rpc.route("/api")])web.run_app(app)Call thepingmethod without arguments:~ curl -s -H "Content-Type: application/json" -H "Accept: application/json" -X POST \
  -d '{"jsonrpc":"2.0", "method": "ping", "id": 1}' \
  http://localhost:8080/api | jq

{
  "jsonrpc": "2.0",
  "result": "pong",
  "id": 1
}Call themultiplywith a couple of positional arguments:~ curl -s -H "Content-Type: application/json" -H "Accept: application/json" -X POST \
  -d '{"jsonrpc":"2.0", "method": "multiply", "params": [3, 5], "id": 1}' \
  http://localhost:8080/api | jq

{
  "jsonrpc": "2.0",
  "result": {
    "a": 3,
    "b": 5,
    "a*b": 15
  },
  "id": 1
}Call thepingargument with invalid keyword argument:~ curl -s -H "Content-Type: application/json" -H "Accept: application/json" -X POST \
  -d '{"jsonrpc":"2.0", "method": "ping", "params": {"key": "value"}, "id": 1}' \
  http://localhost:8080/api | jq

{
  "jsonrpc": "2.0",
  "error": {
    "code": -32602,
    "message": "Invalid params",
    "data": {
      "detail": "TypeError: ping() got an unexpected keyword argument 'key'",
      "request": {
        "jsonrpc": "2.0",
        "method": "ping",
        "params": {
          "key": "value"
        },
        "id": 1
      }
    }
  },
  "id": 1
}