semvec/pypi-packages huggingface dataset:
asreview-notes-export 0.0.7
ASReview notes export extension
Keywords: asreview notes export

[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) 
[![PyPI version](https://badge.fury.io/py/asreview-notes-export.svg)](https://badge.fury.io/py/asreview-notes-export)

# ASReview Notes Export Extension
This is a simple extension for [ASReview](https://github.com/asreview) ![logo](https://raw.githubusercontent.com/asreview/asreview-artwork/e2e6e5ea58a22077b116b9c3d2a15bc3fea585c7/SVGicons/IconELAS/ELASeyes24px24px.svg "ASReview") that can be used to export notes from the ASReview GUI while exporting the labeled or partially labeled dataset as .csv file.

## Installation
ASReview notes-export can be installed from PyPI:
```bash
pip install asreview-notes-export
```

After installation, check if the `asreview-notes-export` package is listed as an
extension. Use the following command:

```bash
asreview --help
```

It should list the `notes_export` subcommand.

## Usage
To export all the records from the dataset with the current state of labels and notes, we can use the following command:
```
asreview notes_export YOUR_ASREVIEW_FILE.asreview
```

To export only the records with notes from the dataset, we can use the following command:
```
asreview notes_export --only-with-notes YOUR_ASREVIEW_FILE.asreview
```

To include the labeling order in the exported dataset, use following optional argument:
```
asreview notes_export --labeling-order YOUR_ASREVIEW_FILE.asreview
``` 

By default, the filename of the output .csv is the same as the name of .asreview file. To save the output to .csv file with a custom filename, we can use the following command. We can either specify the extension:
```
asreview notes_export --only-with-notes YOUR_ASREVIEW_FILE.asreview -o YOUR_OUTPUT_FILENAME.csv
```  

or only the filename will suffice, without an extension, using the following command:
```
asreview notes_export --only-with-notes YOUR_ASREVIEW_FILE.asreview -o YOUR_OUTPUT_FILENAME
``` 

Note that currently only .csv output is supported. 

## License
The extension is published under the [Apache 2.0 license](https://github.com/rohitgarud/asreview-notes-export/blob/main/LICENSE).

## Contact
This is an unofficial extension of ASReview and is developed and maintained by [Rohit Garud](https://github.com/rohitgarud).
parallel-statistics 0.13
Calculating basic statistics in parallel, incrementally
Requires Python: >=3.6
Keywords: MPI,statistics

Overview
--------

This package collects tools which compute weighted statistics on parallel, incremental data, i.e. data being read by multiple processors, a chunk at a time.  

The tools available are:
- ``ParallelSum``
- ``ParallelMean``
- ``ParallelMeanVariance``
- ``ParallelHistogram``
- ``SparseArray``

All assume that mpi4py is being used among the processes, and are passed a communicator object (often ``mpi4py.MPI.COMM_WORLD``).

Installation
------------

For now you can install this package using:

```
pip install parallel_statistics
```

Documentation
-------------

Documentation can be found at https://parallel-statistics.readthedocs.io/

Example
-------

The three tools ``ParallelSum``, ``ParallelMean``, and ``ParallelMeanVariance`` compute statistics in bins, and you add data to them per bin.

The usage pattern for them, and ``ParallelHistogram``, is:

- Create a parallel calculator object in each MPI process
- Have each process read in their own chunks of data and add it using the ``add_data`` methods
- Once complete, call the ``collect`` method to get the combined results.

Here's an example of splitting up data from an HDF5 file, using an example from the DESC tomographic challenge.  You can run it either on its own, or under MPI with different numbers of processors, and the results should be the same:

```python
import mpi4py.MPI
import h5py
import parallel_statistics
import numpy as np

# This data file is available at
# https://portal.nersc.gov/project/lsst/txpipe/tomo_challenge_data/ugrizy/mini_training.hdf5
f = h5py.File("mini_training.hdf5", "r")
comm = mpi4py.MPI.COMM_WORLD

# We must divide up the data between the processes
# Choose the chunk sizes to use here
chunk_size = 1000
total_size = f['redshift_true'].size
nchunk = total_size // chunk_size
if nchunk * chunk_size < total_size:
    nchunk += 1

# Choose the binning in which to put values
nbin = 20
dz = 0.2

# Make our calculator
calc = parallel_statistics.ParallelMeanVariance(size=nbin)

# Loop through the data
for i in range(nchunk):
    # Each process only reads its assigned chunks,
    # otherwise, skip this chunk
    if i % comm.size != comm.rank:
        continue
    # work out the data range to read
    start = i * chunk_size
    end = start + chunk_size

    # read in the input data
    z = f['redshift_true'][start:end]
    r = f['r_mag'][start:end]

    # Work out which bins to use for it
    b = (z / dz).astype(int)

    # add add each one
    for j in range(z.size):
        # skip inf, nan, and sentinel values
        if not r[j] < 30:
            continue
        # add each data point
        calc.add_datum(b[j], r[j])

# Finally, collect the results together
weight, mean, variance = calc.collect(comm)

# Print out results - only the root process gets the data, unless you pass
# mode=allreduce to collect.  Will print out NaNs for bins with no objects in.
if comm.rank == 0:
    for i in range(nbin):
        print(f"z = [{ dz * i :.1f} .. { dz * (i+1) :.1f}]    r = { mean[i] :.2f} ± { variance[i] :.2f}")
```