neulab/docprompting-conala huggingface dataset:
sklearn.utils.extmath.randomized_svd(M, n_components, *, n_oversamples=10, n_iter='auto', power_iteration_normalizer='auto', transpose='auto', flip_sign=True, random_state=0) [source]
 
Computes a truncated randomized SVD.  Parameters 
 
M{ndarray, sparse matrix} 

Matrix to decompose.  
n_componentsint 

Number of singular values and vectors to extract.  
n_oversamplesint, default=10 

Additional number of random vectors to sample the range of M so as to ensure proper conditioning. The total number of random vectors used to find the range of M is n_components + n_oversamples. Smaller number can improve speed but can negatively impact the quality of approximation of singular vectors and singular values.  
n_iterint or ‘auto’, default=’auto’ 

Number of power iterations. It can be used to deal with very noisy problems. When ‘auto’, it is set to 4, unless n_components is small (< .1 * min(X.shape)) n_iter in which case is set to 7. This improves precision with few components.  Changed in version 0.18.   
power_iteration_normalizer{‘auto’, ‘QR’, ‘LU’, ‘none’}, default=’auto’ 

Whether the power iterations are normalized with step-by-step QR factorization (the slowest but most accurate), ‘none’ (the fastest but numerically unstable when n_iter is large, e.g. typically 5 or larger), or ‘LU’ factorization (numerically stable but can lose slightly in accuracy). The ‘auto’ mode applies no normalization if n_iter <= 2 and switches to LU otherwise.  New in version 0.18.   
transposebool or ‘auto’, default=’auto’ 

Whether the algorithm should be applied to M.T instead of M. The result should approximately be the same. The ‘auto’ mode will trigger the transposition if M.shape[1] > M.shape[0] since this implementation of randomized SVD tend to be a little faster in that case.  Changed in version 0.18.   
flip_signbool, default=True 

The output of a singular value decomposition is only unique up to a permutation of the signs of the singular vectors. If flip_sign is set to True, the sign ambiguity is resolved by making the largest loadings for each component in the left singular vectors positive.  
random_stateint, RandomState instance or None, default=0 

The seed of the pseudo random number generator to use when shuffling the data, i.e. getting the random vectors to initialize the algorithm. Pass an int for reproducible results across multiple function calls. See Glossary.     Notes This algorithm finds a (usually very good) approximate truncated singular value decomposition using randomization to speed up the computations. It is particularly fast on large matrices on which you wish to extract only a small number of components. In order to obtain further speed up, n_iter can be set <=2 (at the cost of loss of precision). References  Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions Halko, et al., 2009 https://arxiv.org/abs/0909.4061
 A randomized algorithm for the decomposition of matrices Per-Gunnar Martinsson, Vladimir Rokhlin and Mark Tygert An implementation of a randomized algorithm for principal component analysis A. Szlam et al. 2014
sklearn.utils.extmath.safe_sparse_dot(a, b, *, dense_output=False) [source]
 
Dot product that handle the sparse matrix case correctly.  Parameters 
 
a{ndarray, sparse matrix} 

b{ndarray, sparse matrix} 

dense_outputbool, default=False 

When False, a and b both being sparse will yield sparse output. When True, output will always be a dense array.    Returns 
 
dot_product{ndarray, sparse matrix} 

Sparse if a and b are sparse and dense_output=False.
sklearn.utils.extmath.weighted_mode(a, w, *, axis=0) [source]
 
Returns an array of the weighted modal (most common) value in a. If there is more than one such value, only the first is returned. The bin-count for the modal bins is also returned. This is an extension of the algorithm in scipy.stats.mode.  Parameters 
 
aarray-like 

n-dimensional array of which to find mode(s).  
warray-like 

n-dimensional array of weights for each value.  
axisint, default=0 

Axis along which to operate. Default is 0, i.e. the first axis.    Returns 
 
valsndarray 

Array of modal values.  
scorendarray 

Array of weighted counts for each mode.      See also  
scipy.stats.mode

  Examples >>> from sklearn.utils.extmath import weighted_mode
>>> x = [4, 1, 4, 2, 4, 2]
>>> weights = [1, 1, 1, 1, 1, 1]
>>> weighted_mode(x, weights)
(array([4.]), array([3.]))
 The value 4 appears three times: with uniform weights, the result is simply the mode of the distribution. >>> weights = [1, 3, 0.5, 1.5, 1, 2]  # deweight the 4's
>>> weighted_mode(x, weights)
(array([2.]), array([3.5]))
 The value 2 has the highest score: it appears twice with weights of 1.5 and 2: the sum of these is 3.5.
sklearn.utils.gen_even_slices(n, n_packs, *, n_samples=None) [source]
 
Generator to create n_packs slices going up to n.  Parameters 
 
nint 

n_packsint 

Number of slices to generate.  
n_samplesint, default=None 

Number of samples. Pass n_samples when the slices are to be used for sparse matrix indexing; slicing off-the-end raises an exception, while it works for NumPy arrays.    Yields 
 slice
   Examples >>> from sklearn.utils import gen_even_slices
>>> list(gen_even_slices(10, 1))
[slice(0, 10, None)]
>>> list(gen_even_slices(10, 10))
[slice(0, 1, None), slice(1, 2, None), ..., slice(9, 10, None)]
>>> list(gen_even_slices(10, 5))
[slice(0, 2, None), slice(2, 4, None), ..., slice(8, 10, None)]
>>> list(gen_even_slices(10, 3))
[slice(0, 4, None), slice(4, 7, None), slice(7, 10, None)]
sklearn.utils.graph.single_source_shortest_path_length(graph, source, *, cutoff=None) [source]
 
Return the shortest path length from source to all reachable nodes. Returns a dictionary of shortest path lengths keyed by target.  Parameters 
 
graph{sparse matrix, ndarray} of shape (n, n) 

Adjacency matrix of the graph. Sparse matrix of format LIL is preferred.  
sourceint 

Starting node for path.  
cutoffint, default=None 

Depth to stop the search - only paths of length <= cutoff are returned.     Examples >>> from sklearn.utils.graph import single_source_shortest_path_length
>>> import numpy as np
>>> graph = np.array([[ 0, 1, 0, 0],
...                   [ 1, 0, 1, 0],
...                   [ 0, 1, 0, 1],
...                   [ 0, 0, 1, 0]])
>>> list(sorted(single_source_shortest_path_length(graph, 0).items()))
[(0, 0), (1, 1), (2, 2), (3, 3)]
>>> graph = np.ones((6, 6))
>>> list(sorted(single_source_shortest_path_length(graph, 2).items()))
[(0, 1), (1, 1), (2, 0), (3, 1), (4, 1), (5, 1)]
sklearn.utils.graph_shortest_path.graph_shortest_path()  
Perform a shortest-path graph search on a positive directed or undirected graph.  Parameters 
 
dist_matrixarraylike or sparse matrix, shape = (N,N) 

Array of positive distances. If vertex i is connected to vertex j, then dist_matrix[i,j] gives the distance between the vertices. If vertex i is not connected to vertex j, then dist_matrix[i,j] = 0  
directedboolean 

if True, then find the shortest path on a directed graph: only progress from a point to its neighbors, not the other way around. if False, then find the shortest path on an undirected graph: the algorithm can progress from a point to its neighbors and vice versa.  
methodstring [‘auto’|’FW’|’D’] 

method to use. Options are ‘auto’ : attempt to choose the best method for the current problem ‘FW’ : Floyd-Warshall algorithm. O[N^3] ‘D’ : Dijkstra’s algorithm with Fibonacci stacks. O[(k+log(N))N^2]    Returns 
 
Gnp.ndarray, float, shape = [N,N] 

G[i,j] gives the shortest distance from point i to point j along the graph.     Notes As currently implemented, Dijkstra’s algorithm does not work for graphs with direction-dependent distances when directed == False. i.e., if dist_matrix[i,j] and dist_matrix[j,i] are not equal and both are nonzero, method=’D’ will not necessarily yield the correct result. Also, these routines have not been tested for graphs with negative distances. Negative distances can lead to infinite cycles that must be handled by specialized algorithms.
sklearn.utils.indexable(*iterables) [source]
 
Make arrays indexable for cross-validation. Checks consistent length, passes through None, and ensures that everything can be indexed by converting sparse matrices to csr and converting non-interable objects to arrays.  Parameters 
 
*iterables{lists, dataframes, ndarrays, sparse matrices} 

List of objects to ensure sliceability.
sklearn.utils.metaestimators.if_delegate_has_method(delegate) [source]
 
Create a decorator for methods that are delegated to a sub-estimator This enables ducktyping by hasattr returning True according to the sub-estimator.  Parameters 
 
delegatestring, list of strings or tuple of strings 

Name of the sub-estimator that can be accessed as an attribute of the base object. If a list or a tuple of names are provided, the first sub-estimator that is an attribute of the base object will be used.
sklearn.utils.multiclass.is_multilabel(y) [source]
 
Check if y is in a multilabel format.  Parameters 
 
yndarray of shape (n_samples,) 

Target values.    Returns 
 
outbool 

Return True, if y is in a multilabel format, else `False.     Examples >>> import numpy as np
>>> from sklearn.utils.multiclass import is_multilabel
>>> is_multilabel([0, 1, 0, 1])
False
>>> is_multilabel([[1], [0, 2], []])
False
>>> is_multilabel(np.array([[1, 0], [0, 0]]))
True
>>> is_multilabel(np.array([[1], [0], [0]]))
False
>>> is_multilabel(np.array([[1, 0, 0]]))
True
sklearn.utils.multiclass.type_of_target(y) [source]
 
Determine the type of data indicated by the target. Note that this type is the most specific type that can be inferred. For example:  
binary is more specific but compatible with multiclass. 
multiclass of integers is more specific but compatible with continuous. 
multilabel-indicator is more specific but compatible with multiclass-multioutput.   Parameters 
 
yarray-like 
  Returns 
 
target_typestr 

One of:  ‘continuous’: y is an array-like of floats that are not all integers, and is 1d or a column vector. ‘continuous-multioutput’: y is a 2d array of floats that are not all integers, and both dimensions are of size > 1. ‘binary’: y contains <= 2 discrete values and is 1d or a column vector. ‘multiclass’: y contains more than two discrete values, is not a sequence of sequences, and is 1d or a column vector. ‘multiclass-multioutput’: y is a 2d array that contains more than two discrete values, is not a sequence of sequences, and both dimensions are of size > 1. ‘multilabel-indicator’: y is a label indicator matrix, an array of two dimensions with at least two columns, and at most 2 unique values. ‘unknown’: y is array-like but none of the above, such as a 3d array, sequence of sequences, or an array of non-sequence objects.      Examples >>> import numpy as np
>>> type_of_target([0.1, 0.6])
'continuous'
>>> type_of_target([1, -1, -1, 1])
'binary'
>>> type_of_target(['a', 'b', 'a'])
'binary'
>>> type_of_target([1.0, 2.0])
'binary'
>>> type_of_target([1, 0, 2])
'multiclass'
>>> type_of_target([1.0, 0.0, 3.0])
'multiclass'
>>> type_of_target(['a', 'b', 'c'])
'multiclass'
>>> type_of_target(np.array([[1, 2], [3, 1]]))
'multiclass-multioutput'
>>> type_of_target([[1, 2]])
'multilabel-indicator'
>>> type_of_target(np.array([[1.5, 2.0], [3.0, 1.6]]))
'continuous-multioutput'
>>> type_of_target(np.array([[0, 1], [1, 1]]))
'multilabel-indicator'
sklearn.utils.multiclass.unique_labels(*ys) [source]
 
Extract an ordered array of unique labels.  We don’t allow:

 mix of multilabel and multiclass (single label) targets mix of label indicator matrix and anything else, because there are no explicit labels) mix of label indicator matrices of different sizes mix of string and integer labels    At the moment, we also don’t allow “multiclass-multioutput” input type.  Parameters 
 
*ysarray-likes 
  Returns 
 
outndarray of shape (n_unique_labels,) 

An ordered array of unique labels.     Examples >>> from sklearn.utils.multiclass import unique_labels
>>> unique_labels([3, 5, 5, 5, 7, 7])
array([3, 5, 7])
>>> unique_labels([1, 2, 3, 4], [2, 2, 3, 4])
array([1, 2, 3, 4])
>>> unique_labels([1, 2, 10], [5, 11])
array([ 1,  2,  5, 10, 11])
sklearn.utils.murmurhash3_32()  
Compute the 32bit murmurhash3 of key at seed. The underlying implementation is MurmurHash3_x86_32 generating low latency 32bits hash suitable for implementing lookup tables, Bloom filters, count min sketch or feature hashing.  Parameters 
 
keynp.int32, bytes, unicode or ndarray of dtype=np.int32 

The physical object to hash.  
seedint, default=0 

Integer seed for the hashing algorithm.  
positivebool, default=False 

 True: the results is casted to an unsigned int

from 0 to 2 ** 32 - 1  False: the results is casted to a signed int

from -(2 ** 31) to 2 ** 31 - 1
sklearn.utils.parallel_backend(backend, n_jobs=- 1, inner_max_num_threads=None, **backend_params) [source]
 
Change the default backend used by Parallel inside a with block. If backend is a string it must match a previously registered implementation using the register_parallel_backend function. By default the following backends are available:  ‘loky’: single-host, process-based parallelism (used by default), ‘threading’: single-host, thread-based parallelism, ‘multiprocessing’: legacy single-host, process-based parallelism.  ‘loky’ is recommended to run functions that manipulate Python objects. ‘threading’ is a low-overhead alternative that is most efficient for functions that release the Global Interpreter Lock: e.g. I/O-bound code or CPU-bound code in a few calls to native code that explicitly releases the GIL. In addition, if the dask and distributed Python packages are installed, it is possible to use the ‘dask’ backend for better scheduling of nested parallel calls without over-subscription and potentially distribute parallel calls over a networked cluster of several hosts. It is also possible to use the distributed ‘ray’ backend for distributing the workload to a cluster of nodes. To use the ‘ray’ joblib backend add the following lines: >>> from ray.util.joblib import register_ray  
>>> register_ray()  
>>> with parallel_backend("ray"):  
...     print(Parallel()(delayed(neg)(i + 1) for i in range(5)))
[-1, -2, -3, -4, -5]
 Alternatively the backend can be passed directly as an instance. By default all available workers will be used (n_jobs=-1) unless the caller passes an explicit value for the n_jobs parameter. This is an alternative to passing a backend='backend_name' argument to the Parallel class constructor. It is particularly useful when calling into library code that uses joblib internally but does not expose the backend argument in its own API. >>> from operator import neg
>>> with parallel_backend('threading'):
...     print(Parallel()(delayed(neg)(i + 1) for i in range(5)))
...
[-1, -2, -3, -4, -5]
 Warning: this function is experimental and subject to change in a future version of joblib. Joblib also tries to limit the oversubscription by limiting the number of threads usable in some third-party library threadpools like OpenBLAS, MKL or OpenMP. The default limit in each worker is set to max(cpu_count() // effective_n_jobs, 1) but this limit can be overwritten with the inner_max_num_threads argument which will be used to set this limit in the child processes.  New in version 0.10.
sklearn.utils.random.sample_without_replacement()  
Sample integers without replacement. Select n_samples integers from the set [0, n_population) without replacement.  Parameters 
 
n_populationint 

The size of the set to sample from.  
n_samplesint 

The number of integer to sample.  
random_stateint, RandomState instance or None, default=None 

If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.  
method{“auto”, “tracking_selection”, “reservoir_sampling”, “pool”}, default=’auto’ 

If method == “auto”, the ratio of n_samples / n_population is used to determine which algorithm to use: If ratio is between 0 and 0.01, tracking selection is used. If ratio is between 0.01 and 0.99, numpy.random.permutation is used. If ratio is greater than 0.99, reservoir sampling is used. The order of the selected integers is undefined. If a random order is desired, the selected subset should be shuffled. If method ==”tracking_selection”, a set based implementation is used which is suitable for n_samples <<< n_population. If method == “reservoir_sampling”, a reservoir sampling algorithm is used which is suitable for high memory constraint or when O(n_samples) ~ O(n_population). The order of the selected integers is undefined. If a random order is desired, the selected subset should be shuffled. If method == “pool”, a pool based algorithm is particularly fast, even faster than the tracking selection method. However, a vector containing the entire population has to be initialized. If n_samples ~ n_population, the reservoir sampling method is faster.    Returns 
 
outndarray of shape (n_samples,) 

The sampled subsets of integer. The subset of selected integer might not be randomized, see the method argument.
sklearn.utils.register_parallel_backend(name, factory, make_default=False) [source]
 
Register a new Parallel backend factory. The new backend can then be selected by passing its name as the backend argument to the Parallel class. Moreover, the default backend can be overwritten globally by setting make_default=True. The factory can be any callable that takes no argument and return an instance of ParallelBackendBase. Warning: this function is experimental and subject to change in a future version of joblib.  New in version 0.10.
sklearn.utils.resample(*arrays, replace=True, n_samples=None, random_state=None, stratify=None) [source]
 
Resample arrays or sparse matrices in a consistent way. The default strategy implements one step of the bootstrapping procedure.  Parameters 
 
*arrayssequence of array-like of shape (n_samples,) or (n_samples, n_outputs) 

Indexable data-structures can be arrays, lists, dataframes or scipy sparse matrices with consistent first dimension.  
replacebool, default=True 

Implements resampling with replacement. If False, this will implement (sliced) random permutations.  
n_samplesint, default=None 

Number of samples to generate. If left to None this is automatically set to the first dimension of the arrays. If replace is False it should not be larger than the length of arrays.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for shuffling the data. Pass an int for reproducible results across multiple function calls. See Glossary.  
stratifyarray-like of shape (n_samples,) or (n_samples, n_outputs), default=None 

If not None, data is split in a stratified fashion, using this as the class labels.    Returns 
 
resampled_arrayssequence of array-like of shape (n_samples,) or (n_samples, n_outputs) 

Sequence of resampled copies of the collections. The original arrays are not impacted.      See also  
shuffle

  Examples It is possible to mix sparse and dense arrays in the same run: >>> X = np.array([[1., 0.], [2., 1.], [0., 0.]])
>>> y = np.array([0, 1, 2])

>>> from scipy.sparse import coo_matrix
>>> X_sparse = coo_matrix(X)

>>> from sklearn.utils import resample
>>> X, X_sparse, y = resample(X, X_sparse, y, random_state=0)
>>> X
array([[1., 0.],
       [2., 1.],
       [1., 0.]])

>>> X_sparse
<3x2 sparse matrix of type '<... 'numpy.float64'>'
    with 4 stored elements in Compressed Sparse Row format>

>>> X_sparse.toarray()
array([[1., 0.],
       [2., 1.],
       [1., 0.]])

>>> y
array([0, 1, 0])

>>> resample(y, n_samples=2, random_state=0)
array([0, 1])
 Example using stratification: >>> y = [0, 0, 1, 1, 1, 1, 1, 1, 1]
>>> resample(y, n_samples=5, replace=False, stratify=y,
...          random_state=0)
[1, 1, 1, 0, 1]
sklearn.utils.safe_mask(X, mask) [source]
 
Return a mask which is safe to use on X.  Parameters 
 
X{array-like, sparse matrix} 

Data on which to apply mask.  
maskndarray 

Mask to be used on X.    Returns 
 mask
sklearn.utils.safe_sqr(X, *, copy=True) [source]
 
Element wise squaring of array-likes and sparse matrices.  Parameters 
 
X{array-like, ndarray, sparse matrix} 

copybool, default=True 

Whether to create a copy of X and operate on it or to perform inplace computation (default behaviour).    Returns 
 
X ** 2element wise square
sklearn.utils.shuffle(*arrays, random_state=None, n_samples=None) [source]
 
Shuffle arrays or sparse matrices in a consistent way. This is a convenience alias to resample(*arrays, replace=False) to do random permutations of the collections.  Parameters 
 
*arrayssequence of indexable data-structures 

Indexable data-structures can be arrays, lists, dataframes or scipy sparse matrices with consistent first dimension.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for shuffling the data. Pass an int for reproducible results across multiple function calls. See Glossary.  
n_samplesint, default=None 

Number of samples to generate. If left to None this is automatically set to the first dimension of the arrays. It should not be larger than the length of arrays.    Returns 
 
shuffled_arrayssequence of indexable data-structures 

Sequence of shuffled copies of the collections. The original arrays are not impacted.      See also  
resample

  Examples It is possible to mix sparse and dense arrays in the same run: >>> X = np.array([[1., 0.], [2., 1.], [0., 0.]])
>>> y = np.array([0, 1, 2])

>>> from scipy.sparse import coo_matrix
>>> X_sparse = coo_matrix(X)

>>> from sklearn.utils import shuffle
>>> X, X_sparse, y = shuffle(X, X_sparse, y, random_state=0)
>>> X
array([[0., 0.],
       [2., 1.],
       [1., 0.]])

>>> X_sparse
<3x2 sparse matrix of type '<... 'numpy.float64'>'
    with 3 stored elements in Compressed Sparse Row format>

>>> X_sparse.toarray()
array([[0., 0.],
       [2., 1.],
       [1., 0.]])

>>> y
array([2, 1, 0])

>>> shuffle(y, n_samples=2, random_state=0)
array([0, 1])
sklearn.utils.sparsefuncs.incr_mean_variance_axis(X, *, axis, last_mean, last_var, last_n, weights=None) [source]
 
Compute incremental mean and variance along an axis on a CSR or CSC matrix. last_mean, last_var are the statistics computed at the last step by this function. Both must be initialized to 0-arrays of the proper size, i.e. the number of features in X. last_n is the number of samples encountered until now.  Parameters 
 
XCSR or CSC sparse matrix of shape (n_samples, n_features) 

Input data.  
axis{0, 1} 

Axis along which the axis should be computed.  
last_meanndarray of shape (n_features,) or (n_samples,), dtype=floating 

Array of means to update with the new data X. Should be of shape (n_features,) if axis=0 or (n_samples,) if axis=1.  
last_varndarray of shape (n_features,) or (n_samples,), dtype=floating 

Array of variances to update with the new data X. Should be of shape (n_features,) if axis=0 or (n_samples,) if axis=1.  
last_nfloat or ndarray of shape (n_features,) or (n_samples,), dtype=floating 

Sum of the weights seen so far, excluding the current weights If not float, it should be of shape (n_samples,) if axis=0 or (n_features,) if axis=1. If float it corresponds to having same weights for all samples (or features).  
weightsndarray of shape (n_samples,) or (n_features,), default=None 

If axis is set to 0 shape is (n_samples,) or if axis is set to 1 shape is (n_features,). If it is set to None, then samples are equally weighted.  New in version 0.24.     Returns 
 
meansndarray of shape (n_features,) or (n_samples,), dtype=floating 

Updated feature-wise means if axis = 0 or sample-wise means if axis = 1.  
variancesndarray of shape (n_features,) or (n_samples,), dtype=floating 

Updated feature-wise variances if axis = 0 or sample-wise variances if axis = 1.  
nndarray of shape (n_features,) or (n_samples,), dtype=integral 

Updated number of seen samples per feature if axis=0 or number of seen features per sample if axis=1. If weights is not None, n is a sum of the weights of the seen samples or features instead of the actual number of seen samples or features.     Notes NaNs are ignored in the algorithm.