neulab/docprompting-conala huggingface dataset:
sklearn.datasets.fetch_20newsgroups(*, data_home=None, subset='train', categories=None, shuffle=True, random_state=42, remove=(), download_if_missing=True, return_X_y=False) [source]
 
Load the filenames and data from the 20 newsgroups dataset (classification). Download it if necessary.  
Classes 20  
Samples total 18846  
Dimensionality 1  
Features text   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify a download and cache folder for the datasets. If None, all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
subset{‘train’, ‘test’, ‘all’}, default=’train’ 

Select the dataset to load: ‘train’ for the training set, ‘test’ for the test set, ‘all’ for both, with shuffled ordering.  
categoriesarray-like, dtype=str or unicode, default=None 

If None (default), load all the categories. If not None, list of category names to load (other categories ignored).  
shufflebool, default=True 

Whether or not to shuffle the data: might be important for models that make the assumption that the samples are independent and identically distributed (i.i.d.), such as stochastic gradient descent.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See Glossary.  
removetuple, default=() 

May contain any subset of (‘headers’, ‘footers’, ‘quotes’). Each of these are kinds of text that will be detected and removed from the newsgroup posts, preventing classifiers from overfitting on metadata. ‘headers’ removes newsgroup headers, ‘footers’ removes blocks at the ends of posts that look like signatures, and ‘quotes’ removes lines that appear to be quoting another post. ‘headers’ follows an exact standard; the other filters are not always correct.  
download_if_missingbool, default=True 

If False, raise an IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.22.     Returns 
 
bunchBunch 

Dictionary-like object, with the following attributes.  
datalist of shape (n_samples,) 

The data list to learn.  target: ndarray of shape (n_samples,)

The target labels.  filenames: list of shape (n_samples,)

The path to the location of the data.  DESCR: str

The full description of the dataset.  target_names: list of shape (n_classes,)

The names of target classes.    
(data, target)tuple if return_X_y=True 

 New in version 0.22.
sklearn.datasets.fetch_20newsgroups_vectorized(*, subset='train', remove=(), data_home=None, download_if_missing=True, return_X_y=False, normalize=True, as_frame=False) [source]
 
Load and vectorize the 20 newsgroups dataset (classification). Download it if necessary. This is a convenience function; the transformation is done using the default settings for CountVectorizer. For more advanced usage (stopword filtering, n-gram extraction, etc.), combine fetch_20newsgroups with a custom CountVectorizer, HashingVectorizer, TfidfTransformer or TfidfVectorizer. The resulting counts are normalized using sklearn.preprocessing.normalize unless normalize is set to False.  
Classes 20  
Samples total 18846  
Dimensionality 130107  
Features real   Read more in the User Guide.  Parameters 
 
subset{‘train’, ‘test’, ‘all’}, default=’train’ 

Select the dataset to load: ‘train’ for the training set, ‘test’ for the test set, ‘all’ for both, with shuffled ordering.  
removetuple, default=() 

May contain any subset of (‘headers’, ‘footers’, ‘quotes’). Each of these are kinds of text that will be detected and removed from the newsgroup posts, preventing classifiers from overfitting on metadata. ‘headers’ removes newsgroup headers, ‘footers’ removes blocks at the ends of posts that look like signatures, and ‘quotes’ removes lines that appear to be quoting another post.  
data_homestr, default=None 

Specify an download and cache folder for the datasets. If None, all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
download_if_missingbool, default=True 

If False, raise an IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.20.   
normalizebool, default=True 

If True, normalizes each document’s feature vector to unit norm using sklearn.preprocessing.normalize.  New in version 0.22.   
as_framebool, default=False 

If True, the data is a pandas DataFrame including columns with appropriate dtypes (numeric, string, or categorical). The target is a pandas DataFrame or Series depending on the number of target_columns.  New in version 0.24.     Returns 
 
bunchBunch 

Dictionary-like object, with the following attributes.  data: {sparse matrix, dataframe} of shape (n_samples, n_features)

The input data matrix. If as_frame is True, data is a pandas DataFrame with sparse columns.  target: {ndarray, series} of shape (n_samples,)

The target labels. If as_frame is True, target is a pandas Series.  target_names: list of shape (n_classes,)

The names of target classes.  DESCR: str

The full description of the dataset.  frame: dataframe of shape (n_samples, n_features + 1)

Only present when as_frame=True. Pandas DataFrame with data and target.  New in version 0.24.     
(data, target)tuple if return_X_y is True 

data and target would be of the format defined in the Bunch description above.  New in version 0.20.
sklearn.datasets.fetch_california_housing(*, data_home=None, download_if_missing=True, return_X_y=False, as_frame=False) [source]
 
Load the California housing dataset (regression).  
Samples total 20640  
Dimensionality 8  
Features real  
Target real 0.15 - 5.   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False. 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.20.   
as_framebool, default=False 

If True, the data is a pandas DataFrame including columns with appropriate dtypes (numeric, string or categorical). The target is a pandas DataFrame or Series depending on the number of target_columns.  New in version 0.23.     Returns 
 
datasetBunch 

Dictionary-like object, with the following attributes.  
datandarray, shape (20640, 8) 

Each row corresponding to the 8 feature values in order. If as_frame is True, data is a pandas object.  
targetnumpy array of shape (20640,) 

Each value corresponds to the average house value in units of 100,000. If as_frame is True, target is a pandas object.  
feature_nameslist of length 8 

Array of ordered feature names used in the dataset.  
DESCRstring 

Description of the California housing dataset.  
framepandas DataFrame 

Only present when as_frame=True. DataFrame with data and target.  New in version 0.23.     
(data, target)tuple if return_X_y is True 

 New in version 0.20.      Notes This dataset consists of 20,640 samples and 9 features.
sklearn.datasets.fetch_covtype(*, data_home=None, download_if_missing=True, random_state=None, shuffle=False, return_X_y=False, as_frame=False) [source]
 
Load the covertype dataset (classification). Download it if necessary.  
Classes 7  
Samples total 581012  
Dimensionality 54  
Features int   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See Glossary.  
shufflebool, default=False 

Whether to shuffle dataset.  
return_X_ybool, default=False 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.20.   
as_framebool, default=False 

If True, the data is a pandas DataFrame including columns with appropriate dtypes (numeric). The target is a pandas DataFrame or Series depending on the number of target columns. If return_X_y is True, then (data, target) will be pandas DataFrames or Series as described below.  New in version 0.24.     Returns 
 
datasetBunch 

Dictionary-like object, with the following attributes.  
datandarray of shape (581012, 54) 

Each row corresponds to the 54 features in the dataset.  
targetndarray of shape (581012,) 

Each value corresponds to one of the 7 forest covertypes with values ranging between 1 to 7.  
framedataframe of shape (581012, 53) 

Only present when as_frame=True. Contains data and target.  
DESCRstr 

Description of the forest covertype dataset.  
feature_nameslist 

The names of the dataset columns.  target_names: list

The names of the target columns.    
(data, target)tuple if return_X_y is True 

 New in version 0.20.
sklearn.datasets.fetch_kddcup99(*, subset=None, data_home=None, shuffle=False, random_state=None, percent10=True, download_if_missing=True, return_X_y=False, as_frame=False) [source]
 
Load the kddcup99 dataset (classification). Download it if necessary.  
Classes 23  
Samples total 4898431  
Dimensionality 41  
Features discrete (int) or continuous (float)   Read more in the User Guide.  New in version 0.18.   Parameters 
 
subset{‘SA’, ‘SF’, ‘http’, ‘smtp’}, default=None 

To return the corresponding classical subsets of kddcup 99. If None, return the entire kddcup 99 dataset.  
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders. .. versionadded:: 0.19  
shufflebool, default=False 

Whether to shuffle dataset.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset shuffling and for selection of abnormal samples if subset='SA'. Pass an int for reproducible output across multiple function calls. See Glossary.  
percent10bool, default=True 

Whether to load only 10 percent of the data.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False 

If True, returns (data, target) instead of a Bunch object. See below for more information about the data and target object.  New in version 0.20.   
as_framebool, default=False 

If True, returns a pandas Dataframe for the data and target objects in the Bunch returned object; Bunch return object will also have a frame member.  New in version 0.24.     Returns 
 
dataBunch 

Dictionary-like object, with the following attributes.  
data{ndarray, dataframe} of shape (494021, 41) 

The data matrix to learn. If as_frame=True, data will be a pandas DataFrame.  
target{ndarray, series} of shape (494021,) 

The regression target for each sample. If as_frame=True, target will be a pandas Series.  
framedataframe of shape (494021, 42) 

Only present when as_frame=True. Contains data and target.  
DESCRstr 

The full description of the dataset.  
feature_nameslist 

The names of the dataset columns  target_names: list

The names of the target columns    
(data, target)tuple if return_X_y is True 

 New in version 0.20.
sklearn.datasets.fetch_lfw_pairs(*, subset='train', data_home=None, funneled=True, resize=0.5, color=False, slice_=slice(70, 195, None), slice(78, 172, None), download_if_missing=True) [source]
 
Load the Labeled Faces in the Wild (LFW) pairs dataset (classification). Download it if necessary.  
Classes 2  
Samples total 13233  
Dimensionality 5828  
Features real, between 0 and 255   In the official README.txt this task is described as the “Restricted” task. As I am not sure as to implement the “Unrestricted” variant correctly, I left it as unsupported for now. The original images are 250 x 250 pixels, but the default slice and resize arguments reduce them to 62 x 47. Read more in the User Guide.  Parameters 
 
subset{‘train’, ‘test’, ‘10_folds’}, default=’train’ 

Select the dataset to load: ‘train’ for the development training set, ‘test’ for the development test set, and ‘10_folds’ for the official evaluation set that is meant to be used with a 10-folds cross validation.  
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
funneledbool, default=True 

Download and use the funneled variant of the dataset.  
resizefloat, default=0.5 

Ratio used to resize the each face picture.  
colorbool, default=False 

Keep the 3 RGB channels instead of averaging them to a single gray level channel. If color is True the shape of the data has one more dimension than the shape with color = False.  
slice_tuple of slice, default=(slice(70, 195), slice(78, 172)) 

Provide a custom 2D slice (height, width) to extract the ‘interesting’ part of the jpeg files and avoid use statistical correlation from the background  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.    Returns 
 
dataBunch 

Dictionary-like object, with the following attributes.  
datandarray of shape (2200, 5828). Shape depends on subset. 

Each row corresponds to 2 ravel’d face images of original size 62 x 47 pixels. Changing the slice_, resize or subset parameters will change the shape of the output.  
pairsndarray of shape (2200, 2, 62, 47). Shape depends on subset 

Each row has 2 face images corresponding to same or different person from the dataset containing 5749 people. Changing the slice_, resize or subset parameters will change the shape of the output.  
targetnumpy array of shape (2200,). Shape depends on subset. 

Labels associated to each pair of images. The two label values being different persons or the same person.  
DESCRstring 

Description of the Labeled Faces in the Wild (LFW) dataset.
sklearn.datasets.fetch_lfw_people(*, data_home=None, funneled=True, resize=0.5, min_faces_per_person=0, color=False, slice_=slice(70, 195, None), slice(78, 172, None), download_if_missing=True, return_X_y=False) [source]
 
Load the Labeled Faces in the Wild (LFW) people dataset (classification). Download it if necessary.  
Classes 5749  
Samples total 13233  
Dimensionality 5828  
Features real, between 0 and 255   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
funneledbool, default=True 

Download and use the funneled variant of the dataset.  
resizefloat, default=0.5 

Ratio used to resize the each face picture.  
min_faces_per_personint, default=None 

The extracted dataset will only retain pictures of people that have at least min_faces_per_person different pictures.  
colorbool, default=False 

Keep the 3 RGB channels instead of averaging them to a single gray level channel. If color is True the shape of the data has one more dimension than the shape with color = False.  
slice_tuple of slice, default=(slice(70, 195), slice(78, 172)) 

Provide a custom 2D slice (height, width) to extract the ‘interesting’ part of the jpeg files and avoid use statistical correlation from the background  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False 

If True, returns (dataset.data, dataset.target) instead of a Bunch object. See below for more information about the dataset.data and dataset.target object.  New in version 0.20.     Returns 
 
datasetBunch 

Dictionary-like object, with the following attributes.  
datanumpy array of shape (13233, 2914) 

Each row corresponds to a ravelled face image of original size 62 x 47 pixels. Changing the slice_ or resize parameters will change the shape of the output.  
imagesnumpy array of shape (13233, 62, 47) 

Each row is a face image corresponding to one of the 5749 people in the dataset. Changing the slice_ or resize parameters will change the shape of the output.  
targetnumpy array of shape (13233,) 

Labels associated to each face image. Those labels range from 0-5748 and correspond to the person IDs.  
DESCRstring 

Description of the Labeled Faces in the Wild (LFW) dataset.    
(data, target)tuple if return_X_y is True 

 New in version 0.20.
sklearn.datasets.fetch_olivetti_faces(*, data_home=None, shuffle=False, random_state=0, download_if_missing=True, return_X_y=False) [source]
 
Load the Olivetti faces data-set from AT&T (classification). Download it if necessary.  
Classes 40  
Samples total 400  
Dimensionality 4096  
Features real, between 0 and 1   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
shufflebool, default=False 

If True the order of the dataset is shuffled to avoid having images of the same person grouped.  
random_stateint, RandomState instance or None, default=0 

Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See Glossary.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False 

If True, returns (data, target) instead of a Bunch object. See below for more information about the data and target object.  New in version 0.22.     Returns 
 
dataBunch 

Dictionary-like object, with the following attributes.  data: ndarray, shape (400, 4096)

Each row corresponds to a ravelled face image of original size 64 x 64 pixels.  
imagesndarray, shape (400, 64, 64) 

Each row is a face image corresponding to one of the 40 subjects of the dataset.  
targetndarray, shape (400,) 

Labels associated to each face image. Those labels are ranging from 0-39 and correspond to the Subject IDs.  
DESCRstr 

Description of the modified Olivetti Faces Dataset.    
(data, target)tuple if return_X_y=True 

 New in version 0.22.
sklearn.datasets.fetch_rcv1(*, data_home=None, subset='all', download_if_missing=True, random_state=None, shuffle=False, return_X_y=False) [source]
 
Load the RCV1 multilabel dataset (classification). Download it if necessary. Version: RCV1-v2, vectors, full sets, topics multilabels.  
Classes 103  
Samples total 804414  
Dimensionality 47236  
Features real, between 0 and 1   Read more in the User Guide.  New in version 0.17.   Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
subset{‘train’, ‘test’, ‘all’}, default=’all’ 

Select the dataset to load: ‘train’ for the training set (23149 samples), ‘test’ for the test set (781265 samples), ‘all’ for both, with the training samples first if shuffle is False. This follows the official LYRL2004 chronological split.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See Glossary.  
shufflebool, default=False 

Whether to shuffle dataset.  
return_X_ybool, default=False 

If True, returns (dataset.data, dataset.target) instead of a Bunch object. See below for more information about the dataset.data and dataset.target object.  New in version 0.20.     Returns 
 
datasetBunch 

Dictionary-like object, with the following attributes.  
datasparse matrix of shape (804414, 47236), dtype=np.float64 

The array has 0.16% of non zero values. Will be of CSR format.  
targetsparse matrix of shape (804414, 103), dtype=np.uint8 

Each sample has a value of 1 in its categories, and 0 in others. The array has 3.15% of non zero values. Will be of CSR format.  
sample_idndarray of shape (804414,), dtype=np.uint32, 

Identification number of each sample, as ordered in dataset.data.  
target_namesndarray of shape (103,), dtype=object 

Names of each target (RCV1 topics), as ordered in dataset.target.  
DESCRstr 

Description of the RCV1 dataset.    
(data, target)tuple if return_X_y is True 

 New in version 0.20.
sklearn.datasets.fetch_species_distributions(*, data_home=None, download_if_missing=True) [source]
 
Loader for species distribution dataset from Phillips et. al. (2006) Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.    Returns 
 
dataBunch 

Dictionary-like object, with the following attributes.  
coveragesarray, shape = [14, 1592, 1212] 

These represent the 14 features measured at each point of the map grid. The latitude/longitude values for the grid are discussed below. Missing data is represented by the value -9999.  
trainrecord array, shape = (1624,) 

The training points for the data. Each point has three fields:  train[‘species’] is the species name train[‘dd long’] is the longitude, in degrees train[‘dd lat’] is the latitude, in degrees   
testrecord array, shape = (620,) 

The test points for the data. Same format as the training data.  
Nx, Nyintegers 

The number of longitudes (x) and latitudes (y) in the grid  
x_left_lower_corner, y_left_lower_cornerfloats 

The (x,y) position of the lower-left corner, in degrees  
grid_sizefloat 

The spacing between points of the grid, in degrees       Notes This dataset represents the geographic distribution of species. The dataset is provided by Phillips et. al. (2006). The two species are:  
“Bradypus variegatus” , the Brown-throated Sloth. 
“Microryzomys minutus” , also known as the Forest Small Rice Rat, a rodent that lives in Peru, Colombia, Ecuador, Peru, and Venezuela. For an example of using this dataset with scikit-learn, see examples/applications/plot_species_distribution_modeling.py.  References  
“Maximum entropy modeling of species geographic distributions” S. J. Phillips, R. P. Anderson, R. E. Schapire - Ecological Modelling, 190:231-259, 2006.
sklearn.datasets.get_data_home(data_home=None) → str[source]
 
Return the path of the scikit-learn data dir. This folder is used by some large dataset loaders to avoid downloading the data several times. By default the data dir is set to a folder named ‘scikit_learn_data’ in the user home folder. Alternatively, it can be set by the ‘SCIKIT_LEARN_DATA’ environment variable or programmatically by giving an explicit folder path. The ‘~’ symbol is expanded to the user home folder. If the folder does not already exist, it is automatically created.  Parameters 
 
data_homestr, default=None 

The path to scikit-learn data directory. If None, the default path is ~/sklearn_learn_data.