Package: fastvs
FastVSFastVS (Fast Vector Search) is a Python library designed for exact vector search in dataframes or tables. It provides functionality to work with both PyArrow Tables and Pandas DataFrames, allowing users to perform nearest neighbor searches using various distance metrics. It is most optimized for PyArrow Tables, as it uses the Rust Arrow library under the hood for zero-copy and vectorized computation.InstallationFastVS can be installed using pip:pipinstallfastvsFunctionsFastVS offers the following main functions:search_arrowSearches a PyArrow Table for the k nearest neighbors of a query point.Parameterstable(pyarrow.Table): The table to search.column_name(str): This column should be a list or np array type column, where each element is a vector of floats.query_point(list or np array): The query point.k(int): The number of nearest neighbors to return.metric(str): The metric to use for the search (e.g., "euclidean", "manhattan", "cosine_similarity", "inner_product").ReturnsTuple[List[int], List[float]]: The indices and distances of the k nearest neighbors.Usageimportpyarrowaspafromfastvsimportsearch_arrowindices,distances=search_arrow(your_pyarrow_table,"your_column",[1.0,2.0],5,"cosine_similarity")search_pandasSearches a Pandas DataFrame for the k nearest neighbors of a query point. This function usessearch_tableunder the hood. Note that this function is slower thansearch_arrowdue to the copying of data from the DataFrame to the Arrow table format.Parametersdf(pandas.DataFrame): The DataFrame to search.column_name(str): The column name to search. This column should be a list or np array type column, where each element is a vector of floats.query_point(list or np array): The query point.k(int): The number of nearest neighbors to return.metric(str): The metric to use for the search.ReturnsTuple[List[int], List[float]]: The indices and distances of the k nearest neighbors.Usageimportpandasaspdfromfastvsimportsearch_pandasdf=pd.read_csv("your_dataset.csv")indices,distances=search_pandas(df,"your_column",[1.0,2.0],5,"cosine_similarity")apply_distance_arrowApplies a distance function to a PyArrow table and returns an array of distances.Parameterstable(pyarrow.Table): The table to search.column_name(str): The column name to search. This column should be a list or np array type column, where each element is a vector of floats.query_point(list or np array): The query point.metric(str): The metric to use for the search.Returnspyarrow.Array: The distances in the order of the table.Usageimportpyarrowaspafromfastvsimportapply_distance_arrowtable=pa.Table.from_pandas(your_dataframe)distances=apply_distance_arrow(table,"your_column",[1.0,2.0],"euclidean")apply_distance_pandasApplies a distance function to a Pandas DataFrame and returns a Series of distances. Usesapply_distance_arrowunder the hood.Parametersdf(pandas.DataFrame): The DataFrame to search.column_name(str): The column name to search. This column should be a list or np array type column, where each element is a vector of floats.query_point(list or np array): The query point.metric(str): The metric to use for the search.Returnspandas.Series: The distances as a pandas Series.Usageimportpandasaspdfromfastvsimportapply_distance_pandasdf=pd.read_csv("your_dataset.csv")distances=apply_distance_pandas(df,"your_column",[1.0,2.0],"euclidean")Supported MetricsFastVS supports various distance metrics, including:Euclidean ("euclidean")Manhattan ("manhattan")Inner Product ("inner_product")Cosine Similarity ("cosine_similarity")Euclidean DistanceThe Euclidean distance between two points $P$ and $Q$ in $N$-dimensional space, with $P = (p_1, p_2, ..., p_N)$ and $Q = (q_1, q_2, ..., q_N)$, is defined as:d(P, Q) = \sqrt{\sum_{i=1}^{N} (p_i - q_i)^2}Manhattan DistanceThe Manhattan distance (also known as L1 norm) between two points $P$ and $Q$ in $N$-dimensional space is the sum of the absolute differences of their Cartesian coordinates:d(P, Q) = \sum_{i=1}^{N} |p_i - q_i|Cosine SimilarityCosine similarity measures the cosine of the angle between two vectors$P$and$Q$in an$N$-dimensional space. It is defined as:\text{Cosine Similarity}(P, Q) = \frac{P \cdot Q}{\|P\| \|Q\|}where$P \cdot Q$is the dot product of vectors $P$ and $Q$, and $|P|$ and $|Q|$ are the magnitudes (Euclidean norms) of vectors $P$ and $Q$, respectively.Inner ProductThe inner product (or dot product) between two vectors$P$and$Q$in an$N$-dimensional space is defined as the sum of the products of their corresponding components:\text{Inner Product}(P, Q) = P \cdot Q = \sum_{i=1}^{N} p_i q_iContributionContributions to FastVS are welcome! Please submit your pull requests to the repository or open an issue for any bugs or feature requests.To Dos:Clean up rust codeSupport f32LicenseFastVS is released under the MIT License. See the LICENSE file in the repository for more details.