Vector databases are designed to handle the unique structure of vector embeddings, which are dense vectors of numbers that represent text. They are used in machine learning to index vectors for easy search and retrieval by comparing values and finding those that are most similar to one another.