Using Redis as Vector Database: Basics and Quick Tutorial

Can You Use Redis as a Vector Database?

Redis, traditionally known as a high-performance in-memory data store for caching and real-time analytics, has evolved to support vector data. This allows Redis to store, index, and search high-dimensional vectors used in AI and machine learning tasks. Redis’ ability to handle vector data enhances its role in applications like semantic search, recommendation systems, and natural language processing, which rely on similarity search and ranking.

Key benefits of using Redis for vector storage include:

• High performance: Redis delivers fast data retrieval, making it ideal for real-time AI applications.
• Efficient vector search: Supports similarity search for high-dimensional vectors, ensuring fast and accurate results.
• Integration: Native support for vector indexing and search simplifies AI workloads.
• Low latency: Ideal for applications requiring low-latency, high-speed data access, such as recommendation engines.

In this article:

• What is RediSearch?
• Key Features of Redis as a Vector Database
• pgvector vs. Redis for Vector Databases
• Quick Tutorial: Step-by-Step for Redis as a Vector Database

What is RediSearch?

RediSearch is a search and indexing engine built on top of Redis, designed to provide secondary indexing, full-text search, filtering, and vector similarity search. RediSearch extends Redis with features like tokenization, text indexing, and querying, making it suitable for handling both structured and unstructured data in real time.

In the context of vector data, RediSearch adds significant value by providing vector indexing and similarity search. This enables Redis to perform high-speed, high-dimensional vector searches efficiently. With RediSearch, developers can combine traditional search capabilities (like keyword searches) with advanced vector search (such as nearest neighbor search), allowing for more powerful and flexible data retrieval strategies.

RediSearch It simplifies the development process by offering a unified interface for managing both traditional and vector data in Redis. However, it should be noted that RediSearch mainly works with hash and JSON values and is less useful for other data types within Redis.

Key Features of Redis as a Vector Database

High-Performance Vector Search

Redis delivers high-performance vector search by leveraging its in-memory data storage, enabling extremely fast lookups and similarity comparisons. With the addition of module RediSearch, Redis can support vector indexing structures such as HNSW (hierarchical navigable small world), allowing for quick retrieval of vectors most similar to a target query. This speed is crucial for applications that require instant recommendations or real-time filtering of large datasets, such as e-commerce search engines and chatbots.

In terms of scalability, Redis can handle large-scale, high-dimensional vector datasets across distributed environments if configured and deployed properly. Since search and indexing are accomplished in-memory, Redis achieves millisecond-scale query responses in many scenarios, making it a strong fit for workloads where performance is a priority.

Also note that the distributed cluster version of RediSearch, which scales to a large amount of documents across hundreds of servers, is currently available through Redis Cloud and Redis Enterprise Software, not as part of Redis Open Source.

Flexible Data Modeling

Redis’s flexible data modeling allows you to structure vector data alongside other types within the same database. Unlike some specialized vector databases that store only vectors, Redis enables a unified schema where vectors can be combined with other values. For example, you can use a hash to store a series of blogs’ metadata, then create an index that uses the blog’s category as a tag and its description as a vector. This approach simplifies associating metadata with vector records, which is crucial for performing filtered or hybrid searches. Ultimately, this flexibility reduces the complexity of building full-featured AI applications, as you can store, filter, and retrieve both vector and non-vector data without complex joins or external systems.

Integrated Vector Indexing

Redis incorporates integrated vector indexing directly into its module ecosystem, particularly with RediSearch. This allows developers to build vector indices for high-dimensional data using algorithms that balance speed and accuracy, such as HNSW or FLAT indexes. Efficient indexing enhances application reliability, facilitating rapid similarity searches and supporting advanced query capabilities like filtering on non-vector fields or combining multiple filtering criteria. Redis’s index management features ensure that scaling to millions of vectors does not degrade query performance, making it suitable for production-grade workloads demanding consistent speed at scale.

Integration with AI Workflows

Redis’s compatibility with popular AI frameworks and languages simplifies its integration into machine learning pipelines. Libraries and clients exist for Python, Java, Node.js, and more, along with specialized connectors for AI platforms like LangChain and LlamaIndex. This ensures that vector data produced or consumed in model training, inference, and data preprocessing can be ingested, stored, and queried in Redis without heavy custom development.

Redis also supports streaming and event-driven architectures, potentially making it a unified platform to keep the search indexes updated in time as well. These features expedite deploying AI solutions such as personalization engines or semantic search APIs, as teams can rely on Redis’s proven scalability and consistency to manage both the input vectors and their derived outputs.

Durability and Persistence Options

Redis offers multiple durability configurations to ensure vector data reliability. These include RDB snapshots, which periodically persist the in-memory state to disk, and append-only files (AOF), which log every write operation on disk after it is executed in memory. This resilience is critical in production, as it allows vector databases to recover from outages with minimal loss of accuracy and coverage, which is a mandatory feature for meeting compliance and audit requirements.

Native high availability (HA) solutions like Redis Sentinel further enhance durability, enabling deployments where vector data is automatically mirrored across nodes. Coupled with failover and backup management capabilities, Redis provides flexible persistence strategies that support both performance and reliability requirements. This makes it possible to tune the system either for maximum throughput or for strong durability guarantees, depending on your application’s needs.

Vector Sets

Redis recently introduced vector sets, a new data type designed for efficient vector similarity search in AI applications. Inspired by Redis’s sorted sets, vector sets allow the storage and querying of high-dimensional vector embeddings, which are essential in various AI and machine learning tasks. A vector set associates string elements with vectors, enabling the storage and retrieval of vectors based on their similarity to a given query vector.

Vector sets are tailored for scenarios where you need to store and retrieve vector embeddings (such as text descriptions, images, or any other AI-related data) while performing fast and efficient similarity searches. A key benefit of vector sets is their simple API, which aligns with Redis’s philosophy of delivering high-performance solutions with minimal complexity.

pgvector vs. Redis for Vector Databases

pgvector is an open-source vector database solution that integrates with PostgreSQL and is a common alternative to Redis for vector processing. When choosing between Redis and pgvector for vector databases, the key differences lie in performance, scalability, and compatibility, as discussed below.

Redis is an in-memory data store that offers ultra-low latency and high throughput, making it ideal for real-time applications requiring fast vector searches, such as recommendation engines and fraud detection services. It supports advanced search algorithms like HNSW, which ensure quick retrieval of similar vectors. Redis is horizontally scalable if deployed and configured properly, capable of handling large datasets across distributed environments, but this comes at a higher cost due to its reliance on memory. In the meantime, as mentioned earlier, the distributed cluster version of RediSearch is currently available in Redis Cloud and Redis Enterprise Software, not as a part of the open-source offering.

pgvector is a PostgreSQL extension, making it a great choice for applications already using this relational database. While it may not match Redis’s raw speed, pgvector is still very performant and excels at managing both vector and relational data in a unified schema, allowing for complex queries that combine the two. It is also more cost-effective than Redis, as it leverages disk storage rather than in-memory operations, making it suitable for handling large datasets without the overhead of high-memory usage.

For real-time AI applications, Redis can be a good choice, but for hybrid workloads involving vector and relational data, pgvector provides a more cost-effective and seamless solution.

Quick Tutorial: Step-by-Step for Redis as a Vector Database

In this tutorial, you’ll learn how to use Redis as a vector database by following a simple workflow. This guide will walk you through the process of creating a vector database in Redis, generating vector embeddings, storing and indexing data, and performing vector searches. 

We will be using the redis-py client library, and these instructions are adapted from the Redis documentation.

Step 1: Install Prerequisites

Before you begin, you need to install the necessary dependencies. Start by setting up a Python virtual environment and installing the required packages.

$> python -m venv redis-venv
$> source redis-venv/bin/activate 
$> pip install redis pandas sentence-transformers tabulate

These libraries are required for interacting with Redis, manipulating data, generating text embeddings, and rendering results.

Step 2: Connect to Redis

The next step is to establish a connection to your Redis server. If you’re running Redis locally, the default connection details should work.

import redis

client = redis.Redis(host="localhost", port=6379, decode_responses=True)

Step 3: Prepare Your Dataset

For this tutorial, we’ll use a sample dataset including various attributes, including a description field that will be used to generate vector embeddings.

import json

# Read the demo data from file.
file_name = "boeing.json"

with open(file_name, 'r') as f:
    planes = json.load(f)

# Inspect a single document.
print(planes[0])

The dataset contains plane descriptions that will be turned into vector embeddings for similarity search.

Step 4: Generate Text Embeddings

To perform vector search, we need to generate embeddings for the descriptions using a pre-trained model from the sentence-transformers library. In this case, we’ll use the msmarco-distilbert-base-v4 model.

from sentence_transformers import SentenceTransformer
import numpy as np

# Initialize the model
embedder = SentenceTransformer("msmarco-distilbert-base-v4")

# Extract descriptions from the dataset
descriptions = [plane['description'] for plane in planes]

# Generate embeddings
embeddings = embedder.encode(descriptions).astype(np.float32).tolist()

# Check the dimension of embeddings (usually the output of models is 768-dimensional)
VECTOR_DIMENSION = len(embeddings[0])

Step 5: Store Data in Redis

Now, store the data and its corresponding embeddings in Redis. We will use the RedisJSON module to store both the metadata (plane attributes) and the embeddings.

pipeline = client.pipeline()
for i, plane in enumerate(planes, start=1):
    redis_key = f"planes:{i:03}"
    # Store the plane data as JSON
    plane_data = {
        **plane,
        "description_embeddings": embeddings[i-1]  # Adding the vector embedding
    }
    pipeline.json().set(redis_key, "$", plane_data)

pipeline.execute()
print("pipeline executed")

Step 6: Create an Index

For efficient querying, you must create an index on the vector field. Redis uses the RedisSearch module to handle indexing and querying.

from redis.commands.search.field import TextField, NumericField, VectorField
from redis.commands.search.indexDefinition import IndexDefinition, IndexType

# Create an index for the planes data.
schema = [
    TextField("model"),
    TextField("brand"),
    NumericField("price"),
    TextField("type"),
    TextField("engine_type"),
    NumericField("number_of_engines"),
    NumericField("thrust_per_engine"),
    TextField("description"),
    VectorField(
        "$.description_embeddings",
        "FLAT",
        {
            "TYPE": "FLOAT32",
            "DIM": VECTOR_DIMENSION,
            "DISTANCE_METRIC": "COSINE",
        },
        as_name="vector",
    ),
]

definition = IndexDefinition(prefix=["planes:"], index_type=IndexType.JSON)
res = client.ft("idx:planes_vss").create_index(fields=schema, definition=definition)

This index is designed to search through plane descriptions using the vector embeddings, as well as other fields.

Step 7: Perform a Vector Search

Once the index is created, you can perform a vector search to find planes similar to a query:

queries = [
    "Boeing twin propeller plan",
    "Small Jet",
]

encoded_queries = embedder.encode(queries)
len(encoded_queries)

query = (
    Query('(*)=>[KNN 3 @vector $query_vector AS vector_score]')
    .sort_by('vector_score')
    .return_fields('vector_score', 'id', 'brand', 'model', 'description')
    .dialect(2)
)

output = client.ft('idx:planes_vss').search(
    query,
    {
        'query_vector': np.array(encoded_query, dtype=np.float32).tobytes()
    }
).docs
print(output)

This query will return the top 3 airplanes most similar to the input query based on their vector embeddings.

Step 8: Visualize Search Results

You can further refine your search results and convert them into a more readable format, such as a pandas table.

import pandas as pd

# Prepare the result data.
results_list = [{
    'id': doc.id,
    'description': doc.description,
    'score': round(1 - float(doc.vector_score), 2)
} for doc in output]

# Convert to a pandas dataframe.
results_df = pd.DataFrame(results_list)
print(results_df)

Dragonfly Cloud: The Ultimate Redis Alternative for Vector Database

Dragonfly is a modern, source-available, multi-threaded, Redis-compatible in-memory data store that stands out by delivering unmatched performance and efficiency. Designed from the ground up to disrupt legacy technologies, Dragonfly redefines what an in-memory data store can achieve.

As mentioned above, Redis supports additional functionalities via modules, such as RedisJSON for JSON storage and manipulation and RediSearch for full-text, keyword, and vector indexing and querying. Dragonfly, on the other hand, has these advanced functionalities built-in without additional module management. When you run Dragonfly (on-prem, in the cloud, or via our cloud offering), everything is included by default.

Dragonfly Cloud is a fully managed service from the creators of Dragonfly, handling all operations and delivering effortless scaling so you can focus on what matters without worrying about in-memory data infrastructure anymore. By leveraging Dragonfly’s architecture, users often experience much higher throughput and more than 30% total cost of ownership.

Dragonfly Scales Both Vertically and Horizontally

Dragonfly’s architecture allows a single instance to fully utilize a modern multi-core server, handling up to millions of requests per second (RPS) and 1TB of in-memory data. This high vertical scalability often eliminates the need for clustering—unlike Redis, which typically requires a cluster even on a powerful single server (premature horizontal scaling). As a result, Dragonfly significantly reduces operational overhead while delivering superior performance.

For workloads that exceed even these limits, Dragonfly offers a horizontal scaling solution: Dragonfly Swarm. Swarm seamlessly extends Dragonfly’s capabilities to handle 100 million+ RPS and 100 TB+ of memory capacity, providing a path for massive growth.