RETRIEVE PLANNING 01

An investigation into the Retrieve module of the dspy framework and its potential equivalent in the dspex library reveals a fundamental difference in the current scope and features of the two implementations. While dspy has a robust, first-class retrieval system, dspex currently does not have a direct, built-in equivalent.

Executive Summary

The dspy framework includes a dedicated dspy.Retrieve module and a rich ecosystem of connectors to various vector stores and search engines, making it a powerful tool for building Retrieval-Augmented Generation (RAG) pipelines. This module acts as a core component, allowing programs to dynamically fetch external context to inform language model predictions.

In contrast, the dspex library, based on the provided codebase, does not currently have an equivalent to the dspy.Retrieve module. Its primary focus is on the core prompting and optimization layers of the DSPy paradigm: Signatures, Programs (like Predict), and Teleprompters (SIMBA, BEACON). While RAG patterns can be manually implemented in dspex, the framework lacks the built-in, standardized abstraction for retrieval that dspy provides.

1. Investigation of dspy.Retrieve

The dspy library’s retrieval functionality is a cornerstone of its architecture, designed to seamlessly integrate external knowledge sources into the prompting process.

1.1. Core Component: dspy.Retrieve

• File: dspy/retrieve/retrieve.py
• Purpose: The Retrieve class is a dspy.Module that acts as a placeholder within a program for a retrieval step. When called, it doesn’t contain the retrieval logic itself but instead delegates the task to a globally configured “retrieval model” (RM).
• Mechanism: Its forward method takes a query and an integer k (for the number of documents to retrieve). It then calls dspy.settings.rm(query, k=k), which invokes the actual retrieval backend. This design decouples the program’s logic from the specific retrieval technology being used.

1.2. Retrieval Model (RM) Ecosystem

• Files: dspy/retrieve/*.py
• Purpose: The retrieve directory in dspy is populated with a wide array of concrete RM implementations, each acting as a connector to a specific vector database, search engine, or retrieval library.
• Examples of Implementations:
  • PineconeRM: Connects to Pinecone.
  • ChromaDBRM: Connects to ChromaDB.
  • WeaviateRM: Connects to Weaviate.
  • AzureAISearchRM: Connects to Azure AI Search.
  • QdrantRM: Connects to Qdrant.
  • FaissRM: Uses a local FAISS index.
  • YouRM: Connects to the You.com search API.
  • …and many others.

1.3. Role in the Framework

The dspy.Retrieve module is fundamental to building RAG applications. A typical dspy program, like a ChainOfThought module, can first use a Retrieve step to fetch relevant context from a corpus and then pass that context to an LLM to generate a grounded answer.

# Conceptual dspy RAG program
class RAG(dspy.Module):
  def __init__(self, num_passages=3):
    super().__init__()
    # The program explicitly includes a retrieval step
    self.retrieve = dspy.Retrieve(k=num_passages)
    self.generate_answer = dspy.ChainOfThought("context, question -> answer")

  def forward(self, question):
    context = self.retrieve(question).passages
    prediction = self.generate_answer(context=context, question=question)
    return dspy.Prediction(context=context, answer=prediction.answer)

2. Investigation of dspex for an Equivalent

A thorough review of the dspex codebase shows no direct counterpart to dspy’s retrieval system.

2.1. Codebase Analysis

• Directory Structure: The dspex project structure (adapters/, config/, services/, signature/, teleprompter/) lacks a retrieve/ or retriever/ directory.
• Core Modules:
  • dspex/program.ex and dspex/predict.ex: These modules manage the execution flow of a program, focusing on taking inputs, formatting them for an LLM via an adapter, and parsing the response. They do not contain any logic for querying external data sources.
  • dspex/teleprompter/: This directory contains the BEACON and SIMBA optimizers. Their role is to generate and select optimal prompts (instructions and few-shot demos), not to retrieve documents from a corpus.
  • dspex/client.ex: This module is responsible for the final HTTP request to the LLM provider.
• Conclusion: There are no built-in modules, behaviors, or connectors for vector databases or search engines. The functionality is absent from the current framework.

2.2. Potential Workaround in dspex

While dspex doesn’t offer a built-in Retrieve module, a developer could manually implement a RAG pattern. This would involve:

1. Creating a Custom Retriever Program: A developer would need to create their own Elixir module that implements the DSPEx.Program behaviour.
2. Implementing forward/3: Inside the forward function of this custom program, the developer would write the code to connect to their chosen vector database (e.g., using Req for a REST API or a dedicated Elixir client library if one exists).
3. Composing Programs: This custom retriever program would then be used in sequence with a DSPEx.Predict program. The output of the retriever (the context) would be fed as an input to the predictor.

Conceptual dspex RAG Implementation (Manual):

# 1. Developer-defined retriever program
defmodule MyApp.Retriever do
  use DSPEx.Program

  @impl DSPEx.Program
  def forward(_program, %{query: query}, _opts) do
    # Manual logic to call Pinecone, Weaviate, etc.
    # and get context passages.
    context = query_my_vector_db(query)
    {:ok, %{context: context}}
  end
end

# 2. Composition in application logic
question = "What is DSPEx?"

# Step 1: Manually call the custom retriever
{:ok, %{context: context}} = DSPEx.Program.forward(MyApp.Retriever, %{query: question})

# Step 2: Define a signature that accepts the retrieved context
defmodule MyRAGSignature do
  use DSPEx.Signature, "context, question -> answer"
end

# Step 3: Call a standard Predict program with the context
predictor = DSPEx.Predict.new(MyRAGSignature, :gemini)
DSPEx.Program.forward(predictor, %{context: context, question: question})

This demonstrates that while possible, implementing RAG in dspex is a manual process that lacks the abstraction, standardization, and convenience of dspy’s Retrieve module.

3. Mapping and Final Conclusion

The Retrieve module is a significant component of the dspy framework that is currently absent in dspex. The Elixir-based dspex has focused on faithfully re-implementing the core optimization and prompting mechanisms like Predict, Teleprompter, and Signature. The retrieval functionality, being a large and distinct part of the ecosystem, has not yet been ported or may be on a future roadmap. Users of dspex who require RAG capabilities must build the retrieval logic themselves.