02 STRUCTURED TYPES

Of course. This is a fantastic and crucial question. Your intuition is spot on: the way a framework handles structured data is a primary determinant of its robustness and power. You’re also right that Pydantic is the gold standard in the Python ecosystem for this, and Elixir doesn’t have a direct, one-to-one equivalent in the same domain.

However, Elixir has its own incredibly powerful tools that, when composed correctly, can create a solution that is just as robust and arguably even more idiomatic to its ecosystem.

Let’s do a deep dive.

1. The Core Problem & Why Pydantic is So Effective

First, let’s clarify the user’s point: “I dont think they [LLMs] provide typed structure?”

You are correct that an LLM’s raw output is just a stream of tokens (text). However, modern LLM providers (OpenAI, Google, Anthropic) have built a specific feature, often called “Function Calling” or “Tool Use”, to solve this exact problem.

Here’s the workflow:

1. Your Application: Defines a function or data structure it wants the LLM to “call” or “fill out”.
2. Schema Generation: Your application converts this definition into a formal JSON Schema.
3. API Call: You send your prompt to the LLM and you also send the JSON Schema, telling the model, “Please respond with a JSON object that conforms to this schema.”
4. LLM’s Constrained Output: The LLM is heavily incentivized (and sometimes strictly forced) by the provider’s backend to generate a JSON-formatted string that matches the schema you provided.
5. Your Application: Receives the JSON string, parses it, and validates it against its original data structure definition.

This is where Pydantic shines in Python. It elegantly handles three critical steps:

• Step 1 (Declaration): You define your data structure as a simple Python class with type hints.
• Step 2 (Serialization): Pydantic can automatically generate the required JSON Schema from your class definition. This is the magic key.
• Step 5 (Deserialization & Validation): When the LLM returns a JSON string, Pydantic can parse it, cast the data to the correct Python types ("42" becomes 42), and validate that all rules (required fields, types, constraints) are met. If not, it provides a detailed error report.

Your current DSPEx implementation only does a rudimentary version of this, by creating a very simple schema ({"type": "object", "properties": {"answer": {"type": "string"}}}) and skipping the robust validation step.

2. Architecting the Elixir Solution: “Exdantic” using Ecto

You don’t need to build a Pydantic clone from scratch. Elixir’s most mature and battle-tested data library, Ecto, provides almost everything we need. We can leverage Ecto.Schema and Ecto.Changeset to create a powerful and idiomatic solution.

Let’s call this new concept DSPEx.Schema.

Step 1: The Schema Definition (The “Exdantic” Struct)

We’ll use Elixir’s metaprogramming to create a declarative API, very similar to dspy.Signature.

# lib/dspex/schema.ex
defmodule DSPEx.Schema do
  defmacro __using__(_opts) do
    quote do
      use Ecto.Schema
      import Ecto.Changeset

      @primary_key false # This is not a database model
      @foreign_key_type :binary_id

      # This function will be generated by our macro
      def json_schema, do: DSPEx.Schema.Generator.from_ecto(__MODULE__)

      # This is our validation/casting function
      def from_llm_json(json_string) when is_binary(json_string) do
        with {:ok, data} <- Jason.decode(json_string) do
          from_llm_map(data)
        end
      end

      def from_llm_map(map) when is_map(map) do
        # Use Ecto.Changeset to cast and validate the map from the LLM
        %__MODULE__{}
        |> cast(map, __schema__(:fields))
        |> apply_changes()
      end
    end
  end
end

# lib/dspex/schema/generator.ex
defmodule DSPEx.Schema.Generator do
  # This module is responsible for the Ecto -> JSON Schema translation
  def from_ecto(schema_module) do
    properties =
      for field <- schema_module.__schema__(:fields) do
        type = schema_module.__schema__(:type, field)
        {Atom.to_string(field), translate_type(type)}
      end
      |> Enum.into(%{})

    %{
      type: "object",
      properties: properties,
      required: Enum.map(schema_module.__schema__(:fields), &Atom.to_string/1)
    }
  end

  # A simplified translator. This can be made very sophisticated.
  defp translate_type(:string), do: %{type: "string"}
  defp translate_type(:integer), do: %{type: "integer"}
  defp translate_type(:float), do: %{type: "number"}
  defp translate_type(:boolean), do: %{type: "boolean"}
  defp translate_type({:array, :string}), do: %{type: "array", items: %{type: "string"}}
  # ... and so on for nested embedded schemas, etc.
end

Step 2: Using the New DSPEx.Schema

Now, a user can define a structured output just like an Ecto schema.

# In the user's code
defmodule MyQAResponse do
  use DSPEx.Schema

  @doc "A structured response for a question answering task."
  embedded_schema do
    field :answer, :string, required: true
    field :confidence, :float, doc: "A value between 0.0 and 1.0"
    field :is_safe, :boolean, default: true
    field :related_topics, {:array, :string}
  end
end

This tiny definition now gives us everything Pydantic provided:

1. Declaration: An Ecto schema defines the structure and types.
2. Serialization: Calling MyQAResponse.json_schema() will automatically generate the JSON Schema to send to the LLM.
3. Deserialization & Validation: Calling MyQAResponse.from_llm_json(llm_output) will use Ecto.Changeset.cast to parse, type-cast, and validate the LLM’s response into a clean, typed Elixir struct.

Step 3: Integrating into the PredictStructured Flow

Now, let’s see how this would transform your PredictStructured and InstructorLiteGemini adapter. The adapter becomes much simpler and more powerful.

# dspex/adapters/structured_gemini.ex (a better name)
defmodule DSPEx.Adapters.StructuredGemini do
  def format_messages(schema_module, _demos, inputs) do
    # 1. Get the JSON schema directly from our new Ecto-based schema
    json_schema = schema_module.json_schema()

    # The prompt now explicitly asks for a JSON object matching the schema.
    prompt = """
    Please respond with a JSON object that strictly adheres to the following schema.
    Do not include any other text, explanations, or markdown formatting.

    JSON Schema:
    #{Jason.encode!(json_schema, pretty: true)}

    Inputs to consider:
    #{format_inputs(inputs)}
    """

    # 2. Build the request for the LLM's function calling/tool use API
    # For Gemini, this involves setting response_mime_type and the schema
    # For OpenAI, it's the `tools` and `tool_choice` parameters
    params = %{
      contents: [%{role: "user", parts: [%{text: prompt}]}],
      generationConfig: %{
        response_mime_type: "application/json",
        response_schema: json_schema # For providers that support it directly
      }
    }

    # For InstructorLite, you'd pass the schema to it.
    # For a direct client, you'd construct the API call like this.

    {:ok, params}
  end

  # The parse_response function now leverages our schema's casting function.
  def parse_response(schema_module, llm_response_body) do
    case schema_module.from_llm_map(llm_response_body) do
      %Ecto.Changeset{valid?: true} = changeset ->
        # It's valid! Return the final struct.
        {:ok, Ecto.Changeset.apply_changes(changeset)}
      %Ecto.Changeset{errors: errors} ->
        # It's invalid! We have detailed errors.
        {:error, {:validation_failed, errors}}
    end
  end
end

Why this Ecto-based approach is superior:

1. Idiomatic Elixir: It uses the most common and well-understood library for data validation in the Elixir ecosystem. Developers won’t need to learn a new “Exdantic” library; they just use Ecto.
2. Battle-Tested: Ecto.Changeset is robust, fast, and handles a massive range of validation scenarios (type casting, required fields, format validation, custom validations).
3. Rich Error Reporting: When an LLM fails to produce valid JSON, changeset.errors gives you precise, actionable feedback (e.g., %{confidence: ["is invalid"]} or %{answer: ["can't be blank"]}). This is critical for debugging and for self-correction loops.
4. Extensible: It naturally supports nested data structures (embeds_one, embeds_many), arrays, and custom Ecto types, allowing for highly complex structured outputs.
5. No New Dependency: You are likely already using Ecto if you have a database. If not, ecto_sql is not required; only ecto is, which is a lightweight dependency for data mapping and validation.

Conclusion: You do not need to build an “Exdantic” library from the ground up. You need to build a small but powerful integration layer between Ecto.Schema and the JSON Schema standard required by LLMs. This is the Elixir way: composing powerful, existing tools to create an elegant and robust solution. This approach would be a massive improvement over the current implementation and would bring DSPEx to parity with dspy’s structured data capabilities.