Phase 1: Resource Framework Design
Ash-Inspired Resource Management for ElixirML/DSPEx
Executive Summary
The Resource Framework provides the enterprise-grade foundation for ElixirML/DSPEx, treating programs, optimizations, and configurations as first-class Ash resources. This system enables declarative resource modeling, automatic API generation, real-time subscriptions, and policy-based access control while maintaining seamless integration with the Schema Engine and Variable System.
Design Philosophy
Core Principles
- Resources as First-Class Citizens: Programs, variables, optimizations are all resources
- Declarative Configuration: Define behavior through resource attributes and relationships
- Automatic API Generation: GraphQL, REST, and LiveView APIs generated automatically
- Enterprise Features: Built-in authentication, authorization, and audit trails
- Real-Time Intelligence: Live subscriptions to optimization progress and results
Strategic Advantages
- Enterprise Ready: Production-grade features out of the box
- Developer Productivity: Automatic API generation and admin interfaces
- Scalability: Built on Ash’s proven architecture patterns
- Flexibility: Extensible resource definitions with custom actions
- Integration: Native support for Schema validation and Variable optimization
Architecture Overview
Resource Framework Architecture
├── Core Resource System
│ ├── Program Resources
│ ├── Variable Space Resources
│ ├── Optimization Resources
│ └── Configuration Resources
├── Enterprise Features
│ ├── Authentication & Authorization
│ ├── Policy Engine
│ ├── Audit Logging
│ └── Multi-Tenancy Support
├── API Generation
│ ├── GraphQL API
│ ├── REST API
│ ├── LiveView Admin
│ └── Real-Time Subscriptions
├── Integration Layer
│ ├── Schema Engine Integration
│ ├── Variable System Integration
│ ├── Process Orchestrator Integration
│ └── External System Connectors
└── Developer Experience
├── Resource Definition DSL
├── Admin Dashboard
├── Query Builder
└── Performance Analytics
Core Resource Definitions
1. Program Resource
defmodule ElixirML.Resources.Program do
use Ash.Resource,
domain: ElixirML.Domain,
data_layer: AshPostgres.DataLayer
use ElixirML.Schema.ResourceIntegration
postgres do
table "programs"
repo ElixirML.Repo
references do
reference :variable_space, on_delete: :nilify
end
end
attributes do
uuid_primary_key :id
attribute :name, :string do
allow_nil? false
constraints max_length: 255
end
attribute :description, :string do
constraints max_length: 1000
end
attribute :type, :atom do
allow_nil? false
constraints one_of: [:predict, :chain_of_thought, :react, :custom]
end
schema_attribute :signature_config, ElixirML.Schemas.Program.SignatureConfig do
allow_nil? false
end
schema_attribute :program_config, ElixirML.Schemas.Program.ProgramConfig do
default %{}
end
attribute :status, :atom do
default :draft
constraints one_of: [:draft, :active, :optimizing, :archived]
end
schema_attribute :performance_metrics, ElixirML.Schemas.Program.PerformanceMetrics do
default %{}
end
attribute :version, :integer do
default 1
constraints min: 1
end
timestamps()
end
relationships do
belongs_to :variable_space, ElixirML.Resources.VariableSpace do
allow_nil? true
end
belongs_to :parent_program, ElixirML.Resources.Program do
allow_nil? true
end
has_many :child_programs, ElixirML.Resources.Program do
destination_attribute :parent_program_id
end
has_many :optimization_runs, ElixirML.Resources.OptimizationRun do
destination_attribute :program_id
end
has_many :executions, ElixirML.Resources.Execution do
destination_attribute :program_id
end
has_many :evaluations, ElixirML.Resources.Evaluation do
destination_attribute :program_id
end
many_to_many :tags, ElixirML.Resources.Tag do
through ElixirML.Resources.ProgramTag
source_attribute_on_join_resource :program_id
destination_attribute_on_join_resource :tag_id
end
end
actions do
defaults [:create, :read, :update, :destroy]
create :create_with_variable_space do
accept [:name, :description, :type, :signature_config, :program_config]
argument :variable_definitions, {:array, :map} do
allow_nil? false
end
change fn changeset, _context ->
variable_definitions = Ash.Changeset.get_argument(changeset, :variable_definitions)
# Create variable space
{:ok, variable_space} = ElixirML.Resources.VariableSpace.create(%{
name: "#{Ash.Changeset.get_attribute(changeset, :name)} Variables",
variable_definitions: variable_definitions
})
Ash.Changeset.change_attribute(changeset, :variable_space_id, variable_space.id)
end
end
action :execute, ElixirML.Resources.Execution do
argument :inputs, :map do
allow_nil? false
end
argument :variable_configuration, :map do
default %{}
end
argument :execution_options, :map do
default %{}
end
run ElixirML.Actions.ExecuteProgram
end
action :optimize, ElixirML.Resources.OptimizationRun do
argument :training_data, {:array, :map} do
allow_nil? false
end
argument :optimization_strategy, :atom do
default :simba
constraints one_of: [:simba, :bayesian, :grid_search, :random]
end
argument :optimization_config, :map do
default %{}
end
run ElixirML.Actions.OptimizeProgram
end
action :clone, ElixirML.Resources.Program do
argument :new_name, :string do
allow_nil? false
end
run fn input, context ->
program = input.resource
{:ok, cloned} = ElixirML.Resources.Program.create(%{
name: input.arguments.new_name,
description: "Clone of #{program.name}",
type: program.type,
signature_config: program.signature_config,
program_config: program.program_config,
variable_space_id: program.variable_space_id,
parent_program_id: program.id
})
{:ok, cloned}
end
end
update :update_performance_metrics do
accept []
argument :metrics, :map do
allow_nil? false
end
change fn changeset, _context ->
new_metrics = Ash.Changeset.get_argument(changeset, :metrics)
current_metrics = Ash.Changeset.get_attribute(changeset, :performance_metrics) || %{}
updated_metrics = Map.merge(current_metrics, new_metrics)
Ash.Changeset.change_attribute(changeset, :performance_metrics, updated_metrics)
end
end
destroy :archive do
change fn changeset, _context ->
Ash.Changeset.change_attribute(changeset, :status, :archived)
end
soft? true
end
end
calculations do
calculate :current_performance_score, :float, ElixirML.Calculations.CurrentPerformanceScore
calculate :optimization_count, :integer do
ElixirML.Calculations.OptimizationCount
end
calculate :average_execution_time, :float do
ElixirML.Calculations.AverageExecutionTime
end
calculate :cost_efficiency_ratio, :float do
ElixirML.Calculations.CostEfficiencyRatio
end
calculate :variable_importance_scores, {:array, :map} do
ElixirML.Calculations.VariableImportanceScores
end
end
aggregates do
count :execution_count, :executions
avg :avg_performance, :evaluations, :score
max :best_performance, :evaluations, :score
sum :total_cost, :executions, :cost
end
validations do
validate compare(:version, greater_than: 0), message: "Version must be positive"
validate fn changeset, _context ->
case Ash.Changeset.get_attribute(changeset, :type) do
:custom ->
# Custom programs require additional validation
signature_config = Ash.Changeset.get_attribute(changeset, :signature_config)
if Map.has_key?(signature_config, :custom_implementation) do
:ok
else
{:error, field: :signature_config, message: "Custom programs require custom_implementation"}
end
_ -> :ok
end
end
end
policies do
# Default policy - users can read their own programs
policy action_type(:read) do
authorize_if actor_attribute_equals(:id, :user_id)
end
# Creation policy - authenticated users can create programs
policy action_type(:create) do
authorize_if actor_present()
end
# Update policy - only program owners can update
policy action_type(:update) do
authorize_if actor_attribute_equals(:id, :user_id)
end
# Admin policy - admins can do anything
policy always() do
authorize_if actor_attribute_equals(:role, :admin)
end
end
pub_sub do
module ElixirML.PubSub
prefix "program"
publish_all :create, ["created"]
publish :update, ["updated", :id]
publish :destroy, ["deleted", :id]
# Custom publications for ML-specific events
publish :execute, ["executed", :id]
publish :optimize, ["optimization_started", :id]
end
end
2. Variable Space Resource
defmodule ElixirML.Resources.VariableSpace do
use Ash.Resource,
domain: ElixirML.Domain,
data_layer: AshPostgres.DataLayer
use ElixirML.Schema.ResourceIntegration
postgres do
table "variable_spaces"
repo ElixirML.Repo
end
attributes do
uuid_primary_key :id
attribute :name, :string do
allow_nil? false
constraints max_length: 255
end
attribute :description, :string do
constraints max_length: 1000
end
schema_attribute :variable_definitions, ElixirML.Schemas.Variable.VariableDefinitions do
allow_nil? false
end
schema_attribute :constraints, ElixirML.Schemas.Variable.Constraints do
default %{dependencies: [], validations: []}
end
schema_attribute :optimization_hints, ElixirML.Schemas.Variable.OptimizationHints do
default %{}
end
attribute :discrete_space_size, :integer do
default 1
constraints min: 1
end
attribute :continuous_dimensions, :integer do
default 0
constraints min: 0
end
timestamps()
end
relationships do
has_many :programs, ElixirML.Resources.Program do
destination_attribute :variable_space_id
end
has_many :optimization_runs, ElixirML.Resources.OptimizationRun do
destination_attribute :variable_space_id
end
has_many :configurations, ElixirML.Resources.VariableConfiguration do
destination_attribute :variable_space_id
end
end
actions do
defaults [:create, :read, :update, :destroy]
create :create_from_definitions do
accept [:name, :description]
argument :variable_definitions, {:array, :map} do
allow_nil? false
end
change fn changeset, _context ->
definitions = Ash.Changeset.get_argument(changeset, :variable_definitions)
# Validate and process variable definitions
processed_definitions = process_variable_definitions(definitions)
# Calculate space metrics
discrete_size = calculate_discrete_space_size(processed_definitions)
continuous_dims = calculate_continuous_dimensions(processed_definitions)
changeset
|> Ash.Changeset.change_attribute(:variable_definitions, processed_definitions)
|> Ash.Changeset.change_attribute(:discrete_space_size, discrete_size)
|> Ash.Changeset.change_attribute(:continuous_dimensions, continuous_dims)
end
end
action :generate_configuration, ElixirML.Resources.VariableConfiguration do
argument :strategy, :atom do
default :random
constraints one_of: [:random, :grid_point, :latin_hypercube, :sobol]
end
argument :seed, :integer do
allow_nil? true
end
run ElixirML.Actions.GenerateVariableConfiguration
end
action :validate_configuration, :boolean do
argument :configuration, :map do
allow_nil? false
end
run fn input, _context ->
variable_space = input.resource
configuration = input.arguments.configuration
case ElixirML.Variable.Space.validate_configuration(
convert_to_variable_space(variable_space),
configuration
) do
{:ok, _} -> {:ok, true}
{:error, _} -> {:ok, false}
end
end
end
action :analyze_variable_importance, {:array, :map} do
argument :optimization_runs, {:array, :uuid} do
allow_nil? false
end
run ElixirML.Actions.AnalyzeVariableImportance
end
update :add_variable do
accept []
argument :variable_definition, :map do
allow_nil? false
end
change fn changeset, _context ->
new_var = Ash.Changeset.get_argument(changeset, :variable_definition)
current_definitions = Ash.Changeset.get_attribute(changeset, :variable_definitions)
updated_definitions = [new_var | current_definitions]
# Recalculate space metrics
discrete_size = calculate_discrete_space_size(updated_definitions)
continuous_dims = calculate_continuous_dimensions(updated_definitions)
changeset
|> Ash.Changeset.change_attribute(:variable_definitions, updated_definitions)
|> Ash.Changeset.change_attribute(:discrete_space_size, discrete_size)
|> Ash.Changeset.change_attribute(:continuous_dimensions, continuous_dims)
end
end
end
calculations do
calculate :variable_count, :integer do
ElixirML.Calculations.VariableCount
end
calculate :complexity_score, :float do
ElixirML.Calculations.ComplexityScore
end
calculate :optimization_difficulty, :atom do
ElixirML.Calculations.OptimizationDifficulty
end
end
aggregates do
count :program_count, :programs
count :optimization_count, :optimization_runs
count :configuration_count, :configurations
end
validations do
validate fn changeset, _context ->
definitions = Ash.Changeset.get_attribute(changeset, :variable_definitions)
# Validate unique variable names
names = Enum.map(definitions, & &1["name"])
if length(names) == length(Enum.uniq(names)) do
:ok
else
{:error, field: :variable_definitions, message: "Variable names must be unique"}
end
end
end
defp process_variable_definitions(definitions) do
# Process and validate variable definitions
definitions
|> Enum.map(&normalize_variable_definition/1)
|> Enum.map(&validate_variable_definition/1)
end
defp normalize_variable_definition(definition) do
# Normalize variable definition format
definition
|> Map.put_new("metadata", %{})
|> Map.put_new("constraints", %{})
end
defp validate_variable_definition(definition) do
# Validate individual variable definition
required_fields = ["name", "type"]
case Enum.all?(required_fields, &Map.has_key?(definition, &1)) do
true -> definition
false -> raise "Variable definition missing required fields: #{inspect(required_fields)}"
end
end
defp calculate_discrete_space_size(definitions) do
definitions
|> Enum.map(&discrete_variable_size/1)
|> Enum.reduce(1, &*/2)
end
defp calculate_continuous_dimensions(definitions) do
definitions
|> Enum.count(fn def -> def["type"] in ["float", "integer"] end)
end
defp discrete_variable_size(%{"type" => "choice", "constraints" => %{"choices" => choices}}) do
length(choices)
end
defp discrete_variable_size(%{"type" => "module", "constraints" => %{"modules" => modules}}) do
length(modules)
end
defp discrete_variable_size(_), do: 1
defp convert_to_variable_space(resource) do
# Convert Ash resource to ElixirML.Variable.Space
%ElixirML.Variable.Space{
id: resource.id,
name: resource.name,
variables: convert_variable_definitions(resource.variable_definitions),
constraints: resource.constraints["validations"] || [],
dependencies: resource.constraints["dependencies"] || %{},
metadata: %{
discrete_space_size: resource.discrete_space_size,
continuous_dimensions: resource.continuous_dimensions
}
}
end
defp convert_variable_definitions(definitions) do
definitions
|> Enum.map(&convert_variable_definition/1)
|> Map.new(fn var -> {var.name, var} end)
end
defp convert_variable_definition(definition) do
%ElixirML.Variable{
name: String.to_atom(definition["name"]),
type: String.to_atom(definition["type"]),
default: definition["default"],
constraints: definition["constraints"],
description: definition["description"],
metadata: definition["metadata"]
}
end
end
3. Optimization Run Resource
defmodule ElixirML.Resources.OptimizationRun do
use Ash.Resource,
domain: ElixirML.Domain,
data_layer: AshPostgres.DataLayer
use ElixirML.Schema.ResourceIntegration
postgres do
table "optimization_runs"
repo ElixirML.Repo
references do
reference :program, on_delete: :delete
reference :variable_space, on_delete: :delete
end
end
attributes do
uuid_primary_key :id
attribute :name, :string do
allow_nil? false
constraints max_length: 255
end
attribute :strategy, :atom do
allow_nil? false
constraints one_of: [:simba, :bayesian, :grid_search, :random, :evolutionary]
end
attribute :status, :atom do
default :pending
constraints one_of: [:pending, :running, :completed, :failed, :cancelled]
end
schema_attribute :configuration, ElixirML.Schemas.Optimization.Configuration do
allow_nil? false
end
schema_attribute :training_data, ElixirML.Schemas.Optimization.TrainingData do
allow_nil? false
end
schema_attribute :results, ElixirML.Schemas.Optimization.Results do
default %{}
end
schema_attribute :pareto_frontier, ElixirML.Schemas.Optimization.ParetoFrontier do
default %{points: [], objectives: []}
end
attribute :iterations_completed, :integer do
default 0
constraints min: 0
end
attribute :max_iterations, :integer do
default 100
constraints min: 1
end
attribute :best_score, :float do
constraints min: 0.0, max: 1.0
end
attribute :convergence_score, :float do
constraints min: 0.0, max: 1.0
end
attribute :started_at, :utc_datetime
attribute :completed_at, :utc_datetime
timestamps()
end
relationships do
belongs_to :program, ElixirML.Resources.Program do
allow_nil? false
end
belongs_to :variable_space, ElixirML.Resources.VariableSpace do
allow_nil? false
end
has_many :evaluations, ElixirML.Resources.Evaluation do
destination_attribute :optimization_run_id
end
has_many :configurations_tested, ElixirML.Resources.VariableConfiguration do
destination_attribute :optimization_run_id
end
end
actions do
defaults [:create, :read, :update, :destroy]
create :start_optimization do
accept [:name, :strategy, :configuration, :training_data, :max_iterations]
argument :program_id, :uuid do
allow_nil? false
end
change fn changeset, _context ->
program_id = Ash.Changeset.get_argument(changeset, :program_id)
# Load program and variable space
program = ElixirML.Resources.Program.get!(program_id, load: [:variable_space])
changeset
|> Ash.Changeset.change_attribute(:program_id, program.id)
|> Ash.Changeset.change_attribute(:variable_space_id, program.variable_space.id)
|> Ash.Changeset.change_attribute(:status, :pending)
end
after_action fn changeset, optimization_run, _context ->
# Start the optimization process asynchronously
Task.Supervisor.start_child(ElixirML.TaskSupervisor, fn ->
ElixirML.Optimization.Runner.execute(optimization_run)
end)
{:ok, optimization_run}
end
end
update :update_progress do
accept [:iterations_completed, :best_score, :convergence_score]
argument :new_evaluation, :map do
allow_nil? true
end
argument :pareto_point, :map do
allow_nil? true
end
change fn changeset, _context ->
# Update pareto frontier if new point provided
if pareto_point = Ash.Changeset.get_argument(changeset, :pareto_point) do
current_frontier = Ash.Changeset.get_attribute(changeset, :pareto_frontier)
updated_frontier = add_to_pareto_frontier(current_frontier, pareto_point)
Ash.Changeset.change_attribute(changeset, :pareto_frontier, updated_frontier)
else
changeset
end
end
end
update :complete do
accept []
argument :final_results, :map do
allow_nil? false
end
change fn changeset, _context ->
final_results = Ash.Changeset.get_argument(changeset, :final_results)
changeset
|> Ash.Changeset.change_attribute(:status, :completed)
|> Ash.Changeset.change_attribute(:completed_at, DateTime.utc_now())
|> Ash.Changeset.change_attribute(:results, final_results)
end
end
update :fail do
accept []
argument :error_message, :string do
allow_nil? false
end
change fn changeset, _context ->
error_message = Ash.Changeset.get_argument(changeset, :error_message)
current_results = Ash.Changeset.get_attribute(changeset, :results) || %{}
error_results = Map.put(current_results, :error, error_message)
changeset
|> Ash.Changeset.change_attribute(:status, :failed)
|> Ash.Changeset.change_attribute(:completed_at, DateTime.utc_now())
|> Ash.Changeset.change_attribute(:results, error_results)
end
end
action :cancel do
change fn changeset, _context ->
Ash.Changeset.change_attribute(changeset, :status, :cancelled)
end
end
action :get_best_configuration, :map do
run fn input, _context ->
optimization_run = input.resource
case optimization_run.results do
%{"best_configuration" => config} -> {:ok, config}
_ -> {:error, "No best configuration available"}
end
end
end
end
calculations do
calculate :duration_seconds, :integer do
ElixirML.Calculations.OptimizationDuration
end
calculate :progress_percentage, :float do
ElixirML.Calculations.OptimizationProgress
end
calculate :convergence_status, :atom do
ElixirML.Calculations.ConvergenceStatus
end
calculate :estimated_completion, :utc_datetime do
ElixirML.Calculations.EstimatedCompletion
end
end
aggregates do
count :evaluation_count, :evaluations
avg :average_score, :evaluations, :score
max :best_evaluation_score, :evaluations, :score
end
validations do
validate compare(:iterations_completed, less_than_or_equal_to: :max_iterations),
message: "Completed iterations cannot exceed maximum"
validate fn changeset, _context ->
started_at = Ash.Changeset.get_attribute(changeset, :started_at)
completed_at = Ash.Changeset.get_attribute(changeset, :completed_at)
case {started_at, completed_at} do
{nil, nil} -> :ok
{start, nil} when not is_nil(start) -> :ok
{start, finish} when not is_nil(start) and not is_nil(finish) ->
if DateTime.compare(start, finish) in [:lt, :eq] do
:ok
else
{:error, field: :completed_at, message: "Completion time must be after start time"}
end
_ -> {:error, field: :started_at, message: "Start time required if completion time is set"}
end
end
end
pub_sub do
module ElixirML.PubSub
prefix "optimization"
publish_all :create, ["started"]
publish :update, ["progress", :id]
publish :complete, ["completed", :id]
publish :fail, ["failed", :id]
end
defp add_to_pareto_frontier(current_frontier, new_point) do
current_points = current_frontier["points"] || []
# Simple Pareto frontier update (in real implementation would be more sophisticated)
updated_points = [new_point | current_points]
|> Enum.uniq_by(fn point -> point["configuration"] end)
|> Enum.take(100) # Limit frontier size
%{current_frontier | "points" => updated_points}
end
end
Enterprise Features
1. Authentication & Authorization
defmodule ElixirML.Resources.User do
use Ash.Resource,
domain: ElixirML.Domain,
data_layer: AshPostgres.DataLayer
postgres do
table "users"
repo ElixirML.Repo
end
attributes do
uuid_primary_key :id
attribute :email, :string do
allow_nil? false
constraints format: ~r/^[^\s]+@[^\s]+\.[^\s]+$/
end
attribute :name, :string do
allow_nil? false
constraints max_length: 255
end
attribute :role, :atom do
default :user
constraints one_of: [:user, :admin, :viewer]
end
attribute :organization, :string do
constraints max_length: 255
end
attribute :preferences, :map do
default %{}
end
attribute :api_key_hash, :string do
sensitive? true
end
attribute :last_login_at, :utc_datetime
attribute :active, :boolean, default: true
timestamps()
end
relationships do
has_many :programs, ElixirML.Resources.Program do
destination_attribute :user_id
end
has_many :optimization_runs, ElixirML.Resources.OptimizationRun do
destination_attribute :user_id
end
end
actions do
defaults [:create, :read, :update]
create :register do
accept [:email, :name, :organization]
change fn changeset, _context ->
# Generate API key
api_key = generate_api_key()
api_key_hash = hash_api_key(api_key)
changeset
|> Ash.Changeset.change_attribute(:api_key_hash, api_key_hash)
|> Ash.Changeset.after_action(fn _changeset, user ->
# Return user with API key (only time it's visible)
{:ok, Map.put(user, :api_key, api_key)}
end)
end
end
action :authenticate do
argument :email, :string, allow_nil?: false
argument :api_key, :string, allow_nil?: false
run fn input, _context ->
user = ElixirML.Resources.User.get_by_email!(input.arguments.email)
if verify_api_key(input.arguments.api_key, user.api_key_hash) do
# Update last login
ElixirML.Resources.User.update!(user, %{last_login_at: DateTime.utc_now()})
{:ok, user}
else
{:error, "Invalid credentials"}
end
end
end
update :regenerate_api_key do
change fn changeset, _context ->
api_key = generate_api_key()
api_key_hash = hash_api_key(api_key)
changeset
|> Ash.Changeset.change_attribute(:api_key_hash, api_key_hash)
|> Ash.Changeset.after_action(fn _changeset, user ->
{:ok, Map.put(user, :api_key, api_key)}
end)
end
end
end
validations do
validate unique(:email), message: "Email already exists"
end
defp generate_api_key do
:crypto.strong_rand_bytes(32) |> Base.encode64()
end
defp hash_api_key(api_key) do
:crypto.hash(:sha256, api_key) |> Base.encode64()
end
defp verify_api_key(api_key, hash) do
hash_api_key(api_key) == hash
end
end
2. Policy Engine
defmodule ElixirML.Policies.ProgramPolicy do
use Ash.Policy.SimpleCheck
def describe(_), do: "Program access policy"
def match?(actor, resource, _context) do
cond do
# Admins can access everything
actor.role == :admin -> true
# Users can access their own programs
resource.user_id == actor.id -> true
# Users can read public programs in their organization
resource.visibility == :public and resource.organization == actor.organization -> true
# Viewers can only read
actor.role == :viewer -> false
# Default deny
true -> false
end
end
end
defmodule ElixirML.Policies.OptimizationPolicy do
use Ash.Policy.SimpleCheck
def describe(_), do: "Optimization access policy"
def match?(actor, resource, _context) do
# Load the associated program
program = ElixirML.Resources.Program.get!(resource.program_id)
# Apply program policy
ElixirML.Policies.ProgramPolicy.match?(actor, program, _context)
end
end
3. Audit Logging
defmodule ElixirML.Resources.AuditLog do
use Ash.Resource,
domain: ElixirML.Domain,
data_layer: AshPostgres.DataLayer
postgres do
table "audit_logs"
repo ElixirML.Repo
end
attributes do
uuid_primary_key :id
attribute :action, :string do
allow_nil? false
end
attribute :resource_type, :string do
allow_nil? false
end
attribute :resource_id, :uuid
attribute :actor_id, :uuid
attribute :actor_type, :string, default: "User"
attribute :changes, :map do
default %{}
end
attribute :metadata, :map do
default %{}
end
attribute :ip_address, :string
attribute :user_agent, :string
timestamps(updated_at: false)
end
actions do
defaults [:create, :read]
create :log_action do
accept [:action, :resource_type, :resource_id, :actor_id, :changes, :metadata, :ip_address, :user_agent]
end
end
# No updates or deletes - audit logs are immutable
end
# Audit logging hook
defmodule ElixirML.AuditHook do
use Ash.Notifier
def notify(%Ash.Notifier.Notification{} = notification) do
# Log all resource changes
ElixirML.Resources.AuditLog.create!(%{
action: to_string(notification.action.name),
resource_type: notification.resource |> Module.split() |> List.last(),
resource_id: notification.data.id,
actor_id: notification.actor && notification.actor.id,
changes: extract_changes(notification),
metadata: %{
domain: notification.domain,
timestamp: DateTime.utc_now()
}
})
end
defp extract_changes(notification) do
case notification.changeset do
nil -> %{}
changeset -> Ash.Changeset.get_changes(changeset)
end
end
end
API Generation
1. GraphQL API
defmodule ElixirMLWeb.Schema do
use Absinthe.Schema
use AshGraphql, domains: [ElixirML.Domain]
query do
# Automatically generated queries for all resources
end
mutation do
# Automatically generated mutations
end
subscription do
field :optimization_progress, :optimization_run do
arg :id, non_null(:id)
config fn args, %{context: %{current_user: user}} ->
# Verify user has access to this optimization
optimization = ElixirML.Resources.OptimizationRun.get!(args.id)
if can_access_optimization?(user, optimization) do
{:ok, topic: "optimization:progress:#{args.id}"}
else
{:error, "Access denied"}
end
end
trigger :update_optimization_progress, topic: fn optimization ->
"optimization:progress:#{optimization.id}"
end
end
field :program_execution, :execution do
arg :program_id, non_null(:id)
config fn args, %{context: %{current_user: user}} ->
program = ElixirML.Resources.Program.get!(args.program_id)
if can_access_program?(user, program) do
{:ok, topic: "program:execution:#{args.program_id}"}
else
{:error, "Access denied"}
end
end
trigger :program_executed, topic: fn execution ->
"program:execution:#{execution.program_id}"
end
end
field :pareto_frontier_update, :pareto_point do
arg :optimization_id, non_null(:id)
config fn args, %{context: %{current_user: user}} ->
optimization = ElixirML.Resources.OptimizationRun.get!(args.optimization_id)
if can_access_optimization?(user, optimization) do
{:ok, topic: "pareto:#{args.optimization_id}"}
else
{:error, "Access denied"}
end
end
trigger :pareto_frontier_updated, topic: fn %{optimization_id: id} ->
"pareto:#{id}"
end
end
end
defp can_access_optimization?(user, optimization) do
program = ElixirML.Resources.Program.get!(optimization.program_id)
can_access_program?(user, program)
end
defp can_access_program?(user, program) do
user.role == :admin or program.user_id == user.id
end
end
2. LiveView Admin Dashboard
defmodule ElixirMLWeb.AdminLive do
use ElixirMLWeb, :live_view
use AshAdmin.LiveView
admin_table ElixirML.Resources.Program do
column :name
column :type
column :status
column :current_performance_score
column :optimization_count
column :inserted_at
action :optimize, "Optimize Program"
action :clone, "Clone Program"
action :archive, "Archive"
end
admin_table ElixirML.Resources.OptimizationRun do
column :name
column :strategy
column :status
column :progress_percentage
column :best_score
column :started_at
action :cancel, "Cancel Optimization"
end
admin_table ElixirML.Resources.VariableSpace do
column :name
column :variable_count
column :discrete_space_size
column :continuous_dimensions
column :program_count
action :analyze_importance, "Analyze Variable Importance"
end
def mount(_params, _session, socket) do
# Subscribe to real-time updates
ElixirML.PubSub.subscribe("optimization:*")
ElixirML.PubSub.subscribe("program:*")
{:ok, socket}
end
def handle_info({:optimization_progress, optimization}, socket) do
# Update optimization progress in real-time
send_update(OptimizationProgressComponent,
id: "optimization-#{optimization.id}",
optimization: optimization
)
{:noreply, socket}
end
def handle_info({:program_executed, execution}, socket) do
# Update program execution status
send_update(ProgramStatusComponent,
id: "program-#{execution.program_id}",
execution: execution
)
{:noreply, socket}
end
end
Implementation Strategy
Week 1: Core Resource Foundation
- Program, VariableSpace, and OptimizationRun resources
- Basic CRUD actions and relationships
- Schema Engine integration
- Database migrations
Week 2: Enterprise Features
- Authentication and authorization
- Policy engine implementation
- Audit logging system
- Multi-tenancy support
Week 3: API Generation & Real-Time
- GraphQL API with subscriptions
- REST API endpoints
- LiveView admin dashboard
- Real-time progress updates
Week 4: Advanced Features & Integration
- Variable System integration
- Process Orchestrator integration
- Performance analytics
- Documentation and examples
Success Metrics
Functional Requirements
- Complete CRUD operations for all resources
- Real-time subscriptions for optimization progress
- Policy-based access control
- Automatic API generation
Performance Requirements
- <100ms response time for resource queries
- Support for 10,000+ concurrent subscriptions
- Efficient pagination for large datasets
- Optimized database queries
Enterprise Requirements
- Role-based access control
- Complete audit trail
- Multi-tenant isolation
- API rate limiting and monitoring
This Resource Framework design provides enterprise-grade resource management for ElixirML/DSPEx, enabling declarative resource modeling with automatic API generation, real-time capabilities, and comprehensive enterprise features.