Preview v0.3.0

Vllm

vLLM for Elixir via SnakeBridge - Easy, fast, and cheap LLM serving for everyone. High-throughput LLM inference with PagedAttention, continuous batching, and OpenAI-compatible API.

Stats & Info

  • Latest release: v0.3.0
  • Recent downloads: 132
  • All-time downloads: 132
  • Maintainers: nshkrdotcom

Tags

MIT Hex.pm

Installation

Add vllm to your list of dependencies in mix.exs:

def deps do
  [
    {:vllm, "~> 0.3.0"}
  ]
end

Then run:

mix deps.get

Resources

View Documentation View on Hex.pm View on GitHub