Vibra-Ingenn/Janus: Janus is a API router for AI fashions written in Go and has a Vulkan Model runner · GitHub

Janus is a single Go binary that runs .gguf fashions in your machine (GPU or CPU) and exposes an OpenAI-compatible API. No Python, no Docker, no Ollama required.

Use it your means: name it from the command line (curl, PowerShell, scripts), wire it into Cursor / Cline / any OpenAI consumer — similar native fashions, no matter workflow suits you.

• Local inference — llama.cpp through Vulkan (AMD / Intel / NVIDIA) or CPU fallback
• OpenAI-compatible API — /v1/chat/completions, /v1/fashions
• Hot-swap fashions — change .gguf with out restarting
• Thinking mannequin help — reasoning break up into reasoning_content
• Chat template auto-detection — makes use of the template from GGUF metadata
• Zero dependencies — one .exe on Windows, no Python, no Docker

Disk: plan for the mannequin dimension (usually 2–8 GB per mannequin) plus ~50 MB for Janus + llama.dll.

git clone https://github.com/Vibra-Ingenn/Janus.git
cd Janus
.construct.ps1

construct.ps1 downloads pre-built llama.cpp Vulkan DLLs and compiles distjanus.exe.

Put a .gguf file within the fashions folder. Use the included downloader:

go construct -o distmodelget.exe .cmdmodelget
.distmodelget.exe -repo meta-llama/Llama-3.2-3B-Instruct -file Llama-3.2-3B-Instruct-Q8_0.gguf -out .fashions

Or obtain any GGUF from Hugging Face.

Edit .env:

INFERENCE_BACKEND=vulkan
JANUS_MODEL_PATH=./fashions/Llama-3.2-3B-Instruct-Q8_0.gguf
JANUS_MAX_TOKENS=4096

Opens http://127.0.0.1:8990 in your browser.

curl http://127.0.0.1:8990/well being

git clone https://github.com/Vibra-Ingenn/Janus.git
cd Janus
go mod tidy
go construct -o dist/janus ./cmd/janus
cp .env.instance .env
# edit .env — set JANUS_MODEL_PATH and INFERENCE_BACKEND=cpu if no Vulkan
./dist/janus

On Linux you want libllama.so subsequent to the binary or on LD_LIBRARY_PATH.

curl http://127.0.0.1:8990/v1/chat/completions 
  -H "Content-Type: software/json" 
  -d '{
    "mannequin": "native",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Base URL: http://127.0.0.1:8990/v1

curl http://127.0.0.1:8990/v1/chat/completions 
  -H "Content-Type: software/json" 
  -d '{"mannequin":"native","stream":true,"messages":[{"role":"user","content":"Tell me a joke"}]}'

Base URL:  http://127.0.0.1:8990/v1
API Key:   (go away clean)

cmd/janus/          Main server (OpenAI-compatible API)
cmd/modelget/       Hugging Face mannequin downloader
inside/engine/    llama.cpp Vulkan/CPU backend
inside/bridge/    DLL loader and FFI bindings
inside/singleton/ Single-instance guard
fashions/             Put .gguf information right here (not dedicated)
dist/               janus.exe + llama.dll after construct

Get-Process -Name "janus" -ErrorAction SilentlyContinue | Stop-Process -Force
go check ./...
go construct -o distjanus.exe .cmdjanus
.distjanus.exe

Or simply .run.ps1. For a full construct together with llama DLLs, use .construct.ps1.

Contributions welcome — see CONTRIBUTING.md.

MIT — see LICENSE.