LlamaLink

API Gateway en Go para ejecutar modelos de lenguaje (LLMs) de forma local usando llama.cpp, con autenticación por API keys, rate limiting, cuotas de uso y panel de administración Vue 3.

Características

• API OpenAI-compatible: POST /v1/chat/completions con streaming SSE
• Multi-model hot-swap: Configura muchos modelos, activa uno a la vez sin downtime
• Auth por API keys: Keys con scopes (chat, models, usage, admin), hashing bcrypt
• Rate limiting: Token bucket por API key (configurable, backend memory o Redis)
• Cuotas mensuales: Tracking por modelo o global, webhooks al 90% y 100%
• Admin SPA: Vue 3 + Tailwind CSS con Dashboard, Keys, Models, Usage
• Deployment: Docker, docker-compose, systemd, nginx/Caddy
• WebSocket: Live updates de estado del modelo y stats

Requisitos

• Go 1.23+
• Node.js 20+ (para build del frontend)
• llama-server (GGML/GGUF)
• CUDA 12.x (opcional, para GPU)

Quick Start

1. Build

# Dependencias Go
go mod download

# Build binario
make build

# O directamente
go build -o llamalink ./cmd/llamalink

2. Configuración

cp .env.example .env
# Editar .env - mínimo: ADMIN_TOKEN

3. Arrancar

# Con llama-server externo (ya corriendo en 127.0.0.1:8080)
./llamalink

# O gestionar llama-server internamente
MANAGE_LLAMA_SERVER=true ./llamalink

4. Probar

# Health check
curl http://localhost:8000/health

# Login admin (primera key se crea automáticamente)
ADMIN_TOKEN=tu-token curl http://localhost:8000/api/v1/admin/login \
  -X POST -H "Content-Type: application/json" \
  -d '{"admin_token":"tu-token"}'

# Crear API key
curl http://localhost:8000/v1/keys \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"test-key"}'

# Chat completion
curl http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer $TU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tu-modelo","messages":[{"role":"user","content":"Hello!"}]}'

Variables de Entorno

Variable	Descripción	Default
LLAMALINK_ENV	development o production	development
LLAMALINK_PORT	Puerto HTTP	8000
DATABASE_URL	Connection string SQLite/Postgres	sqlite:///./llamalink.db
MANAGE_LLAMA_SERVER	¿Gestionar llama-server internamente?	true
LLAMA_SERVER_BIN	Path a llama-server	/usr/local/bin/llama-server
LLAMA_SERVER_HOST	Host de llama-server	127.0.0.1
LLAMA_SERVER_PORT	Puerto de llama-server	8080
LLAMA_SERVER_STARTUP_TIMEOUT	Timeout startup (seg)	120
RATE_LIMIT_PER_MINUTE	Requests/min por key	60
ADMIN_TOKEN	Token admin inicial	changeme
LOG_LEVEL	debug, info, warn, error	info

Docker

# Build imagen
make docker/build

# O docker-compose completo (API + llama-runner + nginx)
docker compose -f deploy/docker-compose.yml up -d

# Production con tu modelo
ADMIN_TOKEN=mi-token-secreto docker compose -f deploy/docker-compose.yml up -d

Install

Binary releases (all platforms)

Download pre-built binaries from GitHub Releases:

# Linux/macOS/Windows — pick your arch
curl -L https://github.com/llamalink/llamalink/releases/latest/download/llamalink-linux-amd64.tar.gz | tar -xz
./llamalink

.deb package (Debian/Ubuntu)

# Download latest .deb
curl -L https://github.com/llamalink/llamalink/releases/latest/download/llamalink_VERSION_amd64.deb \
  -o llamalink.deb

# Install
sudo dpkg -i llamalink.deb

# Configure
sudo nano /etc/llamalink/env
sudo systemctl enable llamalink
sudo systemctl start llamalink

Build from source

git clone https://github.com/llamalink/llamalink.git
cd llamalink
make deps build

# Or with .deb package
make deb
sudo dpkg -i dist/llamalink_*.deb

Admin Panel

Accede a http://localhost:8000/admin/ para el panel Vue 3:

• Dashboard: Stats, estado del modelo, tabla de modelos con hot-swap
• API Keys: Crear, listar, revocar keys
• Models: CRUD de modelos, load/unload
• Usage: Charts de uso, logs, quota

API Endpoints

OpenAI-compatible

• POST /v1/chat/completions — Chat con streaming
• GET /v1/models — Lista modelos
• GET /v1/models/active — Modelo activo

Gestión

• GET|POST /v1/keys — Listar/crear keys (admin)
• DELETE /v1/keys/{id} — Revocar key (admin)
• POST /v1/models — Crear modelo (admin)
• POST /v1/models/{name}/load — Hot-swap (admin)
• GET /v1/usage — Uso de tu key
• GET /v1/usage/{key_id} — Uso específico (admin)

Admin API (JSON)

• POST /api/v1/admin/login — Login admin
• GET /api/v1/admin/dashboard — Stats aggregated
• GET /api/v1/admin/keys — Keys con detalles
• POST /api/v1/admin/keys — Crear key
• DELETE /api/v1/admin/keys/{id} — Revocar

Makefile

make build           # Build binario
make run             # Build y ejecutar
make dev             # go run (desarrollo)
make test            # Tests
make lint            # golangci-lint
make clean           # Limpiar build
make migrate/up      # Correr migrations
make frontend/build  # Build Vue SPA
make deb             # Build .deb package (Linux, requires dpkg-deb)
make release         # Build .deb + local release artifacts
make docker/build    # Build imagen Docker

Arquitectura

llamalink/
├── cmd/llamalink/    # Entry point
├── internal/
│   ├── api/          # Gin router, handlers, middleware
│   ├── auth/          # API key service, bcrypt
│   ├── db/           # GORM models, migrations
│   ├── llama/        # Proxy, subprocess manager
│   └── quota/        # Quota check, webhooks
├── web/frontend/     # Vue 3 SPA (build → web/dist/)
├── deploy/           # Docker, systemd, nginx
└── migrations/       # SQL migrations

Deployment

Systemd

sudo cp deploy/llamalink.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable llamalink
sudo systemctl start llamalink

###nginx

sudo cp deploy/nginx.conf /etc/nginx/sites-available/llamalink
sudo ln -s /etc/nginx/sites-available/llamalink /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

License

MIT