Files

T

darroyo 4c9ed3c24b Initial commit: LlamaLink Go rewrite

Complete rewrite from Python/FastAPI to Go/Gin:
- Go backend: auth (API keys + bcrypt), llama.cpp subprocess manager,
  hot-swap multi-model, rate limiting, quota system, webhooks
- Vue 3 SPA admin panel (src/) with Tailwind CSS
- Deployment: Docker multi-stage, docker-compose, nginx, systemd
- GORM/SQLite models: ApiKey, Model, UsageLog, Quota, Webhook
- REST API: /api/v1/admin/* (keys, models, chat, usage, health)
- Embedded frontend via go:embed (build output at web/dist/)

Removed legacy Python artifacts (app/, tests/, pyproject.toml, etc.)

2026-07-30 10:58:55 -04:00

5.3 KiB

Raw Blame History

LlamaLink

API Gateway en Go para ejecutar modelos de lenguaje (LLMs) de forma local usando llama.cpp, con autenticación por API keys, rate limiting, cuotas de uso y panel de administración Vue 3.

Características

API OpenAI-compatible: POST /v1/chat/completions con streaming SSE
Multi-model hot-swap: Configura muchos modelos, activa uno a la vez sin downtime
Auth por API keys: Keys con scopes (chat, models, usage, admin), hashing bcrypt
Rate limiting: Token bucket por API key (configurable, backend memory o Redis)
Cuotas mensuales: Tracking por modelo o global, webhooks al 90% y 100%
Admin SPA: Vue 3 + Tailwind CSS con Dashboard, Keys, Models, Usage
Deployment: Docker, docker-compose, systemd, nginx/Caddy
WebSocket: Live updates de estado del modelo y stats

Requisitos

Go 1.23+
Node.js 20+ (para build del frontend)
llama-server (GGML/GGUF)
CUDA 12.x (opcional, para GPU)

Quick Start

1. Build

# Dependencias Go
go mod download

# Build binario
make build

# O directamente
go build -o llamalink ./cmd/llamalink

2. Configuración

cp .env.example .env
# Editar .env - mínimo: ADMIN_TOKEN

3. Arrancar

# Con llama-server externo (ya corriendo en 127.0.0.1:8080)
./llamalink

# O gestionar llama-server internamente
MANAGE_LLAMA_SERVER=true ./llamalink

4. Probar

# Health check
curl http://localhost:8000/health

# Login admin (primera key se crea automáticamente)
ADMIN_TOKEN=tu-token curl http://localhost:8000/api/v1/admin/login \
  -X POST -H "Content-Type: application/json" \
  -d '{"admin_token":"tu-token"}'

# Crear API key
curl http://localhost:8000/v1/keys \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"test-key"}'

# Chat completion
curl http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer $TU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tu-modelo","messages":[{"role":"user","content":"Hello!"}]}'

Variables de Entorno

Variable	Descripción	Default
`LLAMALINK_ENV`	`development` o `production`	`development`
`LLAMALINK_PORT`	Puerto HTTP	`8000`
`DATABASE_URL`	Connection string SQLite/Postgres	`sqlite:///./llamalink.db`
`MANAGE_LLAMA_SERVER`	¿Gestionar llama-server internamente?	`true`
`LLAMA_SERVER_BIN`	Path a `llama-server`	`/usr/local/bin/llama-server`
`LLAMA_SERVER_HOST`	Host de llama-server	`127.0.0.1`
`LLAMA_SERVER_PORT`	Puerto de llama-server	`8080`
`LLAMA_SERVER_STARTUP_TIMEOUT`	Timeout startup (seg)	`120`
`RATE_LIMIT_PER_MINUTE`	Requests/min por key	`60`
`ADMIN_TOKEN`	Token admin inicial	`changeme`
`LOG_LEVEL`	`debug`, `info`, `warn`, `error`	`info`

Docker

# Build imagen
make docker/build

# O docker-compose completo (API + llama-runner + nginx)
docker compose -f deploy/docker-compose.yml up -d

# Production con tu modelo
ADMIN_TOKEN=mi-token-secreto docker compose -f deploy/docker-compose.yml up -d

Admin Panel

Accede a http://localhost:8000/admin/ para el panel Vue 3:

Dashboard: Stats, estado del modelo, tabla de modelos con hot-swap
API Keys: Crear, listar, revocar keys
Models: CRUD de modelos, load/unload
Usage: Charts de uso, logs, quota

API Endpoints

OpenAI-compatible

POST /v1/chat/completions — Chat con streaming
GET /v1/models — Lista modelos
GET /v1/models/active — Modelo activo

Gestión

GET|POST /v1/keys — Listar/crear keys (admin)
DELETE /v1/keys/{id} — Revocar key (admin)
POST /v1/models — Crear modelo (admin)
POST /v1/models/{name}/load — Hot-swap (admin)
GET /v1/usage — Uso de tu key
GET /v1/usage/{key_id} — Uso específico (admin)

Admin API (JSON)

POST /api/v1/admin/login — Login admin
GET /api/v1/admin/dashboard — Stats aggregated
GET /api/v1/admin/keys — Keys con detalles
POST /api/v1/admin/keys — Crear key
DELETE /api/v1/admin/keys/{id} — Revocar

Makefile

make build       # Build binario
make run         # Build y ejecutar
make dev         # go run (desarrollo)
make test        # Tests
make lint        # golangci-lint
make clean       # Limpiar build
make migrate/up  # Correr migrations
make frontend/build  # Build Vue SPA

Arquitectura

llamalink/
├── cmd/llamalink/    # Entry point
├── internal/
│   ├── api/          # Gin router, handlers, middleware
│   ├── auth/          # API key service, bcrypt
│   ├── db/           # GORM models, migrations
│   ├── llama/        # Proxy, subprocess manager
│   └── quota/        # Quota check, webhooks
├── web/frontend/     # Vue 3 SPA (build → web/dist/)
├── deploy/           # Docker, systemd, nginx
└── migrations/       # SQL migrations

Deployment

Systemd

sudo cp deploy/llamalink.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable llamalink
sudo systemctl start llamalink

###nginx

sudo cp deploy/nginx.conf /etc/nginx/sites-available/llamalink
sudo ln -s /etc/nginx/sites-available/llamalink /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx

License

MIT

5.3 KiB Raw Blame History