darroyo
f88b8e099b
CI / test (push) Failing after 17m30s
- Makefile: add 'deb' and 'release' targets; fix migrate/create syntax - scripts/build-deb.sh: build .deb with systemd unit, postinst/prerm/postrm - .github/workflows/ci.yml: go build, test, lint on PR/push to main - .github/workflows/release.yml: multi-platform build (linux/darwin/windows × amd64/arm64), .deb for Linux, tarballs, draft GitHub Release on tag v* - README.md: add CI/Release badges, Install section with .deb and binary release instructions
236 lines
6.6 KiB
Markdown
236 lines
6.6 KiB
Markdown
# LlamaLink
|
|
|
|
[](https://github.com/llamalink/llamalink/actions/workflows/ci)
|
|
[](https://github.com/llamalink/llamalink/releases/latest)
|
|
|
|
**API Gateway en Go** para ejecutar modelos de lenguaje (LLMs) de forma local usando **llama.cpp**, con autenticación por API keys, rate limiting, cuotas de uso y panel de administración Vue 3.
|
|
|
|
## Características
|
|
|
|
- **API OpenAI-compatible**: `POST /v1/chat/completions` con streaming SSE
|
|
- **Multi-model hot-swap**: Configura muchos modelos, activa uno a la vez sin downtime
|
|
- **Auth por API keys**: Keys con scopes (`chat`, `models`, `usage`, `admin`), hashing bcrypt
|
|
- **Rate limiting**: Token bucket por API key (configurable, backend memory o Redis)
|
|
- **Cuotas mensuales**: Tracking por modelo o global, webhooks al 90% y 100%
|
|
- **Admin SPA**: Vue 3 + Tailwind CSS con Dashboard, Keys, Models, Usage
|
|
- **Deployment**: Docker, docker-compose, systemd, nginx/Caddy
|
|
- **WebSocket**: Live updates de estado del modelo y stats
|
|
|
|
## Requisitos
|
|
|
|
- Go 1.23+
|
|
- Node.js 20+ (para build del frontend)
|
|
- llama-server (GGML/GGUF)
|
|
- CUDA 12.x (opcional, para GPU)
|
|
|
|
## Quick Start
|
|
|
|
### 1. Build
|
|
|
|
```bash
|
|
# Dependencias Go
|
|
go mod download
|
|
|
|
# Build binario
|
|
make build
|
|
|
|
# O directamente
|
|
go build -o llamalink ./cmd/llamalink
|
|
```
|
|
|
|
### 2. Configuración
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# Editar .env - mínimo: ADMIN_TOKEN
|
|
```
|
|
|
|
### 3. Arrancar
|
|
|
|
```bash
|
|
# Con llama-server externo (ya corriendo en 127.0.0.1:8080)
|
|
./llamalink
|
|
|
|
# O gestionar llama-server internamente
|
|
MANAGE_LLAMA_SERVER=true ./llamalink
|
|
```
|
|
|
|
### 4. Probar
|
|
|
|
```bash
|
|
# Health check
|
|
curl http://localhost:8000/health
|
|
|
|
# Login admin (primera key se crea automáticamente)
|
|
ADMIN_TOKEN=tu-token curl http://localhost:8000/api/v1/admin/login \
|
|
-X POST -H "Content-Type: application/json" \
|
|
-d '{"admin_token":"tu-token"}'
|
|
|
|
# Crear API key
|
|
curl http://localhost:8000/v1/keys \
|
|
-H "Authorization: Bearer $ADMIN_TOKEN" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"name":"test-key"}'
|
|
|
|
# Chat completion
|
|
curl http://localhost:8000/v1/chat/completions \
|
|
-H "Authorization: Bearer $TU_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"tu-modelo","messages":[{"role":"user","content":"Hello!"}]}'
|
|
```
|
|
|
|
## Variables de Entorno
|
|
|
|
| Variable | Descripción | Default |
|
|
|---|---|---|
|
|
| `LLAMALINK_ENV` | `development` o `production` | `development` |
|
|
| `LLAMALINK_PORT` | Puerto HTTP | `8000` |
|
|
| `DATABASE_URL` | Connection string SQLite/Postgres | `sqlite:///./llamalink.db` |
|
|
| `MANAGE_LLAMA_SERVER` | ¿Gestionar llama-server internamente? | `true` |
|
|
| `LLAMA_SERVER_BIN` | Path a `llama-server` | `/usr/local/bin/llama-server` |
|
|
| `LLAMA_SERVER_HOST` | Host de llama-server | `127.0.0.1` |
|
|
| `LLAMA_SERVER_PORT` | Puerto de llama-server | `8080` |
|
|
| `LLAMA_SERVER_STARTUP_TIMEOUT` | Timeout startup (seg) | `120` |
|
|
| `RATE_LIMIT_PER_MINUTE` | Requests/min por key | `60` |
|
|
| `ADMIN_TOKEN` | Token admin inicial | `changeme` |
|
|
| `LOG_LEVEL` | `debug`, `info`, `warn`, `error` | `info` |
|
|
|
|
## Docker
|
|
|
|
```bash
|
|
# Build imagen
|
|
make docker/build
|
|
|
|
# O docker-compose completo (API + llama-runner + nginx)
|
|
docker compose -f deploy/docker-compose.yml up -d
|
|
|
|
# Production con tu modelo
|
|
ADMIN_TOKEN=mi-token-secreto docker compose -f deploy/docker-compose.yml up -d
|
|
```
|
|
|
|
## Install
|
|
|
|
### Binary releases (all platforms)
|
|
|
|
Download pre-built binaries from [GitHub Releases](https://github.com/llamalink/llamalink/releases):
|
|
|
|
```bash
|
|
# Linux/macOS/Windows — pick your arch
|
|
curl -L https://github.com/llamalink/llamalink/releases/latest/download/llamalink-linux-amd64.tar.gz | tar -xz
|
|
./llamalink
|
|
```
|
|
|
|
### .deb package (Debian/Ubuntu)
|
|
|
|
```bash
|
|
# Download latest .deb
|
|
curl -L https://github.com/llamalink/llamalink/releases/latest/download/llamalink_VERSION_amd64.deb \
|
|
-o llamalink.deb
|
|
|
|
# Install
|
|
sudo dpkg -i llamalink.deb
|
|
|
|
# Configure
|
|
sudo nano /etc/llamalink/env
|
|
sudo systemctl enable llamalink
|
|
sudo systemctl start llamalink
|
|
```
|
|
|
|
### Build from source
|
|
|
|
```bash
|
|
git clone https://github.com/llamalink/llamalink.git
|
|
cd llamalink
|
|
make deps build
|
|
|
|
# Or with .deb package
|
|
make deb
|
|
sudo dpkg -i dist/llamalink_*.deb
|
|
```
|
|
|
|
## Admin Panel
|
|
|
|
Accede a `http://localhost:8000/admin/` para el panel Vue 3:
|
|
|
|
- **Dashboard**: Stats, estado del modelo, tabla de modelos con hot-swap
|
|
- **API Keys**: Crear, listar, revocar keys
|
|
- **Models**: CRUD de modelos, load/unload
|
|
- **Usage**: Charts de uso, logs, quota
|
|
|
|
## API Endpoints
|
|
|
|
### OpenAI-compatible
|
|
- `POST /v1/chat/completions` — Chat con streaming
|
|
- `GET /v1/models` — Lista modelos
|
|
- `GET /v1/models/active` — Modelo activo
|
|
|
|
### Gestión
|
|
- `GET|POST /v1/keys` — Listar/crear keys (admin)
|
|
- `DELETE /v1/keys/{id}` — Revocar key (admin)
|
|
- `POST /v1/models` — Crear modelo (admin)
|
|
- `POST /v1/models/{name}/load` — Hot-swap (admin)
|
|
- `GET /v1/usage` — Uso de tu key
|
|
- `GET /v1/usage/{key_id}` — Uso específico (admin)
|
|
|
|
### Admin API (JSON)
|
|
- `POST /api/v1/admin/login` — Login admin
|
|
- `GET /api/v1/admin/dashboard` — Stats aggregated
|
|
- `GET /api/v1/admin/keys` — Keys con detalles
|
|
- `POST /api/v1/admin/keys` — Crear key
|
|
- `DELETE /api/v1/admin/keys/{id}` — Revocar
|
|
|
|
## Makefile
|
|
|
|
```bash
|
|
make build # Build binario
|
|
make run # Build y ejecutar
|
|
make dev # go run (desarrollo)
|
|
make test # Tests
|
|
make lint # golangci-lint
|
|
make clean # Limpiar build
|
|
make migrate/up # Correr migrations
|
|
make frontend/build # Build Vue SPA
|
|
make deb # Build .deb package (Linux, requires dpkg-deb)
|
|
make release # Build .deb + local release artifacts
|
|
make docker/build # Build imagen Docker
|
|
```
|
|
|
|
## Arquitectura
|
|
|
|
```
|
|
llamalink/
|
|
├── cmd/llamalink/ # Entry point
|
|
├── internal/
|
|
│ ├── api/ # Gin router, handlers, middleware
|
|
│ ├── auth/ # API key service, bcrypt
|
|
│ ├── db/ # GORM models, migrations
|
|
│ ├── llama/ # Proxy, subprocess manager
|
|
│ └── quota/ # Quota check, webhooks
|
|
├── web/frontend/ # Vue 3 SPA (build → web/dist/)
|
|
├── deploy/ # Docker, systemd, nginx
|
|
└── migrations/ # SQL migrations
|
|
```
|
|
|
|
## Deployment
|
|
|
|
### Systemd
|
|
|
|
```bash
|
|
sudo cp deploy/llamalink.service /etc/systemd/system/
|
|
sudo systemctl daemon-reload
|
|
sudo systemctl enable llamalink
|
|
sudo systemctl start llamalink
|
|
```
|
|
|
|
###nginx
|
|
|
|
```bash
|
|
sudo cp deploy/nginx.conf /etc/nginx/sites-available/llamalink
|
|
sudo ln -s /etc/nginx/sites-available/llamalink /etc/nginx/sites-enabled/
|
|
sudo nginx -t && sudo systemctl reload nginx
|
|
```
|
|
|
|
## License
|
|
|
|
MIT
|