Compare commits

...

54 Commits

Author SHA1 Message Date
darroyo 7b693bee5b Bump version to 1.0.58 2026-07-19 22:21:28 -04:00
darroyo 5cb21ddceb db: fix migration runner - proper schema_migrations checksum handling for existing DBs
- Check if checksum column exists before querying it
- For old DBs (no checksum col): use COUNT(*) to check if migration applied
- For new DBs (has checksum col): use checksum value
- All migrations run in a single transaction
2026-07-19 22:20:55 -04:00
darroyo dfb667e340 Bump version to 1.0.57 2026-07-19 22:18:46 -04:00
darroyo 02c1fd55fb Bump version to 1.0.56 2026-07-19 22:18:42 -04:00
darroyo d8aaaf5ca4 db: fix migration runner to handle existing DB without checksum col 2026-07-19 22:18:38 -04:00
darroyo 867794d846 Bump version to 1.0.55 2026-07-19 22:18:12 -04:00
darroyo f5d3ecfbf3 db: fix migration for existing schema_migrations without checksum column
- Add 0007 migration to ALTER TABLE adding checksum column
- Make migration runner INSERT conditional: if checksum col exists, use it; otherwise omit
- Handles existing databases (no checksum col) and fresh installs (has checksum col) correctly
2026-07-19 22:18:06 -04:00
darroyo a7ae619b76 Bump version to 1.0.54 2026-07-19 22:14:38 -04:00
darroyo 84b185be39 Phase A-E: stability, security, observability, and test coverage
Phase A - Stability:
- Engine.Start(): recovers orphaned jobs (running/queued/waking_up) after crash
- Engine.Stop(): graceful shutdown - cancels all in-flight jobs and waits
- Queue keyed by jobID (not syncPairID): cancel now targets exact job
- Local rsync uses jobCtx (context.Background() replaced)
- Migrations wrapped in transactions; checksums stored

Phase B - Security:
- admin/admin default removed: SYNCSERVER_ADMIN_PASSWORD required on first run
- Path validation: rejects .., leading -, null bytes in sync pair paths
- Rsync flags allowlist: dangerous flags blocked (--rsync-path, -e, --files-from)
- Shell concat in RunRemote replaced with proper sh -c escaping
- knownhosts: replaced custom parser with golang.org/x/crypto/ssh/knownhosts
- RequireAdmin wired: machine CRUD, SSH key ops, settings require admin role
- deploy-keys: uses authorized_keys only (no private key upload)
- Hardcoded /var/lib/syncserver/ssh paths replaced with cfg.SSHDir()

Phase C - Operational:
- /readyz health check: DB query + SSH dir accessibility
- /metrics endpoint: Prometheus text format (jobs, queue, machines)
- Event struct JSON tags: job_id, machine_id, type (snake_case)
- EventBus broadcast: fanned out to all subscribers
- SQLite VACUUM INTO backup: scheduled before cleanup if BackupDir set
- Filesystem job log cleanup: removes .log files for purged jobs
- Backup retention: old backups auto-purged

Phase D - Frontend:
- Schedules page: REST API + full CRUD UI for cron schedules
- Dashboard: cancel button for running/queued jobs
- JobDetail: server-side log download via API
- Settings: displays data_dir from server
- 404 page: proper NotFound component

Phase E - Tests:
- auth_test.go: JWT, bcrypt, middleware, seed (18 tests)
- models_test.go: Job, Machine, SyncPair, Schedule repos (18 tests)
- go test -race: no data races found
2026-07-19 22:14:30 -04:00
darroyo 300555d35f Bump version to 1.0.52 2026-07-19 19:49:10 -04:00
darroyo 6b29a4b419 Bump version to 1.0.51 2026-07-19 19:49:00 -04:00
darroyo 4ccf2fc2d6 Bump version to 1.0.50 2026-07-19 18:17:42 -04:00
darroyo ae33703ef9 Bump version to 1.0.49 2026-07-19 18:17:36 -04:00
darroyo 2285790257 Add Direction mode help text and mirror --delete warning in sync pair form 2026-07-19 18:17:29 -04:00
darroyo 2ec031c9dc Bump version to 1.0.48 2026-07-17 18:40:23 -04:00
darroyo b2172145e7 Always copy source directory contents, not the directory itself 2026-07-17 18:40:16 -04:00
darroyo 340370a43b Bump version to 1.0.47 2026-07-17 18:26:26 -04:00
darroyo 743882eb0e Allow remote-to-remote sync pairs 2026-07-17 18:26:19 -04:00
darroyo 0437a7b248 Bump version to 1.0.46 2026-07-13 15:20:20 -04:00
darroyo f476b7bd8b Fix shutdown UX: nohup wrapper prevents SSH session drop during host shutdown
When the host shuts down, sshd is killed before sending exit-status,
causing Go ssh library to return "wait: remote command exited without
exit status" — treated as failure even though the shutdown worked.

Changes:
- shutdown.go: wrap shutdown-like commands with nohup so the SSH
  session exits cleanly before sshd is killed by shutdown
- IsShutdownCommand() helper detects shutdown/poweroff/halt/reboot commands
- handlers_machines.go: classify expected shutdown-side-effect errors
  (no exit status, connection refused/reset) as success so the UI
  shows a green toast instead of a false error
2026-07-13 15:20:13 -04:00
darroyo e6d550b3ab Bump version to 1.0.45 2026-07-13 13:43:40 -04:00
darroyo 50f73cd656 Fix ApproveFingerprint: extract real host key via SSH instead of writing fingerprint SHA256 to known_hosts
The old ApproveFingerprint passed the SHA256 fingerprint string to
AddKnownHost which expected authorized_key format, causing known_hosts
entries to be corrupted and subsequent SSH connections (including shutdown)
to fail with "host key not found".

Changes:
- dialSSH now returns (conn, fingerprint, pubKey, error) with the raw
  ssh.PublicKey captured from the server
- New ConnectForApproval() wraps dialSSH with strictHostKeyChecking=false
  for the approval handshake
- ApproveFingerprint now opens an SSH connection to the host (non-strict),
  captures the real public key, and writes it in authorized_keys format
  to known_hosts via AddKnownHost
- shutdown.go updated to handle the new 4-value dialSSH return
- Supports optional host_key field in request body for direct key submission
2026-07-13 13:43:33 -04:00
darroyo d12f76ca56 Bump version to 1.0.44 2026-07-13 13:35:02 -04:00
darroyo bc3bc44c4a Add shutdown machine feature with configurable command
Backend:
- New migration: add shutdown_command TEXT column to machines table
- Machine model updated with ShutdownCommand field (Create/GetAll/GetByID/Update)
- MachineRequest/MachineResponse DTOs updated with shutdown_command field
- New ShutdownResponse DTO
- New POST /api/machines/{id}/shutdown handler via SSH
- Refactor sshmanager/testconn.go: extract dialSSH() helper shared with shutdown.go
- New sshmanager/shutdown.go: RunRemoteCommand with 15s timeout

Frontend:
- New shutdownMachine() API helper and ShutdownResponse type in client.ts
- New shutdown_command field in MachineForm
- Power button (amber) in machines table actions
- Shutdown confirmation modal with WoL warning notice
- shutdown_command input field in machine edit/create form
- Machine interface updated with shutdown_command and last_seen_at fields
2026-07-13 10:02:18 -04:00
darroyo bf9bcccde2 Fix MachineResponse: expose last_seen_at/created_at, clean status string
The status field was being decorated with "(last seen ...)" suffix
which broke frontend statusVariant() matching and prevented the
LastSeen column from showing the separate timestamp.

Changes:
- machineToResp() now returns clean status ("online"/"offline")
- MachineResponse includes last_seen_at and created_at as separate fields
- Fix whitespace typo in WakeCheckIntervalSeconds field
2026-07-10 11:05:27 -04:00
darroyo 7a2a7a4935 Add Last Seen column and color-coded status badges in Machines table
- New Last Seen column showing last_seen_at timestamp per machine
- Status badges: online=green (success), offline=red (error), waking/pending=amber
- Update Machine interface with last_seen_at and created_at fields
2026-07-10 10:51:10 -04:00
darroyo 175fdf4dff Bump version to 1.0.40 2026-07-09 21:58:50 -04:00
darroyo 683c489a24 Bump version to 1.0.39 2026-07-09 21:58:46 -04:00
darroyo 5b28b46e6b fix: simplify RunRemote to not re-prefix source or dest
RunRemote was adding user@host: prefixes on top of the paths already
pre-prefixed by engine.go in the remote-to-remote branch, causing
"both remote" rsync errors. The source path was also incorrectly
prefixed with srcUserHost:, making rsync reject the command entirely.

The fix: buildArgs already orders args correctly for rsync (source
then dest), so just use args[-2] and args[-1] as-is without any
additional prefixing.
2026-07-09 21:58:02 -04:00
darroyo fce8962f2b Bump version to 1.0.38 2026-07-09 21:51:25 -04:00
darroyo 147c6d8ead fix: rsync double SSH spec in RunRemote, restore dropped flags in Run
- Fix Bug A: RunRemote was re-prefixing destination with user@host: when
  engine.go:135 already pre-prefixed it for remote-to-remote, causing
  "admin@host:admin@host:/path" to be passed to rsync
- Fix Bug B: Run used args[1:] when inserting -e ssh, silently dropping
  the first rsync flag (e.g. -aP became flag-less)
- Fix Bug C: RunRemote extracted source/dest in wrong order for pull
  direction (buildArgs reverses them but RunRemote assumed push order)
- Add rsync_runner_test.go covering buildArgs, RunRemote prefixing,
  and flag preservation
2026-07-09 21:51:05 -04:00
darroyo 969ccacfc8 Bump version to 1.0.36 2026-07-09 21:25:53 -04:00
darroyo 935fa83b93 fix: add json tags to DeployResult for camelCase serialization
DeployResult struct fields were serializing as PascalCase (Success,
Messages, Errors) but the TypeScript frontend expected camelCase
(success, messages, errors). Adding json:"..." tags fixes the mismatch.
2026-07-09 21:25:49 -04:00
darroyo 0a82f4e777 Bump version to 1.0.35 2026-07-09 21:19:56 -04:00
darroyo 7a88fb861d fix: ssh-keyscan failures are warnings, not fatal in deploy-keys
ssh-keyscan failures no longer set Success=false. Keys were uploaded
successfully which is the critical part. Only session errors remain as
errors, ssh-keyscan failures are advisory.
2026-07-09 21:19:52 -04:00
darroyo 0a42f157f7 Bump version to 1.0.34 2026-07-09 20:58:52 -04:00
darroyo c7337a3bbc fix: remove public key deployment from deploy-keys
Deploy-keys now only:
- Creates /var/lib/syncserver/ssh/keys on remote
- Uploads private keys of all other machines to that path
- Populates known_hosts with all other machine hosts

No longer touches authorized_keys or reads server public key.
2026-07-09 20:58:48 -04:00
darroyo 048b99921e Bump version to 1.0.33 2026-07-09 20:52:37 -04:00
darroyo f7801585a6 feat: deploy all machine keys (mesh)
When deploying keys to a machine, upload ALL private keys from ALL
other machines (not just sync pair peers). Also populate known_hosts
with all other machine hosts. Creates a full mesh where any machine
can SSH to any other.

- sshmanager/deploy.go: change knownHostsHost string parameter to
  knownHostsHosts []string for multi-host ssh-keyscan
- handlers_machines.go: replace sync-pair-based key detection with
  loop over all machines, deduplicating by local key path
2026-07-09 20:52:33 -04:00
darroyo e0252829b3 Bump version to 1.0.32 2026-07-09 20:45:37 -04:00
darroyo cb39df63bc fix: deploy keys StdinPipe order and add structured logging
- sshmanager/deploy.go: StdinPipe() must be called BEFORE Start(),
  not after. Reordered the calls to fix "ssh: StdinPipe after
  process started" error.
- Return DeployResult instead of nil error on SSH dial failure
  so the frontend always gets a parseable response.
- Add slog.Debug for key upload, slog.Warn for ssh-keyscan and
  authorized_keys failures, slog.Info for final result summary.
2026-07-09 20:45:32 -04:00
darroyo be2d2d1a8d Bump version to 1.0.31 2026-07-09 20:38:45 -04:00
darroyo 179b2d8fbd fix: deploy keys modal crash on null/undefined arrays
- sshmanager/deploy.go: initialize DeployResult with empty slices
  instead of nil to prevent null serialization in JSON
- Machines.tsx: use ?? [] fallback for messages and errors arrays
  in deploy keys modal to handle null/undefined gracefully
2026-07-09 20:38:41 -04:00
darroyo c0bf16f132 Bump version to 1.0.30 2026-07-09 20:33:43 -04:00
darroyo ee5372c9fd Bump version to 1.0.29 2026-07-09 20:33:19 -04:00
darroyo dbf998703f feat: add Deploy Keys function to Machines UI
- sshmanager/deploy.go: new DeployKeysToMachine function that uploads
  private keys, populates known_hosts via ssh-keyscan, and adds server
  pub key to authorized_keys on remote machines
- handlers_machines.go: new DeployKeys handler with auto-detection of
  keys needed per sync pair (source->dest uploads dest key, dest->source
  uploads source key)
- router.go: POST /machines/{id}/deploy-keys route
- client.ts: deployKeys() API method
- Machines.tsx: Deploy Keys button + modal with result display
2026-07-09 20:33:04 -04:00
darroyo 5d708e1d7b fix: RunRemote SSH-to-source with pre-installed dest key, eliminating base64 round-trip
- Regenerate qnap.key and baby-nas.key to OpenSSH native format (387 bytes vs 119 PKCS8)
- Pre-install qnap.key on Baby NAS at /var/lib/syncserver/ssh/keys/
- Pre-populate Baby NAS known_hosts with Qnap host keys
- Simplify RunRemote: LXC SSH to Baby NAS, Baby NAS runs rsync with local qnap.key
- Remove wrapper script approach (rsync rejects remote-to-remote)
2026-07-09 20:03:00 -04:00
darroyo 727d8676a6 Bump version to 1.0.27 2026-07-09 18:41:13 -04:00
darroyo f76cdaf140 Bump version to 1.0.26 2026-07-09 18:40:30 -04:00
darroyo afe399d3a9 fix: use git add -f to bypass spurious ignore on cmd/server 2026-07-09 18:40:15 -04:00
darroyo 83cc48e70f Bump version to 1.0.25 2026-07-09 18:30:24 -04:00
darroyo 63e82c502c Bump version to 1.0.24 2026-07-09 18:28:03 -04:00
darroyo cc1519810b chore: stop tracking build artifacts in internal/webui/dist (regenerated by build) 2026-07-09 18:27:46 -04:00
darroyo 3ad22c3dcb fix: show fingerprint approve button even on connection success 2026-07-09 18:27:03 -04:00
50 changed files with 4245 additions and 1098 deletions
+173
View File
@@ -0,0 +1,173 @@
# Revertir RunRemote a approach SSH-a-source sin envío base64
## Estado
Estamos en plan mode (READ-ONLY). Los cambios de código Go están bloqueados. Este documento describe los cambios necesarios para proceder cuando salgamos de plan mode.
## Contexto
Las claves SSH ya están pre-instaladas en cada máquina:
| Máquina | Clave instalada | Pública autorizada en |
|---|---|---|
| Baby NAS (10.5.1.20) | `/var/lib/syncserver/ssh/keys/qnap.key` (OpenSSH nativo, 387 bytes) | qnap.key.pub ya en Qnap `/mnt/HDA_ROOT/.config/ssh/authorized_keys` |
| Qnap (10.5.0.144) | n/a (server) | baby-nas.key.pub ya en Baby NAS `/root/.ssh/authorized_keys` |
| Baby NAS known_hosts | `/var/lib/syncserver/ssh/known_hosts` (3 host keys de Qnap en formato hashed) | — |
Test crítico exitoso:
```
ssh -i /var/lib/syncserver/ssh/keys/baby-nas.key root@10.5.1.20 \
"ssh -i /var/lib/syncserver/ssh/keys/qnap.key -o StrictHostKeyChecking=yes \
admin@10.5.0.144 echo QNAP-FROM-BABY"
→ QNAP-FROM-BABY (exit 0)
```
## Cambios pendientes
### 1. `internal/syncengine/rsync_runner.go`
**Eliminar** la función `buildWrapperScript` (líneas 86-101 del archivo actual).
**Reemplazar** la función `RunRemote` actual con la versión SSH-a-source:
```go
func (r *RsyncRunner) RunRemote(ctx context.Context, pair *SyncPairConfig, src *MachineKeys, dst *MachineKeys, destKey string, onLine func(stream string, line string)) (*RsyncResult, error) {
if src.Port == 0 {
src.Port = 22
}
if src.PrivKey == "" {
src.PrivKey = filepath.Join(r.sshDir, "id_ed25519")
}
if destKey == "" {
destKey = filepath.Join(r.sshDir, "id_ed25519")
}
destUserHost := fmt.Sprintf("%s@%s", dst.SSHUser, dst.Host)
args := r.buildArgs(pair)
innerSSH := fmt.Sprintf("ssh -i %s -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=%s",
destKey, filepath.Join(r.sshDir, "known_hosts"))
rsyncFlags := strings.Join(args[:len(args)-2], " ")
sourcePath := args[len(args)-2]
destPath := args[len(args)-1]
remoteCmd := fmt.Sprintf("rsync %s -e %q %s %s",
rsyncFlags, innerSSH, sourcePath, destUserHost+":"+destPath)
sshArgs := []string{
"-i", src.PrivKey,
"-o", "StrictHostKeyChecking=accept-new",
"-o", "UserKnownHostsFile=" + filepath.Join(r.sshDir, "known_hosts"),
"-p", fmt.Sprintf("%d", src.Port),
fmt.Sprintf("%s@%s", src.SSHUser, src.Host),
}
sshArgs = append(sshArgs, remoteCmd)
cmd := exec.CommandContext(ctx, "ssh", sshArgs...)
return r.runCmd(ctx, cmd, onLine)
}
```
**Modificar** el struct `MachineKeys` para agregar `SSHUser`:
```go
type MachineKeys struct {
Host string
Port int
User string
SSHUser string
PrivKey string
}
```
### 2. `internal/syncengine/engine.go`
Actualizar la llamada a `RunRemote` (líneas 263-283 actuales) para usar la nueva firma con `destPrivKeyPath` como argumento separado:
```go
if isRemoteToRemote(srcMachine, dstMachine) {
destPrivKeyPath, err := e.resolveSSHKey(dstMachine)
if err != nil {
slog.Warn("failed to resolve destination SSH key, using server key", "error", err)
destPrivKeyPath = ""
}
result, err = runner.RunRemote(jobCtx, cfg,
&syncengine.MachineKeys{
Host: srcMachine.Host,
Port: srcMachine.Port,
User: srcMachine.SSHUser,
SSHUser: srcMachine.SSHUser,
PrivKey: privKey,
},
&syncengine.MachineKeys{
Host: dstMachine.Host,
Port: dstMachine.Port,
User: dstMachine.SSHUser,
SSHUser: dstMachine.SSHUser,
PrivKey: destPrivKeyPath,
},
destPrivKeyPath,
onLine)
}
```
### 3. Build y deploy
```bash
go build ./cmd/server
make clean && make package
./scripts/bump-version.sh # bump a 1.0.28
git add -f cmd/server/main.go Makefile internal/syncengine/
git commit -m "fix: RunRemote uses SSH-to-source, dest key pre-installed on source"
git push
# Deploy manual:
scp dist/syncserver_1.0.28_amd64.deb root@10.5.1.30:/root/move-data-nas/dist/
ssh root@10.5.1.30 "cd /root/move-data-nas && dpkg -i dist/syncserver_1.0.28_amd64.deb"
```
### 4. Test
Trigger job 23 desde UI o via API. Verificar en DB:
```bash
ssh root@10.5.1.30 'sqlite3 /var/lib/syncserver/app.db \
"SELECT id, status, error_code, substr(error_message,1,200) FROM jobs WHERE id=24;"'
```
Esperado: `status=success`, `error_code=NULL`, `error_message=NULL`.
## Por qué este approach funciona
Antes:
- LXC SSH a Baby NAS
- Baby NAS ejecuta `echo 'BASE64' | base64 -d > /tmp/syncserver-dest-key && rsync ...`
- El base64 round-trip generaba archivo corrupto o vacío
- Baby NAS OpenSSH 8.9 no podía parsear el formato PKCS8
Ahora:
- LXC SSH a Baby NAS (usa baby-nas.key)
- Baby NAS ejecuta `rsync -e ssh -i /var/lib/syncserver/ssh/keys/qnap.key ...`
- qnap.key YA está en Baby NAS en formato OpenSSH nativo (parseable por 8.9)
- Host key de Qnap YA está en known_hosts de Baby NAS (no prompt)
- Pública de qnap.key YA está en authorized_keys de Qnap (auth pasa)
## Comportamiento esperado
```
job started job_id=24 pair=Peliculas
ssh -i baby-nas.key root@10.5.1.20 'rsync -aP -e "ssh -i qnap.key -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=..." /mnt/storage/multimedia/peliculas admin@10.5.0.144:/share/media/peliculas'
sending incremental file list
...
sent X bytes received Y bytes Z bytes/sec
total size is T speedup is S
job completed job_id=24 pair=Peliculas
```
## Si falla
| Error | Causa | Fix |
|---|---|---|
| Load key invalid format | clave corrupta en Baby NAS | re-copiar qnap.key |
| Permission denied (publickey) | pública no está en Qnap | re-agregar a authorized_keys |
| Host key verification failed | known_hosts desincronizado | re-correr ssh-keyscan |
| Connection timed out | problema L2/L3 real (no era el caso) | investigar red |
+1 -1
View File
@@ -1,5 +1,5 @@
BINARY=syncserver
VERSION?=1.0.23
VERSION?=1.0.58
GO?=go
LDFLAGS=-s -w -X main.version=$(VERSION) -X main.commit=$(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)
BUILD_FLAGS=CGO_ENABLED=0
+1 -1
View File
@@ -20,7 +20,7 @@ import (
"github.com/syncserver/internal/syncengine"
)
var version = "1.0.23"
var version = "1.0.58"
func main() {
cfgPath := flag.String("config", "", "Path to config.yaml")
+33
View File
@@ -11,6 +11,7 @@ type MachineRequest struct {
BroadcastAddr *string `json:"broadcast_addr"`
WakeTimeoutSeconds int `json:"wake_timeout_seconds"`
WakeCheckIntervalSeconds int `json:"wake_check_interval_seconds"`
ShutdownCommand string `json:"shutdown_command,omitempty"`
}
type MachineResponse struct {
@@ -28,6 +29,9 @@ type MachineResponse struct {
FingerprintConfirmed bool `json:"fingerprint_confirmed"`
HostKeyFingerprint *string `json:"host_key_fingerprint"`
Status string `json:"status"`
LastSeenAt *string `json:"last_seen_at"`
CreatedAt string `json:"created_at"`
ShutdownCommand string `json:"shutdown_command"`
}
type TestConnectionResponse struct {
@@ -37,6 +41,12 @@ type TestConnectionResponse struct {
Fingerprint string `json:"fingerprint,omitempty"`
}
type ShutdownResponse struct {
Success bool `json:"success"`
Output string `json:"output,omitempty"`
Error string `json:"error,omitempty"`
}
type SyncPairRequest struct {
Name string `json:"name"`
SourceMachineID *int64 `json:"source_machine_id"`
@@ -74,6 +84,8 @@ type JobResponse struct {
ErrorCode *string `json:"error_code,omitempty"`
DurationSeconds *int64 `json:"duration_seconds,omitempty"`
LogLineCount *int64 `json:"log_line_count,omitempty"`
TotalSizeBytes *int64 `json:"total_size_bytes,omitempty"`
SentBytes *int64 `json:"sent_bytes,omitempty"`
}
type LogLineResponse struct {
@@ -109,3 +121,24 @@ type SettingsInfoResponse struct {
DataDir string `json:"data_dir"`
SSHPubKey string `json:"ssh_pub_key"`
}
type CreateScheduleRequest struct {
SyncPairID int64 `json:"sync_pair_id"`
CronExpr string `json:"cron_expr"`
Enabled bool `json:"enabled"`
}
type UpdateScheduleRequest struct {
CronExpr string `json:"cron_expr"`
Enabled bool `json:"enabled"`
}
type ScheduleResponse struct {
ID int64 `json:"id"`
SyncPairID int64 `json:"sync_pair_id"`
SyncPairName string `json:"sync_pair_name"`
CronExpr string `json:"cron_expr"`
NextRun *string `json:"next_run_at"`
Enabled bool `json:"enabled"`
CreatedAt string `json:"created_at"`
}
+7 -1
View File
@@ -128,7 +128,7 @@ func (h *JobHandler) Cancel(w http.ResponseWriter, r *http.Request) {
}
if h.engine != nil {
h.engine.Cancel(id, j.SyncPairID, true)
h.engine.Cancel(id, true)
}
repo.UpdateStatus(id, "cancelled")
@@ -254,6 +254,12 @@ func jobToResp(j models.Job) JobResponse {
s := j.FinishedAt.Format(time.RFC3339)
resp.FinishedAt = &s
}
if j.TotalSizeBytes > 0 {
resp.TotalSizeBytes = &j.TotalSizeBytes
}
if j.SentBytes > 0 {
resp.SentBytes = &j.SentBytes
}
return resp
}
+242 -22
View File
@@ -4,14 +4,18 @@ import (
"context"
"database/sql"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"strings"
"path/filepath"
"regexp"
"strconv"
"time"
"github.com/go-chi/chi/v5"
"github.com/syncserver/internal/config"
"golang.org/x/crypto/ssh"
"github.com/syncserver/internal/models"
"github.com/syncserver/internal/sshmanager"
"github.com/syncserver/internal/syncengine"
@@ -21,10 +25,11 @@ import (
type MachineHandler struct {
db *sql.DB
engine *syncengine.Engine
cfg *config.Config
}
func NewMachineHandler(db *sql.DB, engine *syncengine.Engine) *MachineHandler {
return &MachineHandler{db: db, engine: engine}
func NewMachineHandler(db *sql.DB, engine *syncengine.Engine, cfg *config.Config) *MachineHandler {
return &MachineHandler{db: db, engine: engine, cfg: cfg}
}
var macRegex = regexp.MustCompile(`^([0-9A-Fa-f]{2}[:-]){5}[0-9A-Fa-f]{2}$`)
@@ -93,6 +98,10 @@ func (h *MachineHandler) Create(w http.ResponseWriter, r *http.Request) {
return
}
shutdownCmd := req.ShutdownCommand
if shutdownCmd == "" {
shutdownCmd = "shutdown now"
}
m := &models.Machine{
Name: req.Name,
Host: req.Host,
@@ -106,6 +115,7 @@ func (h *MachineHandler) Create(w http.ResponseWriter, r *http.Request) {
WakeCheckIntervalSeconds: req.WakeCheckIntervalSeconds,
FingerprintConfirmed: false,
Status: "unknown",
ShutdownCommand: shutdownCmd,
}
repo := models.NewMachineRepository(h.db)
@@ -175,6 +185,9 @@ func (h *MachineHandler) Update(w http.ResponseWriter, r *http.Request) {
if req.WakeCheckIntervalSeconds > 0 {
existing.WakeCheckIntervalSeconds = req.WakeCheckIntervalSeconds
}
if req.ShutdownCommand != "" {
existing.ShutdownCommand = req.ShutdownCommand
}
if err := repo.Update(existing); err != nil {
slog.Error("failed to update machine", "id", id, "error", err)
@@ -221,6 +234,87 @@ func (h *MachineHandler) TestWoL(w http.ResponseWriter, r *http.Request) {
writeJSON(w, map[string]interface{}{"ok": true, "sent": 3})
}
func (h *MachineHandler) Shutdown(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid id")
return
}
repo := models.NewMachineRepository(h.db)
m, err := repo.GetByID(id)
if err == sql.ErrNoRows {
writeError(w, http.StatusNotFound, "machine not found")
return
}
if err != nil {
slog.Error("failed to fetch machine", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch machine")
return
}
knownHostsPath, err := sshmanager.EnsureKnownHosts(h.cfg.SSHDir())
if err != nil {
writeError(w, http.StatusInternalServerError, "failed to ensure known_hosts")
return
}
privKeyPath := filepath.Join(h.cfg.SSHDir(), "id_ed25519")
if m.SSHKeyID != nil {
sshKeyRepo := models.NewSSHKeyRepository(h.db)
sshKey, err := sshKeyRepo.GetByID(*m.SSHKeyID)
if err == nil && sshKey.PrivateKeyPath != "" {
privKeyPath = sshKey.PrivateKeyPath
}
}
cmd := m.ShutdownCommand
if cmd == "" {
cmd = "shutdown now"
}
slog.Info("shutdown requested", "machine_id", id, "name", m.Name, "command", cmd)
result, err := sshmanager.RunRemoteCommand(
context.Background(), m.Host, m.Port, m.SSHUser,
privKeyPath, knownHostsPath, m.FingerprintConfirmed, cmd,
)
if err != nil {
slog.Error("shutdown failed", "machine_id", id, "error", err)
writeJSON(w, ShutdownResponse{Success: false, Error: err.Error()})
return
}
responseSuccess := result.Success
responseOutput := result.Output
responseError := result.Error
if !responseSuccess && m.ShutdownCommand != "" && sshmanager.IsShutdownCommand(m.ShutdownCommand) {
if isExpectedShutdownError(result.Error) {
responseSuccess = true
responseError = ""
if responseOutput == "" {
responseOutput = "shutdown command sent (host session terminated as expected)"
}
}
}
writeJSON(w, ShutdownResponse{
Success: responseSuccess,
Output: responseOutput,
Error: responseError,
})
}
func isExpectedShutdownError(errMsg string) bool {
if errMsg == "" {
return false
}
return strings.Contains(errMsg, "remote command exited without exit status") ||
strings.Contains(errMsg, "connection refused") ||
strings.Contains(errMsg, "connection reset by peer") ||
strings.Contains(errMsg, "use of closed network connection")
}
func (h *MachineHandler) TestConnection(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
@@ -239,13 +333,13 @@ func (h *MachineHandler) TestConnection(w http.ResponseWriter, r *http.Request)
return
}
knownHostsPath, err := sshmanager.EnsureKnownHosts(filepath.Join("/var/lib/syncserver", "ssh"))
knownHostsPath, err := sshmanager.EnsureKnownHosts(h.cfg.SSHDir())
if err != nil {
writeError(w, http.StatusInternalServerError, "failed to ensure known_hosts")
return
}
privKeyPath := filepath.Join("/var/lib/syncserver", "ssh", "id_ed25519")
privKeyPath := filepath.Join(h.cfg.SSHDir(), "id_ed25519")
if m.SSHKeyID != nil {
sshKeyRepo := models.NewSSHKeyRepository(h.db)
sshKey, err := sshKeyRepo.GetByID(*m.SSHKeyID)
@@ -292,30 +386,154 @@ func (h *MachineHandler) ApproveFingerprint(w http.ResponseWriter, r *http.Reque
var req struct {
Fingerprint string `json:"fingerprint"`
HostKey string `json:"host_key"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid request body")
return
}
if req.Fingerprint == "" {
writeError(w, http.StatusBadRequest, "fingerprint is required")
json.NewDecoder(r.Body).Decode(&req)
sshDir := h.cfg.SSHDir()
knownHostsPath, err := sshmanager.EnsureKnownHosts(sshDir)
if err != nil {
writeError(w, http.StatusInternalServerError, "failed to ensure known_hosts")
return
}
if err := repo.UpdateFingerprint(id, true, req.Fingerprint); err != nil {
privKeyPath := filepath.Join(h.cfg.SSHDir(), "id_ed25519")
if m.SSHKeyID != nil {
sshKeyRepo := models.NewSSHKeyRepository(h.db)
sshKey, err := sshKeyRepo.GetByID(*m.SSHKeyID)
if err == nil && sshKey.PrivateKeyPath != "" {
privKeyPath = sshKey.PrivateKeyPath
}
}
var fingerprint, pubKeyLine string
if req.HostKey != "" {
pubKeyLine = req.HostKey
} else {
conn, fp, pubKey, err := sshmanager.ConnectForApproval(
context.Background(), m.Host, m.Port, m.SSHUser,
privKeyPath, knownHostsPath,
)
if err != nil {
writeError(w, http.StatusInternalServerError, fmt.Sprintf("could not retrieve host key: %v", err))
return
}
conn.Close()
fingerprint = fp
pubKeyLine = string(ssh.MarshalAuthorizedKey(pubKey))
}
if fingerprint == "" && req.Fingerprint != "" {
fingerprint = req.Fingerprint
}
if pubKeyLine != "" {
if err := sshmanager.AddKnownHost(sshDir, m.Host, m.Port, []byte(pubKeyLine)); err != nil {
writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to add known_host entry: %v", err))
return
}
}
if err := repo.UpdateFingerprint(id, true, fingerprint); err != nil {
slog.Error("failed to update fingerprint", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to update fingerprint")
return
}
sshDir := filepath.Join("/var/lib/syncserver", "ssh")
if err := sshmanager.AddKnownHost(sshDir, m.Host, m.Port, []byte(req.Fingerprint)); err != nil {
slog.Warn("failed to add known_host entry", "host", m.Host, "error", err)
m.FingerprintConfirmed = true
m.HostKeyFingerprint = &fingerprint
writeJSON(w, machineToResp(*m))
}
m.FingerprintConfirmed = true
m.HostKeyFingerprint = &req.Fingerprint
writeJSON(w, machineToResp(*m))
func (h *MachineHandler) DeployKeys(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid id")
return
}
repo := models.NewMachineRepository(h.db)
m, err := repo.GetByID(id)
if err == sql.ErrNoRows {
writeError(w, http.StatusNotFound, "machine not found")
return
}
if err != nil {
slog.Error("failed to fetch machine", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch machine")
return
}
var req struct {
KnownHostsHost string `json:"known_hosts_host"`
IncludeServerKey bool `json:"include_server_key"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid request body")
return
}
serverKeyPath := h.cfg.SSHDir() + "/id_ed25519"
if m.SSHKeyID != nil {
sshKeyRepo := models.NewSSHKeyRepository(h.db)
sshKey, err := sshKeyRepo.GetByID(*m.SSHKeyID)
if err == nil && sshKey.PrivateKeyPath != "" {
serverKeyPath = sshKey.PrivateKeyPath
}
}
allMachines, err := repo.GetAll()
if err != nil {
slog.Warn("failed to fetch machines for auto-detect", "error", err)
}
var keys []sshmanager.DeployKey
seenKeys := make(map[string]bool)
knownHostsHosts := []string{}
for _, other := range allMachines {
if other.ID == m.ID {
continue
}
knownHostsHosts = append(knownHostsHosts, fmt.Sprintf("%s:%d", other.Host, other.Port))
if other.SSHKeyID != nil {
skRepo := models.NewSSHKeyRepository(h.db)
sk, err := skRepo.GetByID(*other.SSHKeyID)
if err == nil && sk.PrivateKeyPath != "" {
if seenKeys[sk.PrivateKeyPath] {
continue
}
seenKeys[sk.PrivateKeyPath] = true
keys = append(keys, sshmanager.DeployKey{
LocalPath: sk.PrivateKeyPath,
RemotePath: h.cfg.SSHDir() + "/keys/" + filepath.Base(sk.PrivateKeyPath),
Mode: 0600,
})
}
}
}
if len(keys) == 0 {
slog.Info("no keys auto-detected for machine, using empty key list", "machine", m.Name)
}
result, err := sshmanager.DeployKeysToMachine(
context.Background(),
serverKeyPath,
m.Host,
m.Port,
m.SSHUser,
keys,
knownHostsHosts,
h.cfg.SSHDir(),
)
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
writeJSON(w, result)
}
func (h *MachineHandler) Refresh(w http.ResponseWriter, r *http.Request) {
@@ -352,11 +570,10 @@ func (h *MachineHandler) Delete(w http.ResponseWriter, r *http.Request) {
}
func machineToResp(m models.Machine) MachineResponse {
var status string
var lastSeen *string
if m.LastSeenAt != nil {
status = m.Status + " (last seen " + m.LastSeenAt.Format(time.RFC3339) + ")"
} else {
status = m.Status
s := m.LastSeenAt.Format(time.RFC3339)
lastSeen = &s
}
return MachineResponse{
ID: m.ID,
@@ -372,7 +589,10 @@ func machineToResp(m models.Machine) MachineResponse {
WakeCheckIntervalSeconds: m.WakeCheckIntervalSeconds,
FingerprintConfirmed: m.FingerprintConfirmed,
HostKeyFingerprint: m.HostKeyFingerprint,
Status: status,
Status: m.Status,
LastSeenAt: lastSeen,
CreatedAt: m.CreatedAt.Format(time.RFC3339),
ShutdownCommand: m.ShutdownCommand,
}
}
+243
View File
@@ -0,0 +1,243 @@
package api
import (
"database/sql"
"encoding/json"
"log/slog"
"net/http"
"strconv"
"time"
"github.com/go-chi/chi/v5"
"github.com/syncserver/internal/models"
"github.com/syncserver/internal/scheduler"
)
type ScheduleHandler struct {
db *sql.DB
}
func NewScheduleHandler(db *sql.DB) *ScheduleHandler {
return &ScheduleHandler{db: db}
}
func (h *ScheduleHandler) List(w http.ResponseWriter, r *http.Request) {
repo := models.NewScheduleRepository(h.db)
schedules, err := repo.GetAll()
if err != nil {
slog.Error("failed to fetch schedules", "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch schedules")
return
}
pairRepo := models.NewSyncPairRepository(h.db)
pairs, err := pairRepo.GetAll()
if err != nil {
slog.Error("failed to fetch sync pairs", "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch sync pairs")
return
}
pairMap := make(map[int64]string)
for _, p := range pairs {
pairMap[p.ID] = p.Name
}
out := make([]ScheduleResponse, len(schedules))
for i, s := range schedules {
out[i] = scheduleToResp(s, pairMap)
}
writeJSON(w, out)
}
func (h *ScheduleHandler) Get(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid id")
return
}
repo := models.NewScheduleRepository(h.db)
s, err := repo.GetByID(id)
if err == sql.ErrNoRows {
writeError(w, http.StatusNotFound, "schedule not found")
return
}
if err != nil {
slog.Error("failed to fetch schedule", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch schedule")
return
}
pairRepo := models.NewSyncPairRepository(h.db)
pairs, err := pairRepo.GetAll()
if err != nil {
slog.Error("failed to fetch sync pairs", "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch sync pairs")
return
}
pairMap := make(map[int64]string)
for _, p := range pairs {
pairMap[p.ID] = p.Name
}
writeJSON(w, scheduleToResp(*s, pairMap))
}
func (h *ScheduleHandler) Create(w http.ResponseWriter, r *http.Request) {
var req CreateScheduleRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid request body")
return
}
if req.SyncPairID <= 0 {
writeError(w, http.StatusBadRequest, "sync_pair_id is required")
return
}
if req.CronExpr == "" {
writeError(w, http.StatusBadRequest, "cron_expr is required")
return
}
_, err := scheduler.ParseCron(req.CronExpr)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid cron expression: "+err.Error())
return
}
pairRepo := models.NewSyncPairRepository(h.db)
_, err = pairRepo.GetByID(req.SyncPairID)
if err == sql.ErrNoRows {
writeError(w, http.StatusBadRequest, "sync_pair not found")
return
}
if err != nil {
slog.Error("failed to fetch sync pair", "id", req.SyncPairID, "error", err)
writeError(w, http.StatusInternalServerError, "failed to validate sync pair")
return
}
expr, _ := scheduler.ParseCron(req.CronExpr)
var nextRun *time.Time
if expr != nil {
t := scheduler.NextRun(expr, time.Now().UTC())
nextRun = &t
}
s := &models.Schedule{
SyncPairID: req.SyncPairID,
CronExpr: req.CronExpr,
NextRunAt: nextRun,
Enabled: req.Enabled,
}
repo := models.NewScheduleRepository(h.db)
id, err := repo.Create(s)
if err != nil {
slog.Error("failed to create schedule", "error", err)
writeError(w, http.StatusInternalServerError, "failed to create schedule")
return
}
s.ID = id
pairs, _ := pairRepo.GetAll()
pairMap := make(map[int64]string)
for _, p := range pairs {
pairMap[p.ID] = p.Name
}
w.Header().Set("Location", "/api/schedules/"+strconv.FormatInt(id, 10))
writeJSON(w, scheduleToResp(*s, pairMap), http.StatusCreated)
}
func (h *ScheduleHandler) Update(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid id")
return
}
var req UpdateScheduleRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid request body")
return
}
repo := models.NewScheduleRepository(h.db)
existing, err := repo.GetByID(id)
if err == sql.ErrNoRows {
writeError(w, http.StatusNotFound, "schedule not found")
return
}
if err != nil {
slog.Error("failed to fetch schedule", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to fetch schedule")
return
}
if req.CronExpr != "" {
_, err := scheduler.ParseCron(req.CronExpr)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid cron expression: "+err.Error())
return
}
existing.CronExpr = req.CronExpr
expr, _ := scheduler.ParseCron(req.CronExpr)
if expr != nil {
t := scheduler.NextRun(expr, time.Now().UTC())
existing.NextRunAt = &t
}
}
if req.Enabled {
existing.Enabled = true
} else if req.Enabled == false {
existing.Enabled = false
}
if err := repo.Update(existing); err != nil {
slog.Error("failed to update schedule", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to update schedule")
return
}
pairRepo := models.NewSyncPairRepository(h.db)
pairs, _ := pairRepo.GetAll()
pairMap := make(map[int64]string)
for _, p := range pairs {
pairMap[p.ID] = p.Name
}
writeJSON(w, scheduleToResp(*existing, pairMap))
}
func (h *ScheduleHandler) Delete(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeError(w, http.StatusBadRequest, "invalid id")
return
}
repo := models.NewScheduleRepository(h.db)
if err := repo.Delete(id); err != nil {
slog.Error("failed to delete schedule", "id", id, "error", err)
writeError(w, http.StatusInternalServerError, "failed to delete schedule")
return
}
w.WriteHeader(http.StatusNoContent)
}
func scheduleToResp(s models.Schedule, pairMap map[int64]string) ScheduleResponse {
resp := ScheduleResponse{
ID: s.ID,
SyncPairID: s.SyncPairID,
SyncPairName: pairMap[s.SyncPairID],
CronExpr: s.CronExpr,
Enabled: s.Enabled,
CreatedAt: s.CreatedAt.Format(time.RFC3339),
}
if s.NextRunAt != nil {
t := s.NextRunAt.Format(time.RFC3339)
resp.NextRun = &t
}
return resp
}
+39 -4
View File
@@ -3,10 +3,12 @@ package api
import (
"database/sql"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"regexp"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"github.com/syncserver/internal/models"
@@ -22,6 +24,23 @@ func NewSyncPairHandler(db *sql.DB) *SyncPairHandler {
var directionRegex = regexp.MustCompile(`^(push|pull|mirror)$`)
func ValidatePath(path string) error {
path = strings.TrimSpace(path)
if path == "" {
return fmt.Errorf("path cannot be empty")
}
if strings.Contains(path, "..") {
return fmt.Errorf("path cannot contain '..'")
}
if strings.HasPrefix(path, "-") {
return fmt.Errorf("path cannot start with '-'")
}
if strings.Contains(path, "\x00") {
return fmt.Errorf("path cannot contain null bytes")
}
return nil
}
func (h *SyncPairHandler) List(w http.ResponseWriter, r *http.Request) {
repo := models.NewSyncPairRepository(h.db)
pairs, err := repo.GetAll()
@@ -68,6 +87,14 @@ func (h *SyncPairHandler) Create(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "name, source_path and dest_path are required")
return
}
if err := ValidatePath(req.SourcePath); err != nil {
writeError(w, http.StatusBadRequest, "invalid source_path: "+err.Error())
return
}
if err := ValidatePath(req.DestPath); err != nil {
writeError(w, http.StatusBadRequest, "invalid dest_path: "+err.Error())
return
}
if req.Direction == "" {
req.Direction = "push"
}
@@ -75,8 +102,8 @@ func (h *SyncPairHandler) Create(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "direction must be push, pull, or mirror")
return
}
if req.SourceMachineID != nil && req.DestMachineID != nil {
writeError(w, http.StatusBadRequest, "both source and destination cannot be remote machines; one side must be the server (set one MachineID to null)")
if req.SourceMachineID != nil && req.DestMachineID != nil && *req.SourceMachineID == *req.DestMachineID {
writeError(w, http.StatusBadRequest, "source and destination cannot be the same machine")
return
}
if req.RsyncFlags == "" {
@@ -127,12 +154,20 @@ func (h *SyncPairHandler) Update(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, "name, source_path and dest_path are required")
return
}
if err := ValidatePath(req.SourcePath); err != nil {
writeError(w, http.StatusBadRequest, "invalid source_path: "+err.Error())
return
}
if err := ValidatePath(req.DestPath); err != nil {
writeError(w, http.StatusBadRequest, "invalid dest_path: "+err.Error())
return
}
if !directionRegex.MatchString(req.Direction) {
writeError(w, http.StatusBadRequest, "direction must be push, pull, or mirror")
return
}
if req.SourceMachineID != nil && req.DestMachineID != nil {
writeError(w, http.StatusBadRequest, "both source and destination cannot be remote machines; one side must be the server (set one MachineID to null)")
if req.SourceMachineID != nil && req.DestMachineID != nil && *req.SourceMachineID == *req.DestMachineID {
writeError(w, http.StatusBadRequest, "source and destination cannot be the same machine")
return
}
+2 -2
View File
@@ -45,7 +45,7 @@ func (h *SSEHandler) StreamAll(w http.ResponseWriter, r *http.Request) {
select {
case evt := <-events:
data, _ := json.Marshal(evt)
fmt.Fprintf(w, "event: %s\ndata: %s\n\n", evt.Type, data)
fmt.Fprintf(w, "data: %s\n\n", data)
flusher.Flush()
case <-r.Context().Done():
return
@@ -94,7 +94,7 @@ func (h *SSEHandler) StreamJob(w http.ResponseWriter, r *http.Request) {
select {
case evt := <-events:
data, _ := json.Marshal(evt)
fmt.Fprintf(w, "event: %s\ndata: %s\n\n", evt.Type, data)
fmt.Fprintf(w, "data: %s\n\n", data)
flusher.Flush()
case <-r.Context().Done():
return
+98 -18
View File
@@ -3,9 +3,12 @@ package api
import (
"database/sql"
"encoding/json"
"fmt"
"log/slog"
"net/http"
"os"
"runtime/debug"
"sort"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
@@ -34,41 +37,60 @@ func NewServer(cfg *config.Config, db *sql.DB, engine *syncengine.Engine) *Serve
s := &Server{router: r, cfg: cfg, engine: engine}
authHandler := NewAuthHandler(db)
machineHandler := NewMachineHandler(db, engine)
machineHandler := NewMachineHandler(db, engine, cfg)
syncPairHandler := NewSyncPairHandler(db)
scheduleHandler := NewScheduleHandler(db)
jobHandler := NewJobHandler(db, engine)
sseHandler := NewSSEHandler(engine)
sshKeyHandler := NewSSHKeyHandler(db, cfg)
admin := func(h http.Handler) http.Handler {
return auth.RequireAdmin(auth.RequireAuth(h))
}
authGet := func(h http.Handler) http.Handler {
return auth.RequireAuth(h)
}
r.Route("/api", func(r chi.Router) {
r.Route("/auth", func(r chi.Router) {
r.Post("/login", authHandler.Login)
r.Post("/logout", authHandler.Logout)
r.With(auth.RequireAuth).Get("/me", authHandler.Me)
r.With(authGet).Get("/me", authHandler.Me)
})
r.With(auth.RequireAuth).Route("/machines", func(r chi.Router) {
r.With(authGet).Route("/machines", func(r chi.Router) {
r.Get("/", machineHandler.List)
r.Post("/", machineHandler.Create)
r.Post("/refresh", machineHandler.Refresh)
r.With(admin).Post("/", machineHandler.Create)
r.With(admin).Post("/refresh", machineHandler.Refresh)
r.Get("/{id}", machineHandler.Get)
r.Put("/{id}", machineHandler.Update)
r.Delete("/{id}", machineHandler.Delete)
r.With(admin).Put("/{id}", machineHandler.Update)
r.With(admin).Delete("/{id}", machineHandler.Delete)
r.Post("/{id}/test-wol", machineHandler.TestWoL)
r.With(admin).Post("/{id}/shutdown", machineHandler.Shutdown)
r.Post("/{id}/test-connection", machineHandler.TestConnection)
r.Post("/{id}/approve-fingerprint", machineHandler.ApproveFingerprint)
r.With(admin).Post("/{id}/approve-fingerprint", machineHandler.ApproveFingerprint)
r.With(admin).Post("/{id}/deploy-keys", machineHandler.DeployKeys)
})
r.With(auth.RequireAuth).Route("/sync-pairs", func(r chi.Router) {
r.With(authGet).Route("/sync-pairs", func(r chi.Router) {
r.Get("/", syncPairHandler.List)
r.Post("/", syncPairHandler.Create)
r.Get("/{id}", syncPairHandler.Get)
r.Put("/{id}", syncPairHandler.Update)
r.Delete("/{id}", syncPairHandler.Delete)
r.With(admin).Delete("/{id}", syncPairHandler.Delete)
r.Post("/{id}/run", jobHandler.TriggerRun)
})
r.With(auth.RequireAuth).Route("/jobs", func(r chi.Router) {
r.With(authGet).Route("/schedules", func(r chi.Router) {
r.Get("/", scheduleHandler.List)
r.Post("/", scheduleHandler.Create)
r.Get("/{id}", scheduleHandler.Get)
r.Put("/{id}", scheduleHandler.Update)
r.With(admin).Delete("/{id}", scheduleHandler.Delete)
})
r.With(authGet).Route("/jobs", func(r chi.Router) {
r.Get("/", jobHandler.List)
r.Get("/{id}", jobHandler.Get)
r.Post("/{id}/cancel", jobHandler.Cancel)
@@ -77,15 +99,15 @@ func NewServer(cfg *config.Config, db *sql.DB, engine *syncengine.Engine) *Serve
r.Get("/{id}/log/stream", sseHandler.StreamJob)
})
r.With(auth.RequireAuth).Get("/jobs/stream", sseHandler.StreamAll)
r.With(authGet).Get("/jobs/stream", sseHandler.StreamAll)
r.With(auth.RequireAuth).Get("/settings/pubkey", func(w http.ResponseWriter, r *http.Request) {
r.With(authGet).Get("/settings/pubkey", func(w http.ResponseWriter, r *http.Request) {
_, _, pubKey, _ := sshmanager.EnsureServerKey(cfg.SSHDir())
w.Header().Set("Content-Type", "text/plain")
w.Write([]byte(pubKey))
})
r.With(auth.RequireAuth).Get("/settings/info", func(w http.ResponseWriter, r *http.Request) {
r.With(authGet).Get("/settings/info", func(w http.ResponseWriter, r *http.Request) {
_, _, pubKey, _ := sshmanager.EnsureServerKey(cfg.SSHDir())
resp := SettingsInfoResponse{
Version: cfg.Version,
@@ -96,12 +118,12 @@ func NewServer(cfg *config.Config, db *sql.DB, engine *syncengine.Engine) *Serve
json.NewEncoder(w).Encode(resp)
})
r.With(auth.RequireAuth).Route("/ssh-keys", func(r chi.Router) {
r.With(authGet).Route("/ssh-keys", func(r chi.Router) {
r.Get("/", sshKeyHandler.List)
r.Post("/", sshKeyHandler.Create)
r.With(admin).Post("/", sshKeyHandler.Create)
r.Get("/{id}", sshKeyHandler.Get)
r.Delete("/{id}", sshKeyHandler.Delete)
r.Get("/{id}/private", sshKeyHandler.DownloadPrivate)
r.With(admin).Delete("/{id}", sshKeyHandler.Delete)
r.With(admin).Get("/{id}/private", sshKeyHandler.DownloadPrivate)
})
})
@@ -109,6 +131,64 @@ func NewServer(cfg *config.Config, db *sql.DB, engine *syncengine.Engine) *Serve
w.Write([]byte("ok"))
}))
r.Get("/healthz", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("ok"))
}))
r.Get("/readyz", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
db := s.engine.DB()
if _, err := db.Exec("SELECT 1"); err != nil {
http.Error(w, fmt.Sprintf("db query failed: %v", err), http.StatusServiceUnavailable)
return
}
if _, err := db.Exec("SELECT 1"); err != nil {
http.Error(w, fmt.Sprintf("db write test failed: %v", err), http.StatusServiceUnavailable)
return
}
sshDir := s.cfg.SSHDir()
if _, err := os.Stat(sshDir); err != nil {
http.Error(w, fmt.Sprintf("ssh dir not accessible: %v", err), http.StatusServiceUnavailable)
return
}
w.Write([]byte("ok"))
}))
r.Get("/metrics", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain; version=0.0.4")
jobsTotal := s.engine.GetJobsTotal()
var keys []string
for k := range jobsTotal {
keys = append(keys, k)
}
sort.Strings(keys)
for _, status := range keys {
fmt.Fprintf(w, "# HELP syncserver_jobs_total Total jobs by final status\n")
fmt.Fprintf(w, "# TYPE syncserver_jobs_total counter\n")
fmt.Fprintf(w, "syncserver_jobs_total{status=%q} %d\n", status, jobsTotal[status])
}
fmt.Fprintf(w, "# HELP syncserver_jobs_running Currently running jobs\n")
fmt.Fprintf(w, "# TYPE syncserver_jobs_running gauge\n")
fmt.Fprintf(w, "syncserver_jobs_running %d\n", s.engine.GetJobsRunning())
fmt.Fprintf(w, "# HELP syncserver_queue_depth Jobs waiting to run\n")
fmt.Fprintf(w, "# TYPE syncserver_queue_depth gauge\n")
fmt.Fprintf(w, "syncserver_queue_depth %d\n", s.engine.GetQueueDepth())
online, total := s.engine.GetMachineCounts()
fmt.Fprintf(w, "# HELP syncserver_machines_online Online machines count\n")
fmt.Fprintf(w, "# TYPE syncserver_machines_online gauge\n")
fmt.Fprintf(w, "syncserver_machines_online %d\n", online)
fmt.Fprintf(w, "# HELP syncserver_machines_total Total machines\n")
fmt.Fprintf(w, "# TYPE syncserver_machines_total gauge\n")
fmt.Fprintf(w, "syncserver_machines_total %d\n", total)
fmt.Fprintf(w, "# HELP syncserver_up Server is up\n")
fmt.Fprintf(w, "# TYPE syncserver_up gauge\n")
fmt.Fprintf(w, "syncserver_up 1\n")
}))
r.NotFound(func(w http.ResponseWriter, r *http.Request) {
webui.ServeSPA().ServeHTTP(w, r)
})
+411
View File
@@ -0,0 +1,411 @@
package auth
import (
"context"
"database/sql"
"errors"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/golang-jwt/jwt/v5"
_ "modernc.org/sqlite"
)
func TestJWT_Generate(t *testing.T) {
mgr := NewJWTManager("test-secret", 24)
token, expiresAt, err := mgr.Generate(1, "testuser", "admin")
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
if token == "" {
t.Fatal("Generate() returned empty token")
}
if expiresAt.Before(time.Now()) {
t.Fatal("Generate() returned past expiration time")
}
if expiresAt.Before(time.Now().Add(23 * time.Hour)) {
t.Fatal("Generate() expiration time is too early")
}
claims, err := mgr.Validate(token)
if err != nil {
t.Fatalf("Validate() error = %v", err)
}
if claims.UserID != 1 {
t.Errorf("claims.UserID = %d, want 1", claims.UserID)
}
if claims.Username != "testuser" {
t.Errorf("claims.Username = %s, want testuser", claims.Username)
}
if claims.Role != "admin" {
t.Errorf("claims.Role = %s, want admin", claims.Role)
}
}
func TestJWT_Validate_ValidToken(t *testing.T) {
mgr := NewJWTManager("test-secret", 24)
token, _, err := mgr.Generate(42, "alice", "user")
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
claims, err := mgr.Validate(token)
if err != nil {
t.Fatalf("Validate() error = %v", err)
}
if claims.UserID != 42 {
t.Errorf("claims.UserID = %d, want 42", claims.UserID)
}
if claims.Username != "alice" {
t.Errorf("claims.Username = %s, want alice", claims.Username)
}
if claims.Role != "user" {
t.Errorf("claims.Role = %s, want user", claims.Role)
}
}
func TestJWT_Validate_ExpiredToken(t *testing.T) {
mgr := NewJWTManager("test-secret", 0)
token := jwtWithExpiry(time.Now().Add(-1 * time.Hour))
_, err := mgr.Validate(token)
if !errors.Is(err, ErrExpiredToken) {
t.Errorf("Validate() error = %v, want ErrExpiredToken", err)
}
}
func TestJWT_Validate_InvalidToken(t *testing.T) {
mgr := NewJWTManager("test-secret", 24)
tests := []struct {
name string
token string
}{
{"malformed token", "not.a.token"},
{"empty token", ""},
{"random string", "abcdef123456"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := mgr.Validate(tt.token)
if err == nil {
t.Error("Validate() expected error for invalid token")
}
})
}
}
func TestJWT_Validate_WrongSecret(t *testing.T) {
mgr1 := NewJWTManager("secret-one", 24)
mgr2 := NewJWTManager("secret-two", 24)
token, _, err := mgr1.Generate(1, "user", "admin")
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
_, err = mgr2.Validate(token)
if err == nil {
t.Error("Validate() expected error for token signed with different secret")
}
}
func TestPassword_HashPassword_RandomSalts(t *testing.T) {
password := "samepassword123"
hash1, err := HashPassword(password)
if err != nil {
t.Fatalf("HashPassword() error = %v", err)
}
hash2, err := HashPassword(password)
if err != nil {
t.Fatalf("HashPassword() error = %v", err)
}
if string(hash1) == string(hash2) {
t.Error("HashPassword() produced identical hashes for same password")
}
}
func TestPassword_CheckPassword_Correct(t *testing.T) {
password := "mysecretpassword"
hash, err := HashPassword(password)
if err != nil {
t.Fatalf("HashPassword() error = %v", err)
}
if !VerifyPassword(hash, password) {
t.Error("VerifyPassword() returned false for correct password")
}
}
func TestPassword_CheckPassword_Wrong(t *testing.T) {
password := "mysecretpassword"
wrongPassword := "wrongpassword"
hash, err := HashPassword(password)
if err != nil {
t.Fatalf("HashPassword() error = %v", err)
}
if VerifyPassword(hash, wrongPassword) {
t.Error("VerifyPassword() returned true for wrong password")
}
}
func TestMiddleware_RequireAuth_NoToken(t *testing.T) {
InitJWTManager("test-secret", 24)
handler := RequireAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("next handler should not be called")
}))
req := httptest.NewRequest(http.MethodGet, "/", nil)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusUnauthorized {
t.Errorf("RequireAuth() status = %d, want %d", rr.Code, http.StatusUnauthorized)
}
}
func TestMiddleware_RequireAuth_InvalidToken(t *testing.T) {
InitJWTManager("test-secret", 24)
handler := RequireAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("next handler should not be called")
}))
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(&http.Cookie{Name: CookieName, Value: "invalid-token"})
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusUnauthorized {
t.Errorf("RequireAuth() status = %d, want %d", rr.Code, http.StatusUnauthorized)
}
}
func TestMiddleware_RequireAuth_ValidToken(t *testing.T) {
secret := "test-secret"
InitJWTManager(secret, 24)
mgr := GetJWTManager()
token, _, err := mgr.Generate(99, "testuser", "admin")
if err != nil {
t.Fatalf("Generate() error = %v", err)
}
var capturedClaims *Claims
handler := RequireAuth(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
capturedClaims = GetClaims(r.Context())
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.AddCookie(&http.Cookie{Name: CookieName, Value: token})
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Errorf("RequireAuth() status = %d, want %d", rr.Code, http.StatusOK)
}
if capturedClaims == nil {
t.Fatal("RequireAuth() did not set claims in context")
}
if capturedClaims.UserID != 99 {
t.Errorf("capturedClaims.UserID = %d, want 99", capturedClaims.UserID)
}
if capturedClaims.Username != "testuser" {
t.Errorf("capturedClaims.Username = %s, want testuser", capturedClaims.Username)
}
if capturedClaims.Role != "admin" {
t.Errorf("capturedClaims.Role = %s, want admin", capturedClaims.Role)
}
}
func TestMiddleware_RequireAdmin_NonAdmin(t *testing.T) {
InitJWTManager("test-secret", 24)
handler := RequireAdmin(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("next handler should not be called for non-admin")
}))
req := httptest.NewRequest(http.MethodGet, "/", nil)
ctx := context.WithValue(req.Context(), ClaimsCtxKey, &Claims{Role: "user"})
req = req.WithContext(ctx)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusForbidden {
t.Errorf("RequireAdmin() status = %d, want %d", rr.Code, http.StatusForbidden)
}
}
func TestMiddleware_RequireAdmin_NoClaims(t *testing.T) {
InitJWTManager("test-secret", 24)
handler := RequireAdmin(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("next handler should not be called when no claims in context")
}))
req := httptest.NewRequest(http.MethodGet, "/", nil)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusForbidden {
t.Errorf("RequireAdmin() status = %d, want %d", rr.Code, http.StatusForbidden)
}
}
func TestMiddleware_RequireAdmin_Admin(t *testing.T) {
InitJWTManager("test-secret", 24)
called := false
handler := RequireAdmin(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
called = true
w.WriteHeader(http.StatusOK)
}))
req := httptest.NewRequest(http.MethodGet, "/", nil)
ctx := context.WithValue(req.Context(), ClaimsCtxKey, &Claims{Role: "admin"})
req = req.WithContext(ctx)
rr := httptest.NewRecorder()
handler.ServeHTTP(rr, req)
if rr.Code != http.StatusOK {
t.Errorf("RequireAdmin() status = %d, want %d", rr.Code, http.StatusOK)
}
if !called {
t.Error("RequireAdmin() did not call next handler for admin role")
}
}
func TestSeed_SeedsAdminUser(t *testing.T) {
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("sql.Open() error = %v", err)
}
defer db.Close()
_, err = db.Exec(`CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)`)
if err != nil {
t.Fatalf("CREATE TABLE error = %v", err)
}
err = SeedAdmin(db, "admin", "secretpassword123")
if err != nil {
t.Fatalf("SeedAdmin() error = %v", err)
}
var username, role string
err = db.QueryRow("SELECT username, role FROM users WHERE username = 'admin'").Scan(&username, &role)
if err != nil {
t.Fatalf("QueryRow() error = %v", err)
}
if username != "admin" {
t.Errorf("username = %s, want admin", username)
}
if role != "admin" {
t.Errorf("role = %s, want admin", role)
}
}
func TestSeed_Idempotent(t *testing.T) {
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("sql.Open() error = %v", err)
}
defer db.Close()
_, err = db.Exec(`CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)`)
if err != nil {
t.Fatalf("CREATE TABLE error = %v", err)
}
_, err = db.Exec("INSERT INTO users (username, password_hash, role) VALUES ('admin', 'existing-hash', 'admin')")
if err != nil {
t.Fatalf("INSERT error = %v", err)
}
err = SeedAdmin(db, "admin", "newpassword")
if err != nil {
t.Fatalf("SeedAdmin() error = %v", err)
}
var count int
err = db.QueryRow("SELECT COUNT(*) FROM users WHERE username = 'admin'").Scan(&count)
if err != nil {
t.Fatalf("QueryRow() error = %v", err)
}
if count != 1 {
t.Errorf("user count = %d, want 1 (idempotent behavior)", count)
}
}
func TestSeed_EmptyPasswordError(t *testing.T) {
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("sql.Open() error = %v", err)
}
defer db.Close()
_, err = db.Exec(`CREATE TABLE users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
)`)
if err != nil {
t.Fatalf("CREATE TABLE error = %v", err)
}
err = SeedAdmin(db, "admin", "")
if err == nil {
t.Error("SeedAdmin() expected error for empty password on first run")
}
}
func jwtWithExpiry(expiry time.Time) string {
claims := &Claims{
UserID: 1,
Username: "testuser",
Role: "admin",
RegisteredClaims: jwt.RegisteredClaims{
ExpiresAt: jwt.NewNumericDate(expiry),
IssuedAt: jwt.NewNumericDate(time.Now().Add(-2 * time.Hour)),
NotBefore: jwt.NewNumericDate(time.Now().Add(-2 * time.Hour)),
},
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
signed, _ := token.SignedString([]byte("test-secret"))
return signed
}
+4
View File
@@ -50,6 +50,10 @@ func GetClaims(ctx context.Context) *Claims {
return v.(*Claims)
}
func WithAuthAdmin(next http.Handler) http.Handler {
return RequireAdmin(RequireAuth(next))
}
var GlobalJWTManager *JWTManager
func InitJWTManager(secret string, expiryH int) {
+5 -3
View File
@@ -2,6 +2,7 @@ package auth
import (
"database/sql"
"errors"
"log/slog"
)
@@ -9,9 +10,6 @@ func SeedAdmin(db *sql.DB, username, password string) error {
if username == "" {
username = "admin"
}
if password == "" {
password = "admin"
}
var exists bool
err := db.QueryRow("SELECT EXISTS(SELECT 1 FROM users WHERE username = ?)", username).Scan(&exists)
@@ -22,6 +20,10 @@ func SeedAdmin(db *sql.DB, username, password string) error {
return nil
}
if password == "" {
return errors.New("SYNCSERVER_ADMIN_PASSWORD environment variable is required on first run")
}
hash, err := HashPassword(password)
if err != nil {
return err
+2
View File
@@ -32,6 +32,8 @@ type AuthConfig struct {
type SchedulerConfig struct {
Timezone string `yaml:"timezone" env:"SYNCSERVER_SCHEDULER_TZ" default:"UTC"`
RetentionDays int `yaml:"retention_days" env:"SYNCSERVER_RETENTION_DAYS" default:"30"`
BackupDir string `yaml:"backup_dir" env:"SYNCSERVER_BACKUP_DIR"`
BackupRetentionDays int `yaml:"backup_retention_days" env:"SYNCSERVER_BACKUP_RETENTION_DAYS" default:"7"`
}
var globalCfg *Config
+55 -8
View File
@@ -1,7 +1,9 @@
package db
import (
"crypto/sha256"
"database/sql"
"encoding/hex"
"fmt"
"os"
"path/filepath"
@@ -62,35 +64,80 @@ func (db *DB) runMigrationsInternal(mfs embedFS, migrationsRoot string) error {
if _, err := db.Exec(`
CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
checksum TEXT,
applied_at DATETIME DEFAULT CURRENT_TIMESTAMP
)
`); err != nil {
return fmt.Errorf("creating schema_migrations table: %w", err)
}
for _, name := range names {
var applied bool
row := db.QueryRow("SELECT 1 FROM schema_migrations WHERE version = ?", name)
if err := row.Scan(&applied); err == nil {
applied = true
hasChecksumCol := false
if rows, err := db.Query("PRAGMA table_info(schema_migrations)"); err == nil {
for rows.Next() {
var cid int
var cname string
rows.Scan(&cid, &cname, new(string), new(int), new(interface{}), new(int))
if cname == "checksum" {
hasChecksumCol = true
}
}
rows.Close()
}
if applied {
for _, name := range names {
if hasChecksumCol {
var storedChecksum string
row := db.QueryRow("SELECT checksum FROM schema_migrations WHERE version = ?", name)
if err := row.Scan(&storedChecksum); err == nil && storedChecksum != "" {
continue
}
} else {
var count int
row := db.QueryRow("SELECT COUNT(*) FROM schema_migrations WHERE version = ?", name)
if err := row.Scan(&count); err == nil && count > 0 {
continue
}
}
data, err := mfs.ReadFile(filepath.Join(migrationsRoot, name))
if err != nil {
return fmt.Errorf("reading migration %s: %w", name, err)
}
if _, err := db.Exec(string(data)); err != nil {
checksum := sha256.Sum256(data)
checksumHex := hex.EncodeToString(checksum[:])
tx, err := db.Begin()
if err != nil {
return fmt.Errorf("starting transaction for migration %s: %w", name, err)
}
if _, err := tx.Exec(string(data)); err != nil {
tx.Rollback()
return fmt.Errorf("applying migration %s: %w", name, err)
}
if _, err := db.Exec("INSERT INTO schema_migrations (version) VALUES (?)", name); err != nil {
if hasChecksumCol {
if _, err := tx.Exec(
"INSERT INTO schema_migrations (version, checksum) VALUES (?, ?)",
name, checksumHex,
); err != nil {
tx.Rollback()
return fmt.Errorf("recording migration %s: %w", name, err)
}
} else {
if _, err := tx.Exec(
"INSERT INTO schema_migrations (version) VALUES (?)",
name,
); err != nil {
tx.Rollback()
return fmt.Errorf("recording migration %s: %w", name, err)
}
}
if err := tx.Commit(); err != nil {
return fmt.Errorf("committing migration %s: %w", name, err)
}
}
return nil
@@ -0,0 +1,3 @@
-- 0005_machine_shutdown_command.sql
ALTER TABLE machines ADD COLUMN shutdown_command TEXT NOT NULL DEFAULT 'shutdown now';
@@ -0,0 +1,4 @@
-- 0006_progress_totals.sql
ALTER TABLE jobs ADD COLUMN total_size_bytes INTEGER DEFAULT 0;
ALTER TABLE jobs ADD COLUMN sent_bytes INTEGER DEFAULT 0;
@@ -0,0 +1,7 @@
-- Migration 0007: Add checksum column to schema_migrations.
-- This migration is idempotent and safe to re-run.
-- For existing databases (pre-1.0.54): ALTER TABLE adds the checksum column.
-- For fresh databases (1.0.54+): 0001_init.sql now creates the table with checksum column.
-- This migration handles the upgrade path only.
ALTER TABLE schema_migrations ADD COLUMN checksum TEXT DEFAULT '';
+70 -6
View File
@@ -2,6 +2,8 @@ package models
import (
"database/sql"
"fmt"
"strings"
"time"
)
@@ -15,6 +17,8 @@ type Job struct {
LogFile *string `db:"log_file" json:"log_file"`
ErrorMessage *string `db:"error_message" json:"error_message,omitempty"`
ErrorCode *string `db:"error_code" json:"error_code,omitempty"`
TotalSizeBytes int64 `db:"total_size_bytes" json:"total_size_bytes,omitempty"`
SentBytes int64 `db:"sent_bytes" json:"sent_bytes,omitempty"`
CreatedAt time.Time `db:"created_at" json:"created_at"`
}
@@ -43,9 +47,10 @@ func (r *JobRepository) GetByID(id int64) (*Job, error) {
var logFile, errMsg, errCode sql.NullString
err := r.db.QueryRow(`
SELECT id, sync_pair_id, trigger_type, status, started_at, finished_at,
log_file, error_message, error_code, created_at FROM jobs WHERE id = ?`, id).Scan(
log_file, error_message, error_code, total_size_bytes, sent_bytes, created_at
FROM jobs WHERE id = ?`, id).Scan(
&j.ID, &j.SyncPairID, &j.TriggerType, &j.Status, &started, &finished,
&logFile, &errMsg, &errCode, &j.CreatedAt)
&logFile, &errMsg, &errCode, &j.TotalSizeBytes, &j.SentBytes, &j.CreatedAt)
if err != nil {
return nil, err
}
@@ -70,7 +75,7 @@ func (r *JobRepository) GetByID(id int64) (*Job, error) {
func (r *JobRepository) GetAll(limit, offset int) ([]Job, error) {
rows, err := r.db.Query(`
SELECT id, sync_pair_id, trigger_type, status, started_at, finished_at,
log_file, error_message, error_code, created_at
log_file, error_message, error_code, total_size_bytes, sent_bytes, created_at
FROM jobs ORDER BY created_at DESC LIMIT ? OFFSET ?`,
limit, offset)
if err != nil {
@@ -84,7 +89,8 @@ func (r *JobRepository) GetAll(limit, offset int) ([]Job, error) {
var started, finished sql.NullTime
var logFile, errMsg, errCode sql.NullString
if err := rows.Scan(&j.ID, &j.SyncPairID, &j.TriggerType, &j.Status,
&started, &finished, &logFile, &errMsg, &errCode, &j.CreatedAt); err != nil {
&started, &finished, &logFile, &errMsg, &errCode,
&j.TotalSizeBytes, &j.SentBytes, &j.CreatedAt); err != nil {
return nil, err
}
if started.Valid {
@@ -144,11 +150,11 @@ func (r *JobRepository) GetRunningBySyncPair(syncPairID int64) (*Job, error) {
var logFile, errMsg, errCode sql.NullString
err := r.db.QueryRow(`
SELECT id, sync_pair_id, trigger_type, status, started_at, finished_at,
log_file, error_message, error_code, created_at FROM jobs
log_file, error_message, error_code, total_size_bytes, sent_bytes, created_at FROM jobs
WHERE sync_pair_id = ? AND status IN ('queued','waking_up','running')
ORDER BY created_at DESC LIMIT 1`, syncPairID).Scan(
&j.ID, &j.SyncPairID, &j.TriggerType, &j.Status, &started,
&j.FinishedAt, &logFile, &errMsg, &errCode, &j.CreatedAt)
&j.FinishedAt, &logFile, &errMsg, &errCode, &j.TotalSizeBytes, &j.SentBytes, &j.CreatedAt)
if err != nil {
return nil, err
}
@@ -180,3 +186,61 @@ func (r *JobRepository) DeleteFinishedBefore(before time.Time) (int64, error) {
}
return res.RowsAffected()
}
func (r *JobRepository) SetTotals(id int64, totalSize, sentBytes int64) error {
_, err := r.db.Exec(
"UPDATE jobs SET total_size_bytes = ?, sent_bytes = ? WHERE id = ?",
totalSize, sentBytes, id,
)
return err
}
func (r *JobRepository) GetByStatusAny(statuses []string) ([]Job, error) {
if len(statuses) == 0 {
return nil, nil
}
placeholders := make([]string, len(statuses))
args := make([]interface{}, len(statuses))
for i, s := range statuses {
placeholders[i] = "?"
args[i] = s
}
query := fmt.Sprintf(`
SELECT id, sync_pair_id, trigger_type, status, started_at, finished_at,
log_file, error_message, error_code, total_size_bytes, sent_bytes, created_at
FROM jobs WHERE status IN (%s)`, strings.Join(placeholders, ","))
rows, err := r.db.Query(query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
var j Job
var started, finished sql.NullTime
var logFile, errMsg, errCode sql.NullString
if err := rows.Scan(&j.ID, &j.SyncPairID, &j.TriggerType, &j.Status,
&started, &finished, &logFile, &errMsg, &errCode,
&j.TotalSizeBytes, &j.SentBytes, &j.CreatedAt); err != nil {
return nil, err
}
if started.Valid {
j.StartedAt = &started.Time
}
if finished.Valid {
j.FinishedAt = &finished.Time
}
if logFile.Valid {
j.LogFile = &logFile.String
}
if errMsg.Valid {
j.ErrorMessage = &errMsg.String
}
if errCode.Valid {
j.ErrorCode = &errCode.String
}
jobs = append(jobs, j)
}
return jobs, rows.Err()
}
+15 -2
View File
@@ -85,6 +85,19 @@ func (r *JobLogRepository) DeleteBefore(before time.Time) (int64, error) {
return res.RowsAffected()
}
func (r *JobLogRepository) TruncateKeepingHeaderTail(jobID int64, head, tail int) error {
_, err := r.db.Exec(`
DELETE FROM job_logs
WHERE job_id = ?
AND id NOT IN (
SELECT id FROM job_logs WHERE job_id = ? ORDER BY id ASC LIMIT ?
UNION ALL
SELECT id FROM job_logs WHERE job_id = ? ORDER BY id DESC LIMIT ?
)`,
jobID, jobID, head, jobID, tail)
return err
}
type JobWithStats struct {
Job
DurationSeconds *int64 `db:"duration_seconds" json:"duration_seconds"`
@@ -125,7 +138,7 @@ func (r *JobLogRepository) GetAllFiltered(limit, offset int, syncPairID *int64,
SELECT
j.id, j.sync_pair_id, j.trigger_type, j.status,
j.started_at, j.finished_at, j.log_file,
j.error_message, j.error_code, j.created_at,
j.error_message, j.error_code, j.total_size_bytes, j.sent_bytes, j.created_at,
CASE WHEN j.finished_at IS NOT NULL AND j.started_at IS NOT NULL
THEN (j.finished_at - j.started_at) ELSE NULL END as duration_seconds,
(SELECT COUNT(*) FROM job_logs WHERE job_id = j.id) as log_line_count
@@ -148,7 +161,7 @@ func (r *JobLogRepository) GetAllFiltered(limit, offset int, syncPairID *int64,
var durationSeconds sql.NullInt64
if err := rows.Scan(&j.ID, &j.SyncPairID, &j.TriggerType, &j.Status,
&started, &finished, &logFile,
&errMsg, &errCode, &j.CreatedAt,
&errMsg, &errCode, &j.TotalSizeBytes, &j.SentBytes, &j.CreatedAt,
&durationSeconds, &j.LogLineCount); err != nil {
return nil, 0, err
}
+11 -8
View File
@@ -22,6 +22,7 @@ type Machine struct {
Status string `db:"status" json:"status"`
LastSeenAt *time.Time `db:"last_seen_at" json:"last_seen_at"`
CreatedAt time.Time `db:"created_at" json:"created_at"`
ShutdownCommand string `db:"shutdown_command" json:"shutdown_command"`
}
type MachineRepository struct {
@@ -36,11 +37,12 @@ func (r *MachineRepository) Create(m *Machine) (int64, error) {
res, err := r.db.Exec(`
INSERT INTO machines (name, host, port, ssh_user, ssh_key_id, mac_address,
wol_enabled, broadcast_addr, wake_timeout_seconds, wake_check_interval_seconds,
fingerprint_confirmed, host_key_fingerprint, status)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
fingerprint_confirmed, host_key_fingerprint, status, shutdown_command)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
m.Name, m.Host, m.Port, m.SSHUser, m.SSHKeyID, m.MACAddress,
boolToInt(m.WoLEnabled), m.BroadcastAddr, m.WakeTimeoutSeconds,
m.WakeCheckIntervalSeconds, boolToInt(m.FingerprintConfirmed), m.HostKeyFingerprint, m.Status,
m.ShutdownCommand,
)
if err != nil {
return 0, err
@@ -52,7 +54,7 @@ func (r *MachineRepository) GetAll() ([]Machine, error) {
rows, err := r.db.Query(`
SELECT id, name, host, port, ssh_user, ssh_key_id, mac_address,
wol_enabled, broadcast_addr, wake_timeout_seconds, wake_check_interval_seconds,
fingerprint_confirmed, host_key_fingerprint, status, last_seen_at, created_at
fingerprint_confirmed, host_key_fingerprint, status, last_seen_at, created_at, shutdown_command
FROM machines ORDER BY name`)
if err != nil {
return nil, err
@@ -69,7 +71,7 @@ func (r *MachineRepository) GetAll() ([]Machine, error) {
err := rows.Scan(&m.ID, &m.Name, &m.Host, &m.Port, &m.SSHUser, &keyID,
&mac, &m.WoLEnabled, &bcast, &m.WakeTimeoutSeconds,
&m.WakeCheckIntervalSeconds, &m.FingerprintConfirmed,
&hostKeyFP, &m.Status, &lastSeen, &m.CreatedAt)
&hostKeyFP, &m.Status, &lastSeen, &m.CreatedAt, &m.ShutdownCommand)
if err != nil {
return nil, err
}
@@ -103,12 +105,12 @@ func (r *MachineRepository) GetByID(id int64) (*Machine, error) {
err := r.db.QueryRow(`
SELECT id, name, host, port, ssh_user, ssh_key_id, mac_address,
wol_enabled, broadcast_addr, wake_timeout_seconds, wake_check_interval_seconds,
fingerprint_confirmed, host_key_fingerprint, status, last_seen_at, created_at
fingerprint_confirmed, host_key_fingerprint, status, last_seen_at, created_at, shutdown_command
FROM machines WHERE id = ?`, id).Scan(
&m.ID, &m.Name, &m.Host, &m.Port, &m.SSHUser, &keyID,
&mac, &m.WoLEnabled, &bcast, &m.WakeTimeoutSeconds,
&m.WakeCheckIntervalSeconds, &m.FingerprintConfirmed,
&hostKeyFP, &m.Status, &lastSeen, &m.CreatedAt)
&hostKeyFP, &m.Status, &lastSeen, &m.CreatedAt, &m.ShutdownCommand)
if err != nil {
return nil, err
}
@@ -135,12 +137,13 @@ func (r *MachineRepository) Update(m *Machine) error {
_, err := r.db.Exec(`
UPDATE machines SET name=?, host=?, port=?, ssh_user=?, ssh_key_id=?,
mac_address=?, wol_enabled=?, broadcast_addr=?, wake_timeout_seconds=?,
wake_check_interval_seconds=?, fingerprint_confirmed=?, host_key_fingerprint=?, status=?, last_seen_at=?
wake_check_interval_seconds=?, fingerprint_confirmed=?, host_key_fingerprint=?, status=?, last_seen_at=?,
shutdown_command=?
WHERE id=?`,
m.Name, m.Host, m.Port, m.SSHUser, m.SSHKeyID, m.MACAddress,
boolToInt(m.WoLEnabled), m.BroadcastAddr, m.WakeTimeoutSeconds,
m.WakeCheckIntervalSeconds, boolToInt(m.FingerprintConfirmed), m.HostKeyFingerprint,
m.Status, m.LastSeenAt, m.ID,
m.Status, m.LastSeenAt, m.ShutdownCommand, m.ID,
)
return err
}
+602
View File
@@ -0,0 +1,602 @@
package models
import (
"database/sql"
"os"
"testing"
"time"
_ "modernc.org/sqlite"
)
func openTestDB(t *testing.T) *sql.DB {
db, err := sql.Open("sqlite", ":memory:")
if err != nil {
t.Fatalf("failed to open in-memory db: %v", err)
}
migrations := []string{initSchema, migration002, migration003, migration004, migration005, migration006}
for _, m := range migrations {
if _, err := db.Exec(m); err != nil {
t.Fatalf("failed to apply migration: %v", err)
}
}
return db
}
const initSchema = `
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user',
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS ssh_keys (
id INTEGER PRIMARY KEY AUTOINCREMENT,
label TEXT NOT NULL,
private_key_path TEXT NOT NULL,
public_key TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS machines (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
host TEXT NOT NULL,
port INTEGER NOT NULL DEFAULT 22,
ssh_user TEXT NOT NULL DEFAULT 'root',
ssh_key_id INTEGER REFERENCES ssh_keys(id),
mac_address TEXT,
wol_enabled INTEGER NOT NULL DEFAULT 0,
broadcast_addr TEXT,
wake_timeout_seconds INTEGER NOT NULL DEFAULT 120,
wake_check_interval_seconds INTEGER NOT NULL DEFAULT 5,
fingerprint_confirmed INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL DEFAULT 'unknown',
last_seen_at DATETIME,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS sync_pairs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
source_machine_id INTEGER REFERENCES machines(id),
source_path TEXT NOT NULL,
dest_machine_id INTEGER REFERENCES machines(id),
dest_path TEXT NOT NULL,
direction TEXT NOT NULL DEFAULT 'push',
rsync_flags TEXT NOT NULL DEFAULT '-aP',
exclude_patterns TEXT NOT NULL DEFAULT '',
enabled INTEGER NOT NULL DEFAULT 1,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS schedules (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sync_pair_id INTEGER NOT NULL REFERENCES sync_pairs(id) ON DELETE CASCADE,
cron_expr TEXT NOT NULL,
next_run_at DATETIME,
enabled INTEGER NOT NULL DEFAULT 1,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS jobs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
sync_pair_id INTEGER NOT NULL REFERENCES sync_pairs(id),
trigger_type TEXT NOT NULL DEFAULT 'manual',
status TEXT NOT NULL DEFAULT 'queued',
started_at DATETIME,
finished_at DATETIME,
log_file TEXT,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS job_logs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
job_id INTEGER NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
stream TEXT NOT NULL,
content TEXT NOT NULL,
timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
);
CREATE TABLE IF NOT EXISTS sessions (
id TEXT PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL,
expires_at DATETIME NOT NULL,
revoked INTEGER NOT NULL DEFAULT 0,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
`
const migration002 = `
CREATE INDEX IF NOT EXISTS idx_job_logs_job_id ON job_logs(job_id);
CREATE INDEX IF NOT EXISTS idx_jobs_status_created ON jobs(status, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_jobs_sync_pair_id ON jobs(sync_pair_id);
CREATE TABLE IF NOT EXISTS cleanup_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
deleted_before DATETIME NOT NULL,
logs_purged INTEGER NOT NULL DEFAULT 0,
jobs_purged INTEGER NOT NULL DEFAULT 0,
executed_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
`
const migration003 = `
ALTER TABLE jobs ADD COLUMN error_message TEXT;
ALTER TABLE jobs ADD COLUMN error_code TEXT;
`
const migration004 = `
ALTER TABLE machines ADD COLUMN host_key_fingerprint TEXT;
`
const migration005 = `
ALTER TABLE machines ADD COLUMN shutdown_command TEXT NOT NULL DEFAULT 'shutdown now';
`
const migration006 = `
ALTER TABLE jobs ADD COLUMN total_size_bytes INTEGER DEFAULT 0;
ALTER TABLE jobs ADD COLUMN sent_bytes INTEGER DEFAULT 0;
`
func TestJobRepository_CreateAndGetByID(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id, err := repo.Create(1, "manual", "queued")
if err != nil {
t.Fatalf("Create failed: %v", err)
}
job, err := repo.GetByID(id)
if err != nil {
t.Fatalf("GetByID failed: %v", err)
}
if job.ID != id {
t.Errorf("expected ID %d, got %d", id, job.ID)
}
if job.SyncPairID != 1 {
t.Errorf("expected SyncPairID 1, got %d", job.SyncPairID)
}
if job.TriggerType != "manual" {
t.Errorf("expected trigger_type 'manual', got %q", job.TriggerType)
}
if job.Status != "queued" {
t.Errorf("expected status 'queued', got %q", job.Status)
}
}
func TestJobRepository_UpdateStatus_SetsStartedAt(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id, _ := repo.Create(1, "manual", "queued")
err := repo.UpdateStatus(id, "running")
if err != nil {
t.Fatalf("UpdateStatus failed: %v", err)
}
job, _ := repo.GetByID(id)
if job.StartedAt == nil {
t.Fatal("expected StartedAt to be set for 'running' status")
}
if job.Status != "running" {
t.Errorf("expected status 'running', got %q", job.Status)
}
}
func TestJobRepository_UpdateStatus_SetsFinishedAt(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id, _ := repo.Create(1, "manual", "queued")
repo.UpdateStatus(id, "running")
err := repo.UpdateStatus(id, "success")
if err != nil {
t.Fatalf("UpdateStatus failed: %v", err)
}
job, _ := repo.GetByID(id)
if job.FinishedAt == nil {
t.Fatal("expected FinishedAt to be set for 'success' status")
}
if job.Status != "success" {
t.Errorf("expected status 'success', got %q", job.Status)
}
}
func TestJobRepository_UpdateStatus_Failed(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id, _ := repo.Create(1, "manual", "queued")
repo.UpdateStatus(id, "running")
err := repo.UpdateStatus(id, "failed")
if err != nil {
t.Fatalf("UpdateStatus failed: %v", err)
}
job, _ := repo.GetByID(id)
if job.FinishedAt == nil {
t.Fatal("expected FinishedAt to be set for 'failed' status")
}
}
func TestJobRepository_SetError(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id, _ := repo.Create(1, "manual", "queued")
err := repo.SetError(id, "EIO", "disk read failed")
if err != nil {
t.Fatalf("SetError failed: %v", err)
}
job, _ := repo.GetByID(id)
if job.ErrorCode == nil || *job.ErrorCode != "EIO" {
t.Errorf("expected error_code 'EIO', got %v", job.ErrorCode)
}
if job.ErrorMessage == nil || *job.ErrorMessage != "disk read failed" {
t.Errorf("expected error_message 'disk read failed', got %v", job.ErrorMessage)
}
}
func TestJobRepository_GetByStatusAny(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
_, _ = repo.Create(1, "manual", "queued")
id2, _ := repo.Create(1, "manual", "running")
_, _ = repo.Create(1, "manual", "success")
jobs, err := repo.GetByStatusAny([]string{"running"})
if err != nil {
t.Fatalf("GetByStatusAny failed: %v", err)
}
if len(jobs) != 1 {
t.Fatalf("expected 1 job, got %d", len(jobs))
}
if jobs[0].ID != id2 {
t.Errorf("expected job ID %d, got %d", id2, jobs[0].ID)
}
jobs, _ = repo.GetByStatusAny([]string{"queued", "running"})
if len(jobs) != 2 {
t.Fatalf("expected 2 jobs, got %d", len(jobs))
}
}
func TestJobRepository_DeleteFinishedBefore(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id1, _ := repo.Create(1, "manual", "queued")
repo.UpdateStatus(id1, "running")
repo.UpdateStatus(id1, "success")
db.Exec("UPDATE jobs SET finished_at = datetime('2020-01-01 00:00:00') WHERE id = ?", id1)
id2, _ := repo.Create(1, "manual", "queued")
repo.UpdateStatus(id2, "running")
repo.UpdateStatus(id2, "success")
cutoff := time.Now()
deleted, err := repo.DeleteFinishedBefore(cutoff)
if err != nil {
t.Fatalf("DeleteFinishedBefore failed: %v", err)
}
if deleted != 1 {
t.Errorf("expected 1 deleted, got %d", deleted)
}
_, err = repo.GetByID(id1)
if err != sql.ErrNoRows {
t.Errorf("expected id1 to be deleted")
}
_, err = repo.GetByID(id2)
if err != nil {
t.Errorf("expected id2 to still exist")
}
}
func TestJobRepository_SetTotals(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewJobRepository(db)
id, _ := repo.Create(1, "manual", "queued")
err := repo.SetTotals(id, 1024, 512)
if err != nil {
t.Fatalf("SetTotals failed: %v", err)
}
job, _ := repo.GetByID(id)
if job.TotalSizeBytes != 1024 {
t.Errorf("expected TotalSizeBytes 1024, got %d", job.TotalSizeBytes)
}
if job.SentBytes != 512 {
t.Errorf("expected SentBytes 512, got %d", job.SentBytes)
}
}
func TestMachineRepository_CreateAndGetByID(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewMachineRepository(db)
mac := "AA:BB:CC:DD:EE:FF"
bcast := "192.168.1.255"
WolEnabled := true
WolTimeout := 300
machine := &Machine{
Name: "test-machine",
Host: "192.168.1.10",
Port: 22,
SSHUser: "admin",
MACAddress: &mac,
WoLEnabled: WolEnabled,
BroadcastAddr: &bcast,
WakeTimeoutSeconds: WolTimeout,
WakeCheckIntervalSeconds: 10,
Status: "unknown",
ShutdownCommand: "shutdown -h now",
}
id, err := repo.Create(machine)
if err != nil {
t.Fatalf("Create failed: %v", err)
}
retrieved, err := repo.GetByID(id)
if err != nil {
t.Fatalf("GetByID failed: %v", err)
}
if retrieved.Name != "test-machine" {
t.Errorf("expected name 'test-machine', got %q", retrieved.Name)
}
if retrieved.Host != "192.168.1.10" {
t.Errorf("expected host '192.168.1.10', got %q", retrieved.Host)
}
if retrieved.MACAddress == nil || *retrieved.MACAddress != mac {
t.Errorf("expected MAC %q, got %v", mac, retrieved.MACAddress)
}
if !retrieved.WoLEnabled {
t.Error("expected WoLEnabled to be true")
}
if retrieved.BroadcastAddr == nil || *retrieved.BroadcastAddr != bcast {
t.Errorf("expected broadcast %q, got %v", bcast, retrieved.BroadcastAddr)
}
if retrieved.WakeTimeoutSeconds != WolTimeout {
t.Errorf("expected wake timeout %d, got %d", WolTimeout, retrieved.WakeTimeoutSeconds)
}
if retrieved.ShutdownCommand != "shutdown -h now" {
t.Errorf("expected shutdown command 'shutdown -h now', got %q", retrieved.ShutdownCommand)
}
}
func TestMachineRepository_UpdateStatus(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewMachineRepository(db)
id, _ := repo.Create(&Machine{Name: "test", Host: "192.168.1.1", Status: "unknown"})
err := repo.UpdateStatus(id, "online")
if err != nil {
t.Fatalf("UpdateStatus failed: %v", err)
}
machine, _ := repo.GetByID(id)
if machine.Status != "online" {
t.Errorf("expected status 'online', got %q", machine.Status)
}
if machine.LastSeenAt == nil {
t.Error("expected LastSeenAt to be set")
}
}
func TestMachineRepository_GetAll(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewMachineRepository(db)
repo.Create(&Machine{Name: "machine-a", Host: "192.168.1.1", Status: "online"})
repo.Create(&Machine{Name: "machine-b", Host: "192.168.1.2", Status: "offline"})
machines, err := repo.GetAll()
if err != nil {
t.Fatalf("GetAll failed: %v", err)
}
if len(machines) != 2 {
t.Errorf("expected 2 machines, got %d", len(machines))
}
}
func TestSyncPairRepository_CreateAndGetByID(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewSyncPairRepository(db)
sp := &SyncPair{
Name: "backup-data",
SourcePath: "/data",
DestPath: "/backup",
Direction: "push",
RsyncFlags: "-aP --delete",
ExcludePatterns: "*.tmp\n*.log",
Enabled: true,
}
id, err := repo.Create(sp)
if err != nil {
t.Fatalf("Create failed: %v", err)
}
retrieved, err := repo.GetByID(id)
if err != nil {
t.Fatalf("GetByID failed: %v", err)
}
if retrieved.Name != "backup-data" {
t.Errorf("expected name 'backup-data', got %q", retrieved.Name)
}
if retrieved.ExcludePatterns != "*.tmp\n*.log" {
t.Errorf("expected exclude patterns '*.tmp\\n*.log', got %q", retrieved.ExcludePatterns)
}
}
func TestSyncPairRepository_ExcludePatternsList(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewSyncPairRepository(db)
sp := &SyncPair{
Name: "test-pair",
SourcePath: "/src",
DestPath: "/dst",
ExcludePatterns: "*.tmp\n *.log\n \n*.bak",
}
id, _ := repo.Create(sp)
retrieved, _ := repo.GetByID(id)
patterns := retrieved.ExcludePatternsList()
if len(patterns) != 3 {
t.Fatalf("expected 3 patterns, got %d: %v", len(patterns), patterns)
}
if patterns[0] != "*.tmp" {
t.Errorf("expected first pattern '*.tmp', got %q", patterns[0])
}
if patterns[1] != "*.log" {
t.Errorf("expected second pattern '*.log', got %q", patterns[1])
}
if patterns[2] != "*.bak" {
t.Errorf("expected third pattern '*.bak', got %q", patterns[2])
}
}
func TestSyncPairRepository_GetAll(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewSyncPairRepository(db)
repo.Create(&SyncPair{Name: "pair-a", SourcePath: "/a", DestPath: "/b"})
repo.Create(&SyncPair{Name: "pair-b", SourcePath: "/c", DestPath: "/d"})
pairs, err := repo.GetAll()
if err != nil {
t.Fatalf("GetAll failed: %v", err)
}
if len(pairs) != 2 {
t.Errorf("expected 2 pairs, got %d", len(pairs))
}
}
func TestScheduleRepository_CreateAndGetByID(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewScheduleRepository(db)
nextRun := time.Now().Add(1 * time.Hour)
sched := &Schedule{
SyncPairID: 1,
CronExpr: "0 0 * * *",
NextRunAt: &nextRun,
Enabled: true,
}
id, err := repo.Create(sched)
if err != nil {
t.Fatalf("Create failed: %v", err)
}
retrieved, err := repo.GetByID(id)
if err != nil {
t.Fatalf("GetByID failed: %v", err)
}
if retrieved.CronExpr != "0 0 * * *" {
t.Errorf("expected cron '0 0 * * *', got %q", retrieved.CronExpr)
}
if !retrieved.Enabled {
t.Error("expected enabled to be true")
}
}
func TestScheduleRepository_UpdateEnabled(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewScheduleRepository(db)
nextRun := time.Now().Add(1 * time.Hour)
id, _ := repo.Create(&Schedule{SyncPairID: 1, CronExpr: "0 0 * * *", NextRunAt: &nextRun, Enabled: true})
sched, _ := repo.GetByID(id)
sched.Enabled = false
err := repo.Update(sched)
if err != nil {
t.Fatalf("Update failed: %v", err)
}
retrieved, _ := repo.GetByID(id)
if retrieved.Enabled {
t.Error("expected enabled to be false after update")
}
}
func TestScheduleRepository_GetEnabledDue(t *testing.T) {
db := openTestDB(t)
defer db.Close()
repo := NewScheduleRepository(db)
pastRun := time.Now().Add(-1 * time.Hour)
futureRun := time.Now().Add(1 * time.Hour)
_, _ = repo.Create(&Schedule{SyncPairID: 1, CronExpr: "0 0 * * *", NextRunAt: &pastRun, Enabled: true})
_, _ = repo.Create(&Schedule{SyncPairID: 2, CronExpr: "0 0 * * *", NextRunAt: &futureRun, Enabled: true})
_, _ = repo.Create(&Schedule{SyncPairID: 3, CronExpr: "0 0 * * *", NextRunAt: &pastRun, Enabled: false})
due, err := repo.GetEnabledDue(time.Now())
if err != nil {
t.Fatalf("GetEnabledDue failed: %v", err)
}
if len(due) != 1 {
t.Errorf("expected 1 due schedule, got %d", len(due))
}
if due[0].SyncPairID != 1 {
t.Errorf("expected sync_pair_id 1, got %d", due[0].SyncPairID)
}
}
func TestMain(m *testing.M) {
os.Exit(m.Run())
}
+82
View File
@@ -3,7 +3,12 @@ package scheduler
import (
"context"
"database/sql"
"fmt"
"log/slog"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"time"
@@ -124,6 +129,10 @@ func (s *Scheduler) cleanup() {
}
before := time.Now().AddDate(0, 0, -retentionDays)
if s.cfg.Scheduler.BackupDir != "" {
s.backupDB(before)
}
logRepo := models.NewJobLogRepository(s.db)
jobRepo := models.NewJobRepository(s.db)
@@ -141,5 +150,78 @@ func (s *Scheduler) cleanup() {
if deletedLogs > 0 || deletedJobs > 0 {
slog.Info("cleanup: purged old records", "logs_deleted", deletedLogs, "jobs_deleted", deletedJobs, "before", before.Format("2006-01-02"))
s.purgeJobLogFiles(before)
}
s.purgeOldBackups()
}
func (s *Scheduler) backupDB(before time.Time) {
backupDir := s.cfg.Scheduler.BackupDir
if backupDir == "" {
return
}
if err := os.MkdirAll(backupDir, 0700); err != nil {
slog.Error("cleanup: failed to create backup dir", "error", err)
return
}
ts := time.Now().UTC().Format("20060102-150405")
backupPath := filepath.Join(backupDir, fmt.Sprintf("syncserver-%s.db", ts))
if _, err := s.db.Exec(fmt.Sprintf("VACUUM INTO '%s'", backupPath)); err != nil {
slog.Error("cleanup: failed to vacuum into backup", "path", backupPath, "error", err)
return
}
slog.Info("cleanup: database backup created", "path", backupPath)
}
func (s *Scheduler) purgeJobLogFiles(before time.Time) {
logsDir := s.cfg.LogsDir()
entries, err := os.ReadDir(logsDir)
if err != nil {
return
}
for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".log") {
continue
}
jobID := strings.TrimSuffix(entry.Name(), ".log")
id, err := strconv.ParseInt(jobID, 10, 64)
if err != nil {
continue
}
jobRepo := models.NewJobRepository(s.db)
job, err := jobRepo.GetByID(id)
if err != nil || job == nil {
os.Remove(filepath.Join(logsDir, entry.Name()))
continue
}
if job.FinishedAt != nil && job.FinishedAt.Before(before) {
os.Remove(filepath.Join(logsDir, entry.Name()))
}
}
}
func (s *Scheduler) purgeOldBackups() {
backupDir := s.cfg.Scheduler.BackupDir
retention := s.cfg.Scheduler.BackupRetentionDays
if backupDir == "" || retention <= 0 {
return
}
cutoff := time.Now().AddDate(0, 0, -retention)
entries, err := os.ReadDir(backupDir)
if err != nil {
return
}
for _, entry := range entries {
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".db") {
continue
}
info, err := entry.Info()
if err != nil {
continue
}
if info.ModTime().Before(cutoff) {
os.Remove(filepath.Join(backupDir, entry.Name()))
}
}
}
+167
View File
@@ -0,0 +1,167 @@
package sshmanager
import (
"bytes"
"context"
"fmt"
"log/slog"
"os"
"path/filepath"
"time"
"golang.org/x/crypto/ssh"
)
type DeployKey struct {
LocalPath string
RemotePath string
Mode uint32
}
type DeployResult struct {
Success bool `json:"success"`
Messages []string `json:"messages"`
Errors []string `json:"errors"`
}
func DeployKeysToMachine(ctx context.Context, serverKeyPath string, host string, port int, user string, keys []DeployKey, knownHostsHosts []string, sshDir string) (*DeployResult, error) {
result := &DeployResult{Success: true, Messages: []string{}, Errors: []string{}}
addr := fmt.Sprintf("%s:%d", host, port)
keyData, err := os.ReadFile(serverKeyPath)
if err != nil {
return nil, fmt.Errorf("reading server key: %w", err)
}
signer, err := ssh.ParsePrivateKey(keyData)
if err != nil {
return nil, fmt.Errorf("parsing server key: %w", err)
}
knownHostsPath := filepath.Join(sshDir, "known_hosts")
hostKeyCallback, err := NewKnownHostsCallback(knownHostsPath, true)
if err != nil {
slog.Warn("deploy keys: creating host key callback failed, ignoring hosts", "error", err)
hostKeyCallback = ssh.InsecureIgnoreHostKey()
}
cfg := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{ssh.PublicKeys(signer)},
HostKeyCallback: hostKeyCallback,
Timeout: 10 * time.Second,
}
ctx, cancel := context.WithTimeout(ctx, 60*time.Second)
defer cancel()
conn, err := ssh.Dial("tcp", addr, cfg)
if err != nil {
slog.Warn("deploy keys: SSH dial failed", "host", addr, "error", err)
return &DeployResult{Success: false, Messages: []string{}, Errors: []string{fmt.Sprintf("connecting to %s: %v", addr, err)}}, nil
}
defer conn.Close()
remoteSSHDir := sshDir
remoteKeysDir := filepath.Join(remoteSSHDir, "keys")
session, err := conn.NewSession()
if err != nil {
return nil, fmt.Errorf("creating session: %w", err)
}
defer session.Close()
var stdout, stderr bytes.Buffer
session.Stdout = &stdout
session.Stderr = &stderr
if err := session.Run(fmt.Sprintf("mkdir -p %s && chmod 700 %s", remoteKeysDir, remoteKeysDir)); err != nil {
return nil, fmt.Errorf("creating remote ssh dir: %s %w", stderr.String(), err)
}
result.Messages = append(result.Messages, fmt.Sprintf("Created %s on %s", remoteKeysDir, host))
for _, k := range keys {
keyContent, err := os.ReadFile(k.LocalPath)
if err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("reading local key %s: %v", k.LocalPath, err))
result.Success = false
continue
}
mode := k.Mode
if mode == 0 {
mode = 0600
}
sess2, err := conn.NewSession()
if err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("session for key upload: %v", err))
result.Success = false
continue
}
defer sess2.Close()
sess2.Stdout = &stdout
sess2.Stderr = &stderr
slog.Debug("deploy: uploading key", "local", k.LocalPath, "remote", k.RemotePath, "host", host)
cmd := fmt.Sprintf("cat > %s && chmod 0%o %s", k.RemotePath, mode, k.RemotePath)
stdin, err := sess2.StdinPipe()
if err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("stdin pipe for %s: %v", k.RemotePath, err))
result.Success = false
sess2.Close()
continue
}
if err := sess2.Start(cmd); err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("starting command for %s: %v", k.RemotePath, err))
result.Success = false
sess2.Close()
continue
}
_, err = stdin.Write(keyContent)
if err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("writing key %s: %v", k.RemotePath, err))
result.Success = false
stdin.Close()
sess2.Close()
continue
}
stdin.Close()
if err := sess2.Wait(); err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("uploading %s: %v (stderr: %s)", k.RemotePath, err, stderr.String()))
result.Success = false
continue
}
result.Messages = append(result.Messages, fmt.Sprintf("Uploaded %s to %s:%s", filepath.Base(k.LocalPath), host, k.RemotePath))
}
if len(knownHostsHosts) > 0 {
for _, khHost := range knownHostsHosts {
session2, err := conn.NewSession()
if err != nil {
result.Errors = append(result.Errors, fmt.Sprintf("session for ssh-keyscan %s: %v", khHost, err))
continue
}
session2.Stdout = &stdout
session2.Stderr = &stderr
err = session2.Run(fmt.Sprintf("ssh-keyscan -H -p %s 2>/dev/null >> %s/known_hosts", khHost, remoteSSHDir))
session2.Close()
if err != nil {
slog.Warn("deploy: ssh-keyscan failed", "host", khHost, "error", err)
result.Errors = append(result.Errors, fmt.Sprintf("ssh-keyscan %s: %v (stderr: %s)", khHost, err, stderr.String()))
} else {
result.Messages = append(result.Messages, fmt.Sprintf("Populated known_hosts with %s", khHost))
}
}
}
slog.Info("deploy keys result", "host", host, "success", result.Success, "messages", len(result.Messages), "errors", len(result.Errors))
return result, nil
}
+20
View File
@@ -8,6 +8,7 @@ import (
"strings"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
type KnownHost struct {
@@ -17,6 +18,23 @@ type KnownHost struct {
Fingerprint string
}
func NewKnownHostsCallback(knownHostsPath string, strictHostKeyChecking bool) (ssh.HostKeyCallback, error) {
if !strictHostKeyChecking {
return ssh.InsecureIgnoreHostKey(), nil
}
if knownHostsPath == "" {
return ssh.InsecureIgnoreHostKey(), nil
}
_, err := os.Stat(knownHostsPath)
if os.IsNotExist(err) {
return ssh.InsecureIgnoreHostKey(), nil
}
if err != nil {
return nil, err
}
return knownhosts.New(knownHostsPath)
}
func EnsureKnownHosts(sshDir string) (string, error) {
path := filepath.Join(sshDir, "known_hosts")
f, err := os.OpenFile(path, os.O_CREATE|os.O_RDONLY, 0644)
@@ -27,6 +45,8 @@ func EnsureKnownHosts(sshDir string) (string, error) {
return path, nil
}
// AddKnownHost stores the host key in standard ssh known_hosts format (hostname keytype base64key).
// NOTE: Existing entries in known_hosts may need to be regenerated if they were stored in a different format.
func AddKnownHost(sshDir, host string, port int, keyData []byte) error {
path := filepath.Join(sshDir, "known_hosts")
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
+82
View File
@@ -0,0 +1,82 @@
package sshmanager
import (
"bytes"
"context"
"fmt"
"strings"
"time"
)
type ShutdownResult struct {
Success bool
Output string
Error string
}
func IsShutdownCommand(cmd string) bool {
c := strings.TrimSpace(strings.ToLower(cmd))
for _, p := range []string{
"shutdown", "poweroff", "halt", "reboot",
"sudo shutdown", "sudo poweroff", "sudo halt", "sudo reboot",
"systemctl poweroff", "systemctl halt", "systemctl reboot",
} {
if strings.HasPrefix(c, p) {
return true
}
}
return false
}
func RunRemoteCommand(ctx context.Context, host string, port int, user, privKeyPath, knownHostsPath string, strictHostKeyChecking bool, command string) (*ShutdownResult, error) {
conn, _, _, err := dialSSH(ctx, host, port, user, privKeyPath, knownHostsPath, strictHostKeyChecking)
if err != nil {
return &ShutdownResult{Success: false, Error: err.Error()}, nil
}
defer conn.Close()
session, err := conn.NewSession()
if err != nil {
return &ShutdownResult{Success: false, Error: fmt.Sprintf("session: %v", err)}, nil
}
defer session.Close()
var stdout, stderr bytes.Buffer
session.Stdout = &stdout
session.Stderr = &stderr
effectiveCmd := command
if IsShutdownCommand(command) {
effectiveCmd = fmt.Sprintf(
"nohup %s >/dev/null 2>&1 </dev/null & sleep 1; echo shutdown_initiated",
command,
)
}
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
done := make(chan error, 1)
go func() {
done <- session.Run(effectiveCmd)
}()
select {
case err := <-done:
if err != nil {
return &ShutdownResult{
Success: false,
Error: fmt.Sprintf("exec: %v, stderr: %s", err, stderr.String()),
}, nil
}
return &ShutdownResult{
Success: true,
Output: stdout.String(),
}, nil
case <-ctx.Done():
return &ShutdownResult{
Success: false,
Error: "command timed out after 15 seconds",
}, nil
}
}
+30 -28
View File
@@ -8,7 +8,6 @@ import (
"fmt"
"net"
"os"
"path/filepath"
"strings"
"time"
@@ -22,40 +21,35 @@ type ConnResult struct {
Fingerprint string
}
func TestSSHConnection(ctx context.Context, host string, port int, user, privKeyPath, knownHostsPath string, strictHostKeyChecking bool) (*ConnResult, error) {
func dialSSH(ctx context.Context, host string, port int, user, privKeyPath, knownHostsPath string, strictHostKeyChecking bool) (*ssh.Client, string, ssh.PublicKey, error) {
addr := fmt.Sprintf("%s:%d", host, port)
auths := []ssh.AuthMethod{}
if privKeyPath != "" {
key, err := os.ReadFile(privKeyPath)
if err != nil {
return nil, fmt.Errorf("reading private key: %w", err)
return nil, "", nil, fmt.Errorf("reading private key: %w", err)
}
signer, err := ssh.ParsePrivateKey(key)
if err != nil {
return nil, fmt.Errorf("parsing private key: %w", err)
return nil, "", nil, fmt.Errorf("parsing private key: %w", err)
}
auths = append(auths, ssh.PublicKeys(signer))
}
var capturedFingerprint string
var capturedPubKey ssh.PublicKey
callback, err := NewKnownHostsCallback(knownHostsPath, strictHostKeyChecking)
if err != nil {
return nil, "", nil, fmt.Errorf("creating host key callback: %w", err)
}
hostKeyCallback := func(hostname string, remote net.Addr, key ssh.PublicKey) error {
h := sha256.Sum256(key.Marshal())
capturedFingerprint = "SHA256:" + base64.RawStdEncoding.EncodeToString(h[:])
if strictHostKeyChecking && knownHostsPath != "" {
kh, err := GetKnownHost(filepath.Dir(knownHostsPath), host, port)
if err != nil {
return fmt.Errorf("checking known_hosts: %w", err)
}
if kh == nil {
return fmt.Errorf("host key not found in known_hosts: %s", hostname)
}
wantFP := kh.Fingerprint
if capturedFingerprint != wantFP {
return fmt.Errorf("host key mismatch: got %s, want %s", capturedFingerprint, wantFP)
}
}
return nil
capturedPubKey = key
return callback(hostname, remote, key)
}
cfg := &ssh.ClientConfig{
@@ -71,23 +65,27 @@ func TestSSHConnection(ctx context.Context, host string, port int, user, privKey
conn, err := ssh.Dial("tcp", addr, cfg)
if err != nil {
if strings.Contains(err.Error(), "known_hosts") || strings.Contains(err.Error(), "host key") {
return &ConnResult{
Success: false,
Error: fmt.Sprintf("host key verification failed: %v", err),
Fingerprint: capturedFingerprint,
}, nil
return nil, capturedFingerprint, capturedPubKey, fmt.Errorf("host key verification failed: %v", err)
}
return nil, capturedFingerprint, capturedPubKey, fmt.Errorf("connection failed: %v", err)
}
return conn, capturedFingerprint, capturedPubKey, nil
}
func TestSSHConnection(ctx context.Context, host string, port int, user, privKeyPath, knownHostsPath string, strictHostKeyChecking bool) (*ConnResult, error) {
conn, fingerprint, _, err := dialSSH(ctx, host, port, user, privKeyPath, knownHostsPath, strictHostKeyChecking)
if err != nil {
return &ConnResult{
Success: false,
Error: fmt.Sprintf("connection failed: %v", err),
Fingerprint: capturedFingerprint,
Error: err.Error(),
Fingerprint: fingerprint,
}, nil
}
defer conn.Close()
session, err := conn.NewSession()
if err != nil {
return &ConnResult{Success: false, Error: fmt.Sprintf("session: %v", err), Fingerprint: capturedFingerprint}, nil
return &ConnResult{Success: false, Error: fmt.Sprintf("session: %v", err), Fingerprint: fingerprint}, nil
}
defer session.Close()
@@ -99,13 +97,17 @@ func TestSSHConnection(ctx context.Context, host string, port int, user, privKey
return &ConnResult{
Success: false,
Error: fmt.Sprintf("exec: %v, stderr: %s", err, stderr.String()),
Fingerprint: capturedFingerprint,
Fingerprint: fingerprint,
}, nil
}
return &ConnResult{
Success: true,
Output: stdout.String(),
Fingerprint: capturedFingerprint,
Fingerprint: fingerprint,
}, nil
}
func ConnectForApproval(ctx context.Context, host string, port int, user, privKeyPath, knownHostsPath string) (*ssh.Client, string, ssh.PublicKey, error) {
return dialSSH(ctx, host, port, user, privKeyPath, knownHostsPath, false)
}
+160 -28
View File
@@ -25,16 +25,24 @@ type Engine struct {
mu sync.RWMutex
stopped bool
lastProbeAt atomic.Int64
jobsWG sync.WaitGroup
stopCh chan struct{}
jobsTotal map[string]int64
jobsTotalMu sync.Mutex
}
type Event struct {
Type string
JobID int64
MachineID int64
Key string
Value string
Line string
Stream string
Type string `json:"type"`
JobID int64 `json:"job_id"`
MachineID int64 `json:"machine_id"`
Key string `json:"key,omitempty"`
Value string `json:"value,omitempty"`
Line string `json:"line,omitempty"`
Stream string `json:"stream,omitempty"`
Progress *ProgressFields `json:"progress,omitempty"`
TotalBytes int64 `json:"total_bytes,omitempty"`
SentBytes int64 `json:"sent_bytes,omitempty"`
}
func New(database interface{ SQLDB() *sql.DB }, cfg *config.Config) *Engine {
@@ -43,12 +51,56 @@ func New(database interface{ SQLDB() *sql.DB }, cfg *config.Config) *Engine {
cfg: cfg,
queue: NewQueue(),
eventBus: NewEventBus(200),
stopCh: make(chan struct{}),
jobsTotal: map[string]int64{
"success": 0,
"failed": 0,
"cancelled": 0,
},
}
return e
}
func (e *Engine) Start() {}
func (e *Engine) Stop() {}
func (e *Engine) Start() {
e.recoverOrphanedJobs()
slog.Info("engine started")
}
func (e *Engine) Stop() {
e.mu.Lock()
if e.stopped {
e.mu.Unlock()
return
}
e.stopped = true
e.mu.Unlock()
close(e.stopCh)
runningIDs := e.queue.RunningJobs()
for _, id := range runningIDs {
e.queue.Cancel(id, false)
}
e.jobsWG.Wait()
slog.Info("engine stopped")
}
func (e *Engine) recoverOrphanedJobs() {
jobRepo := models.NewJobRepository(e.db)
jobs, err := jobRepo.GetByStatusAny([]string{"queued", "waking_up", "running"})
if err != nil {
slog.Warn("failed to recover orphaned jobs", "error", err)
return
}
for _, j := range jobs {
slog.Warn("recovered orphaned job, marking as failed",
"job_id", j.ID, "pair_id", j.SyncPairID, "status", j.Status)
jobRepo.UpdateStatus(j.ID, "failed")
jobRepo.SetError(j.ID, "crash_recovery",
fmt.Sprintf("job was %s when server shut down unexpectedly", j.Status))
}
}
func (e *Engine) SubscribeJob(jobID int64) (chan Event, func()) {
return e.eventBus.Subscribe(jobID)
@@ -81,9 +133,12 @@ func (e *Engine) wakeMachine(ctx context.Context, m *models.Machine) {
}
func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
if e.queue.IsRunning(pairID) {
existingJobID, _ := e.queue.GetJobID(pairID)
return fmt.Errorf("%w: job %d is already running", ErrAlreadyRunning, existingJobID)
if e.queue.IsRunning(jobID) {
return fmt.Errorf("%w: job %d is already running", ErrAlreadyRunning, jobID)
}
if existingJobID, exists := e.queue.GetByPair(pairID); exists {
return fmt.Errorf("%w: job %d is already running for this sync pair", ErrAlreadyRunning, existingJobID)
}
jobCtx, cancel := context.WithCancel(ctx)
@@ -96,7 +151,10 @@ func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
if enqueueErr != nil {
return enqueueErr
}
defer e.queue.Dequeue(pairID)
defer e.queue.Dequeue(jobID)
e.jobsWG.Add(1)
defer e.jobsWG.Done()
pairRepo := models.NewSyncPairRepository(e.db)
pair, err := pairRepo.GetByID(pairID)
@@ -243,7 +301,17 @@ func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
}
}
var lastFileName string
onLine := func(stream, line string) {
if stream == "stdout" && isProgressOnlyLine(line) {
if p := parseProgressFields(line); p != nil {
e.emit(Event{Type: "progress", JobID: jobID, Line: line, Stream: stream, Progress: p, Value: lastFileName})
}
return
}
if stream == "stdout" && isFileNameLine(line) {
lastFileName = line
}
f, _ := os.OpenFile(logPath, os.O_APPEND|os.O_WRONLY, 0644)
if f != nil {
fmt.Fprintln(f, line)
@@ -264,25 +332,33 @@ func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
destPrivKeyPath, err := e.resolveSSHKey(dstMachine)
if err != nil {
slog.Warn("failed to resolve destination SSH key, using server key", "error", err)
destPrivKeyPath = filepath.Join(e.cfg.SSHDir(), "id_ed25519")
destPrivKeyPath = ""
}
destPrivKeyBytes, err := os.ReadFile(destPrivKeyPath)
if err != nil {
slog.Warn("failed to read destination SSH key, using empty", "error", err)
destPrivKeyBytes = []byte{}
}
result, err = runner.RunRemote(jobCtx, cfg, RemoteMachine{
result, err = runner.RunRemote(jobCtx, cfg,
&MachineKeys{
Host: srcMachine.Host,
Port: srcMachine.Port,
SSHUser: srcMachine.SSHUser,
PrivKey: privKey,
DestPrivKey: string(destPrivKeyBytes),
}, onLine)
},
&MachineKeys{
Host: dstMachine.Host,
Port: dstMachine.Port,
SSHUser: dstMachine.SSHUser,
PrivKey: destPrivKeyPath,
},
destPrivKeyPath,
onLine)
} else {
result, err = runner.Run(jobCtx, cfg, onLine)
}
flush()
stats := result.Stats
if stats != nil && stats.TotalSize > 0 {
e.emit(Event{Type: "progress_total", JobID: jobID, TotalBytes: stats.TotalSize, SentBytes: stats.SentBytes})
}
if err != nil {
if jobCtx.Err() != nil {
code := "cancelled_shutdown"
@@ -294,6 +370,7 @@ func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
e.setJobError(jobID, code, msg)
e.setJobStatus(jobID, "cancelled")
e.emit(Event{Type: "status", JobID: jobID, Key: "status", Value: "cancelled", Line: msg})
e.persistAndClose(jobID)
return jobCtx.Err()
}
e.setJobStatus(jobID, "failed")
@@ -312,11 +389,15 @@ func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
e.setJobStatus(jobID, "failed")
e.setJobError(jobID, errCode, errMsg)
e.emit(Event{Type: "status", JobID: jobID, Key: "status", Value: "failed", Line: errMsg})
e.persistAndClose(jobID)
return fmt.Errorf("rsync exited with code %d: %s", result.ExitCode, result.Stderr)
}
e.setJobStatus(jobID, "success")
e.emit(Event{Type: "status", JobID: jobID, Key: "status", Value: "success"})
if stats != nil {
e.persistJobTotals(jobID, stats)
}
slog.Info("job completed", "job_id", jobID, "pair", pair.Name)
e.persistAndClose(jobID)
return nil
@@ -324,6 +405,23 @@ func (e *Engine) Run(ctx context.Context, jobID int64, pairID int64) error {
func (e *Engine) persistAndClose(jobID int64) {
e.eventBus.CloseJobChannels(jobID)
logRepo := models.NewJobLogRepository(e.db)
count, err := logRepo.CountByJobID(jobID)
if err == nil && count > 2000 {
if truncateErr := logRepo.TruncateKeepingHeaderTail(jobID, 50, 100); truncateErr != nil {
slog.Warn("failed to truncate job logs", "job_id", jobID, "error", truncateErr)
}
}
}
func (e *Engine) persistJobTotals(jobID int64, stats *RsyncStats) {
if stats == nil {
return
}
jobRepo := models.NewJobRepository(e.db)
if err := jobRepo.SetTotals(jobID, stats.TotalSize, stats.SentBytes); err != nil {
slog.Warn("failed to persist job totals", "job_id", jobID, "error", err)
}
}
func (e *Engine) resolveSSHKey(machine *models.Machine) (string, error) {
@@ -338,17 +436,18 @@ func (e *Engine) resolveSSHKey(machine *models.Machine) (string, error) {
return sshKey.PrivateKeyPath, nil
}
func (e *Engine) Cancel(jobID int64, syncPairID int64, byUser bool) bool {
if e.queue.IsRunning(syncPairID) {
e.queue.Cancel(syncPairID, byUser)
return true
}
return false
func (e *Engine) Cancel(jobID int64, byUser bool) bool {
return e.queue.Cancel(jobID, byUser)
}
func (e *Engine) setJobStatus(jobID int64, status string) {
jobRepo := models.NewJobRepository(e.db)
jobRepo.UpdateStatus(jobID, status)
if status == "success" || status == "failed" || status == "cancelled" {
e.jobsTotalMu.Lock()
e.jobsTotal[status]++
e.jobsTotalMu.Unlock()
}
}
func (e *Engine) setJobLogFile(jobID int64, path string) {
@@ -452,3 +551,36 @@ func (e *Engine) ProbeAllMachines() {
}
wg.Wait()
}
func (e *Engine) GetJobsTotal() map[string]int64 {
e.jobsTotalMu.Lock()
defer e.jobsTotalMu.Unlock()
return e.jobsTotal
}
func (e *Engine) GetJobsRunning() int64 {
return int64(len(e.queue.RunningJobs()))
}
func (e *Engine) GetQueueDepth() int64 {
return int64(e.queue.Len())
}
func (e *Engine) GetMachineCounts() (online, total int64) {
machineRepo := models.NewMachineRepository(e.db)
ms, err := machineRepo.GetAll()
if err != nil {
return 0, 0
}
for _, m := range ms {
total++
if m.Status == "online" {
online++
}
}
return online, total
}
func (e *Engine) DB() *sql.DB {
return e.db
}
+6 -28
View File
@@ -8,20 +8,17 @@ import (
type EventBus struct {
subscribers map[int64]map[chan Event]struct{}
mu sync.RWMutex
global chan Event
bufferSize int
globalSubs []globalSub
}
type globalSub struct {
ch chan Event
done chan struct{}
}
func NewEventBus(bufferSize int) *EventBus {
return &EventBus{
subscribers: make(map[int64]map[chan Event]struct{}),
global: make(chan Event, bufferSize),
bufferSize: bufferSize,
globalSubs: nil,
}
@@ -51,32 +48,10 @@ func (eb *EventBus) Subscribe(jobID int64) (chan Event, func()) {
func (eb *EventBus) SubscribeGlobal() (chan Event, func()) {
ch := make(chan Event, eb.bufferSize)
done := make(chan struct{})
eb.mu.Lock()
eb.globalSubs = append(eb.globalSubs, globalSub{ch: ch, done: done})
eb.globalSubs = append(eb.globalSubs, globalSub{ch: ch})
eb.mu.Unlock()
go func() {
defer func() {
if r := recover(); r != nil {
slog.Error("SubscribeGlobal goroutine panicked", "reason", r)
}
close(ch)
}()
for {
select {
case evt := <-eb.global:
select {
case ch <- evt:
default:
slog.Warn("global event subscriber buffer full, dropping event", "type", evt.Type)
}
case <-done:
return
}
}
}()
return ch, func() {
close(done)
eb.mu.Lock()
for i, s := range eb.globalSubs {
if s.ch == ch {
@@ -85,6 +60,7 @@ func (eb *EventBus) SubscribeGlobal() (chan Event, func()) {
}
}
eb.mu.Unlock()
close(ch)
}
}
@@ -102,10 +78,12 @@ func (eb *EventBus) Publish(evt Event) {
}
}
for _, sub := range eb.globalSubs {
select {
case eb.global <- evt:
case sub.ch <- evt:
default:
slog.Warn("global event bus full, dropping event", "type", evt.Type)
slog.Warn("global event subscriber buffer full, dropping event", "type", evt.Type)
}
}
}
+95
View File
@@ -24,6 +24,15 @@ type ProgressLine struct {
XferedBytes int64
}
type ProgressFields struct {
FileBytes int64 `json:"file_bytes"`
Pct int `json:"pct"`
SpeedBps int64 `json:"speed_bps"`
EtaSeconds int `json:"eta_seconds"`
XfrDone int `json:"xfr_done"`
XfrTotal int `json:"xfr_total"`
}
var (
progressRegex = regexp.MustCompile(`\s*([\d,]+)\s+([\d,]+)\s+([\d%]+)\s*`)
sentRegex = regexp.MustCompile(`sent\s+([\d,]+)\s+bytes`)
@@ -32,6 +41,24 @@ var (
filesRegex = regexp.MustCompile(`Number of files: ([\d,]+)`)
)
var perFileProgressRegex = regexp.MustCompile(
`^\s*(\d{1,3}(?:,\d{3})+)\s+(\d+)%\s+(\d+\.\d+)([kMG])B/s\s+(\d+:\d{2}:\d{2})(.*)`,
)
var xfrRegex = regexp.MustCompile(`xfr#(\d+).*to-chk=(\d+)/(\d+)`)
func parseXfrSuffix(suffix string) (done, total int) {
m := xfrRegex.FindStringSubmatch(suffix)
if m == nil {
return 0, 0
}
done, _ = strconv.Atoi(m[1])
t, _ := strconv.Atoi(m[2])
_ = t
total, _ = strconv.Atoi(m[3])
return done, total
}
func ParseProgressLine(line string) *ProgressLine {
if strings.Contains(line, "files to consider") || strings.Contains(line, "files...") {
return &ProgressLine{Phase: "scanning"}
@@ -74,3 +101,71 @@ func ParseFinalStats(output string) *RsyncStats {
}
return stats
}
func isProgressOnlyLine(line string) bool {
return perFileProgressRegex.MatchString(line)
}
func parseProgressFields(line string) *ProgressFields {
m := perFileProgressRegex.FindStringSubmatch(line)
if m == nil {
return nil
}
bytes, _ := strconv.ParseInt(strings.ReplaceAll(m[1], ",", ""), 10, 64)
pct, _ := strconv.Atoi(m[2])
speed, _ := strconv.ParseFloat(m[3], 64)
unit := m[4]
eta := m[5]
suffix := m[6]
speedBps := int64(speed * 1e6)
switch unit {
case "k", "K":
speedBps = int64(speed * 1e3)
case "m", "M":
speedBps = int64(speed * 1e6)
case "g", "G":
speedBps = int64(speed * 1e9)
}
etaSecs := 0
parts := strings.Split(eta, ":")
if len(parts) == 3 {
h, _ := strconv.Atoi(parts[0])
m, _ := strconv.Atoi(parts[1])
s, _ := strconv.Atoi(parts[2])
etaSecs = h*3600 + m*60 + s
}
pf := &ProgressFields{
FileBytes: bytes,
Pct: pct,
SpeedBps: speedBps,
EtaSeconds: etaSecs,
}
if suffix != "" {
done, total := parseXfrSuffix(suffix)
pf.XfrDone = done
pf.XfrTotal = total
}
return pf
}
func isFileNameLine(line string) bool {
if line == "" || strings.TrimSpace(line) == "" {
return false
}
if strings.Contains(line, "sending incremental file list") ||
strings.Contains(line, "building file list") ||
strings.Contains(line, "cannot open") ||
strings.Contains(line, "skipping non-regular") ||
strings.HasPrefix(line, "sent ") ||
strings.HasPrefix(line, "total ") ||
strings.HasPrefix(line, "Number of files:") ||
strings.Contains(line, "bytes received") {
return false
}
return !isProgressOnlyLine(line)
}
+124
View File
@@ -0,0 +1,124 @@
package syncengine
import "testing"
func TestIsProgressOnlyLine(t *testing.T) {
cases := []struct {
name string
line string
expect bool
}{
{"per-file progress 0%", " 32,768 0% 0.00kB/s 0:00:00", true},
{"per-file progress 7%", " 2,260,893,696 7% 51.14MB/s 0:08:45", true},
{"per-file progress with xfr suffix", " 67,141,632 0% 32.02MB/s 0:15:06 (xfr#1, to-chk=4/10)", true},
{"filename line", "Dragon Ball Sleeping Princess in Devil's Castle (1987)/", false},
{"sending incremental file list header", "sending incremental file list", false},
{"sent bytes stats", "sent 123,456 bytes received 789 bytes 12.34kB/s", false},
{"total size stats", "total size is 999,999,999 speedup is 1.23", false},
{"Number of files stats", "Number of files: 10", false},
{"building file list", "building file list ...", false},
{"empty line", "", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := isProgressOnlyLine(tc.line)
if got != tc.expect {
t.Errorf("isProgressOnlyLine(%q) = %v, want %v", tc.line, got, tc.expect)
}
})
}
}
func TestParseProgressFields(t *testing.T) {
cases := []struct {
name string
line string
wantPct int
wantSpeedBps int64
wantEtaSeconds int
wantXfrDone int
wantXfrTotal int
}{
{
name: "progress 0% with kB/s",
line: " 32,768 0% 0.00kB/s 0:00:00",
wantPct: 0,
wantSpeedBps: 0,
wantEtaSeconds: 0,
},
{
name: "progress 7% with MB/s",
line: " 2,260,893,696 7% 51.14MB/s 0:08:45",
wantPct: 7,
wantSpeedBps: 51_140_000,
wantEtaSeconds: 8*60 + 45,
},
{
name: "progress with xfr suffix",
line: " 67,141,632 0% 32.02MB/s 0:15:06 (xfr#1, to-chk=4/10)",
wantPct: 0,
wantSpeedBps: 32_020_000,
wantEtaSeconds: 15*60 + 6,
wantXfrDone: 1,
wantXfrTotal: 10,
},
{
name: "progress with GB/s",
line: " 1,234,567,890 50% 1.23GB/s 0:01:30",
wantPct: 50,
wantSpeedBps: 1_230_000_000,
wantEtaSeconds: 1*60 + 30,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
p := parseProgressFields(tc.line)
if p == nil {
t.Fatalf("parseProgressFields(%q) returned nil, want non-nil", tc.line)
}
if p.Pct != tc.wantPct {
t.Errorf("pct = %d, want %d", p.Pct, tc.wantPct)
}
if p.SpeedBps != tc.wantSpeedBps {
t.Errorf("speedBps = %d, want %d", p.SpeedBps, tc.wantSpeedBps)
}
if p.EtaSeconds != tc.wantEtaSeconds {
t.Errorf("etaSeconds = %d, want %d", p.EtaSeconds, tc.wantEtaSeconds)
}
if tc.wantXfrTotal > 0 && p.XfrDone != tc.wantXfrDone {
t.Errorf("xfrDone = %d, want %d", p.XfrDone, tc.wantXfrDone)
}
if tc.wantXfrTotal > 0 && p.XfrTotal != tc.wantXfrTotal {
t.Errorf("xfrTotal = %d, want %d", p.XfrTotal, tc.wantXfrTotal)
}
})
}
}
func TestIsFileNameLine(t *testing.T) {
cases := []struct {
name string
line string
expect bool
}{
{"directory path", "Dragon Ball Sleeping Princess in Devil's Castle (1987)/", true},
{"file path", "Dragon Ball Sleeping Princess in Devil's Castle (1987)/Dragon Ball...WEBDL-2160p.mkv", true},
{"sending incremental file list header", "sending incremental file list", false},
{"sent stats", "sent 12,345 bytes received 1,234 bytes", false},
{"total size stats", "total size is 999,999,999", false},
{"Number of files", "Number of files: 10", false},
{"progress line", " 2,260,893,696 7% 51.14MB/s 0:08:45", false},
{"empty", "", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got := isFileNameLine(tc.line)
if got != tc.expect {
t.Errorf("isFileNameLine(%q) = %v, want %v", tc.line, got, tc.expect)
}
})
}
}
+44 -19
View File
@@ -14,59 +14,84 @@ type Queue struct {
type RunInfo struct {
JobID int64
SyncPairID int64
Cancel func()
ByUser bool
CancelledBy bool
}
func NewQueue() *Queue {
return &Queue{runs: make(map[int64]*RunInfo)}
}
func (q *Queue) Enqueue(syncPairID, jobID int64, cancel func()) error {
func (q *Queue) Enqueue(syncPairID, jobID int64, cancelFn func()) error {
q.mu.Lock()
defer q.mu.Unlock()
if _, exists := q.runs[syncPairID]; exists {
if _, exists := q.runs[jobID]; exists {
return ErrAlreadyRunning
}
q.runs[syncPairID] = &RunInfo{JobID: jobID, Cancel: cancel}
for _, info := range q.runs {
if info.SyncPairID == syncPairID {
return ErrAlreadyRunning
}
}
q.runs[jobID] = &RunInfo{JobID: jobID, SyncPairID: syncPairID, Cancel: cancelFn}
return nil
}
func (q *Queue) Dequeue(syncPairID int64) {
func (q *Queue) Dequeue(jobID int64) {
q.mu.Lock()
defer q.mu.Unlock()
delete(q.runs, syncPairID)
delete(q.runs, jobID)
}
func (q *Queue) IsRunning(syncPairID int64) bool {
func (q *Queue) IsRunning(jobID int64) bool {
q.mu.Lock()
defer q.mu.Unlock()
_, exists := q.runs[syncPairID]
_, exists := q.runs[jobID]
return exists
}
func (q *Queue) GetJobID(syncPairID int64) (int64, bool) {
func (q *Queue) GetByPair(syncPairID int64) (jobID int64, exists bool) {
q.mu.Lock()
defer q.mu.Unlock()
info, exists := q.runs[syncPairID]
if !exists {
return 0, false
}
for _, info := range q.runs {
if info.SyncPairID == syncPairID {
return info.JobID, true
}
}
return 0, false
}
func (q *Queue) Cancel(syncPairID int64, byUser bool) {
func (q *Queue) Cancel(jobID int64, byUser bool) bool {
q.mu.Lock()
defer q.mu.Unlock()
if info, exists := q.runs[syncPairID]; exists && info.Cancel != nil {
info.ByUser = byUser
if info, exists := q.runs[jobID]; exists && info.Cancel != nil {
info.CancelledBy = byUser
info.Cancel()
return true
}
return false
}
func (q *Queue) IsCancelledByUser(syncPairID int64) bool {
func (q *Queue) IsCancelledByUser(jobID int64) bool {
q.mu.Lock()
defer q.mu.Unlock()
info, exists := q.runs[syncPairID]
return exists && info.ByUser
info, exists := q.runs[jobID]
return exists && info.CancelledBy
}
func (q *Queue) RunningJobs() []int64 {
q.mu.Lock()
defer q.mu.Unlock()
ids := make([]int64, 0, len(q.runs))
for id := range q.runs {
ids = append(ids, id)
}
return ids
}
func (q *Queue) Len() int {
q.mu.Lock()
defer q.mu.Unlock()
return len(q.runs)
}
+15 -13
View File
@@ -7,7 +7,7 @@ import (
func TestQueue(t *testing.T) {
q := NewQueue()
if q.IsRunning(1) {
if q.IsRunning(100) {
t.Error("queue should be empty")
}
@@ -16,35 +16,37 @@ func TestQueue(t *testing.T) {
err := q.Enqueue(1, 100, cancel)
if err != nil {
t.Errorf("Enqueue(1) unexpected error: %v", err)
t.Errorf("Enqueue(1, 100) unexpected error: %v", err)
}
if !q.IsRunning(1) {
t.Error("queue should contain syncPair 1")
if !q.IsRunning(100) {
t.Error("queue should contain job 100")
}
jobID, ok := q.GetJobID(1)
jobID, ok := q.GetByPair(1)
if !ok || jobID != 100 {
t.Errorf("GetJobID(1) = %d, %v, want 100, true", jobID, ok)
t.Errorf("GetByPair(1) = %d, %v, want 100, true", jobID, ok)
}
err = q.Enqueue(1, 200, nil)
if err != ErrAlreadyRunning {
t.Errorf("Enqueue(1) again = %v, want ErrAlreadyRunning", err)
t.Errorf("Enqueue(1, 200) = %v, want ErrAlreadyRunning", err)
}
q.Cancel(1, true)
ok = q.Cancel(100, true)
if !ok {
t.Error("Cancel(100) should return true")
}
if !cancelCalled {
t.Error("Cancel should have called the cancel func")
}
if !q.IsCancelledByUser(1) {
t.Error("IsCancelledByUser should return true after Cancel(1, true)")
if !q.IsCancelledByUser(100) {
t.Error("IsCancelledByUser should return true after Cancel(100, true)")
}
q.Dequeue(1)
q.Dequeue(100)
q.Dequeue(1)
if q.IsRunning(1) {
if q.IsRunning(100) {
t.Error("queue should be empty after Dequeue")
}
}
+198 -146
View File
@@ -2,13 +2,99 @@ package syncengine
import (
"context"
"encoding/base64"
"fmt"
"io"
"log/slog"
"os/exec"
"path/filepath"
"strconv"
"strings"
)
var allowedRsyncFlags = map[string]bool{
"-v": true,
"-vv": true,
"-q": true,
"-h": true,
"-P": true,
"-n": true,
"-z": true,
"-c": true,
"-u": true,
"-W": true,
"-i": true,
"-a": true,
"-r": true,
"-l": true,
"-t": true,
"-p": true,
"-g": true,
"-o": true,
"-D": true,
"--verbose": true,
"--quiet": true,
"--help": true,
"--partial": true,
"--partial-dir": true,
"--delay-updates": true,
"--delete": true,
"--delete-before": true,
"--delete-after": true,
"--delete-excluded": true,
"--exclude": true,
"--exclude-from": true,
"--dry-run": true,
"--compress": true,
"--skip-compress": true,
"--whole-file": true,
"--checksum": true,
"--update": true,
"--existing": true,
"--ignore-existing": true,
"--remove-source-files": true,
"--chmod": true,
"--owner": true,
"--group": true,
"--perms": true,
"--executability": true,
"--acls": true,
"--xattrs": true,
"--numeric-ids": true,
"--fake-super": true,
"--bwlimit": true,
"--max-size": true,
"--min-size": true,
"--append": true,
"--append-verify": true,
"--itemize-changes": true,
}
var blockedRsyncFlags = map[string]bool{
"--rsync-path": true,
"-e": true,
"--files-from": true,
"--read-batch": true,
"--write-batch": true,
"--log-file": true,
}
func isSafeRsyncFlag(flag string) bool {
if allowedRsyncFlags[flag] {
return true
}
safePrefixes := []string{
"-a", "-v", "-z", "-P", "-n", "-c", "-u", "-W", "-i",
"--exclude=", "--chmod=", "--bwlimit=", "--max-size=", "--min-size=",
"--partial-dir=", "--skip-compress=",
}
for _, p := range safePrefixes {
if strings.HasPrefix(flag, p) {
return true
}
}
return false
}
type RsyncResult struct {
ExitCode int
Stdout string
@@ -40,15 +126,124 @@ func NewRsyncRunner(sshDir, privKey string) *RsyncRunner {
}
func (r *RsyncRunner) Run(ctx context.Context, pair *SyncPairConfig, onLine func(stream string, line string)) (*RsyncResult, error) {
args := r.buildArgs(pair)
cmd := r.buildRsyncCmd(ctx, pair)
return r.runCmd(ctx, cmd, onLine)
}
func (r *RsyncRunner) buildRsyncCmd(ctx context.Context, pair *SyncPairConfig) *exec.Cmd {
args := r.buildArgs(pair)
cmd := exec.CommandContext(ctx, "rsync", args...)
if r.privKey != "" {
sshCmd := fmt.Sprintf("ssh -i %s -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=%s",
r.privKey, strings.TrimRight(r.sshDir, "/")+"/known_hosts")
cmd.Args = append([]string{"rsync", "-e", sshCmd}, args[1:]...)
cmd.Args = append([]string{"rsync", "-e", sshCmd}, args...)
}
return cmd
}
func (r *RsyncRunner) buildArgs(pair *SyncPairConfig) []string {
var args []string
flags := strings.Fields(pair.RsyncFlags)
for _, flag := range flags {
if strings.HasPrefix(flag, "-") {
if blockedRsyncFlags[flag] {
slog.Warn("blocked dangerous rsync flag", "flag", flag)
continue
}
if !isSafeRsyncFlag(flag) {
slog.Warn("disallowed rsync flag", "flag", flag)
continue
}
}
args = append(args, flag)
}
for _, pattern := range pair.ExcludePatterns {
args = append(args, "--exclude="+pattern)
}
if pair.Direction == "mirror" {
args = append(args, "--delete")
}
args = append(args, "--")
src := ensureDirSlash(pair.Source)
if pair.Direction == "pull" {
args = append(args, pair.Dest, src)
} else {
args = append(args, src, pair.Dest)
}
return args
}
// ensureDirSlash guarantees the source path is treated by rsync as a
// directory whose contents are copied, regardless of whether the user
// supplied a trailing slash. This avoids the common foot-gun where
// "rsync host:/path/series /dest/" creates /dest/series/<contents> nested
// inside an extra "series" subdirectory.
func ensureDirSlash(p string) string {
if strings.HasSuffix(p, "/") {
return p
}
return p + "/"
}
type MachineKeys struct {
Host string
Port int
SSHUser string
PrivKey string
}
func (r *RsyncRunner) RunRemote(ctx context.Context, pair *SyncPairConfig, src *MachineKeys, dst *MachineKeys, destKey string, onLine func(stream string, line string)) (*RsyncResult, error) {
if src.Port == 0 {
src.Port = 22
}
if src.PrivKey == "" {
src.PrivKey = filepath.Join(r.sshDir, "id_ed25519")
}
if destKey == "" {
destKey = filepath.Join(r.sshDir, "id_ed25519")
}
args := r.buildArgs(pair)
innerSSH := fmt.Sprintf("ssh -i %s -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=%s",
destKey, filepath.Join(r.sshDir, "known_hosts"))
rsyncFlags := args[:len(args)-2]
sourcePath := args[len(args)-2]
destPath := args[len(args)-1]
var rsyncCmd []string
rsyncCmd = append(rsyncCmd, "rsync")
rsyncCmd = append(rsyncCmd, "-e")
rsyncCmd = append(rsyncCmd, innerSSH)
rsyncCmd = append(rsyncCmd, rsyncFlags...)
rsyncCmd = append(rsyncCmd, sourcePath, destPath)
remoteCmd := "rsync"
for _, arg := range rsyncFlags {
remoteCmd += " " + strconv.Quote(arg)
}
remoteCmd += " -e " + strconv.Quote(innerSSH) + " " + strconv.Quote(sourcePath) + " " + strconv.Quote(destPath)
remoteCmd = "sh -c " + strconv.Quote(remoteCmd)
sshArgs := []string{
"-i", src.PrivKey,
"-o", "StrictHostKeyChecking=accept-new",
"-o", "UserKnownHostsFile=" + filepath.Join(r.sshDir, "known_hosts"),
"-p", fmt.Sprintf("%d", src.Port),
fmt.Sprintf("%s@%s", src.SSHUser, src.Host),
}
sshArgs = append(sshArgs, remoteCmd)
cmd := exec.CommandContext(ctx, "ssh", sshArgs...)
return r.runCmd(ctx, cmd, onLine)
}
func (r *RsyncRunner) runCmd(ctx context.Context, cmd *exec.Cmd, onLine func(stream string, line string)) (*RsyncResult, error) {
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("stdout pipe: %w", err)
@@ -133,146 +328,3 @@ func (r *RsyncRunner) Run(ctx context.Context, pair *SyncPairConfig, onLine func
return result, nil
}
func (r *RsyncRunner) buildArgs(pair *SyncPairConfig) []string {
var args []string
flags := strings.Fields(pair.RsyncFlags)
args = append(args, flags...)
for _, pattern := range pair.ExcludePatterns {
args = append(args, "--exclude="+pattern)
}
if pair.Direction == "mirror" {
args = append(args, "--delete")
}
if pair.Direction == "pull" {
args = append(args, pair.Dest, pair.Source)
} else {
args = append(args, pair.Source, pair.Dest)
}
return args
}
type RemoteMachine struct {
Host string
Port int
SSHUser string
PrivKey string
DestPrivKey string
}
func (r *RsyncRunner) RunRemote(ctx context.Context, pair *SyncPairConfig, remote RemoteMachine, onLine func(stream string, line string)) (*RsyncResult, error) {
args := r.buildArgs(pair)
var remoteCmd string
if remote.DestPrivKey != "" {
encodedKey := base64.StdEncoding.EncodeToString([]byte(remote.DestPrivKey))
innerSSH := fmt.Sprintf(`ssh -i /tmp/syncserver-dest-key -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=/dev/null`)
rsyncPart := fmt.Sprintf("rsync %s -e %q",
strings.Join(args, " "), innerSSH)
remoteCmd = fmt.Sprintf(
`echo '%s' | base64 -d > /tmp/syncserver-dest-key && chmod 600 /tmp/syncserver-dest-key && %s; STATUS=$?; rm -f /tmp/syncserver-dest-key; exit $STATUS`,
encodedKey, rsyncPart)
} else {
remoteCmd = "rsync " + strings.Join(args, " ")
}
sshArgs := []string{
"-i", remote.PrivKey,
"-o", "StrictHostKeyChecking=accept-new",
"-o", "UserKnownHostsFile=" + strings.TrimRight(r.sshDir, "/") + "/known_hosts",
"-p", fmt.Sprintf("%d", remote.Port),
fmt.Sprintf("%s@%s", remote.SSHUser, remote.Host),
}
sshArgs = append(sshArgs, remoteCmd)
cmd := exec.CommandContext(ctx, "ssh", sshArgs...)
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, fmt.Errorf("stdout pipe: %w", err)
}
stderr, err := cmd.StderrPipe()
if err != nil {
return nil, fmt.Errorf("stderr pipe: %w", err)
}
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("starting ssh: %w", err)
}
var outLines, errLines []string
done := make(chan struct{})
go func() {
br := io.Reader(stdout)
buf := make([]byte, 4096)
for {
n, err := br.Read(buf)
if n > 0 {
line := strings.TrimRight(string(buf[:n]), "\r\n")
if line != "" {
outLines = append(outLines, line)
if onLine != nil {
onLine("stdout", line)
}
}
}
if err != nil {
break
}
}
select {
case <-done:
default:
close(done)
}
}()
go func() {
br := io.Reader(stderr)
buf := make([]byte, 4096)
for {
n, err := br.Read(buf)
if n > 0 {
line := strings.TrimRight(string(buf[:n]), "\r\n")
if line != "" {
errLines = append(errLines, line)
if onLine != nil {
onLine("stderr", line)
}
}
}
if err != nil {
break
}
}
select {
case <-done:
default:
close(done)
}
}()
err = cmd.Wait()
<-done
result := &RsyncResult{
ExitCode: 0,
Stdout: strings.Join(outLines, "\n"),
Stderr: strings.Join(errLines, "\n"),
Stats: ParseFinalStats(strings.Join(outLines, "\n")),
}
if err != nil {
if exitErr, ok := err.(*exec.ExitError); ok {
result.ExitCode = exitErr.ExitCode()
} else {
result.ExitCode = -1
}
}
return result, nil
}
+312
View File
@@ -0,0 +1,312 @@
package syncengine
import (
"context"
"strings"
"testing"
)
type remoteCmdTest struct {
srcHost string
srcUser string
srcPath string
dstHost string
dstUser string
dstPath string
direction string
}
func (tc remoteCmdTest) build() (srcArg, dstArg string) {
runner := NewRsyncRunner("/tmp/ssh", "")
args := runner.buildArgs(&SyncPairConfig{
Source: tc.srcPath,
Dest: tc.dstPath,
Direction: tc.direction,
})
sourcePath := args[len(args)-2]
destPath := args[len(args)-1]
return sourcePath, destPath
}
func TestBuildArgs_Push(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "")
pair := &SyncPairConfig{
Source: "/local/src",
Dest: "admin@10.5.0.144:/remote/dst",
Direction: "push",
RsyncFlags: "-aP",
ExcludePatterns: []string{},
}
args := runner.buildArgs(pair)
if args[0] != "-aP" {
t.Errorf("first flag = %q, want %q", args[0], "-aP")
}
if args[len(args)-2] != "/local/src/" {
t.Errorf("source = %q, want %q (auto-appended trailing slash)", args[len(args)-2], "/local/src/")
}
if args[len(args)-1] != "admin@10.5.0.144:/remote/dst" {
t.Errorf("dest = %q, want %q", args[len(args)-1], "admin@10.5.0.144:/remote/dst")
}
}
func TestBuildArgs_Pull(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "")
pair := &SyncPairConfig{
Source: "admin@10.5.0.144:/remote/src",
Dest: "/local/dst",
Direction: "pull",
RsyncFlags: "-aP",
ExcludePatterns: []string{},
}
args := runner.buildArgs(pair)
if args[len(args)-2] != "/local/dst" {
t.Errorf("pull: second-to-last (dest) = %q, want %q", args[len(args)-2], "/local/dst")
}
if args[len(args)-1] != "admin@10.5.0.144:/remote/src/" {
t.Errorf("pull: last (source) = %q, want %q (auto-appended trailing slash)", args[len(args)-1], "admin@10.5.0.144:/remote/src/")
}
}
func TestBuildArgs_Mirror(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "")
pair := &SyncPairConfig{
Source: "/local/src",
Dest: "admin@10.5.0.144:/remote/dst",
Direction: "mirror",
RsyncFlags: "-aP",
ExcludePatterns: []string{},
}
args := runner.buildArgs(pair)
foundDelete := false
for _, a := range args {
if a == "--delete" {
foundDelete = true
break
}
}
if !foundDelete {
t.Errorf("mirror args = %v, want --delete present", args)
}
}
func TestBuildArgs_MultiTokenFlags(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "")
pair := &SyncPairConfig{
Source: "/local/src",
Dest: "admin@10.5.0.144:/remote/dst",
Direction: "push",
RsyncFlags: "-aP --partial",
ExcludePatterns: []string{},
}
args := runner.buildArgs(pair)
if args[0] != "-aP" {
t.Errorf("first flag = %q, want %q", args[0], "-aP")
}
if args[1] != "--partial" {
t.Errorf("second flag = %q, want %q", args[1], "--partial")
}
}
func TestBuildArgs_ExcludePatterns(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "")
pair := &SyncPairConfig{
Source: "/local/src",
Dest: "admin@10.5.0.144:/remote/dst",
Direction: "push",
RsyncFlags: "-aP",
ExcludePatterns: []string{"*.tmp", ".DS_Store"},
}
args := runner.buildArgs(pair)
var excludes []string
for _, a := range args {
if strings.HasPrefix(a, "--exclude=") {
excludes = append(excludes, a)
}
}
if len(excludes) != 2 {
t.Errorf("excludes = %v, want 2 exclude entries", excludes)
}
if excludes[0] != "--exclude=*.tmp" {
t.Errorf("exclude[0] = %q, want %q", excludes[0], "--exclude=*.tmp")
}
}
func TestRunRemote_PushDestNoDoublePrefix(t *testing.T) {
tc := remoteCmdTest{
srcHost: "10.5.1.10",
srcUser: "root",
srcPath: "/share/homes/admin/media",
dstHost: "10.5.0.144",
dstUser: "admin",
dstPath: "admin@10.5.0.144:/share/media/peliculas",
direction: "push",
}
_, dstArg := tc.build()
if countOccurrences(dstArg, "@") > 1 {
t.Errorf("push dest %q has double SSH spec", dstArg)
}
if dstArg != "admin@10.5.0.144:/share/media/peliculas" {
t.Errorf("push dest = %q, want 'admin@10.5.0.144:/share/media/peliculas'", dstArg)
}
}
func TestRunRemote_PushSrcAndDestStayAsIs(t *testing.T) {
tc := remoteCmdTest{
srcHost: "10.5.1.10",
srcUser: "root",
srcPath: "/share/homes/admin/media",
dstHost: "10.5.0.144",
dstUser: "admin",
dstPath: "/share/media/peliculas",
direction: "push",
}
srcArg, dstArg := tc.build()
if srcArg != "/share/homes/admin/media/" {
t.Errorf("push src = %q, want '/share/homes/admin/media/' (auto-appended trailing slash, no user@host: prefix added)", srcArg)
}
if dstArg != "/share/media/peliculas" {
t.Errorf("push dst = %q, want raw path '/share/media/peliculas'", dstArg)
}
}
func TestRunRemote_PullSrcAndDestStayAsIs(t *testing.T) {
tc := remoteCmdTest{
srcHost: "10.5.0.144",
srcUser: "admin",
srcPath: "admin@10.5.0.144:/share/media/peliculas",
dstHost: "10.5.1.10",
dstUser: "root",
dstPath: "/share/data",
direction: "pull",
}
srcArg, dstArg := tc.build()
if countOccurrences(srcArg, "@") > 1 {
t.Errorf("pull src %q has double SSH spec", srcArg)
}
if srcArg != "/share/data" {
t.Errorf("pull src (rsync dest) = %q, want raw '/share/data'", srcArg)
}
if !strings.HasPrefix(dstArg, "admin@10.5.0.144:/") {
t.Errorf("pull dst (rsync src) = %q, want 'admin@10.5.0.144:/...' prefix", dstArg)
}
}
func TestBuildArgs_AutoAppendsTrailingSlashToSource(t *testing.T) {
cases := []struct {
name string
source string
dest string
direction string
wantSrc string
}{
{
name: "push, source without trailing slash",
source: "/mnt/storage/multimedia/series",
dest: "/share/media/series",
direction: "push",
wantSrc: "/mnt/storage/multimedia/series/",
},
{
name: "push, source already has trailing slash (idempotent)",
source: "/mnt/storage/multimedia/series/",
dest: "/share/media/series",
direction: "push",
wantSrc: "/mnt/storage/multimedia/series/",
},
{
name: "push, remote source without trailing slash",
source: "admin@baby-nas:/mnt/storage/multimedia/series",
dest: "/share/media/series",
direction: "push",
wantSrc: "admin@baby-nas:/mnt/storage/multimedia/series/",
},
{
name: "pull, source without trailing slash",
source: "admin@baby-nas:/mnt/storage/multimedia/series",
dest: "/share/media/series",
direction: "pull",
wantSrc: "admin@baby-nas:/mnt/storage/multimedia/series/",
},
{
name: "mirror, source without trailing slash",
source: "/mnt/storage/multimedia/series",
dest: "admin@10.5.0.144:/share/media/series",
direction: "mirror",
wantSrc: "/mnt/storage/multimedia/series/",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "")
pair := &SyncPairConfig{
Source: tc.source,
Dest: tc.dest,
Direction: tc.direction,
}
args := runner.buildArgs(pair)
var gotSrc, gotDest string
if tc.direction == "pull" {
gotDest = args[len(args)-2]
gotSrc = args[len(args)-1]
} else {
gotSrc = args[len(args)-2]
gotDest = args[len(args)-1]
}
if gotSrc != tc.wantSrc {
t.Errorf("source = %q, want %q (dest must never be touched)", gotSrc, tc.wantSrc)
}
if gotDest != tc.dest {
t.Errorf("dest = %q, want %q (dest must never be normalized)", gotDest, tc.dest)
}
})
}
}
func countOccurrences(s, substr string) int {
return strings.Count(s, substr)
}
func TestRun_FlagsPreservedWithPrivKey(t *testing.T) {
runner := NewRsyncRunner("/tmp/ssh", "/tmp/ssh/id_ed25519")
pair := &SyncPairConfig{
Source: "/local/src",
Dest: "admin@10.5.0.144:/remote/dst",
Direction: "push",
RsyncFlags: "-aP --partial",
ExcludePatterns: []string{},
}
cmd := runner.buildRsyncCmd(context.Background(), pair)
if cmd.Args[0] != "rsync" {
t.Errorf("cmd.Args[0] = %q, want 'rsync'", cmd.Args[0])
}
if cmd.Args[1] != "-e" {
t.Errorf("cmd.Args[1] = %q, want '-e' (the -e flag for ssh)", cmd.Args[1])
}
if !strings.Contains(cmd.Args[2], "ssh -i") {
t.Errorf("cmd.Args[2] = %q, want ssh -i ...", cmd.Args[2])
}
hasAP := false
hasPartial := false
for i, a := range cmd.Args {
if a == "-aP" && i > 2 {
hasAP = true
}
if a == "--partial" && i > 2 {
hasPartial = true
}
}
if !hasAP {
t.Errorf("cmd.Args = %v, want -aP flag preserved (not dropped)", cmd.Args)
}
if !hasPartial {
t.Errorf("cmd.Args = %v, want --partial flag present", cmd.Args)
}
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-14
View File
@@ -1,14 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<base href="/">
<title>SyncServer</title>
<script type="module" crossorigin src="./assets/index-B2TqqDPF.js"></script>
<link rel="stylesheet" crossorigin href="./assets/index-BepSbXPY.css">
</head>
<body>
<div id="root"></div>
</body>
</html>
+1 -1
View File
@@ -111,7 +111,7 @@ else
NEW_VERSION=$(grep -oE 'var version = "[0-9]+\.[0-9]+\.[0-9]+"' cmd/server/main.go | grep -oE '[0-9]+\.[0-9]+\.[0-9]+')
[ "$NEW_VERSION" != "$CURRENT_VERSION" ] || die "Version bump failed"
git add cmd/server/main.go Makefile
git add -f cmd/server/main.go Makefile
git commit -m "Bump version to $NEW_VERSION" >/dev/null
git push || die "git push failed"
success "Bumped to $NEW_VERSION and pushed"
+22 -1
View File
@@ -6,6 +6,7 @@ import {
NavLink,
Outlet,
useNavigate,
Link,
} from 'react-router-dom';
import { useState, useEffect } from 'react';
import { Toaster } from 'sonner';
@@ -20,6 +21,8 @@ import {
LogOut,
Menu,
X,
Clock,
ArrowLeft,
} from 'lucide-react';
import { ErrorBoundary } from './components/ErrorBoundary';
import { Spinner } from './components/ui/Spinner';
@@ -34,11 +37,13 @@ import JobHistory from './pages/JobHistory';
import JobDetail from './pages/JobDetail';
import SettingsPage from './pages/Settings';
import SSHKeys from './pages/SSHKeys';
import Schedules from './pages/Schedules';
const navItems = [
{ to: '/', label: 'Dashboard', icon: LayoutDashboard },
{ to: '/machines', label: 'Machines', icon: Server },
{ to: '/sync-pairs', label: 'Sync Pairs', icon: GitCompare },
{ to: '/schedules', label: 'Schedules', icon: Clock },
{ to: '/jobs', label: 'Jobs', icon: History },
{ to: '/ssh-keys', label: 'SSH Keys', icon: Key },
{ to: '/settings', label: 'Settings', icon: SettingsIcon },
@@ -217,6 +222,21 @@ function Layout() {
);
}
function NotFound() {
return (
<div className="flex flex-col items-center justify-center min-h-[50vh] gap-3">
<p className="text-2xl font-bold text-fg">404</p>
<p className="text-fg-muted">Page not found</p>
<Button variant="secondary" asChild>
<Link to="/">
<ArrowLeft className="h-4 w-4" />
Back to Dashboard
</Link>
</Button>
</div>
);
}
export default function App() {
return (
<BrowserRouter>
@@ -233,12 +253,13 @@ export default function App() {
<Route path="/" element={<Dashboard />} />
<Route path="/machines" element={<Machines />} />
<Route path="/sync-pairs" element={<SyncPairs />} />
<Route path="/schedules" element={<Schedules />} />
<Route path="/jobs" element={<JobHistory />} />
<Route path="/jobs/:id" element={<JobDetail />} />
<Route path="/ssh-keys" element={<SSHKeys />} />
<Route path="/settings" element={<SettingsPage />} />
</Route>
<Route path="*" element={<Navigate to="/" />} />
<Route path="*" element={<NotFound />} />
</Routes>
</ErrorBoundary>
</BrowserRouter>
+42
View File
@@ -53,6 +53,9 @@ export interface Machine {
fingerprint_confirmed: boolean;
host_key_fingerprint: string | null;
status: string;
last_seen_at: string | null;
created_at: string;
shutdown_command: string;
}
export interface TestConnectionResponse {
@@ -116,3 +119,42 @@ export interface SettingsInfo {
data_dir: string;
ssh_pub_key: string;
}
export interface DeployKeysResponse {
success: boolean;
messages: string[];
errors: string[];
}
export interface DeployKeysOptions {
known_hosts_host?: string;
}
export async function deployKeys(machineId: number, options?: DeployKeysOptions): Promise<DeployKeysResponse> {
return api<DeployKeysResponse>(`/api/machines/${machineId}/deploy-keys`, {
method: 'POST',
body: options ?? {},
});
}
export interface ShutdownResponse {
success: boolean;
output?: string;
error?: string;
}
export interface Schedule {
id: number;
sync_pair_id: number;
sync_pair_name: string;
cron_expr: string;
next_run_at: string | null;
enabled: boolean;
created_at: string;
}
export async function shutdownMachine(machineId: number): Promise<ShutdownResponse> {
return api<ShutdownResponse>(`/api/machines/${machineId}/shutdown`, {
method: 'POST',
});
}
+3 -1
View File
@@ -25,7 +25,7 @@ export type BadgeVariant = VariantProps<typeof badgeVariants>['variant']
const STATUS_TO_VARIANT: Record<string, BadgeVariant> = {
running: 'running',
pending: 'pending',
waking: 'waking',
waking: 'pending',
success: 'success',
error: 'error',
failed: 'error',
@@ -33,6 +33,8 @@ const STATUS_TO_VARIANT: Record<string, BadgeVariant> = {
cancelled: 'neutral',
info: 'info',
unknown: 'neutral',
online: 'success',
offline: 'error',
}
export function statusVariant(status: string): BadgeVariant {
+27 -1
View File
@@ -1,6 +1,6 @@
import { useEffect, useState } from 'react';
import { Link } from 'react-router-dom';
import { Server, Activity, HardDrive, Clock, Plus } from 'lucide-react';
import { Server, Activity, HardDrive, Clock, Plus, XCircle } from 'lucide-react';
import { api, Machine, Job, SyncPair } from '../api/client';
import { Badge } from '@/components/ui/Badge';
import { Button } from '@/components/ui/Button';
@@ -12,6 +12,7 @@ import { Skeleton } from '@/components/ui/Skeleton';
import { statusVariant, statusLabel } from '@/lib/status';
import { formatRelativeTime } from '@/lib/utils';
import { subscribeMachineStatus } from '@/lib/sse';
import { toast } from 'sonner';
export default function Dashboard() {
const [machines, setMachines] = useState<Machine[]>([]);
@@ -48,6 +49,17 @@ export default function Dashboard() {
const pairName = (id: number) => pairs.find(p => p.id === id)?.name ?? `Pair ${id}`;
async function cancelJob(jobId: number) {
try {
await api(`/api/jobs/${jobId}/cancel`, { method: 'POST' });
toast.success('Job cancelled');
const j = await api<Job[]>(`/api/jobs?limit=5`);
setJobs(j);
} catch (e: unknown) {
toast.error((e as Error).message);
}
}
const online = machines.filter(m => m.status.startsWith('online')).length;
const todayJobs = jobs.filter(j => {
if (!j.started_at) return false;
@@ -156,6 +168,7 @@ export default function Dashboard() {
<TableHead>Sync Pair</TableHead>
<TableHead>Status</TableHead>
<TableHead>Started</TableHead>
<TableHead className="w-16">Actions</TableHead>
</TableRow>
</TableHeader>
<TableBody>
@@ -178,6 +191,19 @@ export default function Dashboard() {
<TableCell className="text-fg-muted text-xs">
{j.started_at ? formatRelativeTime(j.started_at) : '-'}
</TableCell>
<TableCell>
{['queued', 'waking_up', 'running'].includes(j.status) && (
<Button
variant="ghost"
size="icon-sm"
onClick={() => cancelJob(j.id)}
className="text-rose-400 hover:text-rose-300 hover:bg-rose-500/10"
title="Cancel job"
>
<XCircle className="h-3.5 w-3.5" />
</Button>
)}
</TableCell>
</TableRow>
))}
</TableBody>
+155 -3
View File
@@ -1,6 +1,6 @@
import { useEffect, useState, useRef, useCallback } from 'react';
import { useParams, Link } from 'react-router-dom';
import { api } from '../api/client';
import { api, apiRaw } from '../api/client';
import type { Job, LogLine, SyncPair } from '../api/client';
import { Badge } from '@/components/ui/Badge';
import { Button } from '@/components/ui/Button';
@@ -28,18 +28,32 @@ import {
Terminal,
AlertCircle,
Ban,
ChevronDown,
Activity,
} from 'lucide-react';
import { toast } from 'sonner';
import { statusVariant, statusLabel, getErrorCodeInfo } from '@/lib/status';
import { formatDuration } from '@/lib/utils';
import { cn } from '@/lib/utils';
interface SSEProgress {
file_bytes: number;
pct: number;
speed_bps: number;
eta_seconds: number;
xfr_done: number;
xfr_total: number;
}
interface SSEEvent {
type: string;
job_id: number;
status?: string;
line?: string;
stream?: string;
progress?: SSEProgress;
totalBytes?: number;
sentBytes?: number;
}
export default function JobDetail() {
@@ -56,6 +70,10 @@ export default function JobDetail() {
const [cancelModal, setCancelModal] = useState(false);
const [cancelReason, setCancelReason] = useState('');
const [errorModal, setErrorModal] = useState(false);
const [progress, setProgress] = useState<SSEProgress | null>(null);
const [finalTotals, setFinalTotals] = useState<{ totalBytes: number; sentBytes: number } | null>(null);
const [hasMoreLogs, setHasMoreLogs] = useState(false);
const [loadingMore, setLoadingMore] = useState(false);
useEffect(() => {
loadJob();
@@ -78,6 +96,13 @@ export default function JobDetail() {
return updated;
});
}
if (evt.type === 'progress' && evt.progress) {
setProgress(evt.progress);
}
if (evt.type === 'progress_total' && evt.totalBytes !== undefined && evt.sentBytes !== undefined) {
setFinalTotals({ totalBytes: evt.totalBytes, sentBytes: evt.sentBytes });
setProgress(null);
}
};
}
return () => esRef.current?.close();
@@ -109,8 +134,10 @@ export default function JobDetail() {
)) ?? [];
if (offset === 0) {
setLogs(ls);
setHasMoreLogs(ls.length === 1000);
} else {
setLogs(prev => [...prev, ...ls]);
setHasMoreLogs(ls.length === 1000);
}
} catch {}
}
@@ -130,12 +157,33 @@ export default function JobDetail() {
}
}
function downloadLog() {
async function downloadLog() {
try {
const resp = await apiRaw(`/api/jobs/${id}/log/download`);
if (resp.ok && resp.body) {
const blob = await resp.blob();
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `job-${id}.log`;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
URL.revokeObjectURL(url);
} else {
fallbackDownload();
}
} catch {
fallbackDownload();
}
}
function fallbackDownload() {
const allLines = [
...logs.map(l => `[${l.timestamp}] [${l.stream}] ${l.content}`),
...liveLines.map(l => `[LIVE] [${l.stream}] ${l.text}`),
];
const blob = new Blob([allLines.join('\n')], { type: 'text/plain' });
const blob = new Blob(allLines as string[], { type: 'text/plain' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
@@ -321,6 +369,10 @@ export default function JobDetail() {
</div>
)}
{(progress || finalTotals) && (
<TransferProgress progress={progress} finalTotals={finalTotals} />
)}
<Card className="flex flex-col min-h-0">
<div className="flex items-center justify-between px-4 py-3 border-b border-border shrink-0">
<div className="flex items-center gap-2">
@@ -378,6 +430,23 @@ export default function JobDetail() {
</>
)}
<div ref={logEndRef} />
{hasMoreLogs && (
<div className="flex justify-center py-2">
<Button
variant="secondary"
size="sm"
onClick={async () => {
setLoadingMore(true);
await loadLogs(logs.length);
setLoadingMore(false);
}}
disabled={loadingMore}
>
<ChevronDown className="h-4 w-4" />
{loadingMore ? 'Loading...' : 'Load more'}
</Button>
</div>
)}
</div>
</Card>
@@ -452,3 +521,86 @@ function LogLine({
</div>
);
}
function formatBytes(bytes: number): string {
if (bytes === 0) return '0 B';
const units = ['B', 'kB', 'MB', 'GB', 'TB'];
const i = Math.floor(Math.log(bytes) / Math.log(1024));
return `${(bytes / Math.pow(1024, i)).toFixed(1)} ${units[i]}`;
}
function formatSpeed(bps: number): string {
if (bps === 0) return '0 B/s';
const units = ['B/s', 'kB/s', 'MB/s', 'GB/s'];
const i = Math.floor(Math.log(bps) / Math.log(1000));
return `${(bps / Math.pow(1000, i)).toFixed(1)} ${units[i]}`;
}
function TransferProgress({
progress,
finalTotals,
}: {
progress: SSEProgress | null;
finalTotals: { totalBytes: number; sentBytes: number } | null;
}) {
const globalPct = finalTotals && finalTotals.totalBytes > 0
? Math.round((finalTotals.sentBytes / finalTotals.totalBytes) * 100)
: null;
return (
<Card>
<div className="p-4 space-y-3">
<div className="flex items-center gap-2">
<Activity className="h-4 w-4 text-accent" />
<span className="text-xs font-semibold text-fg-muted uppercase tracking-wider">
Transfer Progress
</span>
</div>
{progress && (
<>
<div className="space-y-1.5">
<div className="flex justify-between text-xs text-fg-muted">
<span>Current file</span>
<span>{progress.pct}%</span>
</div>
<div className="h-2 bg-border rounded-full overflow-hidden">
<div
className="h-full bg-accent transition-all duration-300 rounded-full"
style={{ width: `${progress.pct}%` }}
/>
</div>
</div>
<div className="flex justify-between text-xs text-fg-muted">
<span className="font-mono">
xfr#{(progress.xfr_done).toLocaleString()}/{progress.xfr_total > 0 ? progress.xfr_total.toLocaleString() : '?'}
</span>
<span className="font-mono">
{formatSpeed(progress.speed_bps)}
</span>
<span className="font-mono">
ETA {progress.eta_seconds > 0 ? `${Math.floor(progress.eta_seconds / 60)}m ${progress.eta_seconds % 60}s` : '-'}
</span>
</div>
</>
)}
{finalTotals && (
<div className="space-y-1.5">
<div className="flex justify-between text-xs text-fg-muted">
<span>Total transferred</span>
<span>{globalPct}% {formatBytes(finalTotals.sentBytes)} / {formatBytes(finalTotals.totalBytes)}</span>
</div>
<div className="h-2 bg-border rounded-full overflow-hidden">
<div
className="h-full bg-emerald-500 transition-all duration-300 rounded-full"
style={{ width: `${globalPct ?? 0}%` }}
/>
</div>
</div>
)}
</div>
</Card>
);
}
+185 -10
View File
@@ -1,10 +1,11 @@
import { useEffect, useState } from 'react';
import { api, Machine, SSHKey, TestConnectionResponse } from '../api/client';
import { api, Machine, SSHKey, TestConnectionResponse, deployKeys, DeployKeysResponse, shutdownMachine, ShutdownResponse } from '../api/client';
import { Button } from '@/components/ui/Button';
import { Input } from '@/components/ui/Input';
import { Label } from '@/components/ui/Label';
import { Select, SelectValue, SelectTrigger, SelectContent, SelectItem } from '@/components/ui/Select';
import { Badge } from '@/components/ui/Badge';
import { statusVariant } from '@/lib/status';
import {
Modal,
ModalContent,
@@ -26,7 +27,7 @@ import {
import { EmptyState } from '@/components/ui/EmptyState';
import { CopyButton } from '@/components/ui/CopyButton';
import { Card } from '@/components/ui/Card';
import { Pencil, Trash2, Plus, Server, Zap, Cable } from 'lucide-react';
import { Pencil, Trash2, Plus, Server, Zap, Cable, KeyRound, Power } from 'lucide-react';
import { toast } from 'sonner';
import { cn } from '@/lib/utils';
import { subscribeMachineStatus } from '@/lib/sse';
@@ -43,6 +44,7 @@ type MachineForm = {
broadcast_addr: string;
wake_timeout_seconds: number;
wake_check_interval_seconds: number;
shutdown_command: string;
};
const defaultForm: MachineForm = {
@@ -57,6 +59,7 @@ const defaultForm: MachineForm = {
broadcast_addr: '',
wake_timeout_seconds: 180,
wake_check_interval_seconds: 5,
shutdown_command: 'shutdown now',
};
export default function Machines() {
@@ -68,6 +71,8 @@ export default function Machines() {
const [loading, setLoading] = useState(false);
const [probing, setProbing] = useState(false);
const [connModal, setConnModal] = useState<{ machine: Machine | null; result: TestConnectionResponse | null; loading: boolean }>({ machine: null, result: null, loading: false });
const [deployModal, setDeployModal] = useState<{ machine: Machine | null; result: DeployKeysResponse | null; loading: boolean }>({ machine: null, result: null, loading: false });
const [shutdownModal, setShutdownModal] = useState<{ machine: Machine | null; result: ShutdownResponse | null; loading: boolean }>({ machine: null, result: null, loading: false });
useEffect(() => {
load();
@@ -117,6 +122,7 @@ export default function Machines() {
broadcast_addr: m.broadcast_addr || '',
wake_timeout_seconds: m.wake_timeout_seconds || 180,
wake_check_interval_seconds: m.wake_check_interval_seconds || 5,
shutdown_command: m.shutdown_command || 'shutdown now',
});
setModalOpen(true);
}
@@ -148,6 +154,7 @@ export default function Machines() {
broadcast_addr: form.broadcast_addr || null,
wake_timeout_seconds: Number(form.wake_timeout_seconds),
wake_check_interval_seconds: Number(form.wake_check_interval_seconds),
shutdown_command: form.shutdown_command || 'shutdown now',
};
await api(form.id ? `/api/machines/${form.id}` : '/api/machines', {
method: form.id ? 'PUT' : 'POST',
@@ -213,6 +220,20 @@ export default function Machines() {
setConnModal({ machine: null, result: null, loading: false });
}
async function handleDeployKeys(m: Machine) {
setDeployModal({ machine: m, result: null, loading: true });
try {
const result = await deployKeys(m.id);
setDeployModal({ machine: m, result, loading: false });
} catch (e: unknown) {
setDeployModal({ machine: m, result: { success: false, messages: [], errors: [(e as Error).message] }, loading: false });
}
}
function closeDeployModal() {
setDeployModal({ machine: null, result: null, loading: false });
}
function keyLabel(id: number | null) {
if (!id) return 'Server Key';
const k = sshKeys.find(k => k.id === id);
@@ -263,6 +284,7 @@ export default function Machines() {
<TableHead>WoL</TableHead>
<TableHead>WoL Timeout</TableHead>
<TableHead>Status</TableHead>
<TableHead>Last Seen</TableHead>
<TableHead className="w-24">Actions</TableHead>
</TableRow>
</TableHeader>
@@ -297,6 +319,11 @@ export default function Machines() {
<TableCell>
<StatusBadge status={m.status} />
</TableCell>
<TableCell>
<span className="text-xs text-fg-muted">
{m.last_seen_at ? new Date(m.last_seen_at).toLocaleString() : <span className="text-fg-subtle"></span>}
</span>
</TableCell>
<TableCell>
<div className="flex items-center gap-1">
<Button
@@ -315,6 +342,14 @@ export default function Machines() {
>
<Cable className="h-3.5 w-3.5" />
</Button>
<Button
variant="ghost"
size="icon-sm"
onClick={() => handleDeployKeys(m)}
title="Deploy SSH Keys"
>
<KeyRound className="h-3.5 w-3.5" />
</Button>
{m.wol_enabled && (
<Button
variant="ghost"
@@ -325,6 +360,15 @@ export default function Machines() {
<Zap className="h-3.5 w-3.5" />
</Button>
)}
<Button
variant="ghost"
size="icon-sm"
onClick={() => setShutdownModal({ machine: m, result: null, loading: false })}
title="Shutdown"
className="text-amber-400 hover:text-amber-300 hover:bg-amber-500/10"
>
<Power className="h-3.5 w-3.5" />
</Button>
<Button
variant="ghost"
size="icon-sm"
@@ -514,6 +558,20 @@ export default function Machines() {
</div>
</div>
)}
<div className="space-y-1.5">
<Label htmlFor="shutdown_command">Shutdown Command</Label>
<Input
id="shutdown_command"
placeholder="shutdown now"
value={form.shutdown_command}
onChange={e =>
setForm({ ...form, shutdown_command: e.target.value })
}
/>
<p className="text-xs text-fg-subtle">
Command sent via SSH to power off the machine. Leave blank to use the default.
</p>
</div>
</ModalBody>
<ModalFooter>
<Button
@@ -551,6 +609,76 @@ export default function Machines() {
</ModalContent>
</Modal>
<Modal open={shutdownModal.machine !== null} onOpenChange={v => !v && setShutdownModal({ machine: null, result: null, loading: false })}>
<ModalContent size="md">
<ModalHeader>
<ModalTitle>Shutdown Machine</ModalTitle>
<ModalDescription>
Are you sure you want to power off <strong>{shutdownModal.machine?.name}</strong>? You will need physical or remote access to power it back on.
</ModalDescription>
</ModalHeader>
<ModalBody className="space-y-4">
{shutdownModal.machine?.wol_enabled && (
<div className="rounded-card bg-sky-500/10 border border-sky-500/30 p-3 text-xs text-sky-300">
Wake-on-LAN is enabled you can power this machine back on remotely.
</div>
)}
{shutdownModal.loading && (
<div className="flex items-center justify-center py-8">
<div className="h-6 w-6 border-2 border-accent border-t-transparent rounded-full animate-spin" />
</div>
)}
{!shutdownModal.loading && shutdownModal.result && (
shutdownModal.result.success ? (
<div className="rounded-card bg-emerald-500/10 border border-emerald-500/30 p-4">
<p className="text-sm font-medium text-emerald-400">Shutdown command sent</p>
{shutdownModal.result.output && (
<pre className="text-xs text-fg-muted mt-1 whitespace-pre-wrap">{shutdownModal.result.output}</pre>
)}
</div>
) : (
<div className="rounded-card bg-rose-500/10 border border-rose-500/30 p-4">
<p className="text-sm font-medium text-rose-400">Shutdown failed</p>
<p className="text-xs text-fg-muted mt-1">{shutdownModal.result.error}</p>
</div>
)
)}
</ModalBody>
<ModalFooter>
<Button variant="secondary" onClick={() => setShutdownModal({ machine: null, result: null, loading: false })}>
{shutdownModal.result ? 'Close' : 'Cancel'}
</Button>
{!shutdownModal.result && (
<Button
variant="danger-solid"
loading={shutdownModal.loading}
onClick={async () => {
if (!shutdownModal.machine) return;
setShutdownModal(s => ({ ...s, loading: true }));
try {
const result = await shutdownMachine(shutdownModal.machine.id);
setShutdownModal({ machine: shutdownModal.machine, result, loading: false });
if (result.success) {
toast.success(`Shutdown command sent to ${shutdownModal.machine.name}`);
} else {
toast.error(`Shutdown failed: ${result.error}`);
}
} catch (e) {
setShutdownModal({
machine: shutdownModal.machine,
result: { success: false, error: (e as Error).message },
loading: false,
});
}
}}
>
Shutdown
</Button>
)}
</ModalFooter>
</ModalContent>
</Modal>
<Modal open={connModal.machine !== null} onOpenChange={v => !v && closeConnModal()}>
<ModalContent size="md">
<ModalHeader>
@@ -607,7 +735,7 @@ export default function Machines() {
<Button variant="secondary" onClick={closeConnModal}>
Close
</Button>
{!connModal.loading && connModal.result && !connModal.result.success && connModal.result.fingerprint && connModal.machine && !connModal.machine.fingerprint_confirmed && (
{!connModal.loading && connModal.result && connModal.result.fingerprint && connModal.machine && !connModal.machine.fingerprint_confirmed && (
<Button onClick={handleApproveFingerprint}>
Approve &amp; Trust Fingerprint
</Button>
@@ -615,16 +743,63 @@ export default function Machines() {
</ModalFooter>
</ModalContent>
</Modal>
<Modal open={deployModal.machine !== null} onOpenChange={v => !v && closeDeployModal()}>
<ModalContent size="md">
<ModalHeader>
<ModalTitle>Deploy SSH Keys</ModalTitle>
<ModalDescription>
{deployModal.machine?.name} ({deployModal.machine?.host}:{deployModal.machine?.port})
</ModalDescription>
</ModalHeader>
<ModalBody className="space-y-4">
{deployModal.loading && (
<div className="flex flex-col items-center justify-center py-8 gap-3">
<div className="h-6 w-6 border-2 border-accent border-t-transparent rounded-full animate-spin" />
<p className="text-sm text-fg-muted">Deploying keys...</p>
</div>
)}
{!deployModal.loading && deployModal.result && (
<div className="space-y-4">
{deployModal.result.success ? (
<div className="rounded-card bg-emerald-500/10 border border-emerald-500/30 p-4">
<p className="text-sm font-medium text-emerald-400 mb-2">Deployment successful</p>
<ul className="space-y-1">
{(deployModal.result.messages ?? []).map((msg, i) => (
<li key={i} className="text-xs text-fg-muted flex items-start gap-2">
<span className="text-emerald-400 mt-0.5">&#10003;</span>
{msg}
</li>
))}
</ul>
</div>
) : (
<div className="rounded-card bg-rose-500/10 border border-rose-500/30 p-4">
<p className="text-sm font-medium text-rose-400 mb-2">Deployment failed</p>
<ul className="space-y-1">
{(deployModal.result.errors ?? []).map((err, i) => (
<li key={i} className="text-xs text-fg-muted flex items-start gap-2">
<span className="text-rose-400 mt-0.5">&#10007;</span>
{err}
</li>
))}
</ul>
</div>
)}
</div>
)}
</ModalBody>
<ModalFooter>
<Button variant="secondary" onClick={closeDeployModal}>
Close
</Button>
</ModalFooter>
</ModalContent>
</Modal>
</div>
);
}
function StatusBadge({ status }: { status: string }) {
const variant =
status === 'online'
? 'success'
: status === 'offline'
? 'neutral'
: 'info';
return <Badge variant={variant} label={status} />;
return <Badge variant={statusVariant(status)} label={status} />;
}
+334
View File
@@ -0,0 +1,334 @@
import { useEffect, useState } from 'react';
import { api, SyncPair, Schedule } from '../api/client';
import { Button } from '@/components/ui/Button';
import { Input } from '@/components/ui/Input';
import { Label } from '@/components/ui/Label';
import { Select, SelectValue, SelectTrigger, SelectContent, SelectItem } from '@/components/ui/Select';
import {
Modal,
ModalContent,
ModalHeader,
ModalTitle,
ModalDescription,
ModalBody,
ModalFooter,
} from '@/components/ui/Modal';
import { PageHeader } from '@/components/ui/PageHeader';
import {
Table,
TableHeader,
TableBody,
TableHead,
TableRow,
TableCell,
} from '@/components/ui/Table';
import { EmptyState } from '@/components/ui/EmptyState';
import { Badge } from '@/components/ui/Badge';
import { Card } from '@/components/ui/Card';
import { Trash2, Plus, Clock, Pencil, AlertTriangle } from 'lucide-react';
import { toast } from 'sonner';
import { cn } from '@/lib/utils';
type ScheduleForm = {
id: number | undefined;
sync_pair_id: number | null;
cron_expr: string;
enabled: boolean;
};
const defaultForm: ScheduleForm = {
id: undefined,
sync_pair_id: null,
cron_expr: '',
enabled: true,
};
export default function Schedules() {
const [schedules, setSchedules] = useState<Schedule[]>([]);
const [syncPairs, setSyncPairs] = useState<SyncPair[]>([]);
const [modalOpen, setModalOpen] = useState(false);
const [deleteId, setDeleteId] = useState<number | null>(null);
const [form, setForm] = useState<ScheduleForm>(defaultForm);
const [loading, setLoading] = useState(false);
useEffect(() => {
load();
}, []);
async function load() {
try {
const [s, p] = await Promise.all([
api<Schedule[]>('/api/schedules'),
api<SyncPair[]>('/api/sync-pairs'),
]);
setSchedules(s);
setSyncPairs(p);
} catch {}
}
function openCreate() {
setForm(defaultForm);
setModalOpen(true);
}
async function handleSubmit(e: React.FormEvent) {
e.preventDefault();
if (!form.sync_pair_id) {
toast.error('Sync pair is required');
return;
}
if (!form.cron_expr.trim()) {
toast.error('Cron expression is required');
return;
}
setLoading(true);
try {
await api(form.id ? `/api/schedules/${form.id}` : '/api/schedules', {
method: form.id ? 'PUT' : 'POST',
body: {
sync_pair_id: form.sync_pair_id,
cron_expr: form.cron_expr,
enabled: form.enabled,
},
});
setModalOpen(false);
toast.success(form.id ? 'Schedule updated' : 'Schedule created');
load();
} catch (e: unknown) {
toast.error((e as Error).message);
} finally {
setLoading(false);
}
}
async function handleToggleEnabled(schedule: Schedule) {
try {
await api(`/api/schedules/${schedule.id}`, {
method: 'PUT',
body: {
cron_expr: schedule.cron_expr,
enabled: !schedule.enabled,
},
});
toast.success(`Schedule ${schedule.enabled ? 'disabled' : 'enabled'}`);
load();
} catch (e: unknown) {
toast.error((e as Error).message);
}
}
async function handleDelete() {
if (deleteId === null) return;
try {
await api(`/api/schedules/${deleteId}`, { method: 'DELETE' });
toast.success('Schedule deleted');
setDeleteId(null);
load();
} catch (e: unknown) {
toast.error((e as Error).message);
}
}
function syncPairName(id: number) {
const p = syncPairs.find(p => p.id === id);
return p ? p.name : `Sync Pair ${id}`;
}
function formatNextRun(nextRun: string | null) {
if (!nextRun) return 'Not scheduled';
const d = new Date(nextRun);
return d.toLocaleString();
}
return (
<div className="space-y-6">
<PageHeader
title="Schedules"
description="Automate sync pair execution with cron-based scheduling"
actions={
<Button onClick={openCreate} size="sm">
<Plus className="h-4 w-4" />
Add Schedule
</Button>
}
/>
<Card>
<div className="p-0">
{schedules.length === 0 ? (
<EmptyState
icon={<Clock className="h-5 w-5" />}
title="No schedules"
description="Create a schedule to automate sync pair execution"
action={
<Button onClick={openCreate} size="sm">
<Plus className="h-4 w-4" />
Add Schedule
</Button>
}
/>
) : (
<Table>
<TableHeader>
<TableRow>
<TableHead>Sync Pair</TableHead>
<TableHead>Cron Expression</TableHead>
<TableHead>Next Run</TableHead>
<TableHead>Status</TableHead>
<TableHead className="w-28">Actions</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{schedules.map(s => (
<TableRow key={s.id}>
<TableCell className="font-medium">{syncPairName(s.sync_pair_id)}</TableCell>
<TableCell>
<code className="text-xs bg-surface-raised px-2 py-1 rounded font-mono">
{s.cron_expr}
</code>
</TableCell>
<TableCell className="text-fg-muted text-sm">
{formatNextRun(s.next_run_at)}
</TableCell>
<TableCell>
<button
onClick={() => handleToggleEnabled(s)}
className={cn(
'relative inline-flex h-5 w-9 items-center rounded-full transition-colors focus:outline-none focus:ring-2 focus:ring-accent/40 focus:ring-offset-2 focus:ring-offset-canvas',
s.enabled ? 'bg-emerald-500/20' : 'bg-surface-raised'
)}
>
<span
className={cn(
'inline-block h-3.5 w-3.5 transform rounded-full bg-fg-muted transition-transform',
s.enabled ? 'translate-x-4 bg-emerald-400' : 'translate-x-1 bg-fg-subtle'
)}
/>
</button>
</TableCell>
<TableCell>
<div className="flex items-center gap-1">
<Button
variant="ghost"
size="icon-sm"
onClick={() => setDeleteId(s.id)}
className="text-rose-400 hover:text-rose-300 hover:bg-rose-500/10"
title="Delete"
>
<Trash2 className="h-3.5 w-3.5" />
</Button>
</div>
</TableCell>
</TableRow>
))}
</TableBody>
</Table>
)}
</div>
</Card>
<Modal open={modalOpen} onOpenChange={setModalOpen}>
<ModalContent size="md">
<ModalHeader>
<ModalTitle>Add Schedule</ModalTitle>
<ModalDescription>
Schedule automated sync pair execution using cron syntax.
Format: &quot;minute hour day-of-month month day-of-week&quot;
</ModalDescription>
</ModalHeader>
<form onSubmit={handleSubmit}>
<ModalBody className="space-y-4">
<div className="space-y-1.5">
<Label htmlFor="sch-sync-pair" required>
Sync Pair
</Label>
<Select
value={form.sync_pair_id?.toString() ?? ''}
onValueChange={v => setForm({ ...form, sync_pair_id: Number(v) })}
>
<SelectTrigger id="sch-sync-pair">
<SelectValue placeholder="Select a sync pair" />
</SelectTrigger>
<SelectContent>
{syncPairs.map(p => (
<SelectItem key={p.id} value={p.id.toString()}>
{p.name}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div className="space-y-1.5">
<Label htmlFor="sch-cron" required>
Cron Expression
</Label>
<Input
id="sch-cron"
placeholder="0 2 * * *"
value={form.cron_expr}
onChange={e => setForm({ ...form, cron_expr: e.target.value })}
className="font-mono"
/>
<p className="text-xs text-fg-subtle">
Format: &quot;m h dom mon dow&quot; (5 fields, no seconds)
<br />
Examples: &quot;0 2 * * *&quot; (daily at 2am), &quot;0 */6 * * *&quot; (every 6 hours)
</p>
</div>
<div
className={cn(
'flex items-center gap-2 rounded-card p-3 transition-colors',
form.enabled
? 'bg-accent/5 border border-accent/20'
: 'bg-surface-raised border border-border'
)}
>
<input
type="checkbox"
id="sch-enabled"
checked={form.enabled}
onChange={e => setForm({ ...form, enabled: e.target.checked })}
className="h-4 w-4 rounded border-border accent-accent"
/>
<Label htmlFor="sch-enabled" className="cursor-pointer mb-0">
Enable this schedule
</Label>
</div>
</ModalBody>
<ModalFooter>
<Button
type="button"
variant="secondary"
onClick={() => setModalOpen(false)}
>
Cancel
</Button>
<Button type="submit" loading={loading}>
Add Schedule
</Button>
</ModalFooter>
</form>
</ModalContent>
</Modal>
<Modal open={deleteId !== null} onOpenChange={v => !v && setDeleteId(null)}>
<ModalContent size="sm">
<ModalHeader>
<ModalTitle>Delete Schedule</ModalTitle>
<ModalDescription>
Are you sure you want to delete this schedule?
</ModalDescription>
</ModalHeader>
<ModalFooter>
<Button variant="secondary" onClick={() => setDeleteId(null)}>
Cancel
</Button>
<Button variant="danger-solid" onClick={handleDelete}>
Delete
</Button>
</ModalFooter>
</ModalContent>
</Modal>
</div>
);
}
+21 -2
View File
@@ -3,16 +3,21 @@ import { CopyButton } from '@/components/ui/CopyButton';
import { Button } from '@/components/ui/Button';
import { PageHeader } from '@/components/ui/PageHeader';
import { Card, CardHeader, CardTitle, CardBody } from '@/components/ui/Card';
import { Key, Download, Terminal } from 'lucide-react';
import { Key, Download, Terminal, HardDrive } from 'lucide-react';
import { api, SettingsInfo } from '../api/client';
export default function Settings() {
const [pubKey, setPubKey] = useState('');
const [dataDir, setDataDir] = useState('');
const [loading, setLoading] = useState(true);
useEffect(() => {
Promise.all([
fetch('/api/settings/pubkey', { credentials: 'include' })
.then(r => (r.ok ? r.text() : ''))
.then(t => setPubKey(t))
.then(t => setPubKey(t)),
api<SettingsInfo>('/api/settings/info').then(info => setDataDir(info.data_dir)),
])
.catch(() => {})
.finally(() => setLoading(false));
}, []);
@@ -75,6 +80,20 @@ export default function Settings() {
</CardBody>
</Card>
<Card>
<CardHeader>
<div className="flex items-center gap-2">
<div className="rounded-card bg-accent/10 p-1">
<HardDrive className="h-4 w-4 text-accent" />
</div>
<CardTitle>Data Directory</CardTitle>
</div>
</CardHeader>
<CardBody>
<div className="text-sm font-mono text-fg">{dataDir || '-'}</div>
</CardBody>
</Card>
<Card>
<CardHeader>
<div className="flex items-center gap-2">
+45 -3
View File
@@ -26,7 +26,7 @@ import {
import { EmptyState } from '@/components/ui/EmptyState';
import { Badge } from '@/components/ui/Badge';
import { Card } from '@/components/ui/Card';
import { Play, Trash2, Plus, GitCompare, ArrowRight, ArrowLeft, Pencil } from 'lucide-react';
import { Play, Trash2, Plus, GitCompare, ArrowRight, ArrowLeft, Pencil, AlertTriangle } from 'lucide-react';
import { toast } from 'sonner';
import { cn } from '@/lib/utils';
@@ -111,6 +111,14 @@ export default function SyncPairs() {
toast.error('Source and destination paths are required');
return;
}
if (
form.source_machine_id !== null &&
form.dest_machine_id !== null &&
form.source_machine_id === form.dest_machine_id
) {
toast.error('Source and destination cannot be the same machine');
return;
}
setLoading(true);
try {
await api(form.id ? `/api/sync-pairs/${form.id}` : '/api/sync-pairs', {
@@ -287,8 +295,8 @@ export default function SyncPairs() {
<ModalTitle>{form.id ? 'Edit Sync Pair' : 'Add Sync Pair'}</ModalTitle>
<ModalDescription>
{form.id
? 'Update the configuration for this sync pair'
: 'Define a new source and destination for data syncing'}
? 'Update this sync pair. Direction: push (src→dst), pull (dst←src), or mirror (push + --delete).'
: 'Define source and destination for data syncing. Pick a direction: push (src→dst), pull (dst←src), or mirror (push + --delete).'}
</ModalDescription>
</ModalHeader>
<form onSubmit={handleSubmit}>
@@ -325,6 +333,9 @@ export default function SyncPairs() {
))}
</SelectContent>
</Select>
<p className="text-xs text-fg-subtle">
Where the data lives. Pick <span className="font-medium">Local server</span> if it&apos;s on this machine, otherwise the remote machine holding the data.
</p>
</div>
<div className="space-y-1.5">
<Label htmlFor="sp-dest-machine">Dest Machine</Label>
@@ -346,6 +357,9 @@ export default function SyncPairs() {
))}
</SelectContent>
</Select>
<p className="text-xs text-fg-subtle">
Where to copy the data. Can be the same machine (no-op) or any remote. Remote-to-remote is supported.
</p>
</div>
</div>
<div className="grid grid-cols-2 gap-3">
@@ -361,6 +375,9 @@ export default function SyncPairs() {
setForm({ ...form, source_path: e.target.value })
}
/>
<p className="text-xs text-fg-subtle">
Directory on the source machine. Its <span className="font-medium">contents</span> will be copied into the destination. Trailing <code className="font-mono">/</code> is optional.
</p>
</div>
<div className="space-y-1.5">
<Label htmlFor="sp-dest-path" required>
@@ -374,6 +391,9 @@ export default function SyncPairs() {
setForm({ ...form, dest_path: e.target.value })
}
/>
<p className="text-xs text-fg-subtle">
Directory on the destination machine where the source contents will land.
</p>
</div>
</div>
<div className="grid grid-cols-2 gap-3">
@@ -394,6 +414,12 @@ export default function SyncPairs() {
<SelectItem value="mirror">Mirror</SelectItem>
</SelectContent>
</Select>
<p className="text-xs text-fg-subtle">
<span className="font-medium">Push</span> copies source destination.{' '}
<span className="font-medium">Pull</span> reverses it (dest source).{' '}
<span className="font-medium">Mirror</span> is like push but adds{' '}
<code className="font-mono">--delete</code> (see warning).
</p>
</div>
<div className="space-y-1.5">
<Label htmlFor="sp-rsync-flags">Rsync Flags</Label>
@@ -407,6 +433,22 @@ export default function SyncPairs() {
/>
</div>
</div>
{form.direction === 'mirror' && (
<div
role="alert"
className="flex items-start gap-2 rounded-card p-3 bg-amber-500/10 border border-amber-500/30"
>
<AlertTriangle className="h-4 w-4 text-amber-400 shrink-0 mt-0.5" />
<div className="text-xs text-amber-200 leading-relaxed">
<span className="font-semibold">Mirror will delete files.</span>{' '}
With <code className="font-mono">--delete</code>, any file in the
destination that doesn&apos;t exist in the source is permanently
removed. Double-check both paths before saving a wrong
destination (e.g. <code className="font-mono">/</code> or{' '}
<code className="font-mono">/home</code>) can wipe data.
</div>
</div>
)}
<div className="space-y-1.5">
<Label htmlFor="sp-exclude">Exclude Patterns</Label>
<Textarea