SyncServer

Self-hosted file sync orchestrator with Wake-on-LAN and web UI. Orchestrates rsync over SSH between multiple machines from a single central server.

Features

• Web UI: React-based interface to manage machines, sync pairs, and job history
• Wake-on-LAN: Power on remote machines before syncing (UDP magic packet)
• Rsync over SSH: Efficient file synchronization with progress tracking
• Scheduled syncs: Cron-style scheduler for automated backups
• Live progress: Server-Sent Events (SSE) for real-time job status
• Single binary: No dynamic dependencies, pure Go with embedded SQLite

Architecture

┌─────────────────────────────────────┐
│         SyncServer (Go binary)        │
│  ┌─────────────────────────────────┐ │
│  │  Web UI (React, embedded)       │ │
│  │  REST API (chi router)          │ │
│  │  Auth (JWT + bcrypt)            │ │
│  │  Scheduler (cron)               │ │
│  │  Sync Engine (rsync runner)     │ │
│  │  WoL (UDP magic packet)         │ │
│  │  SQLite (embedded)              │ │
│  └─────────────────────────────────┘ │
└──────────────┬────────────────────────┘
               │ SSH + UDP
     ┌─────────┴──────────┐
     ▼                   ▼
  Machine A          Machine B
  (rsync+sshd)       (rsync+sshd)

Network Requirements for Wake-on-LAN

Critical: The magic packet (WoL) only works when the SyncServer is on the same L2 network (same broadcast domain) as the target machine.

Proxmox LXC Setup

• Use bridge networking (vmbr0) — NOT NAT
• The LXC must share the same VLAN/LAN as the machines it needs to wake
• No special LXC privileges required (works with unprivileged containers)
• SO_BROADCAST works in unprivileged LXC containers

If your LXC is on a different subnet, use the broadcast_addr field on the machine to specify the broadcast address of the target machine's subnet (e.g., 192.168.1.255).

BIOS/UEFI Requirements on Target Machines

• Enable Wake-on-LAN in BIOS/UEFI
• Set to "Power On by PCIe" or "Wake by PCI-E"
• Some motherboards require both PCIe and LAN WoL to be enabled

Installation

Quick Start (Development)

git clone https://github.com/syncserver/syncserver
cd syncserver

# Run with default settings (data in ./data)
SYNCSERVER_ADMIN_USER=admin SYNCSERVER_ADMIN_PASSWORD=secret \
  ./syncserver --addr :8080

# Open http://your-server:8080

Production Install (.deb)

# Transfer to your Proxmox LXC (Debian/Ubuntu)
scp dist/syncserver_*.deb root@your-lxc:/tmp/

# On the LXC:
dpkg -i /tmp/syncserver_*.deb
# postinst will:
#   - Create syncserver user
#   - Create /var/lib/syncserver/{data,ssh,logs}
#   - Install config at /etc/syncserver/config.yaml
#   - Enable and start the systemd service

# To configure the admin user on first run:
# Set environment variables BEFORE first start (in /etc/syncserver/config.yaml or systemd unit):

Configuration

Edit /etc/syncserver/config.yaml:

data_dir: /var/lib/syncserver
addr: :8080

auth:
  jwt_secret: "your-secret-here"  # Generate with: openssl rand -hex 32
  jwt_expiry_hours: 24

scheduler:
  timezone: UTC

Or use environment variables (override config file):

SYNCSERVER_ADMIN_USER=admin SYNCSERVER_ADMIN_PASSWORD=secret \
SYNCSERVER_JWT_SECRET=your-secret \
SYNCSERVER_DATA_DIR=/var/lib/syncserver \
/usr/bin/syncserver --config /etc/syncserver/config.yaml

Setup: SSH Keys

Step 1: Get the Server's Public Key

In the UI, go to Settings to copy the server's SSH public key. It looks like:

ssh-ed25519 AAAA... syncserver

Step 2: Add the Key to Remote Machines

On each remote machine, add the public key to ~/.ssh/authorized_keys:

# As root on the remote machine:
echo "ssh-ed25519 AAAA... syncserver" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Or use ssh-copy-id:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@remote-machine
# Then replace the key in the remote's authorized_keys with the syncserver key

Step 3: Register Machines in the UI

Go to Machines → Add Machine:

• Name: e.g., backup-nas
• Host: IP or hostname (e.g., 192.168.1.100)
• SSH Port: 22
• SSH User: root (or your preferred user)
• SSH Key: leave empty to use the server's default key

For Wake-on-LAN:

• Enable "Wake-on-LAN"
• Enter the MAC address (e.g., AA:BB:CC:DD:EE:FF)
• Optionally set a broadcast address (leave empty for 255.255.255.255)
• Configure timeout and check interval

Creating Sync Pairs

Go to Sync Pairs → Add Sync Pair:

Field	Description
Name	Descriptive name, e.g., backup-home
Source	Machine + path (or "Local server" for the syncserver itself)
Destination	Machine + path
Direction	push (source→dest), pull (dest→source), mirror (bidirectional with delete)
Rsync Flags	Default: -aP (archive, progress). Add --delete for mirror mode.
Exclude Patterns	One pattern per line, e.g., *.tmp, node_modules/

Scheduling

Add a schedule to any sync pair:

Cron Expr	Description
0 2 * * *	Daily at 2 AM
0 */6 * * *	Every 6 hours
0 9-17 * * 1-5	Business hours, weekdays
*/15 * * * *	Every 15 minutes

All schedules run in UTC by default. Set scheduler.timezone in config to change.

Job States

State	Description
queued	Waiting for a worker
waking_up	Sending WoL magic packet and waiting for machine
running	rsync in progress
success	Completed successfully
failed	Error during sync
cancelled	User cancelled the job

Logs

Job logs are stored at /var/lib/syncserver/logs/<job_id>.log.

Application logs go to systemd journal:

journalctl -u syncserver -f

Upgrade

# On the LXC:
dpkg -i /tmp/syncserver-new-version.deb
# The service will automatically restart

Config at /etc/syncserver/config.yaml is preserved (conffile).

Build from Source

# Requires: Go 1.22+, Node 18+, npm

./scripts/build.sh
# Output: packaging/debian/usr/bin/syncserver

./scripts/package-deb.sh
# Output: dist/syncserver_VERSION_amd64.deb

For ARM64:

GOARCH=arm64 ./scripts/build.sh
ARCH=arm64 ./scripts/package-deb.sh

Data Storage

/var/lib/syncserver/
├── data/
│   └── app.db        # SQLite database (machines, sync_pairs, jobs, etc.)
├── ssh/
│   ├── id_ed25519     # Server's private SSH key
│   ├── id_ed25519.pub # Server's public SSH key
│   └── known_hosts    # Host keys of registered machines
└── logs/
    └── <job_id>.log   # Per-job rsync output logs

Recommended: Bind mount /var/lib/syncserver to persistent storage outside the LXC root filesystem. This way you can recreate the LXC without losing data.

In Proxmox LXC config (/etc/pve/lxc/<id>.conf):

mp0: /mnt/data/syncserver,mp=/var/lib/syncserver,backup=0

Troubleshooting

WoL doesn't work

1. Verify the LXC is on the same L2 network as the target
2. Check ip link in the LXC shows the bridge interface
3. Try: tcpdump -i <iface> udp port 9 in the LXC while triggering WoL
4. Verify WoL is enabled in the target machine's BIOS
5. Try sending WoL from another machine on the same subnet to isolate the issue

SSH connection fails

1. Verify the public key is in ~/.ssh/authorized_keys on the target
2. Check sshd is running on the target machine
3. Try manually: ssh -i /var/lib/syncserver/ssh/id_ed25519 user@host
4. Check the target's sshd_config: PubkeyAuthentication yes, AuthorizedKeysFile .ssh/authorized_keys

rsync fails with "connection unexpectedly closed"

• Usually means SSH key auth failed
• Check the job log for the exact SSH error
• Try running the rsync command manually from the server with -v flag

Machine shows "unknown" status

• Status is updated when a job runs or when you test SSH
• It does not auto-poll; this is by design to avoid network load

API Reference

All API endpoints require authentication (JWT cookie).

Method	Path	Description
POST	/api/auth/login	Login with username/password
POST	/api/auth/logout	Clear session
GET	/api/auth/me	Current user info
GET/POST	/api/machines	List/create machines
GET/PUT/DELETE	/api/machines/{id}	Get/update/delete machine
GET/POST	/api/sync-pairs	List/create sync pairs
GET/PUT/DELETE	/api/sync-pairs/{id}	Get/update/delete sync pair
POST	/api/sync-pairs/{id}/run	Trigger manual sync
GET	/api/jobs	List jobs
POST	/api/jobs/{id}/cancel	Cancel a running job
GET	/api/jobs/{id}/log	Stream job log
GET	/api/jobs/stream	SSE stream of all job events
GET	/api/settings/pubkey	Get server's SSH public key

License

MIT