SysAdmin & Self-hosting

This page summarizes practical knowledge shared in the group about running home servers, small services, and general system administration with a bias towards Debian and similar systems.

Debian is a frequent choice for self-hosting:

• Use Debian Stable for servers
  • Predictable updates and long-term support.
  • Ideal for “set and forget” services like web servers, Matrix homeservers, VPN endpoints, and small databases.

• Situations where people moved to testing/unstable:
  • Need for newer versions of virtualization software (e.g. VirtualBox) or GPU drivers.
  • More recent language runtimes (Node.js, Python, etc.) than stable provides.

Recommendations:

• For internet-exposed services: stay on stable unless there is a compelling reason.
• If you must use testing:
  • Pin packages where possible (`/etc/apt/preferences.d`).
  • Keep backups you can quickly restore on stable.

Common hardening patterns that came up repeatedly:

• Automatic security updates
  • On Debian/Ubuntu: install `unattended-upgrades` and enable it:

      sudo apt install unattended-upgrades
      sudo dpkg-reconfigure unattended-upgrades
      

• Firewall basics
  • Keep a default-deny posture:
    • Allow only SSH + specific service ports (e.g. 80/443 for web, 51820 for WireGuard).
    • Use `ufw` or plain `iptables/nftables` as you prefer.

• fail2ban
  • Protects SSH and web services against brute force.
    • Install:

      sudo apt install fail2ban

    • Copy a jail config: `/etc/fail2ban/jail.local`
    • At minimum, enable `sshd` jail.
  • For web apps behind nginx, enable HTTP auth and nginx jails to throttle abusive clients.

• Backups
  • Use `borg`, `restic`, or `rsnapshot` for simple pull-style backups.
  • Test restore regularly.

Several directions emerged in the discussions:

• VirtualBox on Linux
  • Useful for desktop experimentation, but:
    • Can break with kernel updates (DKMS issues).
    • On some distros/releases it has weaker security support.
  • Typical fix when VMs stop booting:
    • Reinstall kernel headers and VirtualBox DKMS modules:

        sudo apt install --reinstall \
            "linux-headers-$(uname -r)" \
            virtualbox-dkms dkms
        

• virt-manager on Linux
  • Harder to setup than VirtualBox, but:
    • By default uses libvirt and KVM (better performance than VirtualBox).
    • VirtualBox is a type-2 hypervisor and KVM is a type-1 hypervisor.
    • Provides a GUI which makes it easier to manage a higher number of VMs.
    • Can manage remote VMs (via network).
    • Supports running VM snapshots.
  • Example on how to setup virt-manager on Arch Linux:

    # Download the script
    curl -LO https://github.com/oliveiraleo/virt-manager-config-arch-linux/raw/main/virt-manager-setup.sh && \
    # Give it exec permission and run
    chmod +x virt-manager-setup.sh && \
    ./virt-manager-setup.sh

• The source code repository is hosted on GitHub
• Tips for self-hosting:
  • libvirt's networking docs
  • Instead of creating a bridge on the host or dealing with a ton of firewall rules (as suggested in the point above), for self-hosted setups where you can configure the routes of the upstream router, it is easier to use the "routed" forward mode.
  • If using the routed mode, use the host's IP as the GW IP of your route (remember to configure the VM's firewall as it will expose service ports to the host's network).
  • If exposing the environment to the internet, it is recommended to setup a firewall (such as OPNSense) to better control networking aspects of the host and its VM's.

• Docker for Services
  • Often used to encapsulate:
    • Databases (PostgreSQL, MariaDB).
    • Web apps (Node.js, Python, Go).
    • Reverse proxies (nginx, Traefik) as containers.
  • Helpful patterns:
    • Compose file per “stack” (e.g. `docker-compose.yml` under `/srv/<service>`).
    • Bind-mount configuration to host (so you can edit configs without rebuilding images).

• When to use containers vs bare metal
  • Containers:
    • Easy deployment and rollback.
    • Good for isolating apps with conflicting dependencies.
  • Bare metal / systemd services:
    • Simpler in very small setups.
    • Avoids Docker as a single point of failure.

nginx frequently appears as the unified entry point for web services:

• Run nginx on the host (or in a well-secured container) listening on ports 80/443.
• Use SNI + reverse proxying to route traffic to internal services:
  • `matrix.example.com` → Matrix homeserver
  • `git.example.com` → Gitea/Forgejo
  • `cloud.example.com` → Nextcloud

• Let’s Encrypt:
  • Use `certbot` or built-in nginx plugins.
  • Automate renewal with systemd timers or `cron`.

Common self-hosted components include:

• Git hosting: Gitea or Forgejo.
• Static site / blog: nginx serving static content generated by Hugo, Jekyll or similar.
• Personal cloud: Nextcloud for file sync & contacts/calendar.
• RSS / Fediverse:
  • Miniflux, Tiny Tiny RSS, or self-hosted ActivityPub platforms.

Core design rule seen in chat: keep your stack as simple as possible. Fewer moving parts = fewer 3 AM emergencies.