homeserver initial commit

- ansible - docker-compose - Kubernetes_deployments
2025-02-12 20:10:56 +02:00
commit e5e8aa6b87
70 changed files with 2860 additions and 0 deletions
--- a/docker_compose/.gitignore
+++ b/docker_compose/.gitignore
@ -0,0 +1 @@
+.env
--- a/docker_compose/README.md
+++ b/docker_compose/README.md
@ -0,0 +1,246 @@
+Homeserver Notes
+================
+
+# Future Plan
+
+  - Add authentication frontend like Authentik which will handle the authentication
+  - Add Nextcloud
+  - Add Gitea
+
+# List of Service Running on Homeserver
+
+    - Adguard
+    - Plex
+    - Sonarr
+    - Radarr
+    - qbittorrent
+    - Portainer
+    - Jackett
+    - Jellyfin
+    - Wireguard
+
+# List of Basic CLI tools installed on server
+
+    - ca-certificates
+    - curl
+    - gnupg
+    - lsb-release
+    - ntp
+    - ncdu
+    - net-tools
+    - apache2-utils
+    - apt-transport-https
+    - htop
+
+
+# Firewall Rules (Currently Disabled)
+
+I am using ufw to set different firewall rules. As I go I will update the rules
+
+    ```
+    sudo ufw default allow outgoing
+    sudo ufw default allow incoming
+    sudo ufw allow from 192.168.1.0/24
+    sudo ufw allow 443
+    sudo ufw allow 80
+    sudo ufw enable
+    ```
+
+# Hardware Transcoding for Jellyfin
+
+The media stream applications such as Jellyfin and Plex uses transcoding to
+convert video format which might be necessary if the end user device does not
+support some video formats or resolution. If hardware transcoding is not enabled
+Plex/Jellyfin uses software based transcoding which is resource intensive.
+
+Most of the new CPU/GPU support HW based transcoding. For our Ryzen 5 2500U
+processor, we have HW transcoding. Here is the process to enable it:
+
+    ```
+    sudo apt-get update
+    sudo apt-get install vainfo mesa-va-drivers libva2 libva-utils
+
+    # Run the following command to make sure VA API is working properly
+    vainfo
+
+    # Add the following to services to the Jellyfin container compose file
+    devices:
+      - /dev/dri/renderD128:/dev/dri/renderD128  # VA-API device for hardware acceleration
+    group_add:
+      - video  # Add the container to the 'video' group
+
+    ```
+
+
+# Traefik Reverse proxy
+
+   - Traefik is modern HTTP reverse proxy and load balancerthat can be used to route
+     traffic to different internal containers or ports based on subdomain name.
+
+   - In addition to that It can also automatically handle SSL certificate genertion
+     and renewal for HTTPS automatically handle SSL certificate genertion
+     and renewal for HTTPS.
+
+
+## Configuration
+
+In order to get wildcard certificates from LetsEncrypt, I will be using DNS challange
+method. DNS challange method is one of the methods provided by LetsEncrypt to verify
+the ownership of the domain by adding specific DNS records.
+
+To do that with cloudflare, I have created a new API token with name _CF_DNS_API_TOKEN_
+and saved it as docker secret under ~/docker/secrets directory
+```
+# To save the appdata for traefik3, created the following folders
+mkdir -p ~/docker/appdata/traefik3/acme
+mkdir -p ~/docker/appdata/traefik3/rules/udms
+
+# To save teh LetsEncrypt certificate, created the following file
+touch acme.json
+chmod 600 acme.json  # without 600, Traefik will not start
+
+# To save logs, created following files
+touch traefik.log
+touch access.log
+```
+After creating the Docker Compose file, add these TLS options like this:
+```
+# Under DOCKERDIR/appdata/traefik3/rules/udms/tls-opts.yml
+tls:
+  options:
+    tls-opts:
+      minVersion: VersionTLS12
+      cipherSuites:
+        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
+        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
+        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
+        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
+        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
+        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
+        - TLS_AES_128_GCM_SHA256
+        - TLS_AES_256_GCM_SHA384
+        - TLS_CHACHA20_POLY1305_SHA256
+        - TLS_FALLBACK_SCSV # Client is doing version fallback. See RFC 7507
+      curvePreferences:
+        - CurveP521
+        - CurveP384
+      sniStrict: true
+```
+Add the middleware Basic Auth:
+```
+# Under DOCKERDIR/appdata/traefik3/rules/udms/middlewares-basic-auth.yml
+http:
+  middlewares:
+    middlewares-basic-auth:
+      basicAuth:
+        # users:
+        #   - "user:password"
+        usersFile: "/run/secrets/basic_auth_credentials"
+        realm: "Traefik 3 Basic Auth"
+
+```
+Add middleware rate limited to prevent DDoS attack
+```
+# Under DOCKERDIR/appdata/traefik3/rules/udms/middlewares-rate-limit.yaml
+http:
+  middlewares:
+    middlewares-rate-limit:
+      rateLimit:
+        average: 100
+        burst: 50
+```
+Add secure headers middleware
+```
+# Under DOCKERDIR/appdata/traefik3/rules/udms/middlewares-secure-headers.yaml
+http:
+  middlewares:
+    middlewares-secure-headers:
+      headers:
+        accessControlAllowMethods:
+          - GET
+          - OPTIONS
+          - PUT
+        accessControlMaxAge: 100
+        hostsProxyHeaders:
+          - "X-Forwarded-Host"
+        stsSeconds: 63072000
+        stsIncludeSubdomains: true
+        stsPreload: true
+        # forceSTSHeader: true # This is a good thing but it can be tricky. Enable after everything works.
+        customFrameOptionsValue: SAMEORIGIN # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
+        contentTypeNosniff: true
+        browserXssFilter: true
+        referrerPolicy: "same-origin"
+        permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=(), usb=(), vr=()"
+        customResponseHeaders:
+          X-Robots-Tag: "none,noarchive,nosnippet,notranslate,noimageindex," # disable search engines from indexing home server
+          server: "" # hide server info from visitors
+```
+
+## Networking
+
+Create a default Bridge network for the Traefik
+
+# Wireguard VPN setup
+
+In order for qbittorrent container to use the wireguard VPN tunnel
+wireguard container has been added to the qbittorrent docker compose
+file.
+  - qbittorrent container depends on the wireguard container. If
+    wireguard container is down, qbittorrent network will not work.
+
+  - Since, qbittorrent is using the wireguard container, port 9500
+    has been forwared to the host 9500 port from the wireguard container
+
+  -  qbittorrent is using wireguard network interface. So, to access
+     the qbittorrent GUI, iptables rules had to be setup. Also, when the pc restarts
+     the wireguard container IP might change.
+
+      ```
+      # Forward traffic coming to port 9500 on the host to port 9500 on the WireGuard container
+      sudo iptables -t nat -A PREROUTING -p tcp --dport 9500 -j DNAT --to-destination 172.18.0.6:9500
+
+      # Forward traffic from the WireGuard container back to the host's port 9500
+      sudo iptables -t nat -A POSTROUTING -p tcp -d 172.18.0.6 --dport 9500 -j MASQUERADE
+      ```
+  - We can check the host ip geolocation by the following command. In that way
+    we can verify VPN is working.
+    ```
+    docker exec -it qbittorrent curl ipinfo.io
+
+    {
+      "ip": "1.2.3.4",
+      "hostname": "1.2.3.4.in-addr.arpa",
+      "city": "Amsterdam",
+      "region": "North Holland",
+      "country": "NL",
+      "loc": "55.3740,41.8897",
+      "org": "Some Company",
+      "postal": "1234",
+      "timezone": "Europe/Amsterdam",
+      "readme": "https://ipinfo.io/missingauth"
+    }
+    ```
+  - We can check the wireguard VPN connection status with the following command
+    ```
+    docker exec -it wireguard wg
+
+    interface: wg0
+      public key: <public key>
+      private key: (hidden)
+      listening port: 56791
+      fwmark: 0xca6c
+
+    peer: <public key>
+      preshared key: (hidden)
+      endpoint: <ip>:51820
+      allowed ips: 0.0.0.0/0, ::/0
+      latest handshake: 1 minute, 47 seconds ago
+      transfer: 12.69 MiB received, 822.64 KiB sent
+      persistent keepalive: every 15 seconds
+    ```
+
+# FAQ
+
+1. How to get the plex claim?
+-> Go the the url and login: https://www.plex.tv/claim/
--- a/docker_compose/jackett.yaml
+++ b/docker_compose/jackett.yaml
@ -0,0 +1,18 @@
+version: "3"
+services:  
+  jackett:
+    image: "linuxserver/jackett"
+    container_name: "jackett"
+    env_file:
+        ./.env
+    volumes:
+      - ${DOCKERDIR}/appdata/jackett:/config
+      - ${DATADIR}/downloads:/downloads
+      - "/etc/localtime:/etc/localtime:ro"
+    ports:
+      - "9117:9117"
+    restart: unless-stopped
+    environment:
+      - PUID=${PUID}
+      - PGID=${PGID}
+      - TZ=${TZ}
--- a/docker_compose/jellyfin.yaml
+++ b/docker_compose/jellyfin.yaml
@ -0,0 +1,38 @@
+  services:
+    jellyfin:
+      image: jellyfin/jellyfin
+      container_name: jellyfin
+      env_file:
+        - ./.env
+      volumes:
+        - ${DOCKERDIR}/appdata/jellyfin:/config
+        - ${DATADIR}/downloads:/downloads
+        - type: bind
+          source: ${DATADIR}
+          target: /media
+          read_only: true
+      ports:
+        - "8096:8096"  # Optional, if you rely on Traefik for routing, this can be removed.
+      restart: 'unless-stopped'
+      devices:
+        - /dev/dri/renderD128:/dev/dri/renderD128  # VA-API device for hardware acceleration
+      group_add:
+        - video  # Add the container to the video group necessary for accesing /dev/dri
+      environment:
+        - PUID=${PUID}
+        - PGID=${PGID}
+        - TZ=${TZ}
+        - UMASK_SET=002
+      labels:
+        - "traefik.enable=true"
+        - "traefik.http.routers.jellyfin-rtr.rule=Host(`jellyfin.${DOMAINNAME}`)"
+        - "traefik.http.routers.jellyfin-rtr.entrypoints=websecure"
+        - "traefik.http.routers.jellyfin-rtr.service=jellyfin-svc"
+        - "traefik.http.services.jellyfin-svc.loadbalancer.server.port=8096"
+        - "traefik.http.routers.traefik-rtr.middlewares=middlewares-rate-limit@file,middlewares-secure-headers@file"
+      networks:
+        - t3_proxy
+
+  networks:
+    t3_proxy:
+      external: true
--- a/docker_compose/plex.yaml
+++ b/docker_compose/plex.yaml
@ -0,0 +1,40 @@
+version: '3.5'
+services:
+  plex:
+    image: plexinc/pms-docker 
+    container_name: plex
+    env_file:
+        ./.env
+    environment:
+      - PLEX_UID=${PUID}
+      - PLEX_GID=${PGID}
+      - TZ=${TZ}
+      - VERSION=docker
+      - PLEX_CLAIM=${PLEX_CLAIM}
+    ports:
+      - "32400:32400/tcp"
+      - "3005:3005/tcp"
+      - "8324:8324/tcp"
+      - "32469:32469/tcp"
+      - "1899:1900/udp"
+      - "32410:32410/udp"
+      - "32412:32412/udp"
+      - "32413:32413/udp"
+      - "32414:32414/udp"
+    volumes:
+      - ${DOCKERDIR}/appdata/plex:/config
+      - ${DATADIR}/tvshows:/tv
+      - ${DATADIR}/movies:/movies
+    restart: unless-stopped
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.plex-rtr.rule=Host(`plex.${DOMAINNAME}`)"
+      - "traefik.http.routers.plex-rtr.entrypoints=websecure"
+      - "traefik.http.routers.plex-rtr.service=plex-svc"
+      - "traefik.http.services.plex-svc.loadbalancer.server.port=32400"
+      - "traefik.http.routers.traefik-rtr.middlewares=middlewares-rate-limit@file,middlewares-secure-headers@file"
+    networks:
+      - t3_proxy
+networks:
+  t3_proxy:
+    external: true
--- a/docker_compose/portainer.yaml
+++ b/docker_compose/portainer.yaml
@ -0,0 +1,35 @@
+version: "3"
+services:
+  portainer:
+    image: portainer/portainer-ce:latest
+    ports:
+      - 9000:9000
+    volumes:
+      - /home/taqi/docker/portainer/data:/data
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+    restart: unless-stopped
+    env_file:
+      - ./.env
+    networks:
+      - t3_proxy
+    labels:
+      - "traefik.enable=true"
+      # HTTP Routers
+      - "traefik.http.routers.portainer-rtr.entrypoints=websecure"
+      - "traefik.http.routers.portainer-rtr.rule=Host(`portainer.${DOMAINNAME}`)"
+      # HTTP Services
+      - "traefik.http.routers.portainer-rtr.tls=true"
+      - "traefik.http.routers.portainer-rtr.service=portainer-svc"
+      - "traefik.http.services.portainer-svc.loadbalancer.server.port=9000"
+      - "traefik.http.routers.traefik-rtr.middlewares=middlewares-rate-limit@file,middlewares-secure-headers@file"
+    command: 
+      --http-enabled
+    environment:
+      - TZ=${TZ}
+      - DOMAINNAME=${DOMAINNAME}
+volumes:
+  data:
+
+networks:
+  t3_proxy:
+    external: true
--- a/docker_compose/qbittorrent.yaml
+++ b/docker_compose/qbittorrent.yaml
@ -0,0 +1,52 @@
+version: "3"
+
+services:
+  wireguard:
+    image: linuxserver/wireguard:latest
+    container_name: wireguard
+    env_file:
+        ./.env
+    cap_add:
+      - NET_ADMIN
+      - SYS_MODULE
+    environment:
+      - PUID=${PUID}
+      - PGID=${PGID}
+      - TZ=Europe/Helsinki
+    volumes:
+      - ${WIREGUARD_CONFIG}:/config/wg0.conf
+      - /lib/modules:/lib/modules
+    ports:
+      - 51820:51820/udp
+      - 9500:9500  # qbittorrent
+    sysctls:
+      - net.ipv4.conf.all.src_valid_mark=1
+      - net.ipv6.conf.all.disable_ipv6=0
+    restart: unless-stopped
+    networks:
+      dockercompose_default:
+        ipv4_address: 172.18.0.100
+
+
+  qbittorrent: 
+    image: "linuxserver/qbittorrent"
+    container_name: "qbittorrent"
+    env_file:
+        ./.env
+    volumes:
+      - ${DOCKERDIR}/appdata/qbittorrent:/config
+      - ${DATADIR}/downloads:/downloads
+    restart: unless-stopped
+    environment:
+      - PUID=${PUID}
+      - PGID=${PGID}
+      - TZ=${TZ}
+      - UMASK_SET=002
+      - WEBUI_PORT=9500
+    network_mode: service:wireguard 
+    depends_on:
+      - wireguard
+
+networks:
+  dockercompose_default:
+    external: true
--- a/docker_compose/radarr.yaml
+++ b/docker_compose/radarr.yaml
@ -0,0 +1,21 @@
+version: "3"
+services:  
+  radarr:
+    image: "linuxserver/radarr"
+    container_name: "radarr"
+    env_file:
+        ./.env
+    volumes:
+      - ${DOCKERDIR}/appdata/radarr:/config
+      - ${DATADIR}/downloads:/downloads
+      - ${DATADIR}/movies:/movies
+      - "/etc/localtime:/etc/localtime:ro"
+    ports:
+      - "7878:7878"
+    restart: always
+    environment:
+      - PUID=${PUID}
+      - PGID=${PGID}
+      - TZ=${TZ}
+    networks:
+      - bridge
--- a/docker_compose/sonarr.yaml
+++ b/docker_compose/sonarr.yaml
@ -0,0 +1,19 @@
+version: "3"
+services:
+    sonarr:
+      image: "linuxserver/sonarr"
+      container_name: "sonarr"
+      env_file:
+          ./.env
+      volumes:
+        - ${DOCKERDIR}/appdata/sonarr:/config
+        - ${DATADIR}/downloads:/downloads
+        - ${DATADIR}/tvshows:/tvshows
+        - "/etc/localtime:/etc/localtime:ro"
+      ports:
+          - "8989:8989"
+      restart: always
+      environment:
+        - PUID=${PUID}
+        - PGID=${PGID}
+        - TZ=${TZ}
--- a/docker_compose/traefikv3.yaml
+++ b/docker_compose/traefikv3.yaml
@ -0,0 +1,96 @@
+version: '3.8'
+
+networks:
+  t3_proxy:
+    name: t3_proxy
+    driver: bridge
+    ipam:
+      config:
+        - subnet: 192.168.90.0/24
+
+secrets:
+  basic_auth_credentials:
+    file: $DOCKERDIR/secrets/basic_auth_credentials
+  cf_dns_api_token:
+    file: $DOCKERDIR/secrets/cf_dns_api_token
+
+services:
+  traefik:
+    container_name: traefik
+    image: traefik:3.0
+    restart: unless-stopped
+    env_file:
+      - ./.env
+    networks:
+      t3_proxy:
+        ipv4_address: 192.168.90.254
+    command:
+      - --entrypoints.web.address=:80
+      - --entrypoints.websecure.address=:443
+      - --entrypoints.traefik.address=:8080
+      - --entrypoints.websecure.http.tls=true
+      # The following two options redirects http request at port 80 to https
+      - --entrypoints.web.http.redirections.entrypoint.to=websecure
+      - --entrypoints.web.http.redirections.entrypoint.scheme=https
+      - --entrypoints.web.http.redirections.entrypoint.permanent=true
+      - --api=true
+      - --api.dashboard=true
+      # - --api.insecure=true
+      - --entrypoints.websecure.forwardedHeaders.trustedIPs=$CLOUDFLARE_IPS,$LOCAL_IPS
+      - --log=true
+      - --log.filePath=/logs/traefik.log
+      - --log.level=DEBUG
+      - --accessLog=true
+      - --accessLog.filePath=/logs/access.log
+      - --accessLog.bufferingSize=100
+      - --accessLog.filters.statusCodes=204-299,400-499,500-599
+      - --providers.docker=true
+      - --providers.docker.network=t3_proxy
+      - --entrypoints.websecure.http.tls.options=tls-opts@file
+      - --entrypoints.websecure.http.tls.certresolver=dns-cloudflare
+      - --entrypoints.websecure.http.tls.domains[0].main=$DOMAINNAME
+      - --entrypoints.websecure.http.tls.domains[0].sans=*.$DOMAINNAME
+      - --providers.file.directory=/rules
+      - --providers.file.watch=true
+      - --certificatesResolvers.dns-cloudflare.acme.storage=/acme.json
+      - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.provider=cloudflare
+      - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.resolvers=1.1.1.1:53,1.0.0.1:53
+    ports:
+      # - 80:80
+      - 443:443
+      - 8080:8080
+      # - target: 80
+      #   published: 80
+      #   protocol: tcp
+      #   mode: host
+      # - target: 443
+      #   published: 443
+      #   protocol: tcp
+      #   mode: host
+      # - target: 8080
+      #   published: 8585
+      #   protocol: tcp
+      #   mode: host
+    volumes:
+      - $DOCKERDIR/appdata/traefik3/rules/$HOSTNAME:/rules
+      - /var/run/docker.sock:/var/run/docker.sock:ro
+      - $DOCKERDIR/appdata/traefik3/acme/acme.json:/acme.json
+      - $DOCKERDIR/logs/$HOSTNAME/traefik:/logs
+    environment:
+      - PUID=${PUID}
+      - PGID=${PGID}
+      - TZ=$TZ
+      - CF_DNS_API_TOKEN_FILE=/run/secrets/cf_dns_api_token
+      - HTPASSWD_FILE=/run/secrets/basic_auth_credentials
+      - DOMAINNAME=${DOMAINNAME}
+    secrets:
+      - cf_dns_api_token
+      - basic_auth_credentials
+    labels:
+      - "traefik.enable=true"
+      - "traefik.http.routers.dashboard.tls=true" 
+      - "traefik.http.routers.traefik-rtr.entrypoints=websecure"
+      - "traefik.http.routers.traefik-rtr.rule=Host(`traefik.${DOMAINNAME}`)"
+      - "traefik.http.routers.traefik-rtr.service=api@internal"
+        # Middlewares
+      - "traefik.http.routers.traefik-rtr.middlewares=middlewares-rate-limit@file,middlewares-secure-headers@file,middlewares-basic-auth@file"