updated readme and restructure project

This commit is contained in:
2025-02-28 20:04:52 +02:00
parent 8e8d1a65e2
commit efdaba6169
60 changed files with 109 additions and 120 deletions

225
README.md
View File

@ -1,173 +1,158 @@
# Homeserver Setup # 🏠 Homeserver Setup Guide: Kubernetes on Proxmox
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
``` ```
© 2023 Taqi Tahmid © 2023 Taqi Tahmid
``` ```
This is the top level directory for the homeserver setup. It contains the > Build your own modern homelab with Kubernetes on Proxmox! This guide walks
following directories: > you through setting up a complete home server infrastructure with essential
> self-hosted services.
1. `ansible`: Contains the ansible playbooks and roles for setting up the ## 🌟 Highlights
homeserver. Proxmox is used as the hypervisor, so the playbooks are written with
that in mind. The Kubernetes cluster is set up using K3s. This has not been
automated yet and is a manual process.
2. `docker`: Contains the docker-compose files for setting up the different - Fully automated setup using Ansible
services. Right now Kubernetes is preferred over docker-compose, so this - Production-grade Kubernetes (K3s) cluster
directory is not actively maintained. - High-availability Proxmox configuration
- Popular self-hosted applications ready to deploy
3. `kubernetes`: Contains the kubernetes manifests and helm charts for setting ## 📁 Repository Structure
up the different services.
- `ansible/` - Automated provisioning with Ansible playbooks. [Ansible Guide](ansible/README.md)
- `kubernetes/` - K8s manifests and Helm charts. [Kubernetes Guide](kubernetes/README.md)
- `docker/` - Legacy docker-compose files (Kubernetes preferred). [Docker Guide](docker/README.md)
# Services ## 🚀 Running Services
The following services are set up on the homeserver: - ✨ AdGuard Home - Network-wide ad blocking
1. AdGuard Home - 🐳 Private Docker Registry
2. Private Docker Registry - 🎬 Jellyfin Media Server
3. Jellyfin - 🌐 Portfolio Website
4. My Portfolio Website - 🗄️ PostgreSQL Database
5. Postgres Database - 📦 Pocketbase Backend
6. Pocketbase Backend - 💻 Gitea Git Server
In future the following services are planned to be set up: ### 📋 Coming Soon
1. Nextcloud - Nextcloud
2. Gitea - Monitoring Stack
3. Monitoring Stack
## 💻 Hardware Setup
# Homeserver Setup - 2x Mini PCs with Intel N1000 CPUs
- 16GB RAM each
- 500GB SSDs
- 1Gbps networking
- Proxmox Cluster Configuration
I have two mini PCs with Intel N1000 CPUs and 16GB of RAM each and 500GB SSDs. ## 🛠️ Installation Steps
Both of them are running Proxmox and are connected to a 1Gbps network. The
Proxmox is set up in a cluster configuration.
There are four VMs dedicated for the Kubernetes cluster. The VMs are running ### 1. Setting up Proxmox Infrastructure
Ubuntu 22.04 and have 4GB of RAM and 2 CPUs each. The VMs are connected to a
bridge network so that they can communicate with each other. Two VMs are
confiugred as dual control plane and worker nodes and two VMs are configured as
worker nodes. The Kubernetes cluster is set up using K3s.
## Proxmox Installation #### Proxmox Base Installation
- Boot mini PCs from Proxmox USB drive
- Install on SSD and configure networking
- Set up cluster configuration
> 📚 Reference: [Official Proxmox Installation Guide](https://pve.proxmox.com/wiki/Installation)
The Proxmox installation is done on the mini PCs. The installation is done by #### 3. Cloud Image Implementation
booting from a USB drive with the Proxmox ISO. The installation is done on the
SSD and the network is configured during the installation. The Proxmox is
configured in a cluster configuration.
Ref: [proxmox-installation](https://pve.proxmox.com/wiki/Installation) Cloud images provide:
- 🚀 Pre-configured, optimized disk images
- 📦 Minimal software footprint
- ⚡ Quick VM deployment
- 🔧 Cloud-init support for easy customization
## Promox VM increase disk size These lightweight images are perfect for rapid virtual machine deployment in
your homelab environment.
Access the Proxmox Web Interface: #### Proxmox VM Disk Management
1. Log in to the Proxmox web interface. **Expanding VM Disk Size:**
Select the VM: 1. Access Proxmox web interface
2. In the left sidebar, click on the VM to resize. 2. Select target VM
Resize the Disk: 3. Navigate to Hardware tab
3. Go to the Hardware tab. 4. Choose disk to resize
4. Select the disk to resize (e.g., scsi0, ide0, etc.). 5. Click Resize and enter new size (e.g., 50G)
5. Click on the Resize button in the toolbar.
6. Enter 50G (or 50000 for 50GB) in the size field.
After that login to the VM and create a new partition or add to existing one **Post-resize VM Configuration:**
``` ```bash
# Access VM and configure partitions
sudo fdisk /dev/sda sudo fdisk /dev/sda
Press p to print the partition table. # Key commands:
Press d to delete the existing partition (note: ensure data is safe). # p - print partition table
Press n to create a new partition and follow the prompts. Make sure to use the # d - delete partition
same starting sector as the previous partition to avoid data loss. Press w to # n - create new partition
write changes and exit. # w - write changes
sudo mkfs.ext4 /dev/sdaX sudo mkfs.ext4 /dev/sdaX
``` ```
## Proxmox Passthrough Physical Disk to VM #### Physical Disk Passthrough
Pass physical disks (e.g., NVME storage) to VMs:
Ref: [proxmox-pve](https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM))
It is possible to pass through a physical disk attached to the hardware to
the VM. This implementation passes through a NVME storage to the
dockerhost VM.
```bash ```bash
# List the disk by-id with lsblk # List disk IDs
lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" \ lsblk |awk 'NR==1{print $0" DEVICE-ID(S)"}NR>1{dev=$1;printf $0" ";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");print "";}'|grep -v -E 'part|lvm'
";system("find /dev/disk/by-id -lname \"*"dev"\" -printf \" %p\"");\
print "";}'|grep -v -E 'part|lvm'
# Hot plug or add the physical device as a new SCSI disk # Add disk to VM (example for VM ID 103)
qm set 103 -scsi2 /dev/disk/by-id/usb-WD_BLACK_SN770_1TB_012938055C4B-0:0 qm set 103 -scsi2 /dev/disk/by-id/usb-WD_BLACK_SN770_1TB_012938055C4B-0:0
# Check with the following command # Verify configuration
grep 5C4B /etc/pve/qemu-server/103.conf grep 5C4B /etc/pve/qemu-server/103.conf
# After that reboot the VM and verify with lsblk command
lsblk
``` ```
> 📚 Reference: [Proxmox Disk Passthrough Guide](https://pve.proxmox.com/wiki/Passthrough_Physical_Disk_to_Virtual_Machine_(VM))
## Setup Master and worker Nodes for K3s ### 2. Kubernetes Cluster Setup
The cluster configuration consists of 4 VMs configured as 2 master and 2 worker k3s nodes. #### K3s Cluster Configuration
The master nodes also function as worker nodes. Setting up a 4-node cluster (2 master + 2 worker):
**Master Node 1:**
```bash ```bash
# On the first master run the following command
curl -sfL https://get.k3s.io | sh -s - server --cluster-init --disable servicelb curl -sfL https://get.k3s.io | sh -s - server --cluster-init --disable servicelb
# This will generate a token under /var/lib/rancher/k3s/server/node-token
# Which will be required to for adding nodes to the cluster
# On the second master run the following command
export TOKEN=<token>
export MASTER1_IP=<ip>
curl -sfL https://get.k3s.io | \
sh -s - server --server https://${MASTER1_IP}:6443 \
--token ${TOKEN} --disable servicelb
# Similarly on the worker nodes run the following command
export TOKEN=<token>
export MASTER1_IP=<ip>
curl -sfL https://get.k3s.io | \
K3S_URL=https://${MASTER1_IP}:6443 K3S_TOKEN=${TOKEN} sh -
``` ```
## Configure Metallb load balancer for k3s **Master Node 2:**
The metallb loadbalancer is used for services instead of k3s default
servicelb as it offers advanced features and supports IP address pools for load
balancer configuration.
```bash ```bash
# On any of the master nodes run the following command export TOKEN=<token>
kubectl apply -f \ export MASTER1_IP=<ip>
https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml curl -sfL https://get.k3s.io | sh -s - server --server https://${MASTER1_IP}:6443 --token ${TOKEN} --disable servicelb
```
# Ensure that metallb is installed with the following command **Worker Nodes:**
```bash
export TOKEN=<token>
export MASTER1_IP=<ip>
curl -sfL https://get.k3s.io | K3S_URL=https://${MASTER1_IP}:6443 K3S_TOKEN=${TOKEN} sh -
```
#### MetalLB Load Balancer Setup
```bash
# Install MetalLB
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml
# Verify installation
kubectl get pods -n metallb-system kubectl get pods -n metallb-system
# On the host machine apply the metallb cofigmap under metallb directory # Apply configuration
kubectl apply -f /home/taqi/homeserver/k3s-infra/metallb/metallbConfig.yaml kubectl apply -f /home/taqi/homeserver/k3s-infra/metallb/metallbConfig.yaml
```
# Test that the loadbalancer is working with nginx deployment **Quick Test:**
```bash
# Deploy test nginx
kubectl create namespace nginx kubectl create namespace nginx
kubectl create deployment nginx --image=nginx -n nginx kubectl create deployment nginx --image=nginx -n nginx
kubectl expose deployment nginx --port=80 --type=LoadBalancer -n nginx kubectl expose deployment nginx --port=80 --type=LoadBalancer -n nginx
# If nginx service gets an external IP and it is accessible from browser then # Cleanup after testing
the configuration is complete
kubectl delete namespace nginx kubectl delete namespace nginx
``` ```
## Cloud Image for VMs ## 🤝 Contributing
A cloud image is a pre-configured disk image that is optimized for use in cloud Contributions welcome! Feel free to open issues or submit PRs.
environments. These images typically include a minimal set of software and
configurations that are necessary to run in a virtualized environment,
such as a cloud or a hypervisor like Proxmox. Cloud images are designed to be
lightweight and can be quickly deployed to create new virtual machines. They
often come with cloud-init support, which allows for easy customization and
initialization of instances at boot time.
The cloud iamges are used for setting up the VMs in Proxmox. No traditional ## 📝 License
template is used for setting up the VMs. The cloud images are available for
download from the following link: MIT License - feel free to use this as a template for your own homelab!
[Ubuntu 22.04](https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img)

View File

@ -14,6 +14,10 @@ gitea:
password: password password: password
email: email email: email
image:
repository: gitea/gitea
tag: 1.23.4
postgresql: postgresql:
enabled: false enabled: false