Compare commits

..

43 Commits

Author SHA1 Message Date
8d943a3899 "Updates" 2026-02-02 02:40:54 +01:00
03e352b742 "Updates" 2026-02-02 01:29:43 +01:00
1c78e6c45f Updates 2026-02-01 21:02:41 +01:00
1a5c941725 "Updates" 2026-02-01 12:59:44 +01:00
bebb44a081 "Updates" 2026-01-31 02:09:04 +01:00
8f76639b96 "Updates" 2026-01-30 20:44:27 +01:00
9c72fa3738 "Updates" 2026-01-28 18:51:05 +01:00
493eb0ed90 Updates 2026-01-28 16:54:06 +01:00
213c2836f9 "Updates" 2026-01-28 16:49:36 +01:00
8d300e0995 "Updates" 2026-01-28 16:46:36 +01:00
677655c669 "Updates" 2026-01-28 08:53:58 +01:00
fd529a1807 "Updates" 2026-01-28 08:53:23 +01:00
78af6f512d "Updates" 2026-01-24 18:07:14 +01:00
a748807df6 "Updates" 2026-01-24 18:03:30 +01:00
bd07f1aceb "Updates" 2026-01-23 12:48:01 +01:00
412fa82ff3 DreamCanvas setup : to be reviewed 2026-01-21 19:20:00 +01:00
b580137ee8 Added AI server setup : to be reviewed 2026-01-21 19:10:32 +01:00
30d560f804 WG helper script 2026-01-20 09:06:51 +01:00
e19aff248d Added gitea-mirror 2026-01-19 08:34:20 +01:00
b956b07d1e Added Dockscribe 2026-01-19 08:31:58 +01:00
36706eb968 "Updates" 2026-01-14 12:09:05 +01:00
f3f02cbc34 "Updates" 2026-01-09 13:48:36 +01:00
c052ffa8c1 "Updates" 2026-01-09 11:36:54 +01:00
8bdc09988e "Updates" 2026-01-09 11:35:00 +01:00
879c7c517f Updates 2026-01-08 15:20:50 +01:00
9eed584abd "Updates" 2026-01-08 15:19:27 +01:00
93005d4254 "Updates" 2026-01-07 18:59:09 +01:00
996b0f6f27 "Updates" 2026-01-07 18:55:58 +01:00
4d3ed75b8e "Updates" 2026-01-07 18:55:23 +01:00
41007376b7 "Updates" 2026-01-06 23:44:10 +01:00
ad26b5b831 "Updates" 2026-01-06 23:42:57 +01:00
5684d7468c "Updates" 2026-01-06 23:30:56 +01:00
5ede101c2d "Updates" 2026-01-06 23:28:53 +01:00
ae640011ed "Updates" 2026-01-06 19:47:48 +01:00
ebfde7d1bd "Updates" 2026-01-06 11:03:46 +01:00
308b0a332e "Updates" 2026-01-06 10:48:27 +01:00
e237591ab1 Updated topology 2026-01-04 15:56:28 +01:00
7d93883b3e "Updates" 2026-01-04 15:53:16 +01:00
0360b18451 "Updates" 2026-01-03 01:24:24 +01:00
706e4c72f5 "Updates" 2025-12-30 10:56:57 +01:00
633a73efc4 "Updates" 2025-12-27 23:35:36 +01:00
dbfcd3c79a Updates 2025-12-26 11:47:37 +01:00
152dc3fa23 "Updates" 2025-12-26 11:12:43 +01:00
478 changed files with 89938 additions and 320 deletions

View File

@@ -0,0 +1,143 @@
# Installing Webmin and Docker on Ubuntu
This guide walks you through installing Webmin on Ubuntu and expanding logical volumes via Webmins interface. Additionally, it covers Docker installation on Ubuntu.
---
## Part 1: Installing Webmin on Ubuntu
Webmin is a web-based interface for managing Unix-like systems, making tasks such as user management, server configuration, and software installation easier.
### Step 1: Update Your System
Before installing Webmin, update your system to ensure all packages are up to date.
```bash
sudo apt update && sudo apt upgrade -y
```
### Step 2: Add the Webmin Repository and Key
To add the Webmin repository, download and run the setup script.
```bash
curl -o setup-repos.sh https://raw.githubusercontent.com/webmin/webmin/master/setup-repos.sh
sudo sh setup-repos.sh
```
### Step 3: Install Webmin
With the repository set up, install Webmin:
```bash
sudo apt-get install webmin --install-recommends
```
### Step 4: Access Webmin
Once installed, Webmin runs on port 10000. You can access it by opening a browser and navigating to:
```
https://<your-server-ip>:10000
```
If you are using a firewall, allow traffic on port 10000:
```bash
sudo ufw allow 10000
```
You can now log in to Webmin using your system's root credentials.
---
## Part 2: Expanding a Logical Volume Using Webmin
Expanding a logical volume through Webmins Logical Volume Management (LVM) interface is a simple process.
### Step 1: Access Logical Volume Management
Log in to Webmin and navigate to:
**Hardware > Logical Volume Management**
Here, you can manage physical volumes, volume groups, and logical volumes.
### Step 2: Add a New Physical Volume
If you've added a new disk or partition to your system, you need to allocate it to a volume group before expanding the logical volume. To do this:
1. Locate your volume group in the Logical Volume Management module.
2. Click **Add Physical Volume**.
3. Select the new partition or RAID device and click **Add to volume group**. This action increases the available space in the group.
### Step 3: Resize the Logical Volume
To extend a logical volume:
1. In the **Logical Volumes** section, locate the logical volume you wish to extend.
2. Select **Resize**.
3. Specify the additional space or use all available free space in the volume group.
4. Click **Apply** to resize the logical volume.
### Step 4: Resize the Filesystem
After resizing the logical volume, expand the filesystem to match:
1. Click on the logical volume to view its details.
2. For supported filesystems like ext2, ext3, or ext4, click **Resize Filesystem**. The filesystem will automatically adjust to the new size of the logical volume.
---
## Part 3: Installing Docker on Ubuntu
This section covers installing Docker on Ubuntu.
### Step 1: Remove Older Versions
If you have previous versions of Docker installed, remove them:
```bash
sudo apt remove docker docker-engine docker.io containerd runc
```
### Step 2: Add Docker's Official GPG Key and Repository
Add Dockers GPG key and repository to your systems Apt sources:
```bash
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
```
### Step 3: Install Docker
Now, install Docker:
```bash
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
### Step 4: Post-Installation Steps
To allow your user to run Docker commands without `sudo`, add your user to the Docker group:
```bash
sudo usermod -aG docker $USER
newgrp docker
```
Test your Docker installation by running the following command:
```bash
docker run hello-world
```
For more information, visit the official [Docker installation page](https://docs.docker.com/engine/install/ubuntu/).

View File

@@ -0,0 +1,182 @@
# How to Install the Latest Version of NVIDIA CUDA on Ubuntu 22.04 LTS
If youre looking to unlock the power of your NVIDIA GPU for scientific computing, machine learning, or other parallel workloads, CUDA is essential. Follow this step-by-step guide to install the latest CUDA release on Ubuntu 22.04 LTS.
## Prerequisites
Before proceeding with installing CUDA, ensure your system meets the following requirements:
- **Ubuntu 22.04 LTS** This version is highly recommended for stability and compatibility.
- **NVIDIA GPU + Drivers** CUDA requires having an NVIDIA GPU along with proprietary Nvidia drivers installed.
To check for an NVIDIA GPU, open a terminal and run:
```bash
lspci | grep -i NVIDIA
```
If an NVIDIA GPU is present, it will be listed. If not, consult NVIDIAs documentation on installing the latest display drivers.
## Step 1: Install Latest NVIDIA Drivers
Install the latest NVIDIA drivers matched to your GPU model and CUDA version using Ubuntus built-in Additional Drivers utility:
1. Open **Settings -> Software & Updates -> Additional Drivers**
2. Select the recommended driver under the NVIDIA heading
3. Click **Apply Changes** and **Reboot**
Verify the driver installation by running:
```bash
nvidia-smi
```
This should print details on your NVIDIA GPU and driver version.
## Step 2: Add the CUDA Repository
Add NVIDIAs official repository to your system to install CUDA:
1. Visit NVIDIAs CUDA Download Page and select "Linux", "x86_64", "Ubuntu", "22.04", "deb(network)"
2. Copy the repository installation commands for Ubuntu 22.04:
```bash
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
```
Run these commands to download repository metadata and add the apt source.
## Step 3: Install CUDA Toolkit
Install CUDA using apt:
```bash
sudo apt-get -y install cuda
```
Press **Y** to proceed and allow the latest supported version of the CUDA toolkit to install.
## Step 4: Configure Environment Variables
Update environment variables to recognize the CUDA compiler, tools, and libraries:
Open `/etc/profile.d/cuda.sh` and add the following configuration:
```bash
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
```
Save changes and refresh environment variables:
```bash
source /etc/profile.d/cuda.sh
```
Alternatively, reboot to load the updated environment variables.
## Step 5: Verify Installation
Validate the installation:
1. Check the `nvcc` compiler version:
```bash
nvcc --version
```
This should display details on the CUDA compile driver, including the installed version.
2. Verify GPU details with NVIDIA SMI:
```bash
nvidia-smi
```
# Optional: Setting Up cuDNN with CUDA: A Comprehensive Guide
This guide will walk you through downloading cuDNN from NVIDIA's official site, extracting it, copying the necessary files to the CUDA directory, and setting up environment variables for CUDA.
## Step 1: Download cuDNN
1. **Visit the NVIDIA cuDNN Archive**:
Navigate to the [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
2. **Select the Version**:
Choose the appropriate version of cuDNN compatible with your CUDA version. For this guide, we'll assume you are downloading `cudnn-linux-x86_64-8.9.7.29_cuda12-archive`.
3. **Download the Archive**:
Download the `tar.xz` file to your local machine.
## Step 2: Extract cuDNN
1. **Navigate to the Download Directory**:
Open a terminal and navigate to the directory where the archive was downloaded.
```bash
cd ~/Downloads
```
2. **Extract the Archive**:
Use the `tar` command to extract the contents of the archive.
```bash
tar -xvf cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar.xz
```
This will create a directory named `cudnn-linux-x86_64-8.9.7.29_cuda12-archive`.
## Step 3: Copy cuDNN Files to CUDA Directory
1. **Navigate to the Extracted Directory**:
Move into the directory containing the extracted cuDNN files.
```bash
cd cudnn-linux-x86_64-8.9.7.29_cuda12-archive
```
2. **Copy Header Files**:
Copy the header files to the CUDA include directory.
```bash
sudo cp include/cudnn*.h /usr/local/cuda-12.5/include/
```
3. **Copy Library Files**:
Copy the library files to the CUDA lib64 directory.
```bash
sudo cp lib/libcudnn* /usr/local/cuda-12.5/lib64/
```
4. **Set Correct Permissions**:
Ensure the copied files have the appropriate permissions.
```bash
sudo chmod a+r /usr/local/cuda-12.5/include/cudnn*.h /usr/local/cuda-12.5/lib64/libcudnn*
```
## Step 4: Set Up Environment Variables
1. **Open Your Shell Profile**:
Open your `.bashrc` or `.bash_profile` file in a text editor.
```bash
nano ~/.bashrc
```
2. **Add CUDA to PATH and LD_LIBRARY_PATH**:
Add the following lines to set the environment variables for CUDA. This example assumes CUDA 12.5.
```bash
export PATH=/usr/local/cuda-12.5/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.5/lib64:$LD_LIBRARY_PATH
```
3. **Apply the Changes**:
Source the file to apply the changes immediately.
```bash
source ~/.bashrc
```
## Verification
1. **Check CUDA Installation**:
Verify that CUDA is correctly set up by running:
```bash
nvcc --version
```
2. **Check cuDNN Installation**:
Optionally, you can compile and run a sample program to ensure cuDNN is working correctly.
By following these steps, you will have downloaded and installed cuDNN, integrated it into your CUDA setup, and configured your environment variables for smooth operation. This ensures that applications requiring both CUDA and cuDNN can run without issues.

View File

@@ -0,0 +1,85 @@
# OPTIONAL: Setting NVIDIA GPU Power Limit at System Startup
## Overview
This guide explains how to set the power limit for NVIDIA GPUs at system startup using a systemd service. This ensures the power limit setting is persistent across reboots.
## Steps
### 1. Create and Configure the Service File
1. Open a terminal and create a new systemd service file:
```bash
sudo nano /etc/systemd/system/nvidia-power-limit.service
```
2. Add the following content to the file, replacing `270` with the desired power limit (e.g., 270 watts for your GPUs):
- For Dual GPU Setup:
```ini
[Unit]
Description=Set NVIDIA GPU Power Limit
[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -i 0 -pl 270
ExecStart=/usr/bin/nvidia-smi -i 1 -pl 270
[Install]
WantedBy=multi-user.target
```
- For Quad GPU Setup:
```ini
[Unit]
Description=Set NVIDIA GPU Power Limit
[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -i 0 -pl 270
ExecStart=/usr/bin/nvidia-smi -i 1 -pl 270
ExecStart=/usr/bin/nvidia-smi -i 2 -pl 270
ExecStart=/usr/bin/nvidia-smi -i 3 -pl 270
[Install]
WantedBy=multi-user.target
```
Save and close the file.
### 2. Apply and Enable the Service
1. Reload the systemd manager configuration:
```bash
sudo systemctl daemon-reload
```
2. Enable the service to ensure it runs at startup:
```bash
sudo systemctl enable nvidia-power-limit.service
```
### 3. (Optional) Start the Service Immediately
To apply the power limit immediately without rebooting:
```bash
sudo systemctl start nvidia-power-limit.service
```
## Verification
Check the power limits using `nvidia-smi`:
```bash
nvidia-smi -q -d POWER
```
Look for the "Power Management" section to verify the new power limits.
By following this guide, you can ensure that your NVIDIA GPUs have a power limit set at every system startup, providing consistent and controlled power usage for your GPUs.

View File

@@ -0,0 +1,103 @@
# Ollama & OpenWebUI Docker Setup
## Ollama with Nvidia GPU
Ollama makes it easy to get up and running with large language models locally.
To run Ollama using an Nvidia GPU, follow these steps:
### Step 1: Install the NVIDIA Container Toolkit
#### Install with Apt
1. **Configure the repository**:
```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
```
2. **Install the NVIDIA Container Toolkit packages**:
```bash
sudo apt-get install -y nvidia-container-toolkit
```
#### Install with Yum or Dnf
1. **Configure the repository**:
```bash
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo \
| sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
```
2. **Install the NVIDIA Container Toolkit packages**:
```bash
sudo yum install -y nvidia-container-toolkit
```
### Step 2: Configure Docker to Use Nvidia Driver
```bash
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```
### Step 3: Start the Container
```bash
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --restart always --name ollama ollama/ollama
```
## Running Multiple Instances with Specific GPUs
You can run multiple instances of the Ollama server and assign specific GPUs to each instance. In my server, I have 4 Nvidia 3090 GPUs, which I use as described below:
### Ollama Server for GPUs 0 and 1
```bash
docker run -d --gpus '"device=0,1"' -v ollama:/root/.ollama -p 11435:11434 --restart always --name ollama1 --network ollama-network ollama/ollama
```
### Ollama Server for GPUs 2 and 3
```bash
docker run -d --gpus '"device=2,3"' -v ollama:/root/.ollama -p 11436:11434 --restart always --name ollama2 --network ollama-network ollama/ollama
```
## Running Models Locally
Once the container is up and running, you can execute models using:
```bash
docker exec -it ollama ollama run llama3.1
```
```bash
docker exec -it ollama ollama run llama3.1:70b
```
```bash
docker exec -it ollama ollama run qwen2.5-coder:1.5b
```
```bash
docker exec -it ollama ollama run deepseek-v2
```
### Try Different Models
Explore more models available in the [Ollama library](https://github.com/ollama/ollama).
## OpenWebUI Installation
To install and run OpenWebUI, use the following command:
```bash
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```

View File

@@ -0,0 +1,48 @@
### Wiki: Updating Docker Containers for Ollama and OpenWebUI
This guide explains the steps to update Docker containers for **Ollama** and **OpenWebUI**. Follow the instructions below to stop, remove, pull new images, and run the updated containers.
---
## Ollama
### Steps to Update
1. **Stop Existing Containers**
2. **Remove Existing Containers**
3. **Pull the Latest Ollama Image**
4. **Run Updated Containers**
For GPU devices 0 and 1:
```bash
docker stop ollama
docker rm ollama
docker pull ollama/ollama
docker run -d --gpus '"device=0,1"' -v ollama:/root/.ollama -p 11434:11434 --restart always --name ollama -e OLLAMA_KEEP_ALIVE=1h ollama/ollama
```
For NVIDIA jetson/cpu
```bash
docker stop ollama
docker rm ollama
docker pull ollama/ollama
docker run -d -v ollama:/root/.ollama -p 11434:11434 --restart always --name ollama -e OLLAMA_KEEP_ALIVE=1h ollama/ollama
```
---
## OpenWebUI
```bash
docker stop open-webui
docker rm open-webui
docker pull ghcr.io/open-webui/open-webui:main
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```
---
### Notes
- Make sure to adjust GPU allocation or port numbers as necessary for your setup.
- The `OLLAMA_KEEP_ALIVE` environment variable is set to `1h` to maintain the container alive for an hour after inactivity.

View File

@@ -0,0 +1,58 @@
# Running SearXNG with Custom Settings in Docker
## Overview
This guide walks you through the steps to run a SearXNG instance in Docker using a custom `settings.yml` configuration file. This setup is ideal for users who want to customize their SearXNG instance without needing to rebuild the Docker image every time they make a change.
## Prerequisites
- **Docker**: Ensure Docker is installed on your machine. Verify the installation by running `docker --version`.
- **Git**: For cloning the SearXNG repository, make sure Git is installed.
## Steps
### 1. Use the Official Image or Clone the SearXNG Repository
You can pull the official image directly from Docker Hub:
```bash
docker pull docker.io/searxng/searxng:latest
```
### 2. Customize `settings.yml`
Place your custom `settings.yml` file in the directory of your choice. Ensure that this file is configured according to your needs, including enabling JSON responses if required.
### 3. Run the SearXNG Docker Container
Run the Docker container using your custom `settings.yml` file. Choose the appropriate command based on whether you are using the official image or a custom build.
#### For the Official Image:
```bash
docker run -d -p 4000:8080 --restart always --name searxng -v ./settings.yml:/etc/searxng/settings.yml searxng/searxng:latest
```
#### Command Breakdown:
- `-d`: Runs the container in detached mode.
- `-p 4000:8080`: Maps port 8080 in the container to port 4000 on your host machine.
- `-v ./settings.yml:/etc/searxng/settings.yml`: Mounts the custom `settings.yml` file into the container.
- `searxng/searxng:latest` or `searxng/searxng`: The Docker image being used.
### 4. Access SearXNG
Once the container is running, you can access your SearXNG instance by navigating to `http://<hostname>:4000` in your web browser.
### 5. Testing JSON Output
To verify that the JSON output is correctly configured, you can use `curl` or a similar tool:
```bash
curl http://<hostname>:4000/search?q=python&format=json
```
This should return search results in JSON format.
### 5. Configuration URL for OpenWebUI
http://<hostname>:4000/search?q=<query>

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,123 @@
# ComfyUI Docker Setup with GGUF Support and ComfyUI Manager
This guide provides detailed steps to build and run **ComfyUI** with **GGUF support** and **ComfyUI Manager** using Docker. The GGUF format is optimized for quantized models, and ComfyUI Manager is included for easy node management.
## Prerequisites
Before starting, ensure you have the following installed on your system:
- **Docker**
- **NVIDIA GPU with CUDA support** (if using GPU acceleration)
- **Create Directory structure for git repo Models and Checkpoints**
```bash
mkdir -p ~/dev-ai/vison/models/checkpoints
```
### 1. Clone the ComfyUI Repository
First, navigate to `~/dev-ai/vison` directory and clone the ComfyUI repository to your local machine:
```bash
cd ~/dev-ai/vison
```
```bash
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
```
### 2. Create the Dockerfile
Copy the provided `Dockerfile` in the root of your ComfyUI directory. This file contains the necessary configurations for building the Docker container with GGUF support.
### 3. Build the Docker Image
```bash
docker build -t comfyui-gguf:latest .
```
This will create a Docker image named `comfyui-gguf:latest` with both **ComfyUI Manager** and **GGUF support** built in.
### 4. Run the Docker Container
Once the image is built, you can run the Docker container with volume mapping for your models.
```bash
docker run --name comfyui -p 8188:8188 --gpus all \
-v /home/mukul/dev-ai/vison/models:/app/models \
-d comfyui-gguf:latest
```
This command maps your local `models` directory to `/app/models` inside the container and exposes ComfyUI on port `8188`.
### 5. Download and Place Checkpoint Models
Download and place your civitai checkpoint models in the `checkpoints` directory inside the container:
https://civitai.com/models/139562/realvisxl-v50
To use GGUF models or other safetensor models, follow the steps below to download them directly into the `checkpoints` directory.
1. **Navigate to the Checkpoints Directory**:
```bash
cd /home/mukul/dev-ai/vison/models/checkpoints
```
2. **Download `flux1-schnell-fp8.safetensors`**:
```bash
wget https://huggingface.co/Comfy-Org/flux1-schnell/resolve/main/flux1-schnell-fp8.safetensors?download=true -O flux1-schnell-fp8.safetensors
```
3. **Download `flux1-dev-fp8.safetensors`**:
```bash
wget https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev-fp8.safetensors?download=true -O flux1-dev-fp8.safetensors
```
These commands will place the corresponding `.safetensors` files into the `checkpoints` directory.
### 6. Access ComfyUI
After starting the container, access the ComfyUI interface in your web browser:
```bash
http://<your-server-ip>:8188
```
Replace `<your-server-ip>` with your server's IP address or use `localhost` if you're running it locally.
### 7. Using GGUF Models
In the ComfyUI interface:
- Use the **UnetLoaderGGUF** node (found in the `bootleg` category) to load GGUF models.
- Ensure your GGUF files are correctly named and placed in the `/app/models/checkpoints` directory for detection by the loader node.
### 8. Managing Nodes with ComfyUI Manager
With **ComfyUI Manager** built into the image:
- **Install** missing nodes as needed when uploading workflows.
- **Enable/Disable** conflicting nodes from the ComfyUI Manager interface.
### 9. Stopping and Restarting the Docker Container
To stop the running container:
```bash
docker stop comfyui
```
To restart the container:
```bash
docker start comfyui
```
### 10. Logs and Troubleshooting
To view the container logs:
```bash
docker logs comfyui
```
This will provide details if anything goes wrong or if you encounter issues with GGUF models or node management.

View File

@@ -0,0 +1,33 @@
# Base image with Python 3.11 and CUDA 12.5 support
FROM nvidia/cuda:12.5.0-runtime-ubuntu22.04
# Install system dependencies
RUN apt-get update && apt-get install -y \
git \
python3-pip \
libgl1-mesa-glx \
&& rm -rf /var/lib/apt/lists/*
# Set working directory
WORKDIR /app
# Copy the cloned ComfyUI repository
COPY . /app
# Install Python dependencies
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
# Clone and install ComfyUI Manager
RUN git clone https://github.com/ltdrdata/ComfyUI-Manager.git /app/custom_nodes/ComfyUI-Manager && \
pip install -r /app/custom_nodes/ComfyUI-Manager/requirements.txt
# Clone and install GGUF support for ComfyUI
RUN git clone https://github.com/city96/ComfyUI-GGUF.git /app/custom_nodes/ComfyUI-GGUF && \
pip install --upgrade gguf
# Expose the port used by ComfyUI
EXPOSE 8188
# Run ComfyUI with the server binding to 0.0.0.0
CMD ["python3", "main.py", "--listen", "0.0.0.0"]

View File

@@ -0,0 +1,113 @@
## Running uncensored models on the NVIDIA Jetson Orin Nano Super Developer Kit
This guide is aimed at helping you set up uncensored models seamlessly on your Jetson Orin Nano, ensuring you can run powerful image generation models on this compact, yet powerful device.
This tutorial will walk you through each step of the process. Even if you're starting from a fresh installation, following along should ensure everything is set up correctly. And if anything doesnt work as expected, feel free to reach out—I'll keep this guide updated to keep it running smoothly.
---
## Lets Dive In
### Step 1: Installing Miniconda and Setting Up a Python Environment
First, we need to install Miniconda on your Jetson Nano. This will allow us to create an isolated Python environment for managing dependencies. Let's set up our project environment.
```bash
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh
chmod +x Miniconda3-latest-Linux-aarch64.sh
./Miniconda3-latest-Linux-aarch64.sh
conda update conda
```
Now, we create and activate a Python 3.10 environment for our project.
```bash
conda create -n comfyui python=3.10
conda activate comfyui
```
### Step 2: Installing CUDA, cuDNN, TensorRT, and Verifying nvcc
```bash
Preconfigured on JetPack 6.1!
```
Next, confirm that CUDA is installed correctly by checking the `nvcc` version.
```bash
nvcc --version
```
### Step 3: Installing PyTorch, TorchVision, and TorchAudio
Now let's install the essential libraries for image generation: PyTorch, TorchVision, and Torchaudio from here [devpi - cu12.6](http://jetson.webredirect.org/jp6/cu126)
```bash
pip install https://pypi.jetson-ai-lab.dev/jp6/cu126/+f/5cf/9ed17e35cb752/torch-2.5.0-cp310-cp310-linux_aarch64.whl
pip install https://pypi.jetson-ai-lab.dev/jp6/cu126/+f/9d2/6fac77a4e832a/torchvision-0.19.1a0+6194369-cp310-cp310-linux_aarch64.whl
pip install https://pypi.jetson-ai-lab.dev/jp6/cu126/+f/812/4fbc4ba6df0a3/torchaudio-2.5.0-cp310-cp310-linux_aarch64.whl
```
### Step 4: Cloning the Project Repository
Now, we clone the necessary source code for the project from GitHub. This will include the files for running uncensored models from civtai.com.
```bash
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI
```
### Step 5: Installing Project Dependencies
Next, install the required dependencies for the project by running the `requirements.txt` file.
```bash
pip install -r requirements.txt
```
### Step 6: Resolving Issues with NumPy (if necessary)
If you encounter issues with NumPy, such as compatibility problems, you can fix it by downgrading to a version below 2.0.
```bash
pip install "numpy<2"
```
### Step 7: Running ComfyUI
Finally, we can run ComfyUI to check if everything is set up properly. Start the app with the following command:
```bash
python main.py --listen 0.0.0.0
```
---
## Great! Now that you've got ComfyUI up and running, its time to load your first uncensored model.
1. Navigate to [civitai.com](https://civitai.com) and select a model. For example, you can choose the following model:
[RealVisionBabes v1.0](https://civitai.com/models/543456?modelVersionId=604282)
2. Download the model file: [realvisionbabes_v10.safetensors](https://civitai.com/api/download/models/604282?type=Model&format=SafeTensor&size=pruned&fp=fp16)
3. Place it inside the `models/checkpoints` folder.
4. Download the VAE file: [ClearVAE_V2.3_fp16.pt](https://civitai.com/api/download/models/604282?type=VAE)
5. Place it inside the `models/vae` folder.
---
## You're all set to launch your first run!
Visit the provided URL by ComfyUI (`http://jetson:8188`) on your Jetson Nano.
Go to the [ControlNet reference demo](https://civitai.com/posts/3943573), download the workflow (also available in the repo as workflow-api.json) and, import it in comfyUI.
And hit the "Queue Prompt" button, and watch the magic unfold!
Happy generating! 🎉

View File

@@ -0,0 +1,91 @@
You are an expert prompt generator. Your task is to transform user requests into detailed, vivid, and imaginative prompts that can be used to generate visually captivating images with a diffusion model. You should:
- **Analyze** the user's request carefully and extract the key visual elements.
- **Generate a prompt** that describes the image in clear and evocative terms, ensuring its visually rich and imaginative.
- **Ensure details** are specific about the atmosphere, setting, colors, lighting, textures, and unique characteristics.
- **Use creative language** to enhance the visual quality, whether the style is realistic, surreal, or abstract.
- **Consider mood and style** (e.g., dark and moody, bright and lively, minimalist, detailed, etc.).
Here are some examples to guide you:
---
### **Few-Shot Examples:**
**Example 1:**
**User Request:**
"I want to see a futuristic cityscape at night with neon lights and flying cars."
**Generated Prompt:**
"A vibrant futuristic cityscape at night, with towering skyscrapers that stretch towards a starless sky. The buildings are covered in shimmering neon lights—bright blues, purples, and pinks—casting colorful reflections onto the glossy streets below. Flying cars zip through the air, leaving glowing trails in their wake, while holographic billboards advertise virtual products. The atmosphere is electric, bustling with energy and technology, as a soft mist rises from the ground, adding a touch of mystery to the scene."
---
**Example 2:**
**User Request:**
"Id love a serene mountain landscape with a calm lake and a small wooden cabin."
**Generated Prompt:**
"A tranquil mountain landscape at dawn, with majestic snow-capped peaks towering in the distance. A serene, glassy lake reflects the vibrant colors of the early morning sky—soft pinks, oranges, and purples. A small, rustic wooden cabin sits by the lakeshore, its smoke rising from the chimney, blending gently with the mist above the water. Pine trees surround the cabin, their dark green needles adding depth to the peaceful scene. The air is crisp, and the whole environment exudes a sense of quiet solitude."
---
**Example 3:**
**User Request:**
"I want a mystical creature in a dark enchanted forest."
**Generated Prompt:**
"A mystical creature standing tall in the heart of a dark, enchanted forest. The creature has the body of a lion, but its fur is deep indigo, shimmering with silver flecks like stars. Its eyes glow with an ethereal light, casting an otherworldly glow across the forest floor. The forest is dense with towering trees whose bark is twisted, covered in glowing moss. Fog weaves through the trees, and mysterious flowers glow faintly in the shadows. The atmosphere is magical, filled with the sense of an ancient, forgotten world full of wonder."
---
**Example 4:**
**User Request:**
"Can you create a vibrant sunset on a tropical beach with palm trees?"
**Generated Prompt:**
"A stunning tropical beach at sunset, where the sky is ablaze with fiery hues of red, orange, and pink, melting into the calm blue of the ocean. The golden sand is warm, and the gentle waves lap against the shore. Silhouetted palm trees frame the scene, their long leaves swaying in the soft breeze. The sun is just dipping below the horizon, casting a golden glow across the water. The atmosphere is peaceful yet vibrant, with the serene sounds of the ocean adding to the beauty of the moment."
---
**Example 5:**
**User Request:**
"Imagine an underwater scene with colorful coral reefs and exotic fish."
**Generated Prompt:**
"A vibrant underwater scene, where the sunlight filters down through crystal-clear water, illuminating the colorful coral reefs below. The corals are in shades of purple, pink, and yellow, teeming with life. Schools of exotic fish dart through the scene—brightly colored in hues of electric blue, orange, and green. The water is calm, with soft ripples distorting the light, while gentle seaweed sways with the current. The scene is peaceful and full of life, a kaleidoscope of color beneath the ocean's surface."
---
### **User Request:**
"Please create a scene with a magical waterfall in a forest."
**Generated Prompt:**
"A breathtaking magical waterfall cascading down from a high cliff, surrounded by an ancient forest. The water sparkles with iridescent hues, as if glowing with a soft, mystical light. Lush green foliage and towering trees frame the waterfall, with delicate vines hanging down like natures curtains. Mist rises from the base of the waterfall, creating a rainbow in the air. Sunlight filters through the canopy above, casting dappled light across the mossy rocks and the peaceful forest floor. The atmosphere is serene, almost dreamlike, filled with the sound of the waters soothing rush."
---
### **User Request:**
"I want to see an alien landscape on another planet with strange rock formations."
**Generated Prompt:**
"A surreal alien landscape on a distant planet, bathed in the pale light of two suns setting on the horizon. The ground is rocky, with bizarre rock formations that defy gravity, twisting and spiraling upward like ancient sculptures. The sky above is a vibrant shade of purple, dotted with swirling clouds and distant stars. The air is thick with an otherworldly mist, and strange, bioluminescent plants glow faintly in the twilight. The scene is alien and unearthly, with a sense of wonder and curiosity as the landscape stretches endlessly into the unknown."
---
### **User Request:**
"Could you create a winter scene with a frozen lake and a snowman?"
**Generated Prompt:**
"A peaceful winter scene with a frozen lake covered in a smooth sheet of ice, reflecting the soft pale blue of the overcast sky. Snow gently falls from the sky, coating the landscape in a thick layer of white. A cheerful snowman stands at the edge of the lake, its coal-black eyes and carrot nose adding a touch of whimsy to the quiet surroundings. Snow-covered pine trees line the shore, their branches weighed down by the snow. The air is crisp and fresh, and the entire scene feels calm, still, and full of the quiet beauty of winter."
---
### **End of Few-Shot Examples**

View File

@@ -0,0 +1,129 @@
{
"1": {
"inputs": {
"ckpt_name": "realvisionbabes_v10.safetensors"
},
"class_type": "CheckpointLoaderSimple",
"_meta": {
"title": "Load Checkpoint"
}
},
"2": {
"inputs": {
"stop_at_clip_layer": -1,
"clip": [
"1",
1
]
},
"class_type": "CLIPSetLastLayer",
"_meta": {
"title": "CLIP Set Last Layer"
}
},
"3": {
"inputs": {
"text": "amateur, instagram photo, beautiful face",
"clip": [
"2",
0
]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"4": {
"inputs": {
"text": "Watermark, Text, censored, deformed, bad anatomy, disfigured, poorly drawn face, mutated, ugly, cropped, worst quality, low quality, mutation, poorly drawn, abnormal eye proportion, bad\nart, ugly face, messed up face, high forehead, professional photo shoot, makeup, photoshop, doll, plastic_doll, silicone, anime, cartoon, fake, filter, airbrush, 3d max, infant, featureless, colourless, impassive, shaders, two heads, crop,",
"clip": [
"2",
0
]
},
"class_type": "CLIPTextEncode",
"_meta": {
"title": "CLIP Text Encode (Prompt)"
}
},
"5": {
"inputs": {
"seed": 411040191827786,
"steps": 30,
"cfg": 3,
"sampler_name": "dpmpp_2m_sde",
"scheduler": "normal",
"denoise": 1,
"model": [
"1",
0
],
"positive": [
"3",
0
],
"negative": [
"4",
0
],
"latent_image": [
"6",
0
]
},
"class_type": "KSampler",
"_meta": {
"title": "KSampler"
}
},
"6": {
"inputs": {
"width": 512,
"height": 768,
"batch_size": 1
},
"class_type": "EmptyLatentImage",
"_meta": {
"title": "Empty Latent Image"
}
},
"7": {
"inputs": {
"samples": [
"5",
0
],
"vae": [
"8",
0
]
},
"class_type": "VAEDecode",
"_meta": {
"title": "VAE Decode"
}
},
"8": {
"inputs": {
"vae_name": "ClearVAE_V2.3_fp16.pt"
},
"class_type": "VAELoader",
"_meta": {
"title": "Load VAE"
}
},
"9": {
"inputs": {
"filename_prefix": "ComfyUI",
"images": [
"7",
0
]
},
"class_type": "SaveImage",
"_meta": {
"title": "Save Image"
}
}
}

View File

@@ -0,0 +1,316 @@
# Running DeepSeek-R1-0528 (FP8 Hybrid) with KTransformers
This guide provides instructions to run the DeepSeek-R1-0528 model locally using a hybrid FP8 (GPU) and Q4_K_M GGUF (CPU) approach with KTransformers, managed via Docker. This setup is optimized for high-end hardware (e.g., NVIDIA RTX 4090, high-core count CPU, significant RAM).
**Model Version:** DeepSeek-R1-0528
**KTransformers Version (Working):** `approachingai/ktransformers:v0.2.4post1-AVX512`
## Table of Contents
1. [Prerequisites](#prerequisites)
2. [Model Preparation](#model-preparation)
* [Step 2a: Download FP8 Base Model (Host)](#step-2a-download-fp8-base-model-host)
* [Step 2b: Download Q4\_K\_M GGUF Model (Host)](#step-2b-download-q4_k_m-gguf-model-host)
* [Step 2c: Merge Models (Inside Docker)](#step-2c-merge-models-inside-docker)
* [Step 2d: Set Ownership & Permissions (Host)](#step-2d-set-ownership--permissions-host)
3. [Running the Model with KTransformers](#running-the-model-with-ktransformers)
* [Single GPU (e.g., 1x RTX 4090)](#single-gpu-eg-1x-rtx-4090)
* [Multi-GPU (e.g., 2x RTX 4090)](#multi-gpu-eg-2x-rtx-4090)
4. [Testing the Server](#testing-the-server)
5. [Key Server Parameters](#key-server-parameters)
6. [Notes on KTransformers v0.3.1](#notes-on-ktransformers-v031)
7. [Available Optimize Config YAMLs (for reference)](#available-optimize-config-yamls-for-reference)
8. [Troubleshooting Tips](#troubleshooting-tips)
---
## 1. Prerequisites
* **Hardware:**
* NVIDIA GPU with FP8 support (e.g., RTX 40-series, Hopper series).
* High core-count CPU (e.g., Intel Xeon, AMD Threadripper).
* Significant System RAM (ideally 512GB for larger GGUF experts and context). The Q4_K_M experts for a large model can consume 320GB+ alone.
* Fast SSD (NVMe recommended) for model storage.
* **Software (on Host):**
* Linux OS (Ubuntu 24.04 LTS recommended).
* NVIDIA Drivers (ensure they are up-to-date and support your GPU and CUDA version).
* Docker Engine.
* NVIDIA Container Toolkit (for GPU access within Docker).
* Conda or a Python virtual environment manager.
* Python 3.9+
* `huggingface_hub` and `hf_transfer`
* Git (for cloning KTransformers if you need to inspect YAMLs or contribute).
---
## 2. Model Preparation
We assume your models will be downloaded and stored under `/home/mukul/dev-ai/models` on your host system. This path will be mounted into the Docker container as `/models`. Adjust paths if your setup differs.
### Step 2a: Download FP8 Base Model (Host)
Download the official DeepSeek-R1-0528 FP8 base model components.
```bash
# Ensure that correct packages are installed. Conda is recommended for environemnt management.
pip install -U huggingface_hub hf_transfer
export HF_HUB_ENABLE_HF_TRANSFER=1 # For faster downloads
```
```bash
# Define your host model directory
HOST_MODEL_DIR="/home/mukul/dev-ai/models"
BASE_MODEL_HF_ID="deepseek-ai/DeepSeek-R1-0528"
LOCAL_BASE_MODEL_PATH="${HOST_MODEL_DIR}/${BASE_MODEL_HF_ID}"
mkdir -p "${LOCAL_BASE_MODEL_PATH}"
echo "Downloading base model to: ${LOCAL_BASE_MODEL_PATH}"
huggingface-cli download --resume-download "${BASE_MODEL_HF_ID}" \
--local-dir "${LOCAL_BASE_MODEL_PATH}"```
```
### Step 2b: Download Q4_K_M GGUF Model (Host)
Download the Unsloth Q4_K_M GGUF version of DeepSeek-R1-0528 using the attached python script.
### Step 2c: Merge Models (Inside Docker)
This step uses the KTransformers Docker image to merge the FP8 base and Q4\_K\_M GGUF weights.
```bash
docker stop ktransformers
docker run --rm --gpus '"device=1"' \
-v /home/mukul/dev-ai/models:/models \
--name ktransformers \
-itd approachingai/ktransformers:v0.2.4post1-AVX512
docker exec -it ktransformers /bin/bash
```
```bash
python merge_tensors/merge_safetensor_gguf.py \
--safetensor_path /models/deepseek-ai/DeepSeek-R1-0528 \
--gguf_path /models/unsloth/DeepSeek-R1-0528-GGUF/Q4_K_M \
--output_path /models/mukul/DeepSeek-R1-0528-GGML-FP8-Hybrid/Q4_K_M_FP8
```
### Step 2d: Set Ownership & Permissions (Host)
After Docker creates the merged files, fix ownership and permissions on the host.
```bash
HOST_OUTPUT_DIR_QUANT="/home/mukul/dev-ai/models/mukul/DeepSeek-R1-0528-GGML-FP8-Hybrid/Q4_K_M_FP8" # As defined above
echo "Setting ownership for merged files in: ${HOST_OUTPUT_DIR_QUANT}"
sudo chown -R $USER:$USER "${HOST_OUTPUT_DIR_QUANT}"
sudo find "${HOST_OUTPUT_DIR_QUANT}" -type f -exec chmod 664 {} \;
sudo find "${HOST_OUTPUT_DIR_QUANT}" -type d -exec chmod 775 {} \;
echo "Ownership and permissions set. Verification:"
ls -la "${HOST_OUTPUT_DIR_QUANT}"
```
---
## 3. Running the Model with KTransformers
Ensure the Docker image `approachingai/ktransformers:v0.2.4post1-AVX512` is pulled.
### Single GPU (e.g., 1x RTX 4090)
**1. Start Docker Container:**
```bash
# Stop any previous instance
docker stop ktransformers || true # Allow if not running
docker rm ktransformers || true # Allow if not existing
# Define your host model directory
HOST_MODEL_DIR="/home/mukul/dev-ai/models"
TARGET_GPU="1" # Specify GPU ID, e.g., "0", "1", or "all"
docker run --rm --gpus "\"device=${TARGET_GPU}\"" \
-v "${HOST_MODEL_DIR}:/models" \
-p 10002:10002 \
--name ktransformers \
-itd approachingai/ktransformers:v0.2.4post1-AVX512
docker exec -it ktransformers /bin/bash
```
**2. Inside the Docker container shell, launch the server:**
```bash
# Set environment variable for PyTorch CUDA memory allocation
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
CONTAINER_MERGED_MODEL_PATH="/models/mukul/DeepSeek-R1-0528-GGML-FP8-Hybrid/Q4_K_M_FP8"
CONTAINER_BASE_MODEL_CONFIG_PATH="/models/deepseek-ai/DeepSeek-R1-0528"
# Launch server
python3 ktransformers/server/main.py \
--gguf_path "${CONTAINER_MERGED_MODEL_PATH}" \
--model_path "${CONTAINER_BASE_MODEL_CONFIG_PATH}" \
--model_name KVCache-ai/DeepSeek-R1-0528-q4km-fp8 \
--cpu_infer 57 \
--max_new_tokens 16384 \
--cache_lens 24576 \
--cache_q4 true \
--temperature 0.6 \
--top_p 0.95 \
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-fp8-linear-ggml-experts.yaml \
--force_think \
--use_cuda_graph \
--host 0.0.0.0 \
--port 10002
```
*Note: The `--optimize_config_path` still refers to a `DeepSeek-V3` YAML. This V3 config is compatible and recommended.
### Multi-GPU (e.g., 2x RTX 4090)
**1. Start Docker Container:**
```bash
# Stop any previous instance
docker stop ktransformers || true
docker rm ktransformers || true
# Define your host model directory
HOST_MODEL_DIR="/home/mukul/dev-ai/models"
TARGET_GPUS="0,1" # Specify GPU IDs
docker run --rm --gpus "\"device=${TARGET_GPUS}\"" \
-v "${HOST_MODEL_DIR}:/models" \
-p 10002:10002 \
--name ktransformers \
-itd approachingai/ktransformers:v0.2.4post1-AVX512
docker exec -it ktransformers /bin/bash
```
**2. Inside the Docker container shell, launch the server:**
```bash
# Set environment variable (optional for multi-GPU, but can be helpful)
# export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
# Define container paths
CONTAINER_MERGED_MODEL_PATH="/models/mukul/DeepSeek-R1-0528-GGML-FP8-Hybrid/Q4_K_M_FP8"
CONTAINER_BASE_MODEL_CONFIG_PATH="/models/deepseek-ai/DeepSeek-R1-0528"
# Launch server
python3 ktransformers/server/main.py \
--gguf_path "${CONTAINER_MERGED_MODEL_PATH}" \
--model_path "${CONTAINER_BASE_MODEL_CONFIG_PATH}" \
--model_name KVCache-ai/DeepSeek-R1-0528-q4km-fp8 \
--cpu_infer 57 \
--max_new_tokens 24576 \
--cache_lens 32768 \
--cache_q4 true \
--temperature 0.6 \
--top_p 0.95 \
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu-fp8-linear-ggml-experts.yaml \
--force_think \
--use_cuda_graph \
--host 0.0.0.0 \
--port 10002
```
*Note: The `--optimize_config_path` still refers to a `DeepSeek-V3` YAML. This is intentional.*
---
## 4. Testing the Server
Once the server is running inside Docker (look for "Uvicorn running on http://0.0.0.0:10002"), open a **new terminal on your host machine** and test with `curl`:
```bash
curl http://localhost:10002/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "KVCache-ai/DeepSeek-R1-0528-q4km-fp8",
"messages": [{"role": "user", "content": "Explain the concept of Mixture of Experts in large language models in a simple way."}],
"max_tokens": 250,
"temperature": 0.6,
"top_p": 0.95
}'
```
A JSON response containing the model's output indicates success.
---
## 5. Key Server Parameters
* `--gguf_path`: Path inside the container to your **merged** hybrid model files.
* `--model_path`: Path inside the container to the **original base model's** directory (containing `config.json`, `tokenizer.json`, etc.). KTransformers needs this for model configuration.
* `--model_name`: Arbitrary name for the API endpoint. Used in client requests.
* `--cpu_infer`: Number of CPU threads for GGUF expert inference. Tune based on your CPU cores (e.g., `57` for a 56-core/112-thread CPU might leave some cores for other tasks, or you could try higher).
* `--max_new_tokens`: Maximum number of tokens the model can generate in a single response.
* `--cache_lens`: Maximum KV cache size in tokens. Directly impacts context length capacity and VRAM usage.
* `--cache_q4`: (Boolean) If `true`, quantizes the KV cache to 4-bit. **Crucial for saving VRAM**, especially with long contexts.
* `--temperature`, `--top_p`: Control generation randomness.
* `--optimize_config_path`: Path to the KTransformers YAML file defining the layer offloading strategy (FP8 on GPU, GGUF on CPU). **Essential for the hybrid setup.**
* `--force_think`: (KTransformers specific) Potentially related to how the model processes or plans.
* `--use_cuda_graph`: Enables CUDA graphs for potentially faster GPU execution by reducing kernel launch overhead.
* `--host`, `--port`: Network interface and port for the server.
* `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`: Environment variable to help PyTorch manage CUDA memory more flexibly and potentially avoid OOM errors.
---
## 6. Notes on KTransformers v0.3.1
As of 2025-06-02, the `approachingai/ktransformers:v0.3.1-AVX512` image was reported as **not working** with the provided single GPU or multi-GPU configuration.
**Attempted Docker Start Command (v0.3.1 - Non-Functional):**
```bash
# docker stop ktransformers # (if attempting to switch)
# docker run --rm --gpus '"device=0,1"' \
# -v /home/mukul/dev-ai/models:/models \
# -p 10002:10002 \
# --name ktransformers \
# -itd approachingai/ktransformers:v0.3.1-AVX512
#
# docker exec -it ktransformers /bin/bash
```
**Attempted Server Launch (v0.3.1 - Non-Functional):**
```bash
# # Inside the v0.3.1 Docker container shell
# PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python3 ktransformers/server/main.py \
# --gguf_path /models/mukul/DeepSeek-R1-0528-GGML-FP8-Hybrid/Q4_K_M_FP8 \
# --model_path /models/deepseek-ai/DeepSeek-R1-0528 \
# --model_name KVCache-ai/DeepSeek-R1-0528-q4km-fp8 \
# --cpu_infer 57 \
# --max_new_tokens 32768 \
# --cache_lens 65536 \
# --cache_q4 true \
# --temperature 0.6 \
# --top_p 0.95 \
# --optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-multi-gpu-fp8-linear-ggml-experts.yaml \
# --force_think \
# --use_cuda_graph \
# --host 0.0.0.0 \
# --port 10002
```
Stick to `approachingai/ktransformers:v0.2.4post1-AVX512` for the configurations described above until compatibility issues with newer versions are resolved for this specific model and setup.
---
## 7. Available Optimize Config YAMLs (for reference)
The KTransformers repository contains various optimization YAML files. The ones used in this guide are for `DeepSeek-V3` but are being applied to `DeepSeek-R1-0528`. Their direct compatibility or optimality for R1-0528 should be verified. If KTransformers releases specific YAMLs for DeepSeek-R1-0528, those should be preferred.
Reference list of some `DeepSeek-V3` YAMLs (path `ktransformers/optimize/optimize_rules/` inside the container):
```
DeepSeek-V3-Chat-amx.yaml
DeepSeek-V3-Chat-fp8-linear-ggml-experts-serve-amx.yaml
DeepSeek-V3-Chat-fp8-linear-ggml-experts-serve.yaml
DeepSeek-V3-Chat-fp8-linear-ggml-experts.yaml
DeepSeek-V3-Chat-multi-gpu-4.yaml
DeepSeek-V3-Chat-multi-gpu-8.yaml
DeepSeek-V3-Chat-multi-gpu-fp8-linear-ggml-experts.yaml
DeepSeek-V3-Chat-multi-gpu-marlin.yaml
DeepSeek-V3-Chat-multi-gpu.yaml
DeepSeek-V3-Chat-serve.yaml
DeepSeek-V3-Chat.yaml
```

View File

@@ -0,0 +1,65 @@
from huggingface_hub import hf_hub_download, list_repo_files # Import list_repo_files
import os
# Configuration
repo_id = "unsloth/DeepSeek-R1-0528-GGUF"
folder_in_repo = "Q4_K_M"
file_extension = ".gguf"
# Expand the tilde (~) to the user's home directory
local_base_dir = os.path.expanduser("~/dev-ai/models/unsloth/DeepSeek-R1-0528-GGUF")
# Create base directory
# The hf_hub_download function will create the directory if it doesn't exist
# when local_dir_use_symlinks=False. However, explicit creation is fine.
os.makedirs(local_base_dir, exist_ok=True)
# Download files
print(f"Listing files from {repo_id} in folder {folder_in_repo} with extension {file_extension}...")
try:
all_repo_files = list_repo_files(repo_id, repo_type='model')
files_to_download = [
f for f in all_repo_files
if f.startswith(folder_in_repo + "/") and f.endswith(file_extension)
]
if not files_to_download:
print(f"No files found in '{folder_in_repo}' with extension '{file_extension}'.")
else:
print(f"Found {len(files_to_download)} file(s) to download.")
for filename_in_repo in files_to_download:
print(f"Downloading {filename_in_repo}...")
# The filename parameter in hf_hub_download should be the path within the repo
# The local_dir parameter specifies where the file (maintaining its repo path structure)
# will be saved under.
# For example, if filename_in_repo is "UD-Q4_K_XL/file.gguf",
# it will be saved as local_base_dir/UD-Q4_K_XL/file.gguf
try:
downloaded_file_path = hf_hub_download(
repo_id=repo_id,
filename=filename_in_repo, # This is the path of the file within the repository
local_dir=local_base_dir,
local_dir_use_symlinks=False,
# Set resume_download=True if you want to resume interrupted downloads
# resume_download=True,
)
# The hf_hub_download function returns the full path to the downloaded file.
# The way files are saved when local_dir is used can be tricky.
# If filename_in_repo is "folder/file.txt", it will be saved as "local_dir/folder/file.txt".
# If you want all files directly in local_base_dir without the repo's folder structure,
# you would need to adjust the local_dir or rename/move the file post-download.
# However, for GGUF files from a specific folder, saving them under that folder structure locally is usually fine.
print(f"Successfully downloaded and saved to: {downloaded_file_path}")
# If you want to confirm the exact path as per your original print statement's intent:
# expected_local_path = os.path.join(local_base_dir, filename_in_repo)
# print(f"Saved to: {expected_local_path}")
except Exception as e:
print(f"Error downloading {filename_in_repo}: {str(e)}")
except Exception as e:
print(f"Error listing files from repository: {str(e)}")
print("Download process complete.")

View File

@@ -0,0 +1,137 @@
# **Port Forwarding Magic: Set Up Bolt.New with Remote Ollama Server and Qwen2.5-Coder:32B**
This guide demonstrates how to use **port forwarding** to connect your local **Bolt.New** setup to a **remote Ollama server**, solving issues with apps that dont allow full customization. Well use the open-source [Bolt.New repository](https://github.com/coleam00/bolt.new-any-llm) as our example, and well even show you how to extend the context length for the popular **Qwen2.5-Coder:32B model**.
If you encounter installation issues, submit an [issue](https://github.com/coleam00/bolt.new-any-llm/issues) or contribute by forking and improving this guide.
---
## **What You'll Learn**
- Clone and configure **Bolt.New** for your local development.
- Use **SSH tunneling** to seamlessly forward traffic to a remote server.
- Extend the context length of AI models for enhanced capabilities.
- Run **Bolt.New** locally.
---
## **Prerequisites**
Download and install Node.js from [https://nodejs.org/en/download/](https://nodejs.org/en/download/).
---
## **Step 1: Clone the Repository**
1. Open Terminal.
2. Clone the repository:
```bash
git clone https://github.com/coleam00/bolt.new-any-llm.git
```
---
## **Step 2: Stop Local Ollama Service**
If Ollama is already running on your machine, stop it to avoid conflicts with the remote server.
- **Stop the service**:
```bash
sudo systemctl stop ollama.service
```
- **OPTIONAL: Disable it from restarting**:
```bash
sudo systemctl disable ollama.service
```
---
## **Step 3: Forward Local Traffic to the Remote Ollama Server**
To forward all traffic from `localhost:11434` to your remote Ollama server (`ai.mtcl.lan:11434`), set up SSH tunneling:
1. Open a terminal and run:
```bash
ssh -L 11434:ai.mtcl.lan:11434 mukul@ai.mtcl.lan
```
- Replace `mukul` with your remote username.
- Replace `ai.mtcl.lan` with your server's hostname or IP.
2. Keep this terminal session running while using Bolt.New. This ensures your app communicates with the remote server as if its local.
---
## **Step 4: OPTIONAL: Extend Ollama Model Context Length**
By default, Ollama models have a context length of 2048 tokens. For tasks requiring larger input, extend this limit for **Qwen2.5-Coder:32B**:
1. SSH into your remote server:
```bash
ssh mukul@ai.mtcl.lan
```
2. Access the Docker container running Ollama:
```bash
docker exec -it ollama /bin/bash
```
3. Create a `Modelfile`:
While inside the Docker container, run the following commands to create the Modelfile:
```bash
echo "FROM qwen2.5-coder:32b" > /tmp/Modelfile
echo "PARAMETER num_ctx 32768" >> /tmp/Modelfile
```
If you prefer, you can use cat to directly create the file:
```bash
cat > /tmp/Modelfile << EOF
FROM qwen2.5-coder:32b
PARAMETER num_ctx 32768
EOF
```
4. Create the new model:
```bash
ollama create -f /tmp/Modelfile qwen2.5-coder-extra-ctx:32b
```
5. Verify the new model:
```bash
ollama list
```
You should see `qwen2.5-coder-extra-ctx:32b` listed.
6. Exit the Docker container:
```bash
exit
```
---
## **Step 5: Run Bolt.New Without Docker**
1. **Install Dependencies**
Navigate to the cloned repository:
```bash
cd bolt.new-any-llm
pnpm install
```
2. **Start the Development Server**
Run:
```bash
pnpm run dev
```
---
## **Summary**
This guide walks you through setting up **Bolt.New** with a **remote Ollama server**, ensuring seamless communication through SSH tunneling. Weve also shown you how to extend the context length for **Qwen2.5-Coder:32B**, making it ideal for advanced development tasks.
With this setup:
- Youll offload heavy computation to your remote server.
- Your local machine remains light and responsive.
- Buggy `localhost` configurations? No problem—SSH tunneling has you covered.
Credits: [Bolt.New repository](https://github.com/coleam00/bolt.new-any-llm).
Lets build something amazing! 🚀

View File

@@ -0,0 +1,107 @@
### **Guide to Set Up Bridge Networking on Ubuntu for Virtual Machines**
This guide explains how to configure bridge networking on Ubuntu to allow virtual machines (VMs) to directly access the network, obtaining their own IP addresses from the DHCP server.
By following this guide, you can successfully set up bridge networking, enabling your virtual machines to directly access the network as if they were standalone devices.
---
#### **Step 1: Identify Your Primary Network Interface**
The primary network interface is the one currently used by the server for network access. Identify it with the following command:
```bash
ip link show
```
Look for the name of the interface (e.g., `enp8s0`) with `state UP`.
---
#### **Step 2: Backup Your Current Network Configuration**
Before making any changes, back up the existing netplan configuration file:
```bash
sudo cp /etc/netplan/00-installer-config.yaml /etc/netplan/00-installer-config.yaml.bak
```
---
#### **Step 3: Configure the Bridge**
Edit the netplan configuration file:
```bash
sudo nano /etc/netplan/00-installer-config.yaml
```
Replace its content with the following, adjusted for your environment:
```yaml
network:
version: 2
ethernets:
enp8s0:
dhcp4: no
bridges:
br0:
interfaces: [enp8s0]
dhcp4: true
```
- `enp8s0`: Your physical network interface.
- `br0`: The new bridge interface that will be used by the virtual machines and the host.
Save and exit the file.
---
#### **Step 4: Apply the Configuration**
Apply the new network configuration to create the bridge:
```bash
sudo netplan apply
```
---
#### **Step 5: Verify the Bridge Configuration**
Check that the bridge `br0` is active and has an IP address:
```bash
ip addr show br0
```
You should see an output like this:
```plaintext
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 46:10:cc:63:f4:37 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.10/24 metric 100 brd 192.168.1.255 scope global dynamic br0
valid_lft 7102sec preferred_lft 7102sec
```
---
#### **Step 6: Configure Virtual Machines to Use the Bridge**
For VMs created with tools like `virt-manager` or `virsh`:
1. When configuring the VMs network interface, choose **Bridge** as the network source.
2. Set `br0` as the bridge interface.
3. The VM will now obtain an IP address dynamically from the same DHCP server as the host.
For `virt-manager`:
- Go to **Add Hardware > Network**.
- Choose **Bridge br0** as the source.
---
#### **Step 7: Test the Setup**
1. Start a VM and ensure it obtains a dynamic IP address from the network.
2. Test connectivity by pinging the gateway or external servers from the VM.
---
### **Key Considerations**
1. **Dynamic IP for Host:** The host server's IP address will now be associated with the bridge (`br0`) instead of the physical interface (`enp8s0`). This is expected behavior.
2. **Backup Configuration:** Always maintain a backup of your original network configuration to revert changes if needed.
3. **Network Manager vs. Netplan:** Use only one method (`netplan` or `nmcli`) for managing network configurations to avoid conflicts.
4. **Alternative Access:** If you are working on a remote server, ensure alternative access (e.g., a second network interface) before applying network changes.

View File

@@ -28,7 +28,7 @@ all:
bernie:
ansible_connection: ssh
ansible_user: pleb
pearl:
patrick:
ansible_connection: ssh
ansible_user: pleb
```
@@ -46,7 +46,7 @@ sandy ansible_connection=ssh ansible_user=pleb
krabs ansible_connection=ssh ansible_user=pleb
sheldon ansible_connection=ssh ansible_user=pleb
bernie ansible_connection=ssh ansible_user=pleb
pearl ansible_connection=ssh ansible_user=pleb
patrick ansible_connection=ssh ansible_user=pleb
```
```bash
@@ -604,7 +604,7 @@ vim ~/ansible/ping.yaml
```
```yaml
- name: Pinging Turing Pi (shells)
- name: Pinging Turing Pi
hosts: all
tasks:
- name: Ping my hosts

6
Divers/Bbox (admin).md Normal file
View File

@@ -0,0 +1,6 @@
Bbox (admin)
========================
^6r*qRe@E31nqk
192\.168.1.254
androidapp://fr.bouyguestelecom.ecm.android

File diff suppressed because one or more lines are too long

14
Divers/DNS Servers.md Normal file
View File

@@ -0,0 +1,14 @@
DNS Servers
========================
| # | DNS | Adresses IPv4 | Adresses IPv6 |
| --- | --- | --- | --- |
| 1 | [**Cloudflare 1.1.1.1**] | 1.1.1.1, 1.0.0.1 | 2606:4700:4700::1111,2606:4700:4700::1001 |
| 2 | [Cisco OpenDNS Home](https://www.opendns.com/) | 208.67.222.222, 208.67.220.220 | 2620:119:35::35, 2620:119:53::53 |
| 3 | [Neustar UltraDNS Public](https://www.publicdns.neustar/) | 64.6.64.6, 64.6.65.6 | 2620:74:1b::1:1, 2620:74:1c::2:2 |
| 4 | [NextDNS](https://nextdns.io/fr) | 45.90.28.0, 45.90.30.0 | 2a07:a8c0::, 2a07:a8c1::|
| 5 | [Google Public DNS](https://developers.google.com/speed/public-dns) | 8.8.8.8, 8.8.4.4 | 2001:4860:4860::8888, 2001:4860:4860::8844 |
| 6 | [**Quad9**](https://www.quad9.net/) | 9.9.9.9, 149.112.112.112 | 2620:fe::fe, 2620:fe::9 |
| 7 | [Comodo Secure DNS](https://www.comodo.com/secure-dns/) | 8.26.56.26, 8.20.247.20 | |
| 8 | [Yandex.DNS](https://dns.yandex.com/ ) | 77.88.8.8, 77.88.8.1 | 2a02:6b8::feed:0ff, 2a02:6b8:0:1::feed:0ff |
| 9 | [SafeDNS](https://www.safedns.com/fr/) | 195.46.39.39, 195.46.39.40 | 2001:67c:2778::3939, 2001:67c:2778::3940 |

674
Divers/Dockscribe/LICENSE Normal file
View File

@@ -0,0 +1,674 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.

View File

@@ -0,0 +1,65 @@
# Dockscribe
[![download](https://img.shields.io/badge/Download-gray?logo=github)](#Download)
[![release](https://img.shields.io/github/v/release/patricksthannon/Dockscribe)](https://github.com/patricksthannon/Dockscribe/releases/latest)
[![buy-me-a-coffee](https://img.shields.io/badge/Buy_Me_a_Coffee-ffdd00?logo=buy-me-a-coffee&logoColor=black)](https://buymeacoffee.com/patricksthannon)
**CLI tool to pull short descriptions of all currently running docker containers**
_As I was using the `brew leaves | xargs brew desc --eval-all`[^1] command for homebrew, which lists short descriptions of all current brew packages, I decided I would like a tool similar to this for Docker. Therefor, **Dockscribe** was born._<br/>
This script will scan for all of your currently running docker containers, source descriptions from dockerhub and github, then output the results. In some cases, if the docker container is a dependency for a parent container (ie as "immich-machine-learning" for immich) it will fallback to the parent base description.
## Dependencies:
Dockscribe will automatically download `jq` by either using apt get, homebrew (mac os), or downloading from the static binary.
_jq is built around the concept of filters that work over a stream of JSON._
<a id="download"></a>
## Install Instructions
1. Install using scripts below for either Linux or MacOs.
```
# Linux with curl | into default PATH /.local/bin/
curl -L https://raw.githubusercontent.com/patricksthannon/Dockscribe/refs/heads/main/dockscribe.sh -o ~/.local/bin/dockscribe.sh
chmod +x ~/.local/bin/dockscribe.sh
# Linux with wget | into default PATH /.local/bin/
wget -O ~/.local/bin/docksribe.sh "https://raw.githubusercontent.com/patricksthannon/Dockscribe/refs/heads/main/dockscribe.sh" && chmod +x ~/.local/bin/dockscribe.sh
# Mac OS with curl | into default PATH /usr/local/bin/
curl -L https://raw.githubusercontent.com/patricksthannon/Dockscribe/refs/heads/main/dockscribe.sh -o /usr/local/bin/dockscribe.sh && chmod +x /usr/local/bin/dockscribe.sh
```
2. Test script.
```
dockscribe.sh
```
Output Example:
```
actualbudget/actual-server — Actual server & web app
binwiederhier/ntfy — Send push notifications to your phone or desktop via PUT/POST
ghcr.io/corentinth/it-tools — Collection of handy online tools for developers, with great UX.
ghcr.io/gethomepage/homepage — A highly customizable homepage (or startpage / application dashboard) with Docker and service API integrations.
ghcr.io/immich-app/immich-machine-learning — High performance self-hosted photo and video management solution.
ghcr.io/immich-app/immich-server — High performance self-hosted photo and video management solution.
ghcr.io/jordan-dalby/bytestash — A code snippet storage solution written in React & node.js
ghcr.io/linkwarden/linkwarden — ⚡️⚡️⚡️ Self-hosted collaborative bookmark manager to collect, organize, and preserve webpages, articles, and documents.
gitea/gitea — Gitea: Git with a cup of tea - A painless self-hosted Git service.
jellyfin/jellyfin — The Free Software Media Browser
lscr.io/linuxserver/plex — A Plex Media Server container, brought to you by LinuxServer.io.
lscr.io/linuxserver/syncthing — A Syncthing container, brought to you by LinuxServer.io.
neosmemo/memos — A privacy-first, lightweight note-taking service. Easily capture and share your great thoughts.
postgres — The PostgreSQL object-relational database system provides reliability and data integrity.
redis — Redis is the worlds fastest data platform for caching, vector search, and NoSQL databases.
tensorchord/pgvecto-rs — Scalable Vector Search in Postgres. Revolutionize Vector Search, not Database. pgvector alternative.
vaultwarden/server — Alternative implementation of the Bitwarden server API in Rust, including the Web Vault.
```
If you like this project, consider [buying me a coffee](https://www.buymeacoffee.com/patricksthannon) to keep me motivated! I appreciate it!
<a href="https://www.buymeacoffee.com/patricksthannon" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 40px !important;width: 172px !important;" ></a>
[^1]: Reference: https://apple.stackexchange.com/questions/101090/list-of-all-packages-installed-using-homebrew

View File

@@ -0,0 +1,177 @@
#!/bin/bash
# $$$$$$$\ $$\ $$\ $$\
# $$ __$$\ $$ | \__|$$ |
# $$ | $$ | $$$$$$\ $$$$$$$\ $$ | $$\ $$$$$$$\ $$$$$$$\ $$$$$$\ $$\ $$$$$$$\ $$$$$$\
# $$ | $$ |$$ __$$\ $$ _____|$$ | $$ |$$ _____|$$ _____|$$ __$$\ $$ |$$ __$$\ $$ __$$\
# $$ | $$ |$$ / $$ |$$ / $$$$$$ / \$$$$$$\ $$ / $$ | \__|$$ |$$ | $$ |$$$$$$$$ |
# $$ | $$ |$$ | $$ |$$ | $$ _$$< \____$$\ $$ | $$ | $$ |$$ | $$ |$$ ____|
# $$$$$$$ |\$$$$$$ |\$$$$$$$\ $$ | \$$\ $$$$$$$ |\$$$$$$$\ $$ | $$ |$$$$$$$ |\$$$$$$$\
# \_______/ \______/ \_______|\__| \__|\_______/ \_______|\__| \__|\_______/ \_______|
# By PatrickstHannon https://github.com/patricksthannon
# === jq dependency check & install ===
check_jq() {
if command -v jq >/dev/null 2>&1; then
return 0
fi
echo "jq not found. Attempting to install..."
OS="$(uname -s)"
ARCH="$(uname -m)"
get_jq_url() {
case "$OS" in
Darwin)
echo "https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64"
;;
Linux)
case "$ARCH" in
x86_64|amd64) echo "https://github.com/stedolan/jq/releases/latest/download/jq-linux64" ;;
aarch64|arm64) echo "https://github.com/stedolan/jq/releases/latest/download/jq-linuxarm64" ;;
armv7*) echo "https://github.com/stedolan/jq/releases/latest/download/jq-linuxarm" ;;
*) echo "" ;;
esac
;;
*)
echo ""
;;
esac
}
if [[ "$OS" == "Darwin" ]]; then
if command -v brew >/dev/null 2>&1; then
echo "Installing jq via Homebrew..."
brew install jq && return 0
else
echo "Homebrew not found. Will attempt to download static jq binary."
fi
fi
if [[ "$OS" == "Linux" ]]; then
if command -v apt-get >/dev/null 2>&1; then
echo "Installing jq via apt-get..."
sudo apt-get update && sudo apt-get install -y jq && return 0
elif command -v yum >/dev/null 2>&1; then
echo "Installing jq via yum..."
sudo yum install -y jq && return 0
else
echo "No supported package manager found."
fi
fi
jq_url=$(get_jq_url)
if [[ -z "$jq_url" ]]; then
echo "Unsupported OS or architecture for automatic jq binary download."
return 1
fi
echo "Downloading jq from $jq_url ..."
tmp_jq="$(mktemp)"
if curl -L --fail --silent --show-error -o "$tmp_jq" "$jq_url"; then
chmod +x "$tmp_jq"
if sudo mv "$tmp_jq" /usr/local/bin/jq 2>/dev/null; then
echo "jq installed to /usr/local/bin/jq"
return 0
else
local_bin="$HOME/.local/bin"
mkdir -p "$local_bin"
mv "$tmp_jq" "$local_bin/jq"
echo "jq installed to $local_bin/jq"
echo "Make sure $local_bin is in your PATH."
return 0
fi
else
echo "Failed to download jq binary."
rm -f "$tmp_jq"
return 1
fi
}
if ! check_jq; then
echo "jq is required but could not be installed."
exit 1
fi
# === Docker description fetch functions ===
get_dockerhub_desc() {
local repo="$1"
local url="https://hub.docker.com/v2/repositories/${repo}/"
local json
json=$(curl -s -f "$url" 2>/dev/null | tr -d '\000-\037') || return 1
local desc repo_url
desc=$(echo "$json" | jq -r '.description' 2>/dev/null)
repo_url=$(echo "$json" | jq -r '.repo_url' 2>/dev/null)
if [[ "$desc" != "null" && -n "$desc" ]]; then
echo "$desc"
return 0
elif [[ "$repo_url" =~ github\.com ]]; then
get_github_desc "$repo_url"
return $?
fi
return 1
}
get_github_desc() {
local repo_url="$1"
local api_url=${repo_url/https:\/\/github.com\//https:\/\/api.github.com\/repos\/}
local desc
desc=$(curl -s -f "$api_url" 2>/dev/null | jq -r '.description' 2>/dev/null)
if [[ $? -eq 0 && "$desc" != "null" && -n "$desc" ]]; then
echo "$desc"
return 0
fi
local trimmed="$repo_url"
while [[ "$trimmed" =~ .*/.*-.* ]]; do
trimmed=$(echo "$trimmed" | sed -E 's|-?[^/-]+$||')
local fallback_api_url=${trimmed/https:\/\/github.com\//https:\/\/api.github.com\/repos\/}
desc=$(curl -s -f "$fallback_api_url" 2>/dev/null | jq -r '.description' 2>/dev/null)
if [[ $? -eq 0 && "$desc" != "null" && -n "$desc" ]]; then
echo "$desc"
return 0
fi
done
return 1
}
get_description() {
local image="$1"
if [[ "$image" == lscr.io/* ]]; then
local repo="${image#lscr.io/}"
get_dockerhub_desc "$repo" && return 0
elif [[ "$image" == ghcr.io/* ]]; then
local repo="${image#ghcr.io/}"
local url="https://github.com/${repo}"
get_github_desc "$url" && return 0
elif [[ "$image" == docker.io/* ]]; then
local repo="${image#docker.io/}"
get_dockerhub_desc "$repo" && return 0
elif [[ "$image" == *"/"* ]]; then
get_dockerhub_desc "$image" && return 0
else
get_dockerhub_desc "library/${image}" && return 0
fi
return 1
}
# === Main logic base ===
images=$(docker ps --format '{{.Image}}' | cut -d':' -f1 | sort -u)
for image in $images; do
desc=$(get_description "$image")
if [[ -z "$desc" ]]; then
desc="[no description found]"
fi
echo -e "$image$desc"
done

13
Divers/Laidout.md Normal file
View File

@@ -0,0 +1,13 @@
Laidout
========================
### Install from source
```bash
sudo pacman -S git gcc-multilib pkg-config libpng12 readline libx11 libxext libxi libxft libcups imlib2 fontconfig freetype2 openssl graphicsmagick mesa glu ftgl
yaourt -S makedepend
svn checkout svn://svn.code.sf.net/p/laidout/code/trunk laidout-svn
cd laidout-svn
git clone http://github.com/tomlechner/laxkit.git laxkit
```

View File

@@ -0,0 +1,22 @@
Loïc GENTIL (infos hébergement)
========================
| Service | Login | Password |
| :---- | ----- | :---: |
| **Google** | loic.gentil.kine@gmail.com | YnVoU\#PcA4DPWgh |
| **Clé dauthentification (TOTP)** | agxc b4io nto7 pyqo 4x2e we3d ek46 2j5t | |
| **Google Analytics** [https://www.google.com/webmasters/tools/dashboard?hl=en\&authuser=0\&siteUrl=http://loicgentil.fr/](https://www.google.com/webmasters/tools/dashboard?hl=en&authuser=0&siteUrl=http://loicgentil.fr/) | UA-87786839-1 | |
| **Os2Switch** [https://cpanel.loicgentil.fr](https://cpanel.loicgentil.fr) | loicgentil | o%ke3EFnigdccL |
| \--\> Lune 1 | sc1loicgenti | e\#aoSF@dy2wZi2 |
| \--\> Lune 2 | sc2loicgenti | gV\!wXH^Bhl3\&P& |
| \--\> Lune 3 | sc3loicgenti | wPexaZ5f\#wXhsq |
| \--\> Lune 4 | sc4loicgenti | bECzpTU$^zv2NA |
| **FTP** [ftp.loicgentil.fr](http://ftp.loicgentil.fr) | loicgentil | SdMTp39\*4gyX5\! |
| **Fabform** [https://fabform.io](https://fabform.io) | loicgentil@loicgentil.fr | 7\!q2h$^3FTB7vH |
| **OpenStreetMap** [https://www.openstreetmap.org](https://www.openstreetmap.org) [https://umap.openstreetmap.fr/fr/map/loic-gentil\_1123879](https://umap.openstreetmap.fr/fr/map/loic-gentil_1123879) | loicgentil@loicgentil.fr | 7\!q2h$^3FTB7vH |
| **Maptiler** [https://www.maptiler.com](https://www.maptiler.com) | loicgentil@loicgentil.fr | 7\!q2h$^3FTB7vH |
| Vimeo [https://vimeo.com](https://vimeo.com) | loicgentil@loicgentil.fr | 7\!q2h$^3FTB7vH |
| **Wordpress** [https://wordpress.com/fr/](https://wordpress.com/fr/) | loicgentil@loicgentil.fr | YnVoU\#PcA4DPWgh |
[Google Search Console](https://search.google.com/search-console?resource_id=https%3A%2F%2Floicgentil.fr%2F&hl=en)
[https://freesitemapgenerator.com/](https://freesitemapgenerator.com/) ([loic.gentil.kine@gmail.com](mailto:loic.gentil.kine@gmail.com) / osteopathe)

View File

@@ -32,79 +32,3 @@ dns_ovh_application_key = 3f8bdfed17f848d8
dns_ovh_application_secret = 6946758d7515ecef108aeb286bf3c7d0
dns_ovh_consumer_key = 94b2ddf482d36421a33aa6b3aa515956
```
#### Zone DNS de delmar.bzh
```txt
$TTL 3600
@ IN SOA dns106.ovh.net. tech.ovh.net. (2025122600 86400 3600 3600000 300)
IN NS dns106.ovh.net.
IN NS ns106.ovh.net.
IN MX 100 mx3.mail.ovh.net.
IN MX 50 mx2.mail.ovh.net.
IN MX 10 mx1.mail.ovh.net.
IN MX 0 mx0.mail.ovh.net.
IN A 213.186.33.5
IN TXT "v=spf1 include:mx.ovh.com ~all"
IN TXT "1|www.delmar.bzh"
IN TXT "openpgp4fpr:E22A8974BD3F45E3A827AEB891AFB168A1EAD35C"
IN TXT "google-site-verification=j7RPCRYeiAgvZ4uHOD3ZQ4uqi-vPQ-UUmyVD9WXv4t8"
IN CAA 0 issue "letsencrypt.org"
_autodiscover._tcp IN SRV 0 0 443 pro1.mail.ovh.net.
_dmarc IN TXT "v=DMARC1;p=none;sp=none;aspf=r;"
affine IN A 176.188.240.123
asm IN A 176.188.240.123
auth IN A 176.188.240.123
autoconfig IN CNAME autodiscover.mail.ovh.net.
autodiscover IN CNAME mailconfig.ovh.net.
books IN A 176.188.240.123
bookstack IN A 176.188.240.123
cap IN A 176.188.240.123
cloud IN A 176.188.240.123
crm IN A 176.188.240.123
cs IN A 176.188.240.123
ctr IN A 176.188.240.123
cvs IN A 176.188.240.123
dev IN A 176.188.240.123
dia IN A 176.188.240.123
dkr IN A 176.188.240.123
dolibarr IN A 176.188.240.123
draw IN A 176.188.240.123
ftp IN CNAME delmar.bzh.
gen IN A 176.188.240.123
git IN A 176.188.240.123
gotify IN A 176.188.240.123
home-assistant IN A 176.188.240.123
homepage IN A 176.188.240.123
it IN A 176.188.240.123
jellyfin IN A 176.188.240.123
jellyseerr IN A 176.188.240.123
kontadenn IN A 176.188.240.123
mailbear IN A 176.188.240.123
minio IN A 176.188.240.123
mon IN A 176.188.240.123
nds IN A 176.188.240.123
nsns IN A 176.188.240.123
octoprint IN A 176.188.240.123
ovhemp1116203-selector1._domainkey 60 IN CNAME ovhemp1116203-selector1._domainkey.274927.cq.dkim.mail.ovh.net.
ovhemp1116203-selector2._domainkey 60 IN CNAME ovhemp1116203-selector2._domainkey.274928.az.dkim.mail.ovh.net.
pan IN A 176.188.240.123
paperless IN A 176.188.240.123
pdf IN A 176.188.240.123
penpot IN A 176.188.240.123
pip IN A 176.188.240.123
rallly IN A 176.188.240.123
scanopy IN A 176.188.240.123
search IN A 176.188.240.123
send IN A 176.188.240.123
shop IN A 176.188.240.123
stream IN A 176.188.240.123
tpml IN A 176.188.240.123
twip IN A 176.188.240.123
ugo IN A 176.188.240.123
vault IN A 176.188.240.123
wizarr IN A 176.188.240.123
www IN A 176.188.240.123
xcd IN A 176.188.240.123
zik IN A 176.188.240.123
```

View File

@@ -1,66 +1,63 @@
#### Zone DNS modifiée de delmar.bzh
```txt
$TTL 3600
@ IN SOA dns106.ovh.net. tech.ovh.net. (2025102306 86400 3600 3600000 300)
@ IN SOA dns106.ovh.net. tech.ovh.net. (2026012101 86400 3600 3600000 300)
IN NS dns106.ovh.net.
IN NS ns106.ovh.net.
IN MX 10 mail.delmar.bzh.
IN MX 100 mx3.mail.ovh.net.
IN MX 50 mx2.mail.ovh.net.
IN MX 10 mx1.mail.ovh.net.
IN MX 0 mx0.mail.ovh.net.
IN A 213.186.33.5
IN TXT "v=spf1 include:mx.ovh.com ~all"
IN TXT "1|www.delmar.bzh"
IN TXT "openpgp4fpr:E22A8974BD3F45E3A827AEB891AFB168A1EAD35C"
IN TXT "google-site-verification=j7RPCRYeiAgvZ4uHOD3ZQ4uqi-vPQ-UUmyVD9WXv4t8"
IN CAA 0 issue "letsencrypt.org"
202510e._domainkey IN TXT "v=DKIM1; k=ed25519; h=sha256; p=zqH42wS9S4UgMXBrPKZxJSi45eEjewXJsboHRq7p30E="
202510r._domainkey IN TXT ( "v=DKIM1; k=rsa; h=sha256; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoqg5JMaBF3fDBZ36KRULZR7HnuytV0SmuezMZyPP4KOPyhMdK2aqe12ct7JwCs5dCBPD3nGxqqjHgPDXv4l+pqVIjuGKNYKOfVMvHxlMQWAudyyMfYwj8ve/wCKB3LKsOo+y7JFMNYyWw+/UIBqM2gY4Y2aOCR/tpZo+wQGYQPzkHHE7gl" "81pT0YwtHR6FZZM7vrKC3qdZ8iglfIUvwsCOl+B4alYPfWSpuKT8moWP58LQX8qgDdWLYvMzN7X4ljW2lNiXdxHcrGsPCCE87ROregNNpImKMuIZl/dn7MJey+8NzAqPo2PT9nUmTEpQ+1Ty52TrBwiT/2JeB+gdeJnQIDAQAB" )
_dmarc IN TXT "v=DMARC1; p=reject; rua=mailto:postmaster@delmar.bzh; ruf=mailto:postmaster@delmar.bzh"
_imaps._tcp IN SRV 0 1 993 mail.delmar.bzh.
_smtp._tls IN TXT "v=TLSRPTv1; rua=mailto:postmaster@delmar.bzh"
_submissions._tcp IN SRV 0 1 465 mail.delmar.bzh.
_autodiscover._tcp IN SRV 0 0 443 pro1.mail.ovh.net.
_dmarc IN TXT "v=DMARC1;p=none;sp=none;aspf=r;"
affine IN A 176.188.240.123
asm IN A 176.188.240.123
auth IN A 176.188.240.123
autoconfig IN A 176.188.240.123
autodiscover IN A 176.188.240.123
autoconfig IN CNAME autodiscover.mail.ovh.net.
autodiscover IN CNAME mailconfig.ovh.net.
books IN A 176.188.240.123
bookstack IN A 176.188.240.123
cap IN A 176.188.240.123
cloud IN A 176.188.240.123
cpt IN A 176.188.240.123
cnvrt IN A 176.188.240.123
crm IN A 176.188.240.123
cs IN A 176.188.240.123
ctr IN A 176.188.240.123
cvs IN A 176.188.240.123
dev IN A 176.188.240.123
dia IN A 176.188.240.123
dkr IN A 176.188.240.123
dolibarr IN A 176.188.240.123
draw IN A 176.188.240.123
ftp IN A 176.188.240.123
ftp IN CNAME delmar.bzh.
gen IN A 176.188.240.123
git IN A 176.188.240.123
gotify IN A 176.188.240.123
home-assistant IN A 176.188.240.123
homepage IN A 176.188.240.123
inv IN A 176.188.240.123
it IN A 176.188.240.123
jellyfin IN A 176.188.240.123
jellyseerr IN A 176.188.240.123
kontadenn IN A 176.188.240.123
mail IN A 176.188.240.123
mail IN TXT "v=spf1 a ra=postmaster -all"
mail IN TXT "v=spf1 mx ra=postmaster -all"
mailbear IN A 176.188.240.123
minio IN A 176.188.240.123
mon IN A 176.188.240.123
nds IN A 176.188.240.123
nsns IN A 176.188.240.123
octoprint IN A 176.188.240.123
ovhemp1116203-selector1._domainkey 60 IN CNAME ovhemp1116203-selector1._domainkey.274927.cq.dkim.mail.ovh.net.
ovhemp1116203-selector2._domainkey 60 IN CNAME ovhemp1116203-selector2._domainkey.274928.az.dkim.mail.ovh.net.
paperless IN A 176.188.240.123
pdf IN A 176.188.240.123
penpot IN A 176.188.240.123
rallly IN A 176.188.240.123
pip IN A 176.188.240.123
scanopy IN A 176.188.240.123
search IN A 176.188.240.123
send IN A 176.188.240.123
shop IN A 176.188.240.123
ssm IN A 176.188.240.123
stream IN A 176.188.240.123
tpml IN A 176.188.240.123
twip IN A 176.188.240.123
@@ -69,4 +66,4 @@ vault IN A 176.188.240.123
wizarr IN A 176.188.240.123
www IN A 176.188.240.123
xcd IN A 176.188.240.123
```
zik IN A 176.188.240.123

View File

@@ -1,70 +1,75 @@
### Zone DNS
```
#### Zone DNS de delmar.bzh
```txt
$TTL 3600
@ IN SOA dns106.ovh.net. tech.ovh.net. (2025100947 86400 3600 3600000 60)
IN NS dns106.ovh.net.
IN NS ns106.ovh.net.
IN MX 50 mx2.mail.ovh.net.
IN MX 5 mx1.mail.ovh.net.
IN MX 100 mx3.mail.ovh.net.
IN MX 1 mx0.mail.ovh.net.
IN A 213.186.33.5
IN TXT "v=spf1 include:mx.ovh.com ~all"
IN TXT "1|www.delmar.bzh"
IN TXT "openpgp4fpr:E22A8974BD3F45E3A827AEB891AFB168A1EAD35C"
IN TXT "google-site-verification=j7RPCRYeiAgvZ4uHOD3ZQ4uqi-vPQ-UUmyVD9WXv4t8"
IN CAA 0 issue "letsencrypt.org"
@ IN SOA dns106.ovh.net. tech.ovh.net. (2025122600 86400 3600 3600000 300)
IN NS dns106.ovh.net.
IN NS ns106.ovh.net.
IN MX 100 mx3.mail.ovh.net.
IN MX 50 mx2.mail.ovh.net.
IN MX 10 mx1.mail.ovh.net.
IN MX 0 mx0.mail.ovh.net.
IN A 213.186.33.5
IN TXT "v=spf1 include:mx.ovh.com ~all"
IN TXT "1|www.delmar.bzh"
IN TXT "openpgp4fpr:E22A8974BD3F45E3A827AEB891AFB168A1EAD35C"
IN TXT "google-site-verification=j7RPCRYeiAgvZ4uHOD3ZQ4uqi-vPQ-UUmyVD9WXv4t8"
IN CAA 0 issue "letsencrypt.org"
_autodiscover._tcp IN SRV 0 0 443 pro1.mail.ovh.net.
_dmarc IN TXT "v=DMARC1;p=none;sp=none;aspf=r;"
_dmarc IN TXT "v=DMARC1;p=none;sp=none;aspf=r;"
affine IN A 176.188.240.123
asm IN A 176.188.240.123
auth IN A 176.188.240.123
autoconfig IN CNAME autodiscover.mail.ovh.net.
autodiscover IN CNAME mailconfig.ovh.net.
books IN A 176.188.240.123
bookstack IN A 176.188.240.123
cap IN A 176.188.240.123
cloud IN A 176.188.240.123
crm IN A 176.188.240.123
cs IN A 176.188.240.123
ctr IN A 176.188.240.123
cvs IN A 176.188.240.123
dev IN A 176.188.240.123
dia IN A 176.188.240.123
dkr IN A 176.188.240.123
dolibarr IN A 176.188.240.123
draw IN A 176.188.240.123
ftp IN CNAME delmar.bzh.
gen IN A 176.188.240.123
git IN A 176.188.240.123
gotify IN A 176.188.240.123
home-assistant IN A 176.188.240.123
homepage IN A 176.188.240.123
it IN A 176.188.240.123
jellyfin IN A 176.188.240.123
jellyseerr IN A 176.188.240.123
kontadenn IN A 176.188.240.123
mailbear IN A 176.188.240.123
minio IN A 176.188.240.123
mon IN A 176.188.240.123
nds IN A 176.188.240.123
nsns IN A 176.188.240.123
octoprint IN A 176.188.240.123
ovhemp1116203-selector1._domainkey 60 IN CNAME ovhemp1116203-selector1._domainkey.274927.cq.dkim.mail.ovh.net.
ovhemp1116203-selector2._domainkey 60 IN CNAME ovhemp1116203-selector2._domainkey.274928.az.dkim.mail.ovh.net.
autoconfig IN CNAME autodiscover.mail.ovh.net.
autodiscover IN CNAME mailconfig.ovh.net.
ftp IN CNAME delmar.bzh.
3dm IN A 176.188.240.123
affine IN A 176.188.240.123
asm IN A 176.188.240.123
books IN A 176.188.240.123
bookstack IN A 176.188.240.123
cap IN A 176.188.240.123
cloud IN A 176.188.240.123
cpt IN A 176.188.240.123
cs IN A 176.188.240.123
ctr IN A 176.188.240.123
cvs IN A 176.188.240.123
dev IN A 176.188.240.123
dkr IN A 176.188.240.123
dolibarr IN A 176.188.240.123
draw IN A 176.188.240.123
gen IN A 176.188.240.123
git IN A 176.188.240.123
gotify IN A 176.188.240.123
home-assistant IN A 176.188.240.123
homepage IN A 176.188.240.123
it IN A 176.188.240.123
jellyfin IN A 176.188.240.123
jellyseerr IN A 176.188.240.123
kontadenn IN A 176.188.240.123
mailbear IN A 176.188.240.123
minio IN A 176.188.240.123
mmm IN A 176.188.240.123
nds IN A 176.188.240.123
nsns IN A 176.188.240.123
octoprint IN A 176.188.240.123
paperless IN A 176.188.240.123
pdf IN A 176.188.240.123
penpot IN A 176.188.240.123
rallly IN A 176.188.240.123
search IN A 176.188.240.123
send IN A 176.188.240.123
shop IN A 176.188.240.123
ssm IN A 176.188.240.123
stream IN A 176.188.240.123
tpml IN A 176.188.240.123
twip IN A 176.188.240.123
ugo IN A 176.188.240.123
vault IN A 176.188.240.123
wizarr IN A 176.188.240.123
wud IN A 176.188.240.123
www IN A 176.188.240.123
pan IN A 176.188.240.123
paperless IN A 176.188.240.123
pdf IN A 176.188.240.123
penpot IN A 176.188.240.123
pip IN A 176.188.240.123
rallly IN A 176.188.240.123
scanopy IN A 176.188.240.123
search IN A 176.188.240.123
send IN A 176.188.240.123
shop IN A 176.188.240.123
stream IN A 176.188.240.123
tpml IN A 176.188.240.123
twip IN A 176.188.240.123
ugo IN A 176.188.240.123
vault IN A 176.188.240.123
wizarr IN A 176.188.240.123
www IN A 176.188.240.123
xcd IN A 176.188.240.123
zik IN A 176.188.240.123
```

View File

@@ -0,0 +1,7 @@
carrd (corinne)
========================
> <https://carrd.co/dashboard>
clg35760@gmail.com / chuck-cargo-poker
> <https://clg354.carrd.co>

View File

@@ -0,0 +1,64 @@
# Version control
.git
.gitignore
.github
# Node.js
node_modules
# We don't exclude bun.lock* as it's needed for the build
npm-debug.log
yarn-debug.log
yarn-error.log
# Build outputs
dist
build
.next
out
# Environment variables
.env
.env.local
.env.development
.env.test
.env.production
# IDE and editor files
.idea
.vscode
*.swp
*.swo
*~
# OS files
.DS_Store
Thumbs.db
# Test coverage
coverage
.nyc_output
# Docker
Dockerfile
.dockerignore
docker-compose.yml
docker-compose.*.yml
# Documentation
README.md
LICENSE
docs
# Temporary files
tmp
temp
*.tmp
*.temp
# Logs
logs
*.log
# Cache
.cache
.npm

View File

@@ -0,0 +1,186 @@
# Gitea Mirror Configuration
# Copy this to .env and update with your values
# ===========================================
# CORE CONFIGURATION
# ===========================================
# Application Configuration
NODE_ENV=production
HOST=0.0.0.0
PORT=4321
# Database Configuration
# For self-hosted, SQLite is used by default
DATABASE_URL=sqlite://data/gitea-mirror.db
# Security
# Generate with: openssl rand -base64 32
BETTER_AUTH_SECRET=change-this-to-a-secure-random-string-in-production
BETTER_AUTH_URL=http://localhost:4321
# PUBLIC_BETTER_AUTH_URL=https://your-domain.com # Optional: Set this if accessing from different origins (e.g., IP and domain)
# ENCRYPTION_SECRET=optional-encryption-key-for-token-encryption # Generate with: openssl rand -base64 48
# ===========================================
# DOCKER CONFIGURATION (Optional)
# ===========================================
# Docker Registry Configuration
DOCKER_REGISTRY=ghcr.io
DOCKER_IMAGE=raylabshq/gitea-mirror:
DOCKER_TAG=latest
# ===========================================
# GITHUB CONFIGURATION
# All settings can also be configured via web UI
# ===========================================
# Basic GitHub Settings
# GITHUB_USERNAME=your-github-username
# GITHUB_TOKEN=your-github-personal-access-token
# GITHUB_TYPE=personal # Options: personal, organization
# Repository Selection
# PRIVATE_REPOSITORIES=false
# PUBLIC_REPOSITORIES=true
# INCLUDE_ARCHIVED=false
# SKIP_FORKS=false
# MIRROR_STARRED=false
# STARRED_REPOS_ORG=starred # Organization name for starred repos
# Organization Settings
# MIRROR_ORGANIZATIONS=false
# PRESERVE_ORG_STRUCTURE=false
# ONLY_MIRROR_ORGS=false
# Mirror Strategy
# MIRROR_STRATEGY=preserve # Options: preserve, single-org, flat-user, mixed
# Advanced GitHub Settings
# SKIP_STARRED_ISSUES=false # Enable lightweight mode for starred repos
# ===========================================
# GITEA CONFIGURATION
# All settings can also be configured via web UI
# ===========================================
# Basic Gitea Settings
# GITEA_URL=http://gitea:3000
# GITEA_TOKEN=your-local-gitea-token
# GITEA_USERNAME=your-local-gitea-username
# GITEA_ORGANIZATION=github-mirrors # Default organization for single-org strategy
# Repository Settings
# GITEA_ORG_VISIBILITY=public # Options: public, private, limited, default
# GITEA_MIRROR_INTERVAL=8h # Mirror sync interval (e.g., 30m, 1h, 8h, 24h) - automatically enables scheduler
# GITEA_LFS=false # Enable LFS support
# GITEA_CREATE_ORG=true # Auto-create organizations
# GITEA_PRESERVE_VISIBILITY=false # Preserve GitHub repo visibility in Gitea
# Template Settings (for using repository templates)
# GITEA_TEMPLATE_OWNER=template-owner
# GITEA_TEMPLATE_REPO=template-repo
# Topic Settings
# GITEA_ADD_TOPICS=true # Add topics to repositories
# GITEA_TOPIC_PREFIX=gh- # Prefix for topics
# Fork Handling
# GITEA_FORK_STRATEGY=reference # Options: skip, reference, full-copy
# ===========================================
# MIRROR OPTIONS
# Control what gets mirrored from GitHub
# ===========================================
# Release and Metadata
# MIRROR_RELEASES=false # Mirror GitHub releases
# RELEASE_LIMIT=10 # Maximum number of releases to mirror per repository
# MIRROR_WIKI=false # Mirror wiki content
# Issue Tracking (requires MIRROR_METADATA=true)
# MIRROR_METADATA=false # Master toggle for metadata mirroring
# MIRROR_ISSUES=false # Mirror issues
# MIRROR_PULL_REQUESTS=false # Mirror pull requests
# MIRROR_LABELS=false # Mirror labels
# MIRROR_MILESTONES=false # Mirror milestones
# ===========================================
# AUTOMATION CONFIGURATION
# Schedule automatic mirroring
# ===========================================
# Basic Schedule Settings
# SCHEDULE_ENABLED=false # When true, auto-imports and mirrors all repos on startup (v3.5.3+)
# SCHEDULE_INTERVAL=3600 # Interval in seconds or cron expression (e.g., "0 2 * * *")
# GITEA_MIRROR_INTERVAL=8h # Mirror sync interval (5m, 30m, 1h, 8h, 24h, 1d, 7d) - also triggers auto-start
# AUTO_IMPORT_REPOS=true # Automatically discover and import new GitHub repositories during syncs
# DELAY=3600 # Legacy: same as SCHEDULE_INTERVAL, kept for backward compatibility
# Execution Settings
# SCHEDULE_CONCURRENT=false # Allow concurrent mirror operations
# SCHEDULE_BATCH_SIZE=10 # Number of repos to process in parallel
# SCHEDULE_PAUSE_BETWEEN_BATCHES=5000 # Pause between batches (ms)
# Retry Configuration
# SCHEDULE_RETRY_ATTEMPTS=3
# SCHEDULE_RETRY_DELAY=60000 # Delay between retries (ms)
# SCHEDULE_TIMEOUT=3600000 # Max time for a mirror operation (ms)
# SCHEDULE_AUTO_RETRY=true
# Update Detection
# SCHEDULE_ONLY_MIRROR_UPDATED=false # Only mirror repos with updates
# SCHEDULE_UPDATE_INTERVAL=86400000 # Check for updates interval (ms)
# SCHEDULE_SKIP_RECENTLY_MIRRORED=true
# SCHEDULE_RECENT_THRESHOLD=3600000 # Skip if mirrored within this time (ms)
# Maintenance
# SCHEDULE_CLEANUP_BEFORE_MIRROR=false # Run cleanup before mirroring
# Notifications
# SCHEDULE_NOTIFY_ON_FAILURE=true
# SCHEDULE_NOTIFY_ON_SUCCESS=false
# SCHEDULE_LOG_LEVEL=info # Options: error, warn, info, debug
# SCHEDULE_TIMEZONE=UTC
# ===========================================
# DATABASE CLEANUP CONFIGURATION
# Automatic cleanup of old events and data
# ===========================================
# Basic Cleanup Settings
# CLEANUP_ENABLED=false
# CLEANUP_RETENTION_DAYS=7 # Days to keep events
# Repository Cleanup (v3.4.0+)
# CLEANUP_DELETE_FROM_GITEA=false # Delete repos from Gitea
# CLEANUP_DELETE_IF_NOT_IN_GITHUB=false # Auto-remove repos that no longer exist in GitHub
# CLEANUP_ORPHANED_REPO_ACTION=archive # Options: skip, archive, delete
# CLEANUP_DRY_RUN=true # Test mode without actual deletion (set to false for production)
# Protected Repositories (comma-separated)
# CLEANUP_PROTECTED_REPOS=important-repo,critical-project
# Cleanup Execution
# CLEANUP_BATCH_SIZE=10
# CLEANUP_PAUSE_BETWEEN_DELETES=2000 # Pause between deletions (ms)
# ===========================================
# AUTHENTICATION CONFIGURATION
# ===========================================
# Header Authentication (for Reverse Proxy SSO)
# Enable automatic authentication via reverse proxy headers
# HEADER_AUTH_ENABLED=false
# HEADER_AUTH_USER_HEADER=X-Authentik-Username
# HEADER_AUTH_EMAIL_HEADER=X-Authentik-Email
# HEADER_AUTH_NAME_HEADER=X-Authentik-Name
# HEADER_AUTH_AUTO_PROVISION=false
# HEADER_AUTH_ALLOWED_DOMAINS=example.com,company.org
# ===========================================
# OPTIONAL FEATURES
# ===========================================
# TLS/SSL Configuration
# GITEA_SKIP_TLS_VERIFY=false # WARNING: Only use for testing

View File

@@ -0,0 +1 @@
use flake

39
Divers/gitea-mirror/.gitignore vendored Normal file
View File

@@ -0,0 +1,39 @@
# build output
dist/
# generated types
.astro/
# dependencies
node_modules/
# logs
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
# environment variables
.env
.env.production
# database files
data/gitea-mirror.db
# macOS-specific files
.DS_Store
# jetbrains setting folder
.idea/
# Custom CA certificates (exclude actual certs but keep README)
certs/*.crt
certs/*.pem
certs/*.cer
!certs/README.md
# Nix build artifacts
result
result-*
.direnv/

View File

@@ -0,0 +1,46 @@
# Repository Guidelines
## Project Structure & Module Organization
- `src/` app code
- `components/` (React, PascalCase files), `pages/` (Astro/API routes), `lib/` (domain + utilities, kebab-case), `hooks/`, `layouts/`, `styles/`, `tests/`, `types/`, `data/`, `content/`.
- `scripts/` operational TS scripts (DB init, recovery): e.g., `scripts/manage-db.ts`.
- `drizzle/` SQL migrations; `data/` runtime SQLite (`gitea-mirror.db`).
- `public/` static assets; `dist/` build output.
- Key config: `astro.config.mjs`, `tsconfig.json` (alias `@/* → src/*`), `bunfig.toml` (test preload), `.env(.example)`.
## Build, Test, and Development Commands
- Prereq: Bun `>= 1.2.9` (see `package.json`).
- Setup: `bun run setup` install deps and init DB.
- Dev: `bun run dev` start Astro dev server.
- Build: `bun run build` produce `dist/`.
- Preview/Start: `bun run preview` (static preview) or `bun run start` (SSR entry).
- Database: `bun run db:generate|migrate|push|studio` and `bun run manage-db init|check|fix|reset-users`.
- Tests: `bun test` | `bun run test:watch` | `bun run test:coverage`.
- Docker: see `docker-compose.yml` and variants in repo root.
## Coding Style & Naming Conventions
- Language: TypeScript, Astro, React.
- Indentation: 2 spaces; keep existing semicolon/quote style in touched files.
- Components: PascalCase `.tsx` in `src/components/` (e.g., `MainLayout.tsx`).
- Modules/utils: kebab-case in `src/lib/` (e.g., `gitea-enhanced.ts`).
- Imports: prefer alias `@/…` (configured in `tsconfig.json`).
- Do not introduce new lint/format configs; follow current patterns.
## Testing Guidelines
- Runner: Bun test (`bun:test`) with preload `src/tests/setup.bun.ts` (see `bunfig.toml`).
- Location/Names: `**/*.test.ts(x)` under `src/**` (examples in `src/lib/**`).
- Scope: add unit tests for new logic and API route tests for handlers.
- Aim for meaningful coverage on DB, auth, and mirroring paths.
## Commit & Pull Request Guidelines
- Commits: short, imperative, scoped when helpful (e.g., `lib: fix token parsing`, `ui: align buttons`).
- PRs must include:
- Summary, rationale, and testing steps/commands.
- Linked issues (e.g., `Closes #123`).
- Screenshots/gifs for UI changes.
- Notes on DB/migration or .env impacts; update `docs/`/CHANGELOG if applicable.
## Security & Configuration Tips
- Never commit secrets. Copy `.env.example``.env` and fill values; prefer `bun run startup-env-config` to validate.
- SQLite files live in `data/`; avoid committing generated DBs.
- Certificates (if used) reside in `certs/`; manage locally or via Docker secrets.

View File

@@ -0,0 +1,474 @@
# Changelog
All notable changes to the Gitea Mirror project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- Git LFS (Large File Storage) support for mirroring (#74)
- New UI checkbox "Mirror LFS" in Mirror Options
- Automatic LFS object transfer when enabled
- Documentation for Gitea server LFS requirements
- Repository "ignored" status to skip specific repos from mirroring (#75)
- Repositories can be marked as ignored to exclude from all operations
- Scheduler automatically skips ignored repositories
- Enhanced error handling for all metadata mirroring operations
- Individual try-catch blocks for issues, PRs, labels, milestones
- Operations continue even if individual components fail
- Support for BETTER_AUTH_TRUSTED_ORIGINS environment variable (#63)
- Enables access via multiple URLs (local IP + domain)
- Comma-separated trusted origins configuration
- Proper documentation for multi-URL access patterns
- Comprehensive fix report documentation
### Fixed
- Fixed metadata mirroring authentication errors (#68)
- Changed field checking from `username` to `defaultOwner` in metadata functions
- Added proper field validation for all metadata operations
- Fixed automatic mirroring scheduler issues (#72)
- Improved interval parsing and error handling
- Fixed OIDC authentication 500 errors with Authentik (#73)
- Added URL validation in Better Auth configuration
- Prevented undefined URL errors in auth callback
- Fixed SSL certificate handling in Docker (#48)
- NODE_EXTRA_CA_CERTS no longer gets overridden
- Proper preservation of custom CA certificates
- Fixed reverse proxy base domain issues (#63)
- Better handling of custom subdomains
- Support for trusted origins configuration
- Fixed configuration persistence bugs (#49)
- Config merging now preserves all fields
- Retention period settings no longer reset
- Fixed sync failures with improved error handling (#51)
- Comprehensive error wrapping for all operations
- Better error messages and logging
### Improved
- Enhanced logging throughout metadata mirroring operations
- Detailed success/failure messages for each component
- Configuration details logged for debugging
- Better configuration state management
- Proper merging of loaded configs with defaults
- Preservation of user settings on refresh
- Updated documentation
- Added LFS feature documentation
- Updated README with new features
- Enhanced CLAUDE.md with repository status definitions
## [3.7.1] - 2025-09-14
### Fixed
- Cleanup archiving for mirror repositories now works reliably (refs #84; awaiting user confirmation).
- Gitea rejects names violating the AlphaDashDot rule; archiving a mirror now uses a sanitized rename strategy (`archived-<name>`), with a timestamped fallback on conflicts or validation errors.
- Owner resolution during cleanup no longer uses the GitHub owner by mistake. It prefers `mirroredLocation`, falls back to computed Gitea owner via configuration, and verifies location with a presence check to avoid `GetUserByName` 404s.
- Repositories UI crash resolved when cleanup marked repos as archived.
- Added `"archived"` to repository/job status enums, fixing Zod validation errors on the Repositories page.
### Changed
- Archiving logic for mirror repos is non-destructive by design: data is preserved, repo is renamed with an archive marker, and mirror interval is reduced (besteffort) to minimize sync attempts.
- Cleanup service updates DB to `status: "archived"` and `isArchived: true` on successful archive path.
### Notes
- This release addresses the scenario where a GitHub source disappears (deleted/banned), ensuring Gitea backups are preserved even when using `CLEANUP_DELETE_IF_NOT_IN_GITHUB=true` with `CLEANUP_ORPHANED_REPO_ACTION=archive`.
- No database migration required.
## [3.2.6] - 2025-08-09
### Fixed
- Added missing release asset mirroring functionality (APK, ZIP, Binary files)
- Release assets (attachments) are now properly downloaded from GitHub and uploaded to Gitea
- Fixed missing metadata component configuration checks
### Added
- Full support for mirroring release assets/attachments
- Debug logging for metadata component configuration to help troubleshoot mirroring issues
- Download and upload progress logging for release assets
### Improved
- Enhanced release mirroring to include all associated binary files and attachments
- Better visibility into which metadata components are enabled/disabled
- More detailed logging during the release asset transfer process
### Notes
This patch adds the missing functionality to mirror release assets (APK, ZIP, Binary files, etc.) that was reported in Issue #68. Previously only release metadata was being mirrored, now all attachments are properly transferred to Gitea.
## [3.2.5] - 2025-08-09
### Fixed
- Fixed critical authentication issue in releases mirroring that was still using encrypted tokens
- Added missing repository existence check for releases mirroring function
- Fixed "user does not exist [uid: 0]" error specifically affecting GitHub releases synchronization
### Improved
- Enhanced releases mirroring with duplicate detection to avoid errors on re-runs
- Better error handling and logging for release operations with [Releases] prefix
- Added individual release error handling to continue mirroring even if some releases fail
### Notes
This patch completes the authentication fixes started in v3.2.4, specifically addressing the releases mirroring function that was accidentally missed in the previous update.
## [3.2.4] - 2025-08-09
### Fixed
- Fixed critical authentication issue causing "user does not exist [uid: 0]" errors during metadata mirroring (Issue #68)
- Fixed inconsistent token handling across Gitea API calls
- Fixed metadata mirroring functions attempting to operate on non-existent repositories
- Fixed organization creation failing silently without proper error messages
### Added
- Pre-flight authentication validation for all Gitea operations
- Repository existence verification before metadata mirroring
- Graceful fallback to user account when organization creation fails due to permissions
- Authentication validation utilities for debugging configuration issues
- Diagnostic test scripts for troubleshooting authentication problems
### Improved
- Enhanced error messages with specific guidance for authentication failures
- Better identification and logging of permission-related errors
- More robust organization creation with retry logic and better error handling
- Consistent token decryption across all API operations
- Clearer error reporting for metadata mirroring failures
### Security
- Fixed potential exposure of encrypted tokens in API calls
- Improved token handling to ensure proper decryption before use
## [3.2.0] - 2025-07-31
### Fixed
- Fixed Zod validation error in activity logs by correcting invalid "success" status values to "synced"
- Resolved activity fetch API errors that occurred after mirroring operations
### Changed
- Improved error handling and validation for mirror job status tracking
- Enhanced reliability of organization creation and mirroring processes
### Internal
- Consolidated Gitea integration modules for better maintainability
- Improved test coverage for mirror operations
## [3.1.1] - 2025-07-30
### Fixed
- Various bug fixes and stability improvements
## [3.1.0] - 2025-07-21
### Added
- Support for GITHUB_EXCLUDED_ORGS environment variable to filter out specific organizations during discovery
- New textarea UI component for improved form inputs in configuration
### Fixed
- Fixed test failures related to mirror strategy configuration location
- Corrected organization repository routing logic for different mirror strategies
- Fixed starred repositories organization routing bug
- Resolved SSO and OIDC authentication issues
### Improved
- Enhanced organization configuration for better repository routing control
- Better handling of mirror strategies in test suite
- Improved error handling in authentication flows
## [3.0.0] - 2025-07-17
### 🔴 Breaking Changes
- **Authentication System Overhaul**: Migrated from JWT to Better Auth session-based authentication
- **Login Method Changed**: Users now log in with email instead of username
- **Environment Variables**: `JWT_SECRET` renamed to `BETTER_AUTH_SECRET`, new `BETTER_AUTH_URL` required
- **API Endpoints**: Authentication endpoints moved from `/api/auth/login` to `/api/auth/[...all]`
### Added
- **Token Encryption**: All GitHub and Gitea tokens now encrypted with AES-256-GCM
- **SSO/OIDC Support**: Enterprise authentication with OAuth providers (Google, Azure AD, Okta, Authentik, etc.)
- **Header Authentication**: Support for reverse proxy authentication headers (Authentik, Authelia, Traefik Forward Auth)
- **OAuth Provider**: Gitea Mirror can act as an OIDC provider for other applications
- **Automated Migration**: Docker containers auto-migrate from v2 to v3
- **Session Management**: Improved security with session-based authentication
- **Database Migration System**: Drizzle Kit for better schema management
- **Zod v4 Compatibility**: Updated to Zod v4 for schema validation
### Improved
- **Security**: Enhanced error handling and security practices throughout
- **Documentation**: Comprehensive migration guide for v2 to v3 upgrade
- **User Management**: Better Auth provides improved user lifecycle management
- **Database Schema**: Optimized with proper indexes and relationships
- **Password Hashing**: Using bcrypt via Better Auth for secure password storage
### Fixed
- Mirroring issues for starred repositories
- Various security vulnerabilities in authentication system
- Improved error handling across all API endpoints
### Migration Required
- All users must re-authenticate after upgrade
- Existing tokens will be automatically encrypted
- Database schema updates applied automatically
- See [Migration Guide](MIGRATION_GUIDE.md) for detailed instructions
## [2.22.0] - 2025-07-07
### Added
- Comprehensive mobile and responsive design support across the entire application
- New drawer UI component for enhanced mobile navigation
- Mobile-specific layouts for major components (ActivityLog, Header, Organization, Repository)
- Mobile screenshots in documentation showcasing responsive design
### Improved
- Enhanced mobile user experience with optimized layouts for smaller screens
- Updated organization list cards with better mobile responsiveness
- Better touch interaction support throughout the application
### Fixed
- Type definition issues resolved
- Removed unnecessary console.log statements
### Documentation
- Updated README with mobile usage instructions and screenshots
- Added mobile-specific documentation sections
## [2.20.1] - 2025-07-07
### Fixed
- Fixed mixed mode organization strategy not persisting after page refresh
- Added missing "mixed" case handler in GiteaConfigForm component
- Enhanced getMirrorStrategy function to properly detect mixed mode configuration
- Updated dependencies to latest versions
## [2.20.0] - 2025-07-07
### Changed
- **BREAKING**: Repository moved from `arunavo4/gitea-mirror` to `RayLabsHQ/gitea-mirror`
- Docker images now hosted at `ghcr.io/raylabshq/gitea-mirror`
- Updated all repository references and links to new organization
- License changed from MIT to GNU General Public License v3.0
### Fixed
- Updated GitHub API endpoint for version checking to use new repository location
- Corrected all documentation references to point to RayLabsHQ organization
### Security
- Removed test security script after confirming vulnerability resolution
- Updated base Docker image to version 1.2.18-alpine
### Documentation
- Added repository migration notice in README
- Updated quickstart guide with new repository URLs
- Updated LXC deployment documentation with new repository location
## [2.18.0] - 2025-06-24
### Added
- Fourth organization strategy "Mixed Mode" that combines aspects of existing strategies
- Personal repositories go to a single configurable organization
- Organization repositories preserve their GitHub organization structure
- "Override Options" info button in Organization Strategy component explaining customization features
- Organization overrides via edit buttons on organization cards
- Repository overrides via inline destination editor
- Starred repositories behavior and priority hierarchy
### Improved
- Simplified mixed strategy implementation to reuse existing database fields
- Enhanced organization strategy UI with comprehensive override documentation
- Better visual indicators for the new mixed strategy with orange color theme
## [2.17.0] - 2025-06-24
### Added
- Custom destination control for individual repositories with inline editing
- Organization-level destination overrides with visual destination editor
- Personal repositories organization override configuration option
- Visual indicators for starred repositories (⭐ icon) in repository list
- Repository-level destination override API endpoint
- Destination customization priority hierarchy system
- "View on Gitea" buttons for organizations with smart tooltip states
### Changed
- Enhanced repository table with destination column showing both GitHub org and Gitea destination
- Updated organization cards to display custom destinations with visual indicators
- Improved getGiteaRepoOwnerAsync to support repository-level destination overrides
### Improved
- Better visual feedback for custom destinations with badges and inline editing
- Enhanced user experience with hover-based edit buttons
- Comprehensive destination customization documentation in README
## [2.16.3] - 2025-06-20
### Added
- Custom 404 error page with helpful navigation links
- HoverCard components for better UX in configuration forms
### Improved
- Replaced popover components with hover cards for information tooltips
- Enhanced user experience with responsive hover interactions
## [2.16.2] - 2025-06-17
### Added
- Bulk actions for repository management with selection support
### Improved
- Enhanced organization card display with status badges and improved layout
## [2.16.1] - 2025-06-17
### Improved
- Improved repository owner handling and mirror strategy in Gitea integration
- Updated label for starred repositories organization for consistency
## [2.16.0] - 2025-06-17
### Added
- Enhanced OrganizationConfiguration component with improved layout and metadata options
- New GitHubMirrorSettings component with better organization and flexibility
- Enhanced starred repositories content selection and improved layout
### Improved
- Enhanced configuration interface layout and spacing across multiple components
- Streamlined OrganizationStrategy component with cleaner imports and better organization
- Improved responsive layout for larger screens in configuration forms
- Better icon usage and clarity in configuration components
- Enhanced tooltip descriptions and component organization
- Improved version comparison logic in health API
- Enhanced issue mirroring logic for starred repositories
### Fixed
- Fixed mirror to single organization functionality
- Resolved organization strategy layout issues
- Cleaned up unused imports across multiple components
### Refactored
- Simplified component structures by removing unused imports and dependencies
- Enhanced layout flexibility in GitHubConfigForm and GiteaConfigForm components
- Improved component organization and code clarity
- Removed ConnectionsForm and useMirror hook for better code organization
## [2.14.0] - 2025-06-17
### Added
- Enhanced UI components with @radix-ui/react-accordion dependency for improved configuration interface
### Fixed
- Mirror strategies now properly route repositories based on selected strategy
- Starred repositories now correctly go to the designated starred repos organization
- Organization routing for single-org and flat-user strategies
### Improved
- Documentation now explains all three mirror strategies (preserve, single-org, flat-user)
- Added detailed mirror strategy configuration guide
- Updated CLAUDE.md with mirror strategy architecture information
- Enhanced Docker Compose development configuration
## [2.13.2] - 2025-06-15
### Improved
- Enhanced documentation design and layout
- Updated README with improved formatting and content
## [2.13.1] - 2025-06-15
### Added
- Docker Hub authentication for Docker Scout security scanning
- Comprehensive Docker workflow consolidation with build, push & security scan
### Improved
- Enhanced CI/CD pipeline reliability with better error handling
- Updated Bun base image to latest version for improved security
- Migrated from Trivy to Docker Scout for more comprehensive security scanning
- Enhanced Docker workflow with wait steps for image availability
### Fixed
- Docker Scout action integration issues and image reference problems
- Workflow reliability improvements with proper error handling
- Security scanning workflow now continues on security issues without failing the build
### Changed
- Updated package dependencies to latest versions
- Consolidated multiple Docker workflows into single comprehensive workflow
- Enhanced security scanning with Docker Scout integration
## [2.13.0] - 2025-06-15
### Added
- Enhanced Configuration Interface with collapsible components and improved organization strategy UI
- Wiki Mirroring Support in configuration settings
- Auto-Save Functionality for all config forms, eliminating manual save buttons
- Live Refresh functionality with configuration status hooks and enhanced UI components
- Enhanced API Config Handling with mapping functions for UI and database structures
- Secure Error Responses with createSecureErrorResponse for consistent error handling
- Automatic Database Cleanup feature with configuration options and API support
- Enhanced Job Recovery with improved database schema and recovery mechanisms
- Fork tags to repository UI and enhanced organization cards with repository breakdown
- Skeleton loaders and better loading state management across the application
### Improved
- Navigation context and component loading states across the application
- Card components alignment and styling consistency
- Error logging and structured error message parsing
- HTTP client standardization across the application
- Database initialization and management processes
- Visual consistency with updated icons and custom logo integration
### Fixed
- Repository mirroring status inconsistencies
- Organizations getting stuck on mirroring status when empty
- JSON parsing errors and improved error handling
- Broken documentation links in README
- Various UI contrast and alignment issues
### Changed
- Migrated testing framework to Bun and updated test configurations
- Implemented graceful shutdown and enhanced job recovery capabilities
- Replaced SiGitea icons with custom logo
- Updated various dependencies for improved stability and performance
## [2.12.0] - 2025-01-27
### Fixed
- Fixed SQLite "no such table: mirror_jobs" error during application startup
- Implemented automatic database table creation during database initialization
- Resolved database schema inconsistencies between development and production environments
### Improved
- Enhanced database initialization process with automatic table creation and indexing
- Added comprehensive error handling for database table creation
- Integrated database repair functionality into application startup for better reliability
## [2.5.3] - 2025-05-22
### Added
- Enhanced JWT_SECRET handling with auto-generation and persistence for improved security
- Updated Proxmox LXC deployment instructions and replaced deprecated script
## [2.5.2] - 2024-11-22
### Fixed
- Fixed version information in health API for Docker deployments by setting npm_package_version environment variable in entrypoint script
## [2.5.1] - 2024-10-01
### Fixed
- Fixed Docker entrypoint script to prevent unnecessary `bun install` on container startup
- Removed redundant dependency installation in Docker containers for pre-built images
- Fixed "PathAlreadyExists" errors during container initialization
### Changed
- Improved database initialization in Docker entrypoint script
- Added additional checks for TypeScript versions of database management scripts
## [2.5.0] - 2024-09-15
Initial public release with core functionality:
### Added
- GitHub to Gitea repository mirroring
- User authentication and management
- Dashboard with mirroring statistics
- Configuration management for mirroring settings
- Support for organization mirroring
- Automated mirroring with configurable schedules
- Docker multi-architecture support (amd64, arm64)
- LXC container deployment scripts

View File

@@ -0,0 +1,317 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
Gitea Mirror is a self-hosted web application that automatically mirrors repositories from GitHub to Gitea instances. It's built with Astro (SSR mode), React, and runs on the Bun runtime with SQLite for data persistence.
**Key capabilities:**
- Mirrors public, private, and starred GitHub repos to Gitea
- Supports metadata mirroring (issues, PRs as issues, labels, milestones, releases, wiki)
- Git LFS support
- Multiple authentication methods (email/password, OIDC/SSO, header auth)
- Scheduled automatic syncing with configurable intervals
- Auto-discovery of new repos and cleanup of deleted repos
- Multi-user support with encrypted token storage (AES-256-GCM)
## Development Commands
### Setup and Installation
```bash
# Install dependencies
bun install
# Initialize database (first time setup)
bun run setup
# Clean start (reset database)
bun run dev:clean
```
### Development
```bash
# Start development server (http://localhost:4321)
bun run dev
# Build for production
bun run build
# Preview production build
bun run preview
# Start production server
bun run start
```
### Testing
```bash
# Run all tests
bun test
# Run tests in watch mode
bun test:watch
# Run tests with coverage
bun test:coverage
```
**Test configuration:**
- Test runner: Bun's built-in test runner (configured in `bunfig.toml`)
- Setup file: `src/tests/setup.bun.ts` (auto-loaded via bunfig.toml)
- Timeout: 5000ms default
- Tests are colocated with source files using `*.test.ts` pattern
### Database Management
```bash
# Database operations via Drizzle
bun run db:generate # Generate migrations from schema
bun run db:migrate # Run migrations
bun run db:push # Push schema changes directly
bun run db:studio # Open Drizzle Studio (database GUI)
bun run db:check # Check schema consistency
# Database utilities via custom scripts
bun run manage-db init # Initialize database
bun run manage-db check # Check database health
bun run manage-db fix # Fix database issues
bun run manage-db reset-users # Reset all users
bun run cleanup-db # Delete database file
```
### Utility Scripts
```bash
# Recovery and diagnostic scripts
bun run startup-recovery # Recover from crashes
bun run startup-recovery-force # Force recovery
bun run test-recovery # Test recovery mechanism
bun run test-shutdown # Test graceful shutdown
# Environment configuration
bun run startup-env-config # Load config from env vars
```
## Architecture
### Tech Stack
- **Frontend:** Astro v5 (SSR mode) + React v19 + Shadcn UI + Tailwind CSS v4
- **Backend:** Astro API routes (Node adapter, standalone mode)
- **Runtime:** Bun (>=1.2.9)
- **Database:** SQLite via Drizzle ORM
- **Authentication:** Better Auth (session-based)
- **APIs:** GitHub (Octokit with throttling plugin), Gitea REST API
### Directory Structure
```
src/
├── components/ # React components (UI, features)
│ ├── ui/ # Shadcn UI components
│ ├── repositories/ # Repository management components
│ ├── organizations/ # Organization management components
│ └── ...
├── pages/ # Astro pages and API routes
│ ├── api/ # API endpoints (Better Auth integration)
│ │ ├── auth/ # Authentication endpoints
│ │ ├── github/ # GitHub operations
│ │ ├── gitea/ # Gitea operations
│ │ ├── sync/ # Mirror sync operations
│ │ ├── job/ # Job management
│ │ └── ...
│ └── *.astro # Page components
├── lib/ # Core business logic
│ ├── db/ # Database (Drizzle ORM)
│ │ ├── schema.ts # Database schema with Zod validation
│ │ ├── index.ts # Database instance and table exports
│ │ └── adapter.ts # Better Auth SQLite adapter
│ ├── github.ts # GitHub API client (Octokit)
│ ├── gitea.ts # Gitea API client
│ ├── gitea-enhanced.ts # Enhanced Gitea operations (metadata)
│ ├── scheduler-service.ts # Automatic mirroring scheduler
│ ├── cleanup-service.ts # Activity log cleanup
│ ├── repository-cleanup-service.ts # Orphaned repo cleanup
│ ├── auth.ts # Better Auth configuration
│ ├── config.ts # Configuration management
│ ├── helpers.ts # Mirror job creation
│ ├── utils/ # Utility functions
│ │ ├── encryption.ts # AES-256-GCM token encryption
│ │ ├── config-encryption.ts # Config token encryption
│ │ ├── duration-parser.ts # Parse intervals (e.g., "8h", "30m")
│ │ ├── concurrency.ts # Concurrency control utilities
│ │ └── mirror-strategies.ts # Mirror strategy logic
│ └── ...
├── types/ # TypeScript type definitions
├── tests/ # Test utilities and setup
└── middleware.ts # Astro middleware (auth, session)
scripts/ # Utility scripts
├── manage-db.ts # Database management CLI
├── startup-recovery.ts # Crash recovery
└── ...
```
### Key Architectural Patterns
#### 1. Database Schema and Validation
- **Location:** `src/lib/db/schema.ts`
- **Pattern:** Drizzle ORM tables + Zod schemas for validation
- **Key tables:**
- `configs` - User configuration (GitHub/Gitea settings, mirror options)
- `repositories` - Tracked repositories with metadata
- `organizations` - GitHub organizations with destination overrides
- `mirrorJobs` - Mirror job queue and history
- `activities` - Activity log for dashboard
- `user`, `session`, `account` - Better Auth tables
**Important:** All config tokens (GitHub/Gitea) are encrypted at rest using AES-256-GCM. Use helper functions from `src/lib/utils/config-encryption.ts` to decrypt.
#### 2. Mirror Job System
- **Location:** `src/lib/helpers.ts` (createMirrorJob)
- **Flow:**
1. User triggers mirror via API endpoint
2. `createMirrorJob()` creates job record with status "pending"
3. Job processor (in API routes) performs GitHub → Gitea operations
4. Job status updated throughout: "mirroring" → "success"/"failed"
5. Events published via SSE for real-time UI updates
#### 3. GitHub ↔ Gitea Mirroring
- **GitHub Client:** `src/lib/github.ts` - Octokit with rate limit tracking
- **Gitea Client:** `src/lib/gitea.ts` - Basic repo operations
- **Enhanced Gitea:** `src/lib/gitea-enhanced.ts` - Metadata mirroring (issues, PRs, releases)
**Mirror strategies (configured per user):**
- `preserve` - Maintain GitHub org structure in Gitea
- `single-org` - All repos into one Gitea org
- `flat-user` - All repos under user account
- `mixed` - Personal repos in one org, org repos preserve structure
**Metadata mirroring:**
- Issues transferred with comments, labels, assignees
- PRs converted to issues (Gitea API limitation - cannot create PRs)
- Tagged with "pull-request" label
- Title prefixed with `[PR #number] [STATUS]`
- Body includes commit history, file changes, merge status
- Releases mirrored with assets
- Labels and milestones preserved
- Wiki content cloned if enabled
- **Sequential processing:** Issues/PRs mirrored one at a time to prevent out-of-order creation (see `src/lib/gitea-enhanced.ts`)
#### 4. Scheduler Service
- **Location:** `src/lib/scheduler-service.ts`
- **Features:**
- Cron-based or interval-based scheduling (uses `duration-parser.ts`)
- Auto-start on boot when `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` is set
- Auto-import new GitHub repos
- Auto-cleanup orphaned repos (archive or delete)
- Respects per-repo mirror intervals (not Gitea's default 24h)
- **Concurrency control:** Uses `src/lib/utils/concurrency.ts` for batch processing
#### 5. Authentication System
- **Location:** `src/lib/auth.ts`, `src/lib/auth-client.ts`
- **Better Auth integration:**
- Email/password (always enabled)
- OIDC/SSO providers (configurable via UI)
- Header authentication for reverse proxies (Authentik, Authelia)
- **Session management:** Cookie-based, validated in Astro middleware
- **User helpers:** `src/lib/utils/auth-helpers.ts`
#### 6. Environment Configuration
- **Startup:** `src/lib/env-config-loader.ts` + `scripts/startup-env-config.ts`
- **Pattern:** Environment variables can pre-configure settings, but users can override via web UI
- **Encryption:** `ENCRYPTION_SECRET` for tokens, `BETTER_AUTH_SECRET` for sessions
#### 7. Real-time Updates
- **Events:** `src/lib/events.ts` + `src/lib/events/realtime.ts`
- **Pattern:** Server-Sent Events (SSE) for live dashboard updates
- **Endpoints:** `/api/sse` - client subscribes to job/repo events
### Testing Patterns
**Unit tests:**
- Colocated with source: `filename.test.ts` alongside `filename.ts`
- Use Bun's built-in assertions and mocking
- Mock external APIs (GitHub, Gitea) using `src/tests/mock-fetch.ts`
**Integration tests:**
- Located in `src/tests/`
- Test database operations with in-memory SQLite
- Example: `src/lib/db/index.test.ts`
**Test utilities:**
- `src/tests/setup.bun.ts` - Global test setup (loaded via bunfig.toml)
- `src/tests/mock-fetch.ts` - Fetch mocking utilities
### Important Development Notes
1. **Path Aliases:** Use `@/` for imports (configured in `tsconfig.json`)
```typescript
import { db } from '@/lib/db';
```
2. **Token Encryption:** Always use encryption helpers when dealing with tokens:
```typescript
import { getDecryptedGitHubToken, getDecryptedGiteaToken } from '@/lib/utils/config-encryption';
```
3. **API Route Pattern:** Astro API routes in `src/pages/api/` should:
- Check authentication via Better Auth
- Validate input with Zod schemas
- Handle errors gracefully
- Return JSON responses
4. **Database Migrations:**
- Schema changes: Update `src/lib/db/schema.ts`
- Generate migration: `bun run db:generate`
- Review generated SQL in `drizzle/` directory
- Apply: `bun run db:migrate` (or `db:push` for dev)
5. **Concurrency Control:**
- Use utilities from `src/lib/utils/concurrency.ts` for batch operations
- Respect rate limits (GitHub: 5000 req/hr authenticated, Gitea: varies)
- Issue/PR mirroring is sequential to maintain chronological order
6. **Duration Parsing:**
- Use `parseInterval()` from `src/lib/utils/duration-parser.ts`
- Supports: "30m", "8h", "24h", "7d", cron expressions, or milliseconds
7. **Graceful Shutdown:**
- Services implement cleanup handlers (see `src/lib/shutdown-manager.ts`)
- Recovery system in `src/lib/recovery.ts` handles interrupted jobs
## Common Development Workflows
### Adding a new mirror option
1. Update Zod schema in `src/lib/db/schema.ts` (e.g., `giteaConfigSchema`)
2. Update TypeScript types in `src/types/config.ts`
3. Add UI control in settings page component
4. Update API handler in `src/pages/api/config/`
5. Implement logic in `src/lib/gitea.ts` or `src/lib/gitea-enhanced.ts`
### Debugging mirror failures
1. Check mirror jobs: `bun run db:studio` → `mirrorJobs` table
2. Review activity logs: Dashboard → Activity tab
3. Check console logs for API errors (GitHub/Gitea rate limits, auth issues)
4. Use diagnostic scripts: `bun run test-recovery`
### Adding authentication provider
1. Update Better Auth config in `src/lib/auth.ts`
2. Add provider configuration UI in settings
3. Test with `src/tests/test-gitea-auth.ts` patterns
4. Update documentation in `docs/SSO-OIDC-SETUP.md`
## Docker Deployment
- **Dockerfile:** Multi-stage build (bun base → build → production)
- **Entrypoint:** `docker-entrypoint.sh` - handles CA certs, user permissions, database init
- **Compose files:**
- `docker-compose.alt.yml` - Quick start (pre-built image, minimal config)
- `docker-compose.yml` - Full setup (build from source, all env vars)
- `docker-compose.dev.yml` - Development with hot reload
## Additional Resources
- **Environment Variables:** See `docs/ENVIRONMENT_VARIABLES.md` for complete list
- **Development Workflow:** See `docs/DEVELOPMENT_WORKFLOW.md`
- **SSO Setup:** See `docs/SSO-OIDC-SETUP.md`
- **Contributing:** See `CONTRIBUTING.md` for code guidelines and scope
- **Graceful Shutdown:** See `docs/GRACEFUL_SHUTDOWN.md` for crash recovery details

View File

@@ -0,0 +1,182 @@
# Contributing to Gitea Mirror
Thank you for your interest in contributing to Gitea Mirror! This document provides guidelines and instructions for contributing to the open-source version of the project.
## 🎯 Project Overview
Gitea Mirror is an open-source, self-hosted solution for mirroring GitHub repositories to Gitea instances. This guide provides everything you need to know about contributing to the project.
## 🚀 Getting Started
1. Fork the repository
2. Clone your fork:
```bash
git clone https://github.com/yourusername/gitea-mirror.git
cd gitea-mirror
```
3. Install dependencies:
```bash
bun install
```
4. Set up your environment:
```bash
cp .env.example .env
# Edit .env with your configuration
```
5. Start development:
```bash
bun run dev
```
## 🛠 Development Workflow
### Running the Application
```bash
# Development mode
bun run dev
# Build for production
bun run build
# Run tests
bun test
```
### Database Management
```bash
# Initialize database
bun run init-db
# Reset database
bun run cleanup-db && bun run init-db
```
## 📝 Code Guidelines
### General Principles
1. **Keep it Simple**: Gitea Mirror should remain easy to self-host
2. **Focus on Core Features**: Prioritize repository mirroring and synchronization
3. **Database**: Use SQLite for simplicity and portability
4. **Dependencies**: Minimize external dependencies for easier deployment
### Code Style
- Use TypeScript for all new code
- Follow the existing code formatting (Prettier is configured)
- Write meaningful commit messages
- Add tests for new features
### Scope of Contributions
This project focuses on personal/small team use cases. Please keep contributions aligned with:
- Core mirroring functionality
- Self-hosted simplicity
- Minimal external dependencies
- SQLite as the database
- Single-instance deployments
## 🐛 Reporting Issues
1. Check existing issues first
2. Use issue templates when available
3. Provide clear reproduction steps
4. Include relevant logs and screenshots
## 🎯 Pull Request Process
1. Create a feature branch:
```bash
git checkout -b feature/your-feature-name
```
2. Make your changes following the code guidelines
3. Test your changes:
```bash
# Run tests
bun test
# Build and check
bun run build:oss
```
4. Commit your changes:
```bash
git commit -m "feat: add new feature"
```
5. Push to your fork and create a Pull Request
### PR Requirements
- Clear description of changes
- Tests for new functionality
- Documentation updates if needed
- No breaking changes without discussion
- Passes all CI checks
## 🏗 Architecture Overview
```
src/
├── components/ # React components
├── lib/ # Core utilities
│ ├── db/ # Database queries (SQLite only)
│ ├── github/ # GitHub API integration
│ ├── gitea/ # Gitea API integration
│ └── utils/ # Helper functions
├── pages/ # Astro pages
│ └── api/ # API endpoints
└── types/ # TypeScript types
```
## 🧪 Testing
```bash
# Run all tests
bun test
# Run tests in watch mode
bun test:watch
# Run with coverage
bun test:coverage
```
## 📚 Documentation
- Update README.md for user-facing changes
- Add JSDoc comments for new functions
- Update .env.example for new environment variables
## 💡 Feature Requests
We welcome feature requests! When proposing new features, please consider:
- Does it enhance the core mirroring functionality?
- Will it benefit self-hosted users?
- Can it be implemented without complex external dependencies?
- Does it maintain the project's simplicity?
## 🤝 Community
- Be respectful and constructive
- Help others in issues and discussions
- Share your use cases and feedback
## 📄 License
By contributing, you agree that your contributions will be licensed under the same license as the project (MIT).
## Questions?
Feel free to open an issue for any questions about contributing!
---
Thank you for helping make Gitea Mirror better! 🎉

View File

@@ -0,0 +1,169 @@
# Nix Distribution - Ready to Use!
## Current Status: WORKS NOW
Your Nix package is **already distributable**! Users can run it directly from GitHub without any additional setup on your end.
## How Users Will Use It
### Simple: Just Run From GitHub
```bash
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
```
That's it! No releases, no CI, no infrastructure needed. It works right now.
---
## What Happens When They Run This?
1. **Nix fetches** your repo from GitHub
2. **Nix reads** `flake.nix` and `flake.lock`
3. **Nix builds** the package on their machine
4. **Nix runs** the application
5. **Result cached** in `/nix/store` for reuse
---
## Do You Need CI or Releases?
### For Basic Usage: **NO**
Users can already use it from GitHub. No CI or releases required.
### For CI Validation: **Already Set Up**
GitHub Actions validates builds on every push with Magic Nix Cache (free, no setup).
---
## Next Steps (Optional)
### Option 1: Release Versioning (2 minutes)
**Why:** Users can pin to specific versions
**How:**
```bash
# When ready to release
git tag v3.8.11
git push origin v3.8.11
# Users can then pin to this version
nix run github:RayLabsHQ/gitea-mirror/v3.8.11
```
No additional CI needed - tags work automatically with flakes!
### Option 2: Submit to nixpkgs (Long Term)
**Why:** Maximum discoverability and trust
**When:** After package is stable and well-tested
**How:** Submit PR to https://github.com/NixOS/nixpkgs
---
## Files Created
### Essential (Already Working)
- `flake.nix` - Package definition
- `flake.lock` - Dependency lock file
- `.envrc` - direnv integration
### Documentation
- `NIX.md` - Quick reference for users
- `docs/NIX_DEPLOYMENT.md` - Complete deployment guide
- `docs/NIX_DISTRIBUTION.md` - Distribution guide for you (maintainer)
- `README.md` - Updated with Nix instructions
### CI (Already Set Up)
- `.github/workflows/nix-build.yml` - Builds and validates on Linux + macOS
### Updated
- `.gitignore` - Added Nix artifacts
---
## Comparison: Your Distribution Options
| Setup | Time | User Experience | What You Need |
|-------|------|----------------|---------------|
| **Direct GitHub** | 0 min | Slow (build from source) | Nothing! Works now |
| **+ Git Tags** | 2 min | Versionable | Just push tags |
| **+ nixpkgs** | Hours | Official/Trusted | PR review process |
**Recommendation:** Direct GitHub works now. Add git tags for versioning. Consider nixpkgs submission once stable.
---
## Testing Your Distribution
You can test it right now:
```bash
# Test direct GitHub usage
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Test with specific commit
nix run github:RayLabsHQ/gitea-mirror/$(git rev-parse HEAD)
# Validate flake
nix flake check
```
---
## User Documentation Locations
Users will find instructions in:
1. **README.md** - Installation section (already updated)
2. **NIX.md** - Quick reference
3. **docs/NIX_DEPLOYMENT.md** - Detailed guide
All docs include the correct commands with experimental features flags.
---
## When to Release New Versions
### For Git Tag Releases:
```bash
# 1. Update version in package.json
vim package.json
# 2. Update version in flake.nix (line 17)
vim flake.nix # version = "3.8.12";
# 3. Commit and tag
git add package.json flake.nix
git commit -m "chore: bump version to v3.8.12"
git tag v3.8.12
git push origin main
git push origin v3.8.12
```
Users can then use: `nix run github:RayLabsHQ/gitea-mirror/v3.8.12`
### No Release Needed For:
- Bug fixes
- Small changes
- Continuous updates
Users can always use latest from main: `nix run github:RayLabsHQ/gitea-mirror`
---
## Summary
**Ready to distribute RIGHT NOW**
- Just commit and push your `flake.nix`
- Users can run directly from GitHub
- CI validates builds automatically
**Optional: Submit to nixpkgs**
- Maximum discoverability
- Official Nix repository
- Do this once package is stable
See `docs/NIX_DISTRIBUTION.md` for complete details!

View File

@@ -0,0 +1,60 @@
# syntax=docker/dockerfile:1.4
FROM oven/bun:1.3.3-debian AS base
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 make g++ gcc wget sqlite3 openssl ca-certificates \
&& rm -rf /var/lib/apt/lists/*
# ----------------------------
FROM base AS deps
COPY package.json ./
COPY bun.lock* ./
RUN bun install --frozen-lockfile
# ----------------------------
FROM deps AS builder
COPY . .
RUN bun run build
RUN mkdir -p dist/scripts && \
for script in scripts/*.ts; do \
bun build "$script" --target=bun --outfile=dist/scripts/$(basename "${script%.ts}.js"); \
done
# ----------------------------
FROM deps AS pruner
RUN bun install --production --frozen-lockfile
# ----------------------------
FROM base AS runner
WORKDIR /app
COPY --from=pruner /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/docker-entrypoint.sh ./docker-entrypoint.sh
COPY --from=builder /app/scripts ./scripts
COPY --from=builder /app/drizzle ./drizzle
ENV NODE_ENV=production
ENV HOST=0.0.0.0
ENV PORT=4321
ENV DATABASE_URL=file:data/gitea-mirror.db
# Create directories and setup permissions
RUN mkdir -p /app/certs && \
chmod +x ./docker-entrypoint.sh && \
mkdir -p /app/data && \
addgroup --system --gid 1001 nodejs && \
adduser --system --uid 1001 gitea-mirror && \
chown -R gitea-mirror:nodejs /app/data && \
chown -R gitea-mirror:nodejs /app/certs
USER gitea-mirror
VOLUME /app/data
EXPOSE 4321
HEALTHCHECK --interval=30s --timeout=5s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:4321/api/health || exit 1
ENTRYPOINT ["./docker-entrypoint.sh"]

619
Divers/gitea-mirror/LICENSE Normal file
View File

@@ -0,0 +1,619 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS

189
Divers/gitea-mirror/NIX.md Normal file
View File

@@ -0,0 +1,189 @@
# Nix Deployment Quick Reference
## TL;DR
```bash
# From GitHub (no clone needed!)
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Or from local clone
nix run --extra-experimental-features 'nix-command flakes' .#gitea-mirror
```
Secrets auto-generate, database auto-initializes, and the web UI starts at http://localhost:4321.
**Note:** If you have flakes enabled in your nix config, you can omit `--extra-experimental-features 'nix-command flakes'`
---
## Installation Options
### 1. Run Without Installing (from GitHub)
```bash
# Latest version from main branch
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Pin to specific version
nix run github:RayLabsHQ/gitea-mirror/v3.8.11
```
### 2. Install to Profile
```bash
# Install from GitHub
nix profile install --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Run the installed binary
gitea-mirror
```
### 3. Use Local Clone
```bash
# Clone and run
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
nix run --extra-experimental-features 'nix-command flakes' .#gitea-mirror
```
### 4. NixOS System Service
```nix
# configuration.nix
{
inputs.gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
services.gitea-mirror = {
enable = true;
betterAuthUrl = "https://mirror.example.com"; # For production
openFirewall = true;
};
}
```
### 5. Development (Local Clone)
```bash
nix develop --extra-experimental-features 'nix-command flakes'
# or
direnv allow # Handles experimental features automatically
```
---
## Enable Flakes Permanently (Recommended)
To avoid typing `--extra-experimental-features` every time, add to `~/.config/nix/nix.conf`:
```
experimental-features = nix-command flakes
```
---
## What Gets Auto-Generated?
On first run, the wrapper automatically:
1. Creates `~/.local/share/gitea-mirror/` (or `$DATA_DIR`)
2. Generates `BETTER_AUTH_SECRET``.better_auth_secret`
3. Generates `ENCRYPTION_SECRET``.encryption_secret`
4. Initializes SQLite database
5. Runs startup recovery and repair scripts
6. Starts the application
---
## Key Commands
```bash
# Database management
gitea-mirror-db init # Initialize database
gitea-mirror-db check # Health check
gitea-mirror-db fix # Fix issues
# Development (add --extra-experimental-features 'nix-command flakes' if needed)
nix develop # Enter dev shell
nix build # Build package
nix flake check # Validate flake
nix flake update # Update dependencies
```
---
## Environment Variables
All vars from `docker-compose.alt.yml` are supported:
```bash
DATA_DIR="$HOME/.local/share/gitea-mirror"
PORT=4321
HOST="0.0.0.0"
BETTER_AUTH_URL="http://localhost:4321"
# Secrets (auto-generated if not set)
BETTER_AUTH_SECRET=auto-generated
ENCRYPTION_SECRET=auto-generated
# Concurrency (for perfect ordering, set both to 1)
MIRROR_ISSUE_CONCURRENCY=3
MIRROR_PULL_REQUEST_CONCURRENCY=5
```
---
## NixOS Module Options
```nix
services.gitea-mirror = {
enable = true;
package = ...; # Override package
dataDir = "/var/lib/gitea-mirror"; # Data location
user = "gitea-mirror"; # Service user
group = "gitea-mirror"; # Service group
host = "0.0.0.0"; # Bind address
port = 4321; # Listen port
betterAuthUrl = "http://..."; # External URL
betterAuthTrustedOrigins = "..."; # CORS origins
mirrorIssueConcurrency = 3; # Concurrency
mirrorPullRequestConcurrency = 5; # Concurrency
environmentFile = null; # Optional secrets file
openFirewall = true; # Open firewall
};
```
---
## Comparison: Docker vs Nix
| Feature | Docker | Nix |
|---------|--------|-----|
| **Config Required** | BETTER_AUTH_SECRET | None (auto-generated) |
| **Startup** | `docker-compose up` | `nix run .#gitea-mirror` |
| **Service** | Docker daemon | systemd (NixOS) |
| **Updates** | `docker pull` | `nix flake update` |
| **Reproducible** | Image-based | Hash-based |
---
## Full Documentation
- **[docs/NIX_DEPLOYMENT.md](docs/NIX_DEPLOYMENT.md)** - Complete deployment guide
- NixOS module configuration
- Home Manager integration
- Production deployment examples
- Migration from Docker
- Troubleshooting guide
- **[docs/NIX_DISTRIBUTION.md](docs/NIX_DISTRIBUTION.md)** - Distribution guide for maintainers
- How users consume the package
- CI build caching
- Releasing new versions
- Submitting to nixpkgs
---
## Key Features
- **Zero-config deployment** - Runs immediately without setup
- **Auto-secret generation** - Secure secrets created and persisted
- **Startup recovery** - Handles interrupted jobs automatically
- **Graceful shutdown** - Proper signal handling
- **Health checks** - Built-in monitoring support
- **Security hardening** - NixOS module includes systemd protections
- **Docker parity** - Same behavior as `docker-compose.alt.yml`

View File

@@ -0,0 +1,454 @@
<p align="center">
<img src=".github/assets/logo.png" alt="Gitea Mirror Logo" width="120" />
<h1>Gitea Mirror</h1>
<p><i>Automatically mirror repositories from GitHub to your self-hosted Gitea instance.</i></p>
<p align="center">
<a href="https://github.com/RayLabsHQ/gitea-mirror/releases/latest"><img src="https://img.shields.io/github/v/tag/RayLabsHQ/gitea-mirror?label=release" alt="release"/></a>
<a href="https://github.com/RayLabsHQ/gitea-mirror/actions/workflows/astro-build-test.yml"><img src="https://img.shields.io/github/actions/workflow/status/RayLabsHQ/gitea-mirror/astro-build-test.yml?branch=main" alt="build"/></a>
<a href="https://github.com/RayLabsHQ/gitea-mirror/pkgs/container/gitea-mirror"><img src="https://img.shields.io/badge/ghcr.io-container-blue?logo=github" alt="container"/></a>
<a href="https://github.com/RayLabsHQ/gitea-mirror/blob/main/LICENSE"><img src="https://img.shields.io/github/license/RayLabsHQ/gitea-mirror" alt="license"/></a>
</p>
</p>
## 🚀 Quick Start
```bash
# Fastest way - using the simplified Docker setup
docker compose -f docker-compose.alt.yml up -d
# Access at http://localhost:4321
```
First user signup becomes admin. Configure GitHub and Gitea through the web interface!
<p align="center">
<img src=".github/assets/dashboard.png" alt="Dashboard" width="600" />
<img src=".github/assets/dashboard_mobile.png" alt="Dashboard Mobile" width="200" />
</p>
## ✨ Features
- 🔁 Mirror public, private, and starred GitHub repos to Gitea
- 🏢 Mirror entire organizations with flexible strategies
- 🎯 Custom destination control for repos and organizations
- 📦 **Git LFS support** - Mirror large files with Git LFS
- 📝 **Metadata mirroring** - Issues, pull requests (as issues), labels, milestones, wiki
- 🚫 **Repository ignore** - Mark specific repos to skip
- 🔐 Secure authentication with Better Auth (email/password, SSO, OIDC)
- 📊 Real-time dashboard with activity logs
- ⏱️ Scheduled automatic mirroring with configurable intervals
- 🔄 **Auto-discovery** - Automatically import new GitHub repositories (v3.4.0+)
- 🧹 **Repository cleanup** - Auto-remove repos deleted from GitHub (v3.4.0+)
- 🎯 **Proper mirror intervals** - Respects configured sync intervals (v3.4.0+)
- 🗑️ Automatic database cleanup with configurable retention
- 🐳 Dockerized with multi-arch support (AMD64/ARM64)
## 📸 Screenshots
<div align="center">
<img src=".github/assets/repositories.png" alt="Repositories" width="600" />
<img src=".github/assets/repositories_mobile.png" alt="Rrepositories Mobile" width="200" />
</div>
<div align="center">
<img src=".github/assets/organisation.png" alt="Organisations" width="600" />
<img src=".github/assets/organisation_mobile.png" alt="Organisations Mobile" width="200" />
</div>
## Installation
### Docker (Recommended)
We provide two Docker Compose options:
#### Option 1: Quick Start (docker-compose.alt.yml)
Perfect for trying out Gitea Mirror or simple deployments:
```bash
# Clone repository
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
# Start with simplified setup
docker compose -f docker-compose.alt.yml up -d
# Access at http://localhost:4321
```
**Features:**
- ✅ Pre-built image - no building required
- ✅ Minimal configuration needed
- ✅ Data stored in `./data` directory
- ✅ Configure everything through web UI
- ✅ Automatic user/group ID mapping
**Best for:**
- First-time users
- Testing and evaluation
- Simple deployments
- When you prefer web-based configuration
#### Option 2: Full Setup (docker-compose.yml)
For production deployments with environment-based configuration:
```bash
# Start with full configuration options
docker compose up -d
```
**Features:**
- ✅ Build from source or use pre-built image
- ✅ Complete environment variable configuration
- ✅ Support for custom CA certificates
- ✅ Advanced mirror settings (forks, wiki, issues)
- ✅ Multi-registry support
**Best for:**
- Production deployments
- Automated/scripted setups
- Advanced mirror configurations
- When using self-signed certificates
#### Using Pre-built Image Directly
```bash
docker pull ghcr.io/raylabshq/gitea-mirror:v3.1.1
```
### Configuration Options
#### Quick Start Configuration (docker-compose.alt.yml)
Minimal `.env` file (optional - has sensible defaults):
```bash
# Custom port (default: 4321)
PORT=4321
# User/Group IDs for file permissions (default: 1000)
PUID=1000
PGID=1000
# Session secret (auto-generated if not set)
BETTER_AUTH_SECRET=your-secret-key-change-this-in-production
```
All other settings are configured through the web interface after starting.
#### Full Setup Configuration (docker-compose.yml)
Supports extensive environment variables for automated deployment. See the full [docker-compose.yml](docker-compose.yml) for all available options including GitHub tokens, Gitea URLs, mirror settings, and more.
📚 **For a complete list of all supported environment variables, see the [Environment Variables Documentation](docs/ENVIRONMENT_VARIABLES.md).**
### LXC Container (Proxmox)
```bash
# One-line install on Proxmox VE
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/gitea-mirror.sh)"
```
See the [Proxmox VE Community Scripts](https://community-scripts.github.io/ProxmoxVE/scripts?id=gitea-mirror) for more details.
### Nix/NixOS
Zero-configuration deployment with Nix:
```bash
# Run immediately - no setup needed!
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Or build and run locally
nix build --extra-experimental-features 'nix-command flakes'
./result/bin/gitea-mirror
# Or install to profile
nix profile install --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
gitea-mirror
```
**NixOS users** - add to your configuration:
```nix
{
inputs.gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
services.gitea-mirror = {
enable = true;
betterAuthUrl = "https://mirror.example.com";
openFirewall = true;
};
}
```
Secrets auto-generate, database auto-initializes. See [NIX.md](NIX.md) for quick reference or [docs/NIX_DEPLOYMENT.md](docs/NIX_DEPLOYMENT.md) for full documentation.
### Manual Installation
```bash
# Install Bun
curl -fsSL https://bun.sh/install | bash
# Setup and run
bun run setup
bun run dev
```
## Usage
1. **First Time Setup**
- Navigate to http://localhost:4321
- Create admin account (first user signup)
- Configure GitHub and Gitea connections
2. **Mirror Strategies**
- **Preserve Structure**: Maintains GitHub organization structure
- **Single Organization**: All repos go to one Gitea organization
- **Flat User**: All repos under your Gitea user account
- **Mixed Mode**: Personal repos in one org, organization repos preserve structure
3. **Customization**
- Click edit buttons on organization cards to set custom destinations
- Override individual repository destinations in the table view
- Starred repositories automatically go to a dedicated organization
## Advanced Features
### Git LFS (Large File Storage)
Mirror Git LFS objects along with your repositories:
- Enable "Mirror LFS" option in Settings → Mirror Options
- Requires Gitea server with LFS enabled (`LFS_START_SERVER = true`)
- Requires Git v2.1.2+ on the server
### Metadata Mirroring
Transfer complete repository metadata from GitHub to Gitea:
- **Issues** - Mirror all issues with comments and labels
- **Pull Requests** - Transfer PR discussions to Gitea
- **Labels** - Preserve repository labels
- **Milestones** - Keep project milestones
- **Wiki** - Mirror wiki content
- **Releases** - Transfer GitHub releases with assets
Enable in Settings → Mirror Options → Mirror metadata
### Repository Management
- **Ignore Status** - Mark repositories to skip from mirroring
- **Automatic Cleanup** - Configure retention period for activity logs
- **Scheduled Sync** - Set custom intervals for automatic mirroring
### Automatic Syncing & Synchronization
Gitea Mirror provides powerful automatic synchronization features:
#### Features (v3.4.0+)
- **Auto-discovery**: Automatically discovers and imports new GitHub repositories
- **Repository cleanup**: Removes repositories that no longer exist in GitHub
- **Proper intervals**: Mirrors respect your configured sync intervals (not Gitea's default 24h)
- **Smart scheduling**: Only syncs repositories that need updating
- **Auto-start on boot** (v3.5.3+): Automatically imports and mirrors all repositories when `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` is set - no manual clicks required!
#### Configuration via Web Interface (Recommended)
Navigate to the Configuration page and enable "Automatic Syncing" with your preferred interval.
#### Configuration via Environment Variables
**🚀 Set it and forget it!** With these environment variables, Gitea Mirror will automatically:
1. **Import** all your GitHub repositories on startup (no manual import needed!)
2. **Mirror** them to Gitea immediately
3. **Keep them synchronized** based on your interval
4. **Auto-discover** new repos you create/star on GitHub
5. **Clean up** repos you delete from GitHub
```bash
# Option 1: Enable automatic scheduling (triggers auto-start)
SCHEDULE_ENABLED=true
SCHEDULE_INTERVAL=3600 # Check every hour (or use cron: "0 * * * *")
# Option 2: Set mirror interval (also triggers auto-start)
GITEA_MIRROR_INTERVAL=8h # Every 8 hours
# Other examples: 5m, 30m, 1h, 24h, 1d, 7d
# Advanced: Use cron expressions for specific times
SCHEDULE_INTERVAL="0 2 * * *" # Daily at 2 AM (optimize bandwidth usage)
# Auto-import new repositories (default: true)
AUTO_IMPORT_REPOS=true
# Auto-cleanup orphaned repositories
CLEANUP_DELETE_IF_NOT_IN_GITHUB=true
CLEANUP_ORPHANED_REPO_ACTION=archive # 'archive' (recommended) or 'delete'
CLEANUP_DRY_RUN=false # Set to true to test without changes
```
**Important Notes**:
- **Auto-Start**: When `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` is set, the service automatically imports all GitHub repositories and mirrors them on startup. No manual "Import" or "Mirror" button clicks required!
- The scheduler checks every minute for tasks to run. The `GITEA_MIRROR_INTERVAL` determines how often each repository is actually synced. For example, with `8h`, each repo syncs every 8 hours from its last successful sync.
**🛡️ Backup Protection Features**:
- **No Accidental Deletions**: Repository cleanup is automatically skipped if GitHub is inaccessible (account deleted, banned, or API errors)
- **Archive Never Deletes Data**: The `archive` action preserves all repository data:
- Regular repositories: Made read-only using Gitea's archive feature
- Mirror repositories: Renamed with `archived-` prefix (Gitea API limitation prevents archiving mirrors)
- Failed operations: Repository remains fully accessible even if marking as archived fails
- **Manual Sync on Demand**: Archived mirrors stay in Gitea with automatic syncs disabled; trigger `Manual Sync` from the Repositories page whenever you need fresh data.
- **The Whole Point of Backups**: Your Gitea mirrors are preserved even when GitHub sources disappear - that's why you have backups!
- **Strongly Recommended**: Always use `CLEANUP_ORPHANED_REPO_ACTION=archive` (default) instead of `delete`
## Troubleshooting
### Reverse Proxy Configuration
If using a reverse proxy (e.g., nginx proxy manager) and experiencing issues with JavaScript files not loading properly, try enabling HTTP/2 support in your proxy configuration. While not required by the application, some proxy configurations may have better compatibility with HTTP/2 enabled. See [issue #43](https://github.com/RayLabsHQ/gitea-mirror/issues/43) for reference.
## Development
```bash
# Install dependencies
bun install
# Run development server
bun run dev
# Run tests
bun test
# Build for production
bun run build
```
## Technologies
- **Frontend**: Astro, React, Shadcn UI, Tailwind CSS v4
- **Backend**: Bun runtime, SQLite, Drizzle ORM
- **APIs**: GitHub (Octokit), Gitea REST API
- **Auth**: Better Auth with session-based authentication
## Security
### Token Encryption
- All GitHub and Gitea API tokens are encrypted at rest using AES-256-GCM
- Encryption is automatic and transparent to users
- Set `ENCRYPTION_SECRET` environment variable for production deployments
- Falls back to `BETTER_AUTH_SECRET` if not set
### Password Security
- User passwords are securely hashed by Better Auth
- Never stored in plaintext
- Secure cookie-based session management
## Authentication
Gitea Mirror supports multiple authentication methods. **Email/password authentication is the default and always enabled.**
### 1. Email & Password (Default)
The standard authentication method. First user to sign up becomes the admin.
### 2. Single Sign-On (SSO) with OIDC
Enable users to sign in with external identity providers like Google, Azure AD, Okta, Authentik, or any OIDC-compliant service.
**Configuration:**
1. Navigate to Settings → Authentication & SSO
2. Click "Add Provider"
3. Enter your OIDC provider details:
- Issuer URL (e.g., `https://accounts.google.com`)
- Client ID and Secret from your provider
- Use the "Discover" button to auto-fill endpoints
**Redirect URL for your provider:**
```
https://your-domain.com/api/auth/sso/callback/{provider-id}
```
Need help? The [SSO & OIDC guide](docs/SSO-OIDC-SETUP.md) now includes a working Authentik walkthrough plus troubleshooting tips. If you upgraded from a version earlier than v3.8.10 and see `TypeError … url.startsWith` after the callback, delete the old provider and add it again using the Discover button (see [#73](https://github.com/RayLabsHQ/gitea-mirror/issues/73) and [#122](https://github.com/RayLabsHQ/gitea-mirror/issues/122)).
### 3. Header Authentication (Reverse Proxy)
Perfect for automatic authentication when using reverse proxies like Authentik, Authelia, or Traefik Forward Auth.
**Environment Variables:**
```bash
# Enable header authentication
HEADER_AUTH_ENABLED=true
# Header names (customize based on your proxy)
HEADER_AUTH_USER_HEADER=X-Authentik-Username
HEADER_AUTH_EMAIL_HEADER=X-Authentik-Email
HEADER_AUTH_NAME_HEADER=X-Authentik-Name
# Auto-provision new users
HEADER_AUTH_AUTO_PROVISION=true
# Restrict to specific email domains (optional)
HEADER_AUTH_ALLOWED_DOMAINS=example.com,company.org
```
**How it works:**
- Users authenticated by your reverse proxy are automatically logged in
- No additional login step required
- New users can be auto-provisioned if enabled
- Falls back to regular authentication if headers are missing
**Example Authentik Configuration:**
```nginx
# In your reverse proxy configuration
proxy_set_header X-Authentik-Username $authentik_username;
proxy_set_header X-Authentik-Email $authentik_email;
proxy_set_header X-Authentik-Name $authentik_name;
```
### 4. OAuth Applications (Act as Identity Provider)
Gitea Mirror can also act as an OIDC provider for other applications. Register OAuth applications in Settings → Authentication & SSO → OAuth Applications tab.
**Use cases:**
- Allow other services to authenticate using Gitea Mirror accounts
- Create service-to-service authentication
- Build integrations with your Gitea Mirror instance
## Known Limitations
### Pull Request Mirroring Implementation
Pull requests **cannot be created as actual PRs** in Gitea due to API limitations. Instead, they are mirrored as **enriched issues** with comprehensive metadata.
**Why real PR mirroring isn't possible:**
- Gitea's API doesn't support creating pull requests from external sources
- Real PRs require actual Git branches with commits to exist in the repository
- Would require complex branch synchronization and commit replication
- The mirror relationship is one-way (GitHub → Gitea) for repository content
**How we handle Pull Requests:**
PRs are mirrored as issues with rich metadata including:
- 🏷️ Special "pull-request" label for identification
- 📌 [PR #number] prefix in title with status indicators ([MERGED], [CLOSED])
- 👤 Original author and creation date
- 📝 Complete commit history (up to 10 commits with links)
- 📊 File changes summary with additions/deletions
- 📁 List of modified files (up to 20 files)
- 💬 Original PR description and comments
- 🔀 Base and head branch information
- ✅ Merge status tracking
This approach preserves all important PR information while working within Gitea's API constraints. The PRs appear in Gitea's issue tracker with clear visual distinction and comprehensive details.
## Contributing
Contributions are welcome! Please read our [Contributing Guidelines](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests.
## License
GNU General Public License v3.0 - see [LICENSE](LICENSE) file for details.
## Star History
<a href="https://www.star-history.com/#RayLabsHQ/gitea-mirror&type=date&legend=bottom-right">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=RayLabsHQ/gitea-mirror&type=date&theme=dark&legend=bottom-right" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=RayLabsHQ/gitea-mirror&type=date&legend=bottom-right" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=RayLabsHQ/gitea-mirror&type=date&legend=bottom-right" />
</picture>
</a>
## Support
- 📖 [Documentation](https://github.com/RayLabsHQ/gitea-mirror/tree/main/docs)
- 🔐 [Custom CA Certificates](docs/CA_CERTIFICATES.md)
- 🐛 [Report Issues](https://github.com/RayLabsHQ/gitea-mirror/issues)
- 💬 [Discussions](https://github.com/RayLabsHQ/gitea-mirror/discussions)
- 🔧 [Proxmox VE Script](https://community-scripts.github.io/ProxmoxVE/scripts?id=gitea-mirror)

View File

@@ -0,0 +1,22 @@
// @ts-check
import { defineConfig } from 'astro/config';
import tailwindcss from '@tailwindcss/vite';
import react from '@astrojs/react';
import node from '@astrojs/node';
// https://astro.build/config
export default defineConfig({
output: 'server',
adapter: node({
mode: 'standalone',
}),
vite: {
plugins: [tailwindcss()],
build: {
rollupOptions: {
external: ['bun', 'bun:*'],
},
},
},
integrations: [react()]
});

2086
Divers/gitea-mirror/bun.lock Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,6 @@
[test]
# Set test timeout to 5 seconds (5000ms) to prevent hanging tests
timeout = 5000
# Preload the setup file
preload = ["./src/tests/setup.bun.ts"]

View File

@@ -0,0 +1,236 @@
# CA Certificates Configuration
This document explains how to configure custom Certificate Authority (CA) certificates for Gitea Mirror when connecting to self-signed or privately signed Gitea instances.
## Overview
When your Gitea instance uses a self-signed certificate or a certificate signed by a private Certificate Authority (CA), you need to configure Gitea Mirror to trust these certificates.
## Common SSL/TLS Errors
If you encounter any of these errors, you need to configure CA certificates:
- `UNABLE_TO_VERIFY_LEAF_SIGNATURE`
- `SELF_SIGNED_CERT_IN_CHAIN`
- `UNABLE_TO_GET_ISSUER_CERT_LOCALLY`
- `CERT_UNTRUSTED`
- `unable to verify the first certificate`
## Configuration by Deployment Method
### Docker
#### Method 1: Volume Mount (Recommended)
1. Create a certificates directory:
```bash
mkdir -p ./certs
```
2. Copy your CA certificate(s):
```bash
cp /path/to/your-ca-cert.crt ./certs/
```
3. Update `docker-compose.yml`:
```yaml
version: '3.8'
services:
gitea-mirror:
image: raylabs/gitea-mirror:latest
volumes:
- ./data:/app/data
- ./certs:/usr/local/share/ca-certificates:ro
environment:
- NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/your-ca-cert.crt
```
4. Restart the container:
```bash
docker-compose down && docker-compose up -d
```
#### Method 2: Custom Docker Image
Create a `Dockerfile`:
```dockerfile
FROM raylabs/gitea-mirror:latest
# Copy CA certificates
COPY ./certs/*.crt /usr/local/share/ca-certificates/
# Update CA certificates
RUN update-ca-certificates
# Set environment variable
ENV NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/your-ca-cert.crt
```
Build and use:
```bash
docker build -t my-gitea-mirror .
```
### Native/Bun
#### Method 1: Environment Variable
```bash
export NODE_EXTRA_CA_CERTS=/path/to/your-ca-cert.crt
bun run start
```
#### Method 2: .env File
Add to your `.env` file:
```
NODE_EXTRA_CA_CERTS=/path/to/your-ca-cert.crt
```
#### Method 3: System CA Store
**Ubuntu/Debian:**
```bash
sudo cp your-ca-cert.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
```
**RHEL/CentOS/Fedora:**
```bash
sudo cp your-ca-cert.crt /etc/pki/ca-trust/source/anchors/
sudo update-ca-trust
```
**macOS:**
```bash
sudo security add-trusted-cert -d -r trustRoot \
-k /Library/Keychains/System.keychain your-ca-cert.crt
```
### LXC Container (Proxmox VE)
1. Enter the container:
```bash
pct enter <container-id>
```
2. Create certificates directory:
```bash
mkdir -p /usr/local/share/ca-certificates
```
3. Copy your CA certificate:
```bash
cat > /usr/local/share/ca-certificates/your-ca.crt
```
(Paste certificate content and press Ctrl+D)
4. Update the systemd service:
```bash
cat >> /etc/systemd/system/gitea-mirror.service << EOF
Environment="NODE_EXTRA_CA_CERTS=/usr/local/share/ca-certificates/your-ca.crt"
EOF
```
5. Reload and restart:
```bash
systemctl daemon-reload
systemctl restart gitea-mirror
```
## Multiple CA Certificates
### Option 1: Bundle Certificates
```bash
cat ca-cert1.crt ca-cert2.crt ca-cert3.crt > ca-bundle.crt
export NODE_EXTRA_CA_CERTS=/path/to/ca-bundle.crt
```
### Option 2: System CA Store
```bash
# Copy all certificates
cp *.crt /usr/local/share/ca-certificates/
update-ca-certificates
```
## Verification
### 1. Test Gitea Connection
Use the "Test Connection" button in the Gitea configuration section.
### 2. Check Logs
**Docker:**
```bash
docker logs gitea-mirror
```
**Native:**
Check terminal output
**LXC:**
```bash
journalctl -u gitea-mirror -f
```
### 3. Manual Certificate Test
```bash
openssl s_client -connect your-gitea-domain.com:443 -CAfile /path/to/ca-cert.crt
```
## Best Practices
1. **Certificate Security**
- Keep CA certificates secure
- Use read-only mounts in Docker
- Limit certificate file permissions
- Regularly update certificates
2. **Certificate Management**
- Use descriptive certificate filenames
- Document certificate purposes
- Track certificate expiration dates
- Maintain certificate backups
3. **Production Deployment**
- Use proper SSL certificates when possible
- Consider Let's Encrypt for public instances
- Implement certificate rotation procedures
- Monitor certificate expiration
## Troubleshooting
### Certificate not being recognized
- Ensure the certificate is in PEM format
- Check that `NODE_EXTRA_CA_CERTS` points to the correct file
- Restart the application after adding certificates
### Still getting SSL errors
- Verify the complete certificate chain is included
- Check if intermediate certificates are needed
- Ensure the certificate matches the server hostname
### Certificate expired
- Check validity: `openssl x509 -in cert.crt -noout -dates`
- Update with new certificate from your CA
- Restart Gitea Mirror after updating
## Certificate Format
Certificates must be in PEM format. Example:
```
-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJAKl8bUgMdErlMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV
[... certificate content ...]
-----END CERTIFICATE-----
```
If your certificate is in DER format, convert it:
```bash
openssl x509 -inform der -in certificate.cer -out certificate.crt
```

View File

@@ -0,0 +1,21 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "new-york",
"rsc": false,
"tsx": true,
"tailwind": {
"config": "",
"css": "src/styles/global.css",
"baseColor": "neutral",
"cssVariables": true,
"prefix": ""
},
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"iconLibrary": "lucide"
}

View File

@@ -0,0 +1,32 @@
# Data Directory
This directory contains the SQLite database file for the Gitea Mirror application.
## Files
- `gitea-mirror.db`: The main database file. This file is **not** committed to the repository as it may contain sensitive information like tokens.
## Important Notes
- **Never commit `gitea-mirror.db` to the repository** as it may contain sensitive information like GitHub and Gitea tokens.
- The application will create this database file automatically on first run.
## Database Initialization
To initialize the database for real data mode, run:
```bash
pnpm init-db
```
This will create the necessary tables. On first launch, you'll be guided through creating an admin account with your chosen credentials.
## User Management
To reset users (for testing the first-time setup flow), run:
```bash
pnpm reset-users
```
This will remove all users and their associated data from the database, allowing you to test the signup flow.

View File

@@ -0,0 +1,61 @@
# Minimal Gitea Mirror deployment
# Only includes what CANNOT be configured via the Web UI
# Everything else can be set up through the web interface after deployment
services:
gitea-mirror:
image: ghcr.io/raylabshq/gitea-mirror:latest
container_name: gitea-mirror
restart: unless-stopped
ports:
- "${PORT:-4321}:4321"
user: ${PUID:-1000}:${PGID:-1000}
volumes:
- ./data:/app/data
environment:
# === ABSOLUTELY REQUIRED ===
# This MUST be set and CANNOT be changed via UI
- BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET} # Min 32 chars, required for sessions
- BETTER_AUTH_URL=${BETTER_AUTH_URL:-http://localhost:4321}
- BETTER_AUTH_TRUSTED_ORIGINS=${BETTER_AUTH_TRUSTED_ORIGINS:-http://localhost:4321}
# === CORE SETTINGS ===
# These are technically required but have working defaults
- NODE_ENV=production
- DATABASE_URL=file:data/gitea-mirror.db
- HOST=0.0.0.0
- PORT=4321
- PUBLIC_BETTER_AUTH_URL=${PUBLIC_BETTER_AUTH_URL:-http://localhost:4321}
# Optional concurrency controls (defaults match in-app defaults)
# If you want perfect ordering of issues and PRs, set these at 1
- MIRROR_ISSUE_CONCURRENCY=${MIRROR_ISSUE_CONCURRENCY:-3}
- MIRROR_PULL_REQUEST_CONCURRENCY=${MIRROR_PULL_REQUEST_CONCURRENCY:-5}
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=3", "--spider", "http://localhost:4321/api/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 15s
# === QUICK START ===
#
# 1. Create a .env file with only ONE required variable:
# BETTER_AUTH_SECRET=your-32-character-minimum-secret-key-here
#
# 2. Run:
# docker-compose -f docker-compose.alt.yml up -d
#
# 3. Access at http://localhost:4321
#
# 4. Sign up for an account (first user becomes admin)
#
# 5. Configure everything else through the web UI:
# - GitHub credentials
# - Gitea credentials
# - Mirror settings
# - Scheduling options
# - Auto-import settings
# - Cleanup preferences
#
# That's it! Everything else can be configured via the web interface.

View File

@@ -0,0 +1,114 @@
# Development environment with local Gitea instance for testing
# Run with: docker compose -f docker-compose.dev.yml up -d
services:
# Local Gitea instance for testing
gitea:
image: gitea/gitea:latest
container_name: gitea
restart: unless-stopped
entrypoint: ["/tmp/gitea-dev-init.sh"]
environment:
- USER_UID=1000
- USER_GID=1000
- GITEA__database__DB_TYPE=sqlite3
- GITEA__database__PATH=/data/gitea/gitea.db
- GITEA__server__DOMAIN=localhost
- GITEA__server__ROOT_URL=http://localhost:3001/
- GITEA__server__SSH_DOMAIN=localhost
- GITEA__server__SSH_PORT=2222
- GITEA__server__START_SSH_SERVER=true
- GITEA__security__INSTALL_LOCK=true
- GITEA__service__DISABLE_REGISTRATION=false
- GITEA__log__MODE=console
- GITEA__log__LEVEL=Info
ports:
- "3001:3000"
- "2222:22"
volumes:
- gitea-data:/data
- gitea-config:/etc/gitea
- ./scripts/gitea-app.ini:/tmp/app.ini:ro
- ./scripts/gitea-dev-init.sh:/tmp/gitea-dev-init.sh:ro
networks:
- gitea-network
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:3000/"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
# Development service connected to local Gitea
gitea-mirror-dev:
# For dev environment, always build from local sources
build:
context: .
dockerfile: Dockerfile
platforms:
- linux/amd64
- linux/arm64
container_name: gitea-mirror-dev
restart: unless-stopped
ports:
- "4321:4321"
volumes:
- gitea-mirror-data:/app/data
# Mount custom CA certificates - choose one option:
# Option 1: Mount individual CA certificates from certs directory
# - ./certs:/app/certs:ro
# Option 2: Mount system CA bundle (if your CA is already in system store)
# - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro
depends_on:
- gitea
environment:
- NODE_ENV=development
- DATABASE_URL=file:data/gitea-mirror.db
- HOST=0.0.0.0
- PORT=4321
- BETTER_AUTH_SECRET=dev-secret-key
# GitHub/Gitea Mirror Config
- GITHUB_USERNAME=${GITHUB_USERNAME:-your-github-username}
- GITHUB_TOKEN=${GITHUB_TOKEN:-your-github-token}
- GITHUB_EXCLUDED_ORGS=${GITHUB_EXCLUDED_ORGS:-}
- SKIP_FORKS=${SKIP_FORKS:-false}
- PRIVATE_REPOSITORIES=${PRIVATE_REPOSITORIES:-false}
- MIRROR_ISSUES=${MIRROR_ISSUES:-false}
- MIRROR_WIKI=${MIRROR_WIKI:-false}
- MIRROR_STARRED=${MIRROR_STARRED:-false}
- MIRROR_ORGANIZATIONS=${MIRROR_ORGANIZATIONS:-false}
- PRESERVE_ORG_STRUCTURE=${PRESERVE_ORG_STRUCTURE:-false}
- ONLY_MIRROR_ORGS=${ONLY_MIRROR_ORGS:-false}
- SKIP_STARRED_ISSUES=${SKIP_STARRED_ISSUES:-false}
- GITEA_URL=http://gitea:3000
- GITEA_TOKEN=${GITEA_TOKEN:-your-local-gitea-token}
- GITEA_USERNAME=${GITEA_USERNAME:-your-local-gitea-username}
- GITEA_ORGANIZATION=${GITEA_ORGANIZATION:-github-mirrors}
- GITEA_ORG_VISIBILITY=${GITEA_ORG_VISIBILITY:-public}
- DELAY=${DELAY:-3600}
# Optional: Skip TLS verification (insecure, use only for testing)
# - GITEA_SKIP_TLS_VERIFY=${GITEA_SKIP_TLS_VERIFY:-false}
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:4321/api/health"]
interval: 30s
timeout: 5s
retries: 3
start_period: 5s
networks:
- gitea-network
# Define named volumes for data persistence
volumes:
gitea-data: # Gitea data volume
gitea-config: # Gitea config volume
gitea-mirror-data: # Gitea Mirror database volume
# Define networks
networks:
gitea-network:
name: gitea-network
# Let Docker Compose manage this network
# If you need to use an existing network, uncomment the line below
# external: true

View File

@@ -0,0 +1,85 @@
# Gitea Mirror deployment configuration
# Standard deployment with automatic database maintenance
services:
gitea-mirror:
image: ${DOCKER_REGISTRY:-ghcr.io}/${DOCKER_IMAGE:-raylabshq/gitea-mirror}:${DOCKER_TAG:-latest}
build:
context: .
dockerfile: Dockerfile
platforms:
- linux/amd64
- linux/arm64
cache_from:
- ${DOCKER_REGISTRY:-ghcr.io}/${DOCKER_IMAGE:-raylabshq/gitea-mirror}:${DOCKER_TAG:-latest}
container_name: gitea-mirror
restart: unless-stopped
ports:
- "4321:4321"
volumes:
- gitea-mirror-data:/app/data
# Mount custom CA certificates - choose one option:
# Option 1: Mount individual CA certificates from certs directory
# - ./certs:/app/certs:ro
# Option 2: Mount system CA bundle (if your CA is already in system store)
# - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro
environment:
# For a complete list of all supported environment variables, see:
# docs/ENVIRONMENT_VARIABLES.md or .env.example
- NODE_ENV=production
- DATABASE_URL=file:data/gitea-mirror.db
- HOST=0.0.0.0
- PORT=4321
- BETTER_AUTH_SECRET=${BETTER_AUTH_SECRET:-your-secret-key-change-this-in-production}
- BETTER_AUTH_URL=${BETTER_AUTH_URL:-http://localhost:4321}
# Optional: ENCRYPTION_SECRET will be auto-generated if not provided
# - ENCRYPTION_SECRET=${ENCRYPTION_SECRET:-}
# GitHub/Gitea Mirror Config
- GITHUB_USERNAME=${GITHUB_USERNAME:-}
- GITHUB_TOKEN=${GITHUB_TOKEN:-}
- GITHUB_EXCLUDED_ORGS=${GITHUB_EXCLUDED_ORGS:-}
- SKIP_FORKS=${SKIP_FORKS:-false}
- PRIVATE_REPOSITORIES=${PRIVATE_REPOSITORIES:-false}
- MIRROR_ISSUES=${MIRROR_ISSUES:-false}
- MIRROR_WIKI=${MIRROR_WIKI:-false}
- MIRROR_STARRED=${MIRROR_STARRED:-false}
- MIRROR_ORGANIZATIONS=${MIRROR_ORGANIZATIONS:-false}
- PRESERVE_ORG_STRUCTURE=${PRESERVE_ORG_STRUCTURE:-false}
- ONLY_MIRROR_ORGS=${ONLY_MIRROR_ORGS:-false}
- SKIP_STARRED_ISSUES=${SKIP_STARRED_ISSUES:-false}
- MIRROR_ISSUE_CONCURRENCY=${MIRROR_ISSUE_CONCURRENCY:-3}
- MIRROR_PULL_REQUEST_CONCURRENCY=${MIRROR_PULL_REQUEST_CONCURRENCY:-5}
- GITEA_URL=${GITEA_URL:-}
- GITEA_TOKEN=${GITEA_TOKEN:-}
- GITEA_USERNAME=${GITEA_USERNAME:-}
- GITEA_ORGANIZATION=${GITEA_ORGANIZATION:-github-mirrors}
- GITEA_ORG_VISIBILITY=${GITEA_ORG_VISIBILITY:-public}
- DELAY=${DELAY:-3600}
# Scheduling and Sync Configuration (Issue #72 fixes)
- SCHEDULE_ENABLED=${SCHEDULE_ENABLED:-false}
- GITEA_MIRROR_INTERVAL=${GITEA_MIRROR_INTERVAL:-8h}
- AUTO_IMPORT_REPOS=${AUTO_IMPORT_REPOS:-true}
- AUTO_MIRROR_REPOS=${AUTO_MIRROR_REPOS:-false}
# Repository Cleanup Configuration
- CLEANUP_DELETE_IF_NOT_IN_GITHUB=${CLEANUP_DELETE_IF_NOT_IN_GITHUB:-false}
- CLEANUP_ORPHANED_REPO_ACTION=${CLEANUP_ORPHANED_REPO_ACTION:-archive}
- CLEANUP_DRY_RUN=${CLEANUP_DRY_RUN:-true}
# Optional: Skip TLS verification (insecure, use only for testing)
# - GITEA_SKIP_TLS_VERIFY=${GITEA_SKIP_TLS_VERIFY:-false}
# Header Authentication (for Reverse Proxy SSO)
- HEADER_AUTH_ENABLED=${HEADER_AUTH_ENABLED:-false}
- HEADER_AUTH_USER_HEADER=${HEADER_AUTH_USER_HEADER:-X-Authentik-Username}
- HEADER_AUTH_EMAIL_HEADER=${HEADER_AUTH_EMAIL_HEADER:-X-Authentik-Email}
- HEADER_AUTH_NAME_HEADER=${HEADER_AUTH_NAME_HEADER:-X-Authentik-Name}
- HEADER_AUTH_AUTO_PROVISION=${HEADER_AUTH_AUTO_PROVISION:-false}
- HEADER_AUTH_ALLOWED_DOMAINS=${HEADER_AUTH_ALLOWED_DOMAINS:-}
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=3", "--spider", "http://localhost:4321/api/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 15s
# Define named volumes for database persistence
volumes:
gitea-mirror-data: # Database volume

View File

@@ -0,0 +1,227 @@
#!/bin/sh
set -e
# Ensure data directory exists
mkdir -p /app/data
# Handle custom CA certificates
if [ -d "/app/certs" ] && [ "$(ls -A /app/certs/*.crt 2>/dev/null)" ]; then
echo "Custom CA certificates found, configuring Node.js to use them..."
# Combine all CA certificates into a bundle for Node.js
CA_BUNDLE="/app/certs/ca-bundle.crt"
> "$CA_BUNDLE"
for cert in /app/certs/*.crt; do
if [ -f "$cert" ]; then
echo "Adding certificate: $(basename "$cert")"
cat "$cert" >> "$CA_BUNDLE"
echo "" >> "$CA_BUNDLE" # Add newline between certificates
fi
done
# Set Node.js to use the custom CA bundle
export NODE_EXTRA_CA_CERTS="$CA_BUNDLE"
echo "NODE_EXTRA_CA_CERTS set to: $NODE_EXTRA_CA_CERTS"
# For Bun compatibility, also set the CA bundle in system location if writable
if [ -f "/etc/ssl/certs/ca-certificates.crt" ] && [ -w "/etc/ssl/certs/" ]; then
echo "Appending custom certificates to system CA bundle..."
cat "$CA_BUNDLE" >> /etc/ssl/certs/ca-certificates.crt
fi
else
echo "No custom CA certificates found in /app/certs"
fi
# Check if system CA bundle is mounted and use it (only if not already set)
if [ -z "$NODE_EXTRA_CA_CERTS" ] && [ -f "/etc/ssl/certs/ca-certificates.crt" ] && [ ! -L "/etc/ssl/certs/ca-certificates.crt" ]; then
# Check if it's a mounted file (not the default symlink)
if [ "$(stat -c '%d' /etc/ssl/certs/ca-certificates.crt 2>/dev/null)" != "$(stat -c '%d' / 2>/dev/null)" ] || \
[ "$(stat -f '%d' /etc/ssl/certs/ca-certificates.crt 2>/dev/null)" != "$(stat -f '%d' / 2>/dev/null)" ]; then
echo "System CA bundle mounted, configuring Node.js to use it..."
export NODE_EXTRA_CA_CERTS="/etc/ssl/certs/ca-certificates.crt"
echo "NODE_EXTRA_CA_CERTS set to: $NODE_EXTRA_CA_CERTS"
fi
fi
# Optional: If GITEA_SKIP_TLS_VERIFY is set, configure accordingly
if [ "$GITEA_SKIP_TLS_VERIFY" = "true" ]; then
echo "Warning: GITEA_SKIP_TLS_VERIFY is set to true. This is insecure!"
export NODE_TLS_REJECT_UNAUTHORIZED=0
fi
# Generate a secure BETTER_AUTH_SECRET if one isn't provided or is using the default value
BETTER_AUTH_SECRET_FILE="/app/data/.better_auth_secret"
JWT_SECRET_FILE="/app/data/.jwt_secret" # Old file for backward compatibility
if [ "$BETTER_AUTH_SECRET" = "your-secret-key-change-this-in-production" ] || [ -z "$BETTER_AUTH_SECRET" ]; then
# Check if we have a previously generated secret
if [ -f "$BETTER_AUTH_SECRET_FILE" ]; then
echo "Using previously generated BETTER_AUTH_SECRET"
export BETTER_AUTH_SECRET=$(cat "$BETTER_AUTH_SECRET_FILE")
# Check for old JWT_SECRET file for backward compatibility
elif [ -f "$JWT_SECRET_FILE" ]; then
echo "Migrating from old JWT_SECRET to BETTER_AUTH_SECRET"
export BETTER_AUTH_SECRET=$(cat "$JWT_SECRET_FILE")
# Save to new file
echo "$BETTER_AUTH_SECRET" > "$BETTER_AUTH_SECRET_FILE"
chmod 600 "$BETTER_AUTH_SECRET_FILE"
# Optionally remove old file after successful migration
rm -f "$JWT_SECRET_FILE"
else
echo "Generating a secure random BETTER_AUTH_SECRET"
# Try to generate a secure random string using OpenSSL
if command -v openssl >/dev/null 2>&1; then
GENERATED_SECRET=$(openssl rand -hex 32)
else
# Fallback to using /dev/urandom if openssl is not available
echo "OpenSSL not found, using fallback method for random generation"
GENERATED_SECRET=$(head -c 32 /dev/urandom | sha256sum | cut -d' ' -f1)
fi
export BETTER_AUTH_SECRET="$GENERATED_SECRET"
# Save the secret to a file for persistence across container restarts
echo "$GENERATED_SECRET" > "$BETTER_AUTH_SECRET_FILE"
chmod 600 "$BETTER_AUTH_SECRET_FILE"
fi
echo "BETTER_AUTH_SECRET has been set to a secure random value"
fi
# Generate a secure ENCRYPTION_SECRET if one isn't provided
ENCRYPTION_SECRET_FILE="/app/data/.encryption_secret"
if [ -z "$ENCRYPTION_SECRET" ]; then
# Check if we have a previously generated secret
if [ -f "$ENCRYPTION_SECRET_FILE" ]; then
echo "Using previously generated ENCRYPTION_SECRET"
export ENCRYPTION_SECRET=$(cat "$ENCRYPTION_SECRET_FILE")
else
echo "Generating a secure random ENCRYPTION_SECRET"
# Generate a 48-character secret for encryption
if command -v openssl >/dev/null 2>&1; then
GENERATED_ENCRYPTION_SECRET=$(openssl rand -base64 36)
else
# Fallback to using /dev/urandom if openssl is not available
echo "OpenSSL not found, using fallback method for encryption secret generation"
GENERATED_ENCRYPTION_SECRET=$(head -c 36 /dev/urandom | base64 | tr -d '\n' | head -c 48)
fi
export ENCRYPTION_SECRET="$GENERATED_ENCRYPTION_SECRET"
# Save the secret to a file for persistence across container restarts
echo "$GENERATED_ENCRYPTION_SECRET" > "$ENCRYPTION_SECRET_FILE"
chmod 600 "$ENCRYPTION_SECRET_FILE"
fi
echo "ENCRYPTION_SECRET has been set to a secure random value"
fi
# Skip dependency installation entirely for pre-built images
# Dependencies are already installed during the Docker build process
# Initialize the database if it doesn't exist
# Note: Drizzle migrations will be run automatically when the app starts (see src/lib/db/index.ts)
if [ ! -f "/app/data/gitea-mirror.db" ]; then
echo "Database not found. It will be created and initialized via Drizzle migrations on first app startup..."
# Create empty database file so migrations can run
touch /app/data/gitea-mirror.db
else
echo "Database already exists, Drizzle will check for pending migrations on startup..."
fi
# Extract version from package.json and set as environment variable
if [ -f "package.json" ]; then
export npm_package_version=$(grep -o '"version": *"[^"]*"' package.json | cut -d'"' -f4)
echo "Setting application version: $npm_package_version"
fi
# Initialize configuration from environment variables if provided
echo "Checking for environment configuration..."
if [ -f "dist/scripts/startup-env-config.js" ]; then
echo "Loading configuration from environment variables..."
bun dist/scripts/startup-env-config.js
ENV_CONFIG_EXIT_CODE=$?
elif [ -f "scripts/startup-env-config.ts" ]; then
echo "Loading configuration from environment variables..."
bun scripts/startup-env-config.ts
ENV_CONFIG_EXIT_CODE=$?
else
echo "Environment configuration script not found. Skipping."
ENV_CONFIG_EXIT_CODE=0
fi
# Log environment config result
if [ $ENV_CONFIG_EXIT_CODE -eq 0 ]; then
echo "✅ Environment configuration loaded successfully"
else
echo "⚠️ Environment configuration loading completed with warnings"
fi
# Run startup recovery to handle any interrupted jobs
echo "Running startup recovery..."
if [ -f "dist/scripts/startup-recovery.js" ]; then
echo "Running startup recovery using compiled script..."
bun dist/scripts/startup-recovery.js --timeout=30000
RECOVERY_EXIT_CODE=$?
elif [ -f "scripts/startup-recovery.ts" ]; then
echo "Running startup recovery using TypeScript script..."
bun scripts/startup-recovery.ts --timeout=30000
RECOVERY_EXIT_CODE=$?
else
echo "Warning: Startup recovery script not found. Skipping recovery."
RECOVERY_EXIT_CODE=0
fi
# Log recovery result
if [ $RECOVERY_EXIT_CODE -eq 0 ]; then
echo "✅ Startup recovery completed successfully"
elif [ $RECOVERY_EXIT_CODE -eq 1 ]; then
echo "⚠️ Startup recovery completed with warnings"
else
echo "❌ Startup recovery failed with exit code $RECOVERY_EXIT_CODE"
fi
# Run repository status repair to fix any inconsistent mirroring states
echo "Running repository status repair..."
if [ -f "dist/scripts/repair-mirrored-repos.js" ]; then
echo "Running repository repair using compiled script..."
bun dist/scripts/repair-mirrored-repos.js --startup
REPAIR_EXIT_CODE=$?
elif [ -f "scripts/repair-mirrored-repos.ts" ]; then
echo "Running repository repair using TypeScript script..."
bun scripts/repair-mirrored-repos.ts --startup
REPAIR_EXIT_CODE=$?
else
echo "Warning: Repository repair script not found. Skipping repair."
REPAIR_EXIT_CODE=0
fi
# Log repair result
if [ $REPAIR_EXIT_CODE -eq 0 ]; then
echo "✅ Repository status repair completed successfully"
else
echo "⚠️ Repository status repair completed with warnings (exit code $REPAIR_EXIT_CODE)"
fi
# Function to handle shutdown signals
shutdown_handler() {
echo "🛑 Received shutdown signal, forwarding to application..."
if [ ! -z "$APP_PID" ]; then
kill -TERM "$APP_PID"
wait "$APP_PID"
fi
exit 0
}
# Set up signal handlers
trap 'shutdown_handler' TERM INT HUP
# Start the application
echo "Starting Gitea Mirror..."
bun ./dist/server/entry.mjs &
APP_PID=$!
# Wait for the application to finish
wait "$APP_PID"

View File

@@ -0,0 +1,354 @@
# Development Workflow
This guide covers the development workflow for the open-source Gitea Mirror.
## Getting Started
### Prerequisites
- Bun >= 1.2.9
- Node.js >= 20
- Git
- GitHub account (for API access)
- Gitea instance (for testing)
### Initial Setup
1. **Clone the repository**:
```bash
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
```
2. **Install dependencies and seed the SQLite database**:
```bash
bun run setup
```
3. **Configure environment (optional)**:
```bash
cp .env.example .env
# Edit .env with your settings
```
4. **Start the development server**:
```bash
bun run dev
```
## Development Commands
| Command | Description |
|---------|-------------|
| `bun run dev` | Start the Bun + Astro dev server with hot reload |
| `bun run build` | Build the production bundle |
| `bun run preview` | Preview the production build locally |
| `bun test` | Run the Bun test suite |
| `bun test:watch` | Run tests in watch mode |
| `bun run db:studio` | Launch Drizzle Kit Studio |
## Project Structure
```
gitea-mirror/
├── src/ # Application UI, API routes, and services
│ ├── components/ # React components rendered inside Astro pages
│ ├── pages/ # Astro pages and API routes (e.g., /api/*)
│ ├── lib/ # Core logic: GitHub/Gitea clients, scheduler, recovery, db helpers
│ │ ├── db/ # Drizzle adapter + schema
│ │ ├── modules/ # Module wiring (jobs, integrations)
│ │ └── utils/ # Shared utilities
│ ├── hooks/ # React hooks
│ ├── content/ # In-app documentation and templated content
│ ├── layouts/ # Shared layout components
│ ├── styles/ # Tailwind CSS entrypoints
│ └── types/ # TypeScript types
├── scripts/ # Bun scripts for DB management and maintenance
├── www/ # Marketing site (Astro + MDX use cases)
├── public/ # Static assets served by Vite/Astro
└── tests/ # Dedicated integration/unit test helpers
```
## Feature Development
### Adding a New Feature
1. **Create feature branch**:
```bash
git checkout -b feature/my-feature
```
2. **Plan your changes**:
- UI components live in `src/components/`
- API endpoints live in `src/pages/api/`
- Database logic is under `src/lib/db/` (schema + adapter)
- Shared types are in `src/types/`
3. **Implement the feature**:
**Example: Adding a new API endpoint**
```typescript
// src/pages/api/my-endpoint.ts
import type { APIRoute } from 'astro';
import { getUserFromCookie } from '@/lib/auth-utils';
export const GET: APIRoute = async ({ request }) => {
const user = await getUserFromCookie(request);
if (!user) {
return new Response('Unauthorized', { status: 401 });
}
// Your logic here
return new Response(JSON.stringify({ data: 'success' }), {
headers: { 'Content-Type': 'application/json' }
});
};
```
4. **Write tests**:
```typescript
// src/lib/my-feature.test.ts
import { describe, it, expect } from 'bun:test';
describe('My Feature', () => {
it('should work correctly', () => {
expect(myFunction()).toBe('expected');
});
});
```
5. **Update documentation**:
- Add JSDoc comments
- Update README/docs if needed
- Document API changes
## Database Development
### Schema Changes
1. **Modify schema**:
```typescript
// src/lib/db/schema.ts
export const myTable = sqliteTable('my_table', {
id: text('id').primaryKey(),
name: text('name').notNull(),
createdAt: integer('created_at').notNull(),
});
```
2. **Generate migration**:
```bash
bun run db:generate
```
3. **Apply migration**:
```bash
bun run db:migrate
```
### Writing Queries
```typescript
// src/lib/db/queries/my-queries.ts
import { db } from '../index';
import { myTable } from '../schema';
export async function getMyData(userId: string) {
return db.select()
.from(myTable)
.where(eq(myTable.userId, userId));
}
```
## Testing
### Unit Tests
```bash
# Run all tests
bun test
# Run specific test file
bun test auth
# Watch mode
bun test:watch
# Coverage
bun test:coverage
```
### Manual Testing Checklist
- [ ] Feature works as expected
- [ ] No console errors
- [ ] Responsive on mobile
- [ ] Handles errors gracefully
- [ ] Loading states work
- [ ] Form validation works
- [ ] API returns correct status codes
## Debugging
### VSCode Configuration
Create `.vscode/launch.json`:
```json
{
"version": "0.2.0",
"configurations": [
{
"type": "bun",
"request": "launch",
"name": "Debug Bun",
"program": "${workspaceFolder}/src/index.ts",
"cwd": "${workspaceFolder}",
"env": {
"NODE_ENV": "development"
}
}
]
}
```
### Debug Logging
```typescript
// Development only logging
if (import.meta.env.DEV) {
console.log('[Debug]', data);
}
```
## Code Style
### TypeScript
- Use strict mode
- Define interfaces for all data structures
- Avoid `any` type
- Use proper error handling
### React Components
- Use functional components
- Implement proper loading states
- Handle errors with error boundaries
- Use TypeScript for props
### API Routes
- Always validate input
- Return proper status codes
- Use consistent error format
- Document with JSDoc
## Git Workflow
### Commit Messages
Follow conventional commits:
```
feat: add repository filtering
fix: resolve sync timeout issue
docs: update API documentation
style: format code with prettier
refactor: simplify auth logic
test: add user creation tests
chore: update dependencies
```
### Pull Request Process
1. Create feature branch
2. Make changes
3. Write/update tests
4. Update documentation
5. Create PR with description
6. Address review feedback
7. Squash and merge
## Performance
### Development Tips
- Use React DevTools
- Monitor bundle size
- Profile database queries
- Check memory usage
### Optimization
- Lazy load components
- Optimize images
- Use database indexes
- Cache API responses
## Common Issues
### Port Already in Use
```bash
# Use different port
PORT=3001 bun run dev
```
### Database Locked
```bash
# Reset database
bun run cleanup-db
bun run init-db
```
### Type Errors
```bash
# Check types
bunx tsc --noEmit
```
## Release Process
1. **Update version**:
```bash
npm version patch # or minor/major
```
2. **Update CHANGELOG.md**
3. **Build and test**:
```bash
bun run build
bun test
```
4. **Create release**:
```bash
git tag v2.23.0
git push origin v2.23.0
```
5. **Create GitHub release**
## Contributing
1. Fork the repository
2. Create your feature branch
3. Commit your changes
4. Push to your fork
5. Create a Pull Request
## Resources
- [Astro Documentation](https://docs.astro.build)
- [Bun Documentation](https://bun.sh/docs)
- [Drizzle ORM](https://orm.drizzle.team)
- [React Documentation](https://react.dev)
- [TypeScript Handbook](https://www.typescriptlang.org/docs/)
## Getting Help
- Check existing [issues](https://github.com/yourusername/gitea-mirror/issues)
- Join [discussions](https://github.com/yourusername/gitea-mirror/discussions)
- Read the [FAQ](./FAQ.md)

View File

@@ -0,0 +1,416 @@
# Environment Variables Documentation
This document provides a comprehensive list of all environment variables supported by Gitea Mirror. These can be used to configure the application via Docker or other deployment methods.
## Environment Variables and UI Interaction
When environment variables are set:
1. They are loaded on application startup
2. Values are stored in the database on first load
3. The UI will display these values and they can be modified
4. UI changes are saved to the database and persist
5. Environment variables provide initial defaults but don't override UI changes
**Note**: Some critical settings like `GITEA_LFS`, `MIRROR_RELEASES`, and `MIRROR_METADATA` will be visible and configurable in the UI even when set via environment variables.
## Table of Contents
- [Core Configuration](#core-configuration)
- [GitHub Configuration](#github-configuration)
- [Gitea Configuration](#gitea-configuration)
- [Mirror Options](#mirror-options)
- [Automation Configuration](#automation-configuration)
- [Database Cleanup Configuration](#database-cleanup-configuration)
- [Authentication Configuration](#authentication-configuration)
- [Docker Configuration](#docker-configuration)
## Core Configuration
Essential application settings required for running Gitea Mirror.
| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `NODE_ENV` | Application environment | `production` | No |
| `HOST` | Server host binding | `0.0.0.0` | No |
| `PORT` | Server port | `4321` | No |
| `DATABASE_URL` | Database connection URL | `sqlite://data/gitea-mirror.db` | No |
| `BETTER_AUTH_SECRET` | Secret key for session signing (generate with: `openssl rand -base64 32`) | - | Yes |
| `BETTER_AUTH_URL` | Primary base URL for authentication. This should be the main URL where your application is accessed. | `http://localhost:4321` | No |
| `PUBLIC_BETTER_AUTH_URL` | Client-side auth URL for multi-origin access. Set this to your primary domain when you need to access the app from different origins (e.g., both IP and domain). The client will use this URL for all auth requests instead of the current browser origin. | - | No |
| `BETTER_AUTH_TRUSTED_ORIGINS` | Trusted origins for authentication requests. Comma-separated list of URLs. Use this to specify additional access URLs (e.g., local IP + domain: `http://10.10.20.45:4321,https://gitea-mirror.mydomain.tld`), SSO providers, reverse proxies, etc. | - | No |
| `ENCRYPTION_SECRET` | Optional encryption key for tokens (generate with: `openssl rand -base64 48`) | - | No |
## GitHub Configuration
Settings for connecting to and configuring GitHub repository sources.
### Basic Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITHUB_USERNAME` | Your GitHub username | - | - |
| `GITHUB_TOKEN` | GitHub personal access token (requires repo and admin:org scopes) | - | - |
| `GITHUB_TYPE` | GitHub account type | `personal` | `personal`, `organization` |
### Repository Selection
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `PRIVATE_REPOSITORIES` | Include private repositories | `false` | `true`, `false` |
| `PUBLIC_REPOSITORIES` | Include public repositories | `true` | `true`, `false` |
| `INCLUDE_ARCHIVED` | Include archived repositories | `false` | `true`, `false` |
| `SKIP_FORKS` | Skip forked repositories | `false` | `true`, `false` |
| `MIRROR_STARRED` | Mirror starred repositories | `false` | `true`, `false` |
| `STARRED_REPOS_ORG` | Organization name for starred repos | `starred` | Any string |
### Organization Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `MIRROR_ORGANIZATIONS` | Mirror organization repositories | `false` | `true`, `false` |
| `PRESERVE_ORG_STRUCTURE` | Preserve GitHub organization structure in Gitea | `false` | `true`, `false` |
| `ONLY_MIRROR_ORGS` | Only mirror organization repos (skip personal) | `false` | `true`, `false` |
| `MIRROR_STRATEGY` | Repository organization strategy | `preserve` | `preserve`, `single-org`, `flat-user`, `mixed` |
### Advanced Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SKIP_STARRED_ISSUES` | Enable lightweight mode for starred repos (skip issues) | `false` | `true`, `false` |
## Gitea Configuration
Settings for the destination Gitea instance.
### Connection Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_URL` | Gitea instance URL | - | Valid URL |
| `GITEA_TOKEN` | Gitea access token | - | - |
| `GITEA_USERNAME` | Gitea username | - | - |
| `GITEA_ORGANIZATION` | Default organization for single-org strategy | `github-mirrors` | Any string |
### Repository Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_ORG_VISIBILITY` | Default organization visibility | `public` | `public`, `private`, `limited`, `default` |
| `GITEA_MIRROR_INTERVAL` | Mirror sync interval - **automatically enables scheduled mirroring when set** | `8h` | Duration string (e.g., `30m`, `1h`, `8h`, `24h`, `1d`) or seconds |
| `GITEA_LFS` | Enable LFS support (requires LFS on Gitea server) - Shows in UI | `false` | `true`, `false` |
| `GITEA_CREATE_ORG` | Auto-create organizations | `true` | `true`, `false` |
| `GITEA_PRESERVE_VISIBILITY` | Preserve GitHub repo visibility in Gitea | `false` | `true`, `false` |
### Template Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_TEMPLATE_OWNER` | Template repository owner | - | Any string |
| `GITEA_TEMPLATE_REPO` | Template repository name | - | Any string |
### Topic Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_ADD_TOPICS` | Add topics to repositories | `true` | `true`, `false` |
| `GITEA_TOPIC_PREFIX` | Prefix for repository topics | - | Any string |
### Fork Handling
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_FORK_STRATEGY` | How to handle forked repositories | `reference` | `skip`, `reference`, `full-copy` |
### Additional Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `GITEA_SKIP_TLS_VERIFY` | Skip TLS certificate verification (WARNING: insecure) | `false` | `true`, `false` |
## Mirror Options
Control what content gets mirrored from GitHub to Gitea.
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `MIRROR_RELEASES` | Mirror GitHub releases | `false` | `true`, `false` |
| `RELEASE_LIMIT` | Maximum number of releases to mirror per repository | `10` | Number (1-100) |
| `MIRROR_WIKI` | Mirror wiki content | `false` | `true`, `false` |
| `MIRROR_METADATA` | Master toggle for metadata mirroring | `false` | `true`, `false` |
| `MIRROR_ISSUES` | Mirror issues (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_PULL_REQUESTS` | Mirror pull requests (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_LABELS` | Mirror labels (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_MILESTONES` | Mirror milestones (requires MIRROR_METADATA=true) | `false` | `true`, `false` |
| `MIRROR_ISSUE_CONCURRENCY` | Number of issues processed in parallel. Set above `1` to speed up mirroring at the risk of out-of-order creation. | `3` | Integer ≥ 1 |
| `MIRROR_PULL_REQUEST_CONCURRENCY` | Number of pull requests processed in parallel. Values above `1` may cause ordering differences. | `5` | Integer ≥ 1 |
> **Ordering vs Throughput:** Metadata now mirrors sequentially by default to preserve chronology. Increase the concurrency variables only if you can tolerate minor out-of-order entries.
## Automation Configuration
Configure automatic scheduled mirroring.
### Basic Schedule Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_ENABLED` | Enable automatic mirroring. **When set to `true`, automatically imports and mirrors all repositories on startup** (v3.5.3+) | `false` | `true`, `false` |
| `SCHEDULE_INTERVAL` | Interval in seconds or cron expression. **Supports cron syntax for scheduled runs** (e.g., `"0 2 * * *"` for 2 AM daily) | `3600` | Number (seconds) or cron string |
| `DELAY` | Legacy: same as SCHEDULE_INTERVAL | `3600` | Number (seconds) |
> **🚀 Auto-Start Feature (v3.5.3+)**
> Setting either `SCHEDULE_ENABLED=true` or `GITEA_MIRROR_INTERVAL` triggers auto-start functionality where the service will:
> 1. **Import** all GitHub repositories on startup
> 2. **Mirror** them to Gitea immediately
> 3. **Continue syncing** at the configured interval
> 4. **Auto-discover** new repositories
> 5. **Clean up** deleted repositories (if configured)
>
> This eliminates the need for manual button clicks - perfect for Docker/Kubernetes deployments!
> **⏰ Scheduling with Cron Expressions**
> Use cron expressions in `SCHEDULE_INTERVAL` to run at specific times:
> - `"0 2 * * *"` - Daily at 2 AM
> - `"0 */6 * * *"` - Every 6 hours
> - `"0 0 * * 0"` - Weekly on Sunday at midnight
> - `"0 3 * * 1-5"` - Weekdays at 3 AM (Monday-Friday)
>
> This is useful for optimizing bandwidth usage during low-activity periods.
### Execution Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_CONCURRENT` | Allow concurrent mirror operations | `false` | `true`, `false` |
| `SCHEDULE_BATCH_SIZE` | Number of repos to process in parallel | `10` | Number |
| `SCHEDULE_PAUSE_BETWEEN_BATCHES` | Pause between batches (milliseconds) | `5000` | Number |
### Retry Configuration
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_RETRY_ATTEMPTS` | Number of retry attempts | `3` | Number |
| `SCHEDULE_RETRY_DELAY` | Delay between retries (milliseconds) | `60000` | Number |
| `SCHEDULE_TIMEOUT` | Max time for a mirror operation (milliseconds) | `3600000` | Number |
| `SCHEDULE_AUTO_RETRY` | Automatically retry failed operations | `true` | `true`, `false` |
### Update Detection
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `AUTO_IMPORT_REPOS` | Automatically discover and import new GitHub repositories during scheduled syncs | `true` | `true`, `false` |
| `AUTO_MIRROR_REPOS` | Automatically mirror newly imported repositories during scheduled syncs (no manual “Mirror All” required) | `false` | `true`, `false` |
| `SCHEDULE_ONLY_MIRROR_UPDATED` | Only mirror repos with updates | `false` | `true`, `false` |
| `SCHEDULE_UPDATE_INTERVAL` | Check for updates interval (milliseconds) | `86400000` | Number |
| `SCHEDULE_SKIP_RECENTLY_MIRRORED` | Skip recently mirrored repos | `true` | `true`, `false` |
| `SCHEDULE_RECENT_THRESHOLD` | Skip if mirrored within this time (milliseconds) | `3600000` | Number |
### Maintenance & Notifications
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `SCHEDULE_CLEANUP_BEFORE_MIRROR` | Run cleanup before mirroring | `false` | `true`, `false` |
| `SCHEDULE_NOTIFY_ON_FAILURE` | Send notifications on failure | `true` | `true`, `false` |
| `SCHEDULE_NOTIFY_ON_SUCCESS` | Send notifications on success | `false` | `true`, `false` |
| `SCHEDULE_LOG_LEVEL` | Logging level | `info` | `error`, `warn`, `info`, `debug` |
| `SCHEDULE_TIMEZONE` | Timezone for scheduling | `UTC` | Valid timezone string |
## Database Cleanup Configuration
Configure automatic cleanup of old events and data.
### Basic Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `CLEANUP_ENABLED` | Enable automatic cleanup | `false` | `true`, `false` |
| `CLEANUP_RETENTION_DAYS` | Days to keep events | `7` | Number |
### Repository Cleanup
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `CLEANUP_DELETE_FROM_GITEA` | Delete repositories from Gitea | `false` | `true`, `false` |
| `CLEANUP_DELETE_IF_NOT_IN_GITHUB` | Delete repos not found in GitHub (automatically enables cleanup) | `true` | `true`, `false` |
| `CLEANUP_ORPHANED_REPO_ACTION` | Action for orphaned repositories. **Note**: `archive` is recommended to preserve backups | `archive` | `skip`, `archive`, `delete` |
| `CLEANUP_DRY_RUN` | Test mode without actual deletion | `false` | `true`, `false` |
| `CLEANUP_PROTECTED_REPOS` | Comma-separated list of protected repository names | - | Comma-separated strings |
**🛡️ Safety Features (Backup Protection)**:
- **GitHub Failures Don't Delete Backups**: Cleanup is automatically skipped if GitHub API returns errors (404, 403, connection issues)
- **Archive Never Deletes**: The `archive` action ALWAYS preserves repository data, it never deletes
- **Graceful Degradation**: If marking as archived fails, the repository remains fully accessible in Gitea
- **The Purpose of Backups**: Your mirrors are preserved even when GitHub sources disappear - that's the whole point!
**Archive Behavior (Aligned with Gitea API)**:
- **Regular repositories**: Uses Gitea's native archive feature (PATCH `/repos/{owner}/{repo}` with `archived: true`)
- Makes repository read-only while preserving all data
- **Mirror repositories**: Uses rename strategy (Gitea API returns 422 for archiving mirrors)
- Renamed with `archived-` prefix for clear identification
- Description updated with preservation notice and timestamp
- Mirror interval set to 8760h (1 year) to minimize sync attempts
- Repository remains fully accessible and cloneable
- **Manual Sync Option**: Archived mirrors are still available on the Repositories page with automatic syncs disabled—use the `Manual Sync` action to refresh them on demand.
### Execution Settings
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `CLEANUP_BATCH_SIZE` | Number of items to process per batch | `10` | Number |
| `CLEANUP_PAUSE_BETWEEN_DELETES` | Pause between deletions (milliseconds) | `2000` | Number |
## Authentication Configuration
Configure authentication methods and SSO.
### Header Authentication (Reverse Proxy SSO)
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `HEADER_AUTH_ENABLED` | Enable header-based authentication | `false` | `true`, `false` |
| `HEADER_AUTH_USER_HEADER` | Header containing username | `X-Authentik-Username` | Header name |
| `HEADER_AUTH_EMAIL_HEADER` | Header containing email | `X-Authentik-Email` | Header name |
| `HEADER_AUTH_NAME_HEADER` | Header containing display name | `X-Authentik-Name` | Header name |
| `HEADER_AUTH_AUTO_PROVISION` | Auto-create users from headers | `false` | `true`, `false` |
| `HEADER_AUTH_ALLOWED_DOMAINS` | Comma-separated list of allowed email domains | - | Comma-separated domains |
## Docker Configuration
Settings specific to Docker deployments.
| Variable | Description | Default | Options |
|----------|-------------|---------|---------|
| `DOCKER_REGISTRY` | Docker registry URL | `ghcr.io` | Registry URL |
| `DOCKER_IMAGE` | Docker image name | `raylabshq/gitea-mirror:` | Image name |
| `DOCKER_TAG` | Docker image tag | `latest` | Tag name |
## Example Docker Compose Configuration
Here's an example of how to use these environment variables in a `docker-compose.yml` file:
```yaml
version: '3.8'
services:
gitea-mirror:
image: ghcr.io/raylabshq/gitea-mirror:latest
container_name: gitea-mirror
environment:
# Core Configuration
- NODE_ENV=production
- DATABASE_URL=file:data/gitea-mirror.db
- BETTER_AUTH_SECRET=your-secure-secret-here
# Primary access URL:
- BETTER_AUTH_URL=https://gitea-mirror.mydomain.tld
# Additional access URLs (local network + SSO providers):
# - BETTER_AUTH_TRUSTED_ORIGINS=http://10.10.20.45:4321,http://192.168.1.100:4321,https://auth.provider.com
# GitHub Configuration
- GITHUB_USERNAME=your-username
- GITHUB_TOKEN=ghp_your_token_here
- PRIVATE_REPOSITORIES=true
- MIRROR_STARRED=true
- SKIP_FORKS=false
# Gitea Configuration
- GITEA_URL=http://gitea:3000
- GITEA_USERNAME=admin
- GITEA_TOKEN=your-gitea-token
- GITEA_ORGANIZATION=github-mirrors
- GITEA_ORG_VISIBILITY=public
# Mirror Options
- MIRROR_RELEASES=true
- MIRROR_WIKI=true
- MIRROR_METADATA=true
- MIRROR_ISSUES=true
- MIRROR_PULL_REQUESTS=true
# Automation
- SCHEDULE_ENABLED=true
- SCHEDULE_INTERVAL=3600
# Cleanup
- CLEANUP_ENABLED=true
- CLEANUP_RETENTION_DAYS=30
volumes:
- ./data:/app/data
ports:
- "4321:4321"
```
## Authentication URL Configuration
### Multiple Access URLs
To allow access to Gitea Mirror through multiple URLs (e.g., local IP and public domain), you need to configure both server and client settings:
**Example Configuration:**
```bash
# Primary URL (required) - where the auth server is hosted
BETTER_AUTH_URL=https://gitea-mirror.mydomain.tld
# Client-side URL (optional) - tells the browser where to send auth requests
# Set this to your primary domain when accessing from different origins
PUBLIC_BETTER_AUTH_URL=https://gitea-mirror.mydomain.tld
# Additional trusted origins (optional) - origins allowed to make auth requests
BETTER_AUTH_TRUSTED_ORIGINS=http://10.10.20.45:4321,http://192.168.1.100:4321
```
This setup allows you to:
- Access via local network IP: `http://10.10.20.45:4321`
- Access via public domain: `https://gitea-mirror.mydomain.tld`
- Auth requests from the IP will be sent to the domain (via `PUBLIC_BETTER_AUTH_URL`)
- Each origin requires separate login due to browser cookie isolation
**Important:** When accessing from different origins (IP vs domain), you'll need to log in separately on each origin as cookies cannot be shared across different origins for security reasons.
### Trusted Origins
The `BETTER_AUTH_TRUSTED_ORIGINS` variable serves multiple purposes:
1. **SSO/OIDC Providers**: When using external authentication providers (Google, Authentik, Okta)
2. **Reverse Proxies**: When running behind nginx, Traefik, or other proxies
3. **Cross-Origin Requests**: When the frontend and backend are on different domains
4. **Development**: When testing from different URLs
**Example Scenarios:**
```bash
# For Authentik SSO integration
BETTER_AUTH_TRUSTED_ORIGINS=https://authentik.company.com,https://auth.company.com
# For reverse proxy setup
BETTER_AUTH_TRUSTED_ORIGINS=https://proxy.internal,https://public.domain.com
# For development with multiple environments
BETTER_AUTH_TRUSTED_ORIGINS=http://localhost:3000,http://192.168.1.100:3000
```
**Important Notes:**
- All URLs from `BETTER_AUTH_URL` are automatically trusted
- URLs must be complete with protocol (http/https)
- Multiple origins are separated by commas
- No trailing slashes needed
## Notes
1. **First Run**: Environment variables are loaded when the container starts. The configuration is applied after the first user account is created.
2. **UI Priority**: Manual changes made through the web UI will be preserved. Environment variables only set values for empty fields.
3. **Token Security**: All tokens are encrypted before being stored in the database.
4. **Auto-Enabling Features**: Certain environment variables automatically enable features when set:
- `GITEA_MIRROR_INTERVAL` - Automatically enables scheduled mirroring
- `CLEANUP_DELETE_IF_NOT_IN_GITHUB=true` - Automatically enables repository cleanup
- `SCHEDULE_INTERVAL` or `DELAY` - Automatically enables the scheduler
5. **Backward Compatibility**: The `DELAY` variable is maintained for backward compatibility but `SCHEDULE_INTERVAL` is preferred.
6. **Required Scopes**: The GitHub token requires the following scopes:
- `repo` (full control of private repositories)
- `admin:org` (read organization data)
- Additional scopes may be required for specific features
For more examples and detailed configuration, see the `.env.example` file in the repository.

View File

@@ -0,0 +1,249 @@
# Graceful Shutdown and Enhanced Job Recovery
This document describes the graceful shutdown and enhanced job recovery capabilities implemented in gitea-mirror v2.8.0+.
## Overview
The gitea-mirror application now includes comprehensive graceful shutdown handling and enhanced job recovery mechanisms designed specifically for containerized environments. These features ensure:
- **No data loss** during container restarts or shutdowns
- **Automatic job resumption** after application restarts
- **Clean termination** of all active processes and connections
- **Container-aware design** optimized for Docker/LXC deployments
## Features
### 1. Graceful Shutdown Manager
The shutdown manager (`src/lib/shutdown-manager.ts`) provides centralized coordination of application termination:
#### Key Capabilities:
- **Active Job Tracking**: Monitors all running mirroring/sync jobs
- **State Persistence**: Saves job progress to database before shutdown
- **Callback System**: Allows services to register cleanup functions
- **Timeout Protection**: Prevents hanging shutdowns with configurable timeouts
- **Signal Coordination**: Works with signal handlers for proper container lifecycle
#### Configuration:
- **Shutdown Timeout**: 30 seconds maximum (configurable)
- **Job Save Timeout**: 10 seconds per job (configurable)
### 2. Signal Handlers
The signal handler system (`src/lib/signal-handlers.ts`) ensures proper response to container lifecycle events:
#### Supported Signals:
- **SIGTERM**: Docker stop, Kubernetes pod termination
- **SIGINT**: Ctrl+C, manual interruption
- **SIGHUP**: Terminal hangup, service reload
- **Uncaught Exceptions**: Emergency shutdown on critical errors
- **Unhandled Rejections**: Graceful handling of promise failures
### 3. Enhanced Job Recovery
Building on the existing recovery system, new enhancements include:
#### Shutdown-Aware Processing:
- Jobs check for shutdown signals during execution
- Automatic state saving when shutdown is detected
- Proper job status management (interrupted vs failed)
#### Container Integration:
- Docker entrypoint script forwards signals correctly
- Startup recovery runs before main application
- Recovery timeouts prevent startup delays
## Usage
### Basic Operation
The graceful shutdown system is automatically initialized when the application starts. No manual configuration is required for basic operation.
### Testing
Test the graceful shutdown functionality:
```bash
# Run the integration test
bun run test-shutdown
# Clean up test data
bun run test-shutdown-cleanup
# Run unit tests
bun test src/lib/shutdown-manager.test.ts
bun test src/lib/signal-handlers.test.ts
```
### Manual Testing
1. **Start the application**:
```bash
bun run dev
# or in production
bun run start
```
2. **Start a mirroring job** through the web interface
3. **Send shutdown signal**:
```bash
# Send SIGTERM (recommended)
kill -TERM <process_id>
# Or use Ctrl+C for SIGINT
```
4. **Verify job state** is saved and can be resumed on restart
### Container Testing
Test with Docker:
```bash
# Build and run container
docker build -t gitea-mirror .
docker run -d --name test-shutdown gitea-mirror
# Start a job, then stop container
docker stop test-shutdown
# Restart and verify recovery
docker start test-shutdown
docker logs test-shutdown
```
## Implementation Details
### Shutdown Flow
1. **Signal Reception**: Signal handlers detect termination request
2. **Shutdown Initiation**: Shutdown manager begins graceful termination
3. **Job State Saving**: All active jobs save current progress to database
4. **Service Cleanup**: Registered callbacks stop background services
5. **Connection Cleanup**: Database connections and resources are released
6. **Process Termination**: Application exits with appropriate code
### Job State Management
During shutdown, active jobs are updated with:
- `inProgress: false` - Mark as not currently running
- `lastCheckpoint: <timestamp>` - Record shutdown time
- `message: "Job interrupted by application shutdown - will resume on restart"`
- Status remains as `"imported"` (not `"failed"`) to enable recovery
### Recovery Integration
The existing recovery system automatically detects and resumes interrupted jobs:
- Jobs with `inProgress: false` and incomplete status are candidates for recovery
- Recovery runs during application startup (before serving requests)
- Jobs resume from their last checkpoint with remaining items
## Configuration
### Environment Variables
```bash
# Optional: Adjust shutdown timeout (default: 30000ms)
SHUTDOWN_TIMEOUT=30000
# Optional: Adjust job save timeout (default: 10000ms)
JOB_SAVE_TIMEOUT=10000
```
### Docker Configuration
The Docker entrypoint script includes proper signal handling:
```dockerfile
# Signals are forwarded to the application process
# SIGTERM is handled gracefully with 30-second timeout
# Container stops cleanly without force-killing processes
```
### Kubernetes Configuration
For Kubernetes deployments, configure appropriate termination grace period:
```yaml
apiVersion: v1
kind: Pod
spec:
terminationGracePeriodSeconds: 45 # Allow time for graceful shutdown
containers:
- name: gitea-mirror
# ... other configuration
```
## Monitoring and Debugging
### Logs
The application provides detailed logging during shutdown:
```
🛑 Graceful shutdown initiated by signal: SIGTERM
📊 Shutdown status: 2 active jobs, 1 callbacks
📝 Step 1: Saving active job states...
Saving state for job abc-123...
✅ Saved state for job abc-123
🔧 Step 2: Executing shutdown callbacks...
✅ Shutdown callback 1 completed
💾 Step 3: Closing database connections...
✅ Graceful shutdown completed successfully
```
### Status Endpoints
Check shutdown manager status via API:
```bash
# Get current status (if application is running)
curl http://localhost:4321/api/health
```
### Troubleshooting
**Problem**: Jobs not resuming after restart
- **Check**: Startup recovery logs for errors
- **Verify**: Database contains interrupted jobs with correct status
- **Test**: Run `bun run startup-recovery` manually
**Problem**: Shutdown timeout reached
- **Check**: Job complexity and database performance
- **Adjust**: Increase `SHUTDOWN_TIMEOUT` environment variable
- **Monitor**: Database connection and disk I/O during shutdown
**Problem**: Container force-killed
- **Check**: Container orchestrator termination grace period
- **Adjust**: Increase grace period to allow shutdown completion
- **Monitor**: Application shutdown logs for timing issues
## Best Practices
### Development
- Always test graceful shutdown during development
- Use the provided test scripts to verify functionality
- Monitor logs for shutdown timing and job state persistence
### Production
- Set appropriate container termination grace periods
- Monitor shutdown logs for performance issues
- Use health checks to verify application readiness after restart
- Consider job complexity when planning maintenance windows
### Monitoring
- Track job recovery success rates
- Monitor shutdown duration metrics
- Alert on forced terminations or recovery failures
- Log analysis for shutdown pattern optimization
## Future Enhancements
Planned improvements for future versions:
1. **Configurable Timeouts**: Environment variable configuration for all timeouts
2. **Shutdown Metrics**: Prometheus metrics for shutdown performance
3. **Progressive Shutdown**: Graceful degradation of service capabilities
4. **Job Prioritization**: Priority-based job saving during shutdown
5. **Health Check Integration**: Readiness probes during shutdown process

View File

@@ -0,0 +1,486 @@
# Nix Deployment Guide
This guide covers deploying Gitea Mirror using Nix flakes. The Nix deployment follows the same minimal configuration philosophy as `docker-compose.alt.yml` - secrets are auto-generated, and everything else can be configured via the web UI.
## Prerequisites
- Nix 2.4+ installed
- For NixOS module: NixOS 23.05+
### Enable Flakes (Recommended)
To enable flakes permanently and avoid typing flags, add to `/etc/nix/nix.conf` or `~/.config/nix/nix.conf`:
```
experimental-features = nix-command flakes
```
**Note:** If you don't enable flakes globally, add `--extra-experimental-features 'nix-command flakes'` to all nix commands shown below.
## Quick Start (Zero Configuration!)
### Run Immediately - No Setup Required
```bash
# Run directly from the flake (local)
nix run --extra-experimental-features 'nix-command flakes' .#gitea-mirror
# Or from GitHub (once published)
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# If you have flakes enabled globally, simply:
nix run .#gitea-mirror
```
That's it! On first run:
- Secrets (`BETTER_AUTH_SECRET` and `ENCRYPTION_SECRET`) are auto-generated
- Database is automatically created and initialized
- Startup recovery and repair scripts run automatically
- Access the web UI at http://localhost:4321
Everything else (GitHub credentials, Gitea settings, mirror options) is configured through the web interface after signup.
### Development Environment
```bash
# Enter development shell with all dependencies
nix develop --extra-experimental-features 'nix-command flakes'
# Or use direnv for automatic environment loading (handles flags automatically)
echo "use flake" > .envrc
direnv allow
```
### Build and Install
```bash
# Build the package
nix build --extra-experimental-features 'nix-command flakes'
# Run the built package
./result/bin/gitea-mirror
# Install to your profile
nix profile install --extra-experimental-features 'nix-command flakes' .#gitea-mirror
```
## What Happens on First Run?
Following the same pattern as the Docker deployment, the Nix package automatically:
1. **Creates data directory**: `~/.local/share/gitea-mirror` (or `$DATA_DIR`)
2. **Generates secrets** (stored securely in data directory):
- `BETTER_AUTH_SECRET` - Session authentication (32-char hex)
- `ENCRYPTION_SECRET` - Token encryption (48-char base64)
3. **Initializes database**: SQLite database with Drizzle migrations
4. **Runs startup scripts**:
- Environment configuration loader
- Crash recovery for interrupted jobs
- Repository status repair
5. **Starts the application** with graceful shutdown handling
## NixOS Module - Minimal Deployment
### Simplest Possible Configuration
Add to your NixOS configuration (`/etc/nixos/configuration.nix`):
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
};
outputs = { nixpkgs, gitea-mirror, ... }: {
nixosConfigurations.your-hostname = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
gitea-mirror.nixosModules.default
{
# That's it! Just enable the service
services.gitea-mirror.enable = true;
}
];
};
};
}
```
Apply with:
```bash
sudo nixos-rebuild switch
```
Access at http://localhost:4321, sign up (first user is admin), and configure everything via the web UI.
### Production Configuration
For production with custom domain and firewall:
```nix
{
services.gitea-mirror = {
enable = true;
host = "0.0.0.0";
port = 4321;
betterAuthUrl = "https://mirror.example.com";
betterAuthTrustedOrigins = "https://mirror.example.com";
openFirewall = true;
};
# Optional: Use with nginx reverse proxy
services.nginx = {
enable = true;
virtualHosts."mirror.example.com" = {
locations."/" = {
proxyPass = "http://127.0.0.1:4321";
proxyWebsockets = true;
};
enableACME = true;
forceSSL = true;
};
};
}
```
### Advanced: Manual Secret Management
If you prefer to manage secrets manually (e.g., with sops-nix or agenix):
1. Create a secrets file:
```bash
# /var/lib/gitea-mirror/secrets.env
BETTER_AUTH_SECRET=your-32-character-minimum-secret-key-here
ENCRYPTION_SECRET=your-encryption-secret-here
```
2. Reference it in your configuration:
```nix
{
services.gitea-mirror = {
enable = true;
environmentFile = "/var/lib/gitea-mirror/secrets.env";
};
}
```
### Full Configuration Options
```nix
{
services.gitea-mirror = {
enable = true;
package = gitea-mirror.packages.x86_64-linux.default; # Override package
dataDir = "/var/lib/gitea-mirror";
user = "gitea-mirror";
group = "gitea-mirror";
host = "0.0.0.0";
port = 4321;
betterAuthUrl = "https://mirror.example.com";
betterAuthTrustedOrigins = "https://mirror.example.com";
# Concurrency controls (match docker-compose.alt.yml)
mirrorIssueConcurrency = 3; # Set to 1 for perfect chronological order
mirrorPullRequestConcurrency = 5; # Set to 1 for perfect chronological order
environmentFile = null; # Optional secrets file
openFirewall = true;
};
}
```
## Service Management (NixOS)
```bash
# Start the service
sudo systemctl start gitea-mirror
# Stop the service
sudo systemctl stop gitea-mirror
# Restart the service
sudo systemctl restart gitea-mirror
# Check status
sudo systemctl status gitea-mirror
# View logs
sudo journalctl -u gitea-mirror -f
# Health check
curl http://localhost:4321/api/health
```
## Environment Variables
All variables from `docker-compose.alt.yml` are supported:
```bash
# === AUTO-GENERATED (Don't set unless you want specific values) ===
BETTER_AUTH_SECRET # Auto-generated, stored in data dir
ENCRYPTION_SECRET # Auto-generated, stored in data dir
# === CORE SETTINGS (Have good defaults) ===
DATA_DIR="$HOME/.local/share/gitea-mirror"
DATABASE_URL="file:$DATA_DIR/gitea-mirror.db"
HOST="0.0.0.0"
PORT="4321"
NODE_ENV="production"
# === BETTER AUTH (Override for custom domains) ===
BETTER_AUTH_URL="http://localhost:4321"
BETTER_AUTH_TRUSTED_ORIGINS="http://localhost:4321"
PUBLIC_BETTER_AUTH_URL="http://localhost:4321"
# === CONCURRENCY CONTROLS ===
MIRROR_ISSUE_CONCURRENCY=3 # Default: 3 (set to 1 for perfect order)
MIRROR_PULL_REQUEST_CONCURRENCY=5 # Default: 5 (set to 1 for perfect order)
# === CONFIGURE VIA WEB UI (Not needed at startup) ===
# GitHub credentials, Gitea settings, mirror options, scheduling, etc.
# All configured after signup through the web interface
```
## Database Management
The Nix package includes a database management helper:
```bash
# Initialize database (done automatically on first run)
gitea-mirror-db init
# Check database health
gitea-mirror-db check
# Fix database issues
gitea-mirror-db fix
# Reset users
gitea-mirror-db reset-users
```
## Home Manager Integration
For single-user deployments:
```nix
{ config, pkgs, ... }:
let
gitea-mirror = (import (fetchTarball "https://github.com/RayLabsHQ/gitea-mirror/archive/main.tar.gz")).packages.${pkgs.system}.default;
in {
home.packages = [ gitea-mirror ];
# Optional: Run as user service
systemd.user.services.gitea-mirror = {
Unit = {
Description = "Gitea Mirror Service";
After = [ "network.target" ];
};
Service = {
Type = "simple";
ExecStart = "${gitea-mirror}/bin/gitea-mirror";
Restart = "always";
Environment = [
"DATA_DIR=%h/.local/share/gitea-mirror"
"HOST=127.0.0.1"
"PORT=4321"
];
};
Install = {
WantedBy = [ "default.target" ];
};
};
}
```
## Docker Image from Nix (Optional)
You can also use Nix to create a Docker image:
```nix
# Add to flake.nix packages section
dockerImage = pkgs.dockerTools.buildLayeredImage {
name = "gitea-mirror";
tag = "latest";
contents = [ self.packages.${system}.default pkgs.cacert pkgs.openssl ];
config = {
Cmd = [ "${self.packages.${system}.default}/bin/gitea-mirror" ];
ExposedPorts = { "4321/tcp" = {}; };
Env = [
"DATA_DIR=/data"
"DATABASE_URL=file:/data/gitea-mirror.db"
];
Volumes = { "/data" = {}; };
};
};
```
Build and load:
```bash
nix build --extra-experimental-features 'nix-command flakes' .#dockerImage
docker load < result
docker run -p 4321:4321 -v gitea-mirror-data:/data gitea-mirror:latest
```
## Comparison: Docker vs Nix
Both deployment methods follow the same philosophy:
| Feature | Docker Compose | Nix |
|---------|---------------|-----|
| **Configuration** | Minimal (only BETTER_AUTH_SECRET) | Zero config (auto-generated) |
| **Secret Generation** | Auto-generated & persisted | Auto-generated & persisted |
| **Database Init** | Automatic on first run | Automatic on first run |
| **Startup Scripts** | Runs recovery/repair/env-config | Runs recovery/repair/env-config |
| **Graceful Shutdown** | Signal handling in entrypoint | Signal handling in wrapper |
| **Health Check** | Docker healthcheck | systemd timer (optional) |
| **Updates** | `docker pull` | `nix flake update && nixos-rebuild` |
## Troubleshooting
### Check Auto-Generated Secrets
```bash
# For standalone
cat ~/.local/share/gitea-mirror/.better_auth_secret
cat ~/.local/share/gitea-mirror/.encryption_secret
# For NixOS service
sudo cat /var/lib/gitea-mirror/.better_auth_secret
sudo cat /var/lib/gitea-mirror/.encryption_secret
```
### Database Issues
```bash
# Check if database exists
ls -la ~/.local/share/gitea-mirror/gitea-mirror.db
# Reinitialize (deletes all data!)
rm ~/.local/share/gitea-mirror/gitea-mirror.db
gitea-mirror-db init
```
### Permission Issues (NixOS)
```bash
sudo chown -R gitea-mirror:gitea-mirror /var/lib/gitea-mirror
sudo chmod 700 /var/lib/gitea-mirror
```
### Port Already in Use
```bash
# Change port
export PORT=8080
gitea-mirror
# Or in NixOS config
services.gitea-mirror.port = 8080;
```
### View Startup Logs
```bash
# Standalone (verbose output on console)
gitea-mirror
# NixOS service
sudo journalctl -u gitea-mirror -f --since "5 minutes ago"
```
## Updating
### Standalone Installation
```bash
# Update flake lock
nix flake update --extra-experimental-features 'nix-command flakes'
# Rebuild
nix build --extra-experimental-features 'nix-command flakes'
# Or update profile
nix profile upgrade --extra-experimental-features 'nix-command flakes' gitea-mirror
```
### NixOS
```bash
# Update input
sudo nix flake lock --update-input gitea-mirror --extra-experimental-features 'nix-command flakes'
# Rebuild system
sudo nixos-rebuild switch --flake .#your-hostname
```
## Migration from Docker
To migrate from Docker to Nix while keeping your data:
1. **Stop Docker container:**
```bash
docker-compose -f docker-compose.alt.yml down
```
2. **Copy data directory:**
```bash
# For standalone
cp -r ./data ~/.local/share/gitea-mirror
# For NixOS
sudo cp -r ./data /var/lib/gitea-mirror
sudo chown -R gitea-mirror:gitea-mirror /var/lib/gitea-mirror
```
3. **Copy secrets (if you want to keep them):**
```bash
# Extract from Docker volume
docker run --rm -v gitea-mirror_data:/data alpine \
cat /data/.better_auth_secret > better_auth_secret
docker run --rm -v gitea-mirror_data:/data alpine \
cat /data/.encryption_secret > encryption_secret
# Copy to new location
cp better_auth_secret ~/.local/share/gitea-mirror/.better_auth_secret
cp encryption_secret ~/.local/share/gitea-mirror/.encryption_secret
chmod 600 ~/.local/share/gitea-mirror/.*_secret
```
4. **Start Nix version:**
```bash
gitea-mirror
```
## CI/CD Integration
Example GitHub Actions workflow (see `.github/workflows/nix-build.yml`):
```yaml
name: Nix Build
on: [push, pull_request]
permissions:
contents: read
jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix flake check
- run: nix build --print-build-logs
```
This uses:
- **Determinate Nix Installer** - Fast, reliable Nix installation with flakes enabled by default
- **Magic Nix Cache** - Free caching using GitHub Actions cache (no account needed)
## Resources
- [Nix Manual](https://nixos.org/manual/nix/stable/)
- [NixOS Options Search](https://search.nixos.org/options)
- [Nix Pills Tutorial](https://nixos.org/guides/nix-pills/)
- [Project Documentation](../README.md)
- [Docker Deployment](../docker-compose.alt.yml) - Equivalent minimal config

View File

@@ -0,0 +1,322 @@
# Nix Package Distribution Guide
This guide explains how Gitea Mirror is distributed via Nix and how users can consume it.
## Distribution Methods
### Method 1: Direct GitHub Usage (Zero Infrastructure)
**No CI, releases, or setup needed!** Users can consume directly from GitHub:
```bash
# Latest from main branch
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Pin to specific commit
nix run github:RayLabsHQ/gitea-mirror/abc123def
# Pin to git tag
nix run github:RayLabsHQ/gitea-mirror/v3.8.11
```
**How it works:**
1. Nix fetches the repository from GitHub
2. Nix reads `flake.nix` and `flake.lock`
3. Nix builds the package locally on the user's machine
4. Package is cached in `/nix/store` for reuse
**Pros:**
- Zero infrastructure needed
- Works immediately after pushing code
- Users always get reproducible builds
**Cons:**
- Users must build from source (slower first time)
- Requires build dependencies (Bun, etc.)
---
### Method 2: CI Build Caching
The GitHub Actions workflow uses **Magic Nix Cache** (by Determinate Systems) to cache builds:
- **Zero configuration required** - no accounts or tokens needed
- **Automatic** - CI workflow handles everything
- **Uses GitHub Actions cache** - fast, reliable, free
#### How It Works:
1. GitHub Actions builds the package on each push/PR
2. Build artifacts are cached in GitHub Actions cache
3. Subsequent builds reuse cached dependencies (faster CI)
Note: This caches CI builds. Users still build locally, but the flake.lock ensures reproducibility.
---
### Method 3: nixpkgs Submission (Official Distribution)
Submit to the official Nix package repository for maximum visibility.
#### Process:
1. **Prepare package** (already done with `flake.nix`)
2. **Test thoroughly**
3. **Submit PR to nixpkgs:** https://github.com/NixOS/nixpkgs
#### User Experience:
```bash
# After acceptance into nixpkgs
nix run nixpkgs#gitea-mirror
# NixOS configuration
environment.systemPackages = [ pkgs.gitea-mirror ];
```
**Pros:**
- Maximum discoverability (official repo)
- Trusted by Nix community
- Included in NixOS search
- Binary caching by cache.nixos.org
**Cons:**
- Submission/review process
- Must follow nixpkgs guidelines
- Updates require PRs
---
## Current Distribution Strategy
### Phase 1: Direct GitHub (Immediate) ✅
Already working! Users can:
```bash
nix run github:RayLabsHQ/gitea-mirror
```
### Phase 2: CI Build Validation ✅
GitHub Actions workflow validates builds on every push/PR:
- Uses Magic Nix Cache for fast CI builds
- Tests on both Linux and macOS
- No setup required - works automatically
### Phase 3: Version Releases (Optional)
Tag releases for version pinning:
```bash
git tag v3.8.11
git push origin v3.8.11
# Users can then pin:
nix run github:RayLabsHQ/gitea-mirror/v3.8.11
```
### Phase 4: nixpkgs Submission (Long Term)
Once package is stable and well-tested, submit to nixpkgs.
---
## User Documentation
### For Users: How to Install
Add this to your `docs/NIX_DEPLOYMENT.md`:
#### Option 1: Direct Install (No Configuration)
```bash
# Run immediately
nix run --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
# Install to profile
nix profile install --extra-experimental-features 'nix-command flakes' github:RayLabsHQ/gitea-mirror
```
#### Option 2: Pin to Specific Version
```bash
# Pin to git tag
nix run github:RayLabsHQ/gitea-mirror/v3.8.11
# Pin to commit
nix run github:RayLabsHQ/gitea-mirror/abc123def
# Lock in flake.nix
inputs.gitea-mirror.url = "github:RayLabsHQ/gitea-mirror/v3.8.11";
```
#### Option 3: NixOS Configuration
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
gitea-mirror.url = "github:RayLabsHQ/gitea-mirror";
# Or pin to version:
# gitea-mirror.url = "github:RayLabsHQ/gitea-mirror/v3.8.11";
};
outputs = { nixpkgs, gitea-mirror, ... }: {
nixosConfigurations.your-host = nixpkgs.lib.nixosSystem {
modules = [
gitea-mirror.nixosModules.default
{
services.gitea-mirror = {
enable = true;
betterAuthUrl = "https://mirror.example.com";
openFirewall = true;
};
}
];
};
};
}
```
---
## Maintaining the Distribution
### Releasing New Versions
```bash
# 1. Update version in package.json
vim package.json # Update version field
# 2. Update flake.nix version (line 17)
vim flake.nix # Update version = "X.Y.Z";
# 3. Commit changes
git add package.json flake.nix
git commit -m "chore: bump version to vX.Y.Z"
# 4. Create git tag
git tag vX.Y.Z
git push origin main
git push origin vX.Y.Z
# 5. GitHub Actions builds and caches automatically
```
Users can then pin to the new version:
```bash
nix run github:RayLabsHQ/gitea-mirror/vX.Y.Z
```
### Updating Flake Lock
The `flake.lock` file pins all dependencies. Update it periodically:
```bash
# Update all inputs
nix flake update
# Update specific input
nix flake lock --update-input nixpkgs
# Test after update
nix build
nix flake check
# Commit the updated lock file
git add flake.lock
git commit -m "chore: update flake dependencies"
git push
```
---
## Troubleshooting Distribution Issues
### Users Report Build Failures
1. **Check GitHub Actions:** Ensure CI is passing
2. **Test locally:** `nix flake check`
3. **Check flake.lock:** May need update if dependencies changed
### CI Cache Not Working
1. **Check workflow logs:** Review GitHub Actions for errors
2. **Clear cache:** GitHub Actions → Caches → Delete relevant cache
3. **Verify flake.lock:** May need `nix flake update` if dependencies changed
### Version Pinning Not Working
```bash
# Verify tag exists
git tag -l
# Ensure tag is pushed
git ls-remote --tags origin
# Test specific tag
nix run github:RayLabsHQ/gitea-mirror/v3.8.11
```
---
## Advanced: Custom Binary Cache
If you prefer self-hosting instead of Cachix:
### Option 1: S3-Compatible Storage
```nix
# Generate signing key
nix-store --generate-binary-cache-key cache.example.com cache-priv-key.pem cache-pub-key.pem
# Push to S3
nix copy --to s3://my-nix-cache?region=us-east-1 $(nix-build)
```
Users configure:
```nix
substituters = https://my-bucket.s3.amazonaws.com/nix-cache
trusted-public-keys = cache.example.com:BASE64_PUBLIC_KEY
```
### Option 2: Self-Hosted Nix Store
Run `nix-serve` on your server:
```bash
# On server
nix-serve -p 8080
# Behind nginx/caddy
proxy_pass http://localhost:8080;
```
Users configure:
```nix
substituters = https://cache.example.com
trusted-public-keys = YOUR_KEY
```
---
## Comparison: Distribution Methods
| Method | Setup Time | User Speed | Cost | Discoverability |
|--------|-----------|------------|------|-----------------|
| Direct GitHub | 0 min | Slow (build) | Free | Low |
| nixpkgs | Hours/days | Fast (binary) | Free | High |
| Self-hosted cache | 30+ min | Fast (binary) | Server cost | Low |
**Current approach:** Direct GitHub consumption with CI validation using Magic Nix Cache. Users build locally (reproducible via flake.lock). Consider **nixpkgs** submission for maximum reach once the package is mature.
---
## Resources
- [Nix Flakes Documentation](https://nixos.wiki/wiki/Flakes)
- [Magic Nix Cache](https://github.com/DeterminateSystems/magic-nix-cache-action)
- [nixpkgs Contributing Guide](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md)
- [Nix Binary Cache Setup](https://nixos.org/manual/nix/stable/package-management/binary-cache-substituter.html)

View File

@@ -0,0 +1,39 @@
# Gitea Mirror Documentation
This folder contains engineering and operations references for the open-source Gitea Mirror project. Each guide focuses on the parts of the system that still require bespoke explanation beyond the in-app help and the main `README.md`.
## Available Guides
### Core workflow
- **[DEVELOPMENT_WORKFLOW.md](./DEVELOPMENT_WORKFLOW.md)** Set up a local environment, run scripts, and understand the repo layout (app + marketing site).
- **[ENVIRONMENT_VARIABLES.md](./ENVIRONMENT_VARIABLES.md)** Complete reference for every configuration flag supported by the app and Docker images.
### Reliability & recovery
- **[GRACEFUL_SHUTDOWN.md](./GRACEFUL_SHUTDOWN.md)** How signal handling, shutdown coordination, and job persistence work in v3.
- **[RECOVERY_IMPROVEMENTS.md](./RECOVERY_IMPROVEMENTS.md)** Deep dive into the startup recovery workflow and supporting scripts.
### Authentication
- **[SSO-OIDC-SETUP.md](./SSO-OIDC-SETUP.md)** Configure OIDC/SSO providers through the admin UI.
- **[SSO_TESTING.md](./SSO_TESTING.md)** Recipes for local and staging SSO testing (Google, Keycloak, mock providers).
If you are looking for customer-facing playbooks, see the MDX use cases under `www/src/pages/use-cases/`.
## Quick start for local development
```bash
git clone https://github.com/RayLabsHQ/gitea-mirror.git
cd gitea-mirror
bun run setup # installs deps and seeds the SQLite DB
bun run dev # starts the Astro/Bun app on http://localhost:4321
```
The first user you create locally becomes the administrator. All other configuration—GitHub owners, Gitea targets, scheduling, cleanup—is done through the **Configuration** screen in the UI.
## Contributing & support
- 🎯 Contribution guide: [../CONTRIBUTING.md](../CONTRIBUTING.md)
- 📘 Code of conduct: [../CODE_OF_CONDUCT.md](../CODE_OF_CONDUCT.md)
- 🐞 Issues & feature requests: <https://github.com/RayLabsHQ/gitea-mirror/issues>
- 💬 Discussions: <https://github.com/RayLabsHQ/gitea-mirror/discussions>
Security disclosures should follow the process in [../SECURITY.md](../SECURITY.md).

View File

@@ -0,0 +1,170 @@
# Job Recovery and Resume Process Improvements
This document outlines the comprehensive improvements made to the job recovery and resume process to make it more robust to application restarts, container restarts, and application crashes.
## Problems Addressed
The original recovery system had several critical issues:
1. **Middleware-based initialization**: Recovery only ran when the first request came in
2. **Database connection issues**: No validation of database connectivity before recovery attempts
3. **Limited error handling**: Insufficient error handling for various failure scenarios
4. **No startup recovery**: No mechanism to handle recovery before serving requests
5. **Incomplete job state management**: Jobs could remain in inconsistent states
6. **No retry mechanisms**: Single-attempt recovery with no fallback strategies
## Improvements Implemented
### 1. Enhanced Recovery System (`src/lib/recovery.ts`)
#### New Features:
- **Database connection validation** before attempting recovery
- **Stale job cleanup** for jobs older than 24 hours
- **Retry mechanisms** with configurable attempts and delays
- **Individual job error handling** to prevent one failed job from stopping recovery
- **Recovery state tracking** to prevent concurrent recovery attempts
- **Enhanced logging** with detailed job information
#### Key Functions:
- `initializeRecovery()` - Main recovery function with enhanced error handling
- `validateDatabaseConnection()` - Ensures database is accessible
- `cleanupStaleJobs()` - Removes jobs that are too old to recover
- `getRecoveryStatus()` - Returns current recovery system status
- `forceRecovery()` - Bypasses recent attempt checks
- `hasJobsNeedingRecovery()` - Checks if recovery is needed
### 2. Startup Recovery Script (`scripts/startup-recovery.ts`)
A dedicated script that runs recovery before the application starts serving requests:
#### Features:
- **Timeout protection** (default: 30 seconds)
- **Force recovery option** to bypass recent attempt checks
- **Graceful signal handling** (SIGINT, SIGTERM)
- **Detailed logging** with progress indicators
- **Exit codes** for different scenarios (success, warnings, errors)
#### Usage:
```bash
bun scripts/startup-recovery.ts [--force] [--timeout=30000]
```
### 3. Improved Middleware (`src/middleware.ts`)
The middleware now serves as a fallback recovery mechanism:
#### Changes:
- **Checks if recovery is needed** before attempting
- **Shorter timeout** (15 seconds) for request-time recovery
- **Better error handling** with status logging
- **Prevents multiple attempts** with proper state tracking
### 4. Enhanced Database Queries (`src/lib/helpers.ts`)
#### Improvements:
- **Proper Drizzle ORM syntax** for all database queries
- **Enhanced interrupted job detection** with multiple criteria:
- Jobs with no recent checkpoint (10+ minutes)
- Jobs running too long (2+ hours)
- **Detailed logging** of found interrupted jobs
- **Better error handling** for database operations
### 5. Docker Integration (`docker-entrypoint.sh`)
#### Changes:
- **Automatic startup recovery** runs before application start
- **Exit code handling** with appropriate logging
- **Fallback mechanisms** if recovery script is not found
- **Non-blocking execution** - application starts even if recovery fails
### 6. Health Check Integration (`src/pages/api/health.ts`)
#### New Features:
- **Recovery system status** in health endpoint
- **Job recovery metrics** (jobs needing recovery, recovery in progress)
- **Overall health status** considers recovery state
- **Detailed recovery information** for monitoring
### 7. Testing Infrastructure (`scripts/test-recovery.ts`)
A comprehensive test script to verify recovery functionality:
#### Features:
- **Creates test interrupted jobs** with realistic scenarios
- **Verifies recovery detection** and execution
- **Checks final job states** after recovery
- **Cleanup functionality** for test data
- **Comprehensive logging** of test progress
## Configuration Options
### Recovery System Options:
- `maxRetries`: Number of recovery attempts (default: 3)
- `retryDelay`: Delay between attempts in ms (default: 5000)
- `skipIfRecentAttempt`: Skip if recent attempt made (default: true)
### Startup Recovery Options:
- `--force`: Force recovery even if recent attempt was made
- `--timeout`: Maximum time to wait for recovery (default: 30000ms)
## Usage Examples
### Manual Recovery:
```bash
# Run startup recovery
bun run startup-recovery
# Force recovery
bun run startup-recovery-force
# Test recovery system
bun run test-recovery
# Clean up test data
bun run test-recovery-cleanup
```
### Programmatic Usage:
```typescript
import { initializeRecovery, hasJobsNeedingRecovery } from '@/lib/recovery';
// Check if recovery is needed
const needsRecovery = await hasJobsNeedingRecovery();
// Run recovery with custom options
const success = await initializeRecovery({
maxRetries: 5,
retryDelay: 3000,
skipIfRecentAttempt: false
});
```
## Monitoring and Observability
### Health Check Endpoint:
- **URL**: `/api/health`
- **Recovery Status**: Included in response
- **Monitoring**: Can be used with external monitoring systems
### Log Messages:
- **Startup**: Clear indicators of recovery attempts and results
- **Progress**: Detailed logging of recovery steps
- **Errors**: Comprehensive error information for debugging
## Benefits
1. **Reliability**: Jobs are automatically recovered after application restarts
2. **Resilience**: Multiple retry mechanisms and fallback strategies
3. **Observability**: Comprehensive logging and health check integration
4. **Performance**: Efficient detection and processing of interrupted jobs
5. **Maintainability**: Clear separation of concerns and modular design
6. **Testing**: Built-in testing infrastructure for verification
## Migration Notes
- **Backward Compatible**: All existing functionality is preserved
- **Automatic**: Recovery runs automatically on startup
- **Configurable**: All timeouts and retry counts can be adjusted
- **Monitoring**: Health checks now include recovery status
This comprehensive improvement ensures that the gitea-mirror application can reliably handle job recovery in all deployment scenarios, from development to production container environments.

View File

@@ -0,0 +1,226 @@
# SSO and OIDC Setup Guide
This guide explains how to configure Single Sign-On (SSO) and OpenID Connect (OIDC) provider functionality in Gitea Mirror.
## Overview
Gitea Mirror supports three authentication methods:
1. **Email & Password** - Traditional authentication (always enabled)
2. **SSO (Single Sign-On)** - Allow users to authenticate using external OIDC providers
3. **OIDC Provider** - Allow other applications to authenticate users through Gitea Mirror
## Configuration
All SSO and OIDC settings are managed through the web UI in the Configuration page under the "Authentication" tab.
## Setting up SSO (Single Sign-On)
SSO allows your users to sign in using external identity providers like Google, Okta, Azure AD, etc.
### Adding an SSO Provider
1. Navigate to Configuration → Authentication → SSO Providers
2. Click "Add Provider"
3. Fill in the provider details:
#### Required Fields
- **Issuer URL**: The OIDC issuer URL (e.g., `https://accounts.google.com`)
- **Domain**: The email domain for this provider (e.g., `example.com`)
- **Provider ID**: A unique identifier for this provider (e.g., `google-sso`)
- **Client ID**: The OAuth client ID from your provider
- **Client Secret**: The OAuth client secret from your provider
#### Auto-Discovery
If your provider supports OIDC discovery, you can:
1. Enter the Issuer URL
2. Click "Discover"
3. The system will automatically fetch the authorization and token endpoints
#### Manual Configuration
For providers without discovery support, manually enter:
- **Authorization Endpoint**: The OAuth authorization URL
- **Token Endpoint**: The OAuth token exchange URL
- **JWKS Endpoint**: The JSON Web Key Set URL (optional)
- **UserInfo Endpoint**: The user information endpoint (optional)
### Redirect URL
When configuring your SSO provider, use this redirect URL:
```
https://your-domain.com/api/auth/sso/callback/{provider-id}
```
Replace `{provider-id}` with your chosen Provider ID.
### Example: Google SSO Setup
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
2. Create a new OAuth 2.0 Client ID
3. Add authorized redirect URI: `https://your-domain.com/api/auth/sso/callback/google-sso`
4. In Gitea Mirror:
- Issuer URL: `https://accounts.google.com`
- Domain: `your-company.com`
- Provider ID: `google-sso`
- Client ID: [Your Google Client ID]
- Client Secret: [Your Google Client Secret]
- Click "Discover" to auto-fill endpoints
### Example: Okta SSO Setup
1. In Okta Admin Console, create a new OIDC Web Application
2. Set redirect URI: `https://your-domain.com/api/auth/sso/callback/okta-sso`
3. In Gitea Mirror:
- Issuer URL: `https://your-okta-domain.okta.com`
- Domain: `your-company.com`
- Provider ID: `okta-sso`
- Client ID: [Your Okta Client ID]
- Client Secret: [Your Okta Client Secret]
- Click "Discover" to auto-fill endpoints
### Example: Authentik SSO Setup
Working Authentik deployments (see [#134](https://github.com/RayLabsHQ/gitea-mirror/issues/134)) follow these steps:
1. In Authentik, create a new **Application** and OIDC **Provider** (implicit flow works well for testing).
2. Start creating an SSO provider inside Gitea Mirror so you can copy the redirect URL shown (`https://your-domain.com/api/auth/sso/callback/authentik` if you pick `authentik` as your Provider ID).
3. Paste that redirect URL into the Authentik Provider configuration and finish creating the provider.
4. Copy the Authentik issuer URL, client ID, and client secret.
5. Back in Gitea Mirror:
- Issuer URL: the exact value from Authentik (keep any trailing slash Authentik shows).
- Provider ID: match the one you used in step 2.
- Click **Discover** so Gitea Mirror stores the authorization, token, and JWKS endpoints (Authentik publishes them via discovery).
- Domain: enter the email domain you expect to match (e.g. `example.com`).
6. Save the provider and test the login flow.
Notes:
- Make sure `BETTER_AUTH_URL` and (if you serve the UI from multiple origins) `BETTER_AUTH_TRUSTED_ORIGINS` point at the public URL users reach. A mismatch can surface as 500 errors after redirect.
- Authentik must report the users email as verified (default behavior) so Gitea Mirror can auto-link accounts.
- If you created an Authentik provider before v3.8.10 you should delete it and re-add it after upgrading; older versions saved incomplete endpoint data which leads to the `url.startsWith` error explained in the Troubleshooting section.
## Setting up OIDC Provider
The OIDC Provider feature allows other applications to use Gitea Mirror as their authentication provider.
### Creating OAuth Applications
1. Navigate to Configuration → Authentication → OAuth Applications
2. Click "Create Application"
3. Fill in the application details:
- **Application Name**: Display name for the application
- **Application Type**: Web, Mobile, or Desktop
- **Redirect URLs**: One or more redirect URLs (one per line)
4. After creation, you'll receive:
- **Client ID**: Share this with the application
- **Client Secret**: Keep this secure and share only once
### OIDC Endpoints
Applications can use these standard OIDC endpoints:
- **Discovery**: `https://your-domain.com/.well-known/openid-configuration`
- **Authorization**: `https://your-domain.com/api/auth/oauth2/authorize`
- **Token**: `https://your-domain.com/api/auth/oauth2/token`
- **UserInfo**: `https://your-domain.com/api/auth/oauth2/userinfo`
- **JWKS**: `https://your-domain.com/api/auth/jwks`
### Supported Scopes
- `openid` - Required, provides user ID
- `profile` - User's name, username, and profile picture
- `email` - User's email address and verification status
### Example: Configuring Another Application
For an application to use Gitea Mirror as its OIDC provider:
```javascript
// Example configuration for another app
const oidcConfig = {
issuer: 'https://gitea-mirror.example.com',
clientId: 'client_xxxxxxxxxxxxx',
clientSecret: 'secret_xxxxxxxxxxxxx',
redirectUri: 'https://myapp.com/auth/callback',
scope: 'openid profile email'
};
```
## User Experience
### Logging In with SSO
When SSO is configured:
1. Users see tabs for "Email" and "SSO" on the login page
2. In the SSO tab, they can:
- Click a specific provider button (if configured)
- Enter their work email to be redirected to the appropriate provider
### OAuth Consent Flow
When an application requests authentication:
1. Users are redirected to Gitea Mirror
2. If not logged in, they authenticate first
3. They see a consent screen showing:
- Application name
- Requested permissions
- Option to approve or deny
## Security Considerations
1. **Client Secrets**: Store OAuth client secrets securely
2. **Redirect URLs**: Only add trusted redirect URLs for applications
3. **Scopes**: Applications only receive the data for approved scopes
4. **Token Security**: Access tokens expire and can be revoked
## Troubleshooting
### SSO Login Issues
1. **"Invalid origin" error**: Check that your Gitea Mirror URL matches the configured redirect URI
2. **"Provider not found" error**: Ensure the provider is properly configured and enabled
3. **Redirect loop**: Verify the redirect URI in both Gitea Mirror and the SSO provider match exactly
4. **`TypeError: undefined is not an object (evaluating 'url.startsWith')`**: This indicates the stored provider configuration is missing OIDC endpoints. Delete the provider from Gitea Mirror and re-register it using the **Discover** button so authorization/token URLs are saved (see [#73](https://github.com/RayLabsHQ/gitea-mirror/issues/73) and [#122](https://github.com/RayLabsHQ/gitea-mirror/issues/122) for examples).
### OIDC Provider Issues
1. **Application not found**: Ensure the client ID is correct
2. **Invalid redirect URI**: The redirect URI must match exactly what's configured
3. **Consent not working**: Check browser cookies are enabled
## Managing Access
### Revoking SSO Access
Currently, SSO sessions are managed through the identity provider. To revoke access:
1. Log out of Gitea Mirror
2. Revoke access in your identity provider's settings
### Disabling OAuth Applications
To disable an application:
1. Go to Configuration → Authentication → OAuth Applications
2. Find the application
3. Click the delete button
This immediately prevents the application from authenticating new users.
## Best Practices
1. **Use HTTPS**: Always use HTTPS in production for security
2. **Regular Audits**: Periodically review configured SSO providers and OAuth applications
3. **Principle of Least Privilege**: Only grant necessary scopes to applications
4. **Monitor Usage**: Keep track of which applications are accessing your OIDC provider
5. **Secure Storage**: Store client secrets in a secure location, never in code
## Migration Notes
If migrating from the previous JWT-based authentication:
- Existing users remain unaffected
- Users can continue using email/password authentication
- SSO can be added as an additional authentication method

View File

@@ -0,0 +1,193 @@
# Local SSO Testing Guide
This guide explains how to test SSO authentication locally with Gitea Mirror.
## Option 1: Using Google OAuth (Recommended for Quick Testing)
### Setup Steps:
1. **Create a Google OAuth Application**
- Go to [Google Cloud Console](https://console.cloud.google.com/)
- Create a new project or select existing
- Enable Google+ API
- Go to "Credentials" → "Create Credentials" → "OAuth client ID"
- Choose "Web application"
- Add authorized redirect URIs:
- `http://localhost:3000/api/auth/sso/callback/google-sso`
- `http://localhost:9876/api/auth/sso/callback/google-sso`
2. **Configure in Gitea Mirror**
- Go to Configuration → Authentication tab
- Click "Add Provider"
- Select "OIDC / OAuth2"
- Fill in:
- Provider ID: `google-sso`
- Email Domain: `gmail.com` (or your domain)
- Issuer URL: `https://accounts.google.com`
- Click "Discover" to auto-fill endpoints
- Client ID: (from Google Console)
- Client Secret: (from Google Console)
- Save the provider
## Option 2: Using Keycloak (Local Identity Provider)
### Setup with Docker:
```bash
# Run Keycloak
docker run -d --name keycloak \
-p 8080:8080 \
-e KEYCLOAK_ADMIN=admin \
-e KEYCLOAK_ADMIN_PASSWORD=admin \
quay.io/keycloak/keycloak:latest start-dev
# Access at http://localhost:8080
# Login with admin/admin
```
### Configure Keycloak:
1. Create a new realm (e.g., "gitea-mirror")
2. Create a client:
- Client ID: `gitea-mirror`
- Client Protocol: `openid-connect`
- Access Type: `confidential`
- Valid Redirect URIs: `http://localhost:*/api/auth/sso/callback/keycloak`
3. Get credentials from the "Credentials" tab
4. Create test users in "Users" section
### Configure in Gitea Mirror:
- Provider ID: `keycloak`
- Email Domain: `example.com`
- Issuer URL: `http://localhost:8080/realms/gitea-mirror`
- Client ID: `gitea-mirror`
- Client Secret: (from Keycloak)
- Click "Discover" to auto-fill endpoints
## Option 3: Using Mock SSO Provider (Development)
For testing without external dependencies, you can use a mock OIDC provider.
### Using oidc-provider-example:
```bash
# Clone and run mock provider
git clone https://github.com/panva/node-oidc-provider-example.git
cd node-oidc-provider-example
npm install
npm start
# Runs on http://localhost:3001
```
### Configure in Gitea Mirror:
- Provider ID: `mock-provider`
- Email Domain: `test.com`
- Issuer URL: `http://localhost:3001`
- Client ID: `foo`
- Client Secret: `bar`
- Authorization Endpoint: `http://localhost:3001/auth`
- Token Endpoint: `http://localhost:3001/token`
## Testing the SSO Flow
1. **Logout** from Gitea Mirror if logged in
2. Go to `/login`
3. Click on the **SSO** tab
4. Either:
- Click the provider button (e.g., "Sign in with gmail.com")
- Or enter your email and click "Continue with SSO"
5. You'll be redirected to the identity provider
6. Complete authentication
7. You'll be redirected back and logged in
## Troubleshooting
### Common Issues:
1. **"Invalid origin" error**
- Check that `trustedOrigins` in `/src/lib/auth.ts` includes your dev URL
- Restart the dev server after changes
2. **Provider not showing in login**
- Check browser console for errors
- Verify provider was saved successfully
- Check `/api/sso/providers` returns your providers
3. **Redirect URI mismatch**
- Ensure the redirect URI in your OAuth app matches exactly:
`http://localhost:PORT/api/auth/sso/callback/PROVIDER_ID`
4. **CORS errors**
- Add your identity provider domain to CORS allowed origins if needed
### Debug Mode:
Enable debug logging by setting environment variable:
```bash
DEBUG=better-auth:* bun run dev
```
## Testing Different Scenarios
### 1. New User Registration
- Use an email not in the system
- SSO should create a new user automatically
### 2. Existing User Login
- Create a user with email/password first
- Login with SSO using same email
- Should link to existing account
### 3. Domain-based Routing
- Configure multiple providers with different domains
- Test that entering email routes to correct provider
### 4. Organization Provisioning
- Set organizationId on provider
- Test that users are added to correct organization
## Security Testing
1. **Token Expiration**
- Wait for session to expire
- Test refresh flow
2. **Invalid State**
- Modify state parameter in callback
- Should reject authentication
3. **PKCE Flow**
- Enable/disable PKCE
- Verify code challenge works
## Using with Better Auth CLI
Better Auth provides CLI tools for testing:
```bash
# List registered providers
bun run auth:providers list
# Test provider configuration
bun run auth:providers test google-sso
```
## Environment Variables
For production-like testing:
```env
BETTER_AUTH_URL=http://localhost:3000
BETTER_AUTH_SECRET=your-secret-key
```
## Next Steps
After successful SSO setup:
1. Test user attribute mapping
2. Configure role-based access
3. Set up SAML if needed
4. Test with your organization's actual IdP

View File

@@ -0,0 +1,16 @@
import { defineConfig } from "drizzle-kit";
export default defineConfig({
dialect: "sqlite",
schema: "./src/lib/db/schema.ts",
out: "./drizzle",
dbCredentials: {
url: "./data/gitea-mirror.db",
},
verbose: true,
strict: true,
migrations: {
table: "__drizzle_migrations",
schema: "main",
},
});

View File

@@ -0,0 +1,180 @@
CREATE TABLE `accounts` (
`id` text PRIMARY KEY NOT NULL,
`account_id` text NOT NULL,
`user_id` text NOT NULL,
`provider_id` text NOT NULL,
`provider_user_id` text,
`access_token` text,
`refresh_token` text,
`expires_at` integer,
`password` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_accounts_account_id` ON `accounts` (`account_id`);--> statement-breakpoint
CREATE INDEX `idx_accounts_user_id` ON `accounts` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_accounts_provider` ON `accounts` (`provider_id`,`provider_user_id`);--> statement-breakpoint
CREATE TABLE `configs` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`name` text NOT NULL,
`is_active` integer DEFAULT true NOT NULL,
`github_config` text NOT NULL,
`gitea_config` text NOT NULL,
`include` text DEFAULT '["*"]' NOT NULL,
`exclude` text DEFAULT '[]' NOT NULL,
`schedule_config` text NOT NULL,
`cleanup_config` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE TABLE `events` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`channel` text NOT NULL,
`payload` text NOT NULL,
`read` integer DEFAULT false NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_events_user_channel` ON `events` (`user_id`,`channel`);--> statement-breakpoint
CREATE INDEX `idx_events_created_at` ON `events` (`created_at`);--> statement-breakpoint
CREATE INDEX `idx_events_read` ON `events` (`read`);--> statement-breakpoint
CREATE TABLE `mirror_jobs` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`repository_id` text,
`repository_name` text,
`organization_id` text,
`organization_name` text,
`details` text,
`status` text DEFAULT 'imported' NOT NULL,
`message` text NOT NULL,
`timestamp` integer DEFAULT (unixepoch()) NOT NULL,
`job_type` text DEFAULT 'mirror' NOT NULL,
`batch_id` text,
`total_items` integer,
`completed_items` integer DEFAULT 0,
`item_ids` text,
`completed_item_ids` text DEFAULT '[]',
`in_progress` integer DEFAULT false NOT NULL,
`started_at` integer,
`completed_at` integer,
`last_checkpoint` integer,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_user_id` ON `mirror_jobs` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_batch_id` ON `mirror_jobs` (`batch_id`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_in_progress` ON `mirror_jobs` (`in_progress`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_job_type` ON `mirror_jobs` (`job_type`);--> statement-breakpoint
CREATE INDEX `idx_mirror_jobs_timestamp` ON `mirror_jobs` (`timestamp`);--> statement-breakpoint
CREATE TABLE `organizations` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`avatar_url` text NOT NULL,
`membership_role` text DEFAULT 'member' NOT NULL,
`is_included` integer DEFAULT true NOT NULL,
`destination_org` text,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`repository_count` integer DEFAULT 0 NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_organizations_user_id` ON `organizations` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_config_id` ON `organizations` (`config_id`);--> statement-breakpoint
CREATE INDEX `idx_organizations_status` ON `organizations` (`status`);--> statement-breakpoint
CREATE INDEX `idx_organizations_is_included` ON `organizations` (`is_included`);--> statement-breakpoint
CREATE TABLE `repositories` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`config_id` text NOT NULL,
`name` text NOT NULL,
`full_name` text NOT NULL,
`url` text NOT NULL,
`clone_url` text NOT NULL,
`owner` text NOT NULL,
`organization` text,
`mirrored_location` text DEFAULT '',
`is_private` integer DEFAULT false NOT NULL,
`is_fork` integer DEFAULT false NOT NULL,
`forked_from` text,
`has_issues` integer DEFAULT false NOT NULL,
`is_starred` integer DEFAULT false NOT NULL,
`is_archived` integer DEFAULT false NOT NULL,
`size` integer DEFAULT 0 NOT NULL,
`has_lfs` integer DEFAULT false NOT NULL,
`has_submodules` integer DEFAULT false NOT NULL,
`language` text,
`description` text,
`default_branch` text NOT NULL,
`visibility` text DEFAULT 'public' NOT NULL,
`status` text DEFAULT 'imported' NOT NULL,
`last_mirrored` integer,
`error_message` text,
`destination_org` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action,
FOREIGN KEY (`config_id`) REFERENCES `configs`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_repositories_user_id` ON `repositories` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_repositories_config_id` ON `repositories` (`config_id`);--> statement-breakpoint
CREATE INDEX `idx_repositories_status` ON `repositories` (`status`);--> statement-breakpoint
CREATE INDEX `idx_repositories_owner` ON `repositories` (`owner`);--> statement-breakpoint
CREATE INDEX `idx_repositories_organization` ON `repositories` (`organization`);--> statement-breakpoint
CREATE INDEX `idx_repositories_is_fork` ON `repositories` (`is_fork`);--> statement-breakpoint
CREATE INDEX `idx_repositories_is_starred` ON `repositories` (`is_starred`);--> statement-breakpoint
CREATE TABLE `sessions` (
`id` text PRIMARY KEY NOT NULL,
`token` text NOT NULL,
`user_id` text NOT NULL,
`expires_at` integer NOT NULL,
`ip_address` text,
`user_agent` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE UNIQUE INDEX `sessions_token_unique` ON `sessions` (`token`);--> statement-breakpoint
CREATE INDEX `idx_sessions_user_id` ON `sessions` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_sessions_token` ON `sessions` (`token`);--> statement-breakpoint
CREATE INDEX `idx_sessions_expires_at` ON `sessions` (`expires_at`);--> statement-breakpoint
CREATE TABLE `users` (
`id` text PRIMARY KEY NOT NULL,
`name` text,
`email` text NOT NULL,
`email_verified` integer DEFAULT false NOT NULL,
`image` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
`username` text
);
--> statement-breakpoint
CREATE UNIQUE INDEX `users_email_unique` ON `users` (`email`);--> statement-breakpoint
CREATE TABLE `verification_tokens` (
`id` text PRIMARY KEY NOT NULL,
`token` text NOT NULL,
`identifier` text NOT NULL,
`type` text NOT NULL,
`expires_at` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE UNIQUE INDEX `verification_tokens_token_unique` ON `verification_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_verification_tokens_token` ON `verification_tokens` (`token`);--> statement-breakpoint
CREATE INDEX `idx_verification_tokens_identifier` ON `verification_tokens` (`identifier`);

View File

@@ -0,0 +1,64 @@
CREATE TABLE `oauth_access_tokens` (
`id` text PRIMARY KEY NOT NULL,
`access_token` text NOT NULL,
`refresh_token` text,
`access_token_expires_at` integer NOT NULL,
`refresh_token_expires_at` integer,
`client_id` text NOT NULL,
`user_id` text NOT NULL,
`scopes` text NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_access_token` ON `oauth_access_tokens` (`access_token`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_user_id` ON `oauth_access_tokens` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_access_tokens_client_id` ON `oauth_access_tokens` (`client_id`);--> statement-breakpoint
CREATE TABLE `oauth_applications` (
`id` text PRIMARY KEY NOT NULL,
`client_id` text NOT NULL,
`client_secret` text NOT NULL,
`name` text NOT NULL,
`redirect_urls` text NOT NULL,
`metadata` text,
`type` text NOT NULL,
`disabled` integer DEFAULT false NOT NULL,
`user_id` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE UNIQUE INDEX `oauth_applications_client_id_unique` ON `oauth_applications` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_applications_client_id` ON `oauth_applications` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_applications_user_id` ON `oauth_applications` (`user_id`);--> statement-breakpoint
CREATE TABLE `oauth_consent` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`client_id` text NOT NULL,
`scopes` text NOT NULL,
`consent_given` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_oauth_consent_user_id` ON `oauth_consent` (`user_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_consent_client_id` ON `oauth_consent` (`client_id`);--> statement-breakpoint
CREATE INDEX `idx_oauth_consent_user_client` ON `oauth_consent` (`user_id`,`client_id`);--> statement-breakpoint
CREATE TABLE `sso_providers` (
`id` text PRIMARY KEY NOT NULL,
`issuer` text NOT NULL,
`domain` text NOT NULL,
`oidc_config` text NOT NULL,
`user_id` text NOT NULL,
`provider_id` text NOT NULL,
`organization_id` text,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE UNIQUE INDEX `sso_providers_provider_id_unique` ON `sso_providers` (`provider_id`);--> statement-breakpoint
CREATE INDEX `idx_sso_providers_provider_id` ON `sso_providers` (`provider_id`);--> statement-breakpoint
CREATE INDEX `idx_sso_providers_domain` ON `sso_providers` (`domain`);--> statement-breakpoint
CREATE INDEX `idx_sso_providers_issuer` ON `sso_providers` (`issuer`);

View File

@@ -0,0 +1,10 @@
CREATE TABLE `verifications` (
`id` text PRIMARY KEY NOT NULL,
`identifier` text NOT NULL,
`value` text NOT NULL,
`expires_at` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL
);
--> statement-breakpoint
CREATE INDEX `idx_verifications_identifier` ON `verifications` (`identifier`);

View File

@@ -0,0 +1,3 @@
ALTER TABLE `organizations` ADD `public_repository_count` integer;--> statement-breakpoint
ALTER TABLE `organizations` ADD `private_repository_count` integer;--> statement-breakpoint
ALTER TABLE `organizations` ADD `fork_repository_count` integer;

View File

@@ -0,0 +1,18 @@
CREATE TABLE `rate_limits` (
`id` text PRIMARY KEY NOT NULL,
`user_id` text NOT NULL,
`provider` text DEFAULT 'github' NOT NULL,
`limit` integer NOT NULL,
`remaining` integer NOT NULL,
`used` integer NOT NULL,
`reset` integer NOT NULL,
`retry_after` integer,
`status` text DEFAULT 'ok' NOT NULL,
`last_checked` integer NOT NULL,
`created_at` integer DEFAULT (unixepoch()) NOT NULL,
`updated_at` integer DEFAULT (unixepoch()) NOT NULL,
FOREIGN KEY (`user_id`) REFERENCES `users`(`id`) ON UPDATE no action ON DELETE no action
);
--> statement-breakpoint
CREATE INDEX `idx_rate_limits_user_provider` ON `rate_limits` (`user_id`,`provider`);--> statement-breakpoint
CREATE INDEX `idx_rate_limits_status` ON `rate_limits` (`status`);

View File

@@ -0,0 +1,11 @@
-- Step 1: Remove duplicate repositories, keeping the most recently updated one
-- This handles cases where users have duplicate entries from before the unique constraint
DELETE FROM repositories
WHERE rowid NOT IN (
SELECT MAX(rowid)
FROM repositories
GROUP BY user_id, full_name
);
--> statement-breakpoint
-- Step 2: Now create the unique index safely
CREATE UNIQUE INDEX uniq_repositories_user_full_name ON repositories (user_id, full_name);

View File

@@ -0,0 +1,4 @@
ALTER TABLE `accounts` ADD `id_token` text;--> statement-breakpoint
ALTER TABLE `accounts` ADD `access_token_expires_at` integer;--> statement-breakpoint
ALTER TABLE `accounts` ADD `refresh_token_expires_at` integer;--> statement-breakpoint
ALTER TABLE `accounts` ADD `scope` text;

View File

@@ -0,0 +1,18 @@
ALTER TABLE `organizations` ADD `normalized_name` text NOT NULL DEFAULT '';--> statement-breakpoint
UPDATE `organizations` SET `normalized_name` = lower(trim(`name`));--> statement-breakpoint
DELETE FROM `organizations`
WHERE rowid NOT IN (
SELECT MIN(rowid)
FROM `organizations`
GROUP BY `user_id`, `normalized_name`
);--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_organizations_user_normalized_name` ON `organizations` (`user_id`,`normalized_name`);--> statement-breakpoint
ALTER TABLE `repositories` ADD `normalized_full_name` text NOT NULL DEFAULT '';--> statement-breakpoint
UPDATE `repositories` SET `normalized_full_name` = lower(trim(`full_name`));--> statement-breakpoint
DELETE FROM `repositories`
WHERE rowid NOT IN (
SELECT MIN(rowid)
FROM `repositories`
GROUP BY `user_id`, `normalized_full_name`
);--> statement-breakpoint
CREATE UNIQUE INDEX `uniq_repositories_user_normalized_full_name` ON `repositories` (`user_id`,`normalized_full_name`);

View File

@@ -0,0 +1 @@
ALTER TABLE `repositories` ADD `metadata` text;

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,69 @@
{
"version": "7",
"dialect": "sqlite",
"entries": [
{
"idx": 0,
"version": "6",
"when": 1752171873627,
"tag": "0000_init",
"breakpoints": true
},
{
"idx": 1,
"version": "6",
"when": 1752173351102,
"tag": "0001_polite_exodus",
"breakpoints": true
},
{
"idx": 2,
"version": "6",
"when": 1753539600567,
"tag": "0002_bored_captain_cross",
"breakpoints": true
},
{
"idx": 3,
"version": "6",
"when": 1757390828679,
"tag": "0003_open_spacker_dave",
"breakpoints": true
},
{
"idx": 4,
"version": "6",
"when": 1757392620734,
"tag": "0004_grey_butterfly",
"breakpoints": true
},
{
"idx": 5,
"version": "6",
"when": 1757786449446,
"tag": "0005_polite_preak",
"breakpoints": true
},
{
"idx": 6,
"version": "6",
"when": 1761483928546,
"tag": "0006_military_la_nuit",
"breakpoints": true
},
{
"idx": 7,
"version": "6",
"when": 1761534391115,
"tag": "0007_whole_hellion",
"breakpoints": true
},
{
"idx": 8,
"version": "6",
"when": 1761802056073,
"tag": "0008_serious_thena",
"breakpoints": true
}
]
}

9
Divers/gitea-mirror/env.d.ts vendored Normal file
View File

@@ -0,0 +1,9 @@
/// <reference path="./.astro/types.d.ts" />
/// <reference types="astro/client" />
declare namespace App {
interface Locals {
user: import("better-auth").User | null;
session: import("better-auth").Session | null;
}
}

61
Divers/gitea-mirror/flake.lock generated Normal file
View File

@@ -0,0 +1,61 @@
{
"nodes": {
"flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": {
"lastModified": 1731533236,
"narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1761672384,
"narHash": "sha256-o9KF3DJL7g7iYMZq9SWgfS1BFlNbsm6xplRjVlOCkXI=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "08dacfca559e1d7da38f3cf05f1f45ee9bfd213c",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"flake-utils": "flake-utils",
"nixpkgs": "nixpkgs"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
}
},
"root": "root",
"version": 7
}

View File

@@ -0,0 +1,395 @@
{
description = "Gitea Mirror - Self-hosted GitHub to Gitea mirroring service";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = nixpkgs.legacyPackages.${system};
# Build the application
gitea-mirror = pkgs.stdenv.mkDerivation {
pname = "gitea-mirror";
version = "3.8.11";
src = ./.;
nativeBuildInputs = with pkgs; [
bun
];
buildInputs = with pkgs; [
sqlite
openssl
];
configurePhase = ''
export HOME=$TMPDIR
export BUN_INSTALL=$TMPDIR/.bun
export PATH=$BUN_INSTALL/bin:$PATH
'';
buildPhase = ''
# Install dependencies
bun install --frozen-lockfile --no-progress
# Build the application
bun run build
'';
installPhase = ''
mkdir -p $out/lib/gitea-mirror
mkdir -p $out/bin
# Copy the built application
cp -r dist $out/lib/gitea-mirror/
cp -r node_modules $out/lib/gitea-mirror/
cp -r scripts $out/lib/gitea-mirror/
cp package.json $out/lib/gitea-mirror/
# Create entrypoint script that matches Docker behavior
cat > $out/bin/gitea-mirror <<'EOF'
#!/usr/bin/env bash
set -e
# === DEFAULT CONFIGURATION ===
# These match docker-compose.alt.yml defaults
export DATA_DIR=''${DATA_DIR:-"$HOME/.local/share/gitea-mirror"}
export DATABASE_URL=''${DATABASE_URL:-"file:$DATA_DIR/gitea-mirror.db"}
export HOST=''${HOST:-"0.0.0.0"}
export PORT=''${PORT:-"4321"}
export NODE_ENV=''${NODE_ENV:-"production"}
# Better Auth configuration
export BETTER_AUTH_URL=''${BETTER_AUTH_URL:-"http://localhost:4321"}
export BETTER_AUTH_TRUSTED_ORIGINS=''${BETTER_AUTH_TRUSTED_ORIGINS:-"http://localhost:4321"}
export PUBLIC_BETTER_AUTH_URL=''${PUBLIC_BETTER_AUTH_URL:-"http://localhost:4321"}
# Concurrency settings (match docker-compose.alt.yml)
export MIRROR_ISSUE_CONCURRENCY=''${MIRROR_ISSUE_CONCURRENCY:-3}
export MIRROR_PULL_REQUEST_CONCURRENCY=''${MIRROR_PULL_REQUEST_CONCURRENCY:-5}
# Create data directory
mkdir -p "$DATA_DIR"
cd $out/lib/gitea-mirror
# === AUTO-GENERATE SECRETS ===
BETTER_AUTH_SECRET_FILE="$DATA_DIR/.better_auth_secret"
ENCRYPTION_SECRET_FILE="$DATA_DIR/.encryption_secret"
# Generate BETTER_AUTH_SECRET if not provided
if [ -z "$BETTER_AUTH_SECRET" ]; then
if [ -f "$BETTER_AUTH_SECRET_FILE" ]; then
echo "Using previously generated BETTER_AUTH_SECRET"
export BETTER_AUTH_SECRET=$(cat "$BETTER_AUTH_SECRET_FILE")
else
echo "Generating a secure random BETTER_AUTH_SECRET"
GENERATED_SECRET=$(${pkgs.openssl}/bin/openssl rand -hex 32)
export BETTER_AUTH_SECRET="$GENERATED_SECRET"
echo "$GENERATED_SECRET" > "$BETTER_AUTH_SECRET_FILE"
chmod 600 "$BETTER_AUTH_SECRET_FILE"
echo " BETTER_AUTH_SECRET generated and saved to $BETTER_AUTH_SECRET_FILE"
fi
fi
# Generate ENCRYPTION_SECRET if not provided
if [ -z "$ENCRYPTION_SECRET" ]; then
if [ -f "$ENCRYPTION_SECRET_FILE" ]; then
echo "Using previously generated ENCRYPTION_SECRET"
export ENCRYPTION_SECRET=$(cat "$ENCRYPTION_SECRET_FILE")
else
echo "Generating a secure random ENCRYPTION_SECRET"
GENERATED_ENCRYPTION_SECRET=$(${pkgs.openssl}/bin/openssl rand -base64 36)
export ENCRYPTION_SECRET="$GENERATED_ENCRYPTION_SECRET"
echo "$GENERATED_ENCRYPTION_SECRET" > "$ENCRYPTION_SECRET_FILE"
chmod 600 "$ENCRYPTION_SECRET_FILE"
echo " ENCRYPTION_SECRET generated and saved to $ENCRYPTION_SECRET_FILE"
fi
fi
# === DATABASE INITIALIZATION ===
DB_PATH=$(echo "$DATABASE_URL" | sed 's|^file:||')
if [ ! -f "$DB_PATH" ]; then
echo "Database not found. It will be created and initialized via Drizzle migrations on first app startup..."
touch "$DB_PATH"
else
echo "Database already exists, Drizzle will check for pending migrations on startup..."
fi
# === STARTUP SCRIPTS ===
# Initialize configuration from environment variables
echo "Checking for environment configuration..."
if [ -f "dist/scripts/startup-env-config.js" ]; then
echo "Loading configuration from environment variables..."
${pkgs.bun}/bin/bun dist/scripts/startup-env-config.js && \
echo " Environment configuration loaded successfully" || \
echo " Environment configuration loading completed with warnings"
fi
# Run startup recovery
echo "Running startup recovery..."
if [ -f "dist/scripts/startup-recovery.js" ]; then
${pkgs.bun}/bin/bun dist/scripts/startup-recovery.js --timeout=30000 && \
echo " Startup recovery completed successfully" || \
echo " Startup recovery completed with warnings"
fi
# Run repository status repair
echo "Running repository status repair..."
if [ -f "dist/scripts/repair-mirrored-repos.js" ]; then
${pkgs.bun}/bin/bun dist/scripts/repair-mirrored-repos.js --startup && \
echo " Repository status repair completed successfully" || \
echo " Repository status repair completed with warnings"
fi
# === SIGNAL HANDLING ===
shutdown_handler() {
echo "🛑 Received shutdown signal, forwarding to application..."
if [ ! -z "$APP_PID" ]; then
kill -TERM "$APP_PID" 2>/dev/null || true
wait "$APP_PID" 2>/dev/null || true
fi
exit 0
}
trap 'shutdown_handler' TERM INT HUP
# === START APPLICATION ===
echo "Starting Gitea Mirror..."
echo "Access the web interface at $BETTER_AUTH_URL"
${pkgs.bun}/bin/bun dist/server/entry.mjs &
APP_PID=$!
wait "$APP_PID"
EOF
chmod +x $out/bin/gitea-mirror
# Create database management helper
cat > $out/bin/gitea-mirror-db <<'EOF'
#!/usr/bin/env bash
export DATA_DIR=''${DATA_DIR:-"$HOME/.local/share/gitea-mirror"}
mkdir -p "$DATA_DIR"
cd $out/lib/gitea-mirror
exec ${pkgs.bun}/bin/bun scripts/manage-db.ts "$@"
EOF
chmod +x $out/bin/gitea-mirror-db
'';
meta = with pkgs.lib; {
description = "Self-hosted GitHub to Gitea mirroring service";
homepage = "https://github.com/RayLabsHQ/gitea-mirror";
license = licenses.mit;
maintainers = [ ];
platforms = platforms.linux ++ platforms.darwin;
};
};
in
{
packages = {
default = gitea-mirror;
gitea-mirror = gitea-mirror;
};
# Development shell
devShells.default = pkgs.mkShell {
buildInputs = with pkgs; [
bun
sqlite
openssl
];
shellHook = ''
echo "🚀 Gitea Mirror development environment"
echo ""
echo "Quick start:"
echo " bun install # Install dependencies"
echo " bun run dev # Start development server"
echo " bun run build # Build for production"
echo ""
echo "Database:"
echo " bun run manage-db init # Initialize database"
echo " bun run db:studio # Open Drizzle Studio"
'';
};
# NixOS module
nixosModules.default = { config, lib, pkgs, ... }:
with lib;
let
cfg = config.services.gitea-mirror;
in {
options.services.gitea-mirror = {
enable = mkEnableOption "Gitea Mirror service";
package = mkOption {
type = types.package;
default = self.packages.${system}.default;
description = "The Gitea Mirror package to use";
};
dataDir = mkOption {
type = types.path;
default = "/var/lib/gitea-mirror";
description = "Directory to store data and database";
};
user = mkOption {
type = types.str;
default = "gitea-mirror";
description = "User account under which Gitea Mirror runs";
};
group = mkOption {
type = types.str;
default = "gitea-mirror";
description = "Group under which Gitea Mirror runs";
};
host = mkOption {
type = types.str;
default = "0.0.0.0";
description = "Host to bind to";
};
port = mkOption {
type = types.port;
default = 4321;
description = "Port to listen on";
};
betterAuthUrl = mkOption {
type = types.str;
default = "http://localhost:4321";
description = "Better Auth URL (external URL of the service)";
};
betterAuthTrustedOrigins = mkOption {
type = types.str;
default = "http://localhost:4321";
description = "Comma-separated list of trusted origins for Better Auth";
};
mirrorIssueConcurrency = mkOption {
type = types.int;
default = 3;
description = "Number of concurrent issue mirror operations (set to 1 for perfect ordering)";
};
mirrorPullRequestConcurrency = mkOption {
type = types.int;
default = 5;
description = "Number of concurrent PR mirror operations (set to 1 for perfect ordering)";
};
environmentFile = mkOption {
type = types.nullOr types.path;
default = null;
description = ''
Path to file containing environment variables.
Only needed if you want to set BETTER_AUTH_SECRET or ENCRYPTION_SECRET manually.
Otherwise, secrets will be auto-generated and stored in the data directory.
Example:
BETTER_AUTH_SECRET=your-32-character-secret-here
ENCRYPTION_SECRET=your-encryption-secret-here
'';
};
openFirewall = mkOption {
type = types.bool;
default = false;
description = "Open the firewall for the specified port";
};
};
config = mkIf cfg.enable {
users.users.${cfg.user} = {
isSystemUser = true;
group = cfg.group;
home = cfg.dataDir;
createHome = true;
};
users.groups.${cfg.group} = {};
systemd.services.gitea-mirror = {
description = "Gitea Mirror - GitHub to Gitea mirroring service";
after = [ "network.target" ];
wantedBy = [ "multi-user.target" ];
environment = {
DATA_DIR = cfg.dataDir;
DATABASE_URL = "file:${cfg.dataDir}/gitea-mirror.db";
HOST = cfg.host;
PORT = toString cfg.port;
NODE_ENV = "production";
BETTER_AUTH_URL = cfg.betterAuthUrl;
BETTER_AUTH_TRUSTED_ORIGINS = cfg.betterAuthTrustedOrigins;
PUBLIC_BETTER_AUTH_URL = cfg.betterAuthUrl;
MIRROR_ISSUE_CONCURRENCY = toString cfg.mirrorIssueConcurrency;
MIRROR_PULL_REQUEST_CONCURRENCY = toString cfg.mirrorPullRequestConcurrency;
};
serviceConfig = {
Type = "simple";
User = cfg.user;
Group = cfg.group;
ExecStart = "${cfg.package}/bin/gitea-mirror";
Restart = "always";
RestartSec = "10s";
# Security hardening
NoNewPrivileges = true;
PrivateTmp = true;
ProtectSystem = "strict";
ProtectHome = true;
ReadWritePaths = [ cfg.dataDir ];
# Load environment file if specified (optional)
EnvironmentFile = mkIf (cfg.environmentFile != null) cfg.environmentFile;
# Graceful shutdown
TimeoutStopSec = "30s";
KillMode = "mixed";
KillSignal = "SIGTERM";
};
};
# Health check timer (optional monitoring)
systemd.timers.gitea-mirror-healthcheck = mkIf cfg.enable {
description = "Gitea Mirror health check timer";
wantedBy = [ "timers.target" ];
timerConfig = {
OnBootSec = "5min";
OnUnitActiveSec = "5min";
};
};
systemd.services.gitea-mirror-healthcheck = mkIf cfg.enable {
description = "Gitea Mirror health check";
after = [ "gitea-mirror.service" ];
serviceConfig = {
Type = "oneshot";
ExecStart = "${pkgs.curl}/bin/curl -f http://${cfg.host}:${toString cfg.port}/api/health || true";
User = "nobody";
};
};
networking.firewall = mkIf cfg.openFirewall {
allowedTCPPorts = [ cfg.port ];
};
};
};
}
) // {
# Overlay for adding to nixpkgs
overlays.default = final: prev: {
gitea-mirror = self.packages.${final.system}.default;
};
};
}

View File

@@ -0,0 +1,21 @@
---
extends: default
ignore: |
.yamllint
node_modules
templates
unittests/bash
rules:
truthy:
allowed-values: ['true', 'false']
check-keys: False
level: error
line-length: disable
document-start: disable
comments:
min-spaces-from-content: 1
braces:
max-spaces-inside: 2

View File

@@ -0,0 +1,12 @@
apiVersion: v2
name: gitea-mirror
description: Kubernetes helm chart for gitea-mirror
type: application
version: 0.0.1
appVersion: 3.7.2
icon: https://github.com/RayLabsHQ/gitea-mirror/blob/main/.github/assets/logo.png
keywords:
- git
- gitea
sources:
- https://github.com/RayLabsHQ/gitea-mirror

View File

@@ -0,0 +1,307 @@
# gitea-mirror (Helm Chart)
Deploy **gitea-mirror** to Kubernetes using Helm. The chart packages a Deployment, Service, optional Ingress or Gateway API HTTPRoutes, ConfigMap and Secret, a PVC (optional), and an optional ServiceAccount.
- **Chart name:** `gitea-mirror`
- **Type:** `application`
- **App version:** `3.7.2` (default image tag, can be overridden)
---
## Prerequisites
- Kubernetes 1.23+
- Helm 3.8+
- (Optional) Gateway API (v1) if you plan to use `route.*` HTTPRoutes, see https://github.com/kubernetes-sigs/gateway-api/
- (Optional) An Ingress controller if you plan to use `ingress.*`
---
## Quick start
From the repo root (chart path: `helm/gitea-mirror`):
```bash
# Create a namespace (optional)
kubectl create namespace gitea-mirror
# Install with minimal required secrets/values
helm upgrade --install gitea-mirror ./helm/gitea-mirror --namespace gitea-mirror --set "gitea-mirror.github.username=<your-gh-username>" --set "gitea-mirror.github.token=<your-gh-token>" --set "gitea-mirror.gitea.url=https://gitea.example.com" --set "gitea-mirror.gitea.token=<your-gitea-token>"
```
The default Service is `ClusterIP` on port `4321`. You can expose it via Ingress or Gateway API; see below.
---
## Upgrading
Standard Helm upgrade:
```bash
helm upgrade gitea-mirror ./helm/gitea-mirror -n gitea-mirror
```
If you change persistence settings or storage class, a rollout may require PVC recreation.
---
## Uninstalling
```bash
helm uninstall gitea-mirror -n gitea-mirror
```
If you enabled persistence with a PVC the data may persist; delete the PVC manually if you want a clean slate.
---
## Configuration
### Global image & pod settings
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `image.registry` | string | `ghcr.io` | Container registry. |
| `image.repository` | string | `raylabshq/gitea-mirror` | Image repository. |
| `image.tag` | string | `""` | Image tag; when empty, uses the chart `appVersion` (`3.7.2`). |
| `image.pullPolicy` | string | `IfNotPresent` | K8s image pull policy. |
| `imagePullSecrets` | list | `[]` | Image pull secrets. |
| `podSecurityContext.runAsUser` | int | `1001` | UID. |
| `podSecurityContext.runAsGroup` | int | `1001` | GID. |
| `podSecurityContext.fsGroup` | int | `1001` | FS group. |
| `podSecurityContext.fsGroupChangePolicy` | string | `OnRootMismatch` | FS group change policy. |
| `nodeSelector` / `tolerations` / `affinity` / `topologySpreadConstraints` | — | — | Standard scheduling knobs. |
| `extraVolumes` / `extraVolumeMounts` | list | `[]` | Append custom volumes/mounts. |
| `priorityClassName` | string | `""` | Optional Pod priority class. |
### Deployment
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `deployment.port` | int | `4321` | Container port & named `http` port. |
| `deployment.strategy.type` | string | `Recreate` | Update strategy (`Recreate` or `RollingUpdate`). |
| `deployment.strategy.rollingUpdate.maxUnavailable/maxSurge` | string/int | — | Used when `type=RollingUpdate`. |
| `deployment.env` | list | `[]` | Extra environment variables. |
| `deployment.resources` | map | `{}` | CPU/memory requests & limits. |
| `deployment.terminationGracePeriodSeconds` | int | `60` | Grace period. |
| `livenessProbe.*` | — | enabled, `/api/health` | Liveness probe (HTTP GET to `/api/health`). |
| `readinessProbe.*` | — | enabled, `/api/health` | Readiness probe. |
| `startupProbe.*` | — | enabled, `/api/health` | Startup probe. |
> The Pod mounts a volume at `/app/data` (PVC or `emptyDir` depending on `persistence.enabled`).
### Service
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `service.type` | string | `ClusterIP` | Service type. |
| `service.port` | int | `4321` | Service port. |
| `service.clusterIP` | string | `None` | ClusterIP (only when `type=ClusterIP`). |
| `service.externalTrafficPolicy` | string | `""` | External traffic policy (LB). |
| `service.loadBalancerIP` | string | `""` | LoadBalancer IP. |
| `service.loadBalancerClass` | string | `""` | LoadBalancer class. |
| `service.annotations` / `service.labels` | map | `{}` | Extra metadata. |
### Ingress (optional)
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `ingress.enabled` | bool | `false` | Enable Ingress. |
| `ingress.className` | string | `""` | IngressClass name. |
| `ingress.hosts[0].host` | string | `mirror.example.com` | Hostname. |
| `ingress.tls` | list | `[]` | TLS blocks (secret name etc.). |
| `ingress.annotations` | map | `{}` | Controller-specific annotations. |
> The Ingress exposes `/` to the charts Service.
### Gateway API HTTPRoutes (optional)
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `route.enabled` | bool | `false` | Enable Gateway API HTTPRoutes. |
| `route.forceHTTPS` | bool | `true` | If true, create an HTTP route that redirects to HTTPS (301). |
| `route.domain` | list | `["mirror.example.com"]` | Hostnames. |
| `route.gateway` | string | `""` | Gateway name. |
| `route.gatewayNamespace` | string | `""` | Gateway namespace. |
| `route.http.gatewaySection` | string | `""` | SectionName for HTTP listener. |
| `route.https.gatewaySection` | string | `""` | SectionName for HTTPS listener. |
| `route.http.filters` / `route.https.filters` | list | `[]` | Additional filters. (Defaults add HSTS header on HTTPS.) |
### Persistence
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `persistence.enabled` | bool | `true` | Enable persistent storage. |
| `persistence.create` | bool | `true` | Create a PVC from the chart. |
| `persistence.claimName` | string | `gitea-mirror-storage` | PVC name. |
| `persistence.storageClass` | string | `""` | StorageClass to use. |
| `persistence.accessModes` | list | `["ReadWriteOnce"]` | Access modes. |
| `persistence.size` | string | `1Gi` | Requested size. |
| `persistence.volumeName` | string | `""` | Bind to existing PV by name (optional). |
| `persistence.annotations` | map | `{}` | PVC annotations. |
### ServiceAccount (optional)
| Key | Type | Default | Description |
| --- | --- | --- | --- |
| `serviceAccount.create` | bool | `false` | Create a ServiceAccount. |
| `serviceAccount.name` | string | `""` | SA name (defaults to release fullname). |
| `serviceAccount.automountServiceAccountToken` | bool | `false` | Automount token. |
| `serviceAccount.annotations` / `labels` | map | `{}` | Extra metadata. |
---
## Application configuration (`gitea-mirror.*`)
These values populate a **ConfigMap** (non-secret) and a **Secret** (for tokens and sensitive fields). Environment variables from both are consumed by the container.
### Core
| Key | Default | Mapped env |
| --- | --- | --- |
| `gitea-mirror.nodeEnv` | `production` | `NODE_ENV` |
| `gitea-mirror.core.databaseUrl` | `file:data/gitea-mirror.db` | `DATABASE_URL` |
| `gitea-mirror.core.encryptionSecret` | `""` | `ENCRYPTION_SECRET` (Secret) |
| `gitea-mirror.core.betterAuthSecret` | `""` | `BETTER_AUTH_SECRET` |
| `gitea-mirror.core.betterAuthUrl` | `http://localhost:4321` | `BETTER_AUTH_URL` |
| `gitea-mirror.core.betterAuthTrustedOrigins` | `http://localhost:4321` | `BETTER_AUTH_TRUSTED_ORIGINS` |
### GitHub
| Key | Default | Mapped env |
| --- | --- | --- |
| `gitea-mirror.github.username` | `""` | `GITHUB_USERNAME` |
| `gitea-mirror.github.token` | `""` | `GITHUB_TOKEN` (Secret) |
| `gitea-mirror.github.type` | `personal` | `GITHUB_TYPE` |
| `gitea-mirror.github.privateRepositories` | `true` | `PRIVATE_REPOSITORIES` |
| `gitea-mirror.github.skipForks` | `false` | `SKIP_FORKS` |
| `gitea-mirror.github.starredCodeOnly` | `false` | `SKIP_STARRED_ISSUES` |
| `gitea-mirror.github.mirrorStarred` | `false` | `MIRROR_STARRED` |
### Gitea
| Key | Default | Mapped env |
| --- | --- | --- |
| `gitea-mirror.gitea.url` | `""` | `GITEA_URL` |
| `gitea-mirror.gitea.token` | `""` | `GITEA_TOKEN` (Secret) |
| `gitea-mirror.gitea.username` | `""` | `GITEA_USERNAME` |
| `gitea-mirror.gitea.organization` | `github-mirrors` | `GITEA_ORGANIZATION` |
| `gitea-mirror.gitea.visibility` | `public` | `GITEA_ORG_VISIBILITY` |
### Mirror options
| Key | Default | Mapped env |
| --- | --- | --- |
| `gitea-mirror.mirror.releases` | `true` | `MIRROR_RELEASES` |
| `gitea-mirror.mirror.wiki` | `true` | `MIRROR_WIKI` |
| `gitea-mirror.mirror.metadata` | `true` | `MIRROR_METADATA` |
| `gitea-mirror.mirror.issues` | `true` | `MIRROR_ISSUES` |
| `gitea-mirror.mirror.pullRequests` | `true` | `MIRROR_PULL_REQUESTS` |
| `gitea-mirror.mirror.starred` | _(see note above)_ | `MIRROR_STARRED` |
### Automation & cleanup
| Key | Default | Mapped env |
| --- | --- | --- |
| `gitea-mirror.automation.schedule_enabled` | `true` | `SCHEDULE_ENABLED` |
| `gitea-mirror.automation.schedule_interval` | `3600` | `SCHEDULE_INTERVAL` (seconds) |
| `gitea-mirror.cleanup.enabled` | `true` | `CLEANUP_ENABLED` |
| `gitea-mirror.cleanup.retentionDays` | `30` | `CLEANUP_RETENTION_DAYS` |
> **Secrets:** If you set `gitea-mirror.existingSecret` (name of an existing Secret), the chart will **not** create its own Secret and will reference yours instead. Otherwise it creates a Secret with `GITHUB_TOKEN`, `GITEA_TOKEN`, `ENCRYPTION_SECRET`.
---
## Exposing the service
### Using Ingress
```yaml
ingress:
enabled: true
className: "nginx"
hosts:
- host: mirror.example.com
tls:
- secretName: mirror-tls
hosts:
- mirror.example.com
```
This creates an Ingress routing `/` to the service on port `4321`.
### Using Gateway API (HTTPRoute)
```yaml
route:
enabled: true
domain: ["mirror.example.com"]
gateway: "my-gateway"
gatewayNamespace: "gateway-system"
http:
gatewaySection: "http"
https:
gatewaySection: "https"
# Example extra filter already included by default: add HSTS header
```
If `forceHTTPS: true`, the chart emits an HTTP route that redirects to HTTPS with 301. An HTTPS route is always created when `route.enabled=true`.
---
## Persistence & data
By default, the chart provisions a PVC named `gitea-mirror-storage` with `1Gi` and mounts it at `/app/data`. To use an existing PV or tune storage, adjust `persistence.*` in `values.yaml`. If you disable persistence, an `emptyDir` will be used instead.
---
## Environment & health endpoints
The container listens on `PORT` (defaults to `deployment.port` = `4321`) and exposes `GET /api/health` for liveness/readiness/startup probes.
---
## Examples
### Minimal (tokens via chart-managed Secret)
```yaml
gitea-mirror:
github:
username: "gitea-mirror"
token: "<gh-token>"
gitea:
url: "https://gitea.company.tld"
token: "<gitea-token>"
```
### Bring your own Secret
```yaml
gitea-mirror:
existingSecret: "gitea-mirror-secrets"
github:
username: "gitea-mirror"
gitea:
url: "https://gitea.company.tld"
```
Where `gitea-mirror-secrets` contains keys `GITHUB_TOKEN`, `GITEA_TOKEN`, `ENCRYPTION_SECRET`.
---
## Development
Lint the chart:
```bash
yamllint -c helm/gitea-mirror/.yamllint helm/gitea-mirror
```
Tweak probes, resources, and scheduling as needed; see `values.yaml`.
---
## License
This chart is part of the `RayLabsHQ/gitea-mirror` repository. See the repository for licensing details.

View File

@@ -0,0 +1,59 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "gitea-mirror.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "gitea-mirror.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "gitea-mirror.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{/*
Common labels
*/}}
{{- define "gitea-mirror.labels" -}}
helm.sh/chart: {{ include "gitea-mirror.chart" . }}
app: {{ include "gitea-mirror.name" . }}
{{ include "gitea-mirror.selectorLabels" . }}
app.kubernetes.io/version: {{ .Values.image.tag | default .Chart.AppVersion | quote }}
version: {{ .Values.image.tag | default .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}
{{/*
Selector labels
*/}}
{{- define "gitea-mirror.selectorLabels" -}}
app.kubernetes.io/name: {{ include "gitea-mirror.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}
{{/*
ServiceAccount name
*/}}
{{- define "gitea-mirror.serviceAccountName" -}}
{{ .Values.serviceAccount.name | default (include "gitea-mirror.fullname" .) }}
{{- end -}}

View File

@@ -0,0 +1,38 @@
{{- $gm := index .Values "gitea-mirror" -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "gitea-mirror.fullname" . }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
data:
NODE_ENV: {{ $gm.nodeEnv | quote }}
# Core configuration
DATABASE_URL: {{ $gm.core.databaseUrl | quote }}
BETTER_AUTH_SECRET: {{ $gm.core.betterAuthSecret | quote }}
BETTER_AUTH_URL: {{ $gm.core.betterAuthUrl | quote }}
BETTER_AUTH_TRUSTED_ORIGINS: {{ $gm.core.betterAuthTrustedOrigins | quote }}
# GitHub Config
GITHUB_USERNAME: {{ $gm.github.username | quote }}
GITHUB_TYPE: {{ $gm.github.type | quote }}
PRIVATE_REPOSITORIES: {{ $gm.github.privateRepositories | quote }}
MIRROR_STARRED: {{ $gm.github.mirrorStarred | quote }}
SKIP_FORKS: {{ $gm.github.skipForks | quote }}
SKIP_STARRED_ISSUES: {{ $gm.github.starredCodeOnly | quote }}
# Gitea Config
GITEA_URL: {{ $gm.gitea.url | quote }}
GITEA_USERNAME: {{ $gm.gitea.username | quote }}
GITEA_ORGANIZATION: {{ $gm.gitea.organization | quote }}
GITEA_ORG_VISIBILITY: {{ $gm.gitea.visibility | quote }}
# Mirror Options
MIRROR_RELEASES: {{ $gm.mirror.releases | quote }}
MIRROR_WIKI: {{ $gm.mirror.wiki | quote }}
MIRROR_METADATA: {{ $gm.mirror.metadata | quote }}
MIRROR_ISSUES: {{ $gm.mirror.issues | quote }}
MIRROR_PULL_REQUESTS: {{ $gm.mirror.pullRequests | quote }}
# Automation
SCHEDULE_ENABLED: {{ $gm.automation.schedule_enabled| quote }}
SCHEDULE_INTERVAL: {{ $gm.automation.schedule_interval | quote }}
# Cleanup
CLEANUP_ENABLED: {{ $gm.cleanup.enabled | quote }}
CLEANUP_RETENTION_DAYS: {{ $gm.cleanup.retentionDays | quote }}

View File

@@ -0,0 +1,143 @@
{{- $gm := index .Values "gitea-mirror" -}}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "gitea-mirror.fullname" . }}
{{- with .Values.deployment.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
{{- if .Values.deployment.labels }}
{{- toYaml .Values.deployment.labels | nindent 4 }}
{{- end }}
spec:
replicas: 1
strategy:
type: {{ .Values.deployment.strategy.type }}
{{- if eq .Values.deployment.strategy.type "RollingUpdate" }}
rollingUpdate:
maxUnavailable: {{ .Values.deployment.strategy.rollingUpdate.maxUnavailable }}
maxSurge: {{ .Values.deployment.strategy.rollingUpdate.maxSurge }}
{{- end }}
selector:
matchLabels:
{{- include "gitea-mirror.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{- include "gitea-mirror.labels" . | nindent 8 }}
{{- if .Values.deployment.labels }}
{{- toYaml .Values.deployment.labels | nindent 8 }}
{{- end }}
spec:
{{- if (or .Values.serviceAccount.create .Values.serviceAccount.name) }}
serviceAccountName: {{ include "gitea-mirror.serviceAccountName" . }}
{{- end }}
{{- if .Values.priorityClassName }}
priorityClassName: "{{ .Values.priorityClassName }}"
{{- end }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
terminationGracePeriodSeconds: {{ .Values.deployment.terminationGracePeriodSeconds }}
containers:
- name: gitea-mirror
image: {{ .Values.image.registry }}/{{ .Values.image.repository }}:{{ .Values.image.tag | default (printf "v%s" .Chart.AppVersion) }}
imagePullPolicy: {{ .Values.image.pullPolicy }}
envFrom:
- configMapRef:
name: {{ include "gitea-mirror.fullname" . }}
{{- if $gm.existingSecret }}
- secretRef:
name: {{ $gm.existingSecret }}
{{- else }}
- secretRef:
name: {{ include "gitea-mirror.fullname" . }}
{{- end }}
env:
- name: PORT
value: "{{ .Values.deployment.port }}"
{{- if .Values.deployment.env }}
{{- toYaml .Values.deployment.env | nindent 12 }}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.deployment.port }}
{{- if .Values.livenessProbe.enabled }}
livenessProbe:
httpGet:
path: /api/health
port: "http"
initialDelaySeconds: {{ .Values.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.livenessProbe.timeoutSeconds }}
failureThreshold: {{ .Values.livenessProbe.failureThreshold }}
successThreshold: {{ .Values.livenessProbe.successThreshold }}
{{- end }}
{{- if .Values.readinessProbe.enabled }}
readinessProbe:
httpGet:
path: /api/health
port: "http"
initialDelaySeconds: {{ .Values.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.readinessProbe.timeoutSeconds }}
failureThreshold: {{ .Values.readinessProbe.failureThreshold }}
successThreshold: {{ .Values.readinessProbe.successThreshold }}
{{- end }}
{{- if .Values.startupProbe.enabled }}
startupProbe:
httpGet:
path: /api/health
port: "http"
initialDelaySeconds: {{ .Values.startupProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.startupProbe.periodSeconds }}
timeoutSeconds: {{ .Values.startupProbe.timeoutSeconds }}
failureThreshold: {{ .Values.startupProbe.failureThreshold }}
successThreshold: {{ .Values.startupProbe.successThreshold }}
{{- end }}
volumeMounts:
- name: data
mountPath: /app/data
{{- if .Values.extraVolumeMounts }}
{{- toYaml .Values.extraVolumeMounts | nindent 12 }}
{{- end }}
{{- with .Values.deployment.resources }}
resources:
{{- toYaml .Values.deployment.resources | nindent 12 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.topologySpreadConstraints }}
topologySpreadConstraints:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
{{- if .Values.persistence.enabled }}
- name: data
persistentVolumeClaim:
claimName: {{ .Values.persistence.claimName }}
{{- else if not .Values.persistence.enabled }}
- name: data
emptyDir: {}
{{- end }}
{{- if .Values.extraVolumes }}
{{- toYaml .Values.extraVolumes | nindent 8 }}
{{- end }}

View File

@@ -0,0 +1,77 @@
{{- if .Values.route.enabled }}
{{- if .Values.route.forceHTTPS }}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: {{ include "gitea-mirror.fullname" . }}-http
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
spec:
parentRefs:
- name: {{ .Values.route.gateway }}
sectionName: {{ .Values.route.http.gatewaySection }}
namespace: {{ .Values.route.gatewayNamespace }}
hostnames: {{ .Values.route.domain }}
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
{{- with .Values.route.http.filters }}
{{ toYaml . | nindent 4 }}
{{- end }}
{{- else }}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: {{ include "gitea-mirror.fullname" . }}-http
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
spec:
parentRefs:
- name: {{ .Values.route.gateway }}
sectionName: {{ .Values.route.http.gatewaySection }}
namespace: {{ .Values.route.gatewayNamespace }}
hostnames: {{ .Values.route.domain }}
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: {{ include "gitea-mirror.fullname" . }}
port: {{ .Values.service.port }}
{{- with .Values.route.http.filters }}
filters:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: {{ include "gitea-mirror.fullname" . }}-https
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
spec:
parentRefs:
- name: {{ .Values.route.gateway }}
sectionName: {{ .Values.route.https.gatewaySection }}
namespace: {{ .Values.route.gatewayNamespace }}
hostnames: {{ .Values.route.domain }}
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: {{ include "gitea-mirror.fullname" . }}
port: {{ .Values.service.port }}
{{- with .Values.route.https.filters }}
filters:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,40 @@
{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "gitea-mirror.fullname" . }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- . | toYaml | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.className }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ tpl . $ | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ tpl .host $ | quote }}
http:
paths:
- path: {{ .path | default "/" }}
pathType: {{ .pathType | default "Prefix" }}
backend:
service:
name: {{ include "gitea-mirror.fullname" $ }}
port:
number: {{ $.Values.service.port }}
{{- end }}
{{- end }}

View File

@@ -0,0 +1,26 @@
{{- if and .Values.persistence.enabled .Values.persistence.create }}
{{- $gm := index .Values "gitea-mirror" -}}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: {{ .Values.persistence.claimName }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
{{- with .Values.persistence.annotations }}
annotations:
{{ . | toYaml | indent 4}}
{{- end }}
spec:
accessModes:
{{- toYaml .Values.persistence.accessModes | nindent 4 }}
{{- with .Values.persistence.storageClass }}
storageClassName: {{ . }}
{{- end }}
volumeMode: Filesystem
{{- with .Values.persistence.volumeName }}
volumeName: {{ . }}
{{- end }}
resources:
requests:
storage: {{ .Values.persistence.size }}
{{- end }}

View File

@@ -0,0 +1,14 @@
{{- $gm := index .Values "gitea-mirror" -}}
{{- if (empty $gm.existingSecret) }}
apiVersion: v1
kind: Secret
metadata:
name: {{ include "gitea-mirror.fullname" . }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
type: Opaque
stringData:
GITHUB_TOKEN: {{ $gm.github.token | quote }}
GITEA_TOKEN: {{ $gm.gitea.token | quote }}
ENCRYPTION_SECRET: {{ $gm.core.encryptionSecret | quote }}
{{- end }}

View File

@@ -0,0 +1,34 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "gitea-mirror.fullname" . }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
{{- if .Values.service.labels }}
{{- toYaml .Values.service.labels | nindent 4 }}
{{- end }}
annotations:
{{- toYaml .Values.service.annotations | nindent 4 }}
spec:
type: {{ .Values.service.type }}
{{- if eq .Values.service.type "LoadBalancer" }}
{{- if .Values.service.loadBalancerClass }}
loadBalancerClass: {{ .Values.service.loadBalancerClass }}
{{- end }}
{{- if and .Values.service.loadBalancerIP }}
loadBalancerIP: {{ .Values.service.loadBalancerIP }}
{{- end }}
{{- end }}
{{- if .Values.service.externalTrafficPolicy }}
externalTrafficPolicy: {{ .Values.service.externalTrafficPolicy }}
{{- end }}
{{- if and .Values.service.clusterIP (eq .Values.service.type "ClusterIP") }}
clusterIP: {{ .Values.service.clusterIP }}
{{- end }}
ports:
- name: http
port: {{ .Values.service.port }}
protocol: TCP
targetPort: http
selector:
{{- include "gitea-mirror.selectorLabels" . | nindent 4 }}

View File

@@ -0,0 +1,17 @@
{{- if .Values.serviceAccount.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "gitea-mirror.serviceAccountName" . }}
labels:
{{- include "gitea-mirror.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.labels }}
{{- . | toYaml | nindent 4 }}
{{- end }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- . | toYaml | nindent 4 }}
{{- end }}
automountServiceAccountToken: {{ .Values.serviceAccount.automountServiceAccountToken }}
{{- end }}

View File

@@ -0,0 +1,151 @@
image:
registry: ghcr.io
repository: raylabshq/gitea-mirror
# Leave blank to use the Appversion tag
tag: ""
pullPolicy: IfNotPresent
imagePullSecrets: []
podSecurityContext:
runAsUser: 1001
runAsGroup: 1001
fsGroup: 1001
fsGroupChangePolicy: OnRootMismatch
ingress:
enabled: false
className: ""
annotations: {}
hosts:
- host: mirror.example.com
paths:
- path: /
pathType: Prefix
tls: []
# - secretName: chart-example-tls
# hosts:
# - mirror.example.com
route:
enabled: false
forceHTTPS: true
domain: ["mirror.example.com"]
gateway: ""
gatewayNamespace: ""
http:
gatewaySection: ""
filters: []
https:
gatewaySection: ""
filters:
- type: ResponseHeaderModifier
responseHeaderModifier:
add:
- name: Strict-Transport-Security
value: "max-age=31536000; includeSubDomains; preload"
service:
type: ClusterIP
port: 4321
clusterIP: None
annotations: {}
externalTrafficPolicy:
labels: {}
loadBalancerIP:
loadBalancerClass:
deployment:
port: 4321
strategy:
type: Recreate
env: []
terminationGracePeriodSeconds: 60
labels: {}
annotations: {}
resources: {}
livenessProbe:
enabled: true
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
readinessProbe:
enabled: true
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
startupProbe:
enabled: true
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
persistence:
enabled: true
create: true
claimName: gitea-mirror-storage
storageClass: ""
accessModes:
- ReadWriteOnce
size: 1Gi
affinity: {}
nodeSelector: {}
tolerations: []
topologySpreadConstraints: []
extraVolumes: []
extraVolumeMounts: []
serviceAccount:
create: false
name: ""
annotations: {}
labels: {}
automountServiceAccountToken: false
gitea-mirror:
existingSecret: ""
nodeEnv: production
core:
databaseUrl: file:data/gitea-mirror.db
encryptionSecret: ""
betterAuthSecret: ""
betterAuthUrl: "http://localhost:4321"
betterAuthTrustedOrigins: "http://localhost:4321"
github:
username: ""
token: ""
type: personal
privateRepositories: true
mirrorStarred: false
skipForks: false
starredCodeOnly: false
gitea:
url: ""
token: ""
username: ""
organization: "github-mirrors"
visibility: "public"
mirror:
releases: true
wiki: true
metadata: true
issues: true
pullRequests: true
automation:
schedule_enabled: true
schedule_interval: 3600
cleanup:
enabled: true
retentionDays: 30

View File

@@ -0,0 +1,115 @@
{
"name": "gitea-mirror",
"type": "module",
"version": "3.9.2",
"engines": {
"bun": ">=1.2.9"
},
"scripts": {
"setup": "bun install && bun run manage-db init",
"dev": "bunx --bun astro dev",
"dev:clean": "bun run cleanup-db && bun run manage-db init && bunx --bun astro dev",
"build": "bunx --bun astro build",
"cleanup-db": "rm -f gitea-mirror.db data/gitea-mirror.db",
"manage-db": "bun scripts/manage-db.ts",
"init-db": "bun scripts/manage-db.ts init",
"check-db": "bun scripts/manage-db.ts check",
"fix-db": "bun scripts/manage-db.ts fix",
"reset-users": "bun scripts/manage-db.ts reset-users",
"db:generate": "bun drizzle-kit generate",
"db:migrate": "bun drizzle-kit migrate",
"db:push": "bun drizzle-kit push",
"db:pull": "bun drizzle-kit pull",
"db:check": "bun drizzle-kit check",
"db:studio": "bun drizzle-kit studio",
"startup-recovery": "bun scripts/startup-recovery.ts",
"startup-recovery-force": "bun scripts/startup-recovery.ts --force",
"startup-env-config": "bun scripts/startup-env-config.ts",
"test-recovery": "bun scripts/test-recovery.ts",
"test-recovery-cleanup": "bun scripts/test-recovery.ts --cleanup",
"test-shutdown": "bun scripts/test-graceful-shutdown.ts",
"test-shutdown-cleanup": "bun scripts/test-graceful-shutdown.ts --cleanup",
"preview": "bunx --bun astro preview",
"start": "bun dist/server/entry.mjs",
"start:fresh": "bun run cleanup-db && bun run manage-db init && bun dist/server/entry.mjs",
"test": "bun test",
"test:watch": "bun test --watch",
"test:coverage": "bun test --coverage",
"astro": "bunx --bun astro"
},
"overrides": {
"@esbuild-kit/esm-loader": "npm:tsx@^4.21.0",
"devalue": "^5.5.0"
},
"dependencies": {
"@astrojs/check": "^0.9.6",
"@astrojs/mdx": "4.3.12",
"@astrojs/node": "9.5.1",
"@astrojs/react": "^4.4.2",
"@better-auth/sso": "1.4.5",
"@octokit/plugin-throttling": "^11.0.3",
"@octokit/rest": "^22.0.1",
"@radix-ui/react-accordion": "^1.2.12",
"@radix-ui/react-avatar": "^1.1.11",
"@radix-ui/react-checkbox": "^1.3.3",
"@radix-ui/react-collapsible": "^1.1.12",
"@radix-ui/react-dialog": "^1.1.15",
"@radix-ui/react-dropdown-menu": "^2.1.16",
"@radix-ui/react-hover-card": "^1.1.15",
"@radix-ui/react-label": "^2.1.8",
"@radix-ui/react-popover": "^1.1.15",
"@radix-ui/react-progress": "^1.1.8",
"@radix-ui/react-radio-group": "^1.3.8",
"@radix-ui/react-scroll-area": "^1.2.10",
"@radix-ui/react-select": "^2.2.6",
"@radix-ui/react-separator": "^1.1.8",
"@radix-ui/react-slot": "^1.2.4",
"@radix-ui/react-switch": "^1.2.6",
"@radix-ui/react-tabs": "^1.1.13",
"@radix-ui/react-tooltip": "^1.2.8",
"@tailwindcss/vite": "^4.1.17",
"@tanstack/react-virtual": "^3.13.12",
"@types/canvas-confetti": "^1.9.0",
"@types/react": "^19.2.7",
"@types/react-dom": "^19.2.3",
"astro": "^5.16.4",
"bcryptjs": "^3.0.3",
"buffer": "^6.0.3",
"better-auth": "1.4.5",
"canvas-confetti": "^1.9.4",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cmdk": "^1.1.1",
"dotenv": "^17.2.3",
"drizzle-orm": "^0.44.7",
"fuse.js": "^7.1.0",
"jsonwebtoken": "^9.0.3",
"lucide-react": "^0.555.0",
"next-themes": "^0.4.6",
"react": "^19.2.1",
"react-dom": "^19.2.1",
"react-icons": "^5.5.0",
"sonner": "^2.0.7",
"tailwind-merge": "^3.4.0",
"tailwindcss": "^4.1.17",
"tw-animate-css": "^1.4.0",
"typescript": "^5.9.3",
"uuid": "^13.0.0",
"vaul": "^1.1.2",
"zod": "^4.1.13"
},
"devDependencies": {
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^16.3.0",
"@types/bcryptjs": "^3.0.0",
"@types/bun": "^1.3.3",
"@types/jsonwebtoken": "^9.0.10",
"@types/uuid": "^10.0.0",
"@vitejs/plugin-react": "^5.1.1",
"drizzle-kit": "^0.31.7",
"jsdom": "^27.2.0",
"tsx": "^4.21.0",
"vitest": "^4.0.15"
},
"packageManager": "bun@1.3.3"
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Some files were not shown because too many files have changed in this diff Show More