diff --git a/TuringPi/docker-swarm.md b/TuringPi/docker-swarm.md index 5cc1546..9bb60d6 100644 --- a/TuringPi/docker-swarm.md +++ b/TuringPi/docker-swarm.md @@ -110,7 +110,7 @@ sudo iptables -I INPUT -p 50 -j ACCEPT # allows ipsec when secure overlay is ena ### Install docker... -``` +```bash sudo apt install -y docker docker-compose git sudo usermod -aG docker pleb @@ -125,23 +125,23 @@ sudo chmod 666 /var/run/docker.sock #### On master -``` +```bash docker swarm init --advertise-addr 192.168.1.14 ``` -> *Swarm initialized: current node (3kdxixaa86m8pvag6jn0b70ut) is now a manager*. +> *Swarm initialized: current node (z76p66jcv4c1c9qstodlpcfii) is now a manager.* #### On nodes -``` -docker swarm join --token SWMTKN-1-2px1bindhl41x9h6l4ve7x15iwjryr0uf3ekmu7hz4bezjewwh-ae9vv4657zcki160s71vjn75z 192.168.1.14:2377 +```bash +docker swarm join --token SWMTKN-1-27tm53ylrwyesiq3ntfumfixw3gpe2164rnsume8sg07saz12v-1fig1jzehaocodm6hv322l4ku 192.168.1.14:2377 ``` > *To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions*. > Change ip range if necessary ?! -``` +```bash docker network rm ingress # Create in different range docker network create --driver overlay --ingress --subnet 192.168.1.0/16 --gateway 192.168.1.254 ingress @@ -149,27 +149,45 @@ docker network create --driver overlay --ingress --subnet 192.168.1.0/16 --gatew --- -``` +```bash docker node ls +``` +``` +ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION +lqu5nr20g6vt2g1hc5xwm2njg gary Ready Active 29.6.1 +3aqxc8mj1wk2qp1k0llodgyxw pearl Ready Active 29.6.1 +z76p66jcv4c1c9qstodlpcfii * sandy Ready Active Leader 29.6.1 +i62rvg0fh6c0hv49btb1qrfv4 sheldon Ready Active 29.6.1 +``` +```bash docker network ls ``` - +``` +NETWORK ID NAME DRIVER SCOPE +3b8a522fd1ad bridge bridge local +8624696f7cbd docker_gwbridge bridge local +116b95142a3a host host local +jg83q53aljof ingress overlay swarm +46f3064ef5dc none null local +``` --- ## 3. Keepalived > *Load Balancer : install on all nodes* -``` -sudo apt-get -y install keepalived +```bash +sudo apt install keepalived -y ``` ### On master node (sandy) #### Create/Edit -> sudo vim /etc/keepalived/keepalived.conf +```bash +sudo vim /etc/keepalived/keepalived.conf +``` ``` ! Configuration File for keepalived @@ -208,7 +226,9 @@ vrrp_instance VI_1 { #### Create/Edit -> sudo vim /etc/keepalived/keepalived.conf +```bash +sudo vim /etc/keepalived/keepalived.conf +``` ``` ! Configuration File for keepalived @@ -278,7 +298,7 @@ sudo mkdir -p /mnt/ssd echo "UUID=... /mnt/ssd xfs defaults 0 0" | sudo tee -a /etc/fstab # Create brick folder -# sudo mkdir -p /mnt/ssd/brick +sudo mkdir -p /mnt/ssd/brick # Mount sudo mount -a @@ -292,24 +312,48 @@ sudo gluster peer probe pearl > Check -``` -df -h /data/glusterfs/volume/brick +```bash +df -h /mnt/ssd/brick sudo gluster pool list +``` +``` +UUID Hostname State +a40ae0e1-90ec-4ccb-88eb-0a13c6c0de26 gary Connected +fc215f6f-38df-4cce-a0d3-5297e0290d3f sheldon Connected +13233744-1367-4e76-b486-a0e0b8a190e7 pearl Connected +07702514-efbc-4ca6-950d-d9c0b14ff61c localhost Connected +``` +```bash sudo gluster peer status ``` +``` +Number of Peers: 3 + +Hostname: gary +Uuid: a40ae0e1-90ec-4ccb-88eb-0a13c6c0de26 +State: Peer in Cluster (Connected) + +Hostname: sheldon +Uuid: fc215f6f-38df-4cce-a0d3-5297e0290d3f +State: Peer in Cluster (Connected) + +Hostname: pearl +Uuid: 13233744-1367-4e76-b486-a0e0b8a190e7 +State: Peer in Cluster (Connected) + > For 2 nodes: sandy and sheldon for example -``` -sudo gluster volume create docker-volume replica 2 transport tcp sandy:/mnt/ssd sheldon:/mnt/data force +```bash +sudo gluster volume create docker-volume replica 2 transport tcp sandy:/mnt/ssd sheldon:/mnt/ssd force ``` -> For 3 nodes: sandy, sheldon and pearl for example +> For all nodes: sandy, gary, sheldon and pearl for example -``` -sudo gluster volume create dockervolume disperse 3 redundancy 1 sandy:/data/glusterfs/volume/brick sheldon:/data/glusterfs/volume/brick pearl:/data/glusterfs/volume/brick force +```bash +sudo gluster volume create dockervolume disperse 4 redundancy 1 sandy:/mnt/ssd/brick gary:/mnt/ssd/brick sheldon:/mnt/ssd/brick pearl:/mnt/ssd/brick force ``` #### Start Gluster Volume @@ -323,9 +367,9 @@ sudo gluster volume start dockervolume > On all nodes : ``` -sudo mkdir /mnt/data +sudo mkdir /mnt/ssd -echo "localhost:/dockervolume /mnt/data glusterfs defaults,_netdev 0 0" | sudo tee -a /etc/fstab +echo "localhost:/dockervolume /mnt/ssd/brick glusterfs defaults,_netdev 0 0" | sudo tee -a /etc/fstab sudo mount -a @@ -334,10 +378,10 @@ sudo systemctl daemon-reload > Check -> > Persistent shared storage will be "/mnt/data" +> > Persistent shared storage will be "/mnt/ssd/brick" ``` -df -h /mnt/data +df -h /mnt/ssd/brick sudo gluster volume set dockervolume auth.allow 127.0.0.1,192.168.1.14,192.168.1.13,192.168.1.16,192.168.1.19 ``` @@ -351,7 +395,25 @@ sudo gluster volume add-brick **volume** replica X **server:brick** > Option ``` -docker plugin install --alias glusterfs mikebarkmin/glusterfs SERVERS=sandy,gary,sheldon,pearl VOLNAME=dockervolume +docker plugin install --grant-all-permissions \ + trajano/glusterfs-volume-plugin +``` + +# Use in a stack file + +```ỳaml +services: + app: + image: myapp/web:latest + volumes: + - app_data:/app/data + +volumes: + app_data: + driver: trajano/glusterfs-volume-plugin + driver_opts: + glusteropts: "--volfile-server=node-01 --volfile-id=swarm-vol" + subdir: "app_data" ``` ### 5. Portainer @@ -361,7 +423,7 @@ Create container directory ``` sudo mkdir -p /var/lib/docker/volumes -sudo mkdir -p /mnt/data/portainer +sudo mkdir -p /mnt/ssd/brick/portainer ``` > Edit @@ -383,7 +445,7 @@ volumes: ``` volumes: - type: bind - source: /mnt/data/portainer/portainer_data + source: /mnt/ssd/brick/portainer/portainer_data target: /data ```