Added Divers

Divers/Local AI.md

@@ -0,0 +1,43 @@
+# LocalAI
+
+```
+sudo mkdir -p /mnt/data/localai
+```
+
+```bash
+cd /mnt/data
+
+git clone https://github.com/go-skynet/LocalAI
+
+cd LocalAI
+
+# (optional) Checkout a specific LocalAI tag
+# git checkout -b build <TAG>
+
+# copy your models to models/
+wget https://gpt4all.io/models/ggml-gpt4all-j.bin -O models/ggml-gpt4all-j
+
+# cp your-model.bin models/
+
+# Use a template from the examples
+cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/
+
+# (optional) Edit the .env file to set things like context size and threads
+# vim .env
+
+docker-compose up -d --build
+
+# Now API is accessible at localhost:8080
+curl http://localhost:8080/v1/models
+# {"object":"list","data":[{"id":"ggml-gpt4all-j","object":"model"}]}
+
+curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
+     "model": "ggml-gpt4all-j",
+     "messages": [{"role": "user", "content": "How are you?"}],
+     "temperature": 0.9 
+   }'
+
+# {"model":"ggml-gpt4all-j","choices":[{"message":{"role":"assistant","content":"I'm doing well, thanks. How about you?"}}]}
+```
+
+[https://localai.io](https://localai.io "https://localai.io")

Divers/Rockchip RKNN.md

@@ -0,0 +1,156 @@
+# Rockchip RKNN
+
+#### Conda
+
+> [https://docs.conda.io/projects/miniconda/en/latest/](https://docs.conda.io/projects/miniconda/en/latest/ "https://docs.conda.io/projects/miniconda/en/latest/")
+
+```
+mkdir -p ~/miniconda3
+wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O ~/miniconda3/miniconda.sh
+bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
+rm -rf ~/miniconda3/miniconda.sh
+```
+
+> After install :
+
+```
+~/miniconda3/bin/conda init bash
+~/miniconda3/bin/conda init zsh
+```
+
+### Rockchip toolkit
+
+> [https://github.com/rockchip-linux/rknn-toolkit2](https://github.com/rockchip-linux/rknn-toolkit2 "https://github.com/rockchip-linux/rknn-toolkit2")
+
+```
+sudo apt-get install libxslt1-dev zlib1g zlib1g-dev libglib2.0-0 \
+libsm6 libgl1-mesa-glx libprotobuf-dev gcc g++
+
+# Create a new environment
+conda create -n rknn2 python=3.9
+conda activate rknn2
+
+# RKNN-Toolkit2 has a specific dependency on numpy
+pip install numpy==1.16.6
+
+# Install rknn_toolkit2-1.6.0+81f21f4d-cp39-cp39-linux_x86_64.whl 
+cd ~/rknn-toolkit2/rknn-toolkit2/packages/
+pip install rknn_toolkit2-1.6.0+81f21f4d-cp39-cp39-linux_x86_64.whl
+```
+
+----------
+
+# RKNN
+
+##### Installing RKNN LLM and RKNN Toolkit 2
+
+Réf. : [https://github.com/Pelochus/ezrknpu](https://github.com/Pelochus/ezrknpu)
+
+```bash
+sudo apt install cmake g++ make libgl1 libglx-mesa0 -y
+
+cd /mnt/data
+sudo curl https://raw.githubusercontent.com/Pelochus/ezrknpu/main/install.sh | sudo bash
+```
+
+##### Test run
+
+```bash
+GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Pelochus/qwen-1_8B-rk3588 # Running git lfs pull after is usually better
+cd qwen-1_8B-rk3588 && git lfs pull # Pull model
+rkllm qwen-chat-1_8B.rkllm # Run!
+```
+
+---
+
+##### Installation of Miniforge3
+
+```bash
+wget -c https://github.com/conda-forge/miniforge/releases/download/24.9.0-0/Miniforge3-24.9.0-0-Linux-aarch64.sh
+
+chmod 777 Miniforge3-24.9.0-0-Linux-aarch64.sh
+
+bash Miniforge3-24.9.0-0-Linux-aarch64.sh
+```
+
+##### Create RKLLM-Toolkit Conda Environment
+
+```bash
+source ~/miniforge3/bin/activate
+
+conda create -n RKLLM-Toolkit python=3.8
+
+#
+# To activate this environment, use
+#
+#     $ conda activate RKLLM-Toolkit
+#
+# To deactivate an active environment, use
+#
+#     $ conda deactivate
+
+conda activate RKLLM-Toolkit
+```
+
+##### Install RKLLM-Toolkit
+
+```bash
+pip3 install rkllm_toolkit-x.x.x-cp38-cp38-linux_x86_64.whl
+```
+
+##### Install OpenCL
+
+Download the ARM Mali GPU blob from rockchip's repository and put it into /usr/lib/ as follows. And install the firmware for the GPU if not already installed.
+
+```bash
+cd /usr/lib && sudo wget https://github.com/JeffyCN/mirrors/raw/libmali/lib/aarch64-linux-gnu/libmali-valhall-g610-g6p0-x11-wayland-gbm.so
+cd /lib/firmware && sudo wget https://github.com/JeffyCN/mirrors/raw/libmali/firmware/g610/mali_csffw.bin
+```
+
+Add the Mali GPU blob to the OpenCL ICD config file as follows;
+
+```
+sudo apt install mesa-opencl-icd clinfo
+```
+
+On Ubuntu, you may get not found errors, especially huawei links. Ignore them. It looks OK. Proceed with the followings;
+
+```bash
+sudo mkdir -p /etc/OpenCL/vendors
+echo "/usr/lib/libmali-valhall-g610-g6p0-x11-wayland-gbm.so" | sudo tee /etc/OpenCL/vendors/mali.icd
+```
+
+Set the dependencies of the Mali OpenCL as follows;
+
+```bash
+sudo apt install libxcb-dri2-0 libxcb-dri3-0 libwayland-client0 libwayland-server0 libx11-xcb1
+```
+
+Now you can run "clinfo" to check whether OpenCL is working.
+
+```bash
+clinfo...
+```
+
+Check whether some dependencies are missing using ldd command as follows;
+
+```
+ldd /usr/lib/libmali-valhall-g610-g6p0-x11-wayland-gbm.so
+```
+
+Create Symbolic Link for libOpenCL.so
+
+The directory "/usr/lib/aarch64-linux-gnu/" will have "libOpenCL.so.1.0.0". But no "libOpenCL.so" file. In this case, create a symbolic link as follows. You need to log into root account to create this, say, "su -";
+
+```bash
+cd /usr/lib/aarch64-linux-gnu/ && sudo ln -s libOpenCL.so.1.0.0 libOpenCL.so
+```
+
+Copying OpenCL "CL" Folder into "/usr/include"
+
+```bash
+cd ~/ && wget https://www.roselladb.com/download/CLv3.zip
+unzip CLv3.zip
+sudo cp -r CL /usr/include
+sudo reboot
+```