Rockchip RKNN

Conda

https://docs.conda.io/projects/miniconda/en/latest/

mkdir -p ~/miniconda3
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh -O ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh

After install :

~/miniconda3/bin/conda init bash
~/miniconda3/bin/conda init zsh

Rockchip toolkit

https://github.com/rockchip-linux/rknn-toolkit2

sudo apt-get install libxslt1-dev zlib1g zlib1g-dev libglib2.0-0 \
libsm6 libgl1-mesa-glx libprotobuf-dev gcc g++

# Create a new environment
conda create -n rknn2 python=3.9
conda activate rknn2

# RKNN-Toolkit2 has a specific dependency on numpy
pip install numpy==1.16.6

# Install rknn_toolkit2-1.6.0+81f21f4d-cp39-cp39-linux_x86_64.whl 
cd ~/rknn-toolkit2/rknn-toolkit2/packages/
pip install rknn_toolkit2-1.6.0+81f21f4d-cp39-cp39-linux_x86_64.whl

RKNN

Installing RKNN LLM and RKNN Toolkit 2

Réf. : https://github.com/Pelochus/ezrknpu

sudo apt install cmake g++ make libgl1 libglx-mesa0 -y

cd /mnt/data
sudo curl https://raw.githubusercontent.com/Pelochus/ezrknpu/main/install.sh | sudo bash

Test run

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Pelochus/qwen-1_8B-rk3588 # Running git lfs pull after is usually better
cd qwen-1_8B-rk3588 && git lfs pull # Pull model
rkllm qwen-chat-1_8B.rkllm # Run!

Installation of Miniforge3

wget -c https://github.com/conda-forge/miniforge/releases/download/24.9.0-0/Miniforge3-24.9.0-0-Linux-aarch64.sh

chmod 777 Miniforge3-24.9.0-0-Linux-aarch64.sh

bash Miniforge3-24.9.0-0-Linux-aarch64.sh

Create RKLLM-Toolkit Conda Environment

source ~/miniforge3/bin/activate

conda create -n RKLLM-Toolkit python=3.8

#
# To activate this environment, use
#
#     $ conda activate RKLLM-Toolkit
#
# To deactivate an active environment, use
#
#     $ conda deactivate

conda activate RKLLM-Toolkit

Install RKLLM-Toolkit

pip3 install rkllm_toolkit-x.x.x-cp38-cp38-linux_x86_64.whl

Install OpenCL

Download the ARM Mali GPU blob from rockchip's repository and put it into /usr/lib/ as follows. And install the firmware for the GPU if not already installed.

cd /usr/lib && sudo wget https://github.com/JeffyCN/mirrors/raw/libmali/lib/aarch64-linux-gnu/libmali-valhall-g610-g6p0-x11-wayland-gbm.so
cd /lib/firmware && sudo wget https://github.com/JeffyCN/mirrors/raw/libmali/firmware/g610/mali_csffw.bin

Add the Mali GPU blob to the OpenCL ICD config file as follows;

sudo apt install mesa-opencl-icd clinfo

On Ubuntu, you may get not found errors, especially huawei links. Ignore them. It looks OK. Proceed with the followings;

sudo mkdir -p /etc/OpenCL/vendors
echo "/usr/lib/libmali-valhall-g610-g6p0-x11-wayland-gbm.so" | sudo tee /etc/OpenCL/vendors/mali.icd

Set the dependencies of the Mali OpenCL as follows;

sudo apt install libxcb-dri2-0 libxcb-dri3-0 libwayland-client0 libwayland-server0 libx11-xcb1

Now you can run "clinfo" to check whether OpenCL is working.

clinfo...

Check whether some dependencies are missing using ldd command as follows;

ldd /usr/lib/libmali-valhall-g610-g6p0-x11-wayland-gbm.so

Create Symbolic Link for libOpenCL.so

The directory "/usr/lib/aarch64-linux-gnu/" will have "libOpenCL.so.1.0.0". But no "libOpenCL.so" file. In this case, create a symbolic link as follows. You need to log into root account to create this, say, "su -";

cd /usr/lib/aarch64-linux-gnu/ && sudo ln -s libOpenCL.so.1.0.0 libOpenCL.so

Copying OpenCL "CL" Folder into "/usr/include"

cd ~/ && wget https://www.roselladb.com/download/CLv3.zip
unzip CLv3.zip
sudo cp -r CL /usr/include
sudo reboot

4.0 KiB Raw Permalink Blame History