Python版本：3.10
CANN版本：8.0.rc3
操作系统版本：ubuntu_22.04
mindie_1.0.T65

## 910A_MindIE_1.0.t65推理 1. 拉取镜像 ``` docker pull swr.cn-central-221.ovaijisuan.com/wh-aicc-fae/mindie:910a-ascend_23.0.0-cann_8.0.rc3-py_3.10-ubuntu_22.04-aarch64-mindie_1.0.t65 ``` 2. 启动容器，注意添加反斜杠 ``` docker run -it --name 910A_MindIE_1.0.T65_aarch64_dev0-7 --ipc=host --net=host \ --device=/dev/davinci0 \ --device=/dev/davinci1 \ --device=/dev/davinci2 \ --device=/dev/davinci3 \ --device=/dev/davinci4 \ --device=/dev/davinci5 \ --device=/dev/davinci6 \ --device=/dev/davinci7 \ --device=/dev/davinci_manager \ --device=/dev/devmm_svm \ --device=/dev/hisi_hdc \ -v /usr/local/dcmi:/usr/local/dcmi \ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \ -v /usr/local/Ascend/driver/lib64/common:/usr/local/Ascend/driver/lib64/common \ -v /usr/local/Ascend/driver/lib64/driver:/usr/local/Ascend/driver/lib64/driver \ -v /etc/ascend_install.info:/etc/ascend_install.info \ -v /etc/vnpu.cfg:/etc/vnpu.cfg \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /home/aicc:/home/aicc \ swr.cn-central-221.ovaijisuan.com/wh-aicc-fae/mindie:910a-ascend_23.0.0-cann_8.0.rc3-py_3.10-ubuntu_22.04-aarch64-mindie_1.0.t65 /bin/bash ``` 3. 准备权重 使用python脚本下载权重，以下仅为示例 ``` #模型下载，cache_dir替换为本地路径 from modelscope import snapshot_download model_dir = snapshot_download('qwen/Qwen2-7B-Instruct', cache_dir = '') ``` 修改权重路径中`config.json`中的`torch_dtype`为`float16` 4. 推理测试 ``` torchrun --nproc_per_node 卡数 \ --master_port 20038 \ -m examples.run_pa \ --model_path 权重路径 \ --input_text ["输入的token"] \ --is_chat_model \ --max_output_length 输出长度 ``` e.g ``` torchrun --nproc_per_node 8 \ --master_port 20038 \ -m examples.run_pa \ --model_path /home/aicc/checkpoint/ \ --input_text ["What's Deep Learning?"] \ --is_chat_model \ --max_output_length 128 ``` 5. 性能测试 配置环境及环境变量 ``` export HCCL_DETERMINISTIC=0 export LCCL_DETERMINISTIC=0 export HCCL_BUFFSIZE=120 export ATB_WORKSPACE_MEM_ALLOC_GLOBAL=1 export ASCEND_RT_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" cd /usr/local/Ascend/atb-models/tests/modeltest pip install -r requirements.txt ``` 运行测试 ``` #bash run.sh pa_fp16 performance [[input,output]] batchsize modelname model_path device_num bash run.sh pa_fp16 full_GSM8K 5 1 qwen /home/aicc/checkpoint/ 1 bash run.sh pa_fp16 performance [[1024,1024]] 1 qwen /home/aicc/checkpoint/ 1 ``` 6. 拉起mindie-service服务，需先配置`/usr/local/Ascend/mindie/latest/mindie-service/conf/config.json` ``` cd /usr/local/Ascend/mindie/latest/mindie-service/bin/ ./mindieservice_daemon ``` 7. 测试服务 另起一个容器，或宿主机 ``` apt update apt install curl curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{ "inputs": "My name is Olivier and I", "parameters": { "decoder_input_details": true, "details": true, "do_sample": true, "max_new_tokens": 20, "repetition_penalty": 1.03, "return_full_text": false, "seed": null, "temperature": 0.5, "top_k": 10, "top_p": 0.95, "truncate": null, "typical_p": 0.5, "watermark": false } }' http://127.0.0.1:1025/generate ```

在其他AICC使用本镜像时，
1） 在本地arm宿主机通过docker pull 拉取上面镜像到本地（即执行docker pull remote_image_address）
2） 用docker tag 将局点信息和组织名替换成对应版本（即执行 docker tag local_image_address remote_image_address），
3） 用docker push 将修改后的镜像名称推送到局点的swr服务中（即执行docker push remote_image_address）

若您有任何疑虑，您可以在这里反馈您的建议。