Sample Usage Guide【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/geFeature DescriptionThis sample demonstratesPatternMatcherConfigfunctionality using MatMulAdd fusion to GEMM pass, providing two verification methods: online inference and ATC offline model compilation. The pass uses eager style API and fusion interface.Directory Structure├── src │ ├──fuse_matmul_add_pass.cpp // pass implementation file ├── CMakeLists.txt // build script ├── data | ├──es_gen_air.py // export air | ├──es_forward_1.py // build graph using eager style api, call ge api for online inference, pass executed successfully | ├──es_forward_2.py // online inference, EnableConstValueMatch blocking takes effect, pass blocked | ├──es_forward_3.py // online inference, EnableIrAttrMatch blocking takes effect, pass blocked |—— gen_es_api | |——CMakeLists.txt // build script for generating eager style apiEnvironment RequirementsCompiler: GCC 7.3.xPython and dependencies: python3.9, pytorch2.1Environment preparation completed.Implementation StepsDefine classFuseMatMulAndAddPassinheriting fromPatternFusionPass. Override constructor:explicit MatmulAddFusionPass() : PatternFusionPass(PatternMatcherConfigBuilder() .EnableConstValueMatch() .EnableIrAttrMatch().Build()){}Override two functions from base classPatternFusionPass:Patternsdefines matching templates for identifying topologies matching the template in the graph.Replacementdefines replacement part.RegisterFuseMatMulAndAddPassas custom fusion pass with execution phase BeforeInferShape.Program CompilationAssume CANN software package installation directory is INSTALL_PATH, e.g.,/home/HwHiAiUser/Ascend/.Configure environment variables.Run environment setup script from software package:source ${ASCEND_PATH}/set_env.sh${ASCEND_PATH}is cann path under CANN software package installation directory. Replace with actual installation path, e.g.,${INSTALL_PATH}/cann.ModifyCMakeLists.txtas needed.ASCEND_PATH: Default software package path. If$ASCEND_HOME_PATHset via set_env.sh, no modification needed.PASS_SO_DIR: Custom fusion pass dynamic library installation directory name, defaultpass_so_dir.target_include_directories: Required header files. For this sample, no modification needed. For custom development, add header files below the example without deleting existing items. If network has custom operators, add custom operator prototype definition headers.target_link_libraries: Required libraries. For this sample, no modification needed. For custom development, add libraries below the example without deleting existing items.Do not link other SOs from software package to avoid compatibility issues during future upgrades.Execute sequentially:mkdir build cd build cmake ..Run following commands to compile custom pass so and copy compiled dynamic library libfuse_matmul_add_for_matcher_config_sample_pass.so to custom fusion pass directory, where xxx is user-defined directory. Optional parameter-j$(nproc)can be added after make for parallel build tasks,$(nproc)dynamically gets CPU core count.make -j$(nproc) fuse_matmul_add_for_matcher_config_sample_pass make installAfter sample verification, run the following command to clean custom pass so installed under CANN package to avoid affecting subsequent UT/ST:make clean_custom_passProgram ExecutionConfigure environment variables (if already done, skip).Run environment setup script:source ${ASCEND_PATH}/set_env.shReplace${ASCEND_PATH}with actual software package installation path.Set environment variable to dump model graph during compilation:export DUMP_GE_GRAPH1Install es_all.whlpip install --force-reinstall --upgrade --target ${ASCEND_PATH}/python/site-packages/ ${BUILD_PATH}/es_output/whl/es_all-*****.whlReplace${BUILD_PATH}with actual build directory path.Set environment variable to add es_all.so pathexport LD_LIBRARY_PATH${BUILD_PATH}/es_output/lib64:${LD_LIBRARY_PATH}Use ATC offline inference.Enter data directory and execute .py file to export air (uses es python interface for graph construction):python es_gen_air.pyAfter execution, .air format model file named graph.air generated in data directory.Execute ATC tool command (for detailed ATC tool instructions, visit Ascend Documentation and search for ATC Offline Model Compilation Tool), modifysoc_versionbased on actual environment:atc --framework1 --model./graph.air --soc_versionxxx --output./model --input_shapeinput_0:2,3;input_1:3,2After successful run, log shows:Define pattern for MatMulAddFusionPass in matcher config sample Define replacement for MatMulAddFusionPass in matcher config sampleOnline inferenceEnter data directory and execute .py file for online inference (ensure torch_npu plugin installed for online inference), executees_forward_1.py:python es_forward_1.pyFor es_forward_1.py, log shows:Define pattern for MatMulAddFusionPass in matcher config sample Define replacement for MatMulAddFusionPass in matcher config sampleExecutees_forward_2.pyandes_forward_3.pypython es_forward_2.py python es_forward_3.pyFor es_forward_2.py and es_forward_3.py, log shows:Define pattern for MatMulAddFusionPass in matcher config sampleView resultsAfter execution, series of .pbtxt files generated in directory. Compare the following dump graphs:ge_onnx_xxxxx_PreRunBegin.pbtxtdump graph before executionge_onnx_xxxxx_RunCustomPassBeforeInferShape.pbtxtcustom pass dump graph before InferShape executionModel optimized as expected, i.e., MatMul and Add replaced by GEMM.If results not as expected, set following environment variables (for atc command, also add parameter--logdebug) to print logs to screen for troubleshooting.export ASCEND_SLOG_PRINT_TO_STDOUT1 #print logs to screen export ASCEND_GLOBAL_LOG_LEVEL0 #log level debug【免费下载链接】geGEGraph Engine是面向昇腾的图编译器和执行器提供了计算图优化、多流并行、内存复用和模型下沉等技术手段加速模型执行效率减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考