CANN/ascend-transformer-boost ReshapeAndCache C++示例

加速库ReshapeAndCacheOperation C Demo【免费下载链接】ascend-transformer-boost本项目是CANN提供的是一款高效、可靠的Transformer加速库基于华为Ascend AI处理器提供Transformer定制化场景的高性能融合算子。项目地址: https://gitcode.com/cann/ascend-transformer-boost介绍该目录下为加速库ReshapeAndCacheOperation C调用示例。使用说明首先source 对应的CANN和nnal包的安装路径source [cann安装路径]/set_env.sh 默认source /usr/local/Ascend/ascend-toolkit/set_env.shsource [nnal安装路径]/set_env.sh默认source /usr/local/Ascend/nnal/atb/set_env.sh①. 如果使用加速库源码编译source [加速库源码路径]/output/atb/set_env.sh例如 source ./ascend-transformer-boost/output/atb/set_env.sh运行demobash build.sh注意使用cxx_abi0默认时设置D_GLIBCXX_USE_CXX11_ABI为0即g -D_GLIBCXX_USE_CXX11_ABI0 -I ...使用cxx_abi1时更改D_GLIBCXX_USE_CXX11_ABI为1即g -D_GLIBCXX_USE_CXX11_ABI1 -I ...提供的build脚本仅用于编译和运行reshape_and_cache_demo.cpp如需编译其他demo需要替换“reshape_and_cache_demo”为对应的cpp文件名额外说明示例中生成的数据不代表实际场景如需数据生成参考请查看根目录下的python用例目录 tests/apitest/opstest/python/operations/reshape_and_cache/产品支持情况本op在Atlas A2/A3系列和Atlas 推理系列产品上实现有所区别场景说明提供demo分别对应不同产品的基础场景编译运行时需要对应更改build脚本Atlas A2/A3参数设置成员名称取值compressTypeCOMPRESS_TYPE_UNDEFINEDkvCacheCfgK_CACHE_V_CACHE以下demo仅支持在Atlas A2/A3系列上运行。reshape_and_cache_demo.cpptensor名字数据类型数据格式维度信息keyfloat16nd[2, 32, 128]valuefloat16nd[2, 32, 128]keyCachefloat16nd[512, 128, 32, 128]valueCachefloat16nd[512, 128, 32, 128]slotMappingint32nd[2]keyCacheOutfloat16nd[512, 128, 32, 128]valueCacheOutfloat16nd[512, 128, 32, 128]reshape_and_cache_demo_ds1.cpptensor名字数据类型数据格式维度信息keybf16nd[5, 1, 128]valuebf16nd[5, 1, 128]keyCachebf16nd[9, 128, 1, 128]valueCachebf16nd[9, 128, 1, 128]slotMappingint32nd[5]keyCacheOutbf16nd[9, 128, 1, 128]valueCacheOutbf16nd[9, 128, 1, 128]reshape_and_cache_demo_ds2.cpptensor名字数据类型数据格式维度信息keybf16nd[1024, 1, 128]valuebf16nd[1024, 1, 128]keyCachebf16nd[9, 128, 1, 128]valueCachebf16nd[9, 128, 1, 128]slotMappingint32nd[1024]keyCacheOutbf16nd[9, 128, 1, 128]valueCacheOutbf16nd[9, 128, 1, 128]reshape_and_cache_demo_ds3.cpptensor名字数据类型数据格式维度信息keybf16nd[1, 1, 128]valuebf16nd[1, 1, 128]keyCachebf16nd[9, 128, 1, 128]valueCachebf16nd[9, 128, 1, 128]slotMappingint32nd[1]keyCacheOutbf16nd[9, 128, 1, 128]valueCacheOutbf16nd[9, 128, 1, 128]Atlas推理系列产品 reshape_and_cache_inference_demo.cpp相较于A2/A3的demo本示例主要有以下修改点kvCache shape改为[block_num, head_size \times head_num / 16, block_size, 16]。kvCache数据格式改为ACL_FORMAT_FRACTAL_NZ。参数设置成员名称取值compressTypeCOMPRESS_TYPE_UNDEFINEDkvCacheCfgK_CACHE_V_CACHEtensor名字数据类型数据格式维度信息keybf16nd[3, 4, 128]valuebf16nd[3, 4, 128]keyCachebf16nd[512, 32, 128, 16]valueCachebf16nd[512, 32, 128, 16]slotMappingint32nd[3]keyCacheOutbf16nd[512, 32, 128, 16]valueCacheOutbf16nd[512, 32, 128, 16]更改编译脚本为g -D_GLIBCXX_USE_CXX11_ABI$cxx_abi -I ${ATB_HOME_PATH}/include -I ${ASCEND_HOME_PATH}/include -L ${ATB_HOME_PATH}/lib -L ${ASCEND_HOME_PATH}/lib64 reshape_and_cache_inference_demo.cpp demo_util.h -l atb -l ascendcl -o reshape_and_cache_inference_demo运行时调用./reshape_and_cache_inference_demo该demo仅支持在Atlas 推理系列产品上运行【免费下载链接】ascend-transformer-boost本项目是CANN提供的是一款高效、可靠的Transformer加速库基于华为Ascend AI处理器提供Transformer定制化场景的高性能融合算子。项目地址: https://gitcode.com/cann/ascend-transformer-boost创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考