`ijK,SSKrSSKrSSKrSSKrSSKrSSKJr /SQrSSjrSr Sr SSjr Sr S r S rS rSS jrS rSrSr\S:XaB\"5r\"S\5 SSKJr \"\R25 \"\5r\H r\"\5 M gg)N) TensorProto)ScanLoopIfc [R"5nURSSS[SS9 URSSS[S S S 9 URS S S[SSS 9 URSS[S SS 9 URSS[S SS 9 URSS[SSS 9 URSS[ SSS 9 URSS[SSS 9 URSS[SSS 9 URSS[SS S 9 URS!S[SS"S 9 URS#SS$/S%QS&S'9 URS(S)SS*S+S,9 UR SS-9 URS.SS*S/S,9 UR SS09 URS1SS*S2S,9 UR SS39 URS4S5SS*S69 UR SS79 URU5$)8Nz-mz--modelTzonnx model path)requiredtypehelpz-bz --batch_sizeFzbatch size of input)rr defaultr z-sz--sequence_length zsequence length of inputz--past_sequence_lengthzpast sequence length for gpt2z--global_lengthz&number of global tokens for longformerz --samplesiz\number of samples to test. Set it large enough to reduce the variance of performance result.z --thresholdg{Gz?zfThreshold of run time ratio among all nodes. Nodes with larger ratio will show in top expensive nodes.z --thread_numznumber of threads to usez--input_ids_namez"input name for input IDs, for bertz--segment_ids_namez$input name for segment IDs, for bertz--input_mask_namez'input name for attention mask, for bertz--dummy_inputsr )bertgpt2 longformerr zEType of model inputs. The default will create dummy inputs with ones.)rr choicesr z-gz --use_gpu store_truezuse GPU)ractionr )use_gpuz--basic_optimizationz_Enable only basic graph optimizations. By default, all optimizations are enabled in OnnxRuntime)basic_optimizationz--kernel_time_onlyz.Only include the kernel time and no fence time)kernel_time_onlyz-vz --verbose)rr)verbose)argparseArgumentParser add_argumentstrintfloat set_defaults parse_args)argvparsers f/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/onnxruntime_tools/transformers/profiler.pyparse_argumentsr$sx  $ $ &F i$SGXY nu3PQXmn +!& "7 9 0!& !< >  )!& !E G    k m   u w S"Smn *!& $A C  ,!& $C E  +!& $F H  (!& ) Id f  kE,U^_ &  n p  51 ,!&+MO / kE,O &   T ""cSSKJn U"UUU(+USS9nUHnURSU5nM UR5n U $)Nr)create_onnxruntime_sessionT)enable_all_optimization num_threadsenable_profiling)benchmark_helperr'run end_profiling) onnx_model_pathrr thread_num all_inputsr'sessioninputs_ profile_files r# run_profiler5esS;()0EWAW5?:> @G  KKf %((*L r%c[SUS35 [US5n[R"U5nSSS5 [ W[ 5(deU$!,(df  N'=f)Nzloading profile output z ...r)printopenjsonload isinstancelist)r4 opened_file sess_times r#load_profile_jsonr@usW #L> 67 lC KIIk*  ! i & && &  ! s A A"c/n0n0n0nSnUHnUSS:XdMSU;dMSU;dMSUS;dM)USRSS 5RS S 5RS S 5n S US;a%USS S :XaSOSn X;aXU 'OXiU :XdeO U(aMUSSn U [;aMX;aXI==US- ss'XY==S- ss'OUSXI'SXY'URU 5 XxS- nM /SQn Sn UH\n XInXYnU[U5- nX- S-nX;aXiOS nU RUSSUSSU SSSU - SSUSUSU 3 5 U U- n M^ U RSUSS35 U RS5 U RS5 [ UR 5SSS9HOupX- nUU:aMXYnU[U5- nX;aXiOS nU RUSSUS-SSUSU 35 MQ U $) aParse profile data and output nodes in two sections - nodes in the original order, and top expensive nodes. Args: sess_time (List[Dict]): profile data kernel_time_only (bool, optional): Only include items for kernel time. Defaults to False. threshold (int, optional): Minimum ratio of duration among all. Defaults to 0. Returns: List[str]: lines of string for output. rcatNodedurargsop_namename _kernel_time _fence_before _fence_afterproviderCPUExecutionProviderCPUCUDAr )zResults:@----------------------------------------------------------------uTDuration(μs) Percentage Before(Exclusive) After(Inclusive) Calls Provider Node_NamegY@.1f 5.2fz5.1fz$ Top expensive nodes with threshold=z.2f:rPu&Duration(μs) Percentage Provider Namec US$Nr xs r#'parse_profile_results..sqtr%Tkeyreverse)replaceNODES_TYPE_CONTAINING_SUBGRAPHappendrsorteditems)r?r thresholdnode_name_list node_time node_freq node_providertotalitem node_namedevicerFlinesbefore_percentagedurationcallsavg_time percentagerLratios r#parse_profile_resultsrusNIIM E ;& Ud]v~)W[\bWcJcV ,,^R@HHIKMMTWUcegMh T&\)"&v,z":>T"TZ`1/5),(3v===!6l9-G88%$U 3$$)$'+E{ $'( $%%i0 %[ E5: E# '$eEl*&%/ /8/I=+r nBz$/r2CD1IETeLefjKkkmnsmttvwwAACDMCN O  Z'$ LL83qIJ LL LL<=%ioo&7^UYZ   9  $eEl*/8/I=+r ~R d';2hZr)UV [ Lr%c20n0n0n0nSnUHnUSS:XdMSU;dMSU;dMSUS;dM)U(a SUS;aM;USSn U [;aMOX;aX9==US- ss'XI==S- ss'O USX9'SXI'XxS- nSUS;=(a USSS :Hn U (dMX;aXY==US- ss'Xi==S- ss'MUSXY'SXi'M U(aS /n OS /n [UR5S S S9HupX- n XInX;aXYOSnX;aXiOSnU [U5- nU(a,U R USSU SU S-SSUSUSUSU 3 5 MdU R USSU SU S-SSUSU 3 5 M U $)zGroup results by operator name. Args: sess_time (List[Dict]): profile data kernel_time_only (bool): Only include items for kernel time. use_gpu (bool): GPU is used in profiling or not. Returns: List[str]: lines of string for output. rrBrCrDrErFrLr rMuJAverage(μs) Total(μs) Total_Percentage Calls Cpu_Duration Cpu_Calls Nameu3Average(μs) Total(μs) Total_Percentage Calls Namec US$rWrXrYs r#r['group_profile_results..s1Q4r%Tr]rRrSrQrT)rarcrdrrb)r?rrop_time op_records op_cpu_timeop_cpu_recordsrjrkrFis_cpurnrprtrqcpu_time cpu_callsrrs r#group_profile_resultsrsGJKN E ;& Ud]v~)W[\bWcJcJd6l$B6l9-G88! DK/ #q(##'; &' # %[ E4</fDL4LPf4fFv)(DK7("+q0++/;K(./N+58cdJK#GMMOQUV #+2+A;'q/6/HN+a eEl*  LLC.8*Buu}T.B"UG2hZWYZcYddfgnfop r LLHS>H:R d7K2eWTVW^V_` aW Lr%c~[URS55[:Xa[XRS55$S$)Nvalue)r WhichOneofrgetattr)dims r#get_dim_from_type_protors2489P4QUX4X73w/ 0b^bbr%cURRRVs/sHn[U5PM sn$s snfN) tensor_typeshaperr) type_protods r#get_shape_from_type_protors40:0F0F0L0L0P0P Q0P1 #A &0P QQ Qs;c80nUR5GHcn[UR5n/n[U5H-up[ U [ 5(dMUR U5 M/ [U5S:a g[U5S:aXUS'[U5S:aX&US'URRRn U [R[R[R4;deU [R:Xa[RO3U [R:Xa[R O[R"n [R$"XkS9n XUR&'GMf [)U5V s/sHoPM nn U$s sn f)zCreate dummy inputs for ONNX model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples Returns: List[Dict]: list of inputs Nrr dtype)'get_graph_inputs_excluding_initializersrr enumerater<rrblenr elem_typerFLOATINT32INT64numpyfloat32int64int32onesrGrange) onnx_model batch_sizesequence_lengthsamples dummy_inputs graph_inputr symbol_dimsirr data_typedatar3r0s r#create_dummy_inputsrsTL!IIK )+*:*:; &FA#s##""1%' { a  { a $.+a. ! { a $3+a. !$$00:: [.. 0A0A;CTCTUUUU%.+2C2C%CEMM$ (9(99EKKu{{ zz%1)-[%%&)L,).g71,J7 8s Fc JSSKJnJn U"XXV5upn U"UUUSSU U U SS9 n U $)aCreate dummy inputs for BERT model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: List[Dict]: list of inputs r)find_bert_inputsgenerate_test_data{F) test_casesseedr input_ids segment_ids input_maskrandom_mask_length)bert_test_datarr) rrrrinput_ids_namesegment_ids_nameinput_mask_namerrrrrr0s r#create_bert_inputsr=sF*D)9*Vf)x&IJ#J$3/6),,1.70;/97<>J r%cUUUX#-S.n0nUR5GH8n[UR5n[U5HAup[ U [ 5(a!XR 5;a[SU 35eXZX'MC URRRn U [R[R[R4;deU [R:Xa[RO3U [R:Xa[R O[R"n [R$"XS9n XUR&'GM; [)U5Vs/sHoPM nnU$s snf)aCreate dummy inputs for GPT-2 model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length past_sequence_length (int): past sequence length samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rseq_len past_seq_len total_seq_lensymbol is not supported: r)rrr rr<rkeys RuntimeErrorrrrrrrrrrrrrGr)rrrpast_sequence_lengthrsymbolsrrrrrrrrr3r0s r#create_gpt2_inputsras>$!",(? GL!IIK )+*:*:;&FA#s##<<>(A"%>se#DEE"< '  $$00:: [.. 0A0A;CTCTUUUU%.+2C2C%CEMM$ (9(99EKKu{{ zz%1)-[%%&L).g71,J7 8s& E5cJXS.n0nUR5GHhn[UR5n[U5HAup[ U [ 5(a!XR 5;a[SU 35eXZX'MC URRRn U [R[R[R4;deU [R:Xa[RO3U [R:Xa[R O[R"n SUR$;a [R&"XS9n SU SS2SU24'O[R("XS9n XUR$'GMk [+U5Vs/sHoPM nnU$s snf)aCreate dummy inputs for Longformer model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length global_length (int): number of global tokens samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_longformer_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rrrglobalrr N)rrr rr<rrrrrrrrrrrrrrGzerosrr)rrr global_lengthrrrrrrrrrrr3r0s r#create_longformer_inputsrs_ (LGL!IIK )+*:*:;&FA#s##<<>(A"%>se#DEE"< '  $$00:: [.. 0A0A;CTCTUUUU%.+2C2C%CEMM$ (9(99EKKu{{  {'' ';;u6D&'DN]N" #::e5D)-[%%&%L().g71,J7 8s F c URS:a URO[R"SS9nS[R;a[ U5[RS'SSKJn SSKJ n U"U"UR55nSnURS:XaM[X@RURURUR UR"UR$5nOURS:Xa7['X@RURUR(UR5nOrURS :Xa7[+X@RURUR,UR5nO+[/X@RURUR5n[1URUR2UR4URU5n[7U5n[9XpR:UR<5nUR?S 5 UR?S 5 U[AXpR:UR25- nU$) NrF)logicalOMP_NUM_THREADS)r;) OnnxModelrrrz Grouped by operator type:rP)!r/psutil cpu_countosenvironronnxr;rrmodelrrrrrrrrrrrrrr5rrr@rurrerbr) rEr)r;rrr0r4profile_recordsrns r#r,r,s%)__q%8$//f>N>NW\>]K *(+K(8 $%$4 +,JJ F"' OOTEYEY[_[g[g(,(;(;T=R=RTXThThj  f $' OOTEYEY[_[t[t(, 6  l *-j//4K_K_aeasas.2ll< )__dFZFZ\`\h\hi tzz4<<9P9PRVRaRacmnL' 5O !/3H3H$.. YE LL./ LL "?4I4I4<< XXE Lr%__main__ Arguments) setup_loggerr)Fr)NNN)rrr:rrrrrar$r5r@rurrrrrrrr,__name__ argumentsr8r+rrresultslinerXr%r#rs   "8R#j  K\>BcR$V'+(,'+ !H)X(V"J z!I +y!-""#)nG d r%