`iSSKrSSKrSSKJr SSKrSSKrSSKrSSKrSSKJ r SSK J r J r Jr SSKJrJrJrJr SSKJrJrJrJrJrJr SSKJrJrJrJr SS KJrJrJ r J!r! SS K"J#r#J$r$ SS K%J&r& "S S 5r'g)N)Path)onnx_pb)SessionOptionsInferenceSessionGraphOptimizationLevel)QuantizationModeQuantizedValueTypeQuantizedInitializerQuantizedValue) find_by_nameget_elem_index get_mul_nodegenerate_identified_filenameattribute_to_kwarg type_to_name)quantize_nparray quantize_datacompute_scale_zpget_qrange_for_qType) QuantType onnx_domain __producer__ __version__)CreateOpQuantizerCreateDefaultOpQuantizer) ONNXModelc\rSrSrSrSrSrSrSr\ S5r Sr S r S r S rS rS rSrSSjrSrSrSrSSjrSrSrSrSrSrSrg) ONNXQuantizerc [RRU5nURRV s0sHoR U _M sn UlUR RURRV s0sHoR U _M sn 5 UR RURRVs0sHoR U_M sn5 [U5Ul X l X0l X@lXPlSUlU["R$:Xa[&R(R*O[&R(R,UlU["R$:Xa[&R(R*O[&R(R,UlXlXlXlXl/UlUR=5UlUR[@;a$[CSREUR55eURG5Ul$SUl%SUl&SUl'SUl(0Ul)gs sn fs sn fs snf)NFz unsupported quantization mode {}fixed_quantization_range_uint8fixed_quantization_range_int8 fixed_zero fixed_zero_zp)*onnxshape_inference infer_shapesgraph value_infoname value_infosupdateoutputinputrmodel per_channel reduce_rangemodestaticfuse_dynamic_quantrQInt8 onnx_proto TensorProtoINT8UINT8 input_qType weight_qType tensors_rangenodes_to_quantizenodes_to_excludeop_types_to_quantize new_nodescheck_opset_version opset_versionr ValueErrorformatcalculate_quantization_paramsquantization_paramsfixed_qrange_uint8_namefixed_qrange_int8_namefixed_zero_namefixed_zero_zp_namequantized_value_map)selfr0r1r2r3r4r<r;r=r>r?r@viotits l/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/onnxruntime_tools/quantization/onnx_quantizer.py__init__ONNXQuantizer.__init__s$$11%827++2H2HI2HBGGRK2HI u{{7I7I J7I"7I JK u{{7H7H I7H"7H IJu% &(  "':E:X:1166^h^t^t^z^z;G9??;ZJ2277`j`v`v`|`| +!2 0$8!!557yy,,?FFtyyQR R#'#E#E#G (H$&E#+"1$& _J J IsII I#cURRRVs/sH'oR(aURS:XdM%UPM) nnS[U5:wa [ S5eUSR nUS:Xa&[ R"SRU55 gUS:a[ R"SRU55 URRRRUS5 URRRR[RRSS 5/5 S nS UlU$s snf) Nzai.onnxrz$Failed to find proper ai.onnx domainr zThe original model opset version is {}, which does not support node fusions. Please update the model to opset >= 11 for better performance.zThe original model opset version is {}, which does not support quantization. Please update the model to opset >= 11. Updating the model automatically to opset 11. Please verify the quantized model. T)r0 opset_importdomainlenrDversionloggingwarningrEremoveextendr&helper make_opsetidr5)rMopsetai_onnx_domainrCs rQrB!ONNXQuantizer.check_opset_versionRs)#zz//<< T>T>V!W%%..~>%%../@A  ",I-i.>.>q.A9??SH2~4=OOA4F 1(G#VINN355 ",'__Q/ ++3/1D,7C((4 && {3&& {3'--/@.AB&--~.>?i,l 0 &&'=>zzc URb4[UR5S:waURUR;agURUR;agUR bURUR ;agg)NrFT)r>rZr+rkr@r?)rMnodes rQshould_quantizeONNXQuantizer.should_quantizess  ! ! -#&&3(+,3-15$BXBX1X LL 9 9 9  ,d>S>S1SrcUR5 URR5H@nURU5(a [ X5nO [ X5nUR 5 MB UR5 URR5RS5 URR5RRUR5 URR5 [URRl[ URRlURR$)Nr)rr0rjrrrquantize_dequantize_outputsr) ClearFieldrr_rAremove_unused_constantr producer_namerproducer_version)rMr op_quantizers rQquantize_modelONNXQuantizer.quantize_models ((*JJ$$&D##D))0< 7C  ! ! # '   " %%f- &&t~~6 ))+)5 &,7 )zzrcUR[RR:Xa![R R U5nU$[SRUR[UR55e)Nz=Only float type quantization is supported. Weights {} is {}. ) data_typer7r8FLOATr&rorprDrEr+r)rnweightss rQtensor_proto_to_array#ONNXQuantizer.tensor_proto_to_arrayso  J$:$:$@$@ @''00=G\cc  ,{/D/D"EGH HrcP[XRR55nUSL$N)r r0rn)rM input_namerns rQis_input_a_weightONNXQuantizer.is_input_a_weights$":zz/E/E/GH $&&rcUR$r)r1)rMs rQis_per_channelONNXQuantizer.is_per_channelsrc[XRR55nUSL=(a' UR[R R :H$r)r r0rnrr7r8r)rM weight_nameweights rQis_valid_quantize_weight&ONNXQuantizer.is_valid_quantize_weights@k::+A+A+CDT!Vf&6&6*:P:P:V:V&VVrcU[RR:XaURX5$UR X5$)a* Create nodes for dynamic quantization of input and add them to nodes_list. parameter input_name: Name of the input. parameter nodes_list: new nodes are appended to this list. parameter qType: type to quantize to. return: scale_name, zero_point_name, scale_shape, zero_point_shape. )r7r8r9+_get_dynamic_input_quantization_params_int8,_get_dynamic_input_quantization_params_uint8)rMr nodes_listqTypes rQ&_get_dynamic_input_quantization_params4ONNXQuantizer._get_dynamic_input_quantization_paramss: J**// /CCJ[ [@@XXrc2[RRnUS-nUS-n[RR SU/US-/USS9nUR U5 US-n[RR SU/US-/USS9nUR U5 US -n [RR S URS/U S-/U 5n UR U 5 US -n [RR S URS/U S-/U 5n UR U 5 US -n [RR S U RSU RS/U S-/U 5nUR U5 [RRUR[RR/[U5S - /5nURRU5 US-n[RR SURSUR/U/U5nUR U5 [RRURU/S/5nURRU5 X@R//4$)a Create nodes for dynamic quantization of input to int8 and add them to nodes_list parameter input_name: Name of the input. parameter nodes_list: new nodes are appended to this list. return: scale_name, zero_point_name, scale_shape, zero_point_shape. _scale _ReduceMin ReduceMin:0rkeepdims _ReduceMax ReduceMax_AbsAbs_Abs_MaxMaxg@ scale_DivDiv)r7r8r9r&r` make_nodeappendr. make_tensorrIrrr0add_initializerrK)rMrrrinput_scale_namereduce_min_namereduce_min_nodereduce_max_namereduce_max_nodereduce_min_abs_namereduce_min_abs_nodereduce_max_abs_namereduce_max_abs_node abs_max_name abs_max_nodeinitializer_divscale_div_namescale_div_noder~s rQr9ONNXQuantizer._get_dynamic_input_quantization_params_int8s&&++&0$|3++// j\O^bLbKc0?9:0< /*$|3++// j\O^bLbKc0?9:0< /*.6"kk33EOS.)001H1H%QSVWUXY "">2!8!8"b@@rcz[RRnUS-nUS-nUS-n[RR SU/US-/USS9nUR U5 US-n[RR S U/US-/USS9n UR U 5 [RRUR[RR/[U5/5n URRU 5 [RRUR[RR/S /5n URRU 5 US -n [RR S U RSURS/U S-/U 5n UR U 5 US -n[RR SU RSUR/U/U5nUR U5 US-n[RR S URURS/US-/U5nUR U5 US-n[RR SURSU/US-/U5nUR U5 US-n[RR SURUS-/U5nUR U5 US-n[RR SURU/UUS9nUR U5 XE//4$)a Create nodes for dynamic quantization of input to uint8 and add them to nodes_list parameter input_name: Name of the input. parameter nodes_list: new nodes are appended to this list. return: scale_name, zero_point_name, scale_shape, zero_point_shape. r _zero_pointrrrrrrrg _scale_SubSub _scale_Divr_zero_point_Sub_zero_point_Div_zero_point_FloorFloor_zero_point_CastCastto)r7r8r:r&r`rrrrHrrr0rrJr.)rMrrrr input_zp_namerrrrinitializer_qrangeinitializer_qvaluescale_sub_namescale_sub_noderr zp_sub_name zp_sub_node zp_div_name zp_div_node zp_floor_name zp_floor_node zp_cast_name zp_cast_nodes rQr:ONNXQuantizer._get_dynamic_input_quantization_params_uint8-s;&&,,%0"]2 $|3++// j\O^bLbKc0?9:0< /*$|3++// j\O^bLbKc0?9:0< /*"[[44T5Q5QS]SiSiSoSoqs6J56Q5RT ""#56![[44T5I5I:KaKaKgKgiknqmrs ""#56$l2..u7M7Ma7PRaRhRhijRk6l0>0E/FX.)#l2..u~7L7LQ7OQUQmQm6n0@/A>S.)!#44 kk++ED4H4H/J`J`abJc3d-84-?,@+O +& #44 kk++EK4F4Fq4IK[3\_jmq_q^r,79 +&"%88  --g{7I7IM\`L`Kacpq -(!$66 {{,,V]5I5IM?\hmr,s ,'B66rc&URbXR;agURUnUb[U5S:wa[SRX55eUS/n/nUS-nURnUS/n/nUS-n [ R RXVXC5n URRU 5 [ R RU [RRX5n URRU 5 SXX4$) a4 Create initializers and inputs in the graph for zero point and scale of output. Zero point and scale values are obtained from self.quantization_params if specified. parameter param_name: Name of the quantization parameter. return: result, scale_name, zero_point_name, scale_shape, zero_point_shape. )FrVrVrVrVrhz_Quantization parameters should contain zero point and scale. Specified values for output {}: {}rrrrT) rGrZrDrEr;r&r`rr0rr7r8r) rMrparamszero_point_valueszero_point_shapezero_point_namezero_point_type scale_values scale_shape scale_nameinit_zp init_scales rQ_get_quantization_params&ONNXQuantizer._get_quantization_paramsns  # # +zAYAY/Y())*5 >S[A-BBH&B\^ ^$AYK$}4**q {  (* ++))/L\p ""7+[[,,Z9O9O9U9UWbq  "":.Z+OOrNcURUnUS-nUbUbSXEpnOURU5upn n UR(aKUS:Xa[SR U55e[ R RSXiU /U/US-5n U /$US:Xa+[ R RSXiU /U/US-5n U /$UR(aSU[RR:Xa5US-n US-n [ R RS U/XyU /US-5n U /$/nURXnU5upnn[ R RSXiU /U/US-5n X/-$) aj Given an input for a node (which is not a initializer), this function - add nodes to compute zero point and scale for this input if they don't exist. - add new QuantizeLinear node to quantize the input. parameter node: node being quantized in NodeProto format. parameter input_index: index of input in node.input. parameter qType: type to quantize to. parameter given_scale_name: if those inputs need to be quanitzed using this scale tensor. parameter given_zp_name: if those inputs to be quantized using this zeropoint tensor. return: List of newly created nodes in NodeProto format. _quantizedTFzQuantization parameters are not specified for param {}.In static mode quantization params for inputs and outputs of nodes to be quantized are required.rf_QuantizeLinearrrDynamicQuantizeLinear) r/rr4rDrEr&r`rr5r7r8r:r)rMr input_indexrgiven_scale_name given_zp_namer output_name data_foundrzp_name_ qlinear_nodezeropoint_namerjrzp_shapes rQ_get_quantize_input_nodes'ONNXQuantizer._get_quantize_input_nodessZZ ,  </  (}/H/35EGJG484Q4QR\4] 1JGQ ;;U" wF:&((  ;;001AJ\cCdgrfs1;>O1OQL > !T!#{{445E `gGhkvjw5?BS5S U $~%**u 8N8N8T8T/T!+h!6J%/-%?N#';;#8#89PS]R^:ESa9b9CFW9W$YL)>)ECC&u6?Jh$(;;#8#89IJdkKl:E UfHf$hL!>11rc`[URSURR55nUc'[ SR URS55eUnUS-nXg/n[ R"[UR5[ RS9n SU S'[RRU[RR [UR5/U 5n URR#U 5 UR$SS-n [RR'SX/US -5n UR)U 5 U/n U R)U 5 UR$SS -n[RR'S X/US -5nUR)U5 U$) a- Given a node, this function handles bias add by adding a "reshape" node on bias and an "add" node parameter nodes: new nodes would be appended into nodes parameter node: current node (Conv) parameter last_output: output of previous node (input to bias add) return: the name of output rz Expected {} to be an initializer_reshape_shapedtyperir_reshapeReshapereshape _bias_addAddbias_add)r r/r0rnrDrEnponesrZdimsint64r&r`rr7r8INT64rr.rr)rMrjr last_outputquantized_bias_namerreshape_input_datareshape_input_shape reshape_input reshape_shape init_shapereshape_op_output reshape_nodebias_add_inputadd_node_outputadd_nodes rQget_bias_add_nodes ONNXQuantizer.get_bias_add_nodesszdjjmTZZ-C-C-EF >?FFtzzRS}UV V114DD+A V[[!1"((C  a[[,,-@*BXBXB^B^adekepepaq`r-:<  "":. KKNZ7{{,,Y GZ-@9-LN  \"&/0++a.;6;;((@QSfisSst Xrc^URURn[XPRR 55nUR U5n[XRR 55nUR U5n US-n [ RRn US-n [RRSX/US-/US-5n URU 5 [RRSXRS/US-/US-5nURU5 [RRS URUS -/US -5nURU5 [RRS URU /U S -U S9nURU5 U $)M Quantized the bias. Zero Point == 0 and Scale == Input_Scale * Weight_Scale rrMul _scale_noderrz _tmp_quant:0 _tmp_qauntrz_quant_rounded:0_quant_roundedr_noder)rLrr r0rnrr7r8INT32r&r`rrr.)rM bias_namerr new_node_listweight_scale_nameweight_initializer weight_scalebias_initializer bias_datarrrbias_scale_nodequantize_bias_nodebias_rounded_nodebias_cast_nodes rQquantize_bias_dynamic#ONNXQuantizer.quantize_bias_dynamics !44[ALL)*;ZZ=S=S=UV112DE ( ::3I3I3KL../?@ ',6&&,,%0++//8H7\_hks_s^t09M0IK_-![[2259F\F\]^F_:`4=4N3OQZ]iQik/0 KK11';M;T;TW`cuWuVv2;>N2NP./..v/@/G/GJ]I^/BW/L27/9 ^,""rc XR;aURUR$URURn[X@RR 55nUR U5n[XRR 55nUR U5nUS-n X R;aURURn O@X R;aURU5up n O[SRU55e[XRR 55n UR U 5n X-n[R"U5U- R5R[R5n[R"U[RS9R!UR"5n[$R&R)UU 5nURR 5R+U/5 U S-n[R"U[R,S9R!S5n[$R&R)UU5nURR 5R+U/5 U S-n[R."UR0[RS9R!S5n[$R&R)UU5nURR 5R+U/5 XR;de[3XUU[4R6UR8S:aSOS 5nUURU'U $) r%rz@Expected {} to be in quantized value map for static quantizationr rrirrrN)rLq_namerr r0rnrrGrrDrErasarrayroundastypeint32r rr&ro from_arrayr_float32zerosshaper r Initializersize)rMr,rrr.r/r0r1r2rrrinputscale_initializer input_scale bias_scalequantized_data bias_np_datapacked_bias_initializerquantized_bias_scale_namebias_scale_datapacked_bias_scale_initializerquantized_bias_zp_name bias_zp_datapacked_bias_zp_initializerquantized_values rQquantize_bias_static"ONNXQuantizer.quantize_bias_statics 00 0++I6== =!44[ALL)*;ZZ=S=S=UV112DE ( ::3I3I3KL../?@ ',6 11 1#77 CNN  33 3+/+H+H+T (AAq_ffgqrs s!-.> @V@V@X!Y001GH !/ **Y/*<CCELLRXXVzz.AIIJZJ_J_` "&"3"3">">|M`"a  '')@(AB%8($B!**ZrzzBJJ2N(,(9(9(D(D_Vo(p%  '')F(GH"5}!Dxx 0 0AII"M %)%6%6%A%A,Pf%g"  '')C(DE!9!99:9(Ib)?ASA_A_.=.B.BQ.FDR/>  +""rc/n/n/n/nUGHLnURUn XR;abURU n URU R5 URU R5 URU R 5 M[ XRR55n U bgURX(a URO UR5upnURU 5 URU 5 URU5 GMURRU S-URURR55nUc2URXUR5nUR!U5 USnUR"S:XaZUR!UR$5 URURS5 URURS5 GMURUR$S5 URUR$S5 URUR$S5 GMO XeXG4$)aC Given a node, this function quantizes the inputs as follows: - If input is an initializer, quantize the initializer data, replace old initializer with new initializer - Else, add QuantizeLinear nodes to perform quantization parameter node: node being quantized in NodeProto format. parameter indices: input indices to quantize. return: (List of quantized input names, List of zero point names used for input quantization, List of scale names used for input quantization, List of new QuantizeLinear nodes created) rrirfrrhr)r/rLrrrr:r r0rnquantize_weightr<r;find_node_by_namerAr)rr_rkr.)rMrindicesinitializer_use_weight_qType scale_nameszero_point_namesquantized_input_namesrjr node_inputrQrn q_weight_namerrrquantize_input_nodess rQquantize_inputsONNXQuantizer.quantize_inputsNs%  ""KK0J555"&":"::"F""?#=#=> ''(?(?@%,,_-C-CD'z::3I3I3KLK&595I5I6R!2!2X\XhXh6j2 &,,]; ''0"":. $zz;;JIZ,>q,AB)001D1DQ1GH&&|':':1'=>$++L,?,?,BCI#L&LLrcURUR;a<URURnURURUR4$URS-nURS-nURS-nUR U5n[ UR5R5[X R5U5u pp[R"U [RRUS9R!UR"5n [R$R'X5n [R(R+U[,R.R0/U /5n [R(R+XR/U /5nUR2R55R7XU/5 [9URXFU[:R<S5nX0RUR'XEU4$)z :param weight: TensorProto initializer :param qType: type to quantize to :return: quantized weight name, zero point name, scale name rrrr N)r+rLr:rrrrflattentolistrr2rr;r&mappingTENSOR_TYPE_TO_NP_TYPEr rror?r`rr7r8rr0rnr_r r rC)rMrrrQr]rr weight_datar zero_pointscale q_weight_dataq_weight_initializerscale_initializerzero_initializers rQrUONNXQuantizer.quantize_weights ;;$22 2"66v{{CO#**O,C,C_E_E_` ` l2 ++ -[[8+ 008 1>{?R?R?T?[?[?]?STY[l[l?mot2v.1% = 8[8[\a8bckklrlwlwx #00;;MY KK33J @V@V@\@\^`chbij;;2272 |T  '')=Rb(cd)mQX);)G)GO0?  -z11rcnXR;a2URUnURURUR4$[ XR R 55nUc [SU5eURU5nURUn/n/n /n /n /n [U5Hn URX5n[UR5R5[X R 5U5unnnnnUR#U5 U R#U5 U R#U5 U R#U5 U R#U5 M [%UR5nSUU'[&R("U S5R+U5n[S[-U 55HCn [&R("X5R+U5n[&R."UU4U5nME US-nUS-nUS-n[1UUUU[2R4S5nX@RU'[&R("U[6R8R:US9R+UR<5n[6R>RAUU5nUR<U/nBREU[FRHRJUU 5n[6RBREUUUU 5nUR R 5RMUUU/5 UUU4$)Nz{} is not an initializerrrrrrr )'rLr:rrr r0rnrDrrBrangetakerrbrcrr2rlistrr;r rZ concatenater r rCr&rdrerror?r`rr7r8rr_)rMrr< channel_axisrQrnr channel_count rmin_list rmax_listzero_point_list scale_listquantized_per_channel_data_listiper_channel_datarminrmaxrgrhquantized_per_channel_data reshape_dimsquantized_weightschannel_weightsr]rrrjzero_scale_shaperkrls rQquantize_weight_per_channel)ONNXQuantizer.quantize_weight_per_channels 22 2"66{CO#**O,C,C_E_E_` `"; 0F0F0HI  7E E,,[9 l3    *,'}%A&||A< HU ((*1135I,XiXi5jI ED$ E+E   T "   T "  " ": .   e $ + 2 23M N&GMM* %& \"JJ'Fq'IJRRS_`q#=>?A jj)H)KLTTUabO "0A?/SUa b @$l2  - 8+ (mZQX);)G)GO0?  -JJ T\\%H%H%VXX_X_`k`p`pXq #00;;@;;227LJZ\kl  '')=?PRb(cdw 33rcXR;aURUnUS-nURRX0RURR 55nUcGUR UR UR/n[RRSXQ/U5nU$XRS:Xdeg)as Given a value (input/output) which is quantized, add a DequantizeLinear node to dequantize it back to float32 parameter value_name: value to dequantize parameter new_nodes_list: List of new nodes created before processing current node return: None if there is already a DequantizeLinear node that dequantizes it A DequantizeLinear node otherwise _DequantizeLinearNrgr) rLr0rVrAr)r:rrr&r`rr.)rM value_namerQ dqlinear_name dqlinear_nodedqlinear_inputsdequantize_nodes rQ_dequantize_valueONNXQuantizer._dequantize_values 11 1"66zBO&)<=rcURR5RH>nURUR5nUcM#UR R U5 M@ g)z Dequantize output if it is quantized parameter new_nodes_list: List of new nodes created before processing current node return: List of new nodes created N)r0r)r.rr+rAr)rMr.rs rQr!ONNXQuantizer._dequantize_outputssMjj&&(//F"44V[[AO*%%o60rc 6URcgURR5HnURS;aMUR U5(dM-[ URR 5URS5S:waMfURSURR5;d+URSURR5;aMURURSURURS'M 0nURR5HXnURUupE[US5n[US5n[XEUR[UR55X#'MZ U$)N)ClipRelurr)r=r0rjrkrrZinput_name_to_nodesr/keysr.minmaxrr;r)rMrrG tensor_namer|r}s rQrF+ONNXQuantizer.calculate_quantization_paramss_    % JJ$$&D||#33''--4::113DJJqMBCqHzz!}D$6$6$;$;$==QW[WiWiWnWnWpAp040B0B4;;q>0RD  tzz!} -'!--224K++K8JDtQr@rCr1rGrLr2r4r=r,r<)NN)T)__name__ __module__ __qualname____firstlineno__rRrBrrr staticmethodrrrrrrrrrr"r7rRr_rUrrrrF__static_attributes__rrQrrs4&l2A F  4' W Y5An?7BP@62p#J$#L;#z9Mv2B84t0 7#rr)(osstructpathlibrnumpyrr\r&onnx.numpy_helperrr7 onnxruntimerrr quant_utilsr r r r r rrrrrrrrrrrrrregistryrr onnx_modelrrrrrQrsU   &PPccDD``JJA!I #I #r