`i9 VSSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJ r SSK J r J r Jr SSKJrJrJrJr SSKJrJrJrJrJr SSKJrJr SS KJrJr SS KJr SS K J!r! SS K"J#r# SS K$J%r%J&r&J'r' S\4Sjr(SS\4Sjjr)SS\RTSSSSSSS/4 Sjr+\RX/SS\RZ\RZ//SS\'R\4 S\%4Sjjr//SS\RZ\RZ//SS4 S\S\4Sjjr0/SS\RZ\RZ//S4S\S\4Sjjr1g)N)Path)onnx_pb)SessionOptionsInferenceSessionGraphOptimizationLevel)QuantizationModeQuantizedValueTypeQuantizedInitializerQuantizedValue) find_by_nameget_elem_index get_mul_nodegenerate_identified_filenameattribute_to_kwarg) QuantType QuantFormat)QLinearOpsRegistryIntegerOpsRegistry) ONNXModel) ONNXQuantizer) QDQQuantizer)CalibrationDataReadercreate_calibratorCalibrationMethod model_pathc[US5n[5nUR5Ul[R Ul[UR5US/S9n[R"UR55nU$)z Generate model that applies graph optimization (constant folding,etc.) parameter model_path: path to the original onnx model return: optimized onnx model z-optCPUExecutionProvider) providers) rras_posixoptimized_model_filepathrORT_ENABLE_BASICgraph_optimization_levelronnxload)ropt_model_path sess_option_optimized_models f/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/onnxruntime_tools/quantization/quantize.pyoptimize_modelr+so 2*fEN "K+9+B+B+DK(+A+R+RK(,,. H^G_`Aii 7 7 9:O TcU(a9[[[U555nUR5 UR$[ R "[U55$)N)rr+rreplace_gemm_with_matmulmodelr$r%)roptimize onnx_models r* load_modelr2-sE~d:.>?@ ++- 99T*% &&r,Fc [R"S5 US:XdUS:XaUn [R"5n U R U5 U (a[ U 5S:XaCU(a[ [R"55O[ [R"55n [XUS:HXUXhXU 5 nUR5 URR$[S5e)a Given an onnx model, create a quantized onnx model and save it into a file :param model: ModelProto to quantize :param per_channel: quantize weights per channel :param nbits: number of bits to represent quantized data. Currently only supporting 8-bit types :param quantization_mode: Can be one of the QuantizationMode types. IntegerOps: the function will use integer ops. Only ConvInteger and MatMulInteger ops are supported now. QLinearOps: the function will use QLinear ops. Only QLinearConv and QLinearMatMul ops are supported now. :param static: True: The inputs/activations are quantized using static scale and zero point values specified through quantization_params. False: The inputs/activations are quantized using dynamic scale and zero point values computed while running the model. :param symmetric_activation: True: activations are quantized into signed integers. False: activations are quantized into unsigned integers. :param symmetric_weight: True: weights are quantized into signed integers. False: weights are quantized into unsigned integers. :param quantization_params: Dictionary to specify the zero point and scale values for inputs to conv and matmul nodes. Should be specified when static is set to True. The quantization_params should be specified in the following format: { "input_name": [zero_point, scale] }. zero_point should be of type np.uint8 and scale should be of type np.float32. example: { 'resnet_model/Relu_1:0': [np.uint8(0), np.float32(0.019539741799235344)], 'resnet_model/Relu_2:0': [np.uint8(0), np.float32(0.011359662748873234)] } :param nodes_to_quantize: List of nodes names to quantize. When this list is not None only the nodes in this list are quantized. example: [ 'Conv__224', 'Conv__252' ] :param nodes_to_exclude: List of nodes names to exclude. The nodes in this list will be excluded from quantization when it is not None. :param op_types_to_quantize: specify the types of operators to quantize, like ['Conv'] to quantize Conv only. It quantizes all supported operators by default. :return: ModelProto with quantization zonnxruntime.quantization.quantize is deprecated. Please use quantize_static for static quantization, quantize_dynamic for dynamic quantization.r3rz4Only 8 and 7 bit quantization is currently supported)loggingwarning onnx_proto ModelProtoCopyFromlenlistrkeysrrquantize_modelr/ ValueError)r/ per_channelnbitsquantization_modestatic force_fusionssymmetric_activationsymmetric_weightquantization_paramsnodes_to_quantizenodes_to_excludeop_types_to_quantizemode copy_model quantizers r*quantizerN8sx OOij zUaZ **, E"#s+?'@A'EFL4(:(?(?(A#BRVWiWnWnWpRq !*5A:tUe"6M^"68    "$$$OPPr,calibration_data_readerc U[R:wa [S5e[RnU(a[ U5S:Xa[ [R"55n[[U5U 5n[XU S9nURU5 UR5nU[RLa[!UUUUSUUUU U U5 nO[#UUUUSUUUU U U5 nUR%5 UR&R)X5 g)a Given an onnx model and calibration data reader, create a quantized onnx model and save it into a file :param model_input: file path of model to quantize :param model_output: file path of quantized model :param calibration_data_reader: a calibration data reader. It enumerates calibration data and generates inputs for the original model. :param quant_format: QuantFormat{QOperator, QDQ}. QOperator format quantizes the model with quantized operators directly. QDQ format quantize the model by inserting QuantizeLinear/DeQuantizeLinear on the tensor. :param op_types_to_quantize: specify the types of operators to quantize, like ['Conv'] to quantize Conv only. It quantizes all supported operators by default. :param op_types: operators to quantize :param per_channel: quantize weights per channel :param reduce_range: quantize weights with 7-bits. It may improve the accuracy for some models running on non-VNNI machine, especially for per-channel mode :param activation_type: quantization data type of activation :param weight_type: quantization data type of weight :param nodes_to_quantize: List of nodes names to quantize. When this list is not None only the nodes in this list are quantized. example: [ 'Conv__224', 'Conv__252' ] :param nodes_to_exclude: List of nodes names to exclude. The nodes in this list will be excluded from quantization when it is not None. :param optimize_model: optimize model before quantization. :parma use_external_data_format: option used for large size (>2GB) model. Set to False by default. :param calibrate_method: Current calibration methods supported are MinMax and Entropy. Please use CalibrationMethod.MinMax or CalibrationMethod.Entropy as options. z:Static quantization only support uint8 for activation now.r)calibrate_methodTN)rQUInt8r?r QLinearOpsr;r<rr=r2rr collect_data compute_ranger QOperatorrrr>r/save_model_to_file) model_input model_outputrO quant_formatrJr@ reduce_rangeactivation_type weight_typerHrIr+use_external_data_formatrQrKr/ calibrator tensors_rangerMs r*quantize_staticras^)***UVV  & &D 3';#<#A#$6$;$;$=> tK(. 9E"5QabJ34,,.M{,,,!            " !            "  OO&&|Nr,rXrYc 6[Rn U(a[U5S:Xa[[R "55n[ [U5U 5n [U UUU SUUSUUU5 n U R5 U RRX5 g)a Given an onnx model, create a quantized onnx model and save it into a file :param model_input: file path of model to quantize :param model_output: file path of quantized model :param op_types_to_quantize: specify the types of operators to quantize, like ['Conv'] to quantize Conv only. It quantizes all supported operators by default :param per_channel: quantize weights per channel :param reduce_range: quantize weights with 7-bits. It may improve the accuracy for some models running on non-VNNI machine, especially for per-channel mode :param nbits: number of bits to represent quantized data. Currently only supporting 8-bit types :param activation_type: quantization data type of activation :param weight_type: quantization data type of weight :param nodes_to_quantize: List of nodes names to quantize. When this list is not None only the nodes in this list are quantized. example: [ 'Conv__224', 'Conv__252' ] :param nodes_to_exclude: List of nodes names to exclude. The nodes in this list will be excluded from quantization when it is not None. :parma use_external_data_format: option used for large size (>2GB) model. Set to False by default. rFN) r IntegerOpsr;r<rr=r2rrr>r/rW)rXrYrJr@r[r\r]rHrIr+r^rKr/rMs r*quantize_dynamicrdsF  & &D 3';#<#A#$6$;$;$=> tK(. 9E     I OO&&|Nr,c 4[Rn [[U55n U(a[ U5S:Xa[ [ R"55n[U UUU SUUSUUU5 n U R5 U RRX5 g)a2 Given a quantize-aware traning onnx model, create a quantized onnx model and save it into a file :param model_input: file path of model to quantize :param model_output: file path of quantized model :param op_types_to_quantize: specify the types of operators to quantize, like ['Conv'] to quantize Conv only. It quantizes all supported operators by default :param per_channel: quantize weights per channel :param reduce_range: quantize weights with 7-bits. It may improve the accuracy for some models running on non-VNNI machine, especially for per-channel mode :param activation_type: quantization data type of activation :param nodes_to_quantize: List of nodes names to quantize. When this list is not None only the nodes in this list are quantized. example: [ 'Conv__224', 'Conv__252' ] :param nodes_to_exclude: List of nodes names to exclude. The nodes in this list will be excluded from quantization when it is not None. :parma use_external_data_format: option used for large size (>2GB) model. Set to False by default. rFN) r rcr+rr;r<rr=rr>r/rW) rXrYrJr@r[r\r]rHrIr^rKr)rMs r* quantize_qatrfs@  & &D%T+%67O 3';#<#A#$6$;$;$=>    I OO&&|Nr,)T)2osr$onnx.numpy_helperstructr6numpynppathlibrrr8 onnxruntimerrr quant_utilsr r r r r rrrrrrregistryrrr1ronnx_quantizerr qdq_quantizerr calibraterrrr+r2rcrNrVrRMinMaxrardrfr,r*rusZ   &PPccuu/<!)'RR t '4'/:: "'#!%#""$MQf"-!6!6)+ %!&$-$4$4 ) 0 0&(%'#'-2%6%=%=YO-BYO|+-!&"'%.%5%5!*!1!1')&($(.37O$7O#'7Ox')"#!*!1!1&--#%"$*/6Od6O#6Or,