iy SrSSKrSSKJrJr SSKJrJrJr SSK r SSK J r SSK J r SSKJrJrJr SSKJr SS KJrJrJrJrJrJrJrJrJrJr SS KJ r /S Qr!S S S\"4Sjr#S S S\"4Sjr$S S S\"4Sjr%S S S\"4Sjr&"SS\\ 5r'"SS\'5r("SS\(5r)"SS\(5r*"SS\(5r+\(RY\SS\ RZ\ R\SS9r/\(RY\S S!\ R`\ RbS"S9r2\(RY\SSS\ RZS#S$9r3\)RY\S%9r4\)RY\S%9r5\4r6\5r7\(RY\S S!\ R`\ RpS"SS&9r9\(RY\\ Rt\ RZSSSS#S'9r;\(RY\\ RtS\ RxS#S(9r=\(RY\SS\ RZ\ R\SS9r>\*RY\SS\ RZS)9r?\*RY\S S!\ R`\ RbS*9r@\*RY\S S!\ R`\ RpS*9rA\*RY\S+S!\ R`\ RbS,S-9rB\*RY\S+S!\ R`\ RpS,S-9rC\+RY\\R\ R\"5S.9rF\+RY\\R\ R\"S/5S.9rI\+RY\\R\ R\"S5S.9rJS0rKS1rLS2rMS3rNS4rOg)5z6Implements modules used to perform fake quantization.N)ABCabstractmethod)AnyOptionalTuple)Node)Module)PerAxisPerGroup PerTensor)AffineQuantizedMinMaxObserver) AffineQuantizedObserverBaseFixedQParamsObserverHistogramObserver MappingTypeMovingAverageMinMaxObserver%MovingAveragePerChannelMinMaxObserverZeroPointDomain _with_args)default_fixed_qparams_range_0to1_observer,default_fixed_qparams_range_neg1to1_observer)_fake_quantize_affine)FakeQuantizeBase FakeQuantizeFixedQParamsFakeQuantizeFusedMovingAvgObsFakeQuantizeAffineFakeQuantizedisable_fake_quantdisable_observerenable_fake_quantenable_observerdefault_fake_quantdefault_weight_fake_quantdefault_dynamic_fake_quant.default_fixed_qparams_range_neg1to1_fake_quant+default_fixed_qparams_range_0to1_fake_quant*default_symmetric_fixed_qparams_fake_quant'default_affine_fixed_qparams_fake_quant%default_per_channel_weight_fake_quantdefault_embedding_fake_quant!default_embedding_fake_quant_4bitdefault_histogram_fake_quantdefault_fused_act_fake_quantdefault_fused_wt_fake_quant'default_fused_per_channel_wt_fake_quant(fused_wt_fake_quant_range_neg_127_to_1274fused_per_channel_wt_fake_quant_range_neg_127_to_127default_affine_fake_quantdefault_groupwise_fake_quant"default_affine_per_axis_fake_quantqschemez torch.qschemereturncfU[R[R[R4;$N)torchper_channel_symmetricper_channel_affine per_channel_affine_float_qparamsr5s f/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/torchao/quantization/pt2e/fake_quantize.py_is_per_channelr?Gs/  ##    .. cHU[R[R4;$r8)r9per_tensor_symmetricper_tensor_affiner=s r>_is_per_tensorrDOs u1153J3JK KKr@cHU[R[R4;$r8)r9rBr:r=s r>_is_symmetric_quantrFSs u1153N3NO OOr@c*U[R4;$r8)r9r<r=s r>_is_float_qparamsrHWs  .. r@c^\rSrSr%Sr\R \S'\R \S'SU4Sjjr\ S5r \ S 5r \RRSS \SS4S jj5r\RRS 5r\RRSS \SS4S jj5r\RRS5r\S5rSrU=r$)r]aBase fake quantize module. Base fake quantize module Any fake quantize implementation should derive from this class. Concrete fake quantize module should follow the same API. In forward, they will update the statistics of the observed Tensor and fake quantize the input. They should also provide a `calculate_qparams` function that computes the quantization parameters given the collected statistics. fake_quant_enabledobserver_enabledr6Nc>[TU]5 URS[R"S/[R S95 URS[R"S/[R S95 g)z,Set fake_quant_enabled and observer_enabled.rKdtyperLN)super__init__register_bufferr9tensoruint8)self __class__s r>rRFakeQuantizeBase.__init__msT  15<<5;;3WX /qc1UVr@cgr8)rVxs r>forwardFakeQuantizeBase.forwardv r@c gr8rZ)rVkwargss r>calculate_qparams"FakeQuantizeBase.calculate_qparamszr^r@enabledc4U(aSOSURS'gNrNr)rKrVrcs r>r "FakeQuantizeBase.enable_fake_quant~s*1Qq"r@c&URS5 gNF)r rVs r>r#FakeQuantizeBase.disable_fake_quants u%r@c4U(aSOSURS'gre)rLrfs r>r! FakeQuantizeBase.enable_observers(/1Qa r@c&URS5 gri)r!rjs r>r!FakeQuantizeBase.disable_observers U#r@c ,[U40UD6nSUlU$)Nz'torchao.quantization.pt2e.fake_quantize)r __module__)clsr`fake_quant_constructors r> with_argsFakeQuantizeBase.with_argss$!+C!:6!:-V)%%r@rZ)r6N)T)__name__rq __qualname____firstlineno____doc__r9Tensor__annotations__rRrr\rajitexportboolr rr!r classmethodrt__static_attributes__ __classcell__rWs@r>rr]s  $ll"W     YY9999 YY&& YY7t7t77 YY$$&&r@rc^\rSrSr%Sr\R \S'\R \S'\SSS4U4Sjjr \RRS5r S r \RRS 5rU4S jrU4S jrS rU=r$)raSimulate the quantize and dequantize operations in training time. The output of this module is given by:: x_out = ( clamp(round(x/scale + zero_point), quant_min, quant_max) - zero_point ) * scale * :attr:`is_dynamic` indicates whether the fake quantie is a placeholder for dynamic quantization operators (choose_qparams -> q -> dq) or static quantization operators (q -> dq) * :attr:`scale` defines the scale factor used for quantization. * :attr:`zero_point` specifies the quantized value to which 0 in floating point maps to * :attr:`fake_quant_enabled` controls the application of fake quantization on tensors, note that statistics can still be updated. * :attr:`observer_enabled` controls statistics collection on tensors * :attr:`dtype` specifies the quantized dtype that is being emulated with fake-quantization, allowable values are torch.qint8 and torch.quint8. Args: observer (module): Module for observing statistics on input tensors and calculating scale and zero-point. observer_kwargs (optional): Arguments for the observer module Attributes: activation_post_process (Module): User provided module that collects statistics on the input tensor and provides a method to calculate scale and zero-point. scale zero_pointNFc ~>[TU]5 UbUbX#::dS5eURS[R5n[ US5(a([ [ US05S05RSU5n[R"U5RU::dS5eU[R"U5R::dS5eURX#S.5 XES'U"S0UD6Ul URRUl URRUl URRUl[URR 5(a[R"nO[R$nUR'S [R("S /[R"S 95 UR'S [R("S /US 95 URR*UlURR Ul[ URS5(aURR,OSUl[/UR 5(d7[1UR 5(dS[3UR 5-5e[/UR 5Ulg)Nz1quant_min must be less than or equal to quant_maxrPpkeywordszquant_min out of boundzquant_max out of bound) quant_min quant_max is_dynamicr?rOrrch_axiszYOnly per channel and per tensor quantization are supported in fake quantize got qscheme: rZ)rQrRgetr9quint8hasattrgetattriinfominmaxupdateactivation_post_processrrrrHr5floatintrSrTrPrr?rDstris_per_channel) rVobserverrrrobserver_kwargsrPzero_point_dtyperWs r>rRFakeQuantize.__init__s@   Y%:) C )$''>Ex%% #r :JKOOU;;u%))Y6 P8P P6 E 2 6 66 P8P P6  " "#S T(2 %'/'B/'B$55??55??66AA T99AA B B${{ $yy  WellC5 &LM \5<<CS+TU1177 33;; t33Y??  ( ( 0 0 t||,,t||0L0L  $,,  L .dll;r@c6URR5$r8rrarjs r>raFakeQuantize.calculate_qparams++==??r@c$URSS:XGaURUR55 UR5up#UR UR R 5UR URR 5p2UR RUR:waJUR RUR5 URRUR5 UR RU5 URRU5 URSS:XaUR(ac[R"UUR URURURR URR"5nU$[R$"UUR URURR URR"5nU$)NrrN)rLrdetachratordevicershaperesize_copy_rKrr9 fake_quantize_per_channel_affinerrrfake_quantize_per_tensor_affinerVX_scale _zero_points r>r\FakeQuantize.forwards   #q (  ( ( 4"&"8"8": F $**++,t556 zz6<</ ""6<<0'' (9(9: JJ  V $ OO ! !+ .  " "1 % *""::JJOOLL00::00::  99JJOO00::00:: r@cSURSURSURRSURRSUR SUR SURSURS UR3$) Nfake_quant_enabled=, observer_enabled= , quant_min= , quant_max=, dtype= , qscheme=z , ch_axis=, scale= , zero_point=) rKrLrrrrPr5rrrrjs r> extra_reprFakeQuantize.extra_reprs"$"9"9!::MdNcNcMde55??@ TMiMiMsMsLtuZZL 4<<. 4<<.QZZL doo-> @ r@cj>[TU]XU5 URXS-'URXS-'gNrr)rQ_save_to_state_dictrr)rV destinationprefix keep_varsrWs r>r FakeQuantize._save_to_state_dict s6 #KC(, W$%-1__ \)*r@c $>SS/nUHn X)-n X;aXn U S:Xa&URRU R5 O-U S:XdeURRU R5 [R R 5(aHU S:XaURRU 5 MU S:XdeURRU 5 MMU(dMURU 5 M [T U])UUUUUUU5 gr) rrrrr9r| is_scriptingrappendrQ_load_from_state_dict) rV state_dictrlocal_metadatastrict missing_keysunexpected_keys error_msgs local_statenamekeyvalrWs r>r"FakeQuantize._load_from_state_dict's - D-C  o 7?JJ&&syy1<///OO++CII699))++w ((-#|333--c2 , ##C(- . %        r@)rrrPrrr5rr)rvrqrwrxryr9rzr{rrRr|r}rar\rrrrrrs@r>rrs!F << - 1[TU]US9 [UR5[:Xd.UR R S[R 35eXlURRUlURRUl [UR5(dS[UR5-5eg)Nrz's observer must be a zWOnly per tensor quantization is supported FixedQParamsFakeQuantize module, got qscheme:) rQrRtyperrrWrv _observer_ctrrrrDr5r)rVrrWs r>rR!FixedQParamsFakeQuantize.__init___s (+D0015II ~~&&''=>R>[>[=\ ] I&1177 66AAdll++  ?$,,  +r@c2URUR4$r8)rrrjs r>ra*FixedQParamsFakeQuantize.calculate_qparamsmszz4??**r@cSURSURSURSURSURSUR R SUR RSUR3$) z:Define a string representation of the object's attributes.rrrrrrrr) rKrLrrrPrrrr5rjs r>r#FixedQParamsFakeQuantize.extra_reprqs"$"9"9!::MdNcNcMdeZZL doo->?ZZL T-I-I-S-S,TU55??@ 4<<. Z r@)rrr) rvrqrwrxryrRr9r|r}rarrrrs@r>rrVsI   YY++ YY  r@rc :^\rSrSrSr\SS4S\S\S\S\S S 4 U4S jjjr\ RRS \ \ R\ R44S j5r\ RRS \4S j5rS\ RS \ R4SjrSrU=r$)ri|aDefine a fused module to observe the tensor. Fused module that is used to observe the input tensor (compute min/max), compute scale/zero_point and fake_quantize the tensor. This module uses calculation similar MovingAverageMinMaxObserver for the inputs, to compute the min/max values in order to compute the scale/zero_point. The qscheme input in the observer is used to differentiate between symmetric/affine quantization scheme. The output of this module is given by x_out = (clamp(round(x/scale + zero_point), quant_min, quant_max)-zero_point)*scale Similar to :class:`~torchao.quantization.pt2e.FakeQuantize`, and accepts the same attributes as the base class. rrrrrr6Nc >[TU]"XU40UD6 [UR[[ 45(dS5eUR S[R"S/[RS95 UR S[R"S/[RS95 [URR5Ul g)NzLFused observer+fake_quant module only works with MovingAverageMinMaxObserverrKrNrOrL) rQrR isinstancerrrrSr9rTlongrFr5is_symmetric_quant)rVrrrrrWs r>rR&FusedMovingAvgObsFakeQuantize.__init__s iK?K  ( ( (*O P    [   15<<5::3VW /qc1TU"5  ( ( 0 0# r@c6URR5$r8rrjs r>ra/FusedMovingAvgObsFakeQuantize.calculate_qparamsrr@c,SURSURSURSURSURSUR R SUR RSURS UR R3$) Nrrrrrrrrz, reduce_range=) rKrLrrrPrrrr5 reduce_rangerjs r>r(FusedMovingAvgObsFakeQuantize.extra_reprs"$"9"9!::MdNcNcMdeZZL doo->htzzlS55??@ TMiMiMsMsLtu||nOD4P4P4]4]3^ ` r@rc[R"UURURURR URR URURURRURRURRURURUR5 $r8)r9fused_moving_avg_obs_fake_quantrLrKrmin_valmax_valrraveraging_constantrrrrr)rVrs r>r\%FusedMovingAvgObsFakeQuantize.forwards44  ! !  # #  ( ( 0 0  ( ( 0 0 JJ OO  ( ( ; ;  ( ( 2 2  ( ( 2 2 LL     # #  r@)r)rvrqrwrxryrrrrRr9r|r}tuplerzrarrr\rrrs@r>rr|s&4           ( YY@5u||)C#D@@ YY C    %,,  r@rc\rSrSr%Sr\R \S'\R \S'\S4S\ 4Sjjr \RRS\ \R \R 44S j5rS \R S\R 4S jr\RRS\4S j5rS \R$R&S\4SjrSrg)ria2Simulate quantize and dequantize with affine quantization in training time. This fake quantize module supports all granularities including per-tensor, per-axis, per-group, per-block, per-row, and per-token quantization through the affine quantization primitives. It works with AffineQuantizedObserverBase subclasses like AffineQuantizedMinMaxObserver and AffineQuantizedMovingAverageMinMaxObserver. The output of this module is given by:: x_out = fake_quantize_affine(x, block_size, scale, zero_point, ...) Args: observer (module): Module for observing statistics on input tensors and calculating scale and zero-point. Must be a subclass of AffineQuantizedObserverBase. is_dynamic (bool): Whether this is for dynamic quantization. **observer_kwargs: Arguments passed to the observer constructor. Attributes: activation_post_process (AffineQuantizedObserverBase): The observer instance. scale (Tensor): The quantization scale. zero_point (Tensor): The quantization zero point. target_dtype (torch.dtype): The target quantized dtype. quant_min (Optional[int]): Minimum quantization value. quant_max (Optional[int]): Maximum quantization value. zero_point_domain (ZeroPointDomain): Domain of the zero point. mapping_type (MappingType): Mapping type for quantization. is_dynamic (bool): Whether this is dynamic quantization. block_size (Optional[Tuple[int, ...]]): Block size for groupwise quantization. Example:: >>> # Per-tensor fake quantize >>> fq = AffineFakeQuantize( ... observer=AffineQuantizedMinMaxObserver, ... mapping_type=MappingType.ASYMMETRIC, ... target_dtype=torch.uint8, ... granularity=PerTensor(), ... ) >>> x = torch.randn(10, 20) >>> y = fq(x) >>> # Groupwise fake quantize >>> fq = AffineFakeQuantize( ... observer=AffineQuantizedMinMaxObserver, ... mapping_type=MappingType.SYMMETRIC, ... target_dtype=torch.int8, ... granularity=PerGroup(128), ... ) >>> x = torch.randn(10, 256) >>> y = fq(x) rrFrc [RU5 X#S'U"S0UD6UlURRUlURRUlURR UlURR UlURRUlURRUlX l UR [R:Xa[RO[RnURS[R "S/[RS95 URS[R "S/US95 SUlSUlg) NrrrrOrrFrZ)rrRr target_dtyperrzero_point_domain mapping_type granularityrrFLOATr9rrrSrT block_size _initialized)rVrrrrs r>rRAffineFakeQuantize.__init__s) !!$'(2 %DLE E $!88EE55??55??!%!=!=!O!O 88EE77CC$%%)>)>> KK  WellC5 &LM \5<<CS+TU59!r@r6c6URR5$)z=Calculate the quantization parameters (scale and zero_point).rrjs r>ra$AffineFakeQuantize.calculate_qparamss++==??r@rc DUR(dAURUR55 URRUlSUlURSS:XGa:URUR55 UR 5up#UR URR5nUR URR5nURRUR:waJURRUR5 URRUR5 URRU5 URRU5 URRUlURSS:XalURcS5e[UURURURURUR UR"UR$5nU$)NTrrNzblock_size must be set before fake quantization. Ensure the model is run at least one forward pass with both observer and fake_quant disabled.)rrrrrLrarrrrrrrrKrrrrrrs r>r\AffineFakeQuantize.forward$s   ( ( 4"::EEDO $D    #q (  ( ( 4"&"8"8": FYYtzz001F%..)?)?@Kzz6<</ ""6<<0'' (9(9: JJ  V $ OO ! !+ ."::EEDO  " "1 % *??. > . & !!&& Ar@c$SURSURSURSURSURSUR SUR SURS URS URS UR3$) Nrrrrz, target_dtype=z, granularity=z, mapping_type=z, zero_point_domain=rrz , block_size=) rKrLrrrrrrrrrrjs r>rAffineFakeQuantize.extra_reprJs"$"9"9!:; $ 5 567( T^^4DE --.nT=M=M?//* , r@model observer_nodec:URRX5 g)a Converts the fake quantize node in the graph into its quantized representation. This method delegates to the underlying activation_post_process (observer)'s convert method, which handles the creation of quantize_affine/dequantize_affine operations in the graph. Args: model: graph module to convert the fake quantize node in observer_node: the fake quantize node to convert N)rconvert)rVrrs r>rAffineFakeQuantize.convertWs $$,,UBr@) rrrrrrrrrrN)rvrqrwrxryr9rzr{r r~rRr|r}rrar\rrfx GraphModulerrrrZr@r>rrs4l << / """"H YY@5u||)C#D@@$$%,,$L YY  C     CUXX11 C$ Cr@rrT)rrrrPr5riFrN)rrrrrPrr)rrrrPr5rr)rr5rPrrrr)rr5rrPr)rrrrP)rrrrPr5ig0?)rrrrPr5eps)rrrrc[U[RR5(aRURR R SS5Sn[R"SSU5nUS:H=(d US:H$g)zFReturn true if given mod is an instance of FakeQuantize script module..rNz\.___torch_mangle_\d+z4torchao.quantization.pt2e.fake_quantize.FakeQuantizezEtorchao.quantization.pt2e.fake_quantize.FusedMovingAvgObsFakeQuantizeF) rr9r|RecursiveScriptModule_cqualified_namesplitresub)modsuffixrs r>_is_fake_quant_script_moduler&sr#uyy6677&&,,S!4Q7vv.F; J J WVW r@cp[U[5(d[U5(aUR5 gg)zDisable fake quantization for the module. Disable fake quantization for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torchao.quantization.pt2e.disable_fake_quant) N)rrrrrs r>rr4s/#'((,H,M,M  -Nr@cp[U[5(d[U5(aUR5 gg)zEnable fake quantization for the module. Enable fake quantization for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torchao.quantization.pt2e.enable_fake_quant) N)rrrr rs r>r r As/#'((,H,M,M -Nr@cp[U[5(d[U5(aUR5 gg)zDisable observation for this module. Disable observation for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torchao.quantization.pt2e.disable_observer) N)rrrrrs r>rrNs/#'((,H,M,M -Nr@cp[U[5(d[U5(aUR5 gg)zEnable observation for this module. Enable observation for this module, if applicable. Example usage:: # model is any PyTorch model model.apply(torchao.quantization.pt2e.enable_observer) N)rrrr!rs r>r!r![s/#'((,H,M,M -Nr@)Pryrabcrrtypingrrrr9torch.fxrtorch.nnr torchao.quantizationr r r .torchao.quantization.pt2e._affine_quantizationr "torchao.quantization.pt2e.observerrrrrrrrrrr%torchao.quantization.quant_primitivesr__all__r~r?rDrFrHrrrrrrtrrCr"qint8rBr#r$r%r&r'r(r:r)r<r*quint4x2r+r,r-r.r/r0r1 ASYMMETRICrUr2 SYMMETRICint8r3r4rrr rr!rZr@r>r*s= #'' ==    @_LOLLPPTP4 7&sF7&t| #| ~# |# LB LB JbCbCJ"++ ( ,,  # # ,)22 ( ++  & & 3 *33 ( ,, 42J1S1S 92T2./G.P.P 6/Q/+ 3++V'(4(>(> 2 ++  ' ' )?)% ,55 2  2 2 ,,  6  %1$:$: 2  2 2 .. %;%! ,55  ,,  # # 6  =FF ( ,, G  <EE ( ++  & & F+H*Q*Q 2 ++  ' ' +R+',I+R+R ( ++  & & ,S,( "++6kk++  ,5 /88 *'' 9  2;; *&& <   &8%A%A *&& &B&"   !    r@