istSSKrSSKrSSKrSSKrSSKJrJr SSKJr SSK J r J r J r J r JrJrJr SSKJs Js Js Jr SSKJr SSKrSSKJrJr SSKJr SSKJrJ r SSK!J"r" SS K#J$r$J%r%J&r& SS K'J(r(J)r)J*r*J+r+J,r,J-r-J.r.J/r/J0r0J1r1J2r2J3r3J4r4J5r5 SS K6J7r7J8r8 SS K9J:r:J;r; SS Kr>J?r?J@r@ SSKAJBrB SSKCJDrD SSKEJFrFJGrG SSKHJHrH SSKIJJrJJKrK SSKLJMrM SSKNJOrO SSKPJQrQ SSKRJSrS "SS\Q5rTSrUS\R|4SjrVS\RR4SjrYS\ \Z4Sjr[S \M\ \D-4S!jr\S"r]S\ \RR\ 44S#jr`SiS\\RR\RR\ \Z\RR4\ \Z\RR44S%\\\R\ \Z\\R44S&\\ \ \ \Z\ \ 44S'\\ \Z\ 4S(\\ S)\\ S*\\\\ \Z\44S+\\dS,\\dS-\\eS.\\eS/\\eS\F4S0jjrfSjS\\RR\RR4S%\\RS)\\ S*\S(\ S\R~4 S1jjrgS2rhS3riSkS5\RRS6\\RS74S8\ \RR/S4S9\dS:\dS;\e4 S<jjrj\*RS4S=\Z\l-S>\ZS?\*S@\ 4SAjjrm"SBSC\5rn\nR4SD\RR4SEjjrp\.RS4S4S4S$4SF\.SG\eSH\eSI\eSJ\eS\+4 SKjjrrSlSL\eSM\eSN\eSJ\eSO\eS\+4 SPjjrsSmS?\*SQ\+SR\eSS\eST\eSU\eSV\eSW\eSX\eSY\eSZ\3S[\eS\ \B4S\jjrtS]ruS^rvS_\RR4S`jrwS_\RRSa\ 4SbjrxSD\RRSc\ \Z\RR44SdjrySerzSfr{Sgr|S\}4Shjr~g)nN) defaultdict OrderedDict)Enum)AnyCallableDictListOptionalTupleUnion) AnnotateStackAnnotateUnbind)QnnPassManager)QNN_QUANT_TYPE_MAPQNN_TENSOR_TYPE_MAP)OpContextLoader)generate_qnn_executorch_optionget_skip_decomp_tableQnnPartitioner)_soc_info_tableHtpArch QcomChipsetQnnExecuTorchBackendOptionsQnnExecuTorchBackendTypeQnnExecuTorchGpuBackendOptionsQnnExecuTorchGpuPrecisionQnnExecuTorchHtpBackendOptionsQnnExecuTorchHtpPerformanceModeQnnExecuTorchHtpPrecisionQnnExecuTorchLogLevelQnnExecuTorchOpPackageOptionsQnnExecuTorchOptionsQnnExecuTorchProfileLevel)flatbuffer_to_optionoption_to_flatbuffer)QCOM_QNN_COMPILE_SPECQCOM_QUANTIZED_IO)QnnManagerContext)EdgeCompileConfigExirExportedProgramto_edge) CompileSpec)LoweredBackendModule)EdgeProgramManagerto_edge_transform_and_lower)tabulate)core_aten_decompositionsremove_decompositions)ExportedProgram)passes)OperatorSupportBase)Libraryc\rSrSrSrSS\S\S\4SjjrS\RR4S jr S\RR4S jr S \RRS \4S jr Srg)_AnnotationSkipperIa Class used to partition out unwanted graph nodes. e.g. - nodes are prevented from quantization annotation - nodes have been grouped together as a submodule Attributes ---------- fp_node_id_set : set a set contains nodes' name to be left in fp precision fp_node_op_set : set a set contains nodes' target (aten dialect) to be left in fp precision skip_annotated_submodule : bool flag to skip annotated submodule or not Methods ------- should_delegate(n: torch.fx.Node) identify the residual nodes haven't be lowered with fixed-precision should_skip(n: torch.fx.Node) identify the nodes should be kept out with fixed-precision or not is_node_supported(_, node: torch.fx.Node) overridden method for graph partitioning Nfp_node_id_setfp_node_op_setskip_annotated_submodulec(XlX lX0lgNr:r;r<)selfr:r;r<s g/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/executorch/backends/qualcomm/utils/utils.py__init___AnnotationSkipper.__init__bs -,(@%nchURS:H=(a UR[R:g$)N call_function)optargetoperatorgetitemr@rEs rAshould_delegate"_AnnotationSkipper.should_delegatels%tt&G188x7G7G+GGrDctURUR;=(d URUR;$r>)namer:rIr;rLs rA should_skip_AnnotationSkipper.should_skipos+vv,,,OD# UHnTRU5v M g7fr>)rM.0userr@s rA 7_AnnotationSkipper.is_node_supported..usM*$4//55*!) placeholderoutputc3F># UHnTRU5v M g7fr>)rQrXs rAr[r\sJztD,,T22zr]z[QNN Quantizer Annotation]: z | SkippedFT) r<rHallusersrManyrQprintrP)r@_rSs` rAis_node_supported$_AnnotationSkipper.is_node_supportedrs  ( (ww*$M$**MMM''- - 44  &GGz)KJtzzJJ    0 :F GrDr?)NNF)__name__ __module__ __qualname____firstlineno____doc__setboolrBtorchfxNoderMrQrf__static_attributes__rDrAr8r8Is~4#"). AAA#' AHHPUXX]]P4rDr8c*[R"SS9$)NT) enable_aot)exir CaptureConfigrsrDrAqnn_capture_configrxs    ..rDrTc*[R"SS9$)NF)_check_ir_validity)rvr)rsrDrAqnn_edge_configr{s  ! !  rDmodulec^^"SS[RR5mS[RR4UU4SjjmT"U5$)Nc2^\rSrSrSU4SjjrSrSrU=r$)(convert_linear_to_conv2d..Conv2Dc>[TU]5 USLn[RR UR SUR SSSUS9Ul[RRUR"/UR QSPSP765UR l U(a/[RRU5UR l gg)Nr) in_channels out_channels kernel_sizepaddingbias) superrBronnConv2dshapeconv Parameterreshapeweightr)r@rruse_bias __class__s rArB1convert_linear_to_conv2d..Conv2D.__init__s G  4'H"LLO#\\!_ (DI %xx11&..2U&,,2UPQ2UST2UVDII !&!3!3D!9 rDcUR5nUS:XaUR"/URQSP76OUR"S/URQSP76n[R"USS5nUR U5n[R"USS5nUS:XaUR S5nU$UR"URSS6nU$)Nr)dimrrro transposersqueeze)r@xrankress rAforward0convert_linear_to_conv2d..Conv2D.forwards557D*.!) &177&A&19Rqww9RPQ9RA1a(A))A,C//#q!,C%)QY#++b/CJ58KK1Q4PCJrD)rr>)rhrirjrkrBrrr __classcell__)rs@rAConv2Drs :  rDrr|c >[U5n[U[RR5(a1U[ [ U55Vs/sHn[U5PM sn- nUH`n[X5n[U[RR5(dM9[XT"URUR55 Mb UR5H upVT"U5nM U$s snfr>)dir isinstanceror ModuleListrangelenstrgetattrLinearsetattrrrnamed_children) r| attr_strsiattr_str target_attrre sub_modulerreplace_linears rAr0convert_linear_to_conv2d..replace_linearsK fehh11 2 2 %F *<=*s C0)rorModule)r|rrs @@rAconvert_linear_to_conv2drs=0 uxx   & !!rDcSSKnSSKJn [US5nUR 5nSSS5 U"W5R nUR RU5n/nURHn[UR5HupU RS:XdMURU RRRn [ R""U 5n Sn [%U 5S:XaU n Sn USUR&S U U 3n[US 5nUR)U 5 SSS5 UR+U5 M M U$!,(df  GN(=f!,(df  N;=f) ae Dump compiled binaries under the same directory of pte_path. For partitioned graph, there will be multiple files with names f"{method_name}_{index}". 'method_name' refers to the name of a method in the nn.Module that was traced to generate this program, while 'index' indicates the order of execution. Args: pte_path (str): The path of generated pte. rN)deserialize_pte_binaryrb QnnBackendz.binz.dlc/rewb)os#executorch.exir._serialize._programropenreadprogrampathdirnameexecution_plan enumerate delegatesidbackend_delegate_data processedindexdataPyQnnManagerAdaptor StripProtocolrrPwriteappend)pte_pathrrf program_datarctx_path dumpfilesrrdelegateprocessed_bytesbinaryfile_extension dump_files rAdump_context_from_pters<J h vvx  %\2::Gwwx(HI!00$^%=%=>KA{{l*")"?"?&&,,#$ -::?K!'v;!#,F%+N'j.*=*=)>as>BRS )T*aGGFO+  +?1 -  &+*sE E E E) exported_programcSnSnS0pC[U[5(a0UH)nU"U5upg[X65nURU5 M+ O U"U5up7U"UW5 U$)NcRSnSn[U[U0nU[U5"U5$)NcS0p!URRR5GH(up4[URSR 5nUR R[R:XdMRUR RR(dMy[R"URSR UR5nUR5R S:XdS5eUR!5R S:XdS5e[#XR%55nXRU'UR'5 GM+ X4$Nrzfailed to initialize backendzfailed to init context cache) graph_module_modulesitemsr$ compile_specsvaluebackend_options backend_typer kHtpBackend htp_optionsuse_multi_contextsr QnnManagerr InitBackendInitContextCachemaxGetSpillFillBufferSizeDestroy)progmax_sf_buf_size module_mapremoptionsqnn_mgrs rAprocess_exported_programRupdate_spill_fill_size..get_program_info..process_exported_programs(*+RZ))2288:.qq/A/G/GH++88/;;<//;;NNN1<<*00!2C2CG ++-33q86568 00288A=656=&)')G)G)I'O%,qMOO%+;,#. .rDc[R"URSRUR5nUR 5RS:XdS5eUR 5RS:XdS5eUR5nUR5 UU[URSR504$r) rrrrrrrrrr$)r|rspill_fill_sizes rAprocess_lowered_modulePupdate_spill_fill_size..get_program_info..process_lowered_modules)44$$Q'--v/E/EG&&(..!3 S5S S3++-33q8 X:X X8%<<>O OO ",V-A-A!-D-J-JK% rD)r3r-type)rrrdispatchs rAget_program_info0update_spill_fill_size..get_program_infos7 /4  5 "8 W &w//rDcSnUR5H'up4XRRlU"X45 M) g)Nc[[[U55n[U[5(aX R S'gX R S'g)Nr)r,r&r%rr3r_compile_specs)r|rspecs rAset_spec@update_spill_fill_size..update_program..set_specs>46J76STD&/22*.$$Q'+/%%a(rD)rrrr)rrrr|rs rAupdate_program.update_spill_fill_size..update_programs8 0 *//1OFBQ # # / / ? V % 2rDr)rlistrupdate)rrr max_sf_size modules_maprrrs rAupdate_spill_fill_sizer ss +0Z & !""D))$D*:4*@ 'Ok;K   z *% #33C"D ; + rDc[U5 gr>)r )objs rAcanonicalize_programr /s 3rDc[5n[5nU(a9UVs/sH,nU[R[R-;dM*UPM. nn[ X5 U$s snfr>)r1rr decomp_opsrr2) passes_jobsource_decompositionsskip_decompositionsskip_decomp_ops rAget_decomp_tabler3sn46/1#6 "5 ++n.G.GGH "5  /E   s )A" A"Finputscompiler_specsconstant_methodsdynamic_shapes dep_tablerskip_node_id_setskip_node_op_setskip_mutable_buffergenerate_etrecordrc pSn [U[5(dSU0nUR5n U "X5nU "X-5nU "XM5nU "X]5nU "Xm5n0n0nU Vs0sHnU[UUUUU S9/_M nnUR 5Hiunn[ R R UUUUUSS9n[5RUU S9UU'[5RUUUUUS9UU'Mk [U5 [UUUU[5U S9sS S S 5 $s snf!,(df  g =f) a$ Transforms and lowers a given PyTorch module to the QNN backend. Args: module (Union[torch.nn.Module, torch.fx.GraphModule,Dict[str, torch.nn.Module], Dict[str, torch.fx.GraphModule]]): The PyTorch module or fx.GraphModule to be transformed. inputs (Union[Tuple[torch.Tensor], Dict[str, Tuple[torch.Tensor]]]): The input tensors for the module. compiler_specs (Union[List[Any], Dict[str, List[Any]]]): Compiler specifications for Qualcomm AI Engine Direct. constant_methods (Optional[Dict[str, Any]]): An optional dictionary mapping method names to constant values returned by those methods in eager mode. Often used to store configuration information on Edge models. dynamic_shapes (Optional[Dict]): Information about dynamic shapes. dep_table (Optional[Dict]): Dependency table for the transformation passes. passes_job (Optional[Union[OrderedDict, Dict[str, OrderedDict]]]): Ordered dictionary of transformation passes. skip_node_id_set (Optional[set]): Set of node IDs to skip during partitioning. skip_node_op_set (Optional[set]): Set of node operations to skip during partitioning. skip_mutable_buffer (Optional[bool]): Whether to skip delegating the mutable buffer in QNN backend. convert_linear_to_conv2d (Optional[bool]): Whether to convert linear to conv2d in some cases to improve performance in HTP backend. Returns: EdgeProgramManager: The manager for the edge program after transformation and lowering. cUcUVs0sHo"S_M sn$[U[5(aXR5:XaU$UVs0sHo"U_M sn$s snfs snf)a Ensures the input value is a dictionary with keys matching the provided graph names. If the input is not a dictionary or its keys do not match the graph names, a new dictionary is created with the graph names as keys and the input value assigned to each key. Examples: 1. Input is None: >>> ensure_graph_specific_dict(None, ["forward1", "forward2"]) {'forward1': None, 'forward2': None} 2. Input is a single value: >>> ensure_graph_specific_dict(input, ["forward1", "forward2"]) {'forward1': input, 'forward2': input} 3. Input is a non-graph specific dict: >>> ensure_graph_specific_dict({Any: input}, ["forward1", "forward2"]) {'forward1': {Any: input}, 'forward2': {Any: input}} N)rdictkeys)r graph_names graph_names rAensure_graph_specific_dictFto_edge_transform_and_lower_to_qnn..ensure_graph_specific_dictysb& =7BC{${C C eT " "{jjl'BL4?@KjE!K@@DAs A Ar)rrrTrstrict)rrr)transform_passes partitionerrcompile_configrN) rr r!rrroexportrtransform_for_export_pipelineget_to_edge_transform_passesr(r/r{)r|rrrrrrrrrrrr$r" aten_programsr)r#qnn_partitionersreps rA"to_edge_transform_and_lower_to_qnnr2EshA2 fd # #V$++-K ' ** -(-*,/   + *A @ + *sD"D'' D5c[R"S[SS9 [RR XUSS9n[ 5R U5nUR[U55n[US5nUR[55n[ 5RURUUS9n UR"U 6 U$)a TODO: Deprecated capture_program with to_edge_transform_and_lower_to_qnn Captures and transforms a PyTorch module into an Exir exported program. Args: module (Union[torch.nn.Module, torch.fx.GraphModule]): The PyTorch module or fx.GraphModule to be captured. inputs (Tuple[torch.Tensor]): The input tensors for the module. dep_table (Optional[Dict]): Dependency table for the transformation passes. passes_job (OrderedDict, optional): Ordered dictionary of transformation passes. dynamic_shapes (Dict, optional): Information about dynamic shapes. Returns: exir.ExirExportedProgram: The transformed Exir exported program ready for lowering to QNN backend. zNcapture_program is deprecated. Use to_edge_transform_and_lower_to_qnn instead.r stacklevelTr&Fr()warningswarnDeprecationWarningror,rr-run_decompositionsrr*r+r{r.r transform) r|rrrrr1 decomposed_epcore_epedge_epr)s rAcapture_programr>s, MMX   VNSW  XB   7 7 ;B))*::*FGM!-7Gooo/0G%'DD  E  '( NrDcSSKJnJnJnJnJn UR 5n [U 5H&upU RHn XRU'M M( [[U 55Hn URRV s/sH&oRRUS5U :XdM$U PM( n n U"U 5nUSU 3nU"XU5unnnU"UU"UU5UU5nU"X5 U"U5 M UR5 U$s sn f)Nr) erase_nodesfuse_as_graphmodule insert_subgmlegalize_graph topo_sortre)!torch.fx.passes.utils.fuser_utilsr@rArBrCrDpropose_partitionsrnodesmetarrgraphget recompile)gm subgm_tagsubgm_cbptnr@rArBrCrD partitionsr partitionrS node_list sorted_nodessubmodule_namesubgm orig_inputs orig_outputss rA _partition_graph_into_submodulesrYs'')J!*- OOD#$IIi $.3z? # XX^^ +Tyy}}Y/Kq/PD^  !+ %;as++> n, ({L  UN +     B%r'$*LLN I) s #D*Dc /nURRGHnURS:XdMXR;dM'URVs/sHFn[ R "URSRURSRS9PMH nnUR[URUR5[U5U55 URURUSR5R 5 [#SUR$55(dGMURR'U5 URR)[*R,US45nURUlUR/USS9 SSS5 GM UR15 X4$s snf!,(df  GM=f) N call_modulevaldtyperc3Z# UH!oR[R:gv M# g7fr>rIrJrK)rYrEs rAr[:_canonicalize_graph_with_lowered_module..6sDA88x///s)+rc<UR[R:g$r>r`)rZs rA9_canonicalize_graph_with_lowered_module..?sDKK8CSCS4SrD) replace_withdelete_user_cb)rJrHrHrPargsroonesrIrr^rr2 get_submoduletuple set_submodulerrrarbinserting_afterrGrJrKreplace_all_uses_withrL)rMrNredge_prog_mgrsrSarg subgm_input getitem_nodes rA'_canonicalize_graph_with_lowered_modulerrszN 77m # YY(> 99$C 388E?008M8MN$   ! !2$$TYY/{1C^     r"335BB  DDDDXX--d3#%88#9#9 ((q $L)- L%..%1'S/ 43)>LLN ;"43sA GAG G T nn_module sample_input.calibration_cbr:r;fallback_to_cpuc r^^SSKJn SSKJn SSKJn SSKJn Jm UU4Sjn UbUO [5nUbUO [5n[RRXSS9R5n U "U [XV5SS 9nS n[U UU US 9n U"U 5 U RR HPnUR"S :XdMU R%UR&U "U R)UR&555 MR [+U UUS 9un nU(d[-U5nU "U5nUR.R0nUR2Ul[7U5USlU "U [SS9SS 9nSn[U USUS 9n [+U UUS 9un nUR=U5 U U4$! [;S5 U U4s$=f)a Exclude speific operators from quantizer annotation. Skipped operators will defaultly stay in CPU, set 'fallback_to_cpu' to False for trying to delegate them with FP16 precision. e.g.: consider following graph: bias_1 weight_1 input_1 bias_2 weight_2 input_2 | (placeholder) | | (placeholder) | \ | / \ | / \ | / \ | / \ | / \ | / conv2d_1 conv2d_2 (torch.ops.aten.conv2d.default) \ / \ / \_______ _______/ add_1 (torch.ops.aten.add.default) | output If user wants to skip convolution op by names with 'skip_node_id_set' = {"conv2d_1"} "bias_1 / weight_1 / input_1 / input_2 / conv2d_1" will be partitioned out and not annotated / lowered with QNN. [Generated graph] bias_1 weight_1 input_1 input_2 | (placeholder) | | \ | / | \ | / | \ | / | conv2d_1 | \ / \ / \ / lowered_module_1 (QNN fixed precision) | output If user wants to skip convolution op by target with 'skip_node_op_set' = {torch.ops.aten.conv2d.default} "bias_1 / weight_1 / input_1 / conv2d_1, bias_2 / weight_2 / input_2 / conv2d_2" will be partitioned out and not annotated / lowered with QNN. [Generated graph] bias_1 weight_1 input_1 bias_2 weight_2 input_2 | (placeholder) | | (placeholder) | \ | / \ | / \ | / \ | / \ | / \ | / conv2d_1 conv2d_2 (torch.ops.aten.conv2d.default) \ / \ / \__ __/ lowered_module_1 (QNN fixed precision) | output If user wants to delegate the skipped conv2d from above graph with 'fallback_to_cpu' = False: [Generated graph] input_1 input_2 (placeholder) (placeholder) | | \ / lowered_module_2 (QNN fp16 precision) | | lowered_module_1 (QNN fixed precision) | output Args: nn_module (torch.nn.Module): The module to be lowered. quantizer (QnnQuantizer): Instance of QnnQuantizer. compiler_specs (List[CompileSpec]): Compiler specs for Qualcomm AI Engine Direct. sample_input ((torch.Tensor, ...)): Sample input tensors for graph exporting. calibration_cb (callable): Callback function for user-defined calibration. fp_node_id_set ({str, ...}): Set of operator names to be left in fp precision. fp_node_op_set ({torch.ops.aten.xxx, ...}): Set of operator targets to be left in fp precision. fallback_to_cpu (bool): Whether to lower skipped nodes to fp16 or not. Returns: exported_programs: List of programs lowered to QnnBackend (quantized graphs only). r)r)r$)CapabilityBasedPartitioner) convert_pt2e prepare_pt2ec:>T"UT5nXRlU$r>)rrh)rV subgm_namesubgm_preparedrz quantizers rA prepare_subgm&skip_annotation..prepare_subgms#%eY7-7  )rDTr')allows_single_node_partitionannotated_group)rMrNrOrPr[)rMrNrzcFailed to change HTP compiler spec with 'use_fp16' as True, skipped operators will fallback to cpu,)r< skipped_groupcU$r>rs)rVres rArc!skip_annotation..serD)4executorch.backends.qualcomm.serialization.qc_schemar>executorch.backends.qualcomm.serialization.qc_schema_serializer$!torch.fx.passes.infra.partitionerrx'torchao.quantization.pt2e.quantize_pt2eryrzrmror,r|r8rYrJrHrHrkrPrirrrrrkHtpFp16 precisionr%rrdextend)rsr~rrtrur:r;rvrr$rxryrrcapability_partitionerrNrSrn qnn_optioncompile_optionredge_prog_mgrs_fprzs ` @rAskip_annotationrFsNMR(6'A^suN'5'A^suN<<&&yt&LSSUL7>:%) "I3  " L< ""(( 77m #  & & \77 BC )$K %$ L.  07GJ1*=N(88DDK$=$F$FK !&:>&JN1  #"<   =)-"  $ 7+&  +R)+ ' ' /0  ''7 0 ;  / /s A F##F6rop_name soc_model custom_infoc ^^SSKJn SmSS[[[S[[[S[[[4UU4SjjjnSnS n[ S S 9n[ UUS S 9n [U[5(dUOU"U"U5R5U 5n 0n [[4H-n U R5HupU RX5 M M/ SunnnUbTU"USU 5nU"USU 5nURSS5nURSS5nURSS5nUSnO[R "[#U 5U 5nUR%5R&S:XdS5eUR)5SnUR+U5 U"UR-U5U 5nU"UR/U5U 5nUR15 U"UUUUU5nUR3UUS.5 [5UUS0[7S S9S9nUR8UR:R<H[nUR>S:XdM[@RB[URD5;dM>U URF[@RH' O URK[MU 55US'U$)Nr)Pathc^[RRU[U5SS9S[[R 4U4Sjj5ng)NCompositeExplicitAutograd) dispatch_keyrcD>[STR555$)Nc3# UH8n[R"[UR5SURS9v M: g7f)rI)devicer^N)rozerosrjrr^)rYvs rAr[Mfrom_context_binary..implement_op..op_impl..s0)A E!''N6I)sAArjvaluesroutputss rAop_impl:from_context_binary..implement_op..op_impl s& ) rD)rolibraryimplrr Tensor) custom_oprrrs ` rA implement_op)from_context_binary..implement_op sE    s7|2M   D.   rD qnn_in_orderexecutorch_in_orderexecutorch_out_orderc>^SnT SUS3n[[RS5nURU5 T "UT U5 "U 4SjS[R R 5nU(a[U4SjU55O[TR55mU"X$5n [RRU TSS 9n UU U S .$) NzTensor[] inputs(z) -> AnyFRAGMENTc:>^\rSrSrSrU4SjrU4SjrSrU=r$)7from_context_binary..build_graph..Modeli&a The args of forward() can be thought of as what executorch is accepting as input. The getattr inside the forward() can be thought of as qnn context binary. When we first pass in the input, we need to use the executorch's(nn.module) input order. After we get into forward(), we then need to convert input order to qnn's input order. Same as return, when qnn returns the value, we need to reorder them back to executorh's output order. c:>[TU]5 XlX lgr>)rrBrr)r@rrrs rArB@from_context_binary..build_graph..Model.__init__/s "$0!,@)rDcV>^UR(a[U4SjUR55m[[[R[ R 5T5RT5nUR(aURVs/sHo2UPM sn$U$s snf)Nc3.># UH nTUv M g7fr>rsrYrrs rAr[Rfrom_context_binary..build_graph..Model.forward..6s"H6G6!96Gs) rrjrroopsr namespacedefaultr)r@rretidxrs ` rAr?from_context_binary..build_graph..Model.forward4s$$""Hd6G6G"HHFEII'@'@A7'&/ 00*.)B)BC)B#X)BCCsB&)rr) rhrirjrkrlrBrrrr)rrs@rAModelr&s  A  rDrc3\># UH!n[TR55Uv M# g7fr>rrs rAr[;from_context_binary..build_graph..As#I5H% (+5Hs),Tr)r custom_moduler) r6rrdefinerorrrjrr,) rrrrr inputs_str func_protorrmodelrrrs ` rA build_graph(from_context_binary..build_graphs' y*X6 O55zB $Y1 EHHOO 8# I5HI Iv}}'  l9||""5&">#" $  rDc[5nUHnnUR5nURUS5nUc SU35e[R"[ UR 55US9X#R5'Mp U$)Nzunknown qnn data type r])r GetDataTyperKrorrjGetDimsGetName)tensors dtype_maprtr^ dtype_torchs rA build_tensor)from_context_binary..build_tensorOsrmAMMOE#--t4K* L.DUG,L L*${{5+=[QC    rDct[R"[U55n[UR U55$r>)rrrbytesMakeBinaryInfo)ctx_binrrs rApreprocess_binary.from_context_binary..preprocess_binaryYs2%00 *> : W++G455rDF)use_fp16T)rris_from_context_binaryNNN graph_inputs graph_outputsr#zfailed to load context binaryrr) _use_edge_ops)r+rGedge_program_manager)'pathlibrr r intgenerate_htp_compiler_spec%generate_qnn_executorch_compiler_specrr read_bytesrrr setdefaultrKrrrInitr GetGraphNamesAllocateTensorGetGraphInputsGetGraphOutputsrrr+r)_edge_programsrJrHrHrrrIrI meta_ctx_bin to_backendr)rrrrrrrrrrrrtype_mapkrrrrrrr#r bundle_prog edge_prog_mgrrErs ` @rAfrom_context_binaryrs -13748 7 tCy)7 &d3i0 7 'tCy1 7 7 r61%@O:'#N(C((  txj2==? P I')<=NN$DA   &%>?O;L%'; k.99E{?;YG"~t< )oo.CTJ*/EtL . &00 *> :  ||~##q(I*II(**,Q/ z*g44Z@)Lw66zBIN':  [!345(u=M ) )* 5 ; ; A A 44? "'@'@CM'Q3:AFF?// 0 B +8*B*B~&+K&' rDc\rSrSrSrSrSrg) DrawFormatirrrsN)rhrirjrkSVGPYDOTrrrsrDrArrs C ErDrrc[RRX 5n[R"SSS9 U[ R :XaI[USUS3S5nURUR5R55 SSS5 gU[ R:Xa&UR5RUSUS35 g[SUS 35e!,(df  g=f) NzMFor large models such as LLM, it is strongly recommended to use PYDOT format.rr4rz.svgrz.dotzUnknown format .)r4 graph_drawer FxGraphDrawerr6r7rrrr get_dot_graph create_svgr write_raw RuntimeError)titlerrformatrJrs rA draw_graphrs    - -l BE MMW TF!E7$' .! GGE'')446 7/ . :## # ''4&%(=>_VHA677 / .s .C C)ruse_memory_optimizationsuse_node_optimizationsuse_queue_recordinguse_weight_sharingc[5nXlXlX%lX5lXEl[ [RUS9$)a Helper function generating backend options for QNN HTP Args: precision: kGpuPrecisionFp32 - Sets the precision mode to floating point 32-bit (FP32). kGpuPrecisionFp16 - Sets the precision mode to floating point 16-bit (FP16). kGpuPrecisionHybrid - Sets the precision mode to FP16 for storage and FP32 for calculations. kGpuPrecisionUserProvided - Uses the tensor data type provided by the user. use_memory_optimizations: If true, backend will share NATIVE tensor memory based upon analysis of the network topology. use_node_optimizations: If true, backend will fuse compatible operations into one operation to improve performance. use_queue_recording: If true, backend will use queue recording to improve performance. use_weight_sharing: Used with multiple_graphs, where model size will be reduced when operations have the same weights across multiple graphs. Returns: QnnExecuTorchGpuBackendOptions: backend options for QNN GPU. )r gpu_options) rrrrrrrr kGpuBackend)rrrrrr s rAgenerate_gpu_compiler_specr sC812K%+C()?&&9#%7" &-99 rDruse_dlbcruse_slc_allocatorc[5nU(a[RO[RUl[ R UlX%lX5l Xl XEl [[RUS9$)a5 Helper function generating backend options for QNN HTP Args: use_fp16: If true, the model is compiled to QNN HTP fp16 runtime. Note that not all SoC support QNN HTP fp16. Only premium tier SoC like Snapdragon 8 Gen 1 or newer can support HTP fp16. use_dlbc: Deep Learning Bandwidth Compression allows inputs to be compressed, such that the processing bandwidth can be lowered. use_multi_contexts: When multiple contexts are generated inside the same pte, it is possible to reserve a single spill-fill allocation that could be re-used across all the splits. use_weight_sharing: Used with multiple_graphs, where model size will be reduced when operations have the same weights across multiple graphs. use_slc_allocator: Allows user to enable the usage of the System Level Cache Allocator for a given graph. It will help the by reducing overall bandwith on the use case. The feature is only supported by specific SOCs. Returns: QnnExecuTorchHtpBackendOptions: backend options for QNN HTP. )rr)rrr kHtpQuantizedrr kHtpBurstperformance_moderrr r rrr)rr rrr rs rArrso812K  "** & 4 4$C#L#LK %7"%7"#$5! &-99 rDrdebugsaveronline_preparedump_intermediate_outputsprofileoptrace shared_bufferrop_package_options use_mha2shac ^[Vs1sHoRiM n nWU ;a[SU35eU(aU(a[R"SSS9 [ [ UU5n U(a[RO[RU l X]l U(aSU l SU l SU lU(a[R U lO2U(a[R$U lO[R&U lU(aDUR([*R,:Xa&UR.R0(a [S5eXlXMlXlU (a[9U R:5S :aXlXl[A[B[EU 55/$s snf) aM Helper function generating compiler specs for Qualcomm AI Engine Direct Args: soc_model: The SoC you plan to run the compiled model. Please check QcomChipset for supported SoC. SM8450 (Snapdragon 8 Gen 1) SM8475(Snapdragon 8 Gen 1+) SM8550(Snapdragon 8 Gen 2) SM8650(Snapdragon 8 Gen 3) SM8750(Snapdragon 8 Elite) SM8850(Snapdragon 8 Elite Gen 5) backend_options: Options required by different backends. debug: Enable verbose logging. Disclaimer: this option must change in the near future. online_prepare: Compose QNN graph on device if set to True saver: Instead of compiling the model, run QNN Saver. Please check documents of Qualcomm AI Engine Direct SDK. This feature is usually for debugging purpose. dump_intermediate_outputs: If tensor dump is enabled, all intermediate tensors output will be dumped. This option exists for debugging accuracy issues profile: Enable profile the performance of per operator. Note that for now only support kProfileDetailed to profile the performance of each operator with cycle unit. shared_buffer: Enables usage of shared buffer between application and backend for graph I/O. is_from_context_binary: True if current graph comes from pre-built context binary. op_package_options: Optional structure to specify op packages loaded and used by the backend. use_mha2sha: This experimental parameter is used to decide whether to enable multi-head attention to single-head attention pass, aiming to reduce time consumption in AOT and improve performance on HTP. Returns: List[CompileSpec]: Compiler specs for Qualcomm AI Engine Direct. Raises: ValueError: The value QcomChipset is currently not supported. ValueError: Confliction between compiler specs. zunknown SoC model for QNN: zIt is not recommended to turn on both profiling and dump_intermediate_outputs the same time, because dump_intermediate_outputs will cause performance drop.rr4zlibQnnSaver.soT saver_outputzc'use_multi_context' could not function in online prepare mode, please set 'online_prepare' to Falser)#rr ValueErrorr6r7r"rr kLogLevelDebugkLogLevelError log_levelr library_pathrsaver_output_dirr#kProfileOptrace profile_levelkProfileDetailed kProfileOffrrrrrrrrrop_package_infosrrr,r&r%)rrrrrrrrrrrr_supported_soc_modelsqnn_executorch_optionss rArrsh?JJk__kJ--6ykBCC,  O 2 "O  ,, " 1 1$ 8Q4 .>+'+$2@//H/X/X,  % 6 6 ,0I/T/T,   ( (,D,P,P P  ' ' : : 3  ,9(,:)4J1c"4"E"EFJ4F1)4& )+?@V+WX oKsF*c0S[R_S[R_S[R_S[R_S[R_S[R_S[R_S[R _S [R _S [R_S [R_S [R_S [R_S[R_S[R _S[R_S[R_[R[R[RS.E$NSA8295SA8797SM8350SM8450SM8475SM8550SA8255SM8650SM8750SM8850SSG2115PSSG2125PSXR1230PSXR2230PSXR2330PQCS9100SAR2230P)SW6100QCM6490SM8845)rV68V81V69V73V75V79rsrDrAget_soc_to_arch_maprFs4 '++ '++  '++  '++   '++   '++   '++  '++  '++  '++  GKK  GKK  GKK  GKK  GKK  7;;! " GKK# $++;;++) rDc0S[R_S[R_S[R_S[R_S[R _S[R _S[R_S[R_S [R_S [R_S [R_S [R_S [R_S[R_S[R_S[R _S[R"_[R$[R&[R(S.E$r+)rr,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?rsrDrAget_soc_to_chipset_maprHs\ +$$ +$$  +$$  +$$   +$$   +$$   +$$  +$$  +$$  +$$  K((  K((  K((  K((  K((  ;&&! " K((# $$$&&$$) rDrMc^0nURRHnURnSnURS:Xa<SUR;a,[ URSR 55SSnURU;dMoURXE4/5RU5 M /nUR5HOuupGnSRUVs/sHo3RPM sn5n UR[U5Xy/5 MQ [[U/SQSS 95 gs snf) z Print a quick preview of op targets and module stack. Use this to inspect the FX graph and identify module stack, which helps you craft regex or op-target for quantization recipe. NrGnn_module_stackrrz, )z Op Targetz Module StackNodesgrid)headerstablefmt)rJrHrIrHrIrrrrrjoinrPrrdr0) rM supported_opsmodule_metadatarSrIdeepest_module table_rows module_stackrH node_namess rA%show_nn_module_stack_for_quant_reciperVs O 77o %*;tyy*H!$)),=">"E"E"GHLQON ;;- '  & &'? D K KD Q J)8)>)>)@%YYe*>s*CBQ*G!HL  ( I]-IIrDc[5n[R"SU5nU(a$[[UR 5SS5up4O[ SU35e[[URS5SS5upVX5:H=(a XF:$rkrlrrs rAis_qnn_sdk_version_greater_thanr}&r{rDc,[R"5$r>)rGetQNNCtxBinAlignmentrsrDrA get_qnn_context_binary_alignmentr6s  4 4 66rD) NNNNNNFFFr)NNT)FFFF) FFFFFFFFNF)rJrrmr6 collectionsrrenumrtypingrrrr r r r 7executorch.backends.qualcomm.python.PyQnnManagerAdaptorbackendsqualcommpythonrexecutorch.exirrvro$executorch.backends.qualcomm._passesr r5executorch.backends.qualcomm._passes.qnn_pass_managerr2executorch.backends.qualcomm.builders.node_visitorrr3executorch.backends.qualcomm.builders.qnn_constantsr6executorch.backends.qualcomm.partition.qnn_partitionerrrrrrrrrrrrrrrr r!r"r#rr$r%,executorch.backends.qualcomm.utils.constantsr&r'8executorch.backends.qualcomm.utils.qnn_manager_lifecycler(r)r*r++executorch.exir.backend.compile_spec_schemar,&executorch.exir.lowered_backend_moduler- executorch.exir.program._programr.r/r0 torch._decompr1r2torch.export.exported_programr3torch.fxr4 torch.fx.passes.operator_supportr5 torch.libraryr6r8rxr{rrrrrr r _ops OperatorBaserrp GraphModulerrmrnr2r>rYrrrr3rrrrrkGpuPrecisionUserProvidedr rrrFrHrVrYrbrirzr}rrrsrDrArs  0DDDUU NPP  WKKCGI9@!=,=@/// '"UXX__'"T$tCy$NH%-A(BBHV !D)@)@()J$K!626%) $GK&*&**/(-/4#C    S%((// !" S%((&& &' ) C  % %tCu||1D,D'EE FC $s)T#tCy.%99:C tCH~.C TNC ~C {Dk1A,BBCDC smC smC "$C  ~!C "'tn#C $%C R!%" ) %((//588#7#77 8) %,, )~) )  )   )X%P$Z {(xx{( c)* {( ehh223T9: {(  {({({(B)// _Ek_ __ _D HR~~ 8%((*>*> 8 ,E+^+^%)#' $$ &(&"&!& &  & ! &V$$# ---- -  - ! -f &+#(8<mm0m m  m  m $ mmmm!m6mm +m`22ehh.B.B:1UXX))1(1/=((&&/=7;C