iE3SSKrSSKJr SSKJrJrJr SSKrSSKJ r S\RS\ S\ 4Sjr S"S \ S \ S \ S \S\\ S\ S \\\R 44SjjrS\RS\R4SjrS\RS\RS\RS\RS\\R\R44 SjrS\RS\RS\RS\R4Sjr"SS\R*R,5rS#S \ S \ S \ S\ S \\\R 44 SjjrSrS$SjrS$Sjr"S S!\R*R,5rg)%N)partial)OptionalTupleUnion) ModelArgsfreqs scale_factorhigh_freq_factorcvSnSnXC- nXB- n/nUH}nS[R-U- n X:aURU5 M1X:aURX- 5 MKXV:wdeXI- U- X#- - n URSU - U-U- X--5 M [R"XpR UR S9$)Ni )dtypedevice)mathpiappendtorchtensorrr) rr r low_freq_factorold_context_lenlow_freq_wavelenhigh_freq_wavelen new_freqsfreqwavelensmooths d/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/executorch/examples/models/llama/rope.py apply_scalingrsOO&8':Idgg+$  &   T "  '   T0 1#8 88%//A 2F   a&jD0<?&-O P << U\\ JJdimendtheta use_scaledrc |SU[R"SUSUS9SUS-R5U- -- n[R"XRS9nU(aUce[ XtU5n[R "X5R5n[R "U5n [R"U5n X4$)N?rr )r)rarangefloatrroutercossin) r r!r"r#r r rrt freqs_cos freqs_sins rprecompute_freqs_cisr.+s  %,,q#q8C1HFLLNQTTU E  S.A'''e3CD KK ! ' ' )E % I % I  r freqs_cisxcFURnURnUS:XaURURSURSURS4:Xde[UR5VVs/sH!upEXBS- :XdXBS- :XdXBS- :XaUOSPM# nnnOgURURSURS4:Xde[UR5VVs/sHupEUS:XdXBS- :XaUOSPM nnnURU5$s snnfs snnf)Nr r )ndimshape enumerateview)r/r0r6freqs_cis_ndimidr7s rreshape_for_broadcastr=As 66D^^N1772; QWWR["IIII"!''* *ax-1q=AMA I*   1771:qwwr{";;;;AJ177ASTASa1fAX 14AST >>%   Us 3(D%Dxqxkr,r-returncNUR5RURSSS-5RS5upEUR5RURSSS-5RS5upg[ X$5n[ X45nXB-XS-- nXC-XR--n Xb-Xs-- n Xc-Xr--n [ R "X/SS9RS5n [ R "X/SS9RS5n U RU5U RU54$Nr5)r5r r r2 r'reshaper7unbindr=rstackflattentype_as)r>r?r,r-xq_rxq_ixk_rxk_ixq_out_rxq_out_ixk_out_rxk_out_ixq_outxk_outs rapply_rotary_embrTRs##BHHSbMG$;<CCBGJD##BHHSbMG$;<CCBGJD%i6I%i6I$"22H$"22H$"22H$"22H [[(-2 6 > >q AF [[(-2 6 > >q AF >>" v~~b1 11rc>UR5RURSSS-5RS5up4[ X5n[ X#5nX1-XB-- nX2-XA--n[ R "XV/SS9RS5nURU5$rBrD)r?r,r-rLrMrPrQrSs rapply_rotary_emb_to_krVfs##BHHSbMG$;<CCBGJD%i6I%i6I$"22H$"22H [[(-2 6 > >q AF >>" rc^\rSrSrU4SjrS\R S\R S\R S\R 4SjrSrU=r $) RotaryEmbeddingwc">[TU]5 gN)super__init__)self __class__s rr]RotaryEmbedding.__init__xs rr>r?r,r-c$[XX45upVXV4$r[rT)r^r>r?r,r-rRrSs rforwardRotaryEmbedding.forward{s*")G~r) __name__ __module__ __qualname____firstlineno__r]rTensorrc__static_attributes__ __classcell__r_s@rrXrXwsH LL LL<<  << rrXpartial_rotary_factorc [X-5nSU[R"SUSU[RS9R 5U- -- n[R"XR [RS9R U5n[R"Xe5R 5n[R"XU4SS9n[R"U5n[R"U5n X4$)Nr%rr )rrr5rC) intrr&int64r'rrIr(catr)r*) r r!r"rnrrr+embr,r-s rhf_precompute_freqs_cisrts c) *C  LLC6 E K K MPS S U E  SU[[AII  A KK ! ' ' )E ))UN +C #I #I  rcUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Nr5r rC)r7rrr)r0x1x2s r rotate_halfrxsZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''rcNURU5nURU5nURSnUSSU24USUS24pUSSU24USUS24p[R"Xr-[ U5U--U/SS9n [R"X-[ U 5U--U /SS9n X4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. r5.NrC) unsqueezer7rrrrx) qkr)r* position_ids unsqueeze_dim rotary_dimq_rotq_passk_rotk_passq_embedk_embeds rhf_apply_rotary_embrs( -- &C -- &C2Jc;J;&'3 +;)<6c;J;&'3 +;)<6ii%++e*C/0G Nrc^\rSrSrS\4U4SjjrS\RS\RS\RS\R4SjrS \ \RS \ 4S jr S \R4S jr Sr U=r$)Ropeparamsc (>[TU]5 XlURR(aI[ [ URR [URSS5S9Ul[Ul Ov[ [URRURRURR[URSS5S9Ul[5Ul URURRURR cURR"OURR"S-URR$5up#UR'SUSS9 UR'S USS9 g) Nrcpu)rnr)r#r r rr r,F) persistentr-)r\r]r use_hf_roperrtrngetattrr.rrTuse_scaled_roperope_scale_factorr rXhead_dimffn_dim_multipliermax_context_lenrope_freq_baseregister_buffer)r^rr,r-r_s rr] Rope.__init__s;   ;; " "(/'&*kk&G&Gt{{He<)D % %8D !(/$;;66![[::!%!=!=t{{He< )D %%4$5D ! $88 KK ;;119 ++[[0014 KK & &   [)F [)Frr{r|r,r-c&URXX45$r[rb)r^r{r|r,r-s rrc Rope.forward s$$Q9@@r input_posseq_lenc<URR(aUcS5eURR(aUSR5n[R "U5 [R "X0RR:5 URRSX25nURRSX25nXE4$URUnURUnXE4$UbS5eURSUnURSUnXE4$)a: Get the precomputed frequencies for the given input position and sequence length. Args: input_pos (torch.Tensor): The input position tensor. seq_len (int): The sequence length. Returns: Tuple[torch.Tensor, torch.Tensor]: The precomputed frequencies for the given input position and sequence length. Nz4input_pos must be provided when use_kv_cache is Truer5rz.input_pos is unused when use_kv_cache is False) r use_kv_cacheenable_dynamic_shapeitemr_check_is_size_checkrr,narrowr-)r^rrinput_pos_itemr,r-s r get_freqsRope.get_freqss ;; # #% FE F%{{//!*2!3!3!5$$^4 ^kk.I.IIJ NN11!^M  NN11!^M ##!NN95  NN95 ##$ V&V V$x0Ix0I##rindicesc>URUURU4$)z Get the precomputed frequencies for given input indices. Args: indices (torch.Tensor): The input indices tensor. Returns: Tuple[torch.Tensor, torch.Tensor]: The precomputed frequencies for given input indices. )r,r-)r^rs rget_freqs_using_indicesRope.get_freqs_using_indices=s!~~g&w(???r)rTrr.)rfrgrhrirr]rrjrcrrprrrkrlrms@rrrs!Gy!GFA <<A <<A<< A << A%$8ELL#9%$C%$N @u|| @ @rr)g@FNr)r%r)Nr )r functoolsrtypingrrrr+executorch.examples.models.llama.model_argsrrjrprr'boolstrrr.r=rTrVnnModulerXrtrxrrrrerrrs )) A KKSKCK4"&',       3-     #u||# $ ,!U\\!ell!"2 2,,238<<2LQLL2 5<< %&2(   !& :?,,  \\ " ehhoo 2$'',      !  #u||# $  :(@2^@588??^@r