9iSSKrSSKrSSKrSSKrSSKJr SSKJs Jr SSK J s J r SSK JrJrJr "SS\R"5rSS\R&-4SjrSrS r"S S \R"5r"S S \R"5r"SS\R"5r"SS\R"5r"SS\R"5r"SS\R"5rg)N)DropPath to_2tuple trunc_normal_cR^\rSrSrSrSS\R S4U4SjjrSrSr U=r $)MlpzMultilayer perceptron.Nc>[TU]5 U=(d UnU=(d Un[R"X5UlU"5Ul[R"X#5Ul[R"U5UlgN) super__init__nnLinearfc1actfc2Dropoutdrop)self in_featureshidden_features out_features act_layerr __class__s b/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/models/cv/vidt/backbone.pyr Mlp.__init__s] #2{ )8[99[:;99_;JJt$ cURU5nURU5nURU5nURU5nURU5nU$r )rrrr)rxs rforward Mlp.forward!sH HHQK HHQK IIaL HHQK IIaLr)rrrr) __name__ __module__ __qualname____firstlineno____doc__rGELUr r __static_attributes__ __classcell__rs@rrrs'!"&"77 %rri'c ^US-nU)nURS[RS9nURS[RS9nSnXfSS2SS2SS24U-- U-nXwSS2SS2SS24U-- U-n[R"U[RURS9n USU S--U- -n USS2SS2SS2S4U - n USS2SS2SS2S4U - n [R "U SS2SS2SS2SSS24R 5U SS2SS2SS2SSS24R54S S 9RS 5n [R "U SS2SS2SS2SSS24R 5U SS2SS2SS2SSS24R54S S 9RS 5n [R"X4S S 9n U $) aMasked Sinusoidal Positional Encoding Args: x: [PATCH] tokens mask: the padding mask for [PATCH] tokens num_pos_feats: the size of channel dimension temperature: the temperature value scale: the normalization scale Returns: pos: Sinusoidal positional encodings r+)dtypegư>N)r.devicerdim) cumsumtorchfloat32aranger0stacksincosflattencat) rmask num_pos_feats temperaturescalenot_masky_embedx_embedepsdim_tpos_xpos_yposs rmasked_sin_pos_encodingrJ*s$"Q&MuHooau}}o5Gooau}}o5G CBC+c12U:GArs+c12U:G LLemmAHH ME !uz*]: ;E Aq!TM "U *E Aq!TM "U *E KK q!Q1}  ! ! #U1aADqD=%9%=%=%?@  wqz  KK q!Q1}  ! ! #U1aADqD=%9%=%=%?@  wqz  ))UN *C JrcURup#pEURX#U-XU-UU5nURSSSSSS5R5RSXU5nU$)z Args: x: (B, H, W, C) window_size (int): window size Returns: windows: (num_windows*B, window_size, window_size, C) rr-r4r+r1r/)shapeviewpermute contiguous)r window_sizeBHWCwindowss rwindow_partitionrWWsiJA! q{"Kk1A; Aii1aA%:<R1(M Nrc[URSX#-U- U- - 5nURXBU-X1-UUS5nURSSSSSS5R 5RXBUS5nU$)z Args: windows: (num_windows*B, window_size, window_size, C) window_size (int): Window size H (int): Height of image W (int): Width of image Returns: x: (B, H, W, C) rr/r-r4r+r1rL)intrMrNrOrP)rVrQrSrTrRrs rwindow_reverserZhs~ GMM!  3k A BCA Q[(!*:K " &A !Q1a#..055aArBA HrcH^\rSrSrSrSU4SjjrSSjrSrU=r$)ReconfiguredAttentionModulezaWindow based multi-head self attention (W-MSA) module with relative position bias -> extended with RAM. It supports both of shifted and non-shifted window. !!!!!!!!!!! IMPORTANT !!!!!!!!!!! The original attention module in Swin is replaced with the reconfigured attention module in Section 3. All the Args are shared, so only the forward function is modified. See https://arxiv.org/pdf/2110.03921.pdf !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Args: dim (int): Number of input channels. window_size (tuple[int]): The height and width of the window. num_heads (int): Number of attention heads. qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True qk_scale (float | None, optional): Override default qk scale of head_dim ** -0.5 if set attn_drop (float, optional): Dropout ratio of attention weight. Default: 0.0 proj_drop (float, optional): Dropout ratio of output. Default: 0.0 c>[TU]5 XlX lX0lX-nU=(d US-Ul[ R"[R"SUS-S- SUS-S- -U55Ul [R"URS5n [R"URS5n [R"[R"X/55n [R"U S5n U SS2SS2S4U SS2SSS24- n U RSSS5R!5n U SS2SS2S4==URSS- - ss'U SS2SS2S4==URSS- - ss'U SS2SS2S4==SURS-S- -ss'U R#S5nUR%SU5 [ R&"XS-US9Ul[ R*"U5Ul[ R&"X5Ul[ R*"U5Ul[3URS S 9 [ R4"SS 9Ulg) Ngr+rr-r/relative_position_indexr4bias{Gz?stdr2)r r r3rQ num_headsrAr Parameterr6zerosrelative_position_bias_tabler8r9meshgridr<rOrPsumregister_bufferrqkvr attn_dropproj proj_droprSoftmaxsoftmax)rr3rQreqkv_biasqk_scalermrohead_dimcoords_hcoords_wcoordscoords_flattenrelative_coordsr_rs rr $ReconfiguredAttentionModule.__init__s@ &"#/4 -/LL KK[^+a/A A4F4JK! #-$) << 0 0 34<< 0 0 34U^^X,@ABvq1(A)-*./1?@DaAH2II*11 q!Z\ 1 "..q1A5 6 1a D$4$4Q$7!$;; 1a A(8(8(;$;a$?? "1"5"5b"9 64 699S'9I.IIc' I.d77SAzzb) rc  URSURS:XdeURSnXf-nU(dhURuppX-n URXU 5n[R"X/SS9nUR U5n U SS2SU 2SS24U SS2U S2SS24pOUSRuppX-n USRunnnnUU-nUSRXU 5nUSRUUU 5n[R"UUU/SS9nUR U5n U SS2SU 2SS24U SS2XU-2SS24U SS2U U-S2SS24nnnURXU S5n[ X5nURSnURUXf-SURXR-5nURSSSSS5nUSUSUSnnnUUR-nUURS S5-nURURRS5RURSURS-URSURS-S5nURSSS5R5nUURS5-nUbsURSnURUU-UURXw5nURS5RS5n UU -nURSURUU5nUR!U5nUR#U5nUU-RSS5RUXfU 5n!URUSSURXR-5nURSSSSS5nUSUSUSn$n#n"U(aWRUWWSURXR-5nUSS2SS2SS2SS2SS2SS24RSSSSSS 5R5n%U%RSXRUU-S5n%U%SU%Sn'n&[R"U&U#/SS9[R"U'U$/SS9n$n#U"UR-n"U"U#RS S5-n(UbU(U-n(UR!U(5n(UR#U(5n(U(U$-RSS5RUSU 5n)[%U!XiU 5n![R"U!RXU -U 5U)/SS9nUR'U5nUR)U5nUSS2SX-2SS24RXX5n!USS2X-S2SS24n)U!U)4$) a Forward function. RAM module receives [Patch] and [DET] tokens and returns their calibrated ones Args: x: [PATCH] tokens det: [DET] tokens mask: (0/-inf) mask with shape of (num_windows, Wh*Ww, Wh*Ww) or None -> mask for shifted window attention "additional inputs for RAM" cross_attn: whether to use cross-attention [det x patch] (for selective cross-attention) cross_attn_mask: mask for cross-attention Returns: patch_x: the calibrated [PATCH] tokens det_x: the calibrated [DET] tokens rr-r2Nr/r4r+r1rL)rQrMrNr6r=rlrWreshapererOrA transposerhr_rP unsqueezerqrmrZrnro)*rrdetr> cross_attncross_attn_maskrQlocal_map_sizerRrSrTrUNfull_qkv patch_qkvdet_qkv_ori_Hori_Wori_N shifted_xcross_xcross_patch_qkvB_ _patch_qkvpatch_qpatch_kpatch_v patch_attnrelative_position_biasnWtmp0tmp1patch_xdet_qdet_kdet_vpatch_kv cross_patch_k cross_patch_vdet_attndet_xs* rr #ReconfiguredAttentionModule.forwards."d&6&6q&9999&&q) $2JA!AqQA 1(*Axx{H!)!RaR(!3XaQh5Gw1JA!A!"1 AueQEME! !*Idii5!,G 9gs3;Axx{HBQB"HQe) Q->$?!QQVYZYZJZA[)0INN1B/ % $ __Q %%b+*CQ&*nna>>6IK &&q!Q15 $.qM:a=*Q-'DJJ& 1 1"b 99 !%!B!B  ( ( - -b 1"3374  #d&6&6q&99  #d&6&6q&994  "8!?!? q!"Z\ "8"B"B1"EE   AB??28R#1CD>>!$..q1DJ#T^^^)79J\\*- ^^J/ '221a8@@  !-,,q"ann9LM//!Q1a0%aj'!*gajeu .221eUA37>>343FHO'q!Q'(r1a(0118Aq!Q9:2<>55="MH,4A; =M!99mU%;)*,-2YY u7M;<.>E  "EOOB33  &/1H<<)>>(+E!,,Q2::1b!D!+!< IIw||A1ua0%8a @ IIaL NN1 AvvqL/&&qQ2!QUVQ,~r) rmr3rernrorlrhrArqrQ)TNr r )NFN r"r#r$r%r&r r r(r)r*s@rr\r\zs3.-*d $ EErr\c r^\rSrSrSrSSSSSSSS\R \R4 U4S jjrS r S r U=r $) SwinTransformerBlockiEa$Swin Transformer Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_size (int): Window size. shift_size (int): Shift size for SW-MSA. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True qk_scale (float | None, optional): Override default qk scale of head_dim ** -0.5 if set. drop (float, optional): Dropout rate. Default: 0.0 attn_drop (float, optional): Attention dropout rate. Default: 0.0 drop_path (float, optional): Stochastic depth rate. Default: 0.0 act_layer (nn.Module, optional): Activation layer. Default: nn.GELU norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm r@TNr c >[TU]5 XlX lX0lX@lXPlSUR s=::aUR:dS5e S5eU "U5Ul[U[UR5UUUU US9Ul U S:a [U 5O[R"5UlU "U5Ul[!X-5n [#UU U US9UlSUlSUlg)Nrz shift_size must in 0-window_size)rQrerrrsrmror )rrrr)r r r3rerQ shift_size mlp_rationorm1r\rattnrrIdentity drop_pathnorm2rYrmlprSrT)rr3rerQrrrrrsrrmrr norm_layermlp_hidden_dimrs rr SwinTransformerBlock.__init__Ws "&$"DOO6d&6&66Z8ZZ6Z8ZZ6_ / !$"2"23 $b." .0kkm _ S_-*  rc URupgnURURpXyU -UR-:XdS5eUn UR U5nUSS2SX-2SS24USS2X-S2SS24pUR XiX5nUn S=pUR XR -- UR -nUR XR -- UR -n[R"USSUUUU45nURunnnnUunnURU5nURS:a0[R"XR*UR*4SS9nUnOUnSnU(aU U-nU U-n UU4nOU U-n UnURUUU UUS9unn URS:a-[R"UURUR4SS9nOUnUS:dUS:aUSS2SU 2SU 2SS24R5nUR XiU -U5n[R"X/SS9nXR!U5-nXR!UR#UR%U555-nU$) aForward function. Args: x: Input feature, tensor size (B, H*W + DET, C). i.e., binded [PATCH, DET] tokens H, W: Spatial resolution of the input feature. mask_matrix: Attention mask for cyclic shift. "additional inputs' pos: (patch_pos, det_pos) cross_attn: whether to use cross attn [det x [det + patch]] cross_attn_mask: attention mask for cross-attention Returns: x: calibrated & binded [PATCH, DET] tokens input feature has wrong sizeNr)r-r+)shiftsdims)r>rrrr-r2)rMrSrT det_token_numrrNrQFpaddet_pos_linearrr6rollrrPr=rrr)rr mask_matrixrIrrrRLrUrSrTshortcutrorig_xpad_lpad_tpad_rpad_brHpWp patch_posdet_posr attn_mask cross_patchs rr SwinTransformerBlock.forwards"''avvtvv1ED....N0NN. JJqM1fqufa<!AquvqL/3 FF1 !!A(8(8$88DVMI#III  9,K-C"K0I-C!I !+ #- 3 ??Q  9A A 19 !RaR!Q,**,A FF1!eQ  IIqhA & ~~a( ( txx 1 67 7r) rSrTrr3rrrrrrerrQ) r"r#r$r%r&rr' LayerNormr r r(r)r*s@rrrEsB(77LL*XXXrrcN^\rSrSrSr\R S4U4SjjrSrSr U=r $) PatchMergingizPatch Merging Layer Args: dim (int): Number of input channels. norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm Tc>[TU]5 XlU(aSU-OSn[R"SU-USS9UlU"SU-5Ul[R"XSS9UlU"U5Ulg)Nr+r1Fr`) r r r3rr reductionnorm expansionr)rr3rexpand expand_dimrs rr PatchMerging.__init__si !'QWC 1s7JUCq3w' 3?_ rc URupEnXRU-UR-:XdS5eUSS2SX#-2SS24USS2X#-S2SS24pqURXBX65nUS-S:H=(d US-S:HnU(a#[R"USSSUS-SUS-45nUSS2SSS2SSS2SS24n USS2SSS2SSS2SS24n USS2SSS2SSS2SS24n USS2SSS2SSS2SS24n [ R "XX/S5nURUSSU-5nURSSS5n[ R "X/SS9nURU5nURU5nU$) aForward function. Args: x: Input feature, tensor size (B, H*W, C), i.e., binded [PATCH, DET] tokens H, W: Spatial resolution of the input feature. Returns: x: merged [PATCH, DET] tokens; only [PATCH] tokens are reduced in spatial dim, while [DET] tokens is fix-scale rNr+r-rr/r1r2) rMrrNrrr6r=repeatrr) rrrSrTrRrrUr pad_inputx0x1x2x3s rr PatchMerging.forwards''aED....N0NN.1fqufa<!AquvqL/3 FF1 UaZ0QUaZ a!Q1q5!QU34A q!$Q$1a  q!$Q$1a  q!$Q$1a  q!$Q$1a  IIrr& + FF1b!a% jjAq! IIqhA & IIaL NN1 r)r3rrrr r"r#r$r%r&rrr r r(r)r*s@rrrs"(*||D %%%rrc d^\rSrSrSrSSSSSSS\R SSS4 U4S jjrS S jrS r U=r $) BasicLayeriaA basic Swin Transformer layer for one stage. Args: dim (int): Number of feature channels depth (int): Depths of this stage. num_heads (int): Number of attention head. window_size (int): Local window size. Default: 7. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4. qkv_bias (bool, optional): If True, add a learnable bias to query, key, value. Default: True qk_scale (float | None, optional): Override default qk scale of head_dim ** -0.5 if set. drop (float, optional): Dropout rate. Default: 0.0 attn_drop (float, optional): Attention dropout rate. Default: 0.0 drop_path (float | tuple[float], optional): Stochastic depth rate. Default: 0.0 norm_layer (nn.Module, optional): Normalization layer. Default: nn.LayerNorm downsample (nn.Module | None, optional): Downsample layer at the end of the layer. Default: None use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False. rrTNr Fc>[TU]5 X@lUS-UlX lXlXl[R"[U5Vs/sH=n[UUUUS-S:XaSOUS-UUUUU [U [5(aXOU U S9 PM? sn5Ul U bU "XU (+S9UlgSUlgs snf)Nr+r) r3rerQrrrrrsrrmrr)r3rr)r r rQrdepthr3use_checkpointr ModuleListranger isinstancelistblocks downsample)rr3rrerQrrrrsrrmrrrlastrirs rr BasicLayer.__init__/s &%* ,mm16e % 1=1 !#'!"Q!1+2B#!!#i..$,4=% '1=%   "  !(DCDO#DO+% sAB<c URSn[[R"X R- 55UR-n[[R"X0R- 55UR-n [ R "SXS4URS9n [SUR*5[UR*UR*5[UR*S54n [SUR*5[UR*UR*5[UR*S54n Sn U HnU HnXSS2XSS24'U S- n M M [XR5nURSURUR-5nURS5URS5- nURUS:g[S55RUS:H[S55nU(GaURSSunnUU:XaUU:XdG[R "USR5X#4S 9R#[ R$5Sn['XUR(5nUR5nURUS:g[S55RUS:H[S55nURXrU-5RS5RS5n[R*"USUR,4SS 9nOSnSnUU4n[/UR05H`unnX#sUlUlU(aS nUnUnOS nSnSU4nUR6(a8"UUUUUUS 9nMVU"UUUUUS 9nMb UR:b)UR;XU5nUS-S-US-S-nnXUUUU4$XX1X#4$)a Forward function. Args: x: Input feature, tensor size (B, H*W, C). H, W: Spatial resolution of the input feature. det_pos: pos encoding for det token input_mask: padding mask for inputs cross_attn: whether to use cross attn [det x [det + patch]] rr-)r0Nr/r+gYr size)valueTF)rIrr)rMrYnpceilrQr6rgr0slicerrWrNr masked_fillfloatr interpolatetoboolrJr3rr enumeraterrSrTr checkpointr)rrrSrTr input_maskrrRrrimg_maskh_slicesw_slicescnthw mask_windowsr_H_WrrrIn_blkblk _cross_attn_cross_attn_mask_posx_downWhWws rr BasicLayer.forward^s GGAJ---. /$2B2B B ---. /$2B2B B;;21~ahh?!d.../4+++??*,-2DOO3CT-JL!d.../4+++??*,-2DOO3CT-JLA'*A!$q (2B2BC #(()-)9)9D!M 0txxHI)..0O-99/S:PRWX^R_` Os2E#J? .22q5#)A,yy| ee!T%7%7!8COI"O'"#DKK0JE3LCE35" #2 # #' g""))*$46*$4 6/1B ?? &__Q1-F!e\AEa<BFB* *q# #r)rrr3rrrrQ)Frr*s@rrrsA,LL  %-#^j$j$rrc>^\rSrSrSrSU4SjjrSrSrU=r$) PatchEmbedia,Image to Patch Embedding Args: patch_size (int): Patch token size. Default: 4. in_chans (int): Number of input image channels. Default: 3. embed_dim (int): Number of linear projection output channels. Default: 96. norm_layer (nn.Module, optional): Normalization layer. Default: None c>[TU]5 [U5nXlX lX0l[ R"X#XS9UlUbU"U5Ul gSUl g)N) kernel_sizestride) r r r patch_sizein_chans embed_dimrConv2drnr)rrrrrrs rr PatchEmbed.__init__sX z* $ "II ZL  !"9-DIDIrc UR5u p#nX@RS-S:wa8[R"USURSX@RS-- 45nX0RS-S:wa:[R"USSSURSX0RS-- 45nUR U5nUR bURS5URS5peUR S5RSS5nUR U5nURSS5RSURXV5nU$)zForward function.r-rr+r4r/) rrrrrnrr<r~rNr)rrrrSrTr r s rr PatchEmbed.forwards"VVX 1 q! !Q &a!T__Q/!ooa6H2HHIJA q! !Q &a!Q 2Q9K5K KLNA IIaL 99 VVAYq  ! &&q!,A ! A Aq!&&r4>>2BAr)rrrrrn)r1r4`Nrr*s@rrrs% &rrc^\rSrSrSrSSSS/SQ/SQS S S S S S S\R SS /SQSS4U4SjjrSr\ RRS5r SSS/4Sjr SrSU4SjjrSrSrU=r$)SwinTransformeriaUSwin Transformer backbone. A PyTorch impl of : `Swin Transformer: Hierarchical Vision Transformer using Shifted Windows` - https://arxiv.org/pdf/2103.14030 Args: pretrain_img_size (int): Input image size for training the pretrained model, used in absolute position embedding. Default 224. patch_size (int | tuple(int)): Patch size. Default: 4. in_chans (int): Number of input image channels. Default: 3. embed_dim (int): Number of linear projection output channels. Default: 96. depths (tuple[int]): Depths of each Swin Transformer stage. num_heads (tuple[int]): Number of attention head of each stage. window_size (int): Window size. Default: 7. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4. qkv_bias (bool): If True, add a learnable bias to query, key, value. Default: True qk_scale (float): Override default qk scale of head_dim ** -0.5 if set. drop_rate (float): Dropout rate. attn_drop_rate (float): Attention dropout rate. Default: 0. drop_path_rate (float): Stochastic depth rate. Default: 0.2. norm_layer (nn.Module): Normalization layer. Default: nn.LayerNorm. ape (bool): If True, add absolute position embedding to the patch embedding. Default: False. patch_norm (bool): If True, add normalization after patch embedding. Default: True. out_indices (Sequence[int]): Output from which stages. frozen_stages (int): Stages to be frozen (stop grad and set eval mode). -1 means not freezing any args. use_checkpoint (bool): Whether to use checkpointing to save memory. Default: False. r1r4r)r+r+r+)r4r rrTNr g?F)r-r+r4r/c$>[TU]5 Xl[U5UlX@lXlUUlUUlUUl [UUUUR(aUOSS9Ul UR (ax[U5n[U5nUSUS-USUS-/n[R"[R "SUUSUS55Ul[%UR"SS9 [R&"U S9Ul[R*"SU [-U55Vs/sHnUR/5PM nn[R0"5Ul[5UR5Hn[7[9USU--5UUUUUUU U U U U[-USU5[-USUS-5UUUR:a[:OSUURS- :aSOSUS 9nUR2R=U5 M [5UR5Vs/sHn[9USU--5PM nnUUlUH%nU"UU5nS U3nURAUU5 M' URC5 gs snfs snf) N)rrrrrr-rbrc)pr+T)r3rrerQrrrrsrrmrrrrrr)"r r pretrain_img_sizelen num_layersrape patch_norm out_indices frozen_stagesr patch_embedrrrfr6rgabsolute_pos_embedrrpos_droplinspacerjitemrlayersrrrYrappend num_features add_module_freeze_stages)rr$rrrdepthsrerQrrrrs drop_rateattn_drop_ratedrop_path_raterr'r(r)r*rpatches_resolutionrdpri_layerlayerrr2 layer_namers rr SwinTransformer.__init__s, !2f+"$&*&!%)__z$ @ 88 )*; < ":.J!!$ 1 5!!$ 1 5"  ')ll Ay* ?A&t~~3?  A}> %m= "XX// > 3t001A56 6   a!e $6  V     $ $  *DKK 03t7I7I3J JK[[E"/ +1>  .&3#')yy))'D$& ! ^ #)-DKKO & $+ sG)c  URSURSURSn nURU5nURS5URS5peURS5R SS5nUR U5nUR RUSS5nURn[R"USR5XPR-X`R-4S9R[R5Sn/n [!UR"5Hn UR$U n XR&;aSOS n [R("X/SS 9nU "UUUUUU S 9upppVUSS2SUR**2SS24USS2UR**S2SS24pqU S:dMU SS2SUR**2SS24R-X>US5R/SSSS5nU R1U5 M U R1UR-X5US5R/SSSS55 W SS2UR**S2SS24R/SSS5nUR/SSS5nU unnnnUUUUUU4$) aForward function. Args: x: input rgb images mask: input padding masks [0: rgb values, 1: padded values] Returns: patch_outs: multi-scale [PATCH] tokens (four scales are used) these tokens are the first input of the neck decoder det_tgt: final [DET] tokens obtained at the last stage this tokens are the second input of the neck decoder det_pos: the learnable pos encoding for [DET] tokens. these encodings are used to generate reference points in deformable attention rr+r4r-r/NrTFr2)rrr)rMr+rr<r~r-rHrrGrrrrTrr6rrr&r0rSr=rrNrOr1)rrr>rRrr r rHr patch_outsstager<rx_outrSrT patch_outdet_tgt features_0 features_1 features_2 features_3s rr SwinTransformer.forwards"''!*aggaj!''!*a1   Q AFF1IB IIaL " "1a ( MM! NN))!R4 $$}} J   ))))))+,-/BuzzN1>  4??+EKK&E"'*<*[[U] U5 UR5 g)z?Convert the model into training mode while keep layers freezed.N)r rtrainr4)rmoders rrdSwinTransformer.trains ot*40 rcTSnXRR5- n[UR5Hup#XR5- nM XRUR S-UR S-SUR --- nXRUR-- nU$)Nrr-r+)r+flopsrr0r2r9r& num_classes)rrhrr<s rrhSwinTransformer.flopss !!''))!$++.HA [[] "E/ ""T%<%< &((+,014??0BD D ""T%5%555 r)r,r'rSrGrHrrr*r0rTrPrRr2r&r)r+r(rQr-r$)T)r"r#r$r%r&rrr r4r6jitignorerJrVr rdrhr(r)r*s@rrrs<"$|| +^@0" YY.. $' $%3 7.rMP^ rr)mathosnumpyrr6torch.nnrtorch.nn.functional functionalrtorch.utils.checkpointutilsrtimm.models.layersrrrModulerpirJrWrZr\rrrrrrIrrrxs   ++AA")):)."#dgg+ *Z" $H"))HVV299Vr;299;|l$l$^//dhbiihr