9iJHSrSSKrSSKrSSKJrJrJr SSKrSSKJ r SSK J s J r SSK Js Jr SSKJr SSKJrJr SSKJr S&SjrS\S \S \S \4S jr"S S\ R65r"SS\ R65r"SS\ R65r"SS\RR65r"SS\ R65r "SS\ R65r!"SS\ R65r""SS\ R65r#Sr$S'Sjr%"SS \ R65r&"S!S"\RR65r'\RP"\RR\RTS#9"S$S%\55r+g)(a*This ECAPA-TDNN implementation is adapted from https://github.com/speechbrain/speechbrain. Self-Distillation Prototypes Network(SDPN) is a self-supervised learning framework in SV. It comprises a teacher and a student network with identical architecture but different parameters. Teacher/student network consists of three main modules: the encoder for extracting speaker embeddings, multi-layer perceptron for feature transformation, and prototypes for computing soft-distributions between global and local views. EMA denotes Exponential Moving Average. N)AnyDictUnion)Models)MODELS TorchModel)Tasksc[UR5S:XdeUc,UR5R5R 5n[ R "XRURS9R[U5U5URS5:nUc URnUc URn[ R"XBUS9nU$)N)devicedtype)r r ) lenshapemaxlongitemtorcharanger r expand unsqueeze as_tensor)lengthmax_lenr r masks _/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/models/audio/sv/sdpn.pylength_to_maskrs v||  !! !**,##%**, << V\\ ;;A6 K<"$*$4$4Q$7 8D }  ~ ??4V [T U]5 X lX@lXPlX`lXl[R"UUURURURSUUS9Ul g)Nr)rr!r)groupsbias) super__init__r rr!r) padding_modennr,conv) self out_channelsr in_channelsrr!r)r/r0r3 __class__s rr2Conv1d.__init__:sa &   (II     ;;]]   rcURS:Xa2URXRURUR5nOnURS:Xa5URS- UR-n[ R "XS45nO)URS:XaO[SUR-5eURU5nU$)Nsamecausalr rvalidz1Padding must be 'same', 'valid' or 'causal'. Got ) r)_manage_paddingr r!rFpad ValueErrorr5)r6xnum_padwxs rforwardConv1d.forwardXs <<6 !$$Q(8(8$--%)[[2A\\X %''!+t}}[TU]5 [R"UUUS9Ulg)N)epsmomentum)r1r2r4rVnorm)r6 input_sizerYrZr9s rr2BatchNorm1d.__init__}s) NN   rc$URU5$Nr[r6rCs rrFBatchNorm1d.forwardsyy|rr`)gh㈵>g?rMrNrOrPr2rFrRrSrTs@rrVrV{s   rrVcJ^\rSrSr\R S4U4SjjrSrSrU=r $) TDNNBlockr c>[[U] 5 [UUUUUS9UlU"5Ul[ US9Ulg)N)r8r7r r!r/r\)r1rer2r,r5 activationrVr[)r6r8r7r r!rir/r9s rr2TDNNBlock.__init__sD i')#%#   %,<8 rc`URURURU555$r_)r[rir5ras rrFTDNNBlock.forwards"yy1677r)rir5r[) rMrNrOrPr4ReLUr2rFrRrSrTs@rreres779(88rrec8^\rSrSrSU4SjjrSrSrU=r$) Res2NetBlockc >[[U] 5 X-S:XdeX#-S:XdeX-nX#-n[R"[ US- 5Vs/sHn[ UUUUS9PM sn5UlX0lgs snf)Nrr )r r!) r1ror2r4 ModuleListrangereblocksscale) r6r8r7rur r! in_channelhidden_channelir9s rr2Res2NetBlock.__init__s lD*,"a'''#q((( ) %.mm UQY' % (! '!   ( %    % sA9c>/n[[R"XRSS95HYup4US:XaUnO:US:XaURUS- "U5nOURUS- "UW-5nUR U5 M[ [R "USS9nU$)Nr dimr) enumeraterchunkrurtappendcat)r6rCyrxx_iy_is rrFRes2NetBlock.forwards  Azzq ABFAAvakk!a%(-kk!a%(s3 HHSMC IIaQ r)rtru)r rcrTs@rroros  .  rroc2^\rSrSrU4SjrSSjrSrU=r$)SEBlockc>[[U] 5 [XSS9Ul[ R RSS9Ul[X#SS9Ul [ R R5Ul g)Nr r8r7r T)inplace) r1rr2r,conv1rr4rmreluconv2Sigmoidsigmoid)r6r8 se_channelsr7r9s rr2SEBlock.__init__s^ gt%'#1N HHMM$M/ #AO xx'') rchURSnUbM[X#-X1RS9nURS5nUR SSS9nX-R SSS9U- nOUR SSS9nUR URU55nURURU55nXa-$)NrIrr r r$Tr|keepdim) rrr rsummeanrrrr)r6rClengthsLrtotalss rrFSEBlock.forwards GGBK  !'+qJD>>!$DHHDH1E1d3e;A1d+A IIdjjm $ LLA 'u r)rrrrr_rcrTs@rrrs*  rrc6^\rSrSrSU4SjjrSSjrSrU=r$)AttentiveStatisticsPoolingc>[TU]5 SUlX0lU(a[ US-USS5UlO[ XSS5Ul[ R"5Ul[UUSS9Ul g)Ng-q=rr r) r1r2rYglobal_contextretdnnr4Tanhtanhr,r5)r6channelsattention_channelsrr9s rr2#AttentiveStatisticsPooling.__init__sg , !(Q,0BAqIDI!(1EDIGGI *! rcLURSnSUR4SjnUc,[R"URSURS9n[ X#-X1RS9nUR S5nUR(aURSSS 9R5nU"XU- 5upxUR S5RSSU5nUR S5RSSU5n[R"XU/SS 9n OUn URURURU 555n U RUS:H[S 55n [ R""U SS 9n U"X5upx[R"Xx4SS 9n U R S5n U $) NrIr$cX-RU5n[R"XURU5- R S5-RU5R U55nXE4$)Nr$)rrsqrtrpowclamp)rCmr|rYrstds r_compute_statistics?AttentiveStatisticsPooling.forward.._compute_statisticss^E;;s#D**$..--22155::3?EEcJLC9 rr)r rr Trr{z-inf)rrYronesr rrrrfloatrepeatrr5rr masked_fillr@softmax) r6rCrrrrrrrattn pooled_statss rrF"AttentiveStatisticsPooling.forwardsn GGBK*+  ?jjAHH=Ggk1XXF~~a    HHDH1779E+Ae|>!$++Aq!4D--"))!Q2C99as^3DDyy499T?34 5=9yy1%'0 yy$!4 #--a0 r)r5rYrrr)Tr_rcrTs@rrrs ))rrcj^\rSrSrSSSS\R R S4U4SjjrSSjrSr U=r $) SERes2NetBlocki(rrr c >[T U]5 X l[UUSSUUS9Ul[ X"X5U5Ul[UUSSUUS9Ul[X$U5Ul SUl X:wa[UUSS9Ul gg)Nr )r r!rir/r) r1r2r7retdnn1ro res2net_blocktdnn2rse_blockshortcutr,) r6r8r7 res2net_scalerr r!rir/r9s rr2SERes2NetBlock.__init__*s (  !   *,*7hP  !    <H   &"')DM 'rcUnUR(aURU5nURU5nURU5nURU5nUR X5nX-$r_)rrrrr)r6rCrresiduals rrFSERes2NetBlock.forwardSs\ ==}}Q'H JJqM   q ! JJqM MM! %|r)r7rrrrrr_) rMrNrOrPrr4rmr2rFrRrSrTs@rrr(s/ 88=='R  rrc ^\rSrSrSrSS\R R/SQ/SQ/SQSS SS /S Q4 U4S jjrSS jr Sr U=r $) ECAPA_TDNNi`zAn implementation of the speaker embedding model in a paper. "ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification" (https://arxiv.org/abs/2005.07143). cpu)rrrri)rrrr )r r$rr rrT)r r r r r c >[TU]5 [U5[U5:Xde[U5[U5:XdeXPl[R "5UlUR R[UUSUSUSUU S55 [S[U5S- 5H9n UR R[X]S- X]U U XmX}UXS95 M; [USUSUSUSUU SS9Ul [USUU S9Ul [USS-S9Ul[!USS-USS 9Ulg) Nrr )rrr r!rir/rI)r/)rrr$rhr)r1r2rrr4rrrtrrersrmfarasprVasp_bnr,fc)r6r\r lin_neuronsrir kernel_sizes dilationsrrrrr/rxr9s rr2ECAPA_TDNN.__init__fsp 8}L 11118}I... mmo    Q! q   q#h-!+,A KK  UOK"/ + ,&\)!9  - RL RL   bM ":  . RL1)  "Xb\A-=>   q($ rcURSS5n/nURHnU"XS9nURU5 M [R "USSSS9nUR U5nURXS9nURU5nURU5nURSS5RS5nU$![a U"U5nNf=f)zpReturns the embedding vector. Arguments --------- x : torch.Tensor Tensor of shape (batch, time, channel). r r$)rNr{) transposert TypeErrorrrrrrrrsqueeze)r6rCrxllayers rrFECAPA_TDNN.forwards KK1  [[E !- IIaL ! IIbf! $ HHQK HHQH ( KKN GGAJ KK1  % %a (! !H sB==CC)rrrtrrrr_) rMrNrOrP__doc__rr4rmr2rFrRrSrTs@rrr`sD88==+$!F PrrcSnXSU-- :d XSU--:a[RSSS9 [R"5 U"X1- U- 5nU"XA- U- 5nUR SU-S- SU-S- 5 UR 5 UR U[R"S5-5 URU5 URX4S9 UsSSS5 $!,(df  g=f)NchS[R"U[R"S5- 5-S- $)N?@)r%erfr)rCs rnorm_cdf(_no_grad_trunc_normal_..norm_cdfs(TXXa$))B-/00B66rr$zimean is more than 2 std from [a, b] in nn.init.trunc_normal_.The distribution of values may be incorrect.) stacklevelr r)minr) warningswarnrno_graduniform_erfinv_mul_r%radd_clamp_)tensorrrabrl_us r_no_grad_trunc_normal_rs7 1s7{1s7{ 2  ;   qx3& ' ah#% & B AEAI.   C$))B-'( D  ! #+ s BC C-c[XX#U5$r_)r)rrrrrs r trunc_normal_rs !& ::rc@^\rSrSrSU4SjjrSrSrSrU=r$)SDPNHeadic>[TU]5 [US5nUS:Xa[R"X5UlGO.[R"X5/nU(a%UR [R"U55 UR [R"55 [US- 5HxnUR [R"XD55 U(a%UR [R"U55 UR [R"55 Mz UR [R"XE55 [R"U6UlURUR5 gr#) r1r2rr4LinearmlprrVGELUrs Sequentialapply _init_weights) r6in_dimuse_bnnlayers hidden_dimbottleneck_dimlayers_r9s rr2SDPNHead.__init__s gq/ a<yy8DHii34F bnnZ89 MM"'') $7Q;' bii ?@MM".."<= bggi( ( MM"))J? @}}f-DH 4%%&rc[U[R5(am[URSS9 [U[R5(a9UR b+[R RUR S5 gggg)Ng{Gz?)rr) isinstancer4rrweightr0init constant_)r6rs rrSDPNHead._init_weightss` a # # !(( ,!RYY''AFF,>!!!&&!,-?' $rcfURU5n[RRUSSS9nU$)NrIr$)r|p)rr4 functional normalizeras rrFSDPNHead.forwards/ HHQK MM # #A2 # 3r)r)Fri) rMrNrOrPr2rrFrRrSrTs@rrrs% # '0- rrc2^\rSrSrSrU4SjrSrSrU=r$)Combineriz) Combine backbone (ECAPA) and head (MLP) cB>[[U] 5 XlX lgr_)r1rr2backbonehead)r6rrr9s rr2Combiner.__init__#s h&(  rcLURU5nURU5nX4$r_rr)r6rCoutputs rrFCombiner.forward(s% MM! 1yrr) rMrNrOrPrr2rFrRrSrTs@rrrs rr) module_namecT^\rSrSrSrS\\\44U4SjjrSr Sr S Sjr Sr U=r $) SpeakerVerificationSDPNi.z Self-Distillation Prototypes Network (SDPN) effectively facilitates self-supervised speaker representation learning. The specific structure can be referred to in https://arxiv.org/pdf/2308.02774. model_configct>[TU]"X/UQ70UD6 X lX@lURSS:wa [ S5eSUl/SQn[ UR US9Ul[UR[SS55UlUS nURU5 URR5 g) NchannelzFmodelscope error: Currently only 1024-channel ecapa tdnn is supported.P)r&r&r&r&i )rrTpretrained_model) r1r2r# other_configrB feature_dimrembedding_modelrr*_SpeakerVerificationSDPN__load_check_pointeval)r6 model_dirr#argskwargschannels_configpretrained_model_namer9s rr2 SpeakerVerificationSDPN.__init__6s B4B6B("   Y '4 /X 8)    8'(<(<(0d(; =!''9 :  56 !!#rc[UR5S:XaURSS:XdS5eURU5nURR U5nU$)Nr$rr zFmodelscope error: the shape of input audio to model needs to be [1, T])rr)_SpeakerVerificationSDPN__extract_featurer+r)r6audiofeature embeddings rrFSpeakerVerificationSDPN.forwardMsk5;;1$ ** ^] ^((/((11': rc[R"XRS9nX"RSSS9- nUR S5nU$)N) num_mel_binsrTr)Kaldifbankr*rr)r6r6r7s r__extract_feature)SpeakerVerificationSDPN.__extract_featureVs@++e2B2BCLLQL==##A&rchU(d[R"S5n[R"[RR UR U5US9nUSR5VVs0sHupEURSS5U_M nnnURRUSS9 gs snnf)Nr) map_locationteacherzmodule.T)strict) rr loadospathjoinr.itemsreplacer+load_state_dict)r6r2r state_dictkvstate_dict_teas r__load_check_point*SpeakerVerificationSDPN.__load_check_point\s\\%(FZZ GGLL)> ?! #9-335 5 IIi $a '5   ,,^D,I  s1B.)r+r*r#r)r_)rMrNrOrPrrstrrr2rFr5r,rRrSrTs@rr"r".s2 $S#X$. J Jrr")NNN)grgr),rr%rFtypingrrrrtorch.nnr4torch.nn.functionalrr@torchaudio.compliance.kaldi compliancekaldir<modelscope.metainformodelscope.modelsrrmodelscope.utils.constantr rrQr*Moduler,rVrerorrrrrrrrregister_modulespeaker_verificationsdpn_svr"rrrasL ## ++&0+& 3  #  @RYY@F"))&8 84$588??$Nbii8::z5RYY5pll^!H; #ryy#L uxx  22O7Jj7JP7Jr