9iXSSKrSSKJr SSKJr \"5rSSjrSSjrSSjrg) N)nn) get_loggerc URSn[R"SS9n[R"U"[R "SS9"U5[R "SS9"U55SS9[R"U"[R "SS9"U5[R "SS9"U55SS9-nUb4[R"XR-5[R"U5- U- nU$[R"U5U- nU$)a Calculate two logits' the KL div value symmetrically. :param logits1: The first logit. :param logits2: The second logit. :param attention_mask: An optional attention_mask which is used to mask some element out. This is usually useful in token_classification tasks. If the shape of logits is [N1, N2, ... Nn, D], the shape of attention_mask should be [N1, N2, ... Nn] :return: The mean loss. none) reduction)dim)shaper KLDivLosstorchsum LogSoftmaxSoftmaxmean)logits1logits2attention_mask labels_numKLDivlosss j/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/models/nlp/structbert/adv_utils.py_symmetric_kl_divrsr"J LL6 *E 99 bmm#G,jjR ) +  )) "--B'0**$W- / D!yy  !#%*YY~%>?AKL Kzz$*, Kc UbUOSU-nXRRUR55RSU5-nUR S5 SU;aUR S5 SU;aSOUSn USn U (dSn SU;aUR S5 U"S0UDSU0D6n U R n [ X,U 5n [RRX5SRnURSS [S 5S 9RS S S 9Sn[R"[R"U55nU(a[R!S5 U$XS-- nXU--n[R"X- U5n[R""X-U5nU"S0UDSU0D6n U R n[ UUU 5nUU-$)a Calculate the adv loss of the model. :param embedding: Original sentence embedding :param model: The model, or the forward function(including decoder/classifier), accept kwargs as input, output logits :param ori_logits: The original logits outputted from the model function :param ori_loss: The original loss :param adv_grad_factor: This factor will be multiplied by the KL loss grad and then the result will be added to the original embedding. More details please check:https://arxiv.org/abs/1908.04577 The range of this value always be 1e-3~1e-7 :param adv_bound: adv_bound is used to cut the top and the bottom bound of the produced embedding. If not proveded, 2 * sigma will be used as the adv_bound factor :param sigma: The std factor used to produce a 0 mean normal distribution. If adv_bound not proveded, 2 * sigma will be used as the adv_bound factor :param kwargs: the input param used in model function :return: The original loss adds the adv loss Nr input_ids inputs_embedswith_attention_maskFrTinfr keepdimpr!z'Nan occurred when calculating adv loss.gư>datanewsizenormal_poplogitsrr autogradgradnormfloatmaxanyisnanloggerwarningmin) embeddingmodel ori_logitsori_lossadv_grad_factor adv_boundsigmakwargs embedding_1rroutputs v1_logitsremb_grad emb_grad_normis_nan embedding_2 adv_logitsadv_losss rcompute_adv_lossrH3s4'2 E Inn001ABJJ 5K JJ{&  ?##8#F%FM,-N & ()8f8K8GI ZN CD~~""45a8==HMM tuU|"--0S t.1. MYYu{{=1 2F @A4/0H( ::K))K3[AK))K3[AK8f8K8GJ ZHH h rc tUbUOSU-nXRRUR55RSU5-n UR S5 SU;aUR S5 U"S 0UDSU 0D6n U R up[ UU 5[ X<5-n U S- n [RRX5SRnURSS[S5S9RSSS 9Sn[R"[R"U55nU(a[R!S 5 U$X- nXU--n[R"X- U5n[R""X-U5nU"S 0UDSU0D6n U R unn[ UU5[ UU5-nUU-$) ax Calculate the adv loss of the model. This function is used in the pair logits scenario. :param embedding: Original sentence embedding :param model: The model, or the forward function(including decoder/classifier), accept kwargs as input, output logits :param start_logits: The original start logits outputted from the model function :param end_logits: The original end logits outputted from the model function :param ori_loss: The original loss :param adv_grad_factor: This factor will be multiplied by the KL loss grad and then the result will be added to the original embedding. More details please check:https://arxiv.org/abs/1908.04577 The range of this value always be 1e-3~1e-7 :param adv_bound: adv_bound is used to cut the top and the bottom bound of the produced embedding. If not proveded, 2 * sigma will be used as the adv_bound factor :param sigma: The std factor used to produce a 0 mean normal distribution. If adv_bound not proveded, 2 * sigma will be used as the adv_bound factor :param kwargs: the input param used in model function :return: The original loss adds the adv loss rrrrTrr r#r$z,Nan occurred when calculating pair adv loss.r%r&)r7r8 start_logits end_logitsr:r;r<r=r>r?r@v1_logits_start v1_logits_endrrBrCrDrEadv_logits_startadv_logits_endrGs rcompute_adv_loss_pairrPos8'2 E Inn001ABJJ 5K JJ{&  ?#8f8K8G%,^^"O \, .0A!+1< )r rmodelscope.utils.loggerrr4rrHrPr%rrrRs8  . @ $ 9D%) $9r