9i T0SSKrSSKrSSKJrJrJr SSKrSSKJr SSKJ r SSK J r J r SSK Jr SSKJr SSKJrJrJr SS KJr SS KJr S S KJrJr S S KJr \"5rSr\R@"\RB\RDS9"SS\55r#g)N)OptionalTupleUnion)nn)CrossEntropyLoss)assert_device_mapget_device_map)Models)MODELS)AttentionBackboneModelOutputSeq2SeqLMOutputTokenGeneratorOutput)Tasks) get_logger)T5PreTrainedModelT5Stack)T5Configa_ The input argument `head_mask` was split into two arguments `head_mask` and `decoder_head_mask`. Currently, `decoder_head_mask` is set to copy `head_mask`, but this feature is deprecated and will be removed in future versions. If you do not want to use any `decoder_head_mask` now, please set `decoder_head_mask = torch.ones(num_layers, num_heads)`. ) group_key module_namec%^\rSrSr/SQrS/rS%S\4U4SjjjrS%SjrSr Sr S r S r S r S rS rS&S\\R$S\\R&S\\R$S\\R(S\\R&S\\R&S\\R*S\\\\R*S\\\\R*S\\R&S\\R&S\\R$S\\S\\S\\S\\S\\\R&\44"SjjrS'S jrS\R*4S!jrU4S"jrS#rS$rU=r $)(T5ForConditionalGeneration.)zencoder\.embed_tokens\.weightzdecoder\.embed_tokens\.weightzlm_head\.weightzMdecoder\.block\.0\.layer\.1\.EncDecAttention\.relative_attention_bias\.weightconfigc >[TU]U5 URUl[R "UR UR5Ul[R"U5nSUl SUl SUl [X@R5Ul[R"U5nSUl SUl URUl[XPR5Ul[R$"URUR SS9UlUR)5 SUlUS:XaUR-5 gg)NFT)biasauto)super__init__d_model model_dimr Embedding vocab_sizesharedcopydeepcopy is_decoder use_cacheis_encoder_decoderrencodernum_decoder_layers num_layersdecoderLinearlm_head post_initmodel_parallel parallelize)selfr device_mapkwargsencoder_configdecoder_config __class__s m/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/models/nlp/T5/text2text_generation.pyr#T5ForConditionalGeneration.__init__<s  ll6#4#4fnnE v.$)!#( ,1)~{{; v.$(!,1)$*$=$=!~{{; yy1B1BO  $       c$UcN[[URR5[ [ R R555OUUl[UR[URR55 URRUR5 URRUR5 URRURR5Ul SUlg)NT)r lenr*blockrangetorchcuda device_countr4rr2r-r/to first_devicer1)r3r4s r9r2&T5ForConditionalGeneration.parallelizeXs! DLL&&'uzz/F/F/H)I K'1  $//3t||/A/A+BC   1   1||t||'@'@A "r;cURR5 URR5 URRS5UlURRS5UlURRS5UlSUlSUl[RR5 g)NcpuF) r* deparallelizer-rCr/r1r4r@rA empty_cacher3s r9rH(T5ForConditionalGeneration.deparallelizecs} ""$ ""$||u- ||u- ||u- #  r;cUR$N)r$rJs r9get_input_embeddings/T5ForConditionalGeneration.get_input_embeddingsms {{r;c|XlURRU5 URRU5 grM)r$r*set_input_embeddingsr-r3new_embeddingss r9rQ/T5ForConditionalGeneration.set_input_embeddingsps+$  )).9 )).9r;cXlgrMr/rRs r9set_output_embeddings0T5ForConditionalGeneration.set_output_embeddingsus% r;cUR$rMrVrJs r9get_output_embeddings0T5ForConditionalGeneration.get_output_embeddingsx ||r;cUR$rM)r*rJs r9 get_encoder&T5ForConditionalGeneration.get_encoder{r\r;cUR$rM)r-rJs r9 get_decoder&T5ForConditionalGeneration.get_decoder~r\r; input_idsattention_maskdecoder_input_idsdecoder_attention_mask head_maskdecoder_head_maskcross_attn_head_maskencoder_outputspast_key_values inputs_embedsdecoder_inputs_embedslabelsr(output_attentionsoutput_hidden_states return_dictreturnc U bU OURRn UbUOURRnUbRUcOURRURR:Xa![ R "[[5 UnUcURUUU UUUUS9nORU(aK[U[5(d6[US[U5S:aUSOS[U5S:aUSOSS9nUSnUR(a3[RR!UR"R$5 U bUcU cUR'U 5nUR(a[RR!UR"R$5 UR)UR"R$5nUb%UR)UR"R$5nUb%UR)UR"R$5nUb%UR)UR"R$5nUR#UUU U UUUUU UUUS9 nUSnUR(a[RR!URR$5 UR*R)URR$5UlUR)UR*R,R.5nURR0(aUUR2S--nUR+U5nSnU b@[5S S 9nU"UR7S UR9S 55U R7S 55nU(dU4USS-U-nUbU4U-$U$[;UUUR<UR>UR@URBURDUR>UR@S 9 $) a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. T5 is a model with relative position embeddings so you should be able to pad the inputs on both the right and the left. Indices can be obtained using [`T5Tokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for detail. [What are input IDs?](../glossary#input-ids) To know more on how to prepare `input_ids` for pretraining take a look a [T5 Training](./t5#training). attention_mask (`torch.FloatTensor` of shape `(batch_size,sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`T5Tokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) T5 uses the `pad_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). To know more on how to prepare `decoder_input_ids` for pretraining take a look at [T5 Training](./t5#training). decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the encoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. decoder_head_mask (`torch.FloatTensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the self-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(num_heads,)` or `(num_layers, num_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. encoder_outputs (`tuple(tuple(torch.FloatTensor)`, *optional*): Tuple consists of (`last_hidden_state`, `optional`: *hidden_states*, `optional`: *attentions*) `last_hidden_state` of shape `(batch_size, sequence_length, hidden_size)` is a sequence of hidden states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. past_key_values (`tuple(tuple(torch.FloatTensor))` of length `config.n_layers` with each tuple having 4 tensors of shape `(batch_size, num_heads, sequence_length - 1, embed_size_per_head)`): Contains precomputed key and value hidden states of the attention blocks. Can be used to speed up decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. decoder_inputs_embeds (`torch.FloatTensor` of shape `(batch_size, target_sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `decoder_input_ids` you can choose to directly pass an embedded representation. If `past_key_values` is used, optionally only the last `decoder_inputs_embeds` have to be input (see `past_key_values`). This is useful if you want more control over how to convert `decoder_input_ids` indices into associated vectors than the model's internal embedding lookup matrix. If `decoder_input_ids` and `decoder_inputs_embeds` are both unset, `decoder_inputs_embeds` takes the value of `inputs_embeds`. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size]` Returns: Examples: >>> from transformers import T5Tokenizer, T5ForConditionalGeneration >>> tokenizer = T5Tokenizer.from_pretrained("t5-small") >>> model = T5ForConditionalGeneration.from_pretrained("t5-small") >>> # training >>> input_ids = tokenizer("The walks in park", return_tensors="pt").input_ids >>> labels = tokenizer(" cute dog the ", return_tensors="pt").input_ids >>> outputs = model(input_ids=input_ids, labels=labels) >>> loss = outputs.loss >>> logits = outputs.logits >>> # inference >>> input_ids = tokenizer( ... "summarize: studies have shown that owning a dog is good for you", return_tensors="pt" >>> ).input_ids # Batch size 1 >>> outputs = model.generate(input_ids) >>> print(tokenizer.decode(outputs[0], skip_special_tokens=True)) >>> # studies have shown that owning a dog is good for you. N)rcrdrlrgrorprqrr)last_hidden_state hidden_states attentions) rcrdrlrkencoder_hidden_statesencoder_attention_maskrgrir(rorprqgi) ignore_index) losslogitsrkdecoder_hidden_statesdecoder_attentionscross_attentionsencoder_last_hidden_staterxencoder_attentions)#rr(use_return_dictr,r+warningswarn2_T5ForConditionalGeneration__HEAD_MASK_WARNING_MSG FutureWarningr* isinstancer r=r1r@rA set_devicer-rD _shift_rightrCr/weightdevicetie_word_embeddingsr!rviewsizer rkrvrwrru)r3rcrdrerfrgrhrirjrkrlrmrnr(rorprqr5rvdecoder_outputssequence_output lm_logitsr|loss_fctoutputs r9forward"T5ForConditionalGeneration.forwards@"+!6IDKK{{%%)G)GG 5}E$-!  ""ll#-+#"3%9'+OO,H"J"J:"1!"4'!+.a015'!++1-15 O(*    JJ ! !$,,";"; <  "3";@U@] $ 1 1& 9     JJ ! !$,,";"; <),,T\\-F-FGM ,$5$8$8LL--%/!)!/!2!24<<3L3L!M%1)?)B)BLL--*/&,,'1/+"/#1'!5/!5#'  *!,    JJ ! !$,,";"; <<>"#56 BID ]_QR%88?JF*.*:THv% F F+;;"1"?"?.99,==&5&G&G"1"?"?.99  r;c 4Ub USS2SS24nUUUUUUUUS.$)Nr{)rerkrjrdrgrhrir() r3rcpastrdrgrhrir(rjr5s r9prepare_inputs_for_generation8T5ForConditionalGeneration.prepare_inputs_for_generations>  !!RS&)I"+#.,"!2$8"  r;c$URU5$rM)r)r3rns r9%prepare_decoder_input_ids_from_labels@T5ForConditionalGeneration.prepare_decoder_input_ids_from_labelss  ((r;c>[TU]"U0UD6n[[U[R 5(aUS9$USS9$)Nr) sequences)rgeneraterrr@Tensor)r3argsr5rr8s r9r#T5ForConditionalGeneration.generatesJ !4262# *65<< @ @f  FLQi  r;c HUc[RS5 U$SnUHnSnUH2nUURSURUR554-nM4 USR USR :Xde[ U5[ U5:XdeUU4-nM U$)NzHYou might want to consider setting `use_cache=True` to speed up decodingrr)loggerwarning index_selectrCrshaper=)r3rbeam_idxreordered_decoder_pastlayer_past_statesreordered_layer_past_stateslayer_past_states r9_reorder_cache)T5ForConditionalGeneration._reorder_caches < NNZ K!#!% +- '$5 .I$118;;'7'>'>?AMD/D+%6 /q177;L<5 23s;L7MM MM%;+?/&/ ""& &%r;)r-r4r*r/r!r1r$rM)NNNNNNNNNNNNNNNN)NNNNNNN)!__name__ __module__ __qualname____firstlineno___keys_to_ignore_on_load_missing"_keys_to_ignore_on_load_unexpectedrrr2rHrNrQrWrZr^rarr@ LongTensor FloatTensor BoolTensorrrboolrr rrrrr__static_attributes__ __classcell__)r8s@r9rr.s4 '# Y*&x8 #!: &9=>B@DEI9=AE?CHLHL=AEI59,0487;.2!J #E$4$45J ():): ;J $,E,<,<#=J )11A1A(B J $E$5$56 J $,E,=,=#> J '/u||&<J "*%ell0C*D!EJ "*%ell0C*D!EJ  ((9(9:J (00A0A'BJ !!1!12J $D>J $,D>J '/tnJ &d^!J "#5):):#;_#LM#J \,059048<;?046: 4)ELL) &&r;r)$r%rtypingrrrr@rtorch.nnr'transformers.utils.model_parallel_utilsrr modelscope.metainfor modelscope.models.builderr modelscope.outputsr r rmodelscope.utils.constantrmodelscope.utils.loggerrbackbonerr configurationrr__HEAD_MASK_WARNING_MSGregister_moduletext2text_generationT5rrr;r9rs )) %E',66+.0# (( \&!2\& \&r;