9i.zSSKrSSKrSSKJr SSKJr \R"5r"SS\5r"SS\5r g)N)PretrainedConfig)loggerc^\rSrSrSrS U4Sjjr\S5r\S5rSr Sr Sr S r S r U=r$) PlugNLUConfigplugNLUc \>[TU]"SSU0UD6 XlX lX0lX@lXPlXplX`lXl Xl Xl Xl Xl XlXlXlUUlUUlUUlUUlUUlUUlUUlUUlUUlgNlayer_norm_eps)super__init__ vocab_sizeoriginal_vocab_size hidden_sizenum_hidden_layersnum_attention_heads hidden_actintermediate_sizehidden_dropout_probattention_probs_dropout_probmax_position_embeddingstype_vocab_sizeinitializer_rangelr_decay_style weight_decay clip_gradwarmuppre_lnfp16fp32_layernormfp32_embeddinglayernorm_epsilonfp32_tokentypesdec_hidden_layers attn_separate)selfrrrrrrrrrrrrrrrrrr r!r"r$r#r%r&kwargs __class__s h/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/models/nlp/plug/configuration.pyrPlugNLUConfig.__init__s4 D(9DVD$#6 &!2#6 $!2#6 ,H)'>$.!2,("   ,,!2.!2*ch[5nUR5Hup4XBRU'M U$)zAConstructs a `BertConfig` from a Python dictionary of parameters.)ritems__dict__)cls json_objectconfigkeyvalues r* from_dictPlugNLUConfig.from_dictSs1%++-JC#(OOC . r,c[USSS9nUR5nSSS5 UR[R"W55$!,(df  N3=f)z9Constructs a `BertConfig` from a json file of parameters.rzutf-8)encodingN)openreadr5jsonloads)r0 json_filereadertexts r*from_json_filePlugNLUConfig.from_json_file[sC)S7 3v;;=D4}}TZZ-..4 3s A  AcURR5nURR5Hup4X2;aM X@RU'M U$)z;merge values a `BertConfig` from a json file of parameters.)r/keysr.)r'args local_keysr3r4s r* merge_argsPlugNLUConfig.merge_argsbsI]]'') ----/JC !&MM# 0 r,c4[UR55$)N)strto_json_stringr's r*__repr__PlugNLUConfig.__repr__ks4&&())r,cF[R"UR5nU$)z0Serializes this instance to a Python dictionary.)copydeepcopyr/)r'outputs r*to_dictPlugNLUConfig.to_dictnst}}- r,cP[R"UR5SSS9S-$)z*Serializes this instance to a JSON string.T)indent sort_keys )r<dumpsrSrLs r*rKPlugNLUConfig.to_json_stringss zz$,,.dCdJJr,)rr&rr%r r"r!r$rrrrrr#rrrrrrrrrr)TR gelu皙?rchn|?linear{Gz??g镲 ?TTTFFgh㈵>F)__name__ __module__ __qualname____firstlineno__ model_typer classmethodr5rArGrMrSrK__static_attributes__ __classcell__r)s@r*rrsJ"%*!#%%(#("%(.1)-!"#* (" $ %!&#'#$$13+j// * KKr,rcd^\rSrSrSrSrSU4SjjrSrU=r$) PlugNLGConfigxap This is the configuration class to store the configuration of a [`PlugModel`]. It is used to instantiate a PLUG understanding model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the PLUG [PLUG](https://modelscope.cn/models/damo/nlp_plug_text-generation_27B/summary) architecture. Configuration objects inherit from [`PlugNLUConfig`] and can be used to control the model outputs. Read the documentation from [`PlugNLUConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 21504): Padded vocabulary size of the PLUG model for vocab tensor parallel. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`PlugModel`]. original_vocab_size (`int`, *optional*, defaults to 21128): True vocabulary size of the PLUG model. Defines the number of different tokens that can be represented. hidden_size (`int`, *optional*, defaults to 8192): Dimensionality of the encoder layers and the pooler layer. num_hidden_layers (`int`, *optional*, defaults to 24): Number of hidden layers in the Transformer encoder. dec_hidden_layers (`int`, *optional*, defaults to 6): Number of hidden layers in the Transformer decoder. num_attention_heads (`int`, *optional*, defaults to 128): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 32768): Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. hidden_act (`str`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"selu"` and `"gelu_new"` are supported. hidden_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for all fully connected layers in the embeddings, encoder, and pooler. attention_probs_dropout_prob (`float`, *optional*, defaults to 0.1): The dropout ratio for the Transformer Attention. max_position_embeddings (`int`, *optional*, defaults to 2048): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). type_vocab_size (`int`, *optional*, defaults to 3): The vocabulary size of the `token_type_ids` passed when calling [`PlugModel`]. initializer_range (`float`, *optional*, defaults to 0.00707): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. lr_decay_style (`str`, *optional*, defaults to 'linear'): The decay style of learning rate during fine-tunining. If string, `"linear"`, `"cosine"`, `"exponential"`, `"constant"`, `"None"` are supported. weight_decay (`float`, *optional*, defaults to 1e-2): Decoupled weight decay to apply. clip_grad (`float`, *optional*, defaults to 1.0): Maximum gradient norm for gradient clipping. warmup (`float`, *optional*, defaults to 0.01): Ratio of total training steps used for a linear warmup from 0 to `learning_rate`. pre_ln (`boolean`, *optional*, defaults to `True`): Whether or not to apply LayerNorm to the input instead of the output in the blocks. fp16 (`boolean`, *optional*, defaults to `True`): Whether to use fp16 16-bit (mixed) precision training instead of 32-bit training. fp32_layernorm (`boolean`, *optional*, defaults to `True`): Whether to use fp32 32-bit precision LayerNorm training while the argument `fp16` set to `True`. fp32_embedding (`boolean`, *optional*, defaults to `False`): Whether to use fp32 32-bit precision Embedding training while the argument `fp16` set to `True`. fp32_tokentypes (`boolean`, *optional*, defaults to `False`): Whether to use fp32 32-bit precision token types training while the argument `fp16` set to `True`. layernorm_epsilon (`float`, *optional*, defaults to 1e-5): The epsilon to use in the layer normalization layers. attn_separate (`boolean`, *optional*, defaults to `False`): Whether or not to separate query-key-value to query, key, value in the Attention. Example: >>> # The PLUG model has 27B parameters and usually need to run on multiple GPUs. The example given >>> # here only initializes a slice of the model on a single GPU. >>> # Check out the [`~DistributedPipeline.__init__`] method to initialize entire PLUG model. >>> from modelscope.models.nlp.plug import PlugNLGConfig, PlugModel >>> # Initializing a Plug configuration >>> configuration = PlugNLGConfig() >>> # Initializing a model from the configuration >>> model = PlugModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config plugNLGc P>[TU]"SSU0UD6 XlX0lX@lX`lXlXplXlXl Xl Xl Xl Xl XlUUlUUlUUlUUlUUlUUlUUlUUlXPlUUlgr )r rrrrrrrrrrrrrrrrrr r!r"r#r$r%r&)r'rrrrr%rrrrrrrrrrrrrr r!r"r$r#r&r(r)s r*rPlugNLGConfig.__init__s4 D(9DVD$&!2#6 $!2#6 ,H)'>$.!2,("   ,,!2.!2*r,)rr&rr%r r"r!r$rrrrrr#rrrrrrrrr)r\r]r^r_rjr`rarbrcrcrdrerfrgrhrirhTTTFFg-q=F) rkrlrmrn__doc__rorrqrrrss@r*ruruxseN`J"%*!#%#$%(#("%(.1)-!"#* (" $ %!&#($12+2+r,ru) rPr< transformersrmodelscope.utilsrlogging get_loggerrrur r,r*rsA" ).    ZK$ZKzE+ME+r,