9i`SSKrSSKJr SSKrSSKJrJr SSKJ r SSK J r SSK J r SSKJrJrJrJrJrJrJr SSKrSSKJr SSKJr SS KJr SS KJrJ r SS K!J"r"J#r# SS K$J%r% SS K&J'r' SSK(J)r)J*r*J+r+ SSK,J-r-J.r.J/r/ SSK0J1r1J2r2 SSK3J4r4J5r5 SSK6J7r7 SSK8J9r9 SSK:J;r; SSKr> \5"5(aSSK?r?\4"5(a\Sr@\\A\B\SS4rC\\A\S4rD\7"5rE"SS\5rF"SS\F5rGS rHg)!N)ABCabstractmethod)partial)Pool)Lock)AnyDict GeneratorListMappingOptionalUnion)version)Model) MsDataset) TASK_OUTPUTSModelOutputBase) TASK_INPUTScheck_input_type) Preprocessor)Config) FrameworksInvoke ModelFile) create_devicedevice_placement verify_device) read_configsnapshot_download)is_tf_availableis_torch_available) get_logger) compile_model)check_model_from_owner_group)is_modelis_official_hub_path)z torch.Tensorz tf.Tensorz Image.Imagez numpy.ndarrayztorch.nn.Modulec \rSrSrSrSrS\\4SjrS!S\ S\ \\\4S \ \ \\ 4S \ 4S jjr S"S \ \ S \ \ 4SjjrSrS\ 4SjrS\ \\\4S\ \\ \4\44SjrSrS\4SjrSrS\S\\ \44SjrSrS\\S\\ \44SjrSrSrS\S\\ \44SjrS\\ \4S\\ \44SjrS\\ \4S\\ \44Sjr S r!g)#Pipeline,zPipeline base. c UR(aSUS'[U[5(a[R SU35 [U[5(ay[ U5(ai[R SUS35 [ U5(a>[R"U4URS[RURS.UD6$U$U$)NTtrust_remote_codezinitiate model from zinitiate model from location .)devicemodel_prefetched invoked_by device_map) r- isinstancestrloggerinfor(r'rfrom_pretrained device_namerPIPELINEr2)selfmodelkwargss Y/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/pipelines/base.pyinitiate_single_modelPipeline.initiate_single_model0s  ! !*.F& ' eS ! ! KK.ug6 7 eS ! !&:5&A&A KK7wa@ A&e__ ((''!%!????    8 38  8L input_modelsc\/nUH#nURURU55 M% U$N)appendr>)r:rAmodelsr;s r=initiate_multiple_models!Pipeline.initiate_multiple_modelsBs-!E MM$44U; <" r@N config_filer; preprocessorr/c Ub US:XdS5eX`l[U5 X@lURSS5Ul[ U[ 5(d+UR"U40UD6UlUR/Ul OSUlURU5Ul [UR5S:Ul Ub;[R"U5Ul[ R"R%U5nOcUR(dR[ UR[&5(a URnOURR(n[+U5UlUc-UR(d[,R."W5UlOX0lUR(d%UR(a*URS(aUR35UlOSUlUR4[6R8:Xa[;UR5UlSUl[A5Ul!XPl"URSS5Ul#URS 05Ul$g) aBase class for pipeline. If config_file is provided, model and preprocessor will be instantiated from corresponding config. Otherwise, model and preprocessor will be constructed separately. Args: config_file(str, optional): Filepath to configuration file. model: (list of) Model name or model object preprocessor: (list of) Preprocessor object device (str): device str, should be either cpu, cuda, gpu, gpu:X or cuda:X auto_collate (bool): automatically to convert data to tensor or not. compile (bool, optional): Compile the model with torch 2.0, default False compile_options (dict, optional): The compile options if compile=True, default None to use the default params of 'TorchModel.compile'. Ngpuz;`device` and `device_map` cannot be input at the same time!r-Fr&rcompilecompile_options)%r2rr8getr-r3r r>r;rErFlenhas_multiple_modelsr from_filecfgospathdirnamer4 model_dirrrr7rI_get_framework frameworkrtorchrr/_model_preparer_model_prepare_lock _auto_collate_compile_compile_options) r:rHr;rIr/ auto_collater2r<rVs r=__init__Pipeline.__init__Hs0  !U? a$a a?$f!!',?!G%&&33EDVDDJ::,DKDJ77>DK#&t{{#3a#7  "'' 4DH 4I))$**c** JJ  JJ00 "9-DH  (@(@ , < >Z-- -'(8(89DK##'6 ) 9e4 & +[U[RR5(d[ US5(a UR n[U[RR5(dgUR 5 SSKJn U"U5(aURTR5 gg)Nr;r)is_on_same_device) r3rYnnModulehasattrr;evalmodelscope.utils.torch_utilsritor/)r;rir:s r=_prepare_single/Pipeline.prepare_model.._prepare_singlesveUXX__55'7;$;$ eUXX__55 JJL F ''%(r@TN) r[acquirerZrXrrYrPrEr]r#r^r;release)r:rpms` r= prepare_modelPipeline.prepare_models   (((5 &""~~!1!11++![['*)}}&*[['%0*!Et/D/DE%0' $DJJ/}}%24::&L595J5J&L "&D    ((*'s D.returnc^/mURH}n[U[5(aUnO URn[R "U[ R5n[R"U5nTRUR5 M [U4SjT55(d[RST35 gTS$)Nc32># UH oTS:Hv M g7f)rN).0x frameworkss r= *Pipeline._get_framework..s:z! 1 %zsz:got multiple models, but they are in different frameworks r)rEr3r4rVospjoinr CONFIGURATIONrrQrDrXallr5warning)r:rtrVcfg_filerRr}s @r=rWPipeline._get_frameworks A!S!! KK xx 9+B+BCH""8,C   cmm ,:z::: NNLZLY !}r@inputcUR(d%UR(a5URS(a!UR(dUR 5 UR SS5nUR "S 0UD6upVnXSS'XcS'XsS'S[U5R;a[U[5(a SU0nSUS '[U[5(aKUc2/nUH(n URUR"U /UQ70UD65 M* U$UR"X40UD6nU$[U[5(aUR"U/UQ70UD6$UR"U/UQ70UD6nU$) Nr batch_sizepreprocess_paramsforward_paramspostprocess_params LLMPipelinemessagesT is_messagerz)r;rPrErZrupop_sanitize_parameterstype__name__r3listrD_process_single_process_batchr_process_iterator) r:rargsr<rrrroutputeles r=__call__Pipeline.__call__sg JJ433 A&&""$ ZZ d3 @D@Y@YAA=+=&7"##1 '9#$ DJ// /Jud4K4K'E#'F< eT " "! CMM$"6"6s"LT"LV"LM! ,,UI&I y ) )))%A$A&A A))%A$A&AF r@c 00U4$)a this method should sanitize the keyword args to preprocessor params, forward params and postprocess params on '__call__' or '_process_single' method considered to be a normal classmethod with default implementation / output Default Returns: Dict[str, str]: preprocess_params = {} Dict[str, str]: forward_params = {} Dict[str, str]: postprocess_params = pipeline_parameters rz)r:pipeline_parameterss r=rPipeline._sanitize_parameterss2***r@c/N# UHnUR"U/UQ70UD6v M g7frC)r)r:rrr<rs r=rPipeline._process_iterators*C&&s.> ?~~!1!11]]_))"..s3,,s=n=C%_ ll39.9@s9&89 3 %_@ ?s$;4E/6D/%E/ D= 9E EcJ0nUHCnUR5H,upEURU/5nURU5 XbU'M. ME UR5HCn[ X$S[ R 5(dM)[ R"X$5X$'ME U$)Nr)itemsrNrDkeysr3rYTensorcat)r: data_list batch_datasample_preprocessedkv value_lists r=_batchPipeline._batch"s #, +113'^^Ar2 !!!$ *1 4$- "A*-*ELL99 % *- 8 #r@c ^URS5nURS5nURS5n/n[S[U5U5GHn[X-[U55n X- n XU Vs/sHoR"U40UD6PM n n[ UR UR5 UR [R:Xae[R"5 URU 5n UR(aURU 5n UR"U 40UD6n SSS5 O$URU 5n UR"U 40UD6n SSS5 [U 5Hm0n W R5HyupUcM [!U["[$45(aJ[!US[R&5(a[)U5"U4SjU55X'MfUTX'MoUTTS-X'M{ UR*"U 40UD6n UR-U 5 UR/U 5 M GM U$s snf!,(df  N=f!,(df  GN=f)Nrrrrc34># UH nUTTS-v M g7f)r&Nrz)r{e batch_idxs r=r~*Pipeline._process_batch..Os#76-4%&i A $>-4sr&)rNrangerOminrrrXr8rrYrrr\rrrr3tuplerrrrrrD)r:rrr<rrr output_listiendreal_batch_sizepreprocessed_list batched_outrrelementrs @r=rPipeline._process_batch.s#"JJ':;$45#ZZ(<= q#e*j1Aanc%j1C!gOAF!AMA7%67 !"$..$2B2BC>>Z%5%55&*kk2C&D --*.*:*:;*GK&*ll;'E5C'E )#'++.?"@K"&,,{"Mn"MKD#?3 "-"3"3"5JA*%gt}==)'!*ellCC)-g76-476*6 *1);%,Yy1}%ECF#6&&sA.@A""3'""3'!4'2JE! )DCs+2H=04I$AI+-I I I I" cURnU[;Gax[Un[U[5(aSnUHUn[U[[ 45(a[ U5[ U5:XaUn OM<[U[5(dMSUn O UcSnUH nXeS3- nM [U5eUn[U[5(a [X15 g[U[ 5(a<[U[ 5(dS5e[X15HupW[XW5 M g[U[5(aFUR5H1n[U[5(dMX;dM![X8X5 M3 g[SU35e[USS5(d![RSUS35 S Ulgg) NzDinput data format for current pipeline should be one of following:  zinput should be a tuplezinvalid input_type definition _input_has_warnedFtask z input definition is missingT) group_keyrr3rdictrrr4 ValueErrorrziprgetattrr5rr) r:r task_name input_type matched_typeterr_msg input_elers r=rPipeline._check_input]sNN  #$Y/J*d++# #A!%$777d5k1+,L!2$As++'( $ 'eG'S8+($W--!-J*c** 3J..!%//J1JJ/$' $:LA$Q2%;J--#*A!%..1:(A+ !#A*!NOO2E:: NNU9+-IJ K%)D ";r@cURnU[;a3[USS5(d [R SUS35 SUlg[Un/n[ U[[45(aUR5OUnUH8n[ U[[45(dM XQ;dM'URU5 M: [U5S:a[SUSUS 35eg) N_output_has_warnedFrz output keys are missingTrzexpected output keys are z, those z are missing) rrrr5rrr3rrrrDrOr)r:rr output_keys missing_keysrs r=rPipeline._check_outputsNN L (4!5u==yk1IJK*.' "9-  *5,0/+B!D!D IN A!dO455!.##A& | q 8 F&&2^<AB B !r@inputsc URcS5e[UR[5(aS5eUR"U40UD6$)z[Provide default implementation based on preprocess_cfg and user can reimplement it z'preprocess method should be implementedzEdefault implementation does not support using multiple preprocessors.)rIr3r )r:rrs r=rPipeline.preprocesssX  ,W.WW,d//66 T S T6  =+<==r@c URcS5eUR(aS5eUR"U40UD6$)zTProvide default implementation using self.model and user can reimplement it z$forward method should be implementedzFdefault implementation does not support multiple models in a pipeline.)r;rP)r:rrs r=rPipeline.forwardsCzz%M'MM%++u-uu+zz&3N33r@c [S5e)a"If current pipeline support model reuse, common postprocess code should be write here. Args: inputs: input data post_params: post process parameters Return: dict of results: a dict containing outputs of model, each output should have the standard output name. r)NotImplementedError)r:r post_paramss r=rPipeline.postprocesss"-00r@)r\r]r^rrZr[rrRr/r2r8rXrPr;rErIr-)NNNrKTN)NN)"r __module__ __qualname____firstlineno____doc__r>r InputModelrFr4rrr`r rdrurWInputr rr rrrrrrrrrrrr__static_attributes__rzr@r=r*r*,s$T*5E%)>BIM$" BB!BBj$z*::;BB %\4 3E%EFBB BBJ;?;?4*23-4+3C=4 !+F$'eE4;$67'#DcNI$=>'R +=u=-US#X( -DK-$(cN-^(*TB(>>S#X>4d38n4%)#s(^4 1$sCx. 1&*38n 1r@r*c\rSrSrSrSS\S\\\\44Sjjr Sr Sr \ S 5r S \\\4S \\\44S jr\ S 5rS\S \4SjrSrg)DistributedPipelineiaThis pipeline is used to load multi gpu models. What will this class do: 1. Read the global config from the configuration.json 2. Set the multiprocessing method to spawn 3. Open a multiprocessing pool of the world_size to instantiate model pieces. 4. Set the master port and ip 5. Call _instantiate_one to instantiate one model piece, This method should be implemented by the derived class. 6. After the forward method is called, do preprocess in main process and call _forward_one to collect results, and do post process in main process. NOTE: _instantiate_one and _forward_one are class methods, any derived class should implement them and store the model handler in the class field. Nr;rIc PX lSUl[5UlX0l[ R RU5(aXlO[U5Ul[UR5Ul URUR5Ul SUlSUl[!UR5UlSUlURR&Ul[(R*R-SSS9 [/[1UR55n[3UR5UlSU;aSUS'SU;a[5US5O[6R8"S S 5nS S KJnJn U"U5(dU"5n[AU5US'US[ RBS 'US[ RBS'URRE[GURHRJ4SUR0URRLDUD6U5 /Ul'g)NFcpuspawnT)force master_ipz 127.0.0.1 master_porti>% "N.u5DNt~~...txx8 #D$4$45 #( ++ ..wd.CU4??+,t/ f $"-F; ,6&/1v r@cg)zForward the inputs to one model piece. Use the model handler kept in the class field to forward. Args: inputs: The inputs after the preprocessing. Returns: The forward results. Nrz)rrs r=r DistributedPipeline._forward_one#rr@rRcPURS5nUcURS5$U$)Nzmegatron.world_sizezmodel.world_size)safe_get)r:rR m_world_sizes r=r#DistributedPipeline._get_world_size1s-||$9:  << 23 3r@) r\rZr[rRr/r8rXrPrVrrErIr)NNT)rrrrrr4rrr r`rr classmethodr r rrrrrrrrzr@r=rrs"#IM"//$\4 3E%EF/b    d38n%)#s(^    6cr@rc $^SSKJn Sn[U[5(d[U[5(aF[ U5"UR 5VVs0sHupEXDS:wa [UT5OU_M snn5$[U[[45(auS[U5:Xa[R"/5$[US[[45(aU"U5RT5$[ U5"U4SjU55$[U[ R"5(aIUR$R [ R&LaU$[[R("U5T5$[U[R5(aURT5$[U[*[,[[[.[ S545(aU$U"U5S:XaU$U"U5S:XaU$[1S [ U535es snnf) aPrepare the input just before the forward function. This method will move the tensors to the right device. Usually this method does not need to be overridden. Args: data: The data out of the dataloader. device: The device to move data to. Returns: The processed data. r)default_collatec.URR$rC)r r)objs r=get_class_name"collate_fn..get_class_nameFs}}%%%r@ img_metasc3<># UHn[UT5v M g7frC)r)r{rr/s r=r~collate_fn..UsBTjF33TsN InputFeatures DataContainerzUnsupported data type )torch.utils.data.dataloaderr)r3rr rrrrrrOrYrrfloatronpndarraydtypestr_ from_numpybytesr4boolr)rr/r)r,rrs ` r=rr8s<&$D'!:!:Dz  $ [(8z!V$a ?$    D5$- ( ( D ><<# # d1gU| , ,"4(++F3 3:BTBB B D"** % % ::??bgg %Ke..t4f= = D%,, ' 'wwv D5#sE4dD E E   0   0 1$t*>??7 s H )IrSos.pathrTrrabcrr functoolsrrr threadingrtypingrr r r r r rnumpyr5 packagingrmodelscope.models.basermodelscope.msdatasetsrmodelscope.outputsrrmodelscope.pipeline_inputsrrmodelscope.preprocessorsrmodelscope.utils.configrmodelscope.utils.constantrrrmodelscope.utils.devicerrrmodelscope.utils.hubrrmodelscope.utils.import_utilsr r!modelscope.utils.loggerr"rnr#utils.automodel_utilsr%utilr'r(rYrr4rrrr5r*rrrzr@r=rPs  # GGG(+<D1*CC44?M.6@0 * + c5)]OCD 300 1 L1sL1^ z(zz.@r@