6pjV SSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJr SSK J r J r J r J r SSKJrJrJrJr SSKJrJrJrJrJrJrJr SSKJr \"5rSrS rS]S jr S \ \!\ \!44S jr"S^S \ \ \!S\ \ \!S\ \!4Sjjr#Sr$S\%S\!4Sjr&S\!S\%4Sjr'S\!S\!4Sjr(S_S\ \!4Sjjr)S\4Sjr*S`S\!S\!S\ \ \!\4S\ \!4Sjjr+S\S\,4Sjr-S r.S]S!jr/SaS"\ \!S#\,S\!4S$jjr0S%r1S\,4S&jr2SbS'\!S(\ \!S)\ \!SS4S*jjr3S+r4S,\5S\!4S-jr6S.\ \ \ \!\%4S/\ \!S\!4S0jr7S1r8\RrSS2.S3\ \!\4S4\ \5S\ \SS44S5jj5r:ScS6\ \%\!\4S7\!S\ \4S8jjr;0S9S:_S;S<_S=S>_S?S@_SASB_SCSB_SDSE_SFSG_SHSI_SJSK_SLSM_SNSM_SOSP_SQSR_SSST_SUSV_SWSX_SYSZ0Er~~01!4  2  c[RRUS5R5R 5nUS:H$)Ntrue)osenvirongetstriplower)var_namevalues r is_env_truer($s2 JJNN8R ( . . 0 6 6 8E F?rc[[R;a9[R"[5(a[R"[5$U(a[$[ $N)rr!r"getenvr rcn_sites r get_domainr.)s<BJJ&2995F+G+Gyy*++((--r raw_inputcTSn[U[5(ab[5nSU;a0URS5Vs/sHo"R 5PM nnU$UR UR 55 U$[U[5(a[5nUHn[U[5(dMSU;a@UR URS5Vs/sHo3R 5PM sn5 M`UR UR 55 M U$s snfs snf)N,) isinstancestrlistrr$appendextend)r/outputssss rconvert_patternsr:2s F)S!! ) )2)=>)=Aggi)=F> M MM)//+ , M It $ $A!S!!!8MM "E "88: "EFMM!''),  M?#Fs D D% allow_file_patternallow_patternsreturnc^/nU(aURU5 U(aURU5 U(dg/nUHn[U5nSH'nURU5nUS:wdM[XW5nM) USUnUR S5n U S:dMYUSU m[ U4SjS55(aMzUR T5 M U(dg[U5S:XaUS$UV s/sHoRS5PM n n /n [U 6H1n [[U 55S:XaU R U S5 M1 O U (dgSRU 5$s sn f)aExtract common directory prefix from include patterns for server-side filtering. Only processes allow/include patterns (ignore/exclude is irrelevant for prefix). Returns None if no meaningful prefix can be extracted. Algorithm: 1. Merge allow_file_pattern and allow_patterns into one list 2. For each pattern, find position of first wildcard char (* ? [) 3. Extract text before that position, then take directory part (up to last '/') 4. Find longest common directory prefix (path-segment-aware) 5. Return None if no valid common prefix Examples: ['TacExo/*'] -> 'TacExo' ['data/train/*.parquet'] -> 'data/train' ['data/train/*', 'data/valid/*'] -> 'data' ['*.safetensors'] -> None ['TacExo/*', 'OtherDir/*'] -> None (no common prefix) ['data/*/train.csv'] -> 'data' N)*?[/rc3,># UH oT;v M g7fr*).0cdir_parts r -extract_root_from_patterns..xs>oH}osr) r6lenfindminrfindanyr5rzipsetjoin)r;r<patternsextracted_dirspatternfirst_wildcardwcposprefixlast_sepd split_pathscommon_segmentssegmentsrHs @rextract_root_from_patternsr_EsT2H*+' NW!B,,r"Cby!$^!9" .)<<$ a<ix(H>o>>>%%h/#(  >aa  *88A773<K8O% s8}  "  " "8A; /  &  88O $$9s/E#c[RRS5(aURSS5nOURSS5n[R R USURSS55nU$)Nwin\rCr.___)sysplatform startswithrsplitr!pathrRreplace) directoryrpartsmasked_directorys rget_model_masked_directoryrosg ||u%%  q)  a(ww||E!Hh.>.>sE.JK r size_bytescSSKnUS:XagSn[URURUS555nUR SU5n[ X- S5nUSX#3$)Nr0B) BKBMBGBTBPBEBZBYBirc )mathintfloorlogpowround)rpr} size_nameipr8s rconvert_readable_sizerscQEI DJJtxx D1 23A qA jna AS),  r folder_pathcSn[U5RS5H6nUR5(dMXR5R- nM8 U$)Nrr?)rrglobis_filestatst_size)r total_sizerjs rget_folder_sizersHJ[!'', <<>> ))+-- -J- rc&[[US95$)Nr)rrrs rget_readable_folder_sizers [!I JJrrc[R"5RSS5n[R"S[R R US55nUcU$[R R X S-5$)zcache dir precedence: function parameter > environment > ~/.cache/modelscope/hub Args: model_id (str, optional): The model id. Returns: str: the model_id dir if model_id not None, otherwise cache root dir. .cache modelscopeMODELSCOPE_CACHEhubrC)rhomejoinpathr!r+rjrR)rdefault_cache_dir base_paths r get_cache_dirrsj ,,X|D ,'',,'8%@BI (9#bggllc>/##rc[RRS5nU(a[U5R 5$[R "5S- S- $)zFCache root used by ``modelscope_hub`` downloads (not SDK ``.../hub``).rrr)r!r"r#r expanduserr)envs r_modelscope_hub_cache_rootrsA **..+ ,C Cy##%% 99; !L 00rrepo_id repo_type cache_dirc>U(aSU;agUb[U5R5O [5nURS5(dUS3OUnUR SS5upVUR SS5nUR SS5nX4- U- X4- U- U- /n U Hn [ U 5(dM g X5- U- X5- U- US- U- U- US- U- U- X4- U- U- /n U H7n [ U 5(dM[RS U U5 [U 5s $ g) aFind old on-disk cache layouts that ``modelscope_hub`` download misses. ``modelscope_hub`` reuses ``{cache}/{type}s/{owner}/{safe_name}/`` (dots in ``name`` replaced by ``___``) and writes to ``{cache}/{type}s/{owner}--{name}/snapshots/{rev}/``. Older SDKs also stored repos at: - ``{cache}/{owner}/{name}/`` (flat, when ``MODELSCOPE_CACHE`` was set) - ``{cache}/hub/{owner}/{name}/`` (pre-``models/`` restructuring) - ``{cache}/{type}s/{owner}/{name}/`` (unsafed name; hub only checks ``safe_name``) Returns a non-empty legacy path only when the layouts hub already handles are absent, so callers can pass it as ``local_dir`` and avoid re-download. rCNr8rrdrez--rz)Found legacy cache at %s for %s, reusing.) rrrendswithrrk_non_empty_dirloggerinfor3) rrrbasesegmentownerr safe_namesafe_id hub_knownrjlegacy_candidatess rfind_reusable_legacy_repo_dirrsE( c(+4+@4 ? % % '"$ %.%7%7%<%< 1o)G--Q'KE S%(Iooc4(G   *I $   t y  u ut# u uy( % " $   KKCT !t9  " rrjcUR5(dg[UR55$![a gf=fNF)is_dirrOiterdirOSError)rjs rrrs8 ;;==4<<>"" s 1 >>c2[[R;a7[[ [ R "5R555nU$SSKJ n [[ [ R"URS5R555nU$)Nr)version%Y-%m-%d %H:%M:%S) rr!r"r~rrnow timestamprrstrptime__release_datetime__)rtrs rget_release_datetimer ssrzz) x||~//12 3 I '  !!'">">"577@y{ DE Irc&[[U5-$r*)rr.r,s r get_endpointrs :g#6 66r cli_endpointr-cUc [US9$UR5RS5nU(d [US9$URS5(dURS5(d [U-nU$)a^Resolve the ModelScope API endpoint with automatic scheme completion. Priority (highest to lowest): 1. ``cli_endpoint`` (explicit CLI --endpoint argument) 2. Environment variable ``MODELSCOPE_DOMAIN`` 3. Built-in default (https://www.modelscope.cn) Scheme auto-completion: If the resolved value does not start with ``http://`` or ``https://``, ``https://`` is prepended automatically so that callers may pass bare domain names such as ``modelscope.ai``. Args: cli_endpoint: Value from the CLI ``--endpoint`` flag. When *None*, the function falls back to :func:`get_endpoint`. cn_site: Forwarded to :func:`get_endpoint` when *cli_endpoint* is *None*. ``True`` selects the Chinese site, ``False`` the international site. Returns: A fully-qualified endpoint URL, e.g. ``https://www.modelscope.cn``. r,rCzhttp://zhttps://)rr$rstriprhr)rr-endpoints rresolve_endpointrsq0G,,!!#**3/H G,,   y ) )(2E2E 33(83 OrcSn[R"5n[US5nURU5nU(dOUR U5 M,SSS5 UR 5$!,(df  UR 5$=f)Nirb)hashlibsha256openreadupdate hexdigest) file_path BUFFER_SIZE sha256_hashfdatas r compute_hashr@sy"K.."K i !66+&D   t $    ""    ""s .A** Bc[U5nX!:Xd;[R"U5 SU<SU<SU<S3n[R U5 gg)zValidate the file hash is expected, if not, delete the file Args: file_path (str): The file to validate expected_sha256 (str): The expected sha256 hash Returns: bool: True if the file is valid, False otherwise zFile z6 integrity check failed, expected sha256 signature is z , actual is z3, the download may be incomplete, please try again.FT)rr!removererror)rexpected_sha256 file_sha256msgs rfile_integrity_validationrMs@y)K  ) )  5 S r file_namerScommit_messagec[U[5(aU/nUc SUSSU3nURn[RR XQ5n[RR U5(a%[USSS9nUR5nSSS5 OSnWn UH;n X;dM [U 5S:aU RS5(dU S- n XS3- n M= X:wa@[US SS9n[RS US U 35 URU 5 SSS5 URU5 g!,(df  N=f!,(df  N2=f![an U(aSn A gU eSn A ff=f) NzAdd `rz` patterns to rutf-8)encodingr wzWriting z file. Content: )r2r3 model_dirr!rjrRexistsrrrKrrdebugwritepush Exception) reporrSrignore_push_errorrepo_dirrrcurrent_contentcontentrUes radd_content_to_filerbsT (C  : ! ^I;G~~H X1I ww~~i  )S7 3qffhO4 3G  !7|a(8(8(>(>4 "~ %G ! )S7 3q LL8I;.>wiH I GGG 4 .!%4 34 3   G s6D80-E &E8 E E E7$E20E22E7))secondr<)minuterr)houri)dayiQ)weeki: r)monthi' )yeari3Ntsc[R"5U- nUS:ag[Hup#n[X- 5nUcMXT::dM O WSWUS:aSS3$SS3$)zOFormat timestamp in seconds into a human-readable string, relative to now. za few seconds agor|rr8rz ago)time_TIMESINCE_CHUNKSr)rdeltalabeldivider max_valuer's rformat_timesincerst IIK" E rz"%6! eo&  U%7 &7WAeWEAIS6d ;;26d ;;rrowsheadersc [/UQUP76Vs/sHn[SU55PM nnS[U5-R"U6n/nUR UR"U65 UR UR"UVs/sHnSU-PM sn65 UH!nUR UR"U65 M# SR U5$s snfs snf)zy Inspired by: - stackoverflow.com/a/8356620/593036 - stackoverflow.com/questions/9535954/printing-lists-as-tabular-data c3J# UHn[[U55v M g7fr*)rKr3)rFxs rrItabulate..s/3ac#a&kk3s!#z{{:{}}} -r)rPmaxrKformatr5rR)rrcol col_widths row_formatlinesrrows rtabulaters1tyonyesr r>0rnnoofffalserzinvalid truth value )r% ValueError)vals r strtoboolr s8 ))+C 22 33 +C73 44rtimeout lock_filer"c#N# Sn[XS9n[R"5n[R"5U- nUbXQ:a[[U55eUR U(a[ X!U- 5OUS9 Uv UR!5 g![a6 [ RSUS[R"5U- SS35 O[a:nS[U5;a%[ RSU5 [XS9nSnAMeSnAf[atnUR[R[R[R4;a1[ RS URU5 [XS9nSnAGMjeSnAff=fGMu![a. [#U5R%5 g![a gf=ff=f!UR!5 f![a. [#U5R%5 f![a ff=ff=f=f7f) Nrr!z!Still waiting to acquire lock on z (elapsed: z.1fz seconds)zuse SoftFileLock insteadzPFileSystem does not appear to support flock. Falling back to SoftFileLock for %szCEncountered OSError (errno=%d) on %s, Falling back to SoftFileLock.)r rr r3acquirerMrrNotImplementedErrorwarningr rerrnoESTALEENOENT EREMOTEIOreleaserunlink)r#r"default_intervallock start_time elapsed_timers rweak_file_lockr2s  I 8DJ yy{Z/  <#:#i.) )  LL, .DE 0  22     LLN9  KK3I;k$))+XbJbcfIggpq # )SV3f$IH  ww5<<uGGYGGY($IH  + L  Y&&(     LLN  Y&&(   sAH%$B=H%?GFH%=FH% F.D H%D FA'FH% FFH% G$F?=H%? G  G H% G  GH%H"G'&H"' H2H  H" H HH"H HH""H% time_stamp time_zonecSSKnU(dg[U[5(aU$[U[5(GaUR S5(aSU;a USSS-nO3USSR S5up4USSR SS5nUSUS3n[R"US 5RURS 5S 9nUS :wa URURU55nU$/S QnUH7n[R"UU5RURU5S 9s $ [S US35e[U[5(aB[R"XRS 5S9RURU55$[S[!U5S35e![a Mf=f![an[SUS35UeSnAff=f![[4an[SUS35UeSnAff=f)a&Convert a UNIX/string timestamp to a timezone-aware datetime object. Args: time_stamp: UNIX timestamp (int), ISO string, or datetime object time_zone: Target timezone for non-UTC timestamps (default: 'Asia/Shanghai') Returns: Timezone-aware datetime object or None if input is None rNZrdrBz.000000Zrrz%Y-%m-%dT%H:%M:%S.%fZUTC)tzinfo)z%Y-%m-%dT%H:%M:%S.%fz%Y-%m-%dT%H:%M:%Sz%Y-%m-%d %H:%M:%S.%frzUnsupported timestamp format: ''zCannot parse 'zr' as a datetime. Expected formats: 'YYYY-MM-DDTHH:MM:SS[.ffffff]Z' (UTC) or 'YYYY-MM-DDTHH:MM:SS[.ffffff]' (local))tzzCannot convert 'z2' to datetime. Ensure it's a valid UNIX timestamp.zUnsupported type 'z"'. Expected int, str, or datetime.)zoneinfor2rr3rrljustrrkZoneInfo astimezonerr~ fromtimestampr TypeErrortype) r3r4r;rfractiondtformatsfmtrs rconvert_timestamprFsU *h''*c""* ""3''j(!+CR:!=J&0_%:%:3%?ND'|11!S9H$(68*A!6J&&z'>@@G2:2C2CE2JAHAL%x'8'8'CDB  #C!'00& !!(0A0A)0L!MN#!5j\CEE J $ $ ))00799C%%i0:2 2 j!1 22T U  1&! !    -bc  G$ ":,.`a  saB2F>. F>82F-*F>-F>AG - F;7F>:F;;F>> GGG H0HHz.webpz image/webpz.avifz image/avifz.heicz image/heicz.heifz image/heifz.jpgz image/jpegz.jpegz.pngz image/pngz.gifz image/gifz.bmpz image/bmpz.svgz image/svg+xmlz.tiffz image/tiffz.tifz.icoz image/x-iconz.mp4z video/mp4z.webmz video/webmz.avizvideo/x-msvideoz.movzvideo/quicktimez.mkvzvideo/x-matroskamedia_file_pathcSSKnSSKn[RR U5n[RR U5(d[ SU35e[RRU5(d[SU35eURU5up4U(dC[R[RRU5SR55nU(d[SU35e[US5nUR5nUR!U5R#S5nSSS5 S US W3$!,(df  N=f) a Encode image or video file to base64 string. Args: media_file_path (str): Path to the image or video file Returns: str: Base64 encoded string with data URL prefix Raises: FileNotFoundError: If image/video file doesn't exist ValueError: If file is not a valid format rNzImage file not found: zPath is not a file: rzFile is not a valid format: rrzdata:z;base64,)base64 mimetypesr!rjrrFileNotFoundErrorisfiler guess_type_FALLBACK_MIME_TYPESr#splitextr%rr b64encodedecode)rGrIrJ mime_type_ media_file image_data base64_datas rencode_media_to_base64rW[s$gg((9O 77>>/ * *"88I JKK 77>>/ * *//@ABB ''8LI (,, GG  _ -a 0 6 6 8: 77HIJJ ot $ __& &&z299'B  % 9+Xk] 33 % $s 1E  E)T)NNr*)modelN)NTr)z Asia/Shanghai)> contextlibr(rr!rfrrpathlibrtypingrrrrfilelockr r r r modelscope.hub.constantsr rrrrrrmodelscope.utils.loggerrrrr(r.r3r:r_ror~rrrrrrboolrrrrrrrrfloatrrr contextmanagerr2rFrNrWrErrrbs  33BB=== /   .c49n 5(/3*.J% c+J%T#Y'J%c]J%^!c!c!K#K#K #HSM #1D1,04 44c4i()4c] 4n$ 748%) 8C= " .1 F #T09=*/ %#&%"&s)%)1 %59 %P  < <3 <4U38_-.cs$5 /32eCI.2$UO2",d":;22l(7L %S((:";L !$L ;CH;ML b \ \ \ \   L   \  K K K O \ L N K \  !" #$ %,*4C*4C*4r