9iJSrSSKrSSKJr SSKJr SSKJrJrJ r J r J r J r J r SSKJrJr SSKJr SSKJrJrJr SS KJrJr SS KJr SS KJr \"5rS S /r\"SS9"SS55r \"SS9"SS55r!\"SS9"SS55r"\"SS9"SS55r#S"S\ \ \$\4S\#4Sjjr%S\S\&4Sjr'S#S\S\$S\&4Sjjr(S \S\$S\ \"4S!jr)g)$zr(r)c,[UR5$)zQ (property) Size of the blob file as a human-readable string. Example: "42.2K". rrr#s r%size_on_disk_strCachedFileInfo.size_on_disk_strH%T%6%677r)N)__name__ __module__ __qualname____firstlineno____doc__str__annotations__rintfloatpropertyr&r+r/__static_attributes__r2r)r%rrs&NOO 9999998#88r)rc\rSrSr%Sr\\S'\\S'\\S'\ \ \S'\ \S'\ S\4S j5r \ S\4S j5r\ S\4S j5rS rg )CachedRevisionInfoRaFrozen data structure holding information about a revision. Args: commit_hash (`str`): Hash of the revision (unique). Example: `"9338f7b671827df886678df2bdd7cc7b4f36dffd"`. snapshot_path (`Path`): Path to the revision directory in the `snapshots` folder. It contains the exact tree structure as the repo on the Hub. files: (`FrozenSet[CachedFileInfo]`): Set of [`~CachedFileInfo`] describing all files contained in the snapshot. size_on_disk (`int`): Sum of the blob file sizes that are symlink-ed by the revision. last_modified (`float`): Timestamp of the last time the revision has been created/modified. commit_hash snapshot_pathrfiles last_modifiedr!c,[UR5$)z (property) Timestamp of the last time the revision has been modified, returned as a human-readable string. Example: "2 weeks ago". rrDr#s r%last_modified_str$CachedRevisionInfo.last_modified_strl  2 233r)c,[UR5$zV (property) Sum of the blob file sizes as a human-readable string. Example: "42.2K". r.r#s r%r/#CachedRevisionInfo.size_on_disk_strvr1r)c,[UR5$)z3 (property) Total number of files in the revision. )lenrCr#s r%nb_filesCachedRevisionInfo.nb_filess 4::r)r2N)r3r4r5r6r7r8r9rr:rrr;r<rGr/rOr=r2r)r%r?r?Rs{" ^ $$ 43448#88#r)r?c\rSrSr%Sr\\S'\\S'\\S'\\S'\\S'\ \ \S'\ \S '\ \S '\ S \4S j5r \ S \4S j5r\ S \4Sj5rSrg)CachedRepoInfoa>Frozen data structure holding information about a cached repository. Args: repo_id (`str`): Repo id of the repo on the Hub. Example: `"damo/bert-base-chinese"`. repo_type (`Literal["dataset", "model"]`): Type of the cached repo. repo_path (`Path`): Local path to the cached repo. size_on_disk (`int`): Sum of the blob file sizes in the cached repo. nb_files (`int`): Total number of blob files in the cached repo. revisions (`FrozenSet[CachedRevisionInfo]`): Set of [`~CachedRevisionInfo`] describing all revisions cached in the repo. last_accessed (`float`): Timestamp of the last time a blob file of the repo has been accessed. last_modified (`float`): Timestamp of the last time a blob file of the repo has been modified/created. repo_id repo_type repo_pathrrO revisions last_accessedrDr!c,[UR5$)z (property) Last time a blob file of the repo has been accessed, returned as a human-readable string. Example: "2 weeks ago". )rrXr#s r%last_accessed_str CachedRepoInfo.last_accessed_strrIr)c,[UR5$)z (property) Last time a blob file of the repo has been modified, returned as a human-readable string. Example: "2 weeks ago". rFr#s r%rG CachedRepoInfo.last_modified_strrIr)c,[UR5$rKr.r#s r%r/CachedRepoInfo.size_on_disk_strr1r)r2N)r3r4r5r6r7r8r9rr:rr?r;r<rZrGr/r=r2r)r%rRrRs*LNOM+,, 434443448#88r)rRcj\rSrSr%Sr\\S'\\\S'\ \ \S'\ S\ 4Sj5r S\ 4SjrS rg ) ModelScopeCacheInfoaFrozen data structure holding information about the entire cache-system. This data structure is returned by [`scan_cache_dir`] and is immutable. Args: size_on_disk (`int`): Sum of all valid repo sizes in the cache-system. repos (`FrozenSet[CachedRepoInfo]`): Set of [`~CachedRepoInfo`] describing all valid cached repos found on the cache-system while scanning. warnings (`List[CorruptedCacheException]`): List of [`~CorruptedCacheException`] that occurred while scanning the cache. Those exceptions are captured so that the scan can continue. Corrupted repos are skipped from the scan. rreposwarningsr!c,[UR5$)zX (property) Sum of all valid repo sizes in the cache-system as a human-readable string. r.r#s r%r/$ModelScopeCacheInfo.size_on_disk_strs %T%6%677r)c S[S[S[[4Sjn/SQn[ UR SS9VVs/sH(n[ UR SS9H nU"X45PM M* nnn[UUS 9$s snnf) aGenerate a detailed table from the [`ModelScopeCacheInfo`] object. Returns a table with a row per repo and revision (thus multiple rows can appear for a single repo), with columns "repo_id", "repo_type", "revision", "size_on_disk", "nb_files", "last_modified", "local_path". Example: ```py >>> from modelscope.hub.cache_manager import scan_cache_dir >>> ms_cache_info = scan_cache_dir() ModelScopeCacheInfo(...) >>> print(ms_cache_info.export_as_table()) REPO ID REPO TYPE REVISION SIZE ON DISK NB FILES LAST_MODIFIED LOCAL PATH ---------------------- --------- ---------- ------------ -------- ------------- ------------------------------------------------------------- damo/bert-base-chinese model master 2.7M 5 1 week ago ~/.cache/modelscope/hub/models--damo--bert-base-chinese/... damo/structured-bert model master 8.8K 1 1 week ago ~/.cache/modelscope/hub/models--damo--structured-bert/... damo/t5-base model master 893.8M 4 7 months ago ~/.cache/modelscope/hub/models--damo--t5-base/... ``` Returns: `str`: The table as a string. reporevisionr!c URURURSRUR5UR UR UR[UR5/$)zHFormat a single repo and revision into a list of strings for tabulation.z{:>12}) rTrUrAformatr/rOrZrGr8rV)rhris r%format_repo_revisionAModelScopeCacheInfo.export_as_table..format_repo_revisions\ $$ 5 56 &&&&DNN#  r))zREPO IDz REPO TYPEREVISIONz SIZE ON DISKzNB FILES LAST_ACCESSED LAST_MODIFIEDz LOCAL PATHcUR$N)rT)rhs r%5ModelScopeCacheInfo.export_as_table..sDLLr))keycUR$rr)rA)ris r%rsrts X5I5Ir))rowsheaders)rRr?rr8sortedrcrWr)r$rlcolumn_headersrhri table_datas r%export_as_table#ModelScopeCacheInfo.export_as_tables2 ~ += BFs)   tzz/HI I"$IK ! 0K 1I  "   s/A7r2N)r3r4r5r6r7r:r9rrRrr r<r8r/r|r=r2r)r%rarasK  ^ $$*++ 8#88< < r)ra cache_dirr!cBUc [5n[U5R5R5nUR 5(d[ SUS3US9eUR 5(a[SUS35e[5n/nUS- nUS- nUR 5(aGUR5(a2[U[S9upVURU5 URU5 UR 5(aGUR5(a2[U[S9upxURU5 URU5 [U[SS 9upURU 5 URU 5 [[!U5[#S U55US 9$) aScan the entire ModelScope cache-system and return a [`ModelScopeCacheInfo`] structure. Use `scan_cache_dir` to programmatically scan your cache-system. The cache will be scanned repo by repo. If a repo is corrupted, a [`~CorruptedCacheException`] will be thrown internally but captured and returned in the [`~ModelScopeCacheInfo`] structure. Only valid repos get a proper report. ```py >>> from modelscope.hub.utils import scan_cache_dir >>> ms_cache_info = scan_cache_dir() ModelScopeCacheInfo( size_on_disk=3398085269, repos=frozenset({ CachedRepoInfo( repo_id='damo/t5-small', repo_type='model', repo_path=PosixPath(...), size_on_disk=970726914, nb_files=11, revisions=frozenset({ CachedRevisionInfo( commit_hash='master', size_on_disk=970726339, snapshot_path=PosixPath(...), files=frozenset({ CachedFileInfo( file_name='config.json', size_on_disk=1197 file_path=PosixPath(...), blob_path=PosixPath(...), ), CachedFileInfo(...), ... }), ), CachedRevisionInfo(...), ... }), ), CachedRepoInfo(...), ... }), warnings=[ CorruptedCacheException("Snapshots dir doesn't exist in cached repo: ..."), CorruptedCacheException(...), ... ], ) ``` Args: cache_dir (`str` or `Path`, `optional`): Cache directory to scan. Defaults to the default ModelScope cache directory. Raises: `CacheNotFound`: If the cache directory does not exist. `ValueError`: If the cache directory is a file, instead of a directory. Returns: a [`ModelScopeCacheInfo`] object. zCache directory not found: zQ. Please use `cache_dir` argument or set `MODELSCOPE_CACHE` environment variable.)r~z1Scan cache expects a directory but found a file: modelsdatasetsrUT)rUinplacec38# UHoRv M g7frr)r).0rhs r% !scan_cache_dir..s=ut**u)rcrrd)rr expanduserresolveexistsr is_file ValueErrorsetis_dir _scan_dirrupdateextendrra frozensetsum) r~rcrd model_dir dataset_dir model_reposmodel_warnings dataset_reposdataset_warnings other_reposother_warningss r%scan_cache_dirr#s~,. Y**,446I     ))5F G  ? {K\ ]  "%E.0HH$Ij(Ki..00&/ '2#  [!' 2 2 4 4*3 #4+6'  ]#()#,_d#<K LL OON# =u== r)dircUR5(dgUR5(dgUR5(agUR[;agg)z+Check if a directory is valid for scanning.FT)rr is_symlinknameFILES_TO_IGNORE)rs r% _is_valid_dirrs@ ::<< ::<< ~~ xx?" r)rUrc[5n/nUR5HwnU(aURS;aM[U5(dM.UR5H5n[U5(dM[ XaS9nUbUR U5 M5M7 My X44$![ anURU5 SnAMaSnAff=f)zWScan a directory for cached repos and return a set of [`~CachedRepoInfo`] and warnings.)rrhubrN)riterdirrr_scan_cached_repoaddr append) rrUrrcrd owner_dirname_dirinfoes r%rrs EEH[[] y~~)FF Y'' !))+H ** #(G#IIdO$ ,# ?+ #"" #s1B C%B<<CrVc ^UR5(d[SU35e[[U55nURnUR nUR 5RSS5nUS:Xag0m[5nUHn[RRXS5n URSS 5n [RRU 5(dM][U 5n U R!5TU 'UR#[%[RR'US5U U TU R(U TU R*TU R,S 95 M S n U(a,S U;a&UR/S 5S R/S5Sn U(a[1U4SjU55n OUR!5R,n [3U [5U5[7U4SjU55UU S9nT(aA[1STR9555n[1STR9555nO(UR!5nUR*nUR,n[;[=T5UUU[5U/5[7STR9555UUS9$![an[SU35eSnAff=f)z~Scan a single cache repo and return information about it. Any unexpected behavior will raise a [`~CorruptedCacheException`]. zRepo path is not a directory: ___.unknownNz"Failed to load cache information: rRevision)rrrrrrr masterz Revision:,rc3V># UHnTURRv M g7frr)rst_mtimerfile blob_statss r%r$_scan_cached_repo..s(%D1B&0%?%H%H1B&)c3V># UHnTURRv M g7frr)rst_sizers r%rrs'8%6T$DNN3;;%6r)rArCrrBrDc38# UHoRv M g7frr)st_atimerstats r%rr O;N4;Nrc38# UHoRv M g7frr)rrs r%rrrrc38# UHoRv M g7frr)rrs r%rrsF2E$2Er)rOrTrVrUrWrrXrD)rr rr8 cached_filescached_model_revision get_model_idreplace Exceptionrospathjoingetrrrrrbasenamerrrsplitmaxr?rrvaluesrRrN)rVrUcacherrrTrcached_files_info cached_filerrr revision_hashrevision_last_modifiedcached_revisionrepo_last_accessedrepo_last_modified repo_statsrs @r%rrs     %,YK 8: : P$S^4)) % ; ;$$&..uc: i  J$ GGLL,?@ (__Z<ww~~i(( O ) 0 9  ''**;v+>?##5' 2::##-i#8#A#A#-i#8#A#A  $,M / /177 DQGMMM!$%D1B%D"D"+!1!:!:)!)*8%688, O O:;L;L;N OO O:;L;L;N OO^^% '00'00 Z_-.F*2C2C2EFF((  A P%(J1#&NOOPsAJ22 K<K  Krr)F)*r7r dataclassesrpathlibrtypingrrrrr r r modelscope.hub.errorsr r modelscope.hub.utils.cachingrmodelscope.hub.utils.utilsrrrmodelscope.utils.constantrrmodelscope.utils.file_utilsrmodelscope.utils.loggerrloggerrrr?rRrar8rboolrrrr2r)r%rsEB !GGGH=DDH@.  - $888888v $111h $;8;8;8| $Y Y Y z15pE#t),-p9Lpf t  4C$.\\!$\)1.)A\r)