KiDSSKrSSKrSSKrSSKrSSKrSSKrSSKJr SSKJr SSK J r SSK r SSK J r Jr SSKJrJr \R$"S5r"S S \5r"S S \ 5rg) N)suppress)datetime)quote)AbstractBufferedFileAbstractFileSystem)infer_storage_optionstokenizewebhdfsch^\rSrSrSr\"\R"55rSr S!U4Sjjr \ S5r Sr S"SjrS#Sjr\S 5r\S 5r\S 5rS rS rSrS$SjrSrSrSrS%SjrSrSrSrS&Sjr Sr!Sr"S$Sjr#Sr$S$Sjr%Sr&Sr'Sr(S r)U=r*$)'WebHDFSa Interface to HDFS over HTTP using the WebHDFS API. Supports also HttpFS gateways. Four auth mechanisms are supported: insecure: no auth is done, and the user is assumed to be whoever they say they are (parameter ``user``), or a predefined value such as "dr.who" if not given spnego: when kerberos authentication is enabled, auth is negotiated by requests_kerberos https://github.com/requests/requests-kerberos . This establishes a session based on existing kinit login and/or specified principal/password; parameters are passed with ``kerb_kwargs`` token: uses an existing Hadoop delegation token from another secured service. Indeed, this client can also generate such tokens when not insecure. Note that tokens expire, but can be renewed (by a previously specified user) and may allow for proxying. basic-auth: used when both parameter ``user`` and parameter ``password`` are provided. )r webHDFSc  ">UR(ag[TU]"S0U D6 U (aSOSSUSUS3UlX0lU=(d 0Ul0UlU =(d 0UlUbUcUb [S5eX@R S'XPl X`l UbUc [S 5eOUbXPR S 'UbXpR S 'U(aUb [S 5eXl Xl UR5 S [X53Ulg)ab Parameters ---------- host: str Name-node address port: int Port for webHDFS kerberos: bool Whether to authenticate with kerberos for this connection token: str or None If given, use this token on every call to authenticate. A user and user-proxy may be encoded in the token and should not be also given user: str or None If given, assert the user name to connect with password: str or None If given, assert the password to use for basic auth. If password is provided, user must be provided also proxy_to: str or None If given, the user has the authority to proxy, and this value is the user in who's name actions are taken kerb_kwargs: dict Any extra arguments for HTTPKerberosAuth, see ``_ data_proxy: dict, callable or None If given, map data-node addresses. This can be necessary if the HDFS cluster is behind a proxy, running on Docker or otherwise has a mismatch between the host-names given by the name-node and the address by which to refer to them from the client. If a dict, maps host names ``host->data_proxy[host]``; if a callable, full URLs are passed, and function must conform to ``url->data_proxy(url)``. use_https: bool Whether to connect to the Name-node using HTTPS instead of HTTP session_cert: str or Tuple[str, str] or None Path to a certificate file, or tuple of (cert, key) files to use for the requests.Session session_verify: str, bool or None Path to a certificate file to use for verifying the requests.Session. kwargs Nhttpshttpz://:z /webhdfs/v1z_If passing a delegation token, must not set user or proxy_to, as these are encoded in the token delegationzQIf passing a password, the user must also beset in order to set up the basic-authz user.namedoaszJIf using Kerberos auth, do not specify the user, this is handled by kinit.webhdfs_)_cachedsuper__init__urlkerb kerb_kwargsparsproxy ValueErroruserpassword session_certsession_verify_connectr _fsid)selfhostportkerberostokenr!r"proxy_tor data_proxy use_httpsr#r$kwargs __class__s `/var/www/html/dynamic-report/venv/lib/python3.13/site-packages/fsspec/implementations/webhdfs.pyrWebHDFS.__init__.s,r <<  "6"!*g7s4&${S &," %2  8#7  ',IIl #   | < )- +&   (IIf  (2  ),  456 cUR$N)r&r's r1fsid WebHDFS.fsids zzr3c[R"5UlUR(aURURlUR URlUR(a'SSKJ n U"S0URD6URl URbAURb3SSKJn U"URUR5URl ggg)Nr)HTTPKerberosAuth) HTTPBasicAuthr)requestsSessionsessionr#certr$verifyrrequests_kerberosr:rauthr!r" requests.authr;)r'r:r;s r1r%WebHDFS._connects'')    $ 1 1DLL "11  99 : 0 D43C3C DDLL  99 T]]%> 3 -dii GDLL &? r3c UbURU5OSnURUR[USS9-5nUR 5nUR UR 5 UR5US'[RSXr5 URRUR5UUUUS9n U RS;a_U R5n U SS n U SS n U S ;a [U 5eU S ;a [U 5eU S ;a [!U 5e[#U 5eU R'5 U $![[$4a N$f=f)Nz/=)safeopzsending %s with %s)methodrparamsdataallow_redirects)iiiiiRemoteExceptionmessage exception)IllegalArgumentExceptionUnsupportedOperationException)SecurityExceptionAccessControlException)FileNotFoundException)_strip_protocol _apply_proxyrrcopyupdaterupperloggerdebugr>request status_codejsonr PermissionErrorFileNotFoundError RuntimeErrorKeyErrorraise_for_status) r'rHrIpathrKredirectr/rargsouterrmsgexps r1_call WebHDFS._callsQ-1-=t##D)25D+A AB{{} DIIXXZT  )37ll""<<>$ #  ??7 7 ,hhj+,Y7+,[9WW$S/)KK)#..55+C00&s++  )  s D==EEc `U=(d URn[UUUUURUUUS9$)a Parameters ---------- path: str File location mode: str 'rb', 'wb', etc. block_size: int Client buffer size for read-ahead or write buffer autocommit: bool If False, writes to temporary file that only gets put in final location upon commit replication: int Number of copies of file on the cluster, write mode only permissions: str or int posix permissions, write mode only kwargs Returns ------- WebHDFile instance )mode block_sizetempdir autocommit replication permissions) blocksize WebHDFilerp)r'rdrnrorqrrrsr/s r1_open WebHDFS._opens<B 14>>   !LL!##  r3cBUSR5US'USUS'U$)Ntypelengthsize)lower)infos r1 _process_infoWebHDFS._process_infos*F|))+V H~V  r3c[U5S$)Nrd)r )clsrds r1rUWebHDFS._strip_protocols$T*622r3c[U5nURSS5 URSS5 SU;aURS5US'U$)Nrdprotocolusernamer!)r pop)urlpathrgs r1_get_kwargs_from_urlsWebHDFS._get_kwargs_from_urlssG#G,   D!  ''*-CK r3crURSUS9nUR5SnXS'URU5$)N GETFILESTATUSrd FileStatusname)rkr^r~)r'rdrgr}s r1r} WebHDFS.info s<jjtj4xxz,'V !!$''r3cURU5nURSS5nUb[R"US- 5$[ S5e)z=Return the created timestamp of a file as a datetime.datetimemodificationTimeNz5Could not retrieve creation time (modification time).r}getr fromtimestamprar'rdr}mtimes r1createdWebHDFS.createdsIyy+T2  ))%$,7 7RSSr3cURU5nURSS5nUb[R"US- 5$[ S5e)z>Return the modified timestamp of a file as a datetime.datetimerNrz%Could not retrieve modification time.rrs r1modifiedWebHDFS.modifiedsGyy+T2  ))%$,7 7BCCr3c URSUS9nUR5SSnUH1nURU5 URS5S-US-US'M3 U(a [ USS 9$[ S U55$) N LISTSTATUSr FileStatusesr/ pathSuffixrc US$)Nrr)is r1WebHDFS.ls..(sqyr3)keyc3*# UH oSv M g7f)rNr).0r}s r1 WebHDFS.ls..*s954v,5s)rkr^r~rstripsorted)r'rddetailr/rginfosr}s r1ls WebHDFS.ls!sjjDj1 >*<8D   t $;;s+c1D4FFDL %%89 99599 9r3cHURSUS9nUR5S$)z8Total numbers of files, directories and bytes under pathGETCONTENTSUMMARYrContentSummaryrkr^)r'rdrgs r1content_summaryWebHDFS.content_summary,s'jj,4j8xxz*++r3cBURSUSS9nSUR;a\URURS5nURR U5nUR 5 UR 5S$UR 5 UR 5S$)z/Checksum info of file, giving method and resultGETFILECHECKSUMF)rdreLocation FileChecksum)rkheadersrVr>rrcr^)r'rdrglocationout2s r1ukey WebHDFS.ukey1sjj*jF  $((Z)@AH<<##H-D  ! ! #99;~. .  "88:n- -r3cJURS5nUR5S$)zGet user's home directoryGETHOMEDIRECTORYPathr)r'rgs r1home_directoryWebHDFS.home_directory=s"jj+,xxz&!!r3cU(aURSUS9nOURS5nUR5SnUc [S5eUS$)zRetrieve token which can give the same authority to other uses Parameters ---------- renewer: str or None User who may use this token; if None, will be current user GETDELEGATIONTOKEN)renewerTokenz1No token available for this user/security context urlString)rkr^r )r'rrgts r1get_delegation_tokenWebHDFS.get_delegation_tokenBsT **17*CC**12C HHJw  9PQ Q~r3cJURSSUS9nUR5S$)z/Make token live longer. Returns new expiry timeRENEWDELEGATIONTOKENputrIr+longr)r'r+rgs r1renew_delegation_tokenWebHDFS.renew_delegation_tokenSs(jj/UjKxxz&!!r3c&URSSUS9 g)z Stop the token from being usefulCANCELDELEGATIONTOKENrrNrk)r'r+s r1cancel_delegation_tokenWebHDFS.cancel_delegation_tokenXs *5 Fr3c&URSSXS9 g)zSet the permission at path Parameters ---------- path: str location to set (file or directory) mod: str or int posix epresentation or permission, give as oct string, e.g, '777' or 0o777 SETPERMISSIONr)rIrd permissionNr)r'rdmods r1chmod WebHDFS.chmod\s ?5t Lr3cN0nUbX$S'UbX4S'UR"SSUS.UD6 g)zChange owning user and/or groupNownergrouprrIrd)SETOWNERr)r'rdrrr/s r1chown WebHDFS.chownis7  #7O  #7O Ae$A&Ar3c(URSUSUS9 g)z Set file replication factor Parameters ---------- path: str File location (not for directories) replication: int Number of copies of file on the cluster. Should be smaller than number of data nodes; normally 3 on most systems. SETREPLICATIONr)rdrIrrNr)r'rdrrs r1set_replicationWebHDFS.set_replicationrs #$u+ Vr3c &URSSUS9 g)NMKDIRSrrrr'rdr/s r1mkdir WebHDFS.mkdirs 8E 5r3crUSLa!URU5(a [U5eURU5 g)NF)existsFileExistsErrorr)r'rdexist_oks r1makedirsWebHDFS.makedirss/ u T!2!2!$' ' 4r3c &URSSXS9 g)NRENAMEr)rIrd destinationr)r'path1path2r/s r1mv WebHDFS.mvs 8E Ir3c :URSSUU(aSOSS9 g)NDELETEdeletetruefalse)rIrd recursiver)r'rdrr/s r1rm WebHDFS.rms#  )fw  r3c &URU5 gr5)rrs r1rm_fileWebHDFS.rm_files   r3c URU5nSRURU5S[R"S53/5nURUS5n[ R "XF5 SSS5 URXR5 SSS5 g!,(df  N)=f![a< [[5 URU5 SSS5 e!,(df  e=ff=f!,(df  g=f)Nrz.tmp.wb) openjoin_parentsecrets token_hexshutil copyfileobjr BaseExceptionrr`r)r'lpathrpathr/lstream tmp_fnamerstreams r1cp_fileWebHDFS.cp_files YYu $,,u"5w?P?PQS?T>U7V!WXI YYy$/7&&w80 )  0/! /0GGI&110  sM:C3 B*B6B* B' #B**C0C C0 C, (C00C33 DcUR(a-[UR5(aURU5nU$UR(a5URR5Hup#URX#S5nM U$)N)rcallableitemsreplace)r'rkvs r1rVWebHDFS._apply_proxyse ::(4::..zz(+H  ZZ ((*#++A!4+r3) r&rrrr"rr>r#r$rr!) iFNNNNNNFNT)rNNT)rbNTNNFr5)NN)+__name__ __module__ __qualname____firstlineno____doc__strtempfile gettempdirrprrpropertyr7r%rkrv staticmethodr~ classmethodrUrr}rrrrrrrrrrrrrrrrrrrV__static_attributes__ __classcell__r0s@r1r r s/*(%%'(G#H  c7JH$H+ Z 33( TD :, ." "" G MB W6 J  r3r cN^\rSrSrSrU4SjrS SjrSrSrSr Sr S r U=r $) ruiz"A file living in HDFS over webHDFSc >[TU]"X40UD6 UR5nURSS5cUR SS5 URSS5cUR SS5 UR SS5UlUR S5nUR SS5SLaSUR Ul[R RU[[R"555Ulgg)NrsrrirprqF) rrrWrrrsrdtargetosr r(uuiduuid4)r'fsrdr/rpr0s r1rWebHDFile.__init__s ,V, ::mT * 2 JJ}d + ::mT * 2 JJ}d +!::mS9**Y' ::lE *e 3))DK Wc$**,.?@DI 4r3cURRRURURR 5SS0S9nUR 5 g)zWrite one part of a multi-block file upload Parameters ========== final: bool This is the last block, so should complete file, if self.autocommit is True. content-typeapplication/octet-stream)rKrT)r7r>postrbuffergetvaluerc)r'finalrgs r1 _upload_chunkWebHDFile._upload_chunksRggoo"" MM%%'#%?@#  r3cVURR5nSUR;aSup#O Sup#SUS'URR"X#UR 4SS0UD6nURR URS5nS UR;aURRRUS S 0S 9nUR5 URR"S SUR 4SS0UD6nURR URS5Ul gg)zCreate remote file/uploada)APPENDPOST)CREATEPUTr overwritereFrwr:r;)rrDrEN) r/rWrnr7rkrdrVrr>rrcr)r'r/rHrIrgrrs r1_initiate_uploadWebHDFile._initiate_uploads!!# $)) )JB(JB"(F; ggmmB LELVL77'' J(?@ $)) 77??&&>3M"N'D  ! ! #77==6499WuWPVWD GG00j1IJDM r3c[US5n[URU5nX:dXR:agURR SUR XU- SS9nUR 5 SUR;aYURSnURRRURRU55nUR$UR$)Nrr3OPENF)rdoffsetrzrer) maxminr{r7rkrdrcrr>rrVcontent)r'startendrgrrs r1 _fetch_rangeWebHDFile._fetch_rangesE1 $))S! <5II-ggmm 5uu    ${{:.H77??&&tww';';H'EFD<< ;; r3cdURRURUR5 gr5)r7rrdr3r6s r1commitWebHDFile.commits  499dkk*r3cNURRUR5 gr5)r7rrdr6s r1discardWebHDFile.discards  499r3)rrdrsr3r") r#r$r%r&r'rr@rJrTrWrZr.r/r0s@r1rurus+, A"K( +r3ru)loggingr4r rr)r5 contextlibrr urllib.parserr<specrrutilsr r getLoggerrZr rurr3r1rbsZ   ;3   9 %V Vr I$Ir3