*pjNSrSSKrSSKJr SSKJr SSKJrJrJ r J r J r SSK J r SSKJr SSKJr SS KJr SS KJr SS KJr SS KJrJr SS KJr SSKJr SSKJ r J!r!J"r" SSK#J$r$ SSK%J&r& SSK'J(r( SSK)J*r*J+r+ \ "S\SSS9r,Sr-Sr.S\/S\0\/S\/S\1\ 4Sjr2Sr3Sr4Sr5S r6S!r7S"\8S#\9\*S$4S%\ S\9\*S$44S&jr:S'S(.S"\8S)\1\9\/\;4S#\9\*S$4S%\ S*\\4SSS.S/S'S'S0.S"\8S1\S2\?S3\0\/S-S4\0\/S-S5\/S6\/S*\ run_pipeline() -> _run_pipeline_core() -> _make_fallback_or_none().  zL is excluded by include_encodings/exclude_encodings; returning encoding=None) stacklevelg?N)rrr)warningswarn UserWarningrr)rr!r"s Y/var/www/html/pdf-tiff/venv/lib/python3.13/site-packages/chardet/pipeline/orchestrator.py_make_fallback_or_noner,8sO l!H<(K L   ~ X$ N OOg?gffffff?datavalid_candidates.ctxcD/nUGH nUR(a[XU5nXRRUR'U[:aMEUR c0[ U5[ URS[55- UlUR [:aM[XX"R S9nXbRUR'U[:aMUR [:a[XU5nU[:aMUR!U5 GM [#U5$)aEliminate CJK multi-byte candidates that lack genuine multi-byte structure. Four checks are applied in order to each multi-byte candidate: 1. **Structural pair ratio** (valid_pairs / lead_bytes) must be >= ``_CJK_MIN_MB_RATIO``. Catches files with many orphan lead bytes. 2. **Minimum non-ASCII byte count**: the data must contain at least ``_CJK_MIN_NON_ASCII`` bytes > 0x7F. Tiny files with 1-5 high bytes can accidentally form perfect pairs and score 1.0 structurally. 3. **Byte coverage** (non-ASCII bytes in valid multi-byte sequences / total non-ASCII bytes) must be >= ``_CJK_MIN_BYTE_COVERAGE``. Latin text has many high bytes that are NOT consumed by multi-byte pairs; genuine CJK text has nearly all high bytes accounted for. 4. **Lead byte diversity**: the number of distinct lead byte values in valid pairs must be >= ``_CJK_MIN_LEAD_DIVERSITY``. Genuine CJK text draws from a wide repertoire of lead bytes; European false positives cluster in a narrow band (e.g. 0xC0-0xDF for accented Latin). Returns the filtered candidate list. Structural scores are cached in ``ctx.mb_scores`` for reuse in Stage 2b. N)non_ascii_count) is_multibyter mb_scoresname_CJK_MIN_MB_RATIOr5len translater_CJK_MIN_NON_ASCIIr mb_coverage_CJK_MIN_BYTE_COVERAGE_CJK_DIVERSITY_MIN_NON_ASCIIr_CJK_MIN_LEAD_DIVERSITYappendtuple)r1r2r3gatedencmb_score byte_coveragelead_diversitys r+_gate_cjk_candidatesrHks:!#E   /3?H&.MM#(( #++""*&)$i#dnnT:6V2W&W#""%77;30C0CM)6OOCHH %55""&BB!= 0.95) receive a confidence boost proportional to coverage. When nearly all non-ASCII bytes form valid multi-byte pairs, the structural evidence is strong and should increase the candidate's ranking relative to single-byte alternatives whose bigram models may score higher on small samples. Note: boosted confidence values may exceed 1.0 and are used only for relative ranking among candidates. ``run_pipeline`` clamps all confidence values to [0.0, 1.0] before returning to callers. c3B># UHupUT;dM TUv M g7fN).0r8_sc enc_lookups r+ /_score_structural_candidates..s&*;YTtz?Q 4*;s  c3J# UHoR(aMUv M g7frN)r6)rPes r+rSrTsJ#3a>>#3s# #NrIgffffff?cUR$rN)rxs r+._score_structural_candidates..sq||r-Tkeyreverse)r6r8rBlistr_STAT_SCORE_MAX_BYTESrr=getrArrrr sort) r1rKr2r3rJrVvalid_mb single_byteresultsboostedrcoveragerRs @r+_score_structural_candidatesrks#4,++a~~  ++J*;HJ#3JJK '' ( %h % %% G&(G ;<::3??&&qzz373 t  NNJJ H =qzz1;;  NN1  LL+TL: N9+s D4D4resultcURbU$URbSOSn[URURURU5$)zXDefault ``mime_type`` to ``text/plain`` (text) or ``application/octet-stream`` (binary).z text/plainr)r rrrr)rlmimes r+_with_default_mimerosD # !??6<.s'GJSJsrt)rwrsrWc US$)NrXrOrZs r+r\$_run_pipeline_core..psQqTr-Tr^rrI)ru)$r r:r frozensetr,r rrr rrr r _BINARY_RESULTrrrrr rrrHr6r7rcr8rrArd _STRUCTURAL_CONFIDENCE_THRESHOLDrkrrbrarrB)r1rvrwrqrrrsrtrJrur3 candidatesr! bom_resultutf1632_result escape_result magic_result utf8_precheckascii_precheck markup_resultr2rKrDscore_ best_scorerg stat_datas r+_run_pipeline_corersb  C%>TY)>O   D  ARSJ''GJ'GGG % +A  D!J*"5"5"@| -T2N!n&=&=&H +40M!  " " .  " "g - %L~ %M"$'N   " d 0 *$/M ]%;%;w%F  %&&-*@*@@&&'1 !**!,,!**!++  0mWM !n&=&=&H ]%;%;w%F*$; %&7BUVV,D2BCH %&7BUVV24    MM%%chh/E}0sC@s{!((#((E):;  >4@)!, : 9 92! ) G*'?++,IE*:$;,WG %&7BUVV tWo NNr-c [UUUUUUUUUS9 n [X 5n U V s/sHAn U RS:Xa,[U RU R SU R 5OU PMC n n U V s/sHn [U 5PM n n U (d Sn [U 5eU V s/sHUn U R S:a@[U R[U R S5U RU R 5OU PMW sn $s sn fs sn fs sn f)aRun the full detection pipeline. :param data: The raw byte data to analyze. :param encoding_era: Filter candidates to a specific era of encodings. :param max_bytes: Maximum number of bytes to process. :param include_encodings: If not ``None``, only return these encodings. :param exclude_encodings: If not ``None``, never return these encodings. :param no_match_encoding: Encoding returned when no candidate survives. :param empty_input_encoding: Encoding returned for empty input. :param full_ranking: When ``True``, statistical scoring evaluates every candidate so the returned list is complete (``detect_all`` needs this). When ``False``, candidates that provably cannot affect the top of the ranking may be omitted from the tail. :param input_truncated: Pass ``True`` when *data* is already a truncated view of the caller's input (``UniversalDetector`` caps its buffer at ``max_bytes``, which this function cannot see from ``len(data)`` alone). Truncation by the ``max_bytes`` slice here is detected either way; the flag only ever widens it. :returns: A list of :class:`DetectionResult` sorted by confidence descending. )rurqrrrsrtrJzxxNz/pipeline must always return at least one resultg?) rrrrrrr ro RuntimeErrormin) r1rvrwrqrrrsrtrJrurgrimsgs r+ run_pipeliners%@! '+++1! GT+G A ::   ALL$ D     /66g!!$gG6 ?3  A <<#   C c$:AJJ T     7 sAC=/DAD)B__doc__r(chardet._utilsr chardet.enumsrchardet.pipelinerrrrr chardet.pipeline.asciir chardet.pipeline.binaryr chardet.pipeline.bomr chardet.pipeline.escaper chardet.pipeline.languagerchardet.pipeline.magicrchardet.pipeline.markuprrchardet.pipeline.postprocessrchardet.pipeline.statisticalrchardet.pipeline.structuralrrrchardet.pipeline.utf8rchardet.pipeline.utf1632rchardet.pipeline.validityrchardet.registryrrr~rrbstrr}rar,r9r<r>r@r?bytesrBrHfloatboolrkrointrrrOr-r+rs,%0-+:4/R<9 .<89 ' ( $( PP s^PP / P2   "3 3L#-.3 3 <  3x 5 5E#u*-.5L#-.5  5  5 /5pVV?V'hO 04/3% '!hO hOhOhO !~, hO !~, hOhOhOhOhO /hO\'A 04/3% '!A AAA !~, A !~, AAAAA /Ar-