9ilSSKrSSKJr SSKJr SSKJrJrJrJ r J r J r SSK r SSKJr SSKJr SS KJr SS KJr SS KJr SS KJr S SKJr SSKJrJrJr SSKJ r J!r!J"r"J#r# \RH\"S5"SS\555r%g)N)deepcopy)sleep)AnyDictListOptionalTupleUnion)logging)pipeline_requires_extra)MarkDownBatchSampler) benchmark) HPIConfig)PaddlePredictorOption) BasePipeline)DocumentResult LatexResultMarkdownResult)split_original_textssplit_text_recursivetranslate_code_blocktranslate_html_blocktranscA^\rSrSrSrS/rSFS\S\S\S\ S \ \ \\\ 4\ 4S \ S S4U4S jjjrS rS\S S4SjrS\S S4SjrSGSjrSHS\ \\\\R,\\R,4S\ \ S\ \ S\ \ S\ \ S\ \ S\ \ S\ \ S\ \ S\ \ \\4S\ \ S\ \ \\\\4\4S\ \S\ \S\ \S \ \S!\ \S"\ \S#\ \S$\ \S%\ \S&\ \S'\ \S(\ \S)\ \S*\ S+\ S,\ S-\ S.\ S/\ S \4@S0jjrS1rS2rSIS3\\S4\S5\S6\S7\S8\S9\S:\S;\S<\S=\4S>jjrS?\S \4S@jr SA\S \4SBjr!SC\S \4SDjr"SEr#U=r$$)JPP_DocTranslation_Pipeline&z PP_ DocTranslation_Pipeline zPP-DocTranslationNconfigdevice pp_optionuse_hpip hpi_configinitial_predictorreturnc>[TU]X#XES9 USUlXlUR SS5UlSUlSUlU(a"URU5 URU5 [5Ul g)aInitializes the PP_Translation_Pipeline. Args: config (Dict): Configuration dictionary containing various settings. device (str, optional): Device to run the predictions on. Defaults to None. pp_option (PaddlePredictorOption, optional): PaddlePredictor options. Defaults to None. use_hpip (bool, optional): Whether to use the high-performance inference plugin (HPIP) by default. Defaults to False. hpi_config (Optional[Union[Dict[str, Any], HPIConfig]], optional): The default high-performance inference configuration dictionary. Defaults to None. initial_predictor (bool, optional): Whether to initialize the predictor. Defaults to True. )r"r#r$r% pipeline_nameuse_layout_parserTN) super__init__r)r!getr*layout_parsing_pipelinechat_botinintial_visual_predictorinintial_chat_predictorrmarkdown_batch_sampler)selfr!r"r#r$r%r& __class__s v/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/paddlex/inference/pipelines/pp_doctranslation/pipeline.pyr,#PP_DocTranslation_Pipeline.__init__/s}.   $O4 !',?!F'+$   * *6 2  ( ( 0&:&<#cTURbURR5 gg)N)r.close)r3s r5r9 PP_DocTranslation_Pipeline.closeWs%  ' ' 3  ( ( . . 0 4r7cURSS5UlUR(a:URS05RSSS05nURU5Ulg)z Initializes the visual predictor with the given configuration. Args: config (dict): The configuration dictionary containing the necessary parameters for initializing the predictor. Returns: None r*T SubPipelines LayoutParserpipeline_config_errorz)config error for layout_parsing_pipeline!N)r-r*create_pipeliner.)r3r!layout_parsing_configs r5r04PP_DocTranslation_Pipeline.inintial_visual_predictor[sb"(,?!F  ! !$*JJ~r$B$F$F(*UV% !,0+?+?@U+VD (r7cSSKJn URS05RSSS05nU"U5UlSSKJn URS05RS05RS S S 05nU"U5Ulg ) z Initializes the chat predictor with the given configuration. Args: config (dict): The configuration dictionary containing the necessary parameters for initializing the predictor. Returns: None rcreate_chat_bot SubModulesLLM_Chatchat_bot_config_errorzconfig error for llm chat bot!)create_prompt_engineeringPromptEngneeringTranslate_CommonTextpe_config_errorz%config error for translate_pe_config!N)rDr-r/rH translate_pe)r3r!rDchat_bot_configrHtranslate_pe_configs r5r12PP_DocTranslation_Pipeline.inintial_chat_predictoros ' **\26::  $&F G (8 0 JJ|R ( S#R ( S&"$KL 66IJr7c0[R"S5 g)NzPP-Translation Pipeline do not support to call `predict()` directly! Please invoke `visual_predict`, `build_vector`, `chat` sequentially to obtain the result.)r error)r3argskwargss r5predict"PP_DocTranslation_Pipeline.predicts  m  r7inputuse_doc_orientation_classifyuse_doc_unwarpinguse_textline_orientationuse_seal_recognitionuse_table_recognitionuse_formula_recognitionuse_chart_recognitionuse_region_detectionlayout_threshold layout_nmslayout_unclip_ratiolayout_merge_bboxes_modetext_det_limit_side_lentext_det_limit_typetext_det_threshtext_det_box_threshtext_det_unclip_ratiotext_rec_score_threshseal_det_limit_side_lenseal_det_limit_typeseal_det_threshseal_det_box_threshseal_det_unclip_ratioseal_rec_score_thresh#use_wired_table_cells_trans_to_html&use_wireless_table_cells_trans_to_htmluse_table_orientation_classify use_ocr_results_with_table_cellsuse_e2e_wired_table_rec_model use_e2e_wireless_table_rec_modelc +# URS:Xa[R"S5 SS0v URc1[R"S5 UR UR 5 URR"U40SU_SU_SU_S U_S U_S U_S U_S U _SU _SU _SU _SU _SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_S U_S!U_S"U_S#U_6H n!S$U!0n"U"v M g7f)%a This function takes an input image or a list of images and performs various visual prediction tasks such as document orientation classification, document unwarping, general OCR, seal recognition, and table recognition based on the provided flags. Args: input (Union[str, list[str], np.ndarray, list[np.ndarray]]): Input image path, list of image paths, numpy array of an image, or list of numpy arrays. use_doc_orientation_classify (Optional[bool]): Whether to use document orientation classification. use_doc_unwarping (Optional[bool]): Whether to use document unwarping. use_textline_orientation (Optional[bool]): Whether to use textline orientation prediction. use_seal_recognition (Optional[bool]): Whether to use seal recognition. use_table_recognition (Optional[bool]): Whether to use table recognition. use_formula_recognition (Optional[bool]): Whether to use formula recognition. use_region_detection (Optional[bool]): Whether to use region detection. layout_threshold (Optional[float]): The threshold value to filter out low-confidence predictions. Default is None. layout_nms (bool, optional): Whether to use layout-aware NMS. Defaults to False. layout_unclip_ratio (Optional[Union[float, Tuple[float, float]]], optional): The ratio of unclipping the bounding box. Defaults to None. If it's a single number, then both width and height are used. If it's a tuple of two numbers, then they are used separately for width and height respectively. If it's None, then no unclipping will be performed. layout_merge_bboxes_mode (Optional[str], optional): The mode for merging bounding boxes. Defaults to None. text_det_limit_side_len (Optional[int]): Maximum side length for text detection. text_det_limit_type (Optional[str]): Type of limit to apply for text detection. text_det_thresh (Optional[float]): Threshold for text detection. text_det_box_thresh (Optional[float]): Threshold for text detection boxes. text_det_unclip_ratio (Optional[float]): Ratio for unclipping text detection boxes. text_rec_score_thresh (Optional[float]): Score threshold for text recognition. seal_det_limit_side_len (Optional[int]): Maximum side length for seal detection. seal_det_limit_type (Optional[str]): Type of limit to apply for seal detection. seal_det_thresh (Optional[float]): Threshold for seal detection. seal_det_box_thresh (Optional[float]): Threshold for seal detection boxes. seal_det_unclip_ratio (Optional[float]): Ratio for unclipping seal detection boxes. seal_rec_score_thresh (Optional[float]): Score threshold for seal recognition. use_wired_table_cells_trans_to_html (bool): Whether to use wired table cells trans to HTML. use_wireless_table_cells_trans_to_html (bool): Whether to use wireless table cells trans to HTML. use_table_orientation_classify (bool): Whether to use table orientation classification. use_ocr_results_with_table_cells (bool): Whether to use OCR results processed by table cells. use_e2e_wired_table_rec_model (bool): Whether to use end-to-end wired table recognition model. use_e2e_wireless_table_rec_model (bool): Whether to use end-to-end wireless table recognition model. **kwargs (Any): Additional settings to extend functionality. Returns: dict: A dictionary containing the layout parsing result. Fz1The models for layout parser are not initialized.rRNzGThe layout parsing pipeline is not initialized, will initialize it now.rXrYrZr[r\r]r^r_r`rarbrcrdrerfrgrhrirmrjrkrlrnrorprqrrrsrtrulayout_parsing_result)r*r rRr.warningr0r!rU)#r3rWrXrYrZr[r\r]r^r_r`rarbrcrdrerfrgrhrirjrkrlrmrnrorprqrrrsrtrurTrwvisual_predict_ress# r5visual_predict)PP_DocTranslation_Pipeline.visual_predictsb  ! !U * MMM NOP P  ' ' / OOY   * *4;; 7%)%A%A%I%I  & )E & 0 & &> & "6 & #8 & %< & #8 & "6 & . & " & !4 & &> & %< & !4 & ,! & "!4# & $#8% & &#8' & (!4) & *%<+ & ,!4- & .,/ & 0#81 & 2#83 & 41T5 & 64Z7 & 8,J9 & :.N; & <+H= & >.N? & !F()>" % $K & sC0C2c/nURRU5HBnURSnURSnUSUSS.nUR [ U55 MD U$)NrTT) input_path page_indexmarkdown_textspage_continuation_flags)r2sample instances input_pathsappendr)r3rWmarkdown_info_listmarkdown_samplemarkdown_contentr~ markdown_infos r5load_from_markdown-PP_DocTranslation_Pipeline.load_from_markdownsu#::AA%HO.88; (44Q7J(""2+7 M  % %n]&C D I"!r7c /nSn[R"S[U5S35 [R"S5 [U5GHupgUupUS:XasUR 5(a'UR U"UR 555 Sn[R"SUS-S[U5S 35 [ XX45 M[U 5U:aUS :Xa[U5[U 5-U:a US U -- nMUR 5(aN[R"SUS-S[U5S 35 UR U"UR 555 U nGM%[R"SUS-S[U5S 35 UR 5(a'UR U"UR 555 SnUS :XaUR [XU55 GMUS :XdUS :Xa[XX45 GM[SU35e UR 5(a%UR U"UR 555 S RU5$)au Chunks the given markdown blocks into smaller chunks of size `chunk_size` and translates them using the given translate function. Args: md_blocks (list): A list of tuples representing each block of markdown content. Each tuple consists of a string indicating the block type ('text', 'code') and the actual content of the block. chunk_size (int): The maximum size of each chunk. translate_func (callable): A callable that accepts a string argument and returns the translated version of that string. Returns: str: A string containing all the translated chunks concatenated together with newlines between them. rLzSplit the original text into z blockszStarting translation...codezTranslating block r/z...text text_with_htmlhtmlzUnknown block type: ) r infolen enumeratestriprrrr ValueErrorjoin) r3 md_blocks chunk_sizetranslate_functranslation_resultschunkidxblock block_type block_contents r5chunk_translate*PP_DocTranslation_Pipeline.chunk_translate%s! 4S^4DGLM ./#I.JC(- %JV#;;=='..~ekkm/LME 1#a%#i.9IMN$!~]#j0Z65Iu:M 22Z?Vm33E{{}} '9#a%#i.AQQT%UV+22>%++-3PQ)E 1#a%#i.9IMN;;=='..~ekkm/LME''..,]W #33zV7K(%>%';J<%HIIC/F ;;==  & &~ekkm'D E{{.//r7ori_md_info_listtarget_languagertask_description output_format rules_strfew_shot_demo_text_contentfew_shot_demo_key_value_listglossaryllm_request_intervalrNc +^^^^^^^^ ^# TRc1[R"S5 TRTR5 U bSSKJn U "U 5mO TRm[U[5(a'USRS5bTRU5/n[T [5(d [T 5m [U [5(d U bS5eSnU bJU R5H6unn[U[5(aS RU5nXS US 3- nM8 US:waTcUmO TS - mTU- mUUUU UUUUU4 S jnTRR!STTTTTTS 9n[#U5nUU:aUU- nO[%SUSUS35eUHBnUSn['U5nTR)UUU5n[+TUSUSUSUS.5v MD g7f)a Translate the given original text into the specified target language using the configured translation model. Args: ori_md_info_list (List[Dict]): A list of dictionaries containing information about the original markdown text to be translated. target_language (str, optional): The desired target language code. Defaults to "zh". chunk_size (int, optional): The maximum number of characters allowed per chunk when splitting long texts. Defaults to 5000. task_description (str, optional): A description of the task being performed by the translation model. Defaults to None. output_format (str, optional): The desired output format of the translation result. Defaults to None. rules_str (str, optional): Rules or guidelines for the translation model to follow. Defaults to None. few_shot_demo_text_content (str, optional): Demo text content for the translation model. Defaults to None. few_shot_demo_key_value_list (str, optional): Demo text key-value list for the translation model. Defaults to None. glossary (Dict, optional): A dictionary containing terms and their corresponding definitions. Defaults to None. llm_request_interval (float, optional): The interval in seconds between each request to the LLM. Defaults to 0.0. chat_bot_config (Dict, optional): Configuration for the chat bot used in the translation process. Defaults to None. **kwargs: Additional keyword arguments passed to the translation model. Yields: MarkdownResult: A dictionary containing the translation result in the target language. Nz;The LLM chat bot is not initialized,will initialize it now.rrCrrzglossary must be a dictrLu或z:  c > [T5 T RRUT T TTTTS9nTRUS9R SS5nSU;a [ S5eUc [ S5eUR SS5R5nU$)z Translate the given text using the configured translation model. Args: text (str): The text to be translated. Returns: str: The translated text in the target language.  original_textlanguagerrrrr)promptcontentrLz<>zThe translation did not reach the end. This may happen if your chunk_size is too large. Please reduce chunk_size and try again.z#The call to the large model failed.)rrMgenerate_promptgenerate_chat_resultsr- Exceptionreplacerstrip) rr translater/rrrrrr3rrs r5r.translate_funcs & '&&66"(!1+#+E-I7F!66f6EII)UWXI )o  EFF!)))R8??AI r7rz:Chunk size should be greater than the base prompt length (z ), but got .rr~r)rr~rrr)r/r rxr1r!rLrD isinstancelistr-concatenate_markdown_pagesfloatdictitemsrrMrrrrrr)r3rrrrrrrrrrrNrTrD glossary_strkvrbase_prompt_contentbase_prompt_lengthori_mdoriginal_textsrtarget_language_textsr/s` ` ````` ` @r5r$PP_DocTranslation_Pipeline.translate^s4F == OOM   ( ( 5  & *&7H}}H ' . . #'' 5A!% ? ?@P QR .66#()=#> (D))X-=X?XX=   (1a&& 1 A#Rs" - ) 2 +3/;,,4,, <,  @#//??$-''A)E@ !!45 * *#&88JLM_L``klvkwwxy 'F#$45N,^Lr(:RT%034D0EC()!,2&-HH/1FG%-HH/1FG& -0E"cC0@,A&AAN"*:&;;N&3/?+@"@@4 :I!R(*<8'3,  011r7 word_listc"[U5S:Xa [S5e/n/n[U5HNupEUSH#n[U5nXGS'UR U5 M% USHnUR U5 M MP [ UUSSUS.5$)z Concatenate Word content from multiple pages into a single document. Args: word_list (list): A list containing Word data for each page. Returns: tuple: A tuple containing the processed Word document. rz The length of word_list is zero. word_blocksrimagesr~)rr~r)rrrrrr) r3r merged_blocksimagepage_idx page_blocksr block_copyimg_objs r5concatenate_word_pages1PP_DocTranslation_Pipeline.concatenate_word_pages-s y>Q ?@ @ %.y%9 !H$]3%e_ +3<($$Z04'x0 W%1 &:,'l<8   r7latex_info_listc"[U5S:Xa [S5e/n/n[U5HNupEUSH#n[U5nXGS'UR U5 M% USHnUR U5 M MP [ UUUSSS.5$)z Concatenate LaTeX content from multiple pages into a single document. Args: latex_info_list (list): A list containing LaTeX data for each page. Returns: tuple: A tuple containing the processed LaTeX document. rz&The length of latex_info_list is zero. latex_blocksrrr~)rrr~)rrrrrr) r3rr merged_imagesrrrrrs r5concatenate_latex_pages2PP_DocTranslation_Pipeline.concatenate_latex_pagesMs  1 $EF F  %.%? !H$^4%e_ +3<($$Z05 'x0$$W-1 &@ -'-a0>   r7)r/r!r.r2r)rMr*)NNFNF)r'N)NNNNNNNNNNNNNNNNNNNNNNNNFFTTFT) zhi NNNNNNgN)%__name__ __module__ __qualname____firstlineno____doc__entitiesrstrrboolrr rrr,r9rr0r1rUrnpndarrayrr intrzrrrrtuplerrr__static_attributes__ __classcell__)r4s@r5rr&s $$H +/AE"'&=&=&=) &=  &= U4S>9#<=> &= &= &=&=P1(dt>8<,037/3042604/39=%)QU2615-1+//3151515-1+//31515497 @% #+4. @% 'tn @% (~@%"*$@% (~@%'tn@%#5#56@%TN@%&eE53F,L&MN@%#+3-@%"*#@% &c]!@%""%#@%$&e_%@%& ('@%( ()@%*"*#+@%,&c]-@%."%/@%0&e_1@%2 (3@%4 (5@%6.27@%8159@%:)-;@%<+/=@%>(,?@%@+/A@%D E@%D "70x $ $!*.,0&) $Lt*LL L  L  LL%(L'*LL$LL\?2?2?2B   @ t   r7r)&rcopyrtimertypingrrrrr r numpyrutilsr utils.depsr common.batch_samplerrutils.benchmarkr utils.hpirutils.pp_optionrbaserresultrrrrrrr time_methodsrr7r5r so ::28("4?? !D  D  "D  r7