9ilSSKJrJr SSKrSSKrSSKJr SSKJr SSKJ r SSK J r SSK Jr SS KJr S S jr"S S \5rg))AnyDictN) transforms)InterpolationMode) functional)convert)ModeKeys)OfaBasePreprocessorcURS5nURup4U(aSupVOvX4:a9[SU5n[S[XU- -55nX- S-nX- U- nSupO8[SU5n[S[XU- -55nX- S-n X- U - n Supx[R "UXV4[ RS9n U(aw[R"5"U 5RSSS 9n [R"5"[R"U S S 95n U RupeX- S-nX- U- nSup[R"U W WW W/S S 9n U RX4:XdeU $) z& Image resize function for OCR tasks. RGB)@ir)rr) interpolation)dimedge)padding padding_mode)rsizemaxintFresizerBICUBICrToTensorchunk ToPILImagetorchcatpad) imgpatch_image_size is_documentwidthheight new_height new_widthtopbottomleftrightimg_new img_splits l/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/preprocessors/ofa/ocr_recognition.py ocr_resizer2s ++e CHHME ( I ?B 01IR%5%%H!IJJ#0Q6C%2S8FKD%R!12JB$4$G HII$0Q6D$047EKChh '//G '')'2888C '')%))I2*FG '  ,2!.4 ee$UF3&JG <<,? ?? ? Nc^\rSrSrSr\R 4U4SjjrS\\ \ 4S\\ \ 44Sjr S\\ \ 4S\\ \ 44Sjr S\\ \ 4S\\ \ 44Sjr S rU=r$) OfaOcrRecognitionPreprocessor;z- OFA preprocessor for OCR recognition tasks. c >^[[T] "XU/UQ70UD6 [R"U4Sj[R "5[R "TRTRS9/5Tl g)zpreprocess the data Args: cfg(modelscope.utils.config.ConfigDict) : model config model_dir (str): model path, mode: preprocessor mode (model mode) cv>[UTRTRRR SS5S9$)Nr&F)r&)r2r%cfgmodelget)imageselfs r18OfaOcrRecognitionPreprocessor.__init__..Qs/*%% HHNN..}eDFr3)meanstdN) superr5__init__rComposer Normalizer@rApatch_resize_transform)r=r9 model_dirmodeargskwargs __class__s` r1rC&OfaOcrRecognitionPreprocessor.__init__@s~ +   T D48 D : ' #r3datareturncUR[R:XaURU5$UR U5$)N)rHr TRAIN_build_train_sample_build_infer_sample)r=rMs r1__call__&OfaOcrRecognitionPreprocessor.__call__Ys4 99 &++D1 1++D1 1r3c&URU5nUSnUR5R5nSRUSUR5nUR USS9US'[ R"URUSSS/5US'U$) a Building training samples. step 1. Preprocess the data using the logic of `_build_infer_sample` and make sure the label data in the result. step 2. Preprocess the label data. Contains: - do tripe to the label value. - tokenize the label as `target` value without `bos` token. - add `bos` token and remove `eos` token of `target` as `prev_output_tokens`. Args: data (`Dict[str, Any]`): Input data, should contains the key of `image`, `prompt` and `label`, the former refers the image input data, and the later refers the text input data the `label` is the supervised data for training. Return: A dict object, contains source, image, mask, label, target tokens, and previous output tokens data. label NF)add_bostargetrprev_output_tokens) rRstripsplitjoinmax_tgt_length tokenize_textr!r"bos_item)r=rMsamplerYtarget_token_lists r1rQ1OfaOcrRecognitionPreprocessor._build_train_sample_s&))$/"LLN002+,@T-@-@AB--fe-Dx',yy ]]F8,Sb1 2(4#$ r3cURXRS5nURU5nURRR SS5nUR U5nUU[R"S/5S.nSUR;aHURSU;a5XRSn[R"S[US55US 'U$) a& Building inference samples. step 1. Get the pillow image. step 2. Do some transforms for the pillow image as the image input, such as resize, normalize, to tensor etc. step 3. Tokenize the prompt as text input. step 4. Determine Whether or not to add labels to the sample. Args: data (`Dict[str, Any]`): Input data, should contains the key of `image` and `prompt`, the former refers the image input data, and the later refers the text input data. Return: A dict object, contains source, image, image patch mask and label data. r<promptu图片上的文字是什么?T)source patch_image patch_masktextNFKCzzh-hansrV) get_img_pil column_maprFr9r:r;r_r!tensor unicodedata2 normalizer)r=rMr<rgreinputsrarYs r1rR1OfaOcrRecognitionPreprocessor._build_infer_sample{s   oog&>!?@11%8 ##H.LM##F+&,,v.  T__ $)@D)H//&12F*44 24F7O r3)rF)__name__ __module__ __qualname____firstlineno____doc__r INFERENCErCrstrrrSrQrR__static_attributes__ __classcell__)rKs@r1r5r5;s(( 22T#s(^2S#X2 S#X4S>8S#X4S>r3r5)F)typingrrr!rn torchvisionrtorchvision.transformsrrrzhconvrmodelscope.utils.constantr baser r2r5r3r1rs5 "42.%)X^$7^r3