9i$!SSKJrJr SSKrSSKrSSKJr SSKJ r SSK J r SSK J r SSKJr SSKJ r "S S \5rg) )AnyDictN)Image) transforms) load_image)ModeKeys)OfaBasePreprocessorc^\rSrSrSr\R 4U4SjjrS\\ \ 4S\\ \ 44Sjr S\\ \ 4S\\ \ 44Sjr S\\ \ 4S\\ \ 44Sjr S rU=r$) OfaVisualGroundingPreprocessorz. OFA preprocessor for visual grounding tasks. c >[[U] "XU/UQ70UD6 URRR SS5UlUR[R:Xa[R"[R"UR/URS9[R"5[R"UR UR"UR$S9/5Ulg[(R"S[(R*"URUR4[(R,R.S9[(R"5[(R"UR UR"S9/5Ulg) zpreprocess the data Args: cfg(modelscope.utils.config.ConfigDict) : model config model_dir (str): model path, mode: preprocessor mode (model mode) num_binsi)max_size)meanstdmax_image_sizec$URS5$)NRGB)convert)images m/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/preprocessors/ofa/visual_grounding.py9OfaVisualGroundingPreprocessor.__init__..3s emmE2) interpolation)rrN)superr __init__cfgmodelgetrmoderTRAINTCompose RandomResizepatch_image_sizeToTensor Normalizerrrpositioning_transformrResizeInterpolationModeBICUBICpatch_resize_transform)selfr model_dirr"argskwargs __class__s rr'OfaVisualGroundingPreprocessor.__init__s= ,   T D48 D">"F"FH##%$$$))B >+D 'rdatareturncUR[R:XaURU5$UR U5$)N)r"rr#_build_train_sample_build_infer_sample)r/r5s r__call__'OfaVisualGroundingPreprocessor.__call__;s4 99 &++D1 1++D1 1rc URXRS5nURup4///[R"XC/5S.nXRSR 5R S5upgp[R"[U5[U5[U5[U 5/5n [R"[U5[U5[U5[U 5//5US'[R"S/5US'[U5[U5- [U 5[U5- -/n [R"U 5US'URX%5upU S SU S S pS R[U SSSURS - -R555nS R[U SSS URS - -R555nS R[U SSS URS - -R555nS R[U SSS URS - -R555nSRUUUU5nURXRSUR 5nUR"R$R'SS5nURU5nUR)U5nUR)USS9n[R*"UR,USS/5nUU [R"S/5UUX- X- U S.nU$)aq Building training samples. step 1. Preprocessing the image input for model's image input. - get the pillow image. - calculate the target boxes using for getting the exact area in the pillow image for input text by input `region_coord`. in training setting, `region_coord` will be a label data. - getting the target image as patch images and do some transforms such as resize, normalize etc. step 2. Preprocessing the text input for model's source text input. - do the str preprocessing to text input by function `pre_caption`. - build the instruction. the default instruction is ` which region does the text " {} " describe?`, `{}` refer to the text input. - tokenize the instruction as source text input. step 3. Preprocessing the patch image boxes for model's target text input. - quantize the coordinate of selected patch images - concatenate the quantization results by blank - tokenize the result above as target text input. step 4. Get the previous output tokens using target item without eos token. Args: data (`Dict[str, Any]`): Input data, should contains the key of `image` `text` and `region_coord`. Return: A dict object, contains source text input, patch images, patch masks with `Tensor([True])` value, target, previous output tokens, width scale ratio, height scale ratio and region coordinate. r)boxeslabelsareasize region_coord,r=rr>r?r@r zz {} {} {} {}textprompt, which region does the text " {} " describe?F)add_bosNT)source patch_image patch_masktargetprev_output_tokensw_resize_ratioh_resize_ratiorA) get_img_pil column_mapr@torchtensorstripsplitfloatnparrayr*formatintrround pre_captionmax_src_lengthrr r! tokenize_textcatbos_item)r/r5rwh boxes_targetx0y0x1y1regionr?rK patch_boxesresize_hresize_wquant_x0quant_y0quant_x1quant_y1rA src_captionrFrEsrc_item target_itemprev_output_itemsamples rr82OfaVisualGroundingPreprocessor._build_train_sampleAs?>  oog&>!?@zzLL!(  oon=>DDFLL uRy%)U2Yb JK % BirE"IBi ! W"$1# XrU2Y&59uRy+@AB$||D1 V#'#=#= $! (03[5H5K($$ W%a(+t}}q/@AHHJ KM$$ W%a(+t}}q/@AHHJ KM$$ W%a(+t}}q/@AHHJ KM$$ W%a(+t}}q/@AHHJ KM$++Hh,46 &&tOOF,C'D'+':':< ## DF}}[)%%d+(( %))  99dmm["5E%FG&,,v.!"2&l&l"   rc6URXRS5nURup4URU5n[R "UR U- 5n[R "UR U- 5nURXRSUR5nURRRSS5n U RU5n URU 5n U U[R "S/5UUS.n SUR;asURSU;a`UURSR5RS5upnn[!U 5[!U5[!U5[!U5/U S 'U $) ax Building inference samples. step 1. Preprocessing image input for model's image input. - get pillow image from data. - do some transforms to the pillow image, such as resize, normalize etc. step 2. Preprocessing the text input for model's text input. - do the str preprocessing to text input by function `pre_caption`. - build the instruction. the default instruction is ` which region does the text " {} " describe?`, `{}` refer to the text input. - tokenize the instruction as source text input. step 3. Whether or not to add label data which refer to a region coordinate in this task. Args: data (`Dict[str, Any]`): Input data, should contains the key of `image` `text`. Return: A dict object, contains source text input, patch images, patch masks with `Tensor([True])` value, width scale ratio, height scale ratio and label. rrErFrGT)rJrKrLrOrPrArBlabel)rQrRr@r.rSrTr'r]r^rr r!rZr_rUrVrW)r/r5rrbrcrKrOrPrqrFrErrrurerfrgrhs rr92OfaVisualGroundingPreprocessor._build_infer_samplesk0  oog&>!?@zz11%8 d&;&;a&?@d&;&;a&?@&&tOOF,C'D'+':':< ## DF}}[)%%d+&,,v.,,   T__ ,2 #'2(!/116s BB$Ry%)U2Yb JF7O r)rr.r*)__name__ __module__ __qualname____firstlineno____doc__r INFERENCErrstrrr:r8r9__static_attributes__ __classcell__)r3s@rr r s((%N2T#s(^2S#X2 RS#XR4S>Rh0S#X04S>00rr )typingrrnumpyrXrSPILr torchvisionrmodelscope.preprocessors.imagermodelscope.utils.constantrbaser utilsr$r rrrs/ "5.%"v%8vr