9iRSSKJrJr SSKrSSKrSSKJr SSKJ r "SS\ 5r g))AnyDictN)ModeKeys)OfaBasePreprocessorc^\rSrSrSr\R 4U4SjjrS\\ \ 4S\\ \ 44Sjr S\\ \ 4S\\ \ 44Sjr S\\ \ 4S\\ \ 44Sjr S rU=r$) OfaSudokuPreprocessor z# OFA preprocessor for sudoku tasks c>[[U] "XU/UQ70UD6 URRR SS5UlURR SS5UlURR SS5UlUR(Ga/Ul /Ul [S5Hn[S5HnURRUS-5 URRUS-5 US :XaUS :XaMMURRS 5 URRS 5 M M [R"UR5Ul [R"UR5Ul [R"UR!UR 55n[R""URU/5n [R""URU/5n [R""UR$XR&/5Ul [R""UR$XR&/5Ul g g ) zpreprocess the data Args: cfg(modelscope.utils.config.ConfigDict) : model config model_dir (str): model path, mode: preprocessor mode (model mode) promptz solve the sudoku . seg_embeddingFmax_struct_length rrN)superr __init__cfgmodelgetinstruction_textr rinput_puzzle_rowinput_puzzle_colrangeappendtorchtensor zeros_like tokenize_textcatbos_itemeos_item) selfr model_dirmodeargskwargsidxjdx instruct_segrr __class__s c/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/modelscope/preprocessors/ofa/sudoku.pyrOfaSudokuPreprocessor.__init__s #T3CD E59 E=C E!% 2 283H!J!XX\\/5A!%.A3!G    $&D !$&D !Qx 8C))00q9))00q91H--44Q7--44Q7 $ %*LL1F1F$GD !$)LL1F1F$GD ! ++""4#8#89;L$yy$*?*?)NO $yy$*?*?)NO $)II 0--@%BD !$)II 0--@%BD !' datareturncUR[R:XaURU5$UR U5$)N)r%rTRAIN_build_train_sample_build_infer_sample)r#r/s r,__call__OfaSudokuPreprocessor.__call__:s4 99 &++D1 1++D1 1r.cBURU5nUSnUR5R5R5nSR USUR 5nUR USS9US'[R"URUSSS/5US'U$) z build sample for training tasks. step 1. execute the `_build_infer_sample` function to get a batch sample for inference. step 2. process the label data for training. label NF)add_bostargetprev_output_tokens) r4lowerstripsplitjoinmax_tgt_lengthrrr r!)r#r/sampler;target_token_lists r,r3)OfaSudokuPreprocessor._build_train_sample@s))$/"LLN00288:+,@T-@-@AB--fe-Dx',yy ]]F8,Sb1 2(4#$ r.cXSUR;aSU;dS5eXRSnSRUR5R5R 5SUR 5nUR X R-5nUSURUR -nSUS.nUR(aURUS'URUS'S UR;a6URS U;a#S RXRS 5US 'U$) a build sample for inference tasks. step 1. Get the input random masked sudoku text input, which shold be generated like below pseudo code. >>> sudo = np.random.randint(1, 9, size=(9, 9)) # a pseudo sudoku >>> sudo_text = " | ".join(" : ".join(str(c) for c in row) \ >>> for row in sudo) step 2. Limit the length, tokenize the input text and add the bos token to the front of the input as source input. step 3. Add a pseodo ids for every input. textz;there must be `text` column in task key map and source datar9Ng)idsourceseg_row_tokensseg_col_tokenssolutionz {}r8) column_maprAr>r?r@rrrmax_src_lengthr rrformat)r#r/rGsrc_itemrCs r,r4)OfaSudokuPreprocessor._build_infer_sampleQs(Vt^ J I J;OOF+,xx **,2245Ld6L6LMN%%d-B-B&BCKd11D4J4JJLx0   '+'<'"S#X4S>r.r ) typingrrnumpynprmodelscope.utils.constantrbaserr r.r,ras# .%c/cr.