`i&jSSKJrJr SSKJr SSKJr SSKJr SSK J r \"\ 5r "SS\ 5r g) )DictOptional) getLogger)helper) OnnxModel)Fusionc^\rSrSrS\4U4SjjrS\S\4SjrS\S\S\\ 4Sjr S\S\S\\ 4S jr S\S\S\\ 4S jr S r U=r$) FusionGelumodelc(>[TU]USS5 g)NGeluErf)super__init__)selfr __class__s i/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/onnxruntime_tools/transformers/fusion_gelu.pyrFusionGelu.__init__s .input_name_to_nodesoutput_name_to_nodecURXU5(agURXU5(agURXU5 g)N)fuse_1fuse_2fuse_3)rerf_noderrs rfuseFusionGelu.fuses; ;;x6I J J  ;;x6I J J  H3FGrreturnc URSU;agX!RSn[U5S:wdUSRS:wagUSnURR US5(dgURSU;agX%RSn[U5S:wdUSRS:wagUSnURR USSU5nUcgURR USSS 9S:wagURSnURSURS:XaSOSn XRU :XakX&RSn[U5S:wdUSRS:wagUSn URR U S 5(dgU RSn ObURR USX5n U cgURR U S 5(dgXR;agURSn XqXVU /n URRX/UU5(dgURRU 5 [R"S U/U /S 9n S U l URRU 5 g)a This pattern is from PyTorch model Fuse Gelu with Erf into one node: Pattern 1: +-------Mul(0.5)---------------------+ | | | v [root] --> Div -----> Erf --> Add --> Mul --> (B=1.4142...) (1) Pattern 2: +------------------------------------+ | | | v [root] --> Div -----> Erf --> Add --> Mul -->Mul --> (B=1.4142...) (1) (0.5) Note that constant input for Add and Mul could be first or second input: like either A=0.5 or B=0.5 is fine. rNAddMulDiv -?MbP?delta?rinputsoutputs com.microsoftT)outputlenop_typer has_constant_input match_parentfind_constant_inputinputis_safe_to_fuse_nodesnodes_to_removeextendr make_nodedomain nodes_to_addappend)rrrrchildren add_after_erf mul_after_erfdivsubgraph_inputanothermul_halfsubgraph_outputsubgraph_nodes fused_nodes rrFusionGelu.fuse_1s( ??1 %8 8 &q'9: x=A !!4!4!=   zz,,]A>>     "*= = &';';A'>? x=A !!4!4!=   jj%%hq:MN ;  :: ) )#vU ) Cq H 1$**1-1E1Ea1HH!a 009 9*+?+?+BCH8}!Xa[%8%8E%A{H::003??&ooa0Ozz..}eWbH::003??^^3+2215OxPzz//@QSf0CEE  ##N3%%fn5EP_O`a +    ,rc`URSU;agX!RSn[U5S:wdUSRS:wagUSnURR US5(dgURSU;agX%RSn[U5S:wdUSRS:wagUSnURR US5(dgURSU;agX&RSn[U5S:wdUSRS:wagUSnURR USSU5nUcgSn URR USS S 9S:waDURR US SU5n U cgURR U S 5(dgURRUSU5n U cgU RSUR;agXXVU/n U (aU RU 5 URRXRS/UU5(dgURRU 5 [R"S U RS/URS/S9n SU lUR RU 5 g)a This pattern is from Keras model Fuse Gelu with Erf into one node: +------------------------------------------+ | | | v [root] --> Div -----> Erf --> Add --> Mul -->Mul (B=1.4142...) (A=1) (A=0.5) Note that constant input for Add and Mul could be first or second input: like either A=0.5 or B=0.5 is fine. rNr"r#r$r*r%r&r'r(Sqrtg@rr+r.T)r/r0r1r r2r3r4 get_parentr5r<r6r7r8rr9r:r;) rrrrr=r>r?mulr@ sqrt_node root_noderErFs rrFusionGelu.fuse_2hs ??1 %8 8 &q'9: x=A !!4!4!=   zz,,]A>>     "*= = &';';A'>? x=A !!4!4!=   zz,,]C@@     "*= = &';';A'>? x=A !!4!4!= qkjj%%hq:MN ;  :: ) )#vU ) Cq H //VQ@STI ::00C@@JJ))#q2EF      A cii / sK   ! !) ,zz//AQd0CEE  ##N3%%fi6F6Fq6I5JUXU_U_`aUbTcd +    ,rcURSU;agX!RSn[U5S:wdUSRS:wagUSnURR US5(dgURSU;agX%RSn[U5S:wdUSRS:wagUSnURR US5(dgURR USSU5nUcgURR USSS 9nUS:agURRXxS:XaSOSU5n U cgURSU;agX&RSn[U5S:wdUSRS:wagUSn U RSU RS:Xd!U RSU RS:XdgXqXVU /n URRXRS/UU5(dgURRU 5 [R"S U RS/U RS/S 9n S U lURR!U 5 g )a This pattern is from TensorFlow model Fuse Gelu with Erf into one node: +----------------------------------------------+ | | | v [root] --> Mul -----> Erf --> Add --> Mul -->Mul (A=0.7071067690849304) (B=1) (B=0.5) Note that constant input for Add and Mul could be first or second input: like either A=0.5 or B=0.5 is fine. rNr"r#r$r*g`?r'r(rr+r.T)r/r0r1r r2r3r4rJr5r6r7r8rr9r:r;r<) rrrrr=r>rC first_mulirMlast_mulrErFs rrFusionGelu.fuse_3sz ??1 %8 8 &q'9: x=A !!4!4!=   zz,,]A>>     "*= = &';';A'>? x=A !!4!4!= A;zz,,Xs;; JJ++HeQ@ST    JJ * *96HPU * V q5 JJ)))q&QaI\]    ??1 %8 8 &q'9: x=A !!4!4!= A;q!Y%5%5a%88HNN1rbs. "  8 bbr