9i%SrSSKJr SSKJr SSKJr SSKJrJ r SSK r SSK J r J r JrJrJr SSKJr 0r\\S 4\S '/S Qr"S S \5rS \S\SS4SjrS\S\S\S\S\ R6S\S\4SjrS\S\S\S\S\ R6S\S\4Sjr\"S\5 \"S\5 S\S\S\ \\ R64S\S\4 SjrS\4Sjr \S\S\S4S j5r!g)!z This is an experimental new API for PyTorch Distributed. This is actively in development and subject to change or deletion entirely. This is intended as a proving ground for more flexible and object oriented distributed APIs. ) Generator)contextmanager) timedelta)ProtocolUnionN)_current_process_group_set_process_group ProcessGroupReduceOpStore) rendezvousProcessGroupFactory _BACKENDS)r r rregister_backend new_groupcurrent_process_group process_groupcR\rSrSrSrS\S\S\S\S\RS\ S \ 4S jr S r g ) r$z%Protocol for process group factories.storerank world_sizetimeoutdevicekwargsreturnc g)N)selfrrrrrrs X/var/www/html/land-doc-ocr/venv/lib/python3.13/site-packages/torch/distributed/_dist2.py__call__ProcessGroupFactory.__call__'srN)__name__ __module__ __qualname____firstlineno____doc__r intrtorchrobjectr r!__static_attributes__rr#r rr$sR/      r#namefuncrcHU[;a[SUS35eU[U'g)z Register a new process group backend. Args: name: The name of the backend. func: The function to create the process group. Backend z already registeredN)r ValueError)r-r.s r rr2s* y8D6)<=>>IdOr#rrrrrrc rSSKJn [U5S:XdS5eU"XX#5nUR5 [ XU5nUR [R R5 URU[R RU5 UR[R"S5[R RU5 [RR5(a?UR[R"S5[R RU5 U$)Nr)ProcessGroupGlooz'Gloo backend received unexpected kwargscpucuda) torch.distributedr3len_set_sequence_number_for_groupr _set_default_backend BackendTypeGLOO_register_backendr*rr5 is_available) rrrrrrr3 backend_classpgs r _gloo_factoryr@@s3 v;! FFF $U*FM002 e: .BL4499:!9!9!>!> N U\55::M zz    LL ,":":"?"?  Ir#c SSKJn UR5nX7lUR 5H/up[ Xx5(d[ SU35e[XxU 5 M1 U"XX'5n U R5 U RU5 [XU5n U R[RR5 U RU[RRU 5 U $)Nr)ProcessGroupNCCLzUnknown option )r6rBOptions_timeoutitemshasattrKeyErrorsetattrr8eager_connect_single_devicer r9r:NCCLr<) rrrrrrrBoptskvr>r?s r _nccl_factoryrN^s3  # # %DM t_QC01 1 %U*CM002--f5 e: .BL4499:!9!9!>!> N Ir#glooncclbackendc U[;a[SUS35e[R"U5n[ [ [ S555upEnURU5 [U"XEXaU40UD6$)a Create a new process group with the given backend and options. This group is independent and will not be globally registered and thus not usable via the standard torch.distributed.* APIs. Args: backend: The backend to use for the process group. timeout: The timeout for collective operations. device: The device to use for the process group. **kwargs: All remaining arguments are passed to the backend constructor. See the backend specific documentation for details. Returns: A new process group. r0z not registeredzenv://)rr1r*rnextiterr set_timeout)rQrrrrrrs r rr~sn*i8G9O<== \\& !F"4 8(<#=>E g W e: Q& QQr#c[5$)z^ Get the current process group. Thread local method. Returns: The current process group. )rrr#r rrs  " ##r#r?)NNNc#v# [5n[U5 Sv [U5 g![U5 f=f7f)zc Context manager for process groups. Thread local method. Args: pg: The process group to use. N)rr )r?prev_pgs r rrs0$%Gr$ 7#7#s9) 9 69)"r(collections.abcr contextlibrdatetimertypingrrr*torch._C._distributed_c10drr r r r torch.distributed.rendezvousr rdictstr__annotations____all__rrr)rr+r@rNrrrrr#r rcs &%" 4/1 4** +0  (  3 &9 d     LL   <    LL   8''R R R #u||# $R R  R@$|$ $l $y1A'B $ $r#