KKi'~SrSSKJrJr SSKJr SSKJr SSKJ r SSK J r SSK J r SSKJrJr SS KJrJr SS KJrJrJr SS KJr SS KJr SS KJr SSKJ r J!r! SSK"J#r$ \\\S-/\\-\%-4r&\\\\\S-/\\-\%-4r'"SS\5r("SS\(5r)\&\ -\!-r*\ \+-\(-r,"SS\5r-g)z!Configuration for run evaluators.)CallableSequence)Any) Embeddings)BaseLanguageModel)BasePromptTemplate) RunEvaluator)EvaluationResultEvaluationResults)ExampleRun) BaseModel ConfigDictField)override) CRITERIA_TYPE)EmbeddingDistance) EvaluatorTypeStringEvaluator)StringDistanceNc<\rSrSr%Sr\\S'S\\\ 44Sjr Sr g) EvalConfig"zhConfiguration for a given run evaluator. Attributes: evaluator_type: The type of evaluator to use. evaluator_typereturnc>0nUHup#US:XdUcMX1U'M U$)z|Get the keyword arguments for the `load_evaluator` call. Returns: The keyword arguments for the `load_evaluator` call. r)selfkwargsfieldvals k/var/www/html/dynamic-report/venv/lib/python3.13/site-packages/langchain_classic/smith/evaluation/config.py get_kwargsEvalConfig.get_kwargs+s3 JE((CK5M rN) __name__ __module__ __qualname____firstlineno____doc__r__annotations__dictstrrr#__static_attributes__rr%r"rr"s# "! DcN r%rc^\rSrSr%SrSr\S-\S'Sr\S-\S'Sr \S-\S'\ S\ \\ 44U4Sjj5r S rU=r$) SingleKeyEvalConfig9zBConfiguration for a run evaluator that only requires a single key.N reference_keyprediction_key input_keyrcZ>[TU]5nSHnURUS5 M U$)N)r2r3r4)superr#pop)rrkey __class__s r"r#SingleKeyEvalConfig.get_kwargsGs.#%CC JJsD !D r%r)r&r'r(r)r*r2r-r+r3r4rr,rr#r. __classcell__)r9s@r"r0r09sfL $M3:$@!%NC$J%!IsTz BDcNr%r0c\rSrSr%Sr\"\S9r\\\ -\ S'Sr \\ S-\ S'Sr \\ S-\ S'Sr\S-\ S'Sr\S-\ S 'Sr\S-\ S 'Sr\S-\ S '\"S S 9r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5r"SS\5r "S S!\5r!"S"S#\5r""S$S%\5r#"S&S'\#5r$S(r%g)) RunEvalConfigTz#Configuration for a run evaluation.)default_factory evaluatorsNcustom_evaluatorsbatch_evaluatorsr2r3r4eval_llmTarbitrary_types_allowedcd\rSrSr%SrSr\S-\S'Sr\ S-\S'\ Rr \ \S'Sr g)RunEvalConfig.Criteria{zConfiguration for a reference-free criteria evaluator. Attributes: criteria: The criteria to evaluate. llm: The language model to use for the evaluation chain. Ncriteriallmrr)r&r'r(r)r*rIrr+rJrrCRITERIArr.rr%r"CriteriarG{s8 *.-$&-(,  %,(5(>(> >r%rLcd\rSrSr%SrSr\S-\S'Sr\ S-\S'\ Rr \ \S'Sr g)RunEvalConfig.LabeledCriteriazConfiguration for a labeled (with references) criteria evaluator. Attributes: criteria: The criteria to evaluate. llm: The language model to use for the evaluation chain. NrIrJrr)r&r'r(r)r*rIrr+rJrrLABELED_CRITERIArr.rr%r"LabeledCriteriarNs8 *.-$&-(,  %,(5(F(F Fr%rQcp\rSrSr%Sr\R r\\S'Sr \ S-\S'Sr \ S-\S'\ "SS9rS rg) RunEvalConfig.EmbeddingDistancezConfiguration for an embedding distance evaluator. Attributes: embeddings: The embeddings to use for computing the distance. distance_metric: The distance metric to use for computing the distance. rN embeddingsdistance_metricTrDr)r&r'r(r)r*rEMBEDDING_DISTANCErr+rUrrVEmbeddingDistanceEnumr model_configr.rr%r"rrSsE )6(H(H H(, J%,8<.5<!$( r%rc^\rSrSr%Sr\R r\\S'Sr \ S-\S'Sr \ \S'Sr g) RunEvalConfig.StringDistanceaMConfiguration for a string distance evaluator. Attributes: distance: The string distance metric to use (`damerau_levenshtein`, `levenshtein`, `jaro`, or `jaro_winkler`). normalize_score: Whether to normalize the distance to between 0 and 1. Applies only to the Levenshtein and Damerau-Levenshtein distances. rNdistanceTnormalize_scorer)r&r'r(r)r*rSTRING_DISTANCErr+r]StringDistanceEnumr^boolr.rr%r"rr[s3 )6(E(E E.2$t+2 $$r%rcd\rSrSr%Sr\R r\\S'Sr \ S-\S'Sr \ S-\S'Sr g)RunEvalConfig.QAzConfiguration for a QA evaluator. Attributes: prompt: The prompt template to use for generating the question. llm: The language model to use for the evaluation chain. rNrJpromptr)r&r'r(r)r*rQArr+rJrrerr.rr%r"rfrcs9 )6(8(8 8(,  %,,0"T)0r%rfcd\rSrSr%Sr\R r\\S'Sr \ S-\S'Sr \ S-\S'Sr g)RunEvalConfig.ContextQAConfiguration for a context-based QA evaluator. Attributes: prompt: The prompt template to use for generating the question. llm: The language model to use for the evaluation chain. rNrJrerr&r'r(r)r*r CONTEXT_QArr+rJrrerr.rr%r" ContextQArh9 )6(@(@ @(,  %,,0"T)0r%rmcd\rSrSr%Sr\R r\\S'Sr \ S-\S'Sr \ S-\S'Sr g)RunEvalConfig.CoTQArjrNrJrerrkrr%r"CoTQArprnr%rrc<\rSrSr%Sr\R r\\S'Sr g)RunEvalConfig.JsonValidityz,Configuration for a json validity evaluator.rrN) r&r'r(r)r*r JSON_VALIDITYrr+r.rr%r" JsonValidityrt:(5(C(C Cr%rwc<\rSrSr%Sr\R r\\S'Sr g)#RunEvalConfig.JsonEqualityEvaluatorz,Configuration for a json equality evaluator.rrN) r&r'r(r)r*r JSON_EQUALITYrr+r.rr%r"JsonEqualityEvaluatorrzrxr%r}cf\rSrSr%Sr\R r\\S'Sr \ \S'Sr \ \S'Sr \ \S'Sr g ) RunEvalConfig.ExactMatchaConfiguration for an exact match string evaluator. Attributes: ignore_case: Whether to ignore case when comparing strings. ignore_punctuation: Whether to ignore punctuation when comparing strings. ignore_numbers: Whether to ignore numbers when comparing strings. rF ignore_caseignore_punctuationignore_numbersrN)r&r'r(r)r*r EXACT_MATCHrr+rrarrr.rr%r" ExactMatchrs8 )6(A(A A! T!#(D($$r%rcJ\rSrSr%Sr\R r\\S'Sr \ \S'Sr g)RunEvalConfig.RegexMatchzConfiguration for a regex match string evaluator. Attributes: flags: The flags to pass to the regex. Example: `re.IGNORECASE`. rrflagsrN) r&r'r(r)r*r REGEX_MATCHrr+rintr.rr%r" RegexMatchrs" )6(A(A Asr%rc\rSrSr%Sr\R r\\S'Sr \ S-\S'Sr \ S-\S'Sr \S-\S'Sr\S-\S'S rg) RunEvalConfig.ScoreStringaConfiguration for a score string evaluator. This is like the criteria evaluator but it is configured by default to return a score on the scale from 1-10. It is recommended to normalize these scores by setting `normalize_by` to 10. Attributes: criteria: The criteria to evaluate. llm: The language model to use for the evaluation chain. normalize_by: If you want to normalize the score, the denominator to use. If not provided, the score will be between 1 and 10. prompt: The prompt template to use for evaluation. rNrIrJ normalize_byrer)r&r'r(r)r*r SCORE_STRINGrr+rIrrJrrfloatrerr.rr%r" ScoreStringrsV )6(B(B B)--$&-(,  %,%) edl),0"T)0r%rc<\rSrSr%Sr\R r\\S'Sr g) RunEvalConfig.LabeledScoreStringiz3Configuration for a labeled score string evaluator.rrN) r&r'r(r)r*rLABELED_SCORE_STRINGrr+r.rr%r"LabeledScoreStringrsA(5(J(J Jr%rr)&r&r'r(r)r*rlistr@SINGLE_EVAL_CONFIG_TYPECUSTOM_EVALUATOR_TYPEr+rArBBATCH_EVALUATOR_LIKEr2r-r3r4rCrrrYr0rLrQrrrfrmrrrwrr}rrrrr.rr%r"r=r=Tsq-HMIJ,/DDEQ=At12T9@8:>d/047> !%M3:$@!%NC$J%!IsTz B)-H$&-H $L ?& ? G- G / %, % 1 1 1' 1 1# 1D*D D D %( %(1)1.K[Kr%r=).r*collections.abcrrtypingrlangchain_core.embeddingsrlangchain_core.language_modelsrlangchain_core.promptsr langsmithr langsmith.evaluation.evaluatorr r langsmith.schemasr r pydanticrrrtyping_extensionsr0langchain_classic.evaluation.criteria.eval_chainr4langchain_classic.evaluation.embedding_distance.baserrX#langchain_classic.evaluation.schemarr1langchain_classic.evaluation.string_distance.baserr`r,RUN_EVALUATOR_LIKErrr0rr-rr=rr%r"rs'.0<5"N*11&JO'D.((4/1  c]HW%,-((4/1 .*.+\9OK'#- :}KI}Kr%