KKi %SrSSKJr SSKJr SSKJr SSKJr SSK J r SSK J r SSK Jr SS KJrJr SS KJrJr SS KJr SS KJrJr SS KJr SSKJr SSKJrJ r J!r! SSK"J#r# SSK$J%r%J&r&J'r' SSK(J)r)J*r* SSK+J,r,J-r- S\.S\/\04Sjr10\%Rd\!_\%Rf\ _\%Rh\_\%Rj\ _\%Rl\*_\%Rn\_\%Rp\)_\%Rr\ _\%Rt\_\%Rv\_\%Rx\-_\%Rz\,_\%R|\_\%R~\_\%R\_\%R\_\%R\_\%R\\%R\#\%R\0ErF\0\%\G\&\G\-\G\'-4\HS'SS.S\%S\S-S\S\\'-4SjjrISSS.S\\%S\S-S \0S-S\S\/\\'-4 S!jjrJg)"z Loading datasets and evaluators.)Sequence)Any)BaseLanguageModel)Chain)TrajectoryEvalChain)PairwiseStringEvalChain)LabeledPairwiseStringEvalChain)CriteriaEvalChainLabeledCriteriaEvalChain)EmbeddingDistanceEvalChain"PairwiseEmbeddingDistanceEvalChain)ExactMatchStringEvaluator)JsonEqualityEvaluatorJsonValidityEvaluator)JsonEditDistanceEvaluator)JsonSchemaEvaluator)ContextQAEvalChainCotQAEvalChain QAEvalChain)RegexMatchStringEvaluator) EvaluatorType LLMEvalChainStringEvaluator)LabeledScoreStringEvalChainScoreStringEvalChain)PairwiseStringDistanceEvalChainStringDistanceEvalChainurireturncSSKJn U"SU35n[US5$![anSn[U5UeSnAff=f)aLoad a dataset from the [LangChainDatasets on HuggingFace](https://huggingface.co/LangChainDatasets). Args: uri: The uri of the dataset to load. Returns: A list of dictionaries, each representing a row in the dataset. **Prerequisites** ```bash pip install datasets ``` Examples: -------- ```python from langchain_classic.evaluation import load_dataset ds = load_dataset("llm-math") ``` r) load_datasetzXload_dataset requires the `datasets` package. Please install with `pip install datasets`NzLangChainDatasets/train)datasetsr! ImportErrorlist)rr!emsgdatasets f/var/www/html/dynamic-report/venv/lib/python3.13/site-packages/langchain_classic/evaluation/loading.pyr!r!4sY.&)/u56G  !! & : #A% &s! >9>_EVALUATOR_MAPN)llm evaluatorr+kwargsc U[;a.SUS[[R553n[U5e[Un[ U[ 5(a-SSKJn U=(d U"SSSS 9nUR"S S U0UD6$U"S 0UD6$![a* SSK Jn N@![anSn[U5UeSnAff=ff=f![anS US 3n[U5UeSnAff=f)aLoad the requested evaluation chain specified by a string. Parameters ---------- evaluator : EvaluatorType The type of evaluator to load. llm : BaseLanguageModel, optional The language model to use for evaluation, by default None **kwargs : Any Additional keyword arguments to pass to the evaluator. Returns: ------- Chain The loaded evaluation chain. Examples: -------- >>> from langchain_classic.evaluation import load_evaluator, EvaluatorType >>> evaluator = load_evaluator(EvaluatorType.QA) zUnknown evaluator type: z Valid types are: r) ChatOpenAIzCould not import langchain_openai or fallback onto langchain_community. Please install langchain_openai or specify a language model explicitly. It's recommended to install langchain_openai AND specify a language model explicitly.Nzgpt-4*)modelseed temperaturezEvaluation with the z requires a language model to function. Failed to create the default 'gpt-4' model. Please manually provide an evaluation LLM or check your openai credentials.r+) r*r%keys ValueError issubclassrlangchain_openair/r$&langchain_community.chat_models.openai Exceptionfrom_llm)r,r+r-r' evaluator_clsr/r&s r)load_evaluatorr=ss6&&yk!$~':':'<"=!> @ o"9-M-.. ) 27 J'JC%%8#888  "6 ""5 2 2#2?&c*12 2  )&}o655 S/q ( )sNB C CBC B<)B77B<<CC C$ CC$)r+config evaluatorsr>c /nUH?nU(aURU05O0nUR[U4SU00UEUED65 MA U$)aLoad evaluators specified by a list of evaluator types. Parameters ---------- evaluators : Sequence[EvaluatorType] The list of evaluator types to load. llm : BaseLanguageModel, optional The language model to use for evaluation, if none is provided, a default ChatOpenAI gpt-4 model will be used. config : dict, optional A dictionary mapping evaluator types to additional keyword arguments, by default None **kwargs : Any Additional keyword arguments to pass to all evaluators. Returns: ------- List[Chain] The loaded evaluators. Examples: -------- >>> from langchain_classic.evaluation import load_evaluators, EvaluatorType >>> evaluators = [EvaluatorType.QA, EvaluatorType.CRITERIA] >>> loaded_evaluators = load_evaluators(evaluators, criteria="helpfulness") r+)getappendr=)r?r+r>r-loadedr,_kwargss r)load_evaluatorsrEsUBF /5&**Y+2 nYQCQ;Pf;P;PQR  M)K__doc__collections.abcrtypingrlangchain_core.language_modelsrlangchain_classic.chains.baser9langchain_classic.evaluation.agents.trajectory_eval_chainr'langchain_classic.evaluation.comparisonr2langchain_classic.evaluation.comparison.eval_chainr 0langchain_classic.evaluation.criteria.eval_chainr r 4langchain_classic.evaluation.embedding_distance.baser r -langchain_classic.evaluation.exact_match.baser)langchain_classic.evaluation.parsing.baserr2langchain_classic.evaluation.parsing.json_distancer0langchain_classic.evaluation.parsing.json_schemarlangchain_classic.evaluation.qarrr-langchain_classic.evaluation.regex_match.baser#langchain_classic.evaluation.schemarrr/langchain_classic.evaluation.scoring.eval_chainrr1langchain_classic.evaluation.string_distance.baserrstrr%dictr!QACOT_QA CONTEXT_QAPAIRWISE_STRING SCORE_STRINGLABELED_PAIRWISE_STRINGLABELED_SCORE_STRINGAGENT_TRAJECTORYCRITERIALABELED_CRITERIASTRING_DISTANCEPAIRWISE_STRING_DISTANCEEMBEDDING_DISTANCEPAIRWISE_EMBEDDING_DISTANCE JSON_VALIDITY JSON_EQUALITYJSON_EDIT_DISTANCEJSON_SCHEMA_VALIDATION REGEX_MATCH EXACT_MATCHr*type__annotations__r=rEr4rFr)rrs&$</LTYP T  !"c!"d4j!"Nk.0!!#:   4  ))+I &&(C""$7-""$<!!#:**,K$$&@--/Q!6 !6!"$$&?#$((*=88)e$tO'<<><%)@#@# T !@# @#  _ @#L%) %'% T !% 4K %  %  %/ !" %rF