如何在一个评估器中返回多个分数

有时，自定义评估器或摘要评估器返回多个指标会非常有用。例如，如果你有多个由LLM评判器生成的指标，你可以通过一次LLM调用生成多个指标来节省时间和成本，而不是多次LLM调用。

To return multiple scores using the Python SDK, simply return a list of dictionaries/objects of the following form:

[
    # 'key' is the metric name
    # 'score' is the value of a numerical metric
    {"key": string, "score": number},
    # 'value' is the value of a categorical metric
    {"key": string, "value": string},
    ... # You may log as many as you wish
]

To do so with the JS/TS SDK, return an object with a 'results' key and then a list of the above form

{results: [{ key: string, score: number }, ...]};

这些字典中的每一个都可以包含以下任意或全部反馈字段；请查阅链接的文档以获取更多信息。

Example:

- Python：需要 langsmith>=0.2.0
- TypeScript：多分数支持在以下版本中可用 langsmith@0.1.32 及更高版本

def multiple_scores(outputs: dict, reference_outputs: dict) -> list[dict]:
    # Replace with real evaluation logic.
    precision = 0.8
    recall = 0.9
    f1 = 0.85
    return [
        {"key": "precision", "score": precision},
        {"key": "recall", "score": recall},
        {"key": "f1", "score": f1},
    ]

function multipleScores(rootRun: Run, example: Example) {
  // Your evaluation logic here
  return {
      results: [
          { key: "precision", score: 0.8 },
          { key: "recall", score: 0.9 },
          { key: "f1", score: 0.85 },
      ],
  };
}

结果实验的行将显示每个分数。

!多个_scores.png

相关