当您直接调用 LLM 时，在以下情况之外 LangChain 或 LangSmith 支持的集成，您需要提供特定的元数据，以便 LangSmith 可以显示 token 计数、计算成本，并允许您在 运行 中打开 Playground 中使用正确的提供商和模型。

完整的 LLM 追踪有四个要求：

消息格式

When tracing a custom model or a custom input/output format, it must either follow the LangChain format, OpenAI completions format or Anthropic messages format. For more details, refer to the OpenAI Chat Completions or Anthropic Messages 文档。LangChain 格式为：

必须匹配 <code>id</code> 先前 <code>assistant</code> 消息的 <code>tool_calls[i]</code> 条目。仅在 <code>role</code> is <code>tool</code>.

Use this field to send token counts and/or costs with your model's output. See 提供令牌和成本信息 了解更多详情。

 inputs = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Hi, can you tell me the capital of France?"
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The capital of France is Paris."
        },
        {
          "type": "reasoning",
          "text": "The user is asking about..."
        }
      ]
    }
  ]
}

input = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What's the weather in San Francisco?"
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [{"type": "tool_call", "name": "get_weather", "args": {"city": "San Francisco"}, "id": "call_1"}],
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "content": [
        {
          "type": "text",
          "text": "{\"temperature\": \"18°C\", \"condition\": \"Sunny\"}"
        }
      ]
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The weather in San Francisco is 18°C and sunny."
        }
      ]
    }
  ]
}

inputs = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What breed is this dog?"
        },
        {
          "type": "image",
          "url": "https://fastly.picsum.photos/id/237/200/300.jpg?hmac=TmmQSbShHz9CdQm0NkEjx1Dyh_Y984R9LpNrpvH2D_U",
          # alternative to a url, you can provide a base64 encoded image
          # "base64": "<base64 encoded image>",
          "mime_type": "image/jpeg",
        }
      ]
    }
  ]
}

outputs = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "This looks like a Black Labrador."
        }
      ]
    }
  ]
}

input = {
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What is the price of AAPL?"
        }
      ]
    }
  ]
}

output = {
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "server_tool_call",
          "name": "web_search",
          "args": {
            "query": "price of AAPL",
            "type": "search"
          },
          "id": "call_1"
        },
        {
          "type": "server_tool_result",
          "tool_call_id": "call_1",
          "status": "success"
        },
        {
          "type": "text",
          "text": "The price of AAPL is $150.00"
        }
      ]
    }
  ]
}

Convert custom I/O formats into LangSmith compatible formats

如果您使用自定义输入或输出格式，可以使用 process_inputs/processInputs 和 process_outputs/processOutputs 函数将其转换为 LangSmith 兼容格式 @traceable 装饰器上的 （Python）或 traceable 函数 (TS).

process_inputs/processInputs 和 process_outputs/processOutputs 接受函数，允许您在将特定追踪的输入和输出记录到 LangSmith 之前对其进行转换。它们可以访问追踪的输入和输出，并可以返回包含处理数据的新字典。

以下是使用 process_inputs 和 process_outputs to convert a custom I/O format into a LangSmith compatible format:

class OriginalInputs(BaseModel):
    """Your app's custom request shape"""

class OriginalOutputs(BaseModel):
    """Your app's custom response shape."""

class LangSmithInputs(BaseModel):
    """The input format LangSmith expects."""

class LangSmithOutputs(BaseModel):
    """The output format LangSmith expects."""

def process_inputs(inputs: dict) -> dict:
    """Dict -> OriginalInputs -> LangSmithInputs -> dict"""

def process_outputs(output: Any) -> dict:
    """OriginalOutputs -> LangSmithOutputs -> dict"""


@traceable(run_type="llm", process_inputs=process_inputs, process_outputs=process_outputs)
def chat_model(inputs: dict) -> dict:
    """
    Your app's model call. Keeps your custom I/O shape.
    The decorators call process_* to log LangSmith-compatible format.
    """

在追踪中识别自定义模型

使用自定义模型时，建议同时提供以下内容 metadata 字段以在查看轨迹时识别模型，以及在 过滤时.

• - ls_provider：模型的提供商，例如 "openai", "anthropic".
• - ls_model_name：模型的名称，例如 "gpt-5.4-mini", "claude-3-opus-20240229".

from langsmith import traceable

inputs = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "I'd like to book a table for two."},
]
output = {
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "Sure, what time would you like to book the table for?"
            }
        }
    ]
}

@traceable(
    run_type="llm",
    metadata={"ls_provider": "my_provider", "ls_model_name": "my_model"}
)
def chat_model(messages: list):
    return output

chat_model(inputs)

const messages = [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "I'd like to book a table for two." }
];
const output = {
    choices: [
        {
            message: {
                role: "assistant",
                content: "Sure, what time would you like to book the table for?",
            },
        },
    ],
    usage_metadata: {
        input_tokens: 27,
        output_tokens: 13,
        total_tokens: 40,
    },
};

// Can also use one of:
// const output = {
//     message: {
//         role: "assistant",
//         content: "Sure, what time would you like to book the table for?"
//     }
// };
//
// const output = {
//     role: "assistant",
//     content: "Sure, what time would you like to book the table for?"
// };
//
// const output = ["assistant", "Sure, what time would you like to book the table for?"];

const chatModel = traceable(
    async ({ messages }: { messages: { role: string; content: string }[] }) => {
        return output;
    },
    {
        run_type: "llm",
        name: "chat_model",
        metadata: {
            ls_provider: "my_provider",
            ls_model_name: "my_model"
        }
    }
);

await chatModel({ messages });

如果您实现自定义流式处理 chat_model，您可以将输出"归约"为与非流式版本相同的格式。这仅在 Python 中受支持：

def _reduce_chunks(chunks: list):
    all_text = "".join([chunk["choices"][0]["message"]["content"] for chunk in chunks])
    return {"choices": [{"message": {"content": all_text, "role": "assistant"}}]}

@traceable(
    run_type="llm",
    reduce_fn=_reduce_chunks,
    metadata={"ls_provider": "my_provider", "ls_model_name": "my_model"}
)
def my_streaming_chat_model(messages: list):
    for chunk in ["Hello, " + messages[1]["content"]]:
        yield {
            "choices": [
                {
                    "message": {
                        "content": chunk,
                        "role": "assistant",
                    }
                }
            ]
        }

list(
    my_streaming_chat_model(
        [
            {"role": "system", "content": "You are a helpful assistant. Please greet the user."},
            {"role": "user", "content": "assistant"},
        ],
    )
)

要了解更多关于如何使用 metadata 字段的信息，请参阅 添加元数据和标签 指南。要自定义自定义代理运行在消息视图中的显示方式，请参见 自定义消息视图.

提供 token 和成本信息

Token 计数用于启用成本计算，LangSmith 在 追踪项目 UI中显示它们。有两种提供方式：

• - **设置 usage_metadata 在运行树**：调用 get_current_run_tree() / getCurrentRunTree() 在您的 @traceable 函数中并设置 usage_metadata 字段。这不会改变函数的返回值。
• - **在输出中返回 usage_metadata ：在**中包含 usage_metadata 作为函数返回字典中的顶层键。

支持的 usage_metadata 字段

要直接发送成本（用于非线性定价），您还可以包含 input_cost, output_cost和 total_cost 字段。有关配置模型定价和查看 UI 中成本的详细信息，请参阅 成本追踪 page.

Time-to-first-token

如果您使用的是 traceable 或其中一个 SDK 包装器，LangSmith 将自动填充流式 LLM 运行的首 token 时间。但是，如果您直接使用 RunTree API ，则需要向运行树添加一个 new_token 事件，以便正确填充首 token 时间。

示例如下：

from langsmith.run_trees import RunTree
run_tree = RunTree(
    name="CustomChatModel",
    run_type="llm",
    inputs={ ... }
)
run_tree.post()
llm_stream = ...
first_token = None
for token in llm_stream:
    if first_token is None:
      first_token = token
      run_tree.add_event({
        "name": "new_token"
      })
run_tree.end(outputs={ ... })
run_tree.patch()

const runTree = new RunTree({
    name: "CustomChatModel",
    run_type: "llm",
    inputs: { ... },
});
await runTree.postRun();
const llmStream = ...;
let firstToken;
for (const token of llmStream) {
    if (firstToken == null) {
        firstToken = token;
        runTree.addEvent({ name: "new_token" });
    }
}
await runTree.end({
    outputs: { ... },
});
await runTree.patchRun();

相关

• - 自定义插桩：核心 @traceable 和 RunTree patterns.
• - 在追踪函数中访问当前运行（span）：使用 get_current_run_tree() 设置 usage_metadata 和其他字段。
• - 追踪 OpenAI 应用程序：使用 OpenAI 包装器时自动追踪令牌和成本。
• - 追踪 Anthropic 应用程序：使用 Anthropic 包装器时自动追踪令牌和成本。
• - 集成概述：内置 LangSmith 支持的提供商和框架完整列表。