自托管 LangSmith 实例可以处理大量追踪和用户。自托管部署的默认配置可以处理大量负载，您可以配置您的部署以实现更高的规模化。本页面描述了规模化注意事项，并提供了一些示例配置来帮助您配置自托管实例。

有关示例配置，请参阅 LangSmith 规模化示例配置.

摘要

The table below provides an overview comparing different LangSmith configurations for various load patterns (reads / writes):

下面我们将详细介绍读写路径，并提供 values.yaml 一段代码供您开始使用自托管 LangSmith 实例。

追踪摄取（写入路径）

对写入路径造成负载的常见用法：

• - 通过 Python 或 JavaScript LangSmith SDK 引入追踪
• - 通过以下方式引入追踪 @traceable 包装器
• - 通过以下方式提交追踪 /runs/multipart 端点

在追踪引入中发挥重要作用的服务：

• - 平台后端服务：接收引入追踪的初始请求并将追踪放入 Redis 队列
• - Redis 缓存：用于对需要持久化的追踪进行排队
• - 引入队列服务：持久化追踪以供查询
• - ClickHouse：用于存储追踪的持久化存储

When scaling up the write path (trace ingestion), it is helpful to monitor the four services/resources listed above. Here are some typical changes that can help increase performance of trace ingestion:

• - 如果 ClickHouse 接近资源限制，请为其分配更多资源（CPU 和内存）。
• - 如果引入请求响应时间过长，请增加平台后端 pod 的数量。
• - 如果追踪从 Redis 处理的速度不够快，请增加引入队列服务 pod 副本数。
• - 如果发现当前 Redis 实例达到资源限制，请使用更大的 Redis 缓存。这也可能是引入请求耗时长的原因。

追踪查询（读取路径）

对读取路径造成负载的常见用法：

• - 前端用户查看追踪项目或单个追踪
• - 用于查询追踪信息的脚本
• - 访问以下任一 /runs/query or /runs/<run-id> API 端点

在查询追踪中发挥重要作用的服务：

• - 后端服务：接收请求，向 ClickHouse 提交查询，然后响应请求
• - ClickHouse：追踪的持久化存储。这是请求追踪信息时查询的主要数据库。

When scaling up the read path (trace querying), it is helpful to monitor the two services/resources listed above. Here are some typical changes that can help improve performance of trace querying:

• - 增加后端服务 pod 的数量。如果后端服务 pod 达到 1 核 CPU 使用率，这将产生最显著的影响。
• - 为 ClickHouse 分配更多资源（CPU 或内存）。ClickHouse 可能非常消耗资源，但应该会带来更好的性能。
• - 迁移到 复制的 ClickHouse 集群。添加 ClickHouse 副本有助于提高读取性能，但我们建议保持在 5 个副本以下（从 3 个开始）。

如需更精确的指导来了解这如何转化为 helm chart 值，请参阅以下示例 部分。如果您不确定为什么 LangSmith 实例无法处理某种负载模式，请联系 LangChain 团队。

LangSmith 队列的 KEDA 自动扩缩容

我们强烈建议安装 KEDA （Kubernetes 事件驱动自动扩缩容）在您的集群上。KEDA 使 queue 和 ingest-queue 服务能够根据其队列积压大小以及 CPU 和内存自动扩缩容。这可以提高资源利用率并更好地处理流量峰值。

安装 KEDA

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda --create-namespace

配置 KEDA 自动扩缩容

安装 KEDA 后，您可以为以下服务启用基于 KEDA 的自动扩缩容 queue 和 ingest-queue 服务在您的 values.yaml:

queue:
  autoscaling:
    keda:
      enabled: true

ingestQueue:
  autoscaling:
    keda:
      enabled: true

启用 KEDA 后，队列服务将在积压量增长时自动扩容，在积压量被处理时自动缩容。这对于处理可变的追踪数据摄取负载特别有用，无需过度配置资源。

扩展配置的 LangSmith 示例

下面我们提供一些基于预期读写负载的 LangSmith 配置示例。

对于读取负载（追踪查询）：

• - 低意味着大约 5 个用户同时查看追踪（约 10 请求/秒）
• - 中意味着大约 20 个用户同时查看追踪（约 40 请求/秒）
• - 高意味着大约 50 个用户同时查看追踪（约 100 请求/秒）

对于写入负载（追踪数据摄取）：

• - 低意味着每秒提交最多 10 条追踪
• - 中意味着每秒提交最多 100 条追踪
• - 高意味着每秒提交最多 1000 条追踪

低读取，低写入 <a name="low-reads-low-writes"></a>

默认的 LangSmith 配置即可处理此负载。无需自定义资源配置。

低读取，高写入 <a name="low-reads-high-writes"></a>

您的追踪数据摄取规模非常大，但前端同时查询追踪的用户数为个位数。

鉴于此，我们推荐以下配置：

config:
  blobStorage:
    # Please also set the other keys to connect to your blob storage. See configuration section.
    enabled: true
  settings:
    redisRunsExpirySeconds: "3600"
# ttl:
#   enabled: true
#   ttl_period_seconds:
#     longlived: "7776000"  # 90 days (default is 400 days)
#     shortlived: "604800"  # 7 days (default is 14 days)

frontend:
  deployment:
    replicas: 4 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 2
#     maxReplicas: 4

platformBackend:
  deployment:
    replicas: 20 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 8
#     maxReplicas: 20

ingestQueue:
  deployment:
    replicas: 24 # OR enable KEDA autoscaling below
# autoscaling:
#   keda:
#     enabled: true
#     minReplicaCount: 8
#     maxReplicaCount: 24

backend:
  deployment:
    replicas: 5 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 3
#     maxReplicas: 5

## Ensure your Redis cache is at least 26 GB for high write scale
redis:
  external:
    enabled: true
    existingSecretName: langsmith-redis-secret # Set the connection url for your external Redis instance (26+ GB)

clickhouse:
  statefulSet:
    persistence:
      # This may depend on your configured TTL (see config section).
      # We recommend 600Gi for every shortlived TTL day if operating at this scale constantly.
      size: 4200Gi # This assumes 7 days TTL and operating a this scale constantly.
    resources:
      requests:
        cpu: "10"
        memory: "32Gi"
      limits:
        cpu: "16"
        memory: "48Gi"

commonEnv:
  - name: "CLICKHOUSE_ASYNC_INSERT_WAIT_PCT_FLOAT"
    value: "0"

高读取，低写入 <a name="high-reads-low-writes"></a>

You have a relatively low scale of trace ingestions, but many frontend users querying traces and/or have scripts that hit the /runs/query or /runs/<run-id> 频繁访问端点。

鉴于此，我们强烈建议设置一个复制的 ClickHouse 集群，以实现低延迟的高读取扩展。 查看我们的 外部 ClickHouse 文档 获取更多关于如何设置复制 ClickHouse 集群的指导。对于这种负载模式，我们建议使用 3 节点复制设置，集群中的每个副本应具有 8 核以上和 16GB 以上的资源请求，以及 12 核和 32GB 内存的资源限制。

鉴于此，我们建议采用如下配置：

config:
  blobStorage:
    # Please also set the other keys to connect to your blob storage. See configuration section.
    enabled: true

frontend:
  deployment:
    replicas: 2

ingestQueue:
  deployment:
    replicas: 3 # OR enable KEDA autoscaling below
# autoscaling:
#   keda:
#     enabled: true
#     minReplicaCount: 2
#     maxReplicaCount: 3

backend:
  deployment:
    replicas: 40 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 16
#     maxReplicas: 40

# We strongly recommend setting up a replicated clickhouse cluster for this load.
# Update these values as needed to connect to your replicated clickhouse cluster.
clickhouse:
  external:
    # If using a 3 node replicated setup, each replica in the cluster should have resource requests of 8+ cores and 16+ GB memory, and resource limit of 12 cores and 32 GB memory.
    enabled: true
    host: langsmith-ch-clickhouse-replicated.default.svc.cluster.local
    port: "8123"
    nativePort: "9000"
    user: "default"
    password: "password"
    database: "default"
    cluster: "replicated"

中等读取，中等写入 <a name="medium-reads-medium-writes"></a>

这是一个全面的良好配置，应该能够处理大多数 LangSmith 使用模式。在内部测试中，此配置使我们能够扩展到每秒 100 条追踪摄取和每秒 40 个读取请求。

鉴于此，我们建议采用如下配置：

config:
  blobStorage:
    # Please also set the other keys to connect to your blob storage. See configuration section.
    enabled: true
  settings:
    redisRunsExpirySeconds: "3600"

frontend:
  deployment:
    replicas: 2

ingestQueue:
  deployment:
    replicas: 6 # OR enable KEDA autoscaling below
# autoscaling:
#   keda:
#     enabled: true
#     minReplicaCount: 3
#     maxReplicaCount: 6

backend:
  deployment:
    replicas: 16 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 8
#     maxReplicas: 16

redis:
  statefulSet:
    resources:
      requests:
        memory: 13Gi
      limits:
        memory: 13Gi

  # -- For external redis instead use something like below --
  # external:
  #   enabled: true
  #   connectionUrl: "" OR existingSecretName: ""

clickhouse:
  statefulSet:
    persistence:
      # This may depend on your configured TTL.
      # We recommend 60Gi for every shortlived TTL day if operating at this scale constantly.
      size: 420Gi # This assumes 7 days TTL and operating a this scale constantly.
    resources:
      requests:
        cpu: "16"
        memory: "24Gi"
      limits:
        cpu: "28"
        memory: "40Gi"

commonEnv:
  - name: "CLICKHOUSE_ASYNC_INSERT_WAIT_PCT_FLOAT"
    value: "0"

高读取，高写入 <a name="high-reads-high-writes"></a>

You have a very high rate of trace ingestion (approaching 1000 traces submitted per second) and also have many users querying traces on the frontend (over 50 users) and/or scripts that are consistently making requests to /runs/query or /runs/<run-id> endpoints.

鉴于此，我们强烈建议设置一个复制的 ClickHouse 集群，以防止在高写入扩展时读取性能下降。 查看我们的 外部 ClickHouse 文档 for more guidance on how to set up a replicated ClickHouse cluster. For this load pattern, we recommend using a 3 node replicated setup, where each replica in the cluster should have resource requests of 14+ cores and 24+ GB memory, and resource limit of 20 cores and 48 GB memory. We also recommend that each node/instance of ClickHouse has 600 Gi of volume storage for each day of TTL that you enable (as per the configuration below).

总体而言，我们建议采用如下配置：

config:
  blobStorage:
    # Please also set the other keys to connect to your blob storage. See configuration section.
    enabled: true
  settings:
    redisRunsExpirySeconds: "3600"
# ttl:
#   enabled: true
#   ttl_period_seconds:
#     longlived: "7776000"  # 90 days (default is 400 days)
#     shortlived: "604800"  # 7 days (default is 14 days)

frontend:
  deployment:
    replicas: 4 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 2
#     maxReplicas: 4

platformBackend:
  deployment:
    replicas: 20 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 8
#     maxReplicas: 20

ingestQueue:
  deployment:
    replicas: 24 # OR enable KEDA autoscaling below
# autoscaling:
#   keda:
#     enabled: true
#     minReplicaCount: 8
#     maxReplicaCount: 24

backend:
  deployment:
    replicas: 50 # OR enable autoscaling below
# autoscaling:
#   hpa:
#     enabled: true
#     minReplicas: 20
#     maxReplicas: 50

## Ensure your Redis cache is at least 26 GB for high write scale
redis:
  external:
    enabled: true
    existingSecretName: langsmith-redis-secret # Set the connection url for your external Redis instance (26+ GB)

# We strongly recommend setting up a replicated clickhouse cluster for this load.
# Update these values as needed to connect to your replicated clickhouse cluster.
clickhouse:
  external:
    # If using a 3 node replicated setup, each replica in the cluster should have resource requests of 14+ cores and 24+ GB memory, and resource limit of 20 cores and 48 GB memory.
    enabled: true
    host: langsmith-ch-clickhouse-replicated.default.svc.cluster.local
    port: "8123"
    nativePort: "9000"
    user: "default"
    password: "password"
    database: "default"
    cluster: "replicated"

commonEnv:
  - name: "CLICKHOUSE_ASYNC_INSERT_WAIT_PCT_FLOAT"
    value: "0"