Ollama 允许你在本地运行开源大语言模型（LLM），如 Llama 3.1。

Ollama 将模型权重、配置和数据打包成单个包，通过 Modelfile 定义。它优化了设置和配置细节，包括 GPU 使用。

本指南将帮助你开始使用 ChatOllama 聊天模型。有关所有 ChatOllama 功能和配置的详细文档，请前往 API 参考.

概述

集成详情

Ollama 允许你使用具有不同能力的各种模型。以下详情表中的一些字段仅适用于 Ollama 提供的部分模型。

有关支持的模型和模型变体的完整列表，请参阅 Ollama 模型库 并按标签搜索。

模型特性

请参阅下表标题中的链接，了解如何使用特定功能。

设置

请按照 这些说明 来设置和运行本地 Ollama 实例。然后，下载 @langchain/ollama package.

凭据

如果你想获取模型调用的自动追踪，还可以设置你的 LangSmith API 密钥，取消下面的注释：

# export LANGSMITH_TRACING="true"
# export LANGSMITH_API_KEY="your-api-key"

安装

LangChain ChatOllama 集成位于 @langchain/ollama package:

npm install @langchain/ollama @langchain/core

yarn add @langchain/ollama @langchain/core

pnpm add @langchain/ollama @langchain/core

实例化

现在我们可以实例化模型对象并生成聊天补全：

const llm = new ChatOllama({
    model: "llama3",
    temperature: 0,
    maxRetries: 2,
    // other params...
})

调用

const aiMsg = await llm.invoke([
    [
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ],
    ["human", "I love programming."],
])
aiMsg

AIMessage {
  "content": "Je adore le programmation.\n\n(Note: \"programmation\" is the feminine form of the noun in French, but if you want to use the masculine form, it would be \"le programme\" instead.)",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3",
    "created_at": "2024-08-01T16:59:17.359302Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 6399311167,
    "load_duration": 5575776417,
    "prompt_eval_count": 35,
    "prompt_eval_duration": 110053000,
    "eval_count": 43,
    "eval_duration": 711744000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 35,
    "output_tokens": 43,
    "total_tokens": 78
  }
}

console.log(aiMsg.content)

Je adore le programmation.

(Note: "programmation" is the feminine form of the noun in French, but if you want to use the masculine form, it would be "le programme" instead.)

工具

Ollama 现在支持原生工具调用 适用于其部分可用模型。下面的示例演示了如何从 Ollama 模型调用工具。

const weatherTool = tool((_) => "Da weather is weatherin", {
  name: "get_current_weather",
  description: "Get the current weather in a given location",
  schema: z.object({
    location: z.string().describe("The city and state, e.g. San Francisco, CA"),
  }),
});

// Define the model
const llmForTool = new ChatOllama({
  model: "llama3-groq-tool-use",
});

// Bind the tool to the model
const llmWithTools = llmForTool.bindTools([weatherTool]);

const resultFromTool = await llmWithTools.invoke(
  "What's the weather like today in San Francisco? Ensure you use the 'get_current_weather' tool."
);

console.log(resultFromTool);

AIMessage {
  "content": "",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llama3-groq-tool-use",
    "created_at": "2024-08-01T18:43:13.2181Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 2311023875,
    "load_duration": 1560670292,
    "prompt_eval_count": 177,
    "prompt_eval_duration": 263603000,
    "eval_count": 30,
    "eval_duration": 485582000
  },
  "tool_calls": [
    {
      "name": "get_current_weather",
      "args": {
        "location": "San Francisco, CA"
      },
      "id": "c7a9d590-99ad-42af-9996-41b90efcf827",
      "type": "tool_call"
    }
  ],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 177,
    "output_tokens": 30,
    "total_tokens": 207
  }
}

结构化输出

Ollama 原生支持 结构化输出 适用于所有模型，允许你通过调用来强制模型返回特定格式 .withStructuredOutput().

// Define the schema
const Country = z.object({
  name: z.string(),
  capital: z.string(),
  languages: z.array(z.string()),
});

// Define the model
const llm = new ChatOllama({
  model: "llama3.1",
  temperature: 0,
});

// Pass the schema to enforce a specific output format
const structuredLlm = llm.withStructuredOutput(Country);

const result = await structuredLlm.invoke("Tell me about Canada.");
console.log(result);

{
  name: 'Canada',
  capital: 'Ottawa',
  languages: [ 'English', 'French' ]
}

如果您更喜欢通过工具调用使用结构化输出，请传递 method: "functionCalling" option:

// Define the schema
const Sentence = z.object({
  nouns: z.array(z.string()),
});

// Define the model
const llm = new ChatOllama({
  model: "llama3.1",
  temperature: 0,
});

// Use structured output via tool calling
const structuredLlm = llm.withStructuredOutput(Sentence, { method: "functionCalling" });

const result = await structuredLlm.invoke("Extract all nouns: A cat named Luna who is 5 years old and loves playing with yarn. She has grey fur");
console.log(result);

{ nouns: [ 'cat', 'Luna', 'years', 'yarn', 'fur' ] }

多模态模型

Ollama 支持开源多模态模型，例如 LLaVA 在 0.1.15 及更高版本中。 您可以将图像作为消息的 content 字段传递给 multimodal-capable 模型，如下所示：

const imageData = await fs.readFile("../../../../../examples/hotdog.jpg");
const llmForMultiModal = new ChatOllama({
  model: "llava",
  baseUrl: "http://127.0.0.1:11434",
});
const multiModalRes = await llmForMultiModal.invoke([
  new HumanMessage({
    content: [
      {
        type: "text",
        text: "What is in this image?",
      },
      {
        type: "image_url",
        image_url: `data:image/jpeg;base64,${imageData.toString("base64")}`,
      },
    ],
  }),
]);
console.log(multiModalRes);

AIMessage {
  "content": " The image shows a hot dog in a bun, which appears to be a footlong. It has been cooked or grilled to the point where it's browned and possibly has some blackened edges, indicating it might be slightly overcooked. Accompanying the hot dog is a bun that looks toasted as well. There are visible char marks on both the hot dog and the bun, suggesting they have been cooked directly over a source of heat, such as a grill or broiler. The background is white, which puts the focus entirely on the hot dog and its bun. ",
  "additional_kwargs": {},
  "response_metadata": {
    "model": "llava",
    "created_at": "2024-08-01T17:25:02.169957Z",
    "done_reason": "stop",
    "done": true,
    "total_duration": 5700249458,
    "load_duration": 2543040666,
    "prompt_eval_count": 1,
    "prompt_eval_duration": 1032591000,
    "eval_count": 127,
    "eval_duration": 2114201000
  },
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 1,
    "output_tokens": 127,
    "total_tokens": 128
  }
}

API 参考

有关所有功能的详细文档 ChatOllama 特性和配置，请前往 API 参考.