Tavily 爬取集成 - 以编程方式使用文档

Tavily 是一个专为 AI 代理（LLM）构建的搜索引擎，以速度提供实时、准确和真实的结果。Tavily 提供爬取端点，可从起始 URL 执行结构化网页遍历，内置内容提取和智能发现功能。

概述

集成详情

类	包	可序列化	JS 支持	版本
`TavilyCrawl`	`langchain-tavily`	✅	✅	!PyPI - 版本

工具特性

返回工件	原生异步	返回数据	价格
❌	✅	base URL, page URL, raw content, images, favicon	1,000 free credits / month

设置

该集成位于 langchain-tavily package.

pip install -qU langchain-tavily

凭证

我们还需要设置 Tavily API 密钥。您可以通过访问此网站并创建一个账户来获取 API 密钥。

if not os.environ.get("TAVILY_API_KEY"):
    os.environ["TAVILY_API_KEY"] = getpass.getpass("Tavily API key:\n")

实例化

该工具在实例化期间接受以下参数：

- max_depth （可选，int）：从起始 URL 的最大跳转次数。默认值为 1。
- max_breadth （可选，int）：每层最大爬取页面数。默认值为 20。
- limit （可选，int）：最大总爬取页面数。默认值为 50。
- instructions （可选，str）：指导爬虫的自然语言指令。
- select_paths （可选，list[str]）：仅爬取包含这些路径正则表达式的 URL。
- select_domains （可选，list[str]）：仅爬取这些域名正则表达式。
- exclude_paths （可选，list[str]）：跳过包含这些路径正则表达式的 URL。
- exclude_domains （可选，list[str]）：跳过这些域名正则表达式的 URL。
- allow_external （可选，bool）：允许爬虫跟随外部链接。
- extract_depth （可选，str）： "basic" or "advanced" 每个爬取页面的提取深度。
- format （可选，str）： "markdown" or "text" 内容格式。
- include_images （可选，bool）：包含每个页面上发现的图片。

有关可用参数的完整概述，请参阅 Tavily 爬取 API 文档.

from langchain_tavily import TavilyCrawl

tool = TavilyCrawl(
    max_depth=1,
    max_breadth=20,
    limit=50,
    # extract_depth="basic",
    # format="markdown",
    # include_images=False,
    # allow_external=False,
)

调用

使用参数直接调用

Tavily 爬取工具在调用期间接受以下参数：

- url （必需）：开始爬取的起始 URL。
- 以下参数也可以在调用期间设置： instructions, select_paths, select_domains, exclude_paths, exclude_domains, allow_external.

注意：可选参数可供代理动态设置。如果您在实例化期间设置了参数，然后使用不同的值调用工具，工具将使用您在调用期间传递的值。

tool.invoke({"url": "https://docs.tavily.com"})

{
  "base_url": "https://docs.tavily.com",
  "results": [
    {
      "url": "https://docs.tavily.com/",
      "raw_content": "![light logo](https://mintcdn.com/tavilyai/HY1Rnt85q4usR4-R/logo/light.svg?fit=max&auto=format&n=HY1Rnt85q4usR4-R&q=85&s=c5c878011f13d458af0997f3a540eb4f)\n![dark logo](https://mintcdn.com/tavilyai/HY1Rnt85q4usR4-R/logo/dark.svg?fit=max&auto=format&n=HY1Rnt85q4usR4-R&q=85&s=1521677768a1f26b34a9ad86d04c62cc)\n\n##### Welcome\n\n## Documentation Index\n\nFetch the complete documentation index at: <https://docs.tavily.com/llms.txt>\n\nUse this file to discover all available pages before exploring further.\n\n# Build with Tavily\n\nYour journey to state-of-the-art web search starts right here.\n\n## Python SDK\n\n`pip install tavily-python`\n\n## JavaScript SDK\n\n`npm i @tavily/core`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.search(\"Who is Leo Messi?\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.extract(\"https://en.wikipedia.org/wiki/Artificial_intelligence\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.crawl(\"https://docs.tavily.com\", instructions=\"Find all pages on the Python SDK\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.map(\"https://docs.tavily.com\")\n\nprint(response)`\n`from tavily import TavilyClient\n\ntavily_client = TavilyClient(api_key=\"tvly-YOUR_API_KEY\")\nresponse = tavily_client.research(\"What are the latest developments in AI?\")\n\nprint(response)`\n\n### Developer Resources\n\n## API Credits Overview\n\n## Rate Limits\n\n## Playground",
      "favicon": "https://docs.tavily.com/mintlify-assets/_mintlify/favicons/tavilyai/SXaxSfweEU3ftIlh/_generated/favicon/apple-touch-icon.png"
    },
    {
      "url": "https://docs.tavily.com/documentation/about",
      "raw_content": "Performance\n\nCommitted to speed and efficiency, our API guarantees real-time and trusted information. Our team works hard to improve Tavily’s performance over time.\n\nIntegration-friendly\n\nWe appreciate the essence of adaptability. That’s why integrating our API with your existing setup is a breeze. You can choose our Python library, JavaScript package or a simple API call. You can also use Tavily through any of our supported partners such as LangChain and LlamaIndex.\n\nTransparent & Informative\n\nOur detailed documentation ensures you’re never left in the dark. From setup basics to nuanced features, we’ve got you covered.\n\n## \n\nHow does the Search API work? [...] With LLM hallucinations, it’s crucial to optimize for RAG with the right context and information. This is where Tavily comes in, delivering accurate and precise information for your RAG applications.\n\n## \n\nGetting started\n\nSign up for Tavily to get your API key. You get 1,000 free API Credits every month. No credit card required.\n\n## Get your free API key You get 1,000 free API Credits every month. No credit card required.\n\nHead to our API Playground to familiarize yourself with our API.To get started with Tavily’s APIs and SDKs using code, head to our Quickstart Guide and follow the steps.\n\nGot questions? Stumbled upon an issue? Simply intrigued? Don’t hesitate! Our support team is always on standby, eager to assist. Join us, dive deep, and redefine your search experience! Contact us! [...] # About - Tavily Docs\n\nSkip to main content\n\nTavily Docs home pageImage 1: light logoImage 2: dark logo\n\nSearch...\n\nCtrl K Ask AI\n\n   Support\n   Get an API key\n   Get an API key\n\nSearch...\n\nNavigation\n\nGetting Started\n\nAbout\n\nHomeIntroductionAPI & SDKsEcosystemExamplesChangelogHelp\n\n   API Playground\n   Community\n   Blog\n\n##### Getting Started\n\n   About\n   Quickstart\n   Credits & Pricing\n   Rate Limits\n\n##### FAQ\n\n   Frequently Asked Questions\n\nOn this page\n\n   Who are we?\n   What is the Tavily Search Engine?\n   Why choose Tavily?\n   How does the Search API work?\n   Getting started\n\nGetting Started\n\n# About\n\nCopy page\n\nWelcome to Tavily!\n\nCopy page\n\nLooking for a step-by-step tutorial to get started in under 5 minutes? Head to our Quickstart guide and start coding!\n\n## \n\nWho are we?",
      "favicon": "https://docs.tavily.com/mintlify-assets/_mintlify/favicons/tavilyai/SXaxSfweEU3ftIlh/_generated/favicon/apple-touch-icon.png"
    },
    {
      "url": "https://docs.tavily.com/documentation/api-reference/introduction",
      "raw_content": "# Introduction - Tavily Docs\n\nSkip to main content\n\nTavily Docs home pageImage 1: light logoImage 2: dark logo\n\nSearch...\n\n⌘K Ask AI\n\n   Support\n   Get an API key\n   Get an API key\n\nSearch...\n\nNavigation\n\nAPI Reference\n\nIntroduction\n\nHomeIntroductionAPI & SDKsEcosystemExamplesChangelogHelp\n\n   API Playground\n   Community\n   Blog\n\n##### API Reference\n\n   Introduction\n   POST Search\n   POST Extract\n   POST Crawl\n   POST Map\n   Research  \n   GET Usage\n\n##### Enterprise API Reference\n\n   API Key Generator  \n\n##### Python SDK\n\n   Quickstart\n   SDK Reference\n\n##### JavaScript SDK\n\n   Quickstart\n   SDK Reference\n\n##### Best Practices\n\n   Search\n   Extract\n   Crawl\n   Research\n   API Key Management\n\nOn this page\n\n   Base URL\n   Authentication\n   Endpoints\n   Project Tracking\n\nAPI Reference [...] When using the SDKs, you can specify a project using the `project_id` parameter when instantiating the client, or by setting the `TAVILY_PROJECT` environment variable.\n\nTavily Search Next\n\n⌘I\n\nTavily Docs home pageImage 3: light logoImage 4: dark logo\n\nlinkedinxgithubyoutube\n\nResources\n\nAboutTavily Search CrawlerTavily Status\n\nLegal\n\nSecurity & CompliancePrivacy PolicyWebsite Terms of UsePlatform Terms of UseCookie Notice\n\nlinkedinxgithubyoutube\n\nPowered by This documentation is built and hosted on Mintlify, a developer documentation platform\n\nAssistant\n\nResponses are generated using AI and may contain mistakes. [...] API Reference\n\n# Introduction\n\nCopy page\n\nEasily integrate our APIs with your services.\n\nCopy page\n\n> ## Documentation Index\n> \n> \n> Fetch the complete documentation index at: \n> \n> \n> Use this file to discover all available pages before exploring further.\n\n## \n\nBase URL\n\nThe base URL for all requests to the Tavily API is:\n\n

\n\n

      "favicon": "https://docs.tavily.com/mintlify-assets/_mintlify/favicons/tavilyai/SXaxSfweEU3ftIlh/_generated/favicon/apple-touch-icon.png"
    },
    {
      "url": "https://github.com/tavily-ai",
      "raw_content": "JavaScript1.8k239  \n2.   tavily-pythontavily-python Public \nThe Tavily Python SDK allows for easy interaction with the Tavily API, offering the full range of our search, extract, crawl, map, and research functionalities directly from your Python programs. E…\n\nPython1.2k148  \n3.   skillsskills Public \nPython23321  \n4.   meeting-prep-agentmeeting-prep-agent Public \nPrepare for meetings with company and attendee research notes, integrated with Google Calendar through MCP\n\nPython19334  \n5.   tavily-jstavily-js Public \nOfficial TypeScript/JavaScript library for Tavily.\n\nTypeScript839  \n6.   tavily-chattavily-chat Public \nConversational agent that fuses chat data with live web results through Tavily search, extract, and crawl.\n\nTypeScript7216  \n\n### Repositories\n\nLoading\n\nType\n\nSelect type [...] ### Repositories\n\nLoading\n\nType\n\nSelect type\n\nAll Public Sources Forks Archived Mirrors Templates \n\nLanguage\n\nSelect language\n\nAll JavaScript Jupyter Notebook Python TypeScript \n\nSort\n\nSelect order\n\nLast updated Name Stars \n\nShowing 10 of 33 repositories [...] langchain-tavily Public  ### Uh oh! There was an error while loading. [Please reload this page.]( past year of commit activity Python22MIT 1511 Updated Apr 16, 2026  \n   langchainjs Public  Forked from langchain-ai/langchainjs\n🦜🔗 Build context-aware reasoning applications ### Uh oh! There was an error while loading. [Please reload this page.]( past year of commit activity TypeScript 0 MIT 3,20800 Updated Apr 16, 2026",
      "favicon": "https://github.githubassets.com/favicons/favicon.svg"
    }
  ],
  "response_time": 13.45
}

使用 ToolCall 调用

我们也可以使用模型生成的 ToolCall 来调用该工具，在这种情况下将返回一个 ToolMessage：

model_generated_tool_call = {
    "args": {"url": "https://docs.tavily.com", "instructions": "Find API reference pages"},
    "id": "1",
    "name": "tavily_crawl",
    "type": "tool_call",
}
tool_msg = tool.invoke(model_generated_tool_call)

print(tool_msg.content[:400])

在代理中使用

我们可以通过将爬虫工具绑定到模型上，直接在代理中使用它。这样代理就可以动态设置可用的参数，以控制爬取的位置和深度。

if not os.environ.get("OPENAI_API_KEY"):
    os.environ["OPENAI_API_KEY"] = getpass.getpass("OPENAI_API_KEY:\n")

from langchain.chat_models import init_chat_model

model = init_chat_model(model="gpt-5.5", model_provider="openai", temperature=0)

from langchain_tavily import TavilyCrawl
from langchain.agents import create_agent


tavily_crawl_tool = TavilyCrawl(max_depth=1, max_breadth=10, limit=10)

agent = create_agent(model, [tavily_crawl_tool])

user_input = "Crawl https://docs.tavily.com and find the API reference for the search endpoint."

stream = agent.stream_events({"messages": user_input}, version="v3")
for snapshot in stream.values:
    snapshot["messages"][-1].pretty_print()

API 参考

如需了解所有 Tavily Crawl API 功能和配置的详细文档，请前往 API 参考： docs.tavily.com/documentation/api-reference/endpoint/crawl