此示例介绍如何从多个文件路径加载数据。第二个参数是文件扩展名到加载器工厂的映射。每个文件将被传递给匹配的加载器,生成的文档将被连接在一起。
示例文件:
src/document_loaders/example_data/example/
├── example.txt
└── example.csv
src/document_loaders/example_data/example2/
├── example.json
└── example.jsonl
示例代码:
JSONLoader,
JSONLinesLoader,
} from "@langchain/classic/document_loaders/fs/json";
const loader = new MultiFileLoader(
[
"src/document_loaders/example_data/example/example.txt",
"src/document_loaders/example_data/example/example.csv",
"src/document_loaders/example_data/example2/example.json",
"src/document_loaders/example_data/example2/example.jsonl",
],
{
".json": (path) => new JSONLoader(path, "/texts"),
".jsonl": (path) => new JSONLinesLoader(path, "/html"),
".txt": (path) => new TextLoader(path),
".csv": (path) => new CSVLoader(path, "text"),
}
);
const docs = await loader.load();
console.log({ docs });