Step 0:环境准备
建议使用 Python 3.10+,用虚拟环境隔离依赖:
python3 -m venv .venv && source .venv/bin/activate
pip install langchain langchain-community chromadb
pip install pypdf python-docx openpyxl
# 中文向量模型(任选其一)
pip install sentence-transformersStep 1:文档加载与切分
不同格式用不同 Loader,切分时按标题层级切,而不是按固定字数切:
from langchain_community.document_loaders import PyPDFLoader, Docx2txtLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
loader = PyPDFLoader("SOP-205.pdf") # PDF
# loader = Docx2txtLoader("制度手册.docx") # Word
docs = loader.load()
splitter = RecursiveCharacterTextSplitter(
chunk_size=500, chunk_overlap=50,
separators=["\n\n", "\n", "。", "!", "?", " ", ""]
)
chunks = splitter.split_documents(docs)
print(f"切分完成:{len(chunks)} 个片段")要点:图纸类文档建议按图层/标号结构化后再切;表格整表保留不切散;chunk 大小按业务需要调,500 字是经验起点。
Step 2:向量化与入库
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import Chroma
embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-large-zh-v1.5")
vectorstore = Chroma.from_documents(
documents=chunks,
embedding=embeddings,
persist_directory="./chroma_db" # 本地持久化
)Step 3:检索问答
from langchain.chains import RetrievalQA
from langchain_community.llms import Ollama
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
llm = Ollama(model="qwen2.5:14b", base_url="http://localhost:11434")
qa = RetrievalQA.from_chain_type(
llm=llm, chain_type="stuff",
retriever=retriever, return_source_documents=True
)
res = qa.invoke("SOP-205 第 5 步的焊接温度是多少?")
print(res["result"])
print("来源:", [d.metadata.get("source") for d in res["source_documents"]])Step 4:权限与审计(上线前必做)
生产环境必须做三件事:文档级权限过滤(检索时按用户角色过滤片段)、问答日志(记录谁问了什么、引用了哪些文档)、答案反馈(点赞/点踩回流,定期重训)。示例:
# 权限过滤:检索前按部门过滤
def filter_by_permission(docs, role):
return [d for d in docs if role in d.metadata.get("roles", [])]
# 审计:写入日志(伪代码)
log_entry = {"user": user_id, "question": q, "sources": [d.metadata["source"] for d in src]}
write_audit_log(log_entry)本教程为最小可运行版本;生产环境建议补充:多用户隔离、SSO 集成、向量库备份、模型推理加速(vLLM)等。