大模型人工智能微调本地部署AI AgentRAG【免费下载链接】ChatGLM3ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型项目地址https://gitcode.com/gh_mirrors/ch/ChatGLM3点击查看免费下载导读本文基于 ChatGLM3 官方PROMPT_en.md文档系统讲解 ChatGLM3 系列模型采用的全新对话格式Chat Format包括|system|、|user|、|assistant|、|observation|四种角色头的语法规则、特殊 token 的注入防护机制以及多轮对话、工具调用Tool Calling、代码执行Code Interpreter三类场景的完整示例。读完本文你将掌握如何手工构造符合规范的历史消息、如何在工具调用与代码执行任务中正确编排|assistant|的 metadata 与|observation|回填并能结合仓库内composite_demo、tools_using_demo、openai_api_demo的源码验证格式在真实推理链路中的落地方式。为什么 ChatGLM3 需要一套全新对话格式传统的大语言模型对话通常以纯文本拼接 role 标签如Human:/Assistant:这种形式存在两个问题注入攻击风险用户输入的内容可能包含伪造的对话分隔符诱导模型扮演其他角色或覆盖系统指令任务输入不统一Code Interpreter、Tool Agent 等任务需要传递外部工具返回结果代码执行环境等信息纯文本格式无法优雅地区分这些语义不同的输入。为此ChatGLM3 引入了一套全新的对话格式将对话分成多个对话节conversation每个对话节由对话头chat header 内容组成。该设计在仓库 README 中被描述为模型原生支持多轮对话、工具调用Function Call与代码执行Code Interpreter能力的基础参见 README_en.md。对话头Chat Header的语法对话头独占一整行格式为|role|{metadata}|role|部分使用**特殊 tokenspecial token**表示文本形态无法被 tokenizer 编码从机制上杜绝了用户输入伪造角色头的注入攻击metadata部分使用普通文本表示是可选内容。四种角色头及其规则如下角色头含义规则约束|system|系统信息设计中可穿插在对话任意位置但当前实现中仅出现在对话开头|user|用户多个|user|消息不会连续出现|assistant|AI 助手出现之前必须已有|user|消息|observation|外部返回结果必须紧跟在|assistant|消息之后可读性提示文档示例中每个角色特殊 token 前额外加了一个\n这只是为了阅读方便实际使用和 tokenizer 实现中不应加入这个多余的换行。角色枚举在源码中的体现仓库 composite_demo/conversation.py 中Role枚举完整映射了这四种角色并额外细化了工具与解释器两种子角色class Role(Enum): SYSTEM auto() USER auto() ASSISTANT auto() TOOL auto() INTERPRETER auto() OBSERVATION auto() def __str__(self): match self: case Role.SYSTEM: return |system| case Role.USER: return |user| case Role.ASSISTANT | Role.TOOL | Role.INTERPRETER: return |assistant| case Role.OBSERVATION: return |observation|注意Role.TOOL与Role.INTERPRETER在序列化时同样输出为|assistant|其区别完全由 metadata工具名或interpreter体现——这与下文工具调用、代码执行场景的格式完全一致。Conversation.__str__composite_demo/conversation.py则展示了拼接规则普通角色输出角色头\n内容工具调用输出角色头{工具名}\n{调用代码}解释器输出角色头interpreter\n{代码}。场景一多轮对话Multi-turn Dialogue多轮对话只使用三种角色|user|、|assistant|与|system|。完整示例|system| You are ChatGLM3, a large language model trained by Zhipu.AI. Follow the users instructions carefully. Respond using markdown. |user| Hello |assistant| Hello, Im ChatGLM3. What can I assist you today?系统提示词是模型角色设定的核心载体以上面这套 ChatGLM3 默认系统提示词为例它同时约束了遵循用户指令与使用 Markdown 回复两条行为准则。在 transformers 中发起对话仓库 README_en.md 给出了标准调用方式多轮历史直接通过history参数回传from transformers import AutoTokenizer, AutoModel tokenizer AutoTokenizer.from_pretrained(THUDM/chatglm3-6b, trust_remote_codeTrue) model AutoModel.from_pretrained(THUDM/chatglm3-6b, trust_remote_codeTrue, devicecuda) model model.eval() response, history model.chat(tokenizer, 你好, history[]) response, history model.chat(tokenizer, What should I do if I cant sleep at night?, historyhistory)历史记录的组装由 tokenizer 的build_chat_input(query, historyhistory, rolerole)完成见 composite_demo/client.pyhistory 中每个元素为{role: ..., content: ...}字典其中 role 即上文四种角色之一。场景二工具调用Tool Calling工具调用场景的完整格式如下注意|assistant|后携带工具名作为 metadata|system| Answer the following questions as best as you can. You have access to the following tools: [ { name: get_current_weather, description: Get the current weather in a given location, parameters: { type: object, properties: { location: { type: string, description: The city and state, e.g. San Francisco, CA, }, unit: {type: string}, }, required: [location], }, } ] |user| Whats the weather in Beijing today? |assistant| Okay, lets look up the weather in Bejing today. |assistant|get_current_weather python tool_call(locationbeijing, unitcelsius) |observation| {temperature: 22} |assistant| According to the query results, the temperature in Beijing today is 22 degrees Celsius.整个流程可以拆解为四个步骤系统提示注入工具清单系统提示词固定为Answer the following questions as best as you can. You have access to the following tools:随后以 JSON 数组形式给出工具描述name / description / parameters其中 parameters 遵循 JSON Schema 的type、properties、required结构模型发起工具调用模型先输出一段自然语言说明意图然后输出|assistant|get_current_weather作为对话头紧接着输出一段tool_call(locationbeijing, unitcelsius)形式的调用代码外部工具返回结果调用方执行工具后将结果以 JSON 文本放入|observation|角色模型生成最终回答模型基于观察结果组织最终回复。工具调用在 composite_demo 中的实现composite_demo/demo_tool.py 展示了完整的状态机以Role.USER与Role.OBSERVATION作为停止序列stop_sequences[str(r) for r in (Role.USER, Role.OBSERVATION)]流式生成当检测到特殊 token|assistant|时切换到工具消息块检测到|observation|时按换行把输出切分为工具名 调用参数文本构造Role.TOOL对话用正则r([^\n]*)\n(.*?)从代码块中提取参数通过eval(code, {tool_call: tool_call}, {})解析出参数字典调用dispatch_tool(tool, args)执行真实工具见 composite_demo/tool_registry.py并把返回值写回Role.OBSERVATION历史继续下一轮生成。工具的定义与注册通过register_tool装饰器完成composite_demo/tool_registry.py它利用typing.Annotated类型注解自动提取参数名、类型、描述与必填标记例如register_tool def get_weather( city_name: Annotated[str, The name of the city to be queried, True], ) - str: Get the current weather for city_name仅支持 chat 方法的工具调用tools_using_demo工具调用当前仅支持model.chat方法不支持stream_chat见 tools_using_demo/README_en.md。官方流程如下tools [ { name: track, description: Track the real-time price of a specified stock, parameters: { type: object, properties: { symbol: {description: The stock code that needs to be tracked} }, required: [symbol] } }, { name: text-to-speech, description: Convert text to speech, parameters: { type: object, properties: { text: {description: The text that needs to be converted into speech}, voice: {description: The type of voice to use (male, female, etc.)}, speed: {description: The speed of the speech (fast, medium, slow, etc.)} }, required: [text] } } ] system_info {role: system, content: Answer the following questions as best as you can. You have access to the following tools:, tools: tools}发起查询history [system_info] query Help me inquire the price of stock 10111 response, history model.chat(tokenizer, query, historyhistory) print(response)预期输出为 JSON{name: track, parameters: {symbol: 10111}}表示模型需要调用工具track并传入参数symbol。拿到工具执行结果后以roleobservation回填import json result json.dumps({price: 12412}, ensure_asciiFalse) response, history model.chat(tokenizer, result, historyhistory, roleobservation) print(response)roleobservation表明该输入是工具调用返回值而非用户输入不能省略。预期输出Based on your query, after the API call, the price of stock 10111 is 12412.对于更复杂的任务模型可能需要多次调用工具此时可通过判断返回的response是str生成回复还是dict工具调用请求来决定是否继续循环调用。工具调用在 OpenAI 兼容 API 中的解析openai_api_demo/utils.py 的process_response展示了从生成文本还原工具调用的解析逻辑按|assistant|切分输出metadata 部分即工具名代码块内的参数经eval解析后封装为 OpenAI 风格的{name: ..., arguments: ...}process_chatglm_messagesopenai_api_demo/utils.py则把 OpenAI 格式的function角色映射回observation角色、把带工具清单的 system 消息转换为带tools字段的消息。同时注意生成端把|user|与|observation|作为停止 tokenopenai_api_demo/utils.py保证每轮生成在轮到用户或观察输入前收敛。场景三代码执行Code Interpreter代码执行场景使用四种角色|user|、|assistant|、|system|与|observation|其中|assistant|的metadata 只能是interpreter。完整示例|system| You are an intelligent AI assistant, named ChatGLM3. You are connected to a computer, but please note that you cannot connect to the internet. When using Python to solve tasks, you can run the code and get the results. If there are errors in the running results, you need to improve the code as much as possible. You can process files uploaded by users to your computer, and the default storage path for files is /mnt/data/. |user| #File: /mnt/data/metadata.jsonl #Size: 35380 #File uploaded Are there missing or abnormal values in the file? |assistant| Firstly, I will read the file you provided and review its contents to determine if there are missing or abnormal values. We start by loading the file: |assistant|interpreter python import json # Load the uploaded file data [] with open(/mnt/data/metadata.jsonl, r) as file: for line in file: data.append(json.loads(line)) # Display the first few records to get a sense of the data data[:5] |observation| result [{file_name: aceinthehole.png, name: Ace in the Hole, type: survivor, description: Lady Luck always seems to be throwing something good your way.}, {file_name: adrenaline.png, name: Adrenaline, type: survivor, description: You are fuelled by unexpected energy when on the verge of escape.}, {file_name: aftercare.png, name: Aftercare, type: survivor, description: Unlocks potential in ones aura reading ability.}, {file_name: agitation.png, name: Agitation, type: killer, description: You get excited in anticipation of hooking your prey.}, {file_name: alert.png, name: Alert, type: survivor, description: Your acute senses are on high alert.}] |assistant| This file appears to contain metadata about certain items, each with the following fields: - file_name - name - type (e.g. survivor or killer) - description Next, I will check for missing values in these fields. |assistant|interpreter python # Check for missing values in each column missing_values {key: sum(1 for item in data if key not in item) for key in data[0].keys()} missing_values |observation| result {file_name: 0, name: 0, type: 0, description: 0} |assistant| Each field has no missing values. This means that each entry is complete. Next, we can check for any potential outliers. This usually requires deeper analysis and understanding of the data, but I can start by examining some basic statistics and distributions. To do this, we can check the value distribution of the type field to see if there are any unexpected categories. |assistant|interpreter python # Check the distribution of values in the type column type_distribution {key: sum(1 for item in data if item[type] key) for key in set(item[type] for item in data)} type_distribution |observation| result {survivor: 116, killer: 103} |assistant| The type field has two values: survivor and killer. Among them, survivor has 116 entries, while killer has 103 entries. From this preliminary examination, we did not find any obvious abnormal values. If you have any specific standards or other fields that need to be checked, I can provide you with a more detailed analysis. What else do you want me to check? |user| Help me draw a heart |assistant|interpreter python import numpy as np import matplotlib.pyplot as plt # The parametric equation for the shape of a heart def heart(t): x 16 * np.sin(t) ** 3 y 13 * np.cos(t) - 5 * np.cos(2 * t) - 2 * np.cos(3 * t) - np.cos(4 * t) return x, y t np.linspace(0, 2 * np.pi, 1000) x, y heart(t) plt.figure(figsize(6, 6)) plt.plot(x, y, colorred) plt.axis(equal) plt.axis(off) plt.show() |observation| result [Image] |assistant| This is a shape of heart. I used parametric equations to describe this shape and drew it using Matplotlib. If you have any other needs or questions, please feel free to let me know. |user| # End这段长示例揭示了代码解释器的几个关键行为特征系统提示词声明能力边界模型被明确告知连接着电脑但不能联网可用 Python 运行代码并获得结果可处理用户上传文件默认存储路径 /mnt/data/用户以文件元信息开头上传文件以#File:路径与#Size:大小注释作为上下文注入用户消息代码以|assistant|interpreter头开始每个代码块前都有独立的|assistant|interpreter对话头**执行结果以\result代码块回填**文本结果以result块包裹绘图等图像结果以[Image] 占位多轮交互直至# End结束用户以# End注释终止对话。代码解释器在 composite_demo 中的实现composite_demo/demo_ci.py 的生成循环中模型流式输出遇到|assistant|特殊 token 时切到 interpreter 消息块遇到|observation|时用extract_code提取代码块并通过CodeKernel.execute在 Jupyter 内核jupyter_client.KernelManager见 composite_demo/demo_ci.py中真实执行res_type, res execute(code, get_kernel())executecomposite_demo/demo_ci.py会先剥离代码中的|observation|、|assistant|interpreter等格式标记再把内核执行结果分类为text或imageimage/png输出经 base64 解码为 PIL Image文本结果超过truncate_length默认 1024时截断并追加[TRUNCATED]标记。执行结果随后以Role.OBSERVATION写回历史模型据此继续生成下一轮回复形成思考 → 写代码 → 观察结果 → 再思考的闭环。实践手工构造与解析对话格式的通用要点序列化构造 prompt综合 composite_demo/conversation.py 的preprocess_text构造完整 prompt 的顺序为|system|\n{系统提示} # 无工具时 |system|\nAnswer the following questions as best as you can. You have access to the following tools:\n{tools JSON} # 有工具时 |user|\n{内容} |assistant|\n{内容} |assistant|{工具名}\n{工具调用代码} |assistant|interpreter\n{代码} |observation|\n{结果} |assistant|\n # 末尾追加等待模型续写其中工具列表在preprocess_text中会被序列化为缩进 JSONjson.dumps(tools, indent4, ensure_asciiFalse)并拼接在固定引导语之后。反序列化解析生成结果生成结果解析的通用要点综合 composite_demo/conversation.py 与 openai_api_demo/utils.py特殊 token 本身|assistant|、|observation|等需要通过postprocess_text清理避免污染展示文本代码块内容通过围栏正则提取工具调用按metadata行工具名 参数代码块拆分参数通过eval解析为字典流式生成时必须把|user|、|observation|加入停止 token / 停止序列见 openai_api_demo/utils.py 与 composite_demo/demo_tool.py否则模型会一直生成到下一轮用户输入。运行与验证可以在仓库中直接运行完整集成的演示对话 / 工具 / 代码解释器三合一参见 composite_demo/README_en.mdcd composite_demo pip install -r requirements.txt streamlit run main.py集成演示通过 composite_demo/main.py 按--tool、--ci等参数切换 Tool 与 Code Interpreter 模式是观察格式在实际推理链路中流转最直观的入口。此外工具调用的最小复现路径是 tools_using_demo代码解释器的最小复现路径是 composite_demo/demo_ci.py。小结ChatGLM3 的对话格式以特殊 token 角色头为安全边界以metadata承载工具名与解释器标记以|observation|统一外部结果回填从而让多轮对话、工具调用、代码执行三类任务共用同一套输入协议。无论是通过 transformers 的model.chat/stream_chat、OpenAI 兼容 APIopenai_api_demo还是 Streamlit 集成演示都遵循本文所述的同一套格式规范。构造 prompt 时务必遵守角色顺序、metadata 限定、结果回填三条规则并配合|user|/|observation|停止 token 使用即可稳定获得符合预期的生成行为。赞分享大模型人工智能微调本地部署AI AgentRAG【免费下载链接】ChatGLM3ChatGLM3 series: Open Bilingual Chat LLMs | 开源双语对话语言模型项目地址https://gitcode.com/gh_mirrors/ch/ChatGLM3点击查看免费下载相关推荐XTuner 数据集格式全解析增量预训练、单轮对话与多轮对话的统一 JSON 规范XTuner 数据集格式全解析增量预训练、单轮对话与多轮对话的统一 JSON 规范 XTuner 是大语言模型与多模态模型的微调训练引擎其 Supervis大模型模型微调Llama 3.1 Prompt Format 完全指南特殊 Token、多轮对话与工具调用协议Llama 3.1 Prompt Format 完全指南特殊 Token、多轮对话与工具调用协议 本文以 models/llama3_1/prompt_for人工智能大模型基础模型ChatGLM3工具调用开发完全指南从函数注册到多轮对话流程ChatGLM3工具调用开发完全指南从函数注册到多轮对话流程 ChatGLM3是智谱AI和清华大学KEG实验室联合发布的开源双语对话语言模型其中ChatGL大模型人工智能微调本地部署AI AgentRAG创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考